[ROC-RK3568-PC] [Firefly-Android] 10min带你了解Camera的使用

奋斗吧

擅长邻域：未填写

标签： [ROC-RK3568-PC] [Firefly-Android] 10min带你了解Camera的使用 Python博客 51CTO博客

2023-03-26 18:23:29 105浏览

[ROC-RK3568-PC] [Firefly-Android] 10min带你了解Camera的使用，根据板子原理图的MIPICSI接口找到两个sensor是挂在哪个I2C总线上，然后在对应的I2C节点配置两个camera节点，正确配置camera模组的I2C设备

? 博主主页： 【Systemcall小酒屋】

? 博主追寻：热衷于用简单的案例讲述复杂的技术，“假传万卷书，真传一案例”，这是林群院士说过的一句话，另外“成就是最好的老师”，技术既要沉淀，也得分享，成就感的正反馈是支持我持续耕耘的动力！

[ROC-RK3568-PC] [Firefly-Android] 10min带你了解Camera的使用_isp

文章目录

一、MIPI CSI

1.1 Full Mode特点
1.2 Split Mode特点

二、Full Mode配置

2.1 配置sensor端
2.2 csi2_dphy0相关配置
2.3 isp相关配置

三、Split Mode配置

3.1 配置sensor端
3.2 csi2_dphy1/csi2_dphy2相关配置
3.3 isp相关配置

四、软件相关目录
五、单目CAM-8MS1M/双目CAM-2MS2MF摄像头的使用

5.1 使用单目摄像头CAM-8MS1M
5.2 使用双目摄像头CAM-2MS2MF

六、Camera底层调试
七、Android系统使用camera应用
八、IQ文件

一、MIPI CSI

RK3566/RK3568平台仅有一个标准物理mipi csi2 dphy，可以工作在两个模式：full mode 和split mode，拆分为csi2_dphy0/csi2_dphy1/csi2_dphy2三个逻辑dphy

1.1 Full Mode特点

仅使用csi2_dphy0。csi2_dphy0与csi2_dphy1/csi2_dphy2互斥，不可同时使用
data lane最大4 lanes
最大速率2.5Gbps/lane

1.2 Split Mode特点

仅使用csi2_dphy1和csi2_dphy2，两者与 csi2_dphy0 互斥，不可同时使用
csi2_dphy1和csi2_dphy2可同时使用
csi2_dphy1和csi2_dphy2各自的data lane最大是2 lanes
csi2_dphy1对应物理dphy的lane0/lane1
csi2_dphy2对应物理dphy的lane2/lane3
最大速率2.5Gbps/lane

[ROC-RK3568-PC] [Firefly-Android] 10min带你了解Camera的使用_isp_02

一般来说，如果用单目摄像头可以配置full mode，若使用双目摄像头可以配置split mode

二、Full Mode配置

链接关系: sensor->csi2_dphy0->isp

2.1 配置sensor端

需要根据板子原理图的MIPI CSI接口找到sensor是挂在哪个I2C总线上，然后在对应的I2C节点配置camera节点，正确配置camera模组的I2C设备地址、引脚等属性。

如下ROC-RK3568-PC的xc7160配置：

&i2c4 {
    status = "okay";
    XC7160: XC7160b@1b {
        status = "okay";
        compatible = "firefly,xc7160";
        reg = <0x1b>;
        clocks = <&cru CLK_CIF_OUT>;
        clock-names = "xvclk";
        power-domains = <&power RK3568_PD_VI>;
        pinctrl-names = "default";
        pinctrl-0 = <&cif_clk>;

        power-gpios = <&gpio4 RK_PB5 GPIO_ACTIVE_LOW>;
        reset-gpios = <&gpio0 RK_PD5 GPIO_ACTIVE_HIGH>;
        pwdn-gpios = <&gpio4 RK_PB4 GPIO_ACTIVE_HIGH>;

        firefly,clkout-enabled-index = <0>;
        rockchip,camera-module-index = <0>;
        rockchip,camera-module-facing = "back";
        rockchip,camera-module-name = "NC";
        rockchip,camera-module-lens-name = "NC";
        port {
            xc7160_out: endpoint {
                remote-endpoint = <&mipi_in_ucam4>;
                data-lanes = <1 2 3 4>;
            };
        };
    };
};

2.2 csi2_dphy0相关配置

csi2_dphy0与csi2_dphy1/csi2_dphy2互斥,不可同时使用。另外需要使能csi2_dphy_hw节点：

&csi2_dphy0 {
    status = "okay";
    /*
    * dphy0 only used for full mode,
    * full mode and split mode are mutually exclusive
    */
    ports {
        #address-cells = <1>;
        #size-cells = <0>;
        port@0 {
            reg = <0>;
            #address-cells = <1>;
            #size-cells = <0>;
...
            mipi_in_ucam4: endpoint@5 {
                    reg = <5>;
                    remote-endpoint = <&xc7160_out>;
                    data-lanes = <1 2 3 4>;
            };
        };
        port@1 {
            reg = <1>;
            #address-cells = <1>;
            #size-cells = <0>;

            csidphy_out: endpoint@0 {
                    reg = <0>;
                    remote-endpoint = <&isp0_in>;
            };
        };
    };
};

&csi2_dphy_hw {
    status = "okay";
};

&csi2_dphy1 {
    status = "disabled";
};

&csi2_dphy2 {
    status = "disabled";
};

2.3 isp相关配置

其中rkisp_vir0节点的remote-endpoint指向csidphy_out：

&rkisp {
    status = "okay";
};

&rkisp_mmu {
    status = "okay";
};

&rkisp_vir0 {
    status = "okay";
    port {
        #address-cells = <1>;
        #size-cells = <0>;

        isp0_in: endpoint@0 {
            reg = <0>;
            remote-endpoint = <&csidphy_out>;
        };
    };
};

三、Split Mode配置

链接关系:

sensor1->csi_dphy1->isp_vir0sensor2->csi_dphy2->mipi_csi2->vicap->isp_vir1

3.1 配置sensor端

根据板子原理图的MIPI CSI接口找到两个sensor是挂在哪个I2C总线上，然后在对应的I2C节点配置两个camera节点，正确配置camera模组的I2C设备地址、引脚等属性。如下ROC-RK3568-PC的gc2053/gc2093配置：

&i2c4 {
    status = "okay";
    gc2053: gc2053@37 { //IR
        status = "okay";
        compatible = "galaxycore,gc2053";
        reg = <0x37>;

        avdd-supply = <&vcc_camera>;
        power-domains = <&power RK3568_PD_VI>;
        clock-names = "xvclk";
        pinctrl-names = "default";

        clocks = <&pmucru CLK_WIFI>;
        pinctrl-0 = <&refclk_pins>;
        power-gpios = <&gpio0 RK_PD5 GPIO_ACTIVE_HIGH>;//IR_PWR_EN
        pwdn-gpios = <&gpio4 RK_PB5 GPIO_ACTIVE_LOW>;

        firefly,clkout-enabled-index = <1>;
        rockchip,camera-module-index = <0>;
        rockchip,camera-module-facing = "back";
        rockchip,camera-module-name = "YT-RV1109-2-V1";
        rockchip,camera-module-lens-name = "40IR-2MP-F20";
        port {
            gc2053_out: endpoint {
                remote-endpoint = <&dphy1_in>;
                data-lanes = <1 2>;
            };
        };
    };
    gc2093: gc2093b@7e { //RGB
        status = "okay";
        compatible = "galaxycore,gc2093";
        reg = <0x7e>;

        avdd-supply = <&vcc_camera>;
        power-domains = <&power RK3568_PD_VI>;
        clock-names = "xvclk";
        pinctrl-names = "default";
        flash-leds = <&flash_led>;

        pwdn-gpios = <&gpio4 RK_PB4 GPIO_ACTIVE_HIGH>;

        firefly,clkout-enabled-index = <0>;
        rockchip,camera-module-index = <1>;
        rockchip,camera-module-facing = "front";
        rockchip,camera-module-name = "YT-RV1109-2-V1";
        rockchip,camera-module-lens-name = "40IR-2MP-F20";
        port {
            gc2093_out: endpoint {
                remote-endpoint = <&dphy2_in>;
                data-lanes = <1 2>;
            };
        };
    };
};

3.2 csi2_dphy1/csi2_dphy2相关配置

csi2_dphy0与csi2_dphy1/csi2_dphy2互斥,不可同时使用：

&csi2_dphy0 {
    status = "disabled";
};

&csi2_dphy1 {
    status = "okay";
    /*
    * dphy1 only used for split mode,
    * can be used concurrently with dphy2
    * full mode and split mode are mutually exclusive
    */
    ports {
        #address-cells = <1>;
        #size-cells = <0>;

        port@0 {
            reg = <0>;
            #address-cells = <1>;
            #size-cells = <0>;

            dphy1_in: endpoint@1 {
                reg = <1>;
                remote-endpoint = <&gc2053_out>;
                data-lanes = <1 2>;
            };
        };

        port@1 {
            reg = <1>;
            #address-cells = <1>;
            #size-cells = <0>;

            dphy1_out: endpoint@1 {
                reg = <1>;
                remote-endpoint = <&isp0_in>;
            };
        };
    };
};

&csi2_dphy2 {
    status = "okay";
    /*
    * dphy2 only used for split mode,
    * can be used concurrently with dphy1
    * full mode and split mode are mutually exclusive
    */
    ports {
        #address-cells = <1>;
        #size-cells = <0>;

        port@0 {
            reg = <0>;
            #address-cells = <1>;
            #size-cells = <0>;

            dphy2_in: endpoint@1 {
                reg = <1>;
                remote-endpoint = <&gc2093_out>;
                data-lanes = <1 2>;
            };
        };

        port@1 {
            reg = <1>;
            #address-cells = <1>;
            #size-cells = <0>;

            dphy2_out: endpoint@1 {
                reg = <1>;
                remote-endpoint = <&mipi_csi2_input>;
            };
        };
    };
};

&csi2_dphy_hw {
    status = "okay";
};

&mipi_csi2 {
    status = "okay";

    ports {
        #address-cells = <1>;
        #size-cells = <0>;

        port@0 {
            reg = <0>;
            #address-cells = <1>;
            #size-cells = <0>;

            mipi_csi2_input: endpoint@1 {
                reg = <1>;
                remote-endpoint = <&dphy2_out>;
                data-lanes = <1 2>;
            };
        };

        port@1 {
            reg = <1>;
            #address-cells = <1>;
            #size-cells = <0>;

            mipi_csi2_output: endpoint@0 {
                reg = <0>;
                remote-endpoint = <&cif_mipi_in>;
                data-lanes = <1 2>;
            };
        };
    };
};

&rkcif_mipi_lvds {
    status = "okay";
    port {
        cif_mipi_in: endpoint {
            remote-endpoint = <&mipi_csi2_output>;
            data-lanes = <1 2>;
        };
    };
};

&rkcif_mipi_lvds_sditf {
    status = "okay";
    port {
        mipi_lvds_sditf: endpoint {
            remote-endpoint = <&isp1_in>;
            data-lanes = <1 2>;
        };
    };
};

3.3 isp相关配置

其中rkisp_vir0节点的remote-endpoint指向dphy1_out

&rkisp {
    status = "okay";
};

&rkisp_mmu {
    status = "okay";
};

&rkisp_vir0 {
    status = "okay";
    port {
        #address-cells = <1>;
        #size-cells = <0>;

        isp0_in: endpoint@0 {
            reg = <0>;
            remote-endpoint = <&dphy1_out>;
        };
    };
};

&rkisp_vir1 {
    status = "okay";

    port {
        reg = <0>;
        #address-cells = <1>;
        #size-cells = <0>;

        isp1_in: endpoint@0 {
            reg = <0>;
            remote-endpoint = <&mipi_lvds_sditf>;
        };
    };
};

&rkcif_mmu {
    status = "okay";
};

&rkcif {
    status = "okay";
};

四、软件相关目录

Linux Kernel-4.19
|-- arch/arm/boot/dts #DTS配置文件
|-- drivers/phy/rockchip
    |-- phy-rockchip-mipi-rx.c #mipi dphy驱动
    |-- phy-rockchip-csi2-dphy-common.h
    |-- phy-rockchip-csi2-dphy-hw.c
    |-- phy-rockchip-csi2-dphy.c
|-- drivers/media
    |-- platform/rockchip/cif #RKCIF驱动
    |-- platform/rockchip/isp #RKISP驱动
        |-- dev #包含 probe、异步注册、clock、pipeline、 iommu及media/v4l2 framework
        |-- capture #包含 mp/sp/rawwr的配置及 vb2,帧中断处理
        |-- dmarx #包含 rawrd的配置及 vb2,帧中断处理
        |-- isp_params #3A相关参数设置
        |-- isp_stats #3A相关统计
        |-- isp_mipi_luma #mipi数据亮度统计
        |-- regs #寄存器相关的读写操作
        |-- rkisp #isp subdev和entity注册
        |-- csi #csi subdev和mipi配置
        |-- bridge #bridge subdev,isp和ispp交互桥梁
    |-- platform/rockchip/ispp #rkispp驱动
        |-- dev #包含 probe、异步注册、clock、pipeline、 iommu及media/v4l2 framework
        |-- stream #包含 4路video输出的配置及 vb2,帧中断处理
        |-- rkispp #ispp subdev和entity注册
        |-- params #TNR/NR/SHP/FEC/ORB参数设置
        |-- stats #ORB统计信息

五、单目CAM-8MS1M/双目CAM-2MS2MF摄像头的使用

ROC-RK3568-PC已经配置好相应的dts，单目摄像头CAM-8MS1M和双目摄像头CAM-2MS2MF使用互斥，只需包含相应的dtsi文件即可使用单目摄像头CAM-8MS1M或双目摄像头CAM-2MS2MF

5.1 使用单目摄像头CAM-8MS1M

dts的配置默认使用单目摄像头：

diff --git a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts
index 7e2a8b2..14fa027 100755
--- a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts
+++ b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts
@@ -7,6 +7,15 @@
+#include "rk3568-firefly-roc-pc-cam-8ms1m.dtsi"
+//#include "rk3568-firefly-roc-pc-cam-2ms2m.dtsi"

5.2 使用双目摄像头CAM-2MS2MF

diff --git a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts
index 7e2a8b2..14fa027 100755
--- a/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts
+++ b/kernel/arch/arm64/boot/dts/rockchip/rk3568-firefly-roc-pc.dts
@@ -7,6 +7,15 @@
- #include "rk3568-firefly-roc-pc-cam-8ms1m.dtsi"
+//#include "rk3568-firefly-roc-pc-cam-8ms1m.dtsi"
- //#include "rk3568-firefly-roc-pc-cam-2ms2m.dtsi"
+ #include "rk3568-firefly-roc-pc-cam-2ms2m.dtsi"

六、Camera底层调试

使用v4l2-ctl抓取camera数据帧：

v4l2-ctl --verbose -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat='NV12' --stream-mmap=4 --set-selection=target=crop,flags=0,top=0,left=0,width=1920,height=1080 --stream-to=/data/out.yuv

把out.yuv文件拷贝出来通过ubuntu去查看：

ffplay -f rawvideo -video_size 1920x1080 -pix_fmt nv12 out.yuv

七、Android系统使用camera应用

Android系统使用camera的apk打开摄像头需要配置camera3_profiles*.xml，具体可参考Android SDK hardware/rockchip/camera/etc/camera目录下的文件

八、IQ文件

raw摄像头支持的iq文件路径external/camera_engine_rkaiq/iqfiles/isp21, 与以前不一样的地方是iq文件不再采用.xml的方式，而是采用.json的方式。虽有提供xml转json的工具, 但isp20的xml配置转换后也不适用isp21。

若使用raw摄像头sensor，请留意isp21目录所支持的iq文件。

好博客就要一起分享哦！分享海报

此处可发布评论

评论（0）展开评论

暂无评论，快来写一下吧

展开评论

您可能感兴趣的博客

奋斗吧 主页关注

167万+ 人气	865 博客	0 专栏
问答	1 粉丝	0 关注

写博客博客管理

他感兴趣的技术

Java HTML PHP python JavaScript Android MySQL C Linux 微信其他

最热博客更多

1、黑客入侵的常法 4944

2、黑马Mysql从入门到高级 2299

3、Windows怎么激活？Windows激活密钥分享 2278

4、类ChatGPT项目的部署与微调(上)：从LLaMA到Alpaca、Vicuna、BELLE 2040

5、掌握Conda环境迁移的几种方式，从此不在重复配环境 1969

6、分享200+个关于AI的网站 1929

7、小迪笔记 1669

8、Yolov8详解与实战 1430

9、八股文总结 1379

10、JavaScript/jQuery、HTML、CSS 构建 Web IM 远程及时聊天通信程序 1041