Ov5640 Camera Support in APQ8096 Open Embedded

Products Support DragonBoard 820c
#1

Hello,

I am trying ov5640 camera interface in APQ8096 with Open Embedded using kernel version 4.14.53.

Kernel : working/qualcomm/kernel.git - Qualcomm Landing Team kernel
Branch : release/qcomlt-4.14
Commit ID : 1d9f9c9a67bf9e3e8501095a87e3f05e2ae007e4

Media Entities and v4l2 devices are created and pipeline is also created successfully using below command for RDI and PIX interface.

RDI :

media-ctl -d /dev/media0 -l ‘“msm_csiphy1”:1->“msm_csid1”:0[1],“msm_csid1”:1->“msm_ispif1”:0[1],“msm_ispif1”:1->“msm_vfe0_rdi1”:0[1]’

media-ctl -d /dev/media0 -V ‘“ov5640 13-003c”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy1”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid1”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif1”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_rdi1”:0[fmt:UYVY2X8/1920x1080 field:none]’

PIX :

media-ctl -d /dev/media0 -l ‘“msm_csiphy1”:1->“msm_csid1”:0[1],“msm_csid1”:1->“msm_ispif1”:0[1],“msm_ispif1”:1->“msm_vfe0_pix”:0[1]’

media-ctl -d /dev/media0 -V ‘“ov5640 13-003c”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy1”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid1”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif1”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:1[fmt:UYVY1_5X8/1920x1080 field:none]’

But camera preview is not working and it gives error, for RDI it gives Unknown pixelformat error and then failed to preview, and in PIX it does not give pixelformat error but failed to preview.

So please let me know if i am missing something or anything wrong with command of creating pipeline and preview, Or i need to change anything in the ov5640 driver.

Below is some portion of error log when execute command of preview.

RDI :

DISPLAY=:0 gst-launch-1.0 v4l2src device=/dev/video1 ! ‘video/x-raw,format=UYVY,width=1920,height=1080,framerate=30/1’ ! glimagesink

root@dragonboard-820c:~#
‘video/x-raw,format=UYVY,width=1920,height=1080,framerate=30/1’ ! glimagesink ! ’
[ 1336.269131] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 1336.804086] qcom-camss a34000.camss: CSIPHY 3PH HW Version = 0x10000000
[ 1336.804510] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 1336.811143] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 1336.816531] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 1336.822426] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 1336.828270] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 1336.834272] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 1336.840263] qcom-camss a34000.camss: VFE HW Version = 0x70020000
Setting pipeline to PAUSED …
[ 1337.503481] qcom-camss a34000.camss: CSIPHY 3PH HW Version = 0x10000000
[ 1337.503888] qcom-camss a34000.camss: VFE HW Version = 0x70020000
Pipeline is live and does not need PREROLL …
[ 1337.512744] Unknown pixelformat 0x45454270
[ 1337.519183] ------------[ cut here ]------------
[ 1337.523256] WARNING: CPU: 2 PID: 2198 at /home/user/workspace/kernel-source/kernel/drivers/media/v4l2-core/v4l2-ioctl.c:1289 v48
[ 1337.528003] Modules linked in: arc4 rfcomm bnep venus_dec venus_enc q6afe_dai q6asm_dai q6routing q6adm q6asm q6afe q6dsp_common q6core rpme
[ 1337.556460] CPU: 2 PID: 2198 Comm: v4l2src0:src Not tainted 4.14.53+linaro #23
[ 1337.578690] Hardware name: Qualcomm Technologies, Inc. DB820c (DT)
[ 1337.585894] task: ffff8000e6870e00 task.stack: ffff0000128c8000
[ 1337.592404] PC is at v4l_enum_fmt+0x168/0x16d8
[ 1337.598132] LR is at v4l_enum_fmt+0x168/0x16d8
:
[ 1339.101780] bfe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 1339.109593] [] __sys_trace_return+0x0/0x4
[ 1339.117402] —[ end trace 232d54ead7683ec4 ]—
[ 1339.130955] qcom-camss a34000.camss: Cannot get CSI2 transmitter’s pixel clock
[ 1339.132441] qcom-camss a34000.camss: Output is running
[ 1339.137086] qcom-camss a34000.camss: Failed to enable vfe outputs
[ 1339.142205] ------------[ cut here ]------------
[ 1339.148348] WARNING: CPU: 3 PID: 2194 at /home/user/workspace/kernel-source/kernel/drivers/media/v4l2-core/videobuf2-core.c:8960
[ 1339.153038] Modules linked in: arc4 rfcomm bnep venus_dec venus_enc q6afe_dai q6asm_dai q6routing q6adm q6asm q6afe q6dsp_common q6core rpme
[ 1339.181928] CPU: 3 PID: 2194 Comm: gst-launch-1.0 Tainted: G W 4.14.53+linaro #23
[ 1339.204159] Hardware name: Qualcomm Technologies, Inc. DB820c (DT)
[ 1339.212750] task: ffff8000e6870000 task.stack: ffff00000a9e8000
[ 1339.219260] PC is at vb2_buffer_done+0x190/0x1d0
[ 1339.224989] LR is at vfe_flush_buffers+0xc4/0x118
[ 1339.229849] pc : [] lr : [] pstate: 800001c5
[ 1339.234452] sp : ffff00000a9ebaa0
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: [ 1339.241912] x29: ffff00000a9ebaa0 x28: ffff8000eea93580
[ 1339.250508] x27: ffff8000f8a33c00 x26: 0000000000000140
:
[ 1340.022561] [] __sys_trace_return+0x0/0x4
[ 1340.030370] —[ end trace 232d54ead7683ec6 ]—
src0:
poll error 1: Invalid argument (22)
Execution ended after 0:00:01.607946041
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
root@dragonboard-820c:~#

PIX :

DISPLAY=:0 gst-launch-1.0 v4l2src device=/dev/video3 ! ‘video/x-raw,format=NV12,width=1920,height=1080’ ! glimagesink

root@dragonboard-820c:~#
fe0_pix":0[1]'rd-820c:~# media-ctl -d /dev/media0 -l '“msm_csiphy1”:1->“msm_csid1”:0[1],“msm_csid1”:1->“msm_ispif1”:0[1],“msm_ispif1”:1->“msm_f
0x1080 field:none],“msm_vfe0_pix”:1[fmt:UYVY1_5X8/1920x1080 field:none]'1”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:0[fmt:UYVY2X8/190
root@dragonboard-820c:~# DISPLAY=:0 gst-launch-1.0 v4l2src device=/dev/video3 ! ‘video/x-raw,format=NV12,width=1920,height=1080’ ! glimagesink
[ 93.230400] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 93.230795] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 93.238665] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 93.779735] qcom-camss a34000.camss: CSIPHY 3PH HW Version = 0x10000000
[ 93.780148] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 93.786423] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 93.791876] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 93.797869] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 93.803656] qcom-camss a34000.camss: VFE HW Version = 0x70020000
Setting pipeline to PAUSED …
[ 94.452573] qcom-camss a34000.camss: CSIPHY 3PH HW Version = 0x10000000
[ 94.452987] qcom-camss a34000.camss: VFE HW Version = 0x70020000
Pipeline is live and does not need PREROLL …
Got context from element ‘sink’: gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"(GstGLDisplayX11)\ gldisplayx11-0";
Setting pipeline to PLAYING …
New clock: GstSystemClock
[ 94.480005] qcom-camss a34000.camss: Cannot get CSI2 transmitter’s pixel clock
[ 94.487086] qcom-camss a34000.camss: Output is running
[ 94.491953] qcom-camss a34000.camss: Failed to enable vfe outputs
[ 94.497189] ------------[ cut here ]------------
[ 94.503222] WARNING: CPU: 2 PID: 2212 at /home/user/workspace/kernel-source/kernel/drivers/media/v4l2-core/videobuf2-core.c:8960
[ 94.507913] Modules linked in: arc4 rfcomm bnep q6asm_dai q6routing q6afe_dai q6adm q6asm q6afe q6dsp_common q6core venus_enc venus_dec apre
[ 94.536803] CPU: 2 PID: 2212 Comm: gst-launch-1.0 Not tainted 4.14.53+linaro #23
[ 94.559033] Hardware name: Qualcomm Technologies, Inc. DB820c (DT)
[ 94.566584] task: ffff8000e8cf2a00 task.stack: ffff000012760000
[ 94.572919] PC is at vb2_buffer_done+0x190/0x1d0
[ 94.578650] LR is at vfe_flush_buffers+0xc4/0x118
[ 94.583507] pc : [] lr : [] pstate: 800001c5
[ 94.588111] sp : ffff000012763aa0
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: [ 94.595572] x29: ffff000012763aa0 x28: ffff8000e8c85080
[ 94.604169] x27: ffff8000e8d42d00 x26: 0000000000000140
:
[ 95.376221] [] __sys_trace_return+0x0/0x4
[ 95.384030] —[ end trace c9ed99675e01117e ]—
src0:
poll error 1: Invalid argument (22)
Execution ended after 0:00:00.016885521
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
root@dragonboard-820c:~#

#2

There is some work ongoing upstream to fix OV5640 camera MIPI camera driver. On DB410C I had to apply some patches to make it work, coul you please try the following kernel branch and let me know:
https://git.linaro.org/people/loic.poulain/linux.git/log/?h=qcomlt-4.14-d3-mezzanine

#3

Thanks for reply.

I have tried ov5640 driver from qcomlt-4.14-d3-mezzanine branch.

With this driver camera is working (Some Noise is observed) with PIX pipeline and also Capture and Video Recording is working.

But when i tried with RDI pipeline there is crash about Unknown pixelformat 0x45454270, still preview is working with this crash and same with Capture and Video Recording is working but crash is observed.

#4

I am also trying to enable ov5640 camera support in CSI0 port which uses the same CCI pins as CSI1 port. Camera is working with CSI1 port as mentioned in above reply.

In case of CSI0, probe is success and v4l2-ctl devices and media-entities are successfully created. But preview is not working and it gives below error:

root@dragonboard-820c:~#
! glimagesinkrd-820c:~# DISPLAY=:0 gst-launch-1.0 v4l2src device=/dev/video3 ! 'video/x-raw,format=NV12,width=1920,height=1080,framerate=15/1
[ 144.053018] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 144.054475] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 144.059515] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 144.549118] qcom-camss a34000.camss: Failed to power up pipeline: -22
[ 144.549975] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 144.555420] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 144.561263] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 144.567168] qcom-camss a34000.camss: VFE HW Version = 0x70020000
Setting pipeline to PAUSED …
[ 145.145710] qcom-camss a34000.camss: Failed to power up pipeline: -22
ERROR: Pipeline doesn’t want to pause.
Got context from element ‘sink’: gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"(GstGLDisplayX11)\ gldisplayx11-0";
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Could not open device ‘/dev/video3’ for reading and writing.
Additional debug info:
…/…/…/gst-plugins-good-1.12.4/sys/v4l2/v4l2_calls.c(626): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
system error: Invalid argument
Setting pipeline to NULL …
Freeing pipeline …
root@dragonboard-820c:~#

It seems like there is some problem with v4l2-subdev, because whichever video node is setting up using pipeline is disappeared after applying command of pipeline

Output of v4l2-ctl --list-device before creating pipeline:

root@dragonboard-820c:~# v4l2-ctl --list-devices
[ 44.736419] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 44.738465] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 44.741816] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 44.749244] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 44.753991] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 44.759964] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 44.765624] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 44.771833] qcom-camss a34000.camss: VFE HW Version = 0x70020000
Qualcomm Camera Subsystem (platform:a34000.camss):
/dev/video0
/dev/video1
/dev/video2
/dev/video3
/dev/video4
/dev/video5
/dev/video6
/dev/video7

Qualcomm Venus video decoder (platform:qcom-venus):
[ 44.798192] qcom-camss a34000.camss: VFE HW Version = 0x70020000
/dev/video8
/dev/video9

Output of v4l2-ctl --list-device after applying command of pipeline:

root@dragonboard-820c:~# v4l2-ctl --list-devices
[ 70.261082] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 70.261396] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 70.266429] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 70.761531] qcom-camss a34000.camss: Failed to power up pipeline: -22
[ 70.761819] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 70.767477] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 70.773369] qcom-camss a34000.camss: VFE HW Version = 0x70020000
[ 70.779319] qcom-camss a34000.camss: VFE HW Version = 0x70020000
Qualcomm Camera Subsystem (platform:a34000.camss):
/dev/video0
/dev/video1
/dev/video2
/dev/video4
/dev/video5
/dev/video6
/dev/video7

Qualcomm Venus video decoder (platform:qcom-venus):
[ 70.804625] qcom-camss a34000.camss: VFE HW Version = 0x70020000
/dev/video8
/dev/video9

root@dragonboard-820c:~#

Here /dev/video3 is missing which is used by PIX. So please let me know what is the issue here.

And for Camera support with CSI2 port, which uses second CCI bus. But from seeing device tree and this post it seems that there is not support added for second CCI. So is there any possibilities to add support for second CCI interface? Is there any major changes for that or we have to just add pinctrl for gpios?

Please let me know if there is any update on it.

#5

Which command do you apply.

CCI bus is basically a I2c subset, so you can have several slave sensors on the same CCI port, which mitigates the ‘problem’.

#6

Pipeline for PIX:

media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_pix”:0[1]’

media-ctl -d /dev/media0 -V ‘“ov5640 13-003c”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:1[fmt:UYVY1_5X8/1920x1080 field:none]’

Pipeline for RDI:

media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_rdi0”:0[1]’

media-ctl -d /dev/media0 -V ‘“ov5640 13-003c”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_rdi0”:0[fmt:UYVY2X8/1920x1080 field:none]’

Yes, in our configuration CSI0 and CSI1 port is connected with CCI_I2C0_SDA and CCI_I2C0_SCL (CCI0),and probe is success in both CSI0 and CSI1 port. While CSI2 port is connected to CCI_I2C1_SDA and CCI_I2C1_SCL (CCI1),and probe is failed in CSI2 port.

#7

I suspect that you are using two identical camera modules. They are normally configured to respond to the same I2C address (or CCI address), hence they are both responding to the initial device probe at the same address. you need to configure one of the cameras with a different I2C address, if this is not possible, then they need to live on different I2C (CCI) busses.

Can you change the I2C address on one of the modules?

-Lawrence-

#8

Yes, i also tried to change I2C address for one module.

But as of now i am only enabling one CSI port at a time. So when i try to enable camera with CSI0 port other two ports are disabled in device tree. And yes i am trying same ov5640 in all three ports but only one port is enabled at a time.