Imx219 Sony image sensor with dragonboard

Hi,

We are trying to interface a imx219 camera module with the Dargonboard410c (Kernel Version:4.14.96).

We got error: “Failed to allocate required memory”

sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_rdi0”:0[1]’

sudo media-ctl -d /dev/media0 -V '"imx219 4-0010":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_csiphy0":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_csid0":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_ispif0":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_vfe0_rdi0":0[fmt:SBGGR8_1X8/1920x1080 field:none]'

linaro@linaro-alip:~$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-bayer,format=bggr,framerate=30/1,width=1920,height=1080 ! bayer2rgb ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
gstv4l2src.c(658): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.467249103
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...   

[  643.681303] ------------[ cut here ]------------
[  643.681357] WARNING: CPU: 2 PID: 3870 at drivers/media/v4l2-core/videobuf2-core.c:1347 vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[  643.684989] Modules linked in: rfcomm aes_neon_blk bnep arc4 wcn36xx mac80211 btqcomsmd btqca bluetooth cfg80211 ecdh_generic qcom_q6v5_pil qrtr_smd venus_dec venus_enc snd_soc_hdmi_codec imx219 v4l2_common qcom_wcnss_pil qcom_common qcom_glink_smem msm qcom_sysmon remoteproc qmi_helpers adv7511 drm_kms_helper qcom_camss venus_core videobuf2_dma_sg drm v4l2_mem2mem v4l2_fwnode mdt_loader videobuf2_memops leds_qcom_lpg snd_soc_lpass_apq8016 videobuf2_v4l2 snd_soc_lpass_cpu snd_soc_msm8916_digital rtc_pm8xxx videobuf2_core snd_soc_lpass_platform snd_soc_apq8016_sbc snd_soc_msm8916_analog msm_rng videodev rng_core media i2c_qcom_cci qrtr crc32_ce rpmsg_char rmtfs_mem ip_tables x_tables
[  643.735060] CPU: 2 PID: 3870 Comm: v4l2src0:src Not tainted 4.14.96 #1
[  643.757287] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[  643.763886] task: ffff800033616200 task.stack: ffff000009598000
[  643.770758] PC is at vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[  643.776398] LR is at vb2_start_streaming+0xa0/0x1a0 [videobuf2_core]
[  643.782982] pc : [<ffff000000c11cd8>] lr : [<ffff000000c11c68>] pstate: 60000145
[  643.789405] sp : ffff00000959bb80
[  643.796779] x29: ffff00000959bb80 x28: ffff800035598b00 
[  643.799993] x27: 0000000000000001 x26: ffff000000b81448 
[  643.805375] x25: ffff80003a737d00 x24: ffff00000959bd30 
[  643.810671] x23: 0000000000000000 x22: ffff800035424580 
[  643.815965] x21: ffff8000354244e0 x20: ffff800035424350 
[  643.821261] x19: 00000000ffffffe0 x18: 000000000000001f 
[  643.826556] x17: 0000ffff90d94eb8 x16: ffff000008236060 
[  643.831852] x15: 0000000000000002 x14: 000000000000001f 
[  643.837146] x13: 0000000000000000 x12: 0000000000000000 
[  643.842441] x11: 0000000000000000 x10: 0000000000000000 
[  643.847736] x9 : 0000000000000000 x8 : ffff80003af8c188 
[  643.853032] x7 : 0000000000000000 x6 : 000000000000003f 
[  643.858329] x5 : 0000000000000000 x4 : 0000000000000000 
[  643.863623] x3 : ffff800033616200 x2 : 0000000000000000 
[  643.868917] x1 : ffff000000c18400 x0 : 0000000000000004 
[  643.874222] Call trace:
[  643.879506] Exception stack(0xffff00000959ba40 to 0xffff00000959bb80)
[  643.881682] ba40: 0000000000000004 ffff000000c18400 0000000000000000 ffff800033616200
[  643.888280] ba60: 0000000000000000 0000000000000000 000000000000003f 0000000000000000
[  643.896092] ba80: ffff80003af8c188 0000000000000000 0000000000000000 0000000000000000
[  643.903905] baa0: 0000000000000000 0000000000000000 000000000000001f 0000000000000002
[  643.911717] bac0: ffff000008236060 0000ffff90d94eb8 000000000000001f 00000000ffffffe0
[  643.919530] bae0: ffff800035424350 ffff8000354244e0 ffff800035424580 0000000000000000
[  643.927343] bb00: ffff00000959bd30 ffff80003a737d00 ffff000000b81448 0000000000000001
[  643.935155] bb20: ffff800035598b00 ffff00000959bb80 ffff000000c11c68 ffff00000959bb80
[  643.942968] bb40: ffff000000c11cd8 0000000060000145 0000000000000000 ffff800035424350
[  643.950781] bb60: 0000ffffffffffff ffff800035424580 ffff00000959bb80 ffff000000c11cd8
[  643.958606] [<ffff000000c11cd8>] vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[  643.966415] [<ffff000000c13ecc>] vb2_core_streamon+0x134/0x1b8 [videobuf2_core]
[  643.973878] [<ffff000000c54264>] vb2_streamon+0x3c/0x80 [videobuf2_v4l2]
[  643.980993] [<ffff000000c542f0>] vb2_ioctl_streamon+0x48/0x58 [videobuf2_v4l2]
[  643.987991] [<ffff000000b5db70>] v4l_streamon+0x20/0x28 [videodev]
[  643.995018] [<ffff000000b607b0>] __video_do_ioctl+0x2a8/0x2e0 [videodev]
[  644.001179] [<ffff000000b6024c>] video_usercopy+0x1e4/0x480 [videodev]
[  644.008036] [<ffff000000b604f8>] video_ioctl2+0x10/0x20 [videodev]
[  644.014372] [<ffff000000b5c64c>] v4l2_ioctl+0x7c/0x128 [videodev]
[  644.020487] [<ffff000008235854>] do_vfs_ioctl+0xa4/0x8b0
[  644.026646] [<ffff0000082360d8>] SyS_ioctl+0x78/0x98
[  644.032025] Exception stack(0xffff00000959bec0 to 0xffff00000959c000)
[  644.036977] bec0: 0000000000000014 0000000040045612 0000aaaae8271450 0000ffff90c21eb0
[  644.043314] bee0: 0000000000000000 000000000000017f 0000000000000000 0000000000000001
[  644.051127] bf00: 000000000000001d 0000000000000001 0000000000000000 0000000000000000
[  644.058940] bf20: 0000000000000000 0000000000000000 000000000000001f 0000000000000002
[  644.066753] bf40: 0000ffff90e908c8 0000ffff90d94eb8 000000000000001f 0000000000000000
[  644.074564] bf60: 0000aaaae828aed0 0000aaaae8271440 0000ffff90701000 0000ffff90702000
[  644.082378] bf80: 0000000000000002 0000aaaae8271440 0000ffff7c00daf0 0000000000000001
[  644.090196] bfa0: 0000aaaae80aac80 0000ffff8d4c35d0 0000ffff906c75cc 0000ffff8d4c35c0
[  644.098003] bfc0: 0000ffff90c21ebc 0000000060000000 0000000000000014 000000000000001d
[  644.105814] bfe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  644.113628] [<ffff000008083ac0>] el0_svc_naked+0x34/0x38
[  644.121435] ---[ end trace 50c5407b420c97a1 ]---

linaro@linaro-alip:~$ yavta -B capture-mplane -c10 -I -n 4 --requeue-last -f SBGGR8 -s 1920x1080 /dev/video0 -F
Device /dev/video0 opened.
Device Qualcomm Camera Subsystem' on platform:1b0ac00.camss’ (driver ‘qcom-camss’) supports video, capture, with mplanes.
Video format set: SBGGR8 (31384142) 1920x1080 field none, 1 planes:

  • Stride 1920, buffer size 2073600
    Video format: SBGGR8 (31384142) 1920x1080 field none, 1 planes:
  • Stride 1920, buffer size 2073600
    4 buffers requested.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 0/0 mapped at address 0xffffa16ba000.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 1/0 mapped at address 0xffffa14bf000.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 2/0 mapped at address 0xffffa12c4000.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 3/0 mapped at address 0xffffa10c9000.
    Unable to start streaming: Broken pipe (32).
    4 buffers released.

[ 794.759234] ------------[ cut here ]------------
[ 794.759290] WARNING: CPU: 0 PID: 3894 at drivers/media/v4l2-core/videobuf2-core.c:1347 vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[ 794.762916] Modules linked in: rfcomm aes_neon_blk bnep arc4 wcn36xx mac80211 btqcomsmd btqca bluetooth cfg80211 ecdh_generic qcom_q6v5_pil qrtr_smd venus_dec venus_enc snd_soc_hdmi_codec imx219 v4l2_common qcom_wcnss_pil qcom_common qcom_glink_smem msm qcom_sysmon remoteproc qmi_helpers adv7511 drm_kms_helper qcom_camss venus_core videobuf2_dma_sg drm v4l2_mem2mem v4l2_fwnode mdt_loader videobuf2_memops leds_qcom_lpg snd_soc_lpass_apq8016 videobuf2_v4l2 snd_soc_lpass_cpu snd_soc_msm8916_digital rtc_pm8xxx videobuf2_core snd_soc_lpass_platform snd_soc_apq8016_sbc snd_soc_msm8916_analog msm_rng videodev rng_core media i2c_qcom_cci qrtr crc32_ce rpmsg_char rmtfs_mem ip_tables x_tables
[ 794.812992] CPU: 0 PID: 3894 Comm: yavta Tainted: G W 4.14.96 #1
[ 794.835220] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[ 794.842510] task: ffff80002a85aa00 task.stack: ffff00000f968000
[ 794.849296] PC is at vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[ 794.854935] LR is at vb2_start_streaming+0xa0/0x1a0 [videobuf2_core]
[ 794.861521] pc : [] lr : [] pstate: 60000145
[ 794.867944] sp : ffff00000f96bb80
[ 794.875317] x29: ffff00000f96bb80 x28: ffff800035f69500
[ 794.878532] x27: 0000000000000001 x26: ffff000000b81448
[ 794.883915] x25: ffff800035f69700 x24: ffff00000f96bd30
[ 794.889211] x23: 0000000000000000 x22: ffff800035424580
[ 794.894505] x21: ffff8000354244e0 x20: ffff800035424350
[ 794.899800] x19: 00000000ffffffe0 x18: 000000000000021f
[ 794.905095] x17: 0000ffffa19abeb0 x16: ffff000008236060
[ 794.910390] x15: 00000000000002b0 x14: 0000000000000001
[ 794.915685] x13: 0000ffffa10c9000 x12: 0000000000000000
[ 794.920979] x11: 0000000000000000 x10: 0000000000000000
[ 794.926275] x9 : 0000000000000000 x8 : ffff8000335d4088
[ 794.931571] x7 : 0000000000000000 x6 : 000000000000003f
[ 794.936865] x5 : 0000000000000000 x4 : 0000000000000000
[ 794.942161] x3 : ffff80002a85aa00 x2 : 0000000000000000
[ 794.947455] x1 : ffff000000c18400 x0 : 0000000000000004
[ 794.952752] Call trace:
[ 794.958044] Exception stack(0xffff00000f96ba40 to 0xffff00000f96bb80)
[ 794.960220] ba40: 0000000000000004 ffff000000c18400 0000000000000000 ffff80002a85aa00
[ 794.966818] ba60: 0000000000000000 0000000000000000 000000000000003f 0000000000000000
[ 794.974630] ba80: ffff8000335d4088 0000000000000000 0000000000000000 0000000000000000
[ 794.982444] baa0: 0000000000000000 0000ffffa10c9000 0000000000000001 00000000000002b0
[ 794.990257] bac0: ffff000008236060 0000ffffa19abeb0 000000000000021f 00000000ffffffe0
[ 794.998069] bae0: ffff800035424350 ffff8000354244e0 ffff800035424580 0000000000000000
[ 795.005881] bb00: ffff00000f96bd30 ffff800035f69700 ffff000000b81448 0000000000000001
[ 795.013694] bb20: ffff800035f69500 ffff00000f96bb80 ffff000000c11c68 ffff00000f96bb80
[ 795.021506] bb40: ffff000000c11cd8 0000000060000145 0000000000000000 ffff800035424350
[ 795.029319] bb60: 0000ffffffffffff ffff800035424580 ffff00000f96bb80 ffff000000c11cd8
[ 795.037147] [] vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[ 795.044956] [] vb2_core_streamon+0x134/0x1b8 [videobuf2_core]
[ 795.052418] [] vb2_streamon+0x3c/0x80 [videobuf2_v4l2]
[ 795.059534] [] vb2_ioctl_streamon+0x48/0x58 [videobuf2_v4l2]
[ 795.066531] [] v4l_streamon+0x20/0x28 [videodev]
[ 795.073561] [] __video_do_ioctl+0x2a8/0x2e0 [videodev]
[ 795.079727] [] video_usercopy+0x1e4/0x480 [videodev]
[ 795.086585] [] video_ioctl2+0x10/0x20 [videodev]
[ 795.092915] [] v4l2_ioctl+0x7c/0x128 [videodev]
[ 795.099027] [] do_vfs_ioctl+0xa4/0x8b0
[ 795.105185] [] SyS_ioctl+0x78/0x98
[ 795.110565] Exception stack(0xffff00000f96bec0 to 0xffff00000f96c000)
[ 795.115514] bec0: 0000000000000003 0000000040045612 0000fffff91545a4 0000000000000000
[ 795.121855] bee0: 0000ffffa12c3400 0000000000000004 0000000000000000 000000000000000a
[ 795.129667] bf00: 000000000000001d ffffff80ffffffc8 0000000000000000 0000000000000000
[ 795.137478] bf20: 0000000000000000 0000ffffa10c9000 0000000000000001 00000000000002b0
[ 795.145291] bf40: 0000aaaab57e8118 0000ffffa19abeb0 000000000000021f 0000aaaab57e7000
[ 795.153104] bf60: 0000000000000001 0000fffff91547e8 0000aaaae551182c 0000000000000001
[ 795.160915] bf80: 0000000000000000 0000aaaab57d69c8 0000000000000000 00000000001fa400
[ 795.168728] bfa0: 00000000ffffffff 0000fffff9154570 0000aaaab57d4e68 0000fffff9154570
[ 795.176541] bfc0: 0000ffffa19abebc 0000000060000000 0000000000000003 000000000000001d
[ 795.184353] bfe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 795.192167] [] el0_svc_naked+0x34/0x38
[ 795.199974] —[ end trace 50c5407b420c97a2 ]—

We checked the clock and the MIPI lines, We can see data transfer
Thanks,
Osama

I’m also getting similar issue with Raspberry pi camera v1.3 (OV5647) connected using MIPI adapter mezzanine. While using yavta, I’m also getting VFE timeouts in dmesg:

[  622.517402] qcom-camss 1b0ac00.camss: VFE sof timeout
[  623.029290] qcom-camss 1b0ac00.camss: VFE reg update timeout

Does this mean the frame is not received properly from camera sensor? SOF timeout looks suspicious as it corresponds to Start of Frame.

Tagging @Loic if he can help with this!

Thanks,
Mani

Pipeline is not able to start stream.
Could you attach ouput of media-ctl -p after setting the pipeline ?
Which sensor driver are you running? a custom one? a downstream one? any link?

Also please note that the camera sensor driver should support pixelclock v4l2 ctrl in order to work with camss.

Custom driver based on ov5640.c

Media controller API version 4.14.96

Media device information
------------------------
driver          qcom-camss
model           Qualcomm Camera Subsystem
serial          
bus info        
hw revision     0x0
driver version  4.14.96

Device topology
- entity 1: msm_csiphy0 (2 pads, 3 links)
            type Node subtype V4L flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "imx219 4-0010":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "msm_csid0":0 [ENABLED]
		-> "msm_csid1":0 []

- entity 4: msm_csiphy1 (2 pads, 2 links)
            type Node subtype V4L flags 0
            device node name /dev/v4l-subdev1
	pad0: Sink
	pad1: Source
		-> "msm_csid0":0 []
		-> "msm_csid1":0 []

- entity 7: msm_csid0 (2 pads, 4 links)
            type Node subtype V4L flags 0
            device node name /dev/v4l-subdev2
	pad0: Sink
		<- "msm_csiphy0":1 [ENABLED]
		<- "msm_csiphy1":1 []
	pad1: Source
		-> "msm_ispif0":0 [ENABLED]
		-> "msm_ispif1":0 []

- entity 10: msm_csid1 (2 pads, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev3
	pad0: Sink
		<- "msm_csiphy0":1 []
		<- "msm_csiphy1":1 []
	pad1: Source
		-> "msm_ispif0":0 []
		-> "msm_ispif1":0 []

- entity 13: msm_ispif0 (2 pads, 6 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev4
	pad0: Sink
		<- "msm_csid0":1 [ENABLED]
		<- "msm_csid1":1 []
	pad1: Source
		-> "msm_vfe0_rdi0":0 [ENABLED]
		-> "msm_vfe0_rdi1":0 []
		-> "msm_vfe0_rdi2":0 []
		-> "msm_vfe0_pix":0 []

- entity 16: msm_ispif1 (2 pads, 6 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev5
	pad0: Sink
		<- "msm_csid0":1 []
		<- "msm_csid1":1 []
	pad1: Source
		-> "msm_vfe0_rdi0":0 []
		-> "msm_vfe0_rdi1":0 []
		-> "msm_vfe0_rdi2":0 []
		-> "msm_vfe0_pix":0 []

- entity 19: msm_vfe0_rdi0 (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR8_1X8/1920x1080 field:none colorspace:srgb]
		<- "msm_ispif0":1 [ENABLED]
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:SBGGR8_1X8/1920x1080 field:none colorspace:srgb]
		-> "msm_vfe0_video0":0 [ENABLED,IMMUTABLE]

- entity 22: msm_vfe0_video0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		<- "msm_vfe0_rdi0":1 [ENABLED,IMMUTABLE]

- entity 28: msm_vfe0_rdi1 (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev7
	pad0: Sink
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		<- "msm_ispif0":1 []
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		-> "msm_vfe0_video1":0 [ENABLED,IMMUTABLE]

- entity 31: msm_vfe0_video1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Sink
		<- "msm_vfe0_rdi1":1 [ENABLED,IMMUTABLE]

- entity 37: msm_vfe0_rdi2 (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev8
	pad0: Sink
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		<- "msm_ispif0":1 []
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		-> "msm_vfe0_video2":0 [ENABLED,IMMUTABLE]

- entity 40: msm_vfe0_video2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Sink
		<- "msm_vfe0_rdi2":1 [ENABLED,IMMUTABLE]

- entity 46: msm_vfe0_pix (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev9
	pad0: Sink
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb
		 compose.bounds:(0,0)/1920x1080
		 compose:(0,0)/1920x1080]
		<- "msm_ispif0":1 []
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb
		 crop.bounds:(0,0)/1920x1080
		 crop:(0,0)/1920x1080]
		-> "msm_vfe0_video3":0 [ENABLED,IMMUTABLE]

- entity 49: msm_vfe0_video3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Sink
		<- "msm_vfe0_pix":1 [ENABLED,IMMUTABLE]

- entity 87: imx219 4-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev10
	pad0: Source
		[fmt:unknown/0x0@1/30]
		-> "msm_csiphy0":0 [ENABLED,IMMUTABLE]

Unknown format? I think it’s a problem.

I suggest you to add some debugging code in sensor and camss (driver/media/platform/qcom/camss vide/vfe…) source in order to highlight issues in the pipeline, mainly in streamon/set_power/set_fmt code…

Also, camss requests some controls to be implemented, like ‘pixel_clock’ (e.g. [1]), check you correctly implemented them.

https://git.linaro.org/people/loic.poulain/linux.git/commit/?h=qcomlt-4.14-ov5640&id=c3fb3b5b6ea78247a5d485cd620f8d5d1b4ab71c