Gstreamer 1.12 pipeline config for camera issue: _pix vs _rdi0

Products Support DragonBoard410c
#1

Hello,

I just built kernel 17.06.1 (build instruction / links)
with small changes in /arch/arm64/boot/dts/qcom/apq8016-sbc.dtsi
and add of rootfs to existing alip in order to include ov5640 camera driver. I compiled gstreamer 1.12 as instructed.

Based on instructions here, I setup gstreamer video pipeline in the following way:

sudo media-ctl -d /dev/media1 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_pix":0[1]'
sudo media-ctl -d /dev/media1 -V '"ov5640 3-0078":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_csiphy0":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_csid0":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_ispif0":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_vfe0_pix":0[fmt:UYVY8_2X8/2592x1944 field:none]'

and tried acquiring an image with following command:
gst-launch-1.0 v4l2src device=/dev/video3 num-buffers=1 ! 'video/x-raw,format=UYVY,width=2592,height=1944,framerate=10/1' ! jpegenc ! filesink location=image01_5MP.jpg

I can’t get any image with that and the output message is the following:
Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error. Additional debug info: gstbasesrc.c(2939): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: streaming stopped, reason not-negotiated (-4) ERROR: pipeline doesn't want to preroll. Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...

I see no particular problem when I print the pipeline with following command:
sudo media-ctl -d /dev/media1 -p

I get this result as part of the printed elements:
- entity 49: msm_vfe0_video3 (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video3 pad0: Sink <- "msm_vfe0_pix":1 [ENABLED,IMMUTABLE]

However, using msm_vfe0_rdi0 instead of msm_vfe0_pix I can get an image out of the pipeline.
In details, I used following commands:

sudo media-ctl -d /dev/media1 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_rdi0":0[1]'
sudo media-ctl -d /dev/media1 -V '"ov5640 3-0078":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_csiphy0":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_csid0":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_ispif0":0[fmt:UYVY8_2X8/2592x1944 field:none],"msm_vfe0_rdi0":0[fmt:UYVY8_2X8/2592x1944 field:none]'

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! 'video/x-raw,format=UYVY,width=2592,height=1944,framerate=10/1' ! jpegenc ! filesink location=image01_5MP.jpg

So, why does he bother us with something that works at the end of the day ? :slight_smile:
The point is that we have other issues and I don’t know if they are linked to this _pix - _rdi0, /dev/video0 - /dev/video3 choice…

Thanks in advance for your thoughts and comments!
Guerric

#2

Hi,

When using the pix interface you have to use NV12/NV21 output format - the other option (NV16/NV61) is not supported by jpegenc.

From [1]:
Format configuration for NV12/NV21 output (the format on msm_vfe0_pix source pad - for NV12/NV21 must be UYVY8_1_5_X8 - must be set explicitly):

sudo media-ctl -d /dev/media1 -V ‘“ov5645 1-0076”:0[fmt:UYVY8_2X8/1280x960 field:none],“msm_csiphy0”:0[fmt:UYVY8_2X8/1280x960 field:none],“msm_csid0”:0[fmt:UYVY8_2X8/1280x960 field:none],“msm_ispif0”:0[fmt:UYVY8_2X8/1280x960 field:none],“msm_vfe0_pix”:0[fmt:UYVY8_2X8/1280x960 field:none],“msm_vfe0_pix”:1[fmt:UYVY8_1_5X8/1280x960 field:none]’

Just set your resolution and sensor entity name in the above pipeline.

Then ask for NV12 format (not UYVY):

gst-launch-1.0 v4l2src device=/dev/video3 num-buffers=1 ! ‘video/x-raw,format=NV12,width=2592,height=1944,framerate=10/1’ ! jpegenc ! filesink location=image01_5MP.jpg

[1] https://github.com/96boards/documentation/blob/master/ConsumerEdition/DragonBoard-410c/Guides/CameraModule.md

#3

Thanks Todor for your fast reaction.

So if I understand well, use of RDI interface of VFE is completely valid for capturing pictures (with jpegenc).
However, if I want to enjoy Scaling, Croping or Format Conversion, I need PIX interface (with NV12 or NV16 format - if supported by the sink -). And NV12 - NV16 conversion is necessary for h264 encode.

I confirm I’m now able to acquire pictures with both RDI or PIX.
Thanks ! This is clear!

#4

hi @todortomov

I have been trying to perform camera preview using PIX interface NV12 format :

I am able to capture image but for preview I faced pipeline errors as i did :

sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_pix”:0[1]’

sudo media-ctl -d /dev/media0 -V ‘“ov5640 5-003c”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:1[fmt:UYVY1_5X8/1920x1080 field:none]’

img capture :
gst-launch-1.0 v4l2src device=/dev/video3 num-buffers=1 ! ‘video/x-raw,format=NV12,width=1280,height=960,framerate=30/1’ ! jpegenc ! filesink location=image02.jpg

preview :
gst-launch-1.0 v4l2src device=/dev/video3 ! ‘video/x-raw,format=NV12,width=1920,height=1080’ ! glimagesink

Any suggestions to debug this…

#5

Can you indicate which error is reported

#6

Hi,

The error was :

Unable to setup formats: Invalid argument (22)
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.001128906
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

But it was solved by adding frame rate as :

gst-launch-1.0 -e v4l2src device=/dev/video3 ! video/x-raw,format=NV12,width=1280,height=960,framerate=30/1 ! glimagesink

Thanks for the responce :slight_smile:

#7

Interesting, thanks for the solution