RAW10 Bayer format dump into memory

[jox]

Hi,
I am trying to dump SBGGR10 directly into memory via Rdi interface.

sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_rdi0”:0[1]’

sudo media-ctl -d /dev/media0 -V '“imx230 3-0010”:0[fmt:SBGGR10_1X10/2136x1202 field:none],“msm_csiphy0”:0[fmt:SBGGR10_1X10/2136x1202 field:none],“msm_csid0”:0[fmt:SBGGR10_1X10/2136x1202 field:none],“msm_ispif0”:0[fmt:SBGGR10_1X10/2136x1202 field:none],“msm_vfe0_rdi0”:0[fmt:SBGGR10_1X10/2136x1202 field:none]

These all got succeeded.

Then I launched gstreamer to start streaming:

gst-launch-1.0 v4l2src ! ‘video/x-bayer,format=bggr,framerate=30/1,width=2136,height=1202’ ! identity silent=false ! filesink location=test.raw

But I didn’t get any start_stream (v4l2_subdev_video_ops) call in any of the modules in pipeline.

I modified “v4l2src” plugin for SBGGR10 support.

Why start stream is not getting called even after linking the pipeline successfully? Please help.

Best Regards,
Jox

[Mathieu_Adok]

Hi jox,

I just saw your post on my topic.

Do you have any warn/error/logs you could share to us ? Also, what did you modify in the v4l2src plugin ?

Regards,
Mathieu

[jox]

Hi Mathieu,
The changes done in gstreamer side is to enable SBGGR10. That is same as what given in this link http://developer.ridgerun.com/wiki/index.php?title=Compile_gstreamer_on_tegra_X1_and_X2

After issuing gst-launch i am getting below error logs and start stream is not happening.

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
gstv4l2src.c(651): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.014894921
Setting pipeline to PAUSED …
Setting pipeline to READY …

Regards,
Jox

[Mathieu_Adok]

@jox

Hi Mathieu,
The changes done in gstreamer side is to enable SBGGR10. That is same as what given in this link http://developer.ridgerun.com/wiki/index.php?title=Compile_gstreamer_on_tegra_X1_and_X2

After issuing gst-launch i am getting below error logs and start stream is not happening.

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
gstv4l2src.c(651): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.014894921
Setting pipeline to PAUSED …
Setting pipeline to READY …

Regards,
Jox

Okay so you don’t have the same error as me. Maybe it could come from your version of GStreamer !

Could you give a try to yavta, as Todor stated in my own topic ? Just git clone the repository, make it and run it with a command line. Please, share the output to us.

Regards,
Mathieu

[jox]

Hi all,
Now I am able to output frames from the sensor. But the frames does not seems proper and getting below errors in the logs.

[ 1134.728200] imx230 3-0010: start stream success
[ 1135.349495] arm_smmu_context_fault: 47811 callbacks suppressed
[ 1135.349508] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0xffff5000, fsynr=0x33, cb=0
[ 1135.354303] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x00093080, fsynr=0x31, cb=0
[ 1135.379312] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x0017f820, fsynr=0x31, cb=0
[ 1135.379402] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x00182520, fsynr=0x31, cb=0
[ 1135.387961] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x0029e960, fsynr=0x31, cb=0
[ 1135.412579] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x003811e0, fsynr=0x31, cb=0
[ 1135.412697] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x00383760, fsynr=0x31, cb=0
[ 1135.421274] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x004964a0, fsynr=0x31, cb=0
[ 1135.445821] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x005804e0, fsynr=0x31, cb=0
[ 1135.445884] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x00582920, fsynr=0x31, cb=0
[ 1140.349484] arm_smmu_context_fault: 50510 callbacks suppressed
[ 1140.349499] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01a64800, fsynr=0x31, cb=0
[ 1140.368425] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01a8af40, fsynr=0x31, cb=0
[ 1140.368535] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01a8dc40, fsynr=0x31, cb=0
[ 1140.377071] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01ba9260, fsynr=0x31, cb=0
[ 1140.401677] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01c8ef00, fsynr=0x31, cb=0
[ 1140.401772] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01c91c00, fsynr=0x31, cb=0
[ 1140.410292] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01daf680, fsynr=0x31, cb=0
[ 1140.434956] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01e90a60, fsynr=0x31, cb=0
[ 1140.435101] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01e94c40, fsynr=0x31, cb=0
[ 1140.443642] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x01fb09c0, fsynr=0x31, cb=0

Any idea/suggestions on what went wrong here?

Best Regards,
Jox

[todortomov]

Hi Jox,

I suppose that what went wrong is the buffer size calculation and the hardware is trying to write outside of the video buffers.
Please try yavta and see what is the result - as it supports SBGGR10P format and we can see whether the error will appear again or not.

[jox]

Hi Todor,
Thanks for the reply. I was able to dump SBGGR10P from csid TG using yavta in 410c platform.

But when i am trying from 820c platform. it seems like no frames coming out. I noticed the CSID HW version is different in both platform.
Does this make any difference in commands used to enable CSID TG?

I used below command in both platforms.

./yavta --no-query -w ‘0x009f0903 1’ /dev/v4l-subdev2

and configured the pipeline for dumping SBGGR10P.

Best Regards,
Jox

[todortomov]

@jox

But when i am trying from 820c platform. it seems like no frames coming out. I noticed the CSID HW version is different in both platform.

Does this make any difference in commands used to enable CSID TG?

No.

@jox

I used below command in both platforms.

./yavta --no-query -w ‘0x009f0903 1’ /dev/v4l-subdev2

The subdev node could be different between the two platforms, please check that you configure the correct one.

[jox]

@todortomov

v4l-subdev2

Hi Todor,
Thanks for the suggestion. You are right, the subdev is “v4l-subdev3”. I am able to output 2136x1202 form TG with out memory related error logs (pattern seems diagonally aligned while checked using 7yuv tool, while for 1920x1080 it is horizontally aligned).

In both frames dumps from imx230 sensor and TG, X resolution seems to be full. But the Y resolution is missing many lines (This i observed for all the resolutions dumped). I am trying to debug on this data loss.
@todortomov @Loic Any thoughts on this will be really helpful.

Best Regards,
Jox

[todortomov]

Hello Jox,

Good that you have progress.

I’m not sure that I understand fully the “X / Y resolution” problem. Could you explain more, or attach example frames to illustrate the problem (along with the pipelines used to capture them if they differ from these already posted)?

[jox]

Hi Todor,
I have attached some dumps , taken from imx230 and TG in below link. Please check it.

It will be great if you can share/suggest any open source tool or code that will be useful to analyze SBGGR10 format. Currently i am using a trial version of 7yuv tool, which support only 16 bit bayer formats.

Best Regards,
Jox

[todortomov]

Unfortunately I cannot recommend convertor for raw format.

It seems that the video frame does not start and/or end with the buffer which is saved. It looks again like a problem with calculating the buffer size.
Which application did you use to do the streaming?

[jox]

Hi Todor,
I am making use of gstreamer for start streaming and dump into memory. Below is the command.

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! ‘video/x-bayer,format=bggr,width=2136,height=1202’ ! identity silent=false ! filesink location=test.bayer

Best Regards,
Jox

[todortomov]

I would again recommend to try yavta and check the buffer captured with it. The command should be something like:
yavta -B capture-mplane -c10 -I -n 5 --requeue-last -f SBGGR10P -s 2136x1202 /dev/video0 -F

Best regards,
Todor

[jox]

Hi Todor,
I tried with yavta. The frames are having the same behavior. One thing i noticed that the size of the dump is lesser than X * Y * 1.25 bytes.

While i dump using gstreamer, the size is X * Y bytes. Ideally the size should be X * Y *1.25 bytes. Is it an issue with 10 BPP support?

Best Regards,
Jox

[todortomov]

@jox

I tried with yavta. The frames are having the same behavior. One thing i noticed that the size of the dump is lesser than X * Y * 1.25 bytes.

Could you please provide again the pipelines and yavta command lines which you use. Do you see any errors in the kernel log when you run these commands? Could you again attach the dumped files?

@jox

Is it an issue with 10 BPP support?

I have tested the 10-bit raw support when developing it and it worked correctly.

[jox]

Hi @todortomov
I was trying with below command:

yavta -B capture-mplane -c1 -I -n 5 --requeue-last -f SBGGR10 -s 2136x1202 /dev/video0 -F

But when i tried with below command the size seems to be almost close. Please check the attached dump.

yavta -B capture-mplane -c1 -I -n 5 --requeue-last -f SBGGR10P -s 2136x1202 /dev/video0 -F

I am also getting below error while dumping.

[ 2114.219481] imx230 3-0010: start stream success
[ 2114.674490] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0xfff11000, fsynr=0x33, cb=0
[ 2114.674585] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0xfff12040, fsynr=0x33, cb=0
[ 2114.683093] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0xfffa2ee0, fsynr=0x33, cb=0
[ 2114.704291] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0xfffe0920, fsynr=0x33, cb=0
[ 2114.704374] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0xfffe1be0, fsynr=0x33, cb=0
[ 2114.712888] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x0005f7a0, fsynr=0x31, cb=0
[ 2114.737542] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x000db960, fsynr=0x31, cb=0
[ 2114.737603] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x000dc4a0, fsynr=0x31, cb=0
[ 2114.746116] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x0015ada0, fsynr=0x31, cb=0
[ 2114.770803] arm-smmu da0000.arm,smmu: Unhandled context fault: fsr=0x402, iova=0x001d8020, fsynr=0x31, cb=0

Best Regards,
Jox

[ribalda]

Hi Jox

I am trying to get the imx214 work (check CAMSS: Cannot use a 4 lanes CSI sensor (IMX214))

And I stucked way earlier than you :). Would you mind sharing your driver and device tree modifications?

Thanks!

[jox]

Hi,

[imx230 driver](git clone git://github.com/akhilxavi/debian4.14-imx230driver.git)

Check this driver code.

Best Regards,
Jox