How can I capture in resolutions other than 1920x1080 and 1280x720 on 4.14?

Products Support DragonBoard410c
#1

Hello All,

I’m working with this script with the OV5640 on the 4.14 kernel that accepts H Resolution, V Resolution, and framerate as args $1, $2, $3 in this script: https://gist.github.com/RobGries/6518cd9272a19ef8925a271b5bba7721

Usage: ./capture.sh 1280 720 15
Captures a JPEG at 720P from a stream running at 15 fps

The script seems to work fine with 1280x720 (15/30 fps) and 1920x1080 (15 fps), however any other resolution fails and I get an SOF error when stopping gstreamer.

SOF errors on every other resolution:
[ 2690.333039] qcom-camss 1b0ac00.camss: VFE sof timeout
[ 2690.845283] qcom-camss 1b0ac00.camss: VFE reg update timeout

Another observation, is that it appears that the driver registers a whole host of other resolutions other than 1080 and 720, but none seem to work. Is there a different media-ctl pipeline to use with other resolutions?

So far the following resolutions result in the error above:
176x144, 640x480, 720x480, 720x576, 1024x768, and 2592x1944

#2

There are interesting patches on LKML from Maxime Ripard:
https://patchwork.linuxtv.org/project/linux-media/list/?submitter=Maxime+Ripard&q=ov5640

The current configuration is a bit static and I think these patches could help making all modes working.
It is worth a try.

#3

Loic, after some fiddling I was able to apply the entire patchset from Maxime Ripard. I am now building the kernel to see if it works out.

Thanks for the hint! I’ll update this post with my progress.

#4

I’ve successfully compiled the driver with the patchset, however I’m now not able to capture at any resolution. I think that I’ve applied a patch that probably breaks something.

Looking at the patchwork pages for the camera, I saw that Daniel Mack has some interesting patches that seem to be tested with our particular SOC. Do you think it is worth rolling back Maxime’s patches and apply these?
https://patchwork.linuxtv.org/project/linux-media/list/?submitter=7218&state=*

I might have applied a conflicting patch to the driver, I had to apply a few upstream patches that I was missing from my 4.14 fork, I suspect that the “media: ov5640: add support of RGB565 and YUYV formats” might have broke it due to GStreamer having trouble negotiating the input format and getting some insane values for the input resolution.

This history for the ov5640 driver that I’m working with is located here:
https://github.com/RobGries/96b_4.14_kernel/commits/ov5640Patches/drivers/media/i2c/ov5640.c

#5

As a test I reverted back to commit id 5b1afba3c02ea0a00dba0c157b9b0e3ee001082e (media: ov5640: add missing output pixel format setting) effectively reverting all of Maxime’s patches this is so that I can localize where things went wrong.

Now I’m looking to implement Maxime’s patches one by one to see where things start to go wrong. Also I might also give Daniel Mack’s changes a try to see if they help in getting this to work.

#6

I need to have a deeper look but I assume we need to adapt Mani’s patch to return the new calculated pixel clock instead of the static one (96Mhz).

#7

@Loic,

Yes, that should be the way to go. Will see if I can get the patch modified by tomorrow. However, I have broken my OV5640 so won’t be able to test it.

#8

If you need help testing your patch, I have an OV5640 on a 410c to try this out on. Thanks for your help.

#9

I have also took a stab at rebasing Mani’s pixel clock patch on Maxime Ripard’s patchset. I’m currently rebuilding to test this out on my side and will update when I have finished testing it.

Here’s the reworked patch:
https://github.com/RobGries/96b_4.14_kernel/commit/0ccf32e60bb0ab5057ad76bd05630e03e4fbc50c.patch

Last patch didn’t initialize the v4l2_ctrl *pixel_clock properly, I reset the branch and committed the proper patch…

#10

So after correcting the patch and recompiling, I seem to fail on reading register 3034 that is defined as OV5640_REG_SC_PLL_CTRL0.

I believe that this is caused by the change that sets the pixel clock in the ov5640_init_controls. This is probably because the data needed to calculate the sysclk isn’t available when we enter the ov5640_probe method upon boot. I wonder if I could check if the register is empty and set a default value upon a failed NULL check on reg 3034.

What do you guys think?

#11

Well returning 96MHz when reading OV5640_REG_SC_PLL_CTRL0 fails doesn’t seem to do it… I can now at least initialize the camera but capturing from it seems to hang with the CAMSS returning the following errors after stopping GStreamer when it hangs:

[ 526.436431] qcom-camss 1b0ac00.camss: VFE sof timeout
[ 526.947891] qcom-camss 1b0ac00.camss: VFE reg update timeout

I gave it a shot, but I’m afraid that I’m probably missing something.

This is the current state of the driver after my failed patches:

Any ideas?

#12

I have a branch on which I applied upstream patches:
https://git.linaro.org/people/loic.poulain/linux.git/log/?h=qcomlt-4.14-d3-mezzanine

I added patch for pixel clock ctrl as well:
https://git.linaro.org/people/loic.poulain/linux.git/commit/?h=qcomlt-4.14-d3-mezzanine&id=5765d5ad7636a04acdbaf8919c30296d6fb7fbe8

I’ll test it tomorrow since I need to rebuild gstreamer.

#13

I finally have the camera working in my git tree [1], the top patch is a workaround for now but will be converted to clean patches. You can give it a try.

[1] loic.poulain/linux.git - [no description]

#14

Hey Loic,

I tried your most recent patches from the kernel that you’ve linked and patched my fork with them. However, I’m not sure if I’m missing something from my end since whenever I try to capture, I get a whole lot of VFE0 pix0 overflow errors in the console and I cannot capture.

Am I missing something related to the CAMSS driver preventing me from using your latest patch?

#15

Not sure, could you try to build and use the ‘my’ branch to compare ?

That said If the camera is came back to life, I still have issue with non 1080p resolutions ( I think this is due to clock configuration/compute). Scaling however can be used to change the resolution via the pix interface / ISP.

#16

Hey Loic,

I forked your kernel changes into another repo and it seems that I’m getting further into getting this driver to work. I’m now able to capture only in 1080p without SOF Freeze errors, but the file generated has no picture and just black screen.

Is this the point that you left off?

#17

Yes, only video streaming was working. I’ve just updated the tree with new patches which makes frame capture working as well. I suggest you to update your tree accordingly.

Yes the others resolutions are not operational for now. I suggest to use CAMSS scaling capability via compose argument (the scaling will be done in QCOM ISP intead of in OV5640 ISP).

#18

Cool, I’ve pulled your new patchset and I’m about to give it a shot.

Will using the QCOM ISP over the integrated ISP in the OV5640 cause any sort of performance hit? Are we now scaling in software rather than using the integrated hardware in the sensor?

Thanks for clearing that up, I was a little confused at first…

#19

No, we are still using hardware scaling, but using the QCOM SoC ISP instead of the camera integrated one, so I do not expect any performance impact.

#20

After cloning your latest sources, and running the capture.sh script from the first post I still get the same VFE errors from the driver when trying to capture at 1080p.

This is a capture of the logs showing the failure - https://gist.github.com/RobGries/0e57aa7f6aa1008029d1095e5789908d

This is the fork that I’m using - https://github.com/RobGries/96b-Loic-kernel-18.01/commits/qcomlt-4.14-d3-mezzanine

Could this be caused by an incompatible gstreamer application? I’m running the 1.12.2 version from the qualcomm openembedded build IMM.LE.36900 - gstreamer\recipes-multimedia\core - imm/meta-backports - Unnamed repository

What version of Gstreamer are you capturing with?

Also are these media-ctl commands valid? Note: these do not enable scaling, just straight capture at 1920x1080 –
media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_pix”:0[1]’
media-ctl -v -d /dev/media0 -V ‘“ov5640 2-003b”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:1[fmt:UYVY1_5X8/1920x1080 field:none]’