Ov5640 fps issue - 1080p@30fps


#1

Hi all,

I am trying to record video @30fps with ov5640. 1080p settings are correct(vfe receiving 30 frames per second) but the recorded video is only 23fps.
vfe_isr_wm_done is happening only 23 times. Do anyone have idea?
In case of 1080p preview, i am getting 30 frames per second(vfe_isr_wm_done is happening 30 times).
where could be the frame drop? I will post my findings, if i get any.

Thanks,
Pradeep


#2

Which kernel version? which command are you running?


#3

Hi @Loic ,
I am using 4.14.53 kernel(release/qcomlt-4.14 branch).

I am using below pipeline for record:
sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_pix”:0[1]’

sudo media-ctl -d /dev/media0 -V ‘“ov5640 3-0078”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:1[fmt:UYVY1_5X8/1920x1080 field:none]’

gst-launch-1.0 -e v4l2src device=/dev/video3 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! v4l2h264enc extra-controls=“controls,h264_profile=4,video_bitrate=10000000;” ! h264parse ! mp4mux ! filesink location=test.h264

I am using below pipeline for preview:

sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_pix”:0[1]’

sudo media-ctl -d /dev/media0 -V ‘“ov5640 3-0078”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csiphy0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_csid0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_ispif0”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:0[fmt:UYVY2X8/1920x1080 field:none],“msm_vfe0_pix”:1[fmt:UYVY1_5X8/1920x1080 field:none]’

gst-launch-1.0 v4l2src device=/dev/video3 ! ‘video/x-raw,format=NV12,width=1920,height=1080’ ! glimagesink

Please let me know your observations.

Thanks,
Pradeep


#4

could you try to increase buffer count (add num-buffers=12 to v4l2src) and add a queue element between capture en encoding (! queue ! v4l2h264enc... )


#5

Hi @Loic,

I tried, it is not working.
videotestsrc 1080p@30fps and 720p@30fps works fine. Have you observed this issue?

Thanks,
Pradeep


#6

OK, just in case can you try to use the raw interface instead of pix, you’ll need to configure pipeline to msm_vfe0_rdi0 and add a videoconvert element between UYVY8 camera output and v4l2enc (for NV12 convertion).


#7

Hi @Loic,

I tried with RDI as well. Same issue.
Adding queue with buffer size like below to encoder, improved fps rate from 23 to 24-26fps(varying). Issue may be at encoder side.

’queue max-size-bytes=204857600’ ! v4l2h264enc

Default: 10 MB, 204857600 =195MB

Thanks,
Pradeep


#8

Hi @Loic,

Please let me know, have you observed this issue when you tested?

Thanks,
Pradeep


#9

Hi @Loic,

Issue is fixed by increasing the clock frequency from 75 to 76 Mhz. Below is the change. Now 1080p is recording @30fps.

diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c
index 41eef37…5608970 100644
— a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/venus/core.c
@@ -340,7 +340,7 @@ static __maybe_unused int venus_runtime_resume(struct device dev)
{ 1944000, 490000000 }, /
4k UHD @ 60 /
{ 972000, 320000000 }, /
4k UHD @ 30 /
{ 489600, 150000000 }, /
1080p @ 60 /
– { 244800, 75000000 }, /
1080p @ 30 /
++{ 244800, 76000000 }, /
1080p @ 30 */
};

Thanks,
Pradeep


#10

thanks, good to know, adding @svarbanov to check if your change can be applied.


#11

For Info, Below are clock frequencies from android:
/* Encoders /
<972000 490000000 0x55555555>, /
4k UHD @ 30 /
<489600 320000000 0x55555555>, /
1080p @ 60 /
<244800 150000000 0x55555555>, /
1080p @ 30 /
<108000 75000000 0x55555555>, /
720p @ 30 */

/* Decoders /
<1944000 490000000 0xffffffff>, /
4k UHD @ 60 /
< 972000 320000000 0xffffffff>, /
4k UHD @ 30 /
< 489600 150000000 0xffffffff>, /
1080p @ 60 /
< 244800 75000000 0xffffffff>; /
1080p @ 30 */