Frame lost when streaming from an UVC camera

Keeler · January 31, 2020, 12:05pm

Hi there,

I was trying to stream from a 5MP UVC camera over USB 3.0. The command I used was:
GST_DEBUG=2 gst-launch-1.0 v4l2src ! fakesink for testing purpose.

The default resolution is 2592x1944@24fps and pixel format is UYVY422.

I got warnings:

=============================================================
linaro@linaro-alip:~$ GST_DEBUG=2 gst-launch-1.0 v4l2src ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.292036809  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:00.300993582  3472 0xaaaaf178bb20 WARN           v4l2allocator gstv4l2allocator.c:1349:gst_v4l2_allocator_dqbuf:<v4l2src0:pool:src:allocator> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:00.301194729  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1189:gst_v4l2_buffer_pool_dqbuf:<v4l2src0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:00.301247333  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:00.301421709  3472 0xaaaaf178bb20 WARN           v4l2allocator gstv4l2allocator.c:1349:gst_v4l2_allocator_dqbuf:<v4l2src0:pool:src:allocator> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:00.301468168  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:00.301577752  3472 0xaaaaf178bb20 WARN           v4l2allocator gstv4l2allocator.c:1349:gst_v4l2_allocator_dqbuf:<v4l2src0:pool:src:allocator> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:00.301621762  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:00.301733013  3472 0xaaaaf178bb20 WARN           v4l2allocator gstv4l2allocator.c:1349:gst_v4l2_allocator_dqbuf:<v4l2src0:pool:src:allocator> V4L2 provided buffer has bytesused 0 which is too small to include data_offset 0
0:00:00.301977442  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:00.319368900  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:00.355833078  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:00.962214644  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:00.962424177  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.003735415  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.044458994  3472 0xaaaaf178bb20 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 4 - ts: 0:00:00.728786130
0:00:01.092419022  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.128489343  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.170352929  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.212165941  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.252868165  3472 0xaaaaf178bb20 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 4 - ts: 0:00:00.937126880
0:00:01.295832278  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.336316376  3472 0xaaaaf178bb20 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 1 - ts: 0:00:01.020476140
0:00:01.378830590  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.821736643  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.857727484  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:01.899350652  3472 0xaaaaf178bb20 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 4 - ts: 0:00:01.583582141
0:00:02.385016182  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:02.385337955  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:02.825132063  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:02.861468636  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:02.902863677  3472 0xaaaaf178bb20 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 6 - ts: 0:00:02.586981422
0:00:02.950073701  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:02.986007043  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:03.028641727  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:03.069452234  3472 0xaaaaf178bb20 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 3 - ts: 0:00:02.753659578
0:00:03.195335336  3472 0xaaaaf178bb20 WARN          v4l2bufferpool gstv4l2bufferpool.c:1952:gst_v4l2_buffer_pool_process:<v4l2src0:pool:src> Dropping truncated buffer, this is likely a driver bug.
0:00:03.236077769  3472 0xaaaaf178bb20 WARN                 v4l2src gstv4l2src.c:976:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 1 - ts: 0:00:02.920324880
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:03.498884513
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
========================================================================

It looks like there are a lot of frames dropped.

Any idea how to fix this?

Many thanks

Keeler · January 31, 2020, 12:05pm

I tested with other resolutions and framerates:

2592x1944@12fps works fine.
2560x1440@16fps works fine.
1920x1080@30fps works fine.

Still had v4l2bufferpool warning and suggested Uncertain or not enough buffers, enabling copy threshold.

Is this a buffer problem? I tried using queue. It didn’t help much. Any way I can increase buffers for incoming data from uvc camera?

Many thanks.

Loic · January 31, 2020, 5:48pm

Well you can try to increase the number of buffers in the pool with e.g. gst-launch-1.0 v4l2src queue-size=10 but no sure it will help.

Keeler · February 3, 2020, 9:12am

Hi Loic,

Thank you for the reply. I can’t find queue-size property for v4l2src element on DragonBoard 820c. Am I missing something?

Loic · February 4, 2020, 8:21am

Look like it does not exist in newer gstreamer version.
You can try a queue:

gst-launch-1.0 v4l2src ! queue ! fakesink

Keeler · February 4, 2020, 9:05am

Hi Loic,

Thank you for the help. We have tried queue and tried different properties settings of the queue. It didnt help much.

Is it a UVC problem or not enough buffer problem?

Loic · February 5, 2020, 8:23am

Not sure, the fakesink should release the buffer quite quickly so buffer should not be starved in this pipeline. This is either due to the UVC driver or the USB controller driver, some of the buffers you receive seem to be malformed or incomplete:

Dropping truncated buffer, this is likely a driver bug.

Do you have the same issue on a regular desktop/laptop?

Keeler · February 5, 2020, 9:01am

I tried on Windows 10 desktop with gstreamer to stream video to the display using fpsdisplaysink. It worked very well. I can get maximum framerate stably.

Keeler · March 6, 2020, 10:50am

@Loic

I am able to get no frame drops when using full resolution @ 12fps or FHD @ 30 fps. However, if i try to get full resolution @ 24fps or FHD @ 60fps there are a lot of frame losses.

I could be wrong but I suspect that USB3.0’s throughput is not high enough as I can get full resolution @24fps on my PC.

Is there anything in kernel that limits USB 3 throughput?

I was checking buffer.c and noticed there was a patch:

/* FIXME tune these based on pool statistics … */
-static const size_t pool_max[HCD_BUFFER_POOLS] = {

/* platforms without dma-friendly caches might need to
- prevent cacheline sharing…
*/
32,
128,
512,
PAGE_SIZE / 2
/* bigger → allocate pages */
+static size_t pool_max[HCD_BUFFER_POOLS] = {

32, 128, 512, 2048,
};

I wish to know how I can tune these? Would changing back to PAGE_SIZE/2 or increasing 2048 to something bigger make any difference for this problem?

Many thanks