Thank you for the other day.
BTW, I am planning to experiment with various encode parameters for FFmpeg, but I can not find parameters equivalent to “extra-controls” in GStreamer.
I think that parameters such as H.264 profile, level, and quality can not be specified with the FFmpeg standard commands, but how can I specify it?
Can I use FFmpeg to grab frames directly from a camera?
In the example:
ffmpeg -loglevel debug -f rawvideo -pix_fmt nv12 -s:v 1280:720 -r 25 -i ~/Videos/raw/freeway.yuv -c:v h264_v4l2m2m out/out.h264.mp4
It appears that the encoder is encoding frames from an already recorded video. Can I do this with live video?
I tried this and FFmpeg cannot tell that the source is a video capture device, so I think I’m missing something here:
ffmpeg -loglevel debug -f v4l2 -pix_fmt nv12 -s:v 1280:960 -r 30 -i /dev/video3 -c:v h264_v4l2m2m encoded.mp4
Edit: Here’s a log from an unsuccessful run of the above command
dragonboard-410c:~$ ffmpeg -loglevel debug -f v4l2 -pix_fmt nv12 -s:v 1280:720 -r 25 -i /dev/video3 -c:v h264_v4l2m2m out.h264.mp4 ffmpeg version 3.3.3 Copyright (c) 2000-2017 the FFmpeg developers built with gcc 6.2.1 (Linaro GCC 6.2-2016.11) 20161016 configuration: --disable-stripping --enable-pic --enable-shared --enable-pthreads --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-libxcb-shape --cross-prefix=aarch64-linaro-linux- --ld='aarch64-linaro-linux-gcc --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --cc='aarch64-linaro-linux-gcc --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --cxx='aarch64-linaro-linux-g++ --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --arch=aarch64 --target-os=linux --enable-cross-compile --extra-cflags=' -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/work/aarch64-linaro-linux/ffmpeg/3.3.3-r0=/usr/src/debug/ffmpeg/3.3.3-r0 -fdebug-prefix-map=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/x86_64-linux= -fdebug-prefix-map=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c= --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --extra-ldflags='-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed' --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c --enable-hardcoded-tables --libdir=/usr/lib64 --shlibdir=/usr/lib64 --datadir=/usr/share/ffmpeg --disable-mipsdsp --disable-mipsdspr2 --cpu=generic --pkg-config=pkg-config --enable-avcodec --enable-avdevice --enable-avfilter --enable-avformat --disable-avresample --enable-bzlib --enable-gpl --disable-libgsm --disable-indev=jack --disable-libvorbis --enable-lzma --disable-libmp3lame --disable-openssl --enable-postproc --disable-libschroedinger --disable-sdl2 --disable-libspeex --enable-swresample --enable-swscale --enable-libtheora --disable-vaapi --disable-vdpau --disable-libvpx --enable-libx264 --enable-outdev=xv libavutil 55. 58.100 / 55. 58.100 libavcodec 57. 89.100 / 57. 89.100 libavformat 57. 71.100 / 57. 71.100 libavdevice 57. 6.100 / 57. 6.100 libavfilter 6. 82.100 / 6. 82.100 libswscale 4. 6.100 / 4. 6.100 libswresample 2. 7.100 / 2. 7.100 libpostproc 54. 5.100 / 54. 5.100 Splitting the commandline. Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'. Reading option '-f' ... matched as option 'f' (force format) with argument 'v4l2'. Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'nv12'. Reading option '-s:v' ... matched as option 's' (set frame size (WxH or abbreviation)) with argument '1280:720'. Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '25'. Reading option '-i' ... matched as input url with argument '/dev/video3'. Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'h264_v4l2m2m'. Reading option 'out.h264.mp4' ... matched as output url. Finished splitting the commandline. Parsing a group of options: global . Applying option loglevel (set logging level) with argument debug. Successfully parsed a group of options. Parsing a group of options: input url /dev/video3. Applying option f (force format) with argument v4l2. Applying option pix_fmt (set pixel format) with argument nv12. Applying option s:v (set frame size (WxH or abbreviation)) with [ 3623.130796] 1b0c000.qcom,cci supply qcom,gdscr-vdd not found, using dummy regulator argument 1280:720. Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 25. Successfully parsed a group of options. Opening an input file: /dev/video3. [video4linux2,v4l2 @ 0x1f8b6460] fd:3 capabilities:85201000 [video4linux2,v4l2 @ 0x1f8b6460] Not a video capture device. /dev/video3: No such device
did you check this link?
https://trac.ffmpeg.org/wiki/Capture/Webcam
it seems your input device is not a video capture device.
Yeah, I got that from the error…
It doesn’t seem like any of the video interfaces are. On the DB410c how would I properly encode from the camera with ffmpeg? Is this supported?
can you open a terminal and type
$ dmesg -w -d
then plug your camera and provide the new lines printed from dmesg and the output from:
$ v4l2-ctl --list-devices
I’m sorry, I still want to implement with GStreamer instead of FFmpeg as my implementation.
There is no change in the present condition that some required parameters are not valid.
So I will rewrite them, please take care that the setting of the following parameters becomes effective.
Best regards.
If I understand your problem, even if these capabilities are reported as supported, they have no effect on your stream when using gstreamer ? could you please provide the gstreamer command/pipeline you apply ?
For example, let’s focus on h264_entropy_mode.
As for the binary data resulting from encoding and encoding the following two types of pipeline, there is no difference other than the time field of the header of .mp4.
The following example is expected to be encoded by CAVLC:
gst-launch-1.0 filesrc location=test.nv12 ! \
rawvideoparse width=1280 height=960 framerate=15/1 format=23 ! \
v4l2h264enc extra-controls="controls,h264_entropy_mode=0,h264_profile=4,h264_level=10,video_bitrate=256000;" ! \
h264parse ! video/x-h264,stream-format="avc",alignment="au" ! \
mp4mux fragment-duration=10 ! filesink location=test0.mp4
Next example is expected to be encoded by CABAC, but it seems same as CAVLC.
gst-launch-1.0 filesrc location=test.nv12 ! \
rawvideoparse width=1280 height=960 framerate=15/1 format=23 ! \
v4l2h264enc extra-controls="controls,h264_entropy_mode=1,h264_profile=4,h264_level=10,video_bitrate=256000;" ! \
h264parse ! video/x-h264,stream-format="avc",alignment="au" ! \
mp4mux fragment-duration=10 ! filesink location=test1.mp4
Could you please try it?
Thanks, I don’t see any obvious usage of entropy_mode in the venus driver, value seems retrieved by the driver but not really applied (ctr->h264_entropy_mode) (cf: drivers/media/platform/qcom/venus).
We are going to clarify the status (what is really supported and what is planned to support).
Obviously entropy mode is not applied in the driver. Could you open a bug with all controls which you think are not applied at bugs.96boards.org?
To clarify my words, I though the driver would use this entropy_mode value to configure the hardware for a certain entropy encoding usage. But yes, @medianoche, could you please open a bug for non-effective v4l2 controls.
I understand. I will post a bug report.
@medianoche , I’ve attached a first patch to the bug : https://bugs.96boards.org/attachment.cgi?id=213
This should fix the entropy issue, would you be able to test it on your side (kernel and modules rebuild)?
With this patch, mediainfo reports the following:
Thank you for providing the patch.
I could build the kernel and modules without problems.
And I also confirmed that h264_entropy_mode is enabled by installing these on DragonBoard 410c.
Thank you for your continued support and for other parameters as well.
One more patch here which should give you some control on GOP size and b_frame. Note that it does not seem possible to have consecutive b_frames, not sure if it is a normal or fixable behavior.
I’ve performed some tests with the following parameters:
*I: I-frame; P: P-frame; B: B-frame
It looks like a correct sequence.
However, I would like to specify more than 2 value for video_b_frames.
I applied the patch uploaded here, then as long as you specify 0 or 1 it will certainly get the correct result.
But, this does not seem to have changed behavior before applying the patch.
Is it difficult to specify 2 or more for video_b_frames due to specifications?
This patch enables gop-size configuration which was not applied before and ensure gop-size conservation in case of b-frame enablement. However, you should be able to enable b-frames without it.
The encoder firmware reports support of 4 consecutive b-frames max, so this should be possible, for now I’m not able to configure it properly. maybe some incompatible parameters… we are going to have a look.
In the same time could you check this other patch. It enables the deblock filter configuration, allowing to use v4l2 controls h264_loop_filter_mode, h264_loop_filter_alpha_offset and h264_loop_filter_beta_offset.
h264_loop_filter_mode values: 0 (filter mode enabled); 1 (default? filter mode disable); 2 (disabled at slice boundary)
h264_loop_filter_alpha_offset: -6 to 6 (default 0)
h264_loop_filter_beta_offset: -6 to 6 (default 0)
I don’t know easy way to determine if it works as expected, but since you request this control I assume you can check and confirm if the control is correctly applied. let me know…
Thank you for providing a new patch.
h264_loop_filter_mode, h264_loop_filter_alpha_offset, h264_loop_filter_beta_offset, all of these parameters seem to work fine.
Also, the image quality has been improved visibly.
For each parameters, I will think about the final value again.
Thank you for your continuous support.
Would you mind sharing your parameters?
I’m dealing with some artifacting issues with these parameters:
controls,h264_profile=4,video_bitrate=2000000;
