Hardware h264 video encoding in linux

Jerome,

Which cpufreq governor are you using when doing the experiment?

By default, we are using ondemand governor, so 70% cpuload doesn’t really tell us how busy the CPU is. We need to know at which speed it is running.

I recommend doing either

cpufreq-set -g powersave
or
cpufreq-set -g performance

it would set all CPU to run at the lowest or max speed. Then the CPU load will be meaningful.

Hi,

Ok so I gave a try and I got the same results with option :
cpufreq-set -g performance
So all CPU-freq are set at 1.21Ghz and still the same load (70 to 80% on CPU1)

Please find the screenshot (hopefully it works):

Jerome, when you seeing 70% cpu load do you use videotestsrc plugin as a source? This plugin use cpu to fill each pixel of the encoder input buffer.

Hello,

I did noticed that videotestsrc consume a bit of the CPU but I also had the same results with “v4l2src”.
My source is in raw data coming from an EasyCap.

Thanks,

Jérôme.

I checked the encoder driver with simple v4l2 test application, when I don’t touch the encoder input buffers the test application consuming 8% cpu, when it filling the input buffers (including memcpy) it consuming 80% cpu. So most probably there is a memcpy between v4l2src and v4l2enc plugins.

Thanks svarbanov for your feedbacks,

It seems that v4l2enc need some improvement then. For my purpose I wanted to encode some textures coming from a glsource and v4l2enc failed to process the flow.
After several try and researches also in the code found here :https://git.linaro.org/landing-teams/working/qualcomm/pkg/gst-plugins-v4l2.git/tree/refs/heads/debian:/gst , I understand that the current module do not manage any other input but NV12 (even if gst-inspect v4l2enc say otherwise).

I don’t know if someone is having a look into this encoder (the only one supporting HW) but we would really appreciate having an answer to be able to do that :

gst-launch-1.0 gltestsrc ! v4l2enc ! fakesink

Jerome.

Hi,
You linked this project stanimir.varbanov/v4l2-decode.git - [no description] for a demo of h264 hardware decoder, but I tried to compile it and run it, it always fails to find the h264 header in function extract_and_process_header. My stream is a mov file (big buck bunny 720p).
Can you check if this example works or if it needs more explanation on how to use it ? Thanks !

try branch qcom-vidc in that tree.

or try master branch from :

https://git.linaro.org/landing-teams/working/qualcomm/pkg/gst-plugins-v4l2.git

If you run our debian images, there is a package available for this plugin (apt-get install gstreamer1.0-plugins-v4l2).

Thanks for this quick reply.
I don’t find qcom-vidc branch in v4l2-decode git tree, only master and dmabuf_drm. Is it somewhere else ?

I’ll try to learn gstream, never used it ! I’m more used to ffmpeg libs (native C apps) and I was wondering if we could get a libavcodec h264 hardware decoder ?

Thanks again!

ah… sorry… i replied quickly, but looks like i read your post too quick… i didn’t notice you were look at the stand alone test app… i will let Stan reply then, since i haven’t tried to compile it recently…

Hi Looping78, the v4l2-decode test application can parse elementary stream only, try some of these streams
http://www.fastvdo.com/264codec/streams/HD720p/

Stan

For those who are interested in I have managed to make it work with :

gst-launch-1.0 gltestsrc ! video/x-raw,format=“UYVY” ! videoconvert ! video/x-raw,format=“NV12” ! v4l2enc ! filesink location=“enc.h264”

You can force gltestsrc “source” caps to something else than UYVY but not NV12… from what I understand v4l2enc fail to download texture from glcontext that is the reason why you need to use a videoconverter element anyway.

Jérôme.

Hi thanks for pointing that! I now understand better how to decode h264 stream via hardware. Last question I can see 2 video devices on my board, /dev/video32 and /dev/video33 (linux Debian). I used 32 to decode the stream. what is the 33 for ?

Hello All,

I tried the above gst commands but with no luck, pipeline errors, please can anyone give me a gstreamer/ffmpeg command, which allows me to stream from /dev/video0 with hardware encoding enabled. I was able to do SW ffmpeg and the speed I get is 4fps which is bad, I need to do HW encoding can anyone help me

Thanks
ram

Hi all,

Please anyone inform which gstreamer-plugin i will use dragon 410c for android.

manju

The following commands for me :
Videotestsrc:
gst-launch-1.0 videotestsrc ! v4l2enc ! filesink location=raw_dual.mkv

gst-launch-1.0 -vvv videotestsrc ! video/x-raw,width=320,height=240,framerate=30/1,format=NV12 ! v4l2enc ! filesink location=raw_dual.mkv

Webcam:
gst-launch-1.0 -vvv v4l2src device=/dev/video0 ! videoconvert ! video/x-raw,width=320,height=240,framerate=30/1,format=NV12 ! v4l2enc ! filesink location=temp.mkv

Note the conversion to explicit NV12. It is very important as @svarbanov pointed out.

The following command works for streaming over UDP -
gst-launch-1.0 -vvv v4l2src device=/dev/video0 ! videoconvert ! video/x-raw,width=320,height=240,framerate=30/1,format=NV12 ! v4l2enc ! h264parse ! rtph264pay ! udpsink host=192.168.0.13 port=5000

Note the presence of h264parse. v4l2enc gives out byte-stream with aligment = au. rtph264pay does not seem to like that. h264parse converts to avc with alignment = au

Hi, does anyone know if the hardware is capable of encoding multiple streams simultaneously? Has anyone tried it? Specifically, I am interested in recording / encoding from two cameras at once.

I’m considering the 410 for a new design, but I can’t seem to find much detailed information on this part of the SoC. Thanks in advance.

matt

Hi, Matt. You should open two instances of the video encoder driver then you will encode camera streams simultaneously. Of course you should take into account the camera resolutions, I think 2x720p is supported from the hardware without overloading (overheating).

Stan

Thanks Stan, 2x720p would certainly meet our requirements. Unfortunately I do not have a board to test this with. If there’s no documentation that can verify the specs, I guess I’ll have to get the Dragonboard to try.

Has anyone seen a block diagram of this part of the SoC? I wonder if the video must be written to memory first, then consumed by the encoder, or if there is a direct path from the CSI’s to the video codec.