Hardware h264 video encoding in linux

[kazenniy]

Hi all!
It is possible to encode video in h264 using hardware?

[ldts]

Qualcomm supports OpenMAX on this platform

Have a look at this document
https://developer.qualcomm.com/download/db410c/openmax-omx-video-encoder.pdf

[ldts]

I cant find the necessary libraries in the Linux release though. Let me have a deeper look.

[kazenniy]

In this pdf talks about android only. Linux support OpenMAX?

[ldts]

Yes I saw that as well.

After discussing with the development team it is clear that OMX is NOT part of the Linux roadmap.

The supported API for h/w encoding/decoding will be done via the v4l2 mem-to-mem.

Video decoding is currently supported; the team is working on adding the necessary encoding support (tentative release date is the end of the month (October 2015))

If you are not familiar with the API please have a look at this project (functional) code:

https://git.linaro.org/people/stanimir.varbanov/v4l2-decode.git/tree

[kazenniy]

I'll wait the end of October.
Thank you.

[suspendme]

Any reason why v4l2 mem-to-mem was preferred over OMX by Linaro? Thank You.

[ndec]

hi,

OMX is a user space framework that would sit on top of v4l2 mem2mem. So we (linaro) didn't choose v4l2 mem2mem instead of OMX, OMX can still be used with the Linaro kernel, if that is really needed. that might require some modifications/integration of OMX which we choose not to do.

On Linux we didn't see the need to use OMX (and gst-omx), since we could do the same things without them. Basically less code, less maintenance.

[kazenniy]

It is November now. How about hardware video encoding on linux?

[ndec]

hi,

the work has been completed, both the kernel driver support and Gstreamer. It now needs to be merged/integrated into our builds.. I am traveling this week, so won't likely happen this week, but we will get builds next week.

[kazenniy]

Hi!
Please let me know when it's available.

[tdc]

Hello,

ldts-jro mentioned above that hardware video decoding is already supported. Is it supported by one of the gstreamer-1.0 plugins available from the ubuntu repository? If so, which plugin and element should I use? I tried v4l2dec and avdec_h264. Both play H.264 720p pretty smoothly with similar CPU resource usage, and 1080 is very sluggish, so I believe neither was doing hardware decoding. There is also v4l2video32dec. but this one didn't work.

Thanks

[pjakub]

Hi, what formats will encoder support ? What resolutions ?

[pjakub]

Last version: http://builds.96boards.org/releases/dragonboard410c/linaro/debian/15.11/
have video encoder.

Have anyone managed to use it?
I tried doing:
sudo v4l2-ctl -d /dev/video33 --list-formats-ext

which leaves me with:
[ 1043.261462] msm_vidc: warn: Getting buffer requirements failed: -22

in dmesg.

Also trying to change video parameters using
ioctl(fd_, VIDIOC_S_FMT, &fmt)

leaves this error in dmesg:

[ 1264.671951] compressed_rgbd[2316]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006
[ 1264.672431] pgd = ffffffc02d89e000
[ 1264.680689] [00000000] *pgd=00000000adbfe003, *pud=00000000adbfe003, *pmd=0000000000000000

[ 1264.692164] CPU: 0 PID: 2316 Comm: compressed_rgbd Not tainted 4.2.4-linaro-lt-qcom #1
[ 1264.693914] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[ 1264.701652] task: ffffffc0360ebe40 ti: ffffffc024848000 task.ti: ffffffc024848000
[ 1264.708528] PC is at 0x7f8ba54b90
[ 1264.715920] LR is at 0x7f8c5f33d0
[ 1264.719129] pc : [<0000007f8ba54b90>] lr : [<0000007f8c5f33d0>] pstate: 40000000
[ 1264.722488] sp : 0000007fd578fb90
[ 1264.729910] x29: 0000007fd578fb90 x28: 0000000000000000
[ 1264.738136] x27: 0000000000000000 x26: 0000007fd578fc10
[ 1264.743344] x25: 0000007fd578fd28 x24: 0000007f8c6fa360
[ 1264.748723] x23: 0000007f8c6f7000 x22: 0000007f8c6f7000
[ 1264.753923] x21: 0000000000000000 x20: 0000007f8c6f7000
[ 1264.759322] x19: 0000007f8c6fa360 x18: 0000007fd578f8f0
[ 1264.764513] x17: 0000007f8c6fa0e8 x16: 0000007f8ba54b80
[ 1264.769916] x15: 0000007f8bb1e588 x14: 72657473616d2065
[ 1264.775151] x13: 68742068636e7561 x12: 6c20796c6c617574
[ 1264.780402] x11: 0101010101010101 x10: 00000000ffffffff
[ 1264.785732] x9 : 6f6e61206e692027 x8 : 0000000026fcd220
[ 1264.790992] x7 : 0000000000000000 x6 : 0000000000000000
[ 1264.796321] x5 : 0000000000000001 x4 : 0000000000000000
[ 1264.801582] x3 : 0000000000000000 x2 : 0000007f8c526650
[ 1264.806978] x1 : 0000000000000000 x0 : 0000000000000000

any hints?

[svarbanov]

Hi, the encoder resolutions which I tested are 720p and 1080p with h264 codec. What codec did you tried to set format to? Also to verify the encoder works on you board you can use v4l2enc gstremer plugin, which has been adopted to work with vidc driver.

[pjakub]

Thanks for reply!

Have you used v4l2enc gstreamer plugin ? Can you please command that you used ? I tried this, but it fails:

gst-launch-1.0 videotestsrc num-buffers=90 ! video/x-raw, format=YUY2, framerate=30/1, width=1280, height=720 ! queue ! videoconvert ! v4l2enc ! filesink location=test.ts

results in:

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Failed to configure the buffer pool
Additional debug info:
gstbasesrc.c(3101): gst_base_src_decide_allocation_default (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
Configuration is most likely invalid, please report this issue.
ERROR: pipeline doesn't want to preroll.

If I replace v4l2enc with x264enc it works just fine.

output from

gst-inspect-1.0 v4l2enc

lists all supported types, and it should work.

[svarbanov]

Hi,
try with videotestsrc format=NV12, I cannot remember the exact gst pipeline right now, but can share with you next week.

[kazenniy]

Hi!
Anyone have hardware video encoding success story now?

[svarbanov]

gst-launch-1.0 -v -e --gst-debug=v4l2enc:0 videotestsrc ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,profile=high,level\=(string)"5" ! v4l2enc ! queue ! filesink location=/home/linaro/enc.h264

or

gst-launch-1.0 -v -e --gst-debug=v4l2enc:0 videotestsrc ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,profile=high,level\=(string)"5" ! v4l2enc ! h264parse ! mp4mux ! filesink location=/home/linaro/video.mp4

[jtis]

Hello,

Succesfully converting videtestsrc with the following commands :
raw h264:
gst-launch1.0 videotestsrc ! v4l2enc ! filesink location="enc.h264"

container mpeg :
gst-launch1.0 videotestsrc ! v4l2enc ! mpegpsmux ! filesink location="enc.mpeg"

When you using v4l2src be careful to adapt the "src" of the element with the "sink" of v4l2enc. I had to convert the format of my source to NV12 to encode it.

By the way I got ONE CPU loaded at 70% during encoding (others CPU are free).
Is that normal for hardware encoding to use even such CPU's?

Jérôme.