Mpph264enc is too slow

vishnu_kumar · October 21, 2019, 7:03am

Board based on the Rockchip RK3399 - ARM Cortex-A72 Dual-core up to 1.8GHz + Cortex A53 Quad-core up to 1.4GHz

I am working on a mipi camera based application in rock960A with debian os. i removed existing gstreamer packages and built 1.12 version from source so as to make use of hardware encoders.
I am trying to encode the YUY2 frames using mpph264enc gstreamer plugin. pipeline is as follows.

gst-launch-1.0 rkcamsrc device=/dev/video0 ! ‘video/x-raw,format=YUY2,width=1920,height=1080,framrate=30’ ! mpph264enc ! filesink location=test1.mp4 -ve

The performance of the hardware encoder is pretty slow. where in, i could fetch only 10 to 15 fps for 1080P. the encoding rate is in response to current cpu frequency. 15 fps is seen only on performance mode. i am not sure if mpph264enc is actually making using of hardware encoder.

Do i have a fix for this ? or have i done anything wrong with the installation ?

Loic · October 9, 2019, 2:24pm

Could you please try with videotestsrc element instead of rkcamsrc, also there is a typo:
framrate=30 shoudl be framerate=30/1

vishnu_kumar · October 11, 2019, 2:01pm

Hello Loic,

I did test it with videotestsrc. thats how i examined the frame rate. should i enable anything in the kernel source ?!?
Following is the log from gstreamer pipeline.
root@linaro-alip:~# gst-launch-1.0 videotestsrc ! 'video/x-raw,format=YUY2,width=1920,height=1080,framrate=30/1' ! mpph264enc ! fpsdisplaysink video-sink=fakesink text-overlay=false sync=false -ve

Setting pipeline to PAUSED …
mpi: mpp version: dd3f0ee7 author: Herman Chen [enc_impl]: Rename and new enc_impl function
mpp_rt: NOT found ion allocator
mpp_rt: found drm allocator
Pipeline is PREROLLING …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, framrate=(fraction)30/1, pixe
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, framrate=(fraction)30/1, pixel-ae
h264e_api: h264e_config MPP_ENC_SET_RC_CFG bps 7776000 [7290000 : 8262000]
/GstPipeline:pipeline0/GstMppH264Enc:mpph264enc0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, framrate=(fraction)30/1, pixel-e
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, framrate=(fraction)30/1, pixel-e
/GstPipeline:pipeline0/GstMppH264Enc:mpph264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fracto
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ro
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=o
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, fro
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 9, dropped: 0, current: 17.02, average: 17.02
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 18, dropped: 0, current: 16.14, average: 16.57
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 26, dropped: 0, current: 15.85, average: 16.34
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 35, dropped: 0, current: 16.03, average: 16.26
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 44, dropped: 0, current: 16.13, average: 16.23

vishnu_kumar · October 21, 2019, 7:01am

Hi,

i have an update on this topic. i tried to debug further with vpu drivers as below.

# echo 0x100 > /sys/module/rk_vcodec/parameters/debug

:~# gst-launch-1.0 rkcamsrc device=/dev/video0 io-mode=dmabuf !
 video/x-raw,format=YUY2,width=1920,height=1080,framrate=30/1 ! mpph264enc ! fp
sdisplaysink video-sink=fakesink text-overlay=false -ve
Setting pipeline to PAUSED ...
mpi: mpp version: dd3f0ee7 author: Herman Chen [enc_impl]: Rename and new enc_impl function
mpp_rt: NOT found ion allocator
mpp_rt: found drm allocator
rkcamsrc: Using ISP main path......
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[  103.627052] rkisp1_isp_sd_try_fmt drivers/media/platform/rockchip/isp1/rkisp1.c 
[  103.627720] stream 0 crop(0,0)/1920x1080
h264e_api: h264e_config MPP_ENC_SET_RC_CFG bps 15552000 [14580000 : 16524000]
/GstPipeline:pipeline0/GstRKCamSrc:rkcamsrc0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framrate=(fraction)30/1, framerate=(fraction)60/1, interlace-mode=(string)progressive, pixe4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framrate=(fraction)30/1, framerate=(fraction)60/1, interlace-mode=(string)progressive, 4
/GstPipeline:pipeline0/GstMppH264Enc:mpph264enc0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framrate=(fraction)30/1, framerate=(fraction)60/1, interlace-mode=(string)progressive,4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framrate=(fraction)30/1, framerate=(fraction)60/1, interlace-mode=(string)progressive,4
[  103.642069] rkisp1: MIPI mis error: 0x00000303
[  103.642492] rkisp1: MIPI mis error: 0x00000303
[  103.642900] rkisp1: MIPI mis error: 0x00000303
[  103.643307] rkisp1: MIPI mis error: 0x00000303
[  103.644484] rkisp1: MIPI mis error: 0x00300303
requesting 2 MMAP buffers/GstPipeline:pipeline0/GstMppH264Enc:mpph264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fract4
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-r4
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=4
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, fr4
[  103.855189] rk_vcodec: vpu2_enc task: 23 ms
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
[  103.970704] rk_vcodec: vpu2_enc task: 21 ms
[  104.070593] rk_vcodec: vpu2_enc task: 22 ms
[  104.182475] rk_vcodec: vpu2_enc task: 22 ms
[  104.292136] rk_vcodec: vpu2_enc task: 21 ms
[  104.388185] rk_vcodec: vpu2_enc task: 21 ms
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 6, dropped: 0, current: 11.26, average: 11.26
[  104.502755] rk_vcodec: vpu2_enc task: 22 ms
[  104.593283] rk_vcodec: vpu2_enc task: 21 ms
[  104.702783] rk_vcodec: vpu2_enc task: 21 ms
[  104.830163] rk_vcodec: vpu2_enc task: 21 ms
[  104.941063] rk_vcodec: vpu2_enc task: 21 ms
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 11, dropped: 0, current: 9.04, average: 10.13
[  105.061744] rk_vcodec: vpu2_enc task: 22 ms
[  105.170144] rk_vcodec: vpu2_enc task: 21 ms
[  105.273190] rk_vcodec: vpu2_enc task: 22 ms

it looks like, VPU is being used at expected performance but still i get lesser FPS.

Mani · October 22, 2019, 4:24am

@hipboi What is the max FPS you have observed with encoder?

vishnu_kumar · November 7, 2019, 6:45am

Any update on this issue ??

Is this the actual performance or something wrong with my package ?
Has anyone encoded 1080 @30fps with the rk3399 and gstreamer pipeline ?

joshvm · March 27, 2020, 8:13pm

I just tested, I can confirm I get the same performance (about 15fps) at least with the pre-built mpp libraries provided in rk-rootfs. Note that these aren’t super old, late 2019. Maybe you can do better if compiling from source. I’ve had problems with that though. If I build mpp from source, although it builds fine, it crashes with an error like:

mpp_device: mpp_device_send_reg ioctl VPU_IOC_SET_REG failed ret -1 errno 14 Bad address
hal_h264e_vepu2: mpp_device_send_reg failed ret 14
mpp_enc: mpp 0x55809fb880 hal_hw_start failed return 14

Probably once that works, you’d also want to reinstall gstreamer-rockchip.

See: https://github.com/rockchip-linux/mpp/issues/124

You can also test with mpi_enc_test -o test.mkv -w 1920 -h 1088 -f 0 -t 7 to make sure, and it’s clearly about 15fps.