Gstreamer issue of DragonBoard 410c

Products Support DragonBoard410c
#22

So I finished compiling the “good” plugin set and encoding via x264enc appears to work until it crashes due to an oom error.

This is the pipeline I’m attempting to use:
gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1024 ! video/x-raw,format=UYVY,width=1920,height=1080,framerate=30/1 ! videoconvert ! x264enc ! matroskamux ! filesink location='raw_single.mkv' sync=true

Dmesg log:
[ 618.045353] qcom-camss 1b0ac00.camss: Vfe halt timeout [ 618.145472] qcom-camss 1b0ac00.camss: Vfe halt timeout [ 618.245497] qcom-camss 1b0ac00.camss: Vfe halt timeout [ 618.400753] 1b0c000.qcom,cci supply qcom,gdscr-vdd not found, using dummy regulator [ 618.402318] msm_cci_init:1016: hw_version = 0x10000008 [ 666.042085] v4l2src0:src invoked oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0 [ 666.042111] v4l2src0:src cpuset=/ mems_allowed=0 [ 666.052442] CPU: 3 PID: 3993 Comm: v4l2src0:src Not tainted 4.4.23-linaro-lt-qcom #2 [ 666.053919] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT) [ 666.061645] Call trace: [ 666.068335] [<ffffffc0000898e0>] dump_backtrace+0x0/0x15c [ 666.070504] [<ffffffc000089a50>] show_stack+0x14/0x1c [ 666.076060] [<ffffffc000342ef0>] dump_stack+0x98/0xb8 [ 666.081094] [<ffffffc0001b5bb4>] dump_header.isra.6+0x58/0x190 [ 666.086128] [<ffffffc000156edc>] oom_kill_process+0x368/0x42c [ 666.091857] [<ffffffc00015732c>] out_of_memory+0x30c/0x348 [ 666.097674] [<ffffffc00015c0a8>] __alloc_pages_nodemask+0x85c/0x898 [ 666.103057] [<ffffffc000155a78>] filemap_fault+0x198/0x4a4 [ 666.109219] [<ffffffc00023b19c>] ext4_filemap_fault+0x38/0x58 [ 666.114773] [<ffffffc00017a0b4>] __do_fault+0x3c/0x98 [ 666.120589] [<ffffffc00017e878>] handle_mm_fault+0xc00/0x1468 [ 666.125625] [<ffffffc000096dfc>] do_page_fault+0x1f0/0x2b4 [ 666.131354] [<ffffffc0000822b0>] do_mem_abort+0x44/0xa4 [ 666.136733] Exception stack(0xffffffc034a13e20 to 0xffffffc034a13f50) [ 666.141858] 3e20: 0000000000000000 0000000040a0b3fa ffffffffffffffff 0000007f7cc7077c [ 666.148456] 3e40: ffffffc034a13e80 ffffffc0000824f0 ffffff800000200c ffffffc034a13ec0 [ 666.156268] 3e60: ffffffc000cfd000 ffffff8000002010 ffffff8000003000 00000000fffff88b [ 666.164082] 3e80: ffffffc034a13ea0 ffffffc0000893bc 0000000000000008 0000007f744d3070 [ 666.171894] 3ea0: 0000007f78e579b0 ffffffc000085d28 0000000000000000 ffffffc034a10000 [ 666.179706] 3ec0: 00000000ffffffff 0000000000000001 0000000000000000 0000000000000000 [ 666.187519] 3ee0: 000000003f800000 0000000000000000 0000007f7cccbfd8 0000007f3cc89df0 [ 666.195327] 3f00: 0000007f6af3b610 0000007f6af4a600 0000000000000002 0000007f3d73cce0 [ 666.203145] 3f20: 0000007f3d78567e 00000000000003ff 0000000000000010 0000007f7c4b28b8 [ 666.210955] 3f40: 0000007f7c4db098 0000007f7cc70578 [ 666.218768] [<ffffffc000085ab8>] el0_ia+0x18/0x1c [ 666.223486] Mem-Info: [ 666.228329] active_anon:192594 inactive_anon:1962 isolated_anon:0 active_file:30 inactive_file:54 isolated_file:0 unevictable:0 dirty:0 writeback:0 unstable:0 slab_reclaimable:1649 slab_unreclaimable:5283 mapped:2206 shmem:2068 pagetables:665 bounce:0 free:5888 free_pcp:218 free_cma:76 [ 666.240838] DMA free:23552kB min:22528kB low:28160kB high:33792kB active_anon:770376kB inactive_anon:7848kB active_file:308kB inactive_file:276kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:951296kB managed:853300kB mlocked:0kB dirty:0kB writeback:0kB mapped:8972kB shmem:8272kB slab_reclaimable:6596kB slab_unreclaimable:21132kB kernel_stack:2832kB pagetables:2660kB unstable:0kB bounce:0kB free_pcp:672kB local_pcp:120kB free_cma:304kB writeback_tmp:0kB pages_scanned:4272 all_unreclaimable? yes [ 666.285968] lowmem_reserve[]: 0 0 0 [ 666.311636] DMA: 88*4kB (UEC) 201*8kB (UMEC) 684*16kB (UMEC) 141*32kB (UMEC) 21*64kB (UMEC) 5*128kB (UE) 2*256kB (ME) 3*512kB (UME) 1*1024kB (E) 0*2048kB 0*4096kB = 22472kB [ 666.326310] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [ 666.327114] 2426 total pagecache pages [ 666.335697] 0 pages in swap cache [ 666.339253] Swap cache stats: add 0, delete 0, find 0/0 [ 666.342630] Free swap = 0kB [ 666.347670] Total swap = 0kB [ 666.350788] 237824 pages RAM [ 666.353662] 0 pages HighMem/MovableOnly [ 666.356524] 24499 pages reserved [ 666.360084] 16384 pages cma reserved [ 666.363561] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name [ 666.367161] [ 1979] 0 1979 4683 183 9 3 0 0 systemd-journal [ 666.375552] [ 2020] 0 2020 3068 301 8 4 0 -1000 systemd-udevd [ 666.385098] [ 2315] 100 2315 20313 104 9 4 0 0 systemd-timesyn [ 666.394214] [ 2650] 0 2650 1730 123 7 4 0 0 systemd-logind [ 666.403849] [ 2665] 0 2665 1319 61 6 3 0 0 cron [ 666.412963] [ 2679] 0 2679 56495 218 12 3 0 0 rsyslogd [ 666.421644] [ 2689] 105 2689 1446 153 7 3 0 -900 dbus-daemon [ 666.430502] [ 2751] 0 2751 69997 614 24 3 0 0 NetworkManager [ 666.439708] [ 2777] 0 2777 450 36 4 3 0 0 acpid [ 666.448820] [ 2848] 0 2848 59005 1448 17 3 0 0 ModemManager [ 666.457500] [ 2965] 0 2965 58017 218 15 3 0 0 polkitd [ 666.466787] [ 3266] 0 3266 2083 160 8 3 0 -1000 sshd [ 666.475641] [ 3369] 0 3369 695 32 5 3 0 0 netserver [ 666.484076] [ 3472] 0 3472 1244 69 6 4 0 0 bluetoothd [ 666.492916] [ 3479] 0 3479 608 37 5 3 0 0 agetty [ 666.501854] [ 3486] 0 3486 1782 123 7 3 0 0 login [ 666.510535] [ 3676] 0 3676 1848 167 8 4 0 0 wpa_supplicant [ 666.519217] [ 3691] 104 3691 2462 177 8 4 0 0 exim4 [ 666.528504] [ 3702] 0 3702 2246 222 8 3 0 0 systemd [ 666.537184] [ 3706] 0 3706 2780 416 8 3 0 0 (sd-pam) [ 666.546125] [ 3711] 0 3711 1358 73 6 3 0 0 bash [ 666.554894] [ 3715] 0 3715 1123 213 6 3 0 0 dhclient [ 666.563404] [ 3919] 0 3919 3096 221 10 4 0 0 sshd [ 666.572258] [ 3921] 1000 3921 2360 232 8 3 0 0 systemd [ 666.580766] [ 3926] 1000 3926 23284 441 11 3 0 0 (sd-pam) [ 666.589629] [ 3929] 1000 3929 3096 225 10 4 0 0 sshd [ 666.598389] [ 3930] 1000 3930 1371 136 7 3 0 0 bash [ 666.606893] [ 3987] 1000 3987 288405 187847 415 4 0 0 gst-launch-1.0 [ 666.615400] Out of memory: Kill process 3987 (gst-launch-1.0) score 882 or sacrifice child [ 666.624640] Killed process 3987 (gst-launch-1.0) total-vm:1153620kB, anon-rss:745160kB, file-rss:6228kB

#23

hi,

Here are the instructions related to video encode that we are planning to add in the next release notes. If you can review them and try them out, that would be helpful. They are supposed to give you a working video encoder pipeline.

==
Starting with this release a basic video recording GStreamer pipeline is supported involving the camera and video encoder. Currently this has the following limitations:

  • The frame data must be vertically aligned to 32 lines. For the OV5645 camera this means that only 1280×960 resolution is supported.
  • A GStreamer video encoder plugin is needed which is not a part of the release. To enable this the user must patch and rebuild gstreamer1.0-plugins-good package – this is recommended for advanced users only.

To enable the video encoder plugin follow these steps. Install needed packages and dependencies:

sudo apt-get install build-essential fakeroot devscripts quilt
sudo apt-get build-dep gstreamer1.0-plugins-good

Get gstreamer1.0-plugins-good source:

apt-get source gstreamer1.0-plugins-good
cd gst-plugins-good1.0-1.10.4/

Get the video encoder plugin patches. These are the 12 last commits from here. Download them as patch files and to apply them do the following for each patch:

quilt import path_to_patch.patch
quilt push

Edit debian/rules and add --enable-v4l2-probe and --without-libv4l2 options to DEB_CONFIGURE_EXTRA_FLAGS.

Build gstreamer1.0-plugins-good package:

debuild -b -uc -us

Install the newly built gstreamer1.0-plugins-good package:

dpkg -i ../gstreamer1.0-plugins-good_1.10.4-1_arm64.deb

Now you are ready to do video recording! Configure the pipeline:

sudo media-ctl -d /dev/media1 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_pix":0[1]'
sudo media-ctl -d /dev/media1 -V '"ov5645 1-0076":0[fmt:UYVY8_2X8/1280x960 field:none],"msm_csiphy0":0[fmt:UYVY8_2X8/1280x960 field:none],"msm_csid0":0[fmt:UYVY8_2X8/1280x960 field:none],"msm_ispif0":0[fmt:UYVY8_2X8/1280x960 field:none],"msm_vfe0_pix":0[fmt:UYVY8_2X8/1280x960 field:none]'

And do video recording:

gst-launch-1.0 -e v4l2src device=/dev/video3 ! video/x-raw,format=NV12,width=1280,height=960,framerate=30/1 ! v4l2video5h264enc extra-controls="controls,h264_profile=4,video_bitrate=2000000;" ! h264parse ! mp4mux ! filesink location=video.mp4

When using the above command make sure that video device node numbers are correct or check and use the correct ones:

  • examine the output of media-ctl -d /dev/media1 -p and check that /dev/video3 is connected to msm_vfe0_pix
  • examine the output of gst-inspect-1.0 video4linux2 and check the name of the V4L2 H.264 Encoder plugin – e.g. v4l2video5h264enc
#24

Thanks for the early release notes, I’m going to try them against the latest #144 release.
Is this okay?

If not, is there a specific snapshot to use with these?

#25

hi,

you would need to use the latest ‘build’, not the latest ‘release’, e.g.

http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/227/

The ‘snapshots’ folder contain ‘daily’ builds or ‘work-in-progress’ toward the next release.

thanks for your help!

#26

So when running build #227 I was able to fully patch and compile gstreamer-plugins-good, but when I got to the part where I link all parts of the camera subsystem using the media-ctl command below:

sudo media-ctl -d /dev/media1 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_pix":0[1]'
I get the following error:
Failed to enumerate /dev/media1 (-2)

Could this be caused by not having the camera defined in the device tree? I know that on #144 the CSI camera was disabled and needed a patch to the apq8016-sbc.dtsi.

I think that this is the case because when looking through the /dev folder, I do not have a /dev/media1 interface. I only have a /dev/media0 interface.

If I change /dev/media1 for /dev/media0 I get the following error:

Opening media device /dev/media0
Enumerating entities
Found 1 entities
Enumerating pads and links
no such entity “msm_csiphy0”
media_parse_setup_link: Unable to parse link

“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_pix”:0[1]
^
Unable to parse link: Invalid argument (22)

I think that my next step is to dump the device tree from #227 to see if the camera is defined. If there is a patch that exists for #227’s device tree file could you share that here?

#27

that’s right. the camera are not enabled in the DTS , by default.

I forgot to send you the part of the update instructions that correspond to this new build. The following will be included in the release notes as well.

==
Please check commit “dts: Disable camera sensors in dtsi” in the kernel, it should mostly be a matter of reverting this patch. Note that this patch assumes that 2 OV5645 sensors are connected on each CSI port.

#28

Hey Nicolas,

I was able to get the camera working with the encoder after reverting this commit:
https://git.linaro.org/landing-teams/working/qualcomm/kernel.git/commit/?h=release/qcomlt-4.9&id=bffd64e24b45824e495532b0521cb5c5568c6ee0

I left the “front camera” disabled since I am only using a single camera in my implementation. I am now able to link the parts of the qualcomm camera subsystem and set the appropriate resolution and format. After configuring camss I am able to capture a video using the sample pipeline that you provided.

I have created a script to grab the gstreamer sources and patchset to make that a bit easier, would you like me to share it?

Thanks for all of your help!

#29

ok, that’s great! thanks for testing this out!

sure, feel free to share your helper scripts, i am sure it will help others.

#30

If anyone is interested, you can pull my patch script down from hastebin here.

This script will grab prerequisites for building gstreamer-plugins-good, grab the source, and the patches from fnoop’s repo.

#31

Is there a way to configure the pipelines at boot time? I tried adding the media-ctl commands to the /etc/init.d/rc.local file and it seems that it doesn’t execute them properly since the camera doesn’t initalize until I execute media-ctl from the shell.

What would you recommend? Should I create a script and a systemd service to execute them upon boot?

#32

Hi @anon91830841,

The video performance issue can be solved in new BSP 17.04/17.06, but for some reasons, I have to migrate the fixes into BSP 16.09.
Could you please advise what patches I should pick for the performance issue?

  • Update to the latest kernel 4.4 ?
  • Migrate meta-qcom/recipes-multimedia/gstreamer from morty branch?

Anything else I should take care? Thank you!

Best regards,
Dainel

#33

hi @daniel-hung,

most likely you need to ‘backport’ the entire v4l2 codec driver (in the kernel driver/media/platform/qcom/venus), and probably bits and pieces from the v4l2 core as well. You will need to update Gstreamer to 1.10 or 1.12 as well.

It won’t be a trivial task to backport the kernel. And unfortunately we won’t be able to help much on that since our focus is current LTS and newer version only…