Yocto kernel panic

Hi!

I tried to update yocto to zeus, meta-qcom and also used the latest (?) firmware (137). I get more often than not a boot kernel panic.

[    6.418693] asix 1-1.1:1.0 eth0: register 'asix' at usb-ci_hdrc.0-1.1, ASIX AX88772B USB 2.0 Ethernet, ac:db:da:59:4e:8f
[    6.418870] usbcore: registered new interface driver asix
[    6.578523] qcom_wcnss_ctrl remoteproc0:smd-edge.WCNSS_CTRL.-1.-1: WCNSS Version 1.5 1.2
[    8.223845] wcn36xx: mac address: 02:00:c7:c2:bd:0a
[    8.534723] wcn36xx: firmware WLAN version 'WCN v2.0 RadioPhy vRhea_GF_1.12 with 19.2MHz XO' and CRM version 'CNSS-PR-2-0-1-2-c1-74-130449-3'
[    8.534788] wcn36xx: firmware API 1.5.1.2, 41 stations, 2 bssids
[    8.937549] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[    8.937595] Mem abort info:
[    8.945399]   ESR = 0x96000004
[    8.947909]   Exception class = DABT (current EL), IL = 32 bits
[    8.951040]   SET = 0, FnV = 0
[    8.956848]   EA = 0, S1PTW = 0
[    8.959972] Data abort info:
[    8.963011]   ISV = 0, ISS = 0x00000004
[    8.966138]   CM = 0, WnR = 0
[    8.969697] user pgtable: 4k pages, 48-bit VAs, pgdp=00000000fc8b5000
[    8.972830] [0000000000000000] pgd=0000000000000000
[    8.979249] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[    8.983937] Modules linked in: wcn36xx(+) aes_neon_blk wcnss_ctrl qrtr_smd cp210x asix usbnet mii overlay qrtr qcom_wcnss_pil mdt_loader qcom_sysmon qmi_helpers qcom_common qcom_glink_smem qcom_glink_native remoteproc ci_hdrc_msm crct10dif_ce virtio ci_hdrc virtio_ring qcom_rng clk_smd_rpm rpmsg_char sch_fq_codel pkcs8_key_parser
[    8.996470] CPU: 0 PID: 1281 Comm: rngd Not tainted 5.2.32-yocto-tiny #1
[    9.018664] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[    9.025609] pstate: 60000085 (nZCv daIf -PAN -UAO)
[    9.032295] pc : __queue_work+0x90/0x368
[    9.036889] lr : __queue_work+0x8c/0x368
[    9.040965] sp : ffff000010003ad0
[    9.044870] x29: ffff000010003ad0 x28: 0000000000000000 
[    9.048085] x27: 0000000000000000 x26: ffff000010fbd670 
[    9.053467] x25: ffff000010dd8618 x24: 0000000000000004 
[    9.058762] x23: 000000000000002e x22: ffff80007a9f4e00 
[    9.064058] x21: ffff000010dc1018 x20: ffff800077b60eb8 
[    9.069352] x19: 0000000000000000 x18: 0000000000000000 
[    9.074647] x17: 0000000fffffffe1 x16: 0000000fffffffe0 
[    9.079943] x15: 0000000000000000 x14: 000021f535cb9b98 
[    9.085238] x13: 14310064ffffe37c x12: ffff800075c7e220 
[    9.090534] x11: 0000000000000000 x10: 0000000000000000 
[    9.095829] x9 : 0000000000000000 x8 : 0000000000000000 
[    9.101124] x7 : ffff800077b60840 x6 : 0000000000000000 
[    9.106419] x5 : 0000000000000000 x4 : ffff80007a9f4e00 
[    9.111713] x3 : 0000000000000004 x2 : ffff800077b60eb8 
[    9.117008] x1 : 000000007fffffff x0 : 0000000000000000 
[    9.122305] Call trace:
[    9.127599]  __queue_work+0x90/0x368
[    9.129771]  queue_work_on+0x88/0x98
[    9.133591]  ieee80211_queue_work+0x30/0x48
[    9.137150]  ieee80211_rx_handlers+0x1978/0x1fa0
[    9.141056]  ieee80211_prepare_and_rx_handle+0x890/0x1038
[    9.145918]  ieee80211_rx_napi+0x570/0x9b0
[    9.151213]  ieee80211_tasklet_handler+0xcc/0xd0
[    9.155208]  tasklet_action_common.isra.0+0x9c/0x128
[    9.159981]  tasklet_action+0x24/0x30
[    9.164925]  __do_softirq+0x120/0x23c
[    9.168484]  irq_exit+0xb8/0xd8
[    9.172129]  __handle_domain_irq+0x64/0xb8
[    9.175082]  gic_handle_irq+0x50/0xa0
[    9.179248]  el0_irq_naked+0x4c/0x54
[    9.182983] Code: 8b010273 aa1403e0 97fffbae aa0003fc (f9400261) 
[    9.186633] ---[ end trace 08f73dd772ab00ad ]---
[    9.192617] Kernel panic - not syncing: Fatal exception in interrupt
[    9.197310] SMP: stopping secondary CPUs
[    9.203642] Kernel Offset: disabled
[    9.207544] CPU features: 0x0002,24002001
[    9.210756] Memory Limit: none
[    9.214934] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

It is not always the same dump, but it always involves ieee80211_xx_yy

Any hints what goes wrong here?!

Regards,

Matthias

Look like you’re no using the kernel defined by meta-qcom (5.4).

No, it is the linux-yocto-tiny. I will retest with the official linaro

I would not say it is that much better, to be honest.

Also, the new firmware requires new partitioning. We try to develop a product based on the dragonboard. How often is this going to change in the future, since repartitioning with OTA Update is quite difficult, if not inpossible, if it involves moving of partitions.

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd030]
[    0.000000] Linux version 5.4.17+linaro (oe-user@oe-host) (gcc version 9.2.0 (GCC)) #1 SMP PREEMPT Tue Mar 24 08:58:41 UTC 2020
[...]
[    3.219284] usb 1-1.3: new full-speed USB device number 4 using ci_hdrc
/lib/systemd/system/syslogd.service:8: PIDFile= references a path below legacy directory /var/run/, updating /var/run/syslogd.pid → /run/syslogd.pid; please update the unit file accordingly.
Configuration file /lib/systemd/system/qrtr-ns.service is marked executable. Please remove executable permission bits. Proceeding anyway.
/lib/systemd/system/klogd.service:8: PIDFile= references a path below legacy directory /var/run/, updating /var/run/klogd.pid → /run/klogd.pid; please update the unit file accordingly.
/lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/dbus/system_bus_socket → /run/dbus/system_bus_socket; please update the unit file according[    3.436086] random: systemd: uninitialized urandom read (16 bytes read)
ly.
[    3.712966] systemd-journald[203]: Received client request to flush runtime journal.
[    4.887541] mc: Linux media interface: v0.10
[    4.983423] rtc-pm8xxx 200f000.spmi:pm8916@0:rtc@6000: registered as rtc0
[    4.984987] qcom-apq8016-sbc 7702000.sound: error getting cpu dai name
[    5.000533] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.043124] qcom-apq8016-sbc 7702000.sound: error getting cpu dai name
[    5.043164] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.054835] qcom-apq8016-sbc 7702000.sound: error getting cpu dai name
[    5.055509] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.124288] remoteproc remoteproc0: 4080000.remoteproc is available
[    5.127875] qcom-apq8016-sbc 7702000.sound: error getting cpu dai name
[    5.129368] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.154591] videodev: Linux video capture interface: v2.00
[    5.156624] apq8016-lpass-cpu 7708000.lpass: asoc_qcom_lpass_cpu_platform_probe() error getting optional mi2s-osr-clk0: -2
[    5.159011] apq8016-lpass-cpu 7708000.lpass: asoc_qcom_lpass_cpu_platform_probe() error getting optional mi2s-osr-clk1: -2
[    5.195568] apq8016-lpass-cpu 7708000.lpass: asoc_qcom_lpass_cpu_platform_probe() error getting optional mi2s-osr-clk2: -2
[    5.195637] apq8016-lpass-cpu 7708000.lpass: asoc_qcom_lpass_cpu_platform_probe() error getting optional mi2s-osr-clk3: -2
[    5.266849] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    5.266892] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.371371] EXT4-fs (mmcblk0p19): recovery complete
[    5.372189] EXT4-fs (mmcblk0p19): mounted filesystem with ordered data mode. Opts: (null)
[    5.435338] qcom-venus 1d00000.video-codec: Adding to iommu group 0
[    5.598917] qcom-wcnss-pil a204000.wcnss: a204000.wcnss supply vddcx not found, using dummy regulator
[    5.602503] remoteproc remoteproc1: a204000.wcnss is available
[    5.641808] Can't find a SQUASHFS superblock on mmcblk0p14
[    5.645121] remoteproc remoteproc1: powering up a204000.wcnss
[    5.646197] remoteproc remoteproc1: Booting fw image qcom/msm8916/wcnss.mdt, size 7260
[    5.662995] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    5.663038] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.683072] adreno 1c00000.gpu: Adding to iommu group 1
[    5.689424] msm_mdp 1a01000.mdp: Adding to iommu group 2
[    5.689561] msm_mdp 1a01000.mdp: No interconnect support may cause display underflows!
[    5.691953] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    5.712044] adv7511 3-0039: 3-0039 supply dvdd not found, using dummy regulator
[    5.712146] adv7511 3-0039: 3-0039 supply pvdd not found, using dummy regulator
[    5.726160] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.730526] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    5.735251] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.735556] adv7511 3-0039: 3-0039 supply a2vdd not found, using dummy regulator
[    5.746721] EXT4-fs (mmcblk0p21): recovery complete
[    5.747398] alloc_contig_range: [fe088, fe08a) PFNs busy
[    5.753694] EXT4-fs (mmcblk0p21): mounted filesystem with ordered data mode. Opts: (null)
[    5.784506] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    5.784548] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    5.792613] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    5.804883] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    5.805097] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    5.821570] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    5.822062] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    5.880271] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    5.885472] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    5.902999] msm 1a00000.mdss: master bind failed: -517
[    5.903185] platform 1a01000.mdp: Removing from iommu group 2
[    5.920764] ------------[ cut here ]------------
[    5.920809] WARNING: CPU: 1 PID: 238 at drivers/iommu/qcom_iommu.c:348 qcom_iommu_domain_free+0x5c/0x70
[    5.924457] Modules linked in: adv7511 crct10dif_ce msm(+) qcom_wcnss_pil cec venus_core(+) v4l2_mem2mem videobuf2_v4l2 videobuf2_common snd_soc_lpass_apq8016 videodev snd_soc_lpass_cpu snd_soc_lpass_platform qcom_q6v5_mss qcom_q6v5 qcom_common snd_soc_msm8916_analog qcom_rng snd_soc_apq8016_sbc snd_soc_msm8916_digital rtc_pm8xxx qcom_glink_smem mc socinfo mdt_loader sch_fq_codel
[    5.945049] CPU: 1 PID: 238 Comm: systemd-udevd Not tainted 5.4.17+linaro #1
[    5.967242] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[    5.974450] pstate: 00000005 (nzcv daif -PAN -UAO)
[    5.981131] pc : qcom_iommu_domain_free+0x5c/0x70
[    5.985733] lr : iommu_group_release+0x48/0x80
[    5.990502] sp : ffff8000120d35f0
[    5.994840] x29: ffff8000120d35f0 x28: ffff800012a75000 
[    5.998228] x27: 0000000000000100 x26: ffff800010181bc0 
[    6.003609] x25: ffff0000773a2580 x24: 0000000000000003 
[    6.008905] x23: ffff0000796ca010 x22: ffff000079775858 
[    6.014201] x21: ffff00007785e080 x20: ffff00007785e200 
[    6.019495] x19: ffff000079775800 x18: 0000000000000000 
[    6.024790] x17: 0000000000000000 x16: 0000000000000000 
[    6.030085] x15: 0000000000000004 x14: 0000000000000000 
[    6.035381] x13: 0000000000000000 x12: ffff000079dbaed8 
[    6.040676] x11: ffff000079dbada0 x10: 0000000000000000 
[    6.045972] x9 : ffff00000eb2c098 x8 : ffff00000eb2c090 
[    6.051266] x7 : 00000000400c0000 x6 : 0000000000000006 
[    6.056561] x5 : 0000000000000001 x4 : 0000000000000000 
[    6.061856] x3 : 0000000000000000 x2 : 0000000000000000 
[    6.067152] x1 : ffff00007aa64880 x0 : ffff00007785e230 
[    6.072447] Call trace:
[    6.077739]  qcom_iommu_domain_free+0x5c/0x70
[    6.079915]  iommu_group_release+0x48/0x80
[    6.084428]  kobject_put+0x8c/0x1f0
[    6.088429]  kobject_del+0x50/0x68
[    6.091803]  kobject_put+0x74/0x1f0
[    6.095276]  iommu_group_remove_device+0xd8/0xec
[    6.098664]  qcom_iommu_remove_device+0x44/0x60
[    6.103525]  iommu_release_device+0x28/0x40
[    6.107776]  iommu_bus_notifier+0xac/0xc0
[    6.111944]  notifier_call_chain+0x58/0x98
[    6.116111]  blocking_notifier_call_chain+0x54/0x78
[    6.120106]  device_del+0x224/0x348
[    6.124878]  platform_device_del.part.0+0x18/0x88
[    6.128352]  platform_device_unregister+0x20/0x38
[    6.133214]  of_platform_device_destroy+0xdc/0xf0
[    6.137900]  device_for_each_child+0x58/0xa0
[    6.142584]  of_platform_depopulate+0x38/0x78
[    6.147050]  msm_pdev_probe+0x1c0/0x308 [msm]
[    6.151183]  platform_drv_probe+0x50/0xa0
[    6.155522]  really_probe+0xd8/0x428
[    6.159512]  driver_probe_device+0xdc/0x130
[    6.163159]  device_driver_attach+0x6c/0x78
[    6.167065]  __driver_attach+0x9c/0x168
[    6.171230]  bus_for_each_dev+0x6c/0xc0
[    6.175049]  driver_attach+0x20/0x28
[    6.178870]  bus_add_driver+0x138/0x218
[    6.182690]  driver_register+0x60/0x110
[    6.186249]  __platform_driver_register+0x44/0x50
[    6.190187]  msm_drm_register+0x54/0x68 [msm]
[    6.194938]  do_one_initcall+0x50/0x190
[    6.199272]  do_init_module+0x50/0x208
[    6.202916]  load_module+0x1eb8/0x2498
[    6.206735]  __do_sys_finit_module+0xac/0x110
[    6.210470]  __arm64_sys_finit_module+0x1c/0x28
[    6.214899]  el0_svc_common.constprop.0+0x68/0x160
[    6.219238]  el0_svc_handler+0x20/0x80
[    6.224096]  el0_svc+0x8/0xc
[    6.227826] ---[ end trace bfca547262fd152a ]---
[    6.534751] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    6.534793] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    6.547372] msm_mdp 1a01000.mdp: Adding to iommu group 2
[    6.547515] msm_mdp 1a01000.mdp: No interconnect support may cause display underflows!
[    6.561956] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    6.561996] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    6.573168] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    6.574861] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    6.587416] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    6.589511] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.597170] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.607311] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    6.615537] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    6.620308] msm 1a00000.mdss: master bind failed: -517
[    6.650800] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    6.650840] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    6.659617] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    6.667532] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    6.671397] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    6.680575] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.684659] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.699343] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    6.700929] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    6.708369] msm 1a00000.mdss: master bind failed: -517
[    6.753009] usbcore: registered new interface driver cp210x
[    6.753090] usbserial: USB Serial support registered for cp210x
[    6.763402] cp210x 1-1.3:1.0: cp210x converter detected
[    6.770106] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    6.770147] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    6.779098] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    6.783178] usb 1-1.3: cp210x converter now attached to ttyUSB0
[    6.795714] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    6.796494] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    6.803447] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.811725] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.818076] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    6.855544] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    6.855749] msm 1a00000.mdss: master bind failed: -517
[    6.876206] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    6.876246] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    6.890315] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    6.890771] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    6.919410] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    6.919551] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.925109] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    6.933055] remoteproc remoteproc1: remote processor a204000.wcnss is now up
[    6.942286] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    6.948792] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    6.959410] msm 1a00000.mdss: master bind failed: -517
[    6.965084] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    6.968095] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    6.986196] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    6.986639] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    6.993211] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    6.999989] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    7.011410] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    7.014556] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    7.024195] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    7.031195] msm 1a00000.mdss: master bind failed: -517
[    7.062852] qcom_wcnss_ctrl remoteproc1:smd-edge.WCNSS_CTRL.-1.-1: WCNSS Version 1.5 1.2
[    7.064658] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    7.082843] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    7.086139] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    7.100246] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    7.100410] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    7.115681] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    7.125806] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    7.126178] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    7.137605] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    7.140687] msm 1a00000.mdss: master bind failed: -517
[    7.302430] remoteproc remoteproc0: powering up 4080000.remoteproc
[    7.439645] remoteproc remoteproc0: Booting fw image qcom/msm8916/mba.mbn, size 230272
[    7.489769] qcom-q6v5-mss 4080000.remoteproc: MBA booted, loading mpss
[    8.730533] wcn36xx: mac address: 02:00:c7:c2:bd:0a
[    8.748299] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    8.748340] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    8.758861] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    8.765355] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    8.773087] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    8.802465] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    8.802697] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    8.815549] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    8.818068] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    8.828992] msm 1a00000.mdss: master bind failed: -517
[    9.101682] Bluetooth: Core ver 2.22
[    9.101873] NET: Registered protocol family 31
[    9.114808] Bluetooth: HCI device and connection manager initialized
[    9.114854] Bluetooth: HCI socket layer initialized
[    9.132233] Bluetooth: L2CAP socket layer initialized
[    9.132293] Bluetooth: SCO socket layer initialized
[    9.183280] random: crng init done
[    9.183311] random: 7 urandom warning(s) missed due to ratelimiting
[    9.195038] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[    9.195079] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[    9.216923] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[    9.217380] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[    9.225861] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[    9.231812] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    9.238172] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[    9.251571] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[    9.253426] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[    9.262112] msm 1a00000.mdss: master bind failed: -517
[    9.411495] wcn36xx: firmware WLAN version 'WCN v2.0 RadioPhy vRhea_GF_1.12 with 19.2MHz XO' and CRM version 'CNSS-PR-2-0-1-2-c1-74-130449-3'
[    9.411551] wcn36xx: firmware API 1.5.1.2, 41 stations, 2 bssids
[   10.163595] remoteproc remoteproc0: remote processor 4080000.remoteproc is now up
[   10.165889] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[   10.171556] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[   10.186292] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[   10.186730] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[   10.194990] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[   10.200972] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[   10.207465] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[   10.216212] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[   10.224806] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[   10.232831] msm 1a00000.mdss: master bind failed: -517
[   10.242589] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[   10.242634] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[   10.257329] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[   10.257815] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[   10.267651] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[   10.271143] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[   10.281281] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[   10.288796] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[   10.294609] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[   10.304093] msm 1a00000.mdss: master bind failed: -517
[   10.500389] qcom-apq8016-sbc 7702000.sound: error getting codec dai name
[   10.500435] qcom-apq8016-sbc 7702000.sound: Error resolving dai links: -517
[   10.522799] msm 1a00000.mdss: 1a00000.mdss supply vdd not found, using dummy regulator
[   10.524755] msm_mdp 1a01000.mdp: [drm:mdp5_bind [msm]] MDP5 version v1.6
[   10.531389] msm 1a00000.mdss: bound 1a01000.mdp (ops mdp5_ops [msm])
[   10.539783] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[   10.543067] msm_dsi 1a98000.dsi: 1a98000.dsi supply gdsc not found, using dummy regulator
[   10.555581] msm_dsi_manager_register: failed to register mipi dsi host for DSI 0
[   10.559288] msm 1a00000.mdss: failed to bind 1a98000.dsi (ops dsi_ops [msm]): -517
[   10.566718] msm 1a00000.mdss: master bind failed: -517
[   11.331246] Bluetooth: hci0: command 0xfc0b tx timeout

Any help is appreciated!!

Thanks and Regards,

Matthias

It seems to boot from the logs. right?

Not sure what changed for you. But partition table is quite stable. I wonder why your rootfs is mounted from mmcblk0p19 since it supposed to be mmcblk0p14…

We use to build OpenEmbedded build for dragonboard410c: https://releases.linaro.org/96boards/dragonboard410c/linaro/openembedded/20.02/

So maybe you could compare to this reference build.
FYI, instructions are here: https://github.com/96boards/oe-rpb-manifest/blob/qcom/zeus/README.md

Hi Loic,

thanks for your answer.

It did boot, but I wonder if I should like this trace:

[    5.920764] ------------[ cut here ]------------
[    5.920809] WARNING: CPU: 1 PID: 238 at drivers/iommu/qcom_iommu.c:348 qcom_iommu_domain_free+0x5c/0x70
[    5.924457] Modules linked in: adv7511 crct10dif_ce msm(+) qcom_wcnss_pil cec venus_core(+) v4l2_mem2mem videobuf2_v4l2 videobuf2_common snd_soc_lpass_apq8016 videodev snd_soc_lpass_cpu snd_soc_lpass_platform qcom_q6v5_mss qcom_q6v5 qcom_common snd_soc_msm8916_analog qcom_rng snd_soc_apq8016_sbc snd_soc_msm8916_digital rtc_pm8xxx qcom_glink_smem mc socinfo mdt_loader sch_fq_codel
[    5.945049] CPU: 1 PID: 238 Comm: systemd-udevd Not tainted 5.4.17+linaro #1
[    5.967242] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[    5.974450] pstate: 00000005 (nzcv daif -PAN -UAO)
[    5.981131] pc : qcom_iommu_domain_free+0x5c/0x70
[    5.985733] lr : iommu_group_release+0x48/0x80
[    5.990502] sp : ffff8000120d35f0
[    5.994840] x29: ffff8000120d35f0 x28: ffff800012a75000 
[    5.998228] x27: 0000000000000100 x26: ffff800010181bc0 
[    6.003609] x25: ffff0000773a2580 x24: 0000000000000003 
[    6.008905] x23: ffff0000796ca010 x22: ffff000079775858 
[    6.014201] x21: ffff00007785e080 x20: ffff00007785e200 
[    6.019495] x19: ffff000079775800 x18: 0000000000000000 
[    6.024790] x17: 0000000000000000 x16: 0000000000000000 
[    6.030085] x15: 0000000000000004 x14: 0000000000000000 
[    6.035381] x13: 0000000000000000 x12: ffff000079dbaed8 
[    6.040676] x11: ffff000079dbada0 x10: 0000000000000000 
[    6.045972] x9 : ffff00000eb2c098 x8 : ffff00000eb2c090 
[    6.051266] x7 : 00000000400c0000 x6 : 0000000000000006 
[    6.056561] x5 : 0000000000000001 x4 : 0000000000000000 
[    6.061856] x3 : 0000000000000000 x2 : 0000000000000000 
[    6.067152] x1 : ffff00007aa64880 x0 : ffff00007785e230 
[    6.072447] Call trace:
[    6.077739]  qcom_iommu_domain_free+0x5c/0x70
[    6.079915]  iommu_group_release+0x48/0x80
[    6.084428]  kobject_put+0x8c/0x1f0
[    6.088429]  kobject_del+0x50/0x68
[    6.091803]  kobject_put+0x74/0x1f0
[    6.095276]  iommu_group_remove_device+0xd8/0xec
[    6.098664]  qcom_iommu_remove_device+0x44/0x60
[    6.103525]  iommu_release_device+0x28/0x40
[    6.107776]  iommu_bus_notifier+0xac/0xc0
[    6.111944]  notifier_call_chain+0x58/0x98
[    6.116111]  blocking_notifier_call_chain+0x54/0x78
[    6.120106]  device_del+0x224/0x348
[    6.124878]  platform_device_del.part.0+0x18/0x88
[    6.128352]  platform_device_unregister+0x20/0x38
[    6.133214]  of_platform_device_destroy+0xdc/0xf0
[    6.137900]  device_for_each_child+0x58/0xa0
[    6.142584]  of_platform_depopulate+0x38/0x78
[    6.147050]  msm_pdev_probe+0x1c0/0x308 [msm]
[    6.151183]  platform_drv_probe+0x50/0xa0
[    6.155522]  really_probe+0xd8/0x428
[    6.159512]  driver_probe_device+0xdc/0x130
[    6.163159]  device_driver_attach+0x6c/0x78
[    6.167065]  __driver_attach+0x9c/0x168
[    6.171230]  bus_for_each_dev+0x6c/0xc0
[    6.175049]  driver_attach+0x20/0x28
[    6.178870]  bus_add_driver+0x138/0x218
[    6.182690]  driver_register+0x60/0x110
[    6.186249]  __platform_driver_register+0x44/0x50
[    6.190187]  msm_drm_register+0x54/0x68 [msm]
[    6.194938]  do_one_initcall+0x50/0x190
[    6.199272]  do_init_module+0x50/0x208
[    6.202916]  load_module+0x1eb8/0x2498
[    6.206735]  __do_sys_finit_module+0xac/0x110
[    6.210470]  __arm64_sys_finit_module+0x1c/0x28
[    6.214899]  el0_svc_common.constprop.0+0x68/0x160
[    6.219238]  el0_svc_handler+0x20/0x80
[    6.224096]  el0_svc+0x8/0xc
[    6.227826] ---[ end trace bfca547262fd152a ]---

We are coming from https://releases.linaro.org/96boards/dragonboard410c/linaro/rescue/17.09/dragonboard410c_bootloader_emmc_linux-88.zip where the partition scheme was to our best knowledge

cdt,2,,A19F205F-CCD8-4B6D-8F1E-2D9BC24CFFB1,,sbc_1.0_8016.bin
sbl1,512,,DEA0BA2C-CBDD-4805-B4F9-F428251C3E98,,sbl1.mbn
rpm,512,,098DF793-D712-413D-9D4E-89D711772228,,rpm.mbn
tz,1024,,A053AA7F-40B8-4B1C-BA08-2F68AC71A4F4,,tz.mbn
hyp,512,,E1A6A689-0C8D-4CC6-B4E8-55A4320FBD8A,,hyp.mbn
sec,16,65536,303E6AC3-AF15-4C54-9E9B-D9A8FBECF401,,
aboot,1024,,400FFDCD-22E0-47E7-9A23-F16ED9382388,,emmc_appsboot.mbn
boot,65536,,20117F86-E985-4357-B9EE-374BC1D8487D,,u-boot-lyd-db410c.img

now we went to https://releases.linaro.org/96boards/dragonboard410c/linaro/rescue/20.02/dragonboard-410c-bootloader-emmc-linux-137.zip where the partition scheme is (again on what we think is right)

cdt,2,,A19F205F-CCD8-4B6D-8F1E-2D9BC24CFFB1,,sbc_1.0_8016.bin
sbl1,512,,DEA0BA2C-CBDD-4805-B4F9-F428251C3E98,,sbl1.mbn
rpm,512,,098DF793-D712-413D-9D4E-89D711772228,,rpm.mbn
tz,1024,,A053AA7F-40B8-4B1C-BA08-2F68AC71A4F4,,tz.mbn
hyp,512,,E1A6A689-0C8D-4CC6-B4E8-55A4320FBD8A,,hyp.mbn
sec,16,65536,303E6AC3-AF15-4C54-9E9B-D9A8FBECF401,,
modemst1,2048,,EBBEADAF-22C9-E33B-8F5D-0E81686A68CB,,
modemst2,2048,,0A288B1F-22C9-E33B-8F5D-0E81686A68CB,,
fsc,1,,57B90A16-22C9-E33B-8F5D-0E81686A68CB,,
fsg,2048,,638FF8E2-22C9-E33B-8F5D-0E81686A68CB,,fs_image_linux.tar.gz.mbn.img
aboot,1024,,400FFDCD-22E0-47E7-9A23-F16ED9382388,,emmc_appsboot.mbn

with modemst1, modemst2 fsc and fsg partition added. That is why I experience a partition change here. We start at boot, so our setup from the bootlog you see is like this

cdt,2,,A19F205F-CCD8-4B6D-8F1E-2D9BC24CFFB1,,sbc_1.0_8016.bin
sbl1,512,,DEA0BA2C-CBDD-4805-B4F9-F428251C3E98,,sbl1.mbn
rpm,512,,098DF793-D712-413D-9D4E-89D711772228,,rpm.mbn
tz,1024,,A053AA7F-40B8-4B1C-BA08-2F68AC71A4F4,,tz.mbn
hyp,512,,E1A6A689-0C8D-4CC6-B4E8-55A4320FBD8A,,hyp.mbn
sec,16,65536,303E6AC3-AF15-4C54-9E9B-D9A8FBECF401,,
modemst1,2048,,EBBEADAF-22C9-E33B-8F5D-0E81686A68CB,,
modemst2,2048,,0A288B1F-22C9-E33B-8F5D-0E81686A68CB,,
fsc,1,,57B90A16-22C9-E33B-8F5D-0E81686A68CB,,
fsg,2048,,638FF8E2-22C9-E33B-8F5D-0E81686A68CB,,fs_image_linux.tar.gz.mbn.img
aboot,1024,,400FFDCD-22E0-47E7-9A23-F16ED9382388,,emmc_appsboot.mbn
boot,65536,,20117F86-E985-4357-B9EE-374BC1D8487D,,u-boot-db410c.img
devinfo,1024,,1B81E7E6-F50D-419B-A739-2AEEF8DA3335,,
devicedata,1024,,45F2A301-9C70-40E6-84CB-CB1F80AE3629,,

kernel0,128M,,,fastboot,fitImage
kernel1,128M,,,fastboot,fitImage
# Redundant root filesystems
rootfs0,2147M,,,fastboot,image-debug-db410c.squashfs-xz
rootfs1,2147M,,,fastboot,image-debug-db410c.squashfs-xz

# Data partitions
data0,1074M,,83,ext4,
data1,1074M,,83,ext4,

Where we use uboot in boot, have an additional partition for our production data (serial number an what not), and then kernel and rootfs in a/b partitioning scheme. That is why we end up with p19 at the end of the day.

I will have a look at the zeus you pointed me to. It really should all work with the normal kernel, should it not?

Regards,
Matthias

This issue is a wrong warning report from kernel, it should not cause any subsequent trouble. Actually this has been fixed upstream if you want to backport the fix now (faf305c51aeabd1ea2d7131e798ef5f55f4a7750).

Hi Loic,

thanks for the hint, I will check the commit out.

I still see the original dump though, but only sometimes, not 100% reproducable. Do you have any idea, what the cause may be?!

[    8.206287] wcn36xx: firmware WLAN version 'WCN v2.0 RadioPhy vRhea_GF_1.12 with 19.2MHz XO' and CRM version 'CNSS-PR-2-0-1-2-c1-74-130449-3'
[    8.206343] wcn36xx: firmware API 1.5.1.2, 41 stations, 2 bssids
[    8.661776] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[    8.661817] Mem abort info:
[    8.669615]   ESR = 0x96000004
[    8.672132]   EC = 0x25: DABT (current EL), IL = 32 bits
[    8.675261]   SET = 0, FnV = 0
[    8.680728]   EA = 0, S1PTW = 0
[    8.683592] Data abort info:
[    8.686627]   ISV = 0, ISS = 0x00000004
[    8.689753]   CM = 0, WnR = 0
[    8.693312] user pgtable: 4k pages, 48-bit VAs, pgdp=00000000f99a4000
[    8.696445] [0000000000000000] pgd=0000000000000000
[    8.702866] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[    8.707552] Modules linked in: btqcomsmd btqca bluetooth ecdh_generic ecc wcn36xx(+) aes_neon_blk qrtr_smd cp210x overlay crct10dif_ce qcom_wcnss_pil qrtr qcom_q6v5_mss mdt_loader qcom_sysmon rtc_pm8xxx rmtfs_mem qcom_q6v5 qmi_helpers socinfo qcom_rng qcom_common qcom_glink_smem rpmsg_char sch_fq_codel
[    8.717642] CPU: 0 PID: 198 Comm: rngd Not tainted 5.4.17+linaro #1
[    8.739846] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[    8.746098] pstate: 60000085 (nZCv daIf -PAN -UAO)
[    8.753042] pc : __queue_work+0x90/0x378
[    8.757637] lr : __queue_work+0x8c/0x378
[    8.761716] sp : ffff800010003af0
[    8.765620] x29: ffff800010003af0 x28: 0000000000000000 
[    8.768836] x27: 0000000000000000 x26: ffff800011a6d768 
[    8.774218] x25: ffff8000118498e0 x24: 0000000000000100 
[    8.779512] x23: 0000000000000016 x22: ffff00007721d400 
[    8.784809] x21: ffff800011445018 x20: ffff000077956f00 
[    8.790104] x19: 0000000000000000 x18: 0000000000000000 
[    8.795398] x17: 0000000000000000 x16: 0000000000000000 
[    8.800694] x15: 0000000000000000 x14: 000021f535cb9b98 
[    8.805989] x13: 14310064ffffe3e0 x12: ffff0000754cc220 
[    8.811284] x11: 0000000000000000 x10: 0000000000000000 
[    8.816580] x9 : 0000000000000000 x8 : 0000000000000000 
[    8.821874] x7 : ffff000077956840 x6 : 0000000000000000 
[    8.827170] x5 : 0000000000000000 x4 : 0000000fffffffe1 
[    8.832465] x3 : 0000000fffffffe0 x2 : ffff000077956f00 
[    8.837760] x1 : 000000007fffffff x0 : 0000000000000000 
[    8.843055] Call trace:
[    8.848346]  __queue_work+0x90/0x378
[    8.850519]  queue_work_on+0x70/0x90
[    8.854342]  ieee80211_queue_work+0x30/0x48
[    8.857900]  ieee80211_rx_handlers+0x18d8/0x1f48
[    8.861807]  ieee80211_prepare_and_rx_handle+0x6b0/0xff8
[    8.866668]  ieee80211_rx_napi+0x2e0/0x960
[    8.871962]  ieee80211_tasklet_handler+0xcc/0xd0
[    8.875870]  tasklet_action_common.isra.0+0x154/0x1a0
[    8.880643]  tasklet_action+0x24/0x30
[    8.885589]  __do_softirq+0x120/0x23c
[    8.889234]  irq_exit+0xb8/0xd8
[    8.892879]  __handle_domain_irq+0x64/0xb8
[    8.895832]  gic_handle_irq+0x50/0xa0
[    8.899998]  el0_irq_naked+0x4c/0x54
[    8.903735] Code: 8b010273 aa1403e0 97fffab4 aa0003fc (f9400261) 
[    8.907388] ---[ end trace edc562715e4de26b ]---
[    8.913367] Kernel panic - not syncing: Fatal exception in interrupt
[    8.918059] SMP: stopping secondary CPUs
[    8.924393] Kernel Offset: disabled
[    8.928295] CPU features: 0x0002,24002005
[    8.931507] Memory Limit: none
[    8.935684] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

Thanks and Regards,

Matthias

Look like WiFi received a packet, which is then handled by ieee80211 tasklet, but for some reason this leads to NULL dereference. I don’t reproduce this issue, but it would be worth to investigate which pointer is NULL. This can be due to memory corruption, or problem in mac80211 management.

Hi Loic,

This issue caused me some headaches, I was able to find a workaround. There clearly is a race within wcn36xx driver. I just do not have the knowledge to find out, what is wrong. We use iwd, and the daemon starts before the driver is loaded (systemd network.target). This causes a 50% chance of the driver crashing in different ways and AFAIK from some irq, which makes it hard to debug. I believe, wcn36xx_probe has run, but wcn36xx_start is not through, when the irq hits. If I delay the start of iwd until wlan0 is up. I do not see this problem anymore.

Still, there seems to be a bug.

Regards,

Matthias

For the record, with the help of linaro, mac80211 race was fixed.

See this commit:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=52e04b4ce5d03775b6a78f3ed1097480faacc9fd

Regards,

Matthias