We are observing on our several RB3 boards I/O error due to ufs_abort with below kernel console logs
| Line 12333: [ 76.561363][ T8] ------------[ cut here ]------------ | |
|---|---|
| Line 12335: [ 76.574523][ T8] WARNING: CPU: 5 PID: 8 at block/blk-core.c:589 blk_get_request+0x108/0x134 | |
| Line 12336: [ 76.574524][ T8] Modules linked in: ufs_qcom system_heap syscon_reboot_mode spmi_pmic_arb socinfo snd_soc_hdmi_codec smsm smp2p sdhci_msm rtc_pm8xxx rpmhpd rmtfs_mem reset_qcom_pdc reset_qcom_aoss qrtr_tun qrtr_smd qrtr qnoc_sdm845 qcom_tsens qcom_spmi_regulator qcom_q6v5_wcss qcom_q6v5_pas qcom_q6v5_mss qcom_q6v5_adsp qcom_sysmon qcom_q6v5 qcom_hwspinlock qcom_gsbi qcom_glink_rpm qcom_geni_serial qcom_aoss qcom_wdt qcom_spmi_pmic regmap_spmi qcom_rpmh_regulator qcom_pon reboot_mode qcom_pdc qcom_cpufreq_hw qcom_apcs_ipc_mailbox pm8941_pwrkey pm8916_wdt pinctrl_spmi_mpp pinctrl_spmi_gpio pinctrl_sdm845 pinctrl_msm8998 phy_qcom_usb_hs ulpi phy_qcom_qusb2 phy_qcom_qmp page_pool ohci_platform ohci_pci ohci_hcd nvmem_qfprom msm_serial msm mdt_loader lt9611 llcc_slice i2c_qup i2c_qcom_geni qcom_geni_se gpucc_sdm845 glink_ssr qcom_common qcom_smd qcom_glink_smem qcom_glink_native smem gcc_sdm845 gcc_msm8998 fastrpc extcon_usb_gpio dispcc_sdm845 deferred_free_helper cqhci clk_scmi | |
| Line 12337: [ 76.574587][ T8] clk_rpmh | |
| Line 12339: [ 76.591111][ T8] qcom_rpmh cmd_db clk_qcom ax88179_178a ath10k_snoc qmi_helpers ath10k_pci ath10k_core ath asix arm_smmu qcom_scm_64 apr | |
| Line 12340: [ 76.591124][ T8] CPU: 5 PID: 8 Comm: kworker/u16:0 Tainted: G W 5.4.127-android12-6-00329-g8451145ec97b-ab7477403 #1 | |
| Line 12341: [ 76.591125][ T8] Hardware name: Thundercomm Dragonboard 845c (DT) | |
| Line 12342: [ 76.591135][ T8] Workqueue: scsi_tmf_0 0xffffffd62ec1b890 | |
| Line 12344: [ 76.605255][ T8] pstate: 00c00005 (nzcv daif +PAN +UAO) | |
| Line 12345: [ 76.605257][ T8] pc : blk_get_request+0x108/0x134 | |
| Line 12346: [ 76.605264][ T8] lr : __ufshcd_issue_tm_cmd+0x60/0xcd4 | |
| Line 12347: [ 76.605265][ T8] sp : ffffffc010093b00 | |
| Line 12348: [ 76.605266][ T8] x29: ffffffc010093b30 x28: ffffff9bf1f74000 | |
| Line 12349: [ 76.605267][ T8] x27: 00000000ffffff9c x26: 0000000000000080 | |
| Line 12351: [ 76.700990][ T8] x25: ffffffd62ec0ab64 x24: ffffff9bf1f71250 | |
| Line 12352: [ 76.700992][ T8] x23: ffffffc010093c18 x22: 0000000080000000 | |
| Line 12353: [ 76.700993][ T8] x21: ffffffc010093b88 x20: ffffff9bf1f970e0 | |
| Line 12354: [ 76.700994][ T8] x19: 000000000000001f x18: ffffffc010095040 | |
| Line 12355: [ 76.700995][ T8] x17: ffffffd630e6cb48 x16: 00000000000000c0 | |
| Line 12356: [ 76.700996][ T8] x15: ffffffd630169a8c x14: ffffffd6307a6298 | |
| Line 12357: [ 76.700997][ T8] x13: 000000000000000a x12: 000000000000003d | |
| Line 12358: [ 76.700999][ T8] x11: 0000000000000004 x10: 0000000000008000 | |
| Line 12359: [ 76.701000][ T8] x9 : ffffffc010093b98 x8 : 00000000fffffff6 | |
| Line 12360: [ 76.701001][ T8] x7 : 6965636572202939 x6 : ffffffd630e8ccc7 | |
| Line 12361: [ 76.701002][ T8] x5 : 0000000000000000 x4 : 000000000000000c | |
| Line 12362: [ 76.701004][ T8] x3 : ffffffd6288ff8d0 x2 : 0000000000000002 | |
| Line 12363: [ 76.701005][ T8] x1 : 0000000000000023 x0 : ffffff9bf1f970e0 | |
| Line 12364: [ 76.701007][ T8] Call trace: | |
| Line 12365: [ 76.701009][ T8] blk_get_request+0x108/0x134 | |
| Line 12366: [ 76.701011][ T8] __ufshcd_issue_tm_cmd+0x60/0xcd4 | |
| Line 12367: [ 76.701015][ T8] ufshcd_try_to_abort_task+0xc4/0x370 | |
| Line 12369: [ 76.710077][ T8] ufshcd_abort+0x1a0/0x678 | |
| Line 12372: [ 76.734952][ T8] scmd_eh_abort_handler+0x84/0x15c | |
| Line 12373: [ 76.734957][ T8] process_one_work+0x2e0/0x77c | |
| Line 12374: [ 76.734958][ T8] worker_thread+0x3b0/0xa14 | |
| Line 12375: [ 76.734960][ T8] kthread+0x148/0x1b8 | |
| Line 12376: [ 76.734965][ T8] ret_from_fork+0x10/0x18 | |
| Line 12377: [ 76.734969][ T8] —[ end trace d4b259028b8d2c03 ]— | |
| Line 12380: [ 76.747161][ T8] ------------[ cut here ]------------ | |
| Line 12382: [ 76.762482][ T8] WARNING: CPU: 7 PID: 8 at block/blk-mq-tag.c:121 blk_mq_get_tag+0x498/0x54c | |
| Line 12383: [ 76.762483][ T8] Modules linked in: ufs_qcom system_heap syscon_reboot_mode spmi_pmic_arb socinfo snd_soc_hdmi_codec smsm smp2p sdhci_msm rtc_pm8xxx rpmhpd rmtfs_mem reset_qcom_pdc reset_qcom_aoss qrtr_tun qrtr_smd qrtr qnoc_sdm845 qcom_tsens qcom_spmi_regulator qcom_q6v5_wcss qcom_q6v5_pas qcom_q6v5_mss qcom_q6v5_adsp qcom_sysmon qcom_q6v5 qcom_hwspinlock qcom_gsbi qcom_glink_rpm qcom_geni_serial qcom_aoss qcom_wdt qcom_spmi_pmic regmap_spmi qcom_rpmh_regulator qcom_pon reboot_mode qcom_pdc qcom_cpufreq_hw qcom_apcs_ipc_mailbox pm8941_pwrkey pm8916_wdt pinctrl_spmi_mpp pinctrl_spmi_gpio pinctrl_sdm845 pinctrl_msm8998 phy_qcom_usb_hs ulpi phy_qcom_qusb2 phy_qcom_qmp page_pool ohci_platform ohci_pci ohci_hcd nvmem_qfprom msm_serial msm mdt_loader lt9611 llcc_slice i2c_qup i2c_qcom_geni qcom_geni_se gpucc_sdm845 glink_ssr qcom_common qcom_smd qcom_glink_smem qcom_glink_native smem gcc_sdm845 gcc_msm8998 fastrpc extcon_usb_gpio dispcc_sdm845 deferred_free_helper cqhci clk_scmi | |
| Line 12384: [ 76.762547][ T8] clk_rpmh | |
| Line 12386: [ 76.772956][ T8] qcom_rpmh cmd_db clk_qcom ax88179_178a ath10k_snoc qmi_helpers ath10k_pci ath10k_core ath asix arm_smmu qcom_scm_64 apr | |
| Line 12388: [ 76.783084][ T8] CPU: 7 PID: 8 Comm: kworker/u16:0 Tainted: G W 5.4.127-android12-6-00329-g8451145ec97b-ab7477403 #1 | |
| Line 12389: [ 76.783087][ T8] Hardware name: Thundercomm Dragonboard 845c (DT) | |
| Line 12391: [ 76.796704][ T8] Workqueue: scsi_tmf_0 0xffffffd62ec1b890 | |
| Line 12392: [ 76.796705][ T8] pstate: 40c00005 (nZcv daif +PAN +UAO) | |
| Line 12393: [ 76.796707][ T8] pc : blk_mq_get_tag+0x498/0x54c | |
| Line 12394: [ 76.796711][ T8] lr : blk_mq_get_request+0x2a0/0x4cc | |
| Line 12396: [ 76.808838][ T8] sp : ffffffc0100939e0 | |
| Line 12397: [ 76.808839][ T8] x29: ffffffc010093a20 x28: ffffff9bf1f74000 | |
| Line 12398: [ 76.808841][ T8] x27: 00000000ffffff9c x26: 0000000000000080 | |
| Line 12399: [ 76.808843][ T8] x25: ffffff9bf8932c40 x24: 0000000000000000 | |
| Line 12400: [ 76.808844][ T8] x23: 0000000000000000 x22: ffffff9bf1f970e0 | |
| Line 12401: [ 76.808845][ T8] x21: ffffff9bf1f97710 x20: 0000000000000000 | |
| Line 12402: [ 76.808846][ T8] x19: ffffffc010093b00 x18: ffffffc010095050 | |
| Line 12403: [ 76.808847][ T8] x17: ffffffd630e6cb48 x16: 00000000000000c0 | |
| Line 12404: [ 76.808848][ T8] x15: ffffffd630169a8c x14: ffffffd6307a6298 | |
| Line 12405: [ 76.808849][ T8] x13: 000000000000000a x12: 000000000000003d | |
| Line 12406: [ 76.808850][ T8] x11: 0000000000000000 x10: ffffff9bf1f9be00 | |
| Line 12407: [ 76.808852][ T8] x9 : ffffff9bf1fae000 x8 : 0000000000000002 | |
| Line 12408: [ 76.808853][ T8] x7 : 6965636572202939 x6 : ffffffd630e8ccc7 | |
| Line 12409: [ 76.808854][ T8] x5 : 0000000000000000 x4 : 000000000000000c | |
| Line 12410: [ 76.808855][ T8] x3 : ffffffd6288ff8d0 x2 : ffffffc010093b00 | |
| Line 12411: [ 76.808856][ T8] x1 : 0000000000000000 x0 : ffffffc010093b00 | |
| Line 12412: [ 76.808857][ T8] Call trace: | |
| Line 12413: [ 76.808859][ T8] blk_mq_get_tag+0x498/0x54c | |
| Line 12414: [ 76.808860][ T8] blk_mq_get_request+0x2a0/0x4cc | |
| Line 12415: [ 76.808863][ T8] blk_get_request+0x78/0x134 | |
| Line 12416: [ 76.808872][ T8] __ufshcd_issue_tm_cmd+0x60/0xcd4 | |
| Line 12418: [ 76.820998][ T8] ufshcd_try_to_abort_task+0xc4/0x370 | |
| Line 12419: [ 76.820999][ T8] ufshcd_abort+0x1a0/0x678 | |
| Line 12420: [ 76.821003][ T8] scmd_eh_abort_handler+0x84/0x15c | |
| Line 12421: [ 76.821008][ T8] process_one_work+0x2e0/0x77c | |
| Line 12422: [ 76.821010][ T8] worker_thread+0x3b0/0xa14 | |
| Line 12424: [ 76.833137][ T8] kthread+0x148/0x1b8 | |
| Line 12425: [ 76.833142][ T8] ret_from_fork+0x10/0x18 | |
| Line 12426: [ 76.833143][ T8] —[ end trace d4b259028b8d2c04 ]— | |
| Line 12427: [ 76.833174][ T8] Unable to handle kernel NULL pointer dereference at virtual address 000000000000011d | |
| Line 12429: [ 76.845534][ T8] Mem abort info: | |
| Line 12430: [ 77.388102][ T8] ESR = 0x96000045 | |
| Line 12431: [ 77.391943][ T8] EC = 0x25: DABT (current EL), IL = 32 bits | |
| Line 12432: [ 77.398017][ T8] SET = 0, FnV = 0 | |
| Line 12433: [ 77.401814][ T8] EA = 0, S1PTW = 0 | |
| Line 12434: [ 77.405693][ T8] Data abort info: | |
| Line 12435: [ 77.409315][ T8] ISV = 0, ISS = 0x00000045 | |
| Line 12436: [ 77.413892][ T8] CM = 0, WnR = 1 | |
| Line 12437: [ 77.417602][ T8] user pgtable: 4k pages, 39-bit VAs, pgdp=00000001679bd000 | |
| Line 12438: [ 77.424807][ T8] [000000000000011d] pgd=0000000000000000, pud=0000000000000000 | |
| Line 12439: [ 77.432367][ T8] Internal error: Oops: 96000045 [#1] PREEMPT SMP | |
| Line 12440: [ 77.438695][ T8] Modules linked in: ufs_qcom system_heap syscon_reboot_mode spmi_pmic_arb socinfo snd_soc_hdmi_codec smsm smp2p sdhci_msm rtc_pm8xxx rpmhpd rmtfs_mem reset_qcom_pdc reset_qcom_aoss qrtr_tun qrtr_smd qrtr qnoc_sdm845 qcom_tsens qcom_spmi_regulator qcom_q6v5_wcss qcom_q6v5_pas qcom_q6v5_mss qcom_q6v5_adsp qcom_sysmon qcom_q6v5 qcom_hwspinlock qcom_gsbi qcom_glink_rpm qcom_geni_serial qcom_aoss qcom_wdt qcom_spmi_pmic regmap_spmi qcom_rpmh_regulator qcom_pon reboot_mode qcom_pdc qcom_cpufreq_hw qcom_apcs_ipc_mailbox pm8941_pwrkey pm8916_wdt pinctrl_spmi_mpp pinctrl_spmi_gpio pinctrl_sdm845 pinctrl_msm8998 phy_qcom_usb_hs ulpi phy_qcom_qusb2 phy_qcom_qmp page_pool ohci_platform ohci_pci ohci_hcd nvmem_qfprom msm_serial msm mdt_loader lt9611 llcc_slice i2c_qup i2c_qcom_geni qcom_geni_se gpucc_sdm845 glink_ssr qcom_common qcom_smd qcom_glink_smem qcom_glink_native smem gcc_sdm845 gcc_msm8998 fastrpc extcon_usb_gpio dispcc_sdm845 deferred_free_helper cqhci clk_scmi | |
| Line 12441: [ 77.438765][ T8] clk_rpmh qcom_rpmh cmd_db clk_qcom ax88179_178a ath10k_snoc qmi_helpers ath10k_pci ath10k_core ath asix arm_smmu qcom_scm_64 apr | |
| Line 12442: [ 77.539303][ T8] CPU: 7 PID: 8 Comm: kworker/u16:0 Tainted: G W 5.4.127-android12-6-00329-g8451145ec97b-ab7477403 #1 | |
| Line 12443: [ 77.551479][ T8] Hardware name: Thundercomm Dragonboard 845c (DT) | |
| Line 12444: [ 77.557900][ T8] Workqueue: scsi_tmf_0 0xffffffd62ec1b890 | |
| Line 12445: [ 77.563619][ T8] pstate: 60c00005 (nZCv daif +PAN +UAO) | |
| Line 12446: [ 77.569164][ T8] pc : __ufshcd_issue_tm_cmd+0x64/0xcd4 | |
| Line 12447: [ 77.574614][ T8] lr : __ufshcd_issue_tm_cmd+0x60/0xcd4 | |
| Line 12448: [ 77.580070][ T8] sp : ffffffc010093b50 | |
| Line 12449: [ 77.584124][ T8] x29: ffffffc010093bb0 x28: ffffff9bf1f74000 | |
| Line 12450: [ 77.590189][ T8] x27: 00000000ffffff9c x26: 0000000000000080 | |
| Line 12451: [ 77.596246][ T8] x25: ffffffd62ec0ab64 x24: ffffff9bf1f71250 | |
| Line 12452: [ 77.602303][ T8] x23: ffffffc010093c18 x22: 0000000080000000 | |
| Line 12453: [ 77.608366][ T8] x21: ffffffc010093b88 x20: ffffff9bf1f74948 | |
| Line 12454: [ 77.614431][ T8] x19: fffffffffffffff5 x18: ffffffc010095038 | |
| Line 12455: [ 77.620498][ T8] x17: ffffffd630e6cb48 x16: 00000000000000c0 | |
| Line 12456: [ 77.626556][ T8] x15: ffffffd630169a8c x14: ffffffd6307a6298 | |
| Line 12457: [ 77.632621][ T8] x13: 000000000000000a x12: 000000000000003d | |
| Line 12458: [ 77.638682][ T8] x11: 0000000000000000 x10: ffffff9bf1f9be00 | |
| Line 12459: [ 77.644746][ T8] x9 : e2c52ed29de08a00 x8 : e2c52ed29de08a00 | |
| Line 12460: [ 77.650804][ T8] x7 : 6965636572202939 x6 : ffffffd630e8ccc7 | |
| Line 12461: [ 77.656872][ T8] x5 : 0000000000000000 x4 : 000000000000000c | |
| Line 12462: [ 77.662935][ T8] x3 : ffffffd6288ff8d0 x2 : ffffffc010093b00 | |
| Line 12463: [ 77.668993][ T8] x1 : 0000000000000000 x0 : fffffffffffffff5 | |
| Line 12464: [ 77.675053][ T8] Call trace: | |
| Line 12465: [ 77.678234][ T8] __ufshcd_issue_tm_cmd+0x64/0xcd4 | |
| Line 12466: [ 77.683340][ T8] ufshcd_try_to_abort_task+0xc4/0x370 | |
| Line 12467: [ 77.688699][ T8] ufshcd_abort+0x1a0/0x678 | |
| Line 12468: [ 77.693110][ T8] scmd_eh_abort_handler+0x84/0x15c | |
| Line 12469: [ 77.698215][ T8] process_one_work+0x2e0/0x77c | |
| Line 12470: [ 77.702975][ T8] worker_thread+0x3b0/0xa14 | |
| Line 12471: [ 77.707468][ T8] kthread+0x148/0x1b8 | |
| Line 12472: [ 77.711437][ T8] ret_from_fork+0x10/0x18 | |
| Line 12473: [ 77.715757][ T8] Code: a93dffbf a93ea7a9 97f24820 aa0003f3 (f9009415) | |
| Line 12474: [ 77.722603][ T8] —[ end trace d4b259028b8d2c05 ]— | |
| Line 12475: [ 77.727974][ T8] Kernel panic - not syncing: Fatal exception | |
| Line 12476: [ 77.733955][ T8] SMP: stopping secondary CPUs | |
| Line 12477: [ 77.739005][ T8] Kernel Offset: 0x161eb80000 from 0xffffffc010000000 | |
| Line 12478: [ 77.745678][ T8] PHYS_OFFSET: 0xffffffe580000000 | |
| Line 12479: [ 77.750613][ T8] CPU features: 0x00010002,2a80a238 | |
| Line 12480: [ 77.755718][ T8] Memory Limit: none |
can anyone let know what would be the issue is it due to HW flash storage issue or software error since we see this consistently irrespective of kernel v 5.10 or 5.4. we flash Android OS on to these devices.
