Dragonboard 410c Reboot problem of Android

Android system is 6.0.1. After operation of ~10hours, the system restarts. From the logcat, global reference table of system server is full.
I reduce the global reference table size to 2560 in order to make it happen faster (~10mins) without any active apps running. Below is the logcat.


01-01 01:22:31.645403 756 768 F art : art/runtime/indirect_reference_table.cc:115] JNI ERROR (app bug): global reference table overflow (max=2560)
01-01 01:22:31.645478 756 768 F art : art/runtime/indirect_reference_table.cc:115] global reference table dump:
01-01 01:22:31.645504 756 768 F art : art/runtime/indirect_reference_table.cc:115] Last 10 entries (of 2559):
01-01 01:22:31.645522 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2558: 0x13726920 java.lang.ref.WeakReference
01-01 01:22:31.645539 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2557: 0x13468570 com.android.internal.util.AsyncChannel$DeathMonitor
01-01 01:22:31.645555 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2556: 0x13742460 java.lang.ref.WeakReference
01-01 01:22:31.645574 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2555: 0x136dc8c0 java.lang.ref.WeakReference
01-01 01:22:31.645591 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2554: 0x1372c280 java.lang.ref.WeakReference
01-01 01:22:31.645610 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2553: 0x136b4ba0 java.lang.ref.WeakReference
01-01 01:22:31.645630 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2552: 0x1371a300 java.lang.ref.WeakReference
01-01 01:22:31.645646 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2551: 0x136c0c40 java.lang.ref.WeakReference
01-01 01:22:31.645663 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2550: 0x137287e0 java.lang.ref.WeakReference
01-01 01:22:31.645678 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2549: 0x136cef20 java.lang.ref.WeakReference
01-01 01:22:31.645695 756 768 F art : art/runtime/indirect_reference_table.cc:115] Summary:
01-01 01:22:31.645710 756 768 F art : art/runtime/indirect_reference_table.cc:115] 1 of com.android.server.SerialService
01-01 01:22:31.645724 756 768 F art : art/runtime/indirect_reference_table.cc:115] 1 of com.android.server.VibratorService
01-01 01:22:31.645742 756 768 F art : art/runtime/indirect_reference_table.cc:115] 1 of com.android.server.location.FlpHardwareProvider

01-01 01:22:31.647663 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2 of java.lang.ThreadGroup (2 unique instances)
01-01 01:22:31.647677 756 768 F art : art/runtime/indirect_reference_table.cc:115] 1633 of java.lang.ref.WeakReference (1633 unique instances)
01-01 01:22:31.647697 756 768 F art : art/runtime/indirect_reference_table.cc:115] 2 of android.opengl.EGLDisplay (1 unique instances)

01-01 01:22:35.119216 756 768 F art : art/runtime/barrier.cc:90] Check failed: count_ == 0 (count_=-1, 0=0) Attempted to destroy barrier with non zero count
01-01 01:22:35.119320 756 768 F art : art/runtime/runtime.cc:366] Runtime aborting — recursively, so no thread-specific detail!
01-01 01:22:35.119352 756 768 F art : art/runtime/runtime.cc:366]
--------- switch to crash
01-01 01:22:35.119735 756 768 F libc : Fatal signal 6 (SIGABRT), code -6 in tid 768 (Binder_2)
--------- switch to crash
01-01 01:22:35.361678 301 301 F DEBUG : Abort message: ‘art/runtime/indirect_reference_table.cc:115] JNI ERROR (app bug): global reference table overflow (max=2560)’
01-01 01:22:35.361808 301 301 F DEBUG : x0 0000000000000000 x1 0000000000000300 x2 0000000000000006 x3 0000000000000000
01-01 01:22:35.361887 301 301 F DEBUG : x4 0000000000000000 x5 0000000000000001 x6 0000000000000000 x7 0000000000000000
01-01 01:22:35.361953 301 301 F DEBUG : x8 0000000000000083 x9 0000007fa17aca70 x10 0000000000000003 x11 0000000000000170
01-01 01:22:35.362012 301 301 F DEBUG : x12 000000000000000e x13 0000000000004000 x14 0000000000000000 x15 0000000000000000
01-01 01:22:35.362073 301 301 F DEBUG : x16 0000007fa17a56a8 x17 0000007fa1767b6c x18 8000000000000000 x19 0000007f8bceb510
01-01 01:22:35.362130 301 301 F DEBUG : x20 0000007f8bceb450 x21 0000000000000000 x22 0000000000000006 x23 0000007f9e43b000
01-01 01:22:35.362183 301 301 F DEBUG : x24 000000558e61a000 x25 000000558e123c20 x26 000000558dff5960 x27 0000007f9e36e000
01-01 01:22:35.362233 301 301 F DEBUG : x28 000000558e5e8630 x29 0000007f8bce9ad0 x30 0000007fa1765308
01-01 01:22:35.362286 301 301 F DEBUG : sp 0000007f8bce9ad0 pc 0000007fa1767b74 pstate 0000000020000000
01-01 01:22:35.371922 301 301 F DEBUG :
01-01 01:22:35.371922 301 301 F DEBUG : backtrace:
01-01 01:22:35.372078 301 301 F DEBUG : #00 pc 000000000006ab74 /system/lib64/libc.so (tgkill+8)
01-01 01:22:35.372184 301 301 F DEBUG : #01 pc 0000000000068304 /system/lib64/libc.so (pthread_kill+68)
01-01 01:22:35.372254 301 301 F DEBUG : #02 pc 00000000000212f8 /system/lib64/libc.so (raise+28)
01-01 01:22:35.372319 301 301 F DEBUG : #03 pc 000000000001ba98 /system/lib64/libc.so (abort+60)
01-01 01:22:35.372389 301 301 F DEBUG : #04 pc 0000000000436508 /system/lib64/libart.so (art::Runtime::Abort()+324)
01-01 01:22:35.372448 301 301 F DEBUG : #05 pc 0000000000139264 /system/lib64/libart.so (art::LogMessage::~LogMessage()+3136)
01-01 01:22:35.372514 301 301 F DEBUG : #06 pc 0000000000133d08 /system/lib64/libart.so (art::Barrier::~Barrier()+296)
01-01 01:22:35.372571 301 301 F DEBUG : #07 pc 000000000046b0f4 /system/lib64/libart.so (art::ThreadList::Dump(std::__1::basic_ostream<char, std::__1::char_traits >&)+212)
01-01 01:22:35.372628 301 301 F DEBUG : #08 pc 00000000004365f0 /system/lib64/libart.so (art::Runtime::Abort()+556)
01-01 01:22:35.372691 301 301 F DEBUG : #09 pc 0000000000139264 /system/lib64/libart.so (art::LogMessage::~LogMessage()+3136)
01-01 01:22:35.372750 301 301 F DEBUG : #10 pc 0000000000276508 /system/lib64/libart.so (art::IndirectReferenceTable::Add(unsigned int, art::mirror::Object*)+372)
01-01 01:22:35.372804 301 301 F DEBUG : #11 pc 000000000030d4ac /system/lib64/libart.so (art::JavaVMExt::AddGlobalRef(art::Thread*, art::mirror::Object*)+56)
01-01 01:22:35.372861 301 301 F DEBUG : #12 pc 00000000003432cc /system/lib64/libart.so (art::JNI::NewGlobalRef(_JNIEnv*, _jobject*)+320)
01-01 01:22:35.372917 301 301 F DEBUG : #13 pc 0000000000462050 /system/lib64/libart.so (art::thread::CreateNativeThread(_JNIEnv*, _jobject*, unsigned long, bool)+180)
01-01 01:22:35.372982 301 301 F DEBUG : #14 pc 000000007253d2b4 /data/dalvik-cache/arm64/system@framework@boot.oat (offset 0x1f08000)


It seems java.lang.ref.WeakReference (1633) are the main cause but I have no idea where they are from. Could anyone advise how I can debug it?

Thanks in advance!

Hi,

I run SD card Android Image build #118 with a downloaded app “Wifi analyzer” running over the last weekend and the reboot problem also happens. It seems the root cause is inside the Android system. Do anyone have hints to debug it?

Thanks in advance!

I’m not going to claim much expertise in diagnosing Android problems but I’d be interested to know if the new #128 snapshot build (http://builds.96boards.org/snapshots/dragonboard410c/qualcomm/android/128/ ) exhibits the same problem.

Hi danielt,

I reduce the global reference table size for #128 and Android can run for 1 hours without reboot, comparing with ~10mins with #118. The problem seems gone. Anyway, I will test the original #128 overnight and tell the result tomorrow.

I used DB410C_build.sh to download AOSP last time. Could you please advise the command details how I download the branch/tag of #128?

Thanks a lot!

Hi danielt,

The dragonboard has run for over 18 hours without reboot problem. It seems the problem has gone with #128.
I download the following BSP from codeaurora.org but the reboot problem still persists.
March 02, 2017 LA.BR.1.2.7-05610-8x16.0 msm8916_64 LA.BR.1.2.7-05610-8x16.0.xml 06.00.01

Could you please tell which one is #128?

Thanks in advance!

I believe it is built from vLA.BR.1.2.7-01010-8x16.0-4 (available here: https://developer.qualcomm.com/hardware/dragonboard-410c/tools ).

Hi danielt,

I find that the source-manifest.xml in build #128 is exactly the same as the one I am using which is downloaded from la.br.1.2.7-01010-8x16.0. Could you please tell what the difference is between build #128 and la.br.1.2.7-01010-8x16.0?

Thanks a lot!

The exact versions that contribute to the build should be in the pinned-manifest.xml.

Hi danielt,

The pinned-manifest.xml shows that revision number is different in 10 folders. e.g. in pinned-manifest.xml,

in my manifest.xml,

When I run “git log --oneline” under device/qcom/common, the output is as follow,
3edfb88 USB camera patch-permissions changes
3eb4cdf device:qcom:common USB to Ethernet patch
1d4c2f7 Merge “IMS-VT: Handle buttons display in InCall Screen”

I do not see the revision “6ef0f58274a9cdf1c4af9bd395e5e39957353ec4”.

I also run “git fetch” but see no difference. Am I missing something or doing wrong in git? Could you please advise how I find the target revision and pull it to my local repository?

Many thanks!