OP-TEE CA as an android application (apk) build problem

Hi. I’m trying to build optee_example CA as an .apk.

I want to make test CA as an apk file so that user can just touch to execute the application.

I think that if I copy hello_world example code to my increment.cpp file with jni rule and import pre-built libteec.so file with CMake option, it would work.
(If I made a CA using android ndk and run, it would interact with pre-built optee_example TA in my optee_os through client APIs. Please correct me if I’m wrong.)

But, though I build and install the application as a system app, it just repeated execution and dead forever when it is run.


Environment
  • AndroidStudio 3.6.2, kotlin 1.3.71 and CMake 3.10.2(not using Android.mk file) on VMware Ubuntu 18.04 LTS
  • HiKey960 4GB RAM, AOSP+OP-TEE 3.4.2 branch, touchscreen enabled

First, I tried android ndktest project. I could make and run successfully.

Next, tried to build CA.

I referred to this issue

and could follow at this point with using CMake.


I could build apk, install and generate libincrement.so and libteec.so(pre-built arm64 cross-compiled) in /data/app/mypackage/lib/arm64/

but I couldn’t open my package with libc++.so.

This is logcat logs of my application.

2020-06-07 08:27:06.156 21255-21255/? W/main: type=1400 audit(0.0:9124): avc: denied { dac_read_search } for capability=2 scontext=u:r:zygote:s0 tcontext=u:r:zygote:s0 tclass=capability permissive=0
2020-06-07 08:27:06.405 21255-21255/com.example.retest D/AndroidRuntime: Shutting down VM
2020-06-07 08:27:06.407 21255-21255/? E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.retest, PID: 21255
java.lang.UnsatisfiedLinkError: dlopen failed: library “libc++.so” not found
at java.lang.Runtime.loadLibrary0(Runtime.java:1016)
at java.lang.System.loadLibrary(System.java:1669)
at com.example.retest.MainActivity.(MainActivity.kt:30)
at java.lang.Class.newInstance(Native Method)
at android.app.AppComponentFactory.instantiateActivity(AppComponentFactory.java:69)
at androidx.core.app.CoreComponentFactory.instantiateActivity(CoreComponentFactory.java:45)
at android.app.Instrumentation.newActivity(Instrumentation.java:1215)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2869)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3086)
at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1816)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:193)
at android.app.ActivityThread.main(ActivityThread.java:6718)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
2020-06-07 08:27:06.412 21255-21255/? I/Process: Sending signal. PID: 21255 SIG: 9


Though I installed as a system app, the application just repeated execution and dead forever. And I’m stuck in here.

This is my project tree under …/src/main/cpp/

image

* MainActivity capture

* increment.cpp capture



(totally same part with main.c)


This is my CMakeLists.txt file.

cmake_minimum_required(VERSION 3.4.1)

add_library(
teec
SHARED
IMPORTED
)

set_target_properties(
teec
PROPERTIES IMPORTED_LOCATION
${CMAKE_SOURCE_DIR}/jniLibs/arm64-v8a/libteec.so
)

include_directories(${CMAKE_SOURCE_DIR})

add_library(
increment
SHARED
increment.cpp
}

find_library(
log-lib
log
)

target_link_libraries(
increment
teec
${log-lib}
)

Repeated logcat logs

2020-06-07 09:33:36.540 23180-23180/? W/main: type=1400 audit(0.0:9436): avc: denied { dac_read_search } for capability=2 scontext=u:r:zygote:s0 tcontext=u:r:zygote:s0 tclass=capability permissive=0
2020-06-07 09:33:36.560 23180-23180/? I/.example.retes: Late-enabling -Xcheck:jni
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/graphics/drawable/Drawable;->getOpticalInsets()Landroid/graphics/Insets; (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->left:I (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->right:I (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->top:I (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->bottom:I (light greylist, linking)
2020-06-07 09:33:36.946 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/view/View;->getAccessibilityDelegate()Landroid/view/View$AccessibilityDelegate; (light greylist, linking)
2020-06-07 09:33:37.003 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (light greylist, reflection)
2020-06-07 09:33:37.004 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (light greylist, reflection)
2020-06-07 09:33:37.012 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/widget/TextView;->getTextDirectionHeuristic()Landroid/text/TextDirectionHeuristic; (light greylist, linking)
2020-06-07 09:33:37.041 23180-23180/com.example.retest D/test: testestestest


Please help me if you have any idea.

I’ve not really tried this before, but for the error above, can you perhaps try to refer to https://github.com/centricular/android_rtsp_example/issues/3? Something about adding a dummy.cpp file will somehow cause ndk-build to include the c++ library required. Also maybe make sure you’re building to the proper AOSP version in Android Studio.

Thanks for comment.

I’ve tried according to libc++ and ANDROID_STL link.
The guide says

By default, c++_static will be used.


I tried to make llvm libc++.so as a shared library with gradle option.

image

But it showed same error.

Maybe libc++_shared.so have already been built in the app statically.

Considering of the issue above, I think "libc++.so not found" issue is droven by the problem that user application don’t have system library access permission.


I’ve tried to either run as a system app and put libc++.so manually under /data/app/mypackage/lib/arm64/ also, but it runs and dies repeatedly.

It might be in some other problem scope.


my android SDK settings have API level 28(Pie), 29(Q)
and 3.4.2 branch AOSP is API level 28.

image

Sorry. Don’t know enough to really help you here. Do you maybe have libc++.so in /vendor/lib64/ and then try running it not as a system app? Have you also tried setting selinux to permissive first?

I’ve put libc++.so under /vendor/lib64/ but "libc++.so not found" issue occurred again in user level installation.

/vendor directory was also a read-only file system like /system directory. So I should have remount as r/w permission.

image

Maybe I had satisfied permission problem.

image

I guess there might be CMake option mistake or libteec.so file interaction mistake.
I’ll try to enable attaching debugger(it didn’t work) or import libteec.so sources in my Android Studio project to build with.

By the way, I wonder if my assume is correct or not. Do you have any idea?

By the way, I wonder if my assume is correct or not. Do you have any idea?

Yes, theoretically it should work [1] but afaik users who’ve tried had faced issues similar to yours. You can maybe read through the related threads in the faq to see if any of them might be of useful references.

[1] https://optee.readthedocs.io/en/latest/faq/faq.html#q-how-can-i-use-or-call-op-tee-from-native-android-apk-applications

I’m trying to refer the project file of this thread. I remember I’ve found the file before, but I don’t know why I didn’t see until now…
Anyway it seems useful. I’ll post again if I can troubleshoot. Thanks!

Ok, but please note that the project file was created before Treble so things will not be exactly the same. Hope to hear good news from you.