OP-TEE CA as an android application (apk) build problem

[dvyurij]

Hi. I’m trying to build optee_example CA as an .apk.

I want to make test CA as an apk file so that user can just touch to execute the application.

I think that if I copy hello_world example code to my increment.cpp file with jni rule and import pre-built libteec.so file with CMake option, it would work.
(If I made a CA using android ndk and run, it would interact with pre-built optee_example TA in my optee_os through client APIs. Please correct me if I’m wrong.)

But, though I build and install the application as a system app, it just repeated execution and dead forever when it is run.

Environment

• AndroidStudio 3.6.2, kotlin 1.3.71 and CMake 3.10.2(not using Android.mk file) on VMware Ubuntu 18.04 LTS
• HiKey960 4GB RAM, AOSP+OP-TEE 3.4.2 branch, touchscreen enabled

First, I tried android ndktest project. I could make and run successfully.

Next, tried to build CA.

I referred to this issue

Developing Native Android Application with Optee

Hello everyone, I want to develop a native Android application by using optee. I developed a Ta and host application by using optee. Right now, I want to connect Ndk application to optee host application. And, I have some questions about that. How can I call a method of optee host application from Android Ndk app? If I call optee host app in /system/bin folder from Android app, android app can see this executable? (I tried, but it is gone wrong.) Can I put optee host application to another fo…

and could follow at this point with using CMake.

Developing Native Android Application with Optee

I think, I generated build jni files with libteec. Android.mk : LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libteec LOCAL_SRC_FILES := /home/test/optee_android/out/target/product/hikey/system/lib64/libteec.so include $(PREBUILT_SHARED_LIBRARY) include $(CLEAR_VARS) CFG_TEEC_PUBLIC_INCLUDE = /home/test/devel/optee/optee_client/public LOCAL_STATIC_LIBRARIES := libteec LOCAL_MODULE := test LOCAL_SRC_FILES := test.c LOCAL_C_INCLUDES := $(CFG_TEEC_PUBLIC_INCLUDE) \ /home/…

I could build apk, install and generate libincrement.so and libteec.so(pre-built arm64 cross-compiled) in /data/app/mypackage/lib/arm64/

but I couldn’t open my package with libc++.so.

This is logcat logs of my application.

2020-06-07 08:27:06.156 21255-21255/? W/main: type=1400 audit(0.0:9124): avc: denied { dac_read_search } for capability=2 scontext=u:r:zygote:s0 tcontext=u:r:zygote:s0 tclass=capability permissive=0
2020-06-07 08:27:06.405 21255-21255/com.example.retest D/AndroidRuntime: Shutting down VM
2020-06-07 08:27:06.407 21255-21255/? E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.retest, PID: 21255
java.lang.UnsatisfiedLinkError: dlopen failed: library “libc++.so” not found
at java.lang.Runtime.loadLibrary0(Runtime.java:1016)
at java.lang.System.loadLibrary(System.java:1669)
at com.example.retest.MainActivity.(MainActivity.kt:30)
at java.lang.Class.newInstance(Native Method)
at android.app.AppComponentFactory.instantiateActivity(AppComponentFactory.java:69)
at androidx.core.app.CoreComponentFactory.instantiateActivity(CoreComponentFactory.java:45)
at android.app.Instrumentation.newActivity(Instrumentation.java:1215)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2869)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3086)
at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:78)
at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:108)
at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:68)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1816)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:193)
at android.app.ActivityThread.main(ActivityThread.java:6718)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
2020-06-07 08:27:06.412 21255-21255/? I/Process: Sending signal. PID: 21255 SIG: 9

Though I installed as a system app, the application just repeated execution and dead forever. And I’m stuck in here.

This is my project tree under …/src/main/cpp/

* MainActivity capture

* increment.cpp capture

…
(totally same part with main.c)
…

This is my CMakeLists.txt file.

cmake_minimum_required(VERSION 3.4.1)

add_library(
teec
SHARED
IMPORTED
)

set_target_properties(
teec
PROPERTIES IMPORTED_LOCATION
${CMAKE_SOURCE_DIR}/jniLibs/arm64-v8a/libteec.so
)

include_directories(${CMAKE_SOURCE_DIR})

add_library(
increment
SHARED
increment.cpp
}

find_library(
log-lib
log
)

target_link_libraries(
increment
teec
${log-lib}
)

Repeated logcat logs

2020-06-07 09:33:36.540 23180-23180/? W/main: type=1400 audit(0.0:9436): avc: denied { dac_read_search } for capability=2 scontext=u:r:zygote:s0 tcontext=u:r:zygote:s0 tclass=capability permissive=0
2020-06-07 09:33:36.560 23180-23180/? I/.example.retes: Late-enabling -Xcheck:jni
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/graphics/drawable/Drawable;->getOpticalInsets()Landroid/graphics/Insets; (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->left:I (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->right:I (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->top:I (light greylist, linking)
2020-06-07 09:33:36.908 23180-23180/com.example.retest W/.example.retes: Accessing hidden field Landroid/graphics/Insets;->bottom:I (light greylist, linking)
2020-06-07 09:33:36.946 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/view/View;->getAccessibilityDelegate()Landroid/view/View$AccessibilityDelegate; (light greylist, linking)
2020-06-07 09:33:37.003 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (light greylist, reflection)
2020-06-07 09:33:37.004 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (light greylist, reflection)
2020-06-07 09:33:37.012 23180-23180/com.example.retest W/.example.retes: Accessing hidden method Landroid/widget/TextView;->getTextDirectionHeuristic()Landroid/text/TextDirectionHeuristic; (light greylist, linking)
2020-06-07 09:33:37.041 23180-23180/com.example.retest D/test: testestestest

Please help me if you have any idea.

[vchong]

@dvyurij

java.lang.UnsatisfiedLinkError: dlopen failed: library “libc++.so” not found

I’ve not really tried this before, but for the error above, can you perhaps try to refer to https://github.com/centricular/android_rtsp_example/issues/3? Something about adding a dummy.cpp file will somehow cause ndk-build to include the c++ library required. Also maybe make sure you’re building to the proper AOSP version in Android Studio.

[dvyurij]

Thanks for comment.

@vchong

can you perhaps try to refer to https://github.com/centricular/android_rtsp_example/issues/3 ? Something about adding a dummy.cpp file will somehow cause ndk-build to include the c++ library required.

I’ve tried according to libc++ and ANDROID_STL link.
The guide says

By default, c++_static will be used.

I tried to make llvm libc++.so as a shared library with gradle option.

But it showed same error.

Maybe libc++_shared.so have already been built in the app statically.

Developing Native Android Application with Optee

I think, I generated build jni files with libteec. Android.mk : LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libteec LOCAL_SRC_FILES := /home/test/optee_android/out/target/product/hikey/system/lib64/libteec.so include $(PREBUILT_SHARED_LIBRARY) include $(CLEAR_VARS) CFG_TEEC_PUBLIC_INCLUDE = /home/test/devel/optee/optee_client/public LOCAL_STATIC_LIBRARIES := libteec LOCAL_MODULE := test LOCAL_SRC_FILES := test.c LOCAL_C_INCLUDES := $(CFG_TEEC_PUBLIC_INCLUDE) \ /home/…

@tesmnorth

I have libc++.so in /system/lib64/

Considering of the issue above, I think "libc++.so not found" issue is droven by the problem that user application don’t have system library access permission.

@dvyurij

Though I installed as a system app, the application just repeated execution and dead forever.

I’ve tried to either run as a system app and put libc++.so manually under /data/app/mypackage/lib/arm64/ also, but it runs and dies repeatedly.

It might be in some other problem scope.

@vchong

Also maybe make sure you’re building to the proper AOSP version in Android Studio.

my android SDK settings have API level 28(Pie), 29(Q)
and 3.4.2 branch AOSP is API level 28.

[vchong]

@dvyurij

I have libc++.so in /system/lib64/

Sorry. Don’t know enough to really help you here. Do you maybe have libc++.so in /vendor/lib64/ and then try running it not as a system app? Have you also tried setting selinux to permissive first?

[dvyurij]

@vchong

Do you maybe have libc++.so in /vendor/lib64/ and then try running it not as a system app?

I’ve put libc++.so under /vendor/lib64/ but "libc++.so not found" issue occurred again in user level installation.

/vendor directory was also a read-only file system like /system directory. So I should have remount as r/w permission.

@vchong

Have you also tried setting selinux to permissive first?

Maybe I had satisfied permission problem.

I guess there might be CMake option mistake or libteec.so file interaction mistake.
I’ll try to enable attaching debugger(it didn’t work) or import libteec.so sources in my Android Studio project to build with.

By the way, I wonder if my assume is correct or not. Do you have any idea?

@dvyurij

I think that if I copy hello_world example code to my increment.cpp file with jni rule and import pre-built libteec.so file with CMake option, it would work.
(If I made a CA using android ndk and run, it would interact with pre-built optee_example TA in my optee_os through client APIs. Please correct me if I’m wrong. )

[vchong]

By the way, I wonder if my assume is correct or not. Do you have any idea?

Yes, theoretically it should work [1] but afaik users who’ve tried had faced issues similar to yours. You can maybe read through the related threads in the faq to see if any of them might be of useful references.

[1] Frequently Asked Questions — OP-TEE documentation documentation

[dvyurij]

I’m trying to refer the project file of this thread. I remember I’ve found the file before, but I don’t know why I didn’t see until now…
Anyway it seems useful. I’ll post again if I can troubleshoot. Thanks!

[vchong]

Ok, but please note that the project file was created before Treble so things will not be exactly the same. Hope to hear good news from you.