Android + OP-TEE compile error

Hi @vchong ,
A quick update.

I am getting build error at step 3.8
My command : make -j12 TARGET_BUILD_KERNEL=true


including ./tools/test/connectivity/Android.mk …
Starting build with ninja
ninja: Entering directory `.’
ninja: error: ‘out/target/product/hikey/kernel’, needed by ‘out/target/product/hikey/boot.img’, missing and no known rule to make it
make: *** [ninja_wrapper] Error 1

make failed to build some targets (01:15 (mm:ss))


Am thinking to copy build_kernel.sh and get_toolchain.sh from another OPTEE source code, to the present one and execute them. By this , kernel should be compiled, but am skeptical about this approach :slight_frown:

Pls let me know if its correct or any other way to solve the error

EDIT : tried this approach , and it didnt work

Lemme take a look and get back to you.

Sorry, there was a missing line in one of the manifests. Please try again. Should be ok now.

Hi ,
Now , I see kernel being compiled.
But now , I am stuck with jack server issues ( java related ).
I am trying many different solutions provided online, but nothing’s worked yet.

Will reply once I get some break through

Jack server should be automatically started and you shouldn’t have to mess with it.
Are you using a shared server where there are multiple other users/accounts trying to build aosp at the same time? If so, try changing the port number to avoid conflicts.

In ~/.jack-server/config.properties, change
jack.server.service.port=<some port other than 8076>
jack.server.admin.port=<some port other than 8077>

In ~/.jack-settings, change
SERVER_PORT_SERVICE=<some port other than 8076>
SERVER_PORT_ADMIN=<some port other than 8077>

[some text edited for clarity]

Finally , Compilation success.
(Had to use PC with 16GB RAM. Previously, I was trying on PC with 8GB RAM and I am facing jack server issues)

Now , except boot_fat.uefi.img, remaining all files were generated
(l-loader.bin , ptable-linux-8g.img, fip.bin, nvme.img , system.img, cache.img , userdata.img)

I flashed above files and board booted successfully.
There is another file , boot.img , I left that alone .

Now , tried xtest and output as follows :

hikey:/system/bin # xtest
xtest
Run test suite with level=0

TEE test application started with device [(null)]
######################################################

XTEST_TEE_TEST

######################################################

  • XTEST_TEE_1001 Core self tests
    ERR [12408] TEEC:TEEC_OpenSession:502: TEE_IOC_OPEN_SESSION failed
    XTEST_TEE_1001 OK

  • XTEST_TEE_1004 Test User Crypt TA
    ERR [12408] TEEC:TEEC_OpenSession:502: TEE_IOC_OPEN_SESSION failed
    external/optee_test/host/xtest/xtest_1000.c:286: xtest_teec_open_session( &session, &crypt_user_ta_uuid, ((void*)0), &ret_orig) has an unexpected value: 0xffff0000 = TEEC_ERROR_GEN
    ERIC, expected 0x0 = TEEC_SUCCESS
    XTEST_TEE_1004 FAILED

  • XTEST_TEE_1005 Many sessions
    ERR [12408] TEEC:TEEC_OpenSession:502: TEE_IOC_OPEN_SESSION failed
    external/optee_test/host/xtest/xtest_1000.c:341: xtest_teec_open_session(&sessions[i], &concurrent_ta_uuid, ((void*)0), &ret_orig) has an unexpected value: 0xffff0000 = TEEC_ERROR_
    GENERIC, expected 0x0 = TEEC_SUCCESS
    XTEST_TEE_1005 FAILED


lot many lines like the above !!! and at the end , summary is as below -

350 subtests of which 177 failed
70 test cases of which 65 failed
0 test case was skipped
TEE test application done!

key:/system # tee-supplicant &
tee-supplicant &
[1] 9186
hikey:/system # ERR [9186] TEES:main:397: failed to find an OP-TEE supplicant device

So , after all these errors , I flashed boot.img , now , I am not getting adb shell . I guess I can sort this out.

But , how can I solve tee-supplicant error above .
Am I missing anything ?

Thanks for the tip on RAM size!

You can build boot_fat.uefi.img by adding TARGET_BOOTIMAGE_USE_FAT=true when you run make. Both boot_fat.uefi.img and boot.img work. Only difference is the fat (not literally) one gives you a grub menu. If you don’t need grub the boot.img is just as well.

tee-supplicant is loaded automatically as a service so you don’t have to do it manually. If you try to load is a 2nd time you’ll see the error you’re seeing. Only one instance is supported.

Please repo sync to the latest and try again. Would be interested in your xtest results. There should only be errors in the 20k series tests.

Hi ,
with TARGET_BOOTIMAGE_USE_FAT=true parameter as above , boot_fat_uefi.img is generated.
But , when I flash that , hikey board is not getting detected on my PC ( which is a Native Linux PC )

So, I am using boot_fat_uefi.img from https://builds.96boards.org/releases/hikey/linaro/aosp/15.11/

As you said, xtest runs directly, no need to invoke tee-supplicant.

And its result is pretty huge , so , pasted full log in pastebin

http://pastebin.com/74mK5P1s

→ Are the results good. How do I infer OPTEE is working correctly.

→ Now , I did repo sync and compiling. Will test xtest again and post my results

→ If OPTEE is running fine, I will check hello world program next, and subsequently build TA for my device
Hope am on right track ?

→ my boot.img is not working ( board is not getting detected ). What can be the problem, any way to debug it ?
Also , since my main intention is build my own TA. So , should I have my own kernel on the board for my TA to work ?
If not required, I need not worry about this and can use pre-built boot image which I have downloaded from internet !!!

EDIT1:
I updated board after syncing to latest code today morning and ran xtest.
And the results are same

350 subtests of which 177 failed
70 test cases of which 65 failed
0 test case was skipped
TEE test application done!

EDIT2:
Also , am trying to run hello_world from https://github.com/linaro-swg/hello_world
Is secure world 32 bit or 64 bit ?
How to verify that ?

Another update.
I was trying to run hello_world program now.
So,

  1. I prepared stack from https://github.com/OP-TEE/build ( steps 7.3 to 7.5 )
  2. Then , prepared hello_world stack from https://github.com/linaro-swg/hello_world from step 1 to 4, 4th step being “make all”

Now , I got hello_world binary and its TA .
I verified with file command. hello_world binary is a 64bit one as well as xtest binary too

I copied them to /system/bin and /system/lib/optee_armtz respectively.

Now , when I run hello_world, I am getting error like below:
hikey:/system/lib/optee_armtz # hello_world
/system/bin/sh: /system/bin/hello_world: No such file or directory
1|hikey:/system/lib/optee_armtz #

I suspect

  1. My boot.img is not working, so I used a prebuilt one
  2. Some dependencies are missing
    Any pointer in this regard ?

Hi ,
From other threads, I made 2 changes as below :
1.
In device/linaro/hikey/optee-packages.mk, add:
PRODUCT_PACKAGES += tee_helloworld
PRODUCT_PACKAGES += 8aaaf200-2450-11e4-abe2-0002a5d5c51b.ta

In device/linaro/hikey/init.common.usb.rc ,
did setprop sys.usb.configfs 1

Now , I can have adb shell using my boot image file

Also , ran tee_helloworld and xtest with below results

hikey:/system/bin # tee_helloworld
Invoking TA to increment 42
TA incremented value to 43

hikey:/system/bin # xtest
22504 subtests of which 170 failed
70 test cases of which 30 failed
0 test case was skipped
TEE test application done!

@vchong , what do you think about results now ?? :slight_smile:

Another issue am facing now is , my monitor doesnt show any UI ? Its blank.
Its samsung monitor and connected to hikey board via HDMI to DVI cable ?
Do you think , any fault with my setup ?

EDIT : yes, indeed issue with my setup :frowning: .
I replaced with LG monitor having HDMI jack , and UI showed up :thumbsup:

Hi, sorry I just noticed your posts.

You don’t need your own kernel to build your own TA. Use the hello_world TA as a reference or basis for your own. Later, for more advanced TAs, you can refer to the ones in optee_test.

Few questions.

  1. Are you still having problems with your boot.img?
    The boot.img is built as part of the aosp build so you should be able to just use the one in out/target/product/hikey/. Please don’t use binaries built or downloaded from other non-aosp systems as they don’t use the android toolchain and most probably will not work.

  2. Did you run xtest as root or non-root?

  3. Can you list the test numbers that failed?

In any case, since you got hello_world to work, you can start working on your own TA at this point.

For the monitor, the Android build is more fickle than the Debian one. See http://www.96boards.org/documentation/ConsumerEdition/HiKey/HardwareDocs/HardwareNotes.md/ for notes on supported resolutions.

Finally, please remember this very important point. If you run repo sync again at any time in the future to update all the repos, by default all the patches above would be discarded, so you’ll have reapply them again before rebuilding!

Thanks!

Hi @vchong

  1. No problems now. My boot.img is fine
  2. Ran xtest as root
    adb root
    adb remount
    adb shell
    xtest
    This is the sequence of cmds
  3. xtest result is here for your reference : http://pastebin.com/V6JAGuSA

For monitor, it was my mistake. I used a DVI monitor and connected hikey with HDMI-DVI cable and there was no display. Now , I am using another monitor having HDMI input and display is fine now.
I think , the key point here is to use direct HDMI monitor

Yep , now I can work on my TA application. Thanks for your quick support

Now , a quick question :
Dont I need optee_os/kernel src code, to build my TA ?
My question arises in the context that , suppose, I give my board to my colleague so that he can verify his TA.
Does he need to have complete stack ( AOSP+OPTEE) like I have ?

Isnt there any on-board security check between TA and optee_os/kernel before TA loads and executes ?
If so , anybody can load their own TA on the board and run it. Isnt this a security loop-hole ?

Is there any command only to “clean and recompile” kernel ?

I just want to change below files
hi6220-hikey.dtb
kernel
boot.img
boot_fat.uefi.img

I am trying this command, but want to make sure
→ make -j12 TARGET_BUILD_KERNEL=true bootimage

Please provide the output of ps -ef | grep tee.

To compile a TA, you need the TA dev kit, which is built as part of the optee_os build, and also libteec.so from the optee_client build. You don’t need the optee_os source code per se. See build_ta_helloworld_qemu.sh as a reference. Find libteec.so and the export-ta_arm64 dir in the ${ANDROID_PRODUCT_OUT} dir and give it to your colleague. Then he/she can build TAs outside the aosp tree using https://github.com/linaro-swg/hello_world/blob/master/README.md or https://github.com/linaro-swg/hello_world/blob/master/build_ta_helloworld_qemu.sh as a reference. Adjust paths for TA_DEV_KIT_DIR and TEEC_EXPORT accordingly, and be sure to use an aosp toolchain instead!

optee_os will check the TA signature before loading and executing, but since this is just a reference implementation, the key and signature are known. Using and securing a key is platform specific so you should implement it specific to your hardware.

I think make -j12 TARGET_BUILD_KERNEL=true bootimage will rebuild everything you listed except boot_fat.uefi.img. Not sure if make -j12 TARGET_BUILD_KERNEL=true TARGET_BOOTIMAGE_USE_FAT=true bootimage will help or not. Please try it and check the date to see if the files are updated. If not, it shouldn’t take too long to rerun the full build anyway since most of the other stuffs haven’t changed.

Forgot to mention that your colleague will need the header files from /path/to/optee_client/out/export as well.

output as you asked :

hikey:/ # ps | grep tee
root 1862 1 7456 2136 optee_supp 0000000000 S /system/bin/tee-supplicant

ps is neither taking ‘e’ not ‘f’

Now , my next part is communicate with my spi device through TEE.
For that , I am trying to build a driver , which should execute in TEE and communicate with my device.
I will talk to my driver from NW binary ( similar to hello world program )

Is it possible to add new drivers to TEE-OS or should I need to modify the existing code here: optee/optee_os/core/drivers

Is there any reference material to add new drivers to OPTEE ?
Or can you show me right forum to clarify my doubts ?

Thanks

Hi @vchong

I am facing compile errors for Jack server while building AOSP 7 with OPTEE.

I have followed below link which you mentioned in your pervious posts and got compile error at step 3.8

below are the errors

[ 30% 14663/48104] Ensure Jack server is installed and started
FAILED: /bin/bash -c “(prebuilts/sdk/tools/jack-admin install-server prebuilts/sdk/tools/jack-launcher.jar prebuilts/sdk/tools/jack-server-4.8.ALPHA.jar 2>&1 || (exit 0) ) && (JACK_SERVER_VM_ARGUMENTS=”-Dfile.encoding=UTF-8 -XX:+TieredCompilation" prebuilts/sdk/tools/jack-admin start-server 2>&1 || exit 0 ) && (prebuilts/sdk/tools/jack-admin update server prebuilts/sdk/tools/jack-server-4.8.ALPHA.jar 4.8.ALPHA 2>&1 || exit 0 ) && (prebuilts/sdk/tools/jack-admin update jack prebuilts/sdk/tools/jacks/jack-2.28.RELEASE.jar 2.28.RELEASE || exit 47; prebuilts/sdk/tools/jack-admin update jack prebuilts/sdk/tools/jacks/jack-3.36.CANDIDATE.jar 3.36.CANDIDATE || exit 47; prebuilts/sdk/tools/jack-admin update jack prebuilts/sdk/tools/jacks/jack-4.7.BETA.jar 4.7.BETA || exit 47 )"
Jack server already installed in “/home/svaddi/.jack-server”
Communication error with Jack server (58), try ‘jack-diagnose’ or see Jack server log
Failed to contact Jack server: Problem reading /home/svaddi/.jack-server/client.pem. Try ‘jack-diagnose’
Failed to contact Jack server: Problem reading /home/svaddi/.jack-server/client.pem. Try ‘jack-diagnose’
[ 30% 14663/48104] host Java: conscrypt-host (out/host/common/obj/JAVA_LIBRARIES/conscrypt-host_intermediates/classes)
warning: [options] bootstrap class path not set in conjunction with -source 1.7
external/conscrypt/src/openjdk/java/org/conscrypt/Platform.java:45: warning: AlgorithmId is internal proprietary API and may be removed in a future release

can you please support ? (Note : I am using 16GB RAM PC and Java version is 1.8)

Thanks in advance,
Raj.

@deepakmnvl
Ok, thanks! The xtest errors are not really problems with OP-TEE but rather related to selinux. In any case, it’s a known issue and currently being worked on.

Regarding spi driver issue, seems like it’s already answered in https://github.com/OP-TEE/optee_os/issues/1461.

@Rajasekhara_Reddy
What does the Jack server log say? Did you try running jack-diagnose as recommended? Did you make any changes to the Jack server? It usually just runs without any issue or modifications required. Have you tried deleting /home/svaddi/.jack-server and /home/svaddi/.jack-settings and build again?

@vchong,

I didn’t change anything for jack server and tried all the above suggestions mentioned by you already but didn’t worked out :frowning:

By searching in google i have found some solution and tried, it was worked out i.e. kill server and start server.

Now i will flash my image files in Hikey board and check.

If any problem comes will let you know.

Thanks,
Raj.