OpTEE + Android in Hikey960


#106

There’s no info about the error, but all you need is to make sure that a new prm_ptable.img is generated in device/linaro/hikey/installer/hikey960. Then flash this file along with all the other images you have previously.

Anyway, as a side note, not sure how you managed to flash all images previously without errors. There should have been an error when flashing userdata.img.


#107

Hi @vchong,

Last day I just received the HiKey 960 board and trying to flash the AOSP with OP-TEE images on that board.
Followed the https://github.com/vchong/optee_android_manifest/blob/wip/tmp.txt and build the images to flash.

Now after flashing the images I am getting the boot logs as mentioned in https://pastebin.com/47ghy1Ty
and the board is not booting up.

Please help me.

Thanks,
@rabindranath-derebac


#108

Flash log for your reference https://pastebin.com/ZtC09XpH


#109

There’s no proper release of HiKey960 AOSP with OP-TEE support. https://github.com/vchong/optee_android_manifest/blob/wip/tmp.txt is just a poc that hasn’t been updated for a while, so it’s relatively too old and fragile to be used. We don’t have enough resources right now to continue development and support this atm. You can try deleting the line that says BL2_AT_EL3 := 1 in device/linaro/arm-trusted-firmware/plat/hisilicon/hikey960/platform.mk (or set it to 0) and rebuilding again, but if it doesn’t work then sorry, you’ll just have to wait to see if a proper release will ever be done.


#110

Made the change BL2_AT_EL3 := 0
make clean
make TARGET_BUILD_UEFI=true TARGET_TEE_IS_OPTEE=true

The build failed with the following error

#warning “Using deprecated console implementation. Please migrate to MULTI_CONSOLE_API”
^~~~~~~
LD build/hikey960/release/bl2/bl2.elf
LD build/hikey960/release/bl31/bl31.elf
./build/hikey960/release/bl2/bl2_entrypoint.o: In function bl2_entrypoint': (.text.asm.bl2_entrypoint+0x74): undefined reference tobl2_plat_arch_setup’
./build/hikey960/release/bl2/bl2_entrypoint.o: In function bl2_entrypoint': (.text.asm.bl2_entrypoint+0x74): relocation truncated to fit: R_AARCH64_CALL26 against undefined symbolbl2_plat_arch_setup’
./build/hikey960/release/bl2/plat_bl_common.o: In function bl2_early_platform_setup2': plat_bl_common.c:(.text.bl2_early_platform_setup2+0x4): undefined reference tobl2_early_platform_setup’
plat_bl_common.c:(.text.bl2_early_platform_setup2+0x4): relocation truncated to fit: R_AARCH64_JUMP26 against undefined symbol `bl2_early_platform_setup’
make[1]: *** [build/hikey960/release/bl2/bl2.elf] Error 1
make[1]: *** Waiting for unfinished jobs…
BIN build/hikey960/release/bl1.bin

Built build/hikey960/release/bl1.bin successfully

make[1]: Leaving directory `/home/optee/devel/AOSP_960_not_stable/device/linaro/arm-trusted-firmware’

                                   Hikey960 RELEASE fail

pass 0
fail 1
make: *** [out/dist/fip.bin] Error 1
make: Leaving directory `/home/optee/devel/AOSP_960_not_stable/device/linaro/hikey/bootloader’
[ 3% 2174/70097] target C++: libv8src <= external/v8/src/builtins/builtins-boolean.cc
ninja: build stopped: subcommand failed.
19:06:00 ninja failed with: exit status 1

failed to build some targets (06:43 (mm:ss))


#111

./device/linaro/arm-trusted-firmware/tools/fiptool/fiptool info device/linaro/hikey/installer/hikey960/fip.bin
SCP Firmware SCP_BL2: offset=0x200, size=0x35100, cmdline="–scp-fw"
EL3 Runtime Firmware BL31: offset=0x35400, size=0x8010, cmdline="–soc-fw"
Secure Payload BL32 (Trusted OS): offset=0x3D600, size=0x44270, cmdline="–tos-fw"
Non-Trusted Firmware BL33: offset=0x81A00, size=0xF0000, cmdline="–nt-fw"


#112

Best I can offer you now is a possible hack which requires 2 builds. Note that I’ve not tried this before.

  1. Build aosp for hikey960 based on https://source.android.com/setup/build/devices#960hikey. Make sure you can get the board booted up and running with the images built here. Note that this is important. Do not continue to the next step until the board is up and running.

  2. I believe you have a hikey aosp build? In your hikey aosp src tree, replace device/linaro/hikey/uefi-tools dir with the one from your hikey960 aosp src tree. Then, also replace device/linaro/bootloader/arm-trusted-firmware dir with the one from your hikey960 aosp src tree.

  3. In your hikey aosp src tree, in device/linaro/hikey/bootloader/Makefile, replace the line that says $(UEFI_TOOLS_DIR)/uefi-build.sh -b $(TARGET) -D EDK2_OUT_DIR=$(ANDROID_BUILD_TOP)/$(PRODUCT_OUT)/obj/uefi -a $(ATF_DIR) hikey with $(UEFI_TOOLS_DIR)/uefi-build.sh -b $(TARGET) -D EDK2_OUT_DIR=$(ANDROID_BUILD_TOP)/$(PRODUCT_OUT)/obj/uefi -a $(ATF_DIR) hikey960.

  4. Run a build in your hikey aosp src tree. If successful, take below listed files from the out/* dir and adb push them to your hikey960 board at the proper location (same location as they’re on the hikey board).

fip.bin
libteec.so
tee-supplicant
xtest
*.ta

For fip.bin, you have to flash it using the fastboot flash fip fip.bin command instead of adb push.

Sorry but I really can’t afford to spend any more time on this.


#113

Addition to item 2 above.
In your hikey aosp src tree, also replace device/linaro/bootloader/edk2 and device/linaro/bootloader/OpenPlatformPkg dirs with the ones from your hikey960 aosp src tree.

Addition to item 4 above.
In your hikey build, proceed until step 3.3 in https://github.com/linaro-swg/optee_android_manifest/tree/lcr-ref-hikey-o, then complete steps 1-3 above, before continuing with step 3.4.


#114

After doing the above steps I found that
/dev/tee0 and /dev/teepriv0 are not available on the board.

tee-supplicant is not able to start and no TA is executing.

Do I need to flash any other images to get /dev/tee0 and /dev/teepriv0 on the board ?


#115

Try flashing the boot.img you had built earlier using the instructions from https://github.com/vchong/optee_android_manifest/blob/wip/tmp.txt.


#116

Boot failed after flashing the boot.img


#117

Failed how? Anyway, going through the source code the boot.img from step 1 above should have worked as well. Did you do “Building the kernel” from https://source.android.com/setup/build/devices#960hikey?


#118

Hi @vchong
I did the following and not able to bring up the HiKey960 board using OP-TEE with AOSP

1. Build aosp for hikey960 based on https://source.android.com/setup/build/devices#960hikey. 
	Boot successful without OP-TEE.
2. Build AOSP with OP-TEE using  https://github.com/linaro-swg/optee_android_manifest/tree/lcr-ref-hikey-o 

and device/linaro/bootloader/arm-trusted-firmware, device/linaro/hikey/uefi-tools, device/linaro/bootloader/edk2, device/linaro/bootloader/OpenPlatformPkg from #1
3. Flash fip.bin & boot.img from #2 but boot failed
--------------boot log…
Audio_codec: [audio]:Could not find “hisilicon,codec-controller” node.
load_teeos: can not find fb node fingerprint
load_teeos: failed to find fb node ese_config
display: [display]hisi_get_disp_info: hisifb_data is NULL pointer, return!
bfm: : >>>>>enter set_boot_stage
bfm: : bootStageCode is 0x02010003, mntnStageCode is 32
bfm: : <<<<<exit set_boot_stage.
misc: [get_misc_message] mmc_read failed
preboot: [fastboot]: check_unlock_misc_info, line:1273 :get misc message failed.
preboot: [fastboot]: check_powerkey_misc_info, line:1242 :!!!CAUTION!!!:get keypad_operator FAIL!
preboot: [fastboot]: check_volumekey_state, line:943 :!!!CAUTION!!!:get keypad_operator FAIL!
misc: [get_misc_message] mmc_read failed
preboot: [fastboot]: comm_read_misc_cmd, line:501 :get misc message failed.
watchdog: Disable watchdog start
bfm: : [disable_boot_fail_system] boot_fail_system state is 0
rescue: ^^^^^^^^^[rescue_init] ok !
usbloader: bootmode is 4
--------------boot log…

4. Building the kernel from https://source.android.com/setup/build/devices#960hikey2? on #2 environment
	Then executed steps at #3 but the result is same

#119

Don’t use the boot.img from #2. Use the boot.img from #1 environment. If you build the kernel, also build it in the #1 environment.


#120

Hi @vchong thanks for your help.

Finally able to boot the hikey960 board using OP-TEE with AOSP.

  1. Recovery from tools-images-hikey960
    https://github.com/96boards-hikey/tools-images-hikey960

  2. Flash reference build
    https://snapshots.linaro.org/96boards/reference-platform/components/uefi-staging/69/hikey960/release/
    Install UEFI on HiKey960 from Binaries [Instructions]

  3. fip.bin from Fresh build…
    https://github.com/OP-TEE/build

  4. Boot and System images from AOSP960 build
    https://source.android.com/setup/build/devices

  5. Build latest boot image using updated kernel
    https://source.android.com/setup/build/devices

  6. xtest, tee-supplicant and .ta from build https://github.com/linaro-swg/optee_android_manifest/tree/lcr-ref-hikey-o

But getting the error while running the xtest

console:/ # xtest 200
Test ID: 200
Run test suite with level=0

TEE test application started with device [(null)]
######################################################

regression

######################################################

  • regression_2001 Trivial TCP iSocket API tests
    o regression_2001.1 Start server
    regression_2001.1 OK
    o regression_2001.2 TCP Socket open
    D/TC:0 core_mmu_entry_to_finer_grained:631 xlat tables used 3 / 5
    D/TC:0 tee_ta_init_pseudo_ta_session:297 Lookup pseudo TA 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (Secure Storage TA)
    D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (REE)
    D/TC:0 load_elf_from_store:759 ELF load address 0x40005000
    D/TC:0 tee_ta_init_user_ta_session:879 Processing relocations in 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    E/TC:0 tee_ta_init_user_ta_session:901 Invalid TA flag(s) 0x3
    D/TC:0 tee_ta_open_session:563 init session failed 0xffff0005
    external/optee_test/host/xtest/regression_2000.c:298: xtest_teec_open_session( &session, &socket_ta_uuid, ((void*)0), &ret_orig) has an unexpected value: 0xffff0005 = TEEC_ERROR_BAD_FORMAT, expected 0x0 = TEEC_SUCCESS
    regression_2001.2 FAILED
    regression_2001 FAILED

  • regression_2002 Concurrent stressing TCP iSocket API tests
    o regression_2002.1 Stressing with 3 threads
    D/TC:0 tee_ta_init_pseudo_ta_session:297 Lookup pseudo TA 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    *D/TC:1 __wq_rpc:40 sleep thread 1 0x3f045198 0
    *D/TC:2 __wq_rpc:40 sleep thread 2 0x3f045198 0
    *D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (Secure Storage TA)
    D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (REE)
    D/TC:0 load_elf_from_store:759 ELF load address 0x40005000
    D/TC:0 tee_ta_init_user_ta_session:879 Processing relocations in 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    E/TC:0 tee_ta_init_user_ta_session:901 Invalid TA flag(s) 0x3
    D/TC:0 __wq_rpc:40 wake thread 1 0x3f045198 -3
    D/TC:0 tee_ta_open_session:563 init session failed 0xffff0005
    *D/TC:1 tee_ta_init_pseudo_ta_session:297 Lookup pseudo TA 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    D/TC:1 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (Secure Storage TA)
    D/TC:1 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (REE)
    D/TC:1 load_elf_from_store:759 ELF load address 0x40005000
    D/TC:1 tee_ta_init_user_ta_session:879 Processing relocations in 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    E/TC:1 tee_ta_init_user_ta_session:901 Invalid TA flag(s) 0x3
    D/TC:1 __wq_rpc:40 wake thread 2 0x3f045198 -3
    D/TC:1 tee_ta_open_session:563 init session failed 0xffff0005
    *D/TC:2 tee_ta_init_pseudo_ta_session:297 Lookup pseudo TA 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    D/TC:2 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (Secure Storage TA)
    D/TC:2 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (REE)
    D/TC:2 load_elf_from_store:759 ELF load address 0x40005000
    D/TC:2 tee_ta_init_user_ta_session:879 Processing relocations in 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    E/TC:2 tee_ta_init_user_ta_session:901 Invalid TA flag(s) 0x3
    D/TC:2 tee_ta_open_session:563 init session failed 0xffff0005
    external/optee_test/host/xtest/regression_2000.c:538: arg[n].success has an unexpected value: 0x0 = false, expected 0x1 = true
    external/optee_test/host/xtest/regression_2000.c:538: arg[n].success has an unexpected value: 0x0 = false, expected 0x1 = true
    external/optee_test/host/xtest/regression_2000.c:538: arg[n].success has an unexpected value: 0x0 = false, expected 0x1 = true
    regression_2002.1 FAILED
    regression_2002 FAILED

  • regression_2003 Timeout TCP iSocket API tests
    o regression_2003.1 Start server
    regression_2003.1 OK
    o regression_2003.2 TCP Socket open
    D/TC:0 tee_ta_init_pseudo_ta_session:297 Lookup pseudo TA 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (Secure Storage TA)
    D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (REE)
    D/TC:0 load_elf_from_store:759 ELF load address 0x40005000
    D/TC:0 tee_ta_init_user_ta_session:879 Processing relocations in 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    E/TC:0 tee_ta_init_user_ta_session:901 Invalid TA flag(s) 0x3
    D/TC:0 tee_ta_open_session:563 init session failed 0xffff0005
    external/optee_test/host/xtest/regression_2000.c:583: xtest_teec_open_session( &session, &socket_ta_uuid, ((void*)0), &ret_orig) has an unexpected value: 0xffff0005 = TEEC_ERROR_BAD_FORMAT, expected 0x0 = TEEC_SUCCESS
    regression_2003.2 FAILED
    regression_2003 FAILED

  • regression_2004 UDP iSocket API tests
    o regression_2004.1 Start server
    regression_2004.1 OK
    o regression_2004.2 UDP Socket open
    D/TC:0 tee_ta_init_pseudo_ta_session:297 Lookup pseudo TA 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (Secure Storage TA)
    D/TC:0 load_elf:791 Lookup user TA ELF 873bcd08-c2c3-11e6-a937-d0bf9c45c61c (REE)
    D/TC:0 load_elf_from_store:759 ELF load address 0x40005000
    D/TC:0 tee_ta_init_user_ta_session:879 Processing relocations in 873bcd08-c2c3-11e6-a937-d0bf9c45c61c
    E/TC:0 tee_ta_init_user_ta_session:901 Invalid TA flag(s) 0x3
    D/TC:0 tee_ta_open_session:563 init session failed 0xffff0005
    external/optee_test/host/xtest/regression_2000.c:708: xtest_teec_open_session( &session, &socket_ta_uuid, ((void*)0), &ret_orig) has an unexpected value: 0xffff0005 = TEEC_ERROR_BAD_FORMAT, expected 0x0 = TEEC_SUCCESS
    regression_2004.2 FAILED
    regression_2004 FAILED
    ±----------------------------------------------------
    Result of testsuite regression filtered by “200”:
    regression_2001.2 FAILED first error at external/optee_test/host/xtest/regression_2000.c:298
    regression_2001 FAILED
    regression_2002.1 FAILED first error at external/optee_test/host/xtest/regression_2000.c:538
    regression_2002 FAILED
    regression_2003.2 FAILED first error at external/optee_test/host/xtest/regression_2000.c:583
    regression_2003 FAILED
    regression_2004.2 FAILED first error at external/optee_test/host/xtest/regression_2000.c:708
    regression_2004 FAILED
    ±----------------------------------------------------
    18 subtests of which 6 failed
    4 test cases of which 4 failed
    140 test cases was skipped
    TEE test application done!


Filesystem not available
Filesystem not available
#121

You’re welcome! Good job putting the pieces together and thanks for sharing!

Din you run xtest as root? In yes, then try adding these selinux policies:
https://android-review.linaro.org/#/c/17177/12/sepolicy/file_contexts
https://android-review.linaro.org/#/c/17177/12/sepolicy/tee.te
https://android-review.linaro.org/#/c/18062/2/sepolicy/tee.te


#122

Now xtest is running OK except the following few test cases related to SDP
console:/ # xtest 1014
Test ID: 1014
Run test suite with level=0

TEE test application started with device [(null)]
######################################################

regression

######################################################

  • regression_1014 Test secure data path against SDP TAs and pTAs
    o regression_1014.1 SDP: NonSecure client invokes a SDP TA
    Error: failed to query the number of heaps
    external/optee_test/host/xtest/regression_1000.c:1258: ret has an unexpected value: 0x1, expected 0x0
    regression_1014.1 FAILED
    o regression_1014.2 SDP: SDP TA invokes a SDP TA
    Error: failed to query the number of heaps
    external/optee_test/host/xtest/regression_1000.c:1264: ret has an unexpected value: 0x1, expected 0x0
    regression_1014.2 FAILED
    o regression_1014.3 SDP: SDP TA invokes a SDP pTA
    Error: failed to query the number of heaps
    external/optee_test/host/xtest/regression_1000.c:1270: ret has an unexpected value: 0x1, expected 0x0
    regression_1014.3 FAILED
    o regression_1014.4 SDP: NSec CA invokes SDP pTA (should fail)
    Error: failed to query the number of heaps
    regression_1014.4 OK
    regression_1014 FAILED
    ±----------------------------------------------------
    Result of testsuite regression filtered by “1014”:
    regression_1014.1 FAILED first error at external/optee_test/host/xtest/regression_1000.c:1258
    regression_1014.2 FAILED first error at external/optee_test/host/xtest/regression_1000.c:1264
    regression_1014.3 FAILED first error at external/optee_test/host/xtest/regression_1000.c:1270
    regression_1014 FAILED
    ±----------------------------------------------------
    4 subtests of which 3 failed
    1 test case of which 1 failed
    66 test cases was skipped
    TEE test application done!

#123

For xtest 200, did the selinux policies help?

Sigh, one more missing piece. The kernel you specified in

  1. Build latest boot image using updated kernel
    https://source.android.com/setup/build/devices

is not fully patched for SDP, i.e. missing CFG_* for SDP in the defconfig and etc.

Try this. Run https://github.com/linaro-swg/optee_android_manifest/tree/lcr-ref-hikey-o until step 3.4. Then stop. Next cd to kernel/linaro/hisilicon. After that:

make ARCH=arm64 hikey960_defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- -j24

After that, continue with step 2 and 3 in the ‘Building the kernel’ section of https://source.android.com/setup/build/devices#960hikey. In step 2, make sure you copy hi3660-hikey960.dtb and Image file over to the hikey-kernel directory of your hikey960 aosp source tree! Then run step 3 in your hikey960 aosp source tree, and reflash the boot img.


#124

Btw, this hack is so bad and ugly at this point I don’t know if I’d recommend anybody using it.

What are you using this build for?


#125

Basically trying to bring up the board with OP-TEE with AOSP and after that planning to do some hardware decoding related stuff.