OpTEE + Android in Hikey960

tesmnorth · October 17, 2017, 8:37am

thanks a lot. @vchong

tesmnorth · October 30, 2017, 1:55pm

@vchong Can I say error in this topic?

xx@xx:~/optee_android_manifest$ make TARGET_BUILD_UEFI=true TARGET_TEE_IS_OPTEE=true
============================================
PLATFORM_VERSION_CODENAME=P
PLATFORM_VERSION=P
TARGET_PRODUCT=hikey960
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-a
TARGET_CPU_VARIANT=cortex-a73
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-a
TARGET_2ND_CPU_VARIANT=cortex-a73
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.10.0-37-generic-x86_64-Ubuntu-16.04.3-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=OC-DR1
OUT_DIR=out
============================================
[1/1] out/soong/.minibootstrap/minibp out/soong/.bootstrap/build.ninja
[868/869] glob vendor/*/*/Android.bp
[75/75] out/soong/.bootstrap/bin/soong_build out/soong/build.ninja
out/build-hikey960-cleanspec.ninja is missing, regenerating...
out/build-hikey960.ninja is missing, regenerating...
[128/917] including device/linaro/hikey/Android.mk ...
gralloc for hikey960
[237/917] including external/optee_test/Android.mk ...
external/optee_test/Android.mk: warning: xtest: Bad LOCAL_ADDITIONAL_DEPENDENCIES: BUILD_OPTEE_OS 
external/optee_test/Android.mk: error: xtest: LOCAL_ADDITIONAL_DEPENDENCIES must only contain paths (not module names) 
build/make/core/base_rules.mk:103: error: done.
15:03:39 ckati failed with: exit status 1

#### failed to build some targets (01:03 (mm:ss)) ####

vchong · October 30, 2017, 4:03pm

@tesmnorth

Looks like a new rule on master that breaks our makefiles.
In core/base_rules.mk, please delete the lines below and try again.

my_bad_deps := $(strip $(foreach dep,$(filter-out | ||,$(LOCAL_ADDITIONAL_DEPENDENCIES)),\
                 $(if $(findstring /,$(dep)),,$(dep))))
ifneq ($(my_bad_deps),)
$(call pretty-warning,"Bad LOCAL_ADDITIONAL_DEPENDENCIES: $(my_bad_deps)")
$(call pretty-error,"LOCAL_ADDITIONAL_DEPENDENCIES must only contain paths (not module names)")
endif

Note that this is just a hack (which might or might not work) and will need to be fixed properly in the future. Also, please keep track so all the steps you perform so we can trace back any changes if necessary.

tesmnorth · October 31, 2017, 7:37am

@vchong after editing, different fail

24576 bytes (25 kB, 24 KiB) copied, 0,00015803 s, 156 MB/s
++ expr 7805952 - 5
+ BK_PTABLE_LBA=7805947
+ dd if=/tmp/linux-32g.vIHP4x of=sec_ptable.img skip=7805947 bs=4096 count=5
5+0 records in
5+0 records out
20480 bytes (20 kB, 20 KiB) copied, 0,000154821 s, 132 MB/s
+ rm -f /tmp/linux-32g.vIHP4x
make: Leaving directory '/home/xx/optee_android_manifest/device/linaro/hikey/bootloader'
[  1% 1281/75561] target  C: toybox_vendor <= external/toybox/toys/posix/chgrp.c
ninja: build stopped: subcommand failed.
10:34:41 ninja failed with: exit status 1

#### failed to build some targets (01:54 (mm:ss)) ####

vchong · November 1, 2017, 7:25am

Can you run below commands without error?

cd device/linaro/l-loader
PTABLE=linux-32g SECTOR_SIZE=4096 SGDISK=./sgdisk bash -x generate_ptable.sh
cp prm_ptable.img out/dist/
cp prm_ptable.img ../hikey/installer/hikey960

vchong · November 1, 2017, 8:05am

Please ignore post above. The error is something else. Do you have more logs to show more details regarding the error?

tesmnorth · November 1, 2017, 8:20am

+ fakeroot ./sgdisk -n 8:0:+16M -t 8:0700 -u 8:6e53b0bb-fa7e-4206-b607-5ae699e9f066 -c 8:dts /tmp/linux-32g.Lb8gXG
Setting name!
partNum is 7
REALLY setting name!
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
+ fakeroot ./sgdisk -n 9:0:+2M -t 9:0700 -u 9:f1e126a6-ceef-45c1-aace-29f33ac9cf13 -c 9:trustfirmware /tmp/linux-32g.Lb8gXG
Setting name!
partNum is 8
REALLY setting name!
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
+ fakeroot ./sgdisk -n 10:0:+4688M -t 10:8300 -u 10:c3e50923-fb85-4153-b925-759614d4dfcd -c 10:system /tmp/linux-32g.Lb8gXG
Setting name!
partNum is 9
REALLY setting name!
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
+ fakeroot ./sgdisk -n 11:0:+784M -t 11:0700 -u 11:919d7080-d71a-4ae1-9227-e4585210c837 -c 11:vendor /tmp/linux-32g.Lb8gXG
Setting name!
partNum is 10
REALLY setting name!
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
+ fakeroot ./sgdisk -n 12:0:+1M -t 12:0700 -u 12:611eac6b-bc42-4d72-90ac-418569c8e9b8 -c 12:reserved /tmp/linux-32g.Lb8gXG
Setting name!
partNum is 11
REALLY setting name!
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
+ dd if=/tmp/linux-32g.Lb8gXG of=prm_ptable.img bs=4096 count=6
6+0 records in
6+0 records out
24576 bytes (25 kB, 24 KiB) copied, 0,00106425 s, 23,1 MB/s
++ expr 7805952 - 5
+ BK_PTABLE_LBA=7805947
+ dd if=/tmp/linux-32g.Lb8gXG of=sec_ptable.img skip=7805947 bs=4096 count=5
5+0 records in
5+0 records out
20480 bytes (20 kB, 20 KiB) copied, 0,00448043 s, 4,6 MB/s
+ rm -f /tmp/linux-32g.Lb8gXG
make: Leaving directory '/home/xx/optee_android_manifest/device/linaro/hikey/bootloader'
[  1% 1032/75561] target  C: toybox <= external/toybox/toys/other/blkid.c
ninja: build stopped: subcommand failed.
11:18:51 ninja failed with: exit status 1

vchong · November 1, 2017, 2:07pm

Nope, still can’t find any details regarding the error. Are you using parallel builds maybe, e.g. make -jX …? Please try to go up further and look for the detailed error message. Or paste the whole log on a pastebin site.

tesmnorth · November 1, 2017, 3:04pm

@vchong https://pastebin.com/rR7JyPXM

vchong · November 1, 2017, 5:03pm

Ok, now there’s something. I made some changes so please try below from your main build dir.

cd .repo/local_manifest
git pull
cd ../../device/linaro
repo sync uefi-tools OpenPlatformPkg arm-trusted-firmware edk2
cd hikey
repo sync .
cd ../../../

and build again…

tesmnorth · November 2, 2017, 11:17am

@vchong

  WriteRelocations64(): /home/xx/optee_android_manifest/device/linaro/edk2/Build/HiKey960/RELEASE_GCC49/AARCH64/ArmPlatformPkg/PrePi/PeiUniCore/DEBUG/ArmPlatformPrePiUniCore.dll unsupported ELF EM_AARCH64 relocation 0x135.
GNUmakefile:404: recipe for target '/home/xx/optee_android_manifest/device/linaro/edk2/Build/HiKey960/RELEASE_GCC49/AARCH64/ArmPlatformPkg/PrePi/PeiUniCore/DEBUG/ArmPlatformPrePiUniCore.efi' failed
make[1]: Leaving directory '/home/xx/optee_android_manifest/out/target/product/hikey960/obj/uefi/RELEASE_GCC49/AARCH64/ArmPlatformPkg/PrePi/PeiUniCore'


build.py...
 : error 7000: Failed to execute command
	make tbuild [/home/xx/optee_android_manifest/device/linaro/edk2/Build/HiKey960/RELEASE_GCC49/AARCH64/ArmPlatformPkg/PrePi/PeiUniCore]


build.py...
 : error 7000: Failed to execute command
	make tbuild [/home/xx/optee_android_manifest/device/linaro/edk2/Build/HiKey960/RELEASE_GCC49/AARCH64/ArmPkg/Drivers/CpuDxe/CpuDxe]


build.py...
 : error 7000: Failed to execute command
	make tbuild [/home/xx/optee_android_manifest/device/linaro/edk2/Build/HiKey960/RELEASE_GCC49/AARCH64/MdeModulePkg/Core/Dxe/DxeMain]


build.py...
 : error F002: Failed to build module
	/home/xx/optee_android_manifest/device/linaro/edk2/ArmPlatformPkg/PrePi/PeiUniCore.inf [AARCH64, GCC49, RELEASE]

- Failed -
Build end time: 11:54:18, Nov.02 2017
Build total time: 00:01:02

------------------------------------------------------------
                                       Hikey960 RELEASE	fail
------------------------------------------------------------
pass	0
fail	1
Makefile:27: recipe for target '/home/martin/optee_android_manifest/out/dist/fip.bin' failed
make: *** [/home/xx/optee_android_manifest/out/dist/fip.bin] Error 1
make: Leaving directory '/home/xx/optee_android_manifest/device/linaro/hikey/bootloader'
[  2% 2269/76802] target  C++: libv8src <= external/v8/src/builtins/builtins-function.cc
ninja: build stopped: subcommand failed.
11:54:22 ninja failed with: exit status 1

#### failed to build some targets (03:28 (mm:ss)) ####

vchong · November 2, 2017, 3:28pm

cd device/linaro/hikey
git reset --hard HEAD~2 #be very careful when u do this!
repo sync .
cd ../edk2
git clean -xfd #also be careful here
cd ../../../
rm -f out/dist/fip.bin #ignore any warning/error

Build again.

tesmnorth · November 3, 2017, 6:22am

plat/common/aarch64/plat_common.c: In function ‘plat_get_syscnt_freq2’:
plat/common/aarch64/plat_common.c:60:2: warning: ‘plat_get_syscnt_freq’ is deprecated [-Wdeprecated-declarations]
  unsigned long long freq = plat_get_syscnt_freq();
  ^
In file included from plat/common/aarch64/plat_common.c:8:0:
include/plat/common/platform.h:41:20: note: declared here
 unsigned long long plat_get_syscnt_freq(void) __deprecated;
                    ^
  CC      lib/stdlib/assert.c
  CC      lib/stdlib/exit.c
  CC      lib/stdlib/mem.c
  CC      lib/stdlib/printf.c
  CC      lib/stdlib/putchar.c
  CC      lib/stdlib/puts.c
  CC      lib/stdlib/sscanf.c
  CC      lib/stdlib/strchr.c
  CC      lib/stdlib/strcmp.c
  CC      lib/stdlib/strlen.c
  CC      lib/stdlib/strncmp.c
  CC      lib/stdlib/strnlen.c
  CC      lib/stdlib/subr_prf.c
  CC      lib/stdlib/timingsafe_bcmp.c
  CC      drivers/delay_timer/delay_timer.c
  CC      drivers/delay_timer/generic_delay_timer.c
  CC      lib/aarch64/xlat_tables.c
  CC      plat/hisilicon/hikey960/aarch64/hikey960_common.c
  CC      plat/hisilicon/hikey960/hikey960_boardid.c
  AS      lib/cpus/aarch64/cortex_a53.S
  AS      lib/cpus/aarch64/cortex_a72.S
  AS      lib/cpus/aarch64/cortex_a73.S
  AS      plat/hisilicon/hikey960/aarch64/hikey960_helpers.S
  AS      bl31/aarch64/runtime_exceptions.S
  AS      bl31/aarch64/crash_reporting.S
  AS      bl31/aarch64/bl31_entrypoint.S
  AS      plat/common/aarch64/platform_mp_stack.S
  AS      lib/el3_runtime/aarch64/cpu_data.S
bl31/aarch64/runtime_exceptions.S: Assembler messages:
bl31/aarch64/runtime_exceptions.S:157: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:165: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:170: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:175: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:189: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:193: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:197: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:201: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:215: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:219: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:223: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:231: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:245: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:249: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:253: Error: non-constant expression in ".if" statement
bl31/aarch64/runtime_exceptions.S:261: Error: non-constant expression in ".if" statement
Makefile:558: recipe for target 'build/hikey960/release/bl31/runtime_exceptions.o' failed
make[1]: *** [build/hikey960/release/bl31/runtime_exceptions.o] Error 1
make[1]: *** Waiting for unfinished jobs....
  AS      lib/locks/exclusive/aarch64/spinlock.S
  AS      lib/cpus/aarch64/cpu_helpers.S
make[1]: Leaving directory '/home/xx/optee_android_manifest/device/linaro/arm-trusted-firmware'
------------------------------------------------------------
                                       Hikey960 RELEASE	fail
------------------------------------------------------------
pass	0
fail	1
Makefile:45: recipe for target '/home/xx/optee_android_manifest/out/dist/fip.bin' failed
make: *** [/home/xx/optee_android_manifest/out/dist/fip.bin] Error 1
make: Leaving directory '/home/xx/optee_android_manifest/device/linaro/hikey/bootloader'
[  2% 2283/76802] target  C++: libv8src <= external/v8/src/builtins/builtins-number.cc
ninja: build stopped: subcommand failed.
09:13:38 ninja failed with: exit status 1

#### failed to build some targets (03:36 (mm:ss)) ####

vchong · November 3, 2017, 7:37am

I didn’t see this error when trying yesterday. Maybe make distclean in arm-trusted-firmware and try again?

tesmnorth · November 3, 2017, 8:03am

I have done it and rebuilded again. Error is changed. It is same with previous error but Makefile:548: recipe for target 'build/hikey960/release/bl2/early_exceptions.o' failed

It can be different gnu or gcc version?
Edit: I used :
sudo apt-get install gcc-aarch64-linux-gnu
sudo apt-get install gcc-arm-linux-gnueabihf

Edit2: I tried to build it again. Now, It is Makefile:544: recipe for target 'build/hikey960/release/bl1/bl1_exceptions.o' failed . It changed again.

vchong · November 3, 2017, 9:14am

It shouldn’t matter because the makefile downloads and uses its own toochains.

Which arm-trusted-firmware did you clean? It should be the one in device/linaro/.
If 1 is ok, can you try below?

make distclean in arm-trusted-firmware.
cd device/linaro/hikey/bootloader
make tst_fipbin_bld
If build passes, then just continue android build.

tesmnorth · November 3, 2017, 10:54am

It is OK. I clean it in device/linaro/arm-trusted-firmware.

I have run above commands. Result is same.
bl1/aarch64/bl1_exceptions.S: Assembler messages:
bl1/aarch64/bl1_exceptions.S:29: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:35: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:41: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:47: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:57: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:63: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:69: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:75: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:94: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:100: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:106: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:112: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:122: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:128: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:134: Error: non-constant expression in “.if” statement
bl1/aarch64/bl1_exceptions.S:140: Error: non-constant expression in “.if” statement
Makefile:544: recipe for target ‘build/hikey960/release/bl1/bl1_exceptions.o’ failed
make[1]: *** [build/hikey960/release/bl1/bl1_exceptions.o] Error 1
make[1]: *** Waiting for unfinished jobs…
make[1]: Leaving directory ‘/home/xx/optee_android_manifest/device/linaro/arm-trusted-firmware’
------------------------------------------------------------
Hikey960 RELEASE fail
------------------------------------------------------------
pass 0
fail 1
Makefile:45: recipe for target ‘/home/xx/optee_android_manifest/out/dist/fip.bin’ failed
make: *** [/home/xx/optee_android_manifest/out/dist/fip.bin] Error 1

#### failed to build some targets (15 seconds) ####

vchong · November 3, 2017, 11:19am

Builds fine for me.

------------------------------------------------------------
                                       Hikey960 RELEASE	pass
------------------------------------------------------------
pass	1
fail	0

On further research, it might be a toolchain problem. Are you using Ubuntu 16.04? Can you try a linaro toolchain? E.g. http://releases.linaro.org/components/toolchain/binaries/5.4-2017.05/aarch64-linux-gnu/gcc-linaro-5.4.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz and http://releases.linaro.org/components/toolchain/binaries/5.4-2017.05/arm-linux-gnueabihf/gcc-linaro-5.4.1-2017.05-x86_64_arm-linux-gnueabihf.tar.xz.

tesmnorth · November 6, 2017, 11:43am

Yes, I am using Ubuntu 16.04. Now I am trying it. After trying I will edit this comment.
Edit: I think I have done somethings wrongly. Same error.
Edit2: I have installed Ubuntu 14.04.5 on my computer. I will try it again.
Edit3: @vchong

I re-installed Ubuntu as 14.04.5. I did everything step by step. I used linaro toolchain. But, Error is same. More detailed log : https://pastebin.com/Gs8tkN4a

Edit4: @vchong
This is for some tools version problem?

Edit5: @vchong I solved the problem. make TARGET_BUILD_UEFI=true TARGET_TEE_IS_OPTEE=true command needs root priviliges.

tesmnorth · November 7, 2017, 6:31am

Building is finished now. But, What do I need to do in following steps?

#build kernel and boot img
#MUST be after building aosp since we need <out_dir>/ramdisk.img
cd kernel/linaro/hisilicon
../../../device/linaro/l-loader/cc