Hikey Lemaker OPTEE kernel working source code

Sorry, I forgot to mention, you also have to cd to .repo/local_manifests and run git pull. So

cd .repo/local_manifests
git pull
cd ../../
repo sync
# apply patches

If you’re still getting errors, it might help to delete the out folder before rebuilding.

Hi @vchong,

I have tried the above mentioned method with OP-TEE version 2.4.0 and the Master branch.
But in bot the cases it is not working.

I did repo sync the AOSP + OP-TEE code base after modifying .repo/local_manifests/optee.xml file and replacing refs/tags/2.3.0 with refs/tags/2.4.0.

Then applied 4 patches
$ ./android-patchsets/hikey-n-workarounds
$ ./android-patchsets/hikey-optee-4.9
$ ./android-patchsets/hikey-optee-n
$ ./android-patchsets/swg-mods

Build failed in the xtest
external/optee_test/host/xtest/adbg/src/adbg_run.c:97:2: error: implicit declaration of function ‘TAILQ_CONCAT’ is invalid in C99 [-Werror,-Wimplicit-function-declaration]
TAILQ_CONCAT(&Dest_p->cases, &Source_p->cases, link);

Commented that line and then build successful.

Now wrote a sample tcp client TA using iSocket and also wrote a sample tcp server on the Rich OS.
Whenever trying to execute the sample tcp client TA it is crashing in the Socket open with the below mentioned log
DEBUG: USER-TA:sample_tcp_client:142: Socket open res=-65536 failed=0
ERROR: USER-TA: Panic 0x0
ERROR: TEE-CORE: TA panicked with code 0x0 usr_sp 0x40000fb0 usr_lr 0x0
310.322111] type=1400 audit(320.003:4): avc: denied { create } for pid=1908 comm=“tee-supplicant” scontext=u:r:tee:s0 tcontext=u:r:0
tee_tcpclient: TEEC_InvokeCommand failed with code 0xffff3024 origin 0x3

Then I did the following
I did repo sync the AOSP + OP-TEE code base after modifying .repo/local_manifests/optee.xml file and replacing refs/tags/2.3.0 with master.
Then applied 4 patches
$ ./android-patchsets/hikey-n-workarounds
$ ./android-patchsets/hikey-optee-4.9
$ ./android-patchsets/hikey-optee-n
$ ./android-patchsets/swg-mods

In this case the tee-supplicant is not running and getting the error when trying to manually run tee-supplicant.
ERR [3478] TEES:main:601: failed to find an OP-TEE supplicant device

Also found that the tee driver is also not running.

Then I did the
cd .repo/local_manifests
git pull
as mentioned by you in the last post.
But after that the build is failing.

Thanks,
Rabi

2.4.0 requires some patches which are currently being worked on.
Will post an update once they’re done.
Thanks!

Hi @vchong,

How can I write a file in OP-TEE which will stay even after rebooting the board.
I tried with
TEE_CreatePersistentObject
TEE_OpenPersistentObject
But after reboot the file does not exist.

Thanks,
Rabi

@rabindranath-derebac Did you create a new app and .ta file? Or did you reuse/recycle and existing app like hello world?

Currently reusing the hello world app.

From the hello world TA I am creating a file using TEE_CreatePersistentObject()
Then open the file using TEE_OpenPersistentObject()

But after rebooting the board the file is erased which was created using TEE_CreatePersistentObject().

Thanks,
Rabi

Try comparing with https://github.com/OP-TEE/optee_test/blob/master/ta/storage/ta_entry.c and https://github.com/OP-TEE/optee_test/blob/master/ta/storage/storage.c to maybe see if there are any differences.

Tried with the APIs but that does not store the files permanently. So after reboot the files are erased from /data/tee directory.

Is there any way to write file to USB storage or SD card attached to the board?

2.4.0 build is up if you want to give it a try: https://github.com/linaro-swg/optee_android_manifest/tree/hikey-n-4.9-240

For the storage issue, it would help if you include what code you changed, what the build → flash → run procedure is, and what logs the console shows. Basically, steps to reproduce the issue is best. Btw, are you also running xtest when you run the hello world app?

You can secure store to an external emmc module with a microsd adapter, but rpmb has to be enabled in your build. See below for references:

LAS16-504: Secure Storage updates in OP-TEE (slides 7-8)
http://www.hardkernel.com/main/products/prdt_info.php?g_code=G135393511084
Odroid-XU4 eMMC module

Hi @vchong,

I will give a try with the 2.4.0. Thanks for the update.

For the storage issue I just wrote the below mentioned code in the Hello World TA just to verify how to

  •       write persistent object
    
  •       read the persistent object
    
  •      reboot the board and check if the persistent object is still present or not.
    

For RPMB build I did the following
…/optee_client/tee-supplicant/Makefile
commented RPMB_EMU := 1

…/optee_os/mk/config.mk
set CFG_REE_FS ?= n
set CFG_RPMB_FS ?= y
set CFG_RPMB_WRITE_KEY ?= y

For REE FS build I did the following
…/optee_client/tee-supplicant/Makefile
RPMB_EMU := 1

…/optee_os/mk/config.mk
set CFG_REE_FS ?= y
set CFG_RPMB_FS ?= n
set CFG_RPMB_WRITE_KEY ?= n

For both the cases with RPMB and without RPMB the file writing and reading is OK but the file is erased after reboot.

After flash executing hello world: In this case file is not present due to flash

ERROR: USER-TA: Failed to open persistent object, res=0xffff0008
DEBUG: USER-TA:inc_value:172: Calling TEE_CreatePersistentObject
DEBUG: USER-TA:inc_value:209: TEE_WriteObjectData SUCCESS
DEBUG: USER-TA:inc_value:233: TEE_ReadObjectData SUCCESS data=Test write using write data. rbytes=28:28

2nd time executing hello world after flash : In this case file is not created as it is present
DEBUG: USER-TA:inc_value:191: TEE_OpenPersistentObject SUCCESS file= FileTest
DEBUG: USER-TA:inc_value:209: TEE_WriteObjectData SUCCESS
DEBUG: USER-TA:inc_value:233: TEE_ReadObjectData SUCCESS data=Test write using write data. rbytes=28:28

Executing hello world after reboot : In is case file is not present though the file was there before reboot
ERROR: USER-TA: Failed to open persistent object, res=0xffff0008
DEBUG: USER-TA:inc_value:172: Calling TEE_CreatePersistentObject
DEBUG: USER-TA:inc_value:209: TEE_WriteObjectData SUCCESS
DEBUG: USER-TA:inc_value:233: TEE_ReadObjectData SUCCESS data=Test write using write data. rbytes=28:28

##======Code added inside hello world TA==========
//Storage related APIs
DMSG(“Starting STORAGE TEST”);
res = TEE_OpenPersistentObject(TEE_STORAGE_PRIVATE,
filename, sizeof(filename),
TEE_DATA_FLAG_ACCESS_READ |
TEE_DATA_FLAG_ACCESS_WRITE |
TEE_DATA_FLAG_ACCESS_WRITE_META |
TEE_DATA_FLAG_OVERWRITE,
&object);
if (res != TEE_SUCCESS) {
EMSG(“Failed to open persistent object, res=0x%08x”, res);
DMSG(“Calling TEE_CreatePersistentObject”);
res = TEE_CreatePersistentObject(TEE_STORAGE_PRIVATE,
filename, sizeof(filename),
TEE_DATA_FLAG_ACCESS_READ |
TEE_DATA_FLAG_ACCESS_WRITE |
TEE_DATA_FLAG_ACCESS_WRITE_META |
TEE_DATA_FLAG_OVERWRITE,
NULL, NULL, 0, &object);
if (res != TEE_SUCCESS) {
EMSG(“Failed to create persistent object, res=0x%08x”, res);
}
else {
DMSG(“Successfully created file %s”, filename);
TEE_CloseObject( object);
}
}
else
{
DMSG(“TEE_OpenPersistentObject SUCCESS file= %s”, filename);
TEE_CloseObject( object);
}

res = TEE_OpenPersistentObject(TEE_STORAGE_PRIVATE,
            filename, sizeof(filename),
            TEE_DATA_FLAG_ACCESS_READ |
            TEE_DATA_FLAG_ACCESS_WRITE |
            TEE_DATA_FLAG_ACCESS_WRITE_META |
            TEE_DATA_FLAG_OVERWRITE,
            &object);
if (res == TEE_SUCCESS)
{
    size = 28;
    memcpy(buffer, "Test write using write data.",28 );

    res = TEE_WriteObjectData( object, buffer, size );
    if (res == TEE_SUCCESS){
        DMSG("TEE_WriteObjectData SUCCESS");
    }
    else {
        DMSG("TEE_WriteObjectData FAILED res=%d",res);
    }
    TEE_CloseObject( object);
}
else
{
    DMSG("TEE_OpenPersistentObject for writing FAILED res=%d",res);
}

res = TEE_OpenPersistentObject(TEE_STORAGE_PRIVATE,
            filename, sizeof(filename),
            TEE_DATA_FLAG_ACCESS_READ |
            TEE_DATA_FLAG_ACCESS_WRITE |
            TEE_DATA_FLAG_ACCESS_WRITE_META |
            TEE_DATA_FLAG_OVERWRITE,
            &object);
			
if (res == TEE_SUCCESS)
{
    size = 28;
    res = TEE_ReadObjectData( object, &readbuffer, size, &count );
    if (res == TEE_SUCCESS){
        DMSG("TEE_ReadObjectData SUCCESS data=%s rbytes=%d:%d",readbuffer,count, size);
    }
    else {
		DMSG("TEE_ReadObjectData FAILED res=%d",res);
    }
    TEE_CloseObject( object);
}
else
{
    DMSG("TEE_OpenPersistentObject for reading FAILED res=%d",res);
}

DMSG("Done with STORAGE TEST");

Thanks,
Rabi

Where and how are you defining filename? Can you include the full source code?

#standard hello_world/ta/hello_world_ta.c file. The changed/added code are given below

static uint8_t filename[] = “FileTest”; //Defining the file name

static TEE_Result inc_value(uint32_t param_types,
TEE_Param params[4])
{
TEE_Result res = TEE_SUCCESS;
TEE_ObjectHandle object;
char buffer[100], readbuffer[100]; //Buffer to write/read to/from the persistent object
size_t size, count;

uint32_t exp_param_types = TEE_PARAM_TYPES(TEE_PARAM_TYPE_VALUE_INOUT,
					   TEE_PARAM_TYPE_NONE,
					   TEE_PARAM_TYPE_NONE,
					   TEE_PARAM_TYPE_NONE);

DMSG("has been called");
if (param_types != exp_param_types)
	return TEE_ERROR_BAD_PARAMETERS;

DMSG("Got value: %u from NW", params[0].value.a);
params[0].value.a++;
DMSG("Increase value to: %u", params[0].value.a);

//Storage related APIs
DMSG(“Starting STORAGE TEST”);
res = TEE_OpenPersistentObject(TEE_STORAGE_PRIVATE,
filename, sizeof(filename),
TEE_DATA_FLAG_ACCESS_READ |
TEE_DATA_FLAG_ACCESS_WRITE |
TEE_DATA_FLAG_ACCESS_WRITE_META |
TEE_DATA_FLAG_OVERWRITE,
&object);
if (res != TEE_SUCCESS) {
EMSG(“Failed to open persistent object, res=0x%08x”, res);

        DMSG("Calling TEE_CreatePersistentObject");
    	res = TEE_CreatePersistentObject(TEE_STORAGE_PRIVATE,
            	        filename, sizeof(filename),
                    	TEE_DATA_FLAG_ACCESS_READ |
                        TEE_DATA_FLAG_ACCESS_WRITE |
    	                TEE_DATA_FLAG_ACCESS_WRITE_META |
            	        TEE_DATA_FLAG_OVERWRITE,
                    	NULL, NULL, 0, &object);
        if (res != TEE_SUCCESS) {
    	        EMSG("Failed to create persistent object, res=0x%08x", res);
    	}
        else {
    	        DMSG("Successfully created file %s", filename);
		TEE_CloseObject( object);
        }
    }
else
{

		DMSG("TEE_OpenPersistentObject SUCCESS file= %s", filename);
	TEE_CloseObject( object);
}

    res = TEE_OpenPersistentObject(TEE_STORAGE_PRIVATE,
                    filename, sizeof(filename),
                    TEE_DATA_FLAG_ACCESS_READ |
                    TEE_DATA_FLAG_ACCESS_WRITE |
                    TEE_DATA_FLAG_ACCESS_WRITE_META |
                    TEE_DATA_FLAG_OVERWRITE,
                    &object);
    if (res == TEE_SUCCESS) 
{
    	size = 28;
        memcpy(buffer, "Test write using write data.",28 );

        res = TEE_WriteObjectData( object, buffer, size );
    	if (res == TEE_SUCCESS){
            	DMSG("TEE_WriteObjectData SUCCESS");
        }
    	else {
            	DMSG("TEE_WriteObjectData FAILED res=%d",res);
        }
	TEE_CloseObject( object);
}
else
{
            DMSG("TEE_OpenPersistentObject for writing FAILED res=%d",res);
}

    res = TEE_OpenPersistentObject(TEE_STORAGE_PRIVATE,
                    filename, sizeof(filename),
                    TEE_DATA_FLAG_ACCESS_READ |
                    TEE_DATA_FLAG_ACCESS_WRITE |
                    TEE_DATA_FLAG_ACCESS_WRITE_META |
                    TEE_DATA_FLAG_OVERWRITE,
                    &object);
    if (res == TEE_SUCCESS) 
{
    	size = 28;
	res = TEE_ReadObjectData( object, &readbuffer, size, &count );
        if (res == TEE_SUCCESS){
    	        DMSG("TEE_ReadObjectData SUCCESS data=%s rbytes=%d:%d",readbuffer,count, size);
        }
    	else {
            	DMSG("TEE_ReadObjectData FAILED res=%d",res);
        }
	TEE_CloseObject( object);
}
else
{
            DMSG("TEE_OpenPersistentObject for reading FAILED res=%d",res);
}

DMSG("Done with STORAGE TEST");


return TEE_SUCCESS;

}

Hi @vchong,

The build failed and with the following log

[ 74% 37048/49749] target Generated: libpiglitutil_32 <= external/piglit/tests/util/gen_dispatch.py
FAILED: /bin/bash -c “python external/piglit/tests/util/gen_dispatch.py --out-dir out/target/product/hikey/gen/SHARED_LIBRARIES/libpiglitutil_intermediates”
Traceback (most recent call last):
File “external/piglit/tests/util/gen_dispatch.py”, line 31, in
import mako.runtime
ImportError: No module named mako.runtime
ninja: build stopped: subcommand failed.
make: *** [ninja_wrapper] Error 1

make failed to build some targets (06:49:00 (hh:mm:ss))

Thanks,
Rabi

Did you try starting from fresh? 2 other persons were able to build this without issues the day you reported this.

Regarding the storage issue, it seems to be failing at https://github.com/OP-TEE/optee_os/blob/master/lib/libutee/tee_api_objects.c#L415. Not sure why yet. What’s the directory where FileTest is located? In the meantime, if you:

  1. Delete everything under /data/tee
  2. Run xtest 6001. Basically this runs a TEE_CreatePersistentObject.
  3. Note the file created under /data/tee
  4. Reboot

Does the file still exist?

For the AOSP build issue
Fresh check out/repo sync 3 times in empty directory, apply all the patches and followed the instructions as mentioned in https://github.com/linaro-swg/optee_android_manifest/tree/hikey-n-4.9-240 and each and every time I got the same error as I mentioned earlier.

For the storage issue
The files are stored under /data/tee/ directory

Executed the following
1. xtest 6001
2. hello_world (custom version with Open/Create/Read Persistent Object)

root@HiKey:/ ls -la /data/tee/
total 0
drwx------ 4 root root 0 Jan 1 00:03 .
drwxr-xr-x 3 root root 0 Jan 1 00:03 …
drwx------ 2 root root 0 Jan 1 00:03 00F2AA8A5024E411ABE20002A5D5C51B
drwx------ 2 root root 0 Jan 1 00:03 A7F289B6DF8A7A479F9932E90C0AD0A2
root@HiKey:/
root@HiKey:/
root@HiKey:/
root@HiKey:/ ls -la /data/tee/00F2AA8A5024E411ABE20002A5D5C51B/
total 12
drwx------ 2 root root 0 Jan 1 00:03 .
drwx------ 4 root root 0 Jan 1 00:03 …
-rw------- 1 root root 12288 Jan 1 00:03 46696C655465737400
root@HiKey:/ ls -la /data/tee/A7F289B6DF8A7A479F9932E90C0AD0A2/
total 12
drwx------ 2 root root 0 Jan 1 00:03 .
drwx------ 4 root root 0 Jan 1 00:03 …
-rw------- 1 root root 12288 Jan 1 00:03 006E045708FB7196F02E553D02C3A692E9C3EF8AB23453E6F0749CD636E7A88E
root@HiKey:/

Now rebooted the board using
reboot

After reboot the files under /data/tee/ directory is not present
root@HiKey:/ ls -la /data/tee/
total 0
drwxr-xr-x 2 root root 0 May 22 2017 .
drwxr-xr-x 3 root root 0 May 22 2017 …
root@HiKey:/
root@HiKey:/

I just tried building again, twice, with no issues. Did you run repo init first? Sorry if it wasn’t clear but this was what I mean by starting from fresh, since it’s a whole new different repo.

For storage, I’m wondering if /data is being mounted on ram. Can you paste the output mount here? Also the contents of /etc/fstab if there’s one. Also try creating a simple text file under /data/tee, reboot and see if it disappears?

Yes ran the “repo init”. Basically I followed all the steps mentioned in the section #3. But one thing is updated later, i.e, “android-7.1.2_r11”. I was trying repo init with “android-7.1.2_r6” which was previously in the document https://github.com/linaro-swg/optee_android_manifest/tree/hikey-n-4.9-240. I will give a try with fresh repo init with “android-7.1.2_r11” and let you know.

Created a file in /data/tee/ directory and rebooted the system. The file disappears after reboot.

Details of fstab and the mount are given below.
root@HiKey:/ cat /etc/fstab

/etc/fstab

proc /proc proc defaults 0 0
none /tmp ramfs defaults 0 0
none /sys sysfs defaults 0 0
none /debug debugfs defaults 0 0
none /sys/kernel/debug debugfs defaults 0 0

none /var/log ramfs defaults 0 0
none /var/run ramfs defaults 0 0
none /var/lock ramfs defaults 0 0
none /var/tmp ramfs defaults 0 0
root@HiKey:/
root@HiKey:/
root@HiKey:/
root@HiKey:/
root@HiKey:/
root@HiKey:/
root@HiKey:/ mount
rootfs on / type rootfs (rw)
proc on /proc type proc (rw,relatime)
none on /tmp type ramfs (rw,relatime)
none on /sys type sysfs (rw,relatime)
none on /debug type debugfs (rw,relatime)
none on /sys/kernel/debug type debugfs (rw,relatime)
none on /var/log type ramfs (rw,relatime)
none on /var/run type ramfs (rw,relatime)
none on /var/lock type ramfs (rw,relatime)
none on /var/tmp type ramfs (rw,relatime)
tmpfs on /dev type tmpfs (rw,relatime,mode=755)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=000)

Hi ,
pls try below steps and compile ur stack

$ sudo apt-get install python-pip libyaml-dev
$ sudo pip install prettytable Mako pyaml dateutils --upgrade

Thanks @deepakmnvl for the tip! Hope it works.

@rabindranath-derebac On my board, there’s no /etc/fstab. And mount looks like below. Files don’t disappear after reboot.

rootfs on / type rootfs (ro,seclabel,size=933540k,nr_inodes=233385)
tmpfs on /dev type tmpfs (rw,seclabel,nosuid,relatime,mode=755)
devpts on /dev/pts type devpts (rw,seclabel,relatime,mode=600,ptmxmode=000)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,seclabel,relatime)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
none on /acct type cgroup (rw,relatime,cpuacct)
none on /dev/stune type cgroup (rw,relatime,schedtune)
tmpfs on /mnt type tmpfs (rw,seclabel,relatime,mode=755,gid=1000)
none on /config type configfs (rw,relatime)
none on /dev/memcg type cgroup (rw,relatime,memory)
none on /dev/cpuctl type cgroup (rw,relatime,cpu)
none on /dev/cpuset type cgroup (rw,relatime,cpuset,noprefix,release_agent=/sbin/cpuset_release_agent)
pstore on /sys/fs/pstore type pstore (rw,seclabel,relatime)
/sys/kernel/debug on /sys/kernel/debug type debugfs (rw,seclabel,relatime)
/dev/block/mmcblk0p9 on /system type ext4 (ro,seclabel,relatime,data=ordered)
/dev/block/mmcblk0p8 on /cache type ext4 (rw,seclabel,relatime,discard,noauto_da_alloc,data=ordered)
/dev/block/mmcblk0p10 on /data type ext4 (ro,seclabel,relatime,discard,noauto_da_alloc,data=ordered)
tmpfs on /storage type tmpfs (rw,seclabel,relatime,mode=755,gid=1000)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,seclabel,relatime)
adb on /dev/usb-ffs/adb type functionfs (rw,relatime)

Not sure if that’s a difference between the 230 and 240 build? @deepakmnvl are you’re still on 230? Do you mind trying on yours?