Booting Hikey970 with f2fs data partition

Hello,

I am using the following branch to build the images for hikey970.

https://android.googlesource.com/platform/manifest/+/pie-dr1-dev

In order to boot the userdata partition using f2fs, I carried out the following steps:

1. Enabled the f2fs configs in the kernel

CONFIG_F2FS_FS=y
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
CONFIG_F2FS_FS_SECURITY=y
CONFIG_F2FS_CHECK_FS=y
CONFIG_F2FS_FS_ENCRYPTION=y
CONFIG_F2FS_IO_TRACE=y
CONFIG_F2FS_FAULT_INJECTION=y

2. Modified the default fstab to the following

# Android fstab file.
#<src>                                                  <mnt_point>         <type>    <mnt_flags and options>                               <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

#/dev/block/platform/soc/ff3b0000.ufs/by-name/system_a    /system    ext4    ro                                                                   wait
/dev/block/sdd12    /system    ext4    ro                                                                   wait
#/dev/block/platform/soc/ff3b0000.ufs/by-name/cache       /cache     ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
/dev/block/sdd5     /cache     ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
#/dev/block/platform/soc/ff3b0000.ufs/by-name/userdata    /data      ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
#/dev/block/sdd15    /data      ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
/dev/block/sdd15    /data      f2fs    discard,noatime,nosuid,nodev                                 wait,check,fileencryption=software,quota
#/dev/block/sdd13    /data      f2fs    discard,noatime,nosuid,nodev                                 wait,check,fileencryption=software,quota
#/dev/block/sdd15    /data      ext4    discard,noatime,noauto_da_alloc,data=ordered,user_xattr,barrier=1    wait,formattable
/devices/platform/soc/ff37f000.dwmmc1/mmc_host/mmc*      auto       auto    defaults    voldmanaged=sdcard1:auto,encryptable=userdata
/devices/platform/soc/soc:hisi_dwc3/ff100000.dwc3/xhci-hcd.*.auto/usb*               auto               auto      defaults  voldmanaged=usbdisk:auto,encryptable=userdata

3. Built the AOSP using the following command

make -j$(nproc) BOARD_USERDATAIMAGE_FILE_SYSTEM_TYPE=f2fs TARGET_USERIMAGES_USE_F2FS=true

It is observed that after booting, the /data partition is not mounted. But It can be mounted manually using the mount command after booting:

hikey970:/data # root@nand:/mnt/storage2/nishad/f2fs/Hikey_Dir/out/target/product/hikey960# adb shell
hikey970:/ $ su                                                                                                                                         
hikey970:/ # df
Filesystem       1K-blocks    Used Available Use% Mounted on
rootfs             2979360    4028   2975332   1% /
tmpfs              3016084     296   3015788   1% /dev
tmpfs              3016084       0   3016084   0% /mnt
/dev/block/sdd12   4725116 1267380   3457736  27% /system
/dev/block/sdd5     237536     176    237360   1% /cache
hikey970:/ # mount -t f2fs /dev/block/sdd                                                                                                               
sdd    sdd1   sdd10  sdd11  sdd12  sdd13  sdd14  sdd15  sdd2   sdd3   sdd4   sdd5   sdd6   sdd7   sdd8   sdd9
hikey970:/ # mount -t f2fs /dev/block/sdd15 data/                                                                                                       
hikey970:/ # df -h
Filesystem       Size  Used Avail Use% Mounted on
rootfs           2.8G  3.9M  2.8G   1% /
tmpfs            2.8G  296K  2.8G   1% /dev
tmpfs            2.8G     0  2.8G   0% /mnt
/dev/block/sdd12 4.5G  1.2G  3.2G  27% /system
/dev/block/sdd5  232M  176K  232M   1% /cache
/dev/block/sdd15  24G  165M   24G   1% /data
hikey970:/ # cd data/                                                                                                                                   
hikey970:/data # mkdir 1234
hikey970:/data # ls
1234 nativetest nativetest64 
hikey970:/data # 

Why does the f2fs partition not mount while board boot-up?

Thank you for your time,

Regards,
Nishad

Any related error in dmesg or/and logcat?

Hi,

Thanks for getting back.

I did not see any error in particular related to sdd15.
It seems like the mount command for sdd15 isn’t getting executed. While it does get executed for sdd12 and sdd5:

Here is the output of the mount command:

hikey970:/ $ su                                                                                                                                         
hikey970:/ # df -h
Filesystem       Size  Used Avail Use% Mounted on
rootfs           2.8G  3.9M  2.8G   1% /
tmpfs            2.8G  296K  2.8G   1% /dev
tmpfs            2.8G     0  2.8G   0% /mnt
/dev/block/sdd12 4.5G  1.2G  3.2G  27% /system
/dev/block/sdd5  232M  176K  232M   1% /cache
hikey970:/ # mount 
rootfs on / type rootfs (ro,seclabel,size=2979360k,nr_inodes=744840)
tmpfs on /dev type tmpfs (rw,seclabel,nosuid,relatime,mode=755)
devpts on /dev/pts type devpts (rw,seclabel,relatime,mode=600,ptmxmode=000)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,seclabel,relatime)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
tmpfs on /mnt type tmpfs (rw,seclabel,nosuid,nodev,noexec,relatime,mode=755,gid=1000)
none on /acct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct)
none on /dev/memcg type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
none on /dev/stune type cgroup (rw,nosuid,nodev,noexec,relatime,schedtune)
none on /config type configfs (rw,nosuid,nodev,noexec,relatime)
none on /dev/cpuctl type cgroup (rw,nosuid,nodev,noexec,relatime,cpu)
none on /dev/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset,noprefix,release_agent=/sbin/cpuset_release_agent)
cg2_bpf on /dev/cg2_bpf type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,seclabel,nosuid,nodev,noexec,relatime)
/sys/kernel/debug on /sys/kernel/debug type debugfs (rw,seclabel,relatime,mode=755)
/dev/block/sdd12 on /system type ext4 (ro,seclabel,relatime,block_validity,delalloc,barrier,user_xattr,acl)
/dev/block/sdd5 on /cache type ext4 (rw,seclabel,relatime,discard,noauto_da_alloc,data=ordered)
tmpfs on /storage type tmpfs (rw,seclabel,nosuid,nodev,noexec,relatime,mode=755,gid=1000)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,seclabel,relatime)
adb on /dev/usb-ffs/adb type functionfs (rw,relatime)
mtp on /dev/usb-ffs/mtp type functionfs (rw,relatime)

Here are some of the snippets of the boot log:

[    6.004391] ueventd: Parsing file /ueventd.rc...
[    6.009897] ueventd: Parsing file /vendor/ueventd.rc...
[    6.015186] ueventd: Unable to read config file '/vendor/ueventd.rc': open() failed: No such file or directory
[    6.025305] ueventd: Parsing file /odm/ueventd.rc...
[    6.030315] ueventd: Unable to read config file '/odm/ueventd.rc': open() failed: No such file or directory
[    6.040221] ueventd: Parsing file /ueventd.hikey970.rc...
[    6.046171] ueventd: [libfs_mgr]fs_mgr_read_fstab_dt(): failed to read fstab from dt
[    6.160739] ueventd: Coldboot took 0.106 seconds
[    6.184736] Registered swp emulation handler
[    6.190131] audit: type=1400 audit(30.963:3): avc:  denied  { write } for  pid=1 comm="init" name="enabled" dev="sysfs" ino=7799 scontext=u:r:init:s0 
tcontext=u:object_r:sysfs:s0 tclass=file permissive=1
[    6.208335] audit: type=1400 audit(30.983:4): avc:  denied  { open } for  pid=1 comm="init" path="/sys/kernel/mm/transparent_hugepage/enabled" dev="sy
sfs" ino=7799 scontext=u:r:init:s0 tcontext=u:object_r:sysfs:s0 tclass=file permissive=1
[    6.236780] watchdogd: watchdogd started (interval 10, margin 20)!
[    6.236997] EXT4-fs (sdd12): mounted filesystem without journal. Opts: (null)
[    6.243497] EXT4-fs (sdd5): Ignoring removed nomblk_io_submit option
[    6.250947] EXT4-fs (sdd5): recovery complete
[    6.250959] EXT4-fs (sdd5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,nomblk_io_submit
[    6.270982] watchdogd: Failed to open /dev/watchdog: No such file or directory
[    6.311555] [drm]  +.
[    6.313836] [drm] dss_pwm_clk clk_enable successed, ret=0!
[    6.314719] audit: type=1400 audit(31.087:5): avc:  denied  { read } for  pid=2463 comm="e2fsck" name="sdd5" dev="tmpfs" ino=9642 scontext=u:r:fsck:s0
 tcontext=u:object_r:block_device:s0 tclass=blk_file permissive=1
[    6.314738] audit: type=1400 audit(31.087:6): avc:  denied  { open } for  pid=2463 comm="e2fsck" path="/dev/block/sdd5" dev="tmpfs" ino=9642 scontext=
u:r:fsck:s0 tcontext=u:object_r:block_device:s0 tclass=blk_file permissive=1
[    6.314850] audit: type=1400 audit(31.087:7): avc:  denied  { write } for  pid=2463 comm="e2fsck" name="sdd5" dev="tmpfs" ino=9642 scontext=u:r:fsck:s
0 tcontext=u:object_r:block_device:s0 tclass=blk_file permissive=1
[    6.314889] audit: type=1400 audit(31.087:8): avc:  denied  { ioctl } for  pid=2463 comm="e2fsck" path="/dev/block/sdd5" dev="tmpfs" ino=9642 ioctlcmd
=0x127c scontext=u:r:fsck:s0 tcontext=u:object_r:block_device:s0 tclass=blk_file permissive=1
[    6.339094] e2fsck: e2fsck 1.43.3 (04-Sep-2016)
[    6.339094] 
[    6.339129] e2fsck: Pass 1: Checking inodes, blocks, and sizes
[    6.339129] 
[    6.339137] e2fsck: Pass 2: Checking directory structure
[    6.339137] 
[    6.339145] e2fsck: Pass 3: Checking directory connectivity
[    6.339145] 
[    6.339152] e2fsck: Pass 4: Checking reference counts
[    6.339152] 
[    6.339158] e2fsck: Pass 5: Checking group summary information
[    6.339158] 
[    6.339166] e2fsck: cache: 14/65536 files (0.0% non-contiguous), 6196/65536 blocks
[    6.339166] 
[    6.403713] EXT4-fs (sdd5): mounted filesystem with ordered data mode. Opts: discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1
[    6.423494] audit: type=1400 audit(31.195:9): avc:  denied  { setattr } for  pid=1 comm="init" name="slabinfo" dev="proc" ino=4026532068 scontext=u:r:
init:s0 tcontext=u:object_r:proc:s0 tclass=file permissive=1
[    6.424563] audit: type=1400 audit(31.195:10): avc:  denied  { write } for  pid=1 comm="init" name="/" dev="debugfs" ino=1 scontext=u:r:init:s0 tconte
xt=u:object_r:debugfs:s0 tclass=dir permissive=1
[    6.434409] logd.auditd: start
[    6.434416] logd.klogd: 6274894270
[    6.632028] vdc: Waited 170ms for vold
[    6.636238] vdc: Failed: Status(-8): '0: '
[    6.651560] (stk) : timed out waiting for ldisc to be un-installed[    6.671873] logd.daemon: reinit
[    6.675220] logd: Cannot read /data/misc/logd/event-log-tags
[    6.699802] type=1400 audit(31.475:13): avc: denied { write } for pid=56 comm="kdevtmpfs" name="/" dev="devtmpfs" ino=1025 scontext=u:r:kernel:s0 tcon
text=u:object_r:device:s0 tclass=dir permissive=1
[    6.700629] file system registered
[    6.709665] healthd: No battery devices found
[    6.710186] healthd: battery none chg=
[    6.711548] [drm] bl_level is 255
[    6.711551] [drm] bl_level=255.
[    6.736508] type=1400 audit(31.475:13): avc: denied { write } for pid=56 comm="kdevtmpfs" name="/" dev="devtmpfs" ino=1025 scontext=u:r:kernel:s0 tcon
text=u:object_r:device:s0 tclass=dir permissive=1
[    6.747853] read descriptors
[    6.747865] read strings
[    6.748915] udc-core: couldn't find an available UDC - added [g1] to list of pending drivers
[    6.768145] (stk) :ldisc_install = 1
[    6.768145] type=1400 audit(31.475:14): avc: denied { mknod } for pid=56 comm="kdevtmpfs" capability=27 scontext=u:r:kernel:s0 tcontext=u:r:kernel:s0 
tclass=capability permissive=1
[    6.768254] type=1400 audit(31.475:14): avc: denied { mknod } for pid=56 comm="kdevtmpfs" capability=27 scontext=u:r:kernel:s0 tcontext=u:r:kernel:s0 
tclass=capability permissive=1
[    6.768259] type=1400 audit(31.475:15): avc: denied { add_name } for pid=56 comm="kdevtmpfs" name="usb_accessory" scontext=u:r:kernel:s0 tcontext=u:ob
ject_r:device:s0 tclass=dir permissive=1
[    6.768306] type=1400 audit(31.475:15): avc: denied { add_name } for pid=56 comm="kdevtmpfs" name="usb_accessory" scontext=u:r:kernel:s0 tcontext=u:ob
ject_r:device:s0 tclass=dir permissive=1
console:/ $ [    6.887564] [drm] addr = 0x251316788445184, offset = 0x0, size = 18432000!
[    6.894448] [drm] fbdev->smem_start = 0x262144, fbdev->screen_base = 0xffff00000d1c1000

Thank you for your time.

Regards,
Nishad

Did you check that the onboard fstab file corresponds to your custom one?

Hi,

You are right. The onboard fstab file does not correspond to the one I changed.

Here is the onboard fstab file:

hikey970:/ # ls                                                                                                                                         
acct       data            init.hikey970.power.rc init.zygote64_32.rc     plat_seapp_contexts   sepolicy            vendor_file_contexts      
bin        default.prop    init.hikey970.rc       mnt                     plat_service_contexts storage             vendor_hwservice_contexts 
bugreports dev             init.hikey970.usb.rc   odm                     proc                  sys                 vendor_property_contexts  
cache      etc             init.rc                oem                     product               system              vendor_seapp_contexts     
charger    fstab.hikey970  init.usb.configfs.rc   plat_file_contexts      root                  ueventd.hikey970.rc vendor_service_contexts   
config     init            init.usb.rc            plat_hwservice_contexts sbin                  ueventd.rc          vndservice_contexts       
d          init.environ.rc init.zygote32.rc       plat_property_contexts  sdcard                vendor              
hikey970:/ # vim fstab.hikey970                                                                                                                         
/system/bin/sh: vim: not found
127|hikey970:/ # cat fstab.hikey970                                                                                                                     
# Android fstab file.
#<src>                                                  <mnt_point>         <type>    <mnt_flags and options>                               <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

#/dev/block/platform/soc/ff3b0000.ufs/by-name/system_a    /system    ext4    ro                                                                   wait
/dev/block/sdd12    /system    ext4    ro                                                                   wait
#/dev/block/platform/soc/ff3b0000.ufs/by-name/cache       /cache     ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
/dev/block/sdd5     /cache     ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
#/dev/block/platform/soc/ff3b0000.ufs/by-name/userdata    /data      ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
/dev/block/sdd15    /data      ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
/devices/platform/soc/ff37f000.dwmmc1/mmc_host/mmc*      auto       auto    defaults    voldmanaged=sdcard1:auto,encryptable=userdata
/devices/platform/soc/soc:hisi_dwc3/ff100000.dwc3/xhci-hcd.*.auto/usb*               auto               auto      defaults  voldmanaged=usbdisk:auto,encryptable=userdata
hikey970:/ # sudo umount data/                                                                                                                          
/system/bin/sh: sudo: not found
127|hikey970:/ # su                                                                                                                                     
hikey970:/ # 

Can you tell me what I need to do to update it?

Thank you for your time.

Regards,
Nishad

Hi,

Thanks a lot for the pointer!

I did not know that fstab was part of the ramdisk image.
Hence I did not run the build_kernel.sh script using the newly built ramdisk image to build the boot.img.

I used the following procedure to build the images after which the changes made in fstab were visible in the onboard fstab file and the f2fs partition now mounts automatically during boot-up.

  1. Modify the fstab.hikey970 with the required changes.
  2. Build the ramdisk.img
  3. Use the newly built ramdisk.img to build the boot.img
  4. Use the newly built Image.gz and hikey970 device tree to build the system, cache and userdata images.
  5. Flash the images.

Following are the contents of the on-board fstab file after the obove steps were carried out:

hikey970:/ # root@nand:/mnt/storage2/nishad/f2fs/Hikey_Dir/out/target/product/hikey960# adb shell
hikey970:/ $ su
hikey970:/ # df
Filesystem       1K-blocks    Used Available Use% Mounted on
rootfs             2979360    4028   2975332   1% /
tmpfs              3016084     300   3015784   1% /dev
tmpfs              3016084       0   3016084   0% /mnt
/dev/block/sdd12   4725116 1267380   3457736  27% /system
/dev/block/sdd5     237536     176    237360   1% /cache
/dev/block/sdd15  25163776  218016  24945760   1% /data
/data/media       25163776  218016  24945760   1% /mnt/runtime/default/emulated
hikey970:/ # cat fstab.hikey970                                                                                                                         
# Android fstab file.
#<src>                                                  <mnt_point>         <type>    <mnt_flags and options>                               <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK

#/dev/block/platform/soc/ff3b0000.ufs/by-name/system_a    /system    ext4    ro                                                                   wait
/dev/block/sdd12    /system    ext4    ro                                                                   wait
#/dev/block/platform/soc/ff3b0000.ufs/by-name/cache       /cache     ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
/dev/block/sdd5     /cache     ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
#/dev/block/platform/soc/ff3b0000.ufs/by-name/userdata    /data      ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
#/dev/block/sdd15    /data      ext4    discard,noauto_da_alloc,data=ordered,user_xattr,discard,barrier=1    wait
/dev/block/sdd15    /data      f2fs    discard,noatime,nosuid,nodev                                 wait,check,fileencryption=software,quota
/dev/block/sdd15    /data      ext4    discard,noatime,noauto_da_alloc,data=ordered,user_xattr,barrier=1    wait,formattable
/devices/platform/soc/ff37f000.dwmmc1/mmc_host/mmc*      auto       auto    defaults    voldmanaged=sdcard1:auto,encryptable=userdata
/devices/platform/soc/soc:hisi_dwc3/ff100000.dwc3/xhci-hcd.*.auto/usb*               auto               auto      defaults  voldmanaged=usbdisk:auto,encryptable=userdata
hikey970:/ # You have new mail in /var/mail/root
root@nand:/mnt/storage2/nishad/f2fs/Hikey_Dir/out/target/product/hikey960#

Can you please confirm whether this is the right way to do it ?

Thank you for your time.

Regards,
Nishad

It looks good, afaik since fstab is part of the boot image, you can run make bootimg to only rebuild/generate this image. Also, you just have to flash the bootimg (fastboot flash boot ...) not all partitions.

Ok, thanks.

regards,
Nishad