{GUIDE} How to Enable NVME SDD in the PCIE Port ! <<-- HERE

Products Support HiKey970
#1

I figured out how to get the NVME SDD Working Right on Hikey 970 so i figured i would share that with you.

#1 Turn on the necessary Kernel Configs.
Optional
CONFIG_PCI_DEBUG=y

Required
CONFIG_BLK_DEV_NVME=y
CONFIG_BLK_DEV_NVME_SCSI=y
CONFIG_NVME_TARGET=y
CONFIG_NVME_TARGET_LOOP=y
CONFIG_NVMEM=y
CONFIG_ACPI_PCI_SLOT=y

Might Need
CONFIG_SCSI_UFSHCD_PCI=y
CONFIG_SCSI_UFS_DWC_TC_PCI=y

If you dont have the drivers enabled you will see something like this.
It shows with lspci.
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device 2263 (r ev 03)

With the configs turned on lspci looks like this.

hikey970# lspci
00:00.0 PCI bridge: Huawei Technologies Co., Ltd. Device 3670 (rev 01)
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
02:01.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
02:04.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
02:05.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
02:07.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
02:09.0 PCI bridge: PLX Technology, Inc. PEX 8606 6 Lane, 6 Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device 2263 (rev 03)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 07)

After you enable the configs we need to create a partition on the nvme.
Before creating a partition lsblk will show.
hikey970# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 4M 0 disk
sdb 8:16 0 4M 0 disk
sdc 8:32 0 8M 0 disk
sdd 8:48 0 59.6G 0 disk
├─sdd1 8:49 0 1M 0 part
├─sdd2 8:50 0 12M 0 part
├─sdd3 8:51 0 6M 0 part
├─sdd4 8:52 0 12M 0 part
├─sdd5 8:53 0 256M 0 part
├─sdd6 8:54 0 1M 0 part
├─sdd7 8:55 0 80M 0 part
├─sdd8 8:56 0 16M 0 part
├─sdd9 8:57 0 16M 0 part
├─sdd10 8:58 0 16M 0 part
├─sdd11 8:59 0 2M 0 part
└─sdd12 8:60 0 59.1G 0 part /
mmcblk0 179:0 0 59.5G 0 disk
└─mmcblk0p1 179:1 0 59.5G 0 part
nvme0n1 259:0 0 465.8G 0 disk

To create the partition execute the following.

sudo fdisk /dev/nvme0n1

Choose “n” to create a new partition, then “p” then “1” to create a new primary partition. Just use defaults for the sector numbers. Then “w” to write the data to the disk. The lsblk command should show something like:

hikey970# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 4M 0 disk
sdb 8:16 0 4M 0 disk
sdc 8:32 0 8M 0 disk
sdd 8:48 0 59.6G 0 disk
├─sdd1 8:49 0 1M 0 part
├─sdd2 8:50 0 12M 0 part
├─sdd3 8:51 0 6M 0 part
├─sdd4 8:52 0 12M 0 part
├─sdd5 8:53 0 256M 0 part
├─sdd6 8:54 0 1M 0 part
├─sdd7 8:55 0 80M 0 part
├─sdd8 8:56 0 16M 0 part
├─sdd9 8:57 0 16M 0 part
├─sdd10 8:58 0 16M 0 part
├─sdd11 8:59 0 2M 0 part
└─sdd12 8:60 0 59.1G 0 part /
mmcblk0 179:0 0 59.5G 0 disk
└─mmcblk0p1 179:1 0 59.5G 0 part
nvme0n1 259:0 0 465.8G 0 disk
└─nvme0n1p1 259:1 0 465.8G 0 part

NEXT CREATE THE FILESYSTEM.

sudo mkfs -t ext4 /dev/nvme0n1p1

Then create a mount point somewhere convenient:

sudo mkdir /media/nvme

Then mount the new partition on that mount point:

sudo mount /dev/nvme0n1p1 /media/nvme

After mounting the terminal will show.
hikey970# sudo mount /dev/nvme0n1p1 /media/nvme
[ 740.658785] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null)

At this point, the whole thing belongs to root which may not be what’s wanted. To change the ownership to a specific user (with the partition mounted):

sudo chown -R <user>:<user> /media/nvme

To get it to mount every time, add a line to /etc/fstab:
UUID=<nvme UUID> /media/nvme ext4 defaults 0 0

To find the UUID, enter:
sudo blkid

To get some idea of performance, the hdparm command can be used. This shows what I got from my system:
sudo apt-get update
sudo apt-get install hdparm

The UFS MEMORY Performance.
sudo hdparm -tT --direct /dev/sdd12

hikey970# sudo hdparm -tT --direct /dev/sdd12

/dev/sdd12:
Timing O_DIRECT cached reads: 368 MB in 2.01 seconds = 183.38 MB/sec
Timing O_DIRECT disk reads: 610 MB in 3.00 seconds = 203.29 MB/sec

THE NVME Performance
sudo hdparm -tT --direct /dev/nvme0n1p1

hikey970# sudo hdparm -tT --direct /dev/nvme0n1p1

/dev/nvme0n1p1:
Timing O_DIRECT cached reads: 520 MB in 2.01 seconds = 259.18 MB/sec
Timing O_DIRECT disk reads: 804 MB in 3.00 seconds = 267.90 MB/sec

The NVME is actually faster than the UFS.
We can edit the grub.cfg to boot the rootfs from the NVME.

The partition on the NVME will have to be named rootfs.

I Will post more details later. I hope this helps someone.
My kernel tree has the NVME Enabled. It is for ubuntu or Debain NOT Android.

https://github.com/Bigcountry907/linux.git -b hikey970-v4.9-Debain-Working

PS the Nvme SDD i bought and have working is this.

2 Likes
#2

Figured i would add a little of my test results here.
I was able to boot the rootfs on the NVME sdd.

After you create the partition on the NVME name it to rootfs.

sudo e2label /dev/nvme0n1p1 rootfs

Then unsparse and mount the desired system image on the hikey.
simg2img rootfs.sparse.img rootfs.raw.img

Then copy the rootfs from the mounted image to the NVME.
cp -arv /mounted-raw-image/* /media/nvme/

After that boot into ubuntu on the UFS not the NVME.
Run 
sudo fsck /dev/nvme0n1p1

If you disconnect the power or you do not shut down properly you will need to do that.

boot into ubuntu on the UFS not the NVME.
Run
sudo fsck /dev/nvme0n1p1

Other than that all you need to do is add a entry like this to the grub.cfg

menuentry ‘HiKey970 Ubuntu Bionic 18.04 NVME BOOT’ {
search.fs_label rootfs root
set root=($root)
linux /boot/Image-hikey970-v4.9.gz earlycon=pl011,0xfff32000,115200 clk_ignore_unused=true console=tty1 console=ttyAMA6,115200n8 root=/dev/nvme0n1p1 rootwait rw quiet efi=noruntime drm.debug=0xf drm_kms_helper.edid_firmware=edid/1920x1080.bin video=HDMI-A-1:1920x1080@60e consoleblank=0
devicetree /boot/kirin970-hikey970.dtb
}

If you run into trouble boot back into the system using the UFS.
Then run the sudo fsck /dev/nvme0n1p1
Then reboot and the NVME will load correctly.

You can also boot from the sdcard the same way. Set up the sdcard just like the nvme.
Create a partition and name it rootfs.
Copy a rootfs image to the sd card.
Edit grub.cfg.

A great way to run a multiboot system especially for testing builds.

#3

You never had any troubles running this stable? Which kernel version are you using?