Hi all,
We are trying to install Mali G71 drivers, for openCL support.
Currently we have successfully flashed Linaro latest image (based on 4.19.5) using Mani instruction with serial\ssh communication.
we also downloaded G71 drivers from ARM website: mali-G71_r9p0-01rel0_linux_1fbdev.tar.gz
We would appreciate help for next steps.
I keep some notes before @sumit.garg shared with me:
# Add the libmali.so to the OpenCL vendors
$ mkdir -p /etc/OpenCL/vendors/
$ echo "libmali.so" > /etc/OpenCL/vendors/mali.icd
# Install OpenCL client using command:
$ apt install clinfo
# Run OpenCL client tool:
$ clinfo
thanks.
clinfo output:
“Number of platforms 0”
I assume clinfo didn’t find any supported cl HW
maybe i wasn’t clear…we downloaded the drivers to host machine, but not to hikey960 because we d’ont know how to install them (and this is why i started this post)
@Moshe_Eliyahu Following are some instructions that I use to install out-of-tree mali G71 kernel driver and user-space libraries for OpenCL on Debian/hikey960:
1. Update device tree with GPU node [1] via GRUB entry update:
-> scp device tree image to “/boot/” location on hikey960 board.
2. Do reboot and press “e” to edit commands when you see GRUB menu during booting and add following command:
`devicetree /boot/hi3660-hikey960.dtb`
3. Install Linux source on Debian to build modules:
$ apt-get update
$ apt-get install linux-headers-$(uname -r)
4. Download mali bitfrost latest r16 kernel driver from here [2].
5. Build and install steps for mali bitfrost driver:
$ tar -xzf BX301A01B-SW-99002-r16p0-01rel0.tgz
$ make -C /usr/src/linux-headers-4.19.5-hikey/ M=BX301A01B-SW-99002-r16p0-01rel0/driver/product/kernel/drivers/gpu/arm/midgard/ CONFIG_MALI_MIDGARD=m CONFIG_MALI_GATOR_SUPPORT=y CONFIG_MALI_MIDGARD_DVFS=y CONFIG_MALI_EXPERT=y CONFIG_MALI_PLATFORM_FAKE=y CONFIG_MALI_PLATFORM_THIRDPARTY=y CONFIG_MALI_PLATFORM_THIRDPARTY_NAME=devicetree modules
$ insmod BX301A01B-SW-99002-r16p0-01rel0/driver/product/kernel/drivers/gpu/arm/midgard/mali_kbase.ko
6. Install and run ARM Mali GPU user-space libraries and OpenCL tool:
# Download, extract and put the mali library in the correct position
$ wget https://developer.arm.com/-/media/Files/downloads/mali-drivers/user-space/HiKey%20960/mali-G71_r9p0-01rel0_linux_1fbdev.tar.gz
$ tar xvf mali-G71_r9p0-01rel0_linux_1fbdev.tar.gz
$ cp fbdev/libmali.so /usr/lib/aarch64-linux-gnu/
$ cp fbdev/liboffline_compiler_api_gles.so /usr/lib/aarch64-linux-gnu/
# Add the libmali.so to the OpenCL vendors
$ mkdir -p /etc/OpenCL/vendors/
$ echo "libmali.so" > /etc/OpenCL/vendors/mali.icd
# Install OpenCL client using command:
$ apt install clinfo
# Run OpenCL client tool:
$ clinfo
Hopefully, the first two steps you won’t have to repeat by next edk2/UEFi image update as we are trying to add GPU node in device-tree by default.
[1] https://github.com/96boards-hikey/linux/commit/65f19abf8b64cd4a9ad7b3da723b5c84cc07163b
[2] https://developer.arm.com/products/software/mali-drivers/bifrost-kernel
Hi, could you please elaborate on step 1? How do I find and edit the device tree? many thanks
@Omer_Schleifer You can simply edit device tree (arch/arm64/boot/dts/hisilicon/hi3660-hikey960.dts) in Linux repo [1]. Then build it using following instructions:
$ make defconfig ARCH=arm64
$ make dtbs ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
Output device tree blob: arch/arm64/boot/dts/hisilicon/hi3660-hikey960.dtb
BTW, now you don’t need to edit device tree as @Mani had already applied the GPU node patch to Linux tree [2]. So you could just build and update device tree.
[1] manivannan.sadhasivam/hikey.git - [no description]
[2] manivannan.sadhasivam/hikey.git - [no description]
FYi…
We have added a guide [1] in documentation regarding “OpenCL support on Debian/hikey960”.
Please feel free to give it a try and let us know your feedback.
thanks @sumit.garg, i was able to solve all the issues using your posts.
I have quotation:
Currently i have to manually change device tree in grub every boot
after boot i have to manually load the mali_kbase (insmod …mali_kbase.ko)
How can do this automatically?
Great!
For device tree, you may need to wait for next UEFI/edk2 image incorporating PR [1] or else you can build UEFI/edk2 image (part of fip.bin) from source including PR [1] and following this doc [2].
For auto-loading of mali kernel driver, use following steps:
$ sudo mkdir -p /lib/modules/$(uname -r)/extras
$ sudo cp BX301A01B-SW-99002-r16p0-01rel0/driver/product/kernel/drivers/gpu/arm/midgard/mali_kbase.ko /lib/modules/$(uname -r)/extras/
$ sudo depmod -a
Then just reboot your hikey960 with devicetree (mali gpu node enabled) updated.
[1] https://github.com/96boards-hikey/OpenPlatformPkg/pull/131
[2] https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/plat/hikey960.rst
AFAIK with current module build options, DEFAULT_GPU_FREQ_KHZ_MAX defines the max GPU frequency as 5MHz. So you could modify the GPU frequency via defining GPU_FREQ_KHZ_MAX in platform config file [1].
I would suggest you to not define it more than max operating frequency (1.037 GHz) as specified in device-tree gpu node [2].
Also with current module build options, devfreq infrastructure isn’t enabled for GPU device which would configure maximum frequency automatically based on highest operating point found from device tree gpu node [2].
[1] BX301A01B-SW-99002-r16p0-01rel0/driver/product/kernel/drivers/gpu/arm/midgard/platform/devicetree/mali_kbase_config_platform.h
[2] https://github.com/96boards-hikey/linux/commit/65f19abf8b64cd4a9ad7b3da723b5c84cc07163b
Thanks for your instructions @sumit.garg! I’ve just updated my dusty HiKey960 using boot-linaro-stretch-developer-hikey-20190620-32.img.gz from http://snapshots.linaro.org/96boards/hikey/linaro/debian/latest/ and it now works!
I found it a bit weird that https://github.com/96boards-hikey/linux/commit/65f19abf8b64cd4a9ad7b3da723b5c84cc07163b mentions "arm,malit6xx", "arm,mali-midgard", not something like "arm,malig7x","arm,mali-bifrost". Ah, well, it works regardless.
Updating the device tree wasn’t terribly well documented, unfortunately. I’d suggest to add to OpenCL support on Debian/hikey960 - 96Boards the following section:
# cd /root
# git clone https://git.linaro.org/people/manivannan.sadhasivam/hikey.git/
# apt install bison flex libssl-dev -y
# make -j4 defconfig ARCH=arm64
# make -j4 dtbs ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
# find . -name hi3660-hikey960.dtb
./arch/arm64/boot/dts/hisilicon/hi3660-hikey960.dtb
# cp ./arch/arm64/boot/dts/hisilicon/hi3660-hikey960.dtb /boot
Finally, it appears that it is still required to change GRUB on every boot? Would it be possible to update only fip.bin? If so, could you please provide minimal instructions to do so?
Finally, it appears that it is still required to change GRUB on every boot? Would it be possible to update only
fip.bin? If so, could you please provide minimal instructions to do so?
DT should already be updated in fip.bin image. Which snapshot are you using for fip.bin image?
Hi @sumit.garg!
I was following instructions on Linux Host Installation for HiKey960 - 96Boards which led me to Debian Downloads for HiKey960 - 96Boards. Therefore, I used http://snapshots.linaro.org/96boards/reference-platform/components/uefi-staging/latest/hikey960/debug/fip.bin
Should I try http://snapshots.linaro.org/96boards/reference-platform/components/uefi-staging/latest/hikey960/release/fip.bin?
@sumit.garg Two further questions, if I may.
anton@hikey961:~/CK_REPOS/ck-env/platform.init/hikey970-linux$ ./ck-print-temperature
*** Temperature:
[zone 0] 35030
*** Temperature - trip point 0 <passive>:
[zone 0] 65000
*** Temperature - trip point 1 <passive>:
[zone 0] 75000
*** Policy:
[zone 0] step_wise
*** Available policies:
[zone 0] power_allocator step_wise
*** Sustainable power:
[zone 0] 4500
anton@hikey961:~/CK_REPOS/ck-env/platform.init/hikey970-linux$ ./ck-print-cpu-freq
*** Current CPU frequency from scaling:
cat: '/sys/devices/system/cpu/cpu*/cpufreq/scaling_cur_freq': No such file or directory
*** Min CPU frequency:
cat: '/sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_min_freq': No such file or directory
*** Max CPU frequency:
cat: '/sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_max_freq': No such file or directory
*** Available CPU frequencies:
cat: '/sys/devices/system/cpu/cpu*/cpufreq/scaling_available_frequencies': No such file or directory
*** Current CPU governor:
cat: '/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor': No such file or directory
*** Available CPU governors:
cat: '/sys/devices/system/cpu/cpu*/cpufreq/scaling_available_governors': No such file or directory
anton@hikey961:~/CK_REPOS/ck-env/platform.init/hikey970-linux$ ./ck-print-gpu-freq
*** Available GPU frequencies:
cat: /sys/devices/platform/e82c0000.mali/devfreq/e82c0000.mali/available_frequencies: No such file or directory
*** Current GPU frequency:
cat: /sys/devices/platform/e82c0000.mali/devfreq/e82c0000.mali/cur_freq: No such file or directory
cat: /sys/devices/platform/e82c0000.mali/devfreq/e82c0000.mali/available_frequencies: No such file or directory
*** Min GPU frequency:
*** Max GPU frequency:
./ck-print-gpu-freq: line 26: all_freqs: bad array subscript
*** Available GPU governors:
cat: /sys/devices/platform/e82c0000.mali/devfreq/e82c0000.mali/available_governors: No such file or directory
*** Current GPU governor:
cat: /sys/devices/platform/e82c0000.mali/devfreq/e82c0000.mali/governor: No such file or directory
From your answer, I understand that devfreq is not enabled for the GPU, but I didn’t expect any issues for the CPU.
#define GPU_FREQ_KHZ_MAX 807000
into BX301A01B-SW-99002-r16p001rel0/driver/product/kernel/drivers/gpu/arm/midgard/platform/devicetree/mali_kbase_config_platform.h and rebuild/reinstall?
Many thanks!
It seems like there hasn’t been any snapshot after 20th dec. which doesn’t include DT sync fix patch from @Mani [1]. So I would suggest you to build fip.bin from source as per instructions here [2].
[1] https://github.com/96boards-hikey/OpenPlatformPkg/commit/f45d971acd7f5365305e5a07c4eadeff607cb16d
[2] https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/plat/hikey960.rst
New snapshots are here: http://snapshots.linaro.org/reference-platform/components/uefi-staging/latest/hikey960/release/
Sorry I can’t help with BSP release from ARM. For that I think you need to seek help from ARM support [1].
In case you are testing this on Debian then following post may help regarding CPU freq.
Correct.
Many thank @Mani! I assume I can simply download fip.bin, l-loader.bin and prm_ptable.img, and flash them again?
@sumit.garg I’m not seeking any help with the Arm-provided BSP 
. I’m just used to being able to set up CPU and GPU frequencies in this way on boards like HiKey960 and RK3399.
From your link, I assume that’s expected to work on the Linaro-provided BSP too. Unfortunately, the cpufreq directory is empty:
$ sudo ls -la /sys/devices/system/cpu/cpufreq/
total 0
drwxr-xr-x 2 root root 0 Jun 27 23:39 .
drwxr-xr-x 14 root root 0 Jun 27 23:39 ..
while Individual core directories do not exist at all:
$ sudo ls /sys/devices/system/cpu/cpu0/cpufreq
ls: cannot access '/sys/devices/system/cpu/cpu0/cpufreq': No such file or directory
Could this issue perhaps also be solved by updating the device tree to the latest release?
