Anybody actually got Linux up on Hikey 960?

I have followed the instructions on how to get Linaro’s Debian to boot on my new Hikey 960. There are plenty of dead links, references to undefined concepts and file names, and contradictory instructions. I have now made around 30 attempts, but to no avail. I am very far from a newbie, and I am perfectly able to read coherent instructions (preferably ones with some context of what the commands intend to achieve, as this allows us engineers to do some less blind trial and error).

I notice from searching these forums and from googling that I am very far from alone. And I still have not seen anybody who says that they have succeeded!

My question is: Is there a working Linux port to this hardware, and if so, are there coherent and tested instructions for how to install this Linux port?

How do my attempts fail? I get no HDMI output. Not even a signal which the monitor detects. I’ve tried several differently spec’d monitors. I actually don’t even get any flashing LEDs since my initial AOSP boot.

(What have I tried. I tried flashing several version of the firmware. Most appeared to succeed, one attempt hung midway through. I have tried with Linaro’s .sd image on an SD card, and this also gave no HDMI output, but this actually flash the green LEDs and after a while lit up the blue one. Some progress…)

@tege First of all you need to have 96boards UART mezzanine to access serial console as current Debian kernel doesn’t have support for HDMI.

I am not sure which links or references you are pointing at. Standard Debian install instructions are here.

To enable HDMI support, you could refer to following thread:

Thanks for your reply!

I got the serial console running with some help and with educated guessed about the wiring. (The connectors of the mezzanine card and the board connectors don’t fit, nor does the included adapter, due to different pin distance!) And thereby I got a clean boot. But only with SD-card.

The current installation instructions are actively harmful as they forget key things and are misleading. The lack of docs for the mezzanine card makes troubleshooting harder.

At a bare minimum, the install instructions should make it clear that HDMI does not work. Instead, they urge users to connect a monitor, implying that it works.

The install instructions should also make the need for messing with the mezzanine card clear. It wouldn’t hurt to mention how to wire it too, as that is not documented (as it should!) in the board hardware manual.

Furthermore, clarification/cleanup of the UEFI firmware stuff is badly needed. Now one might update with a non-UEFI firmware as old scripts for that are still tempting unsuspecting users. All-in-all, the firmware dist is in a complete disarray.

And last but not least, if the only boot that actually works is from the SD card slot, the documentation should not lead users astray with instructions on how to flash the built-in flash storage.

I decided to return my board as I cannot use hardware with such poor documentation and such poor software support. The final straw was that WiFi does not work with the provided images, and that the two USB-A ports stop working as the system becomes warm. I.e., there is no way to get networking on this system in a reasonable time.

The HiKey960 is the the worst SBC I have ever come across. It is also the most expensive one I have purchased in a very long time.

Documentation for 96boards UART mezzanine is here.

Agree, I have created documentation bug for this.

I am not sure which non-UEFI firmware you are referring to. AFAIK, currently hikey960 uses UEFI to boot up Linux.

Booting from flash works for me using standard Debian install instructions.

Wifi also does works well for me using this guide: Setting up a WIFI connection via command line on Debian/Ubuntu - 96Boards

BTW, its good to share the details of issue you are facing rather than making assumptions.

The mezzanine card documentation you link is for a different product, sold separately. I used the one that comes with the Hikey960 which requires completely manual wiring. I got it to work in the end with some experimentation (and private help from an ex Linaro employee).

And yes, I did locate that aftermarket board and how to connect that. Documentation for the Hikey960 as sold today is what I was interested in.

About firmware. There are several links to different sets. E.g., https://github.com/96boards-hikey/tools-images-hikey960. Here there are 2 scripts and a subdirectory with a tempting name. I tried first recovery-flash.sh which appeared to work, then recovery-flash-uefi.sh (but the version of that script I got had hardwired and obviously incorrect paths, so I modified it). Still, it never ran to completion. I also tried issuing the commands manually looking at the scripts and following instructions scattered in the forums and on the web site.

Then there is the subdirectory install-prebuilt-uefi with a README and another script. I never tried that script. It is fragile documentation from my reading; if one god forbid connects things in a different order, ttyUSB0 and ttyUSB1 will get swapped, and things will become interesting!

Since I couldn’t find proper documentation of the process or the many files one flashed, I cannot intelligently debug the process. “Issue these commands” is not good in my worlds, I want “issue these command which have these intended effects” is what I need!

About WiFi. I did not locate that specific web page. I did find other pages (e.g. on the forum). The instruction I did locate did not lead to working interfaces (but magically disabled the USB-A ports!). There are claims in the forum that WiFi does not work yet for the Hikey 960 but just for the Hikey. There are some “try this” and “try that”, and somebody then claims that it works without explaining what he did to make it work.

There were also a bunch of unbelievable hardware issues with this board, contributing to the uselessness. (1) The bundled heatsink was about 15mm too high to fit in the bundled case. (2) The USB-A ports stopped working when when the board got warm during my WiFI struggles (I saw somewhere on he forum that others experience that too). (3) The adapter pin set needed for the mezzanine board didn’t have the same spacing as the 40-pin connector.

I notice that Amazon US has stopped selling this board, and I cannot say that I am surprised. Amazon UK still sells it, and that’s where I got mine.

I have worked with embedded development for a very long time. I have never come across something which causes the number off issues that this hardware + software + documentation does. I spent a full 12 hours on it without sufficient success. It is clear that I cannot use such a fragile product for my current project (which is optimising the GMP library for the A73). Even if I could coax it into working now, I expect it to cause an unacceptable maintenance burden during its lifetime.

For reference this is what I ordered: https://www.amazon.co.uk/gp/product/B07FNJ97QP

I did wonder how a heatsink and fan would fit, but couldn’t imagine that they hadn’t tried assembling their product before selling it. I assumed the picture was misleading. I was mistaken.

I was only confused about the various cables and pins. I was sure there would be some notes on what they would be user for. Again, I was mistaken.

We engineers have a responsibility of making careful designs and implementing things with as few issues as possible, and then document for others what we’ve done and how others can take it from there. I think we’re seeing a decline here, things are gradually less carefully done. I waste much more than 50% of my time on struggling with others’ poor engineering. As a result, my own GNU hacking has grinded to a halt.

I can understand a certain portion of your frustration, however, a large portion of the problem that you are complaining about is related to the RESELLER (Smartfly) you purchased the item from, rather than the item itself, its manufacturer, or 96boards/linaro.

The UART dongle that yours came with is NOT a mezzanine board, and is NOT SUPPORTED or recommended. In fact, I happen to have one of those exact UART dongles myself, and guess what? I would NEVER plug it into a 96boards SBC, because it is, in fact, INCOMPATIBLE/DANGEROUS since it communicates at 3.3 or 5.0 volts whereas the SBC requires 1.8 volt. If you want to play with a raspberry pi, it would be fine (perhaps?), but it is NOT compatible with a 96boards CE.

I would also like to point out that the vendor you purchased from has included a non-standard FAN. The Hikey960 ships with a low profile COPPER heat sink, not a fan with an aluminum heat sink.

Now as far as software support goes, well, not everybody is suited for working with dev boards.

Still comes up as available on Amazon US. Might be just the old link is no longer valid? The product got a RAM bump several months ago, which came along with a price bump and new listings.

Oops, I followed a link on 96boards.org. My mistake! I now see several variants are still available.

Granted that some of this negative experience is due to their sloppiness.

Sorry if I used the term mezzanine word improperly.

I had not noticed this incompatibility, and this should perhaps worry also Linaro and 96boards! It seems like a reasonable assumption that the kit will operate together.

I would add a HUGE warning about this problem in the 96boards web page if I were Linaro.

“Note that this board does not support HDMI when running GNU/Linux. It requires extra hardware to enable the only way of controlling and setting up GNU/Linux. and the kits currently sold includes a usb-to-uart board which is likely to damage the Hikey due to incompatible voltage levels.”

Not all Hikey960 ships with that heat sink, apparently.

But why should we make things hard for each other? I think somebody familiar with these boards and its many peculiarities could write proper install instructions in a jiffy. Now, lots of people waste lots of time because things are a mess.

Doing software releases and writing useful documentation is an exercise in empathy; one needs to understand the perspective of the receiver. Don’t leave lots of old stuff around. Make proper (tested!) releases of entire software bundles. Either hide detail completely under (tested!) installation programs or else explain the detail so that other engineers can apply their engineering skills. Collect all relevant information is one place in a natural flow.

If you’re using GNU/Linux, you’re using lots of software I have written. Ideally, you’re hardly aware of it, as the software works smoothly and as you reasonably expect. And when you need to understand it better, there is carefully written documentation at your fingertips.

I did not write that software with an elitist or excluding attitude, “not everybody is suited for running my software”.

That isn’t any kind of attitude. It is simply a fact that there is a barrier to working with this kind of hardware. It isn’t going to be a flawless experience with a perfectly checked and rechecked 10 thousand page user manual. The software details are a continually moving target, and you are clearly not appreciating the enormous burden that keeping the details COMPLETE and up to date would be. A burden that frankly, is better placed in software development than in documentation.

And on top of that, gnu really isn’t the development focus for this board. What it is, is a first class AOSP reference board; Using Reference Boards  |  Android Open Source Project – that doesn’t mean, of course, that it won’t run gnu. It just means that the experience is going to be a bit gnarly.

So here is my idea; if you are such a big shot developer for gnu/linux as you say you are (and I’m not questioning that), then how about chipping in some hours to help smooth it out a bit? Patches to code or documentation are graciously accepted.

Now back to the hardware. I’m not really sure what kind of sales related documentation or warning you think would be reasonable. Maybe on every product page, it should have a big red warning saying something like “WARNING: UNOFFICIAL RESELLERS MAY PACKAGE PARTS THAT HAVE NOT BEEN CERTIFIED AS SAFE FOR USE WITH THIS PRODUCT.” – IMO, that would be entirely redundant. If you’re going to buy parts through joe shmoe reseller, then doing so has always been at your own risk. I’m thinking that you may even have fallen into the Amazon trap, where you think (incorrectly) that Amazon is the vendor, when in fact, Amazon was essentially doing what ebay does. When you’re buying something via Amazon, you need to pay attention to the “SOLD BY” label. If it doesn’t say Amazon there, then its an ebay-style transaction.

FYI: I’ve never even attempted to boot gnu on a hikey960. I do run gnu on my dragonboard410c. The dragonboard development is more focused on gnu. The dragonboard820c would be reasonably comparable to the hikey960, assuming that arrow ever gets off their butts and starts selling them again.

So firstly… whilst it hasn’t been totally fun for me to read this thread, many thanks for bringing these things up.

I agree we’ve not done the best job communicating expectations for the Debian snapshots for Hikey960 (for a start this might be the first time you have seen them called “snapshots”) and have just put in a PR to improve things.

We have to be a bit careful about big red warnings about reseller kits. Many resellers provide a useful services (local retailer, consumer rights, interesting kits, etc) and a really big warning risks damaging those businesses along with others. I’ve proposed some text but it is deliberately phrased in a fairly neutral way.

hikey960 takes a “kit of parts” approach working to upstream the parts and eventually land hikey960 support for GNU/Linux via regular distributions. Some of the other 96Boards do have board and/or SoC vendors who focus on issuing GNU/Linux OS images themselves (an approach which is much easier to document). However I don’t think any of these boards have A73 cores in them and IIUC this was your main area of interest.

I certainly sympathise if you want to send the board back. On the other hand if you do decide to soldier on despite the problems we’ll do what we can to help.

PS You mentioned problems with the docs. How much of this is problems with the official docs on 96boards.org and how much of this is links to stale material by the reseller (which seems to scatter gun link to a whole bunch of different places)?

First, let me thank you for your reasoned reply. I wish I had written in the same tone myself!

Your suggested text will surely help. Please also remove the text at the end of the installation instructions, “Ensure HDMI connection to monitor” which can reasonably be interpreted as if such a connection works. In lieu of that text, I think you should point out that one is still very much in need of a serial console.

I understand your concern about resellers. Still, it would be good if potentially board-damaging kits were taken off of the market, either by amazon or by the original seller. Competently assembled kits should replace those.

The main problems with the docs and linked software were, I think:

  1. Incorrect expectations
  2. Suggestion that HDMI would work
  3. Links to different sets of firmware
  4. Firmware sets which seem to contain old scripts and READMEs which cannot all be correct. Are some parts for non-UEFI, perchance?
  5. HOWTO style for the firmware update process, i.e. no background or explanation of what the various steps actually do. Sorting the contradictory scripts and READMEs might be possible if one is told what things mean.

I’m afraid that I will proceed with returning the board. I don’t have a working uart-to-usb interface board, and paid for a fan and case which I cannot use. Also, I have lost a lot of time on this already, and I don’t see a clear path towards an acceptably working system. The fact that USB stopped working while trying to configure WiFi was the straw as it and prevented me from getting operational networking after I failed getting WiFi up. At that time I was made aware (on this thread) about the 3.3V vs 1.8V issue which made me unplug things.

I might try again in the future if the docs/firmware state at 96boards.org looks better. (Right now, boards and mezzanine cards are out-of-stock were I look.)

It would be useful with more Arm boards accessible to a broader FOSS community. Have you considered making some systems available to the GCC Compile Farm?

Thanks Torbjörn.

I’ve made a few more changes to the website and agree it would be better to make clear the the Android release and Debian snapshots use an entirely different firmware.

In terms of when/if to take another look, I’d suggest you don’t worry too much about 96boards.org: the sign a board is in really great shape (and will definitely keep track of the latest softwware) is when the board is sufficiently well supported by the mainline kernel that distros adopt and blog about it. At that point 96boards.org/documentation/<board_name> actually becomes rather spartan: firmware install, board recovery, some guides… and lots of entries in the “Third party operating systems” section!

This is what I will do:

I have returned the badly assembled kit.

I will purchase a Hikey960 board and the good (1.8V) uart mezzanine board soon. I will then follow the updated 96boards instructions and see if I am more lucky, and else provide feedback (aka cry for help).

To be honest about those bad kits… the incompatible parts are pretty much thrown in for free. So you can just take it, and understanding what it actually is, keep it in your spare parts box for when you actually need those parts (i.e., for some other project). The board doesn’t actually need a cooling fan, nor does it actually need a heat sink. In fact, even the “correct” heat sink won’t really do much, because (a) it is incredibly limited surface area, and (b) the heat has to soak THROUGH the RAM before it gets to the heat sink. The reality is that this type of hardware (i.e., SoC’s that are primarily intended for battery powered devices, as opposed to datacenter CPUs) mostly are cooled through the PCB, which can actually be fairly effective with just a gentle breeze blowing across the board.

I don’t actually suggest the “uart mezzanine”. A better choice is the “sensors mezzanine”. Its just $5 more, and in addition to the USB/UART interface, adds on a microcontroller, level shifters, and a bunch of sockets for plugging in peripherals.

Hey @danielt : I just noticed that the “buy” links to seeed for the uart and sensors mezzanine are broken. They just go to the seeed main page rather than the specific product page.

That’s a shame. I’ll pass it on.

Another sneaky approach is to get the (very, very cheap) audio-mezz board and desolder the (dragonboard specific) analog audio header. Naturally this removes any audio capability but still leaves you with a debug UART and a bunch of useful level shifters:
https://www.arrow.com/en/products/114990332/seeed-technology-limited

I think you may have misunderstood. The products are still available for sale, just the links need to be updated to new targets.

It’s OK. We got that as soon as we went looking:

Still a weird problem though. The chain of links isn’t wrong but I think the Seeed site reacts in a special way to permanent redirects… I don’t know if that makes me wish I knew more about web programming or less :wink: