Rebuilding the boot partition on 410c

I have been attempting to recreate the boot partition on the 410c. I didn’t have a Linux box available to do the compile, so I figured it should be possible to rebuild on the target. I followed directions from the 96Boards web site:
https://builds.96boards.org/releases/dragonboard410c/linaro/ubuntu/15.07/
https://github.com/96boards/documentation/wiki/Dragonboard-410c-Boot-Image
Throughout the process I needed to make minor changes to the instructions because I was running on the target. I have included my step-by-step instructions below so you can see what I did.

At the end I get a kernel installed and the new kernel comes up and gives me a prompt on the UART console port. BUT there is a problem with the kernel I have created, it doesn’t start the HDMI port and ALIP/LXDE. Obviously I have missed something while configuring the kernel. My rootfs image is 14790656 bytes, and the rootfs image that is in the 15.07 directory is 14774272 bytes. For some reason my rootfs doesn’t exactly match the posted rootfs.

Can someone tell me what step I missed? Here are my steps (sorry about the poor formatting):

Hardware

  • 410c board and power supply.
  • HDMI Display
  • USB Mouse and Keyboard
  • 8GB SDCard (for installing Linux)
  • 64GB SDCard (for storage to build)
    410c Setup
  • Install Ubuntu. Follow the SDCard install instructions here (use the 8GB SDCard):
    o http://linaro.co/96b-linuxuserguide
  • Set MAC address (do this and the rest of the steps this while logged in on the HDMI monitor)
    o Start ->other -> bash (you will need to re-login and restart bash after each reboot)
    o cd /lib/firmware/wlan
    o sudo vi macaddr0
     Change mac adder to match sticker on board
     :wq
    o reboot
    o Connect to access point
  • Check mac address matches sticker on board.
    o ifconfig
  • install a 64GB SDCard, partition it, and mount it.
    o sudo gdisk /dev/mmcblk1p1
    o Command (? for help): o
    o This option deletes all partitions and creates a new protective MBR.
    o Proceed? (Y/N): y
    o #build a 8GB partition for swap
    o Command (? for help): n
    o Partition number (1-128, default 1):
    o First sector (34-8388574, default = 2048) or {±}size{KMGTP}:
    o Last sector (2048-8388574, default = 8388574) or {±}size{KMGTP}: 8G
    o Current type is ‘Linux filesystem’
    o Hex code or GUID (L to show codes, Enter = 8300): 8200
    o Changed type of partition to ‘Linux swap’
    o #build a 48GB partition for storage
    o Command (? for help): n
    o Partition number (1-128, default 1):
    o First sector (34-8388574, default = 2048) or {±}size{KMGTP}:
    o Last sector (2048-8388574, default = 8388574) or {±}size{KMGTP}:
    o Current type is ‘Linux filesystem’
    o Hex code or GUID (L to show codes, Enter = 8300):
    o
    o Command (? for help): w
    o Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING PARTITIONS!!
    o Do you want to proceed? (Y/N): y
    o
  • Reboot and run the following commands (re-login and restart bash)
    o sudo mkswp /dev/mmcblk1p1
    o sudo mkfs –t ext4 /dev/mmcblk1p2
  • #edit /etc/fstab and add the following line
    /dev/mmcblk1p1 none swap sw 0 0
    o ln –s /media/linaro/5d582b5b-fdfd-492b-a3b7-07adc4651c04/ workspace
  • Reboot and run the following commands (re-login and restart bash)
    o free # ensure swap is mounted
    o df #ensure /home/linaro/workspace is mounted
    o cd workspace
    o sudo chown linaro .
    o sudo chgrp linaro .
  • Install dev tools and support programs
    o sudo apt-get update
    o sudo apt-get install fortune
    o sudo apt-get install linux-headers-$(uname -r) build-essential
    o sudo apt-get install git
    o sudo apt-get install device-tree-compiler
  • fetch the kernel sources and rebuild them
    o git clone -n working/qualcomm/kernel.git - Qualcomm Landing Team kernel
    o cd kernel
    o git checkout -b kernel-15.07 ubuntu-qcom-dragonboard410c-15.07
    o export ARCH=arm64
    o make defconfig distro.config
    o make -j4 Image dtbs
    o make modules
  • build the boot img file and install it onto the boot partition
    o git clone git://codeaurora.org/quic/kernel/skales
    o skales/dtbTool -o dt.img -s 2048 arch/arm64/boot/dts/qcom/
    o wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/ubuntu/latest/initrd.img-*
    o export cmdline=“root=/dev/disk/by-partlabel/rootfs rw rootwait console=ttyMSM0,115200n8”
    o skales/mkbootimg --kernel arch/arm64/boot/Image
    o --ramdisk initrd.img-4.0.0-linaro-lt-qcom
    o --output boot-db410c.img
    o --dt dt.img
    o --pagesize 2048
    o --base 0x80000000
    o --cmdline “$cmdline”
    o sudo dd if=boot-db410c.img of=/dev/mmcblk0p8
    o sync
    o reboot

I can’t give you an exact answer on how you made it slightly broken, but I will tell you this; don’t use the image SIZE difference to determine that the two images are effectively different. The size difference is very small and can be explained by the use of a different compiler.

I think it will be very difficult for anyone to guess at what you actually did to build your boot image, since NOBODY thinks in terms of the sequence of button presses during the process, which is effectively what you pasted.

In fact, what would be more useful than the buttons you pressed, would be a boot log and X log. They will describe what is actually going on in the boot, which can be used to figure out where your setup went wrong.

I also have to question your statement “I didn’t have a Linux box available to do the compile”. Since Linux is free, ANYTHING can be a Linux box.

Hi doitright

You are correct, Linux us free ANYTHING can be a Linux box, and in this case I am using the 410c as my Linux box.
I agree that the file size is not a good indicator that a binary is bad because of possible differences in the compiler code generators. it just means the files are different.

It has been a while since I last used a UNIX box, it was an ICM-3216 from National Semiconductor running SysVr2 in 1985 (6 years before the first Linux kernel was released). Needless to say my Linux debugging skills are close to non-existent.

Thanks for the great tip about looking at the log files. I compared the good log file with the bad log file. At startup the good build says: Linux version 4.0.0-linaro-lt-qcom
and my build says: Linux version 4.0.0-dirty (linaro@linaro-alip)
This of course creates difficulty about 600mS into the boot on when I get the error message: systemd-modules-load[231]: could not open moddep file ‘/lib/modules/4.0.0-dirty/modules.dep.bin’

However it appears that I get into the real trouble about 6 seconds into the boot. On the good build it says:

Aug 17 20:03:03 linaro-alip kernel: [ 6.342768] subsys-pil-tz 1de0000.qcom,venus: venus: Brought out of reset

and on my build it says:

Aug 17 19:45:43 linaro-alip kernel: [ 5.617844] subsys-pil-tz 1de0000.qcom,venus: venus: Invalid firmware metadata

looks like I don’t have the correct binaries needed to run the GPU in my build. What step did I miss when I created the build?

Hmm, that is definitely interesting.

Can you verify that your rootfs is actually being mounted?
Can you check that you have the files /lib/firmware/a300_pfp.fw and /lib/firmware/a300_pm4.fw?
Can you check that you have the path /lib/modules/4.0.0-dirty?

Note that the files you are looking for should be in the rootfs.

You can look here for the ubuntu board support package, which contains the required qcom firmware;

A note about the modules; the linux kernel is picky about loading modules. They need to match the running kernel. Your running kernel is “4.0.0-dirty”, and likely the modules you have match a kernel “4.0.0-linaro-lt-qcom”, it therefore can’t find them, and even if it could, would refuse to load them. As part of your build, you have to put the new modules where they belong.

Yes the rootfs is mounted. I ran ‘df’ and I can see the rootfs. A ‘ls -l’ on /lib/firmware confirms that a300_pm4.fw is there (9220 bytes). ‘uname -r’ confirms that I am running my kernel (4.0.0-dirty). and no I don’t have /lib/modules/4.0.0-dirty.

Offline from another Linux guru I received pointers to some more Linux tribal knowledge, how to unroll the initrd, move my dirty modules into the initrd and roll it up again. gunzip the file initrd file I downloaded from 96boards, then use cpio to expand it, insert the modules/4.0.0-dirty, cpio and gzip it back up. Unfortunately I did something wrong and made things worse. I’ll have go through the logs to figure that out next.

I forgot my standard full disclosure notice in the previous posts. I am an employee of Qualcomm Canada. Any opinions expressed in this or any other post may not reflect the opinions of my employer.

My guess is that your modules are probably just too large to fit in the boot.img. It is fairly constrained for space.

I suggest putting your modules in their correct location in the rootfs.

As you suspected my modules are too large to fit into initrd. I took a look at the /lib/modules/4.0.0-linaro-lt-qcom directory, their modules are exactly the same as the ones in the initrd and take up about 4.5MB. The modules that I built from source are just over 56MB. What do I need to do different in the ‘make modules’ process to get the ‘minimal’ modules that are shipped vs the full modules that are generated?

I did try putting all 56MB into the rootfs, but I must have done something wrong, it still doesn’t boot.

You need to strip the binaries. Use the strip command.

But I still don’t think it would make any sense to mix them into the boot image or initrd. As long as it can mount the rootfs, it can get those modules from the rootfs.

Modules that go in the boot image or initrd should only be those modules that are required in order to get to the point where it can obtain the rest of the modules from the rootfs. That means things like filesystem modules, and storage controllers. If the GPU drivers are modularized (which I hope they aren’t, since the kernel is build specifically for that GPU), then early initialization of graphics could require THOSE modules (and their firmware, and firmware loading modules) to also be included in the initrd.

Further, you will need to duplicate the modules between the initrd and the rootfs, because once the rootfs is mounted at /, it will obscure the initrd.

Now from an ANDROID point of view, (be aware that things are setup a bit different), NO modules are actually required to boot. Once it is up, some things obviously don’t work, wifi being the big one.

Could you post a list of the modules that you have?

Hi doitright: Thanks for your help. I will try stripping the modules and see if they get to the same size as the ones in the distribution.

I should go back to the beginning and explain what I am trying to do instead of getting stuck in the details. Maybe there is a better way to do this. I am ‘trying’ to rebuild the kernel for two reasons: 1) I want to ensure that the online instructions are complete and correct, 2) I want to make a minor change to the kernel to enable the uart0 port on the low speed connector so that I can send characters to a uart serial 2x16 LCD display connected to the port. In theory this should be easy!.

As you can see I have failed at 1). I used the instructions at https://builds.96boards.org/releases/dragonboard410c/linaro/ubuntu/15.07/ and https://github.com/96boards/documentation/wiki/Dragonboard-410c-Boot-Image in my mind I shouldn’t need any other instructions to successfully build a kernel, but as it turns out the instructions don’t work.

I have been successful at 2). I tested that my changes do actually make the uart0 port on the low-speed connector work, unfortunately the side effect of my change is it breaks the display.

Don’t forget that you aren’t actually following the instruction precisely as they are written, since you’ve had to modify them for running from the db410c itself, so what you are doing doesn’t really do a great job of (1).

Actually, I’m looking at those instructions, and it refers to the module building part as optional. I suspect from that, that you probably should be able to light it up without the modules being accessible.

Ooooh, waitaminute. I may be seeing something interesting.
The original instructions say to use root=/dev/ram0
You’re using root=/dev/disk/by-partlabel/rootfs

Remember that you’re using an initrd. rd = ramdisk. /dev/ram0 is a ramdisk.

More explanation: Linux can boot with or without a ramdisk, as long as whichever option you are providing to it has all of the necessary initialization programs, scripts, and modules.

So there are two options; either extract the initrd to the filesystem at the partition referenced by /dev/disk/by-partlabel/rootfs, or tell it to use the ramdisk as the rootfs, and contain within the ramdisk the proper fstab and scripts to overlay the ramdisk with the final rootfs at the proper time to hand control over to it. The original ramdisk is already configured to do this. Since you are building a boot.img WITH a ramdisk, you should tell it to actually use that ramdisk.

Hmm, might not be it. Hard to tell. I’m trying to understand the scripts in the initrd, but there are just so many ways that things can be set up.

Can you post the commandline for the original, working kernel?

Good catch. I’ll try a different cmdline, I selected the cmdline from the next block of instructions because I do have a rootfs in the onboard eMMC. The instructions are not clear on when you should use which cmdline. I suspect an experienced Linux developer would have known what to do.

I took some other liberties with the earlier steps in the process because I was not cross compiling.

This morning I went into the /boot directory and extracted the initrd.img-4.0.0 and opened it up (gunzip, cpio). The contents do not match the contents of the initrd.img-4.0.0-linaro-lt-qcom that is posted for download. I would have thought they would be the same file, but possibly something magic happens in the mkbootimg step. There is an extra 12k of stuff, the initrd in /boot doesn’t have a /lib/modules/Linux-4.0.0-linaro-lt-qcom directory and contents.

P.S. the 410c can rebuild the kernel in about 70 minutes, and make modules in 11 minutes.

mkbootimg is a really stupid and simple program. It does not modify the initrd at all. It literally just glues the three input files together, and adds a bit of spacing and a few headers.

Sorry I can’t post the cmdline for the original working kernel. The “original working kernel” that I am using was downloaded is the release binary image from https://builds.96boards.org/releases/dragonboard410c/linaro/ubuntu/15.07/ I am trying to recreate this image.

I tried changing the cmdline to use /dev/ram0 but I get the error “ALERT! /dev/ram0 does not exist. dropping to a shell!”. from the shell I was able to confirm that /dev/ram0 really doesn’t exist. I tried it with both the ramdisk image that came with the release, and the ramdisk image that was poster (they are different). Same result either way.

I am now working on a i7 box running Ubuntu 12.04LTS. Should the instructions work here???. I got to an early step “export CROSS_COMPILE=<path to your GCC cross compiler>/aarch64-linux-gnu-” but my box doesn’t have tha cross compiler installed. It would be nice if they gave a hint as to where to find the cross compiler, or which version was used to build the kernel.

I have googled my way to a linaro web page on how to install the compiler here: https://wiki.linaro.org/HowTo/BuildArm64Kernel and I am now following the steps at the top for a cross toolchain. Of course the web page is out of date and the install instructions point to a version that no longer exists. I have had to take some liberties with the instructions and hunt down the latest version (version 4.9 from December 2014 I hope). I’ll let you know how it goes once the cross compiler is installed.

Am I the only person that has tried to build the kernel by following the instructions?

The commandline can be seen at /proc/cmdline after you have booted on it, or at the very top of the output of dmesg.

I can’t tell you exactly where to find the aarch64 compiler on ubuntu since I don’t use that distro, but on Fedora, the package is called gcc-aarch64-linux-gnu … surely ubuntu has some similar package. Try something like apt-cache dumpavail or apt-cache search all or something like that, and pipe it through “grep gcc”

Thanks for showing me where to look. The cmdline from the working kernel doesn’t match the cmdline in the online documentation. It is " root=/dev/disk/by-partlabel/rootfs rw rootwait console=tty0 console=ttyMSM0,115200n8 androidboot.emmc=true androidboot.serialno=1f9800c3 androidboot.baseband=apq adv7533_dsi2hdmi.panel=dsi mdss_mdp.panel=1:dsi:0:qcom,mdss_dsi_qcom,cc-debug-8936". Still doesn’t bring up the HDMI display with the kernel I built and the revised command line.

The boot log messages also tell me which compiler was used to build the system. “gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09”.

Note that some of those parameters are actually added in by the bootloader.

I would say that a good place to go at this stage may be to try that original compiler.

That commandline looks a bit odd. Though I am not entirely up on the ubuntu side of the display workings, that has at least the superficial appearance of trying to activate dsi output rather than HDMI. But of course you say that that commandline is the working one… very odd. You may need to look into the adv7533 driver and see if they have changed how it responds to parameters in recent revisions. It may be possible that they were previously ignoring the “panel” parameter, but now are using it to select the dsi output.

The display setup on the board is such that it only actually has a dsi output. There is a gpio controlled SWITCH that determines whether that signal is routed to the high speed expansion header, or to the adv7533 chip. The adv7533 chip is a dsi to hdmi converter. In the Android kernel, the adv7533 driver has control over the switch, and decides which position for the switch to be in, depending on whether or not the “panel” parameter contains the string “hdmi”.

Now having said that, don’t get too worked up about those parameters, could be a red herring. I know that the upstream adv7533 driver is VERY different from the Android version – vastly more advanced. It could very well be that those parameters are just leftovers that have absolutely no impact.

hi there,

long discussion… sorry about the delay, i was out for some time. few notes/questions:

  • can you please post the entire boot log on the uart console?

  • the error you mentioned:

Aug 17 19:45:43 linaro-alip kernel: [ 5.617844] subsys-pil-tz 1de0000.qcom,venus: venus: Invalid firmware metadata

has nothing to do with GPU, or the GPU firmware. Venus is the IP that does Multimedia video, not GPU (which is adreno).

  • the release page has pointers to the cross compiler to use in the ‘how to rebuild the kernel’ section.

  • you do not say where/how you got the initrd-4-0-qcom-lt file which is used when calling mkbootimg tool, can you specify that?