Rebuilding the boot partition on 410c

As you suspected my modules are too large to fit into initrd. I took a look at the /lib/modules/4.0.0-linaro-lt-qcom directory, their modules are exactly the same as the ones in the initrd and take up about 4.5MB. The modules that I built from source are just over 56MB. What do I need to do different in the ‘make modules’ process to get the ‘minimal’ modules that are shipped vs the full modules that are generated?

I did try putting all 56MB into the rootfs, but I must have done something wrong, it still doesn’t boot.

You need to strip the binaries. Use the strip command.

But I still don’t think it would make any sense to mix them into the boot image or initrd. As long as it can mount the rootfs, it can get those modules from the rootfs.

Modules that go in the boot image or initrd should only be those modules that are required in order to get to the point where it can obtain the rest of the modules from the rootfs. That means things like filesystem modules, and storage controllers. If the GPU drivers are modularized (which I hope they aren’t, since the kernel is build specifically for that GPU), then early initialization of graphics could require THOSE modules (and their firmware, and firmware loading modules) to also be included in the initrd.

Further, you will need to duplicate the modules between the initrd and the rootfs, because once the rootfs is mounted at /, it will obscure the initrd.

Now from an ANDROID point of view, (be aware that things are setup a bit different), NO modules are actually required to boot. Once it is up, some things obviously don’t work, wifi being the big one.

Could you post a list of the modules that you have?

Hi doitright: Thanks for your help. I will try stripping the modules and see if they get to the same size as the ones in the distribution.

I should go back to the beginning and explain what I am trying to do instead of getting stuck in the details. Maybe there is a better way to do this. I am ‘trying’ to rebuild the kernel for two reasons: 1) I want to ensure that the online instructions are complete and correct, 2) I want to make a minor change to the kernel to enable the uart0 port on the low speed connector so that I can send characters to a uart serial 2x16 LCD display connected to the port. In theory this should be easy!.

As you can see I have failed at 1). I used the instructions at https://builds.96boards.org/releases/dragonboard410c/linaro/ubuntu/15.07/ and https://github.com/96boards/documentation/wiki/Dragonboard-410c-Boot-Image in my mind I shouldn’t need any other instructions to successfully build a kernel, but as it turns out the instructions don’t work.

I have been successful at 2). I tested that my changes do actually make the uart0 port on the low-speed connector work, unfortunately the side effect of my change is it breaks the display.

Don’t forget that you aren’t actually following the instruction precisely as they are written, since you’ve had to modify them for running from the db410c itself, so what you are doing doesn’t really do a great job of (1).

Actually, I’m looking at those instructions, and it refers to the module building part as optional. I suspect from that, that you probably should be able to light it up without the modules being accessible.

Ooooh, waitaminute. I may be seeing something interesting.
The original instructions say to use root=/dev/ram0
You’re using root=/dev/disk/by-partlabel/rootfs

Remember that you’re using an initrd. rd = ramdisk. /dev/ram0 is a ramdisk.

More explanation: Linux can boot with or without a ramdisk, as long as whichever option you are providing to it has all of the necessary initialization programs, scripts, and modules.

So there are two options; either extract the initrd to the filesystem at the partition referenced by /dev/disk/by-partlabel/rootfs, or tell it to use the ramdisk as the rootfs, and contain within the ramdisk the proper fstab and scripts to overlay the ramdisk with the final rootfs at the proper time to hand control over to it. The original ramdisk is already configured to do this. Since you are building a boot.img WITH a ramdisk, you should tell it to actually use that ramdisk.

Hmm, might not be it. Hard to tell. I’m trying to understand the scripts in the initrd, but there are just so many ways that things can be set up.

Can you post the commandline for the original, working kernel?

Good catch. I’ll try a different cmdline, I selected the cmdline from the next block of instructions because I do have a rootfs in the onboard eMMC. The instructions are not clear on when you should use which cmdline. I suspect an experienced Linux developer would have known what to do.

I took some other liberties with the earlier steps in the process because I was not cross compiling.

This morning I went into the /boot directory and extracted the initrd.img-4.0.0 and opened it up (gunzip, cpio). The contents do not match the contents of the initrd.img-4.0.0-linaro-lt-qcom that is posted for download. I would have thought they would be the same file, but possibly something magic happens in the mkbootimg step. There is an extra 12k of stuff, the initrd in /boot doesn’t have a /lib/modules/Linux-4.0.0-linaro-lt-qcom directory and contents.

P.S. the 410c can rebuild the kernel in about 70 minutes, and make modules in 11 minutes.

mkbootimg is a really stupid and simple program. It does not modify the initrd at all. It literally just glues the three input files together, and adds a bit of spacing and a few headers.

Sorry I can’t post the cmdline for the original working kernel. The “original working kernel” that I am using was downloaded is the release binary image from https://builds.96boards.org/releases/dragonboard410c/linaro/ubuntu/15.07/ I am trying to recreate this image.

I tried changing the cmdline to use /dev/ram0 but I get the error “ALERT! /dev/ram0 does not exist. dropping to a shell!”. from the shell I was able to confirm that /dev/ram0 really doesn’t exist. I tried it with both the ramdisk image that came with the release, and the ramdisk image that was poster (they are different). Same result either way.

I am now working on a i7 box running Ubuntu 12.04LTS. Should the instructions work here???. I got to an early step “export CROSS_COMPILE=<path to your GCC cross compiler>/aarch64-linux-gnu-” but my box doesn’t have tha cross compiler installed. It would be nice if they gave a hint as to where to find the cross compiler, or which version was used to build the kernel.

I have googled my way to a linaro web page on how to install the compiler here: https://wiki.linaro.org/HowTo/BuildArm64Kernel and I am now following the steps at the top for a cross toolchain. Of course the web page is out of date and the install instructions point to a version that no longer exists. I have had to take some liberties with the instructions and hunt down the latest version (version 4.9 from December 2014 I hope). I’ll let you know how it goes once the cross compiler is installed.

Am I the only person that has tried to build the kernel by following the instructions?

The commandline can be seen at /proc/cmdline after you have booted on it, or at the very top of the output of dmesg.

I can’t tell you exactly where to find the aarch64 compiler on ubuntu since I don’t use that distro, but on Fedora, the package is called gcc-aarch64-linux-gnu … surely ubuntu has some similar package. Try something like apt-cache dumpavail or apt-cache search all or something like that, and pipe it through “grep gcc”

Thanks for showing me where to look. The cmdline from the working kernel doesn’t match the cmdline in the online documentation. It is " root=/dev/disk/by-partlabel/rootfs rw rootwait console=tty0 console=ttyMSM0,115200n8 androidboot.emmc=true androidboot.serialno=1f9800c3 androidboot.baseband=apq adv7533_dsi2hdmi.panel=dsi mdss_mdp.panel=1:dsi:0:qcom,mdss_dsi_qcom,cc-debug-8936". Still doesn’t bring up the HDMI display with the kernel I built and the revised command line.

The boot log messages also tell me which compiler was used to build the system. “gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09”.

Note that some of those parameters are actually added in by the bootloader.

I would say that a good place to go at this stage may be to try that original compiler.

That commandline looks a bit odd. Though I am not entirely up on the ubuntu side of the display workings, that has at least the superficial appearance of trying to activate dsi output rather than HDMI. But of course you say that that commandline is the working one… very odd. You may need to look into the adv7533 driver and see if they have changed how it responds to parameters in recent revisions. It may be possible that they were previously ignoring the “panel” parameter, but now are using it to select the dsi output.

The display setup on the board is such that it only actually has a dsi output. There is a gpio controlled SWITCH that determines whether that signal is routed to the high speed expansion header, or to the adv7533 chip. The adv7533 chip is a dsi to hdmi converter. In the Android kernel, the adv7533 driver has control over the switch, and decides which position for the switch to be in, depending on whether or not the “panel” parameter contains the string “hdmi”.

Now having said that, don’t get too worked up about those parameters, could be a red herring. I know that the upstream adv7533 driver is VERY different from the Android version – vastly more advanced. It could very well be that those parameters are just leftovers that have absolutely no impact.

hi there,

long discussion… sorry about the delay, i was out for some time. few notes/questions:

  • can you please post the entire boot log on the uart console?

  • the error you mentioned:

Aug 17 19:45:43 linaro-alip kernel: [ 5.617844] subsys-pil-tz 1de0000.qcom,venus: venus: Invalid firmware metadata

has nothing to do with GPU, or the GPU firmware. Venus is the IP that does Multimedia video, not GPU (which is adreno).

  • the release page has pointers to the cross compiler to use in the ‘how to rebuild the kernel’ section.

  • you do not say where/how you got the initrd-4-0-qcom-lt file which is used when calling mkbootimg tool, can you specify that?

Hi ndec:

First an update. When I build the kernel on a i7 box running Ubuntu the kernel that gets built works. My issues came when I tried to build the kernel on the 410c.

When cross compiling on an i7 the instructions say to use compiler gcc-linaro-4.9-2014.11-x86_64-aarch-linux-gnu, but the boot log messages say the released kernel was actually build with (gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) Note the month is different.

I tried getting the initrd-4.0.0-qcom-lt from two different places. The two files are different but neither works.

  1. download from the 96Boards site: https://builds.96boards.org/releases/dragonboard410c/linaro/ubuntu/15.07/initrd.img-4.0.0-linaro-lt-qcom
  2. from the 410c board by copying /boot/initrd.img

I’ll follow up with a failing log soon.

Quick follow up on the problems I was having rebuilding the kernel on the 410c. It turns out I had run into an known issue with the current release (15.07) that has been identified and fixed, all I needed to do was include the correct patch and the build worked. Here are my ‘revised’ instructions on how to successfully rebuild the kernel on the 410c, you will not there is a cherry-pick command added after fetching the source and before compiling it.

Hardware

  • 410c board and power supply.
  • HDMI Display
  • USB Mouse and Keyboard
  • 8GB SDCard (for installing Linux)
  • 8GB SDCard (for swap space to build)
    410c Setup

Install Ubuntu. Follow the SDCard install instructions here (use the 8GB SDCard):

o http://linaro.co/96b-linuxuserguide

Set MAC address

o Start ->other -> bash
o cd /lib/firmware/wlan
o sudo vi macaddr0
 Change mac adder to match sticker on board
 <esc>:wq
o reboot
o Connect to access point

Check mac address matches sticker on board.

o Start ->other -> bash
o ifconfig

install a 8GB SDCard, partition it, and mount it.

o sudo gdisk /dev/mmcblk1p1
o Command (? for help): o
o This option deletes all partitions and creates a new protective MBR.
o Proceed? (Y/N): y
o #build a 8GB partition for swap
o Command (? for help): n
o Partition number (1-128, default 1):
o First sector (34-8388574, default = 2048) or {±}size{KMGTP}:
o Last sector (2048-8388574, default = 8388574) or {±}size{KMGTP}: 8G
o Current type is ‘Linux filesystem’
o Hex code or GUID (L to show codes, Enter = 8300): 8200
o Changed type of partition to ‘Linux swap’
o
o Command (? for help): w
o Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING PARTITIONS!!
o Do you want to proceed? (Y/N): y
o

Reboot and run the following commands

o sudo mkswp /dev/mmcblk1p1

edit /etc/fstab and add the following line

o sudo vi /etc/fstab
/dev/mmcblk1p1 none swap sw 0 0
<esc>:wq

Reboot and run the following commands

o free # ensure swap is mounted

Install dev tools and support programs

Login to linaro/linaro. Start -> Other -> bash

o sudo apt-get update
o sudo apt-get install linux-headers-$(uname -r) build-essential
o sudo apt-get install linux-headers-$(uname -r) –fix-missing build-essential
o sudo apt-get install git
o sudo apt-get install device-tree-compiler

fetch the kernel sources and rebuild them

o cd /usr/src
o sudo mkdir linux-4.0.0
o sudo chown linaro linux-4.0.0
o sudo chgrp linaro linux-4.0.0
o cd linux-4.0.0
o git clone -n working/qualcomm/kernel.git - Qualcomm Landing Team kernel
o cd kernel
o git checkout -b kernel-15.07 ubuntu-qcom-dragonboard410c-15.07
o git cherry-pick –n fb68837eb7fbd9e65f6b9ed40718caa2b00452f1
o export ARCH=arm64
o make defconfig distro.config
o make -j4 Image dtbs modules
o sudo make INSTALL_MOD_STRIP=1 modules_install
o sudo make install #<-- it would be nice if this worked, then we wouldn’t need to do the remaining steps.

use the ramdisk that you just built

o cp /boot/initrd.img-4.0.0 .
o #wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/ubuntu/latest/initrd.img-*

build the boot img file and install it onto the boot partition

o git clone git://codeaurora.org/quic/kernel/skales
o skales/dtbTool -o dt.img -s 2048 arch/arm64/boot/dts/qcom/
o export cmdline=“root=/dev/disk/by-partlabel/rootfs rw rootwait console=ttyMSM0,115200n8”
o skales/mkbootimg --kernel arch/arm64/boot/Image
o --ramdisk initrd.img-4.0.0
o --output boot-db410c.img
o --dt dt.img
o --pagesize 2048
o --base 0x80000000
o --cmdline “$cmdline”
o sudo dd if=boot-db410c.img of=/dev/mmcblk0p8
o sync
o reboot

thanks for the follow up message, just for the record, the issue you mentioned and that you hit is this one:

https://bugs.96boards.org/show_bug.cgi?id=80

In the following command for building the swap

# Reboot and run the following commands
o	sudo mkswp /dev/mmcblk1p1

should be:

# Reboot and run the following commands
o	sudo mkswap /dev/mmcblk1p1

Hi Linaro,
The source of this problem is https://github.com/96boards/documentation/wiki/Dragonboard-410c-Boot-Image listing "export cmdline=“root=/dev/ram0 rw rootwait console=ttyMSM0,115200n8"”. Can you please change it to "export cmdline=“root=/dev/disk/by-partlabel/rootfs rw rootwait console=ttyMSM0,115200n8"” so that others don’t stumble into the same issue?

Thanks

DISCLAIMER: I am an employee of QuIC Inc. Any opinions expressed in this or any other post may not reflect the opinions of my employer.

hi,

hardcoding the bootargs to use the ‘/dev/disk/by-partlabel/rootfs’ might not be the perfect answer neither. By using such bootargs, we are making the assumptions that:

  1. there is a partition called rootfs (which is the case only if the Linux bootloader were flashed)
  2. there is an initrd in the boot image that started udev properly. otherwise /dev/disk/by-partlabel wouldn’t exist.

So modifying the wiki as suggested here, might help some people, but would break for others.

I will see how this can be improved…