How to generate sdcard installation image?

Products Support DragonBoard410c
#1

Hi All,

I see both Linaro & Qualcomm provide the sdcard installation images for Debian & Android respectively.
http://builds.96boards.org/releases/dragonboard410c/linaro/debian/latest/dragonboard410c_sdcard_install_debian-110.zip
http://builds.96boards.org/releases/dragonboard410c/qualcomm/android/latest/dragonboard410c_sdcard_install_android-118.zip

But it seems no documentation about how to generate these installation image?
Does anyone know how to do this?

In addition, is it possible to generate a installation sdcard for Yocto OE build?

Thanks,
Daniel

#2

hey,

good timing… we are currently working on publishing all these bits… I have to admit it was about time… but it also got pushed back to later… We have been using some internal (non public) tools to generate these images, and it was not easy to reproduce. However all the bits needed are now open source, and i am currently updating our build scripts accordingly.

for such SD images the following ‘components’ are needed:

  1. specific bootloader to boot from SD and load the boot image (kernel) from SD card
  2. the graphical application that is started is based on the RPi NOOBS, and has been modified a bit for our needs
  3. a ‘flashing’ tool that will eventually take the ‘raw’ images that need to be put into eMMC

#1 are now published in http://builds.96boards.org/snapshots/dragonboard410c/linaro/rescue-ng/ (i will move them to the same place as before soon)

#2 is available here nicolas.dechesne/db410c-installer.git - [no description]. We have ready to use images/rootfs that include this app (and all its dependencies) here:
http://snapshots.linaro.org/debian/images/installer-arm64/
http://snapshots.linaro.org/debian/images/installer-armhf/

#3 is available here https://git.linaro.org/people/nicolas.dechesne/db-boot-tools.git

I am combining all these and will update our main build job so that the image we produce on 96baords for the DB410c will use only these tools. So you (or anyone else) will be able to use them too and customize.

There is already a bugzilla for this issue, btw. https://bugs.96boards.org/show_bug.cgi?id=115

And the same tool set has been tested on Hikey, i believe.

PS: for the gory detail: one of the main change behind the scene is that with the current tool the images have to be pre-processed into a custom format to be flashed. With the revamped tool set you can simply drag and drop any release images onto the SD card, and that should work. So you will be able to flash Android, Debian or OE builds (or anything else which produces fastboot ready images).

1 Like
#3

Hello,
In addition using the adapted version of the python ffu2img script will enable burning the Windows IOT images to the eMMC much faster.
The script can be found here:

#4

hi Leonid, I have to admit, i am not at all familiar with the Windows images… but i have been requested a few times to be able to install them using the SD card , like we do for android/linux. so what you are saying is that with ffu2img we could convert the windows images into ‘fastboot’ images, and then flash them like we do for android and linux. Is that correct?

#5

Hi,
Last time i tested it it was more simple then converting. Just running ffu2img.py3 flash.ffu /dev/mmcblk0 was sufficient to burn the system.

#6

@ndec When I try to mksdcard with linaro-jessie-installer-20160913-93.tar.gz, I find there is a very big file inside.
The file is binary\dev\core, and the file size is 127TB! So that I am not able to flash rootfs on sdcard.
Can I delete this file or there is something wrong for my steps?

I use this way to transfer rootfs.tar.gz to rootfs.ext4


# dd if=/dev/zero of=rootfs.ext4 bs=1M count=1024
# mkfs.ext4 -F rootfs.ext4
# mount rootfs.ext4 /mnt -o loop
# cd /mnt;tar xf /somewhere/rootfs.tar.gz
# umount rootfs.ext4

Thanks,
Daniel

#7

hi,

/dev/core is a symlink to /proc/kcore, so that would link to your host PC in fact. If you look at our build job [1], you will see that we do the following:

sudo rm -rf rootfs/dev rootfs/boot rootfs/var/lib/apt/lists
sudo mkdir rootfs/dev rootfs/boot rootfs/var/lib/apt/lists

After extracting the rootfs, you should do the same thing as well.

Also, you will find at [2] my current patch to generate automatically the SD installer from the main build job. I am still working on this patch and testing it, but it should give you some hints. I will hopefully merge that in the next few days. The current status is that I was able to flash a Debian image with the latest installer… Just need to finalize a few things, but it’s looking good now.

[1] https://git.linaro.org/ci/job/configs.git/blob/HEAD:/lt-qcom-debian-images-arm64/builders.sh
[2] https://review.linaro.org/#/c/14310/

#8

hi Daniel,

so I have merged the changed I mentioned earlier, and in the last build:

http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/133/

the recipe to build the SD card for the installer is now using only scripts/tools/blobs which are publicly available.

Here are the main changes between this new SD card installer vs the old one:

  • it is using the latest kernel (instead of the 4.0 based kernel that we used at launch, that was still used for the installer)
  • the GUI hasn’t changed much, however on the backend it is now running on top of Debian/X11. This is the main reason why the image is larger, we will probably need to reduce the rootfs size a bit.
  • the format of the images that can be flashed has been updated, and we can directly put any standard ‘fastboot’ images into the FAT partition on the SD card. Previously the images needed to be post processed with some QCOM tools. So it should be much easier now to put your own boot.img or rootfs.img. There is also a partition.txt file that describe what the partition table should be on the eMMC, the installer will create the GPT based on that file, so users can customize their GPT very easily
  • the boot image and the rootfs used for the installer are now published (as fastboot images)

It comes with a slight cost, and the flashing is a bit slower than it was.

There is one issue that need to be fixed, at the end of the flashing, it won’t reboot, but it restarts the X11 session. So you need to manually power cycle the board. That will be fixed soon.

There are a few things that can be added/improved over time:

  • create an SD image with no images to flash, only the installer, and add support for having the images on USB drive for example
  • add support for fetching files over the network
  • create an SD cards with all images available from 96boards.org (android, oe and debian)
    …

For reference, the job that produces our images is here:

https://git.linaro.org/ci/job/configs.git/blob/HEAD:/lt-qcom-debian-images-arm64/builders.sh#l234

feedback is more than welcome…

cheers

#9

Hi Nicolas

I would like to install custom images from sdcard. I went through the script for building sdcard image. how do i create the rootfs and boot sdcard installer images?

Is there a gui based tool to create sdcard installer images?

Thanks
Ashwin

#10

hi,

I am sorry, but I don’t think I understand the question.

#11

Hi Nicolas

I was referring to the lt-qcom-debian-images-arm64_builders.sh, where you have 2 sets of boot and rootfs images built. I’m guessing one for fastboot and one for sdcard install.

whats the difference between ${VENDOR}-${OS_FLAVOUR}-alip-${PLATFORM_NAME}-${VERSION}.img.gz and ${VENDOR}-${OS_FLAVOUR}-installer-${PLATFORM_NAME}-${VERSION}.img.gz

I would like to know since i’m using inforce images for this.

I saw in your requirements in your first post where you had mentioned a graphical application that is started is based on the RPi NOOBS. What is this for. I assumed that this was the gui based tool to create sdcard installer images.

Thanks
Ashwin

#12

@anon91830841 Thanks! Based on your builders.sh script, I can generate a SD installation image with both Debian & OE-RPB os images inside. Here is my steps. Hope it can help ones who also want to create installation sdcard.

  1. Put the installer images into out/ folder.

sudo apt-get update
sudo apt-get install -y kpartx python-pycurl device-tree-compiler zip libfdt-dev mtools android-tools-fsutils

#######################################
# let's create the SD install image now
#######################################
VENDOR = "linaro"
OS_FLAVOUR = "jessie"
PLATFORM_NAME = "qcom-snapdragon-arm64"
BUILD_NUMBER = "133"
VERSION = "20160915-${BUILD_NUMBER}"

git clone --depth 1 -b master https://git.linaro.org/landing-teams/working/qualcomm/db-boot-tools.git/
# record commit info in build log
cd db-boot-tools
git log -1

# Get SD and EMMC bootloader package
BL_BUILD_NUMBER=<code>wget -q --no-check-certificate -O - https://ci.linaro.org/jenkins/job/lt-qcom-db410c-bootloader/lastSuccessfulBuild/buildNumber</code>
wget --progress=dot -e dotbytes=2M \
     http://builds.96boards.org/snapshots/dragonboard410c/linaro/rescue-ng/${BL_BUILD_NUMBER}/dragonboard410c_bootloader_sd_linux-${BL_BUILD_NUMBER}.zip
wget --progress=dot -e dotbytes=2M \
     http://builds.96boards.org/snapshots/dragonboard410c/linaro/rescue-ng/${BL_BUILD_NUMBER}/dragonboard410c_bootloader_emmc_linux-${BL_BUILD_NUMBER}.zip

unzip -d out dragonboard410c_bootloader_sd_linux-${BL_BUILD_NUMBER}.zip
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/boot-installer-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/${VENDOR}-${OS_FLAVOUR}-installer-${PLATFORM_NAME}-${VERSION}.img.gz
cp boot-installer-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz out/boot.img.gz
cp ${VENDOR}-${OS_FLAVOUR}-installer-${PLATFORM_NAME}-${VERSION}.img.gz out/rootfs.img.gz
gunzip out/{boot,rootfs}.img.gz
  1. Prepare Debian OS images

# OS image [Debian]
mkdir -p os/debian
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/boot-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/${VENDOR}-${OS_FLAVOUR}-alip-${PLATFORM_NAME}-${VERSION}.img.gz
cp boot-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz os/debian/boot.img.gz
cp ${VENDOR}-${OS_FLAVOUR}-alip-${PLATFORM_NAME}-${VERSION}.img.gz os/debian/rootfs.img.gz
gunzip os/debian/{boot,rootfs}.img.gz

cat << EOF >> os/debian/os.json
{
"name": "Linaro Linux Desktop for DragonBoard 410c - Build #${BUILD_NUMBER}",
"url": "http://builds.96boards.org/releases/dragonboard410c",
"version": "${VERSION}",
"release_date": "<code>date +%Y-%m-%d</code>",
"description": "Linaro Linux with LXDE desktop based on Debian (${OS_FLAVOUR}) for DragonBoard 410c"
}
EOF

cp mksdcard flash os/
cp dragonboard410c/linux/partitions.txt os/debian
unzip -d os/debian dragonboard410c_bootloader_emmc_linux-${BL_BUILD_NUMBER}.zip
  1. Prepare OE-RPB OS images (I get the latest OE-RPB images directly.)

# OS image [OE-RPB]
mkdir -p os/oe-rpb
wget https://builds.96boards.org/releases/reference-platform/openembedded/dragonboard410c/latest/rpb/boot--4.4-r0.1-dragonboard-410c-20160704175332-54.img
cp boot--4.4-r0.1-dragonboard-410c-20160704175332-54.img os/oe-rpb/boot.img

wget https://builds.96boards.org/releases/reference-platform/openembedded/dragonboard410c/latest/rpb/rpb-desktop-image-dragonboard-410c-20160704175332-54.rootfs.ext4.gz
mv rpb-desktop-image-dragonboard-410c-20160704175332-54.rootfs.ext4.gz rootfs.ext4.gz
gunzip rootfs.ext4.gz
sudo ext2simg -v rootfs.ext4 os/oe-rpb/rootfs.img

cat << EOF >> os/oe-rpb/os.json
{
"name": "Yocto Linux Desktop for DragonBoard 410c - Build #54",
"url": "http://builds.96boards.org/releases/dragonboard410c",
"version": "20160704175332-54",
"release_date": "<code>date +%Y-%m-%d</code>",
"description": "Reference Platform Build (CE OpenEmbedded) for DragonBoard 410c"
}
EOF

cp dragonboard410c/linux/partitions.txt os/oe-rpb
unzip -d os/oe-rpb dragonboard410c_bootloader_emmc_linux-${BL_BUILD_NUMBER}.zip
  1. Generate os.img & execute mksdcard script

# get size of OS partition
size_os=$(du -sk os | cut -f1)
size_os=$(((($size_os + 1024 - 1) / 1024) * 1024))
size_os=$(($size_os + 200*1024))
# pad for SD image size (including rootfs and bootloaders)
size_img=$(($size_os + 1024*1024 + 300*1024))

# create OS image
sudo rm -f out/os.img
sudo mkfs.fat -a -F32 -n "OS" -C out/os.img $size_os
mkdir -p mnt
sudo mount -o loop out/os.img mnt
sudo cp -r os/* mnt/
sudo umount mnt
sudo ./mksdcard -p dragonboard410c/linux/installer.txt -s $size_img -i out -o db410c_sd_install_rpb_debian.img

zip -j db410c_sd_install_rpb_debian.zip db410c_sd_install_rpb_debian.img

That’s done! You should be able to see these 2 os images after booting from sdcard.

1 Like
SOLVED - Custom rootfs.img breaks SD installer
#13

hi daniel , thanks for sharing back your instructions! I hope it will help ashwin :wink:

#14

Hi Daniel

Thanks for the script. This helps a lot.

Cheers
Ashwin

#15

hi Nicolas

I was able to successfully create a sdinstall image for IFC6309 which works.

But i noticed that once i install the the image from sd card i cannot install other images via fastboot.

ashwin@ashwin-desktop:/media/ashwin/files/IFC6309_Debian_Linux_BSP_880455_V2.0/binaries$ sudo fastboot flash aboot emmc_appsboot.mbn 
target reported max download size of 268435456 bytes
sending 'aboot' (726 KB)...
OKAY [  0.026s]
writing 'aboot'...
OKAY [  0.331s]
finished. total time: 0.357s
ashwin@ashwin-desktop:/media/ashwin/files/IFC6309_Debian_Linux_BSP_880455_V2.0/binaries$ sudo fastboot flash boot boot-ifc6309_v2.0.img 
target reported max download size of 268435456 bytes
sending 'boot' (17674 KB)...
OKAY [  0.558s]
writing 'boot'...
OKAY [  1.419s]
finished. total time: 1.976s
ashwin@ashwin-desktop:/media/ashwin/files/IFC6309_Debian_Linux_BSP_880455_V2.0/binaries$ sudo fastboot flash userdata linaro-jessie-alip-qcom-snapdragon-arm64-20160511-85_2.img 
target reported max download size of 268435456 bytes
erasing 'userdata'...
FAILED (remote: Partition table doesn't exist

As you can see the userdata flashing fails with an error.
Would you know what is going on here.

Thanks
Ashwin P

#16

Hi I need your help badly. I have 3 LG Escape 2 H433 handsets which has gone into dead boot. I learned I could revive them using Qfil or an SD card. Frankly I wish you could help me with the necessary steps or what to do to resolve this issue. Please kindly help me out.

#17

hi,

We aren’t really supporting ‘real devices’ with any of our releases. I doubt you can boot from SD card a ‘real’ device, it requires changes in the initial (proprietary) bootloader to be enabled, as well as changing the boot config. For QFIL (or any other equivalent USB loader), i also doubt you can get it to work on real devices. At any rate, this is not something we would support here on this specific forum, which focuses only on ‘open’ dev boards.

cheers

#18

The above links are broken.

#19

There’s a lot of links above. Which one do you need?

#20

Hello folks, after searching around a lot and trying to get some information from the hardware vendor, I found this topic.

I notice that a lot of the links in post are dead, and I’m not sure how to proceed. Any help would be really appreciated! I am going to post a different thread going in to detail about what I’m trying to achieve but in short:

I have mutiple boards with Linaro installed (model # 6309 SBC Inforce 6309™ Micro Single Board Computer (SBC)).

I have setup one board with all the settings/configuration etc that I want. Now I want to clone an image of this board as it stands, so that I could write that image to the other boards without having to go through every installation/configuration step again on all the boards one by one.

  1. I took one board that had Linaro installed from the factory and installed all the libraries that I needed.
  2. I configured and set up these libraries for my application.
  3. I installed my application and configured everything to work with it.
  4. At this point, I want to freeze this image as it stands into an image file(s).
  5. I want to then reuse this image file(s) for all my other boards so that I could somehow write this image to the other boards and have them ready to go.

What if I copied the userdata partition from the board, and just overwrote that partition on all the other boards? Would that allow me to achieve what I need? I know one of the responses above advises against it, but do I have an alternative?

If coying userdata works, what would be the best way to achieve that? How would I talk to the board to achieve this? (fastboot? dd?)

This is new territory for me and I would really appreciate any guidance in the right direction.

Their documentation talks about taking a clean image, modifying it, and creating another image from it, but I’m unable to get a definitive response about whether I can achieve what I described above.

On an x86/amd64 computer this would be very straightforward, clone the hard drive, and use that image to write it out to other similar hardware.

Again, I would really appreciate any help on the topic, thank you!