How to generate sdcard installation image?

anon91830841 · February 20, 2017, 9:27am

hi,

/dev/core is a symlink to /proc/kcore, so that would link to your host PC in fact. If you look at our build job [1], you will see that we do the following:

sudo rm -rf rootfs/dev rootfs/boot rootfs/var/lib/apt/lists
sudo mkdir rootfs/dev rootfs/boot rootfs/var/lib/apt/lists

After extracting the rootfs, you should do the same thing as well.

Also, you will find at [2] my current patch to generate automatically the SD installer from the main build job. I am still working on this patch and testing it, but it should give you some hints. I will hopefully merge that in the next few days. The current status is that I was able to flash a Debian image with the latest installer… Just need to finalize a few things, but it’s looking good now.

[1] https://git.linaro.org/ci/job/configs.git/blob/HEAD:/lt-qcom-debian-images-arm64/builders.sh
[2] https://review.linaro.org/#/c/14310/

anon91830841 · February 20, 2017, 9:27am

hi Daniel,

so I have merged the changed I mentioned earlier, and in the last build:

http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/133/

the recipe to build the SD card for the installer is now using only scripts/tools/blobs which are publicly available.

Here are the main changes between this new SD card installer vs the old one:

it is using the latest kernel (instead of the 4.0 based kernel that we used at launch, that was still used for the installer)
the GUI hasn’t changed much, however on the backend it is now running on top of Debian/X11. This is the main reason why the image is larger, we will probably need to reduce the rootfs size a bit.
the format of the images that can be flashed has been updated, and we can directly put any standard ‘fastboot’ images into the FAT partition on the SD card. Previously the images needed to be post processed with some QCOM tools. So it should be much easier now to put your own boot.img or rootfs.img. There is also a partition.txt file that describe what the partition table should be on the eMMC, the installer will create the GPT based on that file, so users can customize their GPT very easily
the boot image and the rootfs used for the installer are now published (as fastboot images)

It comes with a slight cost, and the flashing is a bit slower than it was.

There is one issue that need to be fixed, at the end of the flashing, it won’t reboot, but it restarts the X11 session. So you need to manually power cycle the board. That will be fixed soon.

There are a few things that can be added/improved over time:

create an SD image with no images to flash, only the installer, and add support for having the images on USB drive for example
add support for fetching files over the network
create an SD cards with all images available from 96boards.org (android, oe and debian)
…

For reference, the job that produces our images is here:

https://git.linaro.org/ci/job/configs.git/blob/HEAD:/lt-qcom-debian-images-arm64/builders.sh#l234

feedback is more than welcome…

cheers

ashparth · February 20, 2017, 9:27am

Hi Nicolas

I would like to install custom images from sdcard. I went through the script for building sdcard image. how do i create the rootfs and boot sdcard installer images?

Is there a gui based tool to create sdcard installer images?

Thanks
Ashwin

anon91830841 · February 20, 2017, 9:27am

hi,

I am sorry, but I don’t think I understand the question.

ashparth · February 20, 2017, 9:27am

Hi Nicolas

I was referring to the lt-qcom-debian-images-arm64_builders.sh, where you have 2 sets of boot and rootfs images built. I’m guessing one for fastboot and one for sdcard install.

whats the difference between ${VENDOR}-${OS_FLAVOUR}-alip-${PLATFORM_NAME}-${VERSION}.img.gz and ${VENDOR}-${OS_FLAVOUR}-installer-${PLATFORM_NAME}-${VERSION}.img.gz

I would like to know since i’m using inforce images for this.

I saw in your requirements in your first post where you had mentioned a graphical application that is started is based on the RPi NOOBS. What is this for. I assumed that this was the gui based tool to create sdcard installer images.

Thanks
Ashwin

daniel-hung · March 21, 2017, 1:44pm

@anon91830841 Thanks! Based on your builders.sh script, I can generate a SD installation image with both Debian & OE-RPB os images inside. Here is my steps. Hope it can help ones who also want to create installation sdcard.

Put the installer images into out/ folder.


sudo apt-get update
sudo apt-get install -y kpartx python-pycurl device-tree-compiler zip libfdt-dev mtools android-tools-fsutils

#######################################
# let&#039;s create the SD install image now
#######################################
VENDOR = &quot;linaro&quot;
OS_FLAVOUR = &quot;jessie&quot;
PLATFORM_NAME = &quot;qcom-snapdragon-arm64&quot;
BUILD_NUMBER = &quot;133&quot;
VERSION = &quot;20160915-${BUILD_NUMBER}&quot;

git clone --depth 1 -b master https://git.linaro.org/landing-teams/working/qualcomm/db-boot-tools.git/
# record commit info in build log
cd db-boot-tools
git log -1

# Get SD and EMMC bootloader package
BL_BUILD_NUMBER=&lt;code&gt;wget -q --no-check-certificate -O - https://ci.linaro.org/jenkins/job/lt-qcom-db410c-bootloader/lastSuccessfulBuild/buildNumber&lt;/code&gt;
wget --progress=dot -e dotbytes=2M \
     http://builds.96boards.org/snapshots/dragonboard410c/linaro/rescue-ng/${BL_BUILD_NUMBER}/dragonboard410c_bootloader_sd_linux-${BL_BUILD_NUMBER}.zip
wget --progress=dot -e dotbytes=2M \
     http://builds.96boards.org/snapshots/dragonboard410c/linaro/rescue-ng/${BL_BUILD_NUMBER}/dragonboard410c_bootloader_emmc_linux-${BL_BUILD_NUMBER}.zip

unzip -d out dragonboard410c_bootloader_sd_linux-${BL_BUILD_NUMBER}.zip
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/boot-installer-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/${VENDOR}-${OS_FLAVOUR}-installer-${PLATFORM_NAME}-${VERSION}.img.gz
cp boot-installer-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz out/boot.img.gz
cp ${VENDOR}-${OS_FLAVOUR}-installer-${PLATFORM_NAME}-${VERSION}.img.gz out/rootfs.img.gz
gunzip out/{boot,rootfs}.img.gz

Prepare Debian OS images


# OS image [Debian]
mkdir -p os/debian
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/boot-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz
wget http://builds.96boards.org/snapshots/dragonboard410c/linaro/debian/${BUILD_NUMBER}/${VENDOR}-${OS_FLAVOUR}-alip-${PLATFORM_NAME}-${VERSION}.img.gz
cp boot-${VENDOR}-${OS_FLAVOUR}-${PLATFORM_NAME}-${VERSION}.img.gz os/debian/boot.img.gz
cp ${VENDOR}-${OS_FLAVOUR}-alip-${PLATFORM_NAME}-${VERSION}.img.gz os/debian/rootfs.img.gz
gunzip os/debian/{boot,rootfs}.img.gz

cat &lt;&lt; EOF &gt;&gt; os/debian/os.json
{
&quot;name&quot;: &quot;Linaro Linux Desktop for DragonBoard 410c - Build #${BUILD_NUMBER}&quot;,
&quot;url&quot;: &quot;http://builds.96boards.org/releases/dragonboard410c&quot;,
&quot;version&quot;: &quot;${VERSION}&quot;,
&quot;release_date&quot;: &quot;&lt;code&gt;date +%Y-%m-%d&lt;/code&gt;&quot;,
&quot;description&quot;: &quot;Linaro Linux with LXDE desktop based on Debian (${OS_FLAVOUR}) for DragonBoard 410c&quot;
}
EOF

cp mksdcard flash os/
cp dragonboard410c/linux/partitions.txt os/debian
unzip -d os/debian dragonboard410c_bootloader_emmc_linux-${BL_BUILD_NUMBER}.zip

Prepare OE-RPB OS images (I get the latest OE-RPB images directly.)


# OS image [OE-RPB]
mkdir -p os/oe-rpb
wget https://builds.96boards.org/releases/reference-platform/openembedded/dragonboard410c/latest/rpb/boot--4.4-r0.1-dragonboard-410c-20160704175332-54.img
cp boot--4.4-r0.1-dragonboard-410c-20160704175332-54.img os/oe-rpb/boot.img

wget https://builds.96boards.org/releases/reference-platform/openembedded/dragonboard410c/latest/rpb/rpb-desktop-image-dragonboard-410c-20160704175332-54.rootfs.ext4.gz
mv rpb-desktop-image-dragonboard-410c-20160704175332-54.rootfs.ext4.gz rootfs.ext4.gz
gunzip rootfs.ext4.gz
sudo ext2simg -v rootfs.ext4 os/oe-rpb/rootfs.img

cat &lt;&lt; EOF &gt;&gt; os/oe-rpb/os.json
{
&quot;name&quot;: &quot;Yocto Linux Desktop for DragonBoard 410c - Build #54&quot;,
&quot;url&quot;: &quot;http://builds.96boards.org/releases/dragonboard410c&quot;,
&quot;version&quot;: &quot;20160704175332-54&quot;,
&quot;release_date&quot;: &quot;&lt;code&gt;date +%Y-%m-%d&lt;/code&gt;&quot;,
&quot;description&quot;: &quot;Reference Platform Build (CE OpenEmbedded) for DragonBoard 410c&quot;
}
EOF

cp dragonboard410c/linux/partitions.txt os/oe-rpb
unzip -d os/oe-rpb dragonboard410c_bootloader_emmc_linux-${BL_BUILD_NUMBER}.zip

Generate os.img & execute mksdcard script


# get size of OS partition
size_os=$(du -sk os | cut -f1)
size_os=$(((($size_os + 1024 - 1) / 1024) * 1024))
size_os=$(($size_os + 200*1024))
# pad for SD image size (including rootfs and bootloaders)
size_img=$(($size_os + 1024*1024 + 300*1024))

# create OS image
sudo rm -f out/os.img
sudo mkfs.fat -a -F32 -n &quot;OS&quot; -C out/os.img $size_os
mkdir -p mnt
sudo mount -o loop out/os.img mnt
sudo cp -r os/* mnt/
sudo umount mnt
sudo ./mksdcard -p dragonboard410c/linux/installer.txt -s $size_img -i out -o db410c_sd_install_rpb_debian.img

zip -j db410c_sd_install_rpb_debian.zip db410c_sd_install_rpb_debian.img

That’s done! You should be able to see these 2 os images after booting from sdcard.

anon91830841 · February 20, 2017, 9:27am

hi daniel , thanks for sharing back your instructions! I hope it will help ashwin

ashparth · February 20, 2017, 9:27am

Hi Daniel

Thanks for the script. This helps a lot.

Cheers
Ashwin

ashparth · February 20, 2017, 9:27am

hi Nicolas

I was able to successfully create a sdinstall image for IFC6309 which works.

But i noticed that once i install the the image from sd card i cannot install other images via fastboot.

ashwin@ashwin-desktop:/media/ashwin/files/IFC6309_Debian_Linux_BSP_880455_V2.0/binaries$ sudo fastboot flash aboot emmc_appsboot.mbn 
target reported max download size of 268435456 bytes
sending &#039;aboot&#039; (726 KB)...
OKAY [  0.026s]
writing &#039;aboot&#039;...
OKAY [  0.331s]
finished. total time: 0.357s
ashwin@ashwin-desktop:/media/ashwin/files/IFC6309_Debian_Linux_BSP_880455_V2.0/binaries$ sudo fastboot flash boot boot-ifc6309_v2.0.img 
target reported max download size of 268435456 bytes
sending &#039;boot&#039; (17674 KB)...
OKAY [  0.558s]
writing &#039;boot&#039;...
OKAY [  1.419s]
finished. total time: 1.976s
ashwin@ashwin-desktop:/media/ashwin/files/IFC6309_Debian_Linux_BSP_880455_V2.0/binaries$ sudo fastboot flash userdata linaro-jessie-alip-qcom-snapdragon-arm64-20160511-85_2.img 
target reported max download size of 268435456 bytes
erasing &#039;userdata&#039;...
FAILED (remote: Partition table doesn&#039;t exist

As you can see the userdata flashing fails with an error.
Would you know what is going on here.

Thanks
Ashwin P

asuk · May 11, 2017, 3:12am

Hi I need your help badly. I have 3 LG Escape 2 H433 handsets which has gone into dead boot. I learned I could revive them using Qfil or an SD card. Frankly I wish you could help me with the necessary steps or what to do to resolve this issue. Please kindly help me out.

anon91830841 · May 11, 2017, 8:20am

hi,

We aren’t really supporting ‘real devices’ with any of our releases. I doubt you can boot from SD card a ‘real’ device, it requires changes in the initial (proprietary) bootloader to be enabled, as well as changing the boot config. For QFIL (or any other equivalent USB loader), i also doubt you can get it to work on real devices. At any rate, this is not something we would support here on this specific forum, which focuses only on ‘open’ dev boards.

cheers

behlingc · October 27, 2017, 2:22pm

The above links are broken.

danielt · November 3, 2017, 4:53pm

There’s a lot of links above. Which one do you need?

ttsas · February 23, 2018, 8:04pm

Hello folks, after searching around a lot and trying to get some information from the hardware vendor, I found this topic.

I notice that a lot of the links in post are dead, and I’m not sure how to proceed. Any help would be really appreciated! I am going to post a different thread going in to detail about what I’m trying to achieve but in short:

I have mutiple boards with Linaro installed (model # 6309 SBC Inforce 6309™ Micro Single Board Computer (SBC)).

I have setup one board with all the settings/configuration etc that I want. Now I want to clone an image of this board as it stands, so that I could write that image to the other boards without having to go through every installation/configuration step again on all the boards one by one.

I took one board that had Linaro installed from the factory and installed all the libraries that I needed.
I configured and set up these libraries for my application.
I installed my application and configured everything to work with it.
At this point, I want to freeze this image as it stands into an image file(s).
I want to then reuse this image file(s) for all my other boards so that I could somehow write this image to the other boards and have them ready to go.

What if I copied the userdata partition from the board, and just overwrote that partition on all the other boards? Would that allow me to achieve what I need? I know one of the responses above advises against it, but do I have an alternative?

If coying userdata works, what would be the best way to achieve that? How would I talk to the board to achieve this? (fastboot? dd?)

This is new territory for me and I would really appreciate any guidance in the right direction.

Their documentation talks about taking a clean image, modifying it, and creating another image from it, but I’m unable to get a definitive response about whether I can achieve what I described above.

On an x86/amd64 computer this would be very straightforward, clone the hard drive, and use that image to write it out to other similar hardware.

Again, I would really appreciate any help on the topic, thank you!

danielt · February 26, 2018, 12:20pm

I think you’ve probably landed on the wrong topic…

You’ve already ruled out doing this the “right” way (e.g. repeatable build engineering for mass production) and want to do it instead by cloning a board (backups and short runs). It is discussed here:

danielt · February 26, 2018, 12:22pm

Apologies… sent you to the wrong topic (hadn’t realized we’ve discussed the same issue twice recently). This thread is much clearer:

vonneu_code · August 16, 2019, 4:45am

Hello, I’m looking for the android sd installer creation scripts which is located in this private repo. Is it possible to get hold of the script, as I’d like to generate my own android sd installers. ssh://dev-private-git.linaro.org/landing-teams/working/qualcomm/db410c_bootloader.git. I’m looking for the script that does these ./build sdcard_install_android. And this is for the dragonboard 410c

anon91830841 · August 16, 2019, 5:21am

hey,

to be honest, i wasn’t aware we still had references to that private git tree… we haven’t done such a build in more than 2y, this is really old project.

This private git is what we used in the very early days of 410c, and it has been replaced by this tree now:
https://git.linaro.org/landing-teams/working/qualcomm/db-boot-tools.git

unfortunately it looks like we never updated the android build. Note that the Android build was the SD card for the Qualcomm Android release which is probably even older than that.

So maybe we should check what you really want to do. If you want to build a boot SD card with the installer script, then you should check our latest SD image with the Debian builds instead, the tools have been modernized and all scripts are published, see:
https://git.linaro.org/ci/job/configs.git/tree/lt-qcom-debian-images/sdcard-dragonboard-410c.sh
this is where we create the sdcard with the installer, and it uses
https://git.linaro.org/landing-teams/working/qualcomm/db-boot-tools.git/

which is the “latest” and public version of ssh://dev-private-git.linaro.org/landing-teams/working/qualcomm/db410c_bootloader.git.

vonneu_code · August 19, 2019, 2:03am

Thank you Nicolas, that helps a lot, I’m able to build the debian based sd installer with the updated scripts. The old scripts took me in the wrong direction, I’m right now in the process of setting up the android installation image within the sd installer.
Also regarding my requirement, I have a dsi touch display that i’m interfacing with the db410, most part of which is working, I need the custom installer to get the installation of OS done by using the dsi out and not the default hdmi out. Thanks again!!

anon91830841 · August 20, 2019, 2:30am

hi,

I think i meant to share this link that creates the SD card installer with our Debian image:
https://git.linaro.org/ci/job/configs.git/tree/lt-qcom-debian-images/installer-dragonboard-410c.sh

as you can see in the script that generates the SD card, the boot image that is used to boot the “Installer” is using a Linux kernel from our 4.14 release branch. e.g. the Installer is a simple X/Qt application that boots from SD card, and it installs our OS image from SD card into eMMC.

If you want to installer to run from DSI panel instead of HDMI, you need to have a boot kernel that has support for DSI instead of HDMI (and the DTB).