Clone a dragonboard to other dragonboard units

Products Support DragonBoard410c
#8

Infrastructure changes means the link in #1 above would now be spelled differently:
https://git.linaro.org/ci/ubuntu-build-service.git/tree/jessie-arm64-alip

Having said that, the link has nothing to do with cloning images: it is a link to scripts to generate filesystem images from scratch (and it generates images from last years release rather than the current one). As Nico said before, it is generally better practice to generate images than to clone one from a board. Cloning from a board implies you haven’t got adequate infrastructure to effectively deliver updates (including security updates).

#9

another issue with ‘cloning’ an image, is that there are files with UID or keys which are generated during the first boot, so you need to clone before the first boot , but I suppose that if you want to clone it is because you want to clone your own image modification, which you would typically do after the first boot.

This is a very common question that you are asking, we hear that often. Maybe you can explain a bit more why you want to clone image, and we can start a document/wiki with recommendations about how to deal with such use cases?

#10

@anon91830841, imagine that you buy a DragonBoard, and then you want to build something on it (let’s say OpenCV for example), which takes some time, and then you change the configuration, and then you add some swap file, etc, etc. After that you simply want to copy/duplicate the DB to another one… That simply means to copy the content of the eMMC. I guess that’s what being done during manufacturing…
When I install linaro according the instructions (https://www.96boards.org/documentation/ConsumerEdition/DragonBoard-410c/Installation/LinuxFastboot.md.html) I burn 3 images. I want to be able to create these images after I put my code into them… Simple.

#11

ok, this specific use case should be covered with this guide then:

https://www.96boards.org/documentation/ConsumerEdition/DragonBoard-410c/Installation/LinuxSDBoot.md.html

Search for “Creating an image of the internal eMMC”

It does slightly more than “just” one partition, since it copies the entire eMMC image. if you prefer you can limit to making a copy of the partition you care about only (e.g. copy from /dev/mmcblk0pXX instead of /dev/mmcblk0). If you copy a single partition, then you end with a partition image (EXT4 for example), and you can then use fastboot flash to transfer on another board.

they key information in the guide is to make a block device copy, not a filesystem copy (e.g. dd , not cp), and to make the copy while the eMMC is not mounted (or booted).

#12

Thanks. I’ll have a look!

#13

The docs were revamped recently and the redirects aren’t working properly. The guide Nico mentions above is now found at:

#14

Thank you so much @danielt. I had discovered those resources after hunting about a bit and was waiting to report on how it works out after trying it out today.

Will report back. Thanks again.

#16

Hey @ttsas,

Did you ever get around to trying this? We are in a similar situation and I’m curious as to what your results might have been.

#17

@eee Sorry for the late response.

Here’s what I did, created a bootable microsd card using dd and the image dragonboard410c_sdcard_developer_debian-xxx.img (where xxx = some version number).

Then, I booted up into a live session.
Then cloned the installed OS with customizations (on the board’s storage) to an img using dd.
I used an external USB drive, mounted it, and used it as the destination for storing this img.

From here on out, I use the same process, to write this image from the USB, to a new board.

Seems to be working great so far, but I don’t know how it will hold up between hardware changes.

Hope that helps.

#18

So to anyone reading this far, the key (and instructions according to @danielt’s link) seems to be to create a block image of the rootfs partition while this partition is not mounted. This essentially means booting a live OS (96boards provides one) and creating the image from there.

@danielt @anon91830841, I’m still very much interested in the more formal way to configure a custom rootfs for our application as at some point we might bring to (initially low-ish volume) production. This customized OS would result from a combination of your Linaro release (which we’ll likely internally freeze at some point), some additional deb packages from signed repos, and our compiled proprietary source. Are the instructions in @danielt’s last post sufficient, or do I need to turn the complexity nob up a few notches?

#19

I doubt it. They only tell how to burn a pre-built SD card and how to backup the eMMC.

Most of the options are laid out in a pair of posts from myself and @anon91830841 (you’ll see us happily disagree with each other… but this is only really about how much effort it is for distributors to setup their own obs server and run fai):

#20

After jumping around a few topics related to this one, I can see this question getting asked in different contexts and being answered with different solutions depending on the particular need. Can you give (or reference) a set of instructions for mounting and chrooting this image with QEMU? Hopefully what I’m asking for is not redundant, but I have a sneaking suspicion it is.

This method…

At the really simple end this can be a matter of taking our image, unpacking it, loopback mounting it and chroot’ing (with the help of qemu) to update things. Then you can repack and install it as normal.

…seems to be the “smarter” path for our use-case. In short, I need a method to make some filesystem changes (install some packages, disable Bluetooth on boot, create a new user, etc.), drop in our complied application, button it up and flash this as our rootfs. Since IMO this is an appliance-like device, I like the Yocto idea too.

The longer version:

We have a small number of units running Debian and our application inside their functional prototype systems. I need a method to package up our changes and ideally simply reflash the rootfs to implement them. Since the changes are primarily in our application or an updated dependency, the unpack-update-repack method looks it’ll work. I can compile our app + deps on a reference DB410c (or x-compile), take the bins, drop them in a “clean” unpacked Debian, then overwrite the rootfs on the target board. Our current method is to:

  1. SSH in while performing sacrifices to the DHCP gods
  2. Do a bunch of git pulls
  3. Rebuild everything
  4. Cry over our lost productivity

Now that I have more than 2 boards to update in a run I’m really itching for a better way.

BTW, I’ll be happy to make a new topic if this a bit hijackie.

#21

I think it’s pertinent to this topic, so continuing here wouldn’t be too bad. If you do make a topic, please do share a link here.

Either way, I am interested in seeing if this qemu method works well for you.

#22

Loopback mounting the image involves:

  • Decompressing
  • Converting from Android sparse format to raw format: simg2img rootfs.img rootfs.img.raw
  • Mounting: mkdir mountpoint && sudo mount -o loop rootfs.img.raw mountpoint

Once you have the filesystem mounted any programs you need to run that using the target architecture (such as apt-get) need to run with qemu using steps are described here (see the appendix on using chroot): QemuUserEmulation - Debian Wiki .

Note that the above instructions only work on distros that provide qemu-aarch64-static (which in practice usually means Debian or one of its derivatives such as Ubuntu) otherwise you cannot copy the emulator into the rootfs (and probably don’t have the kernel setup to allow it to run alien binaries using the emulator).

Once you are done with the changes umount, convert back using img2simg (if it crashes then you probably need a newer version) and you are good to go.

#23

BTW if the qemu instructions don’t gel for you, another option is to run chroot natively on an AArch64 device. In other words put the rootfs image into a large SD card, mount it on the DB410C and run chroot natively to modify the rootfs.

1 Like
#24

Alright, this is a bit lengthy, but it attempts to describe a reproducible process to edit the 96boards Linaro image while hopefully minimizing complexity. This is a rough (unrefined) process, so if anyone has good input I’d be happy to amend. I was able to:

  1. Download the Linaro rootfs from http://releases.linaro.org/96boards/dragonboard410c/linaro/debian/latest/linaro-buster-developer-dragonboard-410c-528.img.gz (note this image lacks a GUI)
  2. Expand and convert from sparse to raw with simg2img rootfs.img rootfs.img.raw
  3. Mount with sudo mount -o loop rootfs.img.raw mountpoint
  4. Modify rootfs
  5. Repackage
  6. Flash and boot the target

If you’re going to be adding to the image you’ll probably want to expand its size lest you run out of disk (file) space. The method I used was (with rootfs umounted):

  1. Append data (zeros) to the end of your raw image file. In this case we add 1GB: dd if=/dev/zero bs=1G count=1 >> rootfs.img.raw. I have also seen some use fallocate.
  2. Run filesystem check: e2fsck -f rootfs.img.raw
  3. Resize the fs to match the new file size: resize2fs rootfs.img.raw

Alternatively you could mount a temp folder from your host inside the chroot jail and avoid expanding and shrinking your image. This seems like a good idea when building things from source as it’s easy to control where your build products go, but it seems a bit more complicated when you need more room for other OS related stuff (like running apt).

If you haven’t already, set up QEMU as described in @danielt’s link (QemuUserEmulation - Debian Wiki). The takeaway is that you’ll be able to execute chroot mountpoint which will run qemu under the hood for you, or something.

Now you are in your emulated arm64 environment and can run aarch64 binaries, including apt-get to manipulate packages, “natively” build and install your own program, etc. I proved it to myself by cross-compiling a “hello world” in the host environment, copying it over to the chroot, running file on it to get something like ELF 64-bit LSB shared object, ARM aarch64, then running it.

Now it’s time to button everything up and get it on your DB410c. After cleaning up any unwanted files, unmounting, and reconverting to sparse, I used fastboot to flash only the rootfs leaving the bootloader and that other thing alone.

I lack the background knowledge to know exactly which parts of the following are required and which aren’t. In particular I’m not extremely clear on how to re-shrink the altered rootfs gracefully. It seems that converting to a sparse image “handles” the free space with its metadata magic. I did:

  1. Unmount everything related to rootfs. I wanted to do an apt upgrade to have the latest-greatest and openjdk made me mount /proc (mount -t proc proc /proc) in the chroot. You’ll have to unmount any nested mounts before unmounting rootfs.
  2. Convert to sparse: img2simg rootfs.img.raw rootfs.img. Caution: you might overwrite your original rootfs.img.
  3. Put your Dragonboard in fastboot mode with these instructions and flash with fastboot using the microUSB host on the DB410c: fastboot flash rootfs rootfs.img.
  4. Boot the Dragonboard and cry tears of joy while not having to compile for hours!

The enterprising engineer can now crate a development pipeline that uses the ARM architecture without having to compile on the target as well as (perhaps crudely) generating a customized image that can be flashed to many targets.

Alright, now it’s time for someone take this apart.

Using Qemu to update packages on rootfs
"Shutdown now" hangs
#25

Sort of.

The new space will be filled with zeros and img2simg detects this and
writes it out in a much smaller form… however when you flash it with
fastboot the zeros will be written to the media so the flashing part
will take longer than with the orignal image (by more than just the cost
of the new stuff you added).

If you want to minimise flashing time try using the -M option to
resize2fs to shrink it down as much as possible. However if it is only a
few boards then I wouldn’t worry about it.

#26

Sorry my last comment was sent by e-mail and discourse ripped out the quoting (so I’ve just edited to put the quote back).

I certainly wasn’t replying to this bit… it looks like a pretty good summary of this technique to me. I was just filling in the details about shrinking file system images if you want to do that!

#27

This is a really interesting thread for me as I have been looking to have a Dragonboard image that I can easily share. I have arrived at a very similar process to yourself. The only difference is the step to expand the size of the image. I have used truncate (truncate(1) - Linux man page).
Here it is in the context how get up to the modify rootfs step.

$ sudo apt-get install android-tools-adb
$ sudo apt-get install android-tools-fastboot
$ sudo apt install android-tools-fsutils
$ simg2img linaro-stretch-developer-qcom-snapdragon-arm64-20171016-283.img linaro-stretch-developer-qcom-snapdragon-arm64-20171016-283.img.raw
# resize disk
$ truncate -s 2G linaro-stretch-developer-qcom-snapdragon-arm64-20171016-283.img.raw
$ mkdir mnt-point
$ sudo mount -t ext4 -o loop linaro-stretch-developer-qcom-snapdragon-arm64-20171016-283.img.raw mnt-point
$ sudo resize2fs /dev/loop0

After modifying the rootfs then my steps to create an image for flashing is very similar:

$ sudo umount mnt-point
$ img2simg linaro-stretch-developer-qcom-snapdragon-arm64-20171016-283.img.raw linaro-stretch-developer-qcom-snapdragon-arm64-20171016-283_bb.img
#28

I would like to thank you for this gude, it really helped me to get started! I would just like to add that I had problems (SSLError malloc failure with python’s pip) using qemu-user-static package from Ubuntu since it’s pretty outdated (version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.26)). I had to compile qemu 5.0.0 on my own with flags: ./configure --static --disable-system --enable-linux-user --enable-kvm. Then I added aarch64 entry to binfmt with ./qemu-binfmt-conf.sh --debian --qemu-path /usr/local/bin. I also had to copy qemu-aarch64 to chroot jail: cp $(which qemu-aarch64) /mnt/$(which qemu-aarch64).