Initial Confusions

Products Support DragonBoard 820c
#1

I have recently received a Dragonboard 820c with Debian installed with the following kernel:
Linux version 4.11.0-qcomlt (abuild@r2-a19) (gcc version 7.2.1 20171025 (Debian 7.2.0-12) ) #1 SMP PREEMPT Mon Nov 6 23:24:36 UTC 2017

Before flashing the latest snapshots onto this board I thought I would review the current state and I have found several confusing inconsistancies which I hope someone can resolve.

As delivered, the kernel reports that write protect is off on all UFS devices, but ufs-provision_toshiba.xml in dragonboard-820c-bootloader-ufs-linux-31 sets bLUWriteProtect=“1” for LUs 1 to 4. Does this tag have any effect or was it not set when my board was configured?

As I understand it, bit 60 of the attribute flags of a GPT partition entry denotes read-only. lsblk does not seem to recognise this bit as meaning read only while reporting it set in the PARTFLAGS column in each partition of LUs 1 to 4 except partition 32 of LU 4, sti.
In addition, sti (partition 30 of LU 4 now as system and recovery have been removed), in gpt_both4.bin of dragonboard-820c-bootloader-ufs-linux-31 does not have bit 60 of the attribute flags set, whereas all the other partitions of LUs 1 to 4 do. Is sti special?

In patch.xml of dragonboard-820c-bootloader-ufs-linux-31, there is one instance of start_sector=“NUM_DISK_SECTORS-1.” for each LU which reads start_sector=“NUM_DISK_SECTORS-1”. Does the full stop (period) have any significance? If so, then the missing one may produce unexpected results, if not, then why include them at all?

gpt_*.bin files
Partition Alignment
#2

As I stated in my first post, I have recently received a Dragonboard 820c with Debian installed with the following kernel:
Linux version 4.11.0-qcomlt (abuild@r2-a19) (gcc version 7.2.1 20171025 (Debian 7.2.0-12) ) #1 SMP PREEMPT Mon Nov 6 23:24:36 UTC 2017
LXQt 0.11.1
root@linaro-alip:~# cat /etc/debian_version
buster/sid

On my actual hardware, lsblk reports, for each partition:
PHY-SEC = LOG-SEC = 4096 == 1 block
MIN-IO = OPT-IO = 65536 = 16 blocks
ALIGNMENT = (16 - StartingLBA mod 16) mod 16 blocks
ALIGNMENT is zero for sti but not for any other partition, in particular, for persist, userdata, cache and system, which would be expected to hold filesystems.
Is it the case that this no longer matters as the kernel driver or UFS controller takes care of alignment issues?

Again, on the actual hardware, last_parti, at /dev/sdd4, /dev/sde33 and /dev/sdf4, has zero sectors as reported by:
root@linaro-alip:~# fdisk -l -o Device,Start,End,Sectors,Type,Attrs,Name
with the preceding partition taking the remainder of the LU.
Does the patch.xml in dragonboard-820c-bootloader-ufs-linux-31 rectify this? It appears to intend to do so.
Is there any documentation on the xml tag attributes? In particular, what is bHighPriorityLUN?

#3

As my first post seems to have disappeared into the void, :), I thought I would try a few more questions. First, may I say that I am coming from a Linux background and know next to nothing about Android.

What tool is used to generate the gpt_*.bin files in dragonboard-820c-bootloader-ufs-linux-31?
The closest I can get is with fdisk. It is the only tool I know which lets you set the block size of a loop-backed file. However, it insists on starting partitions on 1MB, or 256 block, boundaries. Gdisk will let you change the sector alignment but does not let you change the sector size and loop devices seem to have a sector size of 512. Also, fdisk does not allow you to change the number of partition entries from the default 128. However, both tools do set the correct value of 0x000200 for StartingCHS in the PMBR.

Is there any particular reason for the number of partition entries being the smallest multiple of 4 greater than or equal to the number of actual partitions?

The rootfs partition should have a type GUID of ‘Linux root (ARM-64)’.
The partition type GUIDs in the gpt_*.bin files seem to be random, or are they just internal to Qualcomm?

#4

Wow. A lot of questions most of which I have no clue about.

However the number of partition entries is in my mind. The reason there must be 128 entries is because that in the smallest number permitted by the standard (GPT is described in the UEFI standard).

#5

Before flashing the latest snapshots onto this board I thought I would review the current state and I have found several confusing inconsistancies which I hope someone can resolve.

I suppose I have to attempt to reply here… since i built the bootloader packages here… however be aware that we are a bit in the dark here, and I don’t have all answers…

As delivered, the kernel reports that write protect is off on all UFS devices, but ufs-provision_toshiba.xml in dragonboard-820c-bootloader-ufs-linux-31 sets bLUWriteProtect=“1” for LUs 1 to 4. Does this tag have any effect or was it not set when my board was configured?

First note: There is no guarantee at this point that the board you received was flashed/provisioned with the file we released. While we want to ensure that Arrow uses our files during the board production, the process is not finalized yet. I would expect them to use the same xml file, since it originally comes from QCOM, but no guarantee. On the long term, the plan is to Arrow to use all our released files, but for this batch, the boards were flashed before we released them…

bLUWriteProtect=1 means power-on write protection, =2 would mean permanent write protection. See commit 57d104c153d3d6d7bea60089e80f37501851ed2c in kernel for an initial explanation.

As I understand it, bit 60 of the attribute flags of a GPT partition entry denotes read-only. lsblk does not seem to recognise this bit as meaning read only while reporting it set in the PARTFLAGS column in each partition of LUs 1 to 4 except partition 32 of LU 4, sti.
In addition, sti (partition 30 of LU 4 now as system and recovery have been removed), in gpt_both4.bin of dragonboard-820c-bootloader-ufs-linux-31 does not have bit 60 of the attribute flags set, whereas all the other partitions of LUs 1 to 4 do. Is sti special?

sti is not special, as far as i know… Note that all files released in the bootloader package are coming from here: dragonboard820c - working/qualcomm/db-boot-tools.git - [no description] you can see the changes we did in the partition table.

In patch.xml of dragonboard-820c-bootloader-ufs-linux-31, there is one instance of start_sector=“NUM_DISK_SECTORS-1.” for each LU which reads start_sector=“NUM_DISK_SECTORS-1”. Does the full stop (period) have any significance? If so, then the missing one may produce unexpected results, if not, then why include them at all?

yes, correct. I am seeing that now. from the documentation about the firehose protocol (which describes the patch command) it’s not obvious whether it’s needed or not… Looking at QCOM source code for their own flashing tools (not our open source variant), the ‘.’ is interpreted as a comma when doing string to number conversion, e.g. "// user did this 12.3, so return 12 " , so assuming we can trust that… it’s safe to ignore in this specific example.

#6

On my actual hardware, lsblk reports, for each partition:
PHY-SEC = LOG-SEC = 4096 == 1 block
MIN-IO = OPT-IO = 65536 = 16 blocks
ALIGNMENT = (16 - StartingLBA mod 16) mod 16 blocks
ALIGNMENT is zero for sti but not for any other partition, in particular, for persist, userdata, cache and system, which would be expected to hold filesystems.
Is it the case that this no longer matters as the kernel driver or UFS controller takes care of alignment issues?

i am not sure about this one…

Again, on the actual hardware, last_parti, at /dev/sdd4, /dev/sde33 and /dev/sdf4, has zero sectors as reported by:
root@linaro-alip:~# fdisk -l -o Device,Start,End,Sectors,Type,Attrs,Name
with the preceding partition taking the remainder of the LU.
Does the patch.xml in dragonboard-820c-bootloader-ufs-linux-31 rectify this? It appears to intend to do so

not sure…

Is there any documentation on the xml tag attributes? In particular, what is bHighPriorityLUN?

there is, but it’s not publicly available. bHighPrioirtyLUN, indicates the high priority LU on the UFS device, which I believe is something in the spec to indicate that a LU gets more i/o priority.

#7

What tool is used to generate the gpt_*.bin files in dragonboard-820c-bootloader-ufs-linux-31?

the ‘source’ file is the partition.xml, e.g. partition.xml « linux « dragonboard820c - working/qualcomm/db-boot-tools.git - [no description]

then there is a tool that parses this data, and produces the gpt_*.bin, and also rawprogram and patch xml files. That tool is not open source as far as I am aware… at least not the version we are using these days. however a quick google search will bring an interesting finding … and an old version of the same tool was some time ago released by QCOM here: https://wiki.codeaurora.org/xwiki/bin/download/Snapdragon+Developer+Platforms/WebHome/partitioning_tool.zip. I don’t expect that to ‘work’ now, but it can give you an idea about what it is that they do, assuming you are brave enough to look at this code!

The closest I can get is with fdisk. It is the only tool I know which lets you set the block size of a loop-backed file. However, it insists on starting partitions on 1MB, or 256 block, boundaries. Gdisk will let you change the sector alignment but does not let you change the sector size and loop devices seem to have a sector size of 512. Also, fdisk does not allow you to change the number of partition entries from the default 128. However, both tools do set the correct value of 0x000200 for StartingCHS in the PMBR.

For eMMC , on DB410c, we came up with our own tool for paritioning, which you can find here:
https://git.linaro.org/landing-teams/working/qualcomm/db-boot-tools.git/tree/mksdcard
It reads partition.txt file, not .xml, which you can find in the db410c folder. we haven’t moved to this tool yet for db820c, the main reason being that our tool relies on sgdisk which does not support 4K block size which we would need for UFS… so for 820c, we still rely on the QCOM internal tools. it’s a bit ugly, i am aware.

The rootfs partition should have a type GUID of ‘Linux root (ARM-64)’.
Right. i would take a patch to fix that , now that I gave you all the links!

The partition type GUIDs in the gpt_*.bin files seem to be random, or are they just internal to Qualcomm?
It’s not random, it’s set by QCOM in their own documentation. who knows where/when they are used… we just never dare to change them. It’s not clear if they are used by the ROM code or the XBL proprietary boot code.

#8

Thank you both for your excellent answers.

I have since discovered that the interpretation of the GPT attribute flags is determined by the partition type GUID.
From:
http://www.uefi.org/sites/default/files/resources/UEFI%20Spec%202_7_A%20Sept%206.pdf
page 126, Table 23, Defined GPT Partition Entry - Attributes:
…
Bits 48-63
Reserved for GUID specific use. The use of these bits will vary depending on the PartitionTypeGUID . Only the owner of the PartitionTypeGUID is allowed to modify these bits. They must be preserved if Bits 0–47 are modified.

So it is not surprising that lsblk ignores bit 60, as the type GUID is not one of the recognised linux values defined, for example, here:
https://www.freedesktop.org/wiki/Specifications/DiscoverablePartitionsSpec/
or returned by the l command of fdisk.

@danielt
The UEFI specification, above, only specifies the minimum space to be allotted for partition entries as 16k (page 121, second paragraph).
The default number of entries then follows from the minimum size of an entry of 128 bytes (Table 20. GPT Header, page 123). Unused entries must be set to zero.
However, there would appear to be nothing against setting NumberOfPartitionEntries to less than 128. For example, SizeOfPartitionEntry could be 256 and so only 64 entries would fit into the minimum allotted space.
In fact, in the gpt_*.bin files, NumberOfPartitionEntries is the smallest multiple of 4 greater than or equal to the number of non-zero partition entries. Perhaps this is just a performance optimisation but I was wondering if there was some other reason.

@anon91830841
I am investigating those links you gave :smile:.

#9

I have found some answers in:
https://wiki.codeaurora.org/xwiki/bin/download/Snapdragon+Developer+Platforms/WebHome/partitioning_tool.zip

First, from patch0.xml:
<!–NOTE: This file is used by Trace32 - So make sure to add decimals, i.e. 0x10-10=0, but 0x10-10.=6.–>
This implies to me that names like NUM_DISK_SECTORS could have values that are hex strings but the result must be a decimal string.
If that is the case, then it could be significant even when the value being subtracted is 1 as the result could still be a hex string.
Does anyone know what Trace32 is, in this context? All I can find is a microsoft log viewer and this:
http://www.lauterbach.com/frames.html?home.html.

The file ptool.py contains the code generating the odd start_sector=“NUM_DISK_SECTORS-1” at line 792 so I expect it is an error which has been carried through to later versions of the tool.
The file also contains the error of setting the value of 0x000100 for StartingCHS at line 705.
It is also the source of value for the number of partition entries, at line 741, with the comment:
# Want a multiple of 4 to fill the sector (avoids gdisk warning)
Of course, it does not fill a sector on these chips and is not an optimal I/O size.

#10

@danielt
The UEFI specification, above, only specifies the minimum space to be allotted for partition entries as 16k (page 121, second paragraph).
The default number of entries then follows from the minimum size of an entry of 128 bytes (Table 20. GPT Header, page 123). Unused entries must be set to zero.
However, there would appear to be nothing against setting NumberOfPartitionEntries to less than 128. For example, SizeOfPartitionEntry could be 256 and so only 64 entries would fit into the minimum allotted space.

True.

In fact, in the gpt_*.bin files, NumberOfPartitionEntries is the smallest multiple of 4 greater than or equal to the number of non-zero partition entries. Perhaps this is just a performance optimisation but I was wondering if there was some other reason.

Doesn’t this make the partition table non-conforming? I can’t remember
exactly how many partitons there but I thought it was low single
figures.

We’re gathering info about boot ROM quirks at present (part of the EBBR
effort so if the reason is not
performance or “just a mistake” then it would be good to know it!

#11

I do not think it makes the partition table non-conforming. As far as I can see, the specification is silent on the value of NumberOfPartitionEntries beyond the text in Table 20. GPT Header:
The number of Partition Entries in the GUID Partition Entry array.

Now, just because you have allocated 16 kilobytes of space does not mean you have to use it all. You might want to limit the number of new partitions any tool could create and so set NumberOfPartitionEntries to some number less than 128 or even to the actual number of partitions initially created. PartitionEntryArrayCRC32 is only calculated over the partitions specified by NumberOfPartitionEntries and the only other reference to NumberOfPartitionEntries is in Table 89. Hard Drive Media Device Path, page 326, giving the valid values of a partition number.