Getting aosp-known-good-manifest to sync

Products Support HiKey 960
#1

Hi there
Just joined and trying to get my hikey960 to run a build of Android PIE.

I’m trying to use the manifest provided on https:||github.com|96boards|aosp-known-good-manifests.git

So my steps are…

executing cmd repo init -u https :||github.com|96boards|aosp-known-good-manifests.git -b master -m latest|hikey960.xml

Get https:||gerrit.googlesource.com|git-repo|clone.bundle
Get https:||gerrit.googlesource.com|git-repo
From git-repo - Git at Google
266f74c…f25a370 master → origin/master
Get https:||github.com|96boards|aosp-known-good-manifests.git
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
curl: (22) The requested URL returned error: 404 Not Found
Server does not provide clone.bundle; ignoring.
From https:||github.com|96boards|aosp-known-good-manifests

  • [new branch] master → origin/master

repo has been initialized in /localdev/jamieson/android-builds/AOSP/master-build

…then followed by the sync command like this…

executing repo sync -j24 -f -m latest/hikey960.xm

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0 % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0 % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0 % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0 % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
curl: (22) The requested URL returned error: 404 Not Found
Server does not provide clone.bundle; ignoring.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0 % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0 % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
curl: (22) The requested URL returned error: 404 Not Found
Server does not provide clone.bundle; ignoring.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0 % Total % Received % Xferd Average Speed Time Time Time Current

Can anyone advise what I am missing?

FYI the trace continued on with repeated 404 messages on every repository it tried to access but obviously I didn’t want to copy them

Any help much appreciated

regards
Craig

#2

Its pretty clear that you didn’t understand the instructions.

In point 1 of the repo’s readme:
“Also review the instruction details specifically for devboards: Using Reference Boards  |  Android Open Source Project”

And if you read THAT LINK, it says to use:

repo init -u https://android.googlesource.com/platform/manifest -b master

And THEN you follow step 2 and 3, which is to download the known-good-manifest into .repo/manifests/ (very specific location, do not change it)

AND FINALLY, you repo sync -m it.

1 Like
#3

Thanks for your reply, I’ll try the steps as specified.

Wrongly I thought I could merge the two steps.

#4

Hi,

I have an issue matching this thread. I try to sync AOSP Pie (sort of) for HiKey960 using known-good manifests.
Here’s what I’m doing:
repo init -u https://android.googlesource.com/platform/manifest -b master

wget https://raw.githubusercontent.com/96boards/aosp-known-good-manifests/master/known-good/hikey960/2018-09-25_15%3A33%3A22-pinned-manifest.xml -O .repo/manifests/20180925_153322-pinned-manifest.xml

repo sync -j24 -m 20180925_153322-pinned-manifest.xml

I get the following errors for various git repos:
curl: (22) The requested URL returned error: 404
Server does not provide clone.bundle; ignoring.

I read somewhere in the Internet that I should not bother with it, so I don’t. But later on it asks for authentication via:
android.googlesource.com/new-password (probably hit quota on my IP as indicated here: Downloading the Source  |  Android Open Source Project )
Even if I setup the account and generate .gitcookies it says the following:

fatal: failed to unpack tree object HEAD
error: Cannot checkout platform/vendor/aosp/google_devices/marlin: GitError: cannot initialize work tree for platform/vendor/aosp/google_devices/marlin
atform/test/vti/dashboard

and then:

Checking out project platform/vendor/aosp/google_devices/marlin
Traceback (most recent call last):
File “/home/aosp_pie/.repo/repo/main.py”, line 572, in
_Main(sys.argv[1:])
File “/home/aosp_pie/.repo/repo/main.py”, line 547, in _Main
result = run()
File “/home/aosp_pie/.repo/repo/main.py”, line 540, in
run = lambda: repo._Run(name, gopts, argv) or 0
File “/home/aosp_pie/.repo/repo/main.py”, line 201, in _Run
result = cmd.Execute(copts, cargs)
File “/home/aosp_pie/.repo/repo/subcmds/sync.py”, line 997, in Execute
self._Checkout(all_projects, opt)
File “/home/aosp_pie/.repo/repo/subcmds/sync.py”, line 554, in _Checkout
self._CheckoutWorker(**kwargs)
File “/home/aosp_pie/.repo/repo/subcmds/sync.py”, line 439, in _CheckoutWorker
return self._CheckoutOne(opt, project, *args, **kwargs)
File “/home/aosp_pie/.repo/repo/subcmds/sync.py”, line 474, in _CheckoutOne
project.Sync_LocalHalf(syncbuf, force_sync=opt.force_sync)
File “/home/aosp_pie/.repo/repo/project.py”, line 1492, in Sync_LocalHalf
self._InitWorkTree(force_sync=force_sync, submodules=submodules)
File “/home/aosp_pie/.repo/repo/project.py”, line 2719, in _InitWorkTree
raise GitError("cannot initialize work tree for " + self.name)
error.GitError: cannot initialize work tree for platform/vendor/aosp/google_devices/marlin

If I switch to non-parallel sync:
repo sync -j1 -m 20180925_153322-pinned-manifest.xml
I get the following

Fetching project platform/vendor/aosp/qcom/taimen
fatal: remote error: PERMISSION_DENIED The caller does not have permission

Strangely, these issues appears for platform/vendor/… repositories. The last known good manifest does work, I’m able to sync repositories without any issues and extra work around authentication.
Are the git repositories pointed by older known-good manifests are broken or is it still something related to my setup? Has anyone experience something similar?

#5

Ok, this answer from @doitright sheds some light why I’m having all these troubles: How to Build AOSP Android O - #18 by doitright

So is it expected to be brought back to functioning or the restructuring will turn known good into known no good manifests and we should stop bothering with it?

#6

The idea behind known good manifests isn’t to give you a permanent view into the past. While they can sometimes be helpful for this purpose, the reality is that the known good manifests are there to help you overcome TEMPORARY breakage of the master branch.

Because the master branch is the master branch, it is susceptible to being broken at ANY MOMENT. Having a recent known good manifest helps you by giving you something that has been, to some degree, tested. I.e., it builds, it boots, and it seems to work.

Don’t expect the ancient manifests to ever be maintained. Those that won’t build, never will.

#7

Well, if there’s a list of “known good” manifests and they are even recommended to be used then the expectations are already set. Especially if it’s recommeded approach and there is no single mention that they might be not working. Now you say I shouldn’t be expecting all of them working. You confuse me :slight_smile:
If you change something (vide “repository restructuring”) then a good practice would be to let people know that there are changes introduced and some “known good” may be not working anymore. Otherwise you doitwrong.

Now that’s a quite interesting definition of ancient. I took a known good manifest from a year ago so I could expect it does built and works in HW but I’m unable even to sync it correctly.
I don’t expect them to be maintained so that they always work. I expect them to be not listed among known good while they are not good.

It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.

#8

Your expectations are unreasonable.
Further, it is completely inappropriate for you to be taking out your frustrations on me.

#9

So… the known good manifests do two things.

  1. Via the git SHA-1 references they give a permanent cryptographic record of exactly what build is known to be good.
  2. They point you at the servers where, at the time the manifest was generated, it was possible to fetch the code.

The use of git references (#1) is, as I said, permanent and can be used for regression hunting or testing by people who sync frequenctly. Unfortunately if servers (in this case, third party serves at Google) are reconfigured then #2 may no longer be particularly useful: manual changes might be needed to find a server that still serves the content and in some cases perhaps no public server still provides the content.

#10

Your expectations are unreasonable.

I don’t think so :slight_smile:

#11

@danielt, thanks, I’m aware of how the manifests work. I know they are dependent on 3rd party servers. What I don’t know (because I’m not engaged with AOSP all the time) is that there are some changes impacting repositories introduced, like mentioned restructuring. You guys apparently are in the know. I’m not. Where did you learned about it from? If I knew it from the very begining I wouldn’t spent effort on investigations of things that used to work a couple of months ago, but they don’t now.
That’s why I count on those in the know to warn me about potential issues - this is how I imagine a healthy community would work. @doitright, is that also unreasonable?

#12

What is reasonable, is if you don’t know something, then you ASK.

Politely.

#13

Me? To be honest I learned about it this morning by reading this topic!

Even now I’m still deciding if/whether there is anything we should change w.r.t. documentation. This kind of problem is, unfortunately, “normal” with pinned manifests which by their nature document the historic state of “the internet”. I haven’t tried any old pinned manifests for AOSP recently but often a little hand hacking can bring them back to life (which is why I’m not sure removing old ones is proportionate).

#14

The real reason why we are having this discussion now is that some folks (including me) would like to work on release branches which are unsupported. The only resonable (this seems to be a word-sponsor of this thread :wink: ) way, as you recommended is to travel back in time to find something that is as close to release branch as possible. I assumed that if there’s a list of known good manifests then all of the listed manifests would work. There’s no explicit note that they might not work (or at least I’m not able to find such a note). So my expectation is set accordingly. And that’s why I spent too much effort on investigations because I believed it must work.
I didn’t say anything about removing stuff, but rather making people aware that even known good might be not reliable due to other dependencies (I’m coming from a background where dissapearing repositories without any note are uncommon). I learned it the hard way, the others don’t have to.

#15

I guess the problem is that we don’t know when, where or what will break so generic disclaimers that stuff bitrots would be vague to the point of being useless. Ultimately the known good manifests remain the best tool available to people who want to explore history.

Certainly if you or anyone else figures out how best to mend them to make them downloadable again then we’d want to document that!