I have already noticed this Friday last week and talked to Chris, but then got it accidentally working. So no further action has been taken. But since morning I cannot build artifact builds on mozilla-central for at least OS X anymore because of: 0:17.27 Attempting to find a pushhead containing c3e5497cff1c995821b1c9320fa71f1ef9a8c30e on mozilla-central. 0:17.27 Attempting to find a pushhead containing c3e5497cff1c995821b1c9320fa71f1ef9a8c30e on integration/mozilla-inbound. 0:17.27 Attempting to find a pushhead containing c3e5497cff1c995821b1c9320fa71f1ef9a8c30e on releases/mozilla-beta. 0:18.26 Retrieving the last 50 pushheads starting with id 90002 on integration/mozilla-inbound 0:18.26 Retrieving the last 50 pushheads starting with id 31776 on mozilla-central 0:59.02 Tried 53 pushheads, no built artifacts found. 0:59.06 make[4]: *** [recurse_artifact] Error 1 0:59.06 make[3]: *** [artifact] Error 2 0:59.06 make[2]: *** [default] Error 2 0:59.06 make[1]: *** [realbuild] Error 2 0:59.06 make: *** [build] Error 2 This is a merge commit: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=c3e5497cff1c995821b1c9320fa71f1ef9a8c30e&filter-tier=1&filter-tier=2&filter-tier=3 The merge commit done right before works for me: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=22aaf8bad4df8e5f8c976f1521c213cb37e2dff5&filter-tier=1&filter-tier=2&filter-tier=3 But it installs from a remote pushhead from May 5th, which is 5 days old now: 0:03.76 Attempting to find a pushhead containing 22aaf8bad4df8e5f8c976f1521c213cb37e2dff5 on mozilla-central. 0:03.76 Attempting to find a pushhead containing 22aaf8bad4df8e5f8c976f1521c213cb37e2dff5 on integration/mozilla-inbound. 0:03.76 Attempting to find a pushhead containing 22aaf8bad4df8e5f8c976f1521c213cb37e2dff5 on releases/mozilla-beta. 0:04.81 Retrieving the last 50 pushheads starting with id 90002 on integration/mozilla-inbound 0:04.81 Retrieving the last 50 pushheads starting with id 31775 on mozilla-central 0:21.56 Installing from remote pushhead 33b92d9c40562dab3d7b602368c75619f1d793f7 on mozilla-central For me it happens for debug builds while Mark is trying to build opt.

There are two issues here: one is we stopped producing debug os x buildbot builds, so the artifact code needs to update the artifacts it looks for, and the second, which seems like a real bug, is that we appear to have stopped indexing all the relevant artifacts for buildbot os x opt builds in the taskcluster index, see for instance https://tools.taskcluster.net/task-inspector/#aVkiFaQuRIaVccK5lldcnA/0

This looks like fallout from Bug 1350413. The first build with that ticket has Mac OS X 10.7 opt builds that don't index any artifacts; the tc Tier 2 build that indexed target.dmg is gone: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=0b255199db9d6a6f189b89b7906f99155bde3726 Compare to the push before, where there is a tc Tier 2 build indexing target.dmg: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=6d4525b5aaa834ea4db8d8eab8087d86e34cf57a&selectedJob=96615068

My comment mid-aired with chmanchester's and gives details about his second (real bug). wcosta, dustin: can you folks dig into the indexing and figure out how to get Mac OS X opt build artifacts back into the Task Cluster index?

Comment 4 restores my ability to do debug artifact builds locally.

Comment on attachment 8865569 [details] Bug 1362949 - Update artifact build code to accept buildbot or taskcluster style artifact names for OS X. https://reviewboard.mozilla.org/r/137192/#review140242 Makes sense.

Does that patch answer the question? Anyway, I think Wander has all the state here.

Pushed by cmanchester@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/b230bcb04a19 Update artifact build code to accept buildbot or taskcluster style artifact names for OS X. r=nalexander

Is my help still needed here?

Pushed by kwierso@gmail.com: https://hg.mozilla.org/mozilla-central/rev/e0955584782e Update artifact build code to accept buildbot or taskcluster style artifact names for OS X. r=nalexander a=merge

(In reply to Wander Lairson Costa [:wcosta] from comment #9) > Is my help still needed here? I think so: chmanchester's patch restores Mac OS X debug builds. But we don't have any Mac OS X opt builds in the Task Cluster index at this time (that Chris or I can find). So we need you to either find them for us or get them re-instated in the index, ASAP.

This is only half fixed. Reopening to avoid confusion.

The macosx64-opt build task executes through buildbot-bridge, and there is a specific task in which artifacts are uploaded.

Pushed by cmanchester@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/9aacb2f027d2 Fix macosx64-opt index routes. r=dustin

Pushed by archaeopteryx@coole-files.de: https://hg.mozilla.org/mozilla-central/rev/b9bd73036a92 Fix macosx64-opt index routes. r=dustin a=bustage-fix