Rough outline - Task generation for partner repacks: - pull in partner configs when running the action task (r/o github creds via TC secret service, with graceful fallback for unscoped and testing) - create tasks as necessary per platform: partner repack, signing, repackage, repackage-signing We can start with EME-free builds to prove this out with all public artifacts.

This script uses the Github API (v4, GraphQL) to grab the repack manifest from the specified repo, parse it for partners, and then walk those repos to retrieve all repack.cfg files. It works for both public and private manifests (given sufficient privs on the Github Personal Access Token (PAT) in the latter case). We'd likely have a token in the TC secret service (this is not provisioned yet). Example output for my modified mozilla-EME-free setup: 2018-03-02 22:36:00,088 - DEBUG - __main__.<module>#146: Found configs: { "mozilla-EME-free:dummy": "foo bar baz\n", "mozilla-EME-free:mozilla-EME-free": "aus=\"mozilla-EMEfree\"\ndist_id=\"mozilla-EMEfree\"\ndist_version=\"1.0\"\nlinux-i686=true\nlinux-x86_64=true\nlocales=\"ach af de en-US\"\nmac=true\nwin32=true\nwin64=true\noutput_dir=\"%(platform)s-EME-free/%(locale)s\"\n\n# Upload params\nbucket=\"net-mozaws-prod-delivery-firefox\"\nupload_to_candidates=true\n" } You can see it's not parsing the repack.cfg content yet, which we'll need to do to pick out upload_to_candidates, and possibly bucket & output_dir for beetmover config; and maybe enabled platforms for the graph gen. The idea with this approach is to avoid requirements for git and repo at graph generation time. Only need requests, which should already be available given what we do in nightly action tasks on m-c to look up data for partial generation.

I've been looking into the details of generating the tasks we'll need for this a bit. My idea is to make as much use of the existing transforms for signing, repackage, and repackage-signing as possible. My hope is that if we can get the right bits fed into signing, the downstream jobs should mostly "just work". Some random notes: * We'll almost certainly need new kinds for each of the above steps, because the upstream dependencies start with release-partner-repack, not builds or repacks. * Some of these steps are no-ops, depending on platform. Linux does not need to run any of them AFAIK, Mac only needs signing & repackage, while Windows only needs repackage & repackage-signing. All platforms need beetmover, which sounds like will be handled elsewhere. * We may need some hacks to remove these jobs from Treeherder. The current "release_deps" transform requires each side of a dependency to be in the same treeherder "tier", and the existing signing, repackage, etc. transforms set tier 1 (which is forcing me to set partner repacks to tier 1, at the moment).

I've gotten as far as I can on this for the week. I've managed to get signing tasks generated with some tweaks to transforms that look correct-ish, specifically: * Not setting any upstreamArtifacts for Windows & Linux. Dunno if this will work. We might need to modify the signing tasks to bail on this, or somehow remove them from the graph entirely. (I think the latter will be tough, because of the way kind dependencies/release_deps works.) * Set multiple upstreamArtifacts for Mac. This is hardcoded in the transform right now, but based on what Nick has said, I think this will end up coming from a loop over something in config.params. * Set some routes. This probably need to change once we figure out private artifacts. I've not figured out how the heck I can test this yet - I think I'll need the partner repack tasks to actually produce artifacts, and ideally - some way to run partner repacks + downstreams without running an entire release. https://github.com/mozilla/gecko-dev/compare/master...mozbhearsum:partner-repack-scheduling?expand=1 is my current code, which needs a ton of refactoring and hack removal.

(In reply to Ben Hearsum (:bhearsum) from comment #3) > https://github.com/mozilla/gecko-dev/compare/master...mozbhearsum:partner- > repack-scheduling?expand=1 is my current code, which needs a ton of > refactoring and hack removal. I made a bunch of progress on this in the past week. I've now got a graph that includes (and probably fans out for each partner+locale): * partner repack * signing * repackage * repackage signing * beetmover I've done a lot of checks of the output of signing, repackage, and repackage signing - and I think they're in pretty shape. Beetmover is currently only running for Windows, and is currently busted. More work is needed to get the Linux and Mac jobs scheduled, and it looks like some changes to the beetmover mainfest are needed to make them green.

(In reply to Ben Hearsum (:bhearsum) from comment #4) > (In reply to Ben Hearsum (:bhearsum) from comment #3) > > https://github.com/mozilla/gecko-dev/compare/master...mozbhearsum:partner- > > repack-scheduling?expand=1 is my current code, which needs a ton of > > refactoring and hack removal. > > I made a bunch of progress on this in the past week. I've now got a graph > that includes (and probably fans out for each partner+locale): > * partner repack > * signing > * repackage > * repackage signing > * beetmover > > I've done a lot of checks of the output of signing, repackage, and repackage > signing - and I think they're in pretty shape. Beetmover is currently only > running for Windows, and is currently busted. More work is needed to get the > Linux and Mac jobs scheduled, and it looks like some changes to the > beetmover mainfest are needed to make them green. I've now got what I think to be a graph that's shaped properly. Beetmover is still failing, but I think it's time to get real feedback on this work before proceeding any further. I'll post a series of patches shortly.

If we need jamun releases soon for esr, we probably want to skip partner configs until that code is merged in.

We may want to enable tc partner repacks for beta tomorrow...

Comment on attachment 8969103 [details] [diff] [review] py3.diff https://hg.mozilla.org/build/tools/rev/3b706c033d48107098936703dd3545c009fcffe6

Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/mozilla-inbound/rev/2e43f41eb784 Add max-run-time to signing workers; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/fc930141faf9 Propogate shipping-product/phase in signing; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/e2a4ff5c10f6 Allow forcing artifact access to be via private URLs; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/dd7db84b4f11 [partner-repack] Add parameters needed for doing partner repacks; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/90fddb648f30 [partner-repack] Don't display partner repacks on treeherder; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/991f58ef782b [partner-repack] Add docker image for partner repacks; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/00b9d353a6b7 [partner-repack] Add repack partner builds; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/3773a90e8607 [partner-repack] Sign partner builds; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/e5b269778841 [partner-repack] Repackage signed partner builds; r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/e4aa73efed4d [partner-repack] Beetmove the partner repacks; r=Callek

Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/mozilla-inbound/rev/355bcc77b538 [partner-repack] Sign repacked partner builds; r=Callek CLOSED TREE

https://hg.mozilla.org/mozilla-central/rev/2e43f41eb784 https://hg.mozilla.org/mozilla-central/rev/fc930141faf9 https://hg.mozilla.org/mozilla-central/rev/e2a4ff5c10f6 https://hg.mozilla.org/mozilla-central/rev/dd7db84b4f11 https://hg.mozilla.org/mozilla-central/rev/90fddb648f30 https://hg.mozilla.org/mozilla-central/rev/991f58ef782b https://hg.mozilla.org/mozilla-central/rev/00b9d353a6b7 https://hg.mozilla.org/mozilla-central/rev/3773a90e8607 https://hg.mozilla.org/mozilla-central/rev/e5b269778841 https://hg.mozilla.org/mozilla-central/rev/e4aa73efed4d https://hg.mozilla.org/mozilla-central/rev/355bcc77b538