Bug 1427849 comment 18 has details in how the repackaging step should actually be implemented.

Just FYI, this is basically the geckodriver-fetch tasks in https://phabricator.services.mozilla.com/D20566. Sorry to clash on this, didn't realize it would get action from two people after months of inactivity! (I want it to be able to distribute some browsertime-related changes.)

Nick, don't worry. Your help on that is very much appreciated! And yes, it's a very surprising overlap. :) So I will clearly have a look at your TC jobs/kinds patch today, and generalize the code. I'm sure that saves me a couple of days given that I don't have to collect all the information in how to implement it from various people.

My steps would be:

1. Get your code working and fix the Gecko decision task for the docker parent failure
2. change the kind files to make use of job defaults (or a template - not sure what the difference actually is)

I'm currently also stumbling to get the geckodriver jobs created. Andrew, can you see why that fails? We have the kind dependencies in place, but as it looks like the build jobs aren't generated/created.

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=10c5b9baa022027710ebf51c363dc4fe74c0caad&selectedJob=229629478

It looks like your commit doesn't have any of the task changes in it.. Did you forget to git add maybe? Tbh, I'm kind of surprised the decision task passed there. We should probably make sure it errors out if the specified task isn't recognized.

Doh! I completely missed that and also didn't notice afterward. Now I pushed a try again but only for the fetch task for now:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2929c7f9a5e06800dbc59a8bf6bd2f1790e103fa

This result looks pretty good, and gives me a starting point to optimize the job definitions.

While checking the results I noticed that on Linux we have very large geckodriver binaries of around 41MB. This is unusual for opt (release) builds. I will investigate that tomorrow.

When asking on IRC for further input Mike raised the question why we don't want to have a toolchain job for it. Basically I don't know and replied with the following requirements:

First task:

• We need a geckodriver binary compiled with cargo build --release or similar; and it doesn't matter that much if it comes from the common test package or if we build on our own
• For now we would only need this task to run for Nightly builds
• The task has to fail if geckodriver Rust tests are failing

Second task:

• The geckodriver binary for Windows has to be signed with the same key as the Firefox binary (see bug 1427849)

Third task:

• Once in a while we want to release geckodriver, and need a way to most likely trigger it manually via run-actions or such
• The binary will be packaged as tar.gz, or zip file with an appropriate file name including the release version
• Maybe some other source files will be included (like changelog)
• If possible push all the artifacts to github to produce a release
  (* if necessary also push the code for the release to the release branch on Github)

Mike, do you still think this can all be done via toolchain tasks? Especially the signing task would need special privileges, so we might keep the trusted chain?

It almost sounds like you want to have your own taskcluster taskgraph on the geckodriver github repo. The only non-obvious thing from there is the signing part, but... does it really have to be signed with the same key?

The main development happens in mozilla-central for already quite a time. As such we finally may want to get rid of the github repository at all, even for releases. But that's not what this bug is about and we would consider for now.

Mike, so if you have an advice for an in-tree task config, and dependency chain please let me know. For me it's unclear if we could get it all done by a toolchain task. If it's not doable I would implement the test package way.

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #12)

The main development happens in mozilla-central for already quite a time. As such we finally may want to get rid of the github repository at all, even for releases. But that's not what this bug is about and we would consider for now.

Mike, so if you have an advice for an in-tree task config, and dependency chain please let me know. For me it's unclear if we could get it all done by a toolchain task. If it's not doable I would implement the test package way.

It could all be done by a toolchain task, but it's frustrating. The task would basically look like a build task: set up the Rust toolchain, use all of the build options for macOS and Linux, etc, and then produce just geckodriver. I did these types of task for Android: android-gradle-dependencies looks like a build task, and android-{lint,test,...} look like build tasks. It's been a huge hassle 'cuz these aren't real build tasks.

I recommend not trying to rework the whole flow right now: use fetch tasks and a combine/release task, like I sketch; stitch in the signing tasks; and later on we can replace the fetch-from-build with de novo geckodriver tasks if and when that's valuable.

If it is such a hassle I agree in not doing that now. I simply didn't know about all that. So thanks for your feedback Nick!

I will most likely get back to this bug by next week when I'm back from PTO.

Here an updated try push for the repack job, which works across platforms, and has some common configs moved to the job defaults.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fece25d5bc00cc0d36fc0db232e3effd2cf73486

I will have to check how to transform the remaining configs so it's more dynamic.

The original patch from Nick also contains a script for collecting all the artifacts and offering those from a single task. For now I will drop this file, and will have a look again once the signing work has been done.

The upcoming patch will run the repackage task for geckodriver for each build and every supported platform. The signing task as to be created on bug 1427849 will then only pick-up those tasks which were built based on a nightly build.

Actually supporting CI and Nightly builds are a bit more complex, and would require me to also use a loader or transforms.

Earlier this morning I talked to Mike Hommey and he proposed that we should better use a toolchain task for building geckodriver. It would be similar to the one for cbindgen, additionally for linux32 and win32. With such a task geckodriver also won't have to be built as part of the build task in the future anymore, which is good given that it's not related to the build at all.

What's unclear at the moment is how the signing task can be hooked up to such a toolchain task. Is that possible, and what would we have to obey when creating the task?

Johan could you help out or refer the question to a person who will know it? Thanks!

At the moment, the signing tasks are linked to the build ones thanks to [1].

To me, the easiest way would be to create a new signing kind (say "geckodriver-signing"). The content of the file should be similar to [1], except this line [2]. Line [2] refers to this file[3]. We should create a similar file to [3], which defines the right routes and more importantly, the right location of the upstream artifacts (i.e.: the relative path to geckodriver of the artifacts exposed by the toolchain task).

Please let me know if you need more information.

[1] https://searchfox.org/mozilla-central/rev/b2d35912da5b2acecb0274eb113777d344ffb75e/taskcluster/ci/build-signing/kind.yml#14
[2] https://searchfox.org/mozilla-central/rev/b2d35912da5b2acecb0274eb113777d344ffb75e/taskcluster/ci/build-signing/kind.yml#9
[3] https://searchfox.org/mozilla-central/rev/b2d35912da5b2acecb0274eb113777d344ffb75e/taskcluster/taskgraph/transforms/build_signing.py

So I assume signing will be fine for geckodriver for both the methods we talked about above? I won't have the time to implement the toolchain task in the next weeks, but could finish the extract task now. So we could land the latter, and work on the toolchain task later, maybe after one of Releng got the signing task implemented. Does that sound ok?

Here a try build which only repacks geckodriver for Nightly builds. Note that this was the original idea for this bug. I will file a follow-up bug to get this task converted into a toolchain build task later.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=acbcccfccb14279f91c715fb23314bdc9d9aa1a2

This task extracts the binary of geckodriver from the
common test package and stores it into its own artifact.
For now this task is only run after Nightly build tasks
on supported platforms..

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #19)

Does that sound ok?

👍

Here a hopefully final try build with the review comments addressed:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4cb7555441c050b83f5ca79fc9667d7a3502a1dd

https://hg.mozilla.org/mozilla-central/rev/eba70db52fa5

https://hg.mozilla.org/integration/autoland/rev/eba70db52fa545d7e16047948e06506ab9ade728