Variants cover runtime configuration we set in tasks that tweak how Firefox behaves (typically prefs). We currently define them in a big JSON object in transforms/tests.py, but it's getting unwieldy to maintain and use.

In this bug I plan to:

1. Tests
2. Move definition to a yaml file
3. Create documentation (and checks to ensure it stays up to date)
4. Force an owner to be listed
5. Allow composing variants (e.g, variants: ["fission", "webgl-ipc", "fission+webgl-ipc"])
6. Possibly pull the logic to a new transform file?
7. ???

This makes them easier to find and inspect.

Depends on D129826

This ensures the filters stay alongside the variant definitions. I called the
key 'when' as 'filter' is ambiguous about whether the tasks that match are
being included or excluded.

Depends on D129827

Ensures variants all conform to a well defined schema and don't
contain any extra or unexpected data.

Depends on D129828

This also splits the blurb for the 'test' kind into a new file as it was
getting too long for kinds.rst. Having documentation for every kind in the same
file tends to lead towards uninformative documentation.

Depends on D129829

Depends on D129830

This makes things less confusing as we no longer sneakily override the value
defined in the test configs.

Depends on D129831

This is identical to the 'webgl-ipc' variant and only used in one place.

Depends on D129832

Listing contacts next to the variant will provide a point of contact in case
there are questions around cost / capacity or relative importance of a given
variant.

Often we have two variants that we want to test together. The only way to do
this currently is to create a 3rd variant that is the sum of the previous two.

This feature will allow tasks to define multiple variants using the '+' token.
E.g:

variants: ["fission", "webgl-ipc", "fission+webgl-ipc"]

The third variant there will first apply the 'fission' configuration, then the
'webgl-ipc' configuration (in order).

This replaces the following variants with composites:
webrender-sw-a11y-checks
webrender-sw-fission
webrender-sw-wayland

The full task graph is identical after this change.

Depends on D130399

https://hg.mozilla.org/mozilla-central/rev/98a659f60686
https://hg.mozilla.org/mozilla-central/rev/e38b571d248d
https://hg.mozilla.org/mozilla-central/rev/18eec84f404b
https://hg.mozilla.org/mozilla-central/rev/e1d2518592d4
https://hg.mozilla.org/mozilla-central/rev/7147d8490b9c
https://hg.mozilla.org/mozilla-central/rev/44bf02d34aaf
https://hg.mozilla.org/mozilla-central/rev/070efa174493
https://hg.mozilla.org/mozilla-central/rev/6b374ffe6878
https://hg.mozilla.org/mozilla-central/rev/b7ad64578b84

(In reply to Cristian Tuns from comment #13)

https://hg.mozilla.org/mozilla-central/rev/98a659f60686
https://hg.mozilla.org/mozilla-central/rev/e38b571d248d
https://hg.mozilla.org/mozilla-central/rev/18eec84f404b
https://hg.mozilla.org/mozilla-central/rev/e1d2518592d4
https://hg.mozilla.org/mozilla-central/rev/7147d8490b9c
https://hg.mozilla.org/mozilla-central/rev/44bf02d34aaf
https://hg.mozilla.org/mozilla-central/rev/070efa174493
https://hg.mozilla.org/mozilla-central/rev/6b374ffe6878
https://hg.mozilla.org/mozilla-central/rev/b7ad64578b84

Backed out for breaking local Windows builds (bug 1739520).
https://hg.mozilla.org/mozilla-central/rev/47f2f2add0cae4cb58825876f635b39d4dfc4839

Thanks for the backout, sorry about that!

Looks like it just needs to be joined to gecko_taskgraph.GECKO.

https://hg.mozilla.org/mozilla-central/rev/8e4b3a24e96a
https://hg.mozilla.org/mozilla-central/rev/de45d5de31dd
https://hg.mozilla.org/mozilla-central/rev/906cce4fe844
https://hg.mozilla.org/mozilla-central/rev/4e7fba3ff6ee
https://hg.mozilla.org/mozilla-central/rev/2c7aa54e0414
https://hg.mozilla.org/mozilla-central/rev/baedbe12cb81
https://hg.mozilla.org/mozilla-central/rev/9e7d1bac29a6
https://hg.mozilla.org/mozilla-central/rev/11a908ecd5ae
https://hg.mozilla.org/mozilla-central/rev/c5d8a5c91ddd
https://hg.mozilla.org/mozilla-central/rev/829097878a30
https://hg.mozilla.org/mozilla-central/rev/3db46303ab27