Open Bug 1608834 Opened 5 years ago Updated 2 years ago

Enable chunking in the taskgraph for all mochitest suites

Categories

(Firefox Build System :: Task Configuration, task, P2)

task

Tracking

(Not tracked)

People

(Reporter: ahal, Unassigned)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

+++ This bug was initially created as a clone of Bug #1608833 +++

In bug 1583353 we started chunking test suites in the taskgraph (rather than at test runtime). Though I only enabled it for a subset of suites.

This bug tracks getting the rest of the mochitest suites enabled. To enable taskgraph chunking, remove the suite from this config:
https://searchfox.org/mozilla-central/rev/dd1dafd5c9c05640e76af30b58749076e0199704/taskcluster/taskgraph/transforms/tests.py#1280

I normally like to test that we still run the same set of tests before and after. To do this I:

  1. Push to try scheduling all tests in the suite, both with and without this change.
  2. Clone https://github.com/mozilla/ci-recipes and follow installation instructions.
  3. Run poetry run adr compare_pushes --branch try -r1 <rev1> -r2 <rev2>
  4. It will take awhile (~10-20min).
  5. Make sure the output says all tasks are the same. If the output shows one task ran more manifests than another, then we need to investigate.

The remaining mochitest suites weren't enabled simply because I never got around to them. They might be trivial. Though browser-chrome fission mochitests seemed to fail intermittently a bit more frequently than before. A bit of care will be needed with those.

Assignee: nobody → ahal
Keywords: leave-open
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/a80ee2878a79 [ci] Chunk mochitest-a11y and plain in the taskgraph, r=gbrown

Just noticed that all four mochitest-webgl* suites are entirely defined inside a single manifest:
https://searchfox.org/mozilla-central/source/dom/canvas/test/webgl-conf/generated-mochitest.ini

Taskgraph chunking means we start chunkng by manifests, therefore the maximum number of chunks is <= the number of manifests. Therefore we'll either need to:

  1. Reduce all these suites to a single chunk (looks like ~60min combined runtime).
  2. Generate several manifests, one per chunk we desire.
  3. Implement special logic for this suite, to group them by some metric other than manifests (might need to do this for WPT anyway).
  4. Leave them in the BLACKLIST indefinitely.
Pushed by ahalberstadt@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/60e82c1bd6b1 [taskgraph] Enable taskgraph chunking for mochitest-browser-chrome, r=jmaher

The leave-open keyword is there and there is no activity for 6 months.
:ahal, maybe it's time to close this bug?

Flags: needinfo?(ahal)

We still need to support mochitest-webgl here.

Flags: needinfo?(ahal)
Assignee: ahal → nobody

The leave-open keyword is there and there is no activity for 6 months.
:ahal, maybe it's time to close this bug?

Flags: needinfo?(ahal)

Still a few flavors left, at least mochitest-a11y, possibly a couple others.

Flags: needinfo?(ahal)
Keywords: leave-open
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: