Implement an optimization so tasks only run when their dependencies are scheduled
Categories
(Firefox Build System :: Task Configuration, enhancement, P2)
Tracking
(firefox82 fixed)
Tracking | Status | |
---|---|---|
firefox82 | --- | fixed |
People
(Reporter: ahal, Assigned: ahal)
References
Details
Attachments
(8 files, 1 obsolete file)
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details |
We have a smattering of tasks such as build-signing
, upload-symbols
, etc which we typically only want to run when their associated build is explicitly scheduled.
E.g, if we optimize the linux64/opt
away on autoland, we don't want linux64/opt-upload-symbols
being the cause of bringing it back. Historically we've gotten around this by copying the dependencies' optimization over to the current task, e.g:
https://searchfox.org/mozilla-central/rev/50cb0892948fb4291b9a6b1b30122100ec7d4ef2/taskcluster/taskgraph/transforms/upload_symbols.py#78
Though this doesn't work if the optimization specified on the dependency relies on task specific metadata (like the label). This is specifically an issue with the bugbug
based optimizers (but can theoretically happen with other ones as well).
Tom had started work on an optimizer that handles this properly, and optimizes a task away if its dependencies were also optimized over in:
https://phabricator.services.mozilla.com/D80508
I'm going to take it over and finish it off.
Assignee | ||
Comment 1•4 years ago
|
||
I got tripped up by the docs for Graph.visit_preorder/visit_postorder which
talks about nodes linking to and from other nodes, but makes no mention of
which direction the linking happens practically (in my mind I thought "linking
to" meant going from task -> dependent).
This blurb would have helped me catch my mistake.
Assignee | ||
Comment 2•4 years ago
|
||
This is needed as future revisions will refactor the optimization process to be
two pass. So we can't be sure if a task is optimized or not until the very end.
Depends on D87771
Assignee | ||
Comment 3•4 years ago
|
||
This test takes a long time and is annoying when it's included by default.
Depends on D87772
Assignee | ||
Comment 4•4 years ago
|
||
Assignee | ||
Comment 5•4 years ago
|
||
Depends on D80508
Assignee | ||
Comment 6•4 years ago
|
||
Depends on D87775
Updated•4 years ago
|
Updated•4 years ago
|
Updated•4 years ago
|
Assignee | ||
Updated•4 years ago
|
Comment 8•4 years ago
|
||
bugherder |
Updated•4 years ago
|
Updated•4 years ago
|
Updated•4 years ago
|
Assignee | ||
Comment 9•4 years ago
|
||
Assignee | ||
Comment 10•4 years ago
|
||
This removes boiler plate around adding new test cases.
Depends on D88482
Assignee | ||
Comment 11•4 years ago
|
||
Depends on D88483
Updated•4 years ago
|
Comment 12•4 years ago
|
||
Comment 13•4 years ago
|
||
bugherder |
Assignee | ||
Updated•4 years ago
|
Updated•4 years ago
|
Description
•