Closed
Bug 1461980
Opened 6 years ago
Closed 6 years ago
[run-task] Add ability to use artifacts from parent tasks
Categories
(Firefox Build System :: Task Configuration, task)
Tracking
(firefox62 fixed)
RESOLVED
FIXED
mozilla62
Tracking | Status | |
---|---|---|
firefox62 | --- | fixed |
People
(Reporter: ahal, Assigned: ahal)
References
Details
Attachments
(2 files)
When creating new test suites and tasks, I like to try and avoid mozharness/test packaging in favour of mach/running from the srcdir.
The latter approach is difficult to accomplish if the new task depends on any build artifacts. The mochitest/reftest selftest tasks do this, but it's very crude and not easy to copy over to other types of tasks. Basically all taskgraph does is set an environment variable with the url to the artifact, then there is code in the tasks themselves that will download/set it up.
I'd like a generic way for a task to say "I need these artifacts from this dependency". Then I'd like the run-task script to automatically download (and possibly extract?) them into a known location.
I believe something like this would be a big step towards not needing mozharness anymore.
Comment 1•6 years ago
|
||
This might be a nice intermediate step toward gps's aspiration to have the worker do these downloads for you..
Assignee | ||
Comment 2•6 years ago
|
||
Ooh, that would be even better! But yeah, unless that proposal is imminent it's probably worthwhile having something like this in the meantime. I have some WIP patches going through try atm.
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Comment 5•6 years ago
|
||
mozreview-review |
Comment on attachment 8977041 [details]
Bug 1461980 - [taskgraph] Add ability to download artifacts from dependencies in run-task script,
https://reviewboard.mozilla.org/r/244670/#review251516
::: taskcluster/taskgraph/transforms/job/run_task.py:34
(Diff revision 1)
> # if true, perform a checkout of a comm-central based branch inside the
> # gecko checkout
> Required('comm-checkout'): bool,
>
> + # maps a dependency to a list of artifact names to use from that dependency.
> + # E.g: {"build": ["target.tar.bz2"]}
It might be worth mentioning here where these will be downloaded.
Attachment #8977041 -
Flags: review?(dustin) → review+
Comment 6•6 years ago
|
||
mozreview-review |
Comment on attachment 8977042 [details]
Bug 1461980 - [taskgraph] Use run-task's 'use-artifacts' key to setup artifacts in mochitest and reftest selftests,
https://reviewboard.mozilla.org/r/244672/#review251524
Attachment #8977042 -
Flags: review?(dustin) → review+
Comment hidden (mozreview-request) |
Comment hidden (mozreview-request) |
Assignee | ||
Comment 9•6 years ago
|
||
Comment 10•6 years ago
|
||
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/aceeafcc22c9
[taskgraph] Add ability to download artifacts from dependencies in run-task script, r=dustin
https://hg.mozilla.org/integration/autoland/rev/bbeba6126822
[taskgraph] Use run-task's 'use-artifacts' key to setup artifacts in mochitest and reftest selftests, r=dustin
Comment 11•6 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/aceeafcc22c9
https://hg.mozilla.org/mozilla-central/rev/bbeba6126822
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
status-firefox62:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
Comment 12•6 years ago
|
||
Hah - I had kind of already written these patches. Although I never submitted them for review. That's on me.
The main difference from what I wrote is that I had support for validating the SHA-256 of artifacts. And I think I had support for extracting archives. We should be able to incorporate that into this scheme as follow-ups easily enough though.
Thanks for implementing this. It's a *really* useful feature. It will likely be replaced by features in the workers themselves. But until that day arrives, this bridges the gap quite nicely.
Updated•6 years ago
|
Version: Version 3 → 3 Branch
You need to log in
before you can comment on or make changes to this bug.
Description
•