Closed
Bug 1352477
Opened 8 years ago
Closed 8 years ago
taskgraph ignores run-on-project for fennec-nightlies
Categories
(Release Engineering :: Release Automation: Other, defect, P1)
Release Engineering
Release Automation: Other
Tracking
(firefox55 fixed)
RESOLVED
FIXED
Tracking | Status | |
---|---|---|
firefox55 | --- | fixed |
People
(Reporter: jlorenzo, Assigned: jlorenzo)
References
Details
Attachments
(1 file)
Bug 1317783 showed that even though push-apk specifies "run-on-project" to exclude mozilla-central, it does get included in the central nightly graph.
Comment hidden (mozreview-request) |
Comment 2•8 years ago
|
||
Comment on attachment 8853461 [details]
Bug 1352477 - taskgraph ignores run-on-project for fennec-nightlies
Clearing review flag for now... I provided comments in IRC.
Attachment #8853461 -
Flags: review?(aki)
Assignee | ||
Updated•8 years ago
|
Updated•8 years ago
|
Priority: -- → P1
Comment hidden (mozreview-request) |
Assignee | ||
Comment 4•8 years ago
|
||
This patch makes sure there is no new job created/deleted, with the exception of jamun, where:
> build-linux64-nightly/opt-upload-symbols
> build-linux-nightly/opt-upload-symbols
are now part of the regular jamun decision task. I didn't added them on purpose, initially. However as jamun should be par with beta, I believe we can keep them.
TESTING PROTOCOL
1. Download all the parameters.yml from either treeherder (like [1]), or from the hooks[2]. In my case, I named them parameters-branch-nightly-platform.yml
2. Before the patch is applied, run:
> for name in $(find . -name 'parameters-*.yml' | cut -d'-' -f'2-' | cut -d'.' -f1); do echo "$name" ; ./mach taskgraph target -p parameters-$name.yml > $name-before.list ; done
This generates the task lists for every parameters file. You may need to tweak the files from beta/release/jamun because they are outdated. You may also need:
> export BUILD_NUMBER=1
3. Apply the patch
4. Run:
> for name in $(find . -name 'parameters-*.yml' | cut -d'-' -f'2-' | cut -d'.' -f1); do echo "$name" ; ./mach taskgraph target -p parameters-$name.yml > $name-after.list ; done
5. Concatenate the differences between before and after:
> rm -f concatenated.diff && for name in $(find . -name 'parameters-*.yml' | cut -d'-' -f'2-' | cut -d'.' -f1); do echo $name >> meta.diff && diff <(sort $name-before.list) <(sort $name-after.list) >> concatenated.diff ; done
6. Read concatenated.diff.
In my case, I have 16 different parameters files, which are:
* aurora
* aurora-nightly-desktop
* aurora-nightly-fennec
* beta
* beta-candidate-fennec
* central
* central-nightly-desktop
* central-nightly-fennec
* date
* date-nightly-desktop
* date-nightly-fennec
* inbound
* jamun
* jamun-candidate-fennec
* release
* try
Like said at the beginning, the only difference spotted was in jamun. All the other remained intact.
As a follow up, we need to hardcode the list of expected tasks in an integration test. I haven't looked into it yet.
[1] https://treeherder.mozilla.org/#/jobs?repo=date
[2] https://tools.taskcluster.net/hooks/#project-releng
Assignee | ||
Comment 5•8 years ago
|
||
(In reply to Johan Lorenzo [:jlorenzo] from comment #4)
I also tested this patch on top of bug 1317783, where I could see the push-apk tasks (and only these ones) being filtered out of date-nightly, jamun-candidate, and more importantly: central-nightly.
Comment 6•8 years ago
|
||
mozreview-review |
Comment on attachment 8853461 [details]
Bug 1352477 - taskgraph ignores run-on-project for fennec-nightlies
https://reviewboard.mozilla.org/r/125554/#review128638
r+ with nits.
::: taskcluster/ci/nightly-l10n/kind.yml:36
(Diff revision 2)
> + run-on-projects:
> + - mozilla-central
> + - mozilla-aurora
> + - mozilla-beta
> + - date
> + - jamun
As per an earlier review, I think it's cleaner having the real set of branches land, and then update date and jamun to add them. (It may be messy when we land and back out patches, but I think it'll be cleaner overall.)
If we plan on having a post-migration sweep to clean up all the jamun and date references once we're done with them, that works.
::: taskcluster/ci/upload-symbols/job-template.yml:10
(Diff revision 2)
> deadline-after: 24 hours
> run-on-projects:
> - try
> - - mozilla-beta
> - - mozilla-release
> + - release
> + - date
> + - jamun
same here.
::: taskcluster/taskgraph/target_tasks.py:36
(Diff revision 2)
> - return False
> run_on_projects = set(task.attributes.get('run_on_projects', []))
> return match_run_on_projects(parameters['project'], run_on_projects)
>
>
> +def filter_upload_symbols(task, parameters):
We really should add the 'nightly' parameter to these rather than have this function. Maybe as a followup?
Attachment #8853461 -
Flags: review?(aki) → review+
Assignee | ||
Comment 7•8 years ago
|
||
mozreview-review-reply |
Comment on attachment 8853461 [details]
Bug 1352477 - taskgraph ignores run-on-project for fennec-nightlies
https://reviewboard.mozilla.org/r/125554/#review128638
> We really should add the 'nightly' parameter to these rather than have this function. Maybe as a followup?
I filed bug 1353296
Comment hidden (mozreview-request) |
Pushed by jlorenzo@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5533bb014569
taskgraph ignores run-on-project for fennec-nightlies r=aki
Comment 10•8 years ago
|
||
bugherder |
Assignee | ||
Comment 11•8 years ago
|
||
I had to uplift this patch to aurora, in order to make bug 1357409 happen: https://hg.mozilla.org/releases/mozilla-aurora/rev/4dae386f2c331851b192a3ef493cc139fd562a44
Blocks: 1357409
Updated•7 years ago
|
Updated•7 years ago
|
Updated•7 years ago
|
You need to log in
before you can comment on or make changes to this bug.
Description
•