Closed
Bug 1413219
Opened 7 years ago
Closed 7 years ago
add firefox source task for in-tree release promotion
Categories
(Release Engineering :: Release Automation: Other, enhancement)
Release Engineering
Release Automation: Other
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: kmoir, Assigned: kmoir)
References
Details
Attachments
(4 files, 9 obsolete files)
(deleted),
text/plain
|
Details | |
(deleted),
patch
|
rail
:
review+
kmoir
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
kmoir
:
checked-in+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
mozilla
:
feedback+
mtabara
:
checked-in+
|
Details | Diff | Splinter Review |
aki mentioned that he has heard from gerv that we don't actually need to provide a tarball of source, it would be sufficient to provide link to the revision used and instructions on how to build.
That being said, he mentioned that an easy path forward would be to copy the existing fennec source job into the firefox graph
Assignee | ||
Updated•7 years ago
|
Summary: firefox source builder for in-tree release promotion → add firefox source task for in-tree release promotion
Assignee | ||
Updated•7 years ago
|
Assignee: nobody → kmoir
Assignee | ||
Comment 1•7 years ago
|
||
This adds the following tasks to the taskgraph. I thought there was only one generated source but looking at the fennec graph this includes the ones for each platform as well.
upload-generated-sources-linux-devedition-nightly/opt
upload-generated-sources-linux-nightly/opt
upload-generated-sources-linux64-devedition-nightly/opt
upload-generated-sources-linux64-nightly/opt
upload-generated-sources-macosx64-devedition-nightly/opt
upload-generated-sources-macosx64-nightly/opt
upload-generated-sources-win32-devedition-nightly/opt
upload-generated-sources-win32-nightly/opt
upload-generated-sources-win64-devedition-nightly/opt
upload-generated-sources-win64-nightly/opt
Attachment #8925569 -
Flags: feedback?(mtabara)
Updated•7 years ago
|
Attachment #8925569 -
Flags: feedback?(mtabara) → feedback+
Assignee | ||
Updated•7 years ago
|
Attachment #8925569 -
Flags: review?(mtabara)
Updated•7 years ago
|
Attachment #8925569 -
Flags: review?(mtabara)
Attachment #8925569 -
Flags: review+
Attachment #8925569 -
Flags: feedback+
Assignee | ||
Updated•7 years ago
|
Attachment #8925569 -
Flags: checked-in+
Comment 2•7 years ago
|
||
hmm, I think it's not the same source task: https://dxr.mozilla.org/mozilla-central/source/taskcluster/docs/kinds.rst#73
it was implemeted in https://hg.mozilla.org/mozilla-central/rev/4a6df9405fb5
Assignee | ||
Comment 3•7 years ago
|
||
Rail, I'm not sure I understand. Could you elaboarate? The new tasks include a call to the scripts your reference in the commits in comment 2
"metadata": {
"description": "Upload generated source files from build ([Treeherder push](https://treeherder.mozilla.org/#/jobs?repo=maple&revision=559e44e920bfabfdb4ee39749fb3769a2034407f))",
"name": "upload-generated-sources-linux-devedition-nightly/opt",
"owner": "nalexander@mozilla.com",
"source": "https://hg.mozilla.org/projects/maple/file/559e44e920bfabfdb4ee39749fb3769a2034407f/taskcluster/ci/upload-generated-sources"
},
"payload": {
"cache": {
"level-3-checkouts-36e3b4faee27c3795552": "/builds/worker/checkouts",
"level-3-maple-dotcache-36e3b4faee27c3795552": "/builds/worker/.cache"
},
"command": [
"/builds/worker/bin/run-task",
"--vcs-checkout=/builds/worker/checkouts/gecko",
"--fetch-hgfingerprint",
"--",
"bash",
"-cx",
"cd /builds/worker/checkouts/gecko && ./mach python build/upload_generated_sources.py ${ARTIFACT_URL}\n"
],
"env": {
"ARTIFACT_URL": {
"task-reference": "https://queue.taskcluster.net/v1/task/<build>/artifacts/public/build/target.generated-files.tar.gz"
Comment 4•7 years ago
|
||
The name of the task is a bit misleading. From what I see (https://tools.taskcluster.net/groups/b2GhZ0woTOGyJQawUEmzAw/tasks/DJh2XrdGTjelKkvov4kpgA/details) the generated "source" tarballs are partial to a particular task. This is the list of things in one of the tarballs:
tree -L 2 ✔ 15:56:01
.
├── accessible
│ └── xpcom
├── build
│ └── unix
├── buildid.h
├── dist
│ └── include
├── dom
│ ├── base
│ └── bindings
├── gfx
│ └── cairo
├── ipc
│ └── ipdl
├── js
│ └── src
├── layout
│ └── style
├── mozilla-config.h
├── netwerk
│ └── necko-config.h
├── security
│ └── nss
├── source-repo.h
├── toolkit
│ ├── components
│ └── library
└── xpcom
├── base
└── xpcom-config.h
25 directories, 5 files
Assignee | ||
Comment 5•7 years ago
|
||
work in progress, still quite a few things to fix
Assignee | ||
Comment 6•7 years ago
|
||
Attachment #8925569 -
Attachment is obsolete: true
Attachment #8926208 -
Attachment is obsolete: true
Assignee | ||
Comment 7•7 years ago
|
||
generated part of graph
Assignee | ||
Comment 8•7 years ago
|
||
Comment on attachment 8926978 [details] [diff] [review]
bug1413219.patch
Not sure if this is on the right path or I should take a different approach. I still have to what tasks it depends on and tasks that depend on it
Attachment #8926978 -
Flags: feedback?(rail)
Comment 9•7 years ago
|
||
Comment on attachment 8926978 [details] [diff] [review]
bug1413219.patch
LGTM!
I think you don't need to worry of the previous tasks dependency, because you can build the source tarball any time after the push. Still need to figure out how to beetmove this task though.
Updated•7 years ago
|
Attachment #8926978 -
Flags: feedback?(rail) → feedback+
Assignee | ||
Comment 11•7 years ago
|
||
I'll wait for Mihai's beetmover patches to land and then I'll test to ensure the artifacts are beetmoved
Attachment #8927466 -
Attachment is obsolete: true
Attachment #8927859 -
Flags: review?(rail)
Comment 12•7 years ago
|
||
Comment on attachment 8927859 [details] [diff] [review]
bug1413219-5.patch
Review of attachment 8927859 [details] [diff] [review]:
-----------------------------------------------------------------
Sorry for the delay. SHIP IT!
Attachment #8927859 -
Flags: review?(rail) → review+
Assignee | ||
Comment 13•7 years ago
|
||
Attachment #8927859 -
Flags: checked-in+
Assignee | ||
Comment 14•7 years ago
|
||
From conversation with Mihai today
<mtabara> Mihai Tabara kmoir: https://github.com/mozilla-releng/beetmoverscript/pull/92/files
1:14 PM kmoir: beetmover-dev1.srv.releng.use1.mozilla.com
1:20 PM kmoir: https://archive.mozilla.org/pub/firefox/candidates/57.0b9-candidates/build1/beetmover-checksums/
1:22 PM kmoir: https://hg.mozilla.org/mozilla-central/file/tip/testing/mozharness/scripts/release/generate-checksums.py
-Currently we have beetmover-repackage kind which computes checksums
-the checksum signing task signs the checksums
-beetmover-checksums consumes signed artifacts
-need to check if source needs to be signed
-so we just need to add a beetmover-source kind which depends on existing release-source, update dependencies
-beetmover script should just work
Assignee | ||
Comment 15•7 years ago
|
||
So these are my patches which are not working. Right now, the release-source is generated but not signed. These patches attempt to sign it and but the release- source-signing task is not included. I'm not sure that as suggested in irc, the build-signing kind should depend on the release-source kind. The source isn't generated on push, it is only part of release promotion. Anyways, I'm stuck and would appreciate pointers in the right direction.
Comment 16•7 years ago
|
||
>diff --git a/taskcluster/ci/beetmover-repackage/kind.yml b/taskcluster/ci/beetmover-repackage/kind.yml
>--- a/taskcluster/ci/beetmover-repackage/kind.yml
>+++ b/taskcluster/ci/beetmover-repackage/kind.yml
>@@ -8,16 +8,17 @@ transforms:
> - taskgraph.transforms.name_sanity:transforms
> - taskgraph.transforms.beetmover_repackage_l10n:transforms
> - taskgraph.transforms.beetmover_repackage:transforms
> - taskgraph.transforms.task:transforms
>
> kind-dependencies:
> - repackage-signing
> - partials-signing
>+ - release-source
This should probably be `release-source-signing`, since we want to beetmove the signed bits.
I think we also need to add shipping-phase and shipping-product to release-source.
I think the main problem is that transforms/beetmover_repackage.py is hardcoding kind names:
https://hg.mozilla.org/projects/maple/file/bdb3a072306c/taskcluster/taskgraph/transforms/beetmover_repackage.py#l200
https://hg.mozilla.org/projects/maple/file/bdb3a072306c/taskcluster/taskgraph/transforms/beetmover_repackage.py#l208
https://hg.mozilla.org/projects/maple/file/bdb3a072306c/taskcluster/taskgraph/transforms/beetmover_repackage.py#l224
I think this needs to be rewritten to be more data driven and less logic-driven-with-hardcodes. We may be able to solve this current issue without rewriting.
One solution may be to go back to having the source be a build kind, but a different shipping-phase; then we'd match beetmover-repackage's kind expectations. Another may be to add more hardcodes to beetmover_repackage.py.
Assignee | ||
Comment 17•7 years ago
|
||
Here are updated patches. The release-source-signing task is included in the graph now. However, the beetmover parts are not working. As was noted before, there are values hardcoded in beetmover_repackage.py for signing_name, build_name and repackage_name and these look at the tasks dependencies. There aren't dependencies for release-source and release-source-signing in the graph so I'm wondering if the best approach is create a new transforms for beetmover-source but that seems like a lot of duplicated code.
Attachment #8932142 -
Attachment is obsolete: true
Flags: needinfo?(mtabara)
Assignee | ||
Updated•7 years ago
|
Flags: needinfo?(aki)
Comment 18•7 years ago
|
||
12:23 <•aki> kmoir: hm, i'm thinking you should be using beetmover since this isn't a repackage job. am i wrong?
12:23 <•aki> beetmover moves unsigned/signed bits (build+signing); beetmover-repackage moves repackaged bits. the source task doesn't have a repackage task
12:24 <kmoir> okay I will try that, I was talking to m.ihai about beetmover repackage but perhaps that was in reference to the checksums
12:24 <kmoir> thanks
Flags: needinfo?(aki)
Assignee | ||
Updated•7 years ago
|
Flags: needinfo?(mtabara)
Assignee | ||
Comment 19•7 years ago
|
||
These patches fix the earlier issues. They include beetmoving the checksum for the source build but this is not yet implemented. I could comment out temporarily for testing on maple.
Attachment #8932589 -
Attachment is obsolete: true
Assignee | ||
Updated•7 years ago
|
Attachment #8932623 -
Flags: feedback?(mtabara)
Comment 20•7 years ago
|
||
Comment on attachment 8932623 [details] [diff] [review]
bug1413219-bm3.patch
Review of attachment 8932623 [details] [diff] [review]:
-----------------------------------------------------------------
LGTM!
We already have target.tar.bz2 + *asc files present in the PR[1], hence the existing beetmoverscript package that the dep beetmoverworker is using so we should be all good to go here!
As long as the beetmover-source tasks are being correctly generated and the artifacts are good, it should just work! \o/
[1]: https://github.com/mozilla-releng/beetmoverscript/pull/92/files#diff-ca319768c840fd3d444edc95cd0fd017R103
Attachment #8932623 -
Flags: feedback?(mtabara) → feedback+
Assignee | ||
Comment 21•7 years ago
|
||
Comment on attachment 8932623 [details] [diff] [review]
bug1413219-bm3.patch
Thanks! Now asking for review :-) so I can land on maple
Attachment #8932623 -
Flags: review?(mtabara)
Comment 22•7 years ago
|
||
Attachment #8932623 -
Flags: review?(mtabara) → review+
Assignee | ||
Comment 23•7 years ago
|
||
rebased patch after aki's patches landed last night. The issue I'm having is that my new tasks of release-source-signing and beetmover-source are no longer included in the promote graph
Comment 24•7 years ago
|
||
Still digging to see why the release-source-signing and beetmover tasks aren't showing up in the *full* graph.
https://public-artifacts.taskcluster.net/M9o0FgB4TYepcqg9blT4Tg/0/public/logs/live_backing.log is the latest source run - we seem to be missing tooltool_script.
[task 2017-11-29T20:23:11.876Z] 20:23:11 FATAL - The key 'tooltool_script' could not be determined and is needed for the action 'build'. Please add this to your config or run without that action (ie: --no-{action})
Comment 25•7 years ago
|
||
Assignee | ||
Comment 26•7 years ago
|
||
Attachment #8932623 -
Attachment is obsolete: true
Attachment #8932997 -
Attachment is obsolete: true
Attachment #8933044 -
Attachment is obsolete: true
Assignee | ||
Comment 27•7 years ago
|
||
Attachment #8933063 -
Flags: checked-in+
Comment 28•7 years ago
|
||
So there's a chicken-egg sort of problem with this one.
Facts:
* `Beetmover-source depends` on `release-signing-source` which in turns depends on `release-source`
* there's no nightly builds in the chain of dependencies so there's pretty much no way I can add it nicely
* if I dare to add it as a dependency, the beetmover transform will fail because we can't have more than two dependencies in signing[1]
* we need to somehow slip en-US build in the dependencies along with tweaking the upstreamArtifacts list to get balrog_props file
* this is supposed to be a temporary fix until we manage to kill balrog_props in beetmoverscript
Solutions at hand:
1) Hack via a transform the task definition, after beetmover transforms finishes with it.
2) define a separate beetmover-source which would pretty much copy-paste entire beetmover transform but would amend the needs for source file. This would kill the idea of single-sourcing and would require us to make changes in two places every time we need to amend something in beetmover transform
3) make if/else changes in beetmover transform - I guess this can blow off other tasks as well so it's very risky given our tight timeline
I went on with 1) becausee:
i) it's temporary until we get rid of balrog_props
ii) it's not a singular case - we're hacking stuff alike in [2][3] as well in the graph
iii) it works :)
Please let me know what you think of this.
Thanks!
[1]: https://hg.mozilla.org/projects/maple/file/tip/taskcluster/taskgraph/transforms/beetmover.py#l363
[2]: https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/transforms/l10n.py#227
[3]: https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/transforms/use_toolchains.py#38
Attachment #8935059 -
Flags: feedback?(aki)
Comment 29•7 years ago
|
||
Comment on attachment 8935059 [details] [diff] [review]
Tweak beetmover-source task to grab balrog-props too
This works, and I'm glad. I really don't like this hardcode:
> + job['dependencies']['build'] = u'build-linux64-nightly/opt'
I wonder if we can pass that down.
Let's land this and see if it fixes things, and deal with followups.
Thank you!
Attachment #8935059 -
Flags: feedback?(aki) → feedback+
Comment 30•7 years ago
|
||
We'll probably have to deal with that hardcode before we can create source tasks for fennec or devedition.
Comment 31•7 years ago
|
||
Comment on attachment 8935059 [details] [diff] [review]
Tweak beetmover-source task to grab balrog-props too
https://hg.mozilla.org/projects/maple/rev/a7e9356f160f78b918eb509492933779a0c198fb
Addressed aki's comments too to condition that hardcode to firefox only for now. Better to deal hard failure than bad behavior.
Attachment #8935059 -
Flags: checked-in+
Comment 32•7 years ago
|
||
This worked like a charm!
http://ftp.stage.mozaws.net/pub/firefox/candidates/58.0b12-candidates/build11/
Will track devedition separately I suppose.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•