Closed
Bug 1174206
Opened 9 years ago
Closed 8 years ago
Migrate Android x86 builds to TaskCluster and make Tier 2
Categories
(Firefox Build System :: Task Configuration, task)
Firefox Build System
Task Configuration
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: coop, Assigned: kmoir)
References
Details
(Whiteboard: [taskcluster][mobile][build])
Attachments
(5 files, 1 obsolete file)
(deleted),
patch
|
gbrown
:
review+
|
Details | Diff | Splinter Review |
(deleted),
text/plain
|
Details | |
(deleted),
patch
|
gbrown
:
review+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
Details | Diff | Splinter Review | |
(deleted),
patch
|
gbrown
:
review+
|
Details | Diff | Splinter Review |
Android x86 builds use the fx_desktop_build.py mozharness script invoked with the custom build variant of x86.
Example command line:
/tools/buildbot/bin/python scripts/scripts/fx_desktop_build.py --config builds/releng_base_android_64_builds.py --custom-build-variant-cfg x86 --config balrog/production.py --branch mozilla-inbound --build-pool production
Example build logs:
http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-inbound-android-x86/1434122155/mozilla-inbound-android-x86-bm74-build1-build1142.txt.gz
Comment 1•9 years ago
|
||
How is this different from bug 1118394?
Reporter | ||
Comment 2•9 years ago
|
||
I only see the Android 4.0 API11+ TC build in try treeherder right now, so I want to make sure API9, x86, and debug variants don't get missed.
If you're using that bug to track moving *all* the variants, that's fine. Feel free to dupe.
Blocks: 1118394
Comment 3•9 years ago
|
||
I'm assuming (!) those won't be much more work than the single build. I set up just the one to save load on TC until people are actually looking.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → DUPLICATE
Reporter | ||
Updated•9 years ago
|
Status: RESOLVED → REOPENED
Component: General Automation → Task Configuration
Product: Release Engineering → Taskcluster
QA Contact: catlee
Resolution: DUPLICATE → ---
Reporter | ||
Updated•9 years ago
|
Assignee: nobody → kmoir
Updated•9 years ago
|
Summary: Migrate Android x86 builds to TaskCluster → Migrate Android x86 builds to TaskCluster and make Tier 2
Assignee | ||
Comment 4•8 years ago
|
||
wip patches. They don't work yet due to missing java env on worker.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=028ec64f5e34c9be6df7214c3a00fe518155663f
Assignee | ||
Comment 5•8 years ago
|
||
I added openjdk to the tooltool manifest and it ran green. Will do some further verification to ensure it is correct.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b10ccb969ddc&selectedJob=22887702
Attachment #8764739 -
Attachment is obsolete: true
Assignee | ||
Comment 6•8 years ago
|
||
diff of unzip -l of the old and new apks. Not sure if the different manifest names are just because it was run on try
Assignee | ||
Comment 7•8 years ago
|
||
Comment on attachment 8765083 [details] [diff] [review]
bug1174206.patch
Not sure if it's worth having a creating a second tooltool manifest for android x86 since it's going away and having openjdk in the one for buildbot doesn't seem to harm anything.
Attachment #8765083 -
Flags: review?(gbrown)
Comment 8•8 years ago
|
||
Comment on attachment 8765083 [details] [diff] [review]
bug1174206.patch
Review of attachment 8765083 [details] [diff] [review]:
-----------------------------------------------------------------
I think adding on to the existing tooltool manifest is fine.
In your try run, I notice that the tc build had a substantially longer run-time than the buildbot build -- do you know why?
Attachment #8765083 -
Flags: review?(gbrown) → review+
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/3e53a0aae914
Migrate Android x86 builds to TaskCluster and make Tier 2 r=gbrown
Assignee | ||
Comment 10•8 years ago
|
||
Comment on attachment 8765083 [details] [diff] [review]
bug1174206.patch
http://hg.mozilla.org/integration/mozilla-inbound/rev/3e53a0aae914
Thanks, I'm investigating the issue wrt longer build times. I know with some of the other conversions, the problem was that tc builds were running on less powerful instance types (with less RAM). I'm running another try build now to try and will look at the instance types each run with.
Comment 11•8 years ago
|
||
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6b992274233a
Migrate Android x86 builds to TaskCluster and make Tier 2 r=gbrown
Assignee | ||
Comment 12•8 years ago
|
||
Yes, this is the problem. Buildbot builds currently run on r3.xlarge and the tc worker-type I assigned it is android-api-15 (me.2xlarge) which has less memory. Going to try to use a different worker type definition or create a new one. None of the tc worker type definitions currently use r3.xlarge according to https://github.com/taskcluster/aws-provisioner
Comment 13•8 years ago
|
||
So this caused a problem in production. The issue was that the inheritence chain of config files does the following:
taskcluster/ci/legacy/tasks/builds/android_api_15.yml inherits from
taskcluster/ci/legacy/tasks/builds/android_x86.yml also inherits from
taskcluster/ci/legacy/tasks/builds/mobile_base.yml which inherits from
taskcluster/ci/legacy/tasks/docker_build.yml
The variables passed from the inherit section in android_api_15.yml are the same as android_x86.yml,
variables:
build_name: 'android'
build_type: 'opt'
which resolve in docker_build.yml to the routes:
routes:
- 'index.gecko.v1.{{project}}.revision.linux.{{head_rev}}.{{build_name}}.{{build_type}}'
- 'index.gecko.v1.{{project}}.latest.linux.{{build_name}}.{{build_type}}'
which will collide.
Joel helped debug this, and has steps for reproducing this issue here https://pastebin.mozilla.org/8880122 . This should be a simple matter of changing build_name or build_type, but I'm not sure of the impact of making those changes. Once I find someone who can do the backout, that's what I'll do
Comment 14•8 years ago
|
||
Joel's completed the backout: http://hg.mozilla.org/integration/mozilla-inbound/rev/73732455d235
Assignee | ||
Comment 15•8 years ago
|
||
new patch, running try run. Ironically, I had a different build_name in my patch queue locally but not in the patch I submitted to bugzilla
Assignee | ||
Comment 16•8 years ago
|
||
I'm not sure what the output should be given the fix
Kims-MacBook-Pro:mozilla-central kmoir$ ./mach taskgraph target -p ~/Downloads/parameters.yml
0:00.19 Generating full task set
0:00.56 Starting new HTTPS connection (1): hg.mozilla.org
0:22.41 Generating full task graph
0:22.42 Generating target task set
TaskLabel==Htn2LYRFRGGPONQ6Y7aNRw
TaskLabel==dtGSMvrkQ8aPgYfDEwoCrQ
jmaher: how do you tell if mach is generating the right taskgraph? Do the task graphs correspond to something that you can verify in the taskcluster tools?
Flags: needinfo?(jmaher)
Assignee | ||
Comment 17•8 years ago
|
||
Comment on attachment 8765924 [details] [diff] [review]
bug1174206-2.patch
I also changed the worker type to m3.2xlarge but it seems that linux64-opt was allocated to it even though there are r3.2xlarge in the pool so the build wasn't faster. https://tools.taskcluster.net/aws-provisioner/#opt-linux64/
Attachment #8765924 -
Flags: review?(gbrown)
Comment 18•8 years ago
|
||
:kmoir, this is all cryptic, but actually you solved it because when the patch that landed was there it was an error printed out while running |mach taskgraph target|, you don't see the error.
I have confidence all is well!
Flags: needinfo?(jmaher)
Comment 19•8 years ago
|
||
Comment on attachment 8765924 [details] [diff] [review]
bug1174206-2.patch
Review of attachment 8765924 [details] [diff] [review]:
-----------------------------------------------------------------
You may have it right already, but in light of bug 1283091, maybe double-check the x86 route.
Attachment #8765924 -
Flags: review?(gbrown) → review+
Comment 20•8 years ago
|
||
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/1cee0d296a34
Migrate Android x86 builds to TaskCluster and make Tier 2 r=gbrown
Comment 21•8 years ago
|
||
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/89d45ec9d5aa
Migrate Android x86 builds to TaskCluster and make Tier 2 r=gbrown DONTBUILD
Assignee | ||
Comment 22•8 years ago
|
||
I changed the workertype back to android15 as it had a better pool of high memory instance types
Comment 23•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/1cee0d296a34
https://hg.mozilla.org/mozilla-central/rev/89d45ec9d5aa
Status: REOPENED → RESOLVED
Closed: 9 years ago → 8 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 24•8 years ago
|
||
I have to add another patch to enable on other branches, this was just on try.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 25•8 years ago
|
||
patch to enable on other branches
Attachment #8766768 -
Flags: review?(gbrown)
Updated•8 years ago
|
Attachment #8766768 -
Flags: review?(gbrown) → review+
Comment 26•8 years ago
|
||
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c4fc457de3a7
Migrate Android x86 builds to TaskCluster and make Tier 2 r=gbrown
Assignee | ||
Comment 27•8 years ago
|
||
This is landed on inbound and I've verified on treeherder However, I'm going to ask in the next mobile meeting if Android x86 is still needed. I recall asking in a meeting about six months ago that Android x86 has very small market numbers compared to armv7. Going to confirm if this is still the case and what the future holds for x86 builds from a product standpoint.
Status: REOPENED → RESOLVED
Closed: 8 years ago → 8 years ago
Resolution: --- → FIXED
Comment 28•8 years ago
|
||
bugherder |
Updated•7 years ago
|
Product: TaskCluster → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•