Implemented in bug 1354821, we now generate nightly apk with the aurora apk name. We should update the CI to generate these APKs.

Rail, can you take care of that for me? (or explain me how to do it :)

It's on my radar, leaving NI in place.

I have some questions/comments: 1) We want to upload the generated APK to Google Play using org.mozilla.fennec_aurora, but Nighlty branding. Is this correct? Probably we don't want to advertise this product in this case. I wonder if there are ways to hide it from search/store listings. 2) Do we want to run tests against this binary?

> 1) We want to upload the generated APK to Google Play using > org.mozilla.fennec_aurora, but Nighlty branding. Is this correct? Probably > we don't want to advertise this product in this case. I wonder if there are > ways to hide it from search/store listings. I would not phrase this way: we want to ship nightly using the org.mozilla.fennec_aurora. Why we don't want to advertise it? Besides in the navigator url, it is hard to see fennec_aurora somewhere (I tried). > 2) Do we want to run tests against this binary? The same as the current nightly package. if resources constraints, we should just disable tests on the org.mozilla.fennec package as the population is tiny.

After reading bug 1357351, I think it's pretty clear that we want to generate the existing Nightly with the aurora id, because transitioning from one id to another requires user opt-in. We have many more users on Aurora than Nightly, so when we combine these populations we want to require the least amount of user disruption. I think it's unfortunate that we're going to let the "aurora" name live on in presumed perpetuity here, but the trade off seems right. I'll have a look at this.

Nick, I'm not sure if you're the right person to review this or not so feel free to redirect. As far as I can tell, the only thing we need to do to change the package id is set ANDROID_PACKAGE_NAME in configure.sh. I did this on try, and got builds with "org.mozilla.fennec_aurora" in package-name.txt: https://treeherder.mozilla.org/#/jobs?repo=try&filter-searchStr=android&fromchange=be0abd157bc083de74fe521f58380191aaf0bb54 This patch also removes the aurora/ branding directory on the assumption that it's no longer needed.

Hmm, looks like you are doing the same thing as I did in bug 1354821, no?

Comment on attachment 8865612 [details] [diff] [review] use aurora id for nightly fennec (In reply to Sylvestre Ledru [:sylvestre] from comment #7) > Hmm, looks like you are doing the same thing as I did in bug 1354821, no? We've gone about it a slightly different way it looks like - my version updates the already used mozconfigs. But your version is already reviewed and seems fine, we'll just change the mozconfigs we use. I guess this bug was intended to do that part?

Clarified this bug with Sylvestre on IRC. We're going to switch all of our existing Fennec automation on mozilla-central to use the Aurora id. We still need to do builds with the Nightly id for awhile, so we'll set-up a new job to do that. We do not need to do any tests on the new nightly id build, nor any other types of builds with it (debug, artifact, etc.).

One further clarification: the org.mozilla.fennec_aurora builds will continue to publish to the "aurora" update channel and the same "Firefox Aurora for Developers" Play Store app even when they are being built from mozilla-central. (We will make some changes to that app in terms of title, text, etc. - but that's not relevant to the work here). The org.mozilla.fennec builds will continue to publish to the "nightly" update channel. They have no Play Store app right now, nor will we add one as part of this.

Another question Sylvestre: do we need single locale repacks for the old nightly id?

Nope, i expect translator to use the new nightly id. especially with the availability on gp

I managed to get the scheduling working today as seen in this try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=fdfa6ae69760548e9934f35bb7bad11cf94bcb77 The builds may be right too, I'll check them out tomorrow. My basic approach was to change the default ANDROID_PACKAGE_NAME to org.mozilla.fennec_aurora (so that we don't need to go and override it for all build types), and override it specifically for the new "old-id" builds. This is a bit different than what Sylvestre prepped for in bug 1354821, but I think it will be the cleanest approach.

I worked through a bunch of scheduling and configuration issues today. I've now got what I think is a nearly complete patch pushed to date (https://treeherder.mozilla.org/#/jobs?repo=date&revision=63806a9e29871ea819a6b405a9ed9c2ec5c3a814) and I'll see how things look in the morning. I also spoke with Rail today and we got to wondering if we even need a separate build for the old id. He was thinking that we may not, since it's not delivered via the play store. I'm going to try updating a current Nightly to a current Aurora tomorrow to see if that works without dataloss or other issues. Once I verify that one way or the other I'll detail the update rule changes we'll need when this lands.

A bunch more progress today, including: * Set run-on-projects so old-id builds don't run on branches other than mozilla-central * Make upload-symbols tasks get scheduled for old-id builds * Fix typo that was breaking scheduling of a couple of nightlies entirely I'm not at the point where all builds + nightlies are generated. I still need to do more inspection of the builds themselves to make sure build configuration is correct. The latest builds on https://treeherder.mozilla.org/#/jobs?repo=date are available if anyone else wants to look at them. I also need to find a way to fix the beetmover-signing tasks for the new old-id builds. They're failing because "multi" isn't in the mapping it is generating (eg: https://queue.taskcluster.net/v1/task/fscD3rLQQk6KcGrjqdoLCw/runs/0/artifacts/public/logs/live_backing.log), but I'm not sure why. (In reply to Ben Hearsum (:bhearsum) from comment #14) > I also spoke with Rail today and we got to wondering if we even need a > separate build for the old id. He was thinking that we may not, since it's > not delivered via the play store. I'm going to try updating a current > Nightly to a current Aurora tomorrow to see if that works without dataloss > or other issues. Once I verify that one way or the other I'll detail the > update rule changes we'll need when this lands. I haven't been able to test this. I'm going to punt on it it for now (since I've already got old-id builds more or less working) and may come back to it later.

If we end up keeping the old-id builds, we'll need these rule changes in Balrog: * Before we land the build changes on mozilla-central, lock nightly & aurora channel rules to the current nightly. * Once the first set of new nightlies come out, make the following changes: ** Empty out the current "Fennec-mozilla-central-nightly-latest" blob ** channel: nightly, buildid: <(new nightly buildid), mapping: Fennec-mozilla-central-nightly-old-id-latest ** channel: nightly-old-id, mapping: Fennec-mozilla-central-nightly-old-id-latest ** Update the current "main" aurora rule with buildid: <(new nightly buildid), mapping: Fennec-mozilla-central-nightly-latest ** Update the current "main" nightly rule to switch mapping back to Fennec-mozilla-central-nightly-latest We should do the above on test channels first, just to be safe. I also need to figure out how to get the old-id builds to submit to the Fennec-mozilla-central-nightly-old-id-latest blob. I think we might be able to change build_type in cli.py to do this (similar to hack we used to have for api-9).

I gave the builds a try by hand and everything seems OK. I still need to deal with the beetmover signing, but I think it's about time to get some feedback on this patch. There's a few things going on here: * Removing the aurora branding entirely (it's dead!) * Creating new branding/mozconfigs for old id builds * Change the default ANDROID_PACKAGE_NAME to org.mozilla.fennec_aurora * Add OldId builds on mozilla-central The nightlies on https://treeherder.mozilla.org/#/jobs?repo=date&selectedJob=99252339 are built with this patch (tweaked for usage on Date) if you'd like to play with a build.

Details in PR, probably only need one of you to review this.

Comment on attachment 8868225 [details] set old id platforms in beetmover script mihai, if you like, feel free to ignore r? and blame me in the event of fallout.

Another todo: Make sure we automatically publish the new Fennec nightlies on mozilla-central to Google Play. This can probably happen after the builds are reconfigured and verified.

Commit pushed to master at https://github.com/mozilla/treeherder https://github.com/mozilla/treeherder/commit/774963ecbad7de7bcbcdca1c71005ecc67b65a7c Bug 1357808 - Add support for Android OldId builds (#2470)

(In reply to Jordan Lund (:jlund) from comment #20) > Comment on attachment 8868225 [details] > set old id platforms in beetmover script > > mihai, if you like, feel free to ignore r? and blame me in the event of > fallout. https://hg.mozilla.org/build/puppet/rev/2b8ed48da971 Bug 1357808 - bump beetmoverscript in beetmoverworkers. r=trivial 0.5.3 rolled-out and deployed on all four beetmoverworkers. @bhearsum: feel free to respin any jobs that were previously failing.

Comment on attachment 8868215 [details] [diff] [review] fennec-migration-builds.diff >I'm going to try updating a current Nightly to a current Aurora tomorrow to see if that works without dataloss or other issues. I'd be surprised if that works. Pretty sure it'll install as a separate app. >I gave the builds a try by hand and everything seems OK. I still need to deal with the beetmover signing, but I think it's about time to get some feedback on this patch. There's a few things going on here: >* Removing the aurora branding entirely (it's dead!) >* Creating new branding/mozconfigs for old id builds >* Change the default ANDROID_PACKAGE_NAME to org.mozilla.fennec_aurora >* Add OldId builds on mozilla-central Are we keeping "old id" around as a name for a long time? It'll age less poorly than a "new", and I suppose we want to avoid changing these IDs around again. It does bother me a bit though. >diff --git a/old-configure.in b/old-configure.in >--- a/old-configure.in >+++ b/old-configure.in >@@ -5091,7 +5091,20 @@ if test -z "$MOZ_APP_VERSION_DISPLAY"; t > fi > > if test -z "$ANDROID_PACKAGE_NAME" ; then >- ANDROID_PACKAGE_NAME="org.mozilla.$MOZ_APP_NAME" >+ # When we got rid of the Aurora channel we decided to replace the old >+ # Nightly ANDROID_PACKAGE_NAME with Aurora. To make sure this is inherited >+ # by all mozilla-central based branches we make this the new "default" >+ # for Fennec. Non mozilla-central based branches set ANDROID_PACKAGE_NAME >+ # in their mozconfig, so they will never get this. If there are ever >+ # non-Fennec builds for Android, they will fall into the else block >+ # and use their own default name. >+ # https://bugzilla.mozilla.org/show_bug.cgi?id=1357808 has additional >+ # background on this. >+ if test "$MOZ_APP_NAME" = "fennec"; then >+ ANDROID_PACKAGE_NAME="org.mozilla.fennec_aurora" >+ else >+ ANDROID_PACKAGE_NAME="org.mozilla.$MOZ_APP_NAME" >+ fi > fi It would be better to be explicit here rather than magical, but we can keep if this approach makes the most sense.

(In reply to Aki Sasaki [:aki] from comment #24) > Comment on attachment 8868215 [details] [diff] [review] > fennec-migration-builds.diff > > >I'm going to try updating a current Nightly to a current Aurora tomorrow to see if that works without dataloss or other issues. > > I'd be surprised if that works. Pretty sure it'll install as a separate app. Thanks for that perspective! > >I gave the builds a try by hand and everything seems OK. I still need to deal with the beetmover signing, but I think it's about time to get some feedback on this patch. There's a few things going on here: > >* Removing the aurora branding entirely (it's dead!) > >* Creating new branding/mozconfigs for old id builds > >* Change the default ANDROID_PACKAGE_NAME to org.mozilla.fennec_aurora > >* Add OldId builds on mozilla-central > > Are we keeping "old id" around as a name for a long time? It'll age less > poorly than a "new", and I suppose we want to avoid changing these IDs > around again. It does bother me a bit though. Old id is expected to go away in the short to medium term. The idea is that we don't want to immediately abandon our handful of users on the current Fennec Nightly, but we will stop creating these builds later. org.mozilla.org.fennec_aurora will be the id for our Nightly builds for the forseeable future. > >diff --git a/old-configure.in b/old-configure.in > >--- a/old-configure.in > >+++ b/old-configure.in > >@@ -5091,7 +5091,20 @@ if test -z "$MOZ_APP_VERSION_DISPLAY"; t > > fi > > > > if test -z "$ANDROID_PACKAGE_NAME" ; then > >- ANDROID_PACKAGE_NAME="org.mozilla.$MOZ_APP_NAME" > >+ # When we got rid of the Aurora channel we decided to replace the old > >+ # Nightly ANDROID_PACKAGE_NAME with Aurora. To make sure this is inherited > >+ # by all mozilla-central based branches we make this the new "default" > >+ # for Fennec. Non mozilla-central based branches set ANDROID_PACKAGE_NAME > >+ # in their mozconfig, so they will never get this. If there are ever > >+ # non-Fennec builds for Android, they will fall into the else block > >+ # and use their own default name. > >+ # https://bugzilla.mozilla.org/show_bug.cgi?id=1357808 has additional > >+ # background on this. > >+ if test "$MOZ_APP_NAME" = "fennec"; then > >+ ANDROID_PACKAGE_NAME="org.mozilla.fennec_aurora" > >+ else > >+ ANDROID_PACKAGE_NAME="org.mozilla.$MOZ_APP_NAME" > >+ fi > > fi > > It would be better to be explicit here rather than magical, but we can keep > if this approach makes the most sense. The reason I went with this is to avoid the overhead of touching all the different types of builds when we're on a tight schedule. I think doing it this avoids the need to alter tons of files when we uplift to Beta. I'm happy to follow-up later and try to improve this.

(In reply to Aki Sasaki [:aki] from comment #24) > Comment on attachment 8868215 [details] [diff] [review] > fennec-migration-builds.diff > > >I'm going to try updating a current Nightly to a current Aurora tomorrow to see if that works without dataloss or other issues. > > I'd be surprised if that works. Pretty sure it'll install as a separate app. Correct. These should be totally different Apps and totally different Linux users. I would like to compare the before and after APKs to avoid one or two obvious issues we might see.

(In reply to Nick Alexander :nalexander from comment #26) > (In reply to Aki Sasaki [:aki] from comment #24) > > Comment on attachment 8868215 [details] [diff] [review] > > fennec-migration-builds.diff > > > > >I'm going to try updating a current Nightly to a current Aurora tomorrow to see if that works without dataloss or other issues. > > > > I'd be surprised if that works. Pretty sure it'll install as a separate app. > > Correct. These should be totally different Apps and totally different Linux > users. I would like to compare the before and after APKs to avoid one or > two obvious issues we might see. You can take a look at the builds on https://treeherder.mozilla.org/#/jobs?repo=date if you like. You should see that the OldId builds use org.mozilla.fennec, and the other builds use org.mozilla.fennec_aurora.

More good progress today. I believe I've got all of the scheduling issues sorted out now. The latest set of nightly jobs build org.mozilla.fennec_aurora and org.mozilla.fennec builds. Both post to Balrog, but only org.mozilla.fennec_aurora gets push-apk jobs. The push apk jobs work fine in dry run mode, and I think I may have to wait until we land on mozilla-central to switch to test in non-dry run mode (as described in https://github.com/mozilla-releng/pushapkscript/pull/19/files#diff-04c6e90faac2675aa89e2176d2eec7d8R147). I'm going to have another look at the nightlies tomorrow. If they look good, I'll get a patch ready for full review, and we should be able to get these on mozilla-central next week.

This patch fixes up the remaining issues with the Fennec builds on mozilla-central. Changes since the last patch are: * Make push-apk and push-apk-breakpoint run on mozilla-central * Do not include old-id builds in push-apk dependencies * Adjust BALROG_CHANNEL_SCOPES to let the "nightly" alias (which is mozilla-central AFAICT) ship it nightly, nightly-old-id, and aurora * s/aurora/central/ in all the PUSH_APK aliases and mappings * Enable dry run mode for push apk on central. My hope is that if this looks good we can land it on mozilla-central on Tuesday (because Monday is a holiday for me) with the following procedure: * Lock Balrog updates to the latest nightlies * Land this patch, retrigger nightlies * Set-up test channel rules for the new nightlies, including the watersheds described in comment #16 * Do some update testing * If things look good, disable push-apk dry run and use a closed track (per https://github.com/mozilla-releng/pushapkscript/blob/master/README.rst#2-push-to-a-closed-alpha-track) * Retrigger nightlies again * Do some Google Play testing * If things look good, switch to the live track and unlock nightly updates in Balrog

(In reply to Ben Hearsum (:bhearsum) from comment #29) > My hope is that if this looks good we can land it on mozilla-central on > Tuesday (because Monday is a holiday for me) with the following procedure: > * Lock Balrog updates to the latest nightlies > * Land this patch, retrigger nightlies > * Set-up test channel rules for the new nightlies, including the watersheds > described in comment #16 > * Do some update testing > * If things look good, disable push-apk dry run and use a closed track (per > https://github.com/mozilla-releng/pushapkscript/blob/master/README.rst#2- > push-to-a-closed-alpha-track) > * Retrigger nightlies again > * Do some Google Play testing > * If things look good, switch to the live track and unlock nightly updates > in Balrog Sylvestre, I don't think I need your review on this patch, but how do you feel about this landing procedure? It may end up with Fennec nightly+aurora not updating for a few days, but I think that's not a big deal since Aurora updates are off at the moment anyways?

I'm also curious if we want the beta or alpha track on GP.

(In reply to Aki Sasaki [:aki] from comment #31) > I'm also curious if we want the beta or alpha track on GP. Looks like Aurora is currently set to 'beta', so I'm guessing the 'beta' track is correct. I'll leave the ni? here to doublecheck.

Comment on attachment 8869521 [details] [diff] [review] updated patch for fennec builds on m-c >diff --git a/README.txt b/README.txt >--- a/README.txt >+++ b/README.txt >@@ -1,5 +1,3 @@ >-An explanation of the Mozilla Source Code Directory Structure and links to >-project pages with documentation can be found at: > > https://developer.mozilla.org/en/Mozilla_Source_Code_Directory_Structure > Did you intend to remove these two lines? >diff --git a/taskcluster/taskgraph/loader/push_apk.py b/taskcluster/taskgraph/loader/push_apk.py >--- a/taskcluster/taskgraph/loader/push_apk.py >+++ b/taskcluster/taskgraph/loader/push_apk.py >@@ -27,7 +27,10 @@ def get_dependent_loaded_tasks(config, l > ) > android_tasks = [ > task for task in tasks_with_matching_kind >- if task.attributes.get('build_platform', '').startswith('android') >+ # old-id builds are not shipped through the Play store, so we don't >+ # want them as dependencies. >+ if task.attributes.get('build_platform', '').startswith('android') and \ >+ 'old-id' not in task.attributes.get('build_platform', '') > ] This is a little painful, but I can't say it's that much worse than a `.startswith()`. >diff --git a/taskcluster/taskgraph/util/scriptworker.py b/taskcluster/taskgraph/util/scriptworker.py >--- a/taskcluster/taskgraph/util/scriptworker.py >+++ b/taskcluster/taskgraph/util/scriptworker.py > PUSH_APK_SCOPE_ALIAS_TO_PROJECT = [[ >- 'aurora', set([ >- 'mozilla-aurora', >+ 'central', set([ >+ 'mozilla-central', > ]) > ], [ > 'beta', set([ >@@ -221,7 +224,7 @@ PUSH_APK_SCOPE_ALIAS_TO_PROJECT = [[ > > > PUSH_APK_SCOPES = { >- 'aurora': 'project:releng:googleplay:aurora', >+ 'central': 'project:releng:googleplay:aurora', We're going to have to update scriptworker for this change. Right now scriptworker.constants has the set of restricted scopes, as well as which branches they map to. [1] is project:releng:googleplay:aurora, which maps to the 'auroratest' alias, which maps to mozilla-aurora and date here [2]. We need to make sure 'mozilla-central' is in the appropriate alias before we land on central, and then push out a scriptworker dot release. [1] https://github.com/mozilla-releng/scriptworker/blob/master/scriptworker/constants.py#L193 [2] https://github.com/mozilla-releng/scriptworker/blob/master/scriptworker/constants.py#L228-L232 > # See https://github.com/mozilla-releng/pushapkscript#aurora-beta-release-vs-alpha-beta-production > PUSH_APK_GOOGLE_PLAY_TRACT = { >- 'aurora': 'beta', >+ 'central': 'beta', I have a NI about this, but it's probably correct.

indeed, not a big deal. thanks

(In reply to Aki Sasaki [:aki] from comment #32) > (In reply to Aki Sasaki [:aki] from comment #31) > > I'm also curious if we want the beta or alpha track on GP. > > Looks like Aurora is currently set to 'beta', so I'm guessing the 'beta' > track is correct. I'll leave the ni? here to doublecheck. Because the dry-run option is on, it's not mandatory at this point[1]. However, we may want to actually publish the APK to a subset of Aurora users. At this time, we'll have to use a closed alpha track[2]. If all goes well, we can put back the beta track. [1] https://github.com/mozilla-releng/pushapkscript/tree/678b1dca3881411b40544069cd42c0ec23d1ffc2#1-use-dry_run-true-in-your-task-definition [2] https://github.com/mozilla-releng/pushapkscript/tree/678b1dca3881411b40544069cd42c0ec23d1ffc2#2-push-to-a-closed-alpha-track

(In reply to Aki Sasaki [:aki] from comment #33) > Comment on attachment 8869521 [details] [diff] [review] > updated patch for fennec builds on m-c > > >diff --git a/README.txt b/README.txt > >--- a/README.txt > >+++ b/README.txt > >@@ -1,5 +1,3 @@ > >-An explanation of the Mozilla Source Code Directory Structure and links to > >-project pages with documentation can be found at: > > > > https://developer.mozilla.org/en/Mozilla_Source_Code_Directory_Structure > > > > Did you intend to remove these two lines? Whoops, that was a 'let-me-add-try-syntax' commit, heh. > >diff --git a/taskcluster/taskgraph/util/scriptworker.py b/taskcluster/taskgraph/util/scriptworker.py > >--- a/taskcluster/taskgraph/util/scriptworker.py > >+++ b/taskcluster/taskgraph/util/scriptworker.py > > PUSH_APK_SCOPE_ALIAS_TO_PROJECT = [[ > >- 'aurora', set([ > >- 'mozilla-aurora', > >+ 'central', set([ > >+ 'mozilla-central', > > ]) > > ], [ > > 'beta', set([ > >@@ -221,7 +224,7 @@ PUSH_APK_SCOPE_ALIAS_TO_PROJECT = [[ > > > > > > PUSH_APK_SCOPES = { > >- 'aurora': 'project:releng:googleplay:aurora', > >+ 'central': 'project:releng:googleplay:aurora', > > We're going to have to update scriptworker for this change. > Right now scriptworker.constants has the set of restricted scopes, as well > as which branches they map to. > > [1] is project:releng:googleplay:aurora, which maps to the 'auroratest' > alias, which maps to mozilla-aurora and date here [2]. We need to make sure > 'mozilla-central' is in the appropriate alias before we land on central, and > then push out a scriptworker dot release. > > [1] > https://github.com/mozilla-releng/scriptworker/blob/master/scriptworker/ > constants.py#L193 > [2] > https://github.com/mozilla-releng/scriptworker/blob/master/scriptworker/ > constants.py#L228-L232 I'll update these. > > # See https://github.com/mozilla-releng/pushapkscript#aurora-beta-release-vs-alpha-beta-production > > PUSH_APK_GOOGLE_PLAY_TRACT = { > >- 'aurora': 'beta', > >+ 'central': 'beta', > > I have a NI about this, but it's probably correct. (In reply to Johan Lorenzo [:jlorenzo] from comment #35) > (In reply to Aki Sasaki [:aki] from comment #32) > > (In reply to Aki Sasaki [:aki] from comment #31) > > > I'm also curious if we want the beta or alpha track on GP. > > > > Looks like Aurora is currently set to 'beta', so I'm guessing the 'beta' > > track is correct. I'll leave the ni? here to doublecheck. > > Because the dry-run option is on, it's not mandatory at this point[1]. > However, we may want to actually publish the APK to a subset of Aurora > users. At this time, we'll have to use a closed alpha track[2]. If all goes > well, we can put back the beta track. Thanks for the reminder, I've been having trouble keeping this straight. I'll switch this to "alpha" for now, and we can flip it later.

If this patch looks good I think we can land it after the scriptworker change is rolled out?

Comment on attachment 8870452 [details] [diff] [review] get rid of README change, switch to "alpha" google play track Interdiff looks good.

With Aki and Johan's help, we're getting the scriptworker patch landed. We'll need to land this once the new scriptworker tarballs in place. Once the new version is rolled out I think we can land the Gecko patch?

Comment on attachment 8870505 [details] [diff] [review] bump pushapkscriptworker version I'm sharing the trick :sfraser gave me once, reviews are not mandatory for version bump: https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/HowTo/Hack_on_PuppetAgain#Exceptions_to_Review_Requirement

I tried on Google play with the two following apk: <bhearsum> https://queue.taskcluster.net/v1/task/QHzUA0Q6StabGcR_713cRA/runs/0/artifacts/public/build/target.apk for api15 <bhearsum> https://queue.taskcluster.net/v1/task/WUPLOjK3T1q10-xlad_xzA/runs/0/artifacts/public/build/target.apk for x86 I used the alpha track of the application Firefox aurora with a Closed alpha testing using my account. The upload worked perfectly. I will wait for the app to be available on my android system to check it.

And this is working! I have "Date nightly" on my android, seems to work correctly. Well done!

Pushed by bhearsum@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/301e2cdd5d30 switch mozilla-central fennec builds to org.mozilla.fennec_aurora id and add new builds for org.mozilla.fennec. r=aki,sylvestre,jlorenzo a=dawn

Pushed by bhearsum@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/1a735bdbbfa9 fix flake8 error. a=dawn DONTBUILD

Comment on attachment 8870452 [details] [diff] [review] get rid of README change, switch to "alpha" google play track https://hg.mozilla.org/mozilla-central/rev/301e2cdd5d30ba3b8545c4e7ce9a485f64468aef And a flake8 fix: https://hg.mozilla.org/mozilla-central/rev/1a735bdbbfa9a11e78b7e977d45cf00af85e7b66

Pushed by bhearsum@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/0c52dd8e08a8 fix missing commas causing errors in the nightly decision task. a=dawn DONTBUILD https://hg.mozilla.org/mozilla-central/rev/74de74d7808b Backout for android mochitest bustage. https://hg.mozilla.org/mozilla-central/rev/b7eca9a6017d Backout for android mochitest bustage. https://hg.mozilla.org/mozilla-central/rev/37d777d87200 Backout for android mochitest bustage.

I had to back out for major mochitest bustage. It looks like the default browser logic is being triggered, which causes a support.mozilla.org to happen, which crashes (by design) the test suite. Eg: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&selectedJob=101626233 This didn't show up on Date nor on Try. I could really use a hand debugging this, I've been groveling around the default browser code, but I'm not making any progress.

(In reply to Ben Hearsum (:bhearsum) from comment #48) > I had to back out for major mochitest bustage. It looks like the default > browser logic is being triggered, which causes a support.mozilla.org to > happen, which crashes (by design) the test suite. Eg: > https://treeherder.mozilla.org/#/jobs?repo=mozilla- > central&selectedJob=101626233 > > This didn't show up on Date nor on Try. I could really use a hand debugging > this, I've been groveling around the default browser code, but I'm not > making any progress. My gut feeling is this is "CLOBBER REQUIRED" only. (try clearing the tc caches and touching the CLOBBER file) since a lot of app id stuff is embedded amongst the Java iirc

(In reply to Justin Wood (:Callek) from comment #49) > (In reply to Ben Hearsum (:bhearsum) from comment #48) > > I had to back out for major mochitest bustage. It looks like the default > > browser logic is being triggered, which causes a support.mozilla.org to > > happen, which crashes (by design) the test suite. Eg: > > https://treeherder.mozilla.org/#/jobs?repo=mozilla- > > central&selectedJob=101626233 > > > > This didn't show up on Date nor on Try. I could really use a hand debugging > > this, I've been groveling around the default browser code, but I'm not > > making any progress. > > My gut feeling is this is "CLOBBER REQUIRED" only. (try clearing the tc > caches and touching the CLOBBER file) since a lot of app id stuff is > embedded amongst the Java iirc I doubt it, the Java bits are robust in the face of things that would change AndroidManifest.xml. From the logs, I see: [task 2017-05-24T15:43:16.683265Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): failed to check for update: [task 2017-05-24T15:43:16.683619Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): java.lang.IllegalArgumentException: scheme == null [task 2017-05-24T15:43:16.683915Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at java.net.ProxySelectorImpl.selectOneProxy(ProxySelectorImpl.java:41) [task 2017-05-24T15:43:16.684195Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at java.net.ProxySelectorImpl.select(ProxySelectorImpl.java:32) [task 2017-05-24T15:43:16.684667Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at org.mozilla.gecko.util.ProxySelector.openConnectionWithProxy(ProxySelector.java:36) [task 2017-05-24T15:43:16.684785Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at org.mozilla.gecko.updater.UpdateService.findUpdate(UpdateService.java:388) [task 2017-05-24T15:43:16.685020Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at org.mozilla.gecko.updater.UpdateService.startUpdate(UpdateService.java:280) [task 2017-05-24T15:43:16.685316Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at org.mozilla.gecko.updater.UpdateService.registerForUpdates(UpdateService.java:250) [task 2017-05-24T15:43:16.685525Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at org.mozilla.gecko.updater.UpdateService.onHandleIntent(UpdateService.java:203) [task 2017-05-24T15:43:16.686159Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:65) [task 2017-05-24T15:43:16.686242Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at android.os.Handler.dispatchMessage(Handler.java:99) [task 2017-05-24T15:43:16.686571Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at android.os.Looper.loop(Looper.java:137) [task 2017-05-24T15:43:16.686899Z] 15:43:16 INFO - 05-24 15:42:46.714 E/UpdateService( 2020): at android.os.HandlerThread.run(HandlerThread.java:60) [task 2017-05-24T15:43:16.686972Z] 15:43:16 INFO - 05-24 15:42:46.714 I/UpdateService( 2020): no update available [task 2017-05-24T15:43:16.687117Z] 15:43:16 INFO - 05-24 15:42:47.553 I/GeckoConsole( 1932): PAC file installed from data: URI [task 2017-05-24T15:43:16.687332Z] 15:43:16 INFO - 05-24 15:42:48.354 I/GeckoView( 1932): zerdatime 271361 - chrome startup finished That suggests that the PAC file is being installed _after_ the UpdateService kicks in, although log messages are serialized in interesting ways sometimes. I will try to look further shortly.

Aki, there's a few things we found upon landing, most notably: * Scope errors in the nightly decision task. I'm pretty sure these were caused by missing commas in BALROG_CHANNEL_SCOPES. I was surprised to find a bunch of missing commas in the "default" section too. I'm guessing those scopes had gone previous unused. * Permafailures on mochitests. This was caused by bug 1357356, which implemented a what's new page for Fennec. Sebastian Kaspari provided me with the fix for BrowserApp.java for that. My latest two try pushes on https://treeherder.mozilla.org/#/jobs?repo=try&author=bhearsum@mozilla.com seem to confirm that it works. I also had to fix a minor flake8 error. I wanted to run this patch by you because I'm not 100% sure about the comma fixes in scriptworker.py.

Comment on attachment 8870951 [details] [diff] [review] fix scopes, flake8 issues, and test failures We may still have to adjust role scopes to pass, but these look like good changes.

Pushed by bhearsum@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/cb321835e071 switch mozilla-central fennec builds to org.mozilla.fennec_aurora id and add new builds for org.mozilla.fennec. r=aki,sylvestre,jlorenzo a=dawn

Comment on attachment 8870951 [details] [diff] [review] fix scopes, flake8 issues, and test failures https://hg.mozilla.org/mozilla-central/rev/cb321835e07108b6aa4c91b7fd5a1fee5ee0b68f

The latest patch looks much better. The test failures are fixed, and the nightlies appear to be working fine. The push apk job correctly fired, but it still has the dryrun flag set. I'll flip that tomorrow and trigger more nightlies so we can see an actual submission to the "alpha" track. Balrog submission also looks correct and I intend to set-up test channels tomorrow so we can do update testing there.

Not planning to push this one until tomorrow.

(In reply to Ben Hearsum (:bhearsum) from comment #56) > Created attachment 8871000 [details] [diff] [review] > flip dryrun flag for central push apk > > Not planning to push this one until tomorrow. Aiui, this will push to Google Play automatically once landed + we spin nightlies.

(In reply to Aki Sasaki [:aki] from comment #57) > (In reply to Ben Hearsum (:bhearsum) from comment #56) > > Created attachment 8871000 [details] [diff] [review] > > flip dryrun flag for central push apk > > > > Not planning to push this one until tomorrow. > > Aiui, this will push to Google Play automatically once landed + we spin > nightlies. Yup. But we're still shipping to the "alpha" track, so it won't ship to any real users AFAIK.

Pushed by bhearsum@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/e8da7192201e enable pushes of new Fennec nightlies to Google Play store. r=aki a=dawn

Things are looking pretty good today. The regular nightlies are posting to Balrog and Google Play (on the alpha track), and the OldId builds are posting to Balrog. I've got rules set-up on "auroratest" and "nightlytest" to test the Balrog rules, but I've been unable to change the channel of a Fennec build to actually do that - I'm trying to get some QE help to do this. I'm also trying to test the Google Play updates, and waiting on the new APK to propagate. Liz had to enable some permissions for the releng-automation-aurora account in Google Play to get the upload to work, but that seems fine now. If all of the testing goes well we can make the rule changes from comment #16 on the live channels and then we're done!

(In reply to Ben Hearsum (:bhearsum) from comment #60) > Things are looking pretty good today. The regular nightlies are posting to > Balrog and Google Play (on the alpha track), and the OldId builds are > posting to Balrog. I've got rules set-up on "auroratest" and "nightlytest" > to test the Balrog rules, but I've been unable to change the channel of a > Fennec build to actually do that - I'm trying to get some QE help to do > this. I'm also trying to test the Google Play updates, and waiting on the > new APK to propagate. > > Liz had to enable some permissions for the releng-automation-aurora account > in Google Play to get the upload to work, but that seems fine now. > > If all of the testing goes well we can make the rule changes from comment > #16 on the live channels and then we're done! We also need to update https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/util/scriptworker.py#l234 to change back to the "beta" track, and possibly set a rollout_percentage for now. Sylvestre pointed out that we need to wait until some other parts of the app in the Play Store to be updated before we do this, though.

I verified the org.mozilla.fennec_aurora Google Play updates. Branding looks right on the installed app, and everything functions fine. I'd still like a second set of eyes on it if possible though. And we're still waiting on Softvision to test the Balrog updates for both org.mozilla.fennec_aurora and org.mozilla.fennec.

(In reply to Ben Hearsum (:bhearsum) from comment #62) > I verified the org.mozilla.fennec_aurora Google Play updates. Branding looks > right on the installed app, and everything functions fine. I'd still like a > second set of eyes on it if possible though. And we're still waiting on > Softvision to test the Balrog updates for both org.mozilla.fennec_aurora and > org.mozilla.fennec. bhearsum: link to before and after APKs for fennec_aurora and fennec, please?

(In reply to Nick Alexander :nalexander from comment #63) > (In reply to Ben Hearsum (:bhearsum) from comment #62) > > I verified the org.mozilla.fennec_aurora Google Play updates. Branding looks > > right on the installed app, and everything functions fine. I'd still like a > > second set of eyes on it if possible though. And we're still waiting on > > Softvision to test the Balrog updates for both org.mozilla.fennec_aurora and > > org.mozilla.fennec. > > bhearsum: link to before and after APKs for fennec_aurora and fennec, please? Before (fennec_aurora): https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-15-07-47-11-mozilla-aurora-android-api-15/fennec-54.0a2.multi.android-arm.apk and https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-15-07-47-11-mozilla-aurora-android-x86/fennec-54.0a2.multi.android-i386.apk After (fennec_aurora): https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-24-15-20-55-mozilla-central-android-api-15/fennec-55.0a1.multi.android-arm.apk and https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-24-15-20-55-mozilla-central-android-x86/fennec-55.0a1.multi.android-i386.apk Before (fennnec): https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-24-10-02-19-mozilla-central-android-api-15/fennec-55.0a1.multi.android-arm.apk and https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-24-10-02-19-mozilla-central-android-x86/fennec-55.0a1.multi.android-i386.apk After (fennec): https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-24-15-20-55-mozilla-central-android-api-15-old-id/fennec-55.0a1.multi.android-arm.apk and https://archive.mozilla.org/pub/mobile/nightly/2017/05/2017-05-24-15-20-55-mozilla-central-android-x86-old-id/fennec-55.0a1.multi.android-i386.apk

(In reply to Ben Hearsum (:bhearsum) from comment #64) > (In reply to Nick Alexander :nalexander from comment #63) > > (In reply to Ben Hearsum (:bhearsum) from comment #62) > > > I verified the org.mozilla.fennec_aurora Google Play updates. Branding looks > > > right on the installed app, and everything functions fine. I'd still like a > > > second set of eyes on it if possible though. And we're still waiting on > > > Softvision to test the Balrog updates for both org.mozilla.fennec_aurora and > > > org.mozilla.fennec. > > > > bhearsum: link to before and after APKs for fennec_aurora and fennec, please? > > Before (fennec_aurora): <snip> Thanks, Ben! I've looked at the manifests and I think I see: * fennec_aurora (after) growing features from fennec (before), which makes sense, since fennec (before) has more things enabled; * fennec_aurora (after) agreeing with fennec_aurora (before) for critical things like the package name and sharedUserId, which should allow us to smoothly pave over existing fennec_aurora installs and make them "Nightly" branded instead of "Aurora" branded. Roll on!

We enabled updates through Balrog for the "nightly", "aurora", and "nightly-old-id" channels. Other than discovering https://bugzilla.mozilla.org/show_bug.cgi?id=1368707, that went fine. The only thing left to do here is enable automatic pushes to the Google Play store. We're waiting for Sylvestre to do the first push manually (and update some metadata about the product) before we do that.

When we're ready to do it, I think this is what we need to do to enable automatic pushes?

Comment on attachment 8872652 [details] [diff] [review] switch rollout track to enable automatic pushes Johan just r+ IRL

Pushed by bhearsum@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/e15fb8406612 enable automatic shipping of new Fennec nightlies. r=jlorenzo a=sledru

Comment on attachment 8872652 [details] [diff] [review] switch rollout track to enable automatic pushes https://hg.mozilla.org/mozilla-central/rev/e15fb840661215dc3928c3bf1fea0ca0379511f9

(In reply to Ben Hearsum (:bhearsum) from comment #70) > Comment on attachment 8872652 [details] [diff] [review] > switch rollout track to enable automatic pushes > > https://hg.mozilla.org/mozilla-central/rev/ > e15fb840661215dc3928c3bf1fea0ca0379511f9 We needed https://bugzilla.mozilla.org/show_bug.cgi?id=1368728 and for Sylvestre to disable the "alpha" track on Google Play before this worked, but the latest set of nightlies published automatically \o/. I think we're all done here now. Big thanks to Aki and Johan for helping me teach me about some parts of Fennec automation and Google Play, and to everyone who helped verify this.