Google announced the "Publishing API": https://developers.google.com/android-publisher/ It enables "Uploading new versions of an app" We, the release team, would appreciate to have such feature. For now, the beta & release uploads of the APKs are done by hand. It wastes our time and introduce the risk of human error. Just like we do with Desktop, we would like a more transparent push (why not as part of the push to mirror) Thanks to Benjamin Kerensa for the info.

This is a high priority from relman's perspective as we do spend a fair amount of time uploading to the Play store, this does introduce the potential for human error (although the Play store is pretty good about catching this), and we are a bottleneck in the release process. I think this is the API method that is of interest: https://developers.google.com/android-publisher/api-ref/edits/apks For v1, we can simply upload the APKs. For v2, I would like to see us pull recent changes information from Nucleus (will require Nucleus changes): https://developers.google.com/android-publisher/api-ref/edits/apklistings

This could be also the first step to see aurora in the Play store

Lets take a look this quarter.

I am going to give it a try.

You could set up a test account, or a test product on the existing account to do the development here. The latter may need builds which have a different identifier than org.mozilla.firefox.

(In reply to Nick Thomas [:nthomas] from comment #5) > You could set up a test account Yes but it is not free ($25) > or a test product on the existing account to do the development here. That is what I am planning to do.

Here is a first proposition of the Google play upload. That works fine for me. Some information: * This is the first time that I use mozharness. Don't hesitate to let me know if I used it incorrectly. * I have been testing using the old (and not published) aurora app (org.mozilla.fennec_aurora). * I manage the various google play track (Production, beta or alpha). * I don't really know how sanity checks are done in mozharness. So, I wrote a method to do it. * You will need the client_secrets.json to test (give me your PGP key if you want it). * I haven't find how to do use a list of final arguments like: python push_apk.py --package_name org.mozilla.fennec_aurora --push_apk fennec-35.0a2.multi.android-i386.apk fennec-35.0a2.multi.android-arm.apk Instead, I did: python push_apk.py --package_name org.mozilla.fennec_aurora --push_apk --apk-x86 fennec-35.0a2.multi.android-i386.apk --apk-armv7 fennec-35.0a2.multi.android-arm.apk (note that the arch doesn't matter) * I added an optional arg to manage armv6 files (--apk-armv6). As a sample, the command: python push_apk.py --package_name org.mozilla.fennec_aurora --apk-x86 fennec-35.0a2.multi.android-i386.apk --apk-armv7 fennec-35.0a2.multi.android-arm.apk --push_apk gives: 19:19:05 INFO - MultiFileLogger online at 20141027 19:19:05 in /home/sylvestre/dev/mozilla/push-apk/mozharness/scripts 19:19:05 INFO - Run as push_apk.py --package_name org.mozilla.fennec_aurora --apk-x86 fennec-35.0a2.multi.android-i386.apk --apk-armv7 fennec-35.0a2.multi.android-arm.apk --push_apk 19:19:05 INFO - Dumping config to /home/sylvestre/dev/mozilla/push-apk/mozharness/scripts/logs/localconfig.json. 19:19:05 INFO - {'apk_file_armv7': 'fennec-35.0a2.multi.android-arm.apk', 19:19:05 INFO - 'apk_file_x86': 'fennec-35.0a2.multi.android-i386.apk', 19:19:05 INFO - 'append_to_log': False, 19:19:05 INFO - 'base_work_dir': '/home/sylvestre/dev/mozilla/push-apk/mozharness/scripts', 19:19:05 INFO - 'client_secrets': 'client_secrets.json', 19:19:05 INFO - 'debug_build': False, 19:19:05 INFO - 'log_level': 'info', 19:19:05 INFO - 'log_to_console': True, 19:19:05 INFO - 'opt_config_files': (), 19:19:05 INFO - 'package_name': 'org.mozilla.fennec_aurora', 19:19:05 INFO - 'track': 'alpha', 19:19:05 INFO - 'volatile_config': {'actions': ('push_apk',), 19:19:05 INFO - 'add_actions': None, 19:19:05 INFO - 'no_actions': None}, 19:19:05 INFO - 'work_dir': 'build'} 19:19:05 INFO - ##### 19:19:05 INFO - ##### Running push_apk step. 19:19:05 INFO - ##### 19:19:05 INFO - Running main action method: push_apk 19:19:26 INFO - Version code 2014102200 has been uploaded.Filename "fennec-35.0a2.multi.android-arm.apk" edit_id 04447795179430228625 19:19:47 INFO - Version code 2014102201 has been uploaded.Filename "fennec-35.0a2.multi.android-i386.apk" edit_id 04447795179430228625 19:19:48 INFO - Application "org.mozilla.fennec_aurora" set to track "alpha" for versions [2014102200, 2014102201] 19:20:06 INFO - Edit "04447795179430228625" has been committed 19:20:06 INFO - ##### 19:20:06 INFO - ##### Skipping test step. 19:20:06 INFO - ##### 19:20:06 INFO - Copying logs to upload dir...

(In reply to Sylvestre Ledru [:sylvestre] from comment #8) > * I haven't find how to do use a list of final arguments like: > python push_apk.py --package_name org.mozilla.fennec_aurora --push_apk > fennec-35.0a2.multi.android-i386.apk fennec-35.0a2.multi.android-arm.apk > Instead, I did: > python push_apk.py --package_name org.mozilla.fennec_aurora --push_apk > --apk-x86 fennec-35.0a2.multi.android-i386.apk --apk-armv7 > fennec-35.0a2.multi.android-arm.apk > (note that the arch doesn't matter) The normal way of doing this is by passing the --foo more than once. Eg: --push_apk foo.apk --push_apk bar.apk. I don't know if Mozharness supports that, but it's worth a try...

Comment on attachment 8512121 [details] [diff] [review] push_apk.diff Review of attachment 8512121 [details] [diff] [review]: ----------------------------------------------------------------- This is looking pretty darn good! I haven't tried to run it yet, but I wanted to give some quick feedback first. I'll poke you on IRC about getting the secrets. ::: scripts/push_apk.py @@ +55,5 @@ > + # We are not using alpha but we default to it to avoid mistake > + "default": "alpha" > + }], > + [["--auth"], { > + "dest": "client_secrets", Please make the arg name and destination more closely related in name. It's weird to pass --auth, and then need to look for client_secrets when reading the code. @@ +59,5 @@ > + "dest": "client_secrets", > + "help": "The JSON authentication file", > + "default": "client_secrets.json" > + }], > + [["--package_name"], { Not a blocker, but --package-name is a much more standard way of doing args. If you want to keep the _ that's fine though. @@ +88,5 @@ > + package_name_values = ("org.mozilla.fennec_aurora", > + "org.mozilla.firefox_beta", > + "org.mozilla.firefox") > + > + def __init__(self, require_config_file=False, config={}, We may end up needing a config file for package_name rather than passing it on the command line. Not sure, so this is fine for now. We should know for sure when this gets integrated with the buildbot parts of the release automation. @@ +122,5 @@ > + if not os.path.isfile(self.config['apk_file_x86']): > + self.fatal("Could not find " + self.config['apk_file_x86']) > + > + if not os.path.isfile(self.config['apk_file_armv7']): > + self.fatal("Could not find " + self.config['apk_file_armv7']) You should check the armv6 one too, if it exists. Or we can just dump armv6 support completely, seeing as it dies in the not-too-far-off future. @@ +143,5 @@ > + formatter_class=argparse.RawDescriptionHelpFormatter, > + parents=parent_parsers) > + # Using parse_known_args to make sure it does not fail with the > + # mozharness args > + flags, extra = parser.parse_known_args(sys.argv[1:]) A few things on this method: 1) Accessing sys.argv directly here is really weird. If you need stuff from args, it should be stored and used from the config IMO. 2) Am I reading this right that the Mozharness args (like --push_apk) mirror some flags in the google play api methods? If so, that seems like a bad assumption to make - it'll be more difficult to update this script if the google api changes. 3) It should probably go in a Mixin class, so that it can be used in other scripts (which I'm sure we'll have in the future). BalrogMixin might be a nice simple example of Mixin usage in Mozharness. @@ +232,5 @@ > + > + def test(self): > + """ Test if the connexion can be done """ > + self.check_argument() > + self.connect_to_play() This is a really useful method :)

Here it is!

Jordan, we could use some advice here re: using 3rd party libraries in Mozharness. Sylvestre's patch depends on a few, and it's not really possible to implement this without at least oauth2. This is expected to be run through Buildbot (probably ScriptFactory) eventually, if that matters.

I am using python-googleapi from Debian packages (1.2-3).

(In reply to Ben Hearsum [:bhearsum] from comment #12) > Jordan, we could use some advice here re: using 3rd party libraries in > Mozharness. Sylvestre's patch depends on a few, and it's not really possible > to implement this without at least oauth2. This is expected to be run > through Buildbot (probably ScriptFactory) eventually, if that matters. since it is third party and a specific need for this script, it may be best to isolate this via creating a virtualenv within the work_dir and installing google-api-python-client via or pypi server. * Note this would require us to push a tar ball of the package up to http://pypi.pvt.build.mozilla.org/pub/ so it is available would that solve your issue?

(In reply to Jordan Lund (:jlund) from comment #14) > (In reply to Ben Hearsum [:bhearsum] from comment #12) > > Jordan, we could use some advice here re: using 3rd party libraries in > > Mozharness. Sylvestre's patch depends on a few, and it's not really possible > > to implement this without at least oauth2. This is expected to be run > > through Buildbot (probably ScriptFactory) eventually, if that matters. > > since it is third party and a specific need for this script, it may be best > to isolate this via creating a virtualenv within the work_dir and installing > google-api-python-client via or pypi server. > to create a virtualenv in mozharness, inherit from this mixin[1] and add 'create-virtualenv' to your list of actions. You will also want to add to your config: { # this will pip install it automajically when we call the create-virtualenv action 'virtualenv_modules': ['google-api-python-client'], "find_links": [ # so mozharness knows where to look for the package "http://pypi.pvt.build.mozilla.org/pub", "http://pypi.pub.build.mozilla.org/pub", ], # the path inside the work_dir ('build') of where we will install the env. # pretty sure it's the default and not needed. 'virtual_env': 'venv', } if you are going to be using this on multiple platforms and the virtualenv executable is not in your path (i.e. windows), you will also want to add the following to your config. # At which point, we will want to have a config file per platform for this script rather than only # relying on the script for self.config items: 'exes': {'virtualenv': '/path/to/python /path/to/virtualenv'} as we won't be creating a virtualenv for subprocess commands (we need to avail of oauth2 within the mozharness script itself), we probably have to go this route[2] (recent patch from mshal): alternatively, instead of going with query_python_site_packages_path() and messing with sys.path, you could also activate the newly created virtualenv within the script[3]. This technique does require that you use the same version of python to call mozharness as the virtualenv you created[4]. When calling mozharnesss from ScriptFactory, our buildbot-configs usually use mozharness_python key as the interpreter. You just have to set that explicitly to what you want (e.g. a 2.7 one). That's good practice anyway instead of using the one from a slaves path. [1] http://mxr.mozilla.org/build/source/mozharness/mozharness/base/python.py?rev=678bbcd049ee#60 [2] https://bug1071281.bugzilla.mozilla.org/attachment.cgi?id=8511178 [3] http://mxr.mozilla.org/build/source/mozharness/mozharness/base/python.py#414 [4] https://bugzilla.mozilla.org/show_bug.cgi?id=1071281#c18

(In reply to Jordan Lund (:jlund) from comment #15) > (In reply to Jordan Lund (:jlund) from comment #14) > > (In reply to Ben Hearsum [:bhearsum] from comment #12) > > > Jordan, we could use some advice here re: using 3rd party libraries in > > > Mozharness. Sylvestre's patch depends on a few, and it's not really possible > > > to implement this without at least oauth2. This is expected to be run > > > through Buildbot (probably ScriptFactory) eventually, if that matters. > > > > since it is third party and a specific need for this script, it may be best > > to isolate this via creating a virtualenv within the work_dir and installing > > google-api-python-client via or pypi server. > > > > to create a virtualenv in mozharness, inherit from this mixin[1] and add > 'create-virtualenv' to your list of actions. You will also want to add to > your config: Does this strategy still work when the Mozharness script itself needs extra packages? Note that Sylvestre's script imports oauth2client at the top of the file.

(In reply to Ben Hearsum [:bhearsum] from comment #16) > (In reply to Jordan Lund (:jlund) from comment #15) > > (In reply to Jordan Lund (:jlund) from comment #14) > > > (In reply to Ben Hearsum [:bhearsum] from comment #12) > > > > Jordan, we could use some advice here re: using 3rd party libraries in > > > > Mozharness. Sylvestre's patch depends on a few, and it's not really possible > > > > to implement this without at least oauth2. This is expected to be run > > > > through Buildbot (probably ScriptFactory) eventually, if that matters. > > > > > > since it is third party and a specific need for this script, it may be best > > > to isolate this via creating a virtualenv within the work_dir and installing > > > google-api-python-client via or pypi server. > > > > > > > to create a virtualenv in mozharness, inherit from this mixin[1] and add > > 'create-virtualenv' to your list of actions. You will also want to add to > > your config: > > Does this strategy still work when the Mozharness script itself needs extra > packages? Note that Sylvestre's script imports oauth2client at the top of > the file. Buried in my last comment I noted that we have to do something special due to the fact that it is the script itself needing the extra module. It's not uncommon in mozharness to do the import after we create and activate the virtual env. Notice how here[1], we query_python_site_packages_path() before using the requests module. But I also noted that I feel it's better to activate the virtualenv into the currently running script[2] vs messing with the python interpreters sys.path. I like this as it isolates everything into the mozharness scrict and it can be taken out of our build pool, even run locally, if neeeded. If all of this rubs the wrong way, I'd imagine we will have to add the module to built in python on our create a special virtualenv on our slaves. Both I suppose would require puppet if this was linux only. Does that clear things up or is my logic flawed? :) [1] https://bug1071281.bugzilla.mozilla.org/attachment.cgi?id=8511178 [2] http://mxr.mozilla.org/build/source/mozharness/mozharness/base/python.py?rev=89a6b34448ab#446

I updated the auth mechanism to use pkcs 12. It needs: a account name + a key. The key can be downloaded from the Google play admin. Advantages: * it does not need to login on a website * the code is smaller and easier to read * the login+key are enough * Remove the arg stuff I added the option --service-account for the login and we are still using --credentials for the keyfile I also added the venv stuff but I don't see how this can be used python push_apk.py --package-name org.mozilla.fennec_aurora --apk-x86 fennec-35.0a2.multi.android-i386.apk --apk-armv7 fennec-35.0a2.multi.android-arm.apk --service-account foo@developer.gserviceaccount.com --credentials googleplay.p12 --push_apk

Using the virtual env

Comment on attachment 8517416 [details] [diff] [review] push_apk.diff Review of attachment 8517416 [details] [diff] [review]: ----------------------------------------------------------------- Sorry for the delay in reviewing this. ::: mozharness/mozilla/googleplay.py @@ +18,5 @@ > +from oauth2client import client > + > + > +# GooglePlayMixin {{{1 > +class GooglePlayMixin(object): This is so much nicer with the new authentication, thanks for digging into that! @@ +25,5 @@ > + """ Connect to the google play interface > + """ > + # Load the key in PKCS 12 format that you downloaded from the > + # Google APIs Console when you created your Service account. > + f = file(self.config["credentials"], 'rb') Now that you're in a mixin, you're sharing scope with (potentially) every single other mixin. It's best to add something that indicates this is for google play, eg, "google_play_credentials_file". r=me with that fixed (make sure you update the references in the script itself, too).

Comment on attachment 8517416 [details] [diff] [review] push_apk.diff Here it is: https://hg.mozilla.org/build/mozharness/rev/28e050ccc2aa

It is working. Champagne! python push_apk.py --package-name org.mozilla.firefox_beta --apk-x86 fennec-34.0b8.multi.android-i386.apk --apk-armv7 fennec-34.0b8.multi.android-arm.apk --service-account foo@developer.gserviceaccount.com --credentials googleplay.p12 --push_apk --track production 22:16:26 INFO - MultiFileLogger online at 20141113 22:16:26 in /tmp/mozharness/scripts 22:16:26 INFO - Run as push_apk.py --package-name org.mozilla.firefox_beta --apk-x86 fennec-34.0b8.multi.android-i386.apk --apk-armv7 fennec-34.0b8.multi.android-arm.apk --service-account foo@developer.gserviceaccount.com --credentials googleplay.p12 --push_apk --track production 22:16:26 INFO - Dumping config to /tmp/mozharness/scripts/logs/localconfig.json. 22:16:26 INFO - {'apk_file_armv7': 'fennec-34.0b8.multi.android-arm.apk', 22:16:26 INFO - 'apk_file_x86': 'fennec-34.0b8.multi.android-i386.apk', 22:16:26 INFO - 'append_to_log': False, 22:16:26 INFO - 'base_work_dir': '/tmp/mozharness/scripts', 22:16:26 INFO - 'debug_build': False, 22:16:26 INFO - 'find_links': ('http://pypi.pvt.build.mozilla.org/pub', 22:16:26 INFO - 'http://pypi.pub.build.mozilla.org/pub'), 22:16:26 INFO - 'google_play_credentials_file': 'googleplay.p12', 22:16:26 INFO - 'log_level': 'info', 22:16:26 INFO - 'log_to_console': True, 22:16:26 INFO - 'opt_config_files': (), 22:16:26 INFO - 'package_name': 'org.mozilla.firefox_beta', 22:16:26 INFO - 'pip_index': True, 22:16:26 INFO - 'service_account': 'foo@developer.gserviceaccount.com', 22:16:26 INFO - 'track': 'production', 22:16:26 INFO - 'virtualenv_modules': ('google-api-python-client',), 22:16:26 INFO - 'virtualenv_path': 'venv', 22:16:26 INFO - 'volatile_config': {'actions': ('push_apk',), 22:16:26 INFO - 'add_actions': None, 22:16:26 INFO - 'no_actions': None}, 22:16:26 INFO - 'work_dir': 'build'} 22:16:26 INFO - ##### 22:16:26 INFO - ##### Skipping create-virtualenv step. 22:16:26 INFO - ##### 22:16:26 INFO - ##### 22:16:26 INFO - ##### Running push_apk step. 22:16:26 INFO - ##### 22:16:26 INFO - Running main action method: push_apk 22:16:44 INFO - Version code 2014111217 has been uploaded. Filename "fennec-34.0b8.multi.android-arm.apk" edit_id 09894908989416713749 22:17:04 INFO - Version code 2014111218 has been uploaded. Filename "fennec-34.0b8.multi.android-i386.apk" edit_id 09894908989416713749 22:17:05 INFO - Application "org.mozilla.firefox_beta" set to track "production" for versions [2014111217, 2014111218] 22:17:11 INFO - Edit "09894908989416713749" has been committed 22:17:11 INFO - ##### 22:17:11 INFO - ##### Skipping test step. 22:17:11 INFO - ##### 22:17:11 INFO - Copying logs to upload dir...

I had to reenable the esr build (for android 2.2 and armv6) on the interface itself. Next time, I will use armv6 option to upload it.

(In reply to Sylvestre Ledru [:sylvestre] from comment #22) > It is working. Champagne! Hooray! Feel free to land this patch, then. We'll need to keep this bug open to figure out the buildbot integration piece though.

Oh, it is landed (comment #21) From the relman point of view, we would like the same thing as desktop. (ie, send an email saying please push Fennec version xx build y to the beta channel).

(see: https://travis-ci.org/petemoore/build-mozharness/builds/41038657)

Comment on attachment 8523147 [details] [diff] [review] bug1045531_add-httplib2-to-requirements-txt.patch Review of attachment 8523147 [details] [diff] [review]: ----------------------------------------------------------------- I'm not sure this is necessary. This script will likely end up installing requirements into a virtualenv...

Comment on attachment 8523147 [details] [diff] [review] bug1045531_add-httplib2-to-requirements-txt.patch I'm removing this request for now per the previous comment

Chris, Ben, do you think you will be able to use this script for beta 10 (or the next)? Thanks

(In reply to Sylvestre Ledru [:sylvestre] from comment #30) > Chris, Ben, do you think you will be able to use this script for beta 10 (or > the next)? > Thanks It's usually Nick and Massimo shipping releases, but I doubt it. This still needs buildbot integration, and probably some additional testing before that can happen.

We're pretty busy the next week or so with the release but given that Sylvestre has done his part of the work, it would be good to complete this bug by getting it integrated. Is there additional testing that Sylvestre can do to help push this along? Anything that he can help do wrt buildbot integration?

(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #32) > We're pretty busy the next week or so with the release but given that > Sylvestre has done his part of the work, it would be good to complete this > bug by getting it integrated. Is there additional testing that Sylvestre can > do to help push this along? Anything that he can help do wrt buildbot > integration? It's highly unlikely this is going to get integrated this week. Honestly, the integration piece is probably more work because the testing is so much more complicated. A few of us are already busy figuring out 34.1 release mechanics, and speaking for myself, I can't prioritize this higher than the other work on my plate. I'd like to get him up to speed on how to test things within Buildbot while we're in Portland. I don't like good work like this getting blocked either, but it's bound to happen as long as RelEng is on the hook for the integration pieces.

It looks like we need two main things here still: 1) Deploy credentials to build machines. 2) Write buildbot integration piece. We decided in an e-mail thread to not bother trying to test this in staging. Sylvestre has done manual testing of the script and it's simply not worth the effort trying to run this in staging. I'll try to push this past the finish line in the next couple of weeks.

I just had a discussion with some RelEng folks and deploying this just got more complicated. The current way we do pushes involves putting the necessary keys on all of our build machines. Many people are uncomfortable doing that with the Google Play key, so we're going to need some sort of more limited set of machines that we do these uploads on. Someone also suggested that release runner might be able to do it, which would probably mean a new button on Ship It. Given the above, this is looking like more work than I thought it would be. I may have to put it off depending how my other Q1 work is looking.

(In reply to Ben Hearsum [:bhearsum] from comment #35) > Someone also suggested that release runner might be able to do it, which would probably > mean a new button on Ship It. Having the capability to push an APK from ship-it would be "Wahou" for us (Release managers). What does it involve? Adding a new url for release-runners to scan, add a button in ship-it and update release-runners to watch this url? (or use the one for the build) I could give it a try.

(In reply to Sylvestre Ledru [:sylvestre] from comment #36) > (In reply to Ben Hearsum [:bhearsum] from comment #35) > > Someone also suggested that release runner might be able to do it, which would probably > > mean a new button on Ship It. > Having the capability to push an APK from ship-it would be "Wahou" for us > (Release managers). > What does it involve? Adding a new url for release-runners to scan, add a > button in ship-it and update release-runners to watch this url? (or use the > one for the build) > > I could give it a try. Rail, Catlee, and I talked about this a lot yesterday. This is probably going to be more involved than that, I really don't have a clear picture of what it will be yet. Probably need to talk to security folks first, too.

:fkang I was talking to coop about this and he suggested that this would be a good next bug to work on. The work in this bug is before we had release promotion. The work now would be to use scriptworker to upload apks to the play store via a task. Fennec doesn't use release promotion yet but if we switch the scheduling to TC for fennec, we can run jobs in either buildbot or TC.

I have implemented this here: https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/scripts/push_apk.py This is used by release management now. However, it is executed on my system and having a better integration would help. BTW, Having help in bug 1122509 would help this.

(In reply to Kim Moir [:kmoir] from comment #38) > :fkang > > I was talking to coop about this and he suggested that this would be a good > next bug to work on. The work in this bug is before we had release > promotion. The work now would be to use scriptworker to upload apks to the > play store via a task. Fennec doesn't use release promotion yet but if we > switch the scheduling to TC for fennec, we can run jobs in either buildbot > or TC. Kim, could you clarify where the apk would come from if we don't use release promotion? I'm not familiar with the fennec release process

Since the apk is not generated via release promotion yet, we could just focus on implementing the part that uploads the apk and revisit where the actual apk comes from later.

To clarify my previous comment, Sylvestre already had code the uploads the code, it would be good to get that in a format that we can use in a task for release promotion.

Francis, I think we should instead use this bug as meta and we should open a new bug to work on the integration of the script. wdyt ?

(In reply to Sylvestre Ledru [:sylvestre] [PTO until July 18th] from comment #43) > Francis, I think we should instead use this bug as meta and we should open a > new bug to work on the integration of the script. > wdyt ? Sure. works for me.

Found in triaging. The work for this has been completed in other bugs and it's fully working for a good while now. Thanks to :jlorenzo!