the locale picker has been turned off in bug 694047, until we figure out if or how it makes sense to package up which locales with the new android native UI builds. while we do the research to figure out how we want to do this, and create the right arrangement of locales, we will need individual locales to be packaged up and shipped for testing. this is to get this action item on the build teams priority list. we will probably have the first localizations ready for packaging and testing in a couple of weeks, as soon as a bit of the UI is nailed down, and the localization file structure is set up. will add dependencies for those bugs as needed.

Moving this to release engineering, and fixing up deps. releng, would you please file a bug on fennec native on what "APIs" you want for the single-locale builds? I.e., which make targets should do what? CC me on those, too?

(In reply to chris hofmann from comment #2) > we will probably have the first localizations ready for packaging and > testing in a couple of weeks, as soon as a bit of the UI is nailed down, and > the localization file structure is set up. will add dependencies for those > bugs as needed. Any dep.bugs where we can track progress for this?

We have never successfully repacked an apk; the main culprit, iirc, is the binary manifest. We may see more success if we repackage the apk using similar logic to a 'make package'. The main issue here is that we won't have a populated objdir in an l10n repack situation.

Hrm. Yeah, the resources.arsc is probably a bitch. Even if we find the android build fragment that generates that, we may very well end up with some android-sdk-internal java class that they change happily. I wonder if we can do something in the middle, like we do for tests. Create an intermediate tar ball and upload that to ftp and use it for packaging up single-locale builds.

More thoughts: We probably want the localized content to be in res/values/strings.xml and not res/values-ab-rCD/strings.xml, as that keeps the door open to not set the locale at all. Also, if the process here happens to recreated classes.dex, that'd probably be a feature. Going forward, we may want to switch our localization infrastructure to something that'd generate java code per locale.

Taking the releng portion; per today's meeting Axel's going to work on getting the repack working (thanks Axel!)

If Axel's right and the Android single locale makefiles are backwards compatible with existing logic, this may Just Work. I'll verify in staging.

Not needed, but I noticed this and we might as well clean this up.

IOError: [Errno 2] No such file or directory: 'buildbot-configs/mozilla2/android/mozilla-central/nightly/l10n-mozconfig' Gotta add these to my patch.

FYI, the only modification I'm doing to the mozconfig is to add a --with-l10n-base.

Copied macosx-mobile's l10n-mozconfigs and changed the --enable-application line.

Comment on attachment 579862 [details] [diff] [review] android + android-xul l10n mozconfigs r-, I'd rather use the actual build mozconfigs and tweak them. In particular, I'm not confident that --disable-compile-environment does the right thing 100%. Also, I think we want to make the change through the channels as native progresses, and not all at once, right?

I'd rather land them all at once, since we already have a large and growing number of items to remember to merge over the next 3 merge days. --disable-compile-environment works just fine for existing mobile l10n-mozconfigs, but I can look at copying the build mozconfigs.

(In reply to Axel Hecht [:Pike] from comment #12) > Comment on attachment 579862 [details] [diff] [review] > android + android-xul l10n mozconfigs > > r-, I'd rather use the actual build mozconfigs and tweak them. In > particular, I'm not confident that --disable-compile-environment does the > right thing 100%. Are you talking about tweaking at runtime?

Here's my thinking: --disable-compile-environment is for folks that don't do a whole lot of set up and install of compilers and dev lib packages and whatnot. Just a few usually available tools or mozillabuild. The android sdk kinda falls out of that, IMHO. Now, most checks in configure aren't reviewed with respect to --disable-compile-environment, and I confess I haven't bothered for probably a year if all the checks it's doing are in the right place. Thus my recommendation: For mobile, as it's requiring an SDK and stuff, let's not pretend we don't need a compile environment for the repacks. Practical implications for releng should be zilch I guess, as all slaves are set up to do anything anyway, right? Re the branch landings, if the mobile/android stuff isn't triggered before it's walking up the channels, I don't mind either way.

That makes complete sense to me, but I still don't understand what you mean by "I'd rather use the actual build mozconfigs and tweak them.". With this new context I _think_ it means: overwrite the existing l10n configs with the build configs, add whatever options we need for l10n, commit them. Is that right?

Yes.

Great! WFM too!

This is a bit more involved. I added the l10n-base, and otherwise kept the l10n-mozconfig the exact same as the existing nightly or release mozconfig, except I bumped the android sdk, tools, and toolchain to match Firefox 11. These won't be in use until Firefox 11 merges into those repos, anyway, so I'm just doing the legwork ahead of time. I'm going to attach a diff of each mozconfig vs each l10n-mozconfig right after this for easier review.

Output of for l in `find mozilla2/android{,-xul} -name l10n-mozconfig`; do n=`echo $l | sed -e 's/l10n-//'`; echo $l; diff -U9 $n $l ; done If I'm updating the mozilla-release mozconfigs ahead of time for l10n-mozconfig, I kind of might as well do it for the release and nightly mozconfigs too, since android and android-xul won't build on mozilla-release until Firefox 11 merges in. I'm happy to add that to the patch if wanted.

Comment on attachment 580500 [details] [diff] [review] android + android-xul l10n mozconfigs take 2 Review of attachment 580500 [details] [diff] [review]: ----------------------------------------------------------------- This looks very consistent and makes sense to me, which is as much as I understand about buildbot-configs. I'll make this a feedback+ flag instead of an r+

command: make wget-en-US command: cwd: mozilla-central/obj-l10n/mobile/android/locales command: env: {'UPLOAD_USER': 'ffxbld', 'EN_US_BINARY_URL': 'http://dev-stage01.build.sjc1.mozilla.com/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android', 'MOZ_OBJDIR': 'obj-l10n', 'UPLOAD_SSH_KEY': '~/.ssh/ffxbld_dsa', 'UPLOAD_HOST': 'dev-stage01.build.sjc1.mozilla.com', 'UPLOAD_TO_TEMP': '1'} command: output: /bin/sh: /builds/slave/m-cen-andrd-l10n-3/mozilla-central/obj-l10n/config/nsinstall: No such file or directory /builds/slave/m-cen-andrd-l10n-3/mozilla-central/obj-l10n/config/nsinstall -D /builds/slave/m-cen-andrd-l10n-3/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/ make: /builds/slave/m-cen-andrd-l10n-3/mozilla-central/obj-l10n/config/nsinstall: Command not found make: *** [wget-en-US] Error 127 command: END (0.07s elapsed)

[cltbld@linux-ix-slave05 obj-l10n]$ pwd /builds/slave/m-cen-andrd-l10n-3/mozilla-central/obj-l10n [cltbld@linux-ix-slave05 obj-l10n]$ find . -name nsinstall

Looks like a 'make nsinstall' in obj-l10n/config fixes this. Should I add this in the script, or should it be a makefile dependency?

We should add this to the script. either make -C config or make tier_base should do the trick. tier_base does a whole lot more things on android, which I'm not exactly sure why, so I don't know if using that is a pro or a con. tier_base does include config, though, so that'd be on the safe side.

I think that's going to get me past the repack portion; testing to verify. Another issue I'm hitting is that we're triggering the l10n repacks after the multilocale upload, not after the en-US upload, which is wrong. This will take a not-insignificant amount of MercurialBuildFactory refactoring, but I think I know what to do.

(In reply to Aki Sasaki [:aki] from comment #26) > I think that's going to get me past the repack portion; testing to verify. > Another issue I'm hitting is that we're triggering the l10n repacks after > the multilocale upload, not after the en-US upload, which is wrong. > This will take a not-insignificant amount of MercurialBuildFactory > refactoring, but I think I know what to do. Nope, not an issue: wget-en-US grabs the right apk. However, in 4 of the 5 repack scripts, during make unpack I'm hitting this: extracting: update.locale inflating: removed-files inflating: recommended-addons.json inflating: classes.dex /bin/sh: line 1: cd: fennec: No such file or directory make: *** [/builds/slave/m-cen-andrd-l10n-2/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec] Error 1 command: END (0.52s elapsed)

This should fix the en-US subdir issue when we build multilocale. However, I'm still hitting the above error after clobbering and retriggering.

(In reply to Aki Sasaki [:aki] from comment #27) > Nope, not an issue: wget-en-US grabs the right apk. > However, in 4 of the 5 repack scripts, during make unpack I'm hitting this: > > extracting: update.locale > inflating: removed-files > inflating: recommended-addons.json > inflating: classes.dex > /bin/sh: line 1: cd: fennec: No such file or directory > make: *** > [/builds/slave/m-cen-andrd-l10n-2/mozilla-central/obj-l10n/mobile/android/ > locales/../../../dist/l10n-stage/fennec] Error 1 > command: END (0.52s elapsed) Not sure what this means, in particular the "4 of the 5" part. Do you have STR?

PS: stas can reproduce this, it looks like a problem with having omnijar on now. The solution will be to make INNER_UNMAKE_PACKAGE not change the working directory.

(In reply to Axel Hecht [:Pike] from comment #29) > Not sure what this means, in particular the "4 of the 5" part. Do you have > STR? I think this was because I was futzing manually on the slave and not clobbering. When I rm -rf'ed all workdirs and kicked off all the scripts, I failed on 5/5. (In reply to Axel Hecht [:Pike] from comment #30) > PS: stas can reproduce this, it looks like a problem with having omnijar on > now. > > The solution will be to make INNER_UNMAKE_PACKAGE not change the working > directory. Awesome, good to hear!

Right now it looks like I need the patch in bug 709980 to land on m-c, then set JARSIGNER for Android repacks like we do for Android builds.

This isn't ideal, but saves me from having to add a SetProperty step to ScriptFactory, which would just make me feel dirty. (This needs to be an abs_path). If there's a lot of pushback here, I may end up porting these scripts to mozharness sooner rather than later.

I think we should have everything in place to do the single-locale builds on central. For aurora, the latest patch in bug 709980 still needs to land there as well.

Aurora landed, too.

I'm getting a lot of this: make clobber-zip AB_CD=en-US make[1]: Entering directory `/builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales' rm -f /builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/chrome/en-US.jar \ /builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/chrome/en-US.manifest \ /builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/defaults/pref/mobile-l10n.js rm -f -r /builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/dictionaries \ /builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/hyphenation \ /builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/defaults/profile \ /builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/chrome/en-US make[1]: Leaving directory `/builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales' make[1]: Entering directory `/builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales' make: Entering an unknown directory make: Leaving an unknown directory make: *** ../../../mobile/locales: No such file or directory. Stop. make[1]: *** [libs-da] Error 2 make[1]: Leaving directory `/builds/slave/m-aurora-andrd-l10n-1/mozilla-aurora/obj-l10n/mobile/android/locales' make: *** [repackage-zip-da] Error 2 command: END (0.61s elapsed) m-c too: make clobber-zip AB_CD=en-US make[1]: Entering directory `/builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales' rm -f /builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/chrome/en-US.jar \ /builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/chrome/en-US.manifest \ /builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/defaults/pref/mobile-l10n.js rm -f -r /builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/dictionaries \ /builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/hyphenation \ /builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/defaults/profile \ /builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec/chrome/en-US make[1]: Leaving directory `/builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales' make[1]: Entering directory `/builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales' make: Entering an unknown directory make: Leaving an unknown directory make: *** ../../../mobile/locales: No such file or directory. Stop. make[1]: *** [libs-sk] Error 2 make[1]: Leaving directory `/builds/slave/m-cen-andrd-l10n-4/mozilla-central/obj-l10n/mobile/android/locales' make: *** [repackage-zip-sk] Error 2 command: END (0.62s elapsed)

Filed bug 716842, with a patch. Sorry.

Now that various android + android-xul mozconfig fixes have landed, the diff between the mozconfigs and the l10n-mozconfigs is only a comment + the --with-l10n-base line.

So mobile/android is looking good. mobile/xul isn't: adding: update.locale (stored 0%) make: Entering an unknown directory make: Leaving an unknown directory make: *** /builds/slave/m-cen-andrd-xul-l10n-2/mozilla-central/obj-l10n/mobile/xul/locales/../../../embedding/android: No such file or directory. Stop. make[1]: Leaving directory `/builds/slave/m-cen-andrd-xul-l10n-2/mozilla-central/obj-l10n/mobile/xul/locales' make[1]: *** [repackage-zip] Error 2 make: *** [repackage-zip-et] Error 2 command: END (2.82s elapsed) I'm going to also make sure that my tools changes don't break mobile desktop l10n repacks, although we'll probably turn those off once this lands.

I'd think that embedding/android should be set up in http://mxr.mozilla.org/mozilla-central/source/toolkit/toolkit-makefiles.sh#621, as long as "$MOZ_WIDGET_TOOLKIT" = "android" and "$MOZ_BUILD_APP" = "mobile/xul". Is that true for how you trigger the xul repacks?

Ed apparently broke this in http://hg.mozilla.org/mozilla-central/rev/9a1a4157bf49, no bug, rs=build. wokbok:mozilla-central axelhecht$ hg diff toolkit/toolkit-makefiles.sh diff --git a/toolkit/toolkit-makefiles.sh b/toolkit/toolkit-makefiles.sh --- a/toolkit/toolkit-makefiles.sh +++ b/toolkit/toolkit-makefiles.sh @@ -626,7 +626,7 @@ netwerk/system/android/Makefile widget/android/Makefile " - if [ "$MOZ_BUILD_APP" = "mobile/xul" -o "$MOZ_BUILD_APP" = "b2g"]; then + if [ "$MOZ_BUILD_APP" = "mobile/xul" -o "$MOZ_BUILD_APP" = "b2g" ]; then add_makefiles " embedding/android/Makefile embedding/android/locales/Makefile is the difference between life and death. I'd be happy if someone could fix this while I'm in bed.

(In reply to Axel Hecht [:Pike] from comment #41) > - if [ "$MOZ_BUILD_APP" = "mobile/xul" -o "$MOZ_BUILD_APP" = "b2g"]; then > + if [ "$MOZ_BUILD_APP" = "mobile/xul" -o "$MOZ_BUILD_APP" = "b2g" ]; then > ... > is the difference between life and death. Sorry Axel/Aki, fixed in: https://hg.mozilla.org/mozilla-central/rev/b386494d97eb

Comment on attachment 580500 [details] [diff] [review] android + android-xul l10n mozconfigs take 2 http://hg.mozilla.org/build/buildbot-configs/rev/6a878eed7678

Comment on attachment 579500 [details] [diff] [review] just cleanup, buildbotcustom http://hg.mozilla.org/build/buildbotcustom/rev/dfdecce2fcae

This patch: * adds JARSIGNER which adds nightly-key signing for android (needed for nightlies; needed for releases if we want to test on the tegras); * edits the en-US binary url for nightlies that are multilocale (en-US subdir); * adds the 'config' directory to makeDirs for android only. I'm not fond of this, but right now I'm leaning towards leaving it for a followup bug so we can get single locale repacks. Tested android repacks (successful), linux mobile beta repacks (successful on the cmdln; the manually triggered ones weren't getting the script_repo_revision property for some reason, but I think that's unrelated to my patches). Android-xul is still failing, but I'm leaning towards filing a Fennec l10n bug and not blocking on that. Error: make[2]: Entering directory `/builds/slave/m-cen-andrd-xul-l10n-1/mozilla-central/obj-l10n/embedding/android' make[2]: Leaving directory `/builds/slave/m-cen-andrd-xul-l10n-1/mozilla-central/obj-l10n/embedding/android' make[2]: *** No rule to make target `res/values/strings.xml', needed by `gecko.ap_'. Stop. make[1]: *** [repackage-zip] Error 2 make[1]: Leaving directory `/builds/slave/m-cen-andrd-xul-l10n-1/mozilla-central/obj-l10n/mobile/xul/locales' make: *** [repackage-zip-es-ES] Error 2 command: END (2.03s elapsed)

This enables the nightly repacks for android and android-xul on m-c and m-a. If the android-xul error above is a blocking issue, I can wait to enable android-xul until it's resolved; otherwise I'm leaning towards turning them on.

(In reply to Aki Sasaki [:aki] from comment #45) > Android-xul is still failing, but I'm leaning towards filing a Fennec l10n > bug and not blocking on that. Filed bug 717522 for this issue as well as no xpis.

Comment on attachment 587900 [details] [diff] [review] working mobile l10n scripts http://hg.mozilla.org/build/tools/rev/b4d779f50414

Comment on attachment 587901 [details] [diff] [review] enable android and android-xul single locale nightly repacks http://hg.mozilla.org/build/buildbot-configs/rev/892ec4c2fc4e Android and android-xul single locale repacks should happen after the first m-c and m-a nightlies after the next reconfig (later today?)

We're live; we should get android single locale repacks tonight. I'll be working on android signing + update snippets next, then partner repacks.

Hey Aki, this is great news. where will the builds start showing up? some place lkke ftp://ftp.mozilla.org/pub/mobile/nightly/latest-mozilla-aurora-android/ ?

These seem not to be enabled in production for some reason. Reopening to track that down.

yeah! thanks a lot. same thing is needed for aurora since that is where most of the localization work happens. will aurora builds show up soon too?

Kicked off an aurora nightly (just now); when it finishes the l10n repack jobs should fire.