There are no single-locale Android builds for aurora in http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-aurora-android-l10n/ . The dated directories for single-locale aurora Android builds are also missing. We need these for testing new locales to be added to the multi-locale build before Fx 24 migration day please.

Looking through the logs in http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-aurora-l10n/ the latest successful builds were 20130624004020. 20130625004016 fails for all locales. http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-aurora-l10n/mozilla-aurora-android-l10n_5-unknown-bm58-build1-build6.txt.gz has this error message for sv-SE, and similar for other locales: 02:12:04 ERROR - Traceback (most recent call last): 02:12:04 INFO - File "/builds/slave/m-aurora-and-l10n_5-0000000000/build/mozilla-aurora/toolkit/mozapps/installer/l10n-repack.py", line 47, in <module> 02:12:04 INFO - main() 02:12:04 INFO - File "/builds/slave/m-aurora-and-l10n_5-0000000000/build/mozilla-aurora/toolkit/mozapps/installer/l10n-repack.py", line 43, in main 02:12:04 INFO - l10n.repack(args.build, args.l10n, args.non_resource, NON_CHROME) 02:12:04 INFO - File "/builds/slave/m-aurora-and-l10n_5-0000000000/build/mozilla-aurora/python/mozbuild/mozpack/packager/l10n.py", line 172, in repack 02:12:04 INFO - _repack(app_finder, l10n_finder, copier, formatter, non_chrome) 02:12:04 INFO - File "/builds/slave/m-aurora-and-l10n_5-0000000000/build/mozilla-aurora/python/mozbuild/mozpack/packager/l10n.py", line 83, in _repack 02:12:04 INFO - for e in app.entries 02:12:04 INFO - File "/builds/slave/m-aurora-and-l10n_5-0000000000/build/mozilla-aurora/python/mozbuild/mozpack/packager/l10n.py", line 84, in <genexpr> 02:12:04 INFO - if isinstance(e, ManifestEntryWithRelPath)) 02:12:04 INFO - KeyError: 'assets' 02:12:05 ERROR - make[1]: *** [repackage-zip] Error 1 02:12:05 INFO - make[1]: Leaving directory `/builds/slave/m-aurora-and-l10n_5-0000000000/build/mozilla-aurora/obj-l10n/mobile/android/locales'

June 24th was the last uplift to Aurora, so we've been broken since that. I don't have insight into the actual failure, though. Glandium, Aki - any ideas? I don't see much in the way of changes to https://hg.mozilla.org/releases/mozilla-aurora/log/c19e71975e7f/toolkit/mozapps/installer/l10n-repack.py or https://github.com/mozilla/build-mozharness/commits/master/scripts/mobile_l10n.py.

1) go to http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/mozilla-aurora-l10n/?C=M;O=D (for aurora; central would s,aurora,central,) 2) click on log, search for "ERROR -" 02:10:10 INFO - /builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/obj-l10n/_virtualenv/bin/python /builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/toolkit/mozapps/installer/l10n-repack.py /builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/obj-l10n/mobile/android/locales/../../../dist/l10n-stage/fennec ../../../dist/xpi-stage/locale-ach \ 02:10:11 ERROR - Traceback (most recent call last): 02:10:11 INFO - File "/builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/toolkit/mozapps/installer/l10n-repack.py", line 47, in <module> 02:10:11 INFO - main() 02:10:11 INFO - File "/builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/toolkit/mozapps/installer/l10n-repack.py", line 43, in main 02:10:11 INFO - l10n.repack(args.build, args.l10n, args.non_resource, NON_CHROME) 02:10:11 INFO - File "/builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/python/mozbuild/mozpack/packager/l10n.py", line 172, in repack 02:10:11 INFO - _repack(app_finder, l10n_finder, copier, formatter, non_chrome) 02:10:11 INFO - File "/builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/python/mozbuild/mozpack/packager/l10n.py", line 83, in _repack 02:10:11 INFO - for e in app.entries 02:10:11 INFO - File "/builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/python/mozbuild/mozpack/packager/l10n.py", line 84, in <genexpr> 02:10:11 INFO - if isinstance(e, ManifestEntryWithRelPath)) 02:10:11 INFO - KeyError: 'assets' 02:10:11 ERROR - make[1]: *** [repackage-zip] Error 1 02:10:11 INFO - make[1]: Leaving directory `/builds/slave/m-aurora-and-l10n_1-0000000000/build/mozilla-aurora/obj-l10n/mobile/android/locales' 02:10:11 INFO - make: *** [repackage-zip-ach] Error 2 02:10:11 ERROR - Return code: 2 This error seems to be responsible for many, if not all, of the bustages. 3) wish that we had a better way of displaying these errors than manual log searching... new tbpl?

OK, this doesn't look like a RelEng issue...

the catalan locale is ready for testing ( http://hg.mozilla.org/releases/l10n/mozilla-aurora/ca/file/6900afc7b5ba ). it'd help a lot having these builds.

This sounds related to bug 887121.

Bug 873569, in fact.

I've requested uplift for Bug 887121 and Bug 887115, which together address this issue. (I intended to uplift those two earlier, and forgot.) Thanks for bringing this to my attention, glandium.

Adding verifyme to confirm the builds are present after bug 887121 and bug 887115 land.

I can see builds here on the FTP http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android-l10n/

And more importantly, here also :) http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-aurora-android-l10n/

Yeah, terrible copy & paste (but I did check the right folder at least) :-)

I'm going to mark this fixed, since we clearly have builds in the relevant directories. This is marked 'verifyme'; is that good enough to get QA to check that the builds can be installed and run?

lsblakk: I'm still getting tracking updates about this. Does this need a further uplift? If not, could you update the tracking flags? Thanks!

Marking 24 as fixed given comment #8 is addressed.

This is has become an issue again. ftp://ftp.mozilla.org/pub/mobile/nightly/latest-mozilla-aurora-android-l10n/

I think this is a different issue, I'll file a new bug.