splitting out from bug#655129. For the faster release cadence, some locales are changing from developing on m-c to developing on mozilla-aurora. For those locales, RelEng needs to generate custom updates to migrate m-c l10n nightly m-c users to mozilla-aurora l10n nightly.

This is as simple as copying some set of snippets from mozilla-aurora to mozilla-central on AUS. No actual generation required.

(In reply to comment #1) > This is as simple as copying some set of snippets from mozilla-aurora to > mozilla-central on AUS. No actual generation required. Great to know, thanks nthomas. We'll just wait for the list of locales from bug#655129, and then do that.

per chofmann in bug#655129: For nightly users on mozilla-central, go ahead and do the following migrations: 1) migrate 1 locale on mozilla-central to en-US on mozilla-central. af 2) migrate 29 locales on mozilla-central to the same locale on aurora. ak as br bn-IN da eu gd gu-IN hy-AM is ka kk km kn lg mai mk mn mr nso oc or rm si son sw te ta-LK zu 3) at same time as these updates are published, stop generating nightly builds on mozilla-central for these 30 locales. (once we get these rolled out, chofmann will loop back with another batch of locales to migrate.)

'oc' is a stale locale, can we migrate those users off to en-US as well, please?

(In reply to Axel Hecht [:Pike] from comment #4) > 'oc' is a stale locale, can we migrate those users off to en-US as well, > please? sure, we havent started the migration work yet, so I'll update the locale lists as requested.

per chofmann in bug#655129, and axel in comment#4: For nightly users on mozilla-central, go ahead and do the following migrations: 1) migrate 2 locale on mozilla-central to en-US on mozilla-central. af oc 2) migrate 28 locales on mozilla-central to the same locale on aurora. ak as br bn-IN da eu gd gu-IN hy-AM is ka kk km kn lg mai mk mn mr nso or rm si son sw te ta-LK zu 3) at same time as these updates are published, stop generating nightly builds on mozilla-central for these 30 locales.

Some of those locales are on other products, too. If they want to stop working on central, those peers should probably agree, not sure if chofmann checked that. If so, we want to have mapping actions for those products. Here's the data: sea_central da ka si cal_central da eu gd is ka mk si ta-LK fx_central af ak as bn-IN br da eu gd gu-IN hy-AM is ka kk km kn lg mai mk mn mr nso oc or rm si son ta-LK te zu tb_central af br da eu gd is ka rm si ta-LK fennec_central da eu gd

(In reply to Nick Thomas [:nthomas] from comment #1) > This is as simple as copying some set of snippets from mozilla-aurora to > mozilla-central on AUS. No actual generation required. Lies! We need to change the channel-prefs.js file so that users request updates on the aurora instead of nightly. We can do this with a custom mar file, being careful that we need one for mac and one for everything else. Bug 694873 should really be resolved first, so that we keep serving the channel change for more than a few days.

I think I no longer need Khmer m-c builds.

(In reply to Nick Thomas [:nthomas] from comment #8) > (In reply to Nick Thomas [:nthomas] from comment #1) > > This is as simple as copying some set of snippets from mozilla-aurora to > > mozilla-central on AUS. No actual generation required. > > Lies! We need to change the channel-prefs.js file so that users request > updates on the aurora instead of nightly. We can do this with a custom mar > file, being careful that we need one for mac and one for everything else. > Bug 694873 should really be resolved first, so that we keep serving the > channel change for more than a few days. There are also checks for channel name, app version, and cert that occur now per requirements from Mozilla Security. cc'ing bbondy for input regarding this.

So as I understand you're changing some users from m-c to aurora for some locales. After that change you want that channel to accept MARs from the aurora channel only. You need to also: 1) Update update-settings.ini - ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-central + ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-aurora This indicates which types of MARs will be accepted moving forward. 2) If the updates are going to a user of m-c then you need to have a MARChannelName in the MAR set to firefox-mozilla-central. You can set a specific channel name on an unsigned (not signed) MAR by using: mar [-H MARChannelID] [-V ProductVersion] [-C workingDir] -i unsigned_archive_to_refresh.mar 3) The version which is contained inside the the MAR file needs to be higher than the version that the user will have installed when they take this update. You can refresh a product version on a MAR that hasn't been signed yet by using: mar [-H MARChannelID] [-V ProductVersion] [-C workingDir] -i unsigned_archive_to_refresh.mar 4) You will need to sign the MAR using the private key that is usually used for signing m-c builds. signmar [-C workingDir] -d NSSConfigDir -n certname -s archive.mar out_signed_archive.mar To get a signmar in your build you need to enable add this into your .mozconfig: ac_add_options --enable-signmar If you need to strip the signature off of an existing MAR you can do that with: signmar [-C workingDir] -r signed_input_archive.mar output_archive.mar All of this was written to the best of my knowledge, but I have never tried these steps. It is recommended to: i) Test the channel change itself before widely deploying ii) Test an aurora update after the channel change.

Do you know if the m-c builds you want to change have been getting Nightly updates? When is the last time they have been updated?

Also we would need to know if they plan on getting updates from m-c anytime soon.

Looking in http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-central-l10n/, and comparing to comment #6, we have builds from today for all but the sw locale - so yes, it's likely that the (very small) number of users will have been getting nighty updates, and also we might have some users with defaults/preferences/channel-prefs.js file from installs.

(In reply to Brian R. Bondy [:bbondy] from comment #13) > Also we would need to know if they plan on getting updates from m-c anytime > soon. If a valid translation of that question is 'will this bug be resolved before the next merge from central -> aurora' then no.

No I just wanted to know that if a Nightly update was not issued already for those locales, if it would be at some point soon, but as per Comment 14 one has already been issued.

Re-adding the dependencies now that they have been confirmed to affect this bug (see comment #14) due to the regression caused by bug 746156. The following bugs will all need to be fixed first before this bug: bug 756325 bug 759460 bug 759469 bug 760387

Brian, what's the behaviour if the user gets an unsigned or improperly signed mar? Is the update rejected?

I think we can probably generate a small mar that changes channel-prefs.js and update-settings.js, so that the app then queries and accepts mozilla-aurora updates.

The update would be rejected correct. I think you want to have the mar you're offering be signed with the m-c channel inside the mar, but have the files mentioned in Comment 19 have the aurora channel string.

:catlee, per meeting today, it looks like the locales in comment#6 are ready to migrate a) from locales -> en-US or b) from m-c --> m-aurora. This will also reduce our nightly-repack-load, so thanks for the help here.

bah - lost assignment in mid-air.

(In reply to Brian R. Bondy [:bbondy] from comment #20) > The update would be rejected correct. > I think you want to have the mar you're offering be signed with the m-c > channel inside the mar, but have the files mentioned in Comment 19 have the > aurora channel string. Rejected outright completely (as in the mar could never be applied), or would it prompt the user if they wanted to apply the update?

Rejected outright.

I don't remember the exact error handling but I think it gets deleted and re-downloaded. I think the user gets an error wizard to re-download as well. But of course they'd just get another bad mar in this case.

Brian, if you have a moment could you please take a look at http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/channel-switch/switch-to-aurora.mar I'm hitting nthomas@HD-mandala ~/Desktop/FirefoxNightly.app/Contents/MacOS/updates $ cat last-update.log Performing a background update SOURCE DIRECTORY /Users/nthomas/Desktop/FirefoxNightly.app/Contents/MacOS/updates/0 DESTINATION DIRECTORY /Users/nthomas/Desktop/FirefoxNightly.app/Updated.app DoUpdate: error extracting manifest file failed: 6 calling QuitProgressUI when updating a mac nightly (m-c, de locale). Somehow I'm ending up at http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/updater/updater.cpp#3601 It's signed with the nightly cert for firefox-mozilla-central, the code indicates it's gotten past verifying that already but I'm most paranoid about getting that right.

There's a test update at you can point at with app.update.url.override: https://aus3.mozilla.org/update/3/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/test/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml It's locale and platform agnostic.

FTR, I don't think we should take data from comment 6 as current, that's 1.5 years old.

I checked bug 655129, but couldn't find the reasoning behind moving af nightly users to en_US. I guess not many people are affected, but I guess af can be handled just like the other languages with active translations on Aurora.

(In reply to Axel Hecht [:Pike] from comment #28) > FTR, I don't think we should take data from comment 6 as current, that's 1.5 > years old. I posted to newsgroups yesterday to see if any of these locales had changed their mind.

(In reply to Friedel Wolff from comment #29) > I checked bug 655129, but couldn't find the reasoning behind moving af > nightly users to en_US. I guess not many people are affected, but I guess af > can be handled just like the other languages with active translations on > Aurora. This was from l10n-drivers, but a while ago. If you the official "af" lead, just let us know what your community wants - we are happy to move "af" users on m-c to aurora instead of merging them into "en-US" users on m-c. Just let us know what "af" community would prefer.

(In reply to John O'Duinn [:joduinn] from comment #31) > (In reply to Friedel Wolff from comment #29) > > I checked bug 655129, but couldn't find the reasoning behind moving af > > nightly users to en_US. I guess not many people are affected, but I guess af > > can be handled just like the other languages with active translations on > > Aurora. > > > This was from l10n-drivers, but a while ago. If you the official "af" lead, > just let us know what your community wants - we are happy to move "af" users > on m-c to aurora instead of merging them into "en-US" users on m-c. > > Just let us know what "af" community would prefer. Friedal: ping?

We shouldn't try to resolve issues about single locales in this bug. I've gathered some data on ADIs and activity, and l10n-drivers will put the data up for comments in .l10n, and then follow up here.

I figured out my problem at comment #26 (not precise enough manual hackery). When creating the mar file the (simplified) call is mar -c output.mar <list of files> but the <list of files> must be update.manifest [updatev2.manifest] <files in same order as update manifest> Without the manifest first you get the error in comment #26; without the file ordering you get things like: $ cat updates/last-update.log Performing a background update SOURCE DIRECTORY /Users/nthomas/Desktop/FirefoxNightly.app/Contents/MacOS/updates/0 DESTINATION DIRECTORY /Users/nthomas/Desktop/FirefoxNightly.app/Updated.app PREPARE ADD Contents/MacOS/defaults/pref/channel-prefs.js PREPARE ADD Contents/MacOS/update-settings.ini EXECUTE ADD Contents/MacOS/defaults/pref/channel-prefs.js ### execution failed FINISH ADD Contents/MacOS/defaults/pref/channel-prefs.js FINISH ADD Contents/MacOS/update-settings.ini backup_restore: backup file doesn't exist: Contents/MacOS/update-settings.ini.moz-backup failed: 6 calling QuitProgressUI

Status: * channel changing mar created (see attachment for method) * verified it works on Windows XP (using the service) and mac However * when you query for an Aurora update you get AUS:SVC UpdateService:selectUpdate - skipping update because the update's application version is less than the current application version and nothing is downloaded (Aurora version 22.0a2 is less than nightly 23.0a1) * if you fake the snippets to offer 23.0a1 then the d/l will happen. Mac will apply the update with great success, but the windows service will report this in last-update.log: failed: 23 Per http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/common/errors.h#40 this is a VERSION_DOWNGRADE_ERROR, from the 22.0a2 version encoded in the aurora mar file. So, this at this point we have a choice: 1) Deliberately fake version 23.0a1 in Aurora snippets and when versioning the mar files, and use the channel-switching mar. Would have to do this for some undetermined time until users have upgraded. 2) bite the bullet and repack some set of aurora updates. Something like this: * looping over platforms and locales * unpack mar file * add defaults/prefs/channel-prefs.js and update-settings.js files * recreate mar file with -V 23.0a1 -H firefox-mozilla-central * sign mar file with nightly cert, host on ftp.m.o * create and host snippets This would be a one-off job for each locale, but much more time consuming.

Setting myself as QA Contact so I can test this once ready.

We can move Afrikaans (af) users to Aurora like for the other locales.

To put some context on ADI here, for 2013-04-29 on the nightly channel and version 23.0a1 we got blocklist pings for af (1), bg (4), ca (3), da (2), el (6), fi (3), hr (1), mk (1) for a total of 21. The other locales removed in attachment 741623 [details] [diff] [review] didn't have any pings. We might find slightly more if a week was considered instead (metrics interface is misbehaving so can't get that right now) but that data says there's a big reduction in the locales we need to provide paths for, if any at all.

Results are not hugely changed by considering the week ending 2013-04-29. A total of 145 blocklist pings averaging to 20.7/day. The locales listed in comment #38 drop a bit to accomadate adding en-ZA (1), eu (1), sl(1) https://docs.google.com/a/mozilla.com/spreadsheet/ccc?key=0AiC1WGFz4NMDdFRNQ19SVm5NTzJGR2xkQVVsOWZ2TUE&usp=sharing

catlee pointed out that the locales in bug 863452 were disabled long enough ago to not hit the mar versioning problem in comment #35. So I've created a switch-to-aurora.mar file per attachment 739417 [details], and put on ftp at http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/custom-updates/switch-to-aurora.mar. Snippets have been created on aus3-staging, as the last builds were created before we switched to Balrog (so no dependency on bug 945179). They have a buildID of 99999999999999, which has been excluded from the cron job which cleans up old snippets. Testing had to be done by taking an en-US build and hardcoding the %LOCALE% (after copying app.update.url to app.update.url.override), as we don't keep l10n builds that far back. On Windows 8, Mac 10.8, and Ubuntu 32bit, I successfully updated from the last nightly before bug 863452 took effect [1] in two steps. The first applies switch-to-aurora.mar, which sets up Firefox to query on the aurora channel, and accept mars with channelID of firefox-mozilla-aurora; then the latest complete mar is applied. [1] http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2013/04/2013-04-25-03-08-45-mozilla-central/

Notes for the future, when bug 713159 is resolved, and we want to do another round of this sort of update. If it's nearly merge time, I suggest: * before the merge: disabling the locales on mozilla-central (so they never get a version bump) * after the merge: using the channel change trick to point at aurora builds (which will now have the version which was on central). Would need to set up a rule and a release in Balrog, rather than create snippets If it's early/mid-cycle and we want to update straight away we'll to repack some aurora complete mars: * finish this script I've attached and use it to repack some aurora locales * known deficiencies - signing and Balrog submission In either case, having bug 945179 would be very desirable to write a single rule in Balrog (rather than one per locale).