Bug 901921 partially broke partial updates, leaving users who updated in that window with broken menubars on Mac (and potentally other issues). It looks like the only way to fix that would be to force full updates for the affected users (though it would be nice to confirm that that would fix it). Can we do that easily?

I think we can flip create_partial and create_partial_l10n to False (in our buildbot-configs) for the next nightly, then flip it back. That would update everyone to the latest build via complete, some 65MB. Not everyone would need that but it should catch everyone. Will investigate further.

This gives us the same level of control as for en-US. Didn't change anything when did a master dump.

We'd revert this after one nightly.

There's a chance that other platforms are also borked, AFAIK no-one has looked at that. Will do so now, and might need to change this to just flip the two lines before this addition to False.

Yeah, I don't think was isolated to Mac (that's just where the most visible symptom occurred).

Everyone gets to have fun, but only for one night. btw, I think this is going to put the snippets in /opt/aus2/incoming/2/Firefox/mozilla-central/%(target)s/%(buildid)s/%(locale)s/ instead of %(previous_buildid). The first nightly after it's reverted the complete.txt will get overwritten, and partial.txt added, but AUS should be OK with it.

Comment on attachment 786679 [details] [diff] [review] [buildbotcustom] Lets us disable partials for l10n at platform level https://hg.mozilla.org/build/buildbotcustom/rev/a657fbb412f8

Comment on attachment 786685 [details] [diff] [review] [buidlbot-configs] Disable partials on all platforms https://hg.mozilla.org/build/buildbot-configs/rev/9f601896812a

Reconfig to follow, when we have some other things for 24.0b1 finalized.

Can the same procedure be done for nightlies on the UX branch? There are a few thousand users there and we weren't even able to re-spin a Nightly once we noticed the problem due to bug 846104.

Comment on attachment 786729 [details] [diff] [review] [buildbot-configs] All platforms on ux too https://hg.mozilla.org/build/buildbot-configs/rev/2ccec613fb74

In production ahead of the 3am/4am nightlies for m-c/ux.

Unfortunately, the way the buildbot code is written we didn't get any complete snippets uploaded to the update server. So the latest crop of nightlies aren't being offered at all. The best option at this point is to wait for the nightlies to finish, back out the two changes in buildbot-configs, merge, reconfig the buildbot masters, and force a new set of nightlies on each branch. We should get partials from the early 20130807 nightly to the later one, and everyone older would get a complete.

Hi, I a user that got bit by this bug yesterday and I am not sure how forcing a full update solves the issue. To download a new update, I use the Nightly -> About Nightly menu option. With this issue, I can't open that dialog to download a new update. The Preferences doesn't open either through the missing menu or through Cmd-,. So, how does one do this full upgrade? I ended up just redownloading nightly from the website.

The updated works in the background even if you can't open the dialog to trigger one manually, the full updates are useful for that case.

Comment on attachment 786685 [details] [diff] [review] [buidlbot-configs] Disable partials on all platforms Backed out in http://hg.mozilla.org/build/buildbot-configs/rev/37412aa3bc38

Comment on attachment 786729 [details] [diff] [review] [buildbot-configs] All platforms on ux too http://hg.mozilla.org/build/buildbot-configs/rev/37412aa3bc38

Backouts are in production. I'll manually put the snippets on the update server for UX, and trigger a set of nightlies on m-c.

Updates to the 2013-08-07 UX nightly are being offered now, complete only. On m-c updates are out except for windows, the new buildID is 20130806104538, partials from 20130806030203.

Oops, jumped the gun on m-c, those are still building.

(In reply to Nick Thomas [:nthomas] from comment #20) > Updates to the 2013-08-07 UX nightly are being offered now, complete only. > On m-c updates are out except for windows, the new buildID is > 20130806104538, partials from 20130806030203. Thanks Nick. I can confirm that updates are working on Windows and OS X for the UX branch. The menubar is fixed on OS X after updating.

Turns out I mucked up the config backout in production. To fix https://hg.mozilla.org/build/buildbot-configs/rev/ba0cd32cf7f6 and reconfiged with that. Two Linux and win32 builds cancelled before upload, mac had finished so the bits were moved aside. Redoing those builds, same buildID of 20130807161117.

m-c nightlies worked out at last.