+++ This bug was initially created as a clone of Bug #599871 +++ We killed updates in bug 599871 - we need to turn them back on once bug 600098 gets fixed. This bug tracks that. This bug blocks b7.

AIUI we only need to block updates for 10.5 users using Intel i386 builds (taking as a given that PPC users are going nowhere). We could enable updates for people with working i386/x86_64 universals and just leave the old universal blocked for now. Translating that into AUS-speak, we * point Darwin_x86-gcc3-u-ppc-i386 back to Darwin_Universal-gcc3, or nowhere * revert the chmod in bug 599871 to make updates available for * Darwin_x86-gcc3-u-i386-x86_64 * Darwin_x86_64-gcc3-u-i386-x86_64 That also resolves the red 10.6 nightlies we get from not being able to upload snippets to aus. Is everyone cool with that ?

Sounds good to me

Comment 1 sounds good to me, as an intermediate step while we the fixes for bug 600098 sorted out.

Though one concern is that we still don't have a fix for bug 600362, and even if we did it wouldn't help users on old builds (using the old updater). So even with the fixes for bug 600098, we'll crash on the restart after this update (manual launching after that will work). I'm not sure whether there's a workaround for that (like there was for bug 600098).

(In reply to comment #4) > Though one concern is that we still don't have a fix for bug 600362, and even > if we did it wouldn't help users on old builds (using the old updater). So even > with the fixes for bug 600098, we'll crash on the restart after this update > (manual launching after that will work). I'm not sure whether there's a > workaround for that (like there was for bug 600098). One solution would be bug 600098 comment #35

Good point! Talking it over with beltzner/johnath, I'm not sure it's worth the effort to push an updater-only update for nightly/beta users. A crash on update might be tolerable there, as long as the bug 600098 workaround ensures that manual launches after that are OK. For major releases, we'll get the updater fixes into the minor updates, so the point is moot.

To put some ADUs on this, there are ~125 10.5 nightly users on 4.0b7pre, which is about 9% of all Mac users and about 0.3% of all nightly users.

nthomas tells me that the 10.5 vs. 10.6 breakdown of users is 9% to 91%. If we implement the changes in comment 1 now, users on 10.5 that manually updated to 32/64 universal builds (some subset of the 9%) will start receiving updates, and then will run into bug 600411 and bug 600362 (restart to apply the update and launch of the app after the update will fail). I think that cost is outweighed by the benefit of getting updates for users on 10.6 (91%), so I think we should move ahead with comment 1 for the moment. Once that's done, we probably want to wait for fixes to bug 600098 and bug 600411 before re-enabling updates from Darwin_x86-gcc3-u-ppc-i386 (there's no point in waiting for bug 600362, since that fix won't be useful unless we do a special one-off update that only updates the updater, and I think that's probably not worth it).

I tested on 10.5 with one of the new universals downloaded, pulling updates off our staging system. Those users are persistent they will see * update offer * crash on restarting Firefox when it launches as 64bit (600411) * crash when Firefox launches as 32bit but tries to launch the updater as 64bit * Firefox starting 32bit without applying update If the update was a partial then you get the dialog about downloading the complete, crash, crash, end up where you started. These users are basically stuck where they are, at least until they can manually download a build with the launch bugs fixed.

I think that probably tips the scales towards keeping all updates disabled for now :(

What does this imply for our existing 10.5 beta users (about 7k by last ADU count)

We're also getting to the point where I think we want to, at least, turn updates back on for 10.6 users (about 36,000 by last ADU count) right now.

We just turned updates back on for *only* the Darwin_x86_64-gcc3-u-i386-x86_64 requests (10.6). Here's the current state of things: lrwxrwxrwx 1 ffxbld users 21 Aug 24 20:52 Darwin_ppc-gcc3-u-ppc-i386 -> Darwin_Universal-gcc3 drwxr-xr-x 906 ffxbld firefox 73728 Sep 26 06:25 Darwin_Universal-gcc3 drwxr-xr-x 229 ffxbld users 20480 Sep 30 11:01 Darwin_x86_64-gcc3 lrwxrwxrwx 1 ffxbld users 18 Sep 27 02:35 Darwin_x86_64-gcc3-u-i386-x86_64 -> Darwin_x86_64-gcc3 lrwxrwxrwx 1 root root 9 Sep 30 11:02 Darwin_x86-gcc3-u-i386-x86_64 -> /dev/null lrwxrwxrwx 1 root root 9 Sep 30 11:02 Darwin_x86-gcc3-u-ppc-i386 -> /dev/null

Oh, and the current latest update is from the 27th. A new nightly is spinning which will be pushed out to those users when it completes in 1-2 hours.

Mossop points out that 10.6 users that are still on a pre-bug 571367 build still won't get updates, even with the changes we made today. They will get updates if they're on a more recent build than that, though.

Can we make comment 15 be not true? We should be safe to turn on updates for all requests from OSX 10.6, not just ones coming from x64.

(In reply to comment #15) > Mossop points out that 10.6 users that are still on a pre-bug 571367 build > still won't get updates, even with the changes we made today. They will get > updates if they're on a more recent build than that, though. I don't think this is true. Those users will use an URL such as: https://aus3.mozilla.org/update/3/Firefox/4.0b7pre/20100917030803/Darwin_x86_64-gcc3/en-US/nightly/Darwin%2010.4.0/default/default/update.xml?force=1, which returns an update. To my knowledge, the only users that will not be getting updates are: - Any users of the old style (ppc+i386) Universal builds, identified as Darwin_x86-gcc3-u-ppc-i386 on AUS - Any users of the new style (i386+x86) Universal builds running on 10.5, identified as Darwin_x86-gcc3-u-i386-x86_64 on AUS. 10.6 Users of 64-bit only builds come in as Darwin_x86_64-gcc3; 10.6 users of the new-style Universal (i386+x86_64) come in as Darwin_x86_64-gcc3-u-i386-x86_64 -- both of which have updates enabled. Digging through metrics to confirm this.

(In reply to comment #17) > - Any users of the old style (ppc+i386) Universal builds, identified as > Darwin_x86-gcc3-u-ppc-i386 on AUS These are the users I was referring to in comment 15. My assumption is that most 10.6 nightly users were in this group prior to Sunday.

(In reply to comment #18) > (In reply to comment #17) > > - Any users of the old style (ppc+i386) Universal builds, identified as > > Darwin_x86-gcc3-u-ppc-i386 on AUS > > These are the users I was referring to in comment 15. My assumption is that > most 10.6 nightly users were in this group prior to Sunday. After IRC chatter, I understand. Working on this.

Nick wrote this patch earlier this week. I tested it in build's AUS staging environment and it works AFAICT. After chmod'ing Darwin_x86-gcc3-u-ppc-i386 back to 755 I was offered an update on the 20100917 ppc+i386 build, running on 10.6 and was not offered an update on the same build running on 10.5.

Comment on attachment 480223 [details] [diff] [review] patch to block Darwin 9 users on 4.0*/3.7* bhearsum and morgamic asked me to take a look. Looks ok to me, although I'd like to see a bug # in the comment ("the change to the universal builds" is ambiguous.) I am fine with checking this in, but I would like to (at least) see the AUS (fitnesse) tests run before this gets deployed. morgamic should be able to remind us where these live and how to run them in the next hour. Doing a few manual tests like bug 588412 comment 19 could not hurt.

(In reply to comment #21) > Comment on attachment 480223 [details] [diff] [review] > patch to block Darwin 9 users on 4.0*/3.7* > > bhearsum and morgamic asked me to take a look. Looks ok to me, although I'd > like to see a bug # in the comment ("the change to the universal builds" is > ambiguous.) > > I am fine with checking this in, but I would like to (at least) see the AUS > (fitnesse) tests run before this gets deployed. > > morgamic should be able to remind us where these live and how to run them in > the next hour. Doing a few manual tests like bug 588412 comment 19 could not > hurt. I did these myself: - When running the ppc+i386 nightly from 20100917 on a 10.5 machine I received no update - When running the ppc+i386 nightly from 20100917 on a 10.6 machine I received an update to the 20101001 nightly.

Comment on attachment 480253 [details] [diff] [review] universal diff version of the patch This works fine and tests pass. Thanks Ben.

Comment on attachment 480253 [details] [diff] [review] universal diff version of the patch Checking in index.php; /cvsroot/mozilla/webtools/aus/xml/index.php,v <-- index.php new revision: 1.30; previous revision: 1.29 done foo-ix-blah:xml bhearsum$ cvs tag AUS2_PRODUCTION index.php W index.php : AUS2_PRODUCTION already exists on version 1.29 : NOT MOVING tag to version 1.30 foo-ix-blah:xml bhearsum$ cvs tag -F AUS2_PRODUCTION index.php T index.php

Filed bug 601264 for the AUS update. After it's picked up I'll chmod the dirs to re-enable updates for 10.6 users.

I updated the symlink on aus2: lrwxrwxrwx 1 root root 18 Oct 1 2010 Darwin_x86-gcc3-u-ppc-i386 -> Darwin_x86_64-gcc3 And have confirmed that we're seeing what we want: A 10.6 user running the old ppc+i386 will get an update: https://aus3.mozilla.org/update/3/Firefox/4.0b7pre/20100917030917/Darwin_x86-gcc3-u-ppc-i386/en-US/nightly/Darwin%2010.4.0/default/default/update.xml?force=1 but a 10.5 one will not: https://aus3.mozilla.org/update/3/Firefox/4.0b7pre/20100917030917/Darwin_x86-gcc3-u-ppc-i386/en-US/nightly/Darwin%209.2.0/default/default/update.xml?force=1 Big thanks to Gavin, Morgamic, and Rhelmer.

Looks like all the bugs blocking updates on 10.5 are now fixed, so I think we can re-enable 10.5 updates now as well - I'll try to do some testing tomorrow.

Testing checks out - updating from 2010-09-26 to 2010-10-03 on 10.5 works, aside from the "updater fails to re-launch build" issue (bug 600362). Since we need a solution to that bug for beta users, though, I think we'll want to use nightly users as a test bed for the custom update (without the build config changes from bug 571367).

Agreed; can we make it so?

We're not doing bug 601660, so let's just re-enable Mac updates across the board.

I've backed out the patch on trunk: http://bonsai.mozilla.org/cvsquery.cgi?date=explicit&mindate=2010-10-04+13%3A41&maxdate=2010-10-04+13%3A41 I've also tagged that backout + the change from bug 600289 as AUS2_RTM_201010041714, will file a bug requesting an update of production.

bug 601757 to update it in production.

(In reply to comment #31) > We're not doing bug 601660, so let's just re-enable Mac updates across the > board. Re-enabled: All mac trunk updates are pointing at Darwin_x86_64-gcc3 again: drwxr-xr-x 235 ffxbld users 20480 Oct 4 07:46 Darwin_x86_64-gcc3 lrwxrwxrwx 1 ffxbld users 18 Sep 27 02:35 Darwin_x86_64-gcc3-u-i386-x86_64 -> Darwin_x86_64-gcc3 lrwxrwxrwx 1 root root 18 Oct 4 15:17 Darwin_x86-gcc3-u-i386-x86_64 -> Darwin_x86_64-gcc3 lrwxrwxrwx 1 root root 18 Oct 1 15:21 Darwin_x86-gcc3-u-ppc-i386 -> Darwin_x86_64-gcc3