This takes care of reducing setup time. See bug 659328 for reducing reboots and saving setup time.

OK here is the list of builders that take more than 4mins on setting up (minus tegras). I will analyze each one of them one by one and see where we get to. As seen on the spreadsheet, setup time is very variable considering on cleaning previous runs and downloading builds and others (there is caching to help). try_xp_test-tp4 try_win7_test-tp4 try_win7-debug_test-mochitest-other try_win7_test-scroll try_xp_test-chrome try_xp_test-mochitest-other try_win7_test-mochitest-other try_xp-debug_test-mochitest-other try_win7_test-jsreftest try_win7-debug_test-jsreftest try_win7_test-chrome try_leopard_test-dirty try_snowleopard_test-dirty try_win7_test-nochrome try_snowleopard_test-paint try_xp_test-jsreftest try_snowleopard_test-tp4 try_win7-debug_test-reftest try_win7-debug_test-crashtest try_snowleopard_test-mochitests-2 try_snowleopard_test-mochitest-other try_snowleopard_test-mochitests-3 try_xp-debug_test-reftest try_xp_test-crashtest try_snowleopard_test-scroll try_leopard_test-jsreftest try_win7_test-dirty try_leopard_test-mochitest-other

We should not have to download/unpack pagesets.zip (~3mins on XP) since it doesn't really change. > wget --progress=dot:mega -N http://url.com/to/pageset.zip On snowleopard unpacking the build takes a long time (~1min - less than 5 secs on Fedora). Perhaps we could upload a tar ball instead of a dmg? Or try to optimize "bash ../tools/buildfarm/utils/installdmg.sh firefox-6.0a1.en-US.mac64.dmg" Note that we use two different ways of unpacking dmg giles: Talos jobs use ~/talos-slave/talos-data/installdmg.sh while unit tests use ../tools/buildfarm/utils/installdmg.sh For mochitest-other we run the following command several times: > unzip -o firefox-7.0a1.en-US.linux-i686.tests.zip 'bin*' 'certs*' 'mochitest*' Not that I see that it takes a long time (The first time takes > 10secs & the following 2 secs). Cloning tools takes 20 to 40 secs. chmod_files on windows 7 (not xp) takes a lot of time (bug 544727): > chmod_files chmod files (see msys bug) ( 1 mins, 58 secs ) The same for "removing any old dir" > remove any old working dirs remove old working dirs ( 1 mins, 53 secs ) We should also look to see if we can improve download time (from staging to scl1) as in some case can take a long time: > Download build download ( 1 mins, 18 secs ) We should also look into avoid downloading symbols if not necessarily. We should only download talos.zip if newer as it does not get refreshed so often: > wget --progress=dot:mega -N http://build.mozilla.org/talos/zips/talos.zip It seems that there are a bunch of places for things to be improved. I wonder if we should put all the efforts in bug 650880 and bug 650887 rather than fixing each issue individually.

(In reply to comment #2) > We should not have to download/unpack pagesets.zip (~3mins on XP) since it > doesn't really change. > > wget --progress=dot:mega -N http://url.com/to/pageset.zip Not downloading the pageset was a decision made because of how difficult it was to update it. Now that we have Puppet on the POSIX machines, and OPSI on XP I think keeping it on the machine is the right call. > On snowleopard unpacking the build takes a long time (~1min - less than 5 > secs on Fedora). > Perhaps we could upload a tar ball instead of a dmg? I'm not a fan of this. I think we should be testing the exact things that we ship to users.

(In reply to comment #2) > On snowleopard unpacking the build takes a long time (~1min - less than 5 > secs on Fedora). > Perhaps we could upload a tar ball instead of a dmg? > Or try to optimize "bash ../tools/buildfarm/utils/installdmg.sh > firefox-6.0a1.en-US.mac64.dmg" > Note that we use two different ways of unpacking dmg giles: > Talos jobs use ~/talos-slave/talos-data/installdmg.sh while unit tests use > ../tools/buildfarm/utils/installdmg.sh Where is the talos installdmg.sh found? I can't seem to find it to see if there is a difference between them. Do we see a discrepancy between the time it takes to unpack a dmg on a talos run versus a unittest run? I'm also with bhearsum, I don't think we want to shift to a tar on mac. We want to stay with dmg. The buildfarm installdmg.sh has a 5 second sleep in it. That could be eliminated because there really isn't any point in waiting for the disk-image helpers to go away. They don't hurt anything, they just disappear. And the rest of the code looks correct in that you trap and ensure you call detach even if there's an error (this is very good, because too many diskimage-helpers destroy a machine's resources).

After bug 661656 got resolved we have faster transfer between stage.m.o and the test slaves on the scl colo.

Nothing left on this specific bug. Other bugs have been opened for similar issues.