We need to upgrade buildbot installations, buildbotcustom pulls, and buildbot-configs to current states on SeaMonkey builder machines (buildmaster and slaves). The current state is from somewhere around August 2009, from there on we tried to stay as stable as possible while going for a final 2.0 release. The new minis (including the re-imaged cb-sea-miniosx01, see bug 526206) come with buildbot 0.7.10p1, which we should be using everywhere now, buildbotcustom has seen a number of changes, including stuff for nightly L10n updates, which we also want to adopt, and we also might want to shift more of the code from config into buildbotcustom, like it was done for the main Mozilla masters.

For Linux slaves, did the following (buildbotcustom shouldn't matter on slaves, just did it for completeness - and linux64-01 doesn't even have it): cd /tools/buildbot sudo hg pull -u sudo python setup.py install cd /tools/buildbotcustom/buildbotcustom/ sudo hg pull -u rm /builds/slave/twistd.log rm -rf /builds/slave/comm-*/* Done on cb-sea-linux-tbox, cb-seamonkey-linuxdebug-01, cn-sea-qm-centos5-01, cb-seamonkey-linux64-01. (cb-seamonkey-linux-01 is the master, -02 is down completely because of parallels restrictions.)

On Windows slaves, did the following: cd /d/dist/buildbot/ hg pull -u python setup.py install rm /e/builds/slave/twistd.log* rm -rf /e/builds/slave/comm-*/* Done on cb-sea-win32-tbox, cb-seamonkey-win32-01, cb-seamonkey-win32-02, cn-sea-qm-win2k3-01

On Mac slaves, did the following: cd /tools/buildbot/ sudo hg pull -u sudo python setup.py install rm /builds/slave/twistd.log* rm -rf /builds/slave/comm-*/* Done on cb-seamonkey-osx-01, cb-seamonkey-osx-02, cb-seamonkey-osx-03 (-04 is down for parallels reasons, cb-sea-miniosx01 should already have current code).

On the master (cb-seamonkey-linux-01), did the following: buildbot stop /builds/master cd /tools/buildbot sudo hg pull -u sudo python setup.py install sudo python setup.py install --prefix=/tools/buildbot cd /tools/buildbotcustom/buildbotcustom/ sudo hg revert --all sudo hg import --no-commit https://bugzilla.mozilla.org/attachment.cgi?id=417254 sudo hg import --no-commit -f https://bugzilla.mozilla.org/attachment.cgi?id=417272 rm -rf /builds/slave/comm-*/* cd ~/buildbot-configs/ hg pull -u cd /builds/master && buildbot checkconfig buildbot start . For buildbotcustom, we need to be careful, as we had local patches applied. We know that we need to apply two of them again, so applied them manually after upgrading (one will go upstream soon, I hope, the other should get obsoleted by a better solution). We still had slave directories around from the times when a slave was run on the same VM, so also cleaned those up.

Hrm, I just realized that the Linux and Mac slaves aren't suing the new buildbot yet - just like the master, they need to have it installed in /tools/buildbot So, going into all the Linux and Mac slaves again and doing the following: buildbot stop /builds/slave cd /tools/buildbot sudo python setup.py install --prefix=/tools/buildbot sudo reboot (The reboot should make buildbot start up automatically again.) Done on cn-sea-qm-centos5-01, cb-sea-linux-tbox, cb-seamonkey-linuxdebug-01, cb-seamonkey-linux64-01, cb-seamonkey-osx-01, cb-seamonkey-osx-02, cb-seamonkey-osx-03

For whatever reason MSYS on the Windows slaves didn't set MACHTYPE automatically like it should and that made the compile processes fail, so I added export MACHTYPE=i686-pc-msys to seabld's .bash_profile there, which fixed that for the moment.

One more local path needed for the master: https://bugzilla.mozilla.org/attachment.cgi?id=417299

OK, all trees are back to their previous green/orange states, master ist running current buildbot and buildbotcustom (with above-mentioned patches), all slaves are running current buildbot, this mission has been a success!

I found a better solution for the problem in comment #6 and am deploying it right now, removing the hack from there: echo "export HOSTTYPE MACHTYPE OSTYPE SHELL" >> /etc/profile This is actually a bug in MozillaBuild 1.4, and releng fixed it with the OPSI package in http://hg.mozilla.org/build/opsi-package-sources/file/55fbdeba037f/profilevars/CLIENT_DATA/profilevars.ins which does exactly what the one-liner above achieves.