Once we have all the slaves we need available in the new buildbot pools, we can turn on unit tests for comm-central-trunk SeaMonkey builds.

Until we get the additional slaves, would it be possible to build and upload (but not run) something like a "nightly packaged build+test"?

Not right now, as AFAIK, the factory doesn't support that. Also, I don't have time to even think about this before 2.0b1 builds run, and we are still missing some slave VMs and have instabilities in others, let's not push things to fast there.

I am turning on packaged xpcshell tests permanently on the main SeaMonkey tree, even if bug 541235 mailnews failures are still happening there. For the other suite, I have done a test run for all of them on Linux debug builds, with the following results: crashtest: green (1466/0/15) - http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey-Ports/1264293428.1264293820.28256.gz&fulltext=1 reftest: green (4109/0/294) - http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey-Ports/1264292500.1264295466.13505.gz&fulltext=1 mochitest: orange (T-FAIL L-FAIL) - http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey-Ports/1264292467.1264293906.29221.gz&fulltext=1 This one ended up crashing in libimglib2.so mochitest-other: orange - http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey-Ports/1264292467.1264293435.24275.gz&fulltext=1 mochitest-chrome: T-FAIL L-FAIL Another crash in in libimglib2.so mochitest-browser-chrome: 591/22/12 A number of different errors, probably worth investigating mochitest-a11y: 31463/554411/2 most of those are "JavaScript error: chrome://mochikit/content/a11y/accessible/events.js, line 782: nsIAccessibleEvent is not defined" Please follow up to the errors on different bugs.

(In reply to comment #3) > crashtest: green (1466/0/15) - > reftest: green (4109/0/294) - Could we enable them? (They don't take too long, etc) > mochitest: orange (T-FAIL L-FAIL) - > mochitest-chrome: T-FAIL L-FAIL > mochitest-browser-chrome: 591/22/12 Some failures may be intermittent and/or already filed: running them once was nice, but is not enough (for me) to sort them out yet :-| Would it be possible to run these suites with a low priority (= when truly idled)? > mochitest-a11y: 31463/554411/2 This one is bug 535891.

(In reply to comment #4) > (In reply to comment #3) > > > crashtest: green (1466/0/15) - > > reftest: green (4109/0/294) - > > Could we enable them? (They don't take too long, etc) Actually, reftest took a lot longer than I expected (about 45min), and I'd like to not enable things right now if they don't bring much of a value, as the machines are strained enough with what they are doing. > > mochitest: orange (T-FAIL L-FAIL) - > > mochitest-chrome: T-FAIL L-FAIL > > mochitest-browser-chrome: 591/22/12 > > Some failures may be intermittent and/or already filed: > running them once was nice, but is not enough (for me) to sort them out yet :-| > Would it be possible to run these suites with a low priority (= when truly > idled)? Unfortunately, we only can make them be triggered on every debug build or not at all, and we can't assign different priorities. But that said, if we have actual work being done on them and they don't run overly long, I might be willing to do something there, as this might create actual benefit for trunk. > > mochitest-a11y: 31463/554411/2 > > This one is bug 535891. Thanks, good to know.

(In reply to comment #5) > we can't assign different priorities. I was wondering if prioritizeBuilders() at http://mxr.mozilla.org/build/source/buildbot-configs/seamonkey/master.cfg#51 could be used to achieve a very low prioritization for tests builds? > if we have actual work being done on them and they don't run > overly long, I might be willing to do something there, as this might create > actual benefit for trunk. I'm feeling we're in an egg-chicken situation: it's much harder to get work started/done without a build to check :-|

(In reply to comment #6) > (In reply to comment #5) > > > we can't assign different priorities. > > I was wondering if prioritizeBuilders() at > http://mxr.mozilla.org/build/source/buildbot-configs/seamonkey/master.cfg#51 > could be used to achieve a very low prioritization for tests builds? I have no idea how I would use that to selectively un-prioritize only those trunk test builds. I don't even claim to understand the code I copied there. Let's not take this too far, I don't think it's necessary. > > if we have actual work being done on them and they don't run > > overly long, I might be willing to do something there, as this might create > > actual benefit for trunk. > > I'm feeling we're in an egg-chicken situation: it's much harder to get work > started/done without a build to check :-| I think you misunderstood me, I meant I can turn something on that doesn't run too long, if I know that someone will pay attention and try to fix the problems. I wanted to indicate that I just don't want to put our machines under more strain than necessary without knowing that it will actively help us.

(In reply to comment #7) > I think you misunderstood me Ah, got it: *I would monitor them to file bugs. *I probably wouldn't investigate/fix any of them (or a few at most), at least until I'm done with the "configure" backlog... *And then they're the problem that c-1.9.2 hasn't branched yet, which is blocking some fixes wrt m-c, which may cause some failures. (As we already know for a fact ;-<) In the very short term, I would be interested on enabling all Linux Debug tests for multiple (instead of just 1) builds, once on 2.1/m-c, once on 2.1/m-1.9.2 (if possible). I expect many (bad) failures (as we saw), be that would be enough to file (mochitests-*) bugs this time.

(In reply to comment #8) > *I would monitor them to file bugs. > *I probably wouldn't investigate/fix any of them (or a few at most), > at least until I'm done with the "configure" backlog... OK, those are reasons to have at least some of them running. > In the very short term, I would be interested on enabling all Linux Debug tests > for multiple (instead of just 1) builds, once on 2.1/m-c, once on 2.1/m-1.9.2 > (if possible). Running all suites is out of the question, we just don't have the machine power right now. What I can and will do is putting up mochitest-other again there. mochitest-plain has the risk of running too long and block other builds, esp. as we have fewer Linux slaves than ones for any other platform right now, and reftest/crashtest are too little reward right now as they have been green in the tests.

Now that the boxes we need are all online, I'll let them run a few cycles over night and then I'll turn all test suites on for debug builds on all platforms to fix this bug. \o/

http://hg.mozilla.org/build/buildbot-configs/rev/40fb1c3de8b5 checked in the config changes, tests should come up now! \o/