It looks like we've achieved a < 10% failure rate on mochitests on debug B2G emulators on cedar; we should schedule them across all trunk trees now.

I think this should do the trick.

Diff of a clean/patched linux test master. This looks good. Created via: ./setup-master.py -l ./setup-master.py bm51-tests1-linux bm51-tests1-linux cd bm51-tests1-linux /src/clean/braindump/buildbot-related/builder_list.py master.cfg > ../../clean.dump # apply patch to buildbot-configs /src/clean/braindump/buildbot-related/builder_list.py master.cfg > ../../dirty.dump cd ../.. diff {clean,dirty}.dump > builderdiff

Comment on attachment 8338154 [details] [diff] [review] Schedule debug B2G mochitests on trunk branches, I also see the debug emulator build running sendchanges on fx-team and m-c, so we should see tests running after this patch lands + reconfigs.

Thanks Jonathan!

This was bitrotted quite a bit; reflagging for review. Aki, can you run another builder diff on this to make sure I got it right?

Comment on attachment 8358550 [details] [diff] [review] Schedule debug B2G mochitests on trunk branches, >+BRANCHES['mozilla-b2g18_v1_0_1']['platforms']['emulator']['enable_debug_unittests'] = False We deleted the 1.0.1 branch so this line is causing a KeyError: Traceback (most recent call last): File "/src/clean/braindump/buildbot-related/builder_list.py", line 20, in <module> execfile(os.path.basename(sys.argv[1]), g) File "master.cfg", line 12, in <module> import b2g_config File "/src/jgriffin/buildbot-configs/bm51-tests1-linux/b2g_config.py", line 1443, in <module> BRANCHES['mozilla-b2g18_v1_0_1']['platforms']['emulator']['enable_debug_unittests'] = False KeyError: 'mozilla-b2g18_v1_0_1' r=me with that line removed. Attaching the new builderdiff.

1_0_1 line deleted, and MOCHITEST changed to MOCHITEST_EMULATOR_DEBUG, so that we get all 15 chunks, instead of 9.

Comment on attachment 8358587 [details] [diff] [review] Schedule debug B2G mochitests on trunk branches, Review of attachment 8358587 [details] [diff] [review]: ----------------------------------------------------------------- carry r+ forward

Sheriffs, we're turning on debug B2G emulator mochitests with the next buildbot reconfig. These are green on pine now; we'll deal aggressively with any frequent intermittents by disabling the tests.

(In reply to Jonathan Griffin (:jgriffin) from comment #10) > Sheriffs, we're turning on debug B2G emulator mochitests with the next > buildbot reconfig. These are green on pine now; we'll deal aggressively > with any frequent intermittents by disabling the tests. Stop the Presses! Everything is failing right now because of bug 958732.

The patch in bug 958732 fixes this. We just have to make sure it lands first.

in production

All b2g emulator debug mochitest jobs hidden on b2g-inbound for being orange; the other trees will need doing as soon as jobs have completed on them (and thus are able to be hidden in TBPL).

These are running very green again on b2g-inbound; any reason not to unhide them?

Oh they are hidden? Yes please un-hide again.

Unhidden on b2g-inbound, where I'm always happy to unhide (amusingly, permaorange on them had just landed). But let's see some of that comment 10 aggressive disabling before we unhide them elsewhere.

(In reply to Phil Ringnalda (:philor) from comment #18) > Unhidden on b2g-inbound, where I'm always happy to unhide (amusingly, > permaorange on them had just landed). But let's see some of that comment 10 > aggressive disabling before we unhide them elsewhere. Well if we know that changeset that regressed it we shouldn't blame the test :)

Oh, I don't want that permaorange disabled, I already backed out Nuwa for that (though the end result will probably be disabling a bogus assertion); I mean the rest of the constant but not permaorange failures. They are running "very green" if you look at several pushes where they were all green; they are not so green if you are the single person on the planet who looks at every single orange one on every single tree.

(In reply to Phil Ringnalda (:philor) from comment #20) > Oh, I don't want that permaorange disabled, I already backed out Nuwa for > that (though the end result will probably be disabling a bogus assertion); I > mean the rest of the constant but not permaorange failures. They are running > "very green" if you look at several pushes where they were all green; they > are not so green if you are the single person on the planet who looks at > every single orange one on every single tree. If you see frequent failures that we haven't noticed, please note them here or in bug 929122 and we'll disable them for now.

I'm closing this since the tests are running where we want them too; I realize there's some more effort required to green them up, but let's track that against bug 929122 or file new bugs, as needed.