According to Jonas, we want to green up the debug B2G mochitests ASAP, and roll them out everywhere. So, we won't wait for the mochitest manifest work, but rather will add a new instance of b2g.json that we can use to exclude failing tests.

Will disable failing tests in a separate patch.

Comment on attachment 819923 [details] [diff] [review] Add b2g-debug.json, Review of attachment 819923 [details] [diff] [review]: ----------------------------------------------------------------- This could get ugly fast if we keep creating these things :p

Agreed. Let's hope we move to a manifest before this becomes a big headache.

https://hg.mozilla.org/integration/mozilla-inbound/rev/358f2e245f45 Now I need to get the mozharness script to use it.

I'm not sure if this is the preferred mechanism for doing something like this. I didn't see a way to do this in buildbot-configs without adding additional logic.

Comment on attachment 819992 [details] [diff] [review] Use b2g-debug.json for debug mochitests, Yeah, I wish we had a better way to do this too. Pretty sure this will do the trick.

(In reply to Aki Sasaki [:aki] from comment #6) > Comment on attachment 819992 [details] [diff] [review] > Use b2g-debug.json for debug mochitests, > > Yeah, I wish we had a better way to do this too. > Pretty sure this will do the trick. https://hg.mozilla.org/build/mozharness/rev/18e3bd278a5f

merged to production mozharness

This disables all the currently failing debug mochitests on B2G; we may need to disable others after this, since some failures are likely hiding others. Per jonas' advice, I've not filed bugs for each of these, but all these are marked with 'debug-only failure' to make them easy to find.

Comment on attachment 824111 [details] [diff] [review] Disable failing debug mochitests on B2G, Review of attachment 824111 [details] [diff] [review]: ----------------------------------------------------------------- Looks good to me, though if you are only landing these on cedar for now, I don't think you need to get my r+ each time. I've been keeping a master patch in my local patch queue that I'll get review on once everythings ready to land.

(In reply to Andrew Halberstadt [:ahal] from comment #11) > Comment on attachment 824111 [details] [diff] [review] > Disable failing debug mochitests on B2G, > > Review of attachment 824111 [details] [diff] [review]: > ----------------------------------------------------------------- > > Looks good to me, though if you are only landing these on cedar for now, I > don't think you need to get my r+ each time. I've been keeping a master > patch in my local patch queue that I'll get review on once everythings ready > to land. Good point! https://hg.mozilla.org/projects/cedar/rev/d691b456f44b

This updates b2g-debug.json from the version that's currently running on pine.

This seems to clean things up on pine.

Comment on attachment 8358543 [details] [diff] [review] Disable a couple more debug B2G mochitests, DONTBUILD because NPOTB Review of attachment 8358543 [details] [diff] [review]: ----------------------------------------------------------------- Thanks! So much green \o/

https://hg.mozilla.org/integration/b2g-inbound/rev/746cbddb0b76 Hopefully this does it, but I'll leave this bug open until the tests are running on trunks branches, just in case.

https://hg.mozilla.org/mozilla-central/rev/0774521f4f37 https://hg.mozilla.org/mozilla-central/rev/746cbddb0b76

mochitest-5 in particular is hitting a lot of orange. See the new deps. Doesn't help that the runtime is ~80min. I've also added a few other failures that we're hitting in the debug tests primarily.

I'm good with disabling these tests if they appear more than occasionally: > Depends on: 923329, 961107, 922975, 961021, 960922, 960762, 931672, 960488, 947869

The most egregious are 923329, 961107, and 947869. I'll file a patch for disabling just these.

(In reply to Jonathan Griffin (:jgriffin) from comment #21) > The most egregious are 923329, 961107, and 947869. I'll file a patch for > disabling just these. Nvm, philor beat me to it for the first two, gwagner already did the last.

And now bug 961021; disabling that test.

These are running everywhere now, so am closing this meta-bug.