We're currently running 9 chunks, but these may be too long (at any rate, they commonly exceed 60 minutes). We should expand this to at least 12 chunks.

jgriffin: thanks for filing this. Ideally we'd like each specific job to run 25-30 mins... If we can do that with "only" 12 chunks great... if we need more chunks, go for it!

It's very likely we'll need more chunks then...we can try with 15 initially.

AFAICT, there's no way to change just the number of debug chunks without introducing a new test suite name, e.g., mochitest-debug-1, instead of mochitest-1. I think we are forced to choose either introducing a new suite name, or increasing the number of opt chunks as well. We'd have to update TBPL to understand the new suite name, if we went that route, as well as downstream consumers like OrangeFactor. On the other hand, increasing the number of opt chunks would increase our compute hours per commit, due to the overhead of the additional setup/teardown times. releng, do you have a preference here?

Hm, I don't have strong opinions, but: assuming debug tests will take longer forever (or for the long term, at least), that would probably mean we want a different number of chunks for the long term, and the mochitest-debug-# would make sense. I think this is the case, but I'm not sure. If debug tests are going to converge with opt tests in terms of runtime in the short- or medium-term (not sure if this is possible, but if), increasing the total number of opt+debug chunks (or having slightly fewer debug chunks that take longer) might make sense.

I think debug tests will run much slower for the foreseeable future. I'll go with the different suite name.

This looks good.

Comment on attachment 8341497 [details] [diff] [review] Use 15 chunks for debug B2G mochitests, Thanks Jonathan!

agree with Aki, just note that we'll see possibly different test behaviour because of different chunking between opt and debug.

And different developer behavior, since most people think that if they look in one log and find that the test they are working on runs in (opt) m-5, then they can use try: -b do -p emulator -u mochitest-5 to run their test on opt and debug.

We can fix those issues if they become a real problem. Right now we have to get debug emulator tests running.

https://hg.mozilla.org/build/buildbot-configs/rev/7fd217b1cee3 Note that when this goes live, we'll need to update TBPL to handle the new suite name.

in production