These are now green on cedar (bug 1209327), so let's turn them on for trunk. Given capacity problems on Windows, let's just do this for OSX/linux for now.

Given that media-tests have a very high failure rate on Linux asan, debug, we should avoid these platforms for media-youtube-tests. I'd say Windows is the most important platform to test MSE on -- can we include just Windows 7, for example?

Builders added: + Windows 7 32-bit alder debug test media-youtube-tests + Windows 7 32-bit alder opt test media-youtube-tests + Windows 7 32-bit ash debug test media-youtube-tests + Windows 7 32-bit ash opt test media-youtube-tests + Windows 7 32-bit ash pgo test media-youtube-tests + Windows 7 32-bit b2g-inbound debug test media-youtube-tests + Windows 7 32-bit b2g-inbound opt test media-youtube-tests + Windows 7 32-bit b2g-inbound pgo test media-youtube-tests + Windows 7 32-bit fx-team debug test media-youtube-tests + Windows 7 32-bit fx-team opt test media-youtube-tests + Windows 7 32-bit fx-team pgo test media-youtube-tests + Windows 7 32-bit jamun debug test media-youtube-tests + Windows 7 32-bit jamun opt test media-youtube-tests + Windows 7 32-bit mozilla-aurora debug test media-youtube-tests + Windows 7 32-bit mozilla-aurora opt test media-youtube-tests + Windows 7 32-bit mozilla-aurora pgo test media-youtube-tests + Windows 7 32-bit mozilla-beta debug test media-youtube-tests + Windows 7 32-bit mozilla-beta opt test media-youtube-tests + Windows 7 32-bit mozilla-beta pgo test media-youtube-tests + Windows 7 32-bit mozilla-central debug test media-youtube-tests + Windows 7 32-bit mozilla-central opt test media-youtube-tests + Windows 7 32-bit mozilla-central pgo test media-youtube-tests + Windows 7 32-bit mozilla-inbound debug test media-youtube-tests + Windows 7 32-bit mozilla-inbound opt test media-youtube-tests + Windows 7 32-bit mozilla-inbound pgo test media-youtube-tests + Windows 7 32-bit oak debug test media-youtube-tests + Windows 7 32-bit oak opt test media-youtube-tests + Windows 7 32-bit oak pgo test media-youtube-tests + Windows 7 32-bit try debug test media-youtube-tests + Windows 7 32-bit try opt test media-youtube-tests

Comment on attachment 8702999 [details] [diff] [review] Schedule media-youtube-tests on trunk branches, Review of attachment 8702999 [details] [diff] [review]: ----------------------------------------------------------------- apology for delay, I was on pto. looks great. couple questions. setting to r- until you reset it to r? again. ::: mozilla-tests/config.py @@ +2506,4 @@ > BRANCHES[branch]['platforms'][platform][slave_platform]['debug_unittest_suites'] += bc_suite + dt_debug_suite > > > +# Enable mediatests on gecko >= 44 (bug 1209258, 1221963) this differs than the bug summary: "Bug 1221963 - schedule media-youtube-tests on trunk" and attachment summary suggests: https://bugzilla.mozilla.org/show_bug.cgi?id=1221963#c2 sanity check: are you sure you want to land this all the way to beta immediately? @@ +2515,4 @@ > if slave_platform in branch['platforms'][platform]: > branch['platforms'][platform][slave_platform]['opt_unittest_suites'] += MEDIATESTS > branch['platforms'][platform][slave_platform]['debug_unittest_suites'] += MEDIATESTS > + for slave_platform in ('win7-ix', 'yosemite'): is 'yosemite' used here at all or required to make configs happy? I see it's not mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=1221963#c3 @@ -2544,5 @@ > > BRANCHES['cedar']['platforms']['linux64-asan']['ubuntu64-asan_vm']['opt_unittest_suites'] += MARIONETTE > > -# Enable media-youtube-tests (bug 1209327) > -for slave_platform in ('ubuntu64_vm', 'ubuntu64-asan_vm', 'win7-ix', 'win8_64', 'yosemite'): this list suggests that we used to do more than 'win7-ix' and 'yosemite' on cedar: 'ubuntu64_vm', 'ubuntu64-asan_vm', 'win8_64' iiuc we won't be doing those on cedar anymore. They may be unused platforms as your builder list, https://bugzilla.mozilla.org/show_bug.cgi?id=1221963#c3 suggests that we are not losing any builders but I wanted to point that out

I believe we don't care too much about these tests on linux, just primarily win7 and (I assuming) OSX. Maja, can you confirm?

...and if we do care about yosemite, I need to change 'yosemite' to 'yosemite_r7'.

It's true that win7 and osx are the highest priority, but we care about linux, too. Perhaps the best choice is to use the same platforms as the 'media-tests' suite. I believe that amounts to linux64 (opt,pgo only) as well as win7, win8, yosemite.

Added OSX and linux. Kept win8 out, due to capacity issues. Cedar is removed purposefully; we're only using cedar as a proving ground for new jobs, and once they're scheduled on trunk, they're no longer needed there.

Builders added: + Rev7 MacOSX Yosemite 10.10.5 alder debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 alder opt test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 ash debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 ash opt test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 fx-team debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 fx-team opt test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 jamun debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 jamun opt test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 mozilla-central debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 mozilla-central opt test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 mozilla-inbound debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 mozilla-inbound opt test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 oak debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 oak opt test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 try debug test media-youtube-tests + Rev7 MacOSX Yosemite 10.10.5 try opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 alder opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 ash opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 b2g-inbound opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 fx-team opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 jamun opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 mozilla-central opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 mozilla-inbound opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 oak opt test media-youtube-tests + Ubuntu ASAN VM 12.04 x64 try opt test media-youtube-tests + Ubuntu Code Coverage VM 12.04 x64 try opt test media-youtube-tests + Ubuntu TSAN VM 12.04 x64 try opt test media-youtube-tests + Ubuntu VM 12.04 x64 alder debug test media-youtube-tests + Ubuntu VM 12.04 x64 alder opt test media-youtube-tests + Ubuntu VM 12.04 x64 ash debug test media-youtube-tests + Ubuntu VM 12.04 x64 ash opt test media-youtube-tests + Ubuntu VM 12.04 x64 ash pgo test media-youtube-tests + Ubuntu VM 12.04 x64 b2g-inbound debug test media-youtube-tests + Ubuntu VM 12.04 x64 b2g-inbound opt test media-youtube-tests + Ubuntu VM 12.04 x64 b2g-inbound pgo test media-youtube-tests + Ubuntu VM 12.04 x64 elm debug test media-youtube-tests + Ubuntu VM 12.04 x64 elm opt test media-youtube-tests + Ubuntu VM 12.04 x64 fx-team debug test media-youtube-tests + Ubuntu VM 12.04 x64 fx-team opt test media-youtube-tests + Ubuntu VM 12.04 x64 fx-team pgo test media-youtube-tests + Ubuntu VM 12.04 x64 jamun debug test media-youtube-tests + Ubuntu VM 12.04 x64 jamun opt test media-youtube-tests + Ubuntu VM 12.04 x64 mozilla-central debug test media-youtube-tests + Ubuntu VM 12.04 x64 mozilla-central opt test media-youtube-tests + Ubuntu VM 12.04 x64 mozilla-central pgo test media-youtube-tests + Ubuntu VM 12.04 x64 mozilla-inbound debug test media-youtube-tests + Ubuntu VM 12.04 x64 mozilla-inbound opt test media-youtube-tests + Ubuntu VM 12.04 x64 mozilla-inbound pgo test media-youtube-tests + Ubuntu VM 12.04 x64 oak debug test media-youtube-tests + Ubuntu VM 12.04 x64 oak opt test media-youtube-tests + Ubuntu VM 12.04 x64 oak pgo test media-youtube-tests + Ubuntu VM 12.04 x64 try debug test media-youtube-tests + Ubuntu VM 12.04 x64 try opt test media-youtube-tests + Windows 7 32-bit alder debug test media-youtube-tests + Windows 7 32-bit alder opt test media-youtube-tests + Windows 7 32-bit ash debug test media-youtube-tests + Windows 7 32-bit ash opt test media-youtube-tests + Windows 7 32-bit ash pgo test media-youtube-tests + Windows 7 32-bit b2g-inbound debug test media-youtube-tests + Windows 7 32-bit b2g-inbound opt test media-youtube-tests + Windows 7 32-bit b2g-inbound pgo test media-youtube-tests + Windows 7 32-bit fx-team debug test media-youtube-tests + Windows 7 32-bit fx-team opt test media-youtube-tests + Windows 7 32-bit fx-team pgo test media-youtube-tests + Windows 7 32-bit jamun debug test media-youtube-tests + Windows 7 32-bit jamun opt test media-youtube-tests + Windows 7 32-bit mozilla-central debug test media-youtube-tests + Windows 7 32-bit mozilla-central opt test media-youtube-tests + Windows 7 32-bit mozilla-central pgo test media-youtube-tests + Windows 7 32-bit mozilla-inbound debug test media-youtube-tests + Windows 7 32-bit mozilla-inbound opt test media-youtube-tests + Windows 7 32-bit mozilla-inbound pgo test media-youtube-tests + Windows 7 32-bit oak debug test media-youtube-tests + Windows 7 32-bit oak opt test media-youtube-tests + Windows 7 32-bit oak pgo test media-youtube-tests + Windows 7 32-bit try debug test media-youtube-tests + Windows 7 32-bit try opt test media-youtube-tests Builders removed - Ubuntu ASAN VM 12.04 x64 cedar opt test media-youtube-tests - Ubuntu VM 12.04 x64 cedar debug test media-youtube-tests - Ubuntu VM 12.04 x64 cedar opt test media-youtube-tests - Windows 7 32-bit cedar debug test media-youtube-tests - Windows 7 32-bit cedar opt test media-youtube-tests - Windows 8 64-bit cedar debug test media-youtube-tests - Windows 8 64-bit cedar opt test media-youtube-tests

Comment on attachment 8703894 [details] [diff] [review] Schedule media-youtube-tests on trunk branches, Review of attachment 8703894 [details] [diff] [review]: ----------------------------------------------------------------- lgtm :)

Comment on attachment 8703894 [details] [diff] [review] Schedule media-youtube-tests on trunk branches, https://hg.mozilla.org/build/buildbot-configs/rev/f7d9ee4eab35

In production: https://hg.mozilla.org/build/buildbot-configs/rev/f7d9ee4eab35

On second thought, let's not include Linux at all for media-youtube-tests: https://bugzilla.mozilla.org/show_bug.cgi?id=1176815#c58 Meanwhile, media-tests are doing fine on Linux, so we can leave it at that.

Turning off these on linux per maja_zf's request.

Comment on attachment 8706053 [details] [diff] [review] Turn off media-youtube-tests on linux, https://hg.mozilla.org/build/buildbot-configs/rev/1d5a4826538e