Some MediaSource web platform tests run with inconsistent results when media.mediasource.enabled is set. The suggested approach for unstable tests [1] is to disable the tests. [1] https://github.com/mozilla/gecko-dev/blob/master/testing/web-platform/README.md#disabling-tests I intend to leave this bug open after disabling to track re-enabling the tests.

Testing against revision 4d1793da0b96, the following tests or subtests produce unstable results: mediasource-config-change-webm-av-audio-bitrate.html Parent may PASS or TIMEOUT [Tests webm audio bitrate changes in multiplexed content.] may PASS, or FAIL mediasource-config-change-webm-av-video-bitrate.html Parent may PASS or TIMEOUT [Tests webm video bitrate changes in multiplexed content.] may PASS, or FAIL mediasource-getvideoplaybackquality.html Parent may TIMEOUT or FAIL [Test HTMLVideoElement.getVideoPlaybackQuality() with MediaSource API] may FAIL or TIMEOUT mediasource-play.html Parent may TIMEOUT or PASS [Test normal playback case with MediaSource API] may FAIL or TIMEOUT mediasource-redundant-seek.html [Test redundant fully prebuffered seek] may TIMEOUT or FAIL mediasource-seek-beyond-duration.html [Test seeking beyond media duration] may FAIL or TIMEOUT

Tested on revisions in the range 4d1793da0b96 to d462269d88a9, mediasource-config-change-* tests were found to cause TIMEOUT in the immediately following test. The -config-change-mp4- tests all TIMEOUT themselves when run individually. With only mediasource-config-change-webm-a-bitrate.html and mediasource-config-change-webm-v-bitrate.html TIMEOUTs in the next test were intermittent. With other -config-change-webm- tests, TIMEOUTs in subsequent tests were consistent.

Tested on revision d462269d88a9, running any of the following tests mediasource-buffered.html.ini mediasource-duration.html.ini mediasource-play-then-seek-back.html.ini mediasource-redundant-seek.html mediasource-seek-beyond-duration.html.ini mediasource-seek-during-pending-seek.html.ini mediasource-sourcebuffer-mode.html.ini causes failure of old-tests/submission/Microsoft/sandbox/sandbox_002.htm, which runs much later in the testsuite: Allow autoplay for HTML5 Video inside iframe with sandbox attribute if sandbox='allow-scripts'. ----------------------------------------------------------------------------------------------- Expected PASS, got FAIL event is undefined(stack: callback/<@http://web-platform.test:8000/old-tests/submission/Microsoft/sandbox/sandbox_002.htm:22:17 Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:798:20 callback@http://web-platform.test:8000/old-tests/submission/Microsoft/sandbox/sandbox_002.htm:20:13 )

This is disabling 23 of 41 tests, but it means we can be running 18 of them.

https://hg.mozilla.org/integration/mozilla-inbound/rev/1dbd58c232fd https://tbpl.mozilla.org/?tree=Try&rev=370a0779930a

Set to P5 because this is a tracking bug.

(In reply to Karl Tomlinson (:karlt) from comment #2) > The -config-change-mp4- tests all TIMEOUT themselves when run individually. These are now stable, but failing. Enabled in https://hg.mozilla.org/integration/mozilla-inbound/rev/5d07a03af0b0

mac and windows >= vista should provide the same results now.

Update web reftest to reflect changes. Sounds like more could be re-enabled. Rebase

Comment on attachment 8545582 [details] [diff] [review] update test results for mac > TIMEOUT >- Please leave the blank line at the end as autogenerated results will add it again when the tests are next pulled from upstream. (Only subtest annotations get a blank line. Parent annotations don't have trailing blank lines.)

Carrying r+

Bug 1125776 should have made most of those tests now return reliable results.

with bug 1171379, hopefully it won't be unstable anymore..

The leave-open keyword is there and there is no activity for 6 months. :jya, maybe it's time to close this bug?

the aim was to find the intermittents once the new MSE code went it. obviously worked