Over in but 940273 I'm trying to land a new feature. This involves adding new tests. Unfortunately, when I add my new tests, the dom/media/test directory gets bumped from the M3 chunk to the M2 chunk. When this happens, the dom/media/tests/mochtiest directory starts failing with GMP plugin crashes. Here is a try without any patches applied: https://treeherder.mozilla.org/#/jobs?repo=try&revision=004ce511dbfe Here is a try with my bug 940273 patches applied. Note Windows M3: https://treeherder.mozilla.org/#/jobs?repo=try&revision=075680b3e027 Here is a try with my bug 940273 patch applied, but the tests removed. Note, these tests run *after* the media tests: https://treeherder.mozilla.org/#/jobs?repo=try&revision=da9e5f95a294 Also, if you run dom/media/tests/mochitest locally you also get a failure: ./mach mochitest-plain dom/media/tests/mochitest --run-until-failure 3639 INFO TEST-UNEXPECTED-ERROR | dom/media/tests/mochitest/test_dataChannel_basicAudio.html | This test left crash dumps behind, but we weren't expecting it to! TEST-INFO 3640 INFO TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_dataChannel_basicAudio.html | undefi ned assertion name - Result logged after SimpleTest.finish() The stackdump is not terribly useful, but suggests we are crashing when the exit(0) runs in the plugin shutdown. That code references bug 1073310. PROCESS-CRASH | Main app process exited normally | application crashed [@ KERNELBASE.dll + 0xb3c5f] Crash dump filename: c:\users\bkelly\appdata\local\temp\tmpqmnep9.mozrunner\minidumps\a1057a74-dee9- 4aba-b62f-23c1553f864e.dmp Operating system: Windows NT 6.3.9600 CPU: x86 GenuineIntel family 6 model 60 stepping 3 8 CPUs Crash reason: EXCEPTION_BREAKPOINT Crash address: 0x76963c5f Assertion: Unknown assertion type 0x00000000 Thread 0 (crashed) 0 KERNELBASE.dll + 0xb3c5f eip = 0x76963c5f esp = 0x00b4ecdc ebp = 0x00b4f5f4 ebx = 0x00000450 esi = 0x00000000 edi = 0x00b4f600 eax = 0x00000000 ecx = 0x6242be3c edx = 0x00000000 efl = 0x00200246 Found by: given as instruction pointer in context 1 D2D1Debug2.dll + 0xf4fa eip = 0x5e07f4fb esp = 0x00b4f5fc ebp = 0x00b4f604 Found by: previous frame's frame pointer 2 D2D1Debug2.dll + 0x4b346 eip = 0x5e0bb347 esp = 0x00b4f60c ebp = 0x00b4f668 Found by: previous frame's frame pointer 3 ntdll.dll + 0x49bdd eip = 0x771a9bde esp = 0x00b4f670 ebp = 0x00b4f688 Found by: previous frame's frame pointer 4 ntdll.dll + 0x49b57 eip = 0x771a9b58 esp = 0x00b4f690 ebp = 0x00b4f6d8 Found by: previous frame's frame pointer 5 ntdll.dll + 0x6d5b7 eip = 0x771cd5b8 esp = 0x00b4f6e0 ebp = 0x00b4f778 Found by: previous frame's frame pointer 6 ntdll.dll + 0x6d660 eip = 0x771cd661 esp = 0x00b4f780 ebp = 0x00b4f840 Found by: previous frame's frame pointer 7 kernel32.dll + 0x27f76 eip = 0x76f17f77 esp = 0x00b4f848 ebp = 0x00b4f854 Found by: previous frame's frame pointer 8 msvcr120.dll + 0x33fab eip = 0x5d583fac esp = 0x00b4f85c ebp = 0x00b4f860 Found by: previous frame's frame pointer 9 msvcr120.dll + 0x3427d eip = 0x5d58427e esp = 0x00b4f868 ebp = 0x00b4f8a8 Found by: previous frame's frame pointer 10 msvcr120.dll + 0x342a1 eip = 0x5d5842a2 esp = 0x00b4f8b0 ebp = 0x00b4f8bc Found by: previous frame's frame pointer 11 xul.dll!mozilla::ShutdownXPCOM(nsIServiceManager *) [XPCOMInit.cpp:170842a795a7 : 977 + 0x6] eip = 0x58e860b1 esp = 0x00b4f8c4 ebp = 0x00b4f8ec Found by: previous frame's frame pointer 12 xul.dll!XRE_TermEmbedding [nsEmbedFunctions.cpp:170842a795a7 : 207 + 0x6] eip = 0x5b246ae7 esp = 0x00b4f8f4 ebp = 0x00b4fe8c Found by: call frame info 13 xul.dll!mozilla::ipc::ScopedXREEmbed::Stop() [ScopedXREEmbed.cpp:170842a795a7 : 115 + 0x4] eip = 0x59170f93 esp = 0x00b4f900 ebp = 0x00b4fe8c Found by: call frame info 14 xul.dll!XRE_InitChildProcess [nsEmbedFunctions.cpp:170842a795a7 : 584 + 0x11] eip = 0x5b246056 esp = 0x00b4f908 ebp = 0x00b4fe8c Found by: call frame info 15 plugin-container.exe!content_process_main(int,char * * const) [plugin-container.cpp:170842a795a7 : 211 + 0xa] eip = 0x00f857da esp = 0x00b4fe94 ebp = 0x00b4feac Found by: call frame info 16 plugin-container.exe!wmain [nsWindowsWMain.cpp:170842a795a7 : 117 + 0x6] eip = 0x00f85c01 esp = 0x00b4feb4 ebp = 0x00b4fee0 Found by: call frame info 17 plugin-container.exe!__tmainCRTStartup [crt0.c : 255 + 0x11] eip = 0x00fd0016 esp = 0x00b4fee8 ebp = 0x00b4ff28 Found by: call frame info 18 kernel32.dll + 0x1919e eip = 0x76f0919f esp = 0x00b4ff30 ebp = 0x00b4ff34 Found by: call frame info 19 ntdll.dll + 0x5b5ae eip = 0x771bb5af esp = 0x00b4ff3c ebp = 0x00b4ff7c Found by: previous frame's frame pointer 20 ntdll.dll + 0x5b579 eip = 0x771bb57a esp = 0x00b4ff84 ebp = 0x00b4ff8c Found by: previous frame's frame pointer

From one of the try logs: 13:26:31 WARNING - PROCESS-CRASH | Main app process exited normally | application crashed [@ mozilla::gmp::GMPChild::ProcessingError(mozilla::ipc::HasResultCodes::Result,char const *)] 13:26:31 INFO - Crash dump filename: c:\docume~1\cltbld~1.t-x\locals~1\temp\tmpe7mdvf.mozrunner\minidumps\04212345-962c-4baf-81a3-bb8ea34411a4.dmp 13:26:31 INFO - Operating system: Windows NT 13:26:31 INFO - 5.1.2600 Service Pack 3 13:26:31 INFO - CPU: x86 13:26:31 INFO - GenuineIntel family 6 model 30 stepping 5 13:26:31 INFO - 8 CPUs 13:26:31 INFO - Crash reason: EXCEPTION_BREAKPOINT 13:26:31 INFO - Crash address: 0x1e8f5f5 13:26:31 INFO - Assertion: Unknown assertion type 0x00000000 13:26:31 INFO - Thread 0 (crashed) 13:26:31 INFO - 0 xul.dll!mozilla::gmp::GMPChild::ProcessingError(mozilla::ipc::HasResultCodes::Result,char const *) [GMPChild.cpp:075680b3e027 : 480 + 0x18] 13:26:31 INFO - eip = 0x01e8f5f5 esp = 0x0012f5ac ebp = 0x0012f5b0 ebx = 0x00a6c000 13:26:31 INFO - esi = 0x000001da edi = 0x0369daf8 eax = 0x00000000 ecx = 0x00690ad9 13:26:31 INFO - edx = 0x00750ea0 efl = 0x00000206 13:26:31 INFO - Found by: given as instruction pointer in context 13:26:31 INFO - 1 xul.dll!mozilla::ipc::MessageChannel::MaybeHandleError(mozilla::ipc::HasResultCodes::Result,IPC::Message const &,char const *) [MessageChannel.cpp:075680b3e027 : 1565 + 0x17] 13:26:31 INFO - eip = 0x00e73b57 esp = 0x0012f5b8 ebp = 0x0012f7d4 13:26:31 INFO - Found by: call frame info

12:58:31 INFO - [GMP 3832] WARNING: GMPGetAPI call failed trying to construct decoder.: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\media\gmp\GMPChild.cpp, line 575 12:58:31 INFO - IPDL protocol error: Handler for PGMPVideoDecoder returned error code 12:58:31 INFO - ###!!! [Child][DispatchAsyncMessage] Error: (msgtype=0x5A000D,name=PGMP::Msg_PGMPVideoDecoderConstructor) Processing error: message was deserialized, but the handler returned false (indicating failure) 12:58:31 INFO - Hit MOZ_CRASH(aborting because of MsgProcessingError) at c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\media\gmp\GMPChild.cpp:474

So the stack in comment 0 is from the local run. It seems perhaps the try run is hitting a different problem.

I tried applying the patches from bug 1136986, but they did not help: https://treeherder.mozilla.org/#/jobs?repo=try&revision=17f534254e00

Mass change P2->P3 to align with new Mozilla triage process.