With bug 1471573 this would become more common, but I think the failure is already there (infact, the test is disabled in verify mode). I think there's a race between the mochitest harness trying to remove the crash dumps after the test is finished and BrowserTestUtils.crashBrowser when shouldClearMinidumps is true. If mochitest wins the race, all is well because crashBrowser only removes the file if it exists. If BrowserTestUtils.crashBrowser wins instead, the harness will fail with "This test did not leave any crash dumps behind, but we were expecting some!" (https://searchfox.org/mozilla-central/rev/6ef785903fee6c0b16a1eab79d722373d940fd78/testing/mochitest/tests/SimpleTest/TestRunner.js#544). The simplest fix is to just make process_error.xul call crashBrowser with shouldClearMinidumps set to false. It wasn't done as part of bug 1393800 so maybe there's something I'm missing.

Comment on attachment 8989888 [details] [diff] [review] Patch test-verify is passing with the changes: https://treeherder.mozilla.org/#/jobs?repo=try&revision=e40cf25bbed44402eeb1e5e925f14577e7db91a5.

Comment on attachment 8989888 [details] [diff] [review] Patch LGTM. I must have missed it when working on bug 1393800.

Pushed by mcastelluccio@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/cff5a58124f6 Make test_process_error.xul not delete the dumps, as the mochitest harness is doing it automatically. r=gsvelto

Backed out for crashtest failures on several tests. Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&fromchange=cff5a58124f6d4c33390b7348dc60acbcf1ef718&tochange=63e0f81d87e42f3fb449019c3fb7b4320dbc6cda&filter-searchStr=crashtest&selectedJob=186747820 Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=186747820&repo=mozilla-inbound&lineNumber=1293 Backout link: https://hg.mozilla.org/integration/mozilla-inbound/rev/63e0f81d87e42f3fb449019c3fb7b4320dbc6cda [task 2018-07-06T00:14:54.553Z] 00:14:54 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/accessible/tests/crashtests/last_test_to_unload_testsuite.xul | 10 / 3619 (0%) [task 2018-07-06T00:14:54.562Z] 00:14:54 INFO - JavaScript error: file:///builds/worker/workspace/build/tests/reftest/tests/accessible/tests/crashtests/last_test_to_unload_testsuite.xul, line 10: ReferenceError: SpecialPowers is not defined [task 2018-07-06T00:19:54.557Z] 00:19:54 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/accessible/tests/crashtests/last_test_to_unload_testsuite.xul | load failed: timed out waiting for reftest-wait to be removed [task 2018-07-06T00:19:54.559Z] 00:19:54 INFO - REFTEST INFO | Saved log: START file:///builds/worker/workspace/build/tests/reftest/tests/accessible/tests/crashtests/last_test_to_unload_testsuite.xul [task 2018-07-06T00:19:54.559Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering WaitForTestEnd [task 2018-07-06T00:19:54.560Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] WaitForTestEnd: Adding listeners [task 2018-07-06T00:19:54.564Z] 00:19:54 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot [task 2018-07-06T00:19:54.565Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_TO_FIRE_INVALIDATE_EVENT [task 2018-07-06T00:19:54.565Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: dispatching MozReftestInvalidate [task 2018-07-06T00:19:54.567Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL [task 2018-07-06T00:19:54.570Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for reftest-wait to be removed [task 2018-07-06T00:19:54.570Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///builds/worker/workspace/build/tests/reftest/tests/accessible/tests/crashtests/last_test_to_unload_testsuite.xul [task 2018-07-06T00:19:54.571Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] SendUpdateCanvasForEvent with 0 rects [task 2018-07-06T00:19:54.573Z] 00:19:54 INFO - REFTEST INFO | Saved log: Updating canvas for invalidation [task 2018-07-06T00:19:54.574Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] AfterPaintListener in file:///builds/worker/workspace/build/tests/reftest/tests/accessible/tests/crashtests/last_test_to_unload_testsuite.xul [task 2018-07-06T00:19:54.575Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] SendUpdateCanvasForEvent with 1 rects [task 2018-07-06T00:19:54.575Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] Rect: 0 0 800 1000 [task 2018-07-06T00:19:54.576Z] 00:19:54 INFO - REFTEST INFO | Saved log: Updating canvas for invalidation [task 2018-07-06T00:19:54.577Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL [task 2018-07-06T00:19:54.580Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for reftest-wait to be removed [task 2018-07-06T00:19:54.580Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: STATE_WAITING_FOR_REFTEST_WAIT_REMOVAL [task 2018-07-06T00:19:54.580Z] 00:19:54 INFO - REFTEST INFO | Saved log: [CONTENT] MakeProgress: waiting for reftest-wait to be removed [task 2018-07-06T00:19:54.580Z] 00:19:54 INFO - REFTEST TEST-END | file:///builds/worker/workspace/build/tests/reftest/tests/accessible/tests/crashtests/last_test_to_unload_testsuite.xul

(In reply to Cosmin Sabou [:CosminS] from comment #4) > Backed out for crashtest failures on several tests. It must be bug 1471573, which I pushed together with this.

Pushed by mcastelluccio@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/b6f878099e46 Make test_process_error.xul not delete the dumps, as the mochitest harness is doing it automatically. r=gsvelto