Open Bug 1636939 Opened 4 years ago Updated 2 years ago

Mochitest results in errorsummary.log can contain empty groups

Categories

(Firefox Build System :: Task Configuration, defect, P3)

defect

Tracking

(Not tracked)

People

(Reporter: marco, Unassigned)

References

(Blocks 1 open bug)

Details

Recently, I started seeing test_result lines in Mochitest errorsummary.log files which contain an empty group.

I think it is a regression, as I have code relying on "group" not to be empty which has never failed for a long time.

E.g. in https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=52cf30bf74473e78c0baf8c4d6055910f67a0a37&searchStr=test-linux1804-64-shippable%2Fopt-mochitest-plain-e10s-1&selectedTaskRun=V_lcR6-nRaaOMyciFzg7Jw-0:

{"action": "test_groups", "line": 2, "groups": ["browser/extensions/formautofill/test/mochitest/creditCard/mochitest.ini", "testing/mochitest/tests/Harness_sanity/mochitest.ini", "dom/tests/mochitest/storageevent/mochitest.ini", "dom/tests/mochitest/sessionstorage/mochitest.ini", "browser/components/payments/test/mochitest/mochitest.ini", "image/test/mochitest/mochitest.ini", "dom/vr/test/mochitest/mochitest.ini", "docshell/test/iframesandbox/mochitest.ini", "dom/indexedDB/test/mochitest-intl-api.ini", "dom/credentialmanagement/tests/mochitest/mochitest.ini", "browser/components/payments/test/mochitest/formautofill/mochitest.ini", "layout/forms/test/mochitest.ini", "dom/payments/test/mochitest.ini", "dom/tests/mochitest/ajax/offline/mochitest.ini", "dom/file/tests/mochitest.ini", "netwerk/test/mochitests/mochitest.ini", "toolkit/components/windowcreator/test/mochitest.ini", "dom/canvas/test/crossorigin/mochitest.ini", "modules/libjar/test/mochitest/mochitest.ini", "dom/xhr/tests/mochitest.ini", "testing/mochitest/baselinecoverage/plain/mochitest.ini", "dom/tests/mochitest/keyhandling/mochitest.ini", "widget/tests/mochitest.ini", "intl/uconv/tests/mochitest.ini", "dom/indexedDB/test/mochitest.ini", "netwerk/cookie/test/mochitest/mochitest.ini", "dom/security/test/general/mochitest.ini", "dom/events/test/pointerevents/mochitest.ini", "dom/tests/mochitest/dom-level1-core/mochitest.ini", "parser/htmlparser/tests/mochitest/mochitest.ini", "dom/security/test/https-only/mochitest.ini", "dom/tests/mochitest/orientation/mochitest.ini", "caps/tests/mochitest/mochitest.ini", "security/manager/ssl/tests/mochitest/stricttransportsecurity/mochitest.ini", "dom/localstorage/test/mochitest.ini"]}
{"status": "FAIL", "group": "dom/indexedDB/test/mochitest.ini", "line": 4007, "subtest": "Test timed out.", "action": "test_result", "test": "dom/indexedDB/test/test_readonly_transactions.html", "message": null, "stack": "    SimpleTest.ok@SimpleTest/SimpleTest.js:299:16\n    reportError@SimpleTest/TestRunner.js:128:22\n    TestRunner._checkForHangs@SimpleTest/TestRunner.js:150:18\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    TestRunner.runTests/<@SimpleTest/TestRunner.js:420:16\n    promise callback*TestRunner.runTests@SimpleTest/TestRunner.js:407:48\n    RunSet.runtests@SimpleTest/setup.js:218:14\n    RunSet.runall@SimpleTest/setup.js:197:12\n    hookupTests@SimpleTest/setup.js:294:12\nparseTestManifest@http://mochi.test:8888/manifestLibrary.js:46:13\ngetTestManifest/req.onload@http://mochi.test:8888/manifestLibrary.js:59:28\nEventHandlerNonNull*getTestManifest@http://mochi.test:8888/manifestLibrary.js:55:3\n    hookup@SimpleTest/setup.js:270:20\nEventHandlerNonNull*@http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&consoleLevel=INFO&manifestFile=tests.json&dumpOutputDirectory=%2Ftmp&cleanupCrashes=true:11:1\n", "expected": "PASS"}
{"status": "FAIL", "group": "dom/indexedDB/test/mochitest.ini", "line": 4018, "subtest": "Test timed out.", "action": "test_result", "test": "dom/indexedDB/test/test_readwriteflush_disabled.html", "message": null, "stack": "    SimpleTest.ok@SimpleTest/SimpleTest.js:299:16\n    reportError@SimpleTest/TestRunner.js:128:22\n    TestRunner._checkForHangs@SimpleTest/TestRunner.js:150:18\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    TestRunner.runTests/<@SimpleTest/TestRunner.js:420:16\n    promise callback*TestRunner.runTests@SimpleTest/TestRunner.js:407:48\n    RunSet.runtests@SimpleTest/setup.js:218:14\n    RunSet.runall@SimpleTest/setup.js:197:12\n    hookupTests@SimpleTest/setup.js:294:12\nparseTestManifest@http://mochi.test:8888/manifestLibrary.js:46:13\ngetTestManifest/req.onload@http://mochi.test:8888/manifestLibrary.js:59:28\nEventHandlerNonNull*getTestManifest@http://mochi.test:8888/manifestLibrary.js:55:3\n    hookup@SimpleTest/setup.js:270:20\nEventHandlerNonNull*@http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&consoleLevel=INFO&manifestFile=tests.json&dumpOutputDirectory=%2Ftmp&cleanupCrashes=true:11:1\n", "expected": "PASS"}
{"status": "FAIL", "group": "dom/indexedDB/test/mochitest.ini", "line": 4029, "subtest": "Test timed out.", "action": "test_result", "test": "dom/indexedDB/test/test_remove_index.html", "message": null, "stack": "    SimpleTest.ok@SimpleTest/SimpleTest.js:299:16\n    reportError@SimpleTest/TestRunner.js:128:22\n    TestRunner._checkForHangs@SimpleTest/TestRunner.js:150:18\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    TestRunner.runTests/<@SimpleTest/TestRunner.js:420:16\n    promise callback*TestRunner.runTests@SimpleTest/TestRunner.js:407:48\n    RunSet.runtests@SimpleTest/setup.js:218:14\n    RunSet.runall@SimpleTest/setup.js:197:12\n    hookupTests@SimpleTest/setup.js:294:12\nparseTestManifest@http://mochi.test:8888/manifestLibrary.js:46:13\ngetTestManifest/req.onload@http://mochi.test:8888/manifestLibrary.js:59:28\nEventHandlerNonNull*getTestManifest@http://mochi.test:8888/manifestLibrary.js:55:3\n    hookup@SimpleTest/setup.js:270:20\nEventHandlerNonNull*@http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&consoleLevel=INFO&manifestFile=tests.json&dumpOutputDirectory=%2Ftmp&cleanupCrashes=true:11:1\n", "expected": "PASS"}
{"status": "FAIL", "group": "dom/indexedDB/test/mochitest.ini", "line": 4040, "subtest": "Test timed out.", "action": "test_result", "test": "dom/indexedDB/test/test_remove_objectStore.html", "message": null, "stack": "    SimpleTest.ok@SimpleTest/SimpleTest.js:299:16\n    reportError@SimpleTest/TestRunner.js:128:22\n    TestRunner._checkForHangs@SimpleTest/TestRunner.js:150:18\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    TestRunner.runTests/<@SimpleTest/TestRunner.js:420:16\n    promise callback*TestRunner.runTests@SimpleTest/TestRunner.js:407:48\n    RunSet.runtests@SimpleTest/setup.js:218:14\n    RunSet.runall@SimpleTest/setup.js:197:12\n    hookupTests@SimpleTest/setup.js:294:12\nparseTestManifest@http://mochi.test:8888/manifestLibrary.js:46:13\ngetTestManifest/req.onload@http://mochi.test:8888/manifestLibrary.js:59:28\nEventHandlerNonNull*getTestManifest@http://mochi.test:8888/manifestLibrary.js:55:3\n    hookup@SimpleTest/setup.js:270:20\nEventHandlerNonNull*@http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&consoleLevel=INFO&manifestFile=tests.json&dumpOutputDirectory=%2Ftmp&cleanupCrashes=true:11:1\n", "expected": "PASS"}
{"status": "FAIL", "group": "", "line": 4042, "subtest": "4 test timeouts, giving up.", "action": "test_result", "test": "(SimpleTest/TestRunner.js)", "message": null, "stack": "    SimpleTest.ok@SimpleTest/SimpleTest.js:299:16\n    reportError@SimpleTest/TestRunner.js:128:22\n    TestRunner._checkForHangs@SimpleTest/TestRunner.js:158:20\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    TestRunner.runTests/<@SimpleTest/TestRunner.js:420:16\n    promise callback*TestRunner.runTests@SimpleTest/TestRunner.js:407:48\n    RunSet.runtests@SimpleTest/setup.js:218:14\n    RunSet.runall@SimpleTest/setup.js:197:12\n    hookupTests@SimpleTest/setup.js:294:12\nparseTestManifest@http://mochi.test:8888/manifestLibrary.js:46:13\ngetTestManifest/req.onload@http://mochi.test:8888/manifestLibrary.js:59:28\nEventHandlerNonNull*getTestManifest@http://mochi.test:8888/manifestLibrary.js:55:3\n    hookup@SimpleTest/setup.js:270:20\nEventHandlerNonNull*@http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&consoleLevel=INFO&manifestFile=tests.json&dumpOutputDirectory=%2Ftmp&cleanupCrashes=true:11:1\n", "expected": "PASS"}
{"status": "FAIL", "group": "", "line": 4044, "subtest": "Skipping 26 remaining tests.", "action": "test_result", "test": "(SimpleTest/TestRunner.js)", "message": null, "stack": "    SimpleTest.ok@SimpleTest/SimpleTest.js:299:16\n    reportError@SimpleTest/TestRunner.js:128:22\n    TestRunner._checkForHangs@SimpleTest/TestRunner.js:163:20\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    setTimeout handler*TestRunner._checkForHangs@SimpleTest/TestRunner.js:184:15\n    TestRunner.runTests/<@SimpleTest/TestRunner.js:420:16\n    promise callback*TestRunner.runTests@SimpleTest/TestRunner.js:407:48\n    RunSet.runtests@SimpleTest/setup.js:218:14\n    RunSet.runall@SimpleTest/setup.js:197:12\n    hookupTests@SimpleTest/setup.js:294:12\nparseTestManifest@http://mochi.test:8888/manifestLibrary.js:46:13\ngetTestManifest/req.onload@http://mochi.test:8888/manifestLibrary.js:59:28\nEventHandlerNonNull*getTestManifest@http://mochi.test:8888/manifestLibrary.js:55:3\n    hookup@SimpleTest/setup.js:270:20\nEventHandlerNonNull*@http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&consoleLevel=INFO&manifestFile=tests.json&dumpOutputDirectory=%2Ftmp&cleanupCrashes=true:11:1\n", "expected": "PASS"}
{"status": "FAIL", "group": "", "line": 4047, "subtest": "/tests/dom/indexedDB/test/test_remove_objectStore.html", "action": "test_result", "test": "(SimpleTest/TestRunner.js)", "message": "finished in a non-clean fashion, probably because it didn't call SimpleTest.finish()", "stack": "{u'loaded_test_url': u'/tests/dom/indexedDB/test/test_remove_objectStore.html'}", "expected": "PASS"}
{"status": "ERROR", "group": "", "line": 4048, "subtest": null, "action": "test_result", "test": "(SimpleTest/TestRunner.js)", "message": "Finished in 329989ms", "stack": "{u'runtime': 329989}", "expected": "OK"}

Same in https://firefoxci.taskcluster-artifacts.net/RYmCJO0fQsOor5jwtCCUzw/0/public/test_info//wpt_errorsummary.log:

{"status": "FAIL", "group": "", "line": 79772, "subtest": "Call foo", "action": "test_result", "test": "/html/cross-origin-opener-policy/access-reporting/opener-accessed_openee-coop-ro.https.html", "message": "assert_not_equals: got disallowed value \"timeout\"", "stack": "@https://web-platform.test:8443/html/cross-origin-opener-policy/access-reporting/opener-accessed_openee-coop-ro.https.html:55:22\n", "expected": "TIMEOUT"}
{"status": "TIMEOUT", "group": "", "line": 79773, "subtest": "Call location", "action": "test_result", "test": "/html/cross-origin-opener-policy/access-reporting/opener-accessed_openee-coop-ro.https.html", "message": "Test timed out", "stack": null, "expected": "NOTRUN"}

(as a side note, in this task "suite_end" was not emitted)

Summary: Mochitest results in errorsummary.log contain empty groups → Mochitest and WPT results in errorsummary.log can contain empty groups

WPT is probably bug 1646761.

Summary: Mochitest and WPT results in errorsummary.log can contain empty groups → Mochitest results in errorsummary.log can contain empty groups

Marco, do you know if this is still an issue? Or should we close it out and re-open if it surfaces again.

Severity: -- → S3
Flags: needinfo?(mcastelluccio)
Priority: -- → P3

Yes this is still happening.

Flags: needinfo?(mcastelluccio)
You need to log in before you can comment on or make changes to this bug.