Closed Bug 1372738 Opened 7 years ago Closed 7 years ago

A test timing out may show false uncaught errors in subsequent tests

Categories

(Testing :: Mochitest, defect)

Version 3
defect
Not set
normal

Tracking

(firefox56 fixed)

RESOLVED FIXED
mozilla56
Tracking Status
firefox56 --- fixed

People

(Reporter: standard8, Assigned: standard8)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Investigating bug 1320230, I'm seeing a number of cases where a test times out due to a promise not resolving. A subsequent test then triggers something that resolves the promise, at which point the test output complains that subsequent test failed (with a strange message), whereas it was really the first test's fault. For example: TEST-UNEXPECTED-FAIL | browser/components/search/test/browser_searchbar_smallpanel_keyboard_navigation.js | Test timed out TEST-UNEXPECTED-FAIL | browser/components/search/test/browser_tooManyEnginesOffered.js | A promise chain failed to handle a rejection: - at chrome://mochikit/content/browser-test.js:810 - TypeError: this.SimpleTest.isExpectingUncaughtException is not a function The "not a function" part is due to the test having already been cleaned up. I have a patch to make the uncaught exception and the possible reason for it clearer. I'm not 100% sure it is useful to output the exception as a result of the timed out test, as I suspect it'll normally be "something is undefined", but I'd prefer to err on the side of caution, hence outputting it anyway.
Summary: A test timing out may cause false uncaught errors in subsequent tests → A test timing out may show false uncaught errors in subsequent tests
Comment on attachment 8877327 [details] Bug 1372738 - A test timing out may show false uncaught errors in subsequent tests. https://reviewboard.mozilla.org/r/148676/#review153148 I'm fine with this though an alternative would be to cache the value of SimpleTest.isExpectingUncaughtException() on currentTest when it is timed out.
Attachment #8877327 - Flags: review?(dtownsend) → review+
Comment on attachment 8877327 [details] Bug 1372738 - A test timing out may show false uncaught errors in subsequent tests. https://reviewboard.mozilla.org/r/148676/#review153148 Yeah, that would be a possiblility, but we'd still need the early return as everything else has been cleaned up already. So I think it is reasonable as is.
Pushed by mbanner@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8883b171d188 A test timing out may show false uncaught errors in subsequent tests. r=mossop
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: