Reduce web-platform-tests-backlog timeout multiplier
Categories
(Testing :: General, task)
Tracking
(Not tracked)
People
(Reporter: egao, Assigned: egao)
References
Details
Attachments
(7 files, 1 obsolete file)
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details | |
|
(deleted),
text/x-phabricator-request
|
Details |
:jmaher has moved a lot of tests from web-platform-tests and web-platform-tests-reftest to the backlog side of things in the parent bug.
Tests for features that are not implemented yet, or will not be implemented, are currently run as separate suites.
Since the tests in the backlog side of things are expected to time out by default, let's find a way to safely reduce the time the test harness waits for the timeout result. This would help us reduce the runtime of the tests, which in turn will help us reduce the cost of machine time.
If things go as planned, it may even be possible to run all backlog tests in one chunk, therefore saving us machine time and setup/teardown overhead of multiple chunks.
|
Assignee | |
Comment 1•4 years ago
|
||
|
|
Assignee | |
Updated•4 years ago
|
|
|
Assignee | |
Comment 2•4 years ago
|
||
|
|
Assignee | |
Comment 3•4 years ago
|
||
|
|
Assignee | |
Comment 4•4 years ago
|
||
Changes:
- due to lower timeout multiplier, some subtests do not have the chance to run. These expectations are updated with the possibility that tests will TIMEOUT or NOTRUN.
|
||
Updated•4 years ago
|
Pushed by egao@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/9931a8bb043c adjust test expectations of some web-platform-tests-backlog tests r=jmaher
|
|
Assignee | |
Comment 6•4 years ago
|
||
|
|
||
Updated•4 years ago
|
|
|
||
Updated•4 years ago
|
|
|
||
Updated•4 years ago
|
Pushed by egao@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2b5133957b5e discontinue web-platform-tests-backlog on ccov/devedition platforms r=jmaher
|
|
Assignee | |
Updated•4 years ago
|
|
|
Assignee | |
Updated•4 years ago
|
Pushed by egao@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/3acd697a9131 web-platform-tests-backlog to set timeout-multiplier of 0.25 r=jmaher https://hg.mozilla.org/integration/autoland/rev/745ca9baa9c8 reduce web-platform-tests-backlog chunk counts r=jmaher
|
||
Comment 9•4 years ago
|
||
Backed out for multiple perma failures.
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=306554537&repo=autoland&lineNumber=6014
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=306550797&repo=autoland&lineNumber=6067
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=306550801&repo=autoland&lineNumber=1227
Backout: https://hg.mozilla.org/integration/autoland/rev/575059fe05879f2dd0ca0dac7d522d280391c3fc
|
||
Comment 10•4 years ago
|
||
Pushed by egao@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/024c23596ff6 web-platform-tests-backlog to set timeout-multiplier of 0.25 r=jmaher https://hg.mozilla.org/integration/autoland/rev/1793ec502846 reduce web-platform-tests-backlog chunk counts r=jmaher
|
|
Assignee | |
Comment 11•4 years ago
|
||
Added additional conditional to check if the test_type is testharness.
|
||
Comment 12•4 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 13•4 years ago
|
||
|
|
||
Comment 14•4 years ago
|
||
Comment on attachment 9157684 [details]
Bug 1643177 - add TIMEOUT expectations for some tests in network-error-logging for macosx1014 web-platform-tests-backlog
Revision D80219 was moved to bug 1646319. Setting attachment 9157684 [details] to obsolete.
|
|
Assignee | |
Comment 15•4 years ago
|
||
|
|
Assignee | |
Updated•4 years ago
|
|
|
Assignee | |
Comment 16•4 years ago
|
||
Bug 1646319 is for linux1804-64/debug failures due to this change.
|
|
Assignee | |
Comment 17•4 years ago
|
||
|
|
Assignee | |
Comment 18•4 years ago
|
||
|
|
||
Comment 19•4 years ago
|
||
Pushed by egao@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/5065d048c930 add expectations for some /network-error-logging tests for web-platform-tests-backlog on linux1804-64/debug r=jmaher
|
|
||
Comment 20•4 years ago
|
||
Pushed by egao@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/579780116dbf apply timeout multiplier reduction to web-platform-tests-reftest-backlog and reduce chunks r=jmaher
|
|
||
Comment 21•4 years ago
|
||
Pushed by egao@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/4eab38aba3fe separate web-platform-tests-backlog tests into its own test-set r=jmaher
|
||
Comment 22•4 years ago
|
||
| bugherder | ||
|
||
Comment 23•4 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 24•4 years ago
|
||
Multiplier has been reduced to 0.25. Further tuning should be done in its own bug.
Description
•