Closed Bug 1626089 Opened 5 years ago Closed 5 years ago

Run with multiple processes in CI

Categories

(Testing :: web-platform-tests, enhancement)

Version 3
enhancement
Not set
normal

Tracking

(firefox77 fixed)

RESOLVED FIXED
mozilla77
Tracking Status
firefox77 --- fixed

People

(Reporter: jgraham, Assigned: jgraham)

References

Details

(Whiteboard: [ci-costs-2020:done])

Attachments

(2 files)

This should allow a speedup at the cost of perhaps introducing some instability. It doesn't work on android since we can only run a single copy of the geckoview test app.

This runs wpt with --processes=2 in CI, excluding wdspec tests where this
causes obvious test failures and android where we can't have more than a
single instance. This should provide some speedup at the cost of possibly
increased instability. So following landing this it will be necessary to
look out for an increased rate of intermittents.

Assignee: nobody → james
Status: NEW → ASSIGNED

this seems to be a failuree specifically on windows 7 shippable:
TEST-UNEXPECTED-FAIL | /webrtc/protocol/bundle.https.html | bundles on the first transport and closes the second - assert_not_equals: got disallowed value undefined
TEST-UNEXPECTED-OK | /webrtc/protocol/bundle.https.html | expected TIMEOUT

I couldn't see anything in the log file that was an indicator of what went wrong or what is different from the normal timeout.

Looks like that test is already marked as intermittent on some Windows configurations, so I think it might be enough to mark it as always intermittent on windows opt.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7b7fbd8c6c36ed9043311240922377bf08ce6d20 for a try push with that change.

Whiteboard: [ci-costs-2020:todo]
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla77
Regressions: 1628456
Regressions: 1627743
Regressions: 1626181
Regressions: 1535922

Running this made intermittents too frequent to manage on this platform.

Pushed by nerli@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e4034d6f39e5 Disable multiple processes for fission builds, r=jmaher
Regressions: 1628960
Regressions: 1626943

I've tested backing this out on try, the regressions here do not show up anymore, however, there is this new failure: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception&revision=e28069b1e1cb95c5866d149cc0d5ae8e6b2674ba&selectedJob=297299804

Should we move on with backing this out on autoland?

Additional fixes for the regression don't seem to work in some cases: https://bugzilla.mozilla.org/show_bug.cgi?id=1626943#c5

Flags: needinfo?(aryx.bugmail)

Given that the failure rate in the regression bugs has dropped, the backout is no longer necessary?

Flags: needinfo?(james)
Whiteboard: [ci-costs-2020:todo] → [ci-costs-2020:done]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: