Closed Bug 1673071 Opened 4 years ago Closed 4 years ago

Flip macOS tests to running mostly with WR-enabled

Categories

(Core :: Graphics: WebRender, task)

task

Tracking

()

RESOLVED FIXED
84 Branch
Tracking Status
firefox84 --- fixed

People

(Reporter: kats, Assigned: kats)

References

(Depends on 2 open bugs, Blocks 1 open bug)

Details

Attachments

(4 files)

WebRender is getting turned on for 90%+ of the macOS population in Fx 83. So we should ensure that most of the CI jobs we run for macOS are doing so with WR enabled. We should still leave some sanity tests for the layers codepath.

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=71885dcb6cd8603001ac8367d6ecdabee2de0171

This is a try push where I simply copied all the suites that run for macOS (layers) and put them under macOS (QR). Seems mostly in good shape but a bunch of tests might need disabling and/or fuzzing or expectation fixing.

Depends on: 1673105
Severity: -- → S3

https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=11ca8e035f57725a0fc142ce856bd7bd0cc5df44 should have all the right taskcluster config changes. It basically flips around the current macOS coverage so that all the jobs that were running without WR before are now running with WR. And the jobs that were running with WR are now running on macosx1014-64/debug and macosx1014-64-devedition/opt. I also adjusted some of the test expectations from the last try push but there will be some more tests that I need to skip with WR enabled (at least those covered by bug 1673105, and maybe a couple more).

There are two failures which are due to MOZ_ASSERT failures, but I can't reproduce those locally. Bug 1674411 and bug 1674413 track those and I'm just going to mark those tests as skip-if for now. Bug 1674412 seems intermittent so for now I'll leave it as running and if the frequency of failure is too high we can try to do something about it. Bug 1674060 was filed for failures on windows-ccov-qr but I'm seeing those on macosx-qr as well, so I'm going to skip the two tests involved in that as well.

https://treeherder.mozilla.org/jobs?repo=try&group_state=expanded&revision=f39b076429f66e6de4b24fa4b8ef262821c2cac8 was my last try push. The marionette mozinfo object appears to not have the right value for the webrender field, so my skip-if didn't take effect. And I missed some of the webgl failures since the TreeHerder summary didn't show them all the last time around. Both of those problems should be fixed in this new push:

https://treeherder.mozilla.org/jobs?repo=try&group_state=expanded&revision=f07fa06a4de865fe1a611ed720a1c4cf55d2badd

Assignee: nobody → kats
Status: NEW → ASSIGNED

This patch flips things around so that the non-QR macosx test platforms are
running a "skeleton crew" set of tests, and the QR macosx test platforms are
running all the tests that were previously running with non-QR. As of Fx83
macOS has WebRender enabled by default for 90%+ of the population so it makes
sense to test that configuration more heavily, but leave some tests running
for the non-QR configuration as well.

Depends on D95429

Pushed by kgupta@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/d6857ecf3c81 Update some test expectations for macOS+WR configuration. r=jrmuizel https://hg.mozilla.org/integration/autoland/rev/34d6c0fbfeef Update marionette mozinfo object with correct webrender status. r=whimboo,marionette-reviewers https://hg.mozilla.org/integration/autoland/rev/f2ed8ed80bf2 Disable failing tests. r=jrmuizel,marionette-reviewers,jgilbert https://hg.mozilla.org/integration/autoland/rev/a8ad01797ee0 Run the full slate of test suites on macos-qr. r=jmaher,jrmuizel

Ah, stupid mistake in my updating of the test expectations.

Flags: needinfo?(kats)
Pushed by kgupta@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/5af0cc839a14 Update some test expectations for macOS+WR configuration. r=jrmuizel https://hg.mozilla.org/integration/autoland/rev/2c4c6edf9856 Update marionette mozinfo object with correct webrender status. r=whimboo,marionette-reviewers https://hg.mozilla.org/integration/autoland/rev/129e006c7b76 Disable failing tests. r=jrmuizel,marionette-reviewers,jgilbert https://hg.mozilla.org/integration/autoland/rev/aabd0cfc1ac7 Run the full slate of test suites on macos-qr. r=jmaher,jrmuizel
Regressions: 1675739
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: