Now that the wpt suite is running against geckoview and expectation metadata has been updated to account for that platform's current state, there are still several tests/dirs that are frequently intermittently failing in unexpected ways. Frequent enough that it prevents me from moving these tests from tier-3 to tier-2 (where they get shown by default in treeherder and sheriffs actively file bugs about intermittent failures).

I'm going to disable my way to victory on these tests so I can bump them to tier-2 in the next week or so. (Can't do tier-1, we don't have the capacity at the moment to run these per-push on integration branches, so they only run on mozilla-central and try at the moment. Treeherder's Add New Jobs (Search) feature CAN run these jobs on inbound and autoland if you filter for android x86_64 web-platform.)

Setting the bug as leave-open for now because I expect I'll have to play whackamole on the tests over a few rounds of disablings to get things truly green. I'll resolve the bug when I bump the jobs to tier-2.

This disables a few additional tests and updates the annotations for other disabled tests to point to this bug instead of the general geckoview-wpt bug.

https://hg.mozilla.org/mozilla-central/rev/cd771a0b9c07

https://hg.mozilla.org/mozilla-central/rev/c23eb772cf2d

https://hg.mozilla.org/mozilla-central/rev/64bfa1117b35

https://hg.mozilla.org/mozilla-central/rev/e1080b62268c

This patch runs wpt against opt and debug builds of geckoview as tier-2 jobs on mozilla-central and try.

This takes the android wpt jobs that have previously been constrained to running against mozilla-central and try, and will begin also running them on integration branches.

Depends on D31581

https://hg.mozilla.org/mozilla-central/rev/a4c80ac02e50

They're tier-2 now, so I think we can resolve this bug. I might still use this as a meta bug for tracking all of the gv wpt failures that sheriffs file, at least for a while until they get greened up more.

Bug 1553584 was filed and shows a race condition in the test harness that could make fixing bugs difficult. Bumping these back to tier-3 until that can be fixed.

We need to rehide these until bug 1553584 can be fixed.

I think we should have put the reftests down to Tier 3 and left the other tests as Tier 2.

https://hg.mozilla.org/mozilla-central/rev/217038bea1c7

Taskcluster didn't correctly apply the change back to tier-3 in the previous patch because the wildcard setting the tier to 2 by default beat out the specific line setting tier-3 for the geckoview tests. This patch specifically changes the tier for non-geckoview reftests (fennec runs on x86 emulators) to tier-2, while geckoview reftests (TRA runs on x86_64 emulators) to tier-3.

This change also explicitly sets the non-reftest tests to tier-2 for all android platforms, because they didn't have the permafails that the reftests seem to have.

https://hg.mozilla.org/mozilla-central/rev/61e78fd6c946

Now that Bug 1553584 is fixed, we can unhide these tests again.

https://hg.mozilla.org/mozilla-central/rev/59ed9ee0bd98
https://hg.mozilla.org/mozilla-central/rev/253fa3bd2d74

I'm going to update expectation metadata for a few more tests, then resolve this bug.

https://hg.mozilla.org/mozilla-central/rev/7b99cdff2976

These have been Tier-2 for a while so let's close this bug.

A number of tests have been disabled in the past months: I'll organize the work to be done under Bug 1563334.