Syncing wpt PR 27352 found new untriaged test failures in CI

Tests Affected

New Tests That Don't Pass

/css/css-overscroll-behavior/overscroll-behavior.html: ERROR (Chrome: OK, Safari: ERROR)

CI Results

Gecko CI (Treeherder)
GitHub PR Head

Notes

These updates will be on mozilla-central once bug 1689143 lands.

Note: this bug is for tracking fixing the issues and is not
owned by the wpt sync bot.

This bug is linked to the relevant tests by an annotation in
https://github.com/web-platform-tests/wpt-metadata. These annotations
can be edited using the wpt interop dashboard
https://jgraham.github.io/wptdash/

If this bug is split into multiple bugs, please also update the
annotations, otherwise we are unable to track which wpt issues are
already triaged. Resolving as duplicate or closing this issue should
be cause the bot to automatically update or remove the annotation.

Looking at the .ini file, it looks like this test is already known failing since 2018. I suspect we don't implement the relevant functionality.

This bug here seems to be just about some recent additions to the test (being merged in bug 1689143). No big surprises that the new stuff fails just like the existing stuff.

Sorry, disregard comment 2; that was about a different test that happens to have the same name.

wpt.fyi / live links for the actual test here:
https://wpt.fyi/results/css/css-overscroll-behavior/overscroll-behavior.html
https://wpt.live/css/css-overscroll-behavior/overscroll-behavior.html

The test seems to fail in some test-harness code that gets invoked by the test's smoothScrollByXY function:

Uncaught (in promise) Error: unimplemented
    action_sequence https://wpt.live/resources/testdriver.js:685
    action_sequence https://wpt.live/resources/testdriver.js:329
    send https://wpt.live/resources/testdriver-actions.js:99
    smoothScrollByXY https://wpt.live/css/css-overscroll-behavior/overscroll-behavior.html:78
    actions_promise https://wpt.live/css/css-overscroll-behavior/overscroll-behavior.html:186

jgraham says in Matrix that this is due to a testdriver API that he's planning to implement. Once that's in place, we'll be able to actually run the test and see if we pass or not. :)

jgraham, do you know if there's already a bug tracking the testdriver API in question here, which this should depend on?

I filed bug 1746601

With bug 1746601 fixed the test isn't reporting an error anymore but is marked as expected fail:
https://hg.mozilla.org/mozilla-central/diff/22296ec54ef580ea2112ff6e6f1651eb1ab13297/testing/web-platform/meta/css/css-overscroll-behavior/overscroll-behavior.html.ini

Daniel, could you have a look at this test and check why it's failing? Thanks.

If I delete the .ini file and run the test locally, I see the failure:

 0:10.37 TEST_END: Test OK. Subtests passed 0/1. Unexpected 1
FAIL overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: expected 100 but got 0

verify_y_prevented_and_set_boundary_prevents_x/<@http://web-platform.test:8000/css/css-overscroll-behavior/overscroll-behavior.html:105:18
Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:2590:25

verify_y_prevented_and_set_boundary_prevents_x@http://web-platform.test:8000/css/css-overscroll-behavior/overscroll-behavior.html:104:8
@http://web-platform.test:8000/css/css-overscroll-behavior/overscroll-behavior.html:169:42

The logging isn't super-helpful here because the test is passing the assertion description to the wrong function; I filed bug 1790138 on that.

But in any case, it looks like this is the check that's failing:
https://searchfox.org/mozilla-central/rev/31368c7795f44b7a15531d6c5e52dc97f82cf2d5/testing/web-platform/tests/css/css-overscroll-behavior/overscroll-behavior.html#101,104-105

function verify_y_prevented_and_set_boundary_prevents_x() {
...
  test.step(function() {
    assert_equals(root.scrollTop, 100);

It looks like we're applying the dy from the very first entry in test_cases here, but the test is expecting it to get nerfed:

const test_cases = [
  { x: 200, y: 500, dx: -200, dy: -200 },

If you change dy to e.g. -15 there, then the test failure message says "...but got 85"; whereas the test continues to pass in Chrome (where this particular dy value has no effect).

So it looks like this is a case where we're scrolling (by that first test_cases entry's dy value) when the test expects us not to.

As to why that's happening, I don't know; hiro and botond are probably more familiar with overscroll-behavior than I am and might be able to figure out more here.

Without setting test.events.async.enabled pref to true, sendWeelEvent runs though a different code patch that we normally do in APZ. So, that's the cause?

(In reply to Hiroyuki Ikezoe (:hiro) from comment #9)

Without setting test.events.async.enabled pref to true, sendWeelEvent runs though a different code patch that we normally do in APZ. So, that's the cause?

Hm, James would we have to set this pref in Marionette or maybe only for this test? I don't know.

I don't quite remember if that works or if it only works in CI and will cause crashes locally. We can try it of course.

In the long term we probably need to work out a long-term fix for that class of issues (maybe via bug 1773393).

Depends on D177097

Backed out for causing wpt failures in overscroll-behavior.html

• Backout link
• Push with failures
• Failure Log
• Failure line: TEST-UNEXPECTED-FAIL | /css/css-overscroll-behavior/overscroll-behavior.html | overscroll-behavior prevents scroll-propagation in the area and direction as specified - assert_equals: overscroll-behavior-x: none should only prevent scroll propagation on x axis. expected 0 but got 100

The failure is specific to Android.

I tried to reproduce it locally, but wasn't able to get the test to run at all; I filed bug 1837710.

I'm going to enable the test on desktop, and spin out Android to a follow-up issue.

https://hg.mozilla.org/mozilla-central/rev/bc4abcb892be

Note that the patch that landed in this bug does not green up the failure of this test on wpt.fyi. The patch modifies the test's .ini file to set the pref test.events.async.enabled, but the run that collects wpt.fyi results uses default prefs.

Bug 1773393 should make test.events.async.enabled the default (or make the default behaviour similar to what setting that pref would do, in particular passing synthesized events through APZ), which should green up the test on wpt.fyi.