Syncing wpt PR 31930 found new untriaged test failures in CI

Tests Affected

New Tests That Don't Pass

/css/css-scroll-snap/input/snap-area-overflow-boundary.html: ERROR (Chrome: OK, Safari: TIMEOUT)

CI Results

Gecko CI (Treeherder)
GitHub PR Head

Notes

These updates will be on mozilla-central once bug 1744758 lands.

Note: this bug is for tracking fixing the issues and is not
owned by the wpt sync bot.

This bug is linked to the relevant tests by an annotation in
https://github.com/web-platform-tests/wpt-metadata. These annotations
can be edited using the wpt interop dashboard
https://jgraham.github.io/wptdash/

If this bug is split into multiple bugs, please also update the
annotations, otherwise we are unable to track which wpt issues are
already triaged. Resolving as duplicate or closing this issue should
be cause the bot to automatically update or remove the annotation.

I haven't dug into detail, but I assume it's an edge case. I'd hope we could have time for this case when we fix bug 1530253.

wpt.fyi shows all browsers failing this test, though it shows Firefox specifically as erroring out, vs. other browsers as making it through the test and simply producing unexpected scroll positions (I think).

test results at wpt.fyi: https://wpt.fyi/results/css/css-scroll-snap/input/snap-area-overflow-boundary.html
live test at wpt.live: http://wpt.live/css/css-scroll-snap/input/snap-area-overflow-boundary.html

Chrome and Firefox both time out if I load the test directly (e.g. at the wpt.live link in a regular browser).

But both Chrome and Firefox make it through the test if I run the test locally like so...

./mach wpt --product chrome \
  testing/web-platform/tests/css/css-scroll-snap/input/snap-area-overflow-boundary.html

./mach wpt \
  testing/web-platform/tests/css/css-scroll-snap/input/snap-area-overflow-boundary.html

(We have assert_equals mismatches, but we don't error out.)

jgraham, do you know why wpt.fyi is showing "error" as the Firefox result here?

Aha, if I click "Show details" on wpt.fyi, it shows me this additional information for Firefox's error status there:

ERROR message: Traceback (most recent call last):
  File "/home/test/web-platform-tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 835, in run_func
    self.result = True, self.func(self.protocol, self.url, self.timeout)
  File "/home/test/web-platform-tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 958, in do_testharness
    result = protocol.base.execute_script(
  File "/home/test/web-platform-tests/tools/wptrunner/wptrunner/executors/executormarionette.py", line 84, in execute_script
    return method(script, new_sandbox=False, sandbox=None)
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1664, in execute_async_script
    return self._from_json(rv)
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1504, in _from_json
    unwrapped.append(self._from_json(item))
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1512, in _from_json
    unwrapped[key] = self._from_json(value[key])
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1504, in _from_json
    unwrapped.append(self._from_json(item))
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1512, in _from_json
    unwrapped[key] = self._from_json(value[key])
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1504, in _from_json
    unwrapped.append(self._from_json(item))
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1512, in _from_json
    unwrapped[key] = self._from_json(value[key])
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 1510, in _from_json
    return HTMLElement._from_json(value[key], self)
  File "/home/test/web-platform-tests/_venv3/lib/python3.8/site-packages/marionette_driver/marionette.py", line 357, in _from_json
    raise ValueError("Unrecognised web element")
ValueError: Unrecognised web element

So, there's some sort of marionette_driver issue getting encountered there, though it's not an issue that I can reproduce locally (on Linux) fwiw.

I can reproduce locally with a checkout of wpt, but not in m-c. If I copy over the marionette_driver files into the wpt virtualenv (and make sure to stop it overwriting them), the issue no longer reproduces. So I think what's needed here is a new marionette release.

Okay this test is related to this spec issue which hasn't yet been resolved.

We don't do scroll snap on the main-thread (basically, other than Mac), thus to make one of the test cases in snap-area-overflow-boundary.html pass, we need bug 1773393.