Hello all,

I went to

https://foolip.github.io/ad-hoc-wpt-results-analysis/firefox-lone-failures.html

with Firefox 66.0a1 buildID=20190109214248. The following (randomly chosen) tests

http://w3c-test.org/css/css-backgrounds/background-334.html

http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-clipping-color-3.html

http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-positioning-2.html

http://w3c-test.org/css/css-writing-modes/mongolian-orientation-002.html

are listed as FAILed solely by Firefox but that is just NOT TRUE: Firefox 66 PASSes those 4 tests.

I double-checked with their respective reference files and I see no problem:

http://w3c-test.org/css/css-backgrounds/reference/background-334-ref.xht

http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-clipping-color-3-ref.html

http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-positioning-2-ref.html

http://w3c-test.org/css/css-writing-modes/reference/mongolian-orientation-001-ref.html

http://w3c-test.org/css/CSS2/backgrounds/background-bg-pos-206.xht

is passed by Firefox 66 even though firefox-lone-failures.html and wpt lists it as failed by Firefox 66 on Jan 3, 2019.

I just noticed this bug report is for CSS-grid only ... nevertheless, I think there must be a problem somewhere with some of the test results found to be failed by Firefox!

Gerard,

All of these tests are disabled in web platform tests[1] for Firefox so there appears to be some issue. Could you help diagnose what that might be?

[1] https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/CSS2/backgrounds

Could you help diagnose what that might be?

Okay... I think I found why they have been reported as failures...

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-backgrounds/background-334.html.ini
states
[background-334.html]
disabled: antialiasing

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-backgrounds/background-attachment-local/attachment-local-clipping-color-3.html.ini
states
[attachment-local-clipping-color-3.html]
expected:
if debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-backgrounds/background-attachment-local/attachment-local-positioning-2.html.ini
states
expected:
if debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/css-writing-modes/mongolian-orientation-002.html.ini
states
expected:
if os == "android": PASS
FAIL

https://searchfox.org/mozilla-central/source/testing/web-platform/meta/css/CSS2/backgrounds/background-bg-pos-206.xht.ini
states
expected:
if debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "6.1.7601") and (processor == "x86") and (bits == 32): FAIL
if debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if not debug and not webrender and e10s and (os == "win") and (version == "10.0.15063") and (processor == "x86_64") and (bits == 64): FAIL
if os == "android": FAIL

I do not use Android and I do not use Windows. Under Linux, I see no problem with those tests.

attachment-local-clipping-color-3.html appears to be slightly different from the one submitted over at wpt:

https://dxr.mozilla.org/mozilla-central/source/layout/reftests/backgrounds/attachment-local-clipping-color-3.html
uses a 32px blue square
while the one submitted
http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-clipping-color-3.html
uses "Text" ... but that may be unrelated.

The aqua-yellow-32x32.png image in

http://test.csswg.org/suites/css-backgrounds-3_dev/nightly-unstable/html4/attachment-local-positioning-2.htm

is not fetched and is 404 Not Found. But it is fetched and loaded in

http://w3c-test.org/css/css-backgrounds/background-attachment-local/attachment-local-positioning-2.html

I'll ask Peter Linss about this...

THe updated failing tests for css grid are

/css/css-grid/abspos/grid-positioned-items-implicit-grid-line-001.html [2 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-001.html [3 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-002.html [3 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-003.html [5 subtests]
▶ /css/css-grid/alignment/grid-alignment-style-changes-004.html [5 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-001.html [4 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-002.html [3 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-003.html [3 subtests]
▶ /css/css-grid/alignment/grid-column-axis-self-baseline-synthesized-004.html [3 subtests]
▶ /css/css-grid/alignment/grid-container-baseline-001.html [7 subtests]
▶ /css/css-grid/alignment/grid-content-alignment-second-pass-001.html [6 subtests]
▶ /css/css-grid/alignment/grid-content-alignment-second-pass-002.html [8 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-001.html [4 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-002.html [4 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-003.html [4 subtests]
▶ /css/css-grid/alignment/grid-row-axis-self-baseline-synthesized-004.html [4 subtests]
▶ /css/css-grid/alignment/grid-self-alignment-non-static-positioned-items-008.html [3 subtests]
▶ /css/css-grid/alignment/grid-self-alignment-positioned-items-with-margin-border-padding-016.html [1 subtests]
▶ /css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-001.html [3 subtests]
▶ /css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-002.html [24 subtests]
▶ /css/css-grid/alignment/grid-self-baseline-not-applied-if-sizing-cyclic-dependency-003.html [6 subtests]
▶ /css/css-grid/grid-definition/grid-auto-repeat-max-size-001.html [2 subtests]
▶ /css/css-grid/grid-definition/grid-auto-repeat-min-size-001.html [4 subtests]
▶ /css/css-grid/grid-definition/grid-change-fit-content-argument-001.html [2 subtests]
▶ /css/css-grid/grid-definition/grid-inline-template-columns-rows-resolved-values-001.html [15 subtests]
▶ /css/css-grid/grid-definition/grid-percentage-rows-indefinite-height-001.html [104 subtests]
▶ /css/css-grid/grid-definition/grid-percentage-rows-indefinite-height-002.html [4 subtests]
▶ /css/css-grid/grid-definition/grid-template-columns-rows-resolved-values-001.html [15 subtests]
▶ /css/css-grid/grid-layout-properties.html [5 subtests]
▶ /css/css-grid/grid-model/grid-container-ignores-first-letter-001.html [4 subtests]
▶ /css/css-grid/grid-model/grid-container-ignores-first-line-001.html [4 subtests]
▶ /css/css-grid/grid-model/grid-first-letter-003.html [1 subtests]
▶ /css/css-grid/grid-model/grid-first-line-003.html [1 subtests]
▶ /css/css-grid/grid-model/grid-inline-first-letter-003.html [1 subtests]
▶ /css/css-grid/grid-model/grid-inline-first-line-003.html [1 subtests]
▶ /css/css-grid/implicit-grids/grid-support-grid-auto-columns-rows-002.html [1 subtests]
▶ /css/css-grid/inheritance.html [20 subtests]
▶ /css/css-grid/layout-algorithm/grid-content-distribution-must-account-for-track-sizing-002.html [1 subtests]
▶ /css/css-grid/layout-algorithm/grid-intrinsic-size-with-orthogonal-items.html [7 subtests]
▶ /css/css-grid/layout-algorithm/grid-minimum-contribution-baseline-shim-vertical-lr.html [2 subtests]
▶ /css/css-grid/layout-algorithm/grid-minimum-contribution-baseline-shim-vertical-rl.html [3 subtests]
▶ /css/css-grid/layout-algorithm/grid-minimum-contribution-baseline-shim.html [2 subtests]
▶ /css/css-grid/layout-algorithm/grid-percent-rows-filled-shrinkwrap-001.html [1 subtests]
▶ /css/css-grid/layout-algorithm/grid-percent-rows-spanned-shrinkwrap-001.html [1 subtests]
▶ /css/css-grid/parsing/grid-area-computed.html [25 subtests]
▶ /css/css-grid/parsing/grid-auto-columns-computed.html [25 subtests]
▶ /css/css-grid/parsing/grid-auto-columns-valid.html [2 subtests]
▶ /css/css-grid/parsing/grid-auto-flow-computed.html [4 subtests]
▶ /css/css-grid/parsing/grid-auto-rows-computed.html [25 subtests]
▶ /css/css-grid/parsing/grid-auto-rows-valid.html [2 subtests]
▶ /css/css-grid/parsing/grid-template-areas-computed.html [9 subtests]
▶ /css/css-grid/parsing/grid-template-areas-valid.html [3 subtests]

There is a lot more failing tests for this since the original issue

All the parsing and inheritance ones are bogus per https://github.com/web-platform-tests/wpt/pull/16892.

Just notice there are some shorthand and getComputedStyle() issues on this Firefox-only wpt failure:

css/css-grid/parsing/grid-area-computed.html [13 subtests]

and the spec issues are 1033 and 2529.

I'll turn this into a meta bug since many of the test failures here seems to be from bogus tests.
Please file any real bugs separately and mark it as blocking this bug. Thanks.

I don't think this metabug serves a useful purpose at this point; it looks like we fixed some of these, and for remaining failures, we can track them as-needed on a case by case basis (and comment 0's analysis site https://foolip.github.io/ad-hoc-wpt-results-analysis/firefox-lone-failures.html doesn't load anymore).