Upgrade the WebGL conformance tests being run to 1.0.3, since we are currently a bit far from total conformance and don't want to regress.

WIP

Tests are running and seem to be reporting errors properly on try. Trying to compile an errata that doesn't crash/fail for Linux opt builds.

Comment on attachment 8650546 [details] [diff] [review] 001-nuke-dir.patch Review of attachment 8650546 [details] [diff] [review]: ----------------------------------------------------------------- This looks right.

Pathname of a certain test file is too long for the Windows machines to handle, so it was shortened with this patch.

This patch omits tests that crash while running.

This patch modifies the test manifests to skip the entire ogles section of tests since they seem to randomly cause later tests to fail. However, the tests will still crash on Linux 64 bit debug builds seemingly randomly: https://treeherder.mozilla.org/#/jobs?repo=try&revision=414cd8d1c104 Since the 1.0.1 tests don't seem to have a ogles section, I think it's okay to omit them for now. The random crashes in Linux 64 seem to be caused by certain tests in the glsl section of tests since the tests don't crash when both the ogles and glsl tests are disabled: https://treeherder.mozilla.org/#/jobs?repo=try&revision=927db8e97b8f Seeing as the glsl tests are present in the 1.0.1 version of the suite, we probably shouldn't omit them and find the subset of glsl tests that cause the crashes.

(In reply to Kyle Fung from comment #9) > > Seeing as the glsl tests are present in the 1.0.1 version of the suite, we > probably shouldn't omit them and find the subset of glsl tests that cause > the crashes. To make things easy we could just omit the new glsl tests for now.

Here's an attempt at that: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3bbe1ce3c8e6

(In reply to Kyle Fung from comment #9) > The random crashes in Linux 64 seem to be caused by certain tests in the > glsl section of tests since the tests don't crash when both the ogles and > glsl tests are disabled: > https://treeherder.mozilla.org/#/jobs?repo=try&revision=927db8e97b8f I retriggered the job a couple of times and got a crash in the driver.

Is the crash intermittence of your push too frequent to be acceptable? I've been observing frequencies of around that as well, but I figured that they were infrequent enough to be ignored.

(In reply to Kyle Fung from comment #13) > Is the crash intermittence of your push too frequent to be acceptable? > I've been observing frequencies of around that as well, but I figured that > they were infrequent enough to be ignored. We don't want to knowingly introduce an intermittent. I don't think the current tests crash intermittently, so this warrants further investigation.

I'm going to take this bug back. We're basically ready-to-go here with the exception of the Mesa software driver update.

This doesn't require passing the tests. I have created bug 1246063 for tracking general 1.0.3 failures.

Dropping the alias, and see-also'ing bugs for both 103 failures and 103 conformance.

It's at the point where we're just needing to nail down the remaining failures, particularly the troublesome Android frequent oranges: https://treeherder.mozilla.org/#/jobs?repo=try&revision=244b3c957fd0 Here's my WIP branch: https://github.com/jdashg/gecko-dev/tree/test-103-2

Peter, this is our next step in the WebGL2 work; anyone available on your side?

(In reply to Milan Sreckovic [:milan] from comment #20) > Peter, this is our next step in the WebGL2 work; anyone available on your > side? Ethan will be the main owner for this. And it would be great to use etherpad(or bugzilla) to track these failures and others can know which one they can work on if they are available.

I can handle this.

Review commit: https://reviewboard.mozilla.org/r/48367/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48367/

The folder name "webgl-conformance" is too long. Rename it to "webgl-conf". Review commit: https://reviewboard.mozilla.org/r/48369/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48369/

Review commit: https://reviewboard.mozilla.org/r/48371/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48371/

Review commit: https://reviewboard.mozilla.org/r/48373/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48373/

Review commit: https://reviewboard.mozilla.org/r/48375/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48375/

Review commit: https://reviewboard.mozilla.org/r/48377/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48377/

The name of some tests is too long and exceed maximum length of windows path. So we rename it to shorter name. Review commit: https://reviewboard.mozilla.org/r/48379/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48379/

Review commit: https://reviewboard.mozilla.org/r/48381/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48381/

Review commit: https://reviewboard.mozilla.org/r/48383/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48383/

Review commit: https://reviewboard.mozilla.org/r/48385/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48385/

Review commit: https://reviewboard.mozilla.org/r/48387/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48387/

Review commit: https://reviewboard.mozilla.org/r/48389/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/48389/

Comment on attachment 8744182 [details] MozReview Request: Bug 1193526 - Remove old conformance tests and generated files. r=jgilbert https://reviewboard.mozilla.org/r/48367/#review45291

Comment on attachment 8744183 [details] MozReview Request: Bug 1193526 - Update conformance path and remove unused file. r=jgilbert https://reviewboard.mozilla.org/r/48369/#review45293

Comment on attachment 8744184 [details] MozReview Request: Bug 1193526 - Update all generator files for webgl conformance test 1.0.3. r=jgilbert https://reviewboard.mozilla.org/r/48371/#review45295

Comment on attachment 8744185 [details] MozReview Request: Bug 1193526 - Update mochitest paths. r=jgilbert https://reviewboard.mozilla.org/r/48373/#review45297

Comment on attachment 8744186 [details] MozReview Request: Bug 1193526 - Update tests to 1.0.3 from a8ffce2eb6906f3b910399de9ce82dc5538df95b. r=jgilbert https://reviewboard.mozilla.org/r/48375/#review45299

Comment on attachment 8744188 [details] MozReview Request: Bug 1193526 - Handle video failing to load. r=jgilbert https://reviewboard.mozilla.org/r/48379/#review45301

Comment on attachment 8744193 [details] MozReview Request: Bug 1193526 - Add generated files. r=jgilbert https://reviewboard.mozilla.org/r/48389/#review45305

Comment on attachment 8744191 [details] MozReview Request: Bug 1193526 - Update errata to mark all failures in 1.0.3 webgl conformance tests. r=jgilbert https://reviewboard.mozilla.org/r/48385/#review45307

Comment on attachment 8744190 [details] MozReview Request: Bug 1193526 - Handle image.src being invalid. r=jgilbert https://reviewboard.mozilla.org/r/48383/#review45309

Comment on attachment 8744189 [details] MozReview Request: Bug 1193526 - Handle image failing to load. r=jgilbert https://reviewboard.mozilla.org/r/48381/#review45311

Comment on attachment 8744192 [details] MozReview Request: Bug 1193526 - Add generated files. r=jgilbert https://reviewboard.mozilla.org/r/48387/#review45303 ::: dom/canvas/test/webgl-mochitest/mochitest.ini:86 (Diff revision 1) > [test_webgl_conformance.html] > skip-if = toolkit == 'android' #bug 865443- seperate suite - the non_conf* tests pass except for one on armv6 tests > [test_webgl_compressed_texture_es3.html] > [test_webgl_disjoint_timer_query.html] > [test_webgl_force_enable.html] > +skip-if = (os == 'android') Woah, why is this here? We really want to keep this!

Comment on attachment 8744187 [details] MozReview Request: Bug 1193526 - Rename built_in_varying_array_out_of_bounds to biv_array_oob. r=jgilbert https://reviewboard.mozilla.org/r/48377/#review45313 It looks like this reverts the file. We shouldn't revert this file. We should try to work with the file as-is, and only make minor changes (and upstream them) if we need to.

https://reviewboard.mozilla.org/r/48377/#review45315 ::: dom/canvas/test/webgl-conf/checkout/resources/webgl-test-harness.js:1 (Diff revision 1) > -/* > +// Copyright (c) 2009 The Chromium Authors. All rights reserved. Let's try to use the current version of this file.

Comment on attachment 8744182 [details] MozReview Request: Bug 1193526 - Remove old conformance tests and generated files. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48367/diff/1-2/

Comment on attachment 8744183 [details] MozReview Request: Bug 1193526 - Update conformance path and remove unused file. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48369/diff/1-2/

Comment on attachment 8744184 [details] MozReview Request: Bug 1193526 - Update all generator files for webgl conformance test 1.0.3. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48371/diff/1-2/

Comment on attachment 8744185 [details] MozReview Request: Bug 1193526 - Update mochitest paths. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48373/diff/1-2/

Comment on attachment 8744186 [details] MozReview Request: Bug 1193526 - Update tests to 1.0.3 from a8ffce2eb6906f3b910399de9ce82dc5538df95b. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48375/diff/1-2/

Comment on attachment 8744187 [details] MozReview Request: Bug 1193526 - Rename built_in_varying_array_out_of_bounds to biv_array_oob. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48377/diff/1-2/

Comment on attachment 8744188 [details] MozReview Request: Bug 1193526 - Handle video failing to load. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48379/diff/1-2/

Comment on attachment 8744189 [details] MozReview Request: Bug 1193526 - Handle image failing to load. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48381/diff/1-2/

Comment on attachment 8744190 [details] MozReview Request: Bug 1193526 - Handle image.src being invalid. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48383/diff/1-2/

Comment on attachment 8744191 [details] MozReview Request: Bug 1193526 - Update errata to mark all failures in 1.0.3 webgl conformance tests. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48385/diff/1-2/

Comment on attachment 8744192 [details] MozReview Request: Bug 1193526 - Add generated files. r=jgilbert Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48387/diff/1-2/

Because I removed one part of patch, review board got confused......

Comment on attachment 8744187 [details] MozReview Request: Bug 1193526 - Rename built_in_varying_array_out_of_bounds to biv_array_oob. r=jgilbert This part should be r+. jgilbert, can you set r+ in review board?

Comment on attachment 8744192 [details] MozReview Request: Bug 1193526 - Add generated files. r=jgilbert This part should be r+ as well...

Comment on attachment 8744191 [details] MozReview Request: Bug 1193526 - Update errata to mark all failures in 1.0.3 webgl conformance tests. r=jgilbert This is the exactly one should be reviewed.

Comment on attachment 8744187 [details] MozReview Request: Bug 1193526 - Rename built_in_varying_array_out_of_bounds to biv_array_oob. r=jgilbert https://reviewboard.mozilla.org/r/48377/#review45921

Comment on attachment 8744192 [details] MozReview Request: Bug 1193526 - Add generated files. r=jgilbert https://reviewboard.mozilla.org/r/48387/#review45923

https://hg.mozilla.org/integration/mozilla-inbound/rev/6423d55647ba https://hg.mozilla.org/integration/mozilla-inbound/rev/ff47b156807b https://hg.mozilla.org/integration/mozilla-inbound/rev/6ccf5bdf89f4 https://hg.mozilla.org/integration/mozilla-inbound/rev/df8471a4f59f https://hg.mozilla.org/integration/mozilla-inbound/rev/730c8d4b4e38 https://hg.mozilla.org/integration/mozilla-inbound/rev/f73ccd27684a https://hg.mozilla.org/integration/mozilla-inbound/rev/e510a704c1b0 https://hg.mozilla.org/integration/mozilla-inbound/rev/4688537106ce https://hg.mozilla.org/integration/mozilla-inbound/rev/34836f8c1b9e https://hg.mozilla.org/integration/mozilla-inbound/rev/9068b20e29a0 https://hg.mozilla.org/integration/mozilla-inbound/rev/ce229162e7b3

https://hg.mozilla.org/mozilla-central/rev/6423d55647ba https://hg.mozilla.org/mozilla-central/rev/ff47b156807b https://hg.mozilla.org/mozilla-central/rev/6ccf5bdf89f4 https://hg.mozilla.org/mozilla-central/rev/df8471a4f59f https://hg.mozilla.org/mozilla-central/rev/730c8d4b4e38 https://hg.mozilla.org/mozilla-central/rev/f73ccd27684a https://hg.mozilla.org/mozilla-central/rev/e510a704c1b0 https://hg.mozilla.org/mozilla-central/rev/4688537106ce https://hg.mozilla.org/mozilla-central/rev/34836f8c1b9e https://hg.mozilla.org/mozilla-central/rev/9068b20e29a0 https://hg.mozilla.org/mozilla-central/rev/ce229162e7b3

Was re-disabling the webgl conformance suite on e10s an intended change from this?

(In reply to Ryan VanderMeulen [:RyanVM] from comment #70) > Was re-disabling the webgl conformance suite on e10s an intended change from > this? No, mochitest-errata is updated with correct setting. Maybe I forgot to re-generate the generted-mochitest.html with updated mochitest-errata.