The WG hasn't taken a snapshot of it yet, but current trunk (will be 1.0.4) contains the webgl 2 conformance tests, which we need.

Peter, can Morris or somebody else take care of this? Before London if at all possible - failing tests disabled.

(In reply to Milan Sreckovic [:milan] from comment #1) > Peter, can Morris or somebody else take care of this? Before London if at > all possible - failing tests disabled. I will work on this. (In reply to Jeff Gilbert [:jgilbert] from comment #0) > The WG hasn't taken a snapshot of it yet, but current trunk (will be 1.0.4) > contains the webgl 2 conformance tests, which we need. Jeff, do you mean we pick up WebGL2 conformance test but enable mochitest for 1.0.4? [1]https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/webgl-conf/generate-wrappers-and-manifest.py#12

(In reply to Peter Chang[:pchang] from comment #2) > (In reply to Milan Sreckovic [:milan] from comment #1) > > Peter, can Morris or somebody else take care of this? Before London if at > > all possible - failing tests disabled. > I will work on this. > > (In reply to Jeff Gilbert [:jgilbert] from comment #0) > > The WG hasn't taken a snapshot of it yet, but current trunk (will be 1.0.4) > > contains the webgl 2 conformance tests, which we need. > > Jeff, do you mean we pick up WebGL2 conformance test but enable mochitest > for 1.0.4? > > [1]https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/webgl-conf/ > generate-wrappers-and-manifest.py#12 Pick up the whole 1.0.4/2.0 conformance suite, set the 'current version' to 2.0. I'll talk with the WG about versioning this properly.

(In reply to Jeff Gilbert [:jgilbert] from comment #3) > (In reply to Peter Chang[:pchang] from comment #2) > > (In reply to Milan Sreckovic [:milan] from comment #1) > > > Peter, can Morris or somebody else take care of this? Before London if at > > > all possible - failing tests disabled. > > I will work on this. > > > > (In reply to Jeff Gilbert [:jgilbert] from comment #0) > > > The WG hasn't taken a snapshot of it yet, but current trunk (will be 1.0.4) > > > contains the webgl 2 conformance tests, which we need. > > > > Jeff, do you mean we pick up WebGL2 conformance test but enable mochitest > > for 1.0.4? > > > > [1]https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/webgl-conf/ > > generate-wrappers-and-manifest.py#12 > > Pick up the whole 1.0.4/2.0 conformance suite, set the 'current version' to > 2.0. > I'll talk with the WG about versioning this properly. But why not create two folders to save the mochitest for 1.0.4 and 2.0 conformance suites. With the following configuration, we could verify 1.0.4/2.0 conformance suites. webgl-conf -checkout -conformance -generate-wrappers-and-mainfest.py with current version as 1.0.4 -generated-mochitest.ini -mochitest-errata.ini ... -conformance2 -generate-wrappers-and-mainfest.py with current version as 2.0 -generated-mochitest.ini -mochitest-errata.ini ...

There's no real reason to split them as far as I know. We want to run both.

(In reply to Jeff Gilbert [:jgilbert] from comment #5) > There's no real reason to split them as far as I know. We want to run both. If we set current version as 2.0, then some of test cases won't be generated like [1]. This is the reason I want to split them. https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/webgl-conf/checkout/conformance/context/00_test_list.txt#12

(In reply to Peter Chang[:pchang] from comment #6) > (In reply to Jeff Gilbert [:jgilbert] from comment #5) > > There's no real reason to split them as far as I know. We want to run both. > > If we set current version as 2.0, then some of test cases won't be generated > like [1]. This is the reason I want to split them. > https://dxr.mozilla.org/mozilla-central/source/dom/canvas/test/webgl-conf/ > checkout/conformance/context/00_test_list.txt#12 Oh, I'll get these removed.

Since this is blocking, I'll take this for now.

I tried to upload patch with mozreivew, but there was something wrong. (In reply to Peter Chang[:pchang] from comment #13) > Created attachment 8761203 [details] [diff] [review] > Switch to 2.0 conformance suite and remove unnecessary tests (In reply to Peter Chang[:pchang] from comment #14) > Created attachment 8761205 [details] [diff] [review] > Disable WebGL 2.0 conformance suite crashes/failures on Mac Jeff, I just uploaded the changes to pass 2.0 webgl-conf on Mac. I haven't had time to check failures on windows.

These tests may just need to be updated to match their new locations, if they were merely moved. Preliminarily, I removed them.

Comment on attachment 8762235 [details] [diff] [review] 0003-Bug-1277424-Clear-out-now-missing-tests-from-mochite.patch This isn't really necessary except informationally.

Here's my git-cinnabar branch: https://github.com/jdashg/gecko-cinn/commits/webgl-104 It looks like mozreview is not up to the task of changing thousands of files. Fortunately, those patches don't need review. I neglected to comment out the deqp tests, but you've got that covered in your patches. The main thing you want from my branch is the patch that updates the generator. It's up to you if you want to then work from the branch or just cherry-pick that patch.

Wait for try result https://treeherder.mozilla.org/#/jobs?repo=try&revision=32d7596eab41

(In reply to Peter Chang[:pchang] from comment #21) > Wait for try result > > https://treeherder.mozilla.org/#/jobs?repo=try&revision=32d7596eab41 Submit another try after fixing build break. https://treeherder.mozilla.org/#/jobs?repo=try&revision=a177316ada48

New try result after disabling the tests under image_bitmap_from_image_bitmap folder and some timeout cases in different platforms. https://treeherder.mozilla.org/#/jobs?repo=try&revision=74921a5a48a1

(In reply to Peter Chang[:pchang] from comment #23) > New try result after disabling the tests under > image_bitmap_from_image_bitmap folder and some timeout cases in different > platforms. > > https://treeherder.mozilla.org/#/jobs?repo=try&revision=74921a5a48a1 New try after fixing the parsing error, now OSX could run the whole webgl-conf suite whithout crash. Next step is to run the whole set in windows and linux. https://treeherder.mozilla.org/#/jobs?repo=try&revision=db9800b5026d&selectedJob=22857237

This is the latest try result. https://treeherder.mozilla.org/#/jobs?repo=try&revision=6e07240afbdd&selectedJob=22935397 [OSX] After disabling failures, now we only need to handle "called finish() multiple times" error. [Windows] I still got the timed out crash, need to run the whole test to pass first in local [Linux] There are some missing extension to create WebGL2 context and it caused lots of timed out result. 03:47:13 INFO - JavaScript warning: http://mochi.test:8888/tests/dom/canvas/test/webgl-conf/checkout/js/webgl-test-utils.js, line 1437: Error: WebGL: Failed to create WebGL context: WebGL creation failed: WebGL 2 requires support for the following features: 03:47:13 INFO - clear_buffers 03:47:13 INFO - get_integer64_indexed 03:47:13 INFO - gpu_shader4

(In reply to Peter Chang[:pchang] from comment #25) > This is the latest try result. > https://treeherder.mozilla.org/#/ > jobs?repo=try&revision=6e07240afbdd&selectedJob=22935397 > > [OSX] > After disabling failures, now we only need to handle "called finish() > multiple times" error. > > [Windows] > I still got the timed out crash, need to run the whole test to pass first in > local After handling GL_INVALID_OPERATION during CopyTexImage2D, the testing is still running in my windows machine. Will create bug later.

Peter, about the "called finish() multiple times" problem on try server, the webgl conformance test calls finishTest() twice from [1] and [2]. It's because we haven't implement createImageBitmap with ImageBitmapOptions. I think you can disable these tests until we support the interface. [1] https://github.com/KhronosGroup/WebGL/blob/e2281f5d07c8facd1395c72863b471ef433a65bb/sdk/tests/js/tests/tex-image-and-sub-image-with-image-bitmap-utils.js#L210 [2] https://github.com/KhronosGroup/WebGL/blob/be53f1007412830eed32ac12bd449d71679c6839/sdk/tests/js/tests/tex-image-and-sub-image-2d-with-image-bitmap-from-blob.js#L58

(In reply to Ethan Lin[:ethlin] from comment #27) > Peter, about the "called finish() multiple times" problem on try server, the > webgl conformance test calls finishTest() twice from [1] and [2]. It's > because we haven't implement createImageBitmap with ImageBitmapOptions. I > think you can disable these tests until we support the interface. > > [1] > https://github.com/KhronosGroup/WebGL/blob/ > e2281f5d07c8facd1395c72863b471ef433a65bb/sdk/tests/js/tests/tex-image-and- > sub-image-with-image-bitmap-utils.js#L210 > [2] > https://github.com/KhronosGroup/WebGL/blob/ > be53f1007412830eed32ac12bd449d71679c6839/sdk/tests/js/tests/tex-image-and- > sub-image-2d-with-image-bitmap-from-blob.js#L58 Good finding, I will disable the image_bitmap* related test cases first.

With the following try run[1], the debug build of OSX and windows hit the MOZ_GL_DEBUG_ABORT_ON_ERROR . In my patch, I mark failed test cases(more than 50) on OSX as failure by default. Therefore, it is easy to hit this DEBUG_ABORT for failure test cases. If we want to identify which test causes the DEBUG_ABORT, it will waste me lots of time. One way to avoid the debug problem is to skip these failures on debug build and turn it on when test is passed on opt build. Any other suggestion? [1]https://treeherder.mozilla.org/#/jobs?repo=try&revision=175d1d339fdc

(In reply to Peter Chang[:pchang] from comment #29) > With the following try run[1], the debug build of OSX and windows hit the > MOZ_GL_DEBUG_ABORT_ON_ERROR . In my patch, I mark failed test cases(more > than 50) on OSX as failure by default. Therefore, it is easy to hit this > DEBUG_ABORT for failure test cases. If we want to identify which test causes > the DEBUG_ABORT, it will waste me lots of time. > > One way to avoid the debug problem is to skip these failures on debug build > and turn it on when test is passed on opt build. Any other suggestion? > > [1]https://treeherder.mozilla.org/#/jobs?repo=try&revision=175d1d339fdc Works for me. We can go back over these and try to re-enable them later.

Review commit: https://reviewboard.mozilla.org/r/62522/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62522/

Review commit: https://reviewboard.mozilla.org/r/62524/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62524/

Review commit: https://reviewboard.mozilla.org/r/62526/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62526/

Review commit: https://reviewboard.mozilla.org/r/62528/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62528/

Review commit: https://reviewboard.mozilla.org/r/62530/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62530/

Review commit: https://reviewboard.mozilla.org/r/62532/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62532/

Review commit: https://reviewboard.mozilla.org/r/62534/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62534/

Review commit: https://reviewboard.mozilla.org/r/62536/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62536/

Review commit: https://reviewboard.mozilla.org/r/62538/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62538/

Review commit: https://reviewboard.mozilla.org/r/62540/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62540/

Review commit: https://reviewboard.mozilla.org/r/62574/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62574/

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, https://reviewboard.mozilla.org/r/62522/#review59280

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - https://reviewboard.mozilla.org/r/62524/#review59282

Try result with one week ago https://treeherder.mozilla.org/#/jobs?repo=try&revision=29ad40357dad Try result with latest commit https://treeherder.mozilla.org/#/jobs?repo=try&revision=09441ccb4a6b&selectedJob=23397968

These links aren't actually working for me. Can you try pushing again?

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, Review commit: https://reviewboard.mozilla.org/r/62522/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62522/

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - Review commit: https://reviewboard.mozilla.org/r/62524/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62524/

Comment on attachment 8768251 [details] Bug 1277424 - Temporarily disable deqp test cases, Review commit: https://reviewboard.mozilla.org/r/62526/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62526/

Comment on attachment 8768252 [details] Bug 1277424 - Correct the script path of always-fail.html, Review commit: https://reviewboard.mozilla.org/r/62528/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62528/

Comment on attachment 8768253 [details] Bug 1277424 - Disable image_bitmap* test cases because createImageBitmap API is not fully supported, Review commit: https://reviewboard.mozilla.org/r/62530/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62530/

Comment on attachment 8768254 [details] Bug 1277424 - Remove unnecessary test cases in mochitest-errata.ini, Review commit: https://reviewboard.mozilla.org/r/62532/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62532/

Comment on attachment 8768255 [details] Bug 1277424 - Skip test cases that contain Non-local network connections, Review commit: https://reviewboard.mozilla.org/r/62534/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62534/

Comment on attachment 8768256 [details] Bug 1277424 - Disable process-crash test cases on windows, Review commit: https://reviewboard.mozilla.org/r/62536/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62536/

Comment on attachment 8768257 [details] Bug 1277424 - Mark failed test cases on OSX and Windows as failures, Review commit: https://reviewboard.mozilla.org/r/62538/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62538/

Comment on attachment 8768258 [details] Bug 1277424 - Mark failed test cases on Android and Linux as failures, Review commit: https://reviewboard.mozilla.org/r/62540/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62540/

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, Review commit: https://reviewboard.mozilla.org/r/62574/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62574/

I just update wrapper generator to skip webgl2 test cases on Linux and Android. https://treeherder.mozilla.org/#/jobs?repo=try&revision=61e256c5b02b&selectedJob=23469249 There is only one failure on Linux. I'm working on failures disable for Android.

Review commit: https://reviewboard.mozilla.org/r/62810/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/62810/

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62522/diff/1-2/

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62524/diff/1-2/

Comment on attachment 8768251 [details] Bug 1277424 - Temporarily disable deqp test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62526/diff/1-2/

Comment on attachment 8768252 [details] Bug 1277424 - Correct the script path of always-fail.html, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62528/diff/1-2/

Comment on attachment 8768253 [details] Bug 1277424 - Disable image_bitmap* test cases because createImageBitmap API is not fully supported, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62530/diff/1-2/

Comment on attachment 8768254 [details] Bug 1277424 - Remove unnecessary test cases in mochitest-errata.ini, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62532/diff/1-2/

Comment on attachment 8768255 [details] Bug 1277424 - Skip test cases that contain Non-local network connections, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62534/diff/1-2/

Comment on attachment 8768256 [details] Bug 1277424 - Disable process-crash test cases on windows, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62536/diff/1-2/

Comment on attachment 8768257 [details] Bug 1277424 - Mark failed test cases on OSX and Windows as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62538/diff/1-2/

Comment on attachment 8768258 [details] Bug 1277424 - Mark failed test cases on Android and Linux as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62540/diff/1-2/

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62574/diff/1-2/

Wait for latest try result. https://treeherder.mozilla.org/#/jobs?repo=try&revision=45695ff4c6e6

For Windows failures, I guess it was related to nvidia driver on try server. 05:43:53 INFO - [2608] WARNING: NS_ENSURE_TRUE(ParseTypeAttribute(type, &version)) failed: file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/dom/base/nsScriptLoader.cpp, line 1287 05:43:53 INFO - [2608] WARNING: NS_ENSURE_TRUE(ParseTypeAttribute(type, &version)) failed: file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/dom/base/nsScriptLoader.cpp, line 1287 05:43:53 INFO - JavaScript warning: http://mochi.test:8888/tests/dom/canvas/test/webgl-conf/checkout/js/webgl-test-utils.js, line 1437: Error: WebGL: Refused to create native OpenGL context because of blacklist entry: WEBGL_NATIVE_GL_OLD_NVIDIA 05:43:53 INFO - JavaScript warning: http://mochi.test:8888/tests/dom/canvas/test/webgl-conf/checkout/js/webgl-test-utils.js, line 1437: Error: WebGL: Failed to create WebGL context: WebGL creation failed: 05:43:53 INFO - * Refused to create native OpenGL context because of blacklist entry: WEBGL_NATIVE_GL_OLD_NVIDIA 05:43:53 INFO - * Exhausted GL driver options. I will submit another try to address new linux/android failures.

We may want to consider waiting a couple of days before landing this, as it appears the current slew of intermittent webgl test failures came from the 1.0.3 update (see bug 1284152 comment 9.) Or, perhaps, this is the right time to do the update to 1.0.4, then fix the problems.

Comment on attachment 8768252 [details] Bug 1277424 - Correct the script path of always-fail.html, https://reviewboard.mozilla.org/r/62528/#review59860

Comment on attachment 8768253 [details] Bug 1277424 - Disable image_bitmap* test cases because createImageBitmap API is not fully supported, https://reviewboard.mozilla.org/r/62530/#review59862

Comment on attachment 8768251 [details] Bug 1277424 - Temporarily disable deqp test cases, https://reviewboard.mozilla.org/r/62526/#review59864

Comment on attachment 8768254 [details] Bug 1277424 - Remove unnecessary test cases in mochitest-errata.ini, https://reviewboard.mozilla.org/r/62532/#review59866

Comment on attachment 8768255 [details] Bug 1277424 - Skip test cases that contain Non-local network connections, https://reviewboard.mozilla.org/r/62534/#review59868

Comment on attachment 8768256 [details] Bug 1277424 - Disable process-crash test cases on windows, https://reviewboard.mozilla.org/r/62536/#review59870

https://reviewboard.mozilla.org/r/62538/#review59872 ::: dom/canvas/test/webgl-conf/mochitest-errata.ini:222 (Diff revision 2) > +[generated/test_2_conformance__misc__bad-arguments-test.html] > +# skip partial passed test cases > +skip-if = (os == 'mac') || (os == 'win') > +[generated/test_conformance__misc__bad-arguments-test.html] > +# skip partial passed test cases > +skip-if = (os == 'mac') || (os == 'win') Why are these skipped? Please only skip things if they crash. If they just fail, they should be fail-if.

Comment on attachment 8768258 [details] Bug 1277424 - Mark failed test cases on Android and Linux as failures, https://reviewboard.mozilla.org/r/62540/#review59874

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, https://reviewboard.mozilla.org/r/62574/#review59876 You don't need review for running regeneration scripts.

Comment on attachment 8768727 [details] Bug 1277424 - Update wrapper generator to skip WebGL2 testing for linux and Android, https://reviewboard.mozilla.org/r/62810/#review59894 This should work, but let's clean it up like this: https://github.com/jdashg/gecko-cinn/commit/b774541c8c0e9bdb84c25913e0cb0a82c9e2c7ee

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - https://reviewboard.mozilla.org/r/62524/#review59896

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62522/diff/2-3/

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62524/diff/2-3/

Comment on attachment 8768251 [details] Bug 1277424 - Temporarily disable deqp test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62526/diff/2-3/

Comment on attachment 8768252 [details] Bug 1277424 - Correct the script path of always-fail.html, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62528/diff/2-3/

Comment on attachment 8768253 [details] Bug 1277424 - Disable image_bitmap* test cases because createImageBitmap API is not fully supported, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62530/diff/2-3/

Comment on attachment 8768254 [details] Bug 1277424 - Remove unnecessary test cases in mochitest-errata.ini, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62532/diff/2-3/

Comment on attachment 8768255 [details] Bug 1277424 - Skip test cases that contain Non-local network connections, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62534/diff/2-3/

Comment on attachment 8768256 [details] Bug 1277424 - Disable process-crash test cases on windows, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62536/diff/2-3/

Comment on attachment 8768257 [details] Bug 1277424 - Mark failed test cases on OSX and Windows as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62538/diff/2-3/

Comment on attachment 8768258 [details] Bug 1277424 - Mark failed test cases on Android and Linux as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62540/diff/2-3/

Comment on attachment 8768727 [details] Bug 1277424 - Update wrapper generator to skip WebGL2 testing for linux and Android, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62810/diff/1-2/

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62574/diff/2-3/

https://reviewboard.mozilla.org/r/62810/#review59894 Just apply the suggestion

https://reviewboard.mozilla.org/r/62538/#review59872 > Why are these skipped? > Please only skip things if they crash. If they just fail, they should be fail-if. These test cases were found some result logged after simpleTest.Finish(), so skip them first.

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - https://reviewboard.mozilla.org/r/62524/#review60266

Jeff, this is the latest try result. For Mac and Linux, test are all passed. For windows failure, I believe it is related to nvidia driver issue on try machine. For Android, I hit lots of jemalloc crashes which was a known issue, see bug 1285531. https://treeherder.mozilla.org/#/jobs?repo=try&revision=fab516425e59

Comment on attachment 8768257 [details] Bug 1277424 - Mark failed test cases on OSX and Windows as failures, https://reviewboard.mozilla.org/r/62538/#review60504

Comment on attachment 8768727 [details] Bug 1277424 - Update wrapper generator to skip WebGL2 testing for linux and Android, https://reviewboard.mozilla.org/r/62810/#review60506

Review commit: https://reviewboard.mozilla.org/r/63654/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/63654/

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62522/diff/3-4/

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62524/diff/3-4/

Comment on attachment 8768251 [details] Bug 1277424 - Temporarily disable deqp test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62526/diff/3-4/

Comment on attachment 8768252 [details] Bug 1277424 - Correct the script path of always-fail.html, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62528/diff/3-4/

Comment on attachment 8768253 [details] Bug 1277424 - Disable image_bitmap* test cases because createImageBitmap API is not fully supported, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62530/diff/3-4/

Comment on attachment 8768254 [details] Bug 1277424 - Remove unnecessary test cases in mochitest-errata.ini, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62532/diff/3-4/

Comment on attachment 8768255 [details] Bug 1277424 - Skip test cases that contain Non-local network connections, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62534/diff/3-4/

Comment on attachment 8768256 [details] Bug 1277424 - Disable process-crash test cases on windows, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62536/diff/3-4/

Comment on attachment 8768257 [details] Bug 1277424 - Mark failed test cases on OSX and Windows as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62538/diff/3-4/

Comment on attachment 8768258 [details] Bug 1277424 - Mark failed test cases on Android and Linux as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62540/diff/3-4/

Comment on attachment 8768727 [details] Bug 1277424 - Update wrapper generator to skip WebGL2 testing for linux and Android, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62810/diff/2-3/

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62574/diff/3-4/

Comment on attachment 8770067 [details] Bug 1277424 - Enable webgl.force-enabled by default to bypass old driver problem, https://reviewboard.mozilla.org/r/63654/#review60582

After fixing the OLD_NVIDIA driver, I got one assertion on Mac and lots of unexpected failures on windows with latest master. For windows, I saw lots of warnings below and caused test cases were failed on windows opt build but got passed on windows debug build. It seems like related to latency between opt and debug build. 06:26:04 INFO - JavaScript warning: http://mochi.test:8888/tests/dom/canvas/test/webgl-conf/checkout/js/tests/tex-image-and-sub-image-2d-with-canvas.js, line 161: Error: WebGL: texImage2D: Chosen format/type incured an expensive reformat: 0x1908/0x1401 https://treeherder.mozilla.org/#/jobs?repo=try&revision=f867dfa9c8ac&selectedJob=23719312

Jeff, I believe there are several WebGL2 fixes coming recently. How about I pause this work until Friday and re-run again with latest master. Otherwise, I took too much time to trigger try and figure out the problem on different platforms.

(In reply to Peter Chang[:pchang] from comment #117) > Jeff, I believe there are several WebGL2 fixes coming recently. How about I > pause this work until Friday and re-run again with latest master. Otherwise, > I took too much time to trigger try and figure out the problem on different > platforms. Just get the test runs completing. We can then land it and I can help hotfix the failure markings.

Comment on attachment 8768257 [details] Bug 1277424 - Mark failed test cases on OSX and Windows as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62538/diff/4-5/

Comment on attachment 8768258 [details] Bug 1277424 - Mark failed test cases on Android and Linux as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62540/diff/4-5/

Comment on attachment 8768727 [details] Bug 1277424 - Update wrapper generator to skip WebGL2 testing for linux and Android, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62810/diff/3-4/

Comment on attachment 8770067 [details] Bug 1277424 - Enable webgl.force-enabled by default to bypass old driver problem, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/63654/diff/1-2/

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62574/diff/4-5/

The is the latest try result with 7/12 code base and I already address these failures on windows in my latest patch update. https://treeherder.mozilla.org/#/jobs?repo=try&revision=9fe46588441d

Pushed by pchang@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/20091ff168ea Update WebGL conformance based on commit f115726a5a63d7e01694d4568ca7bd4d83fdaa84, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/e8406e4a842d Update wrapper generater to generate webgl2 wrappers. - r=jgilbert https://hg.mozilla.org/integration/autoland/rev/bd240fb194d3 Temporarily disable deqp test cases, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/d476ac6584c7 Correct the script path of always-fail.html, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/c3e353d44cb2 Disable image_bitmap* test cases because createImageBitmap API is not fully supported, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/0e4bd24e7d80 Remove unnecessary test cases in mochitest-errata.ini, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/d47c6533a014 Skip test cases that contain Non-local network connections, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/09754604e0df Disable process-crash test cases on windows, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/00be1eefb359 Mark failed test cases on OSX and Windows as failures, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/052074c53507 Mark failed test cases on Android and Linux as failures, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/d64559a64736 Update wrapper generator to skip WebGL2 testing for linux and Android, r=jgilbert https://hg.mozilla.org/integration/autoland/rev/a9d81704e73f Enable webgl.force-enabled by default to bypass old driver problem, r=mtseng https://hg.mozilla.org/integration/autoland/rev/1e389ee108ad Update generated test cases, r=jgilbert

sorry had to back this out for various failures like here : https://treeherder.mozilla.org/logviewer.html#?job_id=527872&repo=autoland

Looks like I got different result from win7 and win8 opt Submit a try run for this. https://treeherder.mozilla.org/#/jobs?repo=try&revision=92fb4607664a

Backout by cbook@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/a095db872be5 Backed out changeset 1e389ee108ad https://hg.mozilla.org/integration/autoland/rev/9f6e718d6322 Backed out changeset a9d81704e73f https://hg.mozilla.org/integration/autoland/rev/225cd46440c5 Backed out changeset d64559a64736 https://hg.mozilla.org/integration/autoland/rev/e381ec10e87e Backed out changeset 052074c53507 https://hg.mozilla.org/integration/autoland/rev/098798fac264 Backed out changeset 00be1eefb359 https://hg.mozilla.org/integration/autoland/rev/5e4c8dda4bcf Backed out changeset 09754604e0df https://hg.mozilla.org/integration/autoland/rev/837182eb8ef5 Backed out changeset d47c6533a014 https://hg.mozilla.org/integration/autoland/rev/5978a885f6ae Backed out changeset 0e4bd24e7d80 https://hg.mozilla.org/integration/autoland/rev/9cdf6a9b511e Backed out changeset c3e353d44cb2 https://hg.mozilla.org/integration/autoland/rev/0492d55675d5 Backed out changeset d476ac6584c7 https://hg.mozilla.org/integration/autoland/rev/6332a89b0552 Backed out changeset bd240fb194d3 https://hg.mozilla.org/integration/autoland/rev/619d37626900 Backed out changeset e8406e4a842d https://hg.mozilla.org/integration/autoland/rev/151aaa2db94d Backed out changeset 20091ff168ea for various windows gl test failures

(In reply to Peter Chang[:pchang] from comment #127) > Looks like I got different result from win7 and win8 opt > > Submit a try run for this. > > https://treeherder.mozilla.org/#/jobs?repo=try&revision=92fb4607664a I still didn't know why we got failures on win7 but got passed on win8 for some tests. But I'm going to disable WebGL2 for win8 first and the try result looks better. https://treeherder.mozilla.org/#/jobs?repo=try&revision=220fa10792e2

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, About half of this massive diff is whitespace only changes converting files to Windows line endings. Every time a file with Windows line endings is added to the tree, a kitten is tortured. Think of the kittens. Please run dos2unix on the files changed in this commit or re-do the import on a non-Windows machine. Something like this may work: $ hg log -r . -T '{files}' | xargs dos2unix If your changes are uncommitted, you can query for files with Windows line endings: $ hg files 'set:modified() & eol(dos)' See `hg help fileset` for more.

I forgot that filesets operate on the existing revision. So to find all files modified or added by a commit that contain Windows line endings, you can do this: $ hg files -r 20091ff168ea576cc46943c48eac3cd7cd74ce12 'set:(modified() | added()) & eol(dos)' Unfortunately that prints 4398 files. Inverting to "& !eol(dos)" only prints 39 files.

(In reply to Gregory Szorc [:gps] from comment #130) > Comment on attachment 8768249 [details] > Bug 1277424 - Update WebGL conformance based on commit > f115726a5a63d7e01694d4568ca7bd4d83fdaa84, > > About half of this massive diff is whitespace only changes converting files > to Windows line endings. Every time a file with Windows line endings is > added to the tree, a kitten is tortured. Think of the kittens. > > Please run dos2unix on the files changed in this commit or re-do the import > on a non-Windows machine. > > Something like this may work: > > $ hg log -r . -T '{files}' | xargs dos2unix > > If your changes are uncommitted, you can query for files with Windows line > endings: > > $ hg files 'set:modified() & eol(dos)' > > See `hg help fileset` for more. Thanks for the suggestion. I will submit the updated version to address whitespace issue.

Review commit: https://reviewboard.mozilla.org/r/64178/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64178/

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62522/diff/4-5/

Comment on attachment 8768250 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62524/diff/4-5/

Comment on attachment 8768251 [details] Bug 1277424 - Temporarily disable deqp test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62526/diff/4-5/

Comment on attachment 8768252 [details] Bug 1277424 - Correct the script path of always-fail.html, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62528/diff/4-5/

Comment on attachment 8768253 [details] Bug 1277424 - Disable image_bitmap* test cases because createImageBitmap API is not fully supported, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62530/diff/4-5/

Comment on attachment 8768254 [details] Bug 1277424 - Remove unnecessary test cases in mochitest-errata.ini, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62532/diff/4-5/

Comment on attachment 8768255 [details] Bug 1277424 - Skip test cases that contain Non-local network connections, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62534/diff/4-5/

Comment on attachment 8768256 [details] Bug 1277424 - Disable process-crash test cases on windows, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62536/diff/4-5/

Comment on attachment 8768257 [details] Bug 1277424 - Mark failed test cases on OSX and Windows as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62538/diff/5-6/

Comment on attachment 8768258 [details] Bug 1277424 - Mark failed test cases on Android and Linux as failures, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62540/diff/5-6/

Comment on attachment 8768727 [details] Bug 1277424 - Update wrapper generator to skip WebGL2 testing for linux and Android, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62810/diff/4-5/

Comment on attachment 8770067 [details] Bug 1277424 - Enable webgl.force-enabled by default to bypass old driver problem, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/63654/diff/2-3/

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62574/diff/5-6/

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, https://reviewboard.mozilla.org/r/62522/#review61392 Line endings look good. Thanks for fixing. Unfortunately, since this landed once (and got backed out) the bad history and its diff are forever enshrined in history. This is one scenario bug 1286022 will help with. Even then, try to be more careful about line endings next time. (Yes, we really should have hooks on the server to check for these things.) Feel free to file a bug against the hg.mozilla.org component.

I /think/ what happened here is you used Git on Windows. My recollection is Git's default behavior is to use native line endings in the working directory (core.eol=native). So if you check out a text file with Git on Windows it will use CRLF. I'm guessing you checked out the WebGL conformance tests with Git on Windows then copied those files to a Mercurial checkout. Mercurial's default behavior, by contrast, preserves the bytes checked into source control in the checkout: no normalization takes place (although you can opt into this by enabling and configuring the "eol" extension). Feel free to read http://adaptivepatchwork.com/2012/03/01/mind-the-end-of-your-line/ if you want gory details about how Git deals with line endings. https://help.github.com/articles/dealing-with-line-endings/ is also a useful guide. I recommend configuring your Git config on Windows to not munge things to CRLF: unless you are using Notepad as an editor, your tools shouldn't have a problem with this.

Comment on attachment 8768249 [details] Bug 1277424 - Disable tests under conformance2/texture because of random failures on window 7, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62522/diff/5-6/

Comment on attachment 8768259 [details] Bug 1277424 - Update generated test cases, Review request updated; see interdiff: https://reviewboard.mozilla.org/r/62574/diff/6-7/

MozReview-Commit-ID: 2MtFukhR4BU

Review commit: https://reviewboard.mozilla.org/r/64516/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64516/

Review commit: https://reviewboard.mozilla.org/r/64518/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64518/

Review commit: https://reviewboard.mozilla.org/r/64520/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64520/

Review commit: https://reviewboard.mozilla.org/r/64522/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64522/

Review commit: https://reviewboard.mozilla.org/r/64524/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64524/

Review commit: https://reviewboard.mozilla.org/r/64526/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64526/

Review commit: https://reviewboard.mozilla.org/r/64528/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64528/

Review commit: https://reviewboard.mozilla.org/r/64530/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64530/

Review commit: https://reviewboard.mozilla.org/r/64532/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64532/

Review commit: https://reviewboard.mozilla.org/r/64534/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64534/

Review commit: https://reviewboard.mozilla.org/r/64536/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64536/

Review commit: https://reviewboard.mozilla.org/r/64538/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64538/

Review commit: https://reviewboard.mozilla.org/r/64540/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64540/

Review commit: https://reviewboard.mozilla.org/r/64542/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64542/

Review commit: https://reviewboard.mozilla.org/r/64544/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/64544/

I got the problem when using mozreivew to upload updated patch. So I could only re-submit the changes again. Jeff, Morris, sorry for your inconvenience.

Comment on attachment 8771276 [details] Bug 1277424 - Update wrapper generater to generate webgl2 wrappers. - https://reviewboard.mozilla.org/r/64518/#review61542