Filed by: philringnalda@gmail.com https://treeherder.mozilla.org/logviewer.html#?job_id=100406&repo=autoland https://queue.taskcluster.net/v1/task/endpuP-XRbG8weKtfTUUTA/runs/0/artifacts/public%2Flogs%2Flive_backing.log

I suspect this is very similar to bug 1284075.

Milan, is anyone working on these webgl intermittents? Do we have a bug we could link as a dependency here and elsewhere?

Yes, jgilbert is looking at what's going on, although not necessarily tracking it in this bug. We'll probably create a meta bug (or few) to track this if it turns out to be different problems. We have had a recent ANGLE update (bug 1281687), which landed on June 28th, so that could be related. Some of the failures are Android, so that reduces the chance of it being ANGLE, but we'll find out.

:philor, can you tell when these first started showing up?

This pulls up the ones from the recent activities, but also some older ones: https://bugzilla.mozilla.org/buglist.cgi?short_desc=Intermittent%20webgl-%20crashed&short_desc_type=allwordssubstr&resolution=---&list_id=13104436&query_format=advanced Bug 1235138 is from last year, and looks remarkably similar to the bugs from this batch.

It's reasonable to believe bug 1268747 when it says that the current crisis started with the conformance test update from bug 1193526 at the end of April.

milan: do we have any eta/strategy with fixing the jemalloc bugs/crashes or back out/update the test update that philor mentioned in comment #9 ?

Yes, we were galloping forward - instead of backing out the update to 1.0.3 tests, we updated to 1.0.4 tests in bug 1277424 (just landed this week.) Now we can see about these intermittents or start disabling tests. Peter, Jeff, how do you think we should approach these intermittent errors? I know Vincent is looking at some aspects of this in bug 1268747, but there is too much noise with all these failures.

We should be trapping these in RR and reversing-debugging our way to victory.

I'll note here that historically, we found it wasn't obvious which test to disable. That is, disabling the crashing test moves the crash elsewhere.

(In reply to Milan Sreckovic [:milan] from comment #23) > Yes, we were galloping forward - instead of backing out the update to 1.0.3 > tests, we updated to 1.0.4 tests in bug 1277424 (just landed this week.) > Now we can see about these intermittents or start disabling tests. > > Peter, Jeff, how do you think we should approach these intermittent errors? > I know Vincent is looking at some aspects of this in bug 1268747, but there > is too much noise with all these failures. When upgrading webgl conformance 2.0, I hit lots of jemalloc with Android and I think it would be eariser to reproduce the failures now. Vincent is working on the reproduce environment and should be able to give an update before end of July.

Blanket P3 on WebGL intermittent-failure bugs.