Most mochitest-webgl chunks are retried several times until they fail.

(In reply to Marco Castelluccio [:marco] from comment #0) > Most mochitest-webgl chunks are retried several times until they fail. Well, the tests don't actually fail, they never start.

Do you know what might be causing this?

most likely bug 1372172

Rob, are you still working on bug 1372172? Any idea why this is happening so often with windows10-64-ccov/debug compared to other builds (e.g. windows10-64/debug)? Take a look at https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-tier=3&filter-tier=2&filter-tier=1&filter-searchStr=win%2010%2064%20debug.

we don't know what causes it yet but when the gpu gets used by a test, the instance often hangs (impaired state in ec2). before i went on pto i thought it might be related to the theme change (windows falling back to classic theme in the middle of some gpu intensive task), from bug 1343049 but the patch there seems to have not had any effect. so yes, now that i'm back from pto i am looking at this again.

Comment on attachment 8934939 [details] [diff] [review] Patch Review of attachment 8934939 [details] [diff] [review]: ----------------------------------------------------------------- thanks for the comment linking to the bug

Pushed by mcastelluccio@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/5d29d8150f97 Disable mochitest-webgl tests as they are retried too often, until the infrastructure problem is fixed. r=jmaher

Pushed by mcastelluccio@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/eb74c83f92c2 Add another space before comment in yaml file to avoid linting failure. r=me

https://hg.mozilla.org/mozilla-central/rev/5d29d8150f97 https://hg.mozilla.org/mozilla-central/rev/eb74c83f92c2

:grenade, mochitest-gpu are affected by the same problem (they never start). Do you think it's bug 1372172? Isn't bug 1372172 intermittent? This problem with mochitest-gl and mochitest-gpu is permanent with coverage builds.

i guess it could be the same issue if they're starting, but we lose contact with the instance before the task start is shown in treeherder. if you can link to a treeherder job where the tests should have started, but didn't, i can investigate.

(In reply to Rob Thijssen (:grenade UTC+2) from comment #12) > i guess it could be the same issue if they're starting, but we lose contact > with the instance before the task start is shown in treeherder. if you can > link to a treeherder job where the tests should have started, but didn't, i > can investigate. Here it is: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=d75d996016dcf325c2db2ed8a47af512d07ffacd&filter-searchStr=win%20ccov%20gpu.

I've tried another try push today for mochitest-webgl, still failing: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c57c6fe3ba01f2e3e62b34d6d02ca77c573fe2ac.

The suites are passing now that bug 1372172 has been fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d94fa5d9d9e6cbd707bbffb5d960e7d92a72ad4e.

Comment on attachment 9027519 [details] [diff] [review] Re-enable test suites Review of attachment 9027519 [details] [diff] [review]: ----------------------------------------------------------------- great, I am excited to see all the jobs that failed on win10 getting enabled now:)

Pushed by mcastelluccio@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/102c7135d952 Re-enable some mochitest suites using virtual-with-gpu that were failing with Windows ccov builds. r=jmaher