We currently run Mochitest Skia (Msk) on mozilla-central https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=2afd8fa9bb5df5577e5566468bb423b76c63cc77&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=autophone and have errors the following errors

snorp: If we aren't able to fix these issues with Msk or if it is not a priority, I'll remove Msk from the non-try production runs so we will not be in a perma-orange state. Let me know how you'd like me to handle these.

Milan, do you think you could get someone to look at these failures?

A bit of background here - these used to run, but started failing, or never ran, or we updated the version of Android and then it started failing, or...? George, can you find out some of the history on these?

Snorp asked for these ages ago where they have been available on try but we never got them in shape to run regularly partly due to perma-orange. In an attempt to find out which tests we could run without failures I added them to fx-team mozilla-central mozilla-beta mozilla-aurora mozilla-release in March but removed them from anything but mozilla-central in April since they were causing load issues due to timeouts/perma-orange. If we don't care about these I intend to remove them from mozilla-central before the end of June and just keep them available on try.

I'm going to assume we care or :snorp wouldn't have tagged us, so, George, let's see what more we can find out about these failures.

For some reason I never knew this existed in the first place. As far as I can tell, it looks like these have been around for nearly 3 years? The original bug is bug 893611. It's just a normal mochitest canvas test run on Android but with skiagl preffed on. What puzzles me is that the prefs used for this are the same as the default on android anyway (https://dxr.mozilla.org/mozilla-central/source/mobile/android/app/mobile.js#819), and we don't have any perma-orange on our normal mochitest runs, so why is this failing? Looking at the log, there are only three actual test failures: 452 INFO TEST-UNEXPECTED-FAIL | dom/canvas/test/test_imagebitmap_cropping.html | pixel 100,100 of is 255,255,255,247; expected 255,255,255,255 +/- 5 207 INFO TEST-UNEXPECTED-FAIL | dom/canvas/test/test_imagebitmap.html | pixel 384,0 of is 4,251,0,255; expected 0,255,0,255 +/- 1 538 INFO TEST-UNEXPECTED-FAIL | dom/canvas/test/test_offscreencanvas_dynamic_fallback.html | after dynamic fallback, screenshots should be the same But we do get an EGL error: 05-13 18:21:23.220 E/libEGL ( 3540): eglMakeCurrent:775 error 3002 (EGL_BAD_ACCESS) 05-13 18:21:23.220 W/Adreno-EGL( 3540): <qeglDrvAPI_eglMakeCurrent:2919>: EGL_BAD_ACCESS This may be harmless though. Bob, are there any differences between our "normal" android mochitests run on m-c, and the autophone-based Mochitest-Skia here? Are the devices themselves different?

gbrown can provide more detail I think, but my understanding is the "normal" android mochitests run on emulators while the autophone tests run on actual devices. If you click on the job symbol, you will see the device name in the left panel. We currently have Nexus 4, Nexus 5, Nexus 6, Nexus 6P, Nexus 9 and a smattering of Nexus S which are being phased out as Android 2.3 support leaves the station. I would guess that the different screen sizes might come into play here.

The "normal" tier-1 mochitests on treeherder run on Android 4.3.1 (slight variant of JLS36I), on the android emulator with a screen size of 800 wide x 1280 high. This bug is about autophone, running on various real devices.

bitmaprenderer uses an ImageLayer instead of a CanvasLayer, which takes a totally different codepath to render images than canvas2d, and result in some anti-aliasing differences at the edge of the image. Both codepaths apply some anti-aliasing (canvas2d in this case has 254,255,254 as the edge pixels, and bitmaprenderer has 240,255,240), so it's not a case of AA vs. non-AA, which I think would be problematic. It seems reasonable to fuzz this as AA differences when using different codepaths isn't something we've traditionally been fussed about.

leave open for now, as there are a couple of other issues that need resolving.

Comment on attachment 8779044 [details] [diff] [review] 0001-Bug-1272878-Fuzz-test_bitmaprenderer.html-because-of.patch Review of attachment 8779044 [details] [diff] [review]: ----------------------------------------------------------------- Great, thanks for fix.

Pushed by gwright@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/e2d1e50ec9f5 Fuzz test_bitmaprenderer.html because of differences in our two codepaths with antialiasing r=Morris

Bob, once this hits central we can probably enable Msk on Android 5 and 6: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9916bcddfd85&exclusion_profile=false

https://github.com/mozilla/autophone/commit/b19e073039d9e50db74f7d065ac5e8b698ea7b94 deployed 2016-08-10 ~00:00 PDT I expect the next mozilla-central build will have the merge. /me crosses fingers.

I messed up and enabled Nexus 5 which of course is still failing. When the current push on m-c finishes, I'll update the manifest to disable it again. We'll be running Msk on Nexus 6P only.

disable Nexus 5 on Msk https://github.com/mozilla/autophone/commit/3a2e6bd5f2261d20e59917b3afbf272cad072870 deployed 2016-08-10 ~08:55:00 PDT.

filed bug 1294491 for the assertion errors.

-> fixed

Removing leave-open keyword from resolved bugs, per :sylvestre.