In bug 775227, mochitest content/canvas/test/webgl/test_webgl_conformance_test_suite.html has been disabled on android (in android.json) on suspicion of causing out of memory errors on the Tegras. Additional investigation is required to verify this test's memory use and determine if it can be re-enabled on Android.

I would have appreciated being informed of this before the whole WebGL test suite got disabled. The WebGL mochitest consists of 300+ separate test pages that can be individually disabled. These are the 00_test_list.txt files. There was probably no need to disable the entire test suite. Now WebGL isn't tested at all on Android while heavy changes are underway, so we are likely to have regressions and not catch them.

This is games:p1, webgl-conformance, definitely blocking kilimanjaro and basecamp, and in fact, I don't think we can have a Firefox/Android release until this is fixed.

(In reply to Benoit Jacob [:bjacob] from comment #1) > I would have appreciated being informed of this before the whole WebGL test > suite got disabled. Sorry about that. We are getting rather desperate to turn android mochitests green -- failure rates are so high that most are hidden now anyway. I recognize this was an aggressive change and I'll do what I can to narrow down the issue and get tests re-enabled asap. In the meantime, any help/insight would be appreciated.

I'd suggest logging GC and CC. Not-frequent-enough-GC-and-CC is a common reason for OOM conditions in tests. In the case of the WebGL mochitest, this is less likely to happen as the WebGL mochitest explicitly requests GC and CC before every of its 300+ pages. Still this is only a request/hint and might not be observed. In bug 617453 I'm working on an orthogonal solution to keep the number of live WebGL contexts under a hard limit regardless of GC/CC.

(In reply to Benoit Jacob [:bjacob] from comment #4) > In the case of the WebGL mochitest, this is less likely to happen as the > WebGL mochitest explicitly requests GC and CC before every of its 300+ > pages. Still this is only a request/hint and might not be observed. I think the ways to force a GC from JS all immediately do a GC, so I don't think there's too much danger of it not running. These could be switched to memory pressure events if they aren't already, because those aggressively drop GL contexts. johns may have some ideas on approaches to mitigate this from his experience rigging up AreWeSlimYet.com which opens a bunch of pages and measures memory use.

Confirmed locally that the number of live WebGL contexts while running the mochitest does go up (hit 9 contexts at least) i.e. the requesting GC/CC is not always effective. So this is likely to benefit from 617453.

(In reply to Andrew McCreight [:mccr8] from comment #5) > (In reply to Benoit Jacob [:bjacob] from comment #4) > > In the case of the WebGL mochitest, this is less likely to happen as the > > WebGL mochitest explicitly requests GC and CC before every of its 300+ > > pages. Still this is only a request/hint and might not be observed. > > I think the ways to force a GC from JS all immediately do a GC, so I don't > think there's too much danger of it not running. Like I noted in Comment 6, even though we do call SpecialPowers.DOMWindowUtils.garbageCollect(); before each test page and each test page typically created only 1 WebGL context, I quickly hit 9 WebGL contexts on linux x86-64, so it seems that SpecialPowers.DOMWindowUtils.garbageCollect(); was just a hint (it does make a difference though --- it's worse without it). > These could be switched to > memory pressure events if they aren't already, because those aggressively > drop GL contexts. The solution I'm working on in bug 617453 is akin to this but will cause only the oldest-used WebGL context to be lost when the limit is exceeded, whereas memory-pressure would lose all WebGL contexts.

For what it's worth, this is the code AWSY is using to try to ensure that everything is GC'd -- basically hammering CC and GC five times in a loop, returning to the event loop: https://github.com/Nephyrin/MozAreWeSlimYet/blob/master/mozmill_endurance_test/endurance.js#L116 This is based on what about:memory does, and is due to some objects requiring multiple GC -> CC cycles to fully clean up everything (I think there's a bug on that somewhere)

Eek. My takeaway is that explicitly triggering GC isn't really support. Fortunately the patch pending review in bug 617453 takes an approach that's orthogonal to GC.

Now that bug 617453 has landed, seems to be a good time to re-enable this mochitest. In theory, it should never result in more than 2 WebGL contexts being live at the same time, anymore.

I have not been able to reproduce OOM problems since bug 617453 landed. Try run here looks very good: https://tbpl.mozilla.org/?tree=Try&rev=6026067b0d4f

That try run doesn't have mochitest runs, according to TBPL. Seems to be a general issue with Android at the moment.

(In reply to Benoit Jacob [:bjacob] from comment #13) > That try run doesn't have mochitest runs, according to TBPL. Seems to be a > general issue with Android at the moment. They are hidden (bug 777436 and dependents), you need to use the "Show all Android Tests" link in the TBPL status message on Try (or just add &noignore=1 to the Try link in comment 12).

(In reply to Ed Morley [:edmorley] from comment #15) > They are hidden To clarify: By they, I meant just M1 and M3, out of the mochitests.

Ah! thanks.

Clearing blocker flags.