I am trying to enable a new Android test environment, based on the Android x86 emulator running Android 4.2. In this new test environment, there are intermittent crashes and intermittent failures in many of the content/canvas/test/reftest/webgl-* tests, especially the webgl-clear-test and webgl-color-test variants. Many of these tests are already unreliable on B2G and Tegras: random-if(B2G) random-if(Android&&AndroidVersion<15) My default solution for getting the reftests green would change these to: skip-if(Android&&AndroidVersion>=17) random-if(B2G) random-if(Android&&AndroidVersion<15) Alternately, can someone investigate the crashes soon-ish? One example: https://tbpl.mozilla.org/php/getParsedLog.php?id=30805403&tree=Cedar&full=1 19:24:34 INFO - REFTEST TEST-END | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?__&_____&________ 19:24:34 INFO - REFTEST TEST-START | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?aa&_____&________ 19:24:34 INFO - RESTORE PREFERENCE pref(webgl.force-enabled,false) 19:24:34 INFO - SET PREFERENCE pref(webgl.force-enabled,true) 19:24:34 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?aa&_____&________ | 1084 / 3781 (28%) 19:24:34 INFO - RESTORE PREFERENCE pref(webgl.force-enabled,false) 19:24:34 INFO - SET PREFERENCE pref(webgl.force-enabled,true) 19:24:34 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8856/tests/content/canvas/test/reftest/wrapper.html?green.png | 1084 / 3781 (28%) 19:24:34 INFO - INFO | automation.py | Application ran for: 0:06:02.801106 19:24:34 INFO - INFO | zombiecheck | Reading PID log: /tmp/tmpM5rIsUpidlog 19:24:34 INFO - mozcrash INFO | Downloading symbols from: http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/cedar-android-x86/1384909322/fennec-28.0a1.en-US.android-i386.crashreporter-symbols.zip 19:24:34 INFO - /data/anr/traces.txt not found 19:24:34 INFO - PROCESS-CRASH | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?aa&_____&________ | application crashed [@ jemalloc_crash] 19:24:34 INFO - Crash dump filename: /tmp/tmpe5DitR/68806b2e-44bf-a816-4f66bbcc-76d74827.dmp 19:24:34 INFO - Operating system: Android 19:24:34 INFO - 0.0.0 Linux 2.6.29 #2 PREEMPT Thu Nov 15 13:38:07 CST 2012 i686 generic_x86/sdk_x86/generic_x86:4.2/JOP40C/eng.android-build.20121231.103448:eng/test-keys 19:24:34 INFO - CPU: x86 19:24:34 INFO - GenuineIntel family 6 model 3 stepping 3 19:24:34 INFO - 1 CPU 19:24:34 INFO - 19:24:34 INFO - Crash reason: SIGSEGV 19:24:34 INFO - Crash address: 0x0 19:24:34 INFO - 19:24:34 INFO - Thread 12 (crashed) 19:24:34 INFO - 0 libmozglue.so!jemalloc_crash [jemalloc.c:a24bf8dce031 : 1572 + 0x0] 19:24:34 INFO - eip = 0xaa1c439c esp = 0xa9765f80 ebp = 0xa9765f88 ebx = 0xaa205d4c 19:24:34 INFO - esi = 0xaa09b040 edi = 0xaa09ba58 eax = 0x982b6000 ecx = 0x98200898 19:24:34 INFO - edx = 0x00000000 efl = 0x00210282 19:24:34 INFO - Found by: given as instruction pointer in context 19:24:34 INFO - 1 libmozglue.so!arena_malloc [jemalloc.c:a24bf8dce031 : 4194 + 0x4] 19:24:34 INFO - eip = 0xaa1c67ad esp = 0xa9765f90 ebp = 0xa9765fc8 ebx = 0xaa205d4c 19:24:34 INFO - esi = 0xaa09b040 edi = 0xaa09ba58 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 2 libmozglue.so!__wrap_malloc [jemalloc.c:a24bf8dce031 : 4211 + 0x10] 19:24:34 INFO - eip = 0xaa1c6fe9 esp = 0xa9765fd0 ebp = 0xa9765fe8 ebx = 0xaa205d4c 19:24:34 INFO - esi = 0x00000300 edi = 0x0000001c 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 3 libxul.so!PL_DHashAllocTable [pldhash.cpp:a24bf8dce031 : 72 + 0x7] 19:24:34 INFO - eip = 0xa156156e esp = 0xa9765ff0 ebp = 0xa9766008 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x95e13c00 edi = 0x0000001c 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 4 libxul.so!PL_DHashTableInit [pldhash.cpp:a24bf8dce031 : 222 + 0xb] 19:24:34 INFO - eip = 0xa1561685 esp = 0xa9766010 ebp = 0xa9766038 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x95e13c00 edi = 0x0000001c 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 5 libxul.so!RuleCascadeData::RuleCascadeData(nsIAtom*, bool) [nsCSSRuleProcessor.cpp:a24bf8dce031 : 937 + 0xe] 19:24:34 INFO - eip = 0xa20f7422 esp = 0xa9766040 ebp = 0xa9766088 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x95e13c00 edi = 0x95e13b6c 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 6 libxul.so!nsCSSRuleProcessor::RefreshRuleCascade(nsPresContext*) [nsCSSRuleProcessor.cpp:a24bf8dce031 : 3330 + 0x1c] 19:24:34 INFO - eip = 0xa20f772f esp = 0xa9766090 ebp = 0xa9766158 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x9f059260 edi = 0x96e32c00 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 7 libxul.so!nsCSSRuleProcessor::GetRuleCascade(nsPresContext*) [nsCSSRuleProcessor.cpp:a24bf8dce031 : 3299 + 0x8] 19:24:34 INFO - eip = 0xa20f7b79 esp = 0xa9766160 ebp = 0xa9766178 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x970ad5e0 edi = 0x96e32c00 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 8 libxul.so!nsCSSRuleProcessor::RulesMatching(AnonBoxRuleProcessorData*) [nsCSSRuleProcessor.cpp:a24bf8dce031 : 2448 + 0x9] 19:24:34 INFO - eip = 0xa20f81b0 esp = 0xa9766180 ebp = 0xa97661b8 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa97662b0 edi = 0x9573c700 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 9 libxul.so!EnumRulesMatching<AnonBoxRuleProcessorData> [nsStyleSet.cpp:a24bf8dce031 : 658 + 0x8] 19:24:34 INFO - eip = 0xa213600a esp = 0xa97661c0 ebp = 0xa97661d8 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa97662a4 edi = 0x9573c700 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 10 libxul.so!nsStyleSet::FileRules(bool (*)(nsIStyleRuleProcessor*, void*), RuleProcessorData*, mozilla::dom::Element*, nsRuleWalker*) [nsStyleSet.cpp:a24bf8dce031 : 970 + 0xa] 19:24:34 INFO - eip = 0xa2144ff5 esp = 0xa97661e0 ebp = 0xa9766258 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa97662a4 edi = 0x9573c700 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 11 libxul.so!nsStyleSet::ResolveAnonymousBoxStyle(nsIAtom*, nsStyleContext*) [nsStyleSet.cpp:a24bf8dce031 : 1495 + 0x6] 19:24:34 INFO - eip = 0xa214af24 esp = 0xa9766260 ebp = 0xa97662d8 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x9573c700 edi = 0xa97662a4 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 12 libxul.so!nsCSSFrameConstructor::ConstructRootFrame() [nsCSSFrameConstructor.cpp:a24bf8dce031 : 2517 + 0x9] 19:24:34 INFO - eip = 0xa216eaa3 esp = 0xa97662e0 ebp = 0xa9766328 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x9573c700 edi = 0x9550be90 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 13 libxul.so!PresShell::Initialize(int, int) [nsPresShell.cpp:a24bf8dce031 : 1664 + 0x8] 19:24:34 INFO - eip = 0xa21b16f1 esp = 0xa9766330 ebp = 0xa9766398 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x98566d20 edi = 0x9550be90 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 14 libxul.so!nsContentSink::StartLayout(bool) [nsContentSink.cpp:a24bf8dce031 : 1179 + 0xd] 19:24:34 INFO - eip = 0xa1e0b941 esp = 0xa97663a0 ebp = 0xa97663e8 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x954757b0 edi = 0x0000bb80 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 15 libxul.so!nsHtml5TreeOpExecutor::StartLayout() [nsHtml5TreeOpExecutor.cpp:a24bf8dce031 : 696 + 0x9] 19:24:34 INFO - eip = 0xa1cd10cf esp = 0xa97663f0 ebp = 0xa9766408 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x954757b0 edi = 0x96e88c58 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 16 libxul.so!nsHtml5TreeOperation::Perform(nsHtml5TreeOpExecutor*, nsIContent**) [nsHtml5TreeOperation.cpp:a24bf8dce031 : 678 + 0xa] 19:24:34 INFO - eip = 0xa1cdef88 esp = 0xa9766410 ebp = 0xa9766618 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x96e88c58 edi = 0x96e88c58 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 17 libxul.so!nsHtml5TreeOpExecutor::RunFlushLoop() [nsHtml5TreeOpExecutor.cpp:a24bf8dce031 : 523 + 0xb] 19:24:34 INFO - eip = 0xa1cdfe5b esp = 0xa9766620 ebp = 0xa9766678 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x954757b0 edi = 0x96e88c58 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 18 libxul.so!nsHtml5ExecutorFlusher::Run() [nsHtml5StreamParser.cpp:a24bf8dce031 : 131 + 0x8] 19:24:34 INFO - eip = 0xa1ce0050 esp = 0xa9766680 ebp = 0xa9766698 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa9f3c4c0 edi = 0x00000000 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 19 libxul.so!nsThread::ProcessNextEvent(bool, bool*) [nsThread.cpp:a24bf8dce031 : 610 + 0x5] 19:24:34 INFO - eip = 0xa15a151c esp = 0xa97666a0 ebp = 0xa9766708 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa9f3c4c0 edi = 0x00000000 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 20 libxul.so!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:a24bf8dce031 : 251 + 0x11] 19:24:34 INFO - eip = 0xa1560694 esp = 0xa9766710 ebp = 0xa9766738 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x00000000 edi = 0xa9f6a0c0 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 21 libxul.so!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [MessagePump.cpp:a24bf8dce031 : 85 + 0xb] 19:24:34 INFO - eip = 0xa1735643 esp = 0xa9766740 ebp = 0xa9766788 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa9f01fa0 edi = 0xa9f6a0c0 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 22 libxul.so!MessageLoop::RunInternal() [message_loop.cc:a24bf8dce031 : 220 + 0x6] 19:24:34 INFO - eip = 0xa1720eb8 esp = 0xa9766790 ebp = 0xa97667a8 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa9f6a0c0 edi = 0xa9f3c4c0 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 23 libxul.so!MessageLoop::Run() [message_loop.cc:a24bf8dce031 : 213 + 0x7] 19:24:34 INFO - eip = 0xa1720fe9 esp = 0xa97667b0 ebp = 0xa97667d8 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa9f6a0c0 edi = 0xa9f3c4c0 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 24 libxul.so!nsBaseAppShell::Run() [nsBaseAppShell.cpp:a24bf8dce031 : 161 + 0xd] 19:24:34 INFO - eip = 0xa1c39267 esp = 0xa97667e0 ebp = 0xa9766808 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa9f4f700 edi = 0xa9f3c4c0 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 25 libxul.so!nsAppStartup::Run() [nsAppStartup.cpp:a24bf8dce031 : 267 + 0x5] 19:24:34 INFO - eip = 0xa24fe7ff esp = 0xa9766810 ebp = 0xa9766828 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0x9edb0670 edi = 0x9edb0670 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 26 libxul.so!XREMain::XRE_mainRun() [nsAppRunner.cpp:a24bf8dce031 : 3976 + 0x8] 19:24:34 INFO - eip = 0xa24be418 esp = 0xa9766830 ebp = 0xa9766908 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa9766a8d edi = 0x9edb0670 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 27 libxul.so!XREMain::XRE_main(int, char**, nsXREAppData const*) [nsAppRunner.cpp:a24bf8dce031 : 4044 + 0x8] 19:24:34 INFO - eip = 0xa24bff76 esp = 0xa9766910 ebp = 0xa9766958 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa976699c edi = 0x00000000 19:24:34 INFO - Found by: call frame info 19:24:34 INFO - 28 libxul.so!XRE_main [nsAppRunner.cpp:a24bf8dce031 : 4246 + 0x5] 19:24:34 INFO - eip = 0xa24c01a0 esp = 0xa9766960 ebp = 0xa9766a98 ebx = 0xa3808f9c 19:24:34 INFO - esi = 0xa976699c edi = 0xa9f10520 19:24:34 INFO - Found by: call frame info

https://tbpl.mozilla.org/php/getParsedLog.php?id=30809877&tree=Cedar&full=1#error0 21:13:22 INFO - REFTEST INFO | Loading a blank page 21:13:22 INFO - REFTEST TEST-END | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?aa&_____&preserve 21:13:22 INFO - REFTEST TEST-START | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?__&alpha&preserve 21:13:22 INFO - RESTORE PREFERENCE pref(webgl.force-enabled,false) 21:13:22 INFO - SET PREFERENCE pref(webgl.force-enabled,true) 21:13:22 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?__&alpha&preserve | 1089 / 3781 (28%) 21:13:22 INFO - 21:13:22 INFO - INFO | automation.py | Application ran for: 0:06:22.481850 21:13:22 INFO - INFO | zombiecheck | Reading PID log: /tmp/tmpJEv4dnpidlog 21:13:22 INFO - mozcrash INFO | Downloading symbols from: http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/cedar-android-x86/1384909322/fennec-28.0a1.en-US.android-i386.crashreporter-symbols.zip 21:13:22 INFO - /data/anr/traces.txt not found 21:13:22 INFO - PROCESS-CRASH | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-clear-test.html?__&alpha&preserve | application crashed [@ jemalloc_crash] 21:13:22 INFO - Crash dump filename: /tmp/tmp7A0Iu7/164d3a57-4f34-39e7-5c6a97c9-2c5d40e0.dmp 21:13:22 INFO - Operating system: Android 21:13:22 INFO - 0.0.0 Linux 2.6.29 #2 PREEMPT Thu Nov 15 13:38:07 CST 2012 i686 generic_x86/sdk_x86/generic_x86:4.2/JOP40C/eng.android-build.20121231.103448:eng/test-keys 21:13:22 INFO - CPU: x86 21:13:22 INFO - GenuineIntel family 6 model 3 stepping 3 21:13:22 INFO - 1 CPU 21:13:22 INFO - 21:13:22 INFO - Crash reason: SIGSEGV 21:13:22 INFO - Crash address: 0x0 21:13:22 INFO - 21:13:22 INFO - Thread 12 (crashed) 21:13:22 INFO - 0 libmozglue.so!jemalloc_crash [jemalloc.c:a24bf8dce031 : 1572 + 0x0] 21:13:22 INFO - eip = 0xaa33139c esp = 0xa98d2d70 ebp = 0xa98d2d78 ebx = 0xaa372d4c 21:13:22 INFO - esi = 0xaa208040 edi = 0xaa208668 eax = 0x98fc1000 ecx = 0x98f0091c 21:13:22 INFO - edx = 0x00000000 efl = 0x00210282 21:13:22 INFO - Found by: given as instruction pointer in context 21:13:22 INFO - 1 libmozglue.so!arena_malloc [jemalloc.c:a24bf8dce031 : 4194 + 0x4] 21:13:22 INFO - eip = 0xaa3337ad esp = 0xa98d2d80 ebp = 0xa98d2db8 ebx = 0xaa372d4c 21:13:22 INFO - esi = 0xaa208040 edi = 0xaa208668 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 2 libmozglue.so!__wrap_malloc [jemalloc.c:a24bf8dce031 : 4211 + 0x10] 21:13:22 INFO - eip = 0xaa333fe9 esp = 0xa98d2dc0 ebp = 0xa98d2dd8 ebx = 0xaa372d4c 21:13:22 INFO - esi = 0x00000130 edi = 0x9615b6c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 3 libxul.so!JSAutoStructuredCloneBuffer::copy(unsigned long long const*, unsigned int, unsigned int) [Utility.h:a24bf8dce031 : 144 + 0xa] 21:13:22 INFO - eip = 0xa2aed14d esp = 0xa98d2de0 ebp = 0xa98d2e18 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9a597400 edi = 0x9615b6c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 4 libxul.so!nsInProcessTabChildGlobal::DoSendAsyncMessage(JSContext*, nsAString_internal const&, mozilla::dom::StructuredCloneData const&, JS::Handle<JSObject*>, nsIPrincipal*) [nsInProcessTabChildGlobal.cpp:a24bf8dce031 : 69 + 0x14] 21:13:22 INFO - eip = 0xa1fcf15b esp = 0xa98d2e20 ebp = 0xa98d2e68 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9a597400 edi = 0x9615b6c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 5 libxul.so!nsFrameMessageManager::DispatchAsyncMessageInternal(JSContext*, nsAString_internal const&, mozilla::dom::StructuredCloneData const&, JS::Handle<JSObject*>, nsIPrincipal*) [nsFrameMessageManager.cpp:a24bf8dce031 : 614 + 0x16] 21:13:22 INFO - eip = 0xa1fbcb86 esp = 0xa98d2e70 ebp = 0xa98d2ea8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9a576dd0 edi = 0xa98d30a0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 6 libxul.so!nsFrameMessageManager::DispatchAsyncMessage(nsAString_internal const&, JS::Value const&, JS::Value const&, nsIPrincipal*, JSContext*, unsigned char) [nsFrameMessageManager.cpp:a24bf8dce031 : 645 + 0x10] 21:13:22 INFO - eip = 0xa1fbf634 esp = 0xa98d2eb0 ebp = 0xa98d2f18 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa110f20 edi = 0xa98d30a0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 7 libxul.so!nsInProcessTabChildGlobal::SendAsyncMessage(nsAString_internal const&, JS::Value const&, JS::Value const&, nsIPrincipal*, JSContext*, unsigned char) [nsInProcessTabChildGlobal.h:a24bf8dce031 : 42 + 0x19] 21:13:22 INFO - eip = 0xa1fce535 esp = 0xa98d2f20 ebp = 0xa98d2f48 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x00000004 edi = 0xa98d30b0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 8 libxul.so + 0x3919c7 21:13:22 INFO - eip = 0xa17149c8 esp = 0xa98d2f50 ebp = 0xa98d2f98 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x00000004 edi = 0xa98d30b0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 9 libxul.so!XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode) [XPCWrappedNative.cpp:a24bf8dce031 : 2593 + 0x15] 21:13:22 INFO - eip = 0xa1df4de5 esp = 0xa98d2fa0 ebp = 0xa98d3128 21:13:22 INFO - Found by: previous frame's frame pointer 21:13:22 INFO - 10 libxul.so!XPC_WN_CallMethod(JSContext*, unsigned int, JS::Value*) [XPCWrappedNativeJSOps.cpp:a24bf8dce031 : 1304 + 0x9] 21:13:22 INFO - eip = 0xa1df7e59 esp = 0xa98d3130 ebp = 0xa98d31f8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa98d3168 edi = 0x9b366d80 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 11 libxul.so!js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) [jscntxtinlines.h:a24bf8dce031 : 220 + 0x18] 21:13:22 INFO - eip = 0xa2abb841 esp = 0xa98d3200 ebp = 0xa98d3438 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa1df7ce4 edi = 0xa98d3244 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 12 libxul.so!Interpret [Interpreter.cpp:a24bf8dce031 : 2502 + 0x29] 21:13:22 INFO - eip = 0xa2ab5e2b esp = 0xa98d3440 ebp = 0xa98d39c8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa3956b20 edi = 0x9f20e600 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 13 libxul.so!js::RunScript [Interpreter.cpp:a24bf8dce031 : 420 + 0x9] 21:13:22 INFO - eip = 0xa2abb4ec esp = 0xa98d39d0 ebp = 0xa98d3a38 21:13:22 INFO - Found by: previous frame's frame pointer 21:13:22 INFO - 14 libxul.so!js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) [Interpreter.cpp:a24bf8dce031 : 389 + 0x10] 21:13:22 INFO - eip = 0xa2abb764 esp = 0xa98d3a40 ebp = 0xa98d3c78 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa98d3a84 edi = 0xaa110f20 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 15 libxul.so!js::Invoke(JSContext*, JS::Value const&, JS::Value const&, unsigned int, JS::Value*, JS::MutableHandle<JS::Value>) [Interpreter.cpp:a24bf8dce031 : 513 + 0x22] 21:13:22 INFO - eip = 0xa2abd9ad esp = 0xa98d3c80 ebp = 0xa98d3d38 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa110f20 edi = 0xaa110f20 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 16 libxul.so!JS_CallFunctionValue(JSContext*, JSObject*, JS::Value, unsigned int, JS::Value*, JS::Value*) [jsapi.cpp:a24bf8dce031 : 5032 + 0x29] 21:13:22 INFO - eip = 0xa298c478 esp = 0xa98d3d40 ebp = 0xa98d3da8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa110f20 edi = 0xa98d4028 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 17 libxul.so!nsXPCWrappedJSClass::CallMethod(nsXPCWrappedJS*, unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) [XPCWrappedJSClass.cpp:a24bf8dce031 : 1425 + 0x2b] 21:13:22 INFO - eip = 0xa1def7bf esp = 0xa98d3db0 ebp = 0xa98d3f98 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x00000001 edi = 0xa98d4028 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 18 libxul.so!nsXPCWrappedJS::CallMethod(unsigned short, XPTMethodDescriptor const*, nsXPTCMiniVariant*) [XPCWrappedJS.cpp:a24bf8dce031 : 505 + 0x13] 21:13:22 INFO - eip = 0xa1dec1dd esp = 0xa98d3fa0 ebp = 0xa98d3fd8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9615b700 edi = 0x00000003 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 19 libxul.so!PrepareAndDispatch [xptcstubs_gcc_x86_unix.cpp:a24bf8dce031 : 60 + 0x1b] 21:13:22 INFO - eip = 0xa17154d6 esp = 0xa98d3fe0 ebp = 0xa98d40b8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa98d4020 edi = 0x9726be20 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 20 libxul.so!nsEventListenerManager::HandleEventSubType(nsListenerStruct*, nsIDOMEvent*, mozilla::dom::EventTarget*, nsCxPusher*) [nsEventListenerManager.cpp:a24bf8dce031 : 964 + 0x8] 21:13:22 INFO - eip = 0xa2044714 esp = 0xa98d40c0 ebp = 0xa98d4118 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9a597c40 edi = 0x9726be20 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 21 libxul.so!nsEventListenerManager::HandleEventInternal(nsPresContext*, mozilla::WidgetEvent*, nsIDOMEvent**, mozilla::dom::EventTarget*, nsEventStatus*, nsCxPusher*) [nsEventListenerManager.cpp:a24bf8dce031 : 1041 + 0x16] 21:13:22 INFO - eip = 0xa2044902 esp = 0xa98d4120 ebp = 0xa98d4198 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x97b01510 edi = 0x9a597c40 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 22 libxul.so!nsEventTargetChainItem::HandleEvent(nsEventChainPostVisitor&, ELMCreationDetector&, nsCxPusher*) [nsEventListenerManager.h:a24bf8dce031 : 326 + 0xe] 21:13:22 INFO - eip = 0xa20401f2 esp = 0xa98d41a0 ebp = 0xa98d41e8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa98d4298 edi = 0x00000000 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 23 libxul.so!nsEventTargetChainItem::HandleEventTargetChain(nsTArray<nsEventTargetChainItem>&, nsEventChainPostVisitor&, nsDispatchingCallback*, ELMCreationDetector&, nsCxPusher*) [nsEventDispatcher.cpp:a24bf8dce031 : 313 + 0xc] 21:13:22 INFO - eip = 0xa20405aa esp = 0xa98d41f0 ebp = 0xa98d4238 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa98d4290 edi = 0x9b90b01c 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 24 libxul.so!nsEventDispatcher::Dispatch(nsISupports*, nsPresContext*, mozilla::WidgetEvent*, nsIDOMEvent*, nsEventStatus*, nsDispatchingCallback*, nsCOMArray<mozilla::dom::EventTarget>*) [nsEventDispatcher.cpp:a24bf8dce031 : 609 + 0x5] 21:13:22 INFO - eip = 0xa20417d0 esp = 0xa98d4240 ebp = 0xa98d4308 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x00000000 edi = 0x00000000 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 25 libxul.so!nsEventDispatcher::DispatchDOMEvent(nsISupports*, mozilla::WidgetEvent*, nsIDOMEvent*, nsPresContext*, nsEventStatus*) [nsEventDispatcher.cpp:a24bf8dce031 : 676 + 0x15] 21:13:22 INFO - eip = 0xa2041ace esp = 0xa98d4310 ebp = 0xa98d4368 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x97b01510 edi = 0x9726be20 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 26 libxul.so!nsPresContext::FireDOMPaintEvent(nsInvalidateRequestList*) [nsPresContext.cpp:a24bf8dce031 : 2175 + 0x14] 21:13:22 INFO - eip = 0xa230e560 esp = 0xa98d4370 ebp = 0xa98d43b8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x96f1a000 edi = 0x9726be20 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 27 libxul.so!DelayedFireDOMPaintEvent::Run() [nsPresContext.cpp:a24bf8dce031 : 2382 + 0xb] 21:13:22 INFO - eip = 0xa230e5ce esp = 0xa98d43c0 ebp = 0xa98d43d8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x00000000 edi = 0x00000002 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 28 libxul.so!nsContentUtils::RemoveScriptBlocker() [nsContentUtils.cpp:a24bf8dce031 : 4787 + 0x8] 21:13:22 INFO - eip = 0xa1f85afd esp = 0xa98d43e0 ebp = 0xa98d4428 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x00000000 edi = 0x00000002 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 29 libxul.so!nsViewManager::ProcessPendingUpdatesForView(nsView*, bool) [nsContentUtils.h:a24bf8dce031 : 2201 + 0x4] 21:13:22 INFO - eip = 0xa1f56ae3 esp = 0xa98d4430 ebp = 0xa98d4498 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9b9b7100 edi = 0x9b9902e0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 30 libxul.so!nsViewManager::ProcessPendingUpdates() [nsViewManager.cpp:a24bf8dce031 : 1053 + 0xb] 21:13:22 INFO - eip = 0xa1f56b9a esp = 0xa98d44a0 ebp = 0xa98d44b8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9b9b7100 edi = 0x9b9b7100 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 31 libxul.so!nsRefreshDriver::Tick(long long, mozilla::TimeStamp) [nsRefreshDriver.cpp:a24bf8dce031 : 1215 + 0x8] 21:13:22 INFO - eip = 0xa22cc77d esp = 0xa98d44c0 ebp = 0xa98d45b8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9b480a00 edi = 0x9b9b7100 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 32 libxul.so!mozilla::RefreshDriverTimer::Tick() [nsRefreshDriver.cpp:a24bf8dce031 : 168 + 0x13] 21:13:22 INFO - eip = 0xa22ccea2 esp = 0xa98d45c0 ebp = 0xa98d4628 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x948630b5 edi = 0x00000080 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 33 libxul.so!nsTimerImpl::Fire() [nsTimerImpl.cpp:a24bf8dce031 : 546 + 0x7] 21:13:22 INFO - eip = 0xa1711d2e esp = 0xa98d4630 ebp = 0xa98d4678 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9c56b8b0 edi = 0xa22ccec7 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 34 libxul.so!nsTimerEvent::Run() [nsTimerImpl.cpp:a24bf8dce031 : 630 + 0x8] 21:13:22 INFO - eip = 0xa1711e00 esp = 0xa98d4680 ebp = 0xa98d4698 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa13c4c0 edi = 0x00000000 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 35 libxul.so!nsThread::ProcessNextEvent(bool, bool*) [nsThread.cpp:a24bf8dce031 : 610 + 0x5] 21:13:22 INFO - eip = 0xa170f51c esp = 0xa98d46a0 ebp = 0xa98d4708 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa13c4c0 edi = 0x00000000 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 36 libxul.so!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:a24bf8dce031 : 251 + 0x11] 21:13:22 INFO - eip = 0xa16ce694 esp = 0xa98d4710 ebp = 0xa98d4738 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x00000000 edi = 0xaa16a0c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 37 libxul.so!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [MessagePump.cpp:a24bf8dce031 : 85 + 0xb] 21:13:22 INFO - eip = 0xa18a3643 esp = 0xa98d4740 ebp = 0xa98d4788 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa101fa0 edi = 0xaa16a0c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 38 libxul.so!MessageLoop::RunInternal() [message_loop.cc:a24bf8dce031 : 220 + 0x6] 21:13:22 INFO - eip = 0xa188eeb8 esp = 0xa98d4790 ebp = 0xa98d47a8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa16a0c0 edi = 0xaa13c4c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 39 libxul.so!MessageLoop::Run() [message_loop.cc:a24bf8dce031 : 213 + 0x7] 21:13:22 INFO - eip = 0xa188efe9 esp = 0xa98d47b0 ebp = 0xa98d47d8 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa16a0c0 edi = 0xaa13c4c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 40 libxul.so!nsBaseAppShell::Run() [nsBaseAppShell.cpp:a24bf8dce031 : 161 + 0xd] 21:13:22 INFO - eip = 0xa1da7267 esp = 0xa98d47e0 ebp = 0xa98d4808 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xaa14f700 edi = 0xaa13c4c0 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 41 libxul.so!nsAppStartup::Run() [nsAppStartup.cpp:a24bf8dce031 : 267 + 0x5] 21:13:22 INFO - eip = 0xa266c7ff esp = 0xa98d4810 ebp = 0xa98d4828 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0x9efb0670 edi = 0x9efb0670 21:13:22 INFO - Found by: call frame info 21:13:22 INFO - 42 libxul.so!XREMain::XRE_mainRun() [nsAppRunner.cpp:a24bf8dce031 : 3976 + 0x8] 21:13:22 INFO - eip = 0xa262c418 esp = 0xa98d4830 ebp = 0xa98d4908 ebx = 0xa3976f9c 21:13:22 INFO - esi = 0xa98d4a8d edi = 0x9efb0670 21:13:22 INFO - Found by: call frame info

https://tbpl.mozilla.org/php/getParsedLog.php?id=30811220&tree=Cedar&full=1 21:50:52 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8856/tests/content/canvas/test/reftest/wrapper.html?colors.png | 1205 / 3781 (31%) 21:50:52 INFO - REFTEST TEST-PASS | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-color-test.html?16bpp | image comparison (==) 21:50:52 INFO - REFTEST INFO | Loading a blank page 21:50:52 INFO - REFTEST TEST-END | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-color-test.html?16bpp 21:50:52 INFO - REFTEST TEST-START | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-color-test.html?16bpp&readback 21:50:52 INFO - RESTORE PREFERENCE pref(webgl.prefer-16bpp,false) 21:50:52 INFO - RESTORE PREFERENCE pref(webgl.force-enabled,false) 21:50:52 INFO - SET PREFERENCE pref(webgl.force-enabled,true) 21:50:52 INFO - SET PREFERENCE pref(webgl.prefer-16bpp,true) 21:50:52 INFO - SET PREFERENCE pref(webgl.force-layers-readback,true) 21:50:52 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-color-test.html?16bpp&readback | 1206 / 3781 (31%) 21:50:52 INFO - INFO | automation.py | Application ran for: 0:06:42.468555 21:50:52 INFO - INFO | zombiecheck | Reading PID log: /tmp/tmpuCjtFKpidlog 21:50:52 INFO - mozcrash INFO | Downloading symbols from: http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/cedar-android-x86/1384909322/fennec-28.0a1.en-US.android-i386.crashreporter-symbols.zip 21:50:52 INFO - /data/anr/traces.txt not found 21:50:52 INFO - PROCESS-CRASH | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-color-test.html?16bpp&readback | application crashed [@ jemalloc_crash] 21:50:52 INFO - Crash dump filename: /tmp/tmp8MzKjS/4fab8bfd-36e2-32fd-61196a27-2e7546e6.dmp 21:50:52 INFO - Operating system: Android 21:50:52 INFO - 0.0.0 Linux 2.6.29 #2 PREEMPT Thu Nov 15 13:38:07 CST 2012 i686 generic_x86/sdk_x86/generic_x86:4.2/JOP40C/eng.android-build.20121231.103448:eng/test-keys 21:50:52 INFO - CPU: x86 21:50:52 INFO - GenuineIntel family 6 model 3 stepping 3 21:50:52 INFO - 1 CPU 21:50:52 INFO - 21:50:52 INFO - Crash reason: SIGSEGV 21:50:52 INFO - Crash address: 0x0 21:50:52 INFO - 21:50:52 INFO - Thread 12 (crashed) 21:50:52 INFO - 0 libmozglue.so!jemalloc_crash [jemalloc.c:a24bf8dce031 : 1572 + 0x0] 21:50:52 INFO - eip = 0xaa2b539c esp = 0xa9856380 ebp = 0xa9856388 ebx = 0xaa2f6d4c 21:50:52 INFO - esi = 0x966fb000 edi = 0x966fed40 eax = 0x00000000 ecx = 0x00000000 21:50:52 INFO - edx = 0x96600bf8 efl = 0x00210282 21:50:52 INFO - Found by: given as instruction pointer in context 21:50:52 INFO - 1 libmozglue.so!arena_dalloc [jemalloc.c:a24bf8dce031 : 4627 + 0x4] 21:50:52 INFO - eip = 0xaa2b6c5f esp = 0xa9856390 ebp = 0xa98563c8 ebx = 0xaa2f6d4c 21:50:52 INFO - esi = 0x966fb000 edi = 0x966fed40 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 2 libxul.so!PL_DHashFreeTable [pldhash.cpp:a24bf8dce031 : 78 + 0x7] 21:50:52 INFO - eip = 0xa165154b esp = 0xa98563d0 ebp = 0xa98563e8 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96166140 edi = 0x966fee80 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 3 libxul.so!PL_DHashTableFinish [pldhash.cpp:a24bf8dce031 : 311 + 0xa] 21:50:52 INFO - eip = 0xa1651755 esp = 0xa98563f0 ebp = 0xa9856418 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96166140 edi = 0x966fee80 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 4 libxul.so!mozilla::MaskLayerImageCache::~MaskLayerImageCache() + 0x21 21:50:52 INFO - eip = 0xa165e3dc esp = 0xa9856420 ebp = 0xa9856438 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96166100 edi = 0x00000000 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 5 libxul.so!mozilla::css::Loader::Sheets::~Sheets() [nsBaseHashtable.h:a24bf8dce031 : 52 + 0x8] 21:50:52 INFO - eip = 0xa21f3c92 esp = 0xa9856440 ebp = 0xa9856458 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96166100 edi = 0x00000000 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 6 libxul.so!mozilla::css::Loader::~Loader() [nsAutoPtr.h:a24bf8dce031 : 77 + 0x12] 21:50:52 INFO - eip = 0xa21fa19b esp = 0xa9856460 ebp = 0xa9856478 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96166100 edi = 0x00000000 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 7 libxul.so!mozilla::css::Loader::Release() [Loader.h:a24bf8dce031 : 136 + 0x5] 21:50:52 INFO - eip = 0xa1efb3ca esp = 0xa9856480 ebp = 0xa98564a8 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x95724e80 edi = 0x00000000 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 8 libxul.so!nsIDocument::~nsIDocument() [nsAutoPtr.h:a24bf8dce031 : 887 + 0x8] 21:50:52 INFO - eip = 0xa1f247ca esp = 0xa98564b0 ebp = 0xa98564d8 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96265800 edi = 0x00000000 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 9 libxul.so!nsDocument::~nsDocument() [nsDocument.cpp:a24bf8dce031 : 1459 + 0x7] 21:50:52 INFO - eip = 0xa1f27da6 esp = 0xa98564e0 ebp = 0xa9856528 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96265800 edi = 0x962659f0 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 10 libxul.so!nsHTMLDocument::~nsHTMLDocument() [nsHTMLDocument.cpp:a24bf8dce031 : 199 + 0x78] 21:50:52 INFO - eip = 0xa2038135 esp = 0xa9856530 ebp = 0xa9856548 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96265800 edi = 0xa985660c 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 11 libxul.so!nsHTMLDocument::~nsHTMLDocument() [nsHTMLDocument.cpp:a24bf8dce031 : 201 + 0x5] 21:50:52 INFO - eip = 0xa203815c esp = 0xa9856550 ebp = 0xa9856568 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x96265800 edi = 0xa985660c 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 12 libxul.so!InMemoryDataSource::DeleteCycleCollectable() [nsInMemoryDataSource.cpp:a24bf8dce031 : 859 + 0x8] 21:50:52 INFO - eip = 0xa191817b esp = 0xa9856570 ebp = 0xa9856588 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x0000003f edi = 0xa985660c 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 13 libxul.so!nsDocument::cycleCollection::DeleteCycleCollectable(void*) [nsDocument.h:a24bf8dce031 : 864 + 0xb] 21:50:52 INFO - eip = 0xa1f0f4cf esp = 0xa9856590 ebp = 0xa98565a8 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x0000003f edi = 0xa985660c 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 14 libxul.so!SnowWhiteKiller::~SnowWhiteKiller() [nsCycleCollector.cpp:a24bf8dce031 : 2011 + 0xc] 21:50:52 INFO - eip = 0xa1662ff4 esp = 0xa98565b0 ebp = 0xa98565d8 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x0000003f edi = 0xa985660c 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 15 libxul.so!nsCycleCollector::FreeSnowWhite(bool) [nsCycleCollector.cpp:a24bf8dce031 : 2120 + 0x8] 21:50:52 INFO - eip = 0xa1663090 esp = 0xa98565e0 ebp = 0xa9856618 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xaa0a7054 edi = 0xa985660c 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 16 libxul.so!nsCycleCollector_doDeferredDeletion() [nsCycleCollector.cpp:a24bf8dce031 : 3158 + 0xa] 21:50:52 INFO - eip = 0xa16630de esp = 0xa9856620 ebp = 0xa9856638 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x9f10c0b0 edi = 0xa9856668 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 17 libxul.so!AsyncFreeSnowWhite::Run() [XPCJSRuntime.cpp:a24bf8dce031 : 205 + 0x4] 21:50:52 INFO - eip = 0xa1d5bcd3 esp = 0xa9856640 ebp = 0xa9856698 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x9f10c0b0 edi = 0xa9856668 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 18 libxul.so!nsThread::ProcessNextEvent(bool, bool*) [nsThread.cpp:a24bf8dce031 : 610 + 0x5] 21:50:52 INFO - eip = 0xa169151c esp = 0xa98566a0 ebp = 0xa9856708 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xaa03c4c0 edi = 0x00000000 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 19 libxul.so!NS_ProcessNextEvent(nsIThread*, bool) [nsThreadUtils.cpp:a24bf8dce031 : 251 + 0x11] 21:50:52 INFO - eip = 0xa1650694 esp = 0xa9856710 ebp = 0xa9856738 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x00000000 edi = 0xaa06a0c0 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 20 libxul.so!mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) [MessagePump.cpp:a24bf8dce031 : 85 + 0xb] 21:50:52 INFO - eip = 0xa1825643 esp = 0xa9856740 ebp = 0xa9856788 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xaa001fa0 edi = 0xaa06a0c0 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 21 libxul.so!MessageLoop::RunInternal() [message_loop.cc:a24bf8dce031 : 220 + 0x6] 21:50:52 INFO - eip = 0xa1810eb8 esp = 0xa9856790 ebp = 0xa98567a8 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xaa06a0c0 edi = 0xaa03c4c0 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 22 libxul.so!MessageLoop::Run() [message_loop.cc:a24bf8dce031 : 213 + 0x7] 21:50:52 INFO - eip = 0xa1810fe9 esp = 0xa98567b0 ebp = 0xa98567d8 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xaa06a0c0 edi = 0xaa03c4c0 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 23 libxul.so!nsBaseAppShell::Run() [nsBaseAppShell.cpp:a24bf8dce031 : 161 + 0xd] 21:50:52 INFO - eip = 0xa1d29267 esp = 0xa98567e0 ebp = 0xa9856808 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xaa04f700 edi = 0xaa03c4c0 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 24 libxul.so!nsAppStartup::Run() [nsAppStartup.cpp:a24bf8dce031 : 267 + 0x5] 21:50:52 INFO - eip = 0xa25ee7ff esp = 0xa9856810 ebp = 0xa9856828 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0x9eeb0670 edi = 0x9eeb0670 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 25 libxul.so!XREMain::XRE_mainRun() [nsAppRunner.cpp:a24bf8dce031 : 3976 + 0x8] 21:50:52 INFO - eip = 0xa25ae418 esp = 0xa9856830 ebp = 0xa9856908 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xa9856a8d edi = 0x9eeb0670 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 26 libxul.so!XREMain::XRE_main(int, char**, nsXREAppData const*) [nsAppRunner.cpp:a24bf8dce031 : 4044 + 0x8] 21:50:52 INFO - eip = 0xa25aff76 esp = 0xa9856910 ebp = 0xa9856958 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xa985699c edi = 0x00000000 21:50:52 INFO - Found by: call frame info 21:50:52 INFO - 27 libxul.so!XRE_main [nsAppRunner.cpp:a24bf8dce031 : 4246 + 0x5] 21:50:52 INFO - eip = 0xa25b01a0 esp = 0xa9856960 ebp = 0xa9856a98 ebx = 0xa38f8f9c 21:50:52 INFO - esi = 0xa985699c edi = 0xaa010520 21:50:52 INFO - Found by: call frame info

I tried skipping individual tests, but found that most/all of the canvas reftests need to be skipped to get a clean run. (Android&&AndroidVersion>=17) is the "standard" condition we are using for problem tests in this environment.

This really doesn't look like a WebGL bug. Something in the (extremely basic) WebGL reftests is causing these crashes, but none of these traces has anything relating to WebGL in it. In general, these tests are actually quite reliable on Android, but we don't have a way to mark them as failing for our ARMv6 slaves. I don't remember what the deal is for B2G, but I think it's also a problem with different slaves. Really, we need to classify our test systems better, because I can guarantee that if you ran these tests on basically any Android 4.2 device, they will pass. Let's try to narrow this down a little bit more here, before we start down the path of disabling our way to victory.

Which of the following crashes, and which are fine: == webgl-disable-test.html?nogl wrapper.html?green.png pref(webgl.disabled,true) == webgl-disable-test.html wrapper.html?green.png == webgl-clear-test.html?__&_____&________ wrapper.html?green.png == wrapper.html?green.png wrapper.html?green.png == webgl-clear-test.html?__&_____&________ webgl-clear-test.html?__&_____&________

Comment on attachment 8340540 [details] [diff] [review] skip all canvas reftests on Android 4.2 x86 emu Review of attachment 8340540 [details] [diff] [review]: ----------------------------------------------------------------- Going to r- this for now. Rather than doing this, we should detect that we're running this particular config, and only disable it for this config. For instance, this will break as soon as we add new android device tests.

(In reply to Jeff Gilbert [:jgilbert] from comment #5) > Which of the following crashes, and which are fine: Since the crashes are intermittent, and infrequent, a definitive answer may be elusive. In 10 runs, none of those failed. I'll run more tests today.

(In reply to Jeff Gilbert [:jgilbert] from comment #6) > Going to r- this for now. Rather than doing this, we should detect that > we're running this particular config, and only disable it for this config. There have been a few discussions over the course of the x86 emulator project about what criteria should be used for failing or skipping tests. We considered adding support to detect running on x86, but overall felt that AndroidVersion would suffice, and would be - in the absence of determination of root cause of any given failure - just as likely to be the correct differentiating factor as any other (processor, graphics support, emulator/device, debug/opt, ...). AFAIK we have no way for a test harness to detect that it is running in a particular tbpl config. (Is that what you had in mind? skip-if(RunningOnTbpl&&TbplConfig=="Android 4.2 x86 Opt")?) From the continuous integration/tbpl perspective, each of our current test environments runs a distinct version of Android and there are no plans to have more than one tbpl test configuration for the same Android version. Using AndroidVersion to skip/fail tests for a particular tbpl config means that some tests will be unnecessarily skipped and some will unexpectedly pass on some devices, but since we don't regularly test on all devices, I wouldn't expect a clean run anyway. We settled on AndroidVersion>=17 at a time when it was unclear if the x86 emulator would be running 17 or 18. I'm happy to change that to AndroidVersion==17, if that helps. > For instance, this will break as soon as we add new android device tests. I'm not sure I understand "break". We may skip tests unnecessarily, but otherwise...?

(In reply to Geoff Brown [:gbrown] from comment #7) > (In reply to Jeff Gilbert [:jgilbert] from comment #5) > > Which of the following crashes, and which are fine: > > Since the crashes are intermittent, and infrequent, a definitive answer may > be elusive. In 10 runs, none of those failed. I'll run more tests today. In 100 runs, none of these tests failed. (However in 3 cases, reftest-2 failed sometime after running those tests: the webgl tests passed, dozens of additional tests ran and passed, and then the harness lost connectivity to the device. I don't know if this is related to the webgl tests -- I suspect not.)

A few more examples: https://tbpl.mozilla.org/php/getParsedLog.php?id=31599646&full=1&branch=cedar#error0 https://tbpl.mozilla.org/php/getParsedLog.php?id=31599739&full=1&branch=cedar#error0 https://tbpl.mozilla.org/php/getParsedLog.php?id=31599588&full=1&branch=cedar#error0 https://tbpl.mozilla.org/php/getParsedLog.php?id=31597277&full=1&branch=cedar#error0 https://tbpl.mozilla.org/php/getParsedLog.php?id=31594742&full=1&branch=cedar#error0 I don't see much commonality between the crash stacks (other than jemalloc_crash), but all of the logcats have a similar Javascript Warning just before the crash: 18:24:46 INFO - 12-07 02:23:45.159 E/GeckoConsole( 1900): [JavaScript Warning: "Error: WebGL: Exceeded 2 live WebGL contexts for this principal, losing the least recently used one." {file: "http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-utils.js" line: 59}] 18:24:46 INFO - 12-07 02:23:45.489 E/eglCodecCommon( 1900): glUtilsParamSize: unknow param 0x00000bd0 18:24:46 INFO - 12-07 02:23:46.699 E/GeckoConsole( 1900): [JavaScript Warning: "Error: WebGL: Exceeded 2 live WebGL contexts for this principal, losing the least recently used one." {file: "http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-utils.js" line: 59}] 18:24:46 INFO - 12-07 02:23:47.109 E/eglCodecCommon( 1900): glUtilsParamSize: unknow param 0x00000bd0 18:24:46 INFO - 12-07 02:23:48.339 E/GeckoConsole( 1900): [JavaScript Warning: "Error: WebGL: Exceeded 2 live WebGL contexts for this principal, losing the least recently used one." {file: "http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-utils.js" line: 59}] 18:24:46 INFO - 12-07 02:23:48.719 E/eglCodecCommon( 1900): glUtilsParamSize: unknow param 0x00000bd0 18:24:46 INFO - 12-07 02:23:50.099 E/GeckoConsole( 1900): [JavaScript Warning: "Error: WebGL: Exceeded 2 live WebGL contexts for this principal, losing the least recently used one." {file: "http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-utils.js" line: 59}] 18:24:46 INFO - 12-07 02:23:50.179 F/libc ( 1900): Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 1919 (Gecko) 18:24:46 INFO - 12-07 02:23:50.299 I/DEBUG ( 807): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** Might that be significant?

No, it's fully expected. We actively 'lose' the oldest-used WebGL context when we create a new one, past a certain number of contexts. On mobile, that limit is 2 per principal. And in the test suite, a few hundreds of WebGL contexts are created, all on the same principal. 'Losing' a WebGL context means discarding all its graphics resources. This frees memory, so it shouldn't contribute to the problem.

jgilbert - Can you suggest a way forward here? If I can disable these tests, I can demonstrate passing reftest jobs on Cedar and get the Android x86 tests running on trunk trees. At that point, we can investigate the crashes further with try pushes. As long as the reftests are enabled and crashing, we cannot run Android x86 reftests outside of Cedar/Ash. For the skip-if criteria, please see Comment 8.

(In reply to Geoff Brown [:gbrown] from comment #12) > jgilbert - Can you suggest a way forward here? > > If I can disable these tests, I can demonstrate passing reftest jobs on > Cedar and get the Android x86 tests running on trunk trees. At that point, > we can investigate the crashes further with try pushes. As long as the > reftests are enabled and crashing, we cannot run Android x86 reftests > outside of Cedar/Ash. > > For the skip-if criteria, please see Comment 8. We should absolutely skip-if based on TBPL configs, if we have no reason to suspect that it's an android-version-specific issue. Our skip-if/fail-if isn't just for TBPL. It's also a general assertion of what we think is true. We *don't* think that reftests are crasy on Android. They appear to be crashy on the emulator. (This seems like a pretty big bug for webgl) Thus, we should only disable them for the emulator. If I run this on my phone, I'm almost positive it'll work fine. Skipping tests is awful, because realistically, no one tries to re-enable skipped tests, because there's always other work to do. Effectively, it'll just never get run again. We have these tests for a good reason, and we want to run them on every machine we can manage it. If we need to, we *can* skip it, but we should skip the fewest tests on the fewest platforms that we can manage. To reiterate, these are comically simple tests. Some of them are literally 'clear the canvas to green, and make sure it looks green'. That they crash means that either WebGL or the reftest harness are comically broken on your platform. If it's the former, we should just disable WebGL for the emulator, but that makes the emulator that much worse at doing its job. I also hadn't realized before, but the push you linked to only has an OPT build run, no DEBUG run. DEBUG runs are so ridiculously more important, because we actually get assert coverage. It's entirely possible that the runs here would gives us an easy-to-identify assert if run in DEBUG mode, since we assert a lot of things. Don't add OPT without adding DEBUG. If we have to pick just one, only do DEBUG builds! If the emulator is simple to compile and setup, I should be able to get to this at some point, though it's got a lot of competition for priority. It's quicker if you try a DEBUG build, and run with an env var of MOZ_GL_DEBUG=1, and see if the crash stacks improve. (In reply to Geoff Brown [:gbrown] from comment #9) > (In reply to Geoff Brown [:gbrown] from comment #7) > > (In reply to Jeff Gilbert [:jgilbert] from comment #5) > > > Which of the following crashes, and which are fine: > > > > Since the crashes are intermittent, and infrequent, a definitive answer may > > be elusive. In 10 runs, none of those failed. I'll run more tests today. > > In 100 runs, none of these tests failed. > > (However in 3 cases, reftest-2 failed sometime after running those tests: > the webgl tests passed, dozens of additional tests ran and passed, and then > the harness lost connectivity to the device. I don't know if this is related > to the webgl tests -- I suspect not.) Awesome, let's keep these enabled. Something more subtle is happening here. Let's try DEBUG builds with MOZ_GL_DEBUG. I'm willing to r+ skipping some tests, but I'm not ready to scatter-shot-disable our way to victory.

I ran the tests on a DEBUG build with MOZ_GL_DEBUG defined over the weekend. The frequency of intermittent crashes was significantly reduced compared to release builds, but I finally got 2 crashes, both of which were in eglCreatePbufferSurface -- like bug 948406. I re-tested on Cedar with the patch from bug 948406 applied. The frequency of crashing seems to be reduced: https://tbpl.mozilla.org/?tree=Cedar&showall=1&rev=f392dbd98567 but still occurs: https://tbpl.mozilla.org/php/getParsedLog.php?id=32042649&tree=Cedar Also, sometimes webgl-color-test fails without crashing: https://tbpl.mozilla.org/php/getParsedLog.php?id=32043049&tree=Cedar 13:17:43 INFO - SET PREFERENCE pref(webgl.force-enabled,true) 13:17:43 INFO - SET PREFERENCE pref(webgl.prefer-16bpp,true) 13:17:43 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8856/tests/content/canvas/test/reftest/wrapper.html?colors.png | 1269 / 3832 (33%) 13:17:43 INFO - REFTEST INFO | START http://10.0.2.2:8856/tests/content/canvas/test/reftest/wrapper.html?colors.png 13:17:43 INFO - REFTEST INFO | [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts 13:17:43 INFO - REFTEST INFO | Initializing canvas snapshot 13:17:43 INFO - REFTEST INFO | DoDrawWindow 0,0,800,1000 13:17:43 INFO - REFTEST INFO | [CONTENT] RecordResult fired 13:17:43 INFO - REFTEST INFO | RecordResult fired 13:17:43 INFO - REFTEST TEST-UNEXPECTED-FAIL | http://10.0.2.2:8856/tests/content/canvas/test/reftest/webgl-color-test.html?16bpp | image comparison (==), max difference: 255, number of differing pixels: 1

(In reply to Jeff Gilbert [:jgilbert] from comment #13) > I also hadn't realized before, but the push you linked to only has an OPT > build run, no DEBUG run. DEBUG runs are so ridiculously more important, > because we actually get assert coverage. It's entirely possible that the > runs here would gives us an easy-to-identify assert if run in DEBUG mode, > since we assert a lot of things. Don't add OPT without adding DEBUG. > > If we have to pick just one, only do DEBUG builds! We run Opt tests because we ship Opt. The scope of my project is Android x86 Opt. I understand the sentiment here, but it is unlikely that an attempt will be made to run any Android Debug configuration on tbpl until bug 866795 (or the more focused bug 940068) is resolved. > If the emulator is simple to compile and setup, I should be able to get to > this at some point, though it's got a lot of competition for priority. It's > quicker if you try a DEBUG build, and run with an env var of MOZ_GL_DEBUG=1, > and see if the crash stacks improve. I would not recommend trying to reproduce these results on a local computer -- better to do that on the same hardware that is demonstrating the problem. Currently you would need to work on Cedar (but only Opt builds are available there), or request a loan of an ix machine. I will try to reproduce the crash with a new Debug build, but given my experience so far, I am not optimistic.

(In reply to Geoff Brown [:gbrown] from comment #15) > I will try to reproduce the crash with a new Debug build, but given my > experience so far, I am not optimistic. I tried a couple of times, but could not reproduce the crash. Tests for Android 2.3 became a higher priority, so I have not been working on the Android x86 tests lately. These crashes have persisted though, and can be seen on Ash and Cedar. As luck would have it, we can now see that there is a very similar crash affecting the same tests on Android 2.3 -- bug 1000538!

See https://bugzilla.mozilla.org/show_bug.cgi?id=1000538#c13.