This bug was filed from the Socorro interface and is report bp-6bceca79-3a53-403e-ad69-a95142140521. ============================================================= This signature appeared starting 5/21 and it is happening mostly on Win7, followed by Win8.1 and Win8. There are no comments in the reports. Currently in the top 25 crashers list on nightly. More reports can be found here: https://crash-stats.mozilla.com/report/list?product=Firefox&signature=mozalloc_abort%28char+const%2A+const%29+%7C+NS_DebugBreak+%7C+mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3ADebugAbort%28char+const%2A%2C+int%2C+char+const%2A%2C+char+const%2A%2C+bool%29+%7C+mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3A%7EMessageChannel%28%29+%7C+mozilla%3A%3Alayers%3A%3APImageBridgeParent%3A%3A%7EPImageBridgeParent%28%29 0 mozalloc.dll mozalloc_abort(char const * const) memory/mozalloc/mozalloc_abort.cpp 1 xul.dll NS_DebugBreak xpcom/base/nsDebugImpl.cpp 2 xul.dll mozilla::ipc::MessageChannel::DebugAbort(char const *,int,char const *,char const *,bool) ipc/glue/MessageChannel.cpp 3 xul.dll mozilla::ipc::MessageChannel::~MessageChannel() ipc/glue/MessageChannel.cpp 4 xul.dll mozilla::layers::PImageBridgeParent::~PImageBridgeParent() obj-firefox/ipc/ipdl/PImageBridgeParent.cpp 5 xul.dll mozilla::layers::ImageBridgeParent::~ImageBridgeParent() gfx/layers/ipc/ImageBridgeParent.cpp 6 xul.dll mozilla::layers::ImageBridgeParent::`vector deleting destructor'(unsigned int) 7 xul.dll mozilla::AtomicRefCountedWithFinalize<mozilla::layers::ISurfaceAllocator>::Release() obj-firefox/dist/include/mozilla/layers/AtomicRefCountedWithFinalize.h 8 xul.dll mozilla::layers::DeleteImageBridgeSync gfx/layers/ipc/ImageBridgeChild.cpp 9 xul.dll RunnableFunction<void (*)(mozilla::layers::ImageBridgeChild *,mozilla::layers::ImageBridgeParent *),Tuple2<mozilla::layers::ImageBridgeChild *,mozilla::layers::ImageBridgeParent *> >::Run() ipc/chromium/src/base/task.h 10 xul.dll MessageLoop::RunTask(Task *) ipc/chromium/src/base/message_loop.cc 11 xul.dll MessageLoop::DeferOrRunPendingTask(MessageLoop::PendingTask const &) ipc/chromium/src/base/message_loop.cc 12 xul.dll MessageLoop::DoWork() ipc/chromium/src/base/message_loop.cc 13 xul.dll base::MessagePumpDefault::Run(base::MessagePump::Delegate *) ipc/chromium/src/base/message_pump_default.cc 14 xul.dll MessageLoop::RunHandler() ipc/chromium/src/base/message_loop.cc 15 xul.dll MessageLoop::Run() ipc/chromium/src/base/message_loop.cc 16 xul.dll base::Thread::ThreadMain() ipc/chromium/src/base/thread.cc 17 xul.dll `anonymous namespace'::ThreadFunc(void *) ipc/chromium/src/base/platform_thread_win.cc 18 kernel32.dll kernel32.dll@0xb729

Over in bug 924622 comment 35, Benjamin says: > See also bug 959080 where this is seen on mac as well, so it seems to affect every platform with OMTC enabled. Given the timing, I guess OMTC on Windows is now part of this club.

I guess I should add, the abort message is: ###!!! ABORT: mismatched CxxStackFrame ctor/dtors: file c:\builds\moz2_slave\m-cen-w32-ntly-000000000000000\build\ipc\glue\MessageChannel.cpp, line 1732

This crashed in mozmill testrun too, after: http://hg.mozilla.org/qa/mozmill-tests/file/default/firefox/tests/functional/testToolbar/testHomeButton.js https://crash-stats.mozilla.com/report/index/c76a6fa7-e34f-43f1-9a68-3061e2140610

This is the jenkins log, though I couldn't reproduce the crash, I have seen the same log we had before the crash: > ###!!! [Child][DispatchAsyncMessage] Error: Route error: message sent to unknown actor ID

I got this on Nexus 4 (android 4.4 ) after playing a video fro like 5s

Looking at the following URL shows that the crashes have been started on May 21st: https://crash-stats.mozilla.com/report/list?product=Firefox&range_unit=days&range_value=28&signature=mozalloc_abort%28char+const*+const%29+|+NS_DebugBreak+|+mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3ADebugAbort%28char+const*%2C+int%2C+char+const*%2C+char+const*%2C+bool%29+|+mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3A~MessageChannel%28%29+|+mozilla%3A%3Alayers%3A%3APImageBridgeParent%3A%3A~PImageBridgeParent%28%29#tab-reports Maybe the pushlog helps us a bit here: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=cb9f34f73ebe&tochange=9d8d16695f6a (In reply to David Major [:dmajor] (UTC+12) from comment #2) > > See also bug 959080 where this is seen on mac as well, so it seems to affect every platform with OMTC enabled. > > Given the timing, I guess OMTC on Windows is now part of this club. Well, good tip that is the problem. Bug 899785 landed that day on mozilla-central.

(In reply to Henrik Skupin (:whimboo) from comment #7) > Looking at the following URL shows that the crashes have been started on May > 21st: > > https://crash-stats.mozilla.com/report/ > list?product=Firefox&range_unit=days&range_value=28&signature=mozalloc_abort% > 28char+const*+const%29+|+NS_DebugBreak+|+mozilla%3A%3Aipc%3A%3AMessageChannel > %3A%3ADebugAbort%28char+const*%2C+int%2C+char+const*%2C+char+const*%2C+bool%2 > 9+|+mozilla%3A%3Aipc%3A%3AMessageChannel%3A%3A~MessageChannel%28%29+|+mozilla > %3A%3Alayers%3A%3APImageBridgeParent%3A%3A~PImageBridgeParent%28%29#tab- > reports > > Maybe the pushlog helps us a bit here: > http://hg.mozilla.org/mozilla-central/ > pushloghtml?fromchange=cb9f34f73ebe&tochange=9d8d16695f6a > > (In reply to David Major [:dmajor] (UTC+12) from comment #2) > > > See also bug 959080 where this is seen on mac as well, so it seems to affect every platform with OMTC enabled. > > > > Given the timing, I guess OMTC on Windows is now part of this club. > > Well, good tip that is the problem. Bug 899785 landed that day on > mozilla-central. So we get crashes about 25-30% of the time when running our testruns on XP nodes. I'v done the regression range & it looks like this is the one: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=a6a457fe2a2a&tochange=7a8d540c1cb2 *** 1402826524 - 15.06.2014 13:26:00 *** 500 testruns - NO CRASH AT ALL *** 1402849754 - 15.06.2014 17:14:00 *** CRASHES 17/100 CRASHES

(In reply to daniel.gherasim from comment #8) > I'v done the regression range & it looks like this is the one: > http://hg.mozilla.org/mozilla-central/ > pushloghtml?fromchange=a6a457fe2a2a&tochange=7a8d540c1cb2 Thanks Daniel. But this is not fully done yet. We have too many changesets here, so please continue to find the single changeset, which causes this crash to appear.

(In reply to Henrik Skupin (:whimboo) from comment #9) > (In reply to daniel.gherasim from comment #8) > > I'v done the regression range & it looks like this is the one: > > http://hg.mozilla.org/mozilla-central/ > > pushloghtml?fromchange=a6a457fe2a2a&tochange=7a8d540c1cb2 > > Thanks Daniel. But this is not fully done yet. We have too many changesets > here, so please continue to find the single changeset, which causes this > crash to appear. Done: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=f903c0f48969&tochange=bb0c777636e9

So we're seeing crashes a lot on XP after bug 774388 landed when running our tests. Jacob, any idea what cause this ?

(In reply to David Major [:dmajor] (Away until June 23) from comment #3) > I guess I should add, the abort message is: > ###!!! ABORT: mismatched CxxStackFrame ctor/dtors: file > c:\builds\moz2_slave\m-cen-w32-ntly- > 000000000000000\build\ipc\glue\MessageChannel.cpp, line 1732 This assert, and the call stack in comment 0, are the same as Bug 924622, so this looks like the same bug, and the changeset identified in comment 10 would have merely changed the resulting signature (?). Do you know specifically if the frequency of this crash increased with this changeset, compared to what was seen on bug 924622 ?

Oh or is the new thing here that this is seen on Windows whereas bug 924622 was on Android?

Here's the question: is the changeset identified in comment 10 the first changeset where windows is crashy, or the first changeset where windows crashes *with that particular signature* ? I'm asking because it is not surprising that OMTC-on-windows would make until-now-Android-only OMTC crashes now also happen on Windows. So I suppose that two different things could have happened here: 1) OMTC-on-windows made bug 924622 now also affect Windows, and 2) the changeset identified in comment 10 could have changed its signature. Is that theory validated or invalidated by your experimental data?

Hi Jacob, this happens on Windowx XP with this signature: [@ EMPTY: no crashing thread identified; ERROR_NO_THREAD_LIST]. I ran 500 tests with the build before that changeset & it didn't crash at all. With the build after changeset was applied crashed 15/100. *** 1402708216 DATE *** 500 RUNS / NO CRASH *** URL *** 1402711754 DATE *** 100 RUNS / 15 crashes *** URL Running a simple dummy test will cause the crash (open/close firefox). Details from 'about:support' on the machine I ran the tests (VM XP x86): Graphics Adapter Drivers vga framebuf vga256 vga64k Adapter RAM Unknown Device ID 0x0000 Direct2D Enabled Blocked for your graphics card because of unresolved driver issues. DirectWrite Enabled false (0.0.0.0) GPU #2 Active false GPU Accelerated Windows 0/1 Basic Blocked for your graphics card because of unresolved driver issues. Vendor ID 0x0000 WebGL Renderer Blocked for your graphics card because of unresolved driver issues. windowLayerManagerRemote false AzureCanvasBackend skia AzureContentBackend cairo AzureFallbackCanvasBackend cairo AzureSkiaAccelerated 0

Interesting. I've run the same test on a dedicated XP machine, and I got no crashes in 300 runs. > Graphics > Adapter Description ATI Radeon 3000 Graphics > Adapter Drivers ati2dvag > Adapter RAM Unknown > Device ID 0x9616 > DirectWrite Enabled false (0.0.0.0) > Driver Date 7-3-2012 > Driver Version 8.970.100.3000 > GPU #2 Active false > GPU Accelerated Windows 1/1 Direct3D 9 (OMTC) > Vendor ID 0x1002 > WebGL Renderer Google Inc. -- ANGLE (ATI Radeon 3000 Graphics Direct3D9 vs_3_0 ps_3_0) > windowLayerManagerRemote true > AzureCanvasBackend skia > AzureContentBackend cairo > AzureFallbackCanvasBackend cairo > AzureSkiaAccelerated 0

Thanks for the testing; I'm currently looking deep into this crash; if after a few days I can't fix it, I'll back out the patch that you've identified to be regressing this.

Maybe this crash only happens on virtual machines run under VMware? Looks like the graphics driver cannot be correctly identified. Just a note here, for all of the VMs we do not have 3D acceleration enabled.

Daniel or Andrei, could one of you please check if the crash also appears when you turn on/off 3D graphics support in VMware for this machine? Also are the VMware tools up-to-date in the VM? If not may an update of them fix it?

(In reply to Henrik Skupin (:whimboo) from comment #18) > Maybe this crash only happens on virtual machines run under VMware? Looks > like the graphics driver cannot be correctly identified. Just a note here, > for all of the VMs we do not have 3D acceleration enabled. I'm using Virtual Box for the local machine on which firefox crashes. (In reply to Henrik Skupin (:whimboo) from comment #19) > Daniel or Andrei, could one of you please check if the crash also appears > when you turn on/off 3D graphics support in VMware for this machine? Also > are the VMware tools up-to-date in the VM? If not may an update of them fix > it? I enabled 3d support on my local VM, it shows that it's enabled in dxdiag but in about:support I got the same configuration. Crash still reproduces. Current configuration: > Adapter Description VirtualBox Graphics Adapter > Adapter Drivers VBoxDisp > Adapter RAM Unknown > Device ID 0xbeef > Direct2D Enabled Blocked for your graphics card because of unresolved driver issues. > DirectWrite Enabled false (0.0.0.0) > Driver Date 12-18-2013 > Driver Version 4.3.6.0 > GPU #2 Active false > GPU Accelerated Windows 0/1 Basic Blocked for your graphics card because of unresolved driver issues. > Vendor ID 0x80ee > WebGL Renderer Blocked for your graphics card because of unresolved driver issues. > windowLayerManagerRemote false > AzureCanvasBackend skia > AzureContentBackend cairo > AzureFallbackCanvasBackend cairo > AzureSkiaAccelerated 0

This has a really high failure rate on our CI Windows XP machines, rendering tests on these machines uneffective.

qablocker is not the keyword you want to use. What you want is actually the qa-automation-blocked whiteboard entry.

Note taken; I am still working full time on resolving this class of issues; this green try push gives hope that fixes could land soon: https://tbpl.mozilla.org/?tree=Try&rev=ef203ea5b300

(In reply to Benoit Jacob [:bjacob] from comment #23) > Note taken; I am still working full time on resolving this class of issues; > this green try push gives hope that fixes could land soon: > https://tbpl.mozilla.org/?tree=Try&rev=ef203ea5b300 I've tested this try build and it seems to work fine, I haven't had any crash with it.

I spoke too soon. I did got a crash, twice (much more infrequent than the original) But this is another another crash: > Firefox 33.0a1 Crash Report [@ EMPTY: no crashing thread identified; ERROR_NO_MINIDUMP_HEADER ] which appears to be bug 711568. This with the try build mentioned in comment 23. Seems to be totally unrelated...

My patches have finally landed and stuck, on mozilla-central. So you could use current TBPL builds or wait for tomorrow's Nightly.

(In reply to Benoit Jacob [:bjacob] from comment #26) > My patches have finally landed and stuck, on mozilla-central. So you could > use current TBPL builds or wait for tomorrow's Nightly. On which bug did that happen? I don't see a comment here or on any of the dependent bugs.

But 774388 - which the present bug blocks. Sorry about insufficient communication :-)

Thanks for the info. Adding it accordingly.

We haven't seen any crashes since bug 774388 landed on our Mozmill daily testruns. This appears to be fixed (for our testcases at least).

Great. Yes, that is what I expected. Let's mark this bug accordingly then. I'm defaulting to FIXED but maybe that should be VERIFIED?

Last reported crash for Firefox 33.0a1 was with 20140707030202. That's the date when the fix on the other bug has been landed. Interesting is 32.0a2 which had the last crash with 20140625004001. Maybe that is something different, given the low volume in crashes too. Flagging as verified fixed, but not sure what to do with status-firefox32.

Marking status32 as unaffected as OMTC has been disabled on 32.