Caught this stack while trying to reproduce bug 586909. Crashed in a similar manner. Graphical corruption, sluggish response to commands then the browser just deadlocks. 003dc728 6bb54a77 263a5190 6bba232d 00004000 xul!JSC::ExecutablePool::systemAlloc+0x3bbc70 003dc730 6bba232d 00004000 003dc808 00000000 xul!JSC::ExecutablePool::create+0x17 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\assembler\jit\executableallocator.h @ 125] 003dc748 6bba277e 006111f8 0000005d 3b35aa60 xul!JSC::ExecutableAllocator::poolForSize+0x4d [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\assembler\jit\executableallocator.h @ 199] 003dc770 6bba8a90 006111f8 3a8f3808 3a8f3800 xul!JSC::Yarr::RegexGenerator::compile+0x1e [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\yarr\yarr\regexjit.cpp @ 1469] 003dc938 6bbaa5f6 006111f8 3b35aa60 003dc95c xul!JSC::Yarr::jitCompileRegex+0x130 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\yarr\yarr\regexjit.cpp @ 1504] 003dc964 6bbaa631 3b35aa60 1bbf1000 41928eb8 xul!js::RegExp::compileHelper+0x46 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsregexpinlines.h @ 176] 003dc998 6bbaa6a2 3b35aa60 3b35aa60 003dc9e0 xul!js::RegExp::compile+0x21 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsregexpinlines.h @ 209] 003dc9a8 6bbaffbc 3b35aa60 00000002 003dd118 xul!js::RegExp::create+0x62 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsregexpinlines.h @ 140] 003dc9e0 6bc77513 1462a900 00000002 003dd148 xul!js::RegExp::createObject+0x4c [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsregexpinlines.h @ 158] 003dca48 6bc75a32 00000020 00000000 00000000 xul!js::Parser::primaryExpr+0x1433 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsparse.cpp @ 8283] 003dca7c 6bc75366 00000001 00000000 003dd118 xul!js::Parser::memberExpr+0x82 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsparse.cpp @ 6872] 003dca98 6bc73ced 003dd148 003dd118 003dcac8 xul!js::Parser::unaryExpr+0xa6 [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsparse.cpp @ 6245] 003dd148 006ca0b0 0000001d 00000194 00000029 xul!js::Parser::assignExpr+0x9d [e:\builds\moz2_slave\mozilla-central-win32-nightly\build\js\src\jsparse.cpp @ 5860] WARNING: Frame IP not in any known module. Following frames may be wrong. 003dd14c 00000000 00000194 00000029 00000196 0x6ca0b0

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b4pre) Gecko/20100817 Minefield/4.0b4pre

The ExecutablePool is still tied into the WebKit CRASH() macro when systemAlloc fails, which is what does the write-to-0xbadbeef dance. systemAlloc and systemFree call into VirtualAlloc/VirtualFree in ExecutableAllocatorWin.cpp, and systemAlloc is not returning successfully. Kevin, what were the steps you used to reproduce?

The same as bug 586909, enable d2d on windows and then just browse around till the Firefox window starts showing signs of graphical corruption. From that point it is just a matter of time till you crash. Might need an nVidia graphics card and the latest drivers. My stacks from this have been somewhat varied though this has shown up three times. If it would be helpful I have a dump file from the crash with memory state that I can arrange to get to you.

I've caught the same crash a few times, also while trying to repro bug 586909. I open a number of stories and comments on digg.com (20-30 tabs) scroll through them once, close them, undo close the last 10 and then open up another 10 or so stories. Scrolling through the tabs will cause the back/forward button to disappear and then ff will crash. STR are not solid, sometimes it takes more tab opening, closing and scrolling to trigger the crash. Breakpad fires but does not produce a usable crash report.

So the crash reporter wasn't catching these crashes either? Looks like we might have a problem with OOM errors.

I'm having a heck of a time reproing this on my box: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b5pre) Gecko/20100819 Minefield/4.0b5pre Maybe I'm doing something silly. I followed the instructions in this article to turn on d2d ( http://www.basschouten.com/blog1.php/2010/03/02/presenting-direct2d-hardware-acceleratio ) and followed Scott's instructions in comment 4. It sounds like it could be a general OOM problem unrelated to the new regular expression engine, since the thing that's failing is just a virtual alloc -- is anything other than the regular expression engine suspect?

This does not require direct write, that was a red herring. On my computer just browsing for a while will cause this to start appearing ~1h. Leaving the browser up over night is sure to crash. Opening js heavy sites such as gmail or digg can take a browser that is on the tipping edge of crashing, sluggish ui, black areas where the ui should be, disappearing ui, and cause it to fully crash.

Kevin, are you still experiencing this? (You can still repro despite the fix in bug 589809?) If so I'll switch over to Windows full-time because I can't seem to repro with short spurts of browsing activity.

I saw a crash on TM tinderbox that might be the same as this, except with alloc rather than systemAlloc. Same, or different? http://tinderbox.mozilla.org/showlog.cgi?log=TraceMonkey/1282795751.1282796275.31366.gz&fulltext=1#err1

Resolving as WFM -- feel free to reopen if it crops up again.