Tested on m-c revision 8ea5c983743f: During OOM testing I got the following crash: Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () #0 0x0000000000000000 in ?? () #1 0x00002aaaad244255 in nsQueryInterfaceWithError::operator() (this=0x7fffffffa820, aIID=<optimized out>, answer=<optimized out>) at /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/xpcom/build/nsCOMPtr.cpp:64 #2 0x00002aaaac8d0891 in assign_from_qi_with_error (aIID=..., qi=..., this=0x7fffffffa838) at ../../../dist/include/nsCOMPtr.h:1182 #3 nsCOMPtr (qi=..., this=0x7fffffffa838) at ../../../dist/include/nsCOMPtr.h:591 #4 AdoptNodeIntoOwnerDoc (aParent=0x27bbe10, aNode=0x2d650f0) at /srv/repos/browser/mozilla-central/content/base/src/nsGenericElement.cpp:3719 #5 0x00002aaaac8d0ae8 in nsINode::doInsertChildAt (this=0x27bbe10, aKid=0x2d650f0, aIndex=1, aNotify=false, aChildArray=...) at /srv/repos/browser/mozilla-central/content/base/src/nsGenericElement.cpp:3752 #6 0x00002aaaacbfad6d in nsHtml5TreeOperation::AppendToDocument (this=<optimized out>, aNode=0x2d650f0, aBuilder=<optimized out>) at /srv/repos/browser/mozilla-central/parser/html/nsHtml5TreeOperation.cpp:244 #7 0x00002aaaacbfb27a in nsHtml5TreeOperation::Perform (this=0x1ddc170, aBuilder=0x27c5440, aScriptElement=0x7fffffffae78) at /srv/repos/browser/mozilla-central/parser/html/nsHtml5TreeOperation.cpp:329 The failing allocation is: #0 /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/dist/bin/libmozalloc.so(moz_malloc+0x5f) #1 nsVoidArray::SizeTo(int) at objdir-ff-gcc64dbg/xpcom/build/nsVoidArray.cpp:231 #2 nsVoidArray::InsertElementAt(void*, int) at objdir-ff-gcc64dbg/xpcom/build/nsVoidArray.cpp:457 #3 nsCOMArray_base::InsertObjectAt(nsISupports*, int) at objdir-ff-gcc64dbg/xpcom/build/nsCOMArray.cpp:83 #4 nsCOMPtr<nsIAtom>::get() const at objdir-ff-gcc64dbg/parser/html/../../dist/include/nsCOMPtr.h:790 #5 nsHtml5TreeOpExecutor::RunFlushLoop() at parser/html/nsHtml5TreeOpExecutor.cpp:530 #6 nsHtml5ExecutorReflusher::Run() at parser/html/nsHtml5TreeOpExecutor.cpp:97 #7 nsThread::ProcessNextEvent(bool, bool*) at xpcom/threads/nsThread.cpp:657 #8 NS_ProcessNextEvent_P(nsIThread*, bool) at objdir-ff-gcc64dbg/xpcom/build/nsThreadUtils.cpp:245 #9 mozilla::ipc::MessagePump::Run(base::MessagePump::Delegate*) at ipc/glue/MessagePump.cpp:111 #10 MessageLoop::RunInternal() at ipc/chromium/src/base/message_loop.cc:209 #11 ~AutoRunState at ipc/chromium/src/base/message_loop.cc:487 #12 nsBaseAppShell::Run() at widget/xpwidgets/nsBaseAppShell.cpp:191 #13 nsAppStartup::Run() at toolkit/components/startup/nsAppStartup.cpp:296 #14 /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/dist/bin/libxul.so(XRE_main+0x3a17) #15 /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/dist/bin/firefox-bin() I'm making this s-s because I don't know exactly where the NULL address being executed originates from. If this is a safe pointer (e.g. always initialized to NULL), then feel free to unhide this.

Is there any useful testcase for this? Or do you just have to run in a debugger, set a breakpoint on the allocation and manually return null? Since this is a nsCOMArray of nsIAtoms it's likely that this has to be null and can't end up as other bogus values in this case, but Henri would be the expert here.

(In reply to Daniel Veditz [:dveditz] from comment #1) > Is there any useful testcase for this? Or do you just have to run in a > debugger, set a breakpoint on the allocation and manually return null? There is no testcase as these issues are found using my OOM testing patch + automation around it. Using GDB could certainly work, another way would be to use my patch,finding the right allocation from the patch output (as the addresses are build dependent) and telling Firefox to simulate OOM there (functionality provided by the patch).

Guessing Henri due to nsHtml5ExecutorReflusher::Run in the stack. Probably not a security bug since null comptrs are either initialized that way or release()d and guaranteed to be null.

(In reply to Christian Holler (:decoder) from comment #0) > Tested on m-c revision 8ea5c983743f: > #5 nsHtml5TreeOpExecutor::RunFlushLoop() at > parser/html/nsHtml5TreeOpExecutor.cpp:530 > #6 nsHtml5ExecutorReflusher::Run() at > parser/html/nsHtml5TreeOpExecutor.cpp:97 Are you sure of the revision? These line numbers don't make sense for that revision. do_GetAtom is supposed to use infallible malloc, right?

(In reply to Henri Sivonen (:hsivonen) from comment #4) > (In reply to Christian Holler (:decoder) from comment #0) > > Tested on m-c revision 8ea5c983743f: > > > #5 nsHtml5TreeOpExecutor::RunFlushLoop() at > > parser/html/nsHtml5TreeOpExecutor.cpp:530 > > #6 nsHtml5ExecutorReflusher::Run() at > > parser/html/nsHtml5TreeOpExecutor.cpp:97 > > Are you sure of the revision? These line numbers don't make sense for that > revision. The build is debug+optimized, so there might be inlining. But it could be that the rev is wrong. I'm currently rebuilding and will try to repro on another revision and repost here then.

Duping per bug 737875 comment 11, please reopen if that's wrong.