Tested on m-c revision 8ea5c983743f: During OOM testing I got the following crash: Program received signal SIGSEGV, Segmentation fault. 0x00002aaaad249742 in PL_DHashTableEnumerate (table=0x26c5610, etor= 0x2aaaad043b46 <nsBaseHashtable<nsTrimInt64HashKey, nsTArray<nsRefPtr<nsNavHistoryFolderResultNode>, nsTArrayDefaultAllocator>*, nsTArray<nsRefPtr<nsNavHistoryFolderResultNode>, nsTArrayDefaultAllocator>*>::s_EnumStub(PLDHashTable*, PLDHashEntryHdr*, unsigned int, void*)>, arg=0x7fffffff3090) at /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/xpcom/build/pldhash.cpp:743 743 INCREMENT_RECURSION_LEVEL(table); (gdb) bt #0 0x00002aaaad249742 in PL_DHashTableEnumerate (table=0x26c5610, etor=0x2aaaad043b46 <nsBaseHashtable<nsTrimInt64HashKey, nsTArray<nsRefPtr<nsNavHistoryFolderResultNode>, nsTArrayDefaultAllocator>*, nsTArray<nsRefPtr<nsNavHistoryFolderResultNode>, nsTArrayDefaultAllocator>*>::s_EnumStub(PLDHashTable*, PLDHashEntryHdr*, unsigned int, void*)>, arg=0x7fffffff3090) at /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/xpcom/build/pldhash.cpp:743 #1 0x00002aaaad0471fe in nsBaseHashtable<nsTrimInt64HashKey, nsTArray<nsRefPtr<nsNavHistoryFolderResultNode>, nsTArrayDefaultAllocator>*, nsTArray<nsRefPtr<nsNavHistoryFolderResultNode>, nsTArrayDefaultAllocator>*>::Enumerate (this=0x26c5610, enumFunc=0x2aaaad047250 <TraverseBookmarkFolderObservers(nsTrimInt64HashKey::KeyType, nsNavHistoryResult::FolderObserverList*&, void*)>, userArg=0x7fffffff31b8) at ../../../dist/include/nsBaseHashtable.h:239 #2 0x00002aaaad0512a6 in nsNavHistoryResult::cycleCollection::Traverse (this=<optimized out>, p=<optimized out>, cb=...) at /srv/repos/browser/mozilla-central/toolkit/components/places/nsNavHistoryResult.cpp:4266 #3 0x00002aaaad298b8d in GCGraphBuilder::Traverse (this=<optimized out>, aPtrInfo=0x1ce6558) at /srv/repos/browser/mozilla-central/xpcom/base/nsCycleCollector.cpp:1929 #4 0x00002aaaad298c10 in nsCycleCollector::MarkRoots (this=0x4e8ad0, builder=...) at /srv/repos/browser/mozilla-central/xpcom/base/nsCycleCollector.cpp:2222 #5 0x00002aaaad299336 in BeginCollection (aListener=0x0, this=0x4e8ad0) at /srv/repos/browser/mozilla-central/xpcom/base/nsCycleCollector.cpp:3203 #6 nsCycleCollector::BeginCollection (this=0x4e8ad0, aListener=0x0) at /srv/repos/browser/mozilla-central/xpcom/base/nsCycleCollector.cpp:3150 #7 0x00002aaaad29a764 in nsCycleCollector::Collect (this=0x4e8ad0, aResults=<optimized out>, aTryCollections=5, aListener=0x0) at /srv/repos/browser/mozilla-central/xpcom/base/nsCycleCollector.cpp:3139 #8 0x00002aaaad29a9c4 in nsCycleCollector::Shutdown (this=0x4e8ad0) at /srv/repos/browser/mozilla-central/xpcom/base/nsCycleCollector.cpp:3323 #9 0x00002aaaad29ab55 in nsCycleCollector_shutdown () at /srv/repos/browser/mozilla-central/xpcom/base/nsCycleCollector.cpp:4087 #10 0x00002aaaad24e5c8 in mozilla::ShutdownXPCOM (servMgr=<optimized out>) at /srv/repos/browser/mozilla-central/xpcom/build/nsXPComInit.cpp:669 #11 0x00002aaaac49c83f in ScopedXPCOMStartup::~ScopedXPCOMStartup (this=0x7fffffffb2e0, __in_chrg=<optimized out>) at /srv/repos/browser/mozilla-central/toolkit/xre/nsAppRunner.cpp:1123 #12 0x00002aaaac4a32ad in XRE_main (argc=<optimized out>, argv=<optimized out>, aAppData=<optimized out>) at /srv/repos/browser/mozilla-central/toolkit/xre/nsAppRunner.cpp:3327 #13 0x000000000040187c in do_main (argv=0x7fffffffdae8, argc=5) at /srv/repos/browser/mozilla-central/browser/app/nsBrowserApp.cpp:190 #14 main (argc=<optimized out>, argv=<optimized out>) at /srv/repos/browser/mozilla-central/browser/app/nsBrowserApp.cpp:277 (gdb) x /1i $pc => 0x2aaaad249742 <PL_DHashTableEnumerate(PLDHashTable*, PLDHashEnumerator, void*)+48>: mov (%rax),%edx (gdb) info register rax rax 0x72696465722f2f3a 8244230979748572986 The address in rax is random, it could originate from uninitialized memory. There is no single allocation failure trace for this, because for this test, all calls to fallible allocators which go through nsCOMArray_base::InsertObjectAt(nsISupports*, int) /srv/repos/browser/mozilla-central/objdir-ff-gcc64dbg/xpcom/build/nsCOMArray.cpp:83 will fail, causing lots of allocation failure traces, of which most seem to be non-critical. If the crash trace isn't enough, I can record all allocation failure traces that the test produces and we can triage which of those are irrelevant to find the correct (fault inducing) one. This could of course also be automated but my toolchain isnt that advanced yet.

This is a hard bug to know how to handle or where to send. We should probably just unhide this bug and hope someone gets inspired to dig in.

Getting the stacks for the failed allocations would be a good start here, w/o that I don't think fixing this particular crash is worth it, we'd likely only fix one aspect of it, but not the underlying problem.

I'll try to record all failed allocations then and attach them here :)

Here's the full logfile of me reproducing the issue. Every trace in there that starts with "***** MOZ_OOM Testing: Function matched, returning NULL *****" is actually an allocation trace that is answered with NULL for simulating OOM. I piped this through a script to symbolize those traces. Judging from the symbolized trace, I'd assume that the last one is most relevant, as it is also immediately followed by ###!!! ASSERTION: nsBaseHashtable was not initialized properly.: 'this->mTable.entrySize', file ../../../dist/include/nsBaseHashtable.h, line 234

I'm not sure this bug is going anywhere useful without a testcase :-(

(In reply to Daniel Veditz [:dveditz] from comment #5) > I'm not sure this bug is going anywhere useful without a testcase :-( I disagree. I provided allocation traces in comment 4 which usually help to find the cause of the problem (at least they have helped in the other bugs I filed). If this is unlikely to be a risk, then I'd say we open the bug so more people can look at it.

Unhiding based on IRC conversation. There is no test in this bug which could be anywhere near useful to exploit something here.

Is this still relevant?

Closing this as incomplete for now. I think if someone would have actually looked at the traces, then we could have found this bug, but since nobody is going to do that, we can as well close it. It might also be that this code is gone by now.