(From bug #481683 comment 3; it looks like the code is unchanged from then so i'm logging it separately.) There's probably an OOM handling bug in ExpandHeapLocked, near the end. Here's the code: if (newRegion == NULL) { // Ugh, FUBAR. return false; } This code is probably wrong. All earlier code in this function makes a point of unreserving / decommitting if something fails and we have to back out. Here we return if we can't allocate the memory for a Region, but we're leaving behind a new block array that points to heap blocks for the new region, but no region will map to those blocks. At a minimum this is a storage leak (large allocation fails but the client handles the error return from PleaseAlloc, and now the memory in the blocks array is sitting unused). Might be better to try to allocate the Region object before committing the new blocks array and deleting the old one; this would allow the allocating code to back out properly.

Another case exists in GCHeap::RemoveBlock() // we're removing a region so re-allocate the blocks w/o the blocks for this region HeapBlock *newBlocks = new HeapBlock[newBlocksLen]; In fact there are a few more 'new XXX' calls in this file that do not handle OOM. This bug should include those calls and any others in the GC.

I see multiple phases of work here: 1) call Abort when these allocations fail 2) don't rely on system memory but use the memory GCHeap gets from the OS (tricky) 3) don't require one huge allocation but use an impl with the largest allocation being a page

This should be fixed after 488738 if its still a bug after that change lands