Marc Schifer [mschifer] Reporter
	Description • 11 years ago

Demo crashes while loading file on Windows 7.

https://crash-stats.mozilla.com/report/index/73ded81e-33e3-401f-a286-29b932130930


Processor: AMD Athlon II X4 645 3.10 GHz
Graphics:  ATI Radeon HD 4200 

A 	mozalloc.dll 	mozalloc_abort(char const * const) 	memory/mozalloc/mozalloc_abort.cpp
1 	mozalloc.dll 	mozalloc_handle_oom(unsigned int) 	memory/mozalloc/mozalloc_oom.cpp
2 	mozalloc.dll 	moz_xmalloc 	memory/mozalloc/mozalloc.cpp
3 	xul.dll 	`anonymous namespace'::AddHelper::DoDatabaseWork(mozIStorageConnection *) 	dom/indexeddb/IDBObjectStore.cpp
4 	xul.dll 	mozilla::dom::indexedDB::AsyncConnectionHelper::Run() 	dom/indexeddb/AsyncConnectionHelper.cpp
5 	xul.dll 	mozilla::dom::indexedDB::TransactionThreadPool::TransactionQueue::Run() 	dom/indexeddb/TransactionThreadPool.cpp
6 	xul.dll 	nsThreadPool::Run() 	xpcom/threads/nsThreadPool.cpp
7 	xul.dll 	nsThread::ProcessNextEvent(bool,bool *) 	xpcom/threads/nsThread.cpp
8 	xul.dll 	nsThread::ThreadFunc(void *) 	xpcom/threads/nsThread.cpp
9 	nss3.dll 	_PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c
10 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c
11 	msvcr100.dll 	_callthreadstartex 	f:\\dd\\vctools\\crt_bld\\self_x86\\crt\\src\\threadex.c
12 	msvcr100.dll 	_threadstartex 	f:\\dd\\vctools\\crt_bld\\self_x86\\crt\\src\\threadex.c
13 	kernel32.dll 	kernel32.dll@0x1336a 	
14 	ntdll.dll 	ntdll.dll@0x39f72 	
15 	ntdll.dll 	ntdll.dll@0x39f4

	Wayne Mery (:wsmwk)
	Comment 1 • 11 years ago

URL to demo?

	Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)
	Comment 2 • 11 years ago

Not critical; this is just a generic OOM crash.  This demo uses a lot of memory, and is known to OOM on 32-bit Firefox (e.g. all of our standard Win32 builds).  We usually use 64-bit builds.

	Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)
	Comment 3 • 11 years ago

Luke, IndexedDB stuff might be the cause of your mystery 500MB, now that I think about it.. we should analyze what large chunks of memory are in use here (could probably just break in the allocator on >100MB allocations?)

	Luke Wagner [:luke]
	Comment 4 • 11 years ago

Wow, OOM Allocation Size: 425352344.  I'm guessing this is the
  nsAutoArrayPtr<char> compressed(new char[compressedLength]);
line?

Ben: should we switch this to fallible malloc here given the potentially enormous allocation size?  That would at least give a gracefull OOM message.

Vlad: from a quick scan, I'm pretty sure this is this allocation is stack-scoped.  (Is that right Ben?)  In that case, it probably wasn't the 500MB hanging around until forced GC that I was seeing in about:memory earlier.

Jukka/Alon: looking at this code I think we have like 3 copies of the FS in memory: (1) the original ArrayBuffer object, (2) the uncompressed copy in mCloneWriteInfo.mCloneBuffer created by structured clone, (3) the compressed storage (which is overallocated, so still quite large).  Is this still what we're doing in Emscripten?

njn: here's a nice example of a very temporal spike-causing allocation that about:memory-snapshot-on-oom would be able to catch.

	Alon Zakai (:azakai)
	Comment 5 • 11 years ago

Emscripten loads the data file in an XHR, then does a .set() to copy it into the singleton typed array, and then discards the reference to the xhr and its ArrayBuffer. I am not sure how this relates to the stuff you mention, in particular I don't know what "compression" means here?

	Luke Wagner [:luke]
	Comment 6 • 11 years ago

(In reply to Alon Zakai (:azakai) from comment #5)
This OOM is when something large is getting saved in IndexedDB; I assumed it was the FS, but perhaps its something else?

	Alon Zakai (:azakai)
	Comment 7 • 11 years ago

Oh ok, if IndexedDB is involved then I guess this is using emscripten's IndexedDB caching option. cc'ing ack who wrote that.

	Alon Zakai (:azakai)
	Comment 8 • 11 years ago

Note that the only use of IndexedDB is to be a better cache than the normal disk cache. If it causes OOM problems, we can just not use it, and entrust our fate to the normal cache.

	Ben Turner (not reading bugmail, use the needinfo flag!)
	Comment 9 • 11 years ago

We can fix that to be a fallible memory allocation quite easily (maybe only in main process).

	Kyle Huey (Exited; not receiving bugmail, old account, do not use)
	Comment 10 • 11 years ago

The actual crash is fixed by bug 902909.

Running out of memory ...

	Luke Wagner [:luke]
	Comment 11 • 11 years ago

Ok, guess the rest was just related discussion, so we can dup.