u279076 Reporter
	Description • 10 years ago

This bug was filed from the Socorro interface and is 
report bp-13735765-c09f-4f77-a852-f3e2a2140403.
=============================================================
0 	xul.dll 	nsTArray_base<nsTArrayInfallibleAllocator,nsTArray_CopyWithMemutils>::IncrementLength(unsigned int) 	obj-firefox/dist/include/nsTArray.h
1 	xul.dll 	`anonymous namespace'::WorkerJSRuntime::WorkerJSRuntime(JSRuntime *,mozilla::dom::workers::WorkerPrivate *) 	dom/workers/RuntimeService.cpp
2 	xul.dll 	`anonymous namespace'::WorkerThreadPrimaryRunnable::Run() 	dom/workers/RuntimeService.cpp
3 	xul.dll 	nsThread::ProcessNextEvent(bool,bool *) 	xpcom/threads/nsThread.cpp
4 	xul.dll 	NS_ProcessNextEvent(nsIThread *,bool) 	xpcom/glue/nsThreadUtils.cpp
5 	xul.dll 	mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate *) 	ipc/glue/MessagePump.cpp
6 	xul.dll 	MessageLoop::RunHandler() 	ipc/chromium/src/base/message_loop.cc
7 	xul.dll 	MessageLoop::Run() 	ipc/chromium/src/base/message_loop.cc
8 	xul.dll 	nsThread::ThreadFunc(void *) 	xpcom/threads/nsThread.cpp
9 	nss3.dll 	_PR_NativeRunThread 	nsprpub/pr/src/threads/combined/pruthr.c
10 	nss3.dll 	pr_root 	nsprpub/pr/src/md/windows/w95thred.c
11 	msvcr100.dll 	_callthreadstartex 	f:\dd\vctools\crt_bld\self_x86\crt\src\threadex.c
12 	msvcr100.dll 	_threadstartex 	f:\dd\vctools\crt_bld\self_x86\crt\src\threadex.c
13 	kernel32.dll 	kernel32.dll@0x1495d 	
14 	ntdll.dll 	ntdll.dll@0x498ee 	
15 	ntdll.dll 	ntdll.dll@0x498c4 	

More reports:
https://crash-stats.mozilla.com/report/list?product=Firefox&signature=nsTArray_base%3CnsTArrayInfallibleAllocator%2C+nsTArray_CopyWithMemutils%3E%3A%3AIncrementLength%28unsigned+int%29+%7C+%60anonymous+namespace%27%27%3A%3AWorkerJSRuntime%3A%3AWorkerJSRuntime%28JSRuntime%2A%2C+mozilla%3A%3Adom%3A%3Aworkers%3A%3AWorkerPrivate%2A%29

This crash first showed up on 2014-03-29 following the enabling of GGC in Nightly. There are only three comments so far and both mention crashing PluginContainer.exe. It is currently #18 @ 0.64% on Nightly.

Terrence, could you please look into this?

	Paul Silaghi, QA [:pauly]
	Comment 1 • 10 years ago

Unable to reproduce on http://one-piece.ru/ (extracted from crash comments).
31.0a1 (2014-04-04), win 7 x64

	Terrence Cole [:terrence]
	Comment 2 • 10 years ago

Since bug 992535 landed, the frequency is way down: only a handful a day.

	Rob
	Comment 3 • 10 years ago

I can confirm on WinXP with Nightly bp-9096a7c4-4983-4b5b-956d-c1f142140419 when OOM. My BP's "Crashing Thread" is a close copy of Comment 0 .

Indeed it is quite an "InfallibleAllocator", I did not file because I was OOM and expecting a crash for a while (the UI was toast). While checking my BPs for another Report I looked at this one; still I feel I'm out of the 'Bell Curve' to Report but I was able to trigger it and thought I'd chime in (to confirm and provide a possibly unhelpful BP).

	Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)
	Comment 4 • 10 years ago

This is still showing around 20-40 crashes per day on recent builds. There are 194 crashes in the last 7 days. It's still the #4 top crash for Firefox 31.   I don't see the same pattern of crashes dropping dramatically after bug 992535 landed.

	u279076 Reporter
	Comment 5 • 10 years ago

Thanks for the information, Rob.

Terrence, does Rob's report tell you anything actionable?

	Terrence Cole [:terrence]
	Comment 6 • 10 years ago

Yes, it does. Thank you for the information, Rob!

This almost certainly crashing at the browser's equivalent of CrashAtUnhandlableOOM. We should try to handle this case more gracefully.

Jon, is this likely to just be a shifted signature, or did GGC add to the size of these?

	Ben Turner (not reading bugmail, use the needinfo flag!)
	Comment 7 • 10 years ago

This seems to be just another spot that is likely to crash when we get close to OOM with GGC... Moving to JS for the time being.

	Jon Coppeard (:jonco) (PTO until 14th September)
	Comment 8 • 10 years ago

From the second frame and the fact that the crash reason is EXCEPTION_BREAKPOINT, I think this is the MOZ_CRASH in the CycleCollectedJSRuntime constructor which happens when JS_NewRuntime() fails:

http://dxr.mozilla.org/mozilla-central/source/xpcom/base/CycleCollectedJSRuntime.cpp#469

I don't know why it's reporting this as coming from nsTArray though.

The problem is that we are OOM more often here now the new runtime allocates a 16MB nursery.

	(Away)
	Comment 10 • 10 years ago

Over in bug 1007763 comment 9 I came to the same conclusion: it's a failed JS_NewRuntime() due to OOM. The compiler folded a bunch of MOZ_CRASH calls together and that's why it looks like nsTArray.

This can be reproduced with large Mega downloads since they use tons of memory and exercise this worker codepath.

	u279076 Reporter
	Comment 11 • 10 years ago

This is now the #3 topcrash in Firefox 31.

	Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)
	Comment 12 • 10 years ago

This is currently the #10 topcrash in Firefox 31.0b with 2459/59660 crashes.

	Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)
	Comment 13 • 10 years ago

This remains around the #9 topcrash in Firefox 31.0b with 6447/ 277133 crashes in the last 7 days. 

I noticed that there aren't any crashes yet reported for builds after 6/26.

	Robert Kaiser
	Comment 14 • 10 years ago

Well, 6/26 is the newest beta build that was created so far. Today should be the next one.

	Jon Coppeard (:jonco) (PTO until 14th September)
	Comment 15 • 10 years ago

It seems we are assuming that creating a CycleCollectedJSRuntime is infallible.  With GGC attempting to allocate a 16MB nursery this is definitely no longer the case.

I think the solution here is to change the way the initialization of worker threads is done to allow for the possibility of this failing and report errors.   Unfortunately this will require a bit of refactoring - the constructors for CycleCollectedJSRuntime and WorkerJSRuntime will need to be split into an infallible constructor and a fallible initialisation method, and whatever's further up the stack adapted to handle this correctly.

Andrew, you know more about this area than me, does this sound feasible?

	Andrew McCreight [:mccr8] Assignee
	Comment 16 • 10 years ago

Yeah, I was looking into that before, and it seems pretty easy.  I filed bug 1034611 for that.

	Sylvestre Ledru [:Sylvestre]
	Comment 17 • 10 years ago

Liz, Kairo, do you still a trace of this crash? Thanks

	Robert Kaiser
	Comment 18 • 10 years ago

This signature continues to be high volume in any new 31 beta that comes out: https://crash-stats.mozilla.com/report/list?signature=nsTArray_base%3CnsTArrayInfallibleAllocator%2C+nsTArray_CopyWithMemutils%3E%3A%3AIncrementLength%28unsigned+int%29+%7C+nsThread%3A%3AProcessNextEvent%28bool%2C+bool%2A%29

	alex_mayorga
	Comment 19 • 10 years ago

My SO's Firefox 31 is crashing like this

Report ID 	Date Submitted
bp-77874d66-d57f-4549-8da7-ae21a2140715	15/07/2014	09:54 a.m.

Commenting so we can track this bug...

Let us know if there's anything needed from a crashing profile.

	Sylvestre Ledru [:Sylvestre]
	Comment 20 • 10 years ago

That is too late for this bug. Does it affect 32 too?

	u279076 Reporter
	Comment 21 • 10 years ago

(In reply to Sylvestre Ledru [:sylvestre] from comment #20)
> That is too late for this bug. Does it affect 32 too?

This is currently the #13 topcrash in Firefox 32.0a2.

Source: https://crash-stats.mozilla.com/topcrasher_ranks_bybug/?bug_number=991845

	Andrew McCreight [:mccr8] Assignee
	Comment 22 • 10 years ago

It looks like there are no crashes on trunk since bug 1037510 landed.  It has just landed on Aurora and Beta, so hopefully the signature will go away there, too.

	Lawrence Mandel [:lmandel] (use needinfo)
	Comment 23 • 10 years ago

(In reply to Andrew McCreight [:mccr8] from comment #22)
> It looks like there are no crashes on trunk since bug 1037510 landed.  It
> has just landed on Aurora and Beta, so hopefully the signature will go away
> there, too.

Kairo - Can you please help confirm that this crash has been fixed by bug 1037510?

	Robert Kaiser
	Comment 24 • 10 years ago

(In reply to Lawrence Mandel [:lmandel] from comment #23)
> (In reply to Andrew McCreight [:mccr8] from comment #22)
> > It looks like there are no crashes on trunk since bug 1037510 landed.  It
> > has just landed on Aurora and Beta, so hopefully the signature will go away
> > there, too.
> 
> Kairo - Can you please help confirm that this crash has been fixed by bug
> 1037510?

From all I can see, it's fixed in 32.0b5 (the signature still exists but at very low volume, I saw 5 crashes overall, which is probably some other cause). I also do see it last appearing on 8/4 aurora builds and from what6 I see, nightly is good as well.