Marco Castelluccio [:marco] Reporter
	Description • 8 years ago

There's a higher percentage of crashes with AMD CPUs in Firefox 48.0b99 compared to 48.0b10 (see the attached image).

Mostly due to the signatures:
- js::FrameIter::functionDisplayAtom (bug 1290388);
- js::jit::EnterBaselineMethod (bug 1034706);
- nsGlobalWindow::SetTimeoutOrInterval (bug 1290418).

	Marco Castelluccio [:marco] Reporter
	Comment 1 • 8 years ago

See also: https://crash-analysis.mozilla.com/rkaiser/datil/searchcompare/?common=product%3DFirefox&p1=version%3D48.0b10&p2=version%3D48.0b99

Or, for startup crashes: https://crash-analysis.mozilla.com/rkaiser/datil/searchcompare/?common=product%3DFirefox%26uptime%3D%3C%253D60&p1=version%3D48.0b10&p2=version%3D48.0b99

	Nicolas B. Pierron [:nbp]
	Comment 3 • 8 years ago

(In reply to Marco Castelluccio [:marco] (PTO until August 9) from comment #0)
> - js::jit::EnterBaselineMethod (bug 1034706);

Jan made a patch in Bug 1281759 to prevent us from generating too many branches in the same code sequence, if we are on AMD cpu.  Maybe we could backport this patch, and see if this fixes this issue.

(In reply to Marco Castelluccio [:marco] (PTO until August 9) from comment #0)
> - js::FrameIter::functionDisplayAtom (bug 1290388);
> - nsGlobalWindow::SetTimeoutOrInterval (bug 1290418).

If these are correlated with AMD cpu, then these are likely MSVC issues, and if I recall correctly, we are already compiling multiple times firefox binaries to avoid this AMD cpu issues by luck.

Maybe we have not done so for 48.0b99, or some code patterns are triggering this issue.

	Nicolas B. Pierron [:nbp]
	Comment 4 • 8 years ago

Marco, are we still seeing this issue on 49 and newer?

	Marco Castelluccio [:marco] Reporter
	Comment 5 • 8 years ago

(In reply to Nicolas B. Pierron [:nbp] from comment #4)
> Marco, are we still seeing this issue on 49 and newer?

Unfortunately this is happening in specific builds, so it's hard to tell. I guess we can close it for now and reopen it if we see the pattern again (or just open a new bug when we see the pattern again).

(In reply to Nicolas B. Pierron [:nbp] from comment #3)
> If these are correlated with AMD cpu, then these are likely MSVC issues, and
> if I recall correctly, we are already compiling multiple times firefox
> binaries to avoid this AMD cpu issues by luck.

I asked around and it looks like we're not doing this. It might be worth to consider though. Something like this might also be useful for the Intel crashes that we've been seeing lately (bug 1296630).

	Nicolas B. Pierron [:nbp]
	Comment 6 • 8 years ago

(In reply to Marco Castelluccio [:marco] from comment #5)
> (In reply to Nicolas B. Pierron [:nbp] from comment #3)
> > If these are correlated with AMD cpu, then these are likely MSVC issues, and
> > if I recall correctly, we are already compiling multiple times firefox
> > binaries to avoid this AMD cpu issues by luck.
> 
> I asked around and it looks like we're not doing this. It might be worth to
> consider though. Something like this might also be useful for the Intel
> crashes that we've been seeing lately (bug 1296630).

From #jsapi http://logs.glob.uno/?c=mozilla%23jsapi#c802937:
> 
> nbp> RyanVM: I recall that we were compiling multiple versions of our release builds, in order to 
>      work-around CPU bugs produced by MSVC.
> 
> RyanVM> "sorta"
> RyanVM> yes, it wasn't unheard of to do a respin if we had a "bad" build
> RyanVM> we did that for the AMD crashers too for awhile
> RyanVM> i mean, it was basically "Ship an RC build, see if the crash is present on the Beta channel, > respin if yes"
> RyanVM> that was before the days of release promotion

	Marco Castelluccio [:marco] Reporter
	Comment 7 • 8 years ago

(In reply to Nicolas B. Pierron [:nbp] from comment #6)
> (In reply to Marco Castelluccio [:marco] from comment #5)
> > (In reply to Nicolas B. Pierron [:nbp] from comment #3)
> > > If these are correlated with AMD cpu, then these are likely MSVC issues, and
> > > if I recall correctly, we are already compiling multiple times firefox
> > > binaries to avoid this AMD cpu issues by luck.
> > 
> > I asked around and it looks like we're not doing this. It might be worth to
> > consider though. Something like this might also be useful for the Intel
> > crashes that we've been seeing lately (bug 1296630).
> 
> From #jsapi http://logs.glob.uno/?c=mozilla%23jsapi#c802937:
> > 
> > nbp> RyanVM: I recall that we were compiling multiple versions of our release builds, in order to 
> >      work-around CPU bugs produced by MSVC.
> > 
> > RyanVM> "sorta"
> > RyanVM> yes, it wasn't unheard of to do a respin if we had a "bad" build
> > RyanVM> we did that for the AMD crashers too for awhile
> > RyanVM> i mean, it was basically "Ship an RC build, see if the crash is present on the Beta channel, > respin if yes"
> > RyanVM> that was before the days of release promotion

Yes, this is what we always do for any crash / problem, but we don't have an automated way to check if a build is affected by the AMD bug.