Closed
Bug 1488181
Opened 6 years ago
Closed 6 years ago
Crash reporting hangs if gDllServicesLock is locked
Categories
(Firefox :: General, defect, P4)
Tracking
()
RESOLVED
FIXED
Firefox 64
Tracking | Status | |
---|---|---|
firefox64 | --- | fixed |
People
(Reporter: ccorcoran, Assigned: ccorcoran)
References
Details
Attachments
(1 file)
There is a deadlock if crash reporting is invoked while gDllServicesLock is locked.
Here's the sequence of events causing the deadlock:
THREAD A
1. Aquires gDllServicesLock exclusively
2. Crashes
3. Breakpad launches a thread to write the minidump and waits on it.
THREAD B
4. Loads PSAPI.DLL
5. DllLoadNotification waits on on gDllServicesLock.
This is essentially the same type of issue I dealt with in Bug 1435827. Basically because you can't really predict when the OS loader will interrupt, you can't make any guarantees about lock ordering.
Using a TryAcquire* could fix this immediate issue, though I am still uneasy about using SRW locks for this because they aren't recursion-aware, and loader re-entrancy could cause havoc.
For example if a DLL is loaded within DllLoadNotification, then gDllServicesLock will get locked again in the same thread, and will inadvertently get unlocked early.
I feel that using a semaphore, like Bug 1435827, would eliminate the risk of this behavior.
Comment 1•6 years ago
|
||
(In reply to Carl Corcoran [:ccorcoran] from comment #0)
> There is a deadlock if crash reporting is invoked while gDllServicesLock is
> locked.
>
> Here's the sequence of events causing the deadlock:
>
> THREAD A
> 1. Aquires gDllServicesLock exclusively
> 2. Crashes
> 3. Breakpad launches a thread to write the minidump and waits on it.
What was actual crash that triggered this?
>
> THREAD B
Thread B being the Breakpad thread?
> 4. Loads PSAPI.DLL
(Assuming that the answer to my previous question is, yes)
This is bad. Breakpad and our nsExceptionHandler stuff is all written with the understanding that all kinds of stuff has been interrupted and is in an indeterminate state. We *never* (de)allocate memory once a crash has triggered, for example, because we can't reason about the state of the heap allocator. The fact that we're delayloading PSAPI.DLL upon crashing is the real bug, IMHO.
I wonder if we can just set PSAPI_VERSION to 2 (thus completely eliminating PSAPI from our imports) and call it a day...
Comment 2•6 years ago
|
||
(At least, I am assuming that the reason that PSAPI is being loaded so late is due to delayloading...)
Assignee | ||
Comment 3•6 years ago
|
||
To update this, PSAPI is loaded indirectly by MiniDumpWriteDump, so PSAPI_VERSION won't affect this. I also did confirm that pre-loading psapi.dll prevents the deadlock on Win10.
Assignee | ||
Comment 4•6 years ago
|
||
Comment 5•6 years ago
|
||
Comment on attachment 9010550 [details]
Bug 1488181: Preload psapi.dll to prevent hangs in the exception handler;r=ted
Ted Mielczarek [:ted] [:ted.mielczarek] has approved the revision.
Attachment #9010550 -
Flags: review+
Assignee | ||
Updated•6 years ago
|
Keywords: checkin-needed
Pushed by ccoroiu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9d400e7fa5e6
Preload psapi.dll to prevent hangs in the exception handler;r=ted
Keywords: checkin-needed
Comment 7•6 years ago
|
||
bugherder |
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox64:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 64
Updated•6 years ago
|
Assignee: nobody → ccorcoran
You need to log in
before you can comment on or make changes to this bug.
Description
•