Closed Bug 1488181 Opened 6 years ago Closed 6 years ago

Crash reporting hangs if gDllServicesLock is locked

Categories

(Firefox :: General, defect, P4)

Unspecified
Windows
defect

Tracking

()

RESOLVED FIXED
Firefox 64
Tracking Status
firefox64 --- fixed

People

(Reporter: ccorcoran, Assigned: ccorcoran)

References

Details

Attachments

(1 file)

There is a deadlock if crash reporting is invoked while gDllServicesLock is locked. Here's the sequence of events causing the deadlock: THREAD A 1. Aquires gDllServicesLock exclusively 2. Crashes 3. Breakpad launches a thread to write the minidump and waits on it. THREAD B 4. Loads PSAPI.DLL 5. DllLoadNotification waits on on gDllServicesLock. This is essentially the same type of issue I dealt with in Bug 1435827. Basically because you can't really predict when the OS loader will interrupt, you can't make any guarantees about lock ordering. Using a TryAcquire* could fix this immediate issue, though I am still uneasy about using SRW locks for this because they aren't recursion-aware, and loader re-entrancy could cause havoc. For example if a DLL is loaded within DllLoadNotification, then gDllServicesLock will get locked again in the same thread, and will inadvertently get unlocked early. I feel that using a semaphore, like Bug 1435827, would eliminate the risk of this behavior.
(In reply to Carl Corcoran [:ccorcoran] from comment #0) > There is a deadlock if crash reporting is invoked while gDllServicesLock is > locked. > > Here's the sequence of events causing the deadlock: > > THREAD A > 1. Aquires gDllServicesLock exclusively > 2. Crashes > 3. Breakpad launches a thread to write the minidump and waits on it. What was actual crash that triggered this? > > THREAD B Thread B being the Breakpad thread? > 4. Loads PSAPI.DLL (Assuming that the answer to my previous question is, yes) This is bad. Breakpad and our nsExceptionHandler stuff is all written with the understanding that all kinds of stuff has been interrupted and is in an indeterminate state. We *never* (de)allocate memory once a crash has triggered, for example, because we can't reason about the state of the heap allocator. The fact that we're delayloading PSAPI.DLL upon crashing is the real bug, IMHO. I wonder if we can just set PSAPI_VERSION to 2 (thus completely eliminating PSAPI from our imports) and call it a day...
(At least, I am assuming that the reason that PSAPI is being loaded so late is due to delayloading...)
Depends on: 1488800
To update this, PSAPI is loaded indirectly by MiniDumpWriteDump, so PSAPI_VERSION won't affect this. I also did confirm that pre-loading psapi.dll prevents the deadlock on Win10.
Comment on attachment 9010550 [details] Bug 1488181: Preload psapi.dll to prevent hangs in the exception handler;r=ted Ted Mielczarek [:ted] [:ted.mielczarek] has approved the revision.
Attachment #9010550 - Flags: review+
Keywords: checkin-needed
Pushed by ccoroiu@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/9d400e7fa5e6 Preload psapi.dll to prevent hangs in the exception handler;r=ted
Keywords: checkin-needed
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 64
Assignee: nobody → ccorcoran
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: