Closed Bug 708962 Opened 13 years ago Closed 13 years ago

Intermittent test_eviction.js | test failed (with xpcshell return code: 1) from ###!!! ASSERTION: Potential deadlock detected

Categories

(Toolkit :: Telemetry, defect)

Product:
Toolkit
Component:
Telemetry
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla12

People

(Reporter: philor, Assigned: vladan)

References

Details

(Keywords: assertion, intermittent-failure)

Attachments

(1 file)

Start Telemetry service at application start to avoid deadlock from GetService spinning event loop.
(deleted), patch
benjamin
: review+
Details | Diff | Splinter Review 
https://tbpl.mozilla.org/php/getParsedLog.php?id=7845289&tree=Mozilla-Inbound
Rev4 MacOSX Snow Leopard 10.6 mozilla-inbound debug test xpcshell on 2011-12-08 21:35:26 PST for push 59c363453713

--- Mutex : Connection::sharedDBMutex (currently acquired)
 calling context
stacks_disabled+0[stacks_disabled +0x0]

###!!! Deadlock may happen NOW!

###!!! ASSERTION: Potential deadlock detected:
Cyclical dependency starts at
Mutex : Connection::sharedDBMutex (currently acquired)
Cycle completed at
Mutex : Connection::sharedDBMutex (currently acquired)

###!!! Deadlock may happen NOW!

: 'Error', file /builds/slave/m-in-osx64-dbg/build/obj-firefox/xpcom/build/BlockingResourceBase.cpp, line 161
XRE_TelemetryAccumulate+0x0001B6DE [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x016C084E]
XRE_AddManifestLocation+0x0000798F [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x01C7485F]
XRE_ChildProcessTypeToString+0x0000C30D [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x0000E22D]
XRE_ChildProcessTypeToString+0x0000F920 [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x00011840]
XRE_ChildProcessTypeToString+0x0000C1FC [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x0000E11C]
js::SecurityWrapper<js::Wrapper>::~SecurityWrapper()+0x0028D951 [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x016A4D81]
XRE_TelemetryAccumulate+0x00008E70 [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x016ADFE0]
XRE_TelemetryAccumulate+0x0001AAD0 [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x016BFC40]
XRE_TelemetryAccumulate+0x0001AF6B [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x016C00DB]
XRE_TelemetryAccumulate+0x0001B74F [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x016C08BF]
XRE_AddManifestLocation+0x0000798F [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x01C7485F]
XRE_ChildProcessTypeToString+0x0000C30D [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x0000E22D]
XRE_AddManifestLocation+0x0000918D [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/XUL +0x01C7605D]
PR_JoinThread+0x000001D6 [/Users/cltbld/talos-slave/test/build/NightlyDebug.app/Contents/MacOS/libnspr4.dylib +0x00022E66]
_pthread_start+0x0000014B [/usr/lib/libSystem.B.dylib +0x00039FD6]
###!!! ASSERTION: Potential deadlock detected:
Cyclical dependency starts at
Mutex : Connection::sharedDBMutex (currently acquired)
Cycle completed at
Mutex : Connection::sharedDBMutex (currently acquired)

###!!! Deadlock may happen NOW!

: 'Error', file /builds/slave/m-in-osx64-dbg/build/obj-firefox/xpcom/build/BlockingResourceBase.cpp, line 161
<<<<<<<
PROCESS-CRASH | /Users/cltbld/talos-slave/test/build/xpcshell/tests/extensions/cookie/test/unit/test_eviction.js | application crashed (minidump found)
Crash dump filename: /Users/cltbld/talos-slave/test/build/xpcshell/tests/extensions/cookie/test/unit/3633E3AC-5EC4-4328-BA3B-BA357E2A0466.dmp
Operating system: Mac OS X
                  10.6.8 10K549
CPU: amd64
     family 6 model 23 stepping 10
     2 CPUs

Crash reason:  EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
Crash address: 0x0

Thread 14 (crashed)
 0  libmozalloc.dylib + 0x1ac5
    rbx = 0x706312f8   r12 = 0x056ed350   r13 = 0x706312f8   r14 = 0x0248a1f0
    r15 = 0x02631c70   rip = 0x00022ac5   rsp = 0x056ed310   rbp = 0x056ed320
    Found by: given as instruction pointer in context
 1  XUL + 0x12a1f
    rip = 0x00037a20   rsp = 0x056ed330
    Found by: stack scanning
What's happening here is that Telemetry calls into the service manager while a storage lock is being held, and the service manager spins the event loop causing us to reenter storage and attempt to acquire the lock again.

That getting a service can spin the event loop is pretty terrifying.
Blocks: 699051
Component: Networking: Cookies → Telemetry
Product: Core → Toolkit
QA Contact: networking.cookies → telemetry
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #9)
> That getting a service can spin the event loop is pretty terrifying.

And I mean that in the "we almost certainly have potential sg:crit bugs in code that didn't know that when it was written" sense.
Assignee: nobody → vdjeric
Status: NEW → ASSIGNED

        Attachment #587002 -
        Flags: review?(benjamin)

    
  

        Attachment #587002 -
        Flags: review?(benjamin) → review+

    
    

    
  
https://hg.mozilla.org/integration/mozilla-inbound/rev/a537453b1d66
OS: Mac OS X → All
Hardware: x86_64 → All
Version: Trunk → 12 Branch

    
  
Target Milestone: --- → mozilla12
Version: 12 Branch → Trunk

    
    

    
      
  

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/a537453b1d66
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Whiteboard: [orange]

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: