I don't have reliable STR, but I've hit this same shutdown assertion failure 3 times in 5 minutes (testing a local debug build) when shutting down while the following CNN page is still loading: STR: 1. Using a debug build of Firefox, load http://www.cnn.com/2014/04/08/politics/army-hairstyle-ban-debate 2. When the page starts loading, wait 2–3 seconds then CMD+Q to shutdown. RESULT: About 30% of the time, I hit the following assertion failure: Assertion failure: data, at mozilla/inbound/xpcom/base/nsCycleCollector.cpp:3715 Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000 NS_CycleCollectorSuspect3 (n=<value temporarily unavailable, due to optimizations>, cp=<value temporarily unavailable, due to optimizations>, aRefCnt=<value temporarily unavailable, due to optimizations>, aShouldDelete=<value temporarily unavailable, due to optimizations>) at ThreadLocal.h:123 123 MOZ_ASSERT(initialized()); (gdb) bt #0 NS_CycleCollectorSuspect3 (n=<value temporarily unavailable, due to optimizations>, cp=<value temporarily unavailable, due to optimizations>, aRefCnt=<value temporarily unavailable, due to optimizations>, aShouldDelete=<value temporarily unavailable, due to optimizations>) at ThreadLocal.h:123 #1 0x00000001026a90ed in nsXPCWrappedJS::Release (this=0x1243b3200) at nsISupportsImpl.h:183 #2 0x00000001026f9797 in ~nsPrincipal (this=0x12fd30b80) at nsCOMPtr.h:532 #3 0x00000001026f96ee in ~nsPrincipal [inlined] () at /Users/cpeterson/Code/mozilla/inbound/caps/include/nsPrincipal.h:137 #4 0x00000001026f96ee in ~nsPrincipal (this=0x12fd30b80) at mozalloc.h:137 #5 0x00000001026f95e6 in nsPrincipal::Release (this=0x12fd30b80) at nsPrincipal.cpp:66 #6 0x0000000101f3f744 in ~nsCOMPtr [inlined] () at /Users/cpeterson/Code/mozilla/inbound/objdir-osx/dist/include/nsCOMPtr.h:532 #7 0x0000000101f3f744 in ~imgRequest (this=0x133470f20) at nsCOMPtr.h:529 #8 0x0000000101f3f39e in ~imgRequest [inlined] () at /Users/cpeterson/Code/mozilla/inbound/image/src/imgRequest.h:74 #9 0x0000000101f3f39e in ~imgRequest (this=0x133470f20) at mozalloc.h:74 #10 0x0000000101f3ec4e in imgRequest::Release () at /Users/cpeterson/Code/mozilla/inbound/image/src/imgRequest.h:53 #11 0x0000000101f3ec4e in imgRequest::Release (this=<value temporarily unavailable, due to optimizations>) at /Users/cpeterson/Code/mozilla/inbound/image/src/imgRequest.cpp:53

Andrew: this is shutdown assertion failure is the same NS_CycleCollectorSuspect3 MOZ_ASSERT(data) as bug 991626, but with a different stack trace.

I'd need to see the entire stack, but I guess we're somehow holding a reference to an nsIPrincipal late in shutdown. nsXPCWrappedJS gets all kinds of angry when used off-main-thread, so I don't think that can be the problem. I could certainly believe that some kind of pending load at shutdown would somehow result in a reference hanging around.

Oh, I think bug 913138 might actually fix this, as it seems to make imagelib shutdown earlier.

So, imgRequest is threadsafe, and it keeps nsIPrincipal alive, and nsIPrincipal keeps mainthread only nsWrappedJS alive. This is scary stuff. :/

...but a full stack trace is needed here.

This can't be a threading problem, because nsXPCWrappedJS::Release will abort if we're off the main thread. I think the imgRequest is just keeping the nsIPrincipal alive past CC shutdown. That doesn't sound too bad, though it would be nice to see the entire stack to figure out where exactly this is happening.

Indeed. We have a hard crash there in nsWrappedJS::Release.

Here is a full stack trace from an unoptimized debug build: Assertion failure: data, at /Users/chris/Code/mozilla/inbound/xpcom/base/nsCycleCollector.cpp:3715 Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000 NS_CycleCollectorSuspect3 (n=0x1227b0690, cp=0x0, aRefCnt=0x1227b06c0, aShouldDelete=0x7fff5fbfe49f) at nsCycleCollector.cpp:3715 3715 MOZ_ASSERT(data); (gdb) bt #0 NS_CycleCollectorSuspect3 (n=0x1227b0690, cp=0x0, aRefCnt=0x1227b06c0, aShouldDelete=0x7fff5fbfe49f) at nsCycleCollector.cpp:3715 #1 0x00000001016204bf in nsCycleCollectingAutoRefCnt::decr (this=0x1227b06c0, owner=0x1227b0690, p=0x0, shouldDelete=0x7fff5fbfe49f) at nsISupportsImpl.h:183 #2 0x0000000101650ee5 in nsCycleCollectingAutoRefCnt::decr (this=0x1227b06c0, owner=0x1227b0690, shouldDelete=0x7fff5fbfe49f) at nsISupportsImpl.h:171 #3 0x00000001033560d0 in nsXPCWrappedJS::Release (this=0x1227b0680) at XPCWrappedJS.cpp:250 #4 0x00000001016d5778 in nsXPTCStubBase::Release (this=0x11ac54a40) at /Users/chris/Code/mozilla/inbound/xpcom/reflect/xptcall/src/xptcall.cpp:36 #5 0x00000001033e748b in ~nsCOMPtr (this=0x12d8d9e00) at nsCOMPtr.h:532 #6 0x00000001033e3af5 in ~nsCOMPtr (this=0x12d8d9e00) at nsCOMPtr.h:529 #7 0x00000001033d72f5 in ~nsBasePrincipal (this=0x12d8d9df0) at nsPrincipal.cpp:87 #8 0x00000001033d7943 in ~nsPrincipal (this=0x12d8d9df0) at nsPrincipal.cpp:137 #9 0x00000001033d78e5 in ~nsPrincipal (this=0x12d8d9df0) at nsPrincipal.cpp:137 #10 0x00000001033d78b9 in ~nsPrincipal (this=0x12d8d9df0) at nsPrincipal.cpp:137 #11 0x00000001033d7178 in nsBasePrincipal::Release (this=0x12d8d9df0) at nsPrincipal.cpp:66 #12 0x00000001033d77cf in nsPrincipal::Release (this=0x12d8d9df0) at nsPrincipal.cpp:126 #13 0x00000001016e8d1b in ~nsCOMPtr (this=0x12bbeea28) at nsCOMPtr.h:532 #14 0x00000001016e2d35 in ~nsCOMPtr (this=0x12bbeea28) at nsCOMPtr.h:529 #15 0x0000000102673a89 in ~gfxUserFontData (this=0x12bbeea10) at gfxUserFontSet.h:66 #16 0x00000001026739f5 in ~gfxUserFontData (this=0x12bbeea10) at gfxUserFontSet.h:66 #17 0x0000000102673a19 in ~gfxUserFontData (this=0x12bbeea10) at gfxUserFontSet.h:66 #18 0x000000010264ae51 in ~nsAutoPtr (this=0x13659a718) at nsAutoPtr.h:72 #19 0x0000000102637805 in ~nsAutoPtr (this=0x13659a718) at nsAutoPtr.h:71 #20 0x0000000102620884 in ~gfxFontEntry (this=0x13659a6a0) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxFont.cpp:177 #21 0x00000001026b01ea in ~MacOSFontEntry (this=0x13659a6a0) at gfxMacPlatformFontList.h:40 #22 0x00000001026af6a5 in ~MacOSFontEntry (this=0x13659a6a0) at gfxMacPlatformFontList.h:38 #23 0x00000001026af6c9 in ~MacOSFontEntry (this=0x13659a6a0) at gfxMacPlatformFontList.h:38 #24 0x000000010264b2a2 in gfxFontEntry::Release (this=0x13659a6a0) at gfxFont.h:235 #25 0x000000010264b6dd in ~nsRefPtr (this=0x133db60a8) at nsAutoPtr.h:894 #26 0x0000000102639e75 in ~nsRefPtr (this=0x133db60a8) at nsAutoPtr.h:892 #27 0x0000000102626ef0 in ~gfxFont (this=0x133db6090) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxFont.cpp:1953 #28 0x0000000102611b40 in ~gfxMacFont (this=0x133db6090) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxMacFont.cpp:126 #29 0x0000000102611ac5 in ~gfxMacFont (this=0x133db6090) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxMacFont.cpp:119 #30 0x0000000102611a99 in ~gfxMacFont (this=0x133db6090) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxMacFont.cpp:119 #31 0x0000000102625e62 in gfxFontCache::DestroyFont (this=0x1155b2200, aFont=0x133db6090) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxFont.cpp:1689 #32 0x0000000102625cf6 in gfxFontCache::NotifyExpired (this=0x1155b2200, aFont=0x133db6090) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxFont.cpp:1676 #33 0x0000000102647fac in nsExpirationTracker<gfxFont, 3u>::AgeOneGeneration (this=0x1155b2200) at nsExpirationTracker.h:188 #34 0x0000000102638d81 in nsExpirationTracker<gfxFont, 3u>::AgeAllGenerations (this=0x1155b2200) at nsExpirationTracker.h:212 #35 0x0000000102625ba2 in ~gfxFontCache (this=0x1155b2200) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxFont.cpp:1603 #36 0x0000000102625905 in ~gfxFontCache (this=0x1155b2200) at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxFont.cpp:1591 #37 0x00000001026258c8 in gfxFontCache::Shutdown () at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxFont.cpp:1553 #38 0x000000010265994d in gfxPlatform::Shutdown () at /Users/chris/Code/mozilla/inbound/gfx/thebes/gfxPlatform.cpp:443 #39 0x000000010328c4d2 in LayoutModuleDtor () at nsLayoutModule.cpp:1259 #40 0x00000001016aabdd in ~KnownModule (this=0x11081ef00) at nsComponentManager.h:226 #41 0x00000001016aab95 in ~KnownModule (this=0x11081ef00) at nsComponentManager.h:224 #42 0x00000001016aab5d in ~nsAutoPtr (this=0x1003752c8) at nsAutoPtr.h:72 #43 0x00000001016aab25 in ~nsAutoPtr (this=0x1003752c8) at nsAutoPtr.h:71 #44 0x00000001016aab05 in nsTArrayElementTraits<nsAutoPtr<nsComponentManagerImpl::KnownModule> >::Destruct (e=0x1003752c8) at nsTArray.h:536 #45 0x00000001016aaaa6 in nsTArray_Impl<nsAutoPtr<nsComponentManagerImpl::KnownModule>, nsTArrayInfallibleAllocator>::DestructRange (this=0x10039f460, start=0, count=56) at nsTArray.h:1585 #46 0x00000001016aaa24 in nsTArray_Impl<nsAutoPtr<nsComponentManagerImpl::KnownModule>, nsTArrayInfallibleAllocator>::RemoveElementsAt (this=0x10039f460, start=0, count=56) at nsTArray.h:1302 #47 0x00000001016a79ff in nsTArray_Impl<nsAutoPtr<nsComponentManagerImpl::KnownModule>, nsTArrayInfallibleAllocator>::Clear (this=0x10039f460) at nsTArray.h:1313 #48 0x00000001016a312c in nsComponentManagerImpl::Shutdown (this=0x10039f330) at /Users/chris/Code/mozilla/inbound/xpcom/components/nsComponentManager.cpp:788 #49 0x00000001015c2433 in mozilla::ShutdownXPCOM (servMgr=0x0) at nsXPComInit.cpp:990 #50 0x00000001015c1dc5 in NS_ShutdownXPCOM (servMgr=0x10039f338) at nsXPComInit.cpp:819 #51 0x0000000104d74c8a in ~ScopedXPCOMStartup (this=0x10030f480) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:1202 #52 0x0000000104d74b95 in ~ScopedXPCOMStartup (this=0x10030f480) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:1183 #53 0x0000000104d7d9a2 in XREMain::XRE_main (this=0x7fff5fbfefa0, argc=3, argv=0x7fff5fbff8a8, aAppData=0x7fff5fbff238) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:4113 #54 0x0000000104d7dd6d in XRE_main (argc=3, argv=0x7fff5fbff8a8, aAppData=0x7fff5fbff238, aFlags=0) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:4300 #55 0x000000010000210f in do_main (argc=3, argv=0x7fff5fbff8a8, xreDirectory=0x100322340) at /Users/chris/Code/mozilla/inbound/browser/app/nsBrowserApp.cpp:282 #56 0x0000000100001633 in main (argc=3, argv=0x7fff5fbff8a8) at /Users/chris/Code/mozilla/inbound/browser/app/nsBrowserApp.cpp:643 (gdb)

That is actually a bit different stack, but same crash.

Here is the full stack trace for the imgRequest crash: Assertion failure: data, at /Users/chris/Code/mozilla/inbound/xpcom/base/nsCycleCollector.cpp:3715 Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000 NS_CycleCollectorSuspect3 (n=0x12257da10, cp=0x0, aRefCnt=0x12257da40, aShouldDelete=0x7fff5fbfe41f) at nsCycleCollector.cpp:3715 3715 MOZ_ASSERT(data); (gdb) bt #0 NS_CycleCollectorSuspect3 (n=0x12257da10, cp=0x0, aRefCnt=0x12257da40, aShouldDelete=0x7fff5fbfe41f) at nsCycleCollector.cpp:3715 #1 0x00000001016204bf in nsCycleCollectingAutoRefCnt::decr (this=0x12257da40, owner=0x12257da10, p=0x0, shouldDelete=0x7fff5fbfe41f) at nsISupportsImpl.h:183 #2 0x0000000101650ee5 in nsCycleCollectingAutoRefCnt::decr (this=0x12257da40, owner=0x12257da10, shouldDelete=0x7fff5fbfe41f) at nsISupportsImpl.h:171 #3 0x00000001033560d0 in nsXPCWrappedJS::Release (this=0x12257da00) at XPCWrappedJS.cpp:250 #4 0x00000001016d5778 in nsXPTCStubBase::Release (this=0x130fa90c0) at /Users/chris/Code/mozilla/inbound/xpcom/reflect/xptcall/src/xptcall.cpp:36 #5 0x00000001033e748b in ~nsCOMPtr (this=0x12db73e30) at nsCOMPtr.h:532 #6 0x00000001033e3af5 in ~nsCOMPtr (this=0x12db73e30) at nsCOMPtr.h:529 #7 0x00000001033d72f5 in ~nsBasePrincipal (this=0x12db73e20) at nsPrincipal.cpp:87 #8 0x00000001033d7943 in ~nsPrincipal (this=0x12db73e20) at nsPrincipal.cpp:137 #9 0x00000001033d78e5 in ~nsPrincipal (this=0x12db73e20) at nsPrincipal.cpp:137 #10 0x00000001033d78b9 in ~nsPrincipal (this=0x12db73e20) at nsPrincipal.cpp:137 #11 0x00000001033d7178 in nsBasePrincipal::Release (this=0x12db73e20) at nsPrincipal.cpp:66 #12 0x00000001033d77cf in nsPrincipal::Release (this=0x12db73e20) at nsPrincipal.cpp:126 #13 0x00000001016e8d1b in ~nsCOMPtr (this=0x12224b8f8) at nsCOMPtr.h:532 #14 0x00000001016e2d35 in ~nsCOMPtr (this=0x12224b8f8) at nsCOMPtr.h:529 #15 0x00000001027bbfba in ~imgRequest (this=0x12224b8a0) at /Users/chris/Code/mozilla/inbound/image/src/imgRequest.cpp:81 #16 0x00000001027bbd45 in ~imgRequest (this=0x12224b8a0) at /Users/chris/Code/mozilla/inbound/image/src/imgRequest.cpp:74 #17 0x00000001027bbd19 in ~imgRequest (this=0x12224b8a0) at /Users/chris/Code/mozilla/inbound/image/src/imgRequest.cpp:74 #18 0x00000001027bb749 in imgRequest::Release (this=0x12224b8a0) at /Users/chris/Code/mozilla/inbound/image/src/imgRequest.cpp:53 #19 0x00000001027b9601 in ~nsRefPtr (this=0x113baf918) at nsAutoPtr.h:894 #20 0x00000001027b1115 in ~nsRefPtr (this=0x113baf918) at nsAutoPtr.h:892 #21 0x00000001027a6802 in ~imgCacheEntry (this=0x113baf900) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:502 #22 0x00000001027a67b5 in ~imgCacheEntry (this=0x113baf900) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:500 #23 0x00000001027b535b in imgCacheEntry::Release (this=0x113baf900) at imgLoader.h:62 #24 0x00000001027b942d in ~nsRefPtr (this=0x1003620e8) at nsAutoPtr.h:894 #25 0x00000001027b1785 in ~nsRefPtr (this=0x1003620e8) at nsAutoPtr.h:892 #26 0x00000001027ba205 in nsTArrayElementTraits<nsRefPtr<imgCacheEntry> >::Destruct (e=0x1003620e8) at nsTArray.h:536 #27 0x00000001027ba1d6 in nsTArray_Impl<nsRefPtr<imgCacheEntry>, nsTArrayInfallibleAllocator>::DestructRange (this=0x7fff5fbfe9b8, start=0, count=79) at nsTArray.h:1585 #28 0x00000001027ba154 in nsTArray_Impl<nsRefPtr<imgCacheEntry>, nsTArrayInfallibleAllocator>::RemoveElementsAt (this=0x7fff5fbfe9b8, start=0, count=79) at nsTArray.h:1302 #29 0x00000001027b9fbf in nsTArray_Impl<nsRefPtr<imgCacheEntry>, nsTArrayInfallibleAllocator>::Clear (this=0x7fff5fbfe9b8) at nsTArray.h:1313 #30 0x00000001027b9f79 in ~nsTArray_Impl (this=0x7fff5fbfe9b8) at nsTArray.h:764 #31 0x00000001027b9f55 in ~nsTArray (this=0x7fff5fbfe9b8) at nsIHttpChannelInternal.h:17 #32 0x00000001027b3285 in ~nsTArray (this=0x7fff5fbfe9b8) at nsIHttpChannelInternal.h:17 #33 0x00000001027a9a0e in imgLoader::EvictEntries (this=0x121d0f900, aCacheToClear=@0x121d0f940) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:1515 #34 0x00000001027a87f1 in imgLoader::ClearImageCache (this=0x121d0f900) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:993 #35 0x00000001027a86ed in ~imgLoader (this=0x121d0f900) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:779 #36 0x00000001027a8675 in ~imgLoader (this=0x121d0f900) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:777 #37 0x00000001027a8649 in ~imgLoader (this=0x121d0f900) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:777 #38 0x00000001027a8069 in imgLoader::Release (this=0x121d0f900) at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:732 #39 0x00000001027a98ae in imgLoader::Shutdown () at /Users/chris/Code/mozilla/inbound/image/src/imgLoader.cpp:981 #40 0x00000001027a4f2b in mozilla::image::ShutdownModule () at /Users/chris/Code/mozilla/inbound/image/build/nsImageModule.cpp:103 #41 0x000000010328c4cd in LayoutModuleDtor () at nsLayoutModule.cpp:1258 #42 0x00000001016aabdd in ~KnownModule (this=0x11081ef00) at nsComponentManager.h:226 #43 0x00000001016aab95 in ~KnownModule (this=0x11081ef00) at nsComponentManager.h:224 #44 0x00000001016aab5d in ~nsAutoPtr (this=0x1003752c8) at nsAutoPtr.h:72 #45 0x00000001016aab25 in ~nsAutoPtr (this=0x1003752c8) at nsAutoPtr.h:71 #46 0x00000001016aab05 in nsTArrayElementTraits<nsAutoPtr<nsComponentManagerImpl::KnownModule> >::Destruct (e=0x1003752c8) at nsTArray.h:536 #47 0x00000001016aaaa6 in nsTArray_Impl<nsAutoPtr<nsComponentManagerImpl::KnownModule>, nsTArrayInfallibleAllocator>::DestructRange (this=0x10039f460, start=0, count=56) at nsTArray.h:1585 #48 0x00000001016aaa24 in nsTArray_Impl<nsAutoPtr<nsComponentManagerImpl::KnownModule>, nsTArrayInfallibleAllocator>::RemoveElementsAt (this=0x10039f460, start=0, count=56) at nsTArray.h:1302 #49 0x00000001016a79ff in nsTArray_Impl<nsAutoPtr<nsComponentManagerImpl::KnownModule>, nsTArrayInfallibleAllocator>::Clear (this=0x10039f460) at nsTArray.h:1313 #50 0x00000001016a312c in nsComponentManagerImpl::Shutdown (this=0x10039f330) at /Users/chris/Code/mozilla/inbound/xpcom/components/nsComponentManager.cpp:788 #51 0x00000001015c2433 in mozilla::ShutdownXPCOM (servMgr=0x0) at nsXPComInit.cpp:990 #52 0x00000001015c1dc5 in NS_ShutdownXPCOM (servMgr=0x10039f338) at nsXPComInit.cpp:819 #53 0x0000000104d74c8a in ~ScopedXPCOMStartup (this=0x10030f480) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:1202 #54 0x0000000104d74b95 in ~ScopedXPCOMStartup (this=0x10030f480) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:1183 #55 0x0000000104d7d9a2 in XREMain::XRE_main (this=0x7fff5fbfefa0, argc=3, argv=0x7fff5fbff8a8, aAppData=0x7fff5fbff238) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:4113 #56 0x0000000104d7dd6d in XRE_main (argc=3, argv=0x7fff5fbff8a8, aAppData=0x7fff5fbff238, aFlags=0) at /Users/chris/Code/mozilla/inbound/toolkit/xre/nsAppRunner.cpp:4300 #57 0x000000010000210f in do_main (argc=3, argv=0x7fff5fbff8a8, xreDirectory=0x100322340) at /Users/chris/Code/mozilla/inbound/browser/app/nsBrowserApp.cpp:282 #58 0x0000000100001633 in main (argc=3, argv=0x7fff5fbff8a8) at /Users/chris/Code/mozilla/inbound/browser/app/nsBrowserApp.cpp:643 (gdb)

crash test automation sees this assertion on literally hundreds of web sites for all 3 branches and all platforms. This is so common that the crash test automation is spending most of its time finding and reproducing this assertion. Just reproduced this on shutdown with a fresh Nightly build on Linux x86_64 with http://minnesota.cbslocal.com/2014/04/08/digital-bracelet-company-may-end-up-in-court-after-rash-reports/

Hmm. I was hoping bug 913138 would fix this, but I guess not.

The stack still looks similar to comment 10, except that this is happening: 14 imgLoader::Shutdown() + 22 (imgLoader.cpp:981) 15 mozilla::image::ShutdownModule() + 18 (nsImageModule.cpp:104) 16 LayoutModuleDtor() + 184 (nsLayoutModule.cpp:1259) 17 nsTArray_Impl<nsAutoPtr<nsComponentManagerImpl::KnownModule>, nsTArrayInfallibleAllocator>::RemoveElementsAt(unsigned int, unsigned int) + 124 (nsCOMPtr.h:530) 18 nsComponentManagerImpl::Shutdown() + 389 (nsComponentManager.cpp:790) 19 mozilla::ShutdownXPCOM(nsIServiceManager*) + 1537 (nsXPComInit.cpp:990) 20 ScopedXPCOMStartup::~ScopedXPCOMStartup() + 186 (nsAppRunner.cpp:1203) Turns out I didn't read bug 913138 very closely, so it looks like it actually moved image lib shutdown later, so that's not going to help. The basic problem here is that nsPrincipals go away very late in shutdown with these image things, and those keep alive CSP implementations, which are in JS, so they use XPCWrappedJS. cpeterson filed this on 4-8, so if he was using inbound, it is actually possible this is a regression from bug 913138, rather than something that would be fixed by it. bc, did you only start seeing this all over the place recently? Around 4-9?

We could also adjust the cycle collector to just deal with this situation, at the cost of an additional branch in Suspect, which probably isn't the end of the world. That's probably better than relying on the spinning plates of shutdown to remain reasonable. We'd probably want to set data to some non-zero sentinel value, to differentiate the case where we're on a thread that had a cycle collector that shutdown from the case where we are on a thread that never had a cycle collector. In the latter case, we still want to crash. What do you think, Olli?

(In reply to Andrew McCreight [:mccr8] from comment #14) > bc, did you only start seeing this all over the place recently? Around 4-9? Yes. If it would help I can try to bisect this.

Yeah, that would be useful, thanks. My guess is that this is due to bug 913138, so I'd just start by checking if the landing of that caused this.

(In reply to Andrew McCreight [:mccr8] from comment #17) backing out bug 913138 solved the assertion for me.

Hm. I think bug 913138 is definitely the "cause" here. In that bug, I forced imagelib to shut down _after_ layout/xpc, because layout/xpc depends heavily on gfx/imagelib. And this shows that there's an inverse dependency as well. Yuck. Let me upload a patch that would naively fix this.

bholley: patch fixes the issue for me on Linux x86_64.

(In reply to Andrew McCreight [:mccr8] from comment #15) > We could also adjust the cycle collector to just deal with this situation, > at the cost of an additional branch in Suspect, which probably isn't the end > of the world. That's probably better than relying on the spinning plates of > shutdown to remain reasonable. > > We'd probably want to set data to some non-zero sentinel value, to > differentiate the case where we're on a thread that had a cycle collector > that shutdown from the case where we are on a thread that never had a cycle > collector. In the latter case, we still want to crash. > > What do you think, Olli? Adding anything to Suspect would be unfortunate. It is super hot code. But looks like bholley's patch should be fine here.

Comment on attachment 8405809 [details] [diff] [review] Shut down CAPS and XPConnect after imagelib and gfx. v1 Review of attachment 8405809 [details] [diff] [review]: ----------------------------------------------------------------- bholley wants to get this into a beta real quick, so I'm going to r+ on the condition that someone who knows something about Gecko shutdown reviews it later.

Comment on attachment 8405809 [details] [diff] [review] Shut down CAPS and XPConnect after imagelib and gfx. v1 Should we try to get this in before the B8 go-to-build tomorrow? [Approval Request Comment] Bug caused by (feature/regressing bug #): bug 913138 User impact if declined: crashes Testing completed (on m-c, etc.): Just pushed to m-i Risk to taking this patch (and alternatives if risky): Shutdown order munging is always moderately risky, but this is probably better than the setup we had with bug 913138. String or IDL/UUID changes made by this patch: none

For the release managers: If this doesn't make it into beta, aurora, in my opinion the original regressing patch should be backed out on those branches. The regression is so common that it interferes with crash automation and would prevent me from testing those branches.

Bob, my plan is to have it for beta 8 but I am waiting for this patch to land in m-c before approving it.

"moderately risky" + "reproducible" => verifyme :)

https://hg.mozilla.org/releases/mozilla-aurora/rev/44b74caaeed4 https://hg.mozilla.org/releases/mozilla-beta/rev/afc5f648e247

I rebuilt my local linux x86_64 instance from m-c and could not reproduce. Once the patch lands in beta and aurora I will trigger new builds for all 3 branches in crash test automation and will report back on whether or not the assertion has disappeared.

fyi, i retested a number of urls and could not reproduce on windows or linux or osx. the osx workers are kind of lagging behind due to the backlog but from what i can tell this is fixed. if anything untoward shows up, i'll let you know asap. thanks!

(In reply to Bob Clary [:bc:] from comment #33) > fyi, i retested a number of urls and could not reproduce on windows or linux > or osx. the osx workers are kind of lagging behind due to the backlog but > from what i can tell this is fixed. if anything untoward shows up, i'll let > you know asap. thanks! Bob, did you manage to retest on all 3 Branches?

Florin, yes. The OSX workers are still catching up but I haven't seen this assertion since the new builds created after the patch landed. I would say it is verified on all 3 branches and 3 platforms.

Marking this as verified based on the previous comment... thanks Bob.