+++ This bug was initially created as a clone of Bug #1347625 +++ https://treeherder.mozilla.org/logviewer.html#?job_id=138816775&repo=autoland [task 2017-10-23T08:43:13.301Z] 08:43:13 INFO - GECKO(1485) | [GFX1]: Texture deallocated too late during shutdown [task 2017-10-23T08:43:13.302Z] 08:43:13 INFO - GECKO(1485) | Assertion failure: [GFX1]: Texture deallocated too late during shutdown, at /builds/worker/workspace/build/src/gfx/2d/Logging.h:520 [task 2017-10-23T08:43:48.002Z] 08:43:48 INFO - GECKO(1485) | #01: mozilla::gfx::Log<1, mozilla::gfx::CriticalLogger>::~Log [gfx/2d/Logging.h:524] [task 2017-10-23T08:43:48.002Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.002Z] 08:43:48 INFO - GECKO(1485) | #02: mozilla::layers::DeallocateTextureClient [gfx/layers/client/TextureClient.cpp:330] [task 2017-10-23T08:43:48.003Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.004Z] 08:43:48 INFO - GECKO(1485) | #03: mozilla::layers::TextureClient::Destroy [gfx/layers/client/TextureClient.cpp:430] [task 2017-10-23T08:43:48.004Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.004Z] 08:43:48 INFO - GECKO(1485) | #04: mozilla::layers::TextureClient::~TextureClient [gfx/layers/client/TextureClient.cpp:610] [task 2017-10-23T08:43:48.005Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.005Z] 08:43:48 INFO - GECKO(1485) | #05: mozilla::layers::TextureClient::~TextureClient [gfx/layers/client/TextureClient.cpp:617] [task 2017-10-23T08:43:48.005Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.006Z] 08:43:48 INFO - GECKO(1485) | #06: mozilla::AtomicRefCountedWithFinalize<mozilla::layers::TextureClient>::Release [gfx/layers/AtomicRefCountedWithFinalize.h:144] [task 2017-10-23T08:43:48.007Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.007Z] 08:43:48 INFO - GECKO(1485) | #07: PLDHashTable::~PLDHashTable [xpcom/ds/PLDHashTable.cpp:327] [task 2017-10-23T08:43:48.008Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.008Z] 08:43:48 INFO - GECKO(1485) | #08: mozilla::layers::SourceSurfaceImage::~SourceSurfaceImage [xpcom/ds/nsTHashtable.h:391] [task 2017-10-23T08:43:48.009Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.009Z] 08:43:48 INFO - GECKO(1485) | #09: mozilla::layers::SourceSurfaceImage::~SourceSurfaceImage [gfx/layers/ImageContainer.h:935] [task 2017-10-23T08:43:48.009Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.010Z] 08:43:48 INFO - GECKO(1485) | #10: mozilla::layers::Image::Release [gfx/layers/ImageContainer.h:199] [task 2017-10-23T08:43:48.011Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.012Z] 08:43:48 INFO - GECKO(1485) | #11: mozilla::dom::OutputStreamDriver::StreamListener::~StreamListener [dom/media/CanvasCaptureMediaStream.cpp:74] [task 2017-10-23T08:43:48.013Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.015Z] 08:43:48 INFO - GECKO(1485) | #12: mozilla::dom::OutputStreamDriver::StreamListener::~StreamListener [dom/media/CanvasCaptureMediaStream.cpp:74] [task 2017-10-23T08:43:48.017Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.019Z] 08:43:48 INFO - GECKO(1485) | #13: mozilla::MediaStreamListener::Release [dom/media/MediaStreamListener.h:48] [task 2017-10-23T08:43:48.020Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.021Z] 08:43:48 INFO - GECKO(1485) | #14: mozilla::MediaStream::RemoveAllListenersImpl [dom/media/MediaStreamGraph.cpp:2077] [task 2017-10-23T08:43:48.023Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.024Z] 08:43:48 INFO - GECKO(1485) | #15: mozilla::MediaStream::Destroy()::Message::Run() [task 2017-10-23T08:43:48.025Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.026Z] 08:43:48 INFO - GECKO(1485) | #16: mozilla::MediaStreamGraphImpl::AppendMessage [dom/media/MediaStreamGraph.cpp:1881] [task 2017-10-23T08:43:48.027Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.029Z] 08:43:48 INFO - GECKO(1485) | #17: mozilla::MediaStream::Destroy [dom/media/MediaStreamGraph.cpp:2114] [task 2017-10-23T08:43:48.030Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.031Z] 08:43:48 INFO - GECKO(1485) | #18: mozilla::DOMMediaStream::Destroy [dom/media/DOMMediaStream.cpp:498] [task 2017-10-23T08:43:48.033Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.035Z] 08:43:48 INFO - GECKO(1485) | #19: mozilla::DOMMediaStream::cycleCollection::Unlink [dom/media/DOMMediaStream.cpp:382] [task 2017-10-23T08:43:48.036Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.038Z] 08:43:48 INFO - GECKO(1485) | #20: mozilla::dom::CanvasCaptureMediaStream::cycleCollection::Unlink [mfbt/RefPtr.h:63] [task 2017-10-23T08:43:48.040Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.042Z] 08:43:48 INFO - GECKO(1485) | #21: nsCycleCollector::CollectWhite [xpcom/base/nsCycleCollector.cpp:3403] [task 2017-10-23T08:43:48.044Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.045Z] 08:43:48 INFO - GECKO(1485) | #22: nsCycleCollector::Collect [xpcom/base/nsCycleCollector.cpp:3769] [task 2017-10-23T08:43:48.047Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.048Z] 08:43:48 INFO - GECKO(1485) | #23: nsCycleCollector::ShutdownCollect [xpcom/base/nsCycleCollector.cpp:3687] [task 2017-10-23T08:43:48.050Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.052Z] 08:43:48 INFO - GECKO(1485) | #24: nsCycleCollector_shutdown [mfbt/RefPtr.h:63] [task 2017-10-23T08:43:48.053Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.054Z] 08:43:48 INFO - GECKO(1485) | #25: mozilla::ShutdownXPCOM [xpcom/build/XPCOMInit.cpp:987] [task 2017-10-23T08:43:48.056Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.057Z] 08:43:48 INFO - GECKO(1485) | #26: XRE_TermEmbedding [toolkit/xre/nsEmbedFunctions.cpp:225] [task 2017-10-23T08:43:48.059Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.060Z] 08:43:48 INFO - GECKO(1485) | #27: mozilla::ipc::ScopedXREEmbed::Stop [ipc/glue/ScopedXREEmbed.cpp:109] [task 2017-10-23T08:43:48.061Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.063Z] 08:43:48 INFO - GECKO(1485) | #28: XRE_InitChildProcess [toolkit/xre/nsEmbedFunctions.cpp:708] [task 2017-10-23T08:43:48.064Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.093Z] 08:43:48 INFO - GECKO(1485) | #29: content_process_main [ipc/contentproc/plugin-container.cpp:65] [task 2017-10-23T08:43:48.094Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.095Z] 08:43:48 INFO - GECKO(1485) | #30: main [browser/app/nsBrowserApp.cpp:283] [task 2017-10-23T08:43:48.095Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.096Z] 08:43:48 INFO - GECKO(1485) | #31: libc.so.6 + 0x20830 [task 2017-10-23T08:43:48.096Z] 08:43:48 INFO - [task 2017-10-23T08:43:48.097Z] 08:43:48 INFO - GECKO(1485) | #32: _start

(In reply to Sebastian Hengst [:aryx][:archaeopteryx] (needinfo on intermittent or backout) from comment #0) > [task 2017-10-23T08:43:48.038Z] 08:43:48 INFO - GECKO(1485) | #20: > mozilla::dom::CanvasCaptureMediaStream::cycleCollection::Unlink > [mfbt/RefPtr.h:63] My hunch is this is bug 1407542.

It's related in the sense that they're both about cycle collecting DOMMediaStreams, yes, but I don't think bug 1407542 will be a definite fix for this. Other classes can hold references to DOMMediaStream (though I admit it's unclear if there would be cycles from those). This has probably been rare because it would normally get cycle collected before reaching shutdown. Maybe the chunking on try changed. We could have MediaStreamGraph signal all MediaStreamListeners that shutdown is occurring. MSG should go through shutdown, [1], before graphics, [2], which goes through shutdown before the cycle collector, [3]. Destructing CanvasCaptureMediaStream's OutputStreamDriver::mImage during unlink (so [3] when this fails) is what causes issues. One day we get rid of VideoChunk durations and push straight to the SourceMediaStream (i.e., get rid of that mImage), and this will be a non-problem. [1] http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/xpcom/build/XPCOMInit.cpp#876 [2] http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/xpcom/build/XPCOMInit.cpp#885 [3] http://searchfox.org/mozilla-central/rev/1285ae3e810e2dbf2652c49ee539e3199fcfe820/xpcom/build/XPCOMInit.cpp#985

Hmm, The MediaStreamListener is notified of removal from the graph *after* the graph has cleared its shutdown blocker. That sounds wrong.

(In reply to Andreas Pehrson [:pehrsons] from comment #3) > Hmm, The MediaStreamListener is notified of removal from the graph *after* > the graph has cleared its shutdown blocker. That sounds wrong. Because it was only removed on stream destruction (so on cyclecollector shutdown in the worst case). I'm fixing that.

Comment on attachment 8922761 [details] Bug 1410829 - Signal listeners removed on shutdown so they can clean up. https://reviewboard.mozilla.org/r/193866/#review199090 ::: dom/media/MediaStreamGraph.cpp:2098 (Diff revision 1) > + } > + mTrackListeners.Clear(); > + > + if (SourceMediaStream* source = AsSourceStream()) { > + source->RemoveAllDirectListeners(); > + } Can you update the commit message to explain this? I think I understand, but it's a bit late, so I kind of doubt I'm getting things right. Are we just being explicit here to signal to all the track, direct and stream (that was always the case) listeners that they are being removed, so they can clear their gfx resources references? In the past, we used the fact that the different object's dtor were called, so we wouldn't leak, but we also wouln't release the texture properly, right?

Comment on attachment 8922762 [details] Bug 1410829 - Clean up streams and listeners before unblocking shutdown. https://reviewboard.mozilla.org/r/193868/#review199094

Comment on attachment 8922761 [details] Bug 1410829 - Signal listeners removed on shutdown so they can clean up. https://reviewboard.mozilla.org/r/193866/#review199096 This looks like it's correct, but see below. r+ with a longer/clearer commit message if I understood it right, r- otherwise.

Could you elaborate on "In the past, we used the fact that the different object's dtor were called"? E.g., what would "different object" be? What we now get is the shutdown sequence: [media, gfx, cycleCollector] dtors are called on cycleCollector shutdown when this fails, because that releases images after gfx shutdown and we hit this hard assert. We need to move this cleanup to media shutdown, so that's basically what the patches do.

Comment on attachment 8922763 [details] Bug 1410829 - Clear out mImage on media shutdown. https://reviewboard.mozilla.org/r/193870/#review200854 lgtm.

Pushed by pehrsons@gmail.com: https://hg.mozilla.org/integration/autoland/rev/db0fa983b90f Signal listeners removed on shutdown so they can clean up. r=padenot https://hg.mozilla.org/integration/autoland/rev/d9257f9b2296 Clean up streams and listeners before unblocking shutdown. r=padenot https://hg.mozilla.org/integration/autoland/rev/0253bbb60b2f Clear out mImage on media shutdown. r=jib

https://hg.mozilla.org/mozilla-central/rev/db0fa983b90f https://hg.mozilla.org/mozilla-central/rev/d9257f9b2296 https://hg.mozilla.org/mozilla-central/rev/0253bbb60b2f

Clearing NI.