This is a new topcrash for Thunderbird 38, starting with 38.0.1. #2 crash for 38.1.0 #4 crash for 38.0.1 Earliest crashes found are version 37 beta, eg bp-c4a2ca8b-2a4b-4f7a-bbb0-79a5f2150322 (but I can only go back 6 months). Crash was common throughout version 38 beta process, however ranked only #6. Almost no crashes in aurora and trunk - for the past 6 months only bp-9c88a427-accc-4577-b9b9-76c2f2150419 40.0a1 and bp-a9b6a133-2c04-4ade-85ea-c5c1c2150527 40.0a2 Did not analyze anything - whether all crashes with this signature have the same precise cause, or whether specific graphics drivers are involved. And I don't know, for example, whether we are seeing examples of bug 1160157. That said, rkent had a crash which might fit a graphcs reset bp-d08dbc42-3bf9-4bf7-ba05-7e1632150401 where he writes " rkent: hung for about 15 seconds with white screen, then crashed when I tried to open something". NOTE, we have 3 graphics crashes all new in version 38, all highly ranked, so one might presume they are related count rank 2 3.51% 0.46% 1179 mozilla::layers::SyncObjectD3D11::FinalizeFrame() rank 3 3.37% 0.6% 1134 DrawingContext::FillRectangle(D2D_RECT_F const*, ID2D1Brush*) rank 4 3.36% 0.98% 1131 D2DDeviceContextBase<T>::FillRectangle(D2D_RECT_F const*, ID2D1Brush*) In the version 38 beta process, SyncObjectD3D11 was only ranked 6. The D2D crashes ranked higher, in the top 5. bp-0044d671-c982-4014-8c52-5c5dd2150722 38.0.1 0 xul.dll mozilla::layers::SyncObjectD3D11::FinalizeFrame() gfx/layers/d3d11/TextureD3D11.cpp 1 xul.dll mozilla::layers::ClientLayerManager::ForwardTransaction(bool) gfx/layers/client/ClientLayerManager.cpp 2 xul.dll mozilla::layers::ClientLayerManager::EndTransaction(void (*)(mozilla::layers::PaintedLayer*, gfxContext*, nsIntRegion const&, mozilla::layers::DrawRegionClip, nsIntRegion const&, void*), void*, mozilla::layers::LayerManager::EndTransactionFlags) gfx/layers/client/ClientLayerManager.cpp 3 xul.dll nsDisplayList::PaintRoot(nsDisplayListBuilder*, nsRenderingContext*, unsigned int) layout/base/nsDisplayList.cpp 4 xul.dll nsLayoutUtils::PaintFrame(nsRenderingContext*, nsIFrame*, nsRegion const&, unsigned int, unsigned int) layout/base/nsLayoutUtils.cpp 5 xul.dll PresShell::Paint(nsView*, nsRegion const&, unsigned int) layout/base/nsPresShell.cpp 6 xul.dll nsViewManager::ProcessPendingUpdatesPaint(nsIWidget*) view/nsViewManager.cpp 7 xul.dll nsViewManager::ProcessPendingUpdates() view/nsViewManager.cpp 8 xul.dll nsRefreshDriver::Tick(__int64, mozilla::TimeStamp) layout/base/nsRefreshDriver.cpp 9 xul.dll mozilla::RefreshDriverTimer::TickDriver(nsRefreshDriver*, __int64, mozilla::TimeStamp) layout/base/nsRefreshDriver.cpp 10 xul.dll mozilla::RefreshDriverTimer::Tick() layout/base/nsRefreshDriver.cpp 11 nss3.dll md_UnlockAndPostNotifies nsprpub/pr/src/md/windows/w95cv.c The user of the above crash report has 12 crashes in ~6 days: igd10umd32.dll@0x308b4e bp-04d26b1a-4a7e-4c59-81e2-832d42150722 nvwgf2um.dll@0xaa8e5 bp-7e43dc5c-a6b7-4534-b7ed-848902150721 RtlpLowFragHeapFree | RtlFreeHeap | HeapFree | free | __free_lc_time | msvcr120.dll@0x29d7f | msvcr120.dll@0x1453f | _expandlocale bp-e174ec0c-7ed7-4e3d-aa50-b42c62150720 - Crash Rate - 12 crashes in 6 days is more than the average user, based on a sample of 15 from the last 7 days - this from a subset of 25 users who provided an email address and at least one comment. For these users searched for all of their crashes for 2015. Of the 15 - 4 have only one crash. - of the remaining 11, the average crash rate looks to be only about 2-3 per week. - Signature Breakdown - Breakdown of crash signatures: 7 d3d only 2 d3d+video drivers 2 d3d+drivers+others 4 d3d+others, no driver ("other" sigs tends to be d2d crashes) ** N.B. only 2 users crashed in 31.7 in the past 6 months An example of SyncObjectD3D11 with other signatures who also crashed on TB31.7 2cdd63d5-b4ec-40c0-9aa7-7c4ba2150714 DrawingContext::FillRectangle(D2D_RECT_F const*, ID2D1Brush*) 64389c67-64c7-4b3d-b6ca-e829a2150707 mozilla::layers::SyncObjectD3D11::FinalizeFrame() d18aa487-815f-403d-92a9-b56ba2150707 mozilla::gfx::DrawTargetD2D1::Init(mozilla::gfx::IntSizeTyped<T> const&, mozilla::gfx::SurfaceFormat) 4b2bb72c-7d7a-43d6-b8eb-fe4da2150626 mozilla::gfx::DrawTargetD2D1::Init(mozilla::gfx::IntSizeTyped<T> const&, mozilla::gfx::SurfaceFormat) 6f800444-a246-4fe8-b825-5857f2150604 OOM | small TB31.7 An example of SyncObjectD3D11 with other signatures who also crashed on TB31.7 a0e931ae-c6ea-4958-8616-340732150723 mozilla::layers::SyncObjectD3D11::FinalizeFrame() bp-9219312c-004e-4568-9a4b-9a5372150629 nsTextServicesDocument::FirstTextNodeInNextBlock(nsIContentIterator*) TB31.7 (nsTextServicesDocument was also seen for a few other users) An example of SyncObjectD3D11 with D2D signature a9aa9b51-b8e2-4670-ac7e-686932150714 D2DDeviceContextBase<T>::FillRectangle(D2D_RECT_F const*, ID2D1Brush*) e37178b8-e058-48aa-98e6-b998c2150709 mozilla::layers::SyncObjectD3D11::FinalizeFrame()

70-80% have uptime >1hr rank pct OS 2 3.37% win7 DrawingContext::FillRectangle(D2D_RECT_F const*, ID2D1Brush*) 3 3.36% win8/8.1 D2DDeviceContextBase<T>::FillRectangle(D2D_RECT_F const*, ID2D1Brush*) 4 3.51% win7 65% mozilla::layers::SyncObjectD3D11::FinalizeFrame() win8 32% bp-d4c3620e-774c-45d2-acc3-fb1b32150726 mozilla::layers::SyncObjectD3D11::FinalizeFrame() http://hg.mozilla.org/releases/mozilla-esr38/annotate/b32d199c886c/gfx/layers/client/ClientLayerManager.cpp#l318 mstange@232045 310 ClientLayerManager::EndTransaction(DrawPaintedLayerCallback aCallback, mwoodrow@142225 311 void* aCallbackData, mwoodrow@142225 312 EndTransactionFlags aFlags) jones@48140 313 { mwoodrow@141231 314 if (mWidget) { mwoodrow@141231 315 mWidget->PrepareWindowEffects(); mwoodrow@141231 316 } mwoodrow@142225 317 EndTransactionInternal(aCallback, aCallbackData, aFlags); mwoodrow@177593 318 ForwardTransaction(!(aFlags & END_NO_REMOTE_COMPOSITE)); http://hg.mozilla.org/releases/mozilla-esr38/annotate/b32d199c886c/gfx/layers/client/ClientLayerManager.cpp#l552 mwoodrow@177593 549 ClientLayerManager::ForwardTransaction(bool aScheduleComposite) romaxa@59683 550 { bschouten@247140 551 if (mForwarder->GetSyncObject()) { bschouten@247140 552 mForwarder->GetSyncObject()->FinalizeFrame(); http://hg.mozilla.org/releases/mozilla-esr38/annotate/b32d199c886c/gfx/layers/d3d11/TextureD3D11.cpp#l845 bschouten@250628 844 if (hr == WAIT_TIMEOUT) { bschouten@250628 845 MOZ_CRASH(); TDR?

Do similar crashes show up in FF38? Bas is probably a better person to look at this than me.

(In reply to Jeff Muizelaar [:jrmuizel] from comment #2) > Do similar crashes show up in FF38? Yes https://crash-stats.mozilla.com/search/?product=Firefox&signature=%3Dmozilla%3A%3Alayers%3A%3ASyncObjectD3D11%3A%3AFinalizeFrame%28%29&version=38.0.1&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature But not in as high a rate as Thunderbird - this is a major crash for Thundrebird 38. Per 1143806, 75% of Firefox crash volume for the signature was killed by bug 1131370, fixed in firefox 37 and 38. I can't tell whether or to what extent bug 1131370 would have helped since Thunderbird users ride the esr track. Bug 1143806 took this further in FF 39. However, bug 1143806 has not been uplifted into ESR. Perhaps getting 1143806 into ESR would help Thunderbird and firefox too https://crash-stats.mozilla.com/signature/?product=Firefox&version=38.1.0esr&signature=mozilla%3A%3Alayers%3A%3ASyncObjectD3D11%3A%3AFinalizeFrame%28%29&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&page=1 .... for example bp-ffe1df66-1c4b-4bba-9aa1-319412150722

We just recently built 40beta for Thunderbird, which will have the fix bug 1143806. So I'm going to try to get a users to try it. One of these users tells me crash "usually only happens once per day, after waking PC from sleep or hibernate mode."

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #4) > We just recently built 40beta for Thunderbird, which will have the fix bug > 1143806. So I'm going to try to get a users to try it. > > One of these users tells me crash "usually only happens once per day, after > waking PC from sleep or hibernate mode." I've been in contact with the user who reported this - https://crash-stats.mozilla.com/report/index/7babaf24-6dbc-4f2e-963a-fb5462150720 - and in using version 40 the D3D crash is gone. This should indicate the patch from bug 1143806 was effective. 40.0b1 crash stats seem to agree. It's a bit early for truely solid results, after 3 days of 40.0b1 D3D is only ranked #10, with only two crashes, https://crash-stats.mozilla.com/search/?product=Thunderbird&version=40.0b1&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature bp-c61878ac-19f3-4a47-a2e1-d1a812150730 bp-4463ea4c-b844-4e1a-8255-e1fdd2150730 Compare to version 38.0b6 where D3D has crashes ranking #2, #27 and #38 https://crash-stats.mozilla.com/search/?product=Thunderbird&version=38.0b6&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature Will request uplift of bug 1143806.

Related to Nvidia graphics driver update? Saw this with Firefox Nightly 20150804 on Windows 8.1 64 bit with Nvidia 765M and Intel Graphics HD4600 when upgrading from 353.30-notebook-win8-win7-64bit-international-whql.exe to 353.62-notebook-win8-win7-64bit-international-whql.exe Thunderbird showed black areas, unresponsiveness of UI etc. (like Firefox 39.0 and latest Beta).

(In reply to Aryx [:archaeopteryx][:aryx] from comment #7) > Thunderbird showed black areas, unresponsiveness of UI etc. (like Firefox > 39.0 and latest Beta). Which Thunderbird version? Was Thunderbird running at the time of the driver update?

Thunderbird versions 38.1.0, 40.0b1 and Daily 42.0a1 20150803. They all were running during the graphics driver update (which didn't prompt for a restart).

Bug 1143806 landed in mozilla-esr38 so we will see this in Thunderbird 38.2.0 I'm going to stop tracking this bug for TB 38 unless it appears again as a top crasher. "fixed" for status may be optimistic, feel free to change that if this continues.

Approval Request Comment [Feature/regressing bug #]: Unknown [User impact if declined]: Top crasher [Describe test coverage new/current, TreeHerder]: None [Risks and why]: Low, simply a nullcheck and a subsequent return, crash might move and stay, but won't introduce new risks. [String/UUID change made/needed]: None Let's see if this will address the issue. It should be safe for beta, on nightly and aurora it seems David's patches to restructure D2D/D3D11 usage have taken care of this.

Somewhat contrary to comment 6, currently the signature is #1 and #38 crash for 40.0b1 built ~24-Jul-2015. (and HWA is still enabled in beta) The signature is virtually gone in Thunderbird 38.2.0 (only 3 crashes) bp-78544fe2-401b-4e5d-86db-c50912150818 bp-588397c7-ea80-41b6-8c9d-3d6392150817 bp-04af9f66-56ea-4bc1-be8c-075d42150818 But it must be noted that HWA is disabled in Thunderbird 38.2.0

Comment on attachment 8649521 [details] [diff] [review] Abort synchronization if our device has gone away Patch seems safe to uplift to Beta, and since it's a crash fix we should uplift soon.

Bas, is the fix in Firefox code? I am assuming yes but got confused because the product is "Thunderbird" and that I was unable to find the status-firefox41 flag.

(In reply to Ritu Kothari (:ritu) from comment #13) > Comment on attachment 8649521 [details] [diff] [review] > Abort synchronization if our device has gone away > > Patch seems safe to uplift to Beta, and since it's a crash fix we should > uplift soon. It's gecko, it should affect both.

Unfortunately our nightly builds are broken so we won't be able to verify on Thunderbird for some days

[Tracking Requested - why for this release]: Not sure if this is something we'd track for an Fx40 dot release or not.

Too late for 40.