+++ This bug was initially created as a clone of Bug #950133 +++ This bug is to explore a suggestion that we disable HWA. I'm sure Matt won't mind me quoting him "Reverting 3d acceleration to off in preferences should also happen. It does little or nothing for Thunderbird but messes menus, font and causes crashes (the kind with no crash reporter reports)." Possible directions: 1) disable HWA, 2) no action, 3) some form of mitigation short of complete disabling, 4) create/improve our support articles For some history ... Bug 950133 reenabled hardware acceleration which had previously been disabled in bug 689742, which at that time "was providing a distinctly worse user experience by using discrete GPUs." To further quote bug 689742 "In the intervening two years, the graphics situation has changed, and the use of discrete GPUs apparently now requires use of WebGL, so the negative effects originally experienced should no longer be present (this is as discussed in bug 949756 comment 8)." Instead, our disabling of hardware acceleration but not a myriad of other graphics preferences puts us in an untested situation of graphics code. The current week alone has seen two separate comm-central-only bustages due to the lack of this preference. In light of the changing situation, the decision was made to revert the changes in bug 689742 and restore Thunderbird to the same graphics configuration as Firefox.

Matt, please provide several representative support URLs that span the range of issues seen in support

I currently do not see many bug reports that are clearly HWA. But note, we might not have this class of bugs well triaged. The list of reported bugs, both closed and open: INVALID - Bug 1061971 - Thunderbird 31.1.0 - Aero with HWA enabled causes corrupted Minimise,Close buttons in title bar and fonts in both messages and screen elements/characters are not sharp (driver issue) OPEN - Bug 693852 - High GPU usage with constant status bar animation and direct2d (from bug 689742) OPEN - Bug 1070754 - Menus don't render immediately if hardware acceleration is on crashes ... OPEN - Bug 1056627 - HWA crash - igd10umd32.dll, igd10iumd32.dll, atidxx32.dll, nvwgf2um.dll (UNCONFIRMED possibly other signatures with "igd10" #83, #98, #200, #210, #217, #235, #251, #259, #261, #266) OPEN - Bug 1116885 - (#77 crash) HWA startup crash in DWrite.dll FindElementCommon(ICacheReference*, bool, CacheHeader const&, unsigned int const*, HashElement const*, void const*, IElementKey const&, CachedElementData*) OPEN - Bug 1126982 - (#60 for 31.4.0) HWA crash in CContext::CaptureStateImpl<int>(SAPIPipelineState*) @ 0xffffffffffffffe1 *** combined crash count ranking for Bug 1116885 and Bug 1126982 - #25 for 31.4.0) Related: Bug 949756 - Hit MOZ_CRASH(Couldn't create texture host) at ../../../../mozilla/gfx/layers/composite/TextureHost.cpp:173 during Thunderbird OS X mozmill tests It should also be noted, that as of Bug 1115990 we have "an option to Prefs/Advanced to enable/disable the hardware acceleration"

Accessibility issues https://support.mozilla.org/en-US/questions/1012145 Linux Mint Debian Edition 3.1.2-amd64 Cinnamon 2.0.14 crashes https://support.mozilla.org/en-US/questions/1042068 Blurry display https://support.mozilla.org/en-US/questions/1021568 Blue screen crash https://support.mozilla.org/en-US/questions/1045826 "EVGA PrecisionX 16 v5.2.0" is being superimposed over all tabs. https://support.mozilla.org/en-US/questions/1037163 Overheating of graphics chip https://support.mozilla.org/en-US/questions/1039540 My search Fu is off, I am not finding the grey and black menu ones... other than one that never replied. https://support.mozilla.org/en-US/questions/1043370

I think the argument that having it disabled by default puts us in basically untested territory is a very good reason to keep it enabled.

https://support.mozilla.org/en-US/questions/1046437 Another fuzzy font. Graphics Adapter Description NVIDIA GeForce GTX 660 Adapter Description (GPU #2) Intel(R) HD Graphics 4600 I have not looked until now, but perhaps the issue is dual GPU users.

(In reply to Magnus Melin from comment #4) > I think the argument that having it disabled by default puts us in basically > untested territory is a very good reason to keep it enabled. I think that was part of the rationale for reenabling it. But I not 100% convinced - one would hope that gfx has some automated tests on the baseline code - the non-accelerated related code is still exercised in cases where the vendor or version of driver is blacklisted - surely a non-trivial number of systems - I'm sure gfx is not going to abandon the "old code" I randomly came across two other references while looking for crash reports in the support forum https://support.mozilla.org/en-US/questions/1042594 https://support.mozilla.org/en-US/questions/1044982

Sampling of Thunderbird crashes that may be HWA https://crash-stats.mozilla.com/search/?product=Thunderbird&signature=%24igd10&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature I haven't yet got a good query for Firefox for either bug reports or crashes https://bugzilla.mozilla.org/buglist.cgi?keywords=crash&f1=short_desc&keywords_type=allwords&list_id=12014666&short_desc=hwa%20hardware&o1=nowordssubstr&resolution=---&resolution=INVALID&classification=Client%20Software&classification=Components&chfieldto=Now&query_format=advanced&chfieldfrom=2011-01-01&f2=OP&short_desc_type=anywords&product=Core

#33 31.4.0 crash may HWA Bug 903842 - crash in mozilla::layers::LayerManagerD3D10::VerifyBufferSize @ CDXGISwapChain::ResizeBuffers https://crash-stats.mozilla.com/query/?product=Thunderbird&version=ALL%3AALL&range_value=4&range_unit=weeks&date=&query_search=signature&query_type=is_exactly&query=CDXGISwapChain%3A%3AResizeBuffers%28unsigned+int%2C+unsigned+int%2C+unsigned+int%2C+DXGI_FORMAT%2C+unsigned+int%29&reason=&release_channels=&build_id=&process_type=any&hang_type=any

(In reply to Wayne Mery (:wsmwk) from comment #8) > #33 31.4.0 crash may [be] HWA > Bug 903842 - crash in mozilla::layers::LayerManagerD3D10::VerifyBufferSize @ > CDXGISwapChain::ResizeBuffers https://crash-stats.mozilla.com/report/list?signature=CDXGISwapChain%3A%3AResizeBuffers%28unsigned+int%2C+unsigned+int%2C+unsigned+int%2C+DXGI_FORMAT%2C+unsigned+int%29&product=Thunderbird&query_type=is_exactly&range_unit=weeks&process_type=any&hang_type=any&date=2014-08-15+03%3A00%3A00&range_value=4#tab-reports (as best as I can determine) confirms that Thunderbird crashes began in version 29, which is when HWA was reenabled for Thunderbird. Although crashes are extremely rare until the general release of 31. bp-ebc46f49-9ca2-43fb-a2ce-13cdd2140801 TB29 beta build 20140424045348 bp-64c62870-20e5-4f99-927e-ad1c72140806 TB31 beta build 20140715165256

two reports: http://forums.mozillazine.org/viewtopic.php?f=39&t=2857181&p=14067917#p14067917 http://forums.mozillazine.org/viewtopic.php?p=13693775#p13693775

The reports seem to contradict comment 0 "It does little or nothing for Thunderbird"

i hope it is ok to report that here. if not, feel free to contact me or delete this comment. i found this discussion when trying to find an open bug to attach to. i am running thunderbird on my dual gpu macbook (2010) and it constantly switches on the discrete gpu. this more then halves my battery runtime and i can (as a user) see or feel no benefit for it. i use gfxCardStatus and as soon as i start thunderbird, the nvidia gpu is used, when i quit thunderbird, it switches back to the integrated gpu. interestingly, with firefox that does not happen: sometimes the discrete gnu is used (when needed) but turned of again when not…

(In reply to ben from comment #12) > i use gfxCardStatus and as soon as i start thunderbird, the nvidia gpu is > used, when i quit thunderbird, it switches back to the integrated gpu. > interestingly, with firefox that does not happen: sometimes the discrete gnu > is used (when needed) but turned of again when not… Thanks for reporting this. An approximate of recent history of Mac+GPU is http://mzl.la/1bnMz6V As you can see, we had bug 716796 which was fixed by bug 646043, and currently no open Thunderbird reports about this. This is happenign to you with version 31? Does it happen with 37 beta from http://www.mozilla.org/en-US/thunderbird/channel/ ?

I can not reproduce comment 12 on TB 31.2.0, 31.5.0. 36.0, 37.0, 38.0 or the current nightlies. I have a 2013 retina MBP, with two graphics cards on OS X Yosemite. I didn't use gfxCardStatus to track this though, since you can see what card is being used with Activity Monitor. Perhaps the problem has more to do with gfxCardStatus than TB?

@Wayne thanks for asking. I verified with Thunderbird 31.1.5 and 37beta, both with fresh profile and no extensions installed. Discrete as soon as Thunderbird starts and integrated when i quit it. I have a MacBook Pro 15 2010 with Yosemite. And no, this is not related to gfxCardStatus (Comment 14 https://bugzilla.mozilla.org/show_bug.cgi?id=1131879#c14). It would be the only app having a problem with that and i verified by quitting gfxCardStatus and using the Activity Manager. See the screenshots at imgur (german, but you will get the point). http://imgur.com/P2X2HSn,TfdMqZX,Gr7C6hI#0 (first image: activity manager - no thunderbird, second image with thunderbird, third gfxCardStatus).

Sorry for spam, i forgot to report that i checked that disabling the HWA via layers.accelation unsurprisingly prevents from switching to the discrete graphics.

That's odd. Could you provide details of your two graphics cards? For example, I have an (Integrated) Intel HD Graphics 4000 and (Discrete) NVIDA GeForce GT 650M.

Yes, see my technical specs here: https://support.apple.com/kb/SP582?locale=en_US. In short, i have Core i5 Integrated Graphics, simply called Intel HD Graphics with 256MB [1] and a NVIDIA GeForce GT 330M graphics processor with 256MB of GDDR3 ram[2]. Of interest might be the Apple OpenGL compability table [3]. 1: http://ark.intel.com/de/products/43544/Intel-Core-i5-540M-Processor-3M-Cache-2_53-GHz 2: http://www.geforce.com/hardware/notebook-gpus/geforce-gt-330m/specifications 3: https://support.apple.com/en-us/HT202823

(In reply to Magnus Melin from comment #4) > I think the argument that having it disabled by default puts us in basically > untested territory is a very good reason to keep it enabled. What specifically examples or tests (or lack thereof, and afaik it would have to be core because I don't see how thunderbird would have tests in this area) makes you think it puts us in untested territory?

(In reply to Kent James (:rkent) from comment #11) > The reports seem to contradict comment 0 "It does little or nothing for > Thunderbird" Kent, what part of comment 0? N.B. I failed to quote part of comment 0, which is not a statement by me, but rather further comments from Joshua lifted from bug 950133 comment 0 "Instead, our disabling of hardware acceleration but not a myriad of other graphics preferences puts us in an untested situation of graphics code. The current week alone has seen two separate comm-central-only bustages due to the lack of this preference. In light of the changing situation, the decision was made to revert the changes in bug 689742 and restore Thunderbird to the same graphics configuration as Firefox.".... Joshua, do you remember the context? I don't know bug numbers of the issues you refer to.

Ah, so the two bugs Joshua referred to are (unfixed) Bug 949756 - Hit MOZ_CRASH(Couldn't create texture host) at ../../../../mozilla/gfx/layers/composite/TextureHost.cpp:173 during Thunderbird OS X mozmill tests (fixed) Bug 948555 - Assertion failure: compositor (Passed invalid backend hint), at ../../../../mozilla/gfx/layers/ipc/CompositorParent.cpp:731 But is there significantly more to the rationale of enabling HWA than just two failed bug reports?

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #21) > But is there significantly more to the rationale of enabling HWA than just > two failed bug reports? In general, you're talking about a situation that's akin to saying "we should compile with --disable-some-random-feature"--it's a situation that is nominally supported but has little to no testing infrastructure to make it keep working. Outright disabling hardware acceleration (at least, at that time in December 2013) created a situation where no compositor really existed and went down codepaths that weren't in automated tests and weren't really in the minds of reviewers. The forced disabling also induces combinations that weren't intended to exist, I think. The reaction of the graphics people was "it's better to minimize your changes from Firefox" when I brought these multiple failures up to them.

Thanks. That helps us understand that having it disabled hurts us in tests and potentially breaking builds. So my question then is (more accurately), can we have the feature enabled in builds, but preffed off for releases?

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #23) > Thanks. That helps us understand that having it disabled hurts us in tests > and potentially breaking builds. > > So my question then is (more accurately), can we have the feature enabled in > builds, but preffed off for releases? The preffing off is exactly what caused the issues.

Yes having releases be something else than normal builds is not an option. While disabling might help for some smallish amount of users, we must prioritize the 99.9x% that never have any issues with it. For those, "doing what firefox does" naturally gives the most coverage wrt testing, and amount of people able and willing to actually fix the issues.

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #20) > (In reply to Kent James (:rkent) from comment #11) > > The reports seem to contradict comment 0 "It does little or nothing for > > Thunderbird" > > Kent, what part of comment 0? > > I quoted exactly what was being contradicted, "It does little or nothing for Thunderbird" and yet comment 10 reports showed that HWA made a significant difference in Thunderbird performance, which is not "little or nothing".

(In reply to Kent James (:rkent) from comment #26) > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment > #20) > > (In reply to Kent James (:rkent) from comment #11) > > > The reports seem to contradict comment 0 "It does little or nothing for > > > Thunderbird" > > > > Kent, what part of comment 0? > > > > > > I quoted exactly what was being contradicted, "It does little or nothing for > Thunderbird" and yet comment 10 reports showed that HWA made a significant > difference in Thunderbird performance, which is not "little or nothing". Those citations I gave were to convey bad UI behavior. One person [1], who we don't know and never posted to mozillazine before in his life, saying "TB is very speedy again" is hardly a solid comentary about HWA performance. Frankly, looking at his wording again, it's not perfectly clear to me whether his end state was gfx.direct2d.disabled false or true. [1] http://forums.mozillazine.org/viewtopic.php?f=39&t=2857181&p=14067917#p14067917

https://support.mozilla.org/en-US/questions/1021448#answer-714204 @ bp-04e7adc1-d92e-4d51-9938-200662140922

DO we know if the problematic drivers are the same ones that Firefox struggles with, or are Thunderbird issues unique?

(In reply to Kent James (:rkent) from comment #32) > DO we know if the problematic drivers are the same ones that Firefox > struggles with, or are Thunderbird issues unique? I don't know that it's possible to determine. And they are currently becoming more agressive in finding problematic drivers - for example https://groups.google.com/forum/#!topic/mozilla.dev.platform/htK-tBiM03g - but how sustained that will be and how quickly and well it will be addressed time will tell. So my interest here was to disable this until Firefox fully gets their act together, and us take control of an avoidable issue that has high impact to users. Firefox have the luxury of crashes not generally causing dataloss and they also do a great job of saving and restoring session state. Now, not all Thunderbird driver crashes cause dataloss, but we do not save and restore current operating state very and some percentage of crashes definitely cause loss of data other than session state - so the impact of a crash to Thunderbird users is relatively high.

Maybe what we could do is to disable HWA in the betas to start to get some experience with this?

(In reply to Kent James (:rkent) from comment #34) > Maybe what we could do is to disable HWA in the betas to start to get some > experience with this? What would you want is to look for? (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #33) > (In reply to Kent James (:rkent) from comment #32) > > DO we know if the problematic drivers are the same ones that Firefox > > struggles with, or are Thunderbird issues unique? > > I don't know that it's possible to determine. > > And they are currently becoming more agressive in finding problematic > drivers - for example > https://groups.google.com/forum/#!topic/mozilla.dev.platform/htK-tBiM03g - > but how sustained that will be and how quickly and well it will be addressed > time will tell. Along those lines, it doesn't necessarily correlate to what happens with HWA, but several of the top graphics crashes have similar rankings for Firefox and Thunderbird. So hopefully they will get good attention.

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #35) > (In reply to Kent James (:rkent) from comment #34) > > Maybe what we could do is to disable HWA in the betas to start to get some > > experience with this? > > What would you want is to look for? > My main question is, does disabling HWA cause an noticeable decrease in performance for any users?

https://support.mozilla.org/en-US/questions/1069279 This support topic would indicate we have issues on Adapter Description Intel(R) HD Graphics 4600 on Windows 10. Some of the first feedback I have seen in support for Windows 10.

(In reply to Magnus Melin from comment #25) > Yes having releases be something else than normal builds is not an option. > > While disabling might help for some smallish amount of users, we must > prioritize the 99.9x% that never have any issues with it. For those, "doing > what firefox does" naturally gives the most coverage wrt testing, and amount > of people able and willing to actually fix the issues. That seems like a rather broad statement. Where does 99.9x% come from? (In reply to Kent James (:rkent) from comment #36) > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment > #35) > > (In reply to Kent James (:rkent) from comment #34) > > > Maybe what we could do is to disable HWA in the betas to start to get some > > > experience with this? > > > > What would you want is to look for? > > My main question is, does disabling HWA cause an noticeable decrease in > performance for any users? I have no working knowledge of graphics issues. But looking at bug 527707 and other sources [1] it seems to me that Thunderbird users do not derive significant benefit. Perhaps the benefit is even insignificant to the point of invisibility, though I have never seen that anyone has benchmarked it. Perhaps we should ask, prior to our reenabling HWA, did we have ANY bug or support reports that complain of display speed or scroll speed performance issues that were not the results of backend data or addon issues? I do not know of any. (Note, some HWA performance improvements relate to handling of css layers. Do we have significant use in Thunderbird?) As for crashes - in the top 300 38.0.1 crashes, at least 4% are in graphics drivers [2]. (3.5% in the top 200, of which at least 1/4 are startup. Another 15 are found in the next 100 crashes for another 0.5%) And once we resolve 38.0.1's newest topcrashes and get back to our "typical" crash rate, the total percent for the graphics crashes will change to be in the 6-7% range. I am not asserting all these graphics crashes are the result of HWA - we haven't done a case by case study of the signatures, and I'm not sure it's worth our time to do so. But anecdotally, many user's crashes have been solved by disabling HWA. I doubt any marginal performance increase is worth users living with these crashes, even if some (I have no idea how many) can be avoided by updating graphics drivers. So the question is, how do we effectively address this impact? Possibilities: * file bugs to get these drivers blocklisted * Proactive user education, i.e. not "after the fact" in support requests * Disable HWA again for some period of time while we decide what to do, and make progress. [1] http://geeknizer.com/firefox-gets-gpu-d2d-acceleration/ https://hacks.mozilla.org/2010/09/hardware-acceleration/comment-page-1/ https://blog.mozilla.org/joe/2010/05/25/hardware-accelerating-firefox/ http://blogs.msdn.com/b/ie/archive/2010/09/10/the-architecture-of-full-hardware-acceleration-of-all-web-page-content.aspx [2] crashes in graphics drivers, top 200 crashes 24 0.0041133662 nvwgf2um.dll@0xaa92d 36 0.0025765041 nvumdshim.dll@0x305a8 43 0.0021696877 nvwgf2um.dll@0xaa9a5 60 0.0014690593 nvumdshim.dll@0x309b8 62 0.0014464584 mozilla::gfx::DrawTargetD2D1::Init 74 0.0012430502 nvwgf2um.dll@0xaac25 76 0.0012204493 d2d1.dll@0x26988c 81 0.0011978484 d2d1.dll@0xc50c 84 0.0011526466 nvumdshim.dll@0x31132 86 0.0011300457 nvwgf2um.dll@0x617a0c 87 0.0011074447 nvumdshim.dll@0x3107e 97 0.0010170411 nvumdshim.dll@0x30998 100 0.0009944402 mozilla::layers::DeviceManagerD3D9::CreateTexture 112 0.0008588347 nvwgf2um.dll@0x1712e 124 0.000768431 igd10umd32.dll@0x37ebfb 129 0.0007458301 igd10umd32.dll@0xe3331 131 0.0007232292 igd10umd32.dll@0x37ecbb 134 0.0007232292 nvwgf2um.dll@0x222bc9 135 0.0007006283 igd10umd32.dll@0xe33c1 142 0.0006780274 nvwgf2um.dll@0x231609 150 0.0006328256 nvwgf2um.dll@0xf71fb 153 0.0006102247 nvwgf2um.dll@0x12974d 160 0.0005876237 nvumdshim.dll@0x305d8 163 0.0005650228 nvwgf2um.dll@0xe06ea 164 0.0005650228 d3d11.dll@0xd0589 166 0.0005650228 nvwgf2um.dll@0x11dfeb 174 0.0005424219 nvwgf2um.dll@0xaa7e5 181 0.0004972201 nvumdshim.dll@0x2fcd8 182 0.0004972201 nvwgf2um.dll@0x229ac4 183 0.0004972201 nvwgf2um.dll@0x6e6379 187 0.0004746192 nvumdshim.dll@0x31142 189 0.0004746192 nvumdshim.dll@0x30768 192 0.0004746192 nvwgf2um.dll@0xf651a 200 0.0004520183 nvwgf2um.dll@0x181683

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #38) > That seems like a rather broad statement. Where does 99.9x% come from? Just my qualified guess. Of all the bugs filed, comparatively very few turn out to be about hwa problems.

http://forums.mozillazine.org/viewtopic.php?p=14194031#p14194031 http://forums.mozillazine.org/viewtopic.php?p=14197025#p14197025 https://support.mozilla.org/en-US/questions/1066782

There have seen several reports of freezing when getting new messages. Here are just 2 examples https://support.mozilla.org/en-US/questions/1067713#answer-745184 v38 computer freezes as soon as I start Thunderbird, or get new messages - related to new message notification (unknown video) https://support.mozilla.org/en-US/questions/1068495#answer-745032 v38 getting new mail (unknown video)

2 users (maybe 3?) http://forums.mozillazine.org/viewtopic.php?p=14208251#p14208251 v38 NVIDIA http://forums.mozillazine.org/viewtopic.php?p=14208341#p14208341 v38 header/tabs/button area became black, must move window to force it to be redrawn http://forums.mozillazine.org/viewtopic.php?p=14215163#p14215163 v38 with lightning (v37? is OK) https://bugzilla.mozilla.org/show_bug.cgi?id=1176304#c2 v38 windows freezes/lock up

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #38) > > (In reply to Kent James (:rkent) from comment #36) > > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment > > #35) > > > (In reply to Kent James (:rkent) from comment #34) > > > > Maybe what we could do is to disable HWA in the betas to start to get some > > > > experience with this? > > > > > > What would you want is to look for? > > > > My main question is, does disabling HWA cause an noticeable decrease in > > performance for any users? > > I have no working knowledge of graphics issues. But looking at bug 527707 > and other sources [1] it seems to me that Thunderbird users do not derive > significant benefit... > > I doubt any marginal performance increase is worth users living with these > crashes, even if some (I have no idea how many) can be avoided by updating > graphics drivers... I'm not really sure what you are arguing with me about (or even if I should call it arguing). Neither of us seems to really know what impact HWA has on performance, I proposed that we take a first step in testing this (and implementing this bug) by disabling in beta, but AFAICT you don't support that. Are you against disabling in beta? Are you in favor of disabling immediately at 38.1.0? Are you trying to support my proposal or oppose it?

(In reply to Magnus Melin from comment #39) > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment > #38) > > That seems like a rather broad statement. Where does 99.9x% come from? > > Just my qualified guess. Of all the bugs filed, comparatively very few turn > out to be about hwa problems. If you mean very few are not outright driver issues, I suspect that is true. That is however no consolation to users. Firefox's solution in many cases has been to blocklist the affected systems.

(In reply to Kent James (:rkent) from comment #43) > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment > #38) > > > > (In reply to Kent James (:rkent) from comment #36) > > > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment > > > #35) > > > > (In reply to Kent James (:rkent) from comment #34) > > > > > Maybe what we could do is to disable HWA in the betas to start to get some > > > > > experience with this? > > > > > > > > What would you want is to look for? > > > > > > My main question is, does disabling HWA cause an noticeable decrease in > > > performance for any users? > > > > I have no working knowledge of graphics issues. But looking at bug 527707 > > and other sources [1] it seems to me that Thunderbird users do not derive > > significant benefit... > > > > I doubt any marginal performance increase is worth users living with these > > crashes, even if some (I have no idea how many) can be avoided by updating > > graphics drivers... > > I'm not really sure what you are arguing with me about (or even if I should > call it arguing). Sorry. I do not mean to be arguing. I appreciate that you are offering a path to make progress. I am attempting to provide information in some organized fashion. However, we do seem to have a different perspective. > Neither of us seems to really know what impact HWA has on performance, No, I don't think that is what I am saying above. And I would probably not be advocating for disabling if there is an identified determintal impact to users. I am saying I haven't tested it. I am saying no one has offered information saying "we've got to have this because it is so so much faster". I am saying, I do not think it should have been reenabled without examining or documenting the consequences (but admittedly such is the fate of many features that do not get well tested and reported by our meager test population). Since no one has hard performance data about Thunderbird (not a foriegn concept), in the absence of such data, for me, a reading of the articles is sufficient information. > I proposed that we take a first step in testing this (and > implementing this bug) by disabling in beta, but AFAICT you don't support > that. Are you against disabling in beta? Are you in favor of disabling > immediately at 38.1.0? Are you trying to support my proposal or oppose it? In the absence of someone stepping forward to say that disabling in release is going to hurt users, yes I propose disabling in release and keep it live elsewhere. This allows us to test and collect data with a smallish population, while avoiding affecting the majority of users. Disabling in beta does not acheive this. The alternative is to wage an agressive blocklist campaign, or other mitigation previously mentioned. I will attempt to loop in someone from graphics.

Coming to the party very late (thanks for looping me in Wayne), it'll take me some time to go beyond just skimming through the comments, but let me start with this. Setting layers.acceleration.disabled to true is fully supported and tested in Firefox and should work fine on all platforms. If there are bugs that exist in unaccelerated mode, they're usually fewer and far between compared to the accelerated mode ones, especially on Windows and Linux.

When we're talking about using layers.acceleration.disabled - the "fully supported" is in the situation where all the other graphics and layers preferences are left at the default. OMTC should still work without hardware acceleration (Linux was the last one to show up to that party), but it is possible there will be some text quality deterioration on certain Windows systems. Is Thunderbird using the same downloadable blocklist and internal blocklisting scheme for acceleration as Firefox is? What are the differences in default preferences?

I'll try to summarize my understanding of the issues in this comment. Milan's comment 46 (that disabled HWA is fully supported), does not agree with previous comment 22 and similar comments that disabled HWA is not recommended due to poor support in testing and in Firefox generally. The motivation for using HWA in Thunderbird is almost exclusively about keeping parity with Firefox and its testing. The argument in favor of dropping HWA, as I understand it, is: 1) We have plenty of evidence that it causes problems. 2) We have very little evidence that it improves performance in ways that are useful in Thunderbird. 3) Supporting HWA requires detailed attention to driver blocklists. While to some extent we can follow Firefox in this, we do not have the resources to pursue Thunderbird-specific issues that may need blocklists, nor do we get much benefit from this since our performance limitations are not graphics-related. The original argument to enable HWA, from bug 950133, was: "Instead, our disabling of hardware acceleration but not a myriad of other graphics preferences puts us in an untested situation of graphics code. The current week alone has seen two separate comm-central-only bustages due to the lack of this preference." The enabling of HWA in Thunderbird, I think it is fair to say, was born at least partially from a weariness at chasing comm-central testing failures due to differences between m-c and c-c preferences. That weariness has if anything gotten worse, with perma-oranges in c-c left unfixed for months, and all hope of green trees in beta and aurora abandoned. Wayne's proposal in comment 45 "I propose disabling in release and keep it live elsewhere" means postponing those testing challenges to the release point. As one of they guys who has had to deal with testing issues at that level (and proud of the green tree we have today), I have some issues with that. But I can also test that. Let me try disabling HWA on the comm-esr38 repo, and see what problems occur. (Using try-server for anything other than trunk is a major hassle). If this blows up, any enthusiasm I might have for disabled HWA will be greatly diminished.

Someone asked about the breakdown of issues by OS. I don't have a ready answer for that. But all the crashes I have seen are windows, whose rough breakdown is nwg* 1/2 windows 7, 1/2 windows 8 nvu* win7 nvw* 1/2 windows 7, 1/2 windows 8 idg* 1/2 windows 7, 1/2 windows 8

(In reply to Milan Sreckovic [:milan] from comment #46) > Setting layers.acceleration.disabled to true is fully supported and tested > in Firefox Unless the full test suites are run in both modes, bugs are still bound to sneak in... like with everything that's not default configuration. It's just that this non-default configuration could affect anything.

Bug 1178278 - computer hangs when thunderbird 38 is running

Based on these discussions, I disabled HWA yesterday to see if I noticed. This morning when I started using Thunderbird, I clearly did. While scrolling a simple text message in the message pane, the scrolling is not even, but the top and bottom of the screen are slightly out of sync with each other, giving something that resembles a fold to the look of the moving text. Same thing with a more complex HTML message. I turned HWA on and off several times to make sure that is the cause, and it is. I found it annoying enough that I re-enabled HWA.

Wayne suggested that I add my personal experience. Here it is: I had to switch off "hardware acceleration" since I get graphic refresh problems. On my own machine, the TB and FF windows aren't refreshed after the screen saver goes away. I just had a client who edited a calendar item, and when closing the calendar item window, the piece of the desktop that was behind it didn't refresh, so some graphic artefacts got left lying around, much to the confusion of the user. Switching hardware acceleration off, fixed the problem. I can't confirm Kent's observation. Scrolling a longish message in the message pane works just fine without HWA. Perhaps we should move the discussion away from anecdotal evidence and ask: What does HWA actually do? What parts are actually untested? Is is possible that FF ships with an option that if unselected runs untested code. That's somewhat hard to believe. When I start a "home compiled" daily version of TB, I see this on the console: Crash Annotation GraphicsCriticalError: |[0][GFX1-]: Failed to create the graphics startup lockfile.[GFX1-]: Failed to create the graphics startup lockfile. Crash Annotation GraphicsCriticalError: |[0][GFX1-]: Failed to create the graphics startup lockfile.|[1][GFX1-]: Failed to create the graphics startup lockfile.[GFX1-]: Failed to create the graphics startup lockfile. Crash Annotation GraphicsCriticalError: |[0][GFX1-]: Failed to create the graphics startup lockfile.|[1][GFX1-]: Failed to create the graphics startup lockfile.|[2][GFX1-]: Failed to create the graphics startup lockfile.[GFX1-]: Failed to create the graphics startup lockfile. With HWA I see additionally: [7148] WARNING: DWM not enabled, falling back to software vsync: file h:/mozilla-source/comm-central/mozilla/gfx/thebes/gfxWindowsPlatform.cpp, line 2296 [7148] WARNING: Hardware Vsync support not yet implemented. Falling back to software timers: file h:/mozilla-source/comm-central/mozilla/gfx/thebes/gfxPlatform.cpp, line 2410 And without HWA: [7048] WARNING: Enabling vsync refresh driver: file h:/mozilla-source/comm-central/mozilla/layout/base/nsRefreshDriver.cpp, line 859 That doesn't fill me with a whole lot of confidence either way ;-( BTW: There is a heap of preferences called gfx. ... Does anyone know what they do exactly? All that switching off/on HWA does is toggling gfx.direct2d.disabled and layers.acceleration.disabled.

(In reply to Milan Sreckovic [:milan] from comment #47) > When we're talking about using layers.acceleration.disabled - the "fully > supported" is in the situation where all the other graphics and layers > preferences are left at the default. > > OMTC should still work without hardware acceleration (Linux was the last one > to show up to that party), but it is possible there will be some text > quality deterioration on certain Windows systems. > > Is Thunderbird using the same downloadable blocklist and internal > blocklisting scheme for acceleration as Firefox is? What are the > differences in default preferences? Good question. I would hope it does, but I doubt anyone has tested it. That said, I'm pretty sure in the past a driver has been blocked on at least one of my half dozen machines. Milan, I've looked at https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drivers but it is not apparent - where can one see the current blacklist? Also, bug 604771 provides instructions for spoofing drivers. Are those still state of the art instructions?

Kent, Jorg, thank you for those tests. Please list your graphics info from help | support

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #54) > Milan, I've looked at > https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drivers but it is not > apparent - where can one see the current blacklist? An individual user's blocklist can be found here: * Shipped blocklist: %program_folder%/blocklist.xml * Downloaded blocklist: %profile_folder%/blocklist.xml Note, these locations are different depending on the platform.

My graphics: Graphics Adapter Description Intel(R) HD Graphics 4600 Vendor ID 0x8086 Device ID 0x0416 Adapter RAM Unknown Adapter Drivers igdumdim64 igd10iumd64 igd10iumd64 igdumdim32 igd10iumd32 igd10iumd32 Driver Version 10.18.10.3379 Driver Date 12-20-2013 Adapter Description (GPU #2) NVIDIA GeForce GTX 870M Vendor ID (GPU #2) 0x10de Device ID (GPU #2) 0x1199 Adapter RAM (GPU #2) 3072 Adapter Drivers (GPU #2) nvd3dumx,nvwgf2umx,nvwgf2umx nvd3dum,nvwgf2um,nvwgf2um Driver Version (GPU #2) 9.18.13.3788 Driver Date (GPU #2) 5-19-2014 Direct2D Enabled true DirectWrite Enabled true (6.3.9600.17795) ClearType Parameters ClearType parameters not found WebGL Renderer false GPU Accelerated Windows 1/1 Direct3D 11 AzureCanvasBackend direct2d 1.1 AzureSkiaAccelerated 0 AzureFallbackCanvasBackend cairo AzureContentBackend direct2d 1.1

help | support? You mean about:support in FF? I currently don't have access to the two machines (since I'm away on holidays). The one with the screen saver related problem has an AMD A10-5700 APU (integrated graphics) with driver version 14.12 (current as of Dec. 2014). The other one with the graphic artefacts is a laptop with Intel i3-2350M processor (integrated HD Graphics 3000).

Adapter Description AMD Radeon HD 7500 Series Vendor ID 0x1002 Device ID 0x675d Adapter RAM 1024 Adapter Drivers aticfx64 aticfx64 aticfx64 aticfx32 aticfx32 aticfx32 atiumd64 atidxx64 atidxx64 atiumdag atidxx32 atidxx32 atiumdva atiumd6a atitmm64 Driver Version 14.501.1003.0 Driver Date 11-20-2014 Direct2D Enabled false DirectWrite Enabled false (6.2.9200.17292) ClearType Parameters ClearType parameters not found WebGL Renderer false GPU Accelerated Windows 0 AzureCanvasBackend skia AzureSkiaAccelerated 0 AzureFallbackCanvasBackend cairo AzureContentBackend cairo I originally disabled Hardware acceleration as Thunderbird was delaying paint of the mail list UI after sleep until the mouse travelled over. Then there was a ripple effect as the list was redrawn with the correct elements under the mouse. Disabling HWA appears to have stopped that about 95% not 100% but enough to make it usable, without being excessively annoying. Other than that I do not notice any changes at all.

(In reply to Jorg K from comment #53) > On my own machine, the TB and FF windows aren't refreshed after the screen > saver goes away. Or to use the words of comment #59: > Thunderbird was delaying paint of the mail list UI after [screen saver went away] until the mouse travelled over.

https://support.mozilla.org/en-US/questions/1071840 Hardware acceleration causing alignment issues in TB 38+ http://forums.mozillazine.org/viewtopic.php?f=39&t=2942813 graphics glitches

We have five different setups of Thunderbird in our house on three different computers. Two of the computers have NVIDIA GeForce 500/600-series cards, one has an Intel GMA. The computers with the NVIDIA cards have never had any issues; however, the one with the Intel GMA has had various issues in the past. Currently, HWA is enabled on Firefox on that machine with no issues; but Thunderbird is a different story. This is on a HP SlimLine desktop from 2009, running Windows 7 64-bit, with an Intel GMA 3100. This started happening with Thunderbird 38; 31.7 and before ran just fine. Rather than a traditional crash or mousemess, the Thunderbird on that machine would repeatedly hang the system (!). Mouse would freeze and redraws on anything would stop altogether. Wouldn't respond to any input (e.g. ACPI power). The only thing that could be done was to hold the power button for 5 seconds to power the machine off (or physically pull the plug). The hang would consistently happen right after the "fetching mail N out of N" message appeared in the taskbar. Numerous attempts of clearing caches, disabling features, deleting all the .msf files, etc. didn't help. On a whim, I disabled HWA, and now it has been running for half an hour with no hangs at all.

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #54) > ... > Milan, I've looked at > https://wiki.mozilla.org/Blocklisting/Blocked_Graphics_Drivers but it is not > apparent - where can one see the current blacklist? > > Also, bug 604771 provides instructions for spoofing drivers. Are those still > state of the art instructions? Sorry for the long delay answering this. The end of the wiki page has the instruction for spoofing, matching those in bug 604771. The rest of the wiki page is horribly out of date (we have certainly changed the compiled blocklist since August 2013), and is probably impractical to actually keep it up to date. The downloadable blocklist is cached as mentioned in comment 56.

SO we need to reconsider this, given the many issues we are seeing in Thunderbird 38. I checked again my experience in comment 52, and I am having a hard time reproducing it. I seemed to be the only one who was showing any advantage to HWA. I'm now in favor of disabling HWA for the next Thunderbird 38 release.

Since this bug contains a lot of anecdotal evidence, let me throw in another anecdote for your enjoyment: Back in June I wanted to show my mother what I had been working on, so I installed TB 38.0.1 right after it came out. She wasn't so impressed - OK, the red addresses had gone away. Even less so, since her PC now suddenly froze at times, sometimes so badly, that only holding down the power button could fix it. You know how mothers are, so you'll understand that I couldn't leave her with the machine like this. I took immediate and drastic action, wiped Windows 7 off the machine and installed Linux Mint instead. She was happy. You can't imagine how foolish I felt when I read comment #62 about one month later. I had destroyed a perfectly working machine, since I had no clue that the freezes could have been related to the new TB version I had installed.

For issues where the screen freezes, or goes blank or black or white and the system becomes responsive again (eg without crashing or hanging) we may be seeing the likes of bug 1160157

In Tuesday's meeting, Magnus expressed concern about how do we change the default back to disabled "temporarily" and later change the default back to enabled, within the same major release, while respecting cases where the user has manually changed the setting - which the might have enabled in say version 31, or have disabled in say version 38. We have roughly a week to explore solutions so it's ready for 38.2.0. If necessary, we might have to simplify the logic to have a practical solution, and break some users' settings

Thursday there was some discussion on IRC about how to proceed. One reminder is that speed was not the factor for reenabling HWA in bug 950133. So, speed should not be a factor in deciding whether or how to go forward with disabling it again. Some questions/observations 1. whether there is an obvious/simple patch for saving for future reapplication the user's setting if it was manually disabled since version 31, where bug 950133 blindly reenabled HWA. (so far no one has suggested such a patch. I didn't find any via an incomplete, quick pass of bugzilla, but I'm sure someone has done such a thing) 2. whether it matters whether we save a disabled setting for future reapplication a) a user presumably will recognize the problem symptoms which caused them to disable it b) HWA was enabled prior to vresion 9, and a user will have faced a similar situation when it got enabled in version 31 (I know of no users complaining that it overrode their settings c) how many/what % of users do we guess this could affect (and we must guess, because we have no telemetry) 3. what is the risk/advantages/disadvantages to leaving it disabled for the whole of version 38, with a possible goal of reenabling in 45 IF we work toward getting the stability and UI problems fixed A short history of HWA + Thunderbird: v3.3 enabled - decision in bug 618868, N.B. comment 3 v9 disabled - bug 689742, N.B. comment 0 v31 enabled - bug 950133 (actually v29) v38 TBD - bug 1131879

It's time to make a decision. I think we are generally agreed that HWA is causing lots of problems with no clear benefits, and it should be disabled. On the issue of trying to preserve the pref, I don't think it is worth the effort to write code to preserve a user change. We should just use the normal Mozilla pref management for this. HWA is not such a critical issue that we should give it special treatment.

Well this is an abnormal case. You propose to flip the switch on release code, so we can't predict the implications. Say it makes things rather bad for a lot of users, and we need to flip it back on, the users that explicitly disabled it would be have it force-enabled. But ok, maybe that's not super important. Does the problems affect anything else than windows?

(In reply to Magnus Melin from comment #70) > Does the problems affect anything else than windows? My best recollection is yes. But getting examples from the cited cases when 80?% of our users in SUMO are Windows would be time consuming. Others more active in SUMO, or that are linux users who would pay more attention to other linux users may perhaps be better able to answer. Is this a substantive question, or curiousity? I don't know that it would matter relative to the question of whether to disable.

(In reply to Magnus Melin from comment #70) > Well this is an abnormal case. 1. You propose to flip the switch on release > code, 2. so we can't predict the implications. 3. Say it makes things rather bad > for a lot of users, and we need to flip it back on, 4. the users that > explicitly disabled it would be have it force-enabled. 5. But ok, maybe that's > not super important. Good points 1. Indeed, that does make it unique [1] 2. We can't predict _with certainty_ that is correct. But we can estimate based on past experience 3. yes, we'd need to rebuild quickly to turn it back on 4. yes. But this has happened multiple times in the past, in both directions, as mentioned in comment 68 [with I think the exception of Mac users for whom some graphics bits were not enabled at the same times as windows] 5. given #4 I suspect not super important. But for users who do currently have it disabled we can easily arbitrarily save for later use the disabled state in a placeholder pref, and worry about _how_ to use it at a later date. [1] In giving further thought to how we got here, it occurred to me that until December 2014 most users were still on version 24 - even though version 31 first appeared for Release channel users on July 22, 2014. I no longer recall the full transition history, but most usres were still on version 24 until we unthrottled ~2015-12-17 with the avialability of 31.3.0. There were glimmers of this issue in fall 2014, and even summer, with topics such as https://support.mozilla.org/en-US/questions/1012145 and https://support.mozilla.org/en-US/questions/1021568 but they likely would not have been widely reported until 31.3.0 in December 2014. https://support.mozilla.org/en-US/questions/1037843 may be an example of a 31.3.0 user. There are many procedural and manpower issues which got us here, which no doubt includes insufficient attention/recognition at every level. But for now we certainly lack the resources to address the various crashes and other issues in any reasonable time period

(In reply to Magnus Melin from comment #50) > (In reply to Milan Sreckovic [:milan] from comment #46) > > Setting layers.acceleration.disabled to true is fully supported and tested > > in Firefox > > Unless the full test suites are run in both modes, bugs are still bound to > sneak in... like with everything that's not default configuration. It's just > that this non-default configuration could affect anything. I take Milan's statement to mean all tests are always performed. Milan, for older and newer tests that exercise non-accelerated code, to your knowledge do these tests get disabled by layers.acceleration.disabled=true

I don't think all the tests get run with non-default setting for the acceleration. IIRC, it's more that we run a subset of them with the flag flipped. So, yes, there is some danger of regressing on platforms where acceleration is the default. On a side note, when disabling acceleration on Windows, we want to matchingly set gfx.direct2d.disabled to true as well (when setting layers.acceleration.disabled), just in case.

Comment on attachment 8645223 [details] [diff] [review] Just disable HWA Review of attachment 8645223 [details] [diff] [review]: ----------------------------------------------------------------- I don't seem to be able to convince myself we should do this. Yes, there are bug reports but in the grand scheme of things are they really that many? Besides, a lot of our users would have upgraded already anyway (and decided if things were working out or not). If Fallen wants it, feel free to go ahead though. ::: mail/app/profile/all-thunderbird.js @@ +780,5 @@ > +#ifdef XP_WIN > +// and direct2d support on Windows. > +pref("gfx.direct2d.disabled", true); > +#endif > + nit: trailing whitespace

Matt and Philipp, it would be really good if you could give opinions on this change.

Magnus, I very much understand reluctance. I waited to to make this recommendation until I had some semblance of data - it wasn't evident to me until I started mining bugzilla, crash reports and SUMO - this takes a LOT of time. It's also less clear cut because the symptom range is wide (poor performance, hangs, crashes, bad rendering) and there is not overwhelming data to inform a dead simple decision, like a killer topcrash that we'd rebuild for, or the spectacular dataloss we had a few years ago. I'm not bold enough to claim this affects 10%, 1% or whatever number sounds reasonable. However, I do wish to clearly state for everyone that in my estimation this is more than just "few" reports. And this bug effectively has 20-40 dupes (I'm not going to count the citations) if the other bugs had been duped to this and the users in support had filed bug reports. I do hate making decisions in the abscence of hard data but it is literally impossible to answer any question of whether it only affects a few users, where metrics don't exist, except to guage by experience of dealing with the users in support. That said, in many of the URL's I've cited there are multiple users who have commented. And we know from experience for every report we can see there is some order of magnitude of more who never report their issue. Finally, a) many reports (sorry I can't estimate the percentage) are from users whose device drivers are totally current - like from a few days to a month or two old b) there's ~15-20 more citations I haven't yet added c) several users have reported that they have no problem with Firefox, only problems with Thunderbird - so it's not far fetched to say there must be, in areas of code, fundamental differences in how THunderbird is used that cause some of these problems to not show up on mozilla testing, and perhaps also for most of our pre-release users with even in months of testing (we know from past experience that our test population is not a great representation of our greater user population)

(In reply to Magnus Melin from comment #75) > > If Fallen wants it, feel free to go ahead though. I don't have a strong opinion on this, my recommendation came from what I've heard from Wayne. If we do disable it, it would be nice to find some criteria for enabling it again, otherwise it will likely be disabled for 10+ gecko versions.

Comment on attachment 8645223 [details] [diff] [review] Just disable HWA Aside from mentioned nit code looks fine. Action plan as described previously, in the end I think we should trust Wayne's feeling here since he is closest to the stats.

I concur with the decision to disable HWA by default (I thought I commented in this bug but it might have been on one of the lists) I tested fairly extensively on a 2 core I7 with HWA disabled and couldn't see ant significant perf impact. Tested with some fairly complex HTML compositions that have been troublesome in the past IE scrolling with absolutely positioned divs for one. My conclusion is, that if you have relatively current hardware, the contribution of HWA is negligable. And the enabling may be disastrous for some users.

Comment on attachment 8645223 [details] [diff] [review] Just disable HWA [Triage Comment] https://hg.mozilla.org/releases/comm-esr38/rev/88fa0041588f