[Tracking Requested - why for this release]:Regression since 35.0 Steps To reproduce: 1. Open attached Actual Results: Firefox34.0b2 : 16000-17000msec Aurora35.0a1 :19000-20000msec Nightly36.0a1 :19000-20000msec

Tracking for now - Joel are you able to get more detail on what (and when this) was regressed?

this is going to be hard as we would need to have builds for 35 on trunk prior to the uplift, which we don't have as much of. We do have some nightlies, etc. that we could narrow this down with. We could pull builds from: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/ narrow it down to a day and then have a range; I would wonder if this would match up with other talos regressions we had (bug 1072935 around sept 24). Here are some links to builds we could try: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-04-03-02-02-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-09-03-02-07-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-12-03-02-02-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-16-03-02-04-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-20-03-02-06-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-24-03-02-04-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-25-03-02-03-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/09/2014-09-28-03-02-06-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/10/2014-10-02-03-02-02-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/10/2014-10-06-03-02-02-mozilla-central/firefox-35.0a1.en-US.win32.zip http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014/10/2014-10-10-03-02-01-mozilla-central/firefox-35.0a1.en-US.win32.zip If someone could run this on a win7 machine, that would be helpful, ideally 3+ runs in each build. Otherwise I might be able to pick this up after next week!

18270 18382 18194 https://hg.mozilla.org/mozilla-central/rev/776fa9cf70cd Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140904030202 17911 17947 17867 https://hg.mozilla.org/mozilla-central/rev/6b8da5940f74 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140909030207 17889 17962 17858 https://hg.mozilla.org/mozilla-central/rev/2db5b64f6d49 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140912030202 17728 17833 17860 https://hg.mozilla.org/mozilla-central/rev/3b7921328fc1 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140916030204 17942 18029 18026 https://hg.mozilla.org/mozilla-central/rev/27253887d2cc Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140920030206 17641 17879 17909 https://hg.mozilla.org/mozilla-central/rev/1e2993c99323 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140924030204 17683 18012 17914 https://hg.mozilla.org/mozilla-central/rev/1735ff2bb23e Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140925030203 17679 17658 17548 https://hg.mozilla.org/mozilla-central/rev/6a63bcb6e0d3 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140928030206 19172 18939 18933 https://hg.mozilla.org/mozilla-central/rev/2399d1ae89e9 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141002030202 19317 19267 19193 https://hg.mozilla.org/mozilla-central/rev/0ed32d9a42d6 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141006030202 19180 19083 19002 https://hg.mozilla.org/mozilla-central/rev/50b689feab5f Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141010030201

In addition to the above, Nightly builds between 20140928 - 20141002 17335 17400 17576 https://hg.mozilla.org/mozilla-central/rev/9d66436af432 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140929030205 17843 17830 17741 https://hg.mozilla.org/mozilla-central/rev/7c24470b6b3a Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930030202 18685 18498 18598 https://hg.mozilla.org/mozilla-central/rev/14665b1de5ee Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001030205 19114 18952 18891 https://hg.mozilla.org/mozilla-central/rev/2399d1ae89e9 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141002030202 19376 19238 19222 https://hg.mozilla.org/mozilla-central/rev/5d6ec4dddf14 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141002093155

Thanks Alice0775! That is great stuff. Looking at the nightlies on Oct 1,2,3 - we see the values rising each time, probably a set of regressions. The first one: 18685 18498 18598 https://hg.mozilla.org/mozilla-central/rev/14665b1de5ee Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001030205 has 6 pushes: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=7c24470b6b3a&tochange=14665b1de5ee Unfortunately we don't keep these builds around anymore, or we could easily figure this out. We could push to try at each level and then tomorrow see which of the builds is the problem. Then that gives us a set of ~20-40 changes from a branch or two to investigate. Alice0775, would you have time to do this on a few more builds if I were to get you links to the builds from a try push? The one danger here is I need to do PGO builds as that will be closer to what we do for nightly.

There are probably builds at http://inbound-archive.pub.build.mozilla.org/

doing a little treeherder build directory (to get the magic date id to find these builds), I found 4 pgo builds (out of 4). This will get us closer! 14665b1de5ee: http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32-pgo/1412141411/firefox-35.0a1.en-US.win32.zip 6c824fbb73e8: http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32-pgo/1412130628/firefox-35.0a1.en-US.win32.zip 633ae38b87d7: no build 0ade54570f25: http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32-pgo/1412119814/firefox-35.0a1.en-US.win32.zip 2ae57957e4bb: http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32-pgo/1412087422/firefox-35.0a1.en-US.win32.zip 4475aa556e69: no build

17996 18123 17861 https://hg.mozilla.org/integration/mozilla-inbound/rev/4f90b7fb1918 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140929073348 17844 17761 17786 https://hg.mozilla.org/integration/mozilla-inbound/rev/da12cd0ebe40 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140929103004 17427 18040 17829 https://hg.mozilla.org/integration/mozilla-inbound/rev/c9f50d3b8ee4 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140929133004 17660 17886 17927 https://hg.mozilla.org/integration/mozilla-inbound/rev/051ab1a84d38 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140929163004 17896 17826 17842 https://hg.mozilla.org/integration/mozilla-inbound/rev/6fb508e952b1 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140929193027 17739 17914 17832 https://hg.mozilla.org/integration/mozilla-inbound/rev/a68c5d68a968 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140929223005 17677 17836 18000 https://hg.mozilla.org/integration/mozilla-inbound/rev/afcd7e31bb0b Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930000416 17748 17863 17931 https://hg.mozilla.org/integration/mozilla-inbound/rev/4d131d048131 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930013005 17811 17886 17887 https://hg.mozilla.org/integration/mozilla-inbound/rev/e0776c837c2b Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930011248 18491 18275 18444 https://hg.mozilla.org/integration/mozilla-inbound/rev/a378debe719a Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930043007 18389 18972 18597 https://hg.mozilla.org/integration/mozilla-inbound/rev/514846b70ea1 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930073014 18116 18254 18124 https://hg.mozilla.org/integration/mozilla-inbound/rev/0ade54570f25 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930103007 17804 17945 18021 https://hg.mozilla.org/integration/mozilla-inbound/rev/baafb688f138 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930133013 18205 18235 18412 https://hg.mozilla.org/integration/mozilla-inbound/rev/298b1f34d02a Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930163006 18061 18125 18172 https://hg.mozilla.org/integration/mozilla-inbound/rev/946948bfc6a3 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930193022 17747 17637 17803 https://hg.mozilla.org/integration/mozilla-inbound/rev/d3d0a47e6eb7 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140930223005 17784 17843 17903 https://hg.mozilla.org/integration/mozilla-inbound/rev/1ffb7f0c497f Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001013005 17946 16273 18013 https://hg.mozilla.org/integration/mozilla-inbound/rev/bef7426ecdb2 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001043006 18213 18088 18368 https://hg.mozilla.org/integration/mozilla-inbound/rev/c14d1ef31a7f Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001073014 17861 18034 17996 https://hg.mozilla.org/integration/mozilla-inbound/rev/99be24fa7141 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001103002 17580 17766 17580 https://hg.mozilla.org/integration/mozilla-inbound/rev/7719fd510b39 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001104022 19332 19318 19430 https://hg.mozilla.org/integration/mozilla-inbound/rev/e7422ba80faa Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001133005 18906 18765 18716 https://hg.mozilla.org/integration/mozilla-inbound/rev/146eef5daadc Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001193252 19018 18762 18896 https://hg.mozilla.org/integration/mozilla-inbound/rev/6351a349b35d Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141001223011 18903 19009 19245 https://hg.mozilla.org/integration/mozilla-inbound/rev/0f14e8b69354 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141002013004 19402 19156 19136 https://hg.mozilla.org/integration/mozilla-inbound/rev/13740fca92f2 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141002013346 Suspect: Bug 902762 But, It seems something by new feature...

OR Bug 902952 - Use Direct2D 1.1 on mozilla-central

Jonathan and Bas, can you guys look at your patches in bug 902762 and bug 902952 and see if they would cause the attachment in this bug to regression 18% on windows 7?

bug 1070689 seems quite unlikely unless this involves printing a lot of things to the console.

It is extremely unlikely to be bug 1045739, for a variety of reasons. In particular, the fix for bug 1045739 has us doing less work, not more work. And, also it is about as far away from text rendering as we can get.

(In reply to Joel Maher (:jmaher) from comment #11) > Jonathan and Bas, can you guys look at your patches in bug 902762 and bug > 902952 and see if they would cause the attachment in this bug to regression > 18% on windows 7? If bug 902762 caused a perf regression, we'd expect to see that across all platforms. This being Windows-specific, I think a windows graphics back-end change (i.e. bug 902952) sounds far more suspicious. This bug is filed for Windows; to double-check, can you confirm that we have *not* seen a comparable regression on other platforms?

It should be pretty trivial to experimentally revert the change from bug 902952 (just undo the additions to all.js) to see whether that affects this test.

(In reply to Jonathan Kew (:jfkthame) from comment #15) > It should be pretty trivial to experimentally revert the change from bug > 902952 (just undo the additions to all.js) to see whether that affects this > test. Indeed. Also, it's correct, this might very well be the cause of the regression. I'll have to have a little look at why. There doesn't seem to be any rendering in this test for which the command stream would've changed, but obviously I must be mistaking :-). For what it's worth, direct2d 1.1 is being disabled on Aurora so this is not a big concern for Aurora.

I've tried running this test with, and without Direct2D 1.1. 2 sets of 4 refreshes right after startup for both with and without: 1.0 1: 12408, 15217, 15108, 15799 1.0 2: 14829, 14634, 15308, 14785 1.1 1: 15416, 15239, 15333, 15089 1.1 2: 14127, 15141, 15266, 15035 As far as I can tell there is no significant difference. The deviations seem to be relatively high as well. I've also examined the profile and concluded in both cases the majority of the time is spent in text stuff, as expected, but slightly more so in the layout of the text than in the actual rendering.

The regression range in comment 10 is quite narrow. All we need is a build from rev acf16a586d9f to say which bug caused the regression (and perhaps builds from the start and end of the range too if they are not directly comparable in performance to -inbound/-central builds).

e7422ba80faa: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jmaher@mozilla.com-5b9ea52b9b89/try-win32/firefox-35.0a1.en-US.win32.zip 7f08e93cc2a5: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jmaher@mozilla.com-a0879bbeda99/try-win32/firefox-35.0a1.en-US.win32.zip d954ed24e795: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jmaher@mozilla.com-54e7371e3e20/try-win32/firefox-35.0a1.en-US.win32.zip 165c3fd176ec: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jmaher@mozilla.com-1a621bf1a2df/try-win32/firefox-35.0a1.en-US.win32.zip The only build not in this range: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=7719fd510b39&tochange=e7422ba80faa is acf16a586d9f, which I have another push to try waiting for the build to complete. If somebody has a win7 machine, we could verify the values of the above builds. I will post a link the the build from rev acf16a586d9f when it is available in a few hours.

acf16a586d9f: http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jmaher@mozilla.com-2959735875d6/try-win32/firefox-35.0a1.en-US.win32.zip

17414 17782 17639 https://hg.mozilla.org/try/rev/5b9ea52b9b89 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141223080953 17572 17887 17928 https://hg.mozilla.org/try/rev/a0879bbeda99 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141223081354 17455 17668 17837 https://hg.mozilla.org/try/rev/54e7371e3e20 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141223081454 16249 16148 16145 https://hg.mozilla.org/try/rev/1a621bf1a2df Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141223081555 17977 17869 17971 https://hg.mozilla.org/try/rev/2959735875d6 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20141223132353

Thanks Joel, Alice. That data seems to indicate a ~9% regression in this range: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=165c3fd176ec&tochange=d954ed24e795 and no significant regression in the nearby builds. In particular, I think we can exclude Jonathan's vertical text changes (in the comment 10 range) as the cause.

We're out of time for 35, wontfixing.

Milan, it seems to be a graphic regression, can you help on this? Thanks

Checking with Bas if this is something we expect or are worried about.

(In reply to Milan Sreckovic [:milan] from comment #25) > Checking with Bas if this is something we expect or are worried about. Yes, we fixed some perf issues since, all those fixes should be in aurora at the moment.

(In reply to Bas Schouten (:bas.schouten) from comment #26) > (In reply to Milan Sreckovic [:milan] from comment #25) > > Checking with Bas if this is something we expect or are worried about. > > Yes, we fixed some perf issues since, all those fixes should be in aurora at > the moment. The easiest way to test this btw, is by flipping gfx.direct2d.use1_1 and seeing if there's still a difference on tests.

OK. I am going to mark 36 as fixed then. Thanks

Assuming we're all done here. Please reopen if there's something else to be done.