Talos has detected a Firefox performance regression from push eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e. As author of one of the patches included in that push, we need your help to address this regression. Regressions: 3% tp5o summary windows8-64 pgo e10s 261.46 -> 270.57 3% tp5o summary linux64 opt e10s 384.68 -> 397.29 3% tp5o summary osx-10-10 opt e10s 295.1 -> 303.92 2% tp5o summary windows8-64 opt e10s 325.05 -> 332.67 2% tp5o summary windows7-32 opt e10s 378.33 -> 384.67 You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=4455 On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format. To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running *** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! *** Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

After doing some retriggers, this issue might be caused by the following patch: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=5f494ff3b83be2b5e8529e859d80ca3efacf19f7&tochange=eb3ac9d2775b8c4e7b089cab4d33d953d6605f0e Hi Ehsan, as you are the patch author, can you take a look at this and determine what is the root cause? Thanks!

We might need bug 1321874 if the overhead of a classification lookup is high in e10s mode.

I will have a look at this next week. This needs some time to focus so unfortunately I can't do anything about this at the work week. I'm keeping the needinfo on myself.

thanks :ehsan!

I'm trying to get a sense of whether my fix to bug 1321874 is going to be enough to fix this regression. I have pushed a patchset with all of the necessary fixes to try, and the comparison against the base revision from mozilla-central looks like the below: <https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&showOnlyImportant=0> Looking at tp5o opt e10s, we have: linux64 subtests · graph 389.59 > 381.64 ± 0.85% -2.04% 2.33 (low) 1 / 21 osx-10-10 subtests · graph 302.92 > 296.29 ± 1.13% -2.19% 1.90 (low) 1 / 21 windows7-32 subtests · graph 382.22 > 370.37 ± 2.53% -3.10% 1.20 (low) 1 / 21 I didn't make a PGO build for this push, and have recently triggered a bunch of more Talos runs on the base central revision because the compare tool indicated that more base runs are needed to increase the confidence. Eyeballing the above numbers naively it seems that these fixes are sufficient to fix this regression. Joel, does that seem correct to you? Is there any other investigation that I need to do? (Note that the fixes to bug 1321874 are all under review and are slowly landing, but there are 6 or so separate patches on 3 different bugs...)

doing a lot of retriggers on the base revision- will respond when I have more data- thanks for working on this and I will help out in an hour or so.

this is looking pretty good: https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject=try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&filter=tp5o%20summary&showOnlyImportant=0 the win7 has 1 random outlier which is causing the data to not be so reliable, but the rest of the data sans the outlier show a 2%+ improvement. If we have a path forward this is great- we will wait for the patches to land when they are ready- and if we can get them all in before the next uplift that would be even better :)

(In reply to Joel Maher ( :jmaher) from comment #7) > this is looking pretty good: > https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla- > central&originalRevision=5a536a16e33798fe7b16de35c968d5bc0cbf8448&newProject= > try&newRevision=ec1fad1182dda6157748c217ad110c266d8e8ba2&framework=1&filter=t > p5o%20summary&showOnlyImportant=0 > > the win7 has 1 random outlier which is causing the data to not be so > reliable, but the rest of the data sans the outlier show a 2%+ improvement. Fantastic! > If we have a path forward this is great- we will wait for the patches to > land when they are ready- and if we can get them all in before the next > uplift that would be even better :) It seems that the next uplift is on Jan 23, so yeah I would say the patches will definitely land by then. In the interest of not dropping the ball on this, I'll needinfo myself and will update the bug when all of the patches have landed.

[Tracking Requested - why for this release]: Performance regression on this release.

Declaring this as fixed per bug 1325467 comment 12.

Adjusting tracking flag for 53 based on the duplicate status.