Talos has detected a Firefox performance regression from push: https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=fd895bb95b99a0f0b1d8995279141773aa741703 As author of one of the patches included in that push, we need your help to address this regression. Regressions: 40% tscrollx linux64 opt e10s stylo 0.89 -> 1.25 36% tscrollx linux64-qr opt e10s stylo 2.33 -> 3.17 35% tscrollx linux64 pgo e10s stylo 0.85 -> 1.15 20% tp5o_scroll linux64-qr opt e10s stylo 3.96 -> 4.75 18% tps windows7-32 opt e10s stylo 12.23 -> 14.45 17% tscrollx windows10-64 opt e10s stylo 0.86 -> 1.01 15% tp5o_scroll windows10-64-qr opt e10s stylo 2.82 -> 3.26 15% tp5o_scroll linux64 pgo e10s stylo 1.07 -> 1.23 14% tp5o_scroll linux64 opt e10s stylo 1.15 -> 1.32 9% tp5o linux64 pgo e10s stylo 125.05 -> 135.85 9% tp5o linux64 opt e10s stylo 138.76 -> 150.58 8% tp5o_scroll windows10-64 opt e10s stylo 1.36 -> 1.47 7% displaylist_mutate windows10-64 opt e10s stylo 2,165.74 -> 2,308.45 6% displaylist_mutate windows10-64 pgo e10s stylo 2,054.24 -> 2,181.30 5% a11yr windows7-32 opt e10s stylo 183.52 -> 193.47 5% tp5o_webext linux64 pgo e10s stylo 185.27 -> 195.10 5% tp5o_webext linux64 opt e10s stylo 207.77 -> 217.97 5% displaylist_mutate windows7-32 opt e10s stylo 2,246.73 -> 2,352.68 4% tp5o_webext windows10-64 opt e10s stylo 192.81 -> 200.39 4% tp5o windows7-32 opt e10s stylo 127.69 -> 132.42 4% tp5o windows10-64 opt e10s stylo 130.29 -> 134.93 3% displaylist_mutate linux64 opt e10s stylo 2,025.71 -> 2,091.44 3% tp5o_webext windows7-32 opt e10s stylo 189.17 -> 194.83 3% tp5o windows10-64-qr opt e10s stylo 126.69 -> 130.07 2% tp5o linux64-qr opt e10s stylo 143.86 -> 147.35 You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=16404 On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format. To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running *** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! *** Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Here are the Gecko profiles for tp5o on Linux 64 opt: before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FebsrqnVVQKS2WrvCao-I7Q%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tp5o.zip after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FH0GYztXyRIqPuseLEM5j_Q%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tp5o.zip

And for tscrollx on the same platform: before: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FUnVY9XVxRbKjInMJ1EPiqA%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tscrollx.zip after: https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster.net%2Fv1%2Ftask%2FRcXchFihSZOc5S8oRvGv6Q%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tscrollx.zip

this was backed out, so we will see improvements soon, then more regressions assuming it lands again with no adjustments to perf

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #1) > Here are the Gecko profiles for tp5o on Linux 64 opt: > > before: > https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster. > net%2Fv1%2Ftask%2FebsrqnVVQKS2WrvCao- > I7Q%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_tp5o.zip > > after: > https://perf-html.io/from-url/https%3A%2F%2Fqueue.taskcluster. > net%2Fv1%2Ftask%2FH0GYztXyRIqPuseLEM5j_Q%2Fruns%2F0%2Fartifacts%2Fpublic%2Fte > st_info%2Fprofile_tp5o.zip Whenever I try to look at one of these profiles, I get an error like this: Are you sure this is a profile? Failed to process the file in the zip file at the following path:/profile_tp5o/stackoverflow.com_pagecycle_2/cycle_0.profile

I just tried this and it works for me (the before link). Maybe a hiccup in perf.html

Some investigation of these profiles with Markus and Matt today suggested that the cause of the slowdown may be an interaction with the throttling mechanism used by tests in ASAP mode, rather than actual operations like painting or compositing taking longer.

The backout of bug 1495055 canceled all regressions. Here is just a small subset of the perf regains: == Change summary for alert #16514 (as of Fri, 05 Oct 2018 05:00:41 GMT) == Improvements: 6% tp5o_scroll windows10-64 opt e10s stylo 1.48 -> 1.40 5% displaylist_mutate windows7-32 pgo e10s stylo 2,263.83 -> 2,149.40 5% tp5o osx-10-10 opt e10s stylo 229.55 -> 218.39 3% tp5o_scroll windows7-32 pgo e10s stylo 1.43 -> 1.38 For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=16514

The relanding of that bug will be restricted in its effects to Android, so it shouldn't re-introduce these regressions. Calling this fixed by backout.