Closed Bug 1524189 Opened 6 years ago Closed 6 years ago

4.44 - 22.08% raptor tp6 (linux64, linux64-qr, windows10-64, windows10-64-qr, windows7-32) regression on push 754f9877d4baec49ba19f60b81fad85076aec5fc (Wed Jan 30 2019)

Categories

(Core :: Performance, defect, P3)

Product:
Core
Component:
Performance
Type:
defect

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: igoldan, Assigned: smaug)

References

Details

(Keywords: backlog-deferred, perf, regression)

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=754f9877d4baec49ba19f60b81fad85076aec5fc

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

22% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-loadtime linux64 pgo 458.29 -> 559.50
20% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-loadtime windows10-64-qr opt 364.75 -> 437.33
19% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-loadtime linux64 opt 496.83 -> 590.17
16% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-loadtime windows10-64 pgo 331.27 -> 384.71
16% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-loadtime windows7-32 opt 374.19 -> 433.21
14% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-loadtime windows10-64 opt 383.27 -> 437.83
12% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-loadtime windows7-32 pgo 336.56 -> 378.25
9% raptor-tp6-amazon-firefox windows10-64 pgo 356.29 -> 388.72
9% raptor-tp6-amazon-firefox windows10-64-qr opt 394.62 -> 429.95
8% raptor-tp6-amazon-firefox linux64 pgo 395.87 -> 428.54
8% raptor-tp6-amazon-firefox windows7-32 pgo 351.22 -> 378.06
7% raptor-tp6-yandex-firefox raptor-tp6-yandex-firefox-loadtime linux64-qr opt 292.46 -> 314.17
6% raptor-tp6-amazon-firefox windows7-32 opt 394.53 -> 418.54
6% raptor-tp6-yandex-firefox linux64-qr opt 155.07 -> 164.47
6% raptor-tp6-amazon-firefox windows10-64 opt 401.81 -> 424.80
4% raptor-tp6-amazon-firefox linux64 opt 444.90 -> 464.67

Improvements:

48% raptor-tp6-facebook-firefox raptor-tp6-facebook-firefox-loadtime windows10-64 opt 658.50 -> 344.54
46% raptor-tp6-facebook-firefox raptor-tp6-facebook-firefox-loadtime linux64 opt 638.33 -> 342.58
44% raptor-tp6-facebook-firefox raptor-tp6-facebook-firefox-loadtime linux64-qr opt 662.83 -> 369.96
42% raptor-tp6-facebook-firefox raptor-tp6-facebook-firefox-loadtime windows10-64-qr opt 605.54 -> 349.92
42% raptor-tp6-facebook-firefox raptor-tp6-facebook-firefox-loadtime windows7-32 opt 573.10 -> 333.54
40% raptor-tp6-facebook-firefox raptor-tp6-facebook-firefox-loadtime windows10-64 pgo 584.04 -> 350.42
23% raptor-tp6-yandex-firefox raptor-tp6-yandex-firefox-loadtime windows7-32 opt 298.62 -> 228.62
20% raptor-tp6-facebook-firefox linux64 opt 419.13 -> 334.09
20% raptor-tp6-facebook-firefox windows10-64 opt 417.59 -> 335.74
18% raptor-tp6-facebook-firefox windows10-64-qr opt 413.30 -> 339.39
16% raptor-tp6-facebook-firefox linux64-qr opt 422.32 -> 355.36
15% raptor-tp6-facebook-firefox windows7-32 opt 385.92 -> 327.53
14% raptor-tp6-facebook-firefox windows10-64 pgo 378.52 -> 325.63
14% raptor-tp6-reddit-firefox windows10-64-qr opt 1,012.05 -> 872.89
12% raptor-tp6-reddit-firefox linux64 pgo 831.93 -> 729.02
11% raptor-tp6-reddit-firefox windows10-64 opt 1,000.27 -> 886.96
11% raptor-tp6-yandex-firefox raptor-tp6-yandex-firefox-loadtime windows10-64 pgo 277.40 -> 246.62
11% raptor-tp6-reddit-firefox windows10-64 pgo 877.68 -> 780.44
11% raptor-tp6-facebook-firefox linux64 pgo 349.11 -> 310.99
11% raptor-tp6-imdb-firefox linux64-qr opt 308.74 -> 275.49
10% raptor-tp6-reddit-firefox linux64-qr opt 938.62 -> 843.10
10% raptor-tp6-reddit-firefox windows7-32 opt 944.85 -> 849.32
9% raptor-tp6-facebook-firefox windows7-32 pgo 342.27 -> 309.98
6% raptor-tp6-imdb-firefox raptor-tp6-imdb-firefox-loadtime windows10-64 opt 464.12 -> 437.58
5% raptor-tp6-imdb-firefox raptor-tp6-imdb-firefox-loadtime windows7-32 opt 451.29 -> 430.92
4% raptor-tp6-reddit-firefox raptor-tp6-reddit-firefox-fcp linux64 pgo 252.21 -> 241.83
4% raptor-tp6-reddit-firefox raptor-tp6-reddit-firefox-fcp linux64-qr opt 306.35 -> 295.12
4% raptor-tp6-reddit-firefox raptor-tp6-reddit-firefox-fcp linux64 opt 285.81 -> 275.38
2% raptor-tp6-google-firefox raptor-tp6-google-firefox-loadtime linux64 opt 154.67 -> 150.83
2% raptor-tp6-google-firefox raptor-tp6-google-firefox-loadtime windows10-64 opt 137.71 -> 134.46
2% raptor-tp6-amazon-firefox raptor-tp6-amazon-firefox-fcp windows7-32 pgo 225.10 -> 220.50

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=19049

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Product: Testing → Core
Flags: needinfo?(bugs)

If there are any blocked bugs which aren't related to this issue, do remove them from the list.

The duplicate bug 1524207 warns about Talos perf regressions also. comment 0 applies to these as well.

yup, I'm looking why amazon regressed.
(I was pleasantly surprised about the massive improvement with facebook.)

And thanks for the links to the profiles, super helpful!

Component: General → Performance
Assignee: nobody → bugs

FWIW, this may take some time to fix. Various changes affecting page load will cause improvements and some regressions.

Olli, could you please set a priority for this bug? Thanks!

Flags: needinfo?(bugs)
Priority: -- → P3
Keywords: backlog-deferred

Olli, out of these 3 bugs, namely bug 1506949, bug 1521786, bug 1523158, can you say which one is more related to the improvements?

Flags: needinfo?(bugs)

bug 1523158 has nothing to do with this.
bug 1521786 should help with fcp
bug 1506949 should help with loadtime

Flags: needinfo?(bugs)

Marking this wontfix since this actually happened just after bug 1270059 had improved Amazon and such a lot and here we regressed that stuff a bit, but improved other sites. Overall we still improved a lot.
Amazon numbers went 800 -> 364 -> 384 on win10/pgo, and the very latest number is 357
(bug Bug 1538517 improved that a bit).

Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.