37.99 - 34.09% pinterest PerceptualSpeedIndex / pinterest fcp + 1 more (OSX) regression on Mon July 26 2021
Categories
(Core :: Layout: Scrolling and Overflow, defect)
Tracking
()
Tracking | Status | |
---|---|---|
firefox-esr78 | --- | unaffected |
firefox-esr91 | --- | unaffected |
firefox90 | --- | unaffected |
firefox91 | --- | unaffected |
firefox92 | --- | wontfix |
firefox93 | --- | wontfix |
firefox94 | --- | wontfix |
People
(Reporter: alexandrui, Unassigned)
References
(Regression)
Details
(Keywords: perf, perf-alert, regression)
Perfherder has detected a browsertime performance regression from push ba212049d3c03367539fe10a67dc1b2fb7c42a86. As author of one of the patches included in that push, we need your help to address this regression.
Regressions:
Ratio | Suite | Test | Platform | Options | Absolute values (old vs new) |
---|---|---|---|---|---|
38% | PerceptualSpeedIndex | macosx1015-64-shippable-qr | warm webrender | 435.92 -> 601.50 | |
36% | fcp | macosx1015-64-shippable-qr | warm webrender | 291.88 -> 395.79 | |
34% | fcp | macosx1015-64-shippable-qr | warm webrender | 290.17 -> 389.08 |
Improvements:
Ratio | Suite | Test | Platform | Options | Absolute values (old vs new) |
---|---|---|---|---|---|
28% | paypal | LastVisualChange | macosx1015-64-shippable-qr | cold webrender | 1,685.00 -> 1,220.00 |
17% | paypal | SpeedIndex | macosx1015-64-shippable-qr | cold webrender | 942.04 -> 777.58 |
12% | paypal | PerceptualSpeedIndex | macosx1015-64-shippable-qr | cold webrender | 880.96 -> 774.00 |
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.
For more information on performance sheriffing please see our FAQ.
Comment 1•3 years ago
|
||
Set release status flags based on info from the regressing bug 1722359
Comment 2•3 years ago
|
||
Can you double-check the regression push?
I'm moderately sure my patch shouldn't have caused any behavior change on macOS, since on mac (on desktop in general) we returned zero for that value before.
The only plausible behavior change would be android, and it'd be a perf improvement.
Updated•3 years ago
|
Reporter | ||
Comment 3•3 years ago
|
||
Done.
Comment 5•3 years ago
|
||
Looks like the regressing bug was confirmed by retriggers and a recent look at the graph shows that this regression is still present.
Updated•3 years ago
|
Comment 6•3 years ago
|
||
So, I can reproduce, but I still have no idea how this can happen, fwiw... I verified both before and after my patch that the condition I removed wasn't being hit.
I think there's something off with the pinterest test. So here are a few try runs:
- This is central: https://treeherder.mozilla.org/jobs?repo=try&revision=984a40c2ee9449f3affb9a107f3cedf867001c9e
- This is central with my patch backed out: https://treeherder.mozilla.org/jobs?repo=try&revision=8e0e9d531de00c303498bb0bfa34f2831ab98580
- This is central with my patch backed out, and an assert that the condition I removed is false: https://treeherder.mozilla.org/jobs?repo=try&revision=72fa01a09cd0dfaa1749a636fe4b0a347107daec
- This is central with my patch, but leaving the widget/ bits untouched, just removing the condition: https://treeherder.mozilla.org/jobs?repo=try&revision=9434524f4059c8f41c5b18951c97e0efa110e011
Comment 7•3 years ago
|
||
Backout vs. central-with-assert (which shouldn't have any behavior change since the assert is always false): https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=8e0e9d531de00c303498bb0bfa34f2831ab98580&newProject=try&newRevision=72fa01a09cd0dfaa1749a636fe4b0a347107daec&framework=13&page=1
Central vs. central-with-assert: https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=984a40c2ee9449f3affb9a107f3cedf867001c9e&newProject=try&newRevision=72fa01a09cd0dfaa1749a636fe4b0a347107daec&framework=13&page=1
I can't make any sense of this, tbh. Also the fact that the confidence is Infinity
looks a bit suspicious to me... Jesup, do you know what could explain this?
Updated•3 years ago
|
Comment 8•3 years ago
|
||
Enough work has been done since then that I think we can just close this. Some of the tests are somewhat unstable, this may be an aspect of that (random timing change)
Updated•3 years ago
|
Updated•3 years ago
|
Description
•