Closed Bug 1686220 Opened 4 years ago Closed 4 years ago

5.96 - 27.08% wikipedia / wikipedia LastVisualChange / wikipedia fcp / wikipedia loadtime (linux64-shippable, linux64-shippable-qr, macosx1014-64-shippable-qr) regression on push facbcc4306d1a824cb607cb3bbc54a7b13d1038c (Fri January 8 2021)

Categories

(Core :: Layout: Generated Content, Lists, and Counters, defect)

defect

Tracking

()

RESOLVED FIXED
87 Branch
Tracking Status
firefox-esr78 --- unaffected
firefox84 --- unaffected
firefox85 --- unaffected
firefox86 --- fixed
firefox87 --- fixed

People

(Reporter: Bebe, Assigned: emilio)

References

(Regression)

Details

(Keywords: perf, perf-alert, regression, Whiteboard: [perf:alert:1] )

Attachments

(3 files)

Perfherder has detected a browsertime performance regression from push 04b98b03122fc7cbb8017c01d654cb3d9efecd83. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Ratio Suite Test Platform Options Absolute values (old vs new)
27% wikipedia loadtime linux64-shippable nocondprof warm 604.75 -> 768.50
26% wikipedia loadtime linux64-shippable-qr nocondprof warm webrender 625.29 -> 789.17
24% wikipedia loadtime linux64-shippable-qr nocondprof warm webrender 641.27 -> 792.08
20% wikipedia fcp linux64-shippable-qr nocondprof warm webrender 641.88 -> 771.42
19% wikipedia fcp linux64-shippable nocondprof warm 625.33 -> 745.62
18% wikipedia linux64-shippable nocondprof warm 678.58 -> 801.58
18% wikipedia linux64-shippable-qr nocondprof warm webrender 697.97 -> 824.49
18% wikipedia LastVisualChange linux64-shippable nocondprof warm 980.00 -> 1,156.67
16% wikipedia LastVisualChange linux64-shippable-qr nocondprof warm webrender 993.33 -> 1,156.67
6% wikipedia loadtime macosx1014-64-shippable-qr nocondprof warm webrender 945.29 -> 1,001.62

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Flags: needinfo?(emilio)

Huh, thanks, will investigate.

Assignee: nobody → emilio

Set release status flags based on info from the regressing bug 1685078

https://hg.mozilla.org/mozilla-central/rev/66d167aa16ea#l4.139

Had no equivalent in the new code. This seems important for Wikipedia's
visual metrics.

Flags: needinfo?(emilio)
Pushed by ealvarez@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/6083cdb61f42 Restore bullet image load priority boost which was lost in bug 1685078. r=jrmuizel
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → 86 Branch

There seemed to be some high-confidence progressions in my try push but most of the graphs there remained flat, so taking another look at this.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Flags: needinfo?(emilio)

Do you know how can I see a before / after profile / captures? I wonder why this only affected Linux, too... Maybe the test is bi-modal and now the load event fires the same way that on other platforms or something.

Flags: needinfo?(fstrugariu)

to create gecko profile you just select the job and click the "*** > Create gecko profile" button in treeherder
from the graph the test is not bimodal

Flags: needinfo?(fstrugariu)
Whiteboard: [perf:alert:?]

So I'm looking at this profile, but there don't seem to be any symbols?

Flags: needinfo?(emilio) → needinfo?(fstrugariu)

:Sparky do you know anything about the profiles generated by browsertime?

Flags: needinfo?(fstrugariu) → needinfo?(gmierz2)

When the patch in this bug lands, you'll be able to generate good profiles - there were some issues with symbolication and profile organization that were fixed: https://bugzilla.mozilla.org/show_bug.cgi?id=1686327

Flags: needinfo?(gmierz2)

:emilio as of :spaky patch we don't have gecko profiles generation out of the box for this alert
I think you can cherrypick and run some try builds to get the appropiate data

Whiteboard: [perf:alert:?] → [perf:alert:1]

Emilio, is there an update on this bug? Thanks

Flags: needinfo?(emilio)

It's on my list of things to investigate, but this week and the next I'm basically on PTO (university exams). We can revert the regressing bug on beta meanwhile, so that the regression is scoped to nightly for now.

Attached file Back out bug 1686220 from beta. (deleted) —

This will give me more time to investigate.

Approval Request Comment
[Feature/Bug causing the regression]: bug 1686220
[User impact if declined]: performance regression
[Is this code covered by automated tests?]: yes
[Has the fix been verified in Nightly?]: n/a
[Needs manual test from QE? If yes, steps to reproduce]: n/a
[List of other uplifts needed for the feature/fix]: none
[Is the change risky?]: no
[Why is the change risky/not risky?]: cleanish backout (only test expectations needed merge)
[String changes made/needed]: none

Attachment #9199912 - Flags: approval-mozilla-beta?

Comment on attachment 9199912 [details]
Back out bug 1686220 from beta.

Back out bug 1686220 for causing a perf regression, approved for our next 86 beta, thanks!

Attachment #9199912 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

I guess you mean back out bug 1685078. Bug 1686220 is this one.

Invalidating paint on size available is too soon (we can't decode the
image anyways). Bullet frames didn't use to do this and just reflowed.

Scheduling a paint causes us to do layout way too soon when there's more
important stuff to do for the load of the page.

This is a big improvement on load times and visual metrics in a variety
of websites:

https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=3e012dad47b7f4b22741faa46153e940d079a260&newProject=try&newRevision=ce14456c07318eb3fc8d94d9cff9f41d9d122752&framework=13

Took a little bit to confirm what was going on, but I think it was worth it!

Flags: needinfo?(emilio)
Pushed by ealvarez@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8591dc31ccda Invalidate paint on image load, not on size available. r=tnikkel

Backout did its thing:

== Change summary for alert #28633 (as of Thu, 04 Feb 2021 18:24:59 GMT) ==

Improvements:

Ratio Suite Test Platform Options Absolute values (old vs new)
23% wikipedia loadtime linux64-shippable-qr nocondprof warm webrender 778.04 -> 601.54
18% wikipedia fcp linux64-shippable-qr nocondprof warm webrender 763.29 -> 626.42
16% wikipedia linux64-shippable-qr nocondprof warm webrender 822.71 -> 694.86
15% wikipedia LastVisualChange linux64-shippable-qr nocondprof warm webrender 1,133.33 -> 960.00
7% wikipedia loadtime macosx1014-64-shippable-qr nocondprof warm webrender 985.92 -> 921.08

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=28633

Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → FIXED
Target Milestone: 86 Branch → 87 Branch
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: