Perfherder has detected a browsertime performance regression from push 1716229005d8b97f305bd663b8bc8d3fec4a4f3a. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Improvements:

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

Set release status flags based on info from the regressing bug 1700109

== Change summary for alert #29614 (as of Thu, 08 Apr 2021 05:56:29 GMT) ==

Regressions:

Improvements:

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=29614

Any thoughts on this Mike?

The tresize and tabswitch regressions listed in comment 2 for Linux (only) look small enough in magnitude (less than 1ms on average each) to be worth ignoring.

The page speed regressions and improvements in comment 0 are much more surprising.

Hey jesup, do you know what the current state-of-the-art is for getting comparison profiles for page speed tests?

Making sure rtestard is in the loop.

Some of the regressions relate to sites with video playback (Twitch and Youtube) - could that relate to the 2nd line of text now displayed on tab?

Setting as P1 o reflect the fact we need to investigate in order to understand the source of the issue, it can then be re-assessed once we better understand the root cause

Dusting off my perf chops to see what's going on here.

Going to focus on the 90% Imgur one. Pretty clear signal in the graph:

https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&selected=3383744,1341468505&series=mozilla-central,3423047,1,13&series=autoland,3383744,1,13&timerange=7776000

Generating profiles on these pushes:

Before: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&revision=8aa1b80a8c6b66cccc2e37080fbf6197eee8d801&searchStr=linux%2Cimgur

After: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=H6_U1LgbSfGCYoLnS_5xOg.0&resultStatus=success%2Csuperseded%2Cusercancel%2Crunning%2Cpending%2Crunnable&revision=1716229005d8b97f305bd663b8bc8d3fec4a4f3a&searchStr=linux64%2Cimgur

Here are some profiles for the 90% Imgur regression:

Before: https://share.firefox.dev/3snSLDf

After: https://share.firefox.dev/3uSqYwh

The "After" profile with Proton seems to be loading more JS in the content process than the "Before" profile. Here's a list of JS files that the "after" profile seems to load that the "before" process does not:

• https://adservice.google.ro/adsid/integrator.js?domain=imgur.com
• https://securepubads.g.doubleclick.net/gpt/pubads_impl_2020060103.js

One thing worth noting is that the browser chrome is now larger, and the content area is smaller, which may affect the visual metrics calculations.

Doing a side by side comparison reveals that the proton one actually did a bit better

So jesup notes that the last major frame of difference is that "We value your privacy" banner, and that it appears slightly later in the Proton case. Because that change takes up so much of the content area, it has a relatively high impact on the overall score.

I've zoomed in on the part of each profile where (I believe) the banner is being shown after a setTimeout:

Before: https://share.firefox.dev/3ahxIfg

After: https://share.firefox.dev/2QwCBdi

Notably, there's a long-ish (47.1ms) layout flush in the "after" profile.

Some updates here:

Imgur regressions on Linux 64 cold webrender

I wrote a patch that took the pre-Proton state of Firefox, and inflated the navigation toolbar padding to match Proton, which caused the content area to be the same size in both the before and after cases.

This seems to address the Imgur issues on Linux 64: https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=5bc22fff4a703d278926430e3eae22f5740b04d0&newProject=try&newRevision=379a834aa848613146bc7b0c590336926acf7058&framework=13

So my conclusion is that this "regression" is really just a re-baselining. I don't think it actually is capturing a meaningful regression that would impact our users, so I suggest we accept it as the new baseline with Proton enabled.

YouTube regressions on Windows 10 64-bit webrender (cold and warm)

These regressions appear to have gone away?:

• https://treeherder.mozilla.org/perfherder/graphs?timerange=1209600&series=autoland,3399129,1,13
• https://treeherder.mozilla.org/perfherder/graphs?timerange=1209600&series=autoland,3399140,1,13

So I guess there's nothing to do here.

CNN

The CNN regression also appears to have receded:

• https://treeherder.mozilla.org/perfherder/graphs?timerange=1209600&series=autoland,3381656,1,13

The rest

The rest of the regressions persist. I'm going to see if the change to the content area is the cause for them, too.

After retriggers, the only signal that appears to stick around is reddit ContentfulSpeedIndex opt warm webrender on Windows 10: https://treeherder.mozilla.org/perfherder/compare?originalProject=try&originalRevision=5bc22fff4a703d278926430e3eae22f5740b04d0&newProject=try&newRevision=379a834aa848613146bc7b0c590336926acf7058&framework=13

This regression appears to persist still: https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&series=autoland,3381656,1,13&series=autoland,3400821,1,13&timerange=1209600

Investigating - though note that the Reddit vismet test is Tier 2, so this should not block.

Side by side comparison of Reddit warm run on Windows.

I used sparky's generate_side_by_side.py script with these arguments:

python3 generate_side_by_side.py --base-revision 5bc22fff4a703d278926430e3eae22f5740b04d0 --base-branch try --new-revision 379a834aa848613146bc7b0c590336926acf7058 --new-branch try --platform test-windows10-64-shippable-qr/opt --test-name browsertime-tp6-firefox-reddit-e10s --warm --output ./output

Slowing the video down, it looks like the "after" video starts the load a frame or two later, and the cookie banner at the bottom comes in a few frames after.

The before and after profiles don't reveal anything obviously actionable here. I suspect the change in browser UI has caused a shift in scheduling, or memory layout, or something along those lines, which is causing this small regression.

rtestard: Given my conclusion for the regressions in comment 18, and in this comment, I suggest this bug no longer block MR1.

Thanks for the thorough investigations! Agreed that if this is most likely a re-baselining it should not block Proton.
Leaving this opened as a P3 since it seems like a follow-up is needed to ensure that the proton changes are accounted for moving forward (set a new baseline?)

Any news here?