Perfherder has detected a talos performance regression from push e718c31620b4ab8efc075b58c8f6839589af16d5. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

Improvements:

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the offending patch(es) will be backed out in accordance with our regression policy.

For more information on performance sheriffing please see our FAQ.

I'm not really sure what this error is saying. What is a tresize error, or how would I debug?

in the bug title it has, 25.13 - 21.63%, which is going from bigger to smaller. So, I think I need help understanding how smaller is worse in this case, and what these numbers mean.

I also have no frame of reference for how extreme of a regression this is, to try to figure out what could cause it.

The patch that caused this is primarily doing three things, three pref flips that impact newtab.

1. Add an image to newtab.
2. Turning off newtab highlights section by default.
3. Update Topsites section and section title styling in newtab.
4. Adding a personalize button and new personalize menu in newtab.

Looking at that list, I suspect it's number 4? But even then, it's surprising that flipping a pref that triggers some css and html and an image can cause a regression, unless it's really small.

I couldn't reproduce any regressions locally running this test with and without the regressing patch.

It actually seemed to get better, but hard to tell with noise.

The patch also exclusively makes changes to newtab, and this tresize test doesn't run newtab. They don't seem to be connected.

Is it possible this could be a false positive?

Alexandru, do you have answers for Scott's questions? Thanks

I retriggered both the regression and improvement to confirm they have this culprit.

Does anyone have an update on this, has it been resolved, was it a false positive? Can we close the Jira ticket that this is blocking?
https://jira.mozilla.com/browse/FIDE-451

(In reply to Ron Manning from comment #5)

Does anyone have an update on this, has it been resolved, was it a false positive? Can we close the Jira ticket that this is blocking?
https://jira.mozilla.com/browse/FIDE-451

I tried to debug this a bit more. The caveat here is performance numbers sometime produce noise, and maybe I'm getting false positives, but that's unlikely, it looks pretty consistent to me.

It looks like I was able to reproduce it locally. From here, it looks like removing the highlights section from newtab is causing this test to regress.

Makes no sense, but yeah, pretty sure that's what's happening locally and with perfherder.

I can also confirm I see a regression if I turn off topsites as well. It looked like turning off both topsites and highlights caused an even bigger regression. I boiled it down to the html for these two sections specifically that impact the test, it's not a pref or some other data point, it is the existence of some html on newtab.

I tried to turn off newtab completely, but the local tests started to produce inconsistent numbers, which made it hard to get a result from that.

There is also a blip of newtab when the tests run, which I'm not sure why or where that's coming from. By looking at the test code, I don't see it explicitly opening newtab, and I don't see it using the newtab tab for anything. I suspect the blip of newtab while the test starts might be related.

My hunch is this is less of a regression impacting users, and more likely a regression because of how the test is run, but I'm not sure.

(In reply to Ron Manning from comment #5)

Does anyone have an update on this, has it been resolved, was it a false positive? Can we close the Jira ticket that this is blocking?
https://jira.mozilla.com/browse/FIDE-451

Both the regression and improvement have the same culprit and they are still there.

I wouldn't worry about this and just accept the regression. The change improved new tab performance by 20 ms, while regressing linux trsize by 2 ms. Additionally the regression only showed up on the old graphics backend, webrender wasn't impacted. That particular backend will be obsolete in just a month or two, so you're chasing a perf regression in what is essentially soon to be dead code.

https://treeherder.mozilla.org/perfherder/graphs?highlightAlerts=1&highlightChangelogData=1&series=autoland,3139068,1,1&series=mozilla-central,3142372,1,1&series=mozilla-central,2819287,1,1&series=mozilla-central,3186923,1,1&timerange=5184000