Talos has detected a Firefox performance regression from push: https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=2f1892facca9cf9dd9b5f4180a5b0a71c47d88fb As author of one of the patches included in that push, we need your help to address this regression. Regressions: 9% displaylist_mutate linux64 opt e10s 5,231.88 -> 5,690.80 You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=11025 On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format. To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running *** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! *** Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

:jya, I see that you authored the patches in bug 1404997, Can you take a look at this before too long, I know it might be into January give the holidays.

What does this test do, and where is it located? from the test title, I doubt it has anything to do with WebRTC...

sorry, I should have pointed it out: https://wiki.mozilla.org/Buildbot/Talos/Tests#displaylist_mutate this is a newer test added by dvander recently.

looking at the past 30 days: https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=mozilla-inbound,1577099,1,1&series=autoland,1577173,1,1 we have massive noise on December 9th on inbound and again massive noise on December 18th on autoland after each of those dates, things go back to the usual variation from 4927 to 6086, with everything in between. We can see that even within the november period, it fluctuates greatly. I see no evidence that the change on December 9th and December 18th are nothing but an anomaly of the day. And that bug 1404997 (pushed on December 17th) got anything to do with those two dates. in facts the score following that blip on December 18th, are all lower than the runs that occurred on December 12th.

:jya, thank you, I apologize for randomizing you and agree with your conclusion- this looks like a case of the 'weekends' where we see different patterns during high load vs low load. I recommend closing this as wontfix.

What's happening on week-end that could cause such variation? I would have assumed that treeherder being less busy, it would always score higher if it's based on machine load. But instead it's all over the place, from much slower to much faster.

I have never been able to figure it out: https://elvis314.wordpress.com/2014/10/30/a-case-of-the-weekends/ we still see this often and it gets odd when we have holidays, all-hands, etc. If this were a normal time of the year and we had a normal load, then the weekend pattern could be confirmed.