Talos has detected a Firefox performance regression from push 5d6dac688ec8. As author of one of the patches included in that push, we need your help to address this regression. This is a list of all known regressions and improvements related to the push: https://treeherder.mozilla.org/perf.html#/alerts?id=1257 On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format. To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests#tsvgx https://wiki.mozilla.org/Buildbot/Talos/Tests#tsvg-opacity https://wiki.mozilla.org/Buildbot/Talos/Tests#tp5 https://wiki.mozilla.org/Buildbot/Talos/Tests#tabpaint Reproducing and debugging the regression: If you would like to re-run this Talos test on a potential fix, use try with the following syntax: try: -b o -p win64,linux64,win32 -u none -t svgr[Windows 8,Windows XP,Windows 7],tp5o-e10s[Windows 8,Windows XP,Windows 7],svgr-e10s[Windows 8,Windows XP,Windows 7],other-e10s[Windows 8,Windows XP,Windows 7] --rebuild 5 # add "mozharness: --spsProfile" to generate profile data (we suggest --rebuild 5 to be more confident in the results) To run the test locally and do a more in-depth investigation, first set up a local Talos environment: https://wiki.mozilla.lorg/Buildbot/Talos/Running#Running_locally_-_Source_Code Then run the following command from the directory where you set up Talos: talos --develop -e [path]/firefox -a tsvgx:tsvgr_opacity:tp5o:tabpaint (add --e10s to run tests in e10s mode) Making a decision: As the patch author we need your feedback to help us handle this regression. *** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! *** Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

I have done a lot of retriggers on this and the previous changeset, here is a compare view: https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-inbound&originalRevision=e113053408b6&newProject=mozilla-inbound&newRevision=5d6dac688ec8&framework=1 what regressions we have are: tabpaint: * 22% linux64 e10s * 29% win7 e10s * 31% win8 e10s * 28% winxp e10s tp5o: * 2% linux64 e10s tsvgx: * 3% linux64 e10s * 3% win7 e10s * 3% win8 * 7% win8 e10s there is a 4% improvement on win8 tsvg_opacity. :avih, can you confirm this is expected. I know this is making our tests more accurate- but we need to be well aware that this is regressing e10s vs non-e10s- something we should call out in the release criteria for e10s.

> :avih, can you confirm this is expected. It's definitely not expected. > I know this is making our tests more accurate It does not, or at least not intentionally or knowingly. Its only goal is to prevent the video performance test (bug 1254898) from hanging. > but we need to be well aware that this is regressing e10s vs > non-e10s- something we should call out in the release criteria for e10s. Obviously we don't want and should not accept those diffs, because there are no positive performance side effects to this bug which we know of, other than preventing the video test hang. If the offending patch really is bug 1266181, then we'll need to try a different approach there. Whether you want to back it out or wait for a followup is up to you. I'm fine with either, possibly even preferring the backout since its effect would be faster.

However, important to mention, bug 1266181 definitely does not affect Firefox performance in any way. The patch is in talos, and the most it can do is affect the numbers which talos reports.

jmaher, please resolve this!

So, we don't need to track this from relman's point of view since it doesn't impact users or the release. But, we still do want to make sure that we have accurate judgements of e10s perf vs. non-e10s. We can track on that rationale for now.

when we backed this out, the perf regressions/improvements were reset.

(In reply to Joel Maher (:jmaher) from comment #6) > when we backed this out, the perf regressions/improvements were reset. The performance didn't change. Only the reported numbers did :) but yeah. Thanks for verifying.