We have detected an awsy regression from push: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=9322f0ce835f554a0108f59dba244f32ebd8d492&tochange=90ad57245468c398be0be3fcdb2cdec1b9333232 As author of one of the patches included in that push, we need your help to address this regression. Regressions: 23% Images windows10-64 opt stylo 7,105,802.51 -> 8,765,477.90 22% Images windows10-64 pgo stylo 7,462,464.75 -> 9,090,809.38 4% Heap Unclassified windows10-64 pgo stylo41,276,303.39 -> 43,113,613.28 4% Heap Unclassified windows10-64 opt stylo41,327,290.70 -> 43,066,678.10 4% Resident Memory windows10-64 opt stylo473,359,269.95 -> 490,130,107.88 4% Explicit Memory windows10-64 opt stylo311,803,364.88 -> 322,762,749.88 3% Resident Memory windows10-64 pgo stylo464,851,298.54 -> 480,887,147.40 3% Explicit Memory windows10-64 pgo stylo312,704,988.06 -> 323,460,034.89 You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=11997 On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format. To learn more about the regressing test(s), please see: https://wiki.mozilla.org/AWSY/Tests

Other platforms are affected as well: Regressions: 27% Images windows7-32 opt stylo 5,693,513.40 -> 7,225,337.57 26% Images linux64-stylo-sequential opt stylo-sequential6,140,888.39 -> 7,729,311.24 24% Images osx-10-10 opt stylo 6,498,574.26 -> 8,042,625.19 8% Explicit Memory osx-10-10 opt stylo322,331,602.85 -> 348,534,520.90 6% Heap Unclassified windows7-32 opt stylo34,326,129.16 -> 36,303,555.68 6% Heap Unclassified windows7-32 pgo stylo34,171,244.88 -> 36,113,536.48 4% Explicit Memory windows7-32 opt stylo252,345,702.18 -> 262,951,473.47 4% Explicit Memory windows7-32 pgo stylo252,441,930.67 -> 262,428,847.34 3% Explicit Memory linux64-stylo-sequential opt stylo-sequential310,074,352.13 -> 319,732,109.16 2% Explicit Memory linux64 opt stylo313,311,254.14 -> 320,752,266.83

:Mardak It starting to look like bug 1383599 is responsible for these serious AWSY perf regressions. Please look over this issue. I've retriggered the perf tests, but the results just started to show up.

It looks like the cause for the regression is that AWSY is running with browser.newtabpage.enabled = false. Where before bug 1383599, it was running as if browser.newtabpage.enabled = true (because the pref wasn't being checked). erahm -> jmaher, does the test need to run with enabled = true, and is this where it's being set: https://searchfox.org/mozilla-central/rev/588d8120aa11738657da93e09a03378bcd1ba8ec/testing/marionette/components/marionette.js#89-94 And should it be flipped to true (just remove it as the default is true) as that's what it's been running with?

I am not sure here and :erahm is on pto for another week. if setting this =True fixes this regression we should do that and then leave a pending issue for :erahm.

Try run https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-central&originalRevision=d55b703721d2&newProject=try&newRevision=88fc24d19259&framework=4&filter=osx Shows -25.97% Images osx (for 1 run before and after)

Comment on attachment 8957665 [details] Bug 1444404 - Use default (true) value for browser.newtabpage.enabled for testing. https://reviewboard.mozilla.org/r/226586/#review232440 I am not sure what other tests will be affected by this, I suspect quite a few. Have you tested this on all unittests (try -bo -p all -u all)?

Pushed by edilee@gmail.com: https://hg.mozilla.org/integration/autoland/rev/9a200b344efe Use default (true) value for browser.newtabpage.enabled for testing. r=jmaher

This patch made a change to files for automation tooling that targets out-of-tree release channels without the review from a geckodriver/Marionette peer. I’m adding some notes to the relevant files to prevent this from happening in the future as part of bug 1401129. As I understand it browser.newtabpage.enable = false was broken at least since Firefox 56? Do we know about Firefox 55? My assumption is that it we don’t need to add this pref back for Firefox 55 compatibility, but I’d like to check to be sure.

Just checked 55.0.3, and browser.newtabpage.enabled does set the page to be mostly blank for the default old Tiles about:newtab, and activity stream isn't packaged, so browser.newtabpage.activity-stream.enabled does nothing. And in 56.0.2, browser.newtabpage.enabled also shows a mostly blank page for old Tiles about:newtab, and now activity stream is packaged For 57.0.4, activity stream is the default about:newtab, and browser.newtabpage.enabled doesn't do anything there. So the pref was not entirely functional since 57 (although one could trigger the blank by disabling both activity stream and newtabpage).

Thanks for checking, mardak! I don’t think we should have to worry about this for the geckodriver 0.20.0 release.

I confirm the push from comment 9 resolved this bug's issue: == Change summary for alert #12052 (as of Fri, 09 Mar 2018 21:17:16 GMT) == Improvements: 24% Images windows7-32 pgo stylo 7,615,412.06 -> 5,771,764.67 21% Images linux64-stylo-sequential opt stylo-sequential7,713,135.16 -> 6,126,343.62 20% Images linux64 opt stylo 7,760,082.68 -> 6,170,993.01 20% Images windows10-64 opt stylo 9,022,122.53 -> 7,213,663.75 19% Images windows10-64 pgo stylo 9,182,175.45 -> 7,436,880.46 19% Images windows7-32 opt stylo 7,119,213.66 -> 5,766,443.51 17% Images osx-10-10 opt stylo 7,792,453.23 -> 6,439,950.40 5% Heap Unclassified windows7-32 pgo stylo35,995,663.55 -> 34,084,058.05 5% Heap Unclassified windows7-32 opt stylo36,214,134.73 -> 34,314,121.08 4% Heap Unclassified windows10-64 pgo stylo43,076,576.72 -> 41,142,084.77 4% Explicit Memory windows7-32 pgo stylo263,656,416.52 -> 252,222,456.27 4% Heap Unclassified windows10-64 opt stylo42,931,024.60 -> 41,118,374.97 4% Explicit Memory windows10-64 pgo stylo325,134,727.97 -> 312,914,421.07 4% Explicit Memory windows7-32 opt stylo262,229,848.37 -> 252,390,323.57 4% Explicit Memory windows10-64 opt stylo323,345,299.33 -> 311,483,309.89 3% Explicit Memory osx-10-10 opt stylo352,670,241.26 -> 340,546,597.13 3% Explicit Memory linux64-stylo-sequential opt stylo-sequential320,146,774.54 -> 309,550,103.27 3% Resident Memory windows10-64 pgo stylo481,836,990.81 -> 466,890,741.34 3% Resident Memory windows10-64 opt stylo490,748,101.23 -> 475,774,845.05 3% Heap Unclassified osx-10-10 opt stylo74,111,632.79 -> 71,902,783.67 3% Explicit Memory linux64 opt stylo321,628,888.69 -> 313,598,628.41 For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=12052