Firefox's Nightly and Dev Edition channels enable some debugging features that are not enabled in the Release channel. We should make sure we disable these debugging features when benchmarking so we have consistent performance results for all Firefox channels. https://developer.mozilla.org/docs/Mozilla/Benchmarking - Set environment variable JSGC_DISABLE_POISONING=1 to disable slow GC poisoning. - Set pref javascript.options.asyncstack=false to disable slow JS debugger feature. - Disable Flash plugin inoth b Firefox and Chrome so we are measuring browser performance, not Flash. @ sfink: are there any other settings (in Firefox or Chrome) we should disable when benchmarking Nightly vs Chrome with the Hasal tests?

I filed this in Hasal's GitHub issue list here: https://github.com/Mozilla-TWQA/Hasal/issues/493

I don't know of any that are significant enough to worry about. There are lots of differences (you can find them by searching for NIGHTLY_BUILD or RELEASE_OR_BETA), but most of them aren't going to matter. I see random stuff like the prefs browser.urlbar.oneOffSearches findbar.highlightAll findbar.modalHighlight and many more that I could imagine having an impact (eg when timing search), but I don't know if we want to try to keep track of a huge set of things that mostly don't matter -- and if they do, we might want to know about them. It's really the debugging and diagnostic differences that we care about. I suspect there are more than I've found so far, but I'll add them to the wiki page and file issues if I run into them.

Thanks. It sounds like we don't need to worry about those prefs because, IIUC, they control browser UI features and shouldn't affect web content performance.

ni Shako and myself to follow up. Thanks.

Fixed in Hasal testing framework

run with `--firefox-settings settings/default.json` parameter will enable this feature.