Closed Bug 1532374 Opened 6 years ago Closed 6 years ago

[Shield] Pref-Flip Study: Search in Private Browsing New Tab impact study, Fx66, Release

Categories

(Shield :: Shield Study, enhancement)

enhancement
Not set
normal

Tracking

(firefox66- affected)

RESOLVED FIXED
Tracking Status
firefox66 - affected

People

(Reporter: experimenter, Assigned: mconnor)

References

(Blocks 1 open bug)

Details

Search in Private Browsing New Tab impact study

In Firefox 66 we have the ability to show a new version of about:privatebrowsing that adds a search box, in order to better capture private browsing searches

More information: https://experimenter.services.mozilla.com/experiments/search-in-private-browsing-new-tab-impact-study/

Summary: [Shield] Pref-Flip Study: Search in Private Browsing New Tab impact study → [Shield] Pref-Flip Study: Search in Private Browsing New Tab impact study, Fx66, Release

In Firefox 66 we have the ability to show a new version of about:privatebrowsing that adds a search box, in order to better capture private browsing searches

Search in Private Browsing New Tab impact study

Firefox Release 66

We have finished testing the Search in Private Browsing New Tab impact study experiment.

QA’s recommendation: GREEN - SHIP IT

Reasoning:

  • No new issues have been found during testing.

  • A single known issue has been encountered during testing, Bug 1530706, which is an intermittent issue only on Mac using a Japanese keyboard. We do not consider this issue damaging enough to block the experiment, but we would recommend fixing it as soon as possible.

Testing Summary:

Tested Platforms:

  • Windows 10

  • Mac 10.14

  • Arch Linux 4.14

Tested Firefox versions:

  • Firefox 66RC1
Flags: shield-qa+
Experiment Type: Pref Flip Study What is the preference we will be changing browser.privatebrowsing.searchUI What are the branches of the study and what values should each branch be set to? - Treatment Variant (old UI) 50%: Value: false The variant group will see the old about:privatebrowsing UI without search (and a lot more text) - Treatment Control 50%: Value: true The control group will see the new about:privatebrowsing search functionality launching in Firefox 66 What version and channel do you intend to ship to? 0.9% of Release Firefox 66.0 Are there specific criteria for participants? No filtering required What is your intended go live date and how long will the study run? Mar 19, 2019 - Apr 30, 2019 (42 days) What is the main effect you are looking for and what data will you use to make these decisions? What is the main effect you are looking for and what data will you use to make these decisions? What metrics are you using to measure success We want to see no negative effects on the following metrics: SAP Search volume Organic search volume Ad clicks Ad displays Do you plan on surveying users at the end of the study? No. Strategy and Insights can help create surveys if needed Who is the owner of the data analysis for this study? Ben Miroglio Will this experiment require uplift? False QA Status of your code: Link to more information about this study: https://experimenter.services.mozilla.com/experiments/search-in-private-browsing-new-tab-impact-study/

Untracking since we have this information easily findable now in Experimenter.

experiment was ended on April 29th.

Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.