Closed Bug 1676470 Opened 4 years ago Closed 4 years ago

[Experiment] Pref-Flip Experiment: Warpbuilder Experiment Fx 83.0 to 83.0 Release

Categories

(Shield :: Shield Study, task, P3)

Tracking

(firefox83+ fixed)

RESOLVED FIXED
Tracking Status
firefox83 + fixed

People

(Reporter: experimenter, Assigned: denispal)

References

(Blocks 1 open bug, )

Details

User Story

Delivery Type: Pref Flip Experiment

    What is the preference we will be changing

None

    What are the branches of the experiment and what values should
    each branch be set to?

- Treatment Warp disabled 50%:

Value: None

Users with warpbuilder disabled
        
- Control Warp enabled 50%:

Value: None

Users with warpbuilder enabled
        

    What version and channel do you intend to ship to?

1% of Release Firefox 83.0 to 83.0

    Are there specific criteria for participants?

Prefs:

Experiments:

Any additional filters:
Countries: Canada (CA) France (FR) Germany (DE) United States of America (US) 

Locales: all

    What is your intended go live date and how long will the experiment run?

Dec 01, 2020 - Dec 15, 2020 (14 days)

    What is the main effect you are looking for and what data will you use to
    make these decisions?

We will measure the outcome by an increase or decrease in the probes:

PERF_PAGE_LOAD_TIME_MS
PERF_PAGE_LOAD_TIME_FROM_RESPONSESTART_MS
PERF_DOM_CONTENT_LOADED_TIME_MS
PERF_DOM_CONTENT_LOADED_TIME_FROM_RESPONSESTART_MS
PERF_FIRST_CONTENTFUL_PAINT_MS
PERF_FIRST_CONTENTFUL_PAINT_FROM_RESPONSESTART_MS
PERF_REQUEST_ANIMATION_CALLBACK_PAGELOAD_MS
PERF_REQUEST_ANIMATION_CALLBACK_NON_PAGELOAD_MS
JS_EXECUTION_PROPORTION
TIME_TO_FIRST_INTERACTION_MS
INPUT_EVENT_RESPONSE_MS
KEYPRESS_PRESENT_LATENCY
LOAD_INPUT_EVENT_RESPONSE_MS
LOAD_INPUT_EVENT_RESPONSE_NO_PRELOAD_MS
LOAD_INPUT_EVENT_RESPONSE_PRELOAD_MS
SCALARS_TIMESTAMPS.ABOUT_HOME_TOPSITES_FIRST_PAINT
FX_NEW_WINDOW_MS

    Who is the owner of the data analysis for this experiment?

None

    Will this experiment require uplift?

False

    QA Status of your code:

https://jira.mozilla.com/browse/PI-866

Signed Off as GREEN as per comment - https://experimenter.services.mozilla.com/experiments/warpbuilder-experiment/#comment1239

    Link to more information about this experiment:

https://experimenter.services.mozilla.com/experiments/warpbuilder-experiment/

Warpbuilder Experiment

Mozilla is gathering data on the improvements or regressions caused by the Warpbuilder enhancement. This data will be used to verify whether an observable improvement can be measured in telemetry and to match with local CI results to verify accuracy of those tests.

Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.

Data Science Issue: https://jira.mozilla.com/browse/DO-343
More information: https://experimenter.services.mozilla.com/experiments/warpbuilder-experiment/

[Tracking Requested - why for this release]: this is an experiment turning off warpbuilder for 1% of users.

Warpbuilder QA

Firefox Release 83

We have finished testing the [PI-866] Warpbuilder QA experiment.

Quality status: GREEN - SHIP IT

Why is this experiment green?

  • Delivery mechanism:
    • No issues found.
  • Product specific:
    • No issues found.

Testing Summary:

Tested Platforms:

  • Windows 10 x64
  • Ubuntu 20.04 x64
  • macOS 10.15.7

Tested Firefox versions:

  • Firefox Release 83.0.en-US, en-CA, de, fr (Build ID:20201112153044)
User Story: (updated)
Summary: [Experiment]: Pref-Flip Experiment: Warpbuilder Experiment → [Experiment] Pref-Flip Experiment: Warpbuilder Experiment Fx 83.0 to 83.0 Release
Start Date: 2020-12-01 End Date: 2020-12-15
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
User Story: (updated)
You need to log in before you can comment on or make changes to this bug.