[Experiment] Pref-Flip Experiment: WebRender Intel Performance 70 Fx 70.0 Release
Categories
(Shield :: Shield Study, task, P3)
Tracking
(firefox70- affected)
People
(Reporter: experimenter, Assigned: jbonisteel)
References
()
Details
User Story
Experiment Type: Pref Flip Experiment What is the preference we will be changing gfx.webrender.all.qualified What are the branches of the experiment and what values should each branch be set to? - Treatment enabled 50%: Value: true Control branch - Treatment disabled 50%: Value: false This is the group that we will disable WebRender for so we can compare performance against those that do have WebRender What version and channel do you intend to ship to? 5% of Release Firefox 70.0 Are there specific criteria for participants? Prefs: IMPORTANT: This is a pref flip experiment, please keep it separate from the Rollout (https://experimenter.services.mozilla.com/experiments/webrender-intel-rollout/). Experiments: none Any additional filters: 1. Windows 10 2. normandy.telemetry.main.environment.system.gfx.features.wrQualified.status == 'available' 3. Intel Graphics cards, Windows 10 no battery, small screen: (<= 1920x1200) Sgua NOTE: Filtering on normandy.telemetry.main.environment.system.gfx.features.wrQualified.status == 'available' and an Intel graphics card should incorporate the screen size and battery limits; those are baked into the wrQualified assessment. In comments with Jessie we agreed to filter to single monitors. Countries: all Locales: all What is your intended go live date and how long will the experiment run? Oct 29, 2019 - Nov 26, 2019 (28 days) What is the main effect you are looking for and what data will you use to make these decisions? overall crash reports ≤ 5% increase in crash rate OOM crash reports ≤ 5% increase in crash rate shutdown crashes ≤ 5% increase in crash rate CANVAS_WEBGL_SUCCESS ≤ 5% regression in median of fraction "True" per user COMPOSITE_TIME Median per-user fraction of slow frames < 0.5% (absolute) CONTENT_FRAME_TIME_VSYNC ≤ 5% regression in median of per-user fraction of slow events CONTENT_FULL_PAINT_TIME ≤ 5% regression in fraction of slow paints (> 16 ms), ≤ 5% regression in median of per-user means DEVICE_RESET_REASON ≤ 5% increase in reset rate FX_PAGE_LOAD_MS_2 ≤ 5% regression in median of per-user means Crash (/submission/reset) rates will be measured as events per 1,000 usage hours. A "slow event" means 16 ms or 200% vsync. Do you plan on surveying users at the end of the experiment? No. Strategy and Insights can help create surveys if needed Who is the owner of the data analysis for this experiment? sguha@mozilla.com Will this experiment require uplift? False QA Status of your code: Signed off as GREEN, for more info see https://experimenter.services.mozilla.com/experiments/webrender-intel-performance-70/#comment496. Green Link to more information about this experiment: https://experimenter.services.mozilla.com/experiments/webrender-intel-performance-70/
WebRender Intel Performance 70
The goal of this study is to hold back a small population of eligible users from enabling WebRender, another milestone of which is riding the trains to release 70, in order to assess its effect on performance and engagement metrics. Our specific target for 70 is desktop, Windows 10 with Intel graphics cards and small screens.
Experimenter is the source of truth for details and delivery. Changes to Bugzilla are not reflected in Experimenter and will not change delivery configuration.
More information: https://experimenter.services.mozilla.com/experiments/webrender-intel-performance-70/
Assignee | ||
Comment 1•5 years ago
|
||
[Tracking Requested - why for this release]:
Comment 2•5 years ago
|
||
WebRender Intel Performance
Release 70
We have finished testing the WebRender Intel Performance experiment.
QA’s recommendation: GREEN - SHIP IT
Reasoning:
- We haven’t found any issues during testing.
Testing Summary:
- TestRail: Test suite
Tested Platforms:
- Windows 10 x64
Tested Firefox versions:
- Firefox Release 70
Regards,
Robert Martin
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Comment 3•5 years ago
|
||
I'm going to leave this open since it's yet to launch, but untrack it. Following along in Experimenter instead.
Reporter | ||
Updated•5 years ago
|
Description
•