We decided to go with a 5% sample of opt-out Telemetry usage from release. Bug 1191912 takes care of the sample, but we still need to flip the pref for making Telemetry default to opt-in again on 41.

Moving part 3 from bug 1191912 here, to track the pref change uplifts in one bug.

Comment on attachment 8645787 [details] [diff] [review] Make Telemetry opt-in for Firefox 41 Approval Request Comment [Feature/regressing bug #]: Unified Telemetry [User impact if declined]: We decided to not fully ship Unified Telemetry in 41 due to some unexplained discrepancies. Instead we make Telemetry opt-in again, but enable it opt-out for 5% of users. [Describe test coverage new/current, TreeHerder]: Automation coverage, doing basic manual confirmation. [Risks and why]: Low risk at this point - the feature has been tested on the trains for a while now. The biggest remaining concern here is data quality, which doesn't affect clients. [String/UUID change made/needed]: None.

Comment on attachment 8652272 [details] [diff] [review] Enable opt-out Telemetry sampling for Firefox 41 Approval Request Comment [Feature/regressing bug #]: Unified Telemetry [User impact if declined]: We decided to not fully ship Unified Telemetry in 41 due to some unexplained discrepancies. Instead we make Telemetry opt-in again, but enable it opt-out for 5% of users. [Describe test coverage new/current, TreeHerder]: Automation coverage, doing basic manual confirmation. [Risks and why]: Low risk at this point - the feature has been tested on the trains for a while now. The biggest remaining concern here is data quality, which doesn't affect clients. [String/UUID change made/needed]: None.

Comment on attachment 8645787 [details] [diff] [review] Make Telemetry opt-in for Firefox 41 UT is Opt in pref by default, seems safe. Beta41+.

Comment on attachment 8652272 [details] [diff] [review] Enable opt-out Telemetry sampling for Firefox 41 This is an optout (will only apply for 5% of release population). Beta41+

https://hg.mozilla.org/releases/mozilla-beta/rev/efc43d513eee https://hg.mozilla.org/releases/mozilla-beta/rev/5faa1eae6f68

Tested the sampling feature against live servers on Firefox 41.0b5 (20150827142634) and Firefox 41.0b6 (20150831172306), using Windows 10 x64, Ubuntu 14.04 x64 and Mac OS X 10.10.4. I've used various pref combinations to create specific conditions and triggered Telemetry and FHR pings to confirm that the expected behavior is met in all cases. In order to simulate client data originating from either the 5% or outside of the 5%, I've set specific clientIDs in state.json, per Georg's instructions. Apart from two errors I've been seeing in the Browser Console, there were no issues encountered during testing. Please note that the following errors were thrown no matter the test conditions and across the platforms: -- by Log.jsm:749:0 > 1441020571793 Toolkit.Telemetry ERROR TelemetryStorage::loadAbortedSessionPing > - error removing ping: PingReadError JS Stack trace: > PingReadError@TelemetryStorage.jsm:80:15 < > TelemetryStorageImpl.loadPingFile<@TelemetryStorage.jsm:1440:13 -- by Log.jsm:749:0 > 1441086970653 Toolkit.Telemetry ERROR TelemetryReportingPolicy::get > dataSubmissionPolicyNotifiedDate - Invalid date stored. Georg, what's your take on these?

(In reply to Andrei Vaida, QA [:avaida] from comment #8) > Tested the sampling feature against live servers on Firefox 41.0b5 > (20150827142634) and Firefox 41.0b6 (20150831172306), using Windows 10 x64, > Ubuntu 14.04 x64 and Mac OS X 10.10.4. > > I've used various pref combinations to create specific conditions and > triggered Telemetry and FHR pings to confirm that the expected behavior is > met in all cases. In order to simulate client data originating from either > the 5% or outside of the 5%, I've set specific clientIDs in state.json, per > Georg's instructions. > > Apart from two errors I've been seeing in the Browser Console, there were no > issues encountered during testing. Great to hear! > Please note that the following errors > were thrown no matter the test conditions and across the platforms: > -- by Log.jsm:749:0 > > 1441020571793 Toolkit.Telemetry ERROR TelemetryStorage::loadAbortedSessionPing > > - error removing ping: PingReadError JS Stack trace: > > PingReadError@TelemetryStorage.jsm:80:15 < > > TelemetryStorageImpl.loadPingFile<@TelemetryStorage.jsm:1440:13 > -- by Log.jsm:749:0 Known issue, just weird logging, covered by bug 1191825. > > 1441086970653 Toolkit.Telemetry ERROR TelemetryReportingPolicy::get > > dataSubmissionPolicyNotifiedDate - Invalid date stored. > > Georg, what's your take on these? Known issue, just weird logging, covered by bug 1191461.

Thanks for clarifying those, Georg! This bug is verified fixed on Firefox 41, per Comment 8 and Comment 9.

Confirming this fix with 41.0.2 build 2 (20151014143721) too, across platforms [1]. [1] Windows 7 64-bit, Mac OS X 10.10.4 and Ubuntu 14.04 32-bit