Windows Default Browser Agent - Phase 2 - Toast notification telemetry
Categories
(Toolkit :: Default Browser Agent, enhancement)
Tracking
()
Tracking | Status | |
---|---|---|
firefox80 | --- | fixed |
People
(Reporter: nalexander, Assigned: bytesized)
References
()
Details
Attachments
(7 files, 1 obsolete file)
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-github-pull-request
|
Details | |
(deleted),
text/plain
|
chutten
:
data-review+
|
Details |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details | |
(deleted),
text/x-phabricator-request
|
Details |
This ticket tracks delivering telemetry about interactions with any displayed toasts.
Philosophically such interactions are events, which have special handling in Glean. I'm not sure if it's feasible or desirable to try to mimic this event-y flow; part of this ticket is working out such details.
Phase 1 already includes regularly scheduled telemetry so we'll stick to the same, or a very similar, technical implementation.
Reporter | ||
Updated•5 years ago
|
Reporter | ||
Updated•5 years ago
|
Assignee | ||
Comment 1•5 years ago
|
||
Updated•5 years ago
|
Assignee | ||
Comment 2•5 years ago
|
||
Comment on attachment 9147771 [details]
Bug 1621703 - Windows Default Browser Agent Telemetry r=mhowell,agashlin,nalexander
This patch requires data review. It adds fields to the existing windows default browser agent ping, which is external to Firefox. These are the fields that are being added:
- notification_type - A string indicating whether the notification that was possibly shown was the initial notification, or the followup notification.
- notification_shown - A string indicating whether a notification was shown, not shown, or resulted in an error.
- notification_action - A string indicating what interaction occurred with notification (which button was pressed, how the notification was dismissed).
- client_id - A UUID stored in HKEY_CURRENT_USERS (i.e. it is a per-Windows-user ID). It does not match the UUID used for any other client ID in any other ping.
We are hoping to merge this pretty soon so, if possible, a quick turnaround would be greatly appreciated. Thanks!
Assignee | ||
Comment 3•5 years ago
|
||
Telemetry fields have been added to the pipeline schema in commit 45df778ecc8460f7333c126de808c1c9281457e9.
Assignee | ||
Comment 4•5 years ago
|
||
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Comment 5•5 years ago
|
||
Data review fixed.
Comment 6•5 years ago
|
||
Chutten: when you do a data review here, we'd specifically like your thoughts on whether or not we should be adding a client_id.
Comment 7•5 years ago
|
||
Updated•5 years ago
|
Updated•4 years ago
|
Comment 8•4 years ago
|
||
Despite the requester's client_id
field not being the same as telemetry client_id
, there is likely no need for this field.
- to answer Q1 for a given day, the number of pings where default browser is not Firefox and last browser is Firefox which doesnt require a UUID.
To answer this question for period larger than day e.g. one week a UUID would be required, but is this a required question?
Comment 9•4 years ago
|
||
(In reply to "Saptarshi Guha[:joy]" from comment #8)
Despite the requester's
client_id
field not being the same as telemetryclient_id
, there is likely no need for this field.
Documenting some context from past conversations. The client_id
is a suggestion I made on the Phase 1 of this telemetry. IIRC, the goal of Phase 1 was to be able to faster identify retention events caused by forced changes to the default browser.
to answer Q1 for a given day, the number of pings where default browser is not Firefox and last browser is Firefox which doesnt require a UUID.
My initial concern for the Phase 1 telemetry was that we'd have some clients sending many pings per day. This is a problem we face with our main-ping. Usually, I'd just spot check the ping counts against our main-ping to confirm they're similar, but default browser telemetry will likely report very different user volumes than the main-ping.
Again, this is just for context. My review of the Phase 1 ping was very cursory, so I defer to Saptarshi.
Comment 10•4 years ago
|
||
Here is a summary of discussions with Rachel and Saptarshi regarding whether we need a client_id:
1 Questions we need answered with the data
a Is there a trend shift where users change defaults to Browser X? We should be able to do this without risk of false positives like "1 client generates many pings". Clients can send more than one ping a day and bots might send higher counts.
b What is the share of Firefox users with Firefox set as default moving to Browser X? User was previously interpreted as DAU although recent data explorations in share of MAU that are inactive showed that "Share of DAU that have a characteristic" can be very different from "Share of MAU that have a characteristic". Given churn is measured using MAU it sounds appropriate to use MAU for this measure - i.e What is the share of Firefox MAU with Firefox set as default moving to Browser X?
2 Do a and b above require a client_id?
a does not require a client_id. The team suggested usage of a registry entry on the machine for last ping sent - then the client could check it and not send another until 24 hours have passed. Ensures only one ping per machine is sent without extra telemetry
b MAU measurement requires a client_id. Although do we need MAU measurements, or is DAU good enough?:
- We may need MAU because we measure activity and churn from MAU and measuring only "share of DAU switching defaults" will give us a fairly noisy signal given the fact that MAU and DAU populations are so different.
- We may not need MAU because there is probably a bias around the fact that these users don't necessarily open Firefox on the day they send a ping - which probably implies that this is yet another cohort that will behave differently from DAU and MAU so attempting to map this behaviour to the one of MAU may be bound to failure. In this case there is probably a good argument that observing the trend of DAU that open their computers and switch defaults is probably as good of an approximation as MAU who open their computer and switch defaults when trying to use this as a signal that MAU churn may get impacted.
In summary:
- We'll move forward with the proposal to use a registry entry on the machine for last ping sent in order to avoid daily duplicates and comply with the 'lean data' principle
- We'll be observing the daily share of Firefox installs with Firefox set as default moving to Browser X and sending a daily ping as opposed to making this observation monthly given it's a leaner approach that does not require a client_id and that monthly observations would also not necessarilly be a better approximation. If we find that the daily the share is not a good enough signal we may revisit the decision at this point
Comment 11•4 years ago
|
||
Comment 12•4 years ago
|
||
There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:bytesized, could you have a look please?
For more information, please visit auto_nag documentation.
Assignee | ||
Comment 13•4 years ago
|
||
Yeah, these patches are waiting for some other work to be ready before they can merge.
Assignee | ||
Updated•4 years ago
|
Assignee | ||
Comment 14•4 years ago
|
||
Hopefully things look better this time around. The bug is now public and there is no longer a client id field being sent in the telemetry.
Assignee | ||
Comment 15•4 years ago
|
||
This patch also adds an install-directory-prefixed "Installed" value since RemoveAllRegistryEntries relies on prefixed value names and we no longer use any other install-directory-prefixed value names in the WDBA.
Assignee | ||
Comment 16•4 years ago
|
||
Since the WDBA now accesses registry values whose names are not prefixed, it could have concurrency problems when running at the same time as agents from other installations. This patch adds a mutex to address that problem.
Depends on D80706
Assignee | ||
Comment 17•4 years ago
|
||
Depends on D80707
Assignee | ||
Comment 18•4 years ago
|
||
Comment 19•4 years ago
|
||
Comment 20•4 years ago
|
||
Comment 21•4 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c313c7e92aad
https://hg.mozilla.org/mozilla-central/rev/abfba63bc3c1
https://hg.mozilla.org/mozilla-central/rev/edfbd5a25c68
https://hg.mozilla.org/mozilla-central/rev/2fedf44d9f2a
https://hg.mozilla.org/mozilla-central/rev/63c0fc65889b
Assignee | ||
Updated•2 years ago
|
Description
•