*** GENERAL GOALS ***

We're working on a redesign of permission notifications. We believe the new design would reduce accidental dismissals and improve the overall interaction, but we would like to validate that it actually does.

*** HIGH LEVEL QUESTIONS ANSWERED ***

- Which types of UI interactions are the most common? Any surprises?
- How many permissions are granted and how many denied through these notifications (thus excluding manual changes)?
- How many notifications are dismissed without an explicit choice?
- Do reopened notifications actually show an intention to take an action?
- Are there specific notifications that show a different interaction pattern than the average of all notifications? Any surprises?
- With the new notification design, do the above results change?

*** TELEMETRY ***

Note that we don't record telemetry in private windows, only because the available actions can be different from regular mode. The main difference is that all of the persistent permission options like "Always remember" aren't there, so they really need to be handled separately to avoid skewing results. For notifications with the same choices, there would be no reason not to record in private windows as well, but it's just simpler to use the same check for everything.

=== POPUP_NOTIFICATION_STATS ===

A general keyed histogram recording usage of notifications generated by "PopupNotifications.jsm". The key is the notification ID, an argument specified by the callers of the "show" method. An additional key named "(all)" is recorded regardless of notification ID.

We record statistics for reopened notifications separately, because we want to understand if an intention to interact with the notification actually results in a different permission choice. Anything from 0 to 19 applies to the first time the notification is shown. Anything from 20 to 29 applies to reopened notifications. Each value is recorded at most once per notification.

   0: Notification offered ("show" method called).
      Incremented if a new prompt with the same ID replaces an existing one.
   1: Main action actually triggered (requires the security delay).
   2: Secondary action actually triggered.
   3: Third possible action actually triggered.
   4: Fourth possible action actually triggered.
   5: Dismissed by clicking elsewhere on the page or in the system.
   6: Dismissed by switching to a different tab.
   7: Dismissed by clicking on the "X" icon.
   8: Dismissed by clicking the "Not now" submenu option.
  10: The submenu dropdown arrow has been clicked
      (not necessarily resulting in an action).
  11: The 'Learn More' link for the specific notification has been clicked.

  20: Notification reopened by the user after it was dismissed.
  21: Same as 1 (main action), but on a reopened notification.
  22: ...same as 2, but on a reopened notification.
  23: ...and so on.

=== POPUP_NOTIFICATION_MAIN_ACTION_MS ===

Milliseconds between the time the notification was first offered (and also displayed on the screen, unless differently specified in the "show" method) and the time the main action was triggered. The key is the notification ID, and we also record the "(all)" key.

This is the total timing of the main action since the notification was
created, even if the notification was dismissed in the meantime.

We record 40 buckets between 100 ms and 10 minutes. Simulation:

https://telemetry.mozilla.org/histogram-simulator/#low=100&high=600000&n_buckets=40&kind=exponential&generate=log-normal

Anything less than 500 ms indicates that the action was not executed the first time, because we incurred in the security delay.

=== POPUP_NOTIFICATION_DISMISSAL_MS ===

Milliseconds between the time the notification was first offered and the time it was first dismissed, unless the main action was triggered. The key is the notification ID, and we also record the "(all)" key.

We record 50 buckets between 200 ms and 20 seconds. Simulation:

https://telemetry.mozilla.org/histogram-simulator/#low=200&high=20000&n_buckets=50&kind=exponential&generate=log-normal

*** DURATION OF COLLECTION ***

Collection of the POPUP_NOTIFICATION_STATS histogram is opt-out in release channels and expires in Firefox 50 (bug 1228671).

Collection of the other histograms is opt-in in release channels and expires in Firefox 52 (bug 1257222).

Most data should stop being collected when the work on Permissions Notification has stabilized or when Firefox 52 leaves the Beta channel sometime in 2017, whichever comes first.

*** RESPONSIBLE TEAM ***

Firefox Privacy and Security:

https://wiki.mozilla.org/Firefox/privacyandsecurity