Closed Bug 1806765 Opened 2 years ago Closed 2 years ago

Implement zero-prefix telemetry

Categories

(Firefox :: Address Bar, task, P1)

task

Tracking

()

VERIFIED FIXED
110 Branch
Tracking Status
firefox110 --- verified

People

(Reporter: adw, Assigned: adw)

References

Details

Attachments

(2 files)

For the weather suggestions experiment, we need to record telemetry for the zero-prefix (topsites) view.

https://mozilla-hub.atlassian.net/browse/SNT-333

This adds telemetry to UrlbarView that records the following things related to
the zero-prefix view (i.e., the topsites view):

  • Exposures: How many times the ZP view was shown
  • Engagements: How many times a result was picked in the ZP view
  • Abandonments: How many times the user abandoned the ZP view
  • Dwell time: How long the user was shown the ZP view

I considered adding telemetry specifically for topsites instead of the ZP view
as a whole, but since we have plans to start showing other types of results in
the ZP view, I don't think it's a good idea to rely on one specific type of
result as a proxy for the view itself. What DS and Product want to know about is
the view itself: how many times it was shown, for how long, etc.

This also adds one related scalar related to weather suggestions that counts the
number of times it's shown. This is the same scalar I added in D164778 using a
different, more complex approach.

Attached file request.md (deleted) —
Attachment #9311068 - Flags: data-review?(erichards)

Comment on attachment 9311068 [details]
request.md

Data Review Form

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

  2. Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.

    • Yes, through the normal data-collection opt-out in Firefox settings.
  3. If the request is for permanent data collection, is there someone who will monitor the data over time?

  4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

    • Category 2 - interaction data
  5. Is the data collection request for default-on or default-off?

    • Default-on.
  6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

    • No
  7. Is the data collection covered by the existing Firefox privacy notice?

    • Yes
  8. Does the data collection use a third-party collection tool? If yes, escalate to legal.

    • No

Result

data-review+

Attachment #9311068 - Flags: data-review?(erichards) → data-review+

Thank you so much for going into such detail on the context in the data review. It's extremely helpful :)

Pushed by dwillcoxon@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/bb55d68d351b Implement zero-prefix telemetry and add weather suggestion exposure telemetry. r=dao
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 110 Branch

STR for QA

This bug adds telemetry for the zero-prefix urlbar panel, which is the panel that's opened when you click into the urlbar before you type anything. That means you might end up recording this telemetry more than once as you do verification, e.g. as you click into the urlbar to load about:telemetry, etc.

Abandonment

  1. Click into the urlbar to open the zero-prefix panel
  2. Leave it open for a second or two, doesn't matter how long
  3. Close the panel without picking anything in it
  4. FX_URLBAR_ZERO_PREFIX_DWELL_TIME_MS histogram: Verify a value has been recorded that's approximately equal to the number of milliseconds the panel was open
  5. urlbar.zeroprefix.exposure scalar: Verify incremented (at least) once
  6. urlbar.zeroprefix.abandonment scalar: Verify incremented (at least) once
  7. urlbar.zeroprefix.engagement scalar: Verify not incremented

Engagement

  1. Click into the urlbar to open the zero-prefix panel
  2. Pick any result in the panel
  3. FX_URLBAR_ZERO_PREFIX_DWELL_TIME_MS histogram: Verify a value has been recorded that's approximately equal to the number of milliseconds the panel was open
  4. urlbar.zeroprefix.exposure scalar: Verify incremented (at least) once
  5. urlbar.zeroprefix.abandonment scalar: Verify not incremented
  6. urlbar.zeroprefix.engagement scalar: Verify incremented (at least) once

Weather exposure

  1. Enable weather suggestions: set browser.urlbar.weather.featureGate to true
  2. Click into the urlbar to open the zero-prefix panel
  3. Verify a weather suggestion appears as the first result (If it doesn't, you may need to allow Firefox a few seconds to fetch it. Try closing the panel and then opening it again)
  4. Close the panel without picking anything
  5. contextual.services.quicksuggest.exposure_weather scalar: Verify incremented once and has a "Value" of 1
Flags: qe-verify+
Flags: in-testsuite+

I have verified the following telemetry probes on the latest Firefox Nightly 110.0a1 (Build ID: 20230116095316) on Windows 10 x64, macOS 12.4, and Ubuntu 20.04 x64.

  • Scalars:
    urlbar.zeroprefix.exposure: value increments each time the zero-prefix panel is displayed
    urlbar.zeroprefix.engagement: value increments each time a result from the zero-prefix panel is picked
    urlbar.zeroprefix.abandonment: value increments each time the zero-prefix panel is dismissed without picking a result
    contextual.services.quicksuggest.exposure_weather: value increments each time a weather result is displayed in the zero-prefix panel

  • Histograms:
    FX_URLBAR_ZERO_PREFIX_DWELL_TIME_MS: value recorded that's approximately equal to the number of milliseconds the panel was open

Status: RESOLVED → VERIFIED
Duplicate of this bug: 1805827
Blocks: 1815018
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: