Closed Bug 1459966 Opened 7 years ago Closed 6 years ago

browser/modules/test/browser/browser_UsageTelemetry_urlbar.js is going to permafail because probe FX_URLBAR_SELECTED_RESULT_METHOD expires when the Gecko version number is bumped to 63 on 2018-06-25

Categories

(Firefox :: Address Bar, defect, P1)

defect

Tracking

()

RESOLVED FIXED
Firefox 63
Tracking Status
firefox-esr52 --- unaffected
firefox-esr60 --- unaffected
firefox60 --- unaffected
firefox61 --- unaffected
firefox62 blocking unaffected

People

(Reporter: aryx, Assigned: adw)

References

Details

(Whiteboard: [fxsearch])

Attachments

(2 files)

See bug 1334615 where probes including this one got added. central-as-beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&revision=67f7a605b345437aa7e33d58076cf6b5ac7069ae&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable&selectedJob=177472940 Log: https://treeherder.mozilla.org/logviewer.html#?job_id=177472940&repo=try TEST-UNEXPECTED-FAIL | browser/modules/test/browser/browser_UsageTelemetry_searchbar.js | expected counts should match for FX_SEARCHBAR_SELECTED_RESULT_METHOD index 0 - 6448 == 1 - JS frame :: chrome://mochitests/content/browser/browser/modules/test/browser/browser_UsageTelemetry_searchbar.js :: checkHistogramResults :: line 11
Flags: needinfo?(dzeber)
Flags: needinfo?(dzeber) → needinfo?(adw)
Tracking to make sure this is fixed before the next merge day in mid-June.
Assignee: nobody → adw
Status: NEW → ASSIGNED
Flags: needinfo?(adw)
Priority: -- → P1
How do we usually deal with this? This patch just try-catches the getHistogramById() call and skips the relevant parts of the test when it throws. And then once we bump to 63, we can remove this code altogether.
I'd rather bump the histogram version, and when we'll really decide to not track it anymore, we should remove the code. We still care about analyzing how users use the Address Bar imo. Javaun?
Flags: needinfo?(jmoradi)
I must note that the following histograms are set to expire "never": FX_URLBAR_SELECTED_RESULT_INDEX FX_URLBAR_SELECTED_RESULT_TYPE FX_URLBAR_SELECTED_RESULT_INDEX_BY_TYPE I'm not sure why the *_SELECTED_RESULT_METHOD differ, are those considered more privacy sensitive than the above ones? More computationally expensive? Could we just evalaute changing these (urlbar and searchbar) to "never"?
Flags: needinfo?(dzeber)
I agree that these probes should be set to expire "never", similar to those listed in Comment 5. I'm not sure of a particular reason why these don't already, aside from just the general guideline of setting an expiration version ~5 versions out for new probes. I don't think these are any more privacy sensitive. Flagging Rob as a data steward, as I'm not sure if there's any process around this. Rob, this is the probe you reviewed retroactively in Bug 1437803, Comment 22. Are we OK to just change the expiration version to "never"?
Flags: needinfo?(dzeber) → needinfo?(rrayborn)
Comment on attachment 8976670 [details] Bug 1459966 - Change FX_URLBAR_SELECTED_RESULT_METHOD and FX_SEARCHBAR_SELECTED_RESULT_METHOD histograms to expire never. https://reviewboard.mozilla.org/r/244760/#review251730 Clearing, because likely we'll go another direction.
Attachment #8976670 - Flags: review?(mak77)
Whiteboard: [fxsearch]
Sorry for the late reply. Data Review should be re-requested on probe change for documentation's sake, but this is an obvious approval. e.g. It would be good to just copy past the previous request while requesting permanent collection and I'll give it a quick updated approval. Thanks
Looks like we can proceed.
Flags: needinfo?(jmoradi)
Comment on attachment 8976670 [details] Bug 1459966 - Change FX_URLBAR_SELECTED_RESULT_METHOD and FX_SEARCHBAR_SELECTED_RESULT_METHOD histograms to expire never. https://reviewboard.mozilla.org/r/244760/#review256850
Attachment #8976670 - Flags: review-
Flags: needinfo?(adw)
Since we're close to the merge (+ all hands) and we got pre-approval, we'll land the change, and then complete the data review informationa next week for the official approval. The bug will stay open until that's done.
Keywords: leave-open
Comment on attachment 8976670 [details] Bug 1459966 - Change FX_URLBAR_SELECTED_RESULT_METHOD and FX_SEARCHBAR_SELECTED_RESULT_METHOD histograms to expire never. https://reviewboard.mozilla.org/r/244760/#review257312
Attachment #8976670 - Flags: review?(mak77) → review+
Flags: needinfo?(adw)
Pushed by dwillcoxon@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8836d1ae45c6 Change FX_URLBAR_SELECTED_RESULT_METHOD and FX_SEARCHBAR_SELECTED_RESULT_METHOD histograms to expire never. r=mak
(In reply to Marco Bonardo [::mak] from comment #12) > and then complete the data review informationa next week I'll go ahead and do this now
Attached file data review request (deleted) —
We are requesting data review from rrayborn for the FX_URLBAR_SELECTED_RESULT_METHOD and FX_SEARCHBAR_SELECTED_RESULT_METHOD histograms, initially landed in bug 1334615, and updated here. We're simply updating them from expiring in 63 to expiring never.
Attachment #8985683 - Flags: review?(rrayborn)
Drew, we're building 62.0b1 next Monday, do you feel confident this is in an OK state for the first 62 dev edition release next week?
Flags: needinfo?(adw)
Yes, this bug doesn't affect 62 one way or another.
Flags: needinfo?(adw)
We'd like to close this bug, can we get attention from the data team please? we only need an official post-facto data review, we only got a by-email approval.
Flags: needinfo?(rweiss)
Comment on attachment 8985683 [details] data review request Apologies for missing this. General Notes: The only non-standard part of this request is that collection is indefinite. This should be fine as there's clear assignment and it's low risk 1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate? Yes, histograms + inline (BrowserUsageTelemetry.jsm) 2) Is there a control mechanism that allows the user to turn the data collection on and off? Yes, Telemetry defaults 3) If the request is for permanent data collection, is there someone who will monitor the data over time? Yes, :javaun/:dzeber through fx-search list 4) Using the category system of data types (https://wiki.mozilla.org/Firefox/Data_Collection), what collection type of data do the requested measurements fall under? Category 2 5) Is the data collection request for default-on or default-off? Telemetry defaults 6) Does the instrumentation include the addition of any *new* identifier* (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)? No 7) Is the data collection covered by the existing Firefox privacy notice? Yes 8) Does there need to be a check-in in the future to determine whether to renew the data? No
Flags: needinfo?(rweiss)
Flags: needinfo?(rrayborn)
Attachment #8985683 - Flags: review?(rrayborn) → review+
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 63
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: