Currently we only report search submission URLs for built in search engines. I'm proposing that we expand this list to include any URLs that have the same domain as our built in engines. This allows us to get more information about search hijacking without impacing user privacy.

Data collection review request.

Comment on attachment 8996116 [details] Data Collection Review Request Escalating to Marshall since I believe that the data falls within the Web activity category AND is default-on. Marshall, the configured search URLs we are interested in (hijacked by malware/crapware) might be considered Category 1 since it's part of the environment and wasn't configured by the user. However, for legitimate non-default search engines, this feels like Category 3 to me since it reveals a custom URL that a user chose to put in there.

Mike Kaply and I spoke about this briefly...Mike, if I recall correctly, you were going use a whitelist such that the information you are getting about non-default engines is very limited. Correct?

> Mike Kaply and I spoke about this briefly...Mike, if I recall correctly, you were going use a whitelist such that the information you are getting about non-default engines is very limited. Correct? Yes, you can see the update patch here: https://phabricator.services.mozilla.com/D2165 We only send submission URLs for engines we ship or for common search engines. And they are only sent at startup as part of the default engine information.

Given that, I think this is ok. This is category 3 but satisfies the policy requirement that technical mechanisms mitigate the risk. Practically speaking, the whitelist makes this essentially the same as our current approach.

Comment on attachment 8996116 [details] Data Collection Review Request > 5) List all proposed measurements and indicate the category of data collection > for each measurement, using the Firefox > [data collection categories](https://wiki.mozilla.org/Firefox/Data_Collection) on the Mozilla wiki. > > Search submission URL (URL that would be sent to the search service with NO > search term in it) Could you please add a description of the filtering that you allude to in comment 5? That way the data review request can stand on its own. Other than that, I have all of the info I need to do the data review.

Update with domains called out.

Sorry, I forgot something else. Is the schema of this data collection documented publicly anywhere? It seems like this is not a standard telemetry probe. I would ideally like to link to that documentation in my answer to the first question on https://github.com/mozilla/data-review/blob/master/review.md.

It's sent via the telemetry ping as part of the environment. See: https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html In particular, the settings section: defaultSearchEngineData: {, // data about the current default engine name: <string>, // engine name, e.g. "Yahoo"; or "NONE" if no default loadPath: <string>, // where the engine line is located; missing if no default origin: <string>, // 'default', 'verified', 'unverified', or 'invalid'; based on the presence and validity of the engine's loadPath verification hash. submissionURL: <string> // missing if no default or for user-installed engines },

Comment on attachment 8996736 [details] Updated request with domains called out 1) Is there or will there be **documentation** that describes the schema for the ultimate data set available publicly, complete and accurate? Yes, at https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html in the "defaultSearchEngineData" section under "settings". 2) Is there a control mechanism that allows the user to turn the data collection on and off? Yes, telemetry setting. 3) If the request is for permanent data collection, is there someone who will monitor the data over time?** Yes, Michael Kaply. 4) Using the **[category system of data types](https://wiki.mozilla.org/Firefox/Data_Collection)** on the Mozilla wiki, what collection type of data do the requested measurements fall under? ** Category 1 given the technical mitigations to whitelist search provider URLs. 5) Is the data collection request for default-on or default-off? Default on. 6) Does the instrumentation include the addition of **any *new* identifiers** (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)? No. 7) Is the data collection covered by the existing Firefox privacy notice? Yes. 8) Does there need to be a check-in in the future to determine whether to renew the data? No, permanent.

[Tracking Requested - why for this release]: Additional data for addressing hijacking.

Comment on attachment 8992434 [details] Bug 1476088 - Additional search submission telemetry. Florian Quèze [:florian] has approved the revision. https://phabricator.services.mozilla.com/D2165

Pushed by mozilla@kaply.com: https://hg.mozilla.org/integration/autoland/rev/fb85f9a81670 Additional search submission telemetry. r=florian

Comment on attachment 8992434 [details] Bug 1476088 - Additional search submission telemetry. Approval Request Comment [Feature/Bug causing the regression]: Add additional submission URLs to get a handle on hijacking. [User impact if declined]: None [Is this code covered by automated tests?]: Yes [Has the fix been verified in Nightly?]: No [Needs manual test from QE? If yes, steps to reproduce]: Go to yahoo.com and add Yahoo as a search engine and set it as the default. You do this by clicking the three dots dropdown and adding the engine, then right clicking on the engine and then selecting "set as default". Restart the browser. Go to about:telemetry and search on defaultSearchEngineData.submissionURL Verify that there is a URL present (without this patch, it is empty) [List of other uplifts needed for the feature/fix]: None [Is the change risky?]: Very low. [Why is the change risky/not risky?]: Telemetry reporting only, not user facing [String changes made/needed]:

Comment on attachment 8992434 [details] Bug 1476088 - Additional search submission telemetry. Adds additional Telemetry around search submissions. Includes new automated test coverage. Approved for 62.0b16 and 61.0.2.

Rebased release patch (hg export)

I have reproduced this bug on an affected Nightly build (2018-07-16), and using the STR from comment 16. I can confirm that the URL "https://ro.search.yahoo.com/search?p=&fr=opensearch" is present in about:telemetry, on the latest Builds tested: Nightly 63.0a1 (20180807220134), Beta 62.0b16 (20180808015758) and dot Release 61.0.2 (20180807170231). Verified on Win 10 x64, Mac OS X 10.9 and Ubuntu 16.04 x64.