expose HttpTrafficAnalyzer information in webrequest
Categories
(WebExtensions :: Request Handling, enhancement, P1)
Tracking
(firefox70 fixed)
Tracking | Status | |
---|---|---|
firefox70 | --- | fixed |
People
(Reporter: mixedpuppy, Assigned: mixedpuppy)
References
(Blocks 1 open bug)
Details
Attachments
(1 file, 1 obsolete file)
(deleted),
text/x-phabricator-request
|
Details |
expose tracker information (see TrackingClassification[1] and ClassOfService[2])
This is moved from bug 1545163
Assignee | ||
Updated•6 years ago
|
Assignee | ||
Updated•6 years ago
|
Assignee | ||
Comment 1•5 years ago
|
||
expose classification flags to webrequest and proxy events.
https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsIHttpChannel.idl#510
Assignee | ||
Updated•5 years ago
|
Assignee | ||
Comment 2•5 years ago
|
||
Assignee | ||
Comment 3•5 years ago
|
||
The patch has a rough proposed schema for the classification api in webrequest (and proxy) api. You can look at it for a more detailed view of the api addition. I might remove the "optional" in schema so that one does not need to check for its existence as code below illustrates. Here is a rough sample of how one might use it:
browser.webRequest.onBeforeRequest.addListener(details => {
if (details.classification.firstParty.includes("tracking_ad")) { ... do something ... }
if (details.classification.thirdParty.includes("tracking_ad")) { ... do something ... }
}, ....);
In nsIHttpChannel.idl there is a bit more going on, such is isTrackingResource and isThirdPartyTrackingResource. I don't feel those are necessary as that can be checked via "details.classification.firstParty.length > 0".
If we want a higher level indication of any tracking classification, we could leave "details.classification" undefined if there is no classification.
Comment 4•5 years ago
|
||
I honestly don't have an opinion on this; looks good to me generally, tho. I think better to ask is either :baku or ehsan.
Comment 6•5 years ago
|
||
In nsIHttpChannel.idl there is a bit more going on, such is isTrackingResource and isThirdPartyTrackingResource. I don't feel those are necessary as that can be checked via "details.classification.firstParty.length > 0".
Right! We don't need to expose IsTrackingResource()/IsThirdPartyTrackingResource() if we expose classification flags. These 2 methods are helpers to know what must be consider a tracker, based on the current anti-tracking pref status. This doesn't need to be exposed to web-extensions.
Related to classification flags: we are planning to add more flags. What's the right way to keep web-extension code and nsIHttpChannel.classificationFlags in sync? Should I simply CC you to those bugs? The latest one is bug 1560040.
Comment 7•5 years ago
|
||
If these need to be kept in sync I'd recommend a unit test that ensures it.
Assignee | ||
Comment 9•5 years ago
|
||
(In reply to Andrea Marchesini [:baku] from comment #6)
Related to classification flags: we are planning to add more flags. What's the right way to keep web-extension code and nsIHttpChannel.classificationFlags in sync? Should I simply CC you to those bugs? The latest one is bug 1560040.
I'd just add them where apropriate and r? the right people. See the latest patch. Any addition would require changes ChannelWrapper.webidl/cpp, webrequest_json and the test file. We can probably simplify that somehow later.
Assignee | ||
Comment 10•5 years ago
|
||
Depends on D37162
Updated•5 years ago
|
Assignee | ||
Comment 11•5 years ago
|
||
Comment 12•5 years ago
|
||
Comment 13•5 years ago
|
||
Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=2bfb73091f2137235ed130f2227cd95bf0ecb9e2&selectedJob=261498003
Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=261498003&repo=autoland&lineNumber=2793
Backout link: https://hg.mozilla.org/integration/autoland/rev/34b4421d3e0d66c77f629d1b0ac1df0224ad3a07
[task 2019-08-14T02:36:12.136Z] 02:36:12 INFO - TEST-SKIP | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_captivePortal.js | took 1ms
[task 2019-08-14T02:36:12.136Z] 02:36:12 INFO - Retrying tests that failed when run in parallel.
[task 2019-08-14T02:36:12.144Z] 02:36:12 INFO - TEST-START | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_webRequest_urlclassification.js
[task 2019-08-14T02:36:13.049Z] 02:36:13 WARNING - TEST-UNEXPECTED-FAIL | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_webRequest_urlclassification.js | xpcshell return code: 0
[task 2019-08-14T02:36:13.049Z] 02:36:13 INFO - TEST-INFO took 904ms
[task 2019-08-14T02:36:13.050Z] 02:36:13 INFO - >>>>>>>
[task 2019-08-14T02:36:13.051Z] 02:36:13 INFO - PID 28704 | [28704, Main Thread] WARNING: Couldn't get the user appdata directory. Crash events may not be produced.: file /builds/worker/workspace/build/src/toolkit/crashreporter/nsExceptionHandler.cpp, line 2632
[task 2019-08-14T02:36:13.051Z] 02:36:13 INFO - PID 28704 | [28704, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, kKnownEsrVersion) failed with result 0x80004002: file /builds/worker/workspace/build/src/toolkit/components/resistfingerprinting/nsRFPService.cpp, line 662
[task 2019-08-14T02:36:13.052Z] 02:36:13 INFO - PID 28704 | [28704, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520012: file /builds/worker/workspace/build/src/extensions/permissions/nsPermissionManager.cpp, line 2901
[task 2019-08-14T02:36:13.053Z] 02:36:13 INFO - (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2019-08-14T02:36:13.053Z] 02:36:13 INFO - (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2019-08-14T02:36:13.054Z] 02:36:13 INFO - (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2019-08-14T02:36:13.054Z] 02:36:13 INFO - running event loop
[task 2019-08-14T02:36:13.055Z] 02:36:13 INFO - xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_webRequest_urlclassification.js | Starting check_remote
[task 2019-08-14T02:36:13.055Z] 02:36:13 INFO - (xpcshell/head.js) | test check_remote pending (2)
[task 2019-08-14T02:36:13.056Z] 02:36:13 INFO - TEST-PASS | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_webRequest_urlclassification.js | check_remote
Assignee | ||
Comment 14•5 years ago
|
||
Comment 16•5 years ago
|
||
Comment 17•5 years ago
|
||
bugherder |
Comment 18•5 years ago
|
||
Hello,
Will this fix require manual validation? If yes, please provide some steps to reproduce in order to correctly test it and also, please set the "qe-verify+" flag. Otherwise, could the "qe-verify-" flag be added? Thanks!
Comment 20•5 years ago
|
||
Looks like we never documented this. Who could help describe our current url classification flags?
Assignee | ||
Comment 21•5 years ago
|
||
Since it's a privileged api we shouldn't document on mdn, but we should get that documentation pulled together, perhaps on this bug at first.
Comment 22•5 years ago
|
||
With this bug, webRequest listeners receive an extra "urlClassification" attribute in the details object. This attribute is an object composed of "firstParty" and "thirdParty" properties, each one of them, arrays of classification flags. The possible flags are here:
The following is a description of those flags:
- fingerprinting or fingerprinting_content: the request has been marked as a fingerprinting resource. "fingerprinting_content" is used when the resource is loaded from an origin that has been found to fingerprint but is not considered to participate in tracking, such as a payment provider.
- cryptomining or cryptomining_content: similar to the previous category but for cryptomining resources.
- tracking, tracking_ad, tracking_analytics, tracking_social, tracking_content: for tracking resources we have these categories. "tracking" is the generic one. "ad", "analytics", "social", "content" are used to identify the type of tracker
- socialtracking, socialtracking_facebook, socialtracking_linkedin, socialtracking_twitter: similar to the previous categories but more related to social tracking. The major social networks have their own flag.
- any_basic_tracking: this is a "meta" flag that combines any tracking and fingerprinting flag, excluding tracking_content and fingerprinting_content.
- any_strict_tracking: this is another "meta" flag that combines any tracking and fingerprinting flag including tracking_content and fingerprinting_content
- any_social_tracking: yet another "meta" flag that combines any socialtracking flag.
The "meta" flags are used for tracking-protection and ETP.
Assignee | ||
Updated•5 years ago
|
Description
•