Closed
Bug 1168540
Opened 10 years ago
Closed 5 years ago
[meta] WPT referrer policy tests
Categories
(Core :: DOM: Security, defect, P3)
Core
DOM: Security
Tracking
()
RESOLVED
DUPLICATE
of bug 1409600
People
(Reporter: franziskus, Unassigned)
References
(Blocks 1 open bug)
Details
(Keywords: meta, Whiteboard: [domsecurity-backlog] )
I ran the referrer policy tests from w3c [1] and it looks mixed. On nightly I get 143 pass, 102 fail, and 236 timeout. I'm not sure what the reason for the timouts is, but the fails look legit, e.g.
assert_equals: Reported Referrer URL is 'stripped-referrer'. expected (string) "http://web-platform.test:8000/referrer-policy/no-referrer-when-downgrade/http-csp/same-origin/http-http/worker-request/insecure-protocol.http.html" but got (undefined) undefined
[1] https://github.com/w3c/web-platform-tests/tree/master/referrer-policy
Reporter | ||
Comment 1•9 years ago
|
||
Bug filed for document.origin
https://github.com/w3c/web-platform-tests/issues/1886
Reporter | ||
Comment 2•9 years ago
|
||
Reporter | ||
Updated•9 years ago
|
Reporter | ||
Updated•9 years ago
|
Whiteboard: tpe-seceng
Comment 3•9 years ago
|
||
I tried to the referrer policy tests and here's my result: 589 pass, 33 fail, and 680 timeout.
At the first look into 33 failed cases:
- Bug 1260664: 1 case fails which refers to referrerpolicy attribute. I think it should be referrerPolicy (with capital P) in the test
- Bug 1223838: currently perElementReferrer is disabled by default, therefore setting referrerpolicy attribute in element takes no effect.
- Lbnl, in these web platform tests, we change referrer policy by setting referrerpolicy content attribute.
var elementAttributesForDeliveryMethod = {
"attr-referrer": {referrerpolicy: t._scenario.referrer_policy},
"rel-noreferrer": {rel: "noreferrer"}
};
Looking at this for a while and I found it totally has no effect. I have to change it to referrerPolicy (capital P).
Is there a capitalization issue with this feature, or our implementation is missing? Am I missing any point?
[1] spec and probably [2] may refer to this issue
[1] https://w3c.github.io/webappsec-referrer-policy/#referrer-policy-delivery-referrer-attribute
[2] https://w3c.github.io/webappsec-referrer-policy/#html-iframe-element-attributes
I still don't know why there's too much timeout tests :(
Flags: needinfo?(franziskuskiefer)
Reporter | ||
Comment 4•9 years ago
|
||
(In reply to Thomas Nguyen[:tnguyen][:thomas][:nguyen] from comment #3)
> I tried to the referrer policy tests and here's my result: 589 pass, 33
> fail, and 680 timeout.
Timeouts come from certificate issues in web-platform-tests. You have to either import the certificate or manually accept it before running the tests (also see [4]).
> At the first look into 33 failed cases:
> - Bug 1260664: 1 case fails which refers to referrerpolicy attribute. I
> think it should be referrerPolicy (with capital P) in the test
> - Lbnl, in these web platform tests, we change referrer policy by setting
> referrerpolicy content attribute.
>
> var elementAttributesForDeliveryMethod = {
> "attr-referrer": {referrerpolicy: t._scenario.referrer_policy},
> "rel-noreferrer": {rel: "noreferrer"}
> };
>
> Looking at this for a while and I found it totally has no effect. I have to
> change it to referrerPolicy (capital P).
The content attribute is referrerpolicy. Only the IDL attribute is referrerPolicy (see [1][2][3], the example in [3] looks wrong).
> - Bug 1223838: currently perElementReferrer is disabled by default,
> therefore setting referrerpolicy attribute in element takes no effect.
this is probably the most important thing to do first: enable per element referrer before running the tests. otherwise a lot will obviously fail (just flip the pref manually before starting the tests).
[1] https://github.com/w3c/webappsec-referrer-policy/issues/3
[2] https://rawgit.com/w3c/webappsec-referrer-policy/master/index.html#element-interface-extensions
[3] https://rawgit.com/w3c/webappsec-referrer-policy/master/index.html#referrer-policy-delivery-referrer-attribute
[4] https://github.com/w3c/web-platform-tests/issues/1961
Flags: needinfo?(franziskuskiefer)
Comment 5•9 years ago
|
||
Thanks Frankziskus. Let me try importing certificate and run again.
Updated•9 years ago
|
Reporter | ||
Updated•9 years ago
|
Comment 6•9 years ago
|
||
Thanks Franziskus, ran again ( must enable perElementReferrer), and here's result: total 77 failed cases and 1225 passed cases
- Bug 1260664: 1 failed case in total of 77 cases fail
- Bug 1178337: 28 failed cases in total of 77 cases fail
- Bug 1261298: 38 failed cases in total of 77 cases fail
- Bug 1261003: 38 failed cases in total of 77 cases fail
( 1 failed case may contain several different reasons). I will look at all above dependency bugs before running this again.
Comment 7•9 years ago
|
||
> - Bug 1261003: 38 failed cases in total of 77 cases fail
Edit, should be: Bug 1261003: 76 failed cases in total of 77 cases fail
Comment 8•9 years ago
|
||
Hi Franziskus,
Just want to confirm it with you.
All sub-bugs for this bug were resolved. Can we mark this bug as resolved as well?
Should we be aware of anything else for this bug?
Flags: needinfo?(franziskuskiefer)
Updated•9 years ago
|
Iteration: --- → 49.3 - Jun 6
Reporter | ||
Comment 9•9 years ago
|
||
hm I thought we could close this because most things should be working. But looking at a recent m-c run [1] some tests are disabled. So I think there're two things we should look into before potentially closing this.
* img tests are not enabled yet (see discussions in bug 1265864 and related)
* there are some tests that are not enabled at the moment I think (see [1]). We should investigate why and potentially enable them.
I think we should keep this open to track progress on those things but I wouldn't consider it blocking to enabling the referrer policy attribute (we have basic coverage and know why some are disabled).
[1] https://archive.mozilla.org/pub/firefox/nightly/2016/05/2016-05-25-06-37-10-mozilla-central/mozilla-central_ubuntu32_vm_test_pgo-web-platform-tests-4-bm01-tests1-linux32-build2.txt.gz
Flags: needinfo?(franziskuskiefer)
Comment 10•9 years ago
|
||
(In reply to Franziskus Kiefer [:fkiefer or :franziskus] from comment #9)
Franziskus, thanks a lot for your insightful information!
> * img tests are not enabled yet (see discussions in bug 1265864 and related)
> * there are some tests that are not enabled at the moment I think (see [1]).
> We should investigate why and potentially enable them.
Okay. Thomas and I will keep track on this bug and img tests.
> I think we should keep this open to track progress on those things but I
> wouldn't consider it blocking to enabling the referrer policy attribute (we
> have basic coverage and know why some are disabled).
Sure. Thanks!
Comment 11•9 years ago
|
||
Thanks Ethan and Franziskus
(In reply to Franziskus Kiefer [:fkiefer or :franziskus] from comment #9)
> hm I thought we could close this because most things should be working. But
> looking at a recent m-c run [1] some tests are disabled. So I think there're
> two things we should look into before potentially closing this.
>
> * img tests are not enabled yet (see discussions in bug 1265864 and related)
> * there are some tests that are not enabled at the moment I think (see [1]).
> We should investigate why and potentially enable them.
I looked into the bug 1265864 and then Bug 1193461 and we likely have to disable img test. The reason is python 2.7.11 is not deployed on all try test machines.
I will keep my eye on the bugs, update here once test machines are upgraded.
Just want to confirm, I tried to run all referrer policy tests locally (with python 2.7.11, certainly), and all 1302 test cases passed.
> I think we should keep this open to track progress on those things but I
> wouldn't consider it blocking to enabling the referrer policy attribute (we
> have basic coverage and know why some are disabled).
Totally agree, thanks
Updated•6 years ago
|
Priority: -- → P3
Updated•6 years ago
|
Summary: W3C referrer policy tests → [meta] WPT referrer policy tests
Updated•6 years ago
|
Updated•6 years ago
|
Whiteboard: tpe-seceng → [domsecurity-backlog]
Comment 12•5 years ago
|
||
Close it and track spec in bug 1409600
Updated•5 years ago
|
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE
You need to log in
before you can comment on or make changes to this bug.
Description
•