Closed
Bug 1478195
Opened 6 years ago
Closed 6 years ago
Deploy the fastblock lists to prod
Categories
(Cloud Services :: Operations: Shavar, task, P1)
Cloud Services
Operations: Shavar
Tracking
(Not tracked)
VERIFIED
FIXED
People
(Reporter: francois, Assigned: ckolos)
References
Details
Now that bug 1474412 is complete and has been tested, we can go to prod:
14:11 < rbillings> francois: all tests look good
14:16 < francois> rbillings: great, so we have your OK to move to prod?
14:24 < rbillings> francois: absolutely
Assignee | ||
Comment 1•6 years ago
|
||
done
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 2•6 years ago
|
||
I used the scripts in the safebrowsing-tools repo (https://github.com/fmarier/safebrowsing-tools) to confirm that all of the new lists are advertised by the shavar server:
$ ./get-lists.py | grep fastblock
fastblock1-track-digest256
fastblock1-trackwhite-digest256
fastblock2-track-digest256
fastblock2-trackwhite-digest256
fastblock3-track-digest256
and then requested them all:
$ ./download-list.py
n:3600
i:fastblock1-trackwhite-digest256
u:shavar.services.mozilla.com/fastblock1-trackwhite-digest256/1532636914
i:fastblock2-track-digest256
u:shavar.services.mozilla.com/fastblock2-track-digest256/1532636914
i:fastblock1-track-digest256
u:shavar.services.mozilla.com/fastblock1-track-digest256/1532636914
i:fastblock3-track-digest256
u:shavar.services.mozilla.com/fastblock3-track-digest256/1532636914
i:fastblock2-trackwhite-digest256
u:shavar.services.mozilla.com/fastblock2-trackwhite-digest256/1532636914
However, when I follow these redirects, they all return a 404:
$ curl https://shavar.services.mozilla.com/fastblock1-track-digest256/1532636914
The requested page was not found.
$ curl https://shavar.services.mozilla.com/fastblock1-trackwhite-digest256/1532636914
The requested page was not found.
$ curl https://shavar.services.mozilla.com/fastblock2-track-digest256/1532636914
The requested page was not found.
$ curl https://shavar.services.mozilla.com/fastblock2-trackwhite-digest256/1532636914
The requested page was not found.
$ curl https://shavar.services.mozilla.com/fastblock3-track-digest256/1532636914
The requested page was not found.
The interesting thing is that this domain is different from when I request the tracking protection list:
$ ./download-list.py
n:3600
i:base-track-digest256
u:tracking-protection.cdn.mozilla.net/base-track-digest256/1512580265
That list works fine:
$ curl --head https://tracking-protection.cdn.mozilla.net/base-track-digest256/1512580265
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 57526
Connection: keep-alive
Date: Thu, 26 Jul 2018 13:01:41 GMT
Last-Modified: Wed, 06 Dec 2017 17:11:10 GMT
ETag: "91c9f9b9b6e2b5c7312e0df739362f85"
Accept-Ranges: bytes
Server: AmazonS3
Age: 29447
X-Cache: Hit from cloudfront
Via: 1.1 affec1a0de50d1f690b08b468f7d2156.cloudfront.net (CloudFront)
X-Amz-Cf-Id: gWlNhBSZiimBauIcftKQwtcGOnVO6iB-RnR2kSXGs7Bqx1pseQU0SQ==
and if I use the same hostname for the fastblock lists, it also seems to work:
$ curl --head https://tracking-protection.cdn.mozilla.net/fastblock1-track-digest256/1532636914
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 55830
Connection: keep-alive
Date: Mon, 30 Jul 2018 23:45:30 GMT
Last-Modified: Thu, 26 Jul 2018 20:29:01 GMT
ETag: "1128bd872f4b2685ac13ba81905bcdfd"
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 b63769e2d89c89274acd908e4bfcb9f4.cloudfront.net (CloudFront)
X-Amz-Cf-Id: uXqb45dwDKlF2m8jjT4_Z5Hz4-Z9AFYJd8gMnMWcLXBgF2-ZT33ijg==
Could it be that there's some missing CDN config?
Status: RESOLVED → REOPENED
Flags: needinfo?(ckolos)
Resolution: FIXED → ---
Assignee | ||
Comment 3•6 years ago
|
||
That was indeed the case. Since we add lists so often (note, this is sarcasm :) ), I forgot that in stage the config is different than in prod. In prod, we serve lists only from the CDN. I've changed the configs, re-started all servers and now see the following (using your tools):
./download-list.py
n:3600
i:fastblock1-trackwhite-digest256
u:tracking-protection.cdn.mozilla.net/fastblock1-trackwhite-digest256/1532636914
i:fastblock2-track-digest256
u:tracking-protection.cdn.mozilla.net/fastblock2-track-digest256/1532636914
i:fastblock1-track-digest256
u:tracking-protection.cdn.mozilla.net/fastblock1-track-digest256/1532636914
i:fastblock3-track-digest256
u:tracking-protection.cdn.mozilla.net/fastblock3-track-digest256/1532636914
i:fastblock2-trackwhite-digest256
u:tracking-protection.cdn.mozilla.net/fastblock2-trackwhite-digest256/1532636914
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Flags: needinfo?(ckolos)
Resolution: --- → FIXED
Reporter | ||
Comment 4•6 years ago
|
||
I ran my tests again:
$ ./download-list.py
n:3600
i:fastblock1-trackwhite-digest256
u:tracking-protection.cdn.mozilla.net/fastblock1-trackwhite-digest256/1532636914
i:fastblock2-track-digest256
u:tracking-protection.cdn.mozilla.net/fastblock2-track-digest256/1532636914
i:fastblock1-track-digest256
u:tracking-protection.cdn.mozilla.net/fastblock1-track-digest256/1532636914
i:fastblock3-track-digest256
u:tracking-protection.cdn.mozilla.net/fastblock3-track-digest256/1532636914
i:fastblock2-trackwhite-digest256
u:tracking-protection.cdn.mozilla.net/fastblock2-trackwhite-digest256/1532636914
$ for u in "https://tracking-protection.cdn.mozilla.net/fastblock1-trackwhite-digest256/1532636914 https://tracking-protection.cdn.mozilla.net/fastblock2-track-digest256/1532636914 https://tracking-protection.cdn.mozilla.net/fastblock1-track-digest256/1532636914 https://tracking-protection.cdn.mozilla.net/fastblock3-track-digest256/1532636914 https://tracking-protection.cdn.mozilla.net/fastblock2-trackwhite-digest256/1532636914" ; do curl --head $u ; done
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 19446
Connection: keep-alive
Date: Tue, 31 Jul 2018 19:29:23 GMT
Last-Modified: Thu, 26 Jul 2018 20:29:03 GMT
ETag: "fdb36954b5f1c8a59e53f5a414293865"
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 5b51f6e8f38342d63beb93a0db7a392b.cloudfront.net (CloudFront)
X-Amz-Cf-Id: vAg98D0jZAafczh_e5gHA1hTvrmqMxo96vArG6yvsD2nwPfpykJ0xw==
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 54614
Connection: keep-alive
Date: Tue, 31 Jul 2018 19:29:23 GMT
Last-Modified: Thu, 26 Jul 2018 20:29:04 GMT
ETag: "20c158258c9acb6f8d831b826c3c5c6c"
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 5b51f6e8f38342d63beb93a0db7a392b.cloudfront.net (CloudFront)
X-Amz-Cf-Id: NbfCS-mcWWg_PWnzSK_k_Y3p-KKJxu0TWMZ1tCndcD8164YFK7Q8tA==
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 55830
Connection: keep-alive
Date: Tue, 31 Jul 2018 19:29:24 GMT
Last-Modified: Thu, 26 Jul 2018 20:29:01 GMT
ETag: "1128bd872f4b2685ac13ba81905bcdfd"
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 5b51f6e8f38342d63beb93a0db7a392b.cloudfront.net (CloudFront)
X-Amz-Cf-Id: ED1tYTceJac3aXaZ4I-zUlVO6Py-SJVKcCUkYWSyiiT6KE5kzRMwWg==
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 57494
Connection: keep-alive
Date: Tue, 31 Jul 2018 19:29:24 GMT
Last-Modified: Thu, 26 Jul 2018 20:29:07 GMT
ETag: "3f5efa2a4c6f385cd5ebd32556cee4d3"
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 5b51f6e8f38342d63beb93a0db7a392b.cloudfront.net (CloudFront)
X-Amz-Cf-Id: 0C-CdPlTnfOxBwgGFdI9UnnKDOq8gp2LaqHWSQ5ezeQs7VOJzjqO2w==
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 2037
Connection: keep-alive
Date: Tue, 31 Jul 2018 19:29:24 GMT
Last-Modified: Thu, 26 Jul 2018 20:29:06 GMT
ETag: "369dec062c3353ed8647c9a419472062"
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Miss from cloudfront
Via: 1.1 5b51f6e8f38342d63beb93a0db7a392b.cloudfront.net (CloudFront)
X-Amz-Cf-Id: 3hpB2fu36J5a9iqd-q06yZulcYFsfKhtCVirmosMnnhBX8W_gAsyXg==
Then I created three separate profiles and tested https://mozilla.github.io/tracking-test/fastblock.html successfully in each of them.
I'll let Rebecca mark this as VERIFIED once she's had a chance to run through her tests, but from my point of view it all looks good now.
Comment 5•6 years ago
|
||
Production tests pass, new lists are returned, all 3 Fastblock list profiles work as expected.
Status: RESOLVED → VERIFIED
You need to log in
before you can comment on or make changes to this bug.
Description
•