STR:

1. Launch firefox with *new* profile
2. Open New Tab -> YouTube (for example)
3. Open Options(Preferences) -> Privacy & Security -> Cookies and Site Data
4. (make sure cache(Cached Web Content) size)
5. Open Manage Data
6. Select doubleclick.net and remove it (Remove Selected -> Save Changes -> Remove)

ER:
Only cookies of doubleclick.net is removed.

AR:
Cached Web Content is cleared.

mozregression:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=6c289d94dcca88ecfe3893633a31370119c061c7&tochange=143b9e605ff029b2772687a36ae8e727910f37d9

Thanks for the report!

Backed out changeset a75c0f1d25e5 (Bug 1532203) for xpcshell failures in test_clear_forgetAboutSite.js CLOSED TREE

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&classifiedState=unclassified&selectedJob=234106651&searchStr=eb4021aefeda76ed8fc62bfe55d0ba8829008d07&revision=a75c0f1d25e5cc5e12af608a571d6894365e6583

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=234106651&repo=autoland&lineNumber=1966

Backout: https://hg.mozilla.org/integration/autoland/rev/af8f24f10ce3fb0e5f06484d6f769569967f44d2

We're blocked on bug 1536387 to resolve the (rightly occurring) failures.

Johann, with bug 1536387 now resolved, is this patch safe to re-land? And also safe to uplift to 67 beta?

No, there are other problems, which seem to require a closer investigation. I don't have time for it this week but will try to tackle it next.

Johann, checking in to see if there is any update for this P1 bug?

This also moves SiteDataTestUtils to have it available in toolkit.

This test was not actually testing what it expected, and we need to properly solve that (clearing by hostname across OAs) in bug 1541885.

Landing now, I'd say this is upliftable...

We're making a difficult compromise here, causing bug 1541885 through this patch (for cache). But (obviously) I'd say it's the right call.

https://hg.mozilla.org/mozilla-central/rev/0ac60b9d0ffc
https://hg.mozilla.org/mozilla-central/rev/9268318dee79
https://hg.mozilla.org/mozilla-central/rev/72bccd93ed10

Comment on attachment 9050957 [details]
Bug 1532203 - Implement deleteByHost for NetworkCacheCleaner and ImageCacheCleaner. r=baku

Beta/Release Uplift Approval Request

• Feature/Bug causing the regression: Bug 1523272 + other long standing issues
• User impact if declined: Users will clear their entire cache when deleting sites in the site data manager or using Forget About this Site (bug 1301562)
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: Yes
• If yes, steps to reproduce: See comment 0, feel free to also test if this fixes bug 1301562
• List of other uplifts needed: None, bug 1536387 was uplifted already
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The code here is straightforward, well understood and introduces tests for stuff that was previously untested. Note that we chose to introduce a new regression for users with container tabs (bug 1541885) as a trade-off to fix this issue. This isn't really a "risk" (the effects are well understood), just something to be aware of.
• String changes made/needed: None

I'd like to see it verified by QA on Nightly before uplifting to beta.

I reproduced the issue on Firefox Nightly 67.0a1 (2019-03-03) under Windows 10 (x64) using the STR from Comment 0 and some additional infos from Johann.

The issue is no longer reproducible on latest Nightly 68.0a1 (2019-04-09). Tests were performed on Windows 10 (x64), macOS 10.12 and Ubuntu 18.04 (x64).

Comment on attachment 9050957 [details]
Bug 1532203 - Implement deleteByHost for NetworkCacheCleaner and ImageCacheCleaner. r=baku

Tracked 67 regression + patch with tests + fix verified on nightly. Uplift approved for 67 beta 10, thanks!

https://hg.mozilla.org/releases/mozilla-beta/rev/ac28441f8332
https://hg.mozilla.org/releases/mozilla-beta/rev/4d6bd72cdc0e
https://hg.mozilla.org/releases/mozilla-beta/rev/bf0619178aca

Excuse me.
I'm expecting this feature(Manage Data/Manage Cookies and Site Data) to remove selected(or all) cookies and storage data but not cached contents.
https://support.mozilla.org/en-US/kb/clear-cookies-and-site-data-firefox
"Clear Data" dialog shows "Cookies and Site Data" and "Cached Web Content" as different data.
"Manage Cookies and Site Data" dialog is not joined to cached contents.(no content size, no sites which has cached content but not cookies or storage data)
(the missing piece may be "Manage Cached Web Content", but it doesn't matter here)

Even if this feature is changed to remove cached contents, I cannot recognize which content will be removed.
Steps for example:

1. Clear all data (Options -> Privacy & Security -> Cookies and Site Data -> Clear Data -> Clear)
2. Go to YouTube
3. Go to about:cache -> disk -> List Cache Entries
4. Find "doubleclick.net", remember matched count and Number of entries (3, 87)
5. Open Options -> Privacy & Security -> Cookies and Site Data -> Manage Data
6. Select "doubleclick.net" -> Remove Selected -> Save Changes -> Remove
7. Go to about:cache tab and click Update
   => Number of entries is reduced to 23. Not 87-3.

Also, this may be another story though, there are two site's cookies in "Manage Cookies and Site Data" dialog after visiting https://support.mozilla.org/en-US/kb/get-started-firefox-overview-main-features
support.mozilla.org 5
mozilla.org 3
If I select "support.mozilla.org", Remove Selected, Save Changes, and Remove on confirmation dialog for "support.mozilla.org", cookies of "mozilla.org" are removed too.

(Build ID 20190409221047 now)

What am I misunderstanding?
Are there any document about this something new feature?

(In reply to atlanto from comment #21)

Excuse me.
I'm expecting this feature(Manage Data/Manage Cookies and Site Data) to remove selected(or all) cookies and storage data but not cached contents.
https://support.mozilla.org/en-US/kb/clear-cookies-and-site-data-firefox
"Clear Data" dialog shows "Cookies and Site Data" and "Cached Web Content" as different data.
"Manage Cookies and Site Data" dialog is not joined to cached contents.(no content size, no sites which has cached content but not cookies or storage data)
(the missing piece may be "Manage Cached Web Content", but it doesn't matter here)

I think clearing the cache for that specific origin is intended and helpful.

Even if this feature is changed to remove cached contents, I cannot recognize which content will be removed.
Steps for example:

1. Clear all data (Options -> Privacy & Security -> Cookies and Site Data -> Clear Data -> Clear)
2. Go to YouTube
3. Go to about:cache -> disk -> List Cache Entries
4. Find "doubleclick.net", remember matched count and Number of entries (3, 87)
5. Open Options -> Privacy & Security -> Cookies and Site Data -> Manage Data
6. Select "doubleclick.net" -> Remove Selected -> Save Changes -> Remove
7. Go to about:cache tab and click Update
   => Number of entries is reduced to 23. Not 87-3.

Sorry, I can't really figure out if there's an issue based on these numbers. It could be fine, depending on the way the cache eviction works. If we're deleting a specific entry that you think we should not be deleting, feel free to file a new bug.

Also, this may be another story though, there are two site's cookies in "Manage Cookies and Site Data" dialog after visiting https://support.mozilla.org/en-US/kb/get-started-firefox-overview-main-features
support.mozilla.org 5
mozilla.org 3
If I select "support.mozilla.org", Remove Selected, Save Changes, and Remove on confirmation dialog for "support.mozilla.org", cookies of "mozilla.org" are removed too.

This is expected, since support.mozilla.org may be able to access storage from mozilla.org under the right circumstances.

The issue is no longer reproducible on Firefox Beta 67.0b13. Tests were performed on Windows 10 (x64), macOS 10.13.6 and Ubuntu 18.04 (x64).

I do not see that this bug is fixed (Windows 10). Now it not clears entire disk cache to 0 bytes, but still much more than it should. If you have collected 1 gb cache, after clearing 1 domain with 1 cookie out of dozens domains, you lose almost all of your disk cache (e.g. keeping 2 mb out of 1000 mb). This forces me to use extensions to clear cookies for some domains and not to lose cache for other domains with it.

STR:

1. Create clean profile on nightly
2. Open mozilla.org or twitter.com or reddit.com
3. Open https://commons.wikimedia.org/
4. On wikimedia open "Random file" link many times to collect cache (e.g. 20 mb), or open few large images like https://upload.wikimedia.org/wikipedia/commons/0/05/Citizen_of_Castelo_de_Monsanto.jpg
5. Open about:preferences#privacy and delete cookies for domains from step 2.

ER:
Cache loses only related data.

AS:
About half of the cache or more purges. Sometimes you can see in about:cache that you lose https://upload.wikimedia.org/wikipedia/commons/0/05/Citizen_of_Castelo_de_Monsanto.jpg after clearing cookies for unrelated domains (like twitter/reddit).

(In reply to Qwerty from comment #24)

I do not see that this bug is fixed (Windows 10). Now it not clears entire disk cache to 0 bytes, but still much more than it should. If you have collected 1 gb cache, after clearing 1 domain with 1 cookie out of dozens domains, you lose almost all of your disk cache (e.g. keeping 2 mb out of 1000 mb). This forces me to use extensions to clear cookies for some domains and not to lose cache for other domains with it.

STR:

1. Create clean profile on nightly
2. Open mozilla.org or twitter.com or reddit.com
3. Open https://commons.wikimedia.org/
4. On wikimedia open "Random file" link many times to collect cache (e.g. 20 mb), or open few large images like https://upload.wikimedia.org/wikipedia/commons/0/05/Citizen_of_Castelo_de_Monsanto.jpg
5. Open about:preferences#privacy and delete cookies for domains from step 2.

ER:
Cache loses only related data.

AS:
About half of the cache or more purges. Sometimes you can see in about:cache that you lose https://upload.wikimedia.org/wikipedia/commons/0/05/Citizen_of_Castelo_de_Monsanto.jpg after clearing cookies for unrelated domains (like twitter/reddit).

Yeah, I can confirm, can you please file a new bug and needinfo me there?

Thank you!

Done. Bug 1591362