Closed Bug 1522818 Opened 6 years ago Closed 6 years ago

test_IE_cookies.js is failing on an updated version of windows (from 1709 -> 1803)

Categories

(Firefox :: Migration, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
Firefox 67
Tracking Status
firefox67 --- fixed

People

(Reporter: jmaher, Assigned: mak)

References

Details

Attachments

(1 file)

you can see here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a150b38c7f67a6bf6e65df539a391ba900c224bb&searchStr=xpcshell

this is a perma fail test- skip-if for this yields green xpcshell, here is a snippet from the debug xpcshell in the above link:
21:02:22 INFO - browser/components/migration/tests/unit/test_IE_cookies.js | Starting
21:02:22 INFO - (xpcshell/head.js) | test pending (2)
21:02:22 INFO - (xpcshell/head.js) | test run_next_test 0 finished (2)
21:02:22 INFO - TEST-PASS | browser/components/migration/tests/unit/test_IE_cookies.js | - true == true
21:02:22 INFO - TEST-PASS | browser/components/migration/tests/unit/test_IE_cookies.js | - Added a persistent IE cookie: testvalue; expires=Sat, 26 Jan 2019 21:02:21 GMT - true == true
21:02:22 INFO - TEST-PASS | browser/components/migration/tests/unit/test_IE_cookies.js | - Found the added persistent IE cookie - true == true
21:02:22 INFO - "Found cookie: testcookie=testvalue"
21:02:22 INFO - TEST-PASS | browser/components/migration/tests/unit/test_IE_cookies.js | - Found the expected cookie - "testcookie=testvalue" == "testcookie=testvalue"
21:02:22 INFO - TEST-PASS | browser/components/migration/tests/unit/test_IE_cookies.js | - There are no cookies initially - 0 == 0
21:02:22 INFO - TEST-PASS | browser/components/migration/tests/unit/test_IE_cookies.js | - Resource supported by migrator - true == true
21:02:22 INFO - PID 4212 | [4212, Main Thread] WARNING: This method is lossy. Use GetCanonicalPath !: file z:/build/build/src/xpcom/io/nsLocalFileWin.cpp, line 3263
21:02:22 INFO - PID 4212 | [4212, Main Thread] WARNING: This method is lossy. Use GetCanonicalPath !: file z:/build/build/src/xpcom/io/nsLocalFileWin.cpp, line 3263
21:02:22 INFO - PID 4212 | [4212, Main Thread] WARNING: This method is lossy. Use GetCanonicalPath !: file z:/build/build/src/xpcom/io/nsLocalFileWin.cpp, line 3263
21:02:22 INFO - PID 4212 | [4212, StreamTrans #1] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80470002: file z:/build/build/src/netwerk/base/nsFileStreams.cpp, line 77
21:02:22 WARNING - TEST-UNEXPECTED-FAIL | browser/components/migration/tests/unit/test_IE_cookies.js | - Migrated the expected number of cookies - 0 == 1
21:02:22 INFO - Z:/task_1548361637/build/tests/xpcshell/tests/browser/components/migration/tests/unit/test_IE_cookies.js:null:105
21:02:22 INFO - exiting test

We would like to get this new configuration up and running sooner rather than later, I couldn't determine if there was a quick fix by looking at the source code for the test:
https://searchfox.org/mozilla-central/source/browser/components/migration/tests/unit/test_IE_cookies.js

I would like to see if we can fix this as it is a windows only test. Maybe we need to do something on the host OS? Maybe we need to tweak firefox or the test?

Flags: needinfo?(mak77)
Blocks: 1522896

I started investigating this, and things don't look good.

Starting from Windows Fall Creators Update 1709, cookies and history are no more stored in files, but in a database named WebCacheV01.dat under AppData\Local\Microsoft\Windows\WebCache
This is probably a jetblue/ese database, but it is apparently constantly locked and I can't open it.
Single cookies can be accessed through the Internet(Get|Set)Cookie(A|W) APIs, but there isn't an API to get a list of cookies. You can only get cookies

This means our cookies migrators for Edge/IE are pretty much doing nothing. Or, even worse, the Edge migrator is migrating really old cookies created before this version, if there is still some around (clearing cookies in Edge also clears these).

I see 3 options:

  1. figure out a way to read the database, there must be one because I found a closed source util doing that
  2. give up on migrating cookies, it's likely future versions will change more and we can't keep up
  3. read the cookies through the InternetCookie API for urls imported by history and bookmarks

For the test, we could simply skip it by checking for a "deprecated.cookie" file in CookD, or check Windows version obtained from UpdateUtils.OSVersion.

(In reply to Marco Bonardo [::mak] from comment #1)

I started investigating this, and things don't look good.

Starting from Windows Fall Creators Update 1709, cookies and history are no more stored in files, but in a database named WebCacheV01.dat under AppData\Local\Microsoft\Windows\WebCache
This is probably a jetblue/ese database, but it is apparently constantly locked and I can't open it.
Single cookies can be accessed through the Internet(Get|Set)Cookie(A|W) APIs, but there isn't an API to get a list of cookies. You can only get cookies

Ugh. This sounds like https://bugzilla.mozilla.org/show_bug.cgi?id=1192034 . I couldn't find a sane way around the locking at the time, esp. without admin access.

This means our cookies migrators for Edge/IE are pretty much doing nothing. Or, even worse, the Edge migrator is migrating really old cookies created before this version, if there is still some around (clearing cookies in Edge also clears these).

I see 3 options:

  1. figure out a way to read the database, there must be one because I found a closed source util doing that
  2. give up on migrating cookies, it's likely future versions will change more and we can't keep up
  3. read the cookies through the InternetCookie API for urls imported by history and bookmarks

If Edge is really switching to Chromium, and considering fewer and fewer people are using IE, I would suggest wontfix. We can still do:

For the test, we could simply skip it by checking for a "deprecated.cookie" file in CookD, or check Windows version obtained from UpdateUtils.OSVersion.

to continue testing this on older versions, though just removing support might be easier if we can get buy-in for that from relevant product folks.

(In reply to :Gijs (he/him) from comment #2)

Ugh. This sounds like https://bugzilla.mozilla.org/show_bug.cgi?id=1192034 . I couldn't find a sane way around the locking at the time, esp. without admin access.

From what I can tell, the util I found somehow copies the database to the temp folder and accesses it from there. FWIW, I honestly don't think it's a great idea from a security point of view.

to continue testing this on older versions, though just removing support might be easier if we can get buy-in for that from relevant product folks.

Yeah, I was mostly thinking to a quick fix to unblock bug 1522896, then we could just remove this in a separate bug, with a proper discussion.

Flags: needinfo?(mak77)
Assignee: nobody → mak77
Status: NEW → ASSIGNED
Blocks: 1523681

(In reply to :Gijs (he/him) from comment #2)

If Edge is really switching to Chromium, and considering fewer and fewer people are using IE, I would suggest wontfix.

I wouldn't assume they will take Chromium as a whole-lump and just re-brand it. This kind of treatment for cookies and other migratable data seem like exactly the kind of thing they might port over as they presumably had a reason to do it this way in Edge. This perhaps doesnt change the conclusion though - if there's no practical way to fix the bug.

(In reply to Sam Foster [:sfoster] from comment #5)

(In reply to :Gijs (he/him) from comment #2)

If Edge is really switching to Chromium, and considering fewer and fewer people are using IE, I would suggest wontfix.

I wouldn't assume they will take Chromium as a whole-lump and just re-brand it.

Sure, the problem, as I see it, is that it's likely that change may again push them to move their storage, so we risk to play a Whac-A-Mole game until things stabilize.
Anyway, even if we'd find a way to do the import, that patch will be far more complicate and we couldn't uplift it.
I prefer a whole great or dead discussion to happen in bug 1523681, that I filed with that purpose.

Pushed by mak77@bonardo.net: https://hg.mozilla.org/integration/autoland/rev/f2371ea2c1c7 Skip test_IE_cookies.js when cookies are not stored as files. r=Gijs
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: