Filed as requested in bug 1427273.

Transferred from bug 1427273. We can land this patch without waiting for NSS fix. This patch will not add more breakage without NSS fix because currently non-ASCII paths do not work anyway.

Comment on attachment 8940417 [details] [diff] [review] Use UTF-8 file paths for NSS database Review of attachment 8940417 [details] [diff] [review]: ----------------------------------------------------------------- ::: security/manager/ssl/nsNSSComponent.cpp @@ +1756,5 @@ > return NS_OK; > } > > #if defined(XP_WIN) > + // SQLite always takes UTF-8 file paths regardless of the current system ... but again, NSS doesn't. We should at least document this, if not explore it further (for example, will the PKCS#11 module database still work after this? Please ensure the answer is "yes" before landing this.) Also, this is a larger discussion we should have with the NSS team. If I'm understanding correctly, this would all work properly if NSS only accepted UTF-8 paths but then converted them to unicode when doing file operations on Windows. Since it doesn't, though, things are likely to continue to not work as expected.

I assume this should be landed at the same time as uplifting a fixed NSS snapshot. We can help to do that at the same time.

Pushed by kaie@kuix.de: https://hg.mozilla.org/integration/mozilla-inbound/rev/617446a093ba Use UTF-8 file paths for NSS database. r=keeler

(In reply to David Keeler [:keeler] (use needinfo) from comment #2) > ... but again, NSS doesn't. We should at least document this, if not explore > it further (for example, will the PKCS#11 module database still work after > this? Please ensure the answer is "yes" before landing this.) > Also, this is a larger discussion we should have with the NSS team. If I'm > understanding correctly, this would all work properly if NSS only accepted > UTF-8 paths but then converted them to unicode when doing file operations on > Windows. Since it doesn't, though, things are likely to continue to not work > as expected. I changed NSS so that it consistently uses UTF-8 for "sql:" prefixed configdir. All other file paths will still use ANSI encoding.

Backout by apavel@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/6705010893cd Backed out 4 changesets (bug 1428538, bug 1420060) for mochitest mass failures UPGRADE_NSS_RELEASE on a CLOSED TREE

Backed out for mochitest mass failures Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=7604bcd50b63cc8f3e867d8844d7bab871dc0af3 Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=155649398&repo=mozilla-inbound&lineNumber=3301 Backout: https://hg.mozilla.org/integration/mozilla-inbound/rev/6705010893cd6d0a72f6adbcb81a92f052a041c2

> Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=7604bcd50b63cc8f3e867d8844d7bab871dc0af3 Isn't it cased by bug 1382251? https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=6159c8eb544245f0a3ed8766608202ee72530101 This changeset (just before this backout) did not have so much failures (oranges looks known intermittent.)

Hi, i was just about to update the other bug from that pus as both bugs were backed out. The problematic failures where here: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=7604bcd50b63cc8f3e867d8844d7bab871dc0af3 Reason why this was backed out alongside bug 1420060. Bug 1382251 was also backed out for other mochitest failures, specifically for mochitest assertion failures at /builds/worker/workspace/build/src/obj-firefox/dist/include/mozilla/RefPtr.h:318 Aryx: if you have anything to add here, please do.

The mochitest-headless failures at https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=68e0118f11a2e8a71aaff24b57384b15fa087977&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable start with those NSS patches and end after the backout. Both bugs got backed out because this one also touches UTF-8 and the follow-ups used the other bug in the commit message. If the changes for this bug don't cause any of the seen issues and don't depend on bug 1420060, please reland. Thank you.

(In reply to Sebastian Hengst [:aryx][:archaeopteryx] (needinfo on intermittent or backout) from comment #10) > The mochitest-headless failures at > https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=68e0118f11a2e8a71aaff24b57384b15fa087977&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable > start with those NSS > patches and end after the backout. Both bugs got backed out because this one > also touches UTF-8 and the follow-ups used the other bug in the commit > message. Actually mochitest-headless failures start at bug 1382251 patches at: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=cc3ec37e0dbe422223a699ff76b94cbc11b28b45 (bug 1428538 and bug 1420060 was already landed here.) and end after the backout: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=6159c8eb544245f0a3ed8766608202ee72530101 (bug 1428538 and bug 1420060 was not backed out yet here.) Two bustage fixes were about "build" failures (not about test failures). I think we can just reland bug 1428538 and bug 1420060.

Try run just in case: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7ee0791de4d3263dd951d3e4542243951bdf3002

Pushed by kaie@kuix.de: https://hg.mozilla.org/integration/mozilla-inbound/rev/95e2f1556c17 Use UTF-8 file paths for NSS database. r=keeler

Comment on attachment 8942409 [details] [diff] [review] Reression test to make sure non-ASCII file paths work Review of attachment 8942409 [details] [diff] [review]: ----------------------------------------------------------------- I'd prefer this be a separate, more specialized test rather than changing the original cert overrides test to run with this particular setup (feel free to `hg cp test_cert_overrides_read_only.js` and remove the unnecessary extra leftovers). Also, in the future I'd prefer that tests either be landed at the same time as fixes or as part of a separate bug to avoid complicating our understanding of what landed when (I realize that since someone else pushed this before you marked it "checkin-needed" in this case).

(In reply to David Keeler [:keeler] (use needinfo) from comment #16) > I'd prefer this be a separate, more specialized test rather than changing > the original cert overrides test to run with this particular setup (feel > free to `hg cp test_cert_overrides_read_only.js` and remove the unnecessary > extra leftovers). > Also, in the future I'd prefer that tests either be landed at the same time > as fixes or as part of a separate bug to avoid complicating our > understanding of what landed when (I realize that since someone else pushed > this before you marked it "checkin-needed" in this case). Filed bug 1430973. I'll attach a new patch there.