User Agent:
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.1.2
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:104.0) Gecko/20100101 Thunderbird/104.0

Steps to reproduce:

1. Start Thunderbird 91.
2. Create folders with names containing half-width kana characters in Thunderbird.
   For example, "半角ｶﾅ", "迷惑ﾒｰﾙ", "ﾌﾟﾛﾊﾞｲﾀﾞ" (See upper left of attached image)
3. Put some messages in that folder and make sure they show up in the message list.
4. Start Thunderbird 102 with the same profile.
5. Exit Thunderbird 102 and start it again.

Actual results:

On the Thunderbird, the folders with the half-width Kana names are in the following situation. (See bottom left of attached image)

• The same folder appears twice in the folder pane.
• Message lists in those folders are empty.

The local directory where the messages are stored has the following situation. (See bottom right of attached image)

• Original folder file (mbox) and index file (.msf) are unchanged.
• Index (.msf) files are created with file names that have corruped half-width kana parts.
• Empty directories with the same names as the index files are created.

Expected results:

There is no change in the folders named in half-width katakana, and the saved messages are displayed.

Additional Information:

• This bug has been reported to the MozillaZine.jp forum and I have reproduced and confirmed it.
• This bug is a regression by bug 1746052 and is fixed by bug 1778429.
• I have confirmed that it is fixed in Thunderbird Daily 105.0a1 (2022-08-19). But the beta and release versions are not fixed.

If https://bugzilla.mozilla.org/show_bug.cgi?id=1778429#c27 is correct, then the patch should be picked up by Thunderbird 102.2.0. You can try the candidate from https://archive.mozilla.org/pub/thunderbird/candidates/102.2.0-candidates/build2/

(In reply to Wayne Mery (:wsmwk) from comment #1)

If https://bugzilla.mozilla.org/show_bug.cgi?id=1778429#c27 is correct, then the patch should be picked up by Thunderbird 102.2.0. You can try the candidate from https://archive.mozilla.org/pub/thunderbird/candidates/102.2.0-candidates/build2/

I tried 102.2.0-candidates/build2/ but it doesn't seem to fix.

Just in case, when I checked the build fixed by mozregression, it was the following.

pushlog_url: https://hg.mozilla.org/comm-central/pushloghtml?fromchange=e17f4d5f301989c3780f31e11dfd82eec67f63a9&tochange=609f81cdf45703a328280c152e9bd17c1c3f0c0a

I guess https://hg.mozilla.org/mozilla-central/rev/e0b4c0e343efad5a3f447e33e128aae12b078d9d (bug 1778429) is key to understanding what would need to change, and why likely the rework in bug 1782347 fixed it for trunk

For release/beta, what a mess. This looks like some of it:
https://searchfox.org/comm-esr102/search?q=.ReplaceChar%28&path=&case=false&regexp=false

Or maybe it's easy enough to fix.
https://searchfox.org/comm-esr102/source/mailnews/base/src/nsMessenger.cpp#123

- aResult.ReplaceChar(FILE_PATH_SEPARATOR FILE_ILLEGAL_CHARACTERS, '-');
+ aResult.ReplaceChar(u"" FILE_PATH_SEPARATOR FILE_ILLEGAL_CHARACTERS, '-');

And check the other FILE_PATH_SEPARATOR FILE_ILLEGAL_CHARACTERS usages.

If paasing single-byte string to be compared using a utf16 string it won't work, breaks e.g. for half-width katakana.

Doesn't reproduce on linux since I think "" needs to be in there... and it's obviously not.
I think the fix should do it though. Set off a try run here: https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=2e69ffe5273eb4085c7c78aa51d71b71f6c37244

Windows test build can be downloaded from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YN-3VwjBRtG-qW0yPmYo-A/runs/0/artifacts/public/build/setup.exe

(In reply to Magnus Melin [:mkmelin] from comment #9)

Windows test build can be downloaded from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YN-3VwjBRtG-qW0yPmYo-A/runs/0/artifacts/public/build/setup.exe

I tried downloading the file from that URL, but the size of the downloaded file (setup.exe) is only 838 KB.

Sorry, should be https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YN-3VwjBRtG-qW0yPmYo-A/runs/0/artifacts/public/build/install/sea/target.installer.exe

(In reply to Magnus Melin [:mkmelin] from comment #11)

Sorry, should be https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YN-3VwjBRtG-qW0yPmYo-A/runs/0/artifacts/public/build/install/sea/target.installer.exe

I confirmed that the half-width kana name folders are not corrupted.

Great, thanks for testing!
I could not reproduce the bug on Windows either (with any version). Perhaps the windows filesystem has to be something non-western for it to trigger... My folder names were always fully hashed.

Comment on attachment 9290842 [details]
Bug 1786191 - [esr102 only] Use utf-16 character literals when calling ReplaceChar. r=benc

[Approval Request Comment]
Regression caused by (bug #): bug 1746052
User impact if declined: may get folder duplication and other related bad issues related to folders
Testing completed (on c-c, etc.): c-c got fixed through other refactoring, so this particular fix only applies to 102 and can be tested only there.
Risk to taking this patch (and alternatives if risky): in theory safe

Comment on attachment 9290842 [details]
Bug 1786191 - [esr102 only] Use utf-16 character literals when calling ReplaceChar. r=benc

[Triage Comment]
Approved for esr102

Thunderbird 102.2.1:
https://hg.mozilla.org/releases/comm-esr102/rev/edac1d393f54

Thunderbird 102.2.1:
https://hg.mozilla.org/releases/comm-esr102/rev/c0f48e08688a