Build: 1999110908 os: RH6.0 Unzip the file in the above URL, you'll see two mail folders we used for mail sorting testing. Put those mail folders under you POP mail directory. 1. bring up mozilla mail in C locale, go to sorttest folder which we have been using for Latin-1 testing. Click on Subject title bar, you'll see the subjects are sorted like ... Hector HTML header test Héctor .... 2. Set locale to fr and bring up mozilla mail, go to sorttest folder. This order is right in C locale. Click on Subject title bar, you'll see the subjects sorting order is as same as observed in 1. The correct order in fr locale should be like: ... Hector Héctor HTML header test ....

The sender field is not sorted correctly either.

The sorting for the subject field is not working either for Japanese locales. The Japanese sender's name is not displaying correctly in the thread pane. Due to this problem, the sender's sorting can't be tested for M11. There are inconsistency between Japanese locale names. (please see bug 18362) But in any Japanese locales, the sorting order is not expected. Steps of reproduce: After unzip the file shown in the above URL, you'll get sortjpn mail folder. put the folder under pop mail directory. repeat the operations described in step 1.

We basically calls C library function strxfrm() with setting a locale setlocale(). http://lxr.mozilla.org/seamonkey/source/intl/locale/src/unix/nsCollationUnix.cpp It will help if we can try some other unix machines to see the result of fr_FR.

Correction: Upon Naoki's explanation, the sorting order observed in ja_JP.EUC locale on Linux is correct. So below is the matrix of the results: Browser OS locale sorting 5.0 Linux ja_JP.EUC OK 5.0 Linux ja Wrong 5.0 Linux fr Wrong 5.0 Linux fr_FR Wrong 4.7 Solaris 2.6 ja OK 4.7 Solaris 2.6 japanese OK

Since the sorting order for ja_JP.EUC is correct, change the summary accordingly

The sorting order of Latin-1 data is not correct either on Solaris 4.7 using fr locale. I'll file another bug for 4.7.

Checked it again with locale changed to French at login window. And bring up mozilla in fr_FR locale. I saw the sorting order like: ... Héctor Hector HTML header test ... It looks better. But I have a question left: Why the sorting order of Japanese data is correct when I login with English and switch to ja_JP.EUC later. And I didn't get the correct result when I did sameting to French locale. By the way, there is no Japanese language option at the login window even I have Japanese packages installed.

French locale and some other European locales are supported officially by RH 6. The Japanese locale we are using is not. This accounts for the differences observed with CDE logins and other options you can set via System utility.

I put a debug printf in the build. You should see "nsCollation::Initialize mLocale=xx" (xx is a locale name used for the sorting) in terminal output. Please check tomorrow's build with "fr" and see what kind of locale is actually used.

Below is more info with Naoki's debug printf. It looks like the program picks up the locale set at the login window. If you change the locale later after login, no matter which locale you changed to, the program will pick up C locale, But "ja" locale is an exception. The program can pick up "ja" locale if you switch to it before the browser is started. This explains why the sorting order for Latin-1 data is correct when I set French locale at the login window. And it also explains why the sorting order is inconstent between "ja" and "ja_JP.EUC" locale. That's because both of these Japanese locales are not listed on the login window, but "ja" can be recorgnized later while "ja_JP.EUC" can't. For Japanese locale, the sorting order under ja_JP.EUC locale is inconsistent. Sometimes it is sorted by jis code and sometimes it is sorted in different way.

HI, Bob: This is the locale-specific sorting bug I mentioned last night.

I checked in a fix setlocale argument bug (NULL -> ""). Now, it should pick up current locale value from environment.

I checked today's build (1999111809), the fix is not in. It's still looking at C locale under ja_JP.EUC Naoki, which build should this fix be in?

Yes, the fix is in, it works with fr_FR, so the sorting of French can be tested. If it's not working with ja_JP.EUC, the bug can be reopened and we can focus on that problem.

With build 1999111808, the fix works for Latin 1 locales. For ja_JP.EUC, it's not consistent. I tried about 8 times and only once ja_JP.EUC is picked up. But even it picked up ja_JP.EUC locale, the sorting order of Japanese subjects is not correct. "ja" locale can always be picked up before and after the fix checked in, but the sorting order under "ja" is not correct either.

Clearing FIXED resolution due to reopen.

I have a fix reviewed by tao. There is a problem in the mapping functions between POSIX and XP locale (it doesn't work correctly with any locales which have .xxx like ja_JP.EUC).

I checked in the fix (to be available in tomorrow's build).

With 1999112408-M12 build, ja_JP.EUC locale can be recorgnized as the other locales. But the sorting order under ja_JP.EUC is still not correct.

There is another bug in unix locale service. Locale name to charset name mapping is not working correctly. I am going to investigate.

*** Bug 18362 has been marked as a duplicate of this bug. ***

*** Bug 18362 has been marked as a duplicate of this bug. ***

I checked in the fix for locale to charset name mapping problem. But the result of the Japanese sorting is still not correct. I think the problem is not in mozilla code but the collation key generation implementation (i.e. strxfrm). I attached the result of Japanese collation key generation. The collation keys in Solaris follows JIS order while Redhat 6.0 generates keys seems to have no ordering. Please check tomorrow's Solaris build to check Japanese sort order.

Unfortunately Solaris mozilla build doesn't launch, and we don't have commercial builds for Solaris. Can't use Solaris builds for reference at this point.

Currently, we are waiting for the availability of non Linux Unix builds to verify the problem is Linux specific. Once we make sure that, we can do Japanese collation order as JIS code point which is done for Macintosh and generates a reasonable result. I am moving the target M15.

Do any of the standard Unix commands (e.g., ls) work on RH 6.0? If so, we should take a look at the source for those commands to see what they are doing differently.

"sort" command in ja locale gives correct sorting order.

Reassign to tao for the investigation of "sort" command.

Tao, Since you're more familiar with Unix code than nhotta, we thought you might be able to help nhotta quickly look at the RH 6.0 sort code and help him figure out why nhotta's implementation of the Unix sorting APIs is not working. Talk to nhotta. Thx.

Hi, Naoki: Here is the source code of sort.c. Feel free to let me know if you need further help or the original package that containall the files to compile sort.c. Bounce it back to you. Thanks, Tao

Thanks tao for the information. We still need to test non Linux Unix platform. Then may need to change our implementation depends on the result.

QA contact --> ji

I read the code provided by tao. It uses strcoll() to compare strings. I wrote a simple sort program using strcoll() but the results are not good. I will attach the code and the result.

Any chance that this can be tested on other unix builds?

Can't get mozilla build running on Soalris 2.6. Will talk to Tajima at Sun about this.

Checked both beta1 build and 2000042408 commercial build on Japanese Red Hat 6.1. None of them has right sort order on the subject column of a mail folder.

With Japanese Red Hat 6.2, we did not get the correct sort result using the test program (03/22/00 13:54). The same program gave us the correct result using AIX with Japanese locale. So it's likely the problem is in LC_COLLATE support in Linux. I checked in an option to use charset code point for collation keys (intl.collationKeyAsCodePoint, default is "false"), so it can be enabled for a localized version. I want this bug to be reassign to a unix engineer for more investigation. Reassign to bobj.

Reassigned to nhotta who has been in contact with redhat folks on this issue.

I contacted Redhat again this week, so far no reply.

This is not a client by according to our investigation so far. No plan to change client for this. The work around is to set a pref "intl.collationKeyAsCodePoint" to true then the sort will be done by EUC-JP code point for Linux JA locale. Please reopen if this can be reproduced by other Unix.

Is this a general problem that could be avoided by customizing the pref for Ja Linux build?

Yes, it is possible to set that pref to true for JA linux build.

Added "ja" keyword I tried to add "jartm" keyword, but it is not recognized.

jartm can only be added at bugscape.

Please file a L10N bug to bugscape if we want to set the pref to Linux JA build.

And setting this pref will not cause regression in other area? I am trying to determine if we should ask Allan (l10n) to do this for JA RTM Linux release. Thx.

The code path is slitely different so the testing should be done with this pref on using Japanese Linux.

Tested 2000102612 branch build with the pref on my Ja RH6.2. The sorting for mail is working.

Bugscape bug 3077 has been filed for the l10n build.

I have requested the keyword, but Asa hasn't replied yet. Asa, can you confirm that it can be added?

The keywords can be added to Bugzilla. I'd like to change the keywords from the format "jaRTM" to "nsRTMja" or "nsrtmja" (to have the same case as the existing Bugzilla/scape "rtm" keyword). It would make sense to also change the existing L10N RTM keywords in Bugscape for consistency. While it's not directly related to your needs I'd also like to rename the current rtm keywords in Bugilla and Bugscape to nsrtm. This way all of these vendor specific keywords would be grouped together in the keyword list.

As a workaround, pref of "intl.collationOption" in the file of .../locale/en-US/navigator/navigator.properties can be used in Japanese linux build.