This problem is original posted in China linux forum. A user complained that the Chinese newsgroup names are not readable by using mozilla. The news server is 61.156.20.89. And there are some other news servers that have Chinese newsgroup names: news://news.newsfan.net (IP: 210.12.195.2) news://news.cnnb.net news://zhnews.net (IP: 210.73.88.194)

Do you know what charset is used?

Some info from Chinese linux forum: One of the news servers (61.156.20.89) that hosts Chinese newsgroup names are developed by DNEWS during 1997-1998. They used WinNT4.0, it could be gb2312. I sent out an email to support-dnews@netwinsite.com no reply yet.

The one high lighted in the screen shot looks like have 8 bytes (look at cp1252 mapping table and find the byte) 3/4 in cp1252 == 0xbe << in cp1252 == 0xab >> in cp1252 == 0xbb ^a in cp1252 == 0xaa D- in cp1252 == 0xd0 A ring in cp1252 == 0xc5 C ced in cp1252 = 0xc7 o/ in cp1252 == f8 if we treate these 8 bytes as GB18030 instead of cp1252, then it mean 4 characters (look at my GB18030 / GB2312 map and figure out it's glyph and unicode) beab bbaa d0c5 c7f8 U+7ebe U+564e U+4fe1 U+533a which make sense in Chinese. where is the code which we convert a newsgroup name into Unichar ?

sspitzer@netscape.com: where is the code which convert NNTP group name into PRUnichar ?

The Chinese newsgroup name is in GB2312 (confirm from a user in Chinese linux forum)

nsbeta1- per i18n triage

Does the server tell us what encoding the newsgroups are in? How would we even know what to do with them?

I have chinese newsgroup names working. Right now I have hardcoded the encoding to GB2312. I'm trying to decide how to get the right encoding. We can either assume the system locale, or add UI to set the encoding on a news server. Incidentally, should Chinese newsgroup items work?

Composing or replying displays corrupt characters in the To: Field of composer. I'm not sure where this comes from, but I am looking at it.

If the server doesn't tell us which encoding to use, we should make them settable by the user. Should they be per news server, per newsgroup, or both? If a server or newsgroup is set to a particular encoding, should we always then assume all posts are also in that encoding?

*** Bug 162255 has been marked as a duplicate of this bug. ***

i18n triage team: nsbeta1-

*** Bug 196747 has been marked as a duplicate of this bug. ***

I subcribe to a chinese newsgroup, It display chinese name incorrectly!

in linux version 1.4 final, this problem is still exist! mozilla mailnews can't display chinese newsgroup correctly! you can't subscribe to news.newsfan.net to test this bug!

Dear Michael Kaply, Are you still watching this bug? I think that we can add a new preference value,the default value is "UTF-8".The user can set it from the about:config or some other script file.What we need to do is just add some code to get the preference value. Please let me know your idea. Thanks. Bye,Holy

I'm thinking about a UI for this. What if mail/news prefs - server settings had a dropdown for the default character coding for a server? And then when you open the properties for a particular newsgroup, at the top of the encoding pulldown (and the default) would say "Same as server" or something like that. What do people think of this?

Firstly, I wish Chinese had not been so 'brave' to use non-ASCII names in newsgroup names. Sure, NNTP needs to be updated for better I18N (how old is RFC 1036? A couple of attempts have been made to update it, but until the last time I checked, new RFC hadn't been released) but using gb2312(EUC-CN) is .... Anyway, as for mkaply's question at hand, for mail server, it would work well. For news server, what we need is not per-server default charset but per-hierarchy default charset that would be applied _across_ news servers. (this was implemented in Emacs/Gnus in the mid-to-late 1990's and tin may also have implemented it). For instance, Chinese news servers carrying newsgroups in Chinese also carry other hierarchies in which most postings are in scripts other than Chinese. Of course, we'd not have all these options if everybody follows RFC 2047 in their mail/news header.

I have no idea how I could handle the hiearchy thing in the subscribe dialog. Basically we get a to of newsgroups from the server. Is the only issue here Chinese newsgroups? Or are there Japanese named newsgroups, etc.

No, there isn't. As far as I know, Japanese and Korean were not so 'brave' to make newsgroup names in Shift_JIS/EUC-JP/ISO-2022-JP or EUC-KR. Anyway, note that it's not only newsgroup names that is at stake. If that was the case, per-server setting could work better(because only Chinese Usenet admins. made newsgroup names under their control in GB2312 as far as I know). The problem is that we also need to set the default charset to convert properly to Unicode 'unlabelled' characters in __legacy__ encodings. That is, in alt.chinese.* hierarchy, there are lots of postings with _untagged_ headers in GB2312. Similarly, in han.*, there are a number of postings with _untagged_ headers in EUC-KR. The same is true of tw.* (Big5), de.*, fr.* (ISO-8859-1), and so forth. As for the UI, about:config (or prefs.js) could work. It's not something end-users have to configure themselves. Which character encoding is used on which hierarchy is well known and we (or localizers) can set the appropriate default values.

Sorry I realized that what i wrote is kinda off-topic depending on how you look at it. Perhaps, your idea can work with per-hierarchy (rathar static default charsets configurable only via about:config/prefs.js) provided that the latter is made to override the former.

So if I am to understand you correctly, there are certain common hierarchies where the newsgroup names are always in a particular encoding. So rather than attempting to fix this in a server way, I should see about figuring out how to tell Mozilla, all newsgroups under x.y.z are Chinese names regardless of the server...

Dear Michael and Jungshik, I think that to the end user,per-hierarchy default charset that would be applied _across_ news servers will be the best solution,but per-client default charset is a good enough solution by now :). Maybe we can implement per-hierarchy default charset by steps? Bye,Holy

news.cnnb.net uses chinese newsgroup names for EVERY newsgroup, not just a certain hierarchy. The same is true of news.newsfan.net In these cases, setting the defaults character set of the news server makes much more sense. Can you point me to a public news server that only uses Chinese newsgroup names in one of the hierarchies? Or are we talking about two different problems here?

Yeah, we sorta two different problems. That's why I wrote comment #22. So, did they translate all newsgroup names to Chinese? I lost my word. Anyway, that's only Chinese. I'd stand corrected but no other news admins in any other country does that AFAIK. In all other cases, newsgroup names are all in ASCII (hmm... I may as well not be so sure here given what Chinese did. Who knows what others do...) regardless of whether they belong to Big8/global hierarchy or regional/national hierarchies. Per-hierarchy default charset is necessary NOT for newsgroup names BUT for correctly decoding RFC-2047 violating non-tagged message headers. For instance, in han.* (Korean) hierarchy, a lot of articles are posted like this (at one time, we recommend this practice, but not any more). Subject: Korean string in EUC-KR From: Korean name in EUC-KR <....@.....> Newsgroup: han.comp.os.linux.yyyy <== note that newsgroup name is in ASCII Content-Type: text/plain; charset=EUC-KR Content-Transfer-Encoding: 8bit instead of Subject: =?EUC-KR?B?......?= From: =?EUC-KR?B?......?= <...@....> Content-Type: text/plain; charset=EUC-KR Content-Transfer-Encoding: 8bit Even worse is that some messages (although must be very rare nowadays) don't have C-T header. Per-hierarchy default charset is to cope with situations like these. What we can do is as following: - add UI for per-server default charset - For newsgroups, make per-hiearachy default charset (configurable via about:config and prefs.js) should override as long as newsgroup names are all in ASCII. For non-ASCII newsgroup names (Chinese newsgroup names in GB2312), always use per-server setting (that's the only option in that case) - For mail folders, likewise, per-folder default charset (if different from per-server setting) overrides per-server setting. per-server setting

I think it best if we pay attention to what USEFOR guidelines are on newsgroup names. There is one proposal fro internationalized newsgroups names: http://www.ietf.org/internet-drafts/draft-faerber-i18n-email-netnews-names-00.txt This draft expired in March 2003 but it would be worthwhile finding out what the next plan is. The following two documents, "Usenet Best Practice": section 7.2, and "News Article Format": section 5.5 where newsgroup name synatx is defined. http://www.landfield.com/usefor/drafts/draft-ietf-usefor-useage--1.03.txt http://www.landfield.com/usefor/drafts/draft-ietf-usefor-article-11.txt I would think that a good approach would be go a step toward using Punycode as suggested in the 1st document cited above.

Sorry for my misunderstanding of "per-hierarchy ". I think in the comment#26,the "per-hierarchy default charset" is just a default value of "per-server charset",am I right? For a localized mail/news client,if the localizer set this "per-hierarchy default charset" to "GB2312" in pref.js,then all the "per-server charset" are set to "GB2312". but when the user visiting some newsgroup server which is using "EUC-KR",now he/she can change the "per-server charset" to "EUC-KR". But I don't understand why "per-hiearachy default charset (configurable via about:config and prefs.js) should override as long as newsgroup names are all in ASCII" cause even the newsgroup names are in ASCII,after we decoding the names as "GB2312" or "EUC-KR" ,this names is still correct.

There is NO newsgroup name whose name is in EUC-KR. Koreans are not so 'brave' as Chinese to make newsgroup names in EUC-KR.(if we had wanted that, we would have used UTF-8 - or punycode as Kat pointed out - instead of EUC-KR which should be put into disuse as soon as possible) See my comment #26 as to what per-hierarch default charset is for. It's NOT for newsgroup name decoding (how can it be?) but for decoding of __untagged_ headers and message body. As it stands now, it's __impossible__ to set per-hiearachy charset if newsgroup names are non-ASCII (and non-Unicode based like GB2312). So, per-hierarchy charset would be only applied to newsgroup names in ASCII. Just out of the curiosity, how would you sync Chinese translation of newsgroup names with the original ASCII names? How about news exchanges with other news servers? Have you formed an exclusive-Chinese-server only network?

Dear Jungshik, Thanks for your reply. Actually there are some "stand alone" news servers in China,and they never sync with other servers :(. I never hosted a server like this :). But,the Chinese newsgroup name will help a lot to those people who don't know english,Do you think so? To my own opinion,I'd like to use UTF-8 at verywhere,but there are some policy defined by some government department let us use GB2312 and now GB18030. Bye,Holy

Let me clarify. I have nothing against using Chinese names for Chinese newsgroups. However, that has to be done in a proper and standard-compliant way rather than in an ad-hoc manner as was done on some Chinese news servers. Exactly for the same reason, I didn't like what 'Netpia' (a Koeran company) did with 'domain names' in Korean. Their solution was a hack at best that's not interoperable and works ONLY with MS IE. Eventually, IETF adopted a sound and backward compatibile solution (IDN) and national and international domain registrars implemented IDN rather quickly (there's not much to do on the server-side because punycode used in IDN just works with old name servers). Now the situation is reversed in terms of Korean domain name support because MS IE doesn't yet support IDN while Mozilla and Opera support them very well. As for GB 18030 vs UTF-8, please don't make PRC government (your own) silly by saying that it imposes what it does NOT. As far as I know, all PRC government asks for is to support the conversion between GB18030 and other representations. If PRC government did what you said it does, IDN (mentioned above) that uses punycode (a specially-designed way of representing Unicode characters) wouldn't have been adopted in China. However, cn domain already has a lot of domain names in Chinese registered that are stored on BIND and other name servers as punycode (NOT GB18030).

Do you know that in China There is a policy (from government obviously) that any IT products should support GB18030 or it can not be sold in China. I love my Country but not the government.

Please, don't make me repeat myself. Your understanding of your government's policy is wrong. Mozilla does support GB18030 and there's no problem with the compliance to the policy. This bug has nothing to do with that.

Nobody will sell Mozilla in China,So this never happens. and I don't think that Mozilla has any problem with this policy either. As you said ,the topic is nothing about this bug,so lets talk something about how to resolve this bug.

I added a dropdown list to Mail & Newsgroup Account Settings dialog under the newsgroup server panel. This dropdown list is defaulted to ISO-8859-1. Here you can change the charset used for the server. When subscribing to a newsgroup, they will be displayed in that charset. When newsgroups get added, the default charset that the messages will be displayed in is the defined charset for the server.

What do people think of Jessica's patch?

So, when the new drop down list is added, we have the following: 1. The Main encoding setting option (global) in the Message Display section of the Mail pref. window. 2. The new drop down list per server in Mail & Newsgroup pref settings, (which defaults to ISO-8859-1?). 3. Folder proprties dialog under which encoding choice is listed. Since there is already #3, the new drop down is mainly for discoverability? One thing we should not do is to default to ISO-8859-1. Instead, the new option should default to the global Message Display encoding in #1.

I agree with Kats if his #1 is 'intl.charset.default' configurable in Edit | Preference | Languages | Default Character Coding. I'll comment more later.

jungshik, it is actually "mailnews.view_default_charset", configurable via Edit > Preferences > Mail & Newsgroup > Languages > Display Note that all mail folder charsets are initially set to "mailnews.view_default_charset", which then you can override with per folder charsets via Folder Properties menu. This is why I mentioned that creating this new drop down list would be only adding a bit more discoverability to the folder charset menu via Edit > (Folder) Properties menu. I don't see that this menu will be doing anything more than what we already do with the folder charset drop down list.

> Edit > Preferences > Mail & Newsgroup > Languages > Display Sorry, actually the menu is: Edit > Preferences > Mail & Newsgroups > Message Display > Languages > Character Coding

I take back part of what I said in Comment #37 and Comment #39. The new menu is distinct from the folder charset menu. It sets the server charset and this may be needed to display newsgroup folder names correctly. However, I still believe the default for the new menu should be the global Message Display encoding. We need to make sure that any folders for newsgroups and mail created under a server should now listen to the server default encoding to create a new folder rather than to the global Message Display default.

I believe the patch also currently makes the folder charset for newsgroups that are on a server with a different charset default to the server charset. Are you sure about defaulting to the global mail charset? I thought chinese newsgroup servers was the exception not the rule.

> Are you sure about defaulting to the global mail charset? I think we should look at 2 factors in answering this question. 1. Will defaulting to the global mail charset cause any serious problem? The lisf of mail default charsets is not huge. It is a limited set. And all the charsets contain ASCII as a subset. Since almost all newsgroups names are in ASCII. This then would not cause a problem. 2. From user experience point of view, increasing the number of default charsets may contribute to confusion. Currently loclizers set 3 default encodings for a target language. 1) browser default, 2) mail view default, 3) mail send default. The mail global default applies to the default mail folders such as Inbox and any new folders. Taking Simplified Chinese as an example, the global mail charset will be GB 18030. But if the server default is set to ISO-8859-1 and all the folders under a server also gets set to ISO-8859-1, that would in effect nullify the effect of the global mail charset even before the user does anything. Unless I am mistaken about this, setting the server default to the mail global charset seems to the only sane solution.

Comment on attachment 137265 [details] [diff] [review] using mkaply's patch above adding a pref for server charset As Kats gave exactly the same answer as I was about to give, I'll go through the patch. Firstly, it'd be nice if you make a patch with 'diff -u8 -p' ('8' is arbitrary but the point is that giving more context would help. And '-p' is critical in making your patch readable.). >Index: mailnews/base/prefs/resources/content/am-server.xul >+ <hbox align="center" valign="middle" iscontrolcontainer="true"> >+ <label value="&serverDefaultCharset.label;" control="nntp.charset"/> >+ <menulist hidable="true" hidefor="imap,pop3,movemail" wsm_persist="true" id="nntp.charset" It might be a good idea to make this menu available for imap4/pop3 as well. Perhaps, we can deal with that issue in another bug. For that and consistency with the pref. entry name (mail.string.server......charset'), you may consider using 'mailnews.server.charset' instead of 'nntp.charset'. >+ ref="NC:DecodersRoot" datasources="rdf:charset-menu" >+ preftype="string" prefstring="mail.server.%serverkey%.charset"> >Index: mailnews/news/public/nsINntpIncomingServer.idl >+ /* what charset do we use for the newsgroup server? */ >+ attribute string charset; A bit more extensive comment describing the role of the server default charset would be helpful especially in light of Kats' comment. >Index: mailnews/news/src/nsNewsFolder.cpp >@@ -214,7 +217,34 @@ >+ if (NS_FAILED(rv)) { >+ newsgroupName.AssignWithConversion(name); |CopyASCIItoUTF16(nsDependentCString(name), newsgroupName)| would be better. (there's a patch to add CopyASCIItoUTF16() that accepts 'char *', but it's not yet landed). >@@ -548,7 +578,39 @@ > if (!*uninewsgroupname) return NS_ERROR_FAILURE; > > nsCAutoString newsgroupname; >- newsgroupname.AssignWithConversion(uninewsgroupname); >+ >+ nsCOMPtr <nsINntpIncomingServer> nntpServer; >+ rv = GetNntpServer(getter_AddRefs(nntpServer)); >+ if (NS_FAILED(rv)) return rv; >+ >+ char *dataCharset; >+ rv = nntpServer->GetCharset(&dataCharset); >+ >+ nsCOMPtr<nsICharsetConverterManager> ccm = >+ do_GetService("@mozilla.org/charset-converter-manager;1", &rv); >+ if (NS_SUCCEEDED(rv)) { >+ nsCOMPtr<nsIUnicodeEncoder> encoder; >+ rv = ccm->GetUnicodeEncoder(dataCharset, getter_AddRefs(encoder)); ..... snipped conversion ..... >+ >+ if (NS_FAILED(rv)) { >+ newsgroupname.AssignWithConversion(uninewsgroupname); >+ } > > nsFileSpec path; > nsCOMPtr<nsIFileSpec> pathSpec; newsgroupname obtained here(nsMsgNewsFolder::CreateSubfolder) is used to create a filename on the local file system. Therefore, you should not convert to the server default charset but have to convert to the file system encoding. There's an utility function somewhere in mailnews for that. This is a bit debatable, but I guess that's the best we can do. >@@ -659,8 +723,9 @@ > rv = GetNntpServer(getter_AddRefs(nntpServer)); > if (NS_FAILED(rv)) return rv; > >- PRBool abbreviate = PR_TRUE; >- rv = nntpServer->GetAbbreviate(&abbreviate); >+// PRBool abbreviate = PR_TRUE; >+ PRBool abbreviate = PR_FALSE; //@MAK >+// rv = nntpServer->GetAbbreviate(&abbreviate); > if (NS_FAILED(rv)) return rv; Any reason to turn off the abbreviation here? We're working with PRUnichar[] so that I don't see any risk of chopping names in the middle (well, if we consider Unicode combining character sequence and surrogate pairs, we have to fix nsMsgNewsFolder::AbbreviatePrettyName, but at the moment, there's no need). >@@ -1595,7 +1660,39 @@ GetAsciiName() > > // convert to ASCII > nsCAutoString tmpStr; >- tmpStr.AssignWithConversion(name); <conversion code snipped> > >+ } >+ >+ if (NS_FAILED(rv)) { >+ tmpStr.AssignWithConversion(name.get()); >+ } GetAsciiName() is invoked in places (I was just surprised to know that there's a method whose only role is to convert an nsCString member variable to |char *|) like the following: http://lxr.mozilla.org/seamonkey/source/mailnews/news/src/nsNNTPProtocol.cpp#20 04 2009 rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); 2010 NS_ENSURE_SUCCESS(rv,rv); 2011 } 2012 2013 PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) current group = %s, desired group = %s", this, m_currentGroup.get(), newsgroupName.get())); 2014 // if the current group is the desired group, we can just issue the ARTICLE command 2015 // if not, we have to do a GROUP first 2016 if (!PL_strcmp(m_currentGroup.get(), newsgroupName.get())) 2017 m_nextState = NNTP_SEND_ARTICLE_NUMBER; If you change the way |m_currentGroup| (of which class? I have to check) is stored, you can avoid the conversion here. It's quite a lot more work, but I think we must do it while we're at this. Besides, GetAsciiName() is a misnomer with your change, isn't it? >Index: mailnews/news/src/nsNntpIncomingServer.cpp >+struct GetNewsgroupData { >+ nsIOFileStream* mHostInfoStream; >+ nsINntpIncomingServer* nntpServer; >+}; Wouldn'it better to use a noun instead of a 'verb' here? >@@ -801,7 +811,34 @@ writeGroupToHostInfoFile > >+ char *dataCharset; >+ rv = data->nntpServer->GetCharset(&dataCharset); .... <conversion snipped> >+ if (NS_FAILED(rv)) { >+ nameOnDisk.AssignWithConversion(name.get()); >+ } Here, too, I doubt converting to the server charset is the 'right thing'. In this case, I guess you have to convert to UTF-8. >@@ -856,8 +893,21 @@ > << "begingroups" << MSG_LINEBREAK; > > // XXX todo, sort groups first? >+ nsCOMPtr<nsIMsgFolder> rootFolder; >+ rv = GetRootFolder(getter_AddRefs(rootFolder)); >+ NS_ENSURE_SUCCESS(rv,rv); >+ if (!rootFolder) return NS_ERROR_FAILURE; >+ >+ nsCOMPtr <nsIMsgNewsFolder> newsFolder = do_QueryInterface(rootFolder, &rv); >+ NS_ENSURE_SUCCESS(rv,rv); >+ if (!newsFolder) return NS_ERROR_FAILURE; You don't have to check both conditions. NS_ENSURE_SUCCESS() would suffice. >+ nsCOMPtr <nsINntpIncomingServer> nntpServer; >+ rv = newsFolder->GetNntpServer(getter_AddRefs(nntpServer)); >+ if (NS_FAILED(rv)) return rv; The consistency wouldn't be bad here. Pls, use NS_ENSURE_SUCCESS(rv, rv). >@@ -1123,9 +1173,51 @@ >+ rv = decoder->Convert(aName, &inLength, outString, &outLength); >+ if (NS_SUCCEEDED(rv)) { >+ outString[outLength] = 0; >+ escapedName = nsEscape(NS_ConvertUCS2toUTF8(outString).get(), url_Path); NS_ConvertUCS2toUTF8 is now NS_ConvertUTF16toUTF8. Please, use that name. The same is true of all other similar utility methods/functions/classes in xpcom/string. >+ if (NS_FAILED(rv)) { >+ newsgroupName.AssignWithConversion(aName); >+ escapedName = nsEscape(NS_ConvertUCS2toUTF8(newsgroupName.get()).get(), url_Path); I'm not sure if this round-tripping is necessary. Why not just pass |aName| to nsEscape? >Index: modules/libpref/src/init/mailnews.js > >+pref("mail.server.default.charset", "ISO-8859-1"); Per Kats' comment, we don't need this entry.

can we use the method nsMsgI18NConvertFromUnicode and nsMsgI18NConvertToUnicode in nsMsgI18n.cpp instead for the conversion?

based on the previous patch ,but used nsMsgI18NConvertFromUnicode/ nsMsgI18NConvertToUnicode instead for the conversion. and get the default charset from the mailnews.view_default_charset preference.

Hi Jungshik Shin, Could you please give some comments about the new patch? Thank you very much! Best Regards,Holy

Dear all, I have one question about this bug: Should we change the type of asciiName in interface nsIMsgNewsFolder to a AString instead of a string (rename it to name or something else)?

Merry Christmas! this patch based on the previous patch. fixed the newsgroup name in the news download dialog. fixed when the user click the message but not able to view the message content when subscribe to a Chinese news group. could anyone give it a review? I hope it can land to the trunk before 1.8 release . Many people complain to me that they don't use mozilla mail client because it cannot support chinese newsgroup.Now I am not only a localizer but also a bug fixer,haha :) Best Regards,Holy

Comment on attachment 169507 [details] [diff] [review] a patch with more fixes. >diff -U 8 -r Please, use 'diff -u8 -p' >+++ mozilla/mailnews/news/public/nsIMsgNewsFolder.idl 2004-12-22 >+ readonly attribute AString newsgroupName; > readonly attribute string asciiName; Why do you still need 'asciiName'? >- newsgroupName.AssignWithConversion(name); >+ nsXPIDLCString dataCharset; >+ rv = nntpServer->GetCharset(getter_Copies(dataCharset)); You can avoid getter_Copies if you declare 'charset' as ACString instead of 'string' >+ rv = nsMsgI18NConvertFromUnicode(dataCharset, nsDependentString(uninewsgroupname), newsgroupname); >+ if (NS_FAILED(rv)) { >+ newsgroupname.AssignWithConversion(uninewsgroupname); I've already told you that you MUST use the file system charset instead of the server charset when you deal with local files. Please, use NS_CopyUnicodeToNative or equivalent. In this particular case, refer to http://lxr.mozilla.org/seamonkey/source/mailnews/local/src/nsLocalMailFolder.cp p#780 and do a similar thing with 'hashedNames'. There's one more place with the same problem. >+#include "nsIPref.h" nsIPref has been obsolete for almost 4years. Please, use nsIPrefBranch/nsIPrefService. >+ rv = nsMsgI18NConvertToUnicode(NS_LITERAL_CSTRING("UTF-8"),nsDependentCString(unescapedPath),newsgroupName); >+ NS_ENSURE_SUCCESS(rv,rv); You can simply use CopyUTF8toUTF16 here. Also, don't use old names like '..CopyUCS2toUTF8/ASCII' but use '..CopyUTF16toUTF8/ASCII'.

Hi Jungshik Shin, I've tried with diff -u8 -p but it's not work. holywen@holy-note mozsrc $ diff -u8 -p mailnews.old/ mozilla/mailnews/ > newgroupname.testclean.patch diff: `-8' option is obsolete; use `-U 8' diff: Try `diff --help' for more information. >+ rv = nsMsgI18NConvertFromUnicode(dataCharset, nsDependentString(uninewsgroupname), newsgroupname); >+ if (NS_FAILED(rv)) { >+ newsgroupname.AssignWithConversion(uninewsgroupname); Here I also tried to use the filesystem charset here,but it does not work. I kept asciiName in the interface because so many places they calling the getAsciiName.

(In reply to comment #51) > holywen@holy-note mozsrc $ diff -u8 -p mailnews.old/ mozilla/mailnews/ > diff: `-8' option is obsolete; use `-U 8' > diff: Try `diff --help' for more information. My point is that you should use '-p'. Btw, why don't you use cvs? > >+ rv = nsMsgI18NConvertFromUnicode(dataCharset, > nsDependentString(uninewsgroupname), newsgroupname); > >+ if (NS_FAILED(rv)) { > >+ newsgroupname.AssignWithConversion(uninewsgroupname); > Here I also tried to use the filesystem charset here,but it does not work. What did you mean by not working? Did you try NS_CopyUnicodeToNative? It's declaredin nsNativeCharsetUtils.h > I kept asciiName in the interface because so many places they calling the > getAsciiName. All right (even then, changing names would be a good idea), but I don't feel easy about using 'gb18030' in places like http://lxr.mozilla.org/seamonkey/source/mailnews/news/src/nsNewsFolder.cpp#521

I cannot use cvs diff because I cannot access directly to the cvs server (I am inside a intranet). There is a method nsMsgI18NFileSystemCharset in nsMsgI18N.cpp,I tried to use the return value of this method instead of the dataCharset,then the newsgroup names in the left tree becomes to unknown chars,and when I click an newsgroup it says like "the newsgroup doesnot exists in the server,do you want to unsubscribe?"

please, whatever you do, do not introduce more AssignWithConversion callers. it is deprecated. Use AssignASCII/CopyASCIItoUTF16 if you must.

(In reply to comment #53) > There is a method nsMsgI18NFileSystemCharset in nsMsgI18N.cpp,I tried to You shouldn't use that to convert a newsgroup name stored in UTF-16 to GB18030 to communicate with a news server. You only need to use NS_CopyUnicodeToNative (the opposite way) when making up a file name to use on the disk.

(In reply to comment #53) > There is a method nsMsgI18NFileSystemCharset in nsMsgI18N.cpp,I tried to You shouldn't use that to convert a newsgroup name stored in UTF-16 to GB18030 to communicate with a news server. You only need to use NS_CopyUnicodeToNative (the opposite way) when making up a file name to use on the disk.

(In reply to comment #56) > (In reply to comment #53) > > > There is a method nsMsgI18NFileSystemCharset in nsMsgI18N.cpp,I tried to > > You shouldn't use that to convert a newsgroup name stored in UTF-16 to GB18030 > to communicate with a news server. You only need to use NS_CopyUnicodeToNative > (the opposite way) when making up a file name to use on the disk. > I'm not use this as charset everywhere,just in the createsubfolder function and it give me those error messages. Now I know why,because it calles AddNewsgroup(...) after the conversion. Actually I set my locale to zh_CN.utf-8 and the file names in my mozilla profile are correct. for the GetAsciiName in the GetFolderURL(),I think that we should not call getAsciiName here ,but ,should we use escaped name or we just use utf8 string in the url?

use escaped name for the URL in method GetFolderURL . use LossyCopyUTF16toASCII instead of assignWithConversion. use nsIPrefBranch Instead of nsIPref . use ACString for charset. use -p when diff. Thank Shin very much!

I read through the nsMsgNewsFolder::CreateSubfolder(...) and found that. the nsFileSpec path is initiated but never used after: path += hashedName.get(); I tried to assign different encoded newsgroupname for the hashedName and do subscribe&unsubscribe a Chinese group and check the creation of <groupname>.dat and <groupname>.msf file,it doesn't make any sense. Is it just a piece of code that do nothing? can we remove it?

replacement of the previous patch for the trunk. the previous patch is for mozilla 1.7.3

Comment on attachment 170025 [details] [diff] [review] patch for the trunk(1.8a6). > +++ mozilla/mailnews/news/public/nsIMsgNewsFolder.idl 2005-01-02 05:18:12.375209792 +0800 > - /** the ascii name for the newsgroup. > - * newsgroup names are always ascii, and this can help simply > - * the code, to prevent use from having to get the name (as a wstring) > - * and then converting it back to ascii. > - */ > + readonly attribute AString newsgroupName; > readonly attribute string asciiName; You'd better document the difference between 'newsgroupName' and 'asciiName'. 'asciiName' is not ASCII-only any more. 'unicodeName' may be better than 'newsgroupName'. > [scriptable, uuid(b5037f06-1dd1-11b2-bf61-fcab5b5837a7)] When you change an interface, you have to change the UUID. use 'uuidgen' to generate a new uuid. > +++ mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.h 2005-01-02 04:15:48.507364216 +0800 > +++ mozilla/mailnews/news/src/nsNewsFolder.cpp 2005-01-02 05:17:54.805880736 +0800 > + nsXPIDLCString folderAsciiName; You don't seem to use this variable anywhere. > + rv = nsMsgI18NConvertToUnicode(dataCharset.get(),nsDependentCString(name),newsgroupName); rv = ConvertToUnicode(dataCharset.get(), name, newsgroupName); > + if (NS_FAILED(rv)) return rv; To be consistent, it's better to use one or the other. NS_ENSURE_SUCCESS(rv,rv); > + nsCAutoString dataCharset; > + rv = nntpServer->GetCharset(dataCharset); > + NS_ENSURE_SUCCESS(rv,rv); > + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), nsDependentString(uninewsgroupname), newsgroupname); Now I understand why you're doing this. You're using GB2312 newsgroup names for newsrc/hostinfo files. I think it's not so good an idea. Using UTF-8 is better, which require some more changes elsewhere. > + NS_CopyUnicodeToNative(nsDependentString(uninewsgroupname),filename); > + nsCAutoString hashedName = filename; > rv = NS_MsgHashIfNecessary(hashedName); > path += hashedName.get(); You have to do this way (see comment #50) nsAutoString hashedName = uniNewsgroupName; rv = NS_MsgHashIfNecessary(hashedName); nsAutoString hashedName = uniNewsgroupName; rv = NS_MsgHashIfNecessary(hashedName); NS_CopyUnicodeToNative(hashedName, fileName); path += fileName; > +NS_IMETHODIMP > +nsMsgNewsFolder::GetNewsgroupName(nsAString & name) > +{ > + nsresult rv; > + > + nsXPIDLString newsgroupName; > + rv = GetName(getter_Copies(newsgroupName)); > + NS_ENSURE_SUCCESS(rv,rv); > + name = newsgroupName; > + return NS_OK; > +} Why do we need both |GetNewsgroupName| and |GetName|? > +++ mozilla/mailnews/news/src/nsNntpIncomingServer.cpp 2005-01-02 05:17:23.239679528 +0800 > +nsNntpIncomingServer::SetCharset(const nsACString & aCharset) > +{ > + nsresult rv; > + rv = SetCharValue("charset",ToNewCString(aCharset)); You're leaking here. rv = SetCharValue("charset", PromiseFlatCString(aCharset).get()); You don't have to check both. > + if(!serverCharset || serverCharset.IsEmpty()) if(serverCharset.IsEmpty()) > + nsCOMPtr<nsIPrefBranch> prefBranch(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv)); > + > + if (NS_SUCCEEDED(rv)) > + { > + rv = prefBranch->GetCharPref(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, getter_Copies(defaultCharset)); > + serverCharset.Assign(defaultCharset ? defaultCharset.get() : NS_LITERAL_CSTRING("ISO-8859-1").get()); Can you just use |GetCharValue()|? > + }else{ > + serverCharset.Assign(NS_LITERAL_CSTRING("ISO-8859-1").get()); serverCharset = "ISO-8859-1"; > + aCharset.Assign(serverCharset.get()); aCharset = serverCharset; As mentioned above, if you use UTF-8 instead of the 'server encoding' (GB2312/GB18030), you can simplify code here. > +struct NewsgroupData { > + nsIOFileStream* mHostInfoStream; > + nsINntpIncomingServer* nntpServer; > +}; > + > PRBool > writeGroupToHostInfoFile(nsCString &aElement, void *aData) > { > + NewsgroupData *data = (NewsgroupData*)aData; > nsIOFileStream *stream; > - stream = (nsIOFileStream *)aData; > + stream = data->mHostInfoStream; .... > + nsCAutoString dataCharset; > + rv = data->nntpServer->GetCharset(dataCharset); > + NS_ENSURE_SUCCESS(rv,rv); > + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), nsDependentString(name.get()), nameOnDisk); .... > // XXX todo ",,1,0,0" is a temporary hack, fix it > *stream << nameOnDisk.get() << ",,1,0,0" << MSG_LINEBREAK; > return PR_TRUE; + rv = GetRootFolder(getter_AddRefs(rootFolder)); + NS_ENSURE_SUCCESS(rv,rv); + if (!rootFolder) return NS_ERROR_FAILURE; + nsCOMPtr <nsIMsgNewsFolder> newsFolder = do_QueryInterface(rootFolder, &rv); + NS_ENSURE_SUCCESS(rv,rv); + if (!newsFolder) return NS_ERROR_FAILURE; Just |NS_ENSURE_SUCCESS(rv,rv)| suffices. |if (!...) return NS_ERROR....| is redudant.

(In reply to comment #61) Thank you very much! > You'd better document the difference between 'newsgroupName' and 'asciiName'. > 'asciiName' is not ASCII-only any more. 'unicodeName' may be better > than 'newsgroupName'. > How about change asciiName to encodedName? Then we need to change all the GetAsciiName to GetEncodedName in the nsNNTPProtocol.cpp and nsNewsFolder.cpp > You have to do this way (see comment #50) > > nsAutoString hashedName = uniNewsgroupName; > rv = NS_MsgHashIfNecessary(hashedName); > nsAutoString hashedName = uniNewsgroupName; > rv = NS_MsgHashIfNecessary(hashedName); > NS_CopyUnicodeToNative(hashedName, fileName); > path += fileName; > As the comment#59,the path is not used later,should we remove it? > > > +NS_IMETHODIMP > > +nsMsgNewsFolder::GetNewsgroupName(nsAString & name) > > +{ > > + nsresult rv; > > + > > + nsXPIDLString newsgroupName; > > + rv = GetName(getter_Copies(newsgroupName)); > > + NS_ENSURE_SUCCESS(rv,rv); > > + name = newsgroupName; > > + return NS_OK; > > +} > > Why do we need both |GetNewsgroupName| and |GetName|? in the nsNNTPNewsgroupList.cpp,it need to get the newsgroup name from the interface nsIMsgNewsFolder but not from the class nsNewsFolder,So we need to add a method to the interface for get the newsgroup name. > > > + nsCOMPtr<nsIPrefBranch> prefBranch(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv)); > > + > > + if (NS_SUCCEEDED(rv)) > > + { > > + rv = prefBranch->GetCharPref(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, getter_Copies(defaultCharset)); > > + serverCharset.Assign(defaultCharset ? defaultCharset.get() : NS_LITERAL_CSTRING("ISO-8859-1").get()); > > Can you just use |GetCharValue()|? GetCharValue is used to get value of mail.server.%serverkey%.prefname. So we cannot use it here.

use utf8 in the newsrc/hostinfo file. (actually If we simplify the code in writing to the newsrc/hostinfo file.we need to add some code to convert the names to encoded string when loading the two file) change newsgroupName and asciiName to unicodeName and encodedName in nsIMsgNewsFolder interface. modified other places according to the new comments.

Comment on attachment 170507 [details] [diff] [review] patch use utf8 in the newsrc/hostinfo file Please, read my comment more carefully. You didn't take care of unnecessary uses of '.get()' methods in a few places. Here's an example. + { + rv = prefBranch->GetCharPref(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, getter_Copies(defaultCharset)); + serverCharset.Assign(defaultCharset ? defaultCharset.get() : NS_LITERAL_CSTRING("ISO-8859-1").get()); Btw, I don't like 'encodedName'. Perhaps, 'nativeName' would be better. I'll get to the rest of the patch later.

(In reply to comment #65) > (From update of attachment 170507 [details] [diff] [review] [edit]) > Please, read my comment more carefully. You didn't take care of unnecessary > uses of '.get()' methods in a few places. Here's an example. > > + { > + rv = prefBranch->GetCharPref(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, > getter_Copies(defaultCharset)); > + serverCharset.Assign(defaultCharset ? defaultCharset.get() : > NS_LITERAL_CSTRING("ISO-8859-1").get()); > these piece of code I just copy from the mozilla\mailnews\import\eudora\src\nsEudoraCompose.cpp#628 > Btw, I don't like 'encodedName'. Perhaps, 'nativeName' would be better. > > I'll get to the rest of the patch later. > OK,I will change the encodedName to nativeName,but a nativeName should be a filesystem encoded name?

(In reply to comment #66) > (In reply to comment #65) > > Please, read my comment more carefully. You didn't take care of unnecessary > > uses of '.get()' methods in a few places. Here's an example. ... > these piece of code I just copy from the > mozilla\mailnews\import\eudora\src\nsEudoraCompose.cpp#628 That doesn't justify your unnecessary use of 'get()'. Instead, it means that nsEudoraCompose.cpp needs to be fixed as well.

(In reply to comment #66) > OK,I will change the encodedName to nativeName,but a nativeName should be a > filesystem encoded name? Hmm. yeah... That method belongs to |nsIMsgNewsFolder| so that one may ask 'native to what?' How about 'serversideName' ? This makes me even less happy about adding this to support ugly hacks of some Chinese news servers.

> Hmm. yeah... That method belongs to |nsIMsgNewsFolder| so that one may ask > 'native to what?' > How about 'serversideName' ? > > This makes me even less happy about adding this to support ugly hacks of some > Chinese news servers. > What about rawName? about Chinese newsgroup names,my opinion is that the news protocol should be upgraded to supply not only chinese newsgroup names but also other names.

*** Bug 272753 has been marked as a duplicate of this bug. ***

let's try and get this for 1.1.

Is there anything I can help in the next step?

I'm sorry for delay. Can you see if your patch hasn't bit-rotten? If it has, please update it and I'll review it over the weekend. (btw, 'rawName' might be a reasonable choice)

Comment on attachment 170507 [details] [diff] [review] patch use utf8 in the newsrc/hostinfo file diff -U 8 -p -r mailnews.old/base/prefs/resources/content/am-server.xul mozilla/mailnews/base/prefs/resources/content/am-server.xul --- mailnews.old/base/prefs/resources/content/am-server.xul 2005-03-04 02:08:45.000000000 +0800 +++ mozilla/mailnews/base/prefs/resources/content/am-server.xul 2005-04-13 13:20:05.000000000 +0800 @@ -47,16 +47,20 @@ <page xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul" onload="parent.onPanelLoaded('am-server.xul');"> <script type="application/x-javascript" src="chrome://messenger/content/AccountManager.js"/> <script type="application/x-javascript" src="chrome://messenger/content/am-server.js"/> <script type="application/x-javascript" src="chrome://messenger/content/am-prefs.js"/> <script type="application/x-javascript" src="chrome://messenger/content/amUtils.js"/> + <script type="application/x-javascript"><![CDATA[ + var observerService = Components.classes["@mozilla.org/observer-service;1"].getService(Components.int erfaces.nsIObserverService); + observerService.notifyObservers(null, "charsetmenu-selected", "other"); + ]]></script> <stringbundle id="bundle_messenger" src="chrome://messenger/locale/messenger.properties"/> <broadcaster id="broadcaster_doBiff"/> <broadcaster id="broadcaster_notify"/> <label hidden="true" wsm_persist="true" id="server.type"/> <dialogheader title="&serverSettings.label;"/> @@ -325,9 +329,22 @@ accesskey="&localPath.accesskey;"/> <hbox align="center"> <textbox readonly="true" wsm_persist="true" flex="1" id="server.localPath" datatype="nsIFileSpec" prefstring="mail.server.%serverkey%.directory" class="uri-element"/> <button id="browseForLocalFolder" label="&browseFolder.label;" filepickertitle="&localFolderPicker.label;" accesskey="&browseFolder.accesskey;" oncommand="BrowseForLocalFolders()"/> </hbox> </vbox> + <separator class="thin"/> + <hbox hidefor="imap,pop3,movemail" align="center" valign="middle" iscontrolcontainer="true"> + <label value="&serverDefaultCharset.label;" control="nntp.charset"/> + <menulist hidable="true" hidefor="imap,pop3,movemail" wsm_persist="true" id="nntp.charset" + ref="NC:DecodersRoot" datasources="rdf:charset-menu" + preftype="string" prefstring="mail.server.%serverkey%.charset"> + <template> + <menupopup> + <menuitem label="rdf:http://home.netscape.com/NC-rdf#Name" value="..." uri="..."/> + </menupopup> + </template> + </menulist> + </hbox> </page> Only in mozilla/mailnews/base/prefs/resources/content: am-server.xul~ diff -U 8 -p -r mailnews.old/base/prefs/resources/locale/en-US/am-server-top.dtd mozilla/mailnews/base/prefs/resources/locale/en-US/am-server-top.dtd --- mailnews.old/base/prefs/resources/locale/en-US/am-server-top.dtd 2005-02-09 01:01:47.000000000 +0800 +++ mozilla/mailnews/base/prefs/resources/locale/en-US/am-server-top.dtd 2005-04-13 13:19:30.000000000 +0800 @@ -79,15 +79,16 @@ <!ENTITY newsrcFilePath.label "newsrc file:"> <!ENTITY newsrcFilePath.accesskey "f"> <!ENTITY localPath.label "Local directory:"> <!ENTITY localPath.accesskey "L"> <!ENTITY abbreviate.label "Show newsgroup names in the Mail Folder pane as:"> <!ENTITY abbreviateOn.label "Full names (For example, 'netscape.public.mozilla.mail-news')"> <!ENTITY abbreviateOff.label "Abbreviate names (For example, 'n.p.m.mail-news')"> <!ENTITY advancedButton.label "Advanced..."> +<!ENTITY serverDefaultCharset.label "Default Character Encoding:"> <!ENTITY advancedButton.accesskey "v"> <!ENTITY localFolderPicker.label "Select Local Directory"> <!ENTITY browseFolder.label "Browse..."> <!ENTITY browseFolder.accesskey "B"> <!ENTITY newsrcPicker.label "Select newsrc File"> <!ENTITY browseNewsrc.label "Browse..."> <!ENTITY browseNewsrc.accesskey "e"> diff -U 8 -p -r mailnews.old/news/public/nsIMsgNewsFolder.idl mozilla/mailnews/news/public/nsIMsgNewsFolder.idl --- mailnews.old/news/public/nsIMsgNewsFolder.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsIMsgNewsFolder.idl 2005-04-19 03:37:37.000000000 +0800 @@ -36,27 +36,28 @@ * ***** END LICENSE BLOCK ***** */ #include "nsISupports.idl" #include "nsIMsgFolder.idl" interface nsIMsgWindow; interface nsINntpIncomingServer; -[scriptable, uuid(3716abe4-f6d4-11d2-86d5-004005263078)] +[scriptable, uuid(02699533-7ba8-4e78-94dc-a20374fb2711)] interface nsIMsgNewsFolder : nsISupports { attribute string groupUsername; attribute string groupPassword; - /** the ascii name for the newsgroup. - * newsgroup names are always ascii, and this can help simply - * the code, to prevent use from having to get the name (as a wstring) - * and then converting it back to ascii. + readonly attribute AString unicodeName; + /** the raw name name for the newsgroup (server side name). + * newsgroup names are might not always ascii(bug#126453), and this raw name name + * can help simply the code, to prevent use from having to get the unicode name + * (as a wstring) and then converting it back to server side raw name. */ - readonly attribute string asciiName; + readonly attribute string rawName; readonly attribute nsINntpIncomingServer nntpServer; attribute boolean saveArticleOffline; string getGroupPasswordWithUI(in wstring aPromptString, in wstring aPromptTitle, in nsIMsgWindow aMsgWindow); string getGroupUsernameWithUI(in wstring aPromptString, in wstring aPromptTitle, in nsIMsgWindow aMsgWindow); void forgetGroupUsername(); void forgetGroupPassword(); Only in mozilla/mailnews/news/public: nsIMsgNewsFolder.idl~ diff -U 8 -p -r mailnews.old/news/public/nsINewsDownloadDialogArgs.idl mozilla/mailnews/news/public/nsINewsDownloadDialogArgs.idl --- mailnews.old/news/public/nsINewsDownloadDialogArgs.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsINewsDownloadDialogArgs.idl 2005-04-13 13:22:00.000000000 +0800 @@ -33,19 +33,19 @@ * and other provisions required by the GPL or the LGPL. If you do not delete * the provisions above, a recipient may use your version of this file under * the terms of any one of the MPL, the GPL or the LGPL. * * ***** END LICENSE BLOCK ***** */ #include "nsISupports.idl" -[scriptable, uuid(b5037f06-1dd1-11b2-bf61-fcab5b5837a7)] +[scriptable, uuid(3634327c-392b-4686-adf5-576e6cef9196)] interface nsINewsDownloadDialogArgs: nsISupports { - attribute string groupName; + attribute AString groupName; attribute long articleCount; attribute string serverKey; attribute boolean hitOK; attribute boolean downloadAll; }; %{ C++ #define DOWNLOAD_HEADERS_URL "chrome://messenger/content/downloadheaders.xul" diff -U 8 -p -r mailnews.old/news/public/nsINntpIncomingServer.idl mozilla/mailnews/news/public/nsINntpIncomingServer.idl --- mailnews.old/news/public/nsINntpIncomingServer.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsINntpIncomingServer.idl 2005-04-13 13:22:54.000000000 +0800 @@ -61,16 +61,19 @@ interface nsINntpIncomingServer : nsISup attribute boolean markOldRead; /* abbreviate the newsgroup names in the folder pane? */ attribute boolean abbreviate; /* do we use a single login per server or do we login per group */ attribute boolean singleSignon; + /* the server charset and it may be needed to display newsgroup folder names correctly */ + attribute ACString charset; + /* the server keeps track of all the newsgroups we are subscribed to */ void addNewsgroup(in string name); void removeNewsgroup(in string name); void writeNewsrcFile(); attribute boolean newsrcHasChanged; diff -U 8 -p -r mailnews.old/news/src/nsNewsDownloadDialogArgs.cpp mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.cpp --- mailnews.old/news/src/nsNewsDownloadDialogArgs.cpp 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.cpp 2005-04-13 13:24:48.000000000 +0800 @@ -37,45 +37,41 @@ * ***** END LICENSE BLOCK ***** */ #include "nsNewsDownloadDialogArgs.h" #include "nsCRT.h" #include "nsReadableUtils.h" nsNewsDownloadDialogArgs::nsNewsDownloadDialogArgs() { - mGroupName = ""; mArticleCount = 0; mServerKey = ""; mHitOK = PR_FALSE; mDownloadAll = PR_FALSE; } nsNewsDownloadDialogArgs::~nsNewsDownloadDialogArgs() { } NS_IMPL_ISUPPORTS1(nsNewsDownloadDialogArgs, nsINewsDownloadDialogArgs) -NS_IMETHODIMP nsNewsDownloadDialogArgs::GetGroupName(char * *aGroupName) -{ - NS_ENSURE_ARG_POINTER(aGroupName); - - *aGroupName = ToNewCString(mGroupName); - - return NS_OK; -} -NS_IMETHODIMP nsNewsDownloadDialogArgs::SetGroupName(const char * aGroupName) -{ - NS_ENSURE_ARG_POINTER(aGroupName); - - mGroupName = aGroupName; - - return NS_OK; -} +NS_IMETHODIMP nsNewsDownloadDialogArgs::GetGroupName(nsAString & aGroupName) + { + aGroupName = mGroupName; + + return NS_OK; + } +NS_IMETHODIMP nsNewsDownloadDialogArgs::SetGroupName(const nsAString & aGroupName) + { + + mGroupName = aGroupName; + + return NS_OK; + } NS_IMETHODIMP nsNewsDownloadDialogArgs::GetArticleCount(PRInt32 *aArticleCount) { NS_ENSURE_ARG_POINTER(aArticleCount); *aArticleCount = mArticleCount; return NS_OK; } diff -U 8 -p -r mailnews.old/news/src/nsNewsDownloadDialogArgs.h mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.h --- mailnews.old/news/src/nsNewsDownloadDialogArgs.h 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.h 2005-04-13 13:25:12.000000000 +0800 @@ -47,16 +47,16 @@ class nsNewsDownloadDialogArgs : public public: nsNewsDownloadDialogArgs(); virtual ~nsNewsDownloadDialogArgs(); NS_DECL_ISUPPORTS NS_DECL_NSINEWSDOWNLOADDIALOGARGS private: - nsCString mGroupName; + nsString mGroupName; PRInt32 mArticleCount; nsCString mServerKey; PRBool mHitOK; PRBool mDownloadAll; }; #endif // nsNewsDownloadDialogArgs_h__ diff -U 8 -p -r mailnews.old/news/src/nsNewsFolder.cpp mozilla/mailnews/news/src/nsNewsFolder.cpp --- mailnews.old/news/src/nsNewsFolder.cpp 2005-02-15 08:13:35.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsFolder.cpp 2005-04-19 08:33:14.976769560 +0800 @@ -88,16 +88,18 @@ #include "nsIInterfaceRequestor.h" #include "nsIInterfaceRequestorUtils.h" #include "nsReadableUtils.h" #include "nsNewsDownloader.h" #include "nsIStringBundle.h" #include "nsEscape.h" +#include "nsMsgI18N.h" +#include "nsNativeCharsetUtils.h" // we need this because of an egcs 1.0 (and possibly gcc) compiler bug // that doesn't allow you to call ::nsISupports::GetIID() inside of a class // that multiply inherits from nsISupports static NS_DEFINE_CID(kRDFServiceCID, NS_RDFSERVICE_CID); static NS_DEFINE_CID(kCNewsDB, NS_NEWSDB_CID); // ###tw This really ought to be the most @@ -112,33 +114,33 @@ static NS_DEFINE_CID(kCNewsDB, NS_NEWSDB /////////////////////////////////////////////////////////////////////////////// / nsMsgNewsFolder::nsMsgNewsFolder(void) : nsMsgLineBuffer(nsnull, PR_FALSE), mExpungedBytes(0), mGettingNews(PR_FALSE), mInitialized(PR_FALSE), mOptionLines(""), mUnsubscribedNewsgroupLines(""), m_downloadMessageForOfflineUse(PR_FALSE), m_downloadingMultipleMessages(PR_FALSE), - mReadSet(nsnull), mGroupUsername(nsnull), mGroupPassword(nsnull), mAsciiName(nsnull) + mReadSet(nsnull), mGroupUsername(nsnull), mGroupPassword(nsnull), mRawName(nsnull) { MOZ_COUNT_CTOR(nsNewsFolder); // double count these for now. /* we're parsing the newsrc file, and the line breaks are platform specific. * if MSG_LINEBREAK != CRLF, then we aren't looking for CRLF */ if (PL_strcmp(MSG_LINEBREAK, CRLF)) SetLookingForCRLF(PR_FALSE); } nsMsgNewsFolder::~nsMsgNewsFolder(void) { MOZ_COUNT_DTOR(nsNewsFolder); delete mReadSet; PR_Free(mGroupUsername); PR_Free(mGroupPassword); - PR_Free(mAsciiName); + PR_Free(mRawName); } NS_IMPL_ADDREF_INHERITED(nsMsgNewsFolder, nsMsgDBFolder) NS_IMPL_RELEASE_INHERITED(nsMsgNewsFolder, nsMsgDBFolder) NS_IMETHODIMP nsMsgNewsFolder::QueryInterface(REFNSIID aIID, void** aInstancePtr) { if (!aInstancePtr) return NS_ERROR_NULL_POINTER; @@ -205,20 +207,24 @@ nsMsgNewsFolder::AddNewsgroup(const char nsCOMPtr <nsINntpIncomingServer> nntpServer; rv = GetNntpServer(getter_AddRefs(nntpServer)); if (NS_FAILED(rv)) return rv; nsCAutoString uri(mURI); uri.Append('/'); // URI should use UTF-8 // (see RFC2396 Uniform Resource Identifiers (URI): Generic Syntax) - // since we are forcing it to be latin-1 (IS0-8859-1) - // we can just assign with conversion + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + if (NS_FAILED(rv)) return rv; nsAutoString newsgroupName; - newsgroupName.AssignWithConversion(name); + rv = nsMsgI18NConvertToUnicode(dataCharset.get(),nsDependentCString(name),newsgroupN ame); + if (NS_FAILED(rv)) { + CopyASCIItoUTF16(name, newsgroupName); + } nsCAutoString escapedName; rv = NS_MsgEscapeEncodeURLPath(newsgroupName, escapedName); if (NS_FAILED(rv)) return rv; rv = nntpServer->AddNewsgroup(escapedName.get()); if (NS_FAILED(rv)) return rv; @@ -511,32 +517,35 @@ nsMsgNewsFolder::GetMessages(nsIMsgWindo } NS_IMETHODIMP nsMsgNewsFolder::GetFolderURL(char **url) { NS_ENSURE_ARG_POINTER(url); nsXPIDLCString hostName; nsresult rv = GetHostname(getter_Copies(hostName)); - nsXPIDLCString groupName; - rv = GetAsciiName(getter_Copies(groupName)); + nsXPIDLString groupName; + rv = GetName(getter_Copies(groupName)); NS_ENSURE_SUCCESS(rv, rv); nsCOMPtr<nsIMsgIncomingServer> server; rv = GetServer(getter_AddRefs(server)); NS_ENSURE_SUCCESS(rv, rv); PRInt32 port; PRBool isSecure = PR_FALSE; rv = server->GetIsSecure(&isSecure); NS_ENSURE_SUCCESS(rv, rv); rv = server->GetPort(&port); NS_ENSURE_SUCCESS(rv, rv); const char *newsScheme = (isSecure) ? SNEWS_SCHEME : NEWS_SCHEME; - *url = PR_smprintf("%s//%s:%ld/%s", newsScheme, hostName.get(), port, groupName.get()); + nsXPIDLCString escapedName; + rv = NS_MsgEscapeEncodeURLPath(groupName, escapedName); + NS_ENSURE_SUCCESS(rv, rv); + *url = PR_smprintf("%s//%s:%ld/%s", newsScheme, hostName.get(), port, escapedName.get()); return NS_OK; } NS_IMETHODIMP nsMsgNewsFolder::SetNewsrcHasChanged(PRBool newsrcHasChanged) { nsresult rv; @@ -549,18 +558,28 @@ NS_IMETHODIMP nsMsgNewsFolder::SetNewsrc NS_IMETHODIMP nsMsgNewsFolder::CreateSubfolder(const PRUnichar *uninewsgroupname, nsIMsgWindow *msgWindow) { nsresult rv = NS_OK; NS_ENSURE_ARG_POINTER(uninewsgroupname); if (!*uninewsgroupname) return NS_ERROR_FAILURE; - nsCAutoString newsgroupname; - newsgroupname.AssignWithConversion(uninewsgroupname); + nsCAutoString newsgroupname; + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + if (NS_FAILED(rv)) return rv; + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + if (NS_FAILED(rv)) return rv; + + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), nsDependentString(uninewsgroupname), newsgroupname); + if (NS_FAILED(rv)) { + LossyCopyUTF16toASCII(uninewsgroupname,newsgroupname); + } nsFileSpec path; nsCOMPtr<nsIFileSpec> pathSpec; rv = GetPath(getter_AddRefs(pathSpec)); if (NS_FAILED(rv)) return rv; rv = pathSpec->GetFileSpec(&path); @@ -569,29 +588,32 @@ NS_IMETHODIMP nsMsgNewsFolder::CreateSub // Create an empty database for this mail folder, set its name from the user nsCOMPtr<nsIMsgDatabase> newsDBFactory; nsCOMPtr <nsIMsgDatabase> newsDB; //Now we have a valid directory or we have returned. //Make sure the new folder name is valid // remember, some file systems (like mac) can't handle long file names - nsCAutoString hashedName = newsgroupname; + nsCAutoString fileName; + nsAutoString hashedName(uninewsgroupname); rv = NS_MsgHashIfNecessary(hashedName); - path += hashedName.get(); + NS_CopyUnicodeToNative(hashedName, fileName); + path += fileName.get(); //Now let's create the actual new folder rv = AddNewsgroup(newsgroupname.get(), "", getter_AddRefs(child)); if (NS_SUCCEEDED(rv)) SetNewsrcHasChanged(PR_TRUE); // subscribe UI does this - but maybe we got here through auto-subscribe - if(NS_SUCCEEDED(rv) && child) + if(NS_SUCCEEDED(rv) && child){ + child->SetCharset(dataCharset.get()); NotifyItemAdded(child); - + } return rv; } NS_IMETHODIMP nsMsgNewsFolder::Delete() { nsresult rv = GetDatabase(nsnull); if(NS_SUCCEEDED(rv)) @@ -1124,27 +1146,42 @@ nsMsgNewsFolder::HandleNewsrcLine(char* // // So lines like this in a newsrc file should be ignored: // 3746EF3F.6080309@netscape.com: // 3746EF3F.6080309%40netscape.com: if (PL_strstr(line,"@") || PL_strstr(line,"%40")) // skipping, it contains @ or %40 subscribed = PR_FALSE; + nsCOMPtr<nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + if (NS_FAILED(rv)) return rv; + //now we convert from UTF8 to the server charset encoding because the newsrc file is always saved in UTF8 + nsAutoString unicodeLine; + nsCAutoString rawLine; + CopyUTF8toUTF16(line,unicodeLine); + + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ASSERTION(NS_SUCCEEDED(rv),"failed get sever charset"); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), unicodeLine, rawLine); + if (NS_FAILED(rv)) + LossyCopyUTF16toASCII(unicodeLine,rawLine); + if (subscribed) { // we're subscribed, so add it nsCOMPtr <nsIMsgFolder> child; - rv = AddNewsgroup(line, setStr, getter_AddRefs(child)); + rv = AddNewsgroup(rawLine.get(), setStr, getter_AddRefs(child)); if (NS_FAILED(rv)) return -1; } else { - rv = RememberUnsubscribedGroup(line, setStr); + rv = RememberUnsubscribedGroup(rawLine.get(), setStr); if (NS_FAILED(rv)) return -1; } return 0; } nsresult @@ -1503,20 +1540,22 @@ nsresult nsMsgNewsFolder::CreateBaseMess NS_IMETHODIMP nsMsgNewsFolder::GetNewsrcLine(char **newsrcLine) { nsresult rv; if (!newsrcLine) return NS_ERROR_NULL_POINTER; + nsXPIDLString uninewsgroupname; nsXPIDLCString newsgroupname; - rv = GetAsciiName(getter_Copies(newsgroupname)); + rv = GetName(getter_Copies(uninewsgroupname)); if (NS_FAILED(rv)) return rv; - + CopyUTF16toUTF8(uninewsgroupname, newsgroupname); + nsCAutoString newsrcLineStr; newsrcLineStr = newsgroupname; newsrcLineStr += ':'; if (mReadSet) { nsXPIDLCString setStr; mReadSet->Output(getter_Copies(setStr)); if (NS_SUCCEEDED(rv)) @@ -1575,38 +1614,62 @@ nsMsgNewsFolder::GetOptionLines(char **o } NS_IMETHODIMP nsMsgNewsFolder::OnReadChanged(nsIDBChangeListener * aInstigator) { return SetNewsrcHasChanged(PR_TRUE); } +NS_IMETHODIMP +nsMsgNewsFolder::GetUnicodeName(nsAString & name) +{ + nsresult rv; + if(mUnicodeName.IsEmpty()){ + nsXPIDLString newsgroupName; + rv = GetName(getter_Copies(newsgroupName)); + NS_ENSURE_SUCCESS(rv,rv); + mUnicodeName = newsgroupName; + } + name = mUnicodeName; + return NS_OK; +} NS_IMETHODIMP -nsMsgNewsFolder::GetAsciiName(char **asciiName) +nsMsgNewsFolder::GetRawName(char **rawName) { nsresult rv; - NS_ENSURE_ARG_POINTER(asciiName); - if (!mAsciiName) + NS_ENSURE_ARG_POINTER(rawName); + if (!mRawName) { nsXPIDLString name; rv = GetName(getter_Copies(name)); NS_ENSURE_SUCCESS(rv,rv); // convert to ASCII nsCAutoString tmpStr; - tmpStr.AssignWithConversion(name); + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + NS_ENSURE_SUCCESS(rv,rv); + + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ENSURE_SUCCESS(rv,rv); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), name, tmpStr); + + if (NS_FAILED(rv)) { + LossyCopyUTF16toASCII(name,tmpStr); + } - mAsciiName = nsCRT::strdup(tmpStr.get()); - if (!mAsciiName) return NS_ERROR_OUT_OF_MEMORY; + mRawName = nsCRT::strdup(tmpStr.get()); + if (!mRawName) return NS_ERROR_OUT_OF_MEMORY; } - *asciiName = nsCRT::strdup(mAsciiName); - if (!*asciiName) return NS_ERROR_OUT_OF_MEMORY; + *rawName = nsCRT::strdup(mRawName); + if (!*rawName) return NS_ERROR_OUT_OF_MEMORY; return NS_OK; } NS_IMETHODIMP nsMsgNewsFolder::GetNntpServer(nsINntpIncomingServer **result) { nsresult rv; Only in mozilla/mailnews/news/src: nsNewsFolder.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNewsFolder.h mozilla/mailnews/news/src/nsNewsFolder.h --- mailnews.old/news/src/nsNewsFolder.h 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsFolder.h 2005-04-19 08:33:47.599810104 +0800 @@ -154,18 +154,18 @@ protected: nsMsgKeySet *mReadSet; nsCOMPtr<nsIFileSpec> mNewsrcFilePath; // used for auth news char *mGroupUsername; char *mGroupPassword; - // the name of the newsgroup. - char *mAsciiName; + char *mRawName; + nsString mUnicodeName; PRInt32 mSortOrder; private: nsresult CreateNewsgroupUsernameUrlForSignon(const char *inUriStr, char **result); nsresult CreateNewsgroupPasswordUrlForSignon(const char *inUriStr, char **result); nsresult CreateNewsgroupUrlForSignon(const char *inUriStr, const char *ref, char **result); nsCOMPtr <nsIFileSpec> mFilterFile; Only in mozilla/mailnews/news/src: nsNewsFolder.h~ diff -U 8 -p -r mailnews.old/news/src/nsNntpIncomingServer.cpp mozilla/mailnews/news/src/nsNntpIncomingServer.cpp --- mailnews.old/news/src/nsNntpIncomingServer.cpp 2004-08-27 06:16:52.000000000 +0800 +++ mozilla/mailnews/news/src/nsNntpIncomingServer.cpp 2005-04-19 08:33:31.246296216 +0800 @@ -57,22 +57,25 @@ #include "nsMsgUtils.h" #include "nsIPrompt.h" #include "nsIStringBundle.h" #include "nntpCore.h" #include "nsIWindowWatcher.h" #include "nsITreeColumns.h" #include "nsIDOMElement.h" #include "nsMsgFolderFlags.h" +#include "nsMsgI18N.h" +#include "nsIPrefLocalizedString.h" #define INVALID_VERSION 0 #define VALID_VERSION 1 #define NEW_NEWS_DIR_NAME "News" #define PREF_MAIL_NEWSRC_ROOT "mail.newsrc_root" #define PREF_MAIL_NEWSRC_ROOT_REL "mail.newsrc_root-rel" +#define PREF_MAILNEWS_VIEW_DEFAULT_CHARSET "mailnews.view_default_charset" #define HOSTINFO_FILE_NAME "hostinfo.dat" #define NEWS_DELIMITER '.' // this platform specific junk is so the newsrc filenames we create // will resemble the migrated newsrc filenames. #if defined(XP_UNIX) || defined(XP_BEOS) #define NEWSRC_FILE_PREFIX "newsrc-" @@ -304,16 +307,70 @@ nsresult nsNntpIncomingServer::SetupNews mNewsrcSaveTimer = do_CreateInstance("@mozilla.org/timer;1"); mNewsrcSaveTimer->InitWithFuncCallback(OnNewsrcSaveTimer, (void*)this, timeInMSUint32, nsITimer::TYPE_REPEATING_SLACK); return NS_OK; } NS_IMETHODIMP +nsNntpIncomingServer::SetCharset(const nsACString & aCharset) +{ + nsresult rv; + rv = SetCharValue("charset", PromiseFlatCString(aCharset).get()); + return rv; +} + +NS_IMETHODIMP +nsNntpIncomingServer::GetCharset(nsACString & aCharset) +{ + nsresult rv; + nsXPIDLCString serverCharset; + //first we get the per-server settings mail.server.<serverkey>.charset + rv = GetCharValue("charset",getter_Copies(serverCharset)); + + //if the per-server setting is empty,we get the default charset from + //mailnews.view_default_charset setting and set it as per-server preference. + if(serverCharset.IsEmpty()){ + nsXPIDLCString defaultCharset; + nsCOMPtr<nsIPrefBranch> prefBranch(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv)); + + if (NS_SUCCEEDED(rv)){ + nsCOMPtr<nsIPrefLocalizedString> pls; + rv = prefBranch->GetComplexValue(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, + NS_GET_IID(nsIPrefLocalizedString), getter_AddRefs(pls)); + if (NS_SUCCEEDED(rv)) + { + nsXPIDLString ucsval; + pls->ToString(getter_Copies(ucsval)); + if (ucsval) + { + LossyCopyUTF16toASCII(ucsval, defaultCharset); + } + } + } + + if(defaultCharset.IsEmpty()){ + serverCharset = "ISO-8859-1"; + }else{ + serverCharset = defaultCharset; + } + +#ifdef DEBUG_NEWS + printf("default charset for the server is %s\n", (const char *)defaultCharset); +#endif + + SetCharset(serverCharset); + } + + aCharset = serverCharset; + return NS_OK; +} + +NS_IMETHODIMP nsNntpIncomingServer::WriteNewsrcFile() { nsresult rv; PRBool newsrcHasChanged; rv = GetNewsrcHasChanged(&newsrcHasChanged); if (NS_FAILED(rv)) return rv; @@ -806,17 +863,17 @@ writeGroupToHostInfoFile(nsCString &aEle nsAutoString name; nsresult rv = NS_MsgDecodeUnescapeURLPath(aElement, name); if (NS_FAILED(rv)) { // stop, something is bad. return PR_FALSE; } nsCAutoString nameOnDisk; - nameOnDisk.AssignWithConversion(name.get()); + CopyUTF16toUTF8(name, nameOnDisk); // XXX todo ",,1,0,0" is a temporary hack, fix it *stream << nameOnDisk.get() << ",,1,0,0" << MSG_LINEBREAK; return PR_TRUE; } nsresult nsNntpIncomingServer::WriteHostInfoFile() @@ -1128,19 +1185,41 @@ nsNntpIncomingServer::UpdateSubscribed() NS_IMETHODIMP nsNntpIncomingServer::AddTo(const char *aName, PRBool addAsSubscribed, PRBool aSubscribable, PRBool changeIfExists) { nsresult rv = EnsureInner(); NS_ENSURE_SUCCESS(rv,rv); nsAutoString newsgroupName; - newsgroupName.AssignWithConversion(aName); + char *escapedName = nsnull; + + nsCOMPtr<nsIMsgFolder> rootFolder; + rv = GetRootFolder(getter_AddRefs(rootFolder)); + NS_ENSURE_SUCCESS(rv,rv); + if (!rootFolder) return NS_ERROR_FAILURE; + nsCOMPtr <nsIMsgNewsFolder> newsFolder = do_QueryInterface(rootFolder, &rv); + NS_ENSURE_SUCCESS(rv,rv); + if (!newsFolder) return NS_ERROR_FAILURE; + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = newsFolder->GetNntpServer(getter_AddRefs(nntpServer)); + NS_ENSURE_SUCCESS(rv,rv); + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ENSURE_SUCCESS(rv,rv); + rv = nsMsgI18NConvertToUnicode(dataCharset.get(),nsDependentCString(aName),newsgroup Name); + if (NS_SUCCEEDED(rv)) { + escapedName = nsEscape(NS_ConvertUTF16toUTF8(newsgroupName).get(), url_Path); + } + + if (NS_FAILED(rv)) { + CopyASCIItoUTF16(aName, newsgroupName); + escapedName = nsEscape(aName, url_Path); + } - char *escapedName = nsEscape(NS_ConvertUCS2toUTF8(newsgroupName.get()).get(), url_Path); if (!escapedName) return NS_ERROR_OUT_OF_MEMORY; rv = AddGroupOnServer(escapedName); NS_ENSURE_SUCCESS(rv,rv); rv = mInner->AddTo(escapedName,addAsSubscribed, aSubscribable, changeIfExists); NS_ENSURE_SUCCESS(rv,rv); @@ -1245,17 +1324,27 @@ nsNntpIncomingServer::HandleLine(char* l if (line[0] == '#' || line[0] == '\0') return 0; line[line_size] = 0; if (mHasSeenBeginGroups) { char *commaPos = PL_strchr(line,','); if (commaPos) *commaPos = 0; - nsresult rv = AddTo(line, PR_FALSE, PR_TRUE, PR_TRUE); + //now we convert from UTF8 to the server charset encoding because the hostInfo.dat is always saved in UTF8 + nsAutoString unicodeLine; + nsCAutoString encodedLine; + CopyUTF8toUTF16(line,unicodeLine); + nsCAutoString dataCharset; + nsresult rv = GetCharset(dataCharset); + NS_ASSERTION(NS_SUCCEEDED(rv),"failed to get sever charset"); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), unicodeLine, encodedLine); + if (NS_FAILED(rv)) + LossyCopyUTF16toASCII(unicodeLine,encodedLine); + rv = AddTo(encodedLine.get(), PR_FALSE, PR_TRUE, PR_TRUE); NS_ASSERTION(NS_SUCCEEDED(rv),"failed to add line"); if (NS_SUCCEEDED(rv)) { // since we've seen one group, we can claim we've loaded the hostinfo file mHostInfoLoaded = PR_TRUE; } } else { if (nsCRT::strncmp(line,"begingroups", 11) == 0) { Only in mozilla/mailnews/news/src: nsNntpIncomingServer.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNNTPNewsgroupList.cpp mozilla/mailnews/news/src/nsNNTPNewsgroupList.cpp --- mailnews.old/news/src/nsNNTPNewsgroupList.cpp 2005-02-01 05:53:40.000000000 +0800 +++ mozilla/mailnews/news/src/nsNNTPNewsgroupList.cpp 2005-04-13 13:55:22.000000000 +0800 @@ -319,21 +319,21 @@ nsNNTPNewsgroupList::GetRangeOfArtsToDow nsCOMPtr<nsINewsDownloadDialogArgs> args = do_CreateInstance("@mozilla.org/messenger/newsdownloaddialogargs;1", &rv); if (NS_FAILED(rv)) return rv; NS_ENSURE_SUCCESS(rv,rv); rv = args->SetArticleCount(*last - *first + 1); NS_ENSURE_SUCCESS(rv,rv); - nsXPIDLCString groupName; - rv = m_newsFolder->GetAsciiName(getter_Copies(groupName)); + nsXPIDLString groupName; + rv = m_newsFolder->GetUnicodeName(groupName); NS_ENSURE_SUCCESS(rv,rv); - rv = args->SetGroupName((const char *)groupName); + rv = args->SetGroupName(groupName); NS_ENSURE_SUCCESS(rv,rv); // get the server key nsXPIDLCString serverKey; rv = server->GetKey(getter_Copies(serverKey)); NS_ENSURE_SUCCESS(rv,rv); rv = args->SetServerKey((const char *)serverKey); Only in mozilla/mailnews/news/src: nsNNTPNewsgroupList.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNNTPProtocol.cpp mozilla/mailnews/news/src/nsNNTPProtocol.cpp --- mailnews.old/news/src/nsNNTPProtocol.cpp 2005-02-16 11:18:46.000000000 +0800 +++ mozilla/mailnews/news/src/nsNNTPProtocol.cpp 2005-04-19 03:36:57.000000000 +0800 @@ -1934,17 +1934,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom char *command=0; PRInt32 status = 0; if (m_typeWanted == ARTICLE_WANTED) { if (m_key != nsMsgKey_None) { nsresult rv; nsXPIDLCString newsgroupName; if (m_newsFolder) { - rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); + rv = m_newsFolder->GetRawName(getter_Copies(newsgroupName)); NS_ENSURE_SUCCESS(rv,rv); } PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) current group = %s, desired group = %s", this, m_currentGroup.get(), newsgroupName.get())); // if the current group is the desired group, we can just issue the ARTICLE command // if not, we have to do a GROUP first if (!PL_strcmp(m_currentGroup.get(), newsgroupName.get())) m_nextState = NNTP_SEND_ARTICLE_NUMBER; @@ -2045,17 +2045,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom else if(m_typeWanted == GROUP_WANTED) { nsresult rv = NS_ERROR_NULL_POINTER; NS_ASSERTION(m_newsFolder, "m_newsFolder is null, panic!"); if (!m_newsFolder) return -1; nsXPIDLCString group_name; - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); NS_ASSERTION(NS_SUCCEEDED(rv),"failed to get newsgroup name"); if (NS_FAILED(rv)) return -1; m_firstArticle = 0; m_lastArticle = 0; NS_MsgSACopy(&command, "GROUP "); NS_MsgSACat(&command, (const char *)group_name); @@ -2090,17 +2090,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) doing GROUP for XPAT", this)); nsXPIDLCString group_name; /* for XPAT, we have to GROUP into the group before searching */ if (!m_newsFolder) { NNTP_LOG_NOTE("m_newsFolder is null, panic!"); return -1; } - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); if (NS_FAILED(rv)) return -1; NS_MsgSACopy(&command, "GROUP "); NS_MsgSACat (&command, group_name); // force a GROUP next time m_currentGroup.Truncate(); m_nextState = NNTP_RESPONSE; @@ -2201,17 +2201,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom } else { nsresult rv = NS_OK; nsXPIDLCString group_name; NS_ASSERTION(m_newsFolder, "no newsFolder"); if (m_newsFolder) { - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); } if (m_responseCode == MK_NNTP_RESPONSE_GROUP_NO_GROUP && m_typeWanted == GROUP_WANTED) { PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) group (%s) not found, so unset m_currentGroup",this,(const char *)group_name)); m_currentGroup.Truncate(); m_nntpServer->GroupNotFound(m_msgWindow, group_name.get(), PR_TRUE /* opening */); @@ -2316,17 +2316,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom } PRInt32 nsNNTPProtocol::SendGroupForArticle() { nsresult rv; PRInt32 status = 0; nsXPIDLCString groupname; - rv = m_newsFolder->GetAsciiName(getter_Copies(groupname)); + rv = m_newsFolder->GetRawName(getter_Copies(groupname)); NS_ASSERTION(NS_SUCCEEDED(rv) && groupname.get() && groupname.get()[0], "no group name"); char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "GROUP %.512s" CRLF, (const char *)groupname); @@ -2348,17 +2348,17 @@ nsNNTPProtocol::SetCurrentGroup() nsresult rv; nsXPIDLCString groupname; NS_ASSERTION(m_newsFolder, "no news folder"); if (!m_newsFolder) { m_currentGroup.Truncate(); return NS_ERROR_UNEXPECTED; } - rv = m_newsFolder->GetAsciiName(getter_Copies(groupname)); + rv = m_newsFolder->GetRawName(getter_Copies(groupname)); NS_ASSERTION(NS_SUCCEEDED(rv) && groupname.get()[0], "no group name"); PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) SetCurrentGroup to %s",this,(const char *)groupname)); m_currentGroup = groupname; return NS_OK; } PRInt32 nsNNTPProtocol::SendGroupForArticleResponse() { @@ -3873,17 +3873,17 @@ PRInt32 nsNNTPProtocol::DisplayNewsRC() if (NS_FAILED(rv)) return -1; if (!currFolder) return -1; m_newsFolder = do_QueryInterface(currFolder, &rv); if (NS_FAILED(rv)) return -1; if (!m_newsFolder) return -1; nsXPIDLCString name; - rv = m_newsFolder->GetAsciiName(getter_Copies(name)); + rv = m_newsFolder->GetRawName(getter_Copies(name)); if (NS_FAILED(rv)) return -1; if (!name) return -1; /* send group command to server */ char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "GROUP %.512s" CRLF, (const char *)name); nsCOMPtr<nsIMsgMailNewsUrl> mailnewsurl = do_QueryInterface(m_runningURL); @@ -3963,17 +3963,17 @@ PRInt32 nsNNTPProtocol::DisplayNewsRCRes NS_ASSERTION(NS_SUCCEEDED(rv),"DisplaySubscribedGroup() failed"); if (NS_FAILED(rv)) status = -1; if (status < 0) return status; } else if (m_responseCode == MK_NNTP_RESPONSE_GROUP_NO_GROUP) { nsXPIDLCString name; - rv = m_newsFolder->GetAsciiName(getter_Copies(name)); + rv = m_newsFolder->GetRawName(getter_Copies(name)); if (NS_SUCCEEDED(rv)) { m_nntpServer->GroupNotFound(m_msgWindow, name.get(), PR_FALSE); } PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) NO_GROUP, so unset m_currentGroup", this)); m_currentGroup.Truncate(); } @@ -4394,17 +4394,17 @@ PRInt32 nsNNTPProtocol::XPATResponse(nsI PRInt32 nsNNTPProtocol::ListPrettyNames() { nsXPIDLCString group_name; char outputBuffer[OUTPUT_BUFFER_SIZE]; PRInt32 status = 0; - nsresult rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + nsresult rv = m_newsFolder->GetRawName(getter_Copies(group_name)); PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "LIST PRETTYNAMES %.512s" CRLF, NS_SUCCEEDED(rv) ? (const char *) group_name : ""); nsCOMPtr<nsIMsgMailNewsUrl> mailnewsurl = do_QueryInterface(m_runningURL); if (mailnewsurl) status = SendData(mailnewsurl, outputBuffer); @@ -4471,17 +4471,17 @@ PRInt32 nsNNTPProtocol::ListPrettyNamesR PR_FREEIF(line); return 0; } PRInt32 nsNNTPProtocol::ListXActive() { nsXPIDLCString group_name; nsresult rv; - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); if (NS_FAILED(rv)) return -1; PRInt32 status = 0; char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "LIST XACTIVE %.512s" CRLF, @@ -4624,17 +4624,17 @@ PRInt32 nsNNTPProtocol::SendListGroup() nsresult rv; char outputBuffer[OUTPUT_BUFFER_SIZE]; PRInt32 status = 0; NS_ASSERTION(m_newsFolder,"no newsFolder"); if (!m_newsFolder) return -1; nsXPIDLCString newsgroupName; - rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); + rv = m_newsFolder->GetRawName(getter_Copies(newsgroupName)); NS_ENSURE_SUCCESS(rv,rv); PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "listgroup %.512s" CRLF, newsgroupName.get()); m_articleList = do_CreateInstance(NS_NNTPARTICLELIST_CONTRACTID, &rv); Only in mozilla/mailnews/news/src: nsNNTPProtocol.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNntpService.cpp mozilla/mailnews/news/src/nsNntpService.cpp --- mailnews.old/news/src/nsNntpService.cpp 2005-01-18 02:50:13.000000000 +0800 +++ mozilla/mailnews/news/src/nsNntpService.cpp 2005-04-13 13:56:02.000000000 +0800 @@ -650,18 +650,21 @@ nsNntpService::GetFolderFromUri(const ch // but the *name* of the newsgroup (we are calling ::GetChildNamed()) // is unescaped. see http://bugzilla.mozilla.org/show_bug.cgi?id=210089#c17 // for more about this char *unescapedPath = PL_strdup(path.get() + 1); /* skip the leading slash */ if (!unescapedPath) return NS_ERROR_OUT_OF_MEMORY; nsUnescape(unescapedPath); + nsAutoString newsgroupName; + //the encoding is UTF-8 after unescaped. + CopyUTF8toUTF16(unescapedPath,newsgroupName); nsCOMPtr<nsISupports> subFolder; - rv = rootFolder->GetChildNamed(NS_ConvertASCIItoUCS2(unescapedPath).get() , getter_AddRefs(subFolder)); + rv = rootFolder->GetChildNamed(newsgroupName.get() , getter_AddRefs(subFolder)); PL_strfree(unescapedPath); NS_ENSURE_SUCCESS(rv,rv); return CallQueryInterface(subFolder, aFolder); } NS_IMETHODIMP nsNntpService::CopyMessage(const char * aSrcMessageURI, nsIStreamListener * aMailboxCopyHandler, PRBool moveMessage,

Comment on attachment 170507 [details] [diff] [review] patch use utf8 in the newsrc/hostinfo file diff -U 8 -p -r mailnews.old/base/prefs/resources/content/am-server.xul mozilla/mailnews/base/prefs/resources/content/am-server.xul --- mailnews.old/base/prefs/resources/content/am-server.xul 2005-03-04 02:08:45.000000000 +0800 +++ mozilla/mailnews/base/prefs/resources/content/am-server.xul 2005-04-13 13:20:05.000000000 +0800 @@ -47,16 +47,20 @@ <page xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul" onload="parent.onPanelLoaded('am-server.xul');"> <script type="application/x-javascript" src="chrome://messenger/content/AccountManager.js"/> <script type="application/x-javascript" src="chrome://messenger/content/am-server.js"/> <script type="application/x-javascript" src="chrome://messenger/content/am-prefs.js"/> <script type="application/x-javascript" src="chrome://messenger/content/amUtils.js"/> + <script type="application/x-javascript"><![CDATA[ + var observerService = Components.classes["@mozilla.org/observer-service;1"].getService(Components.int erfaces.nsIObserverService); + observerService.notifyObservers(null, "charsetmenu-selected", "other"); + ]]></script> <stringbundle id="bundle_messenger" src="chrome://messenger/locale/messenger.properties"/> <broadcaster id="broadcaster_doBiff"/> <broadcaster id="broadcaster_notify"/> <label hidden="true" wsm_persist="true" id="server.type"/> <dialogheader title="&serverSettings.label;"/> @@ -325,9 +329,22 @@ accesskey="&localPath.accesskey;"/> <hbox align="center"> <textbox readonly="true" wsm_persist="true" flex="1" id="server.localPath" datatype="nsIFileSpec" prefstring="mail.server.%serverkey%.directory" class="uri-element"/> <button id="browseForLocalFolder" label="&browseFolder.label;" filepickertitle="&localFolderPicker.label;" accesskey="&browseFolder.accesskey;" oncommand="BrowseForLocalFolders()"/> </hbox> </vbox> + <separator class="thin"/> + <hbox hidefor="imap,pop3,movemail" align="center" valign="middle" iscontrolcontainer="true"> + <label value="&serverDefaultCharset.label;" control="nntp.charset"/> + <menulist hidable="true" hidefor="imap,pop3,movemail" wsm_persist="true" id="nntp.charset" + ref="NC:DecodersRoot" datasources="rdf:charset-menu" + preftype="string" prefstring="mail.server.%serverkey%.charset"> + <template> + <menupopup> + <menuitem label="rdf:http://home.netscape.com/NC-rdf#Name" value="..." uri="..."/> + </menupopup> + </template> + </menulist> + </hbox> </page> Only in mozilla/mailnews/base/prefs/resources/content: am-server.xul~ diff -U 8 -p -r mailnews.old/base/prefs/resources/locale/en-US/am-server-top.dtd mozilla/mailnews/base/prefs/resources/locale/en-US/am-server-top.dtd --- mailnews.old/base/prefs/resources/locale/en-US/am-server-top.dtd 2005-02-09 01:01:47.000000000 +0800 +++ mozilla/mailnews/base/prefs/resources/locale/en-US/am-server-top.dtd 2005-04-13 13:19:30.000000000 +0800 @@ -79,15 +79,16 @@ <!ENTITY newsrcFilePath.label "newsrc file:"> <!ENTITY newsrcFilePath.accesskey "f"> <!ENTITY localPath.label "Local directory:"> <!ENTITY localPath.accesskey "L"> <!ENTITY abbreviate.label "Show newsgroup names in the Mail Folder pane as:"> <!ENTITY abbreviateOn.label "Full names (For example, 'netscape.public.mozilla.mail-news')"> <!ENTITY abbreviateOff.label "Abbreviate names (For example, 'n.p.m.mail-news')"> <!ENTITY advancedButton.label "Advanced..."> +<!ENTITY serverDefaultCharset.label "Default Character Encoding:"> <!ENTITY advancedButton.accesskey "v"> <!ENTITY localFolderPicker.label "Select Local Directory"> <!ENTITY browseFolder.label "Browse..."> <!ENTITY browseFolder.accesskey "B"> <!ENTITY newsrcPicker.label "Select newsrc File"> <!ENTITY browseNewsrc.label "Browse..."> <!ENTITY browseNewsrc.accesskey "e"> diff -U 8 -p -r mailnews.old/news/public/nsIMsgNewsFolder.idl mozilla/mailnews/news/public/nsIMsgNewsFolder.idl --- mailnews.old/news/public/nsIMsgNewsFolder.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsIMsgNewsFolder.idl 2005-04-19 03:37:37.000000000 +0800 @@ -36,27 +36,28 @@ * ***** END LICENSE BLOCK ***** */ #include "nsISupports.idl" #include "nsIMsgFolder.idl" interface nsIMsgWindow; interface nsINntpIncomingServer; -[scriptable, uuid(3716abe4-f6d4-11d2-86d5-004005263078)] +[scriptable, uuid(02699533-7ba8-4e78-94dc-a20374fb2711)] interface nsIMsgNewsFolder : nsISupports { attribute string groupUsername; attribute string groupPassword; - /** the ascii name for the newsgroup. - * newsgroup names are always ascii, and this can help simply - * the code, to prevent use from having to get the name (as a wstring) - * and then converting it back to ascii. + readonly attribute AString unicodeName; + /** the raw name name for the newsgroup (server side name). + * newsgroup names are might not always ascii(bug#126453), and this raw name name + * can help simply the code, to prevent use from having to get the unicode name + * (as a wstring) and then converting it back to server side raw name. */ - readonly attribute string asciiName; + readonly attribute string rawName; readonly attribute nsINntpIncomingServer nntpServer; attribute boolean saveArticleOffline; string getGroupPasswordWithUI(in wstring aPromptString, in wstring aPromptTitle, in nsIMsgWindow aMsgWindow); string getGroupUsernameWithUI(in wstring aPromptString, in wstring aPromptTitle, in nsIMsgWindow aMsgWindow); void forgetGroupUsername(); void forgetGroupPassword(); Only in mozilla/mailnews/news/public: nsIMsgNewsFolder.idl~ diff -U 8 -p -r mailnews.old/news/public/nsINewsDownloadDialogArgs.idl mozilla/mailnews/news/public/nsINewsDownloadDialogArgs.idl --- mailnews.old/news/public/nsINewsDownloadDialogArgs.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsINewsDownloadDialogArgs.idl 2005-04-13 13:22:00.000000000 +0800 @@ -33,19 +33,19 @@ * and other provisions required by the GPL or the LGPL. If you do not delete * the provisions above, a recipient may use your version of this file under * the terms of any one of the MPL, the GPL or the LGPL. * * ***** END LICENSE BLOCK ***** */ #include "nsISupports.idl" -[scriptable, uuid(b5037f06-1dd1-11b2-bf61-fcab5b5837a7)] +[scriptable, uuid(3634327c-392b-4686-adf5-576e6cef9196)] interface nsINewsDownloadDialogArgs: nsISupports { - attribute string groupName; + attribute AString groupName; attribute long articleCount; attribute string serverKey; attribute boolean hitOK; attribute boolean downloadAll; }; %{ C++ #define DOWNLOAD_HEADERS_URL "chrome://messenger/content/downloadheaders.xul" diff -U 8 -p -r mailnews.old/news/public/nsINntpIncomingServer.idl mozilla/mailnews/news/public/nsINntpIncomingServer.idl --- mailnews.old/news/public/nsINntpIncomingServer.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsINntpIncomingServer.idl 2005-04-13 13:22:54.000000000 +0800 @@ -61,16 +61,19 @@ interface nsINntpIncomingServer : nsISup attribute boolean markOldRead; /* abbreviate the newsgroup names in the folder pane? */ attribute boolean abbreviate; /* do we use a single login per server or do we login per group */ attribute boolean singleSignon; + /* the server charset and it may be needed to display newsgroup folder names correctly */ + attribute ACString charset; + /* the server keeps track of all the newsgroups we are subscribed to */ void addNewsgroup(in string name); void removeNewsgroup(in string name); void writeNewsrcFile(); attribute boolean newsrcHasChanged; diff -U 8 -p -r mailnews.old/news/src/nsNewsDownloadDialogArgs.cpp mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.cpp --- mailnews.old/news/src/nsNewsDownloadDialogArgs.cpp 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.cpp 2005-04-13 13:24:48.000000000 +0800 @@ -37,45 +37,41 @@ * ***** END LICENSE BLOCK ***** */ #include "nsNewsDownloadDialogArgs.h" #include "nsCRT.h" #include "nsReadableUtils.h" nsNewsDownloadDialogArgs::nsNewsDownloadDialogArgs() { - mGroupName = ""; mArticleCount = 0; mServerKey = ""; mHitOK = PR_FALSE; mDownloadAll = PR_FALSE; } nsNewsDownloadDialogArgs::~nsNewsDownloadDialogArgs() { } NS_IMPL_ISUPPORTS1(nsNewsDownloadDialogArgs, nsINewsDownloadDialogArgs) -NS_IMETHODIMP nsNewsDownloadDialogArgs::GetGroupName(char * *aGroupName) -{ - NS_ENSURE_ARG_POINTER(aGroupName); - - *aGroupName = ToNewCString(mGroupName); - - return NS_OK; -} -NS_IMETHODIMP nsNewsDownloadDialogArgs::SetGroupName(const char * aGroupName) -{ - NS_ENSURE_ARG_POINTER(aGroupName); - - mGroupName = aGroupName; - - return NS_OK; -} +NS_IMETHODIMP nsNewsDownloadDialogArgs::GetGroupName(nsAString & aGroupName) + { + aGroupName = mGroupName; + + return NS_OK; + } +NS_IMETHODIMP nsNewsDownloadDialogArgs::SetGroupName(const nsAString & aGroupName) + { + + mGroupName = aGroupName; + + return NS_OK; + } NS_IMETHODIMP nsNewsDownloadDialogArgs::GetArticleCount(PRInt32 *aArticleCount) { NS_ENSURE_ARG_POINTER(aArticleCount); *aArticleCount = mArticleCount; return NS_OK; } diff -U 8 -p -r mailnews.old/news/src/nsNewsDownloadDialogArgs.h mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.h --- mailnews.old/news/src/nsNewsDownloadDialogArgs.h 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.h 2005-04-13 13:25:12.000000000 +0800 @@ -47,16 +47,16 @@ class nsNewsDownloadDialogArgs : public public: nsNewsDownloadDialogArgs(); virtual ~nsNewsDownloadDialogArgs(); NS_DECL_ISUPPORTS NS_DECL_NSINEWSDOWNLOADDIALOGARGS private: - nsCString mGroupName; + nsString mGroupName; PRInt32 mArticleCount; nsCString mServerKey; PRBool mHitOK; PRBool mDownloadAll; }; #endif // nsNewsDownloadDialogArgs_h__ diff -U 8 -p -r mailnews.old/news/src/nsNewsFolder.cpp mozilla/mailnews/news/src/nsNewsFolder.cpp --- mailnews.old/news/src/nsNewsFolder.cpp 2005-02-15 08:13:35.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsFolder.cpp 2005-04-19 08:33:14.976769560 +0800 @@ -88,16 +88,18 @@ #include "nsIInterfaceRequestor.h" #include "nsIInterfaceRequestorUtils.h" #include "nsReadableUtils.h" #include "nsNewsDownloader.h" #include "nsIStringBundle.h" #include "nsEscape.h" +#include "nsMsgI18N.h" +#include "nsNativeCharsetUtils.h" // we need this because of an egcs 1.0 (and possibly gcc) compiler bug // that doesn't allow you to call ::nsISupports::GetIID() inside of a class // that multiply inherits from nsISupports static NS_DEFINE_CID(kRDFServiceCID, NS_RDFSERVICE_CID); static NS_DEFINE_CID(kCNewsDB, NS_NEWSDB_CID); // ###tw This really ought to be the most @@ -112,33 +114,33 @@ static NS_DEFINE_CID(kCNewsDB, NS_NEWSDB /////////////////////////////////////////////////////////////////////////////// / nsMsgNewsFolder::nsMsgNewsFolder(void) : nsMsgLineBuffer(nsnull, PR_FALSE), mExpungedBytes(0), mGettingNews(PR_FALSE), mInitialized(PR_FALSE), mOptionLines(""), mUnsubscribedNewsgroupLines(""), m_downloadMessageForOfflineUse(PR_FALSE), m_downloadingMultipleMessages(PR_FALSE), - mReadSet(nsnull), mGroupUsername(nsnull), mGroupPassword(nsnull), mAsciiName(nsnull) + mReadSet(nsnull), mGroupUsername(nsnull), mGroupPassword(nsnull), mRawName(nsnull) { MOZ_COUNT_CTOR(nsNewsFolder); // double count these for now. /* we're parsing the newsrc file, and the line breaks are platform specific. * if MSG_LINEBREAK != CRLF, then we aren't looking for CRLF */ if (PL_strcmp(MSG_LINEBREAK, CRLF)) SetLookingForCRLF(PR_FALSE); } nsMsgNewsFolder::~nsMsgNewsFolder(void) { MOZ_COUNT_DTOR(nsNewsFolder); delete mReadSet; PR_Free(mGroupUsername); PR_Free(mGroupPassword); - PR_Free(mAsciiName); + PR_Free(mRawName); } NS_IMPL_ADDREF_INHERITED(nsMsgNewsFolder, nsMsgDBFolder) NS_IMPL_RELEASE_INHERITED(nsMsgNewsFolder, nsMsgDBFolder) NS_IMETHODIMP nsMsgNewsFolder::QueryInterface(REFNSIID aIID, void** aInstancePtr) { if (!aInstancePtr) return NS_ERROR_NULL_POINTER; @@ -205,20 +207,24 @@ nsMsgNewsFolder::AddNewsgroup(const char nsCOMPtr <nsINntpIncomingServer> nntpServer; rv = GetNntpServer(getter_AddRefs(nntpServer)); if (NS_FAILED(rv)) return rv; nsCAutoString uri(mURI); uri.Append('/'); // URI should use UTF-8 // (see RFC2396 Uniform Resource Identifiers (URI): Generic Syntax) - // since we are forcing it to be latin-1 (IS0-8859-1) - // we can just assign with conversion + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + if (NS_FAILED(rv)) return rv; nsAutoString newsgroupName; - newsgroupName.AssignWithConversion(name); + rv = nsMsgI18NConvertToUnicode(dataCharset.get(),nsDependentCString(name),newsgroupN ame); + if (NS_FAILED(rv)) { + CopyASCIItoUTF16(name, newsgroupName); + } nsCAutoString escapedName; rv = NS_MsgEscapeEncodeURLPath(newsgroupName, escapedName); if (NS_FAILED(rv)) return rv; rv = nntpServer->AddNewsgroup(escapedName.get()); if (NS_FAILED(rv)) return rv; @@ -511,32 +517,35 @@ nsMsgNewsFolder::GetMessages(nsIMsgWindo } NS_IMETHODIMP nsMsgNewsFolder::GetFolderURL(char **url) { NS_ENSURE_ARG_POINTER(url); nsXPIDLCString hostName; nsresult rv = GetHostname(getter_Copies(hostName)); - nsXPIDLCString groupName; - rv = GetAsciiName(getter_Copies(groupName)); + nsXPIDLString groupName; + rv = GetName(getter_Copies(groupName)); NS_ENSURE_SUCCESS(rv, rv); nsCOMPtr<nsIMsgIncomingServer> server; rv = GetServer(getter_AddRefs(server)); NS_ENSURE_SUCCESS(rv, rv); PRInt32 port; PRBool isSecure = PR_FALSE; rv = server->GetIsSecure(&isSecure); NS_ENSURE_SUCCESS(rv, rv); rv = server->GetPort(&port); NS_ENSURE_SUCCESS(rv, rv); const char *newsScheme = (isSecure) ? SNEWS_SCHEME : NEWS_SCHEME; - *url = PR_smprintf("%s//%s:%ld/%s", newsScheme, hostName.get(), port, groupName.get()); + nsXPIDLCString escapedName; + rv = NS_MsgEscapeEncodeURLPath(groupName, escapedName); + NS_ENSURE_SUCCESS(rv, rv); + *url = PR_smprintf("%s//%s:%ld/%s", newsScheme, hostName.get(), port, escapedName.get()); return NS_OK; } NS_IMETHODIMP nsMsgNewsFolder::SetNewsrcHasChanged(PRBool newsrcHasChanged) { nsresult rv; @@ -549,18 +558,28 @@ NS_IMETHODIMP nsMsgNewsFolder::SetNewsrc NS_IMETHODIMP nsMsgNewsFolder::CreateSubfolder(const PRUnichar *uninewsgroupname, nsIMsgWindow *msgWindow) { nsresult rv = NS_OK; NS_ENSURE_ARG_POINTER(uninewsgroupname); if (!*uninewsgroupname) return NS_ERROR_FAILURE; - nsCAutoString newsgroupname; - newsgroupname.AssignWithConversion(uninewsgroupname); + nsCAutoString newsgroupname; + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + if (NS_FAILED(rv)) return rv; + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + if (NS_FAILED(rv)) return rv; + + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), nsDependentString(uninewsgroupname), newsgroupname); + if (NS_FAILED(rv)) { + LossyCopyUTF16toASCII(uninewsgroupname,newsgroupname); + } nsFileSpec path; nsCOMPtr<nsIFileSpec> pathSpec; rv = GetPath(getter_AddRefs(pathSpec)); if (NS_FAILED(rv)) return rv; rv = pathSpec->GetFileSpec(&path); @@ -569,29 +588,32 @@ NS_IMETHODIMP nsMsgNewsFolder::CreateSub // Create an empty database for this mail folder, set its name from the user nsCOMPtr<nsIMsgDatabase> newsDBFactory; nsCOMPtr <nsIMsgDatabase> newsDB; //Now we have a valid directory or we have returned. //Make sure the new folder name is valid // remember, some file systems (like mac) can't handle long file names - nsCAutoString hashedName = newsgroupname; + nsCAutoString fileName; + nsAutoString hashedName(uninewsgroupname); rv = NS_MsgHashIfNecessary(hashedName); - path += hashedName.get(); + NS_CopyUnicodeToNative(hashedName, fileName); + path += fileName.get(); //Now let's create the actual new folder rv = AddNewsgroup(newsgroupname.get(), "", getter_AddRefs(child)); if (NS_SUCCEEDED(rv)) SetNewsrcHasChanged(PR_TRUE); // subscribe UI does this - but maybe we got here through auto-subscribe - if(NS_SUCCEEDED(rv) && child) + if(NS_SUCCEEDED(rv) && child){ + child->SetCharset(dataCharset.get()); NotifyItemAdded(child); - + } return rv; } NS_IMETHODIMP nsMsgNewsFolder::Delete() { nsresult rv = GetDatabase(nsnull); if(NS_SUCCEEDED(rv)) @@ -1124,27 +1146,42 @@ nsMsgNewsFolder::HandleNewsrcLine(char* // // So lines like this in a newsrc file should be ignored: // 3746EF3F.6080309@netscape.com: // 3746EF3F.6080309%40netscape.com: if (PL_strstr(line,"@") || PL_strstr(line,"%40")) // skipping, it contains @ or %40 subscribed = PR_FALSE; + nsCOMPtr<nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + if (NS_FAILED(rv)) return rv; + //now we convert from UTF8 to the server charset encoding because the newsrc file is always saved in UTF8 + nsAutoString unicodeLine; + nsCAutoString rawLine; + CopyUTF8toUTF16(line,unicodeLine); + + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ASSERTION(NS_SUCCEEDED(rv),"failed get sever charset"); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), unicodeLine, rawLine); + if (NS_FAILED(rv)) + LossyCopyUTF16toASCII(unicodeLine,rawLine); + if (subscribed) { // we're subscribed, so add it nsCOMPtr <nsIMsgFolder> child; - rv = AddNewsgroup(line, setStr, getter_AddRefs(child)); + rv = AddNewsgroup(rawLine.get(), setStr, getter_AddRefs(child)); if (NS_FAILED(rv)) return -1; } else { - rv = RememberUnsubscribedGroup(line, setStr); + rv = RememberUnsubscribedGroup(rawLine.get(), setStr); if (NS_FAILED(rv)) return -1; } return 0; } nsresult @@ -1503,20 +1540,22 @@ nsresult nsMsgNewsFolder::CreateBaseMess NS_IMETHODIMP nsMsgNewsFolder::GetNewsrcLine(char **newsrcLine) { nsresult rv; if (!newsrcLine) return NS_ERROR_NULL_POINTER; + nsXPIDLString uninewsgroupname; nsXPIDLCString newsgroupname; - rv = GetAsciiName(getter_Copies(newsgroupname)); + rv = GetName(getter_Copies(uninewsgroupname)); if (NS_FAILED(rv)) return rv; - + CopyUTF16toUTF8(uninewsgroupname, newsgroupname); + nsCAutoString newsrcLineStr; newsrcLineStr = newsgroupname; newsrcLineStr += ':'; if (mReadSet) { nsXPIDLCString setStr; mReadSet->Output(getter_Copies(setStr)); if (NS_SUCCEEDED(rv)) @@ -1575,38 +1614,62 @@ nsMsgNewsFolder::GetOptionLines(char **o } NS_IMETHODIMP nsMsgNewsFolder::OnReadChanged(nsIDBChangeListener * aInstigator) { return SetNewsrcHasChanged(PR_TRUE); } +NS_IMETHODIMP +nsMsgNewsFolder::GetUnicodeName(nsAString & name) +{ + nsresult rv; + if(mUnicodeName.IsEmpty()){ + nsXPIDLString newsgroupName; + rv = GetName(getter_Copies(newsgroupName)); + NS_ENSURE_SUCCESS(rv,rv); + mUnicodeName = newsgroupName; + } + name = mUnicodeName; + return NS_OK; +} NS_IMETHODIMP -nsMsgNewsFolder::GetAsciiName(char **asciiName) +nsMsgNewsFolder::GetRawName(char **rawName) { nsresult rv; - NS_ENSURE_ARG_POINTER(asciiName); - if (!mAsciiName) + NS_ENSURE_ARG_POINTER(rawName); + if (!mRawName) { nsXPIDLString name; rv = GetName(getter_Copies(name)); NS_ENSURE_SUCCESS(rv,rv); // convert to ASCII nsCAutoString tmpStr; - tmpStr.AssignWithConversion(name); + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + NS_ENSURE_SUCCESS(rv,rv); + + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ENSURE_SUCCESS(rv,rv); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), name, tmpStr); + + if (NS_FAILED(rv)) { + LossyCopyUTF16toASCII(name,tmpStr); + } - mAsciiName = nsCRT::strdup(tmpStr.get()); - if (!mAsciiName) return NS_ERROR_OUT_OF_MEMORY; + mRawName = nsCRT::strdup(tmpStr.get()); + if (!mRawName) return NS_ERROR_OUT_OF_MEMORY; } - *asciiName = nsCRT::strdup(mAsciiName); - if (!*asciiName) return NS_ERROR_OUT_OF_MEMORY; + *rawName = nsCRT::strdup(mRawName); + if (!*rawName) return NS_ERROR_OUT_OF_MEMORY; return NS_OK; } NS_IMETHODIMP nsMsgNewsFolder::GetNntpServer(nsINntpIncomingServer **result) { nsresult rv; Only in mozilla/mailnews/news/src: nsNewsFolder.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNewsFolder.h mozilla/mailnews/news/src/nsNewsFolder.h --- mailnews.old/news/src/nsNewsFolder.h 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsFolder.h 2005-04-19 08:33:47.599810104 +0800 @@ -154,18 +154,18 @@ protected: nsMsgKeySet *mReadSet; nsCOMPtr<nsIFileSpec> mNewsrcFilePath; // used for auth news char *mGroupUsername; char *mGroupPassword; - // the name of the newsgroup. - char *mAsciiName; + char *mRawName; + nsString mUnicodeName; PRInt32 mSortOrder; private: nsresult CreateNewsgroupUsernameUrlForSignon(const char *inUriStr, char **result); nsresult CreateNewsgroupPasswordUrlForSignon(const char *inUriStr, char **result); nsresult CreateNewsgroupUrlForSignon(const char *inUriStr, const char *ref, char **result); nsCOMPtr <nsIFileSpec> mFilterFile; Only in mozilla/mailnews/news/src: nsNewsFolder.h~ diff -U 8 -p -r mailnews.old/news/src/nsNntpIncomingServer.cpp mozilla/mailnews/news/src/nsNntpIncomingServer.cpp --- mailnews.old/news/src/nsNntpIncomingServer.cpp 2004-08-27 06:16:52.000000000 +0800 +++ mozilla/mailnews/news/src/nsNntpIncomingServer.cpp 2005-04-19 08:33:31.246296216 +0800 @@ -57,22 +57,25 @@ #include "nsMsgUtils.h" #include "nsIPrompt.h" #include "nsIStringBundle.h" #include "nntpCore.h" #include "nsIWindowWatcher.h" #include "nsITreeColumns.h" #include "nsIDOMElement.h" #include "nsMsgFolderFlags.h" +#include "nsMsgI18N.h" +#include "nsIPrefLocalizedString.h" #define INVALID_VERSION 0 #define VALID_VERSION 1 #define NEW_NEWS_DIR_NAME "News" #define PREF_MAIL_NEWSRC_ROOT "mail.newsrc_root" #define PREF_MAIL_NEWSRC_ROOT_REL "mail.newsrc_root-rel" +#define PREF_MAILNEWS_VIEW_DEFAULT_CHARSET "mailnews.view_default_charset" #define HOSTINFO_FILE_NAME "hostinfo.dat" #define NEWS_DELIMITER '.' // this platform specific junk is so the newsrc filenames we create // will resemble the migrated newsrc filenames. #if defined(XP_UNIX) || defined(XP_BEOS) #define NEWSRC_FILE_PREFIX "newsrc-" @@ -304,16 +307,70 @@ nsresult nsNntpIncomingServer::SetupNews mNewsrcSaveTimer = do_CreateInstance("@mozilla.org/timer;1"); mNewsrcSaveTimer->InitWithFuncCallback(OnNewsrcSaveTimer, (void*)this, timeInMSUint32, nsITimer::TYPE_REPEATING_SLACK); return NS_OK; } NS_IMETHODIMP +nsNntpIncomingServer::SetCharset(const nsACString & aCharset) +{ + nsresult rv; + rv = SetCharValue("charset", PromiseFlatCString(aCharset).get()); + return rv; +} + +NS_IMETHODIMP +nsNntpIncomingServer::GetCharset(nsACString & aCharset) +{ + nsresult rv; + nsXPIDLCString serverCharset; + //first we get the per-server settings mail.server.<serverkey>.charset + rv = GetCharValue("charset",getter_Copies(serverCharset)); + + //if the per-server setting is empty,we get the default charset from + //mailnews.view_default_charset setting and set it as per-server preference. + if(serverCharset.IsEmpty()){ + nsXPIDLCString defaultCharset; + nsCOMPtr<nsIPrefBranch> prefBranch(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv)); + + if (NS_SUCCEEDED(rv)){ + nsCOMPtr<nsIPrefLocalizedString> pls; + rv = prefBranch->GetComplexValue(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, + NS_GET_IID(nsIPrefLocalizedString), getter_AddRefs(pls)); + if (NS_SUCCEEDED(rv)) + { + nsXPIDLString ucsval; + pls->ToString(getter_Copies(ucsval)); + if (ucsval) + { + LossyCopyUTF16toASCII(ucsval, defaultCharset); + } + } + } + + if(defaultCharset.IsEmpty()){ + serverCharset = "ISO-8859-1"; + }else{ + serverCharset = defaultCharset; + } + +#ifdef DEBUG_NEWS + printf("default charset for the server is %s\n", (const char *)defaultCharset); +#endif + + SetCharset(serverCharset); + } + + aCharset = serverCharset; + return NS_OK; +} + +NS_IMETHODIMP nsNntpIncomingServer::WriteNewsrcFile() { nsresult rv; PRBool newsrcHasChanged; rv = GetNewsrcHasChanged(&newsrcHasChanged); if (NS_FAILED(rv)) return rv; @@ -806,17 +863,17 @@ writeGroupToHostInfoFile(nsCString &aEle nsAutoString name; nsresult rv = NS_MsgDecodeUnescapeURLPath(aElement, name); if (NS_FAILED(rv)) { // stop, something is bad. return PR_FALSE; } nsCAutoString nameOnDisk; - nameOnDisk.AssignWithConversion(name.get()); + CopyUTF16toUTF8(name, nameOnDisk); // XXX todo ",,1,0,0" is a temporary hack, fix it *stream << nameOnDisk.get() << ",,1,0,0" << MSG_LINEBREAK; return PR_TRUE; } nsresult nsNntpIncomingServer::WriteHostInfoFile() @@ -1128,19 +1185,41 @@ nsNntpIncomingServer::UpdateSubscribed() NS_IMETHODIMP nsNntpIncomingServer::AddTo(const char *aName, PRBool addAsSubscribed, PRBool aSubscribable, PRBool changeIfExists) { nsresult rv = EnsureInner(); NS_ENSURE_SUCCESS(rv,rv); nsAutoString newsgroupName; - newsgroupName.AssignWithConversion(aName); + char *escapedName = nsnull; + + nsCOMPtr<nsIMsgFolder> rootFolder; + rv = GetRootFolder(getter_AddRefs(rootFolder)); + NS_ENSURE_SUCCESS(rv,rv); + if (!rootFolder) return NS_ERROR_FAILURE; + nsCOMPtr <nsIMsgNewsFolder> newsFolder = do_QueryInterface(rootFolder, &rv); + NS_ENSURE_SUCCESS(rv,rv); + if (!newsFolder) return NS_ERROR_FAILURE; + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = newsFolder->GetNntpServer(getter_AddRefs(nntpServer)); + NS_ENSURE_SUCCESS(rv,rv); + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ENSURE_SUCCESS(rv,rv); + rv = nsMsgI18NConvertToUnicode(dataCharset.get(),nsDependentCString(aName),newsgroup Name); + if (NS_SUCCEEDED(rv)) { + escapedName = nsEscape(NS_ConvertUTF16toUTF8(newsgroupName).get(), url_Path); + } + + if (NS_FAILED(rv)) { + CopyASCIItoUTF16(aName, newsgroupName); + escapedName = nsEscape(aName, url_Path); + } - char *escapedName = nsEscape(NS_ConvertUCS2toUTF8(newsgroupName.get()).get(), url_Path); if (!escapedName) return NS_ERROR_OUT_OF_MEMORY; rv = AddGroupOnServer(escapedName); NS_ENSURE_SUCCESS(rv,rv); rv = mInner->AddTo(escapedName,addAsSubscribed, aSubscribable, changeIfExists); NS_ENSURE_SUCCESS(rv,rv); @@ -1245,17 +1324,27 @@ nsNntpIncomingServer::HandleLine(char* l if (line[0] == '#' || line[0] == '\0') return 0; line[line_size] = 0; if (mHasSeenBeginGroups) { char *commaPos = PL_strchr(line,','); if (commaPos) *commaPos = 0; - nsresult rv = AddTo(line, PR_FALSE, PR_TRUE, PR_TRUE); + //now we convert from UTF8 to the server charset encoding because the hostInfo.dat is always saved in UTF8 + nsAutoString unicodeLine; + nsCAutoString encodedLine; + CopyUTF8toUTF16(line,unicodeLine); + nsCAutoString dataCharset; + nsresult rv = GetCharset(dataCharset); + NS_ASSERTION(NS_SUCCEEDED(rv),"failed to get sever charset"); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), unicodeLine, encodedLine); + if (NS_FAILED(rv)) + LossyCopyUTF16toASCII(unicodeLine,encodedLine); + rv = AddTo(encodedLine.get(), PR_FALSE, PR_TRUE, PR_TRUE); NS_ASSERTION(NS_SUCCEEDED(rv),"failed to add line"); if (NS_SUCCEEDED(rv)) { // since we've seen one group, we can claim we've loaded the hostinfo file mHostInfoLoaded = PR_TRUE; } } else { if (nsCRT::strncmp(line,"begingroups", 11) == 0) { Only in mozilla/mailnews/news/src: nsNntpIncomingServer.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNNTPNewsgroupList.cpp mozilla/mailnews/news/src/nsNNTPNewsgroupList.cpp --- mailnews.old/news/src/nsNNTPNewsgroupList.cpp 2005-02-01 05:53:40.000000000 +0800 +++ mozilla/mailnews/news/src/nsNNTPNewsgroupList.cpp 2005-04-13 13:55:22.000000000 +0800 @@ -319,21 +319,21 @@ nsNNTPNewsgroupList::GetRangeOfArtsToDow nsCOMPtr<nsINewsDownloadDialogArgs> args = do_CreateInstance("@mozilla.org/messenger/newsdownloaddialogargs;1", &rv); if (NS_FAILED(rv)) return rv; NS_ENSURE_SUCCESS(rv,rv); rv = args->SetArticleCount(*last - *first + 1); NS_ENSURE_SUCCESS(rv,rv); - nsXPIDLCString groupName; - rv = m_newsFolder->GetAsciiName(getter_Copies(groupName)); + nsXPIDLString groupName; + rv = m_newsFolder->GetUnicodeName(groupName); NS_ENSURE_SUCCESS(rv,rv); - rv = args->SetGroupName((const char *)groupName); + rv = args->SetGroupName(groupName); NS_ENSURE_SUCCESS(rv,rv); // get the server key nsXPIDLCString serverKey; rv = server->GetKey(getter_Copies(serverKey)); NS_ENSURE_SUCCESS(rv,rv); rv = args->SetServerKey((const char *)serverKey); Only in mozilla/mailnews/news/src: nsNNTPNewsgroupList.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNNTPProtocol.cpp mozilla/mailnews/news/src/nsNNTPProtocol.cpp --- mailnews.old/news/src/nsNNTPProtocol.cpp 2005-02-16 11:18:46.000000000 +0800 +++ mozilla/mailnews/news/src/nsNNTPProtocol.cpp 2005-04-19 03:36:57.000000000 +0800 @@ -1934,17 +1934,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom char *command=0; PRInt32 status = 0; if (m_typeWanted == ARTICLE_WANTED) { if (m_key != nsMsgKey_None) { nsresult rv; nsXPIDLCString newsgroupName; if (m_newsFolder) { - rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); + rv = m_newsFolder->GetRawName(getter_Copies(newsgroupName)); NS_ENSURE_SUCCESS(rv,rv); } PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) current group = %s, desired group = %s", this, m_currentGroup.get(), newsgroupName.get())); // if the current group is the desired group, we can just issue the ARTICLE command // if not, we have to do a GROUP first if (!PL_strcmp(m_currentGroup.get(), newsgroupName.get())) m_nextState = NNTP_SEND_ARTICLE_NUMBER; @@ -2045,17 +2045,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom else if(m_typeWanted == GROUP_WANTED) { nsresult rv = NS_ERROR_NULL_POINTER; NS_ASSERTION(m_newsFolder, "m_newsFolder is null, panic!"); if (!m_newsFolder) return -1; nsXPIDLCString group_name; - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); NS_ASSERTION(NS_SUCCEEDED(rv),"failed to get newsgroup name"); if (NS_FAILED(rv)) return -1; m_firstArticle = 0; m_lastArticle = 0; NS_MsgSACopy(&command, "GROUP "); NS_MsgSACat(&command, (const char *)group_name); @@ -2090,17 +2090,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) doing GROUP for XPAT", this)); nsXPIDLCString group_name; /* for XPAT, we have to GROUP into the group before searching */ if (!m_newsFolder) { NNTP_LOG_NOTE("m_newsFolder is null, panic!"); return -1; } - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); if (NS_FAILED(rv)) return -1; NS_MsgSACopy(&command, "GROUP "); NS_MsgSACat (&command, group_name); // force a GROUP next time m_currentGroup.Truncate(); m_nextState = NNTP_RESPONSE; @@ -2201,17 +2201,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom } else { nsresult rv = NS_OK; nsXPIDLCString group_name; NS_ASSERTION(m_newsFolder, "no newsFolder"); if (m_newsFolder) { - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); } if (m_responseCode == MK_NNTP_RESPONSE_GROUP_NO_GROUP && m_typeWanted == GROUP_WANTED) { PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) group (%s) not found, so unset m_currentGroup",this,(const char *)group_name)); m_currentGroup.Truncate(); m_nntpServer->GroupNotFound(m_msgWindow, group_name.get(), PR_TRUE /* opening */); @@ -2316,17 +2316,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom } PRInt32 nsNNTPProtocol::SendGroupForArticle() { nsresult rv; PRInt32 status = 0; nsXPIDLCString groupname; - rv = m_newsFolder->GetAsciiName(getter_Copies(groupname)); + rv = m_newsFolder->GetRawName(getter_Copies(groupname)); NS_ASSERTION(NS_SUCCEEDED(rv) && groupname.get() && groupname.get()[0], "no group name"); char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "GROUP %.512s" CRLF, (const char *)groupname); @@ -2348,17 +2348,17 @@ nsNNTPProtocol::SetCurrentGroup() nsresult rv; nsXPIDLCString groupname; NS_ASSERTION(m_newsFolder, "no news folder"); if (!m_newsFolder) { m_currentGroup.Truncate(); return NS_ERROR_UNEXPECTED; } - rv = m_newsFolder->GetAsciiName(getter_Copies(groupname)); + rv = m_newsFolder->GetRawName(getter_Copies(groupname)); NS_ASSERTION(NS_SUCCEEDED(rv) && groupname.get()[0], "no group name"); PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) SetCurrentGroup to %s",this,(const char *)groupname)); m_currentGroup = groupname; return NS_OK; } PRInt32 nsNNTPProtocol::SendGroupForArticleResponse() { @@ -3873,17 +3873,17 @@ PRInt32 nsNNTPProtocol::DisplayNewsRC() if (NS_FAILED(rv)) return -1; if (!currFolder) return -1; m_newsFolder = do_QueryInterface(currFolder, &rv); if (NS_FAILED(rv)) return -1; if (!m_newsFolder) return -1; nsXPIDLCString name; - rv = m_newsFolder->GetAsciiName(getter_Copies(name)); + rv = m_newsFolder->GetRawName(getter_Copies(name)); if (NS_FAILED(rv)) return -1; if (!name) return -1; /* send group command to server */ char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "GROUP %.512s" CRLF, (const char *)name); nsCOMPtr<nsIMsgMailNewsUrl> mailnewsurl = do_QueryInterface(m_runningURL); @@ -3963,17 +3963,17 @@ PRInt32 nsNNTPProtocol::DisplayNewsRCRes NS_ASSERTION(NS_SUCCEEDED(rv),"DisplaySubscribedGroup() failed"); if (NS_FAILED(rv)) status = -1; if (status < 0) return status; } else if (m_responseCode == MK_NNTP_RESPONSE_GROUP_NO_GROUP) { nsXPIDLCString name; - rv = m_newsFolder->GetAsciiName(getter_Copies(name)); + rv = m_newsFolder->GetRawName(getter_Copies(name)); if (NS_SUCCEEDED(rv)) { m_nntpServer->GroupNotFound(m_msgWindow, name.get(), PR_FALSE); } PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) NO_GROUP, so unset m_currentGroup", this)); m_currentGroup.Truncate(); } @@ -4394,17 +4394,17 @@ PRInt32 nsNNTPProtocol::XPATResponse(nsI PRInt32 nsNNTPProtocol::ListPrettyNames() { nsXPIDLCString group_name; char outputBuffer[OUTPUT_BUFFER_SIZE]; PRInt32 status = 0; - nsresult rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + nsresult rv = m_newsFolder->GetRawName(getter_Copies(group_name)); PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "LIST PRETTYNAMES %.512s" CRLF, NS_SUCCEEDED(rv) ? (const char *) group_name : ""); nsCOMPtr<nsIMsgMailNewsUrl> mailnewsurl = do_QueryInterface(m_runningURL); if (mailnewsurl) status = SendData(mailnewsurl, outputBuffer); @@ -4471,17 +4471,17 @@ PRInt32 nsNNTPProtocol::ListPrettyNamesR PR_FREEIF(line); return 0; } PRInt32 nsNNTPProtocol::ListXActive() { nsXPIDLCString group_name; nsresult rv; - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); if (NS_FAILED(rv)) return -1; PRInt32 status = 0; char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "LIST XACTIVE %.512s" CRLF, @@ -4624,17 +4624,17 @@ PRInt32 nsNNTPProtocol::SendListGroup() nsresult rv; char outputBuffer[OUTPUT_BUFFER_SIZE]; PRInt32 status = 0; NS_ASSERTION(m_newsFolder,"no newsFolder"); if (!m_newsFolder) return -1; nsXPIDLCString newsgroupName; - rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); + rv = m_newsFolder->GetRawName(getter_Copies(newsgroupName)); NS_ENSURE_SUCCESS(rv,rv); PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "listgroup %.512s" CRLF, newsgroupName.get()); m_articleList = do_CreateInstance(NS_NNTPARTICLELIST_CONTRACTID, &rv); Only in mozilla/mailnews/news/src: nsNNTPProtocol.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNntpService.cpp mozilla/mailnews/news/src/nsNntpService.cpp --- mailnews.old/news/src/nsNntpService.cpp 2005-01-18 02:50:13.000000000 +0800 +++ mozilla/mailnews/news/src/nsNntpService.cpp 2005-04-13 13:56:02.000000000 +0800 @@ -650,18 +650,21 @@ nsNntpService::GetFolderFromUri(const ch // but the *name* of the newsgroup (we are calling ::GetChildNamed()) // is unescaped. see http://bugzilla.mozilla.org/show_bug.cgi?id=210089#c17 // for more about this char *unescapedPath = PL_strdup(path.get() + 1); /* skip the leading slash */ if (!unescapedPath) return NS_ERROR_OUT_OF_MEMORY; nsUnescape(unescapedPath); + nsAutoString newsgroupName; + //the encoding is UTF-8 after unescaped. + CopyUTF8toUTF16(unescapedPath,newsgroupName); nsCOMPtr<nsISupports> subFolder; - rv = rootFolder->GetChildNamed(NS_ConvertASCIItoUCS2(unescapedPath).get() , getter_AddRefs(subFolder)); + rv = rootFolder->GetChildNamed(newsgroupName.get() , getter_AddRefs(subFolder)); PL_strfree(unescapedPath); NS_ENSURE_SUCCESS(rv,rv); return CallQueryInterface(subFolder, aFolder); } NS_IMETHODIMP nsNntpService::CopyMessage(const char * aSrcMessageURI, nsIStreamListener * aMailboxCopyHandler, PRBool moveMessage,

Comment on attachment 170507 [details] [diff] [review] patch use utf8 in the newsrc/hostinfo file diff -U 8 -p -r mailnews.old/base/prefs/resources/content/am-server.xul mozilla/mailnews/base/prefs/resources/content/am-server.xul --- mailnews.old/base/prefs/resources/content/am-server.xul 2005-03-04 02:08:45.000000000 +0800 +++ mozilla/mailnews/base/prefs/resources/content/am-server.xul 2005-04-13 13:20:05.000000000 +0800 @@ -47,16 +47,20 @@ <page xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul" onload="parent.onPanelLoaded('am-server.xul');"> <script type="application/x-javascript" src="chrome://messenger/content/AccountManager.js"/> <script type="application/x-javascript" src="chrome://messenger/content/am-server.js"/> <script type="application/x-javascript" src="chrome://messenger/content/am-prefs.js"/> <script type="application/x-javascript" src="chrome://messenger/content/amUtils.js"/> + <script type="application/x-javascript"><![CDATA[ + var observerService = Components.classes["@mozilla.org/observer-service;1"].getService(Components.int erfaces.nsIObserverService); + observerService.notifyObservers(null, "charsetmenu-selected", "other"); + ]]></script> <stringbundle id="bundle_messenger" src="chrome://messenger/locale/messenger.properties"/> <broadcaster id="broadcaster_doBiff"/> <broadcaster id="broadcaster_notify"/> <label hidden="true" wsm_persist="true" id="server.type"/> <dialogheader title="&serverSettings.label;"/> @@ -325,9 +329,22 @@ accesskey="&localPath.accesskey;"/> <hbox align="center"> <textbox readonly="true" wsm_persist="true" flex="1" id="server.localPath" datatype="nsIFileSpec" prefstring="mail.server.%serverkey%.directory" class="uri-element"/> <button id="browseForLocalFolder" label="&browseFolder.label;" filepickertitle="&localFolderPicker.label;" accesskey="&browseFolder.accesskey;" oncommand="BrowseForLocalFolders()"/> </hbox> </vbox> + <separator class="thin"/> + <hbox hidefor="imap,pop3,movemail" align="center" valign="middle" iscontrolcontainer="true"> + <label value="&serverDefaultCharset.label;" control="nntp.charset"/> + <menulist hidable="true" hidefor="imap,pop3,movemail" wsm_persist="true" id="nntp.charset" + ref="NC:DecodersRoot" datasources="rdf:charset-menu" + preftype="string" prefstring="mail.server.%serverkey%.charset"> + <template> + <menupopup> + <menuitem label="rdf:http://home.netscape.com/NC-rdf#Name" value="..." uri="..."/> + </menupopup> + </template> + </menulist> + </hbox> </page> Only in mozilla/mailnews/base/prefs/resources/content: am-server.xul~ diff -U 8 -p -r mailnews.old/base/prefs/resources/locale/en-US/am-server-top.dtd mozilla/mailnews/base/prefs/resources/locale/en-US/am-server-top.dtd --- mailnews.old/base/prefs/resources/locale/en-US/am-server-top.dtd 2005-02-09 01:01:47.000000000 +0800 +++ mozilla/mailnews/base/prefs/resources/locale/en-US/am-server-top.dtd 2005-04-13 13:19:30.000000000 +0800 @@ -79,15 +79,16 @@ <!ENTITY newsrcFilePath.label "newsrc file:"> <!ENTITY newsrcFilePath.accesskey "f"> <!ENTITY localPath.label "Local directory:"> <!ENTITY localPath.accesskey "L"> <!ENTITY abbreviate.label "Show newsgroup names in the Mail Folder pane as:"> <!ENTITY abbreviateOn.label "Full names (For example, 'netscape.public.mozilla.mail-news')"> <!ENTITY abbreviateOff.label "Abbreviate names (For example, 'n.p.m.mail-news')"> <!ENTITY advancedButton.label "Advanced..."> +<!ENTITY serverDefaultCharset.label "Default Character Encoding:"> <!ENTITY advancedButton.accesskey "v"> <!ENTITY localFolderPicker.label "Select Local Directory"> <!ENTITY browseFolder.label "Browse..."> <!ENTITY browseFolder.accesskey "B"> <!ENTITY newsrcPicker.label "Select newsrc File"> <!ENTITY browseNewsrc.label "Browse..."> <!ENTITY browseNewsrc.accesskey "e"> diff -U 8 -p -r mailnews.old/news/public/nsIMsgNewsFolder.idl mozilla/mailnews/news/public/nsIMsgNewsFolder.idl --- mailnews.old/news/public/nsIMsgNewsFolder.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsIMsgNewsFolder.idl 2005-04-19 03:37:37.000000000 +0800 @@ -36,27 +36,28 @@ * ***** END LICENSE BLOCK ***** */ #include "nsISupports.idl" #include "nsIMsgFolder.idl" interface nsIMsgWindow; interface nsINntpIncomingServer; -[scriptable, uuid(3716abe4-f6d4-11d2-86d5-004005263078)] +[scriptable, uuid(02699533-7ba8-4e78-94dc-a20374fb2711)] interface nsIMsgNewsFolder : nsISupports { attribute string groupUsername; attribute string groupPassword; - /** the ascii name for the newsgroup. - * newsgroup names are always ascii, and this can help simply - * the code, to prevent use from having to get the name (as a wstring) - * and then converting it back to ascii. + readonly attribute AString unicodeName; + /** the raw name name for the newsgroup (server side name). + * newsgroup names are might not always ascii(bug#126453), and this raw name name + * can help simply the code, to prevent use from having to get the unicode name + * (as a wstring) and then converting it back to server side raw name. */ - readonly attribute string asciiName; + readonly attribute string rawName; readonly attribute nsINntpIncomingServer nntpServer; attribute boolean saveArticleOffline; string getGroupPasswordWithUI(in wstring aPromptString, in wstring aPromptTitle, in nsIMsgWindow aMsgWindow); string getGroupUsernameWithUI(in wstring aPromptString, in wstring aPromptTitle, in nsIMsgWindow aMsgWindow); void forgetGroupUsername(); void forgetGroupPassword(); Only in mozilla/mailnews/news/public: nsIMsgNewsFolder.idl~ diff -U 8 -p -r mailnews.old/news/public/nsINewsDownloadDialogArgs.idl mozilla/mailnews/news/public/nsINewsDownloadDialogArgs.idl --- mailnews.old/news/public/nsINewsDownloadDialogArgs.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsINewsDownloadDialogArgs.idl 2005-04-13 13:22:00.000000000 +0800 @@ -33,19 +33,19 @@ * and other provisions required by the GPL or the LGPL. If you do not delete * the provisions above, a recipient may use your version of this file under * the terms of any one of the MPL, the GPL or the LGPL. * * ***** END LICENSE BLOCK ***** */ #include "nsISupports.idl" -[scriptable, uuid(b5037f06-1dd1-11b2-bf61-fcab5b5837a7)] +[scriptable, uuid(3634327c-392b-4686-adf5-576e6cef9196)] interface nsINewsDownloadDialogArgs: nsISupports { - attribute string groupName; + attribute AString groupName; attribute long articleCount; attribute string serverKey; attribute boolean hitOK; attribute boolean downloadAll; }; %{ C++ #define DOWNLOAD_HEADERS_URL "chrome://messenger/content/downloadheaders.xul" diff -U 8 -p -r mailnews.old/news/public/nsINntpIncomingServer.idl mozilla/mailnews/news/public/nsINntpIncomingServer.idl --- mailnews.old/news/public/nsINntpIncomingServer.idl 2004-04-18 02:33:14.000000000 +0800 +++ mozilla/mailnews/news/public/nsINntpIncomingServer.idl 2005-04-13 13:22:54.000000000 +0800 @@ -61,16 +61,19 @@ interface nsINntpIncomingServer : nsISup attribute boolean markOldRead; /* abbreviate the newsgroup names in the folder pane? */ attribute boolean abbreviate; /* do we use a single login per server or do we login per group */ attribute boolean singleSignon; + /* the server charset and it may be needed to display newsgroup folder names correctly */ + attribute ACString charset; + /* the server keeps track of all the newsgroups we are subscribed to */ void addNewsgroup(in string name); void removeNewsgroup(in string name); void writeNewsrcFile(); attribute boolean newsrcHasChanged; diff -U 8 -p -r mailnews.old/news/src/nsNewsDownloadDialogArgs.cpp mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.cpp --- mailnews.old/news/src/nsNewsDownloadDialogArgs.cpp 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.cpp 2005-04-13 13:24:48.000000000 +0800 @@ -37,45 +37,41 @@ * ***** END LICENSE BLOCK ***** */ #include "nsNewsDownloadDialogArgs.h" #include "nsCRT.h" #include "nsReadableUtils.h" nsNewsDownloadDialogArgs::nsNewsDownloadDialogArgs() { - mGroupName = ""; mArticleCount = 0; mServerKey = ""; mHitOK = PR_FALSE; mDownloadAll = PR_FALSE; } nsNewsDownloadDialogArgs::~nsNewsDownloadDialogArgs() { } NS_IMPL_ISUPPORTS1(nsNewsDownloadDialogArgs, nsINewsDownloadDialogArgs) -NS_IMETHODIMP nsNewsDownloadDialogArgs::GetGroupName(char * *aGroupName) -{ - NS_ENSURE_ARG_POINTER(aGroupName); - - *aGroupName = ToNewCString(mGroupName); - - return NS_OK; -} -NS_IMETHODIMP nsNewsDownloadDialogArgs::SetGroupName(const char * aGroupName) -{ - NS_ENSURE_ARG_POINTER(aGroupName); - - mGroupName = aGroupName; - - return NS_OK; -} +NS_IMETHODIMP nsNewsDownloadDialogArgs::GetGroupName(nsAString & aGroupName) + { + aGroupName = mGroupName; + + return NS_OK; + } +NS_IMETHODIMP nsNewsDownloadDialogArgs::SetGroupName(const nsAString & aGroupName) + { + + mGroupName = aGroupName; + + return NS_OK; + } NS_IMETHODIMP nsNewsDownloadDialogArgs::GetArticleCount(PRInt32 *aArticleCount) { NS_ENSURE_ARG_POINTER(aArticleCount); *aArticleCount = mArticleCount; return NS_OK; } diff -U 8 -p -r mailnews.old/news/src/nsNewsDownloadDialogArgs.h mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.h --- mailnews.old/news/src/nsNewsDownloadDialogArgs.h 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsDownloadDialogArgs.h 2005-04-13 13:25:12.000000000 +0800 @@ -47,16 +47,16 @@ class nsNewsDownloadDialogArgs : public public: nsNewsDownloadDialogArgs(); virtual ~nsNewsDownloadDialogArgs(); NS_DECL_ISUPPORTS NS_DECL_NSINEWSDOWNLOADDIALOGARGS private: - nsCString mGroupName; + nsString mGroupName; PRInt32 mArticleCount; nsCString mServerKey; PRBool mHitOK; PRBool mDownloadAll; }; #endif // nsNewsDownloadDialogArgs_h__ diff -U 8 -p -r mailnews.old/news/src/nsNewsFolder.cpp mozilla/mailnews/news/src/nsNewsFolder.cpp --- mailnews.old/news/src/nsNewsFolder.cpp 2005-02-15 08:13:35.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsFolder.cpp 2005-04-19 08:33:14.976769560 +0800 @@ -88,16 +88,18 @@ #include "nsIInterfaceRequestor.h" #include "nsIInterfaceRequestorUtils.h" #include "nsReadableUtils.h" #include "nsNewsDownloader.h" #include "nsIStringBundle.h" #include "nsEscape.h" +#include "nsMsgI18N.h" +#include "nsNativeCharsetUtils.h" // we need this because of an egcs 1.0 (and possibly gcc) compiler bug // that doesn't allow you to call ::nsISupports::GetIID() inside of a class // that multiply inherits from nsISupports static NS_DEFINE_CID(kRDFServiceCID, NS_RDFSERVICE_CID); static NS_DEFINE_CID(kCNewsDB, NS_NEWSDB_CID); // ###tw This really ought to be the most @@ -112,33 +114,33 @@ static NS_DEFINE_CID(kCNewsDB, NS_NEWSDB /////////////////////////////////////////////////////////////////////////////// / nsMsgNewsFolder::nsMsgNewsFolder(void) : nsMsgLineBuffer(nsnull, PR_FALSE), mExpungedBytes(0), mGettingNews(PR_FALSE), mInitialized(PR_FALSE), mOptionLines(""), mUnsubscribedNewsgroupLines(""), m_downloadMessageForOfflineUse(PR_FALSE), m_downloadingMultipleMessages(PR_FALSE), - mReadSet(nsnull), mGroupUsername(nsnull), mGroupPassword(nsnull), mAsciiName(nsnull) + mReadSet(nsnull), mGroupUsername(nsnull), mGroupPassword(nsnull), mRawName(nsnull) { MOZ_COUNT_CTOR(nsNewsFolder); // double count these for now. /* we're parsing the newsrc file, and the line breaks are platform specific. * if MSG_LINEBREAK != CRLF, then we aren't looking for CRLF */ if (PL_strcmp(MSG_LINEBREAK, CRLF)) SetLookingForCRLF(PR_FALSE); } nsMsgNewsFolder::~nsMsgNewsFolder(void) { MOZ_COUNT_DTOR(nsNewsFolder); delete mReadSet; PR_Free(mGroupUsername); PR_Free(mGroupPassword); - PR_Free(mAsciiName); + PR_Free(mRawName); } NS_IMPL_ADDREF_INHERITED(nsMsgNewsFolder, nsMsgDBFolder) NS_IMPL_RELEASE_INHERITED(nsMsgNewsFolder, nsMsgDBFolder) NS_IMETHODIMP nsMsgNewsFolder::QueryInterface(REFNSIID aIID, void** aInstancePtr) { if (!aInstancePtr) return NS_ERROR_NULL_POINTER; @@ -205,20 +207,24 @@ nsMsgNewsFolder::AddNewsgroup(const char nsCOMPtr <nsINntpIncomingServer> nntpServer; rv = GetNntpServer(getter_AddRefs(nntpServer)); if (NS_FAILED(rv)) return rv; nsCAutoString uri(mURI); uri.Append('/'); // URI should use UTF-8 // (see RFC2396 Uniform Resource Identifiers (URI): Generic Syntax) - // since we are forcing it to be latin-1 (IS0-8859-1) - // we can just assign with conversion + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + if (NS_FAILED(rv)) return rv; nsAutoString newsgroupName; - newsgroupName.AssignWithConversion(name); + rv = nsMsgI18NConvertToUnicode(dataCharset.get(),nsDependentCString(name),newsgroupN ame); + if (NS_FAILED(rv)) { + CopyASCIItoUTF16(name, newsgroupName); + } nsCAutoString escapedName; rv = NS_MsgEscapeEncodeURLPath(newsgroupName, escapedName); if (NS_FAILED(rv)) return rv; rv = nntpServer->AddNewsgroup(escapedName.get()); if (NS_FAILED(rv)) return rv; @@ -511,32 +517,35 @@ nsMsgNewsFolder::GetMessages(nsIMsgWindo } NS_IMETHODIMP nsMsgNewsFolder::GetFolderURL(char **url) { NS_ENSURE_ARG_POINTER(url); nsXPIDLCString hostName; nsresult rv = GetHostname(getter_Copies(hostName)); - nsXPIDLCString groupName; - rv = GetAsciiName(getter_Copies(groupName)); + nsXPIDLString groupName; + rv = GetName(getter_Copies(groupName)); NS_ENSURE_SUCCESS(rv, rv); nsCOMPtr<nsIMsgIncomingServer> server; rv = GetServer(getter_AddRefs(server)); NS_ENSURE_SUCCESS(rv, rv); PRInt32 port; PRBool isSecure = PR_FALSE; rv = server->GetIsSecure(&isSecure); NS_ENSURE_SUCCESS(rv, rv); rv = server->GetPort(&port); NS_ENSURE_SUCCESS(rv, rv); const char *newsScheme = (isSecure) ? SNEWS_SCHEME : NEWS_SCHEME; - *url = PR_smprintf("%s//%s:%ld/%s", newsScheme, hostName.get(), port, groupName.get()); + nsXPIDLCString escapedName; + rv = NS_MsgEscapeEncodeURLPath(groupName, escapedName); + NS_ENSURE_SUCCESS(rv, rv); + *url = PR_smprintf("%s//%s:%ld/%s", newsScheme, hostName.get(), port, escapedName.get()); return NS_OK; } NS_IMETHODIMP nsMsgNewsFolder::SetNewsrcHasChanged(PRBool newsrcHasChanged) { nsresult rv; @@ -549,18 +558,28 @@ NS_IMETHODIMP nsMsgNewsFolder::SetNewsrc NS_IMETHODIMP nsMsgNewsFolder::CreateSubfolder(const PRUnichar *uninewsgroupname, nsIMsgWindow *msgWindow) { nsresult rv = NS_OK; NS_ENSURE_ARG_POINTER(uninewsgroupname); if (!*uninewsgroupname) return NS_ERROR_FAILURE; - nsCAutoString newsgroupname; - newsgroupname.AssignWithConversion(uninewsgroupname); + nsCAutoString newsgroupname; + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + if (NS_FAILED(rv)) return rv; + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + if (NS_FAILED(rv)) return rv; + + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), nsDependentString(uninewsgroupname), newsgroupname); + if (NS_FAILED(rv)) { + LossyCopyUTF16toASCII(uninewsgroupname,newsgroupname); + } nsFileSpec path; nsCOMPtr<nsIFileSpec> pathSpec; rv = GetPath(getter_AddRefs(pathSpec)); if (NS_FAILED(rv)) return rv; rv = pathSpec->GetFileSpec(&path); @@ -569,29 +588,32 @@ NS_IMETHODIMP nsMsgNewsFolder::CreateSub // Create an empty database for this mail folder, set its name from the user nsCOMPtr<nsIMsgDatabase> newsDBFactory; nsCOMPtr <nsIMsgDatabase> newsDB; //Now we have a valid directory or we have returned. //Make sure the new folder name is valid // remember, some file systems (like mac) can't handle long file names - nsCAutoString hashedName = newsgroupname; + nsCAutoString fileName; + nsAutoString hashedName(uninewsgroupname); rv = NS_MsgHashIfNecessary(hashedName); - path += hashedName.get(); + NS_CopyUnicodeToNative(hashedName, fileName); + path += fileName.get(); //Now let's create the actual new folder rv = AddNewsgroup(newsgroupname.get(), "", getter_AddRefs(child)); if (NS_SUCCEEDED(rv)) SetNewsrcHasChanged(PR_TRUE); // subscribe UI does this - but maybe we got here through auto-subscribe - if(NS_SUCCEEDED(rv) && child) + if(NS_SUCCEEDED(rv) && child){ + child->SetCharset(dataCharset.get()); NotifyItemAdded(child); - + } return rv; } NS_IMETHODIMP nsMsgNewsFolder::Delete() { nsresult rv = GetDatabase(nsnull); if(NS_SUCCEEDED(rv)) @@ -1124,27 +1146,42 @@ nsMsgNewsFolder::HandleNewsrcLine(char* // // So lines like this in a newsrc file should be ignored: // 3746EF3F.6080309@netscape.com: // 3746EF3F.6080309%40netscape.com: if (PL_strstr(line,"@") || PL_strstr(line,"%40")) // skipping, it contains @ or %40 subscribed = PR_FALSE; + nsCOMPtr<nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + if (NS_FAILED(rv)) return rv; + //now we convert from UTF8 to the server charset encoding because the newsrc file is always saved in UTF8 + nsAutoString unicodeLine; + nsCAutoString rawLine; + CopyUTF8toUTF16(line,unicodeLine); + + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ASSERTION(NS_SUCCEEDED(rv),"failed get sever charset"); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), unicodeLine, rawLine); + if (NS_FAILED(rv)) + LossyCopyUTF16toASCII(unicodeLine,rawLine); + if (subscribed) { // we're subscribed, so add it nsCOMPtr <nsIMsgFolder> child; - rv = AddNewsgroup(line, setStr, getter_AddRefs(child)); + rv = AddNewsgroup(rawLine.get(), setStr, getter_AddRefs(child)); if (NS_FAILED(rv)) return -1; } else { - rv = RememberUnsubscribedGroup(line, setStr); + rv = RememberUnsubscribedGroup(rawLine.get(), setStr); if (NS_FAILED(rv)) return -1; } return 0; } nsresult @@ -1503,20 +1540,22 @@ nsresult nsMsgNewsFolder::CreateBaseMess NS_IMETHODIMP nsMsgNewsFolder::GetNewsrcLine(char **newsrcLine) { nsresult rv; if (!newsrcLine) return NS_ERROR_NULL_POINTER; + nsXPIDLString uninewsgroupname; nsXPIDLCString newsgroupname; - rv = GetAsciiName(getter_Copies(newsgroupname)); + rv = GetName(getter_Copies(uninewsgroupname)); if (NS_FAILED(rv)) return rv; - + CopyUTF16toUTF8(uninewsgroupname, newsgroupname); + nsCAutoString newsrcLineStr; newsrcLineStr = newsgroupname; newsrcLineStr += ':'; if (mReadSet) { nsXPIDLCString setStr; mReadSet->Output(getter_Copies(setStr)); if (NS_SUCCEEDED(rv)) @@ -1575,38 +1614,62 @@ nsMsgNewsFolder::GetOptionLines(char **o } NS_IMETHODIMP nsMsgNewsFolder::OnReadChanged(nsIDBChangeListener * aInstigator) { return SetNewsrcHasChanged(PR_TRUE); } +NS_IMETHODIMP +nsMsgNewsFolder::GetUnicodeName(nsAString & name) +{ + nsresult rv; + if(mUnicodeName.IsEmpty()){ + nsXPIDLString newsgroupName; + rv = GetName(getter_Copies(newsgroupName)); + NS_ENSURE_SUCCESS(rv,rv); + mUnicodeName = newsgroupName; + } + name = mUnicodeName; + return NS_OK; +} NS_IMETHODIMP -nsMsgNewsFolder::GetAsciiName(char **asciiName) +nsMsgNewsFolder::GetRawName(char **rawName) { nsresult rv; - NS_ENSURE_ARG_POINTER(asciiName); - if (!mAsciiName) + NS_ENSURE_ARG_POINTER(rawName); + if (!mRawName) { nsXPIDLString name; rv = GetName(getter_Copies(name)); NS_ENSURE_SUCCESS(rv,rv); // convert to ASCII nsCAutoString tmpStr; - tmpStr.AssignWithConversion(name); + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = GetNntpServer(getter_AddRefs(nntpServer)); + NS_ENSURE_SUCCESS(rv,rv); + + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ENSURE_SUCCESS(rv,rv); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), name, tmpStr); + + if (NS_FAILED(rv)) { + LossyCopyUTF16toASCII(name,tmpStr); + } - mAsciiName = nsCRT::strdup(tmpStr.get()); - if (!mAsciiName) return NS_ERROR_OUT_OF_MEMORY; + mRawName = nsCRT::strdup(tmpStr.get()); + if (!mRawName) return NS_ERROR_OUT_OF_MEMORY; } - *asciiName = nsCRT::strdup(mAsciiName); - if (!*asciiName) return NS_ERROR_OUT_OF_MEMORY; + *rawName = nsCRT::strdup(mRawName); + if (!*rawName) return NS_ERROR_OUT_OF_MEMORY; return NS_OK; } NS_IMETHODIMP nsMsgNewsFolder::GetNntpServer(nsINntpIncomingServer **result) { nsresult rv; Only in mozilla/mailnews/news/src: nsNewsFolder.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNewsFolder.h mozilla/mailnews/news/src/nsNewsFolder.h --- mailnews.old/news/src/nsNewsFolder.h 2004-04-18 02:33:16.000000000 +0800 +++ mozilla/mailnews/news/src/nsNewsFolder.h 2005-04-19 08:33:47.599810104 +0800 @@ -154,18 +154,18 @@ protected: nsMsgKeySet *mReadSet; nsCOMPtr<nsIFileSpec> mNewsrcFilePath; // used for auth news char *mGroupUsername; char *mGroupPassword; - // the name of the newsgroup. - char *mAsciiName; + char *mRawName; + nsString mUnicodeName; PRInt32 mSortOrder; private: nsresult CreateNewsgroupUsernameUrlForSignon(const char *inUriStr, char **result); nsresult CreateNewsgroupPasswordUrlForSignon(const char *inUriStr, char **result); nsresult CreateNewsgroupUrlForSignon(const char *inUriStr, const char *ref, char **result); nsCOMPtr <nsIFileSpec> mFilterFile; Only in mozilla/mailnews/news/src: nsNewsFolder.h~ diff -U 8 -p -r mailnews.old/news/src/nsNntpIncomingServer.cpp mozilla/mailnews/news/src/nsNntpIncomingServer.cpp --- mailnews.old/news/src/nsNntpIncomingServer.cpp 2004-08-27 06:16:52.000000000 +0800 +++ mozilla/mailnews/news/src/nsNntpIncomingServer.cpp 2005-04-19 08:33:31.246296216 +0800 @@ -57,22 +57,25 @@ #include "nsMsgUtils.h" #include "nsIPrompt.h" #include "nsIStringBundle.h" #include "nntpCore.h" #include "nsIWindowWatcher.h" #include "nsITreeColumns.h" #include "nsIDOMElement.h" #include "nsMsgFolderFlags.h" +#include "nsMsgI18N.h" +#include "nsIPrefLocalizedString.h" #define INVALID_VERSION 0 #define VALID_VERSION 1 #define NEW_NEWS_DIR_NAME "News" #define PREF_MAIL_NEWSRC_ROOT "mail.newsrc_root" #define PREF_MAIL_NEWSRC_ROOT_REL "mail.newsrc_root-rel" +#define PREF_MAILNEWS_VIEW_DEFAULT_CHARSET "mailnews.view_default_charset" #define HOSTINFO_FILE_NAME "hostinfo.dat" #define NEWS_DELIMITER '.' // this platform specific junk is so the newsrc filenames we create // will resemble the migrated newsrc filenames. #if defined(XP_UNIX) || defined(XP_BEOS) #define NEWSRC_FILE_PREFIX "newsrc-" @@ -304,16 +307,70 @@ nsresult nsNntpIncomingServer::SetupNews mNewsrcSaveTimer = do_CreateInstance("@mozilla.org/timer;1"); mNewsrcSaveTimer->InitWithFuncCallback(OnNewsrcSaveTimer, (void*)this, timeInMSUint32, nsITimer::TYPE_REPEATING_SLACK); return NS_OK; } NS_IMETHODIMP +nsNntpIncomingServer::SetCharset(const nsACString & aCharset) +{ + nsresult rv; + rv = SetCharValue("charset", PromiseFlatCString(aCharset).get()); + return rv; +} + +NS_IMETHODIMP +nsNntpIncomingServer::GetCharset(nsACString & aCharset) +{ + nsresult rv; + nsXPIDLCString serverCharset; + //first we get the per-server settings mail.server.<serverkey>.charset + rv = GetCharValue("charset",getter_Copies(serverCharset)); + + //if the per-server setting is empty,we get the default charset from + //mailnews.view_default_charset setting and set it as per-server preference. + if(serverCharset.IsEmpty()){ + nsXPIDLCString defaultCharset; + nsCOMPtr<nsIPrefBranch> prefBranch(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv)); + + if (NS_SUCCEEDED(rv)){ + nsCOMPtr<nsIPrefLocalizedString> pls; + rv = prefBranch->GetComplexValue(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, + NS_GET_IID(nsIPrefLocalizedString), getter_AddRefs(pls)); + if (NS_SUCCEEDED(rv)) + { + nsXPIDLString ucsval; + pls->ToString(getter_Copies(ucsval)); + if (ucsval) + { + LossyCopyUTF16toASCII(ucsval, defaultCharset); + } + } + } + + if(defaultCharset.IsEmpty()){ + serverCharset = "ISO-8859-1"; + }else{ + serverCharset = defaultCharset; + } + +#ifdef DEBUG_NEWS + printf("default charset for the server is %s\n", (const char *)defaultCharset); +#endif + + SetCharset(serverCharset); + } + + aCharset = serverCharset; + return NS_OK; +} + +NS_IMETHODIMP nsNntpIncomingServer::WriteNewsrcFile() { nsresult rv; PRBool newsrcHasChanged; rv = GetNewsrcHasChanged(&newsrcHasChanged); if (NS_FAILED(rv)) return rv; @@ -806,17 +863,17 @@ writeGroupToHostInfoFile(nsCString &aEle nsAutoString name; nsresult rv = NS_MsgDecodeUnescapeURLPath(aElement, name); if (NS_FAILED(rv)) { // stop, something is bad. return PR_FALSE; } nsCAutoString nameOnDisk; - nameOnDisk.AssignWithConversion(name.get()); + CopyUTF16toUTF8(name, nameOnDisk); // XXX todo ",,1,0,0" is a temporary hack, fix it *stream << nameOnDisk.get() << ",,1,0,0" << MSG_LINEBREAK; return PR_TRUE; } nsresult nsNntpIncomingServer::WriteHostInfoFile() @@ -1128,19 +1185,41 @@ nsNntpIncomingServer::UpdateSubscribed() NS_IMETHODIMP nsNntpIncomingServer::AddTo(const char *aName, PRBool addAsSubscribed, PRBool aSubscribable, PRBool changeIfExists) { nsresult rv = EnsureInner(); NS_ENSURE_SUCCESS(rv,rv); nsAutoString newsgroupName; - newsgroupName.AssignWithConversion(aName); + char *escapedName = nsnull; + + nsCOMPtr<nsIMsgFolder> rootFolder; + rv = GetRootFolder(getter_AddRefs(rootFolder)); + NS_ENSURE_SUCCESS(rv,rv); + if (!rootFolder) return NS_ERROR_FAILURE; + nsCOMPtr <nsIMsgNewsFolder> newsFolder = do_QueryInterface(rootFolder, &rv); + NS_ENSURE_SUCCESS(rv,rv); + if (!newsFolder) return NS_ERROR_FAILURE; + nsCOMPtr <nsINntpIncomingServer> nntpServer; + rv = newsFolder->GetNntpServer(getter_AddRefs(nntpServer)); + NS_ENSURE_SUCCESS(rv,rv); + nsCAutoString dataCharset; + rv = nntpServer->GetCharset(dataCharset); + NS_ENSURE_SUCCESS(rv,rv); + rv = nsMsgI18NConvertToUnicode(dataCharset.get(),nsDependentCString(aName),newsgroup Name); + if (NS_SUCCEEDED(rv)) { + escapedName = nsEscape(NS_ConvertUTF16toUTF8(newsgroupName).get(), url_Path); + } + + if (NS_FAILED(rv)) { + CopyASCIItoUTF16(aName, newsgroupName); + escapedName = nsEscape(aName, url_Path); + } - char *escapedName = nsEscape(NS_ConvertUCS2toUTF8(newsgroupName.get()).get(), url_Path); if (!escapedName) return NS_ERROR_OUT_OF_MEMORY; rv = AddGroupOnServer(escapedName); NS_ENSURE_SUCCESS(rv,rv); rv = mInner->AddTo(escapedName,addAsSubscribed, aSubscribable, changeIfExists); NS_ENSURE_SUCCESS(rv,rv); @@ -1245,17 +1324,27 @@ nsNntpIncomingServer::HandleLine(char* l if (line[0] == '#' || line[0] == '\0') return 0; line[line_size] = 0; if (mHasSeenBeginGroups) { char *commaPos = PL_strchr(line,','); if (commaPos) *commaPos = 0; - nsresult rv = AddTo(line, PR_FALSE, PR_TRUE, PR_TRUE); + //now we convert from UTF8 to the server charset encoding because the hostInfo.dat is always saved in UTF8 + nsAutoString unicodeLine; + nsCAutoString encodedLine; + CopyUTF8toUTF16(line,unicodeLine); + nsCAutoString dataCharset; + nsresult rv = GetCharset(dataCharset); + NS_ASSERTION(NS_SUCCEEDED(rv),"failed to get sever charset"); + rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), unicodeLine, encodedLine); + if (NS_FAILED(rv)) + LossyCopyUTF16toASCII(unicodeLine,encodedLine); + rv = AddTo(encodedLine.get(), PR_FALSE, PR_TRUE, PR_TRUE); NS_ASSERTION(NS_SUCCEEDED(rv),"failed to add line"); if (NS_SUCCEEDED(rv)) { // since we've seen one group, we can claim we've loaded the hostinfo file mHostInfoLoaded = PR_TRUE; } } else { if (nsCRT::strncmp(line,"begingroups", 11) == 0) { Only in mozilla/mailnews/news/src: nsNntpIncomingServer.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNNTPNewsgroupList.cpp mozilla/mailnews/news/src/nsNNTPNewsgroupList.cpp --- mailnews.old/news/src/nsNNTPNewsgroupList.cpp 2005-02-01 05:53:40.000000000 +0800 +++ mozilla/mailnews/news/src/nsNNTPNewsgroupList.cpp 2005-04-13 13:55:22.000000000 +0800 @@ -319,21 +319,21 @@ nsNNTPNewsgroupList::GetRangeOfArtsToDow nsCOMPtr<nsINewsDownloadDialogArgs> args = do_CreateInstance("@mozilla.org/messenger/newsdownloaddialogargs;1", &rv); if (NS_FAILED(rv)) return rv; NS_ENSURE_SUCCESS(rv,rv); rv = args->SetArticleCount(*last - *first + 1); NS_ENSURE_SUCCESS(rv,rv); - nsXPIDLCString groupName; - rv = m_newsFolder->GetAsciiName(getter_Copies(groupName)); + nsXPIDLString groupName; + rv = m_newsFolder->GetUnicodeName(groupName); NS_ENSURE_SUCCESS(rv,rv); - rv = args->SetGroupName((const char *)groupName); + rv = args->SetGroupName(groupName); NS_ENSURE_SUCCESS(rv,rv); // get the server key nsXPIDLCString serverKey; rv = server->GetKey(getter_Copies(serverKey)); NS_ENSURE_SUCCESS(rv,rv); rv = args->SetServerKey((const char *)serverKey); Only in mozilla/mailnews/news/src: nsNNTPNewsgroupList.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNNTPProtocol.cpp mozilla/mailnews/news/src/nsNNTPProtocol.cpp --- mailnews.old/news/src/nsNNTPProtocol.cpp 2005-02-16 11:18:46.000000000 +0800 +++ mozilla/mailnews/news/src/nsNNTPProtocol.cpp 2005-04-19 03:36:57.000000000 +0800 @@ -1934,17 +1934,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom char *command=0; PRInt32 status = 0; if (m_typeWanted == ARTICLE_WANTED) { if (m_key != nsMsgKey_None) { nsresult rv; nsXPIDLCString newsgroupName; if (m_newsFolder) { - rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); + rv = m_newsFolder->GetRawName(getter_Copies(newsgroupName)); NS_ENSURE_SUCCESS(rv,rv); } PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) current group = %s, desired group = %s", this, m_currentGroup.get(), newsgroupName.get())); // if the current group is the desired group, we can just issue the ARTICLE command // if not, we have to do a GROUP first if (!PL_strcmp(m_currentGroup.get(), newsgroupName.get())) m_nextState = NNTP_SEND_ARTICLE_NUMBER; @@ -2045,17 +2045,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom else if(m_typeWanted == GROUP_WANTED) { nsresult rv = NS_ERROR_NULL_POINTER; NS_ASSERTION(m_newsFolder, "m_newsFolder is null, panic!"); if (!m_newsFolder) return -1; nsXPIDLCString group_name; - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); NS_ASSERTION(NS_SUCCEEDED(rv),"failed to get newsgroup name"); if (NS_FAILED(rv)) return -1; m_firstArticle = 0; m_lastArticle = 0; NS_MsgSACopy(&command, "GROUP "); NS_MsgSACat(&command, (const char *)group_name); @@ -2090,17 +2090,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) doing GROUP for XPAT", this)); nsXPIDLCString group_name; /* for XPAT, we have to GROUP into the group before searching */ if (!m_newsFolder) { NNTP_LOG_NOTE("m_newsFolder is null, panic!"); return -1; } - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); if (NS_FAILED(rv)) return -1; NS_MsgSACopy(&command, "GROUP "); NS_MsgSACat (&command, group_name); // force a GROUP next time m_currentGroup.Truncate(); m_nextState = NNTP_RESPONSE; @@ -2201,17 +2201,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom } else { nsresult rv = NS_OK; nsXPIDLCString group_name; NS_ASSERTION(m_newsFolder, "no newsFolder"); if (m_newsFolder) { - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); } if (m_responseCode == MK_NNTP_RESPONSE_GROUP_NO_GROUP && m_typeWanted == GROUP_WANTED) { PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) group (%s) not found, so unset m_currentGroup",this,(const char *)group_name)); m_currentGroup.Truncate(); m_nntpServer->GroupNotFound(m_msgWindow, group_name.get(), PR_TRUE /* opening */); @@ -2316,17 +2316,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom } PRInt32 nsNNTPProtocol::SendGroupForArticle() { nsresult rv; PRInt32 status = 0; nsXPIDLCString groupname; - rv = m_newsFolder->GetAsciiName(getter_Copies(groupname)); + rv = m_newsFolder->GetRawName(getter_Copies(groupname)); NS_ASSERTION(NS_SUCCEEDED(rv) && groupname.get() && groupname.get()[0], "no group name"); char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "GROUP %.512s" CRLF, (const char *)groupname); @@ -2348,17 +2348,17 @@ nsNNTPProtocol::SetCurrentGroup() nsresult rv; nsXPIDLCString groupname; NS_ASSERTION(m_newsFolder, "no news folder"); if (!m_newsFolder) { m_currentGroup.Truncate(); return NS_ERROR_UNEXPECTED; } - rv = m_newsFolder->GetAsciiName(getter_Copies(groupname)); + rv = m_newsFolder->GetRawName(getter_Copies(groupname)); NS_ASSERTION(NS_SUCCEEDED(rv) && groupname.get()[0], "no group name"); PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) SetCurrentGroup to %s",this,(const char *)groupname)); m_currentGroup = groupname; return NS_OK; } PRInt32 nsNNTPProtocol::SendGroupForArticleResponse() { @@ -3873,17 +3873,17 @@ PRInt32 nsNNTPProtocol::DisplayNewsRC() if (NS_FAILED(rv)) return -1; if (!currFolder) return -1; m_newsFolder = do_QueryInterface(currFolder, &rv); if (NS_FAILED(rv)) return -1; if (!m_newsFolder) return -1; nsXPIDLCString name; - rv = m_newsFolder->GetAsciiName(getter_Copies(name)); + rv = m_newsFolder->GetRawName(getter_Copies(name)); if (NS_FAILED(rv)) return -1; if (!name) return -1; /* send group command to server */ char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "GROUP %.512s" CRLF, (const char *)name); nsCOMPtr<nsIMsgMailNewsUrl> mailnewsurl = do_QueryInterface(m_runningURL); @@ -3963,17 +3963,17 @@ PRInt32 nsNNTPProtocol::DisplayNewsRCRes NS_ASSERTION(NS_SUCCEEDED(rv),"DisplaySubscribedGroup() failed"); if (NS_FAILED(rv)) status = -1; if (status < 0) return status; } else if (m_responseCode == MK_NNTP_RESPONSE_GROUP_NO_GROUP) { nsXPIDLCString name; - rv = m_newsFolder->GetAsciiName(getter_Copies(name)); + rv = m_newsFolder->GetRawName(getter_Copies(name)); if (NS_SUCCEEDED(rv)) { m_nntpServer->GroupNotFound(m_msgWindow, name.get(), PR_FALSE); } PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) NO_GROUP, so unset m_currentGroup", this)); m_currentGroup.Truncate(); } @@ -4394,17 +4394,17 @@ PRInt32 nsNNTPProtocol::XPATResponse(nsI PRInt32 nsNNTPProtocol::ListPrettyNames() { nsXPIDLCString group_name; char outputBuffer[OUTPUT_BUFFER_SIZE]; PRInt32 status = 0; - nsresult rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + nsresult rv = m_newsFolder->GetRawName(getter_Copies(group_name)); PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "LIST PRETTYNAMES %.512s" CRLF, NS_SUCCEEDED(rv) ? (const char *) group_name : ""); nsCOMPtr<nsIMsgMailNewsUrl> mailnewsurl = do_QueryInterface(m_runningURL); if (mailnewsurl) status = SendData(mailnewsurl, outputBuffer); @@ -4471,17 +4471,17 @@ PRInt32 nsNNTPProtocol::ListPrettyNamesR PR_FREEIF(line); return 0; } PRInt32 nsNNTPProtocol::ListXActive() { nsXPIDLCString group_name; nsresult rv; - rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); + rv = m_newsFolder->GetRawName(getter_Copies(group_name)); if (NS_FAILED(rv)) return -1; PRInt32 status = 0; char outputBuffer[OUTPUT_BUFFER_SIZE]; PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "LIST XACTIVE %.512s" CRLF, @@ -4624,17 +4624,17 @@ PRInt32 nsNNTPProtocol::SendListGroup() nsresult rv; char outputBuffer[OUTPUT_BUFFER_SIZE]; PRInt32 status = 0; NS_ASSERTION(m_newsFolder,"no newsFolder"); if (!m_newsFolder) return -1; nsXPIDLCString newsgroupName; - rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); + rv = m_newsFolder->GetRawName(getter_Copies(newsgroupName)); NS_ENSURE_SUCCESS(rv,rv); PR_snprintf(outputBuffer, OUTPUT_BUFFER_SIZE, "listgroup %.512s" CRLF, newsgroupName.get()); m_articleList = do_CreateInstance(NS_NNTPARTICLELIST_CONTRACTID, &rv); Only in mozilla/mailnews/news/src: nsNNTPProtocol.cpp~ diff -U 8 -p -r mailnews.old/news/src/nsNntpService.cpp mozilla/mailnews/news/src/nsNntpService.cpp --- mailnews.old/news/src/nsNntpService.cpp 2005-01-18 02:50:13.000000000 +0800 +++ mozilla/mailnews/news/src/nsNntpService.cpp 2005-04-13 13:56:02.000000000 +0800 @@ -650,18 +650,21 @@ nsNntpService::GetFolderFromUri(const ch // but the *name* of the newsgroup (we are calling ::GetChildNamed()) // is unescaped. see http://bugzilla.mozilla.org/show_bug.cgi?id=210089#c17 // for more about this char *unescapedPath = PL_strdup(path.get() + 1); /* skip the leading slash */ if (!unescapedPath) return NS_ERROR_OUT_OF_MEMORY; nsUnescape(unescapedPath); + nsAutoString newsgroupName; + //the encoding is UTF-8 after unescaped. + CopyUTF8toUTF16(unescapedPath,newsgroupName); nsCOMPtr<nsISupports> subFolder; - rv = rootFolder->GetChildNamed(NS_ConvertASCIItoUCS2(unescapedPath).get() , getter_AddRefs(subFolder)); + rv = rootFolder->GetChildNamed(newsgroupName.get() , getter_AddRefs(subFolder)); PL_strfree(unescapedPath); NS_ENSURE_SUCCESS(rv,rv); return CallQueryInterface(subFolder, aFolder); } NS_IMETHODIMP nsNntpService::CopyMessage(const char * aSrcMessageURI, nsIStreamListener * aMailboxCopyHandler, PRBool moveMessage,

changes: encodedName changed to rawName. the mailnews.view_default_charset becomes a complex value in the trunk,So using getComplexValue to get the default charset. Thank you very much for reviewing... I don't know edit attachment will put changes to the comments ... Really sorry for that..

Comment on attachment 181047 [details] [diff] [review] patch use utf8 in the newsrc/hostinfo file Sorry for the delay. Let's do one more iteration. >+[scriptable, uuid(02699533-7ba8-4e78-94dc-a20374fb2711)] > interface nsIMsgNewsFolder : nsISupports { > attribute string groupUsername; > attribute string groupPassword; >+ readonly attribute AString unicodeName; >+ /** the raw name name for the newsgroup (server side name). >+ * newsgroup names are might not always ascii(bug#126453), and this raw name name >+ * can help simply the code, to prevent use from having to get the unicode name >+ * (as a wstring) and then converting it back to server side raw name. > */ >- readonly attribute string asciiName; >+ readonly attribute string rawName; |rawName| isn't accessible from a script so that you should add [noscript]. Alternatively, it has to be changed to getter with an octet array. For now, just adding [noscript] should suffice because it's not used in any script. As for the comment, it needs some polishing. -------Cut-------Here---------------------- |rawName| is an 8-bit string to represent the name of a newsgroup used by a news server. It's offered for the convenience of callers so that they don't have to convert |unicodeName| to the server-side name when communicating with a news server. It's US-ASCII except for some 'stand-alone' Chinese news servers that use GB2312 for newsgroup names violating RFC 1036. For those servers, it's GB2312. However, it can be any other single and multibyte encoding in principle. The encoding of this string is stored in |nsINntpIncomingServer| because that's a server-wide property. ---------Cut----------Here--------------- Also, make it |ACString| > nsMsgNewsFolder::~nsMsgNewsFolder(void) > { > MOZ_COUNT_DTOR(nsNewsFolder); > delete mReadSet; > PR_Free(mGroupUsername); > PR_Free(mGroupPassword); >- PR_Free(mAsciiName); >+ PR_Free(mRawName); > } For mRawName, if you use nsCString, you can avoid alloc/free, can't you? >+ nsCOMPtr<nsINntpIncomingServer> nntpServer; >+ rv = GetNntpServer(getter_AddRefs(nntpServer)); >+ if (NS_FAILED(rv)) return rv; >+ //now we convert from UTF8 to the server charset encoding because the newsrc file is always saved in UTF8 >+ nsAutoString unicodeLine; >+ nsCAutoString encodedLine; >+ CopyUTF8toUTF16(line,unicodeLine); >+ >+ nsCAutoString dataCharset; >+ rv = nntpServer->GetCharset(dataCharset); >+ NS_ASSERTION(NS_SUCCEEDED(rv),"failed get sever charset"); >+ rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), unicodeLine, encodedLine); > nsMsgNewsFolder::GetNewsrcLine(char **newsrcLine) >+ nsXPIDLString uninewsgroupname; > nsXPIDLCString newsgroupname; >- rv = GetAsciiName(getter_Copies(newsgroupname)); >+ rv = GetName(getter_Copies(uninewsgroupname)); > if (NS_FAILED(rv)) return rv; Perhaps, |newsgroupNameUtf16| (with intercap) would be better than |uninewsgroupname|. >+nsMsgNewsFolder::GetRawName(char **rawName) >+ NS_ENSURE_ARG_POINTER(rawName); >+ if (!mRawName) Using ns*CString instead of |char *|, you can see if it's empty and don't have to do null-pointer check. >+ mRawName = nsCRT::strdup(tmpStr.get()); >+ if (!mRawName) return NS_ERROR_OUT_OF_MEMORY; This part is simplified, too. >+ char *mRawName; nsCString mRawName; prefBranch(do_GetService(NS_PREFSERVICE_CONTRACTID, &rv)); >+ >+ if (NS_SUCCEEDED(rv)){ >+ nsCOMPtr<nsIPrefLocalizedString> pls; >+ rv = prefBranch->GetComplexValue(PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, >+ NS_GET_IID(nsIPrefLocalizedString), getter_AddRefs(pls)); >+ if (NS_SUCCEEDED(rv)) >+ { >+ nsXPIDLString ucsval; >+ pls->ToString(getter_Copies(ucsval)); >+ if (ucsval) >+ { >+ LossyCopyUTF16toASCII(ucsval, defaultCharset); >+ } >+ } >+ } Why don't you just use |NS_GetUnicharPreferenceWithDefault| defined at http://lxr.mozilla.org/seamonkey/source/mailnews/base/util/nsMsgUtils.cpp#1059 >- newsgroupName.AssignWithConversion(aName); >+ char *escapedName = nsnull; >+ >+ nsCOMPtr<nsIMsgFolder> rootFolder; >+ rv = GetRootFolder(getter_AddRefs(rootFolder)); >+ NS_ENSURE_SUCCESS(rv,rv); >+ if (!rootFolder) return NS_ERROR_FAILURE; >+ nsCOMPtr <nsIMsgNewsFolder> newsFolder = do_QueryInterface(rootFolder, &rv); >+ NS_ENSURE_SUCCESS(rv,rv); >+ if (!newsFolder) return NS_ERROR_FAILURE; >+ nsCOMPtr <nsINntpIncomingServer> nntpServer; >+ rv = newsFolder->GetNntpServer(getter_AddRefs(nntpServer)); >+ NS_ENSURE_SUCCESS(rv,rv); >+ nsCAutoString dataCharset; >+ rv = nntpServer->GetCharset(dataCharset); Why do you need all these (long and winding) indirections? Can't you just use GetCharset here? This method (|AddTo|) belongs to |nsNntpIncomingServer| (an implementation of |nsINntpIncomingServer| so that it should have |GetCharset|, right?). >@@ -1245,17 +1324,27 @@ nsNntpIncomingServer::HandleLine(char* >+ //now we convert from UTF8 to the server charset encoding because the hostInfo.dat is always saved in UTF8 This comment is over 80 columns. s/the server charset encoding/the server-side character encoding/ >+ nsAutoString unicodeLine; >+ nsCAutoString encodedLine; Perhaps, 'utf16Line' and 'serverSideLine' are better. >+ rv = m_newsFolder->GetRawName(getter_Copies(newsgroupName));

One more thing. If you want mailnews.view_default_charset to be localizable, you have to add it to either messenger.properties

Thanks again for your comments and hints!

Comment on attachment 182461 [details] [diff] [review] patch according to comment#78 r=jshin thanks for the patch and sorry for taking so long. > NS_IMETHODIMP nsMsgNewsFolder::CreateSubfolder(const PRUnichar *uninewsgroupname, nsIMsgWindow *msgWindow) > { > nsresult rv = NS_OK; > > NS_ENSURE_ARG_POINTER(uninewsgroupname); > if (!*uninewsgroupname) return NS_ERROR_FAILURE; > >- nsCAutoString newsgroupname; >- newsgroupname.AssignWithConversion(uninewsgroupname); >+ nsCAutoString newsgroupname; >+ nsCOMPtr <nsINntpIncomingServer> nntpServer; >+ rv = GetNntpServer(getter_AddRefs(nntpServer)); >+ if (NS_FAILED(rv)) return rv; >+ nsCAutoString dataCharset; >+ rv = nntpServer->GetCharset(dataCharset); >+ if (NS_FAILED(rv)) return rv; >+ >+ rv = nsMsgI18NConvertFromUnicode(dataCharset.get(), nsDependentString(uninewsgroupname), newsgroupname); Although it's not you who first used 'uninewsgroupname' here, would you change it to newsgroupNameUtf16? Also, it'd be nice to replace |newsgroupname| with |newsgroupName|. >+ rv = args->SetGroupName(groupName); > NS_ENSURE_SUCCESS(rv,rv); > > // get the server key > nsXPIDLCString serverKey; > rv = server->GetKey(getter_Copies(serverKey)); > NS_ENSURE_SUCCESS(rv,rv); Again, it's not your code, but it'd be nice to fix up indentation of three lines before |NS_ENSURE_SUCCESS|. >+++ mozilla/mailnews/news/src/nsNNTPProtocol.cpp 2005-05-03 11:43:54.608116736 +0800 >@@ -1934,17 +1934,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom > char *command=0; > PRInt32 status = 0; > > if (m_typeWanted == ARTICLE_WANTED) { > if (m_key != nsMsgKey_None) { > nsresult rv; > nsXPIDLCString newsgroupName; > if (m_newsFolder) { >- rv = m_newsFolder->GetAsciiName(getter_Copies(newsgroupName)); >+ rv = m_newsFolder->GetRawName(newsgroupName); > NS_ENSURE_SUCCESS(rv,rv); > } > > PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) current group = %s, desired group = %s", this, m_currentGroup.get(), newsgroupName.get())); > // if the current group is the desired group, we can just issue the ARTICLE command > // if not, we have to do a GROUP first > if (!PL_strcmp(m_currentGroup.get(), newsgroupName.get())) > m_nextState = NNTP_SEND_ARTICLE_NUMBER; Phew, the indentation here is also horrible perhaps due to tabs. Can you fix them all? >@@ -2045,17 +2045,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom > > nsXPIDLCString group_name; >- rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); >+ rv = m_newsFolder->GetRawName(group_name); > NS_ASSERTION(NS_SUCCEEDED(rv),"failed to get newsgroup name"); You're touching the line so that you'd better fix the indentation. >@@ -2090,17 +2090,17 @@ PRInt32 nsNNTPProtocol::SendFirstNNTPCom > PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) doing GROUP for XPAT", this)); > nsXPIDLCString group_name; > > /* for XPAT, we have to GROUP into the group before searching */ > if (!m_newsFolder) { > NNTP_LOG_NOTE("m_newsFolder is null, panic!"); > return -1; > } >- rv = m_newsFolder->GetAsciiName(getter_Copies(group_name)); >+ rv = m_newsFolder->GetRawName(group_name); > if (NS_FAILED(rv)) return -1; > > NS_MsgSACopy(&command, "GROUP "); > NS_MsgSACat (&command, group_name); > > // force a GROUP next time > m_currentGroup.Truncate(); > m_nextState = NNTP_RESPONSE; I'm sorry to ask you to do more chores, but if you don't mind getting cvs-blame, would you fix tabs here as well? >+ //the encoding is UTF-8 after unescaped. after unescaping

Comment on attachment 182461 [details] [diff] [review] patch according to comment#78 can you post a new patch with jshin's comments and then re-request an sr from me. thanks!

Sorry again,Yesterday I installed a leafnode to my linuxbox and did a full test for chinese newsgroup names,I found some other related problems and did some fixes. changes: changes according to comment#81 fix for newsgroupname are displayed as escaped string at compose window. fix for cannot send mail to newsgroup because the groupname are escaped at the compose window. fix for not able to search in subscribe dialog window in case of chinese newsgroup names(page subscribe.xul). fix for not able to unsubscribe in subscribe dialog window in case of chinese newsgroup names(exists before),which caused by the unsubscribe(name) method,some place in .js file,called with escaped name as parameter,and someplace called with unicode name as parameter,now only call with escaped name (same as subscribe(name) ).

hope this is the final version now we handle newsgroupname internally as utf8 string only. the conversion from serverSide name to internal utf8 newsgroup name only happens when communicate with news server.

Comment on attachment 182761 [details] [diff] [review] patch according to comment#81 i'll review the newest patch on Wednesday.

*** Bug 151946 has been marked as a duplicate of this bug. ***

since thunderbird have some file duplicated in the base/content directory, we need to add the same change for those files which we modifed in suite files.

sorry for the delay and thanks for the update. I've gone about a half way through. I'll make sure to finish it no later than Tuesday. Btw, can you give me a couple of news servers with Chinese newsgroup names (in GB2312) I can access freely? Is news.newstan.net one of them?

(In reply to comment #89) Yes news.newsfan.net is one of them... You can try the following servers,they are free to access: news://news.newsfan.net (IP: 210.12.195.2) news://news.cnnb.net news://zhnews.net (IP: 210.73.88.194) Thanks again!

Fix this spacing before checkin: - attribute string newsgroups; + attribute AString newsgroups; attribute string newshost; attribute string newspostUrl; attribute string followupTo;

(In reply to comment #91) > Fix this spacing before checkin: > > - attribute string newsgroups; > + attribute AString newsgroups; OK,Thanks!

Wen, I'm really sorry for my delay. I'll get back to this bug this weekend for sure.

we should get this in for b3 if at all possible.

jshin, any update?

sorry again for the delay. I'll finish reviewing and testing today. there were some hiccups in my builds on linux and mac..

It works fine on Linux (UTF-8 locale) and Mac OS X, but I just realized that it wouldn't on Windows if the file system encoding can't cover characters used in the names of newsgroups (e.g. Chinese newsgroup names on English Windows 9x). Given this, it's a better idea to use UTF-7 or other ASCII-safe(and file system safe name) for *msf and *sbd files. Sorry for the delay and not having pointed this out earlier. If you can make a quick change, that would be great and I'll review as quickly as possible.

(In reply to comment #97) > Given this, it's a better idea to use UTF-7 or other ASCII-safe(and file system > safe name) for *msf and *sbd files. There are utility functions for modified UTF-7 in nsMsgI18n.h

Comment on attachment 184740 [details] [diff] [review] patch for both mozilla suite and thunderbird. r=jshin with the following minor issues addressed. Sorry for the delay. Let's put off the issue I raised yesterday for a while unless you can resolve it quickly as I outlined. There'd be a migration issue, but not many users will be affected by it. There are some functions that are always supposed to get 'UTF-8' |char *| strings. Please, add a comment to that effect wherever that's the case. When you add / modify lines longer than 80 columns, try to split them into shorter lines (I noted some of them below, but not all of them). > Index: mail/base/content/mailCommands.js > var subscribableServer = server.QueryInterface(Components.interfaces.nsISubscribableServer); > - subscribableServer.unsubscribe(folder.name); > + subscribableServer.unsubscribe(encodeURI(folder.name)); Perhaps, not in this bug, but later we want to change nsISubscribable interface to make it more consistent.. Currently, it takes |wstring| for |(un)subscribe|, but takes |string| for other methodes, which I guess is why you just use url-escaping here. > Index: mailnews/compose/src/nsMsgCompUtils.cpp > + // fixme:the newsgroups header must be encoded as the server-side character encoding > + // but this |charset| might be different from it. > + ENCODE_AND_PUSH("Newsgroups: ", PR_FALSE, newsgroupsHeaderVal.get(), charset, PR_FALSE); Actually, as long as RFC 2047 is used, it should be fine in theory/principle. (modern standard-compliant newsreaders like TB wouldn't have any problem) Given that, you may wanna tone down your comment ('must' is too strong. 'had better' may be about right) > @@ -2182,22 +2182,22 @@ NS_IMETHODIMP QuotingOutputStreamListene .... > - compFields->SetNewsgroups(nsnull); > + compFields->SetNewsgroups(NS_LITERAL_STRING("")); Pls, use EmptyString() instead of NS_LITER... > Index: mailnews/compose/src/nsMsgComposeService.cpp > - pMsgCompFields->SetNewsgroups(aNewsgroup); > + pMsgCompFields->SetNewsgroups(NS_ConvertUTF8toUTF16(aNewsgroup)); Perhaps, not here but later we may consider using |AUTF8String| for |nsIMsgCompFields::newsgroup| so that we can avoid the conversion. You have a dozen of them like this. > Index: mailnews/news/public/nsIMsgNewsFolder.idl > + readonly attribute AString unicodeName; > + /**|rawName| is an 8-bit string to represent the name of a newsgroup used by a > + * news server. It's offered for the convenience of callers so that they don't > + * have to convert |unicodeName| to the server-side name when communicating with a ....... snip ... > + * because that's a server-wide property. Please, make this comment fit well within 80 columns > Index: mailnews/news/public/nsINntpIncomingServer.idl > + /* the server charset and it may be needed to display newsgroup folder names correctly */ > + attribute ACString charset; Same here.. > Index: mailnews/news/src/nsNNTPProtocol.cpp > - PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) current group = %s, desired group = %s", this, m_currentGroup.get(), newsgroupName.get())); > + PR_LOG(NNTP,PR_LOG_ALWAYS,("(%p) current group = %s, desired group = %s", this, m_currentGroup.get(), newsgroupName.get())); Will you please use the string concatenation to split the above line into two 'shorter-than-80 columns' lines? > - else > - m_nextState = NNTP_SEND_GROUP_FOR_ARTICLE; > + if (!PL_strcmp(m_currentGroup.get(), newsgroupName.get())) > + m_nextState = NNTP_SEND_ARTICLE_NUMBER; > + else > + m_nextState = NNTP_SEND_GROUP_FOR_ARTICLE; This file is rather messy in terms of indentation. It seems like you used 4 column indentation earlier in the file. Will you do the same here? > if (NS_SUCCEEDED(rv)) { > - m_nntpServer->GroupNotFound(m_msgWindow, name.get(), PR_FALSE); > + m_nntpServer->GroupNotFound(m_msgWindow, NS_ConvertUTF16toUTF8(name).get(), PR_FALSE); > } split the above line to two or three lines and get rid of braces. m_nntpServer->GroupNotFound(m_msgWindow, NS_ConvertUTF16toUTF8(name).get(), PR_FALSE); > Index: mailnews/news/src/nsNewsDownloadDialogArgs.cpp > +NS_IMETHODIMP nsNewsDownloadDialogArgs::GetGroupName(nsAString & aGroupName) > + { > + aGroupName = mGroupName; > + > + return NS_OK; nit: one column off here for |return ....| > Index: mailnews/news/src/nsNewsFolder.cpp > > - nsXPIDLCString newsgroupname; > - rv = GetAsciiName(getter_Copies(newsgroupname)); > + nsXPIDLString newsgroupNameUtf16; > + nsXPIDLCString newsgroupName; > + rv = GetName(getter_Copies(newsgroupNameUtf16)); > if (NS_FAILED(rv)) return rv; > - > + CopyUTF16toUTF8(newsgroupNameUtf16, newsgroupName); don't need |nsXPIDLCString newsgroupName|. Instead, just do : NS_ConvertUTF16toUTF8 newsgroupName(newsgroupNameUtf16); > + if(serverCharset.IsEmpty()){ > + nsXPIDLString defaultCharset; > + rv = NS_GetUnicharPreferenceWithDefault(nsnull,PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, nit: split a long line rv = NS_GetUnicharPreferenceWithDefault(nsnull, PREF_MAILNEWS_VIEW_DEFAULT_CHARSET, NS_LITERAL_STRING("ISO-8859-1"), defaultCharset); > +#ifdef DEBUG_NEWS > + printf("default charset for the server is %s\n", (const char *)defaultCharset); printf("default charset for the server is %s\n", (const char *)defaultCharset); > @@ -806,17 +844,17 @@ writeGroupToHostInfoFile(nsCString &aEle > nsAutoString name; > nsresult rv = NS_MsgDecodeUnescapeURLPath(aElement, name); > if (NS_FAILED(rv)) { > // stop, something is bad. > return PR_FALSE; > } > nsCAutoString nameOnDisk; > - nameOnDisk.AssignWithConversion(name.get()); > + CopyUTF16toUTF8(name, nameOnDisk); NS_ConvertUTF16toUTF8 nameInNewsrc(name); > // XXX todo ",,1,0,0" is a temporary hack, fix it > *stream << nameOnDisk.get() << ",,1,0,0" << MSG_LINEBREAK; *stream << nameInNewsrc.get() << ",,1,0,0" << MSG_LINEBREAK; > + rv = nsMsgI18NConvertToUnicode(dataCharset.get(), nsDependentCString(aName), newsgroupName); rv = nsMsgI18NConvertToUnicode(dataCharset.get(), nsDependentCString(aName), newsgroupName); > @@ -1204,24 +1248,19 @@ nsNntpIncomingServer::Unsubscribe(const > - rv = serverFolder->FindSubFolder(escapedName, getter_AddRefs(newsgroupFolder)); > + rv = serverFolder->FindSubFolder(NS_LossyConvertUCS2toASCII(aUnicharName), getter_AddRefs(newsgroupFolder)); Please, add a comment as to why it's safe to use a lossy converter here (because aUnicharName is already escaped ? ) rv = serverFolder->FindSubFolder( NS_LossyConvertUTF16toASCII(aUnicharName), getter_AddRefs(newsgroupFolder));

Comment on attachment 184740 [details] [diff] [review] patch for both mozilla suite and thunderbird. forgot marking my 'r+'. asking David for sr. David, what do you think of putting off implementiong what I wrote in comment #97 and comment #98?

If it's just English Windows 9x, i.e. win 95 and win 98, and it was broken everywhere before, then this would be a big improvement. But, is there a clear migration strategy? What data files are affected by this change? newsrc.dat? hostinfo.dat? Is the format only changed if the user changes the server char set? Also, have we made sure this works with imap folders with special characters? Imap uses subscription as well, though the strings should all be imap mod-utf8, iirc.

Comment on attachment 184740 [details] [diff] [review] patch for both mozilla suite and thunderbird. sr=bienvenu, as long as imap has been tested.

newsrc/hostdata files are not affected (they use UTF-8), but *msf and *sbd filenames are affected. With this patch, they're stored in the local file system encoding. On Mac OS X and modern Linux - by default -, it's UTF-8 so that there's no problem. However, on other OS' where the file system encoding is not Unicode-based (including all versions of Windows. I should have said this although Windows 2k/XP will not have this problem once bug 162361 is fixed and mailnews moves to nsI*File from nsIFileSpec), there's a problem with non-ASCII newsgroup names (as long as Simp. Chinese windows with SC system locale is used, there's no problem). To support those OS', using modified UTF-7 for *msf and *sbd (as we do for IMAP) is the best, but I don't have time to figure that out myself. If Shaohua can do that quickly, that'll be great. As for the migration, that wouldn't be smooth for those who subscribe to newsgroups with non-ASCII names (currently, only *stand-alone* Chinese newsservers in China use them and I don't expect other news servers to do that). They will have to unsubscribe and subscribe again, I guess because *msf and *sbd file names will be changed. So, this may be a rather significant issue that can block this patch from landing. If you believe this migration issue is too important to let this patch land, we shoudln't check in this patch without addressing the problem. Another important problem: subscribe/unsubscribe with IMAP. Thanks for pointing that out. As long as modifiued UTF-7 doesn't use characters subject to url-escaping, that should be fine, but I want to make sure of that. Perhaps, there are two better alternatives: 1) call encodeURI only for newsserver if it's possible to tell the current server is a newsserver (is it possible now? if not, we may add a method to nsISubscribableServer and a few constants) 2) even better is not to use encodeURI at all (see the 4th paragraph of my comment #99)

Comment on attachment 184740 [details] [diff] [review] patch for both mozilla suite and thunderbird. is this patch ready to check in?

the patch needs a bit more testing to make sure it doesn't break imap subscribe

(In reply to comment #105) > the patch needs a bit more testing to make sure it doesn't break imap subscribe Characters like '%' and '^' can represent themselves in modified UTF-7, but they'll be url-escaped by encodeURI so that the patch as it stands breaks imap subscription to folders with names including such characters. <quote : RFC 2060> In modified UTF-7, printable US-ASCII characters except for "&" represent themselves; that is, characters with octet values 0x20-0x25 and 0x27-0x7e. The character "&" (0x26) is represented by the two- octet sequence "&-". </quote> David, which way do you prefer? 1) add a property ('enum type') to nsISubscribableServer so that we can query whether it's a newsserver or not and use encodeURI only for newsservers. 2) use modified-UTF-7 for *sbd and *msf files for news 1) is easier (for me) than 2), but 2) is a better long-term solution than 1). Besides, if we go with 2), we won't have any migration problem except for newsgroup names with '&' (hmm... is there any?)

how much harder is 2)? I don't believe '&' is legal in newsgroup names...

At first (when I raised the issue for the first time), I thought it'd be rather easy, but about an hour of poking around didn't lead me where changes need to be made. Can you tell me where filenames for '*msf' and '*sbd' are 'constructed' for newsgroups and where they're referred to if you know the answer off the top of your head?

mailnews/base/util/nsNewsSummarySpec.cpp is a good place to start...

please please please please please please don't use % in file or foldernames. they cause serious issues for windows where % is the (env)variable marker.

(In reply to comment #110) > please please please please please please don't use % in file or foldernames. > they cause serious issues for windows where % is the (env)variable marker. This patch doesn't use '%' in filenames. Nor will any future patch for this bug as long as newsgroup names don't have '%' ('%' is not allowed for newsgroup names afaik.). Note that '%' is not a part of Base64 alphabet (used in modified UTF-7) (In reply to comment #109) > mailnews/base/util/nsNewsSummarySpec.cpp is a good place to start... thanks !

Comment on attachment 184740 [details] [diff] [review] patch for both mozilla suite and thunderbird. withdrawing my sr and obsoleting patch, so we can deal with the imap issue.

I think this is the simplest way to use utf7 for .msf .dat file names,but it impact not only news,but local/imap also.

Dear all, I have no access to any imap server with non-ascii folders.Could any one help to test the unsubscribe function with this kind of imap server?

that's a very strange place to put that code, and I don't want to change the .msf file names for anything but news, if we can help it.

(In reply to comment #115) > that's a very strange place to put that code, and I don't want to change the > .msf file names for anything but news, if we can help it. I tried override this ParseURI method in nsNewsFolder.cpp by declare a same method in nsNewsFolder.h,and add this convert2mutf7 line,but it seems not working(the method in nsNewsFolder is never called...). Everytime this method was called,it will update the mPath,which is the key piont of the .msf .dat file names. Could you please point me out a way? Thanks!

(In reply to comment #115) > that's a very strange place to put that code, and I don't want to change the > .msf file names for anything but news, if we can help it. OK,I figured it out. What I need do is make the parseURI method a virtual one in nsMsgDBFolder.h and rewrite it on nsNewsFolder.h&nsNewsFolder.cpp. tested and works fine. I will upload the full patch after I can sync with the mozilla.org's cvs server.

(In reply to comment #112) > (From update of attachment 184740 [details] [diff] [review] [edit]) > withdrawing my sr and obsoleting patch, so we can deal with the imap issue. > I installed a cyrus imap server and tested subscribe/unsubscribe with chinese names folders ,works fine.

(In reply to comment #118) > > withdrawing my sr and obsoleting patch, so we can deal with the imap issue. > > > > I installed a cyrus imap server and tested subscribe/unsubscribe with chinese > names folders ,works fine. Did you get rid of |encodeURI| in subscribe/unsubscribe? See comment #106.

closing down for 1.8b3, let's try and get this in for 1.8b4

(In reply to comment #119) > Did you get rid of |encodeURI| in subscribe/unsubscribe? See comment #106. > I tried to create a folder contains % or ^,thunderbird just closed the "create folder" dialog and nothing happens. tried evolution also,it says that % or ^ is invalid. what's the problem?

(In reply to comment #121) > (In reply to comment #119) > > Did you get rid of |encodeURI| in subscribe/unsubscribe? See comment #106. > > > > I tried to create a folder contains % or ^,thunderbird just closed the "create > folder" dialog > and nothing happens. > tried evolution also,it says that % or ^ is invalid. > what's the problem? At least '^' is allowed and it's Evolution's fault to claim that '^' is invalid. TB does allow you to create a new IMAP folder with '^' in its name. I think nsISubscribale needs to be changed as I mentioned in comment #99. It's quite a lot of job, but it can be done in an ad-hoc fashion (with an understanding that it'll be revisited before long. perhaps after 1.1branch is cut?) so that this bug is fixed properly. I'm making that change now.

Comment on attachment 187989 [details] [diff] [review] full patch to use Modified UTF7 for .msf and .dat file names. >- subscribableServer.unsubscribe(folder.name); >+ subscribableServer.unsubscribe(encodeURI(folder.name)); David and I already wrote that this would break IMAP.

I'll try to upload a new patch combining Shaohua's patch with what I wrote in comment #123 soonish.

(In reply to comment #123) > At least '^' is allowed and it's Evolution's fault to claim that '^' is > invalid. TB does allow you to create a new IMAP folder with '^' in its name. > > I think nsISubscribale needs to be changed as I mentioned in comment #99. It's > quite a lot of job, but it can be done in an ad-hoc fashion (with an > understanding that it'll be revisited before long. perhaps after 1.1branch is > cut?) so that this bug is fixed properly. I'm making that change now. > I tried other charactors also,not only % ^ are forbidden,also ! @ # $ & * ( ) are forbidden in the folder name (TB just close the "New Folder" Dailog and nothing happens). Could you please try to verify that? (In reply to comment #124) > (From update of attachment 187989 [details] [diff] [review] [edit]) > > >- subscribableServer.unsubscribe(folder.name); > >+ subscribableServer.unsubscribe(encodeURI(folder.name)); > > David and I already wrote that this would break IMAP. > If those charactors are not allowed,then ,this code will not break IMAP,right?

See http://www.faqs.org/rfcs/rfc3501.html (RFC 3501)

I did what I wrote I would in comment #99. It turned out to have quite an extensive ripple effect across mailnews/. I can produce imap folders with '^' and other special characters (but not '%'...) as well as non-ASCII names. Subscribed group names are now stored in UTF-8 without escaping, but there are still some places escaped groupnames are used. I wanted to get rid of them, too, but there is a 'bytewise inflation' of UTF-8 to UTF-16 (with ASCIItoUTF16 or AssignWithConversion) which I can't pinpoint

(In reply to comment #115) > that's a very strange place to put that code, and I don't want to change the > .msf file names for anything but news, if we can help it. My patch also touches parseURI, but changes msf filenames only for news/snews (I just added to parseURL in nsMsgDBFolder a check as to whether a scheme is news/snews/nntp without making it virtual and overriding it). Btw, nsNewsSummarySpec is only used when deleting msf files for unsubscribed groups so that changing it doesn't seem to work.

Comment on attachment 188182 [details] [diff] [review] Shaohua's patch plus mine David, can you take a look? The size of this patch is a bout twice as large as before, but that's primarily because I replaced |string| in two interface files with |AUTF8String| or |AString|. Btw, two chunks in nsMsgCompose.cpp are not parts of this patch and I got rid of them in my tree.

Jungshik, it basically looks good. Thx for doing this. There are a few places where you've just commented out old lines of code - those can just be removed, I think. Also, + [noscript] readonly attribute ACString rawName; this doesn't have to be noscript, does it? + // fixme: the newsgroups header had better be decoded using the server-side + // character encoding,but this |charset| might be different from it. is this draft fixme going to be a problem for these GB132 servers? + // to handle non-ASCII newsgroup names, we store them internally + // as escaped. decode and unescape the newsgroup name so we'll + // display a proper name. + nsAutoString unescapedName; rv = NS_MsgDecodeUnescapeURLPath(group, unescapedName); + //NS_ConvertUTF8toUTF16 unescapedName(group); is this part finished, or is NS_MsgDecodeUnescapeURLPath doing this conversion now, so the commented out line can be removed? a -uw diff would probably be easier - the news code is filled with tabs and bad indentation, so thanks for cleaning that up. remove this, and the other dumps: + dump("ReverseStateFromNode: name=" + name); Do you want me to try this patch?

(In reply to comment #131) > Jungshik, it basically looks good. Thx for doing this. There are a few places > where you've just commented out old lines of code - those can just be removed, I > think. Yes, I've just done that in my tree and I'll upload a new '-uw' patch for you. > + [noscript] readonly attribute ACString rawName; > > this doesn't have to be noscript, does it? If we want to allow it in scripts, we have to add a very prominent warning that what one get in JS is a JS string made up of 'byte-extended' characters (i.e. if a string is '0xac 0xb0' in GB2312, what you get in JS is 'U+00AC' 'U+00B0' instead of a single Unicode character corresponding to '0xac 0xb0' in GB2312). This can be reverted back to the proper JS string using nsIScriptableUnicodeConverter, but that's a long and winding alternative to a lot simpler |newsFolder.unicodeName| :-) > + // fixme: the newsgroups header had better be decoded using the server-side > + // character encoding,but this |charset| might be different from it. > > is this draft fixme going to be a problem for these GB132 servers? If someone wants to post to a newsgroup on one of these servers (with GB2312 used on their ends) in an char. encoding other than GB2312-compatible ones, it'll be a problem. I guess that'll be very rare and we can get away with it for now. > rv = NS_MsgDecodeUnescapeURLPath(group, unescapedName); > + //NS_ConvertUTF8toUTF16 unescapedName(group); > > is this part finished, or is NS_MsgDecodeUnescapeURLPath doing this conversion > now, so the commented out line can be removed? Yes, I'll remove it. > remove this, and the other dumps: > + dump("ReverseStateFromNode: name=" + name); I'll remove dump's as well. > Do you want me to try this patch? Yes, that'll be nice. I've done some tests and haven't found a problem, but it's not thorough.

I got rid of commented out lines and dump() statements in JS per David's comment. attachment 188182 [details] [diff] [review] should be used to patch (for testing) while this is for an easier reading (-uw patch)

(In reply to comment #133) > Created an attachment (id=188215) [edit] mailnews/base/public/nsISubscribableServer.idl // for searching - void setSearchValue(in string searchValue); + void setSearchValue(in AString searchValue); should it be AUTF8String here also? does it works with non ascii name search here?

AString and AUTF8String are both intl-safe. (as opposed to ACString and string)

re the noscript, it sounds like you have an excellent reason for making it [noscript], so that's fine. You could add a brief comment to the idl for that. I'll try the patch tomorrow and then put my r= on it.

Comment on attachment 188215 [details] [diff] [review] Shaohua's patch plus mine (-uw patch) with dump/commented line removed sorry for all the nits; rebuilding is taking a long time :-) some nits: + if (NS_SUCCEEDED(url->GetScheme(scheme))) { + isNewsFolder = scheme.EqualsLiteral("news") || can you either use the prevailing braces style in this file, if (a) { .. } or no braces at all. similarly, in nsMsgUtils.cpp, the prevalent braces style isn't K&R. re this change, are we still doing case-insensitive search, or did we drop that? It's unclear to me why the same server would return the same newsgroup with difference cases. Actually, looks like we're using nsCaseInsensitiveStringComparator() down the road, so we're OK? - // force the search string to be lower case - // so that we can do case insensitive searching - ToLowerCase(mSearchValue); +nsMsgNewsFolder::GetUnicodeName(nsAString & name) +{ + nsresult rv; + nsXPIDLString newsgroupName; + rv = GetName(getter_Copies(newsgroupName)); here, you can move the decl of rv to where you first use it. And tiny nit, instead of using NS_ENSURE_SUCCESS(rv, rv), you could do if (NS_SUCCEEDED(rv)) name = newsgroupName; return rv; - uri.Append(escapedName.get()); + uri.Append(escapedName); + //uri.Append(name); missed a commented out line here. rev the uuid on nsINntpIncomingServer.idl there are a number of places where you've changed code that wasn't using the interface parameter name standards - while you're in there, you could change the arg names. e.g., setAsSubscribed(const nsACString &path) should be setAsSubscribed(const nsACString &aPath); The only nit that needs to be fixed is rev'ing the uuid on nsINntpIncomingServer.idl

Comment on attachment 188215 [details] [diff] [review] Shaohua's patch plus mine (-uw patch) with dump/commented line removed asking for sr. I addressed David's nits except for the parameter renaming. I changed some of them to be compliant to our naming convention, but not all of them.

I did some quick tests with imap subscribe of folders with non-alphanumeric characters and it worked fine.

David, do you think we should hold off on this patch until Alpha 2 gets out the door?

I think we might as well go for it so this patch can get testing. The imap risk is actually pretty low. And the non-chinese newsgroup risk is probably pretty low as well.

Comment on attachment 188182 [details] [diff] [review] Shaohua's patch plus mine clear r? flag because David already reviewed the -uw patch

Comment on attachment 188215 [details] [diff] [review] Shaohua's patch plus mine (-uw patch) with dump/commented line removed Now that the trunk is open for beta work, now is a good time for us to get this in. Instead of putting this inline with the XUL file we should instead just put it inside of am-server.js, preferrably inside of either the onInit method or the onPreInit. Please make that change before checking in. Thanks!

Comment on attachment 188215 [details] [diff] [review] Shaohua's patch plus mine (-uw patch) with dump/commented line removed thanks for r/sr. I made a change mentioned by mscott in my tree. (I put it in onPreInit() ) This blocks both 1.8b4 and 1.1alpha and both David and Scott want this to be tested by landing it. My test result indiciates that it works fine, but letting others test it will help fix any possible regression.

BTW: A bustage fix was needed for this patch, a change to a Makefile was missing (are you normally building with enable-static?). Timeless probably checked in the correct for this (it's still building), see http://bonsai.mozilla.org/cvsview2.cgi?diff_mode=context&whitespace_mode=show&subdir=mozilla/mailnews/news/src&command=DIFF_FRAMESET&file=Makefile.in&rev1=1.52&rev2=1.53&root=/cvsroot for the change.

thanks for the note. For debug builds, I don't use 'enable-static'. I'll be more careful with this. I'll mark this as fixed when windows build goes through with timeless' patch.

Just to make this bug complete, another bustage fix was checked in, see http://tinderbox.mozilla.org/bonsai/cvsview2.cgi?diff_mode=context&whitespace_mode=show&subdir=mozilla/mailnews/news/build&command=DIFF_FRAMESET&file=Makefile.in&rev1=1.43&rev2=1.44&root=/cvsroot (the first bustage fix didn't fix it completly).

jshin, after I updated my build from the 19th to todays build (the 25th), I'm getting problems with my virtual folders. Virtual folders with spaces in them like: Review Requests are showing up twice in my folder pane. Once as: Review Requests and once as Review%20Requests related?

Found a bug of the patch... In Thunderbird 1.0+ zh-CN builds,we have set mailnews.view_default_charset to GB2312,but when I added a new news server,the default charset for this server is set to: chrome://messenger/locale/messenger.properties but not GB2312,I checked mailnews.js /mailnews/mailnews.js, line 215 -- pref("mailnews.view_default_charset", "chrome://messenger/locale/messenger.properties"); Is this the reason or we should change NS_GetUnicharPreferenceWithDefault(...) in nsNntpIncomingServer::GetCharset(...) to NS_GetLocalizedUnicharPreferenceWithDefault ?

(In reply to comment #148) > jshin, after I updated my build from the 19th to todays build (the 25th), I'm > getting problems with my virtual folders. > > Virtual folders with spaces in them like: > are showing up twice in my folder pane. Once as: > Review Requests > and once as > Review%20Requests > > related? I thought it might be, but a cursory look didn't lead to any suspicious spot. Moreover, my debug builds (one I made right after landing the patch and the other I've just built with the trunk source) don't have such a problem with a virtual folder whose name has a space. I diffed my tree against the cvs tree and I couldn't find anything likely to cause this difference between you and me. Aha... I made a virtual folder with a build with this patch included. I'll make a virtual folder with an older build and run a recent build after that. re: comment #149 yeah, we should use NS_GetLocalizedUnicharPreferenceWithDefault file a new bug, upload a patch and i'll quickly review it.

(In reply to comment #150) > yeah, we should use NS_GetLocalizedUnicharPreferenceWithDefault > file a new bug, upload a patch and i'll quickly review it. > filed a new Bug 302275 .

Scott, I've just tried 2005-07-25 nightly, but couldn't reproduce the problem you experienced. I made a virtual folder from a search result (with TB 1.0.2). Is there any other way?

*** Bug 147823 has been marked as a duplicate of this bug. ***

Hi shin, It seems that there is a regression on this bug in TB1.5.04. Please have a look on the screenshot.The tree display is broken. Thanks Holy

Dear all, This regression happens on 2.0 develop branch also.should we reopen this bug?