** Observed with 2/1/2000 Win32 build ** The message viewing display is currently controlled by the following preferences: user_pref("font.name.monospace.x-unicode", "Font_Name"); --> plain text mail body user_pref("font.name.serif.x-unicode", "Font_Name"); --> HTML mail body user_pref("font.size.fixed.x-unicode", 12); --> sizes in pixels user_pref("font.size.variable.x-unicode", 16); --> sizes in pixels What is wrong with this is that when we have the font pref UI working in M14, the user will not see the effects of their choices based on the language they choose (e.g. Japanese) but rather on the Unicode font choice no matter what the language of the msg is. So we need to somehow reflect the the language information of the original message as we send the display data to the layout. nhotta or erik should have an idea on how to effect this change in the code. I think we should get this done in synch with the Font Pref UI implementation, which is M14.

beta1 designation because the font pref UI will be done before then and it would not be good for users to have to control the viewing font size with no regard to language and only though the Unicode font category.

Assigning myself as QA contact.

CC'ing a bunch of people.

I'm guessing that MIME messages are converted to HTML in Unicode or UTF-8. Since the charset is Unicode-based, the font engine selects the Unicode font prefs. In order to select language-specific font prefs, we will need to pass the language from the MIME parser to the layout engine. If both HTTP and HTML is sent down, we can insert a Content-Language header into the HTTP part. If only HTML is sent down, we can attach a LANG attribute to the HTML element. However, both Content-Language and LANG are not implemented yet in the backend. We will probably need a separate bug for that, assigned to me. We will also need to move the charset->lang mapper from the device context to Cata's charset manager. Then the MIME engine can convert the MIME charset(s) to HTML LANG attributes. If there are multiple MIME body parts, we may need to attach several LANG attributes to various HTML elements.

Geeze, this sounds scary at this point. I can't tell you how much time I've spent with Naoki & company getting mail messages of all different types and languages to display properly. We do conversion to UTF-8 in libmime for message display of mail messages and attachments. I need to understand better what changes are being discussed here. I am in Mountain View until Feb 12 so perhaps we can have a meeting on this one. - rhp

I don't know the libmime code, but this change shouldn't be that hard. Each MIME body part has a Content-Type header. For example: Content-Type: text/plain; charset=iso-2022-jp libmime then converts that body part from iso-2022-jp to UTF-8, and that does not have to change. In fact, that MUST NOT change. If each body part in MIME gets converted to a <DIV> in HTML, then we simply add a LANG attribute to each <DIV>. Continuing the example: <DIV LANG="ja"> <!-- because iso-2022-jp is Japanese (ja) --> The LANG attribute can be attached to any HTML element, so this will work even if you're not using DIVs. What are you using?

Currently, there are no DIV containers for any parts of the message so this would be new. Also, are we talking header display as well as body? The headers are actually XUL documents now so that is a whole other animal. - rhp

Adding dependency, 26237 - implement LANG attribute handling in layout 26240 - Need a service to map charset name to language About the header display, I think we also want to specify language (but not for thread pane). There should be a way to specify lang in XUL.

If libmime doesn't currently use DIVs or any other type of container for MIME body parts, then we should meet to discuss this. XML does have language attributes, if I recall correctly. May be xml:lang?

momoi-san, Why is this Beta1? What does it look like on a standard Japanese system without extra Unicode (e.g., Cyberbit), Chinese and Korean fonts? My guess is that this will be OK for Beta1.

Scott, You need to see this. - rhp

In the future, we may want to generate LANG attributes to help with spellchecking, hyphenation, etc. So I don't know if it is a good idea to potentially overload this attribute. For example, if we generate LANG=x-western in order to help control font selection, what happens if a user wants to add LANG=fr to inform composer to use the French spell checker? Since this proposed usage if for internal use only, we could use an internal-only attribute (e.g., X-FONT-LANG). But then what happens if someone wants to save the mail to an HTML file?

My thinking on beta1 designation is that forcing users to use Unicode category to control mail display fonts. when in the rest of the components, they use more natural language based font settings, would lead to confusing user experience. All of this is based on the assumption that the font pref UI will be working at beta1. Look of displayis as important is in Mail as in Browser and Composer.

I did some expriment onhow plain text mail messages look in Japanese. When there is no specific setting for x-unicode for MS Gothic -10 and when you have other fonts like "Bitstream" on Japanese Windows, the display is terrible. We should not hoist this kind of sub-stadarnd display experience on Japanese users who use plain text mail (often long ones) a lot. Here's a comparative set of images -- 1) MS Gothic 10 is specified for x-unicode, 2) when there is no font specification at all, i.e. the default.

I agree that Japanese email needs to render reasonably well if we want to get any Beta testing, but for Beta which users do we need to address? For a user that doesn't touch his/her prefs and has the standard set of fonts that come with the Japanese OS, will viewing mail look reasonably good? My guess is that it will look OK because the font code will find a Japanese font to render the CJK Unicode glyphs. Your problem is that the font code is using the ugly Cyberbit font for rendering UTF-8 pages in your config. What happens if you clear out all your prefs and remove Cyberbit and any Chinese and Korean fonts? I think we need to spend some time to design the "right" solution and we don't have much time left for M14...

Bob, the problem with the current situation is that what you get in the way of plain text mail display is mostly left to chances as to what other fonts the users have on their system. If they happen to have some font which comes before a proper Japanese font under Unicode, then that font will be used instead. I don't know how typical it is these days for JPN users to have only the fonts they get with the OSs they purchase. It would be better if we give the display control to users in an easily understandable way, and sooner rather than later. I agree with you that we need to find a "right" solution...and if it needs to take us beyond Beta1, so be it. But please address this problem soon after. I cannot stress enough that "how it looks" is a critical factor in user acceptance.

I agree with Bob that we should use something other than "LANG" for this, so that we don't clash with the normal use of LANG in HTML.

Can we use FACE?

FONT FACE doesn't seem like such a good idea because the MIME body part might be text/html with embedded FONT FACE tags already. Those embedded FONT FACE tags would override the MIME engine's FONT FACEs. If we pass the language group down, then FONT FACEs like "serif" can still have the correct effect. I'm wondering whether using something other than LANG is really a good idea, actually. If we have both LANG and X-FONT-LANG, then inheritance and overriding could become tricky in the HTML engine. The RFC for LANG codes allows private "x-" LANG codes.

Removing beta1 keyword. Erik will look into optimizing the font enumerator for M14 to favor the current locale, so it will priortize Japanese fonts for UTF-8. Post-Beta1, we need to come up with a full solution.

*** Bug 26852 has been marked as a duplicate of this bug. ***

Put beta2, adding ftang to cc. People who have suggestion how this should be done, please put your comments in here.

Although this bug is depending on 26237, it is possible to generate LANG because charset to language mapping is available (26240). So having the language from a charset name then what exactly should be generated? I need an example. Erik, what HTML lines need to be generated if the lang is japanese? It is possible to try that now and see if any side effect of that.

Frank and I discussed it this morning. Frank says that it might be cleaner to avoid putting our "language groups" like "x-western" in HTML LANG attributes, since they aren't really languages. So he suggested passing the font directly. After mapping the charset to a lang group, you can get the font from the prefs API, using a name like "font.name.monospace.x-western". Monospace is the fixed width font. Alternatively, you can select the default HTML font, which is in the pref called "font.default", which will be either serif or sans-serif, which you can then append, e.g. "font.name.serif.x-western". Then, in the HTML, I don't know what the mail client is using, but if you are using DIVs, you might say: <div style="font-family: MS Gothic">...</div> for a MIME body-part that has a Japanese charset like iso-2022-jp.

Would we ever save the HTML format of the emails or is this only for display? If we ever save it, do we want to hard code the fontname? If so, this would not be platform independent and font availability varies per system even on the same platform. Also, is it possible for the HTML email already have <FONT FACE=serif> within the <div ...> and therefore the the <div style="font-family: MS Gothic">...</div> would not have any effect?

> is this only for display? yes, this is for display only

I put the attached code in libmime and did some testing. It seems to be working. But I thought it is not natural the user has to change font per language in order to change the font of mail/news. How about having mail/news specific font setting like, "font.name.serif.mailnews", "font.size.variable.mailnews"? For UI, this could be added to the existing font pref UI or we may have a mail/news specific font setting separately. I also thought that the font change does not have to affect inline attachments. Only the main body needs to be affected.

Putting on [nsbeta2+] radar.

I checked in the backend support. Please try following prefs (everything is disabled as default). "mailnews.language_sensitive_font" - if set to "true" then browser's font setting is used for mail/news If above pref is "false" then you can apply mail/news specific font setting by following prefs. "mailnews.font.name.html" "mailnews.font.size.html" "mailnews.font.name.plain" "mailnews.font.size.plain" If we verify those prefs are working then I can check in the default values and UI.

> If above pref is "false" then you can apply mail/news specific font setting by > following prefs. > "mailnews.font.name.plain" > "mailnews.font.size.plain" Note, that this propably collides with the "mail.fixed_width_messages" (sat in "Prefs|Mail and Newsgroups") pref, logically and in the code. As for the code side, I set "font-family: -moz-fixed" (not yet checked in), if true, and depend on HTMLish defaults (i.e. set nothing), if false.

>Note, that this propably collides with the "mail.fixed_width_messages" yes, we need to resolve this before we activate those prefs. I enabled language_sensitive_font today, so the mail/news font will listen the font pref setting of the user (but the font pref UI is currently gone). So I am closing this bug. If we want to activate the mailnews prefs (font/size), we need the pref UI and need to resolve the issue mentioned by mozilla@bucksch.org.

In 4.x, the option to use Fixed or Varaible-width font apply to cases where the font settings are sensitive to encoding differences, i.e. ...Apperance | Fonts | for the encoding. The current new feature sets fonts only in case when the simply wants the default fonts to be applied to all messages. This in effect means that you are specifying the fonts for your default language only. An obvious way to have both of them working is for the "fixed" or "variable" option to apply only if the "mailnews.language_sensitive_font" is "false". Thus it would be best to have these 2 settings under the same section to make them more understandable. I'll open a new bug to settle this issue.

Reopen this because the default font setting does not cooporate well with the existing plain text related tags in libmime. Need to integrate the default font into the plain text font setting implementation (vairable/fixed). Also there are regresssion for format=flowed (38491, 38511).

nhotta, you're aware of bug 37365, aren't you? I'm not keen on that (37365) bug, but it (and bug 26182 here) is influenced by my bug 31906. About bug 31906: It recognizes nested quotes and generally clean up the tags we emit in mimetpla.cpp. Basically (=simplified) <div class=text-plain><pre>text</pre><blockquote><pre>bla</pre></blockquote><pre>bla</pre></div> with div.text-plain { white-space: -moz-pre-wrap } div.text-plain pre { font-family: inherit; white-space: inherit; margin: inherit; } A relatively recent version of the code can be found at <http://bugzilla.mozilla.org/showattachment.cgi?attach_id=7013>. I leave it up to you, what we'll do. But if you work on it again, please make sure, that things don't break with the fix for 31906.

> About bug 31906 I forgot the if (!obj->options->variable_width_plaintext_p) openingDiv += " style=\"font-family: -moz-fixed\""; //if (obj->options->wrap_long_lines_p) ... XXX pre wrap of course. Note, that I set the fixed case explicitly now; in other cases, the variable width font is inherited, i.e. honors the general "serif/sans-serif" preference. "div.text-plain pre { font-family: inherit }" makes this possible (avoids overwriting of the variable width font with the <pre>). Yes, it's tricky (but your feature is no less :) ). That's because I need to emmit good HTML for other HTML viewers (HTML send), and want to make things configurable via CSS in Mozilla. Suggestions welcome.

Because the change of bug 31906 is quite extensive, I'll hold my change for format=fixed case. Hopefully, we are going to have less conflict (since the new code will inherit the font setting). Meanwhile, I am going to do a small change to format=flowed case to fix the regression bugs.

Here is a current status. Done for html and plain format=flowed. The font pref setting is effective to mail/news view (not controlled by unicode setting, e.g. the user can change "Western" font setting to change us-ascii mail view). For plain text format=fixed, wait for 31906 fix. I have a patch for the current tip, it may need some modification after 31906 fix.

Sorry for saying that so late, but I just realized, that already the implementation strategy of this bug somewhat conflicts with teh goals of my bug 31906. The idea of 31906 is to - hardcode only pure HTML (which even Lynx can understand) - wrap everything in semantical (div) classes - specify all Moz specific messing in CSS. (Given, I don't reach those goals completely yet, but I'm close.) This has several advantages: - It makes emitting good HTML for quoting in HTML send easier. - It enables using a HTML viewer other than NGLayout for Mozilla Mailnews. - The user (and vendor) can easily change the whole behaviour. E.g. if he doesn't like the <pre>, he can use |div.text-flowed pre {whitespace: inherit; font-family: inherit}|. As a result, you won't even know, if there'll be a <pre> or whatever later. Is there any way to make the font setting less intrusive, i.e. let layout choose the right font family? BTW: Is the "sans-serif/serif" setting honor currently?

> Is there any way to make the font setting less intrusive, i.e. let layout > choose the right font family? delete the " family".

We may specify LANG attributes.

nhotta, your recent changes are surely an improvement, especially the option to only get the font without tag. However, the issues I mentioned remain, of course (again for not raising them earlier). I thought, there were other problems with LANG. Maybe setting a CHARSET hint for layout? (Just a guess, I don't know I18N or layout.)

> (again for not raising them earlier). (again sorry for not raising them earlier). Well, I can live with the current state in the short term. It's sub-optimal, but it's not the only one (there is still some other, unrelated hardcoded formatting) :-(. However, I still think, doing this kind of stuff in layout would save us some headache.

I checked in the patch for plain text format=fixed. Ben, let us know if more data is needed for your change, we can provide functions to get charset, lang, font.

fixed in M16

*** Bug 28274 has been marked as a duplicate of this bug. ***

There's no way to still use the unicode font setting, right? What would be the right way to include them, i.e. how should the prefs actually read? I can write the actual code for plain text and flowed. Relevance: Maybe I just don't get it, but the unicode font settings worked just right for me (on Linux with some additional fonts), including Japanese text.

What about +---- use mailnews.font.name/size... true mailnews.font.use (or similar) -+ +- this bug false true +- mailnews.language_sensitive_font -+ false +- unicode where mailnews.font.use defaults to false? > I can write the actual code for plain text and flowed. The actual code might even already be there. Maybe, the only changes are to be done in mimemoz2.cpp. I'll into it, when/if I have more time.

Filed bug 39402 about the unicode font pref.

*** Bug 28274 has been marked as a duplicate of this bug. ***

This doesn't seem to be working. I removed all my font prefs from prefs.js, then went through the font dialog to set my fixed width font pref for "Western" (the default) to "clean" 14. Mail/Viewing Messages says it's using Western character coding and fixed width font, but it's not using clean, it's using courier or something like it to display fixed-width messages. This was the issue in bug 28274, which was marked as a dup of this one; I'm not sure I understand all the issues addressed in this bug, but I think duping 28274 is correct and this one is the one I should reopen. Sorry if that's wrong.

Naoki, would you explain if this is supposed to work for both HTML and Plain text mail msgs? This is not working for Japanese msgs, either, when specific fonts are set for it.

Accepting, I see it does not work in case of format=flowed.

Please wait with this bug for the checkin for bug 31906. It may fix this, and I am tired of seeing my patches botrotting.

Okay, I will notify you before I do changes.

About the Japanese problem momoi mentioned, that happens if the font name is not ASCII. I have a fix for that.

nhotta found out, that my fix for bug 31906 fixed this as well. Has been checked in by rhp. FIXED.

I checked in a fix for Non ASCII font name problem yesterday.

** Checked with 6/9/2000 Win32 build ** I'm going to re-open this because I cannot see the effect of Monospace font setting with Fixed font option to work for either format=fixed or format=flowed plain text msgs. It seems to be working for HTML mail msgs. I use Win98/NT4 Japanese and set the Monospace font to MS Gothic. The size change is effective in these plain text msgs but not font face ones.

I cannot reproduce it. I used 2000060908 win32 build running WinNT 4 J. I tested the second i18n smoketest message which is plain text format=fixed. I changed mono space japanese font from MS Mincho to MS Gothic and I was able to see the face change.

This is very interesting. I've looked through my font list to figuire out just what font we are using. It turns out on my machine, we are using MS-Hei font for GB2312! I have a number of other fonts including big Unicode fonts (3) and then Chinese and Korean fonts from MS. Wonder if erik knows why this is so.

Momoi san, could you isolate the problem and file a separate bug if it only happens under the special environment?

Closing again, please verify with a standard environment and file a separate bug for the problem which happens under the special environment.

Xianglan, please take a look at this bug for verification. Your system should have only standard Windows fonts for Japanese when you look at this bug. I have many other fonts and that is affecting the results. This bug is for the basic verification of font setting and plain test msgs. I have other issues but they will filed as separate bugs.

Verified with 2000062709 build on Ja NT 4.0. It's fixed.