TO reproduce this: 1. Copy and paste a Ja text in unicode to your XUL file as the title text of the top level window. And run the client to open the affected window. 2. On Linux, under Ja env. (ja_JP.ujis), the title text renders as garbage. Same thing would happen on Mac. This is a problem similar to http://bugzilla.mozilla.org/show_bug.cgi?id=7822

Log a bug to keep track of this problem.

Tague, on the Mac, can I copy the code from nsMenu into nsMenuBar for menu title internationalization, or is there something else I need to do? As for the other platforms, I believe XPMenus fixes this issue.

XPmenus should have nothing to do with title bars.

You're right, I mistakenly read this bug as top level menu titles. So why do I have this bug then?

Thought that this is an XPWidgets related problem :-). Who might be the lucky owner, any idea?

Eh, I'm going to assign this to danm with this note: I believe our window titles are as internationalized as they're going to get. Tague, do you agree? If so, lets close this bug...

sarri :- no, this is still a valid bug. right now there is extraneous ToNewCString calls in setting the top level window title. non-english characters are getting lost even on non-english systems. for instance, on my japanese NT box, Japanese page titles are displayed as "....". that doesn't work. the baseline functionality is that the window title has to draw correctly if the langauge can be rendered in the system script -- meaning a japanese system has to correctly get japanese titles, and english system doesn't. i'm not sure how far we want to push owner-draw of menu titles and a full multi-lingual menu title. due to technical limitations, i'm willing to accept the baseline functionality for Mac (Japanese only on a Japanese system), but I think we should push for correct Unicode display on windows and unix if possible.

Tao: could you please attach a Japanese localized XUL file to this bug, so that people have something to test with. You can also view the problem by visiting a Japanese website with a Japanese tittle page. You can use http://www.netscape.com/ja to demonstrate the problem. You should get Netcenter へようこそ" (welcome to Netcenter) in your titlebar, but instead with seamonkey you get .... because of the ToNewCString problem outlined above. Look at the code I gave to Saari for menus as a boilerplate.

how critical is this for beta1? Tao's test case isn't posted yet. Can we figure out if there is a way to get help and right ownership on this?

This bug can be easily reproduced by loading Netscape's Japanese home page, http://www.netscape.com/ja. But, I'll see if there is a dialog I can hack with and also retest this bug. If I attach a Seamonkey *.dtd or *.xul files, they might become outdated when the original *.dtd file got changed and prevent the XUL page from being loaded correctly. Please let me know if this bug can't be reproduced by loading the Japanese page. Inserting the Ja home page to the URL field. Thanks

this is a beta-1 blocker.

jliu (used work for Netscape) have some old code in the gromit tree which use NC stuff (Not Client area) to draw unicode for Window. That might help you solve this one Window.

For window platform, jliu two years ago add the following code to make it display Unicode text. http://lxr.mozilla.org/mozilla/source/cmd/winfe/genframe.cpp#301 219 220 221 #ifdef XP_WIN32 222 // jliu added the following to support CJK caption print 223 int16 CGenericFrame::GetTitleWinCSID() 224 { 225 CWinCX *pContext = GetMainWinContext(); 226 MWContext* pMWContext = NULL; 227 if( pContext == NULL || ( pMWContext = pContext->GetContext() ) == NULL ){ 228 return -1; 229 } 230 INTL_CharSetInfo csi = LO_GetDocumentCharacterSetInfo(pMWContext); 231 232 return INTL_GetCSIWinCSID(csi); 233 } 234 235 //////////////////////////////////////////////////////////////////// 236 // this function is called outside to get the wincsid 237 // and then setup the Caption Font. 238 // 239 void CGenericFrame::SetupCapFont( int16 csid, BOOL force ) 240 { 241 if( !force && ( m_csid == csid ) && ( hCapFont.m_hObject != NULL ) ) 242 return; 243 244 // set the wincsid 245 if( csid <= 0 ) 246 m_csid = CIntlWin::GetSystemLocaleCsid(); 247 else 248 m_csid = csid; 249 250 // set the caption print font 251 if( hCapFont.m_hObject != NULL ) 252 hCapFont.DeleteObject(); 253 254 // get the system caption print font first 255 NONCLIENTMETRICS nc; 256 nc.cbSize = sizeof(NONCLIENTMETRICS); 257 BOOL b = SystemParametersInfo( SPI_GETNONCLIENTMETRICS, sizeof(NONCLIENTMETRICS), &nc, 0 ); 258 XP_ASSERT( b ); 259 260 LOGFONT lf = nc.lfCaptionFont; // get the logic font used to paint the caption 261 csid = CIntlWin::GetSystemLocaleCsid(); 262 if( force || ( m_csid != csid ) ){ // in case user changes system font 263 lf.lfCharSet = IntlGetLfCharset( m_csid ); 264 _tcscpy( lf.lfFaceName, IntlGetUIPropFaceName( m_csid ) ); 265 hCapFont.CreateFontIndirect( &lf ); 266 } 267 268 // create the default font if the above step failure 269 if( hCapFont.m_hObject == NULL ){ 270 hCapFont.CreateFontIndirect( &nc.lfCaptionFont ); 271 m_csid = csid; 272 } 273 } 274 275 ////////////////// 276 // User changed system settings, such as font. Or changed 277 // encoding method, coming from compfrm.cpp 278 // Invalidate everything to generate anew. 279 // 280 LRESULT CGenericFrame::OnSettingChange( WPARAM, LPARAM ) 281 { 282 SetupCapFont( GetTitleWinCSID(), TRUE ); 283 SendMessage(WM_NCPAINT); 284 return 0; 285 } 286 287 288 ////////////////// 289 // Someone called SetWindowText: paint new text. 290 // 291 LRESULT CGenericFrame::OnSetText(WPARAM wParam, LPARAM lParam) 292 { 293 SetupCapFont( GetTitleWinCSID() ); 294 295 // Turn WS_VISIBLE style off before calling Windows to 296 // set the text, then turn it back on again after. 297 // 298 DWORD dwStyle = GetStyle(); 299 if( dwStyle & WS_VISIBLE ) 300 SetWindowLong( m_hWnd, GWL_STYLE, dwStyle & ~WS_VISIBLE ); 301 LRESULT lRet = DefWindowProc(WM_SETTEXT, wParam, lParam); 302 if( dwStyle & WS_VISIBLE ) 303 SetWindowLong( m_hWnd, GWL_STYLE, dwStyle ); 304 305 SendMessage(WM_NCPAINT); // paint non-client area (frame too) 306 307 return lRet; 308 } 309 310 ////////////////// 311 // Non-client area (de)activated: paint it 312 // 313 BOOL CGenericFrame::OnNcActivate(BOOL bActive) 314 { 315 if( m_nFlags & WF_STAYACTIVE ) 316 bActive = TRUE; 317 if( !IsWindowEnabled() ) 318 bActive = FALSE; 319 if( bActive == m_bActive ) 320 return TRUE; 321 322 MSG& msg = AfxGetThreadState()->m_lastSentMsg; 323 if( msg.wParam > 1 ) 324 msg.wParam = TRUE; 325 m_bActive = bActive = msg.wParam; 326 327 // Turn WS_VISIBLE off before calling DefWindowProc, 328 // so DefWindowProc won't paint and thereby cause flicker. 329 // 330 DWORD dwStyle = GetStyle(); 331 if( dwStyle & WS_VISIBLE ) 332 ::SetWindowLong(m_hWnd, GWL_STYLE, (dwStyle & ~WS_VISIBLE)); 333 DefWindowProc(WM_NCACTIVATE, bActive, 0L); // don't call CFrameWnd::OnNcActivate() 334 if( dwStyle & WS_VISIBLE ) 335 ::SetWindowLong(m_hWnd, GWL_STYLE, dwStyle); 336 337 SendMessage( WM_NCPAINT ); // paint non-client area and frame 338 339 return TRUE; 340 } 341 342 343 void CGenericFrame::OnNcPaint() 344 { 345 DWORD dwStyle = GetStyle(); 346 if( !( dwStyle & WS_CAPTION ) ){ 347 CFrameWnd::OnNcPaint(); // do the default one first 348 return; 349 } 350 351 CSize szFrame = (dwStyle & WS_THICKFRAME) ? 352 CSize(GetSystemMetrics(SM_CXSIZEFRAME), 353 GetSystemMetrics(SM_CYSIZEFRAME)) : 354 CSize(GetSystemMetrics(SM_CXFIXEDFRAME), 355 GetSystemMetrics(SM_CYFIXEDFRAME)); 356 357 int dxIcon = GetSystemMetrics(SM_CXSIZE); // width of caption icon/button 358 359 // Compute rectangle 360 CRect rc; // window rect in screen coords 361 GetWindowRect( &rc ); 362 rc.left += szFrame.cx; // frame 363 rc.right -= szFrame.cx; // frame 364 rc.top += szFrame.cy; // top = end of frame 365 rc.bottom = rc.top + GetSystemMetrics(SM_CYCAPTION) // height of caption 366 - GetSystemMetrics(SM_CYBORDER); // minus gray shadow border 367 368 369 MSG& msg = AfxGetThreadState()->m_lastSentMsg; 370 371 // Don't paint if the caption doesn't lie within the region. 372 // 373 if (msg.wParam > 1 && !::RectInRegion((HRGN)msg.wParam, &rc)) { 374 CFrameWnd::OnNcPaint(); // do the default one first 375 return; // quit 376 } 377 378 379 // setup the update region, include the title text region 380 381 CRect rectCaption; 382 GetWindowRect( &rectCaption ); 383 384 CWindowDC dc(this); 385 CFont* hFontOld = dc.SelectObject( &hCapFont ); 386 387 HRGN hRgnCaption = ::CreateRectRgnIndirect(&rc); 388 HRGN hRgnNew = ::CreateRectRgnIndirect(&rc); 389 if( msg.wParam > 1 ){ 390 // wParam is a valid region: subtract caption from it 391 ::CombineRgn( hRgnNew, (HRGN)msg.wParam, hRgnCaption, RGN_OR ); 392 } else { 393 // wParam is not a valid region: create one that's the whole window 394 DeleteObject( hRgnNew ); 395 hRgnNew = ::CreateRectRgnIndirect(&rectCaption); 396 } 397 398 // call the default NcPaint 399 400 WPARAM savewp = msg.wParam; // save original wParam 401 msg.wParam = (WPARAM)hRgnNew; // set new region for DefWindowProc 402 CFrameWnd::OnNcPaint(); // paint the caption bar except title text 403 DeleteObject( hRgnCaption ); // clean up 404 DeleteObject( hRgnNew ); // clean up 405 msg.wParam = savewp; // restore original wParam 406 407 408 // Compute rectangle 409 // Within the basic button rectangle, Windows 95 uses a 1 or 2 pixel border 410 // Icon has 2 pixel border on left, 1 pixel on top/bottom, 0 right 411 // Close box has a 2 pixel border on all sides but left, which is zero 412 // Minimize button has 2 pixel border on all sides but right. 413 414 rectCaption -= rectCaption.TopLeft(); // convert caption rectangle origin to (0,0) 415 rectCaption.left += (szFrame.cx + 2 + dxIcon); 416 rectCaption.right -= (szFrame.cx + 2 + dxIcon); 417 rectCaption.top += szFrame.cy; // top = end of frame 418 rectCaption.bottom = rectCaption.top + GetSystemMetrics(SM_CYCAPTION) // height of caption 419 - GetSystemMetrics(SM_CYBORDER); // minus gray shadow border 420 421 422 // i don't know why MS supports this feature ... 423 if( ( dwStyle & WS_MAXIMIZEBOX ) || ( dwStyle & WS_MINIMIZEBOX ) ){ 424 rectCaption.right -= ( dxIcon + 2 + dxIcon ); 425 if( capStyle != DT_LEFT ) // for NT 3.51 426 rectCaption.right += dxIcon + 4; 427 } 428 429 // create a brush for caption background 430 CBrush brCaption; 431 brCaption.CreateSolidBrush( 432 ::GetSysColor( m_bActive ? COLOR_ACTIVECAPTION : COLOR_INACTIVECAPTION ) ); 433 434 CString strTitle; 435 GetWindowText( strTitle ); 436 437 //dc.FillRect(&rectCaption, &brCaption); 438 dc.SetTextColor(::GetSysColor(m_bActive ? COLOR_CAPTIONTEXT : COLOR_INACTIVECAPTIONTEXT)); 439 dc.SetBkMode(TRANSPARENT); 440 441 int titleLen = _tcslen( strTitle ); 442 UINT cs = capStyle; 443 if( cs != DT_LEFT ){ // it is NT 3.51 444 CSize size = dc.GetTextExtent( strTitle, titleLen ); 445 if( size.cx >= rectCaption.Width() ) 446 cs = DT_LEFT; 447 } 448 CIntlWin::DrawText( m_csid, (HDC)dc, strTitle.GetBuffer( titleLen ), titleLen, 449 &rectCaption, cs|DT_VCENTER|DT_SINGLELINE|DT_END_ELLIPSIS ); 450 451 dc.SelectObject(hFontOld); 452 } 453 454 #endif // #ifdef XP_WIN32 455 The main point is he use OnNcPaint to draw the window title by him self. The code should be simpler because in that time he need also keep track of the csid of the origional text but now we can always count nsString as unicode.

A pox on whoever moved this back to M11 on me. While I appreciate Frank's digging up the windows Nav code to handle this problem, it's not trivial, especially in an XP sense, and garbage in the window titlebar on some systems doesn't stack up in comparison to my other M11 bugs. The moniker "beta-1 blocker" begins to lose its value when it's overused so. I'm leaving it M11 in the vain hope that the combination of that and Help Wanted in the status whiteboard will attract outside interest, but in the interest of honesty, plan on its being shoved back to M15 along with a lot of other bugs, as the deadline approaches. I do remain confused why i18n doesn't want this bug.

Reopening this.....this is not a laterable bug - this is a beta blocker. Japanese window title text does not correctly display on a Japanese system. The window titleing code is incorrect and the owners need to fix this. There is a big misconception that the i18n group is reponsible for anything that doesn't work in languages other than English. That is not the case -- the "core" (non-i18n teams) have the responsibility to fix i18n bugs in their code.

clearing resolution.

Pasting from the Seamonkey Beta critera :- international users will be able to use seamonkey: browse sites, compose, read and write mail in Japanese and Latin 1 languages. this bug falls into that criteria

danm: I know you are busy for a lot of thing. But please understand no one "A pox on whoever moved this back to M11 on me" before 09/17/99 11:59. Please read the "View Bug Activity" log and you will find out it is yourself move this bug from M10 to M11 to yourself at 1999-09-07 18:01:10. Mark this as RESOLVED LATER without fixing it is not acceptable. Thanks tague for reopening it. Please do not feel we are pushing your (we don't). We all know this is not trivial but please do not ignore/underestimate the need. Except tague REOPEN this bug after your mark it "RESOLVED LATER", no one from i18ngrp ever change the target_milestone/bug_status/resolution field of this bug (Again, read "View Bug Activity" for details)

Well, a pox on me, then. I had intended to move this bug past M11. Thought I had. Nevertheless, in marketing's criterion quoted above by tague, I fail to see how garbage in the window title affects useability. I'm calling in an arbitrator.

setting p3 for m14. If by some miracle we finish with the critical beta-blocker problems that affect _every_ user, then we'll see about possibly looking at this.

let's get real, what you are saying is we aren't going to fix things unless they affect english users. that is not an acceptable prioritization of bugs. this is essential for shipping the i18n beta, we have a product goals that we have to make -- and just because "all" users aren't affected doesn't mean they aren't valid goals.

Sigh. C'mon guys, this _is_ us trying to be real. There's no time for this in M11. Garbage in the titlebar doesn't stack up against my other bugs. Something has to slip. This won't be the only one. I suppose you gave me this bug because you're out of time, but so am I.

This isn't a UCan vs I18N issue for us, it is a matter of fixing bugs that affect everyone or only a subset of users. We have to fix the critical bugs that affect everyone first. I would declare and ship beta with this bug if we didn't have time to fix it, so I'm not putting it on my must-fix-for-beta list. I'm willing to listen to any rational argument why I'm full of it, and you're also welcome to petition higher authorities.

Everybody cool down. 1. Please read the previous "Comments From tague@netscape.com 08/25/99 19:46." There are two level of fix: Level 1- Display French/German text correctly on US window/mac/Linux. and display Japanese text correctly on Japanse window/mac/linux. It will display Japanese as garbage in US window/mac/Linux and French/German text as garbage in Japanese window. To do this , it is not too difficult. It need to replace the nsString::ToNewCString code with the correct convert to CP_ACP/SystemScript/locale_charset call. 2. Level 2- Need to use MW_NCPAINT on Window (and other tech on Mac/Linux) . This need jliu's NCPAINT work and is not trival. I think it is reasonable to shoot for level 1 in beta1 and level 2 for post beta. If you feel display title as garbage text is *not important* in beta,then why don't we remove the title display text at all (display blank) so it won't make our Japanese user feel bad? Can you accept that ?

trudelle: Please understand that unfortunately your "a subset of users" is our "everyone".

OK... James Toder kindly submit a simple fix for the Level 1 on window. Can we now move this to M12 and let danm merge that patch ? Dan, the mail is also send to you. But in case you lost it , you can find it in the mozilla-i18n newsgroup. Move this bug back to M12 so danm can check in the patch.

Please understand, "it is a matter of fixing bugs that affect everyone or only a subset of users." is a bad criteria for priority. You won't allow the product go beta if the title always display Y as X and X as Y , right ? (It will then only affect the subset of users which view pages that have title including character X and Y. ) Actually, this IS a bug which affect everyone. The page won't display any Non ASCII in the title correctly, including Tradmark, copyright, registration mark, etc. It IS a bug which affect everyone. Unless your subset of user only referred to those user who do not view the page title which include those characters. For example, the english page http://microsoft.com/presspass/features/1999/09-20sparks.htm won't display correctly on Mac and Linux (Probably not even window, without James' patch)

James Toder's partial unicode-to-multibyte patch checked in.

*** Bug 2426 has been marked as a duplicate of this bug. ***

*** Bug 33597 has been marked as a duplicate of this bug. ***

This bug has been RESOLVED (at least partially) for Linux/Unix/X11 by Frank's patch dated January 4, 2000 (see my comment on bug 2426) in that the window title bar can correctly display non-ISO-8859-1 strings as long as the locale under which window manager is running is the same as the locale/encoding in which the page viewed in browser(i.e. title) is encoded. Unix/Linux/X11 version correctly passes along the page title to window manager using the way specified in X11 ICCCM. By using the corresponding spec for interclient communication under MacOS/Windows might well solve the problem for MacOS/Windows. If you want the Japanese/Russian/Chinese title(encoded in SJIS/ISO-2022-JP/EUC-JP,KOI8-R,Big5,EUC-CN/GB2312) to be correctly rendered in the title bar(**realm of window manager**) of Korean window manager, I guess you have to come up with a way to bypass the "normal" mechanism by which window title bar is rendered. There could be a easy way out provided that the window manager is running under a locale with UTF-8 encoding and both window manager and Mozilla agree on the way to exchange UTF-8 data. However, Linux doesn't yet have such a locale(while Solaris and AIX have). In summary, this problem has to be split into two probelms: 1. the case in which window manager locale/encoding is the same as the web page locale/encoding 2. the case in which they're DIFFERENT. The first case is taken care of by Frank's patch for Unix/Linux/X11 version( The same is true of Netscape 4.x and 3.x) and could be dealt with similarly in MacOS/MS-Windows.

*** Bug 34495 has been marked as a duplicate of this bug. ***

Mass moving M17 bugs to M18

mass-moving all bugs to m21 that are not dofood+, or nsbeta2+

*** Bug 48169 has been marked as a duplicate of this bug. ***

*** Bug 56970 has been marked as a duplicate of this bug. ***

*** Bug 61126 has been marked as a duplicate of this bug. ***

*** Bug 75344 has been marked as a duplicate of this bug. ***

For X11, we can use the NET_WM standard (http://www.freedesktop.org/standards/wm-spec/) to hint compliant window managers of the correct UTF8_STRING for the window title. According to the standard, a _NET_WM_NAME(UTF8_STRING) window property, when set, takes percedence over the WM_NAME(STRING) property. The only window manager supporting this currently is Kwin (KDE 2's WM), but it's adopted by both KDE and GTK+ 2.0. A patch for mozilla/widget/src/gtk/nsWindow.cpp is attached. Note: right-to-left language titles would appear reversed in Kwin since Kwin wouldn't handle BiDi until ported to (upcoming) Qt 3.0. It's not up to Mozilla to do BiDi of the window title, methinks.

13 dups. Marking mostfreq.

sure r=pavlov

+ char * utf8_title = NS_ConvertUCS2toUTF8(aTitle).get(); + That's never released. Should it be?

*** Bug 82950 has been marked as a duplicate of this bug. ***

The first 8 characters of an English title with length > 30 are replaced with 4 garbage characters using buildid 2001062821. see http://www.mybc.com/ for an example Window Manager is kde 2.1.1

Ilya's patch is in (28 June late afternoon). (Responding to Blizzard's comment above: the string isn't leaked; it's just a pointer to a stack-allocated string object's internal data.) And www.mybc.com behaves fine on my machine/build.

*** Bug 82252 has been marked as a duplicate of this bug. ***

Yes, there was a bug causing some garbage characters -- I should've tested it better! I did things the wrong way (e.g. grabbing the .data() once). Anyhow, it's all fixed now and should always work (tried on www.rambler.ru, www.yandex.ru, www.msn.co.il and my own test page). Attaching a diff -u against the CVS ...

With KDE2.2beta1 (with my latest Mozilla patch), I've noticed a random behavior from KDE's window manager Kwin, which adds a garbage character in it's own _NET_WM_VISIBLE_NAME property sometimes. From xprop -spy, I'm certain Mozilla assigns the right value, and KDE doesn't properly conform to the UTF8_STRING standard -- so I will take it as my mission to get that bug in KDE fixed. It's not a problem on Mozilla's side.

I tested this patch witch current trunk and it WFM. Red Hat 7.1/KDE2.1.1

r=blizzard

Please take a look at bug 89651 - could it be caused by these patches?

It's very possible that bug is the same as this one.

I duped my bug 89651 against bug 88384 It's really ugly to see almost all titles in windows being screwed up I am using kde 2.1.1, xfree 4.1.0 Is this one trunk-only? I think this is a blocker of the 0.9.3 release. What will users think of the product when they see those chars?

I just submitted to KDE this problem Please wait until i get a response for them (i invited them in here too)

Instead of |strlen(utf8_title.get())| you can just do |utf8_title.Length()|. The length of an nsCString is stored in it, and should save you having to scan for |0| each time.

r=jag

sr=blizzard

Patch checked in.

I'm wodnering if anybody has paid attention to the following article found in MSDN archive : http://www.microsoft.com/msj/1198/multilang/multilang.html (Supporting Multilanguage Text Layout and complex Scripts with Windows NT 5.0) I guess many have, but seem to have forgotten to bring it up here or haven't brought up here because it's not applicable to Mozilla for some reason unknown to me. Anyway, just in case it hasn't caught attention of anyone involved with this bug, I'm adding this comment. Around the end of the article, four strategies are presented to support multilingual applications for both Windows 9x/ME and Windows NT4/2000. It seems like the fourth approach might as well be considered for use in Mozilla. Below is an excerpt from the article: MSDN> Strategy 4 Detect the system and explicitly call the W APIs for WindowsNT MSDN>and the A routines for Windows 95 and Windows 98. The application MSDN> registers itself as a Unicode application on Windows NT and as an ANSI MSDN> application on Windows 95 and Windows 98. MSDN> The easiest way to implement this approach is to write a set of MSDN> functions, say U routines, that parallel the Win32 W and A routines. Your MSDN> application first calls GetVersionEx to detect the system, and stores MSDN> that information into a global variable: Implementing CreateWindowExU and CreateWindowU the way described above might(or might not) be easier than incorporating jilu's approach ftang dug up (implementing Mozilla's own drawing routine). Last week on Unicode mailing list, somebody asked a question related to this issue and one of answers was to use CreateWindowExW (on Win2k/NT4). Just FYI, I put it up at http://www.jshin.net/~jungshik/i18n/mswintitle.html

I'm sorry to bother again. In my previous comment, I forgot that Mozilla needs to display strings (not supported by the present locale) in the title bar both under Windows 9x/ME and Windows NT4/2k. The approach I suggested solves the problem (at both levels outlined by ftang in his comment back in 1999) for Windows NT4/2k but doesn't solve the 2nd level problem (that is displaying Japanese string in French Windows) for Win 9x/ME. I guess what MS IE 5.x does is follow the strategy 4 (in MSDN article I quoted in my previous article) so that it can display Japanese string in the window title bar in French Windows NT4/2k while it cannot display Japanese string in French Windows 9x/ME (thus, it displays the URL of the page instead). If the goal is to display Japanese string in French Windows 9x/ME as well as in French Windows 2k/NT4, I think adopting jilu's code from NS 4.x into Mozilla is the way to go. If Mozilla can be content with doing what MS IE 5.x does, strategy 4 could be easier (but I'm not sure).

FYI, there are bugs filed about this problem on specific OS's...bug 74753 (Linux), bug 36689 (Mac OS 8/9), and bug 92503 (Mac OS X). Bug 92503 (Mac OS X) appears to have a fix-in-hand.

Changing summary to include titlebar for easy searching.

*** Bug 105921 has been marked as a duplicate of this bug. ***

A number of other bugs that *have* been worked on are similar to this, so to avoid duplication, it's worth merging these together. The issue occurs with all non-english text, where you get ?????.??? type titles. I wonder what happens, though, if moz renders the title correctly, but you haven't got the appropriate character set installed with your OS? anyhow. it's still open.

*** Bug 61676 has been marked as a duplicate of this bug. ***

*** Bug 108418 has been marked as a duplicate of this bug. ***

When the OS doesn't have the appropriate font for the title (as well as the webpage), maybe Mozilla could display a dialog/warning window saying the page won't be displayed correctly, and maybe suggests an URL/method to user to get the font.

*** Bug 109599 has been marked as a duplicate of this bug. ***

Interesting, but Mozilla 0.9.6 (20011120)is broken even on Windows XP, when all Unicode fonts are present. I see lots of '????' in title when I go to http://www.computerra.ru/. Mozilla 0.9.5 did not have this problem (at least on Windows 2000).

Is that Win XP Pro or Win XP Home? People always forget that there're two Win XP. Pro is in WinNT series, whereas Home is in Win95/98/ME series. I don't have WinXP Home, but I would tend to suspect its Unicode support isn't as complete as in Win2k/XP Pro.

It could be due to the difference between Win XP home and Win XP Pro, but more likely is that on Win2k, you set the default language (in Control Panel - Regional Options) to Russian while on Win XP, you didn't. I just sorta confirmed my suspicion. When I tried Mozilla 0.9.5 under Win2k with the default language set to English, the title bar has "?" in place of Cyrillic letters when I visited www.computera.ru. However, after setting my default language (system locale) to Russian (go to Control Panel | Regional Options and press the button 'Set default' and choose 'Russian') and rebooting Win2k(you have to reboot after changing the default language), Cyrillic letters were displayed perfectly fine under Win2k + Mozilla 0.9.5 What people (new to this bug) keep forgetting or don't know is that there are two facets to this bug (this has been mentioned a couple of times before in this thread) 1. not being able to display non-English characters even when Mozilla is running under the locale which supports those characters. That is, under Russian Windows-whatever (9x/ME/NT4/2k/XP)(or Unix/X11, MacOS), Russian characters should be rendered correctly in the title bar. 2. not being able to display characters *outside* the character repertoire of the current locale. That is, Russian characters are not rendered under English/French/Chinese/Japanese/German Windows or Unix/X11 with E/F/C/J/G locale. I believe the first bug has been fixed for a long time (in Netscape 3.x) and what we have been really concerned with is the second one. Therefore, I think the summary line should be changed to something like this: chars outside the char. repetoire of the current locale NOT rendered in title bar Please, note that MS IE certainly solved the second problem as well. No matter what language version of Win2k you use or what default language you choose, the title bar is rendered correctly for all the scripts and languages supported by Win 2k (and XP). Of course, this is not the case under Win 9x/ME. Therefore, our target on Windows platform for this bug is to get the second bug fixed for WinNT4/2k/XP (XP Pro only if XP Home and XP Pro are different in terms of Unicode support).

I asked Unicode mailing list if there's any difference between Win XP Home and Win XP Pro in terms of Unicode support and got the following reply, which agrees with what I had thought before (i.e. both XP Home and XP Pro are descendants of NT. MS attempted the unification of their two lineages of Windows - 9x and NT - when releasing Win2k but failed. However, they finally did it in XP): ------- All versions of XP are derived from NT. The differences between Home and Pro versions are target markets and price. The Home version cannot be centrally managed as a part of a domain and IIRC it doesn't support multiple processors, making it unsuitable most companies. --------

I have zero exprience coding for Windows, but according to what I've found on MSDN, WinNT supports BOOL SetWindowTextW(HWND hWnd, LPCTSTR lpString) natively while Win9x/ME requires something called "Microsoft Layer for Unicode" to be installed to be able to import this function from a DLL. Since we won't redistribute this upgrade to all users, I guess we'll only use SetwindowTextW on WinNT (and XP). See: http://msdn.microsoft.com/library/en-us/winui/windows_07w4.asp http://msdn.microsoft.com/library/en-us/win32/unilayer_4wj7.asp

Could someone tell me how MS-Windows binary is compiled in terms of Unicode support? I'm wondering how Mozilla deals with A and W versions of Win32 API calls. In other words, which of the four strategies outlined in the following article used to make a single binary work with both Win 9x/ME and NT/2k/XP? http://www.microsoft.com/msj/1198/multilang/multilang.htm (I mentioned this article in my comment in July 2001, but there was a spurrious 'l' at the end of the URL) I thought Mozilla uses strategy #4 and has a kind of wrapper to detect the OS it's running under (whether it's Win9x/ME or Win NT4/2k/XP) and invokes A or W version accordingly. However, a quick search with lxr couldn't lead me to such a wrapper. Having dug up James Toder's patch (checked in October 1999 : news://news.mozilla.org/19990920123309.2772.rocketmail@web1604.mail.yahoo.com ) I realized that Mozilla uses strategy #3 (compiling Mozilla as ANSI application with Unicode used internally but converted to the current codepage before invoking 'A' version of Win32 calls) This means that CreateWindowEx() in Mozilla is mapped to CreateWindowExA() instead of CreateWindowExW(). According to what I gather a few months ago on the Unicode mailing list (see http://www.jshin.net/~jungshik/i18n/mswintitle.html), calling SetWindowTextW() for a window created with CreateWindowExA() would not work as intended. By the same token, I guess SendMessageW() instead of SendMessage() (which is actually SendMessageA() in ANSI applications) wouldn't work, either. Adopting strategy #1 (compiling as a pure Unicode application and bundling Unicode layer for Win9x/ME) in the MSDN article aforementioned appears to be nice, but I guess it's not plausible at this point (there could be hundreds of places requiring fixes if we suddenly switch from 'A' calls to 'W' calls) Moreover, bundling Microsoft layer for Unicode may not be compatible with various licenses Mozilla is licensed under. I'll try to find out if there's a way around (Like Ilya, I have no experience with Windows programming, but ....). Of course, we have jilu's routines.

What about this ? Just as a proof of the concept(?) 1. Determine whether the OS is Win 9x/ME or Win NT4/2k/XP (perhaps with GetVersionEx()) and set the flag g_IsWindowsNT (global) during the start-up. 2. In widget/src/nsWindow.cpp if g_IsWindowsNT CreateWindowExW() else CreateWindowExA() .... if g_IsWindowsNT use SetWindowTextW() else use James Toder's routine as it is now Again, I'm just thinking loud as someone with zero-experience with Windows programming and my idea may look crazy

I've just made this patch to solve what Frank calls the second-level problem. It is based on MSDN documents on GetVersionEx() and SendMessage() (I have zero experience with MS-Windows programming) and two-year old patch for what Frank called the first-level problem. I also refered to nsCollationWin.* to figure out what's necessary to call A or W API functions depending on the version of MS-Windows (9x/ME vs NT 4/2k/XP). Since I don't have Visual C++ for MS Windows, I'm not able to test it. It may not even get compiled let alone fixing the bug. It would be nice if it's tested (or taken a look at).

i'd rather that we try to use the W function if it's available instead of when we think it will be available. GetProcAddress http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/dll_0f 8z.asp is what i was thinking of, and it all goes back to the Microsoft Layer for Unicode... still this looks like a good start

Thank you for looking at my patch. I'm not sure whether GetProcAddress() gives us any advantage in this particular case. Below is my thought (which may not be right) based on MSDN articles. W API functions are guaranteed to be available in Win NT 4/2k/XP. By not using GetProcAddress() and using GetVersionEx() to check the availability of W API functions, we only miss the case where MS layer for Unicode is installed under Win 9x/ME. However, MS layer for Unicode cannot do anything that cannot be done with A API functions in Windows 9x/ME as far as I understand. It just makes W API functions available in Win 9X/ME, but W API functions in Win 9x/ME get translated, by MS layer for Unicode, to A API functions which are limited to the repertoire of the locale. That is, under French Windows 9x/ME, calling W API functions with MS layer for Unicode in the hope that Chinese title gets rendered correctly is futile. Therefore, unless something like jilu's patch (rendering window title on our own instead of relying on OS APIs) is done, I guess we cannot render Chinese title under French Windows 9x/ME. If my patch works as intended, I believe Mozilla's behavior matches that of MS IE 5.x under both MS Win NT4/2k/XP and Win 9x/ME except that under the latter MS IE 5.x puts the URL instead of question marks in place of unrenderable characters.

Jungshik Shin, Yes, Win9x/ME should be treated differently from WinNT4/2k. Actually, I've reported this in April: http://bugzilla.mozilla.org/show_bug.cgi?id=75344 but that bug was marked as a dup of this one and the discussion just ended.

one thing it gives us is a bit of correctness. if WINE or something similar decides to implement a W version of this function for a system that doesn't call itself NT then you'll do the wrong thing. your code is akin to using: if (navigator.userAgent.match(/4/)>0) instead of using: if (document.layers) when trying to decide if a browser has layers.

timeless, Thank you for clarification. I didn't think about WINE or other implementation of Windows API. Your point was well taken. There are several other places GetVersionEx() is used to determine the availability of W API functions (use lxr and search for GetVersionEx or W_API). If you want to fix all of them, I guess it's a good idea to raise that issue in a separate bug so that they can be fixed everywhere (instead of not just here) I'm also concerned about the implication of the following statement: Because the program uses run-time dynamic linking, you should not link with the import library when creating the program module. at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/dll_69nr.asp IMHO, here we'd rather concentrate on getting things done the way they're done in other places of Mozilla than try to raise the bar for correctness. BTW, when calling CreateWindowW() in my patch, the third parameter ' "" ' should be LPCWSTR so that I guess ' "" ' should be ' (LPCWSTR) L"" ' (or just ' L"" ' )

A slightly modified patch which I actually applied to get the whole thing compiled with VC++. Unfortunately, it does not solve the problem. I confirmed on Win2k that W version of SendMessage and CreateWindow are called with the page title in LWCWSTR (PRUnichar*), but it didn't work as intended. (the result is exactly the same without the patch on Win 2k).

*** Bug 114792 has been marked as a duplicate of this bug. ***

I've seen the following remarks that comes from a contributor to news://msnews.microsoft.com/microsoft.public.win32.programmer.international I don't know if it makes sense, but if it's true, it may explain why the latest patch does not solve the problem. >>>>>>>>>>> I think, Netscape asks OS to use "MS Sans Serif" to draw the title (which is not multilingual, it's single code page font), while Windows 2000, _unlike NT 4.0_ has new system font - "Microsoft Sans Serif" which is Unicode font (contains Hebrew). Probably this is why my IE 5.5 works differently under NT 4.0 and 2000 - in IE code, unlike Netscape, they did *not* use hard-coded name "MS Sans Serif" and use, instead, MS generic name for system font - therefore under Windows 2000 IE 5.5 uses new, Unicode "Microsoft Sans Serif" that contains Cyrillic!

This is on GNU/Linux, Mozilla nightly Jan 31. LC* settings are as follows: declare -x LC_ALL="POSIX". The problem is with http://overosa.dhs.org/index.html the title is written like this in the source: "&ouml;ver&ouml;sa" -- which is shown as "?ver?sa" in the Mozilla title bar. In the location field history/tab field where the Title is shown, it is shown correctly as "överösa". Font for everything is Lucida, shiped with XFree86 4.2.0

Followup to my last comment: When setting LC_ALL to sv_SE, and LANG to sv_SE, the title is displayed correctly. Just setting LC_MESSAGES=sv_SE does not seem to be enough.

It's natural that just setting LC_MESSAGES without setting LC_CTYPE (or LANG/LC_ALL) doesn't work. It's LC_CTYPE that determines the way Window manager and Mozilla communicates. Simple cases (displaying Swedish title under Swedish locale) like yours was taken care of by Mozilla developers a long time ago. The issue here is to make Swedish title displayed correctly under non-Swedish locales such as Japanese and Chinese locales. Even that now works with KDE(newer version) under Linux although that's not the case yet in MS-Windows.

*** Bug 128174 has been marked as a duplicate of this bug. ***

Re: comment #87 > I think, Netscape asks OS to use "MS Sans Serif" > to draw the title (which is not multilingual, it's single code page > font), while Windows 2000, _unlike NT 4.0_ has new system font - > "Microsoft Sans Serif" which is Unicode font (contains Hebrew). I don't think this would work for Win2k either for all languages : 1) "Sans Serif" or any other like "Times" or "Arial" as a font name is meaningless for other languages such like CJK. 2) It seems that when rendering Unicode strings, Windows tries to concatenate substrings from different fonts. I think this is the best way to go for Mozilla to display Window title string. In this way, there's no need to think about whether it's "MS Sans Serif" or "Microsoft Sans Serif". FYI, the size of "Sans Serif" is too small to contain all Unicode characters.

*** Bug 131558 has been marked as a duplicate of this bug. ***

*** Bug 131868 has been marked as a duplicate of this bug. ***

After reading Frank's comment on bug 97671 (basically the same bug as this one), I gave it yet another shot at fixing this bug under Win 2k/NT/XP. This is certainly not the most efficient (performance-wise) and the prettiest patch, but I just want this to be regarded as a proof of concept from which we can start toward a better fix. Anyway, it works !! With the default locale set to Korean under English Windows 2000, I was able to see a lot of simplified Chinese characters (which are NOT available in X-Windows-949/CP949 for Korean) in the title bar while browsing http://cn.yahoo.com or http://home.netscape.com/zh/cn. I'd be glad to hear feedback, positive or negative, for this patch. BTW, applying this patch, Mozilla will be on par with MS IE 5.x/6.x in terms of being able to display characters NOT native to the present locale in the title bar. MS IE 5.x/6.x can do that under Win NT/2k/XP but CANNOT under Win 9x/ME.

Jungshik: Few months back, I worked on compile moz with -DUNICODE flag; but it just had soooooo many errors and warnings to make moz a true Unicode app. I started to realize that instead of bitting a big fish, we should start with smaller scope. (ie call _::RegisterClassW()_ and selectively call W-APIs) Your approach is exactly what I wanted to do. I am glad it works. :) I can't work on this until post moz 1.0; but we should definitely pursue this approach. Thanks again.

Roy, Thank you for your positive response. I'm also glad that we approached the problem exactly the same way. I almost gave up when I found more and more API calls(A vs W) to be selected depending on the platform (9x/ME vs NT/2k/XP in nsWindow.cpp), but fortunately that 'spree' ended without going much farther. Now as to what to do with this now, I'm not sure. Obviously you're too busy preparing for 1.0 to work on this. Perhaps, it may be also a bit risky(my experience with Win32 programming is limited and I definitely need Win32 experts like you to thoroughly review and polish up my code) to put this in when 1.0 deadline is approaching. On the other hand, it makes me a bit uncomfortable to see this bug left unfixed in 1.0 because there are so many duplicates of this bug (I guess this is one of the most frequently reported and the longest standing bugs in i18n) and Mozilla's archrival MS IE does it right. Could you think of anyone other than you, within or without Netscape, who can pitch in for reviewing and refining my patch so that it can make itself in 1.0? Sorry for the 'babbling'(?), but I feel 'a bit attached to' this bug and like to get this taken care of in 1.0 _if possible_.

*** Bug 135718 has been marked as a duplicate of this bug. ***

Man, here it comes again. Duplicate of this bug. I am getting tired of getting the duplicate every few weeks. Let me try if I can put this in moz 1.0 Making moz milestone to 1.0

Roy, Great to hear that you want to put this in for moz1.0 :-) Have you tried my patch on your build? My build works fine as far as I can tell, but who knows? I may have missed a couple of spots where I should invoke W API calls if available. Perhaps, I've gotta put up my build somewhere so that those interested can play with it because I don't spend much time under MS-Windows. Do you think it's a good idea?

*** Bug 135908 has been marked as a duplicate of this bug. ***

Since there are W and A API's, isn't it better to compile two different versions for Windows, ie Win9x/ME and WinNT4/2k/XP ? We could, at the same time, include some optimal codes for just WinNT4/2k/XP platforms! Furthermore, by having two separate versions, those who want the stable one can always download Win9x/ME version since this works in WinNT/2k/XP as well, whereas the other version is left to those other more courageous users :)

seak, no. i'd much prefer for us to go strictly to the W form and offer the microsoft unicode wrapper for 9x consumers. if there arewany W api's that we need which the wrapper doesn't provide we can role our own fallbacks.

> seak, no. i'd much prefer for us to go strictly to the W form and offer the > microsoft unicode wrapper for 9x consumers. if there arewany W api's that we > need which the wrapper doesn't provide we can role our own fallbacks. In the long run, I fully agree with you that we have to do that. However, as Roy noted, it's not so easy to do that at the moment because there are so many places in Mozilla code that depend on A API's behavior (e.g. assuming LPTSTR is 'char *', which doesn't hold true as soon as we define 'UNICODE' and W APIs are used. It would have been great if Mozilla had been a 'Unicode application' from the beginning, but we can't do anything about things in the past .... ). A short term solution is, I believe, to selectively use W APIs as necessary. Perhaps, a branch has to be made for Mozilla as a 'Unicode application'...

I ran more extensive test with attachment 75613 [details] [diff] [review] applied to the latest trunk and found that it crashed at pages such as http://www.hani.co.kr and http://www.lemond.fr and sometimes http://www.yahoo.co.jp (not always in the last case) although Mozilla could display characters outside the repertoire of the present locale codepage in the title bar. I tried to figure out the cause, but couldn't. I also made more extensive replacements of A API calls with platform-dependent W/A API calls in nsWindow.cpp and nsToolkit.cpp than in bug 75613, but that didn't make any difference. Attached is stack backtrace I obtained from my debug build with attachment 75613 [details] [diff] [review] applied just in case more experienced mozilla hackers and Win32 programmers can figure out the cause of the crash. BTW, I couldn't make an optimized build (make looked for xp_core.h, which appears to have gone away a long time ago)

I think this is too risky for nsbeta1, nsbeta1- this. Jungshik Shin and Roy, please keep work on this but plan to land after we branch so this won't impact the coming release. I would love to see this in the first thing after m1.0 work around is using "tab" instead of "window"

Frank, Thank you for looking into this. I agree with you that this is too risky as it is now and it seems reasonable to push this after m1.0

*** Bug 136683 has been marked as a duplicate of this bug. ***

*** Bug 137143 has been marked as a duplicate of this bug. ***

Comment on attachment 75613 [details] [diff] [review] a new patch for Win2k/NT/XP that does work +// virtual LPCTSTR WindowClass(); +// virtual LPCTSTR WindowPopupClass(); you should delete these functions rather than commenting them out + if ( ::IsWindowUnicode(mWnd) ) + fputs ("Current window is Unicode window\n",stderr); + else + fputs ("Current window is ANSI window\n",stderr); Why not use the IS_UNICODE macro? and, you shouldn't unconditionally write data to stderr... can you wrap that in #ifdef DEBUG or use PR_LOG?

Christian, Thank you for reviewing my patch. I agree with you on both issues(using IS_UNICODE macro and #define DEBUG_jungshik and removing unnnecessary commented-out-lines). I thought I removed all those debug print-out's when I uploaded attachement 75613, but somehow I didn't. Anyway, my patch, as it stands, cannot go in because there's a serious problem as noted in comment #105. It was not meant to be final but for others with more experience in Win32 programming to review and comment on. I've made some more changes since, but that hasn't helped me removing the problem I wrote about in comment #105.

*** Bug 138916 has been marked as a duplicate of this bug. ***

Unfortunately not going into moz 1.0

this iis bad, very bad, since corrently in the latest mac builds, there has been a regression and hebrew titles do not disply in tab titles either- there is no way to know or see a hebrew title when using moz on mac.

*** Bug 143563 has been marked as a duplicate of this bug. ***

from comment #114 >know or see a hebrew title when using moz on mac. simon: Were we able to support hebrew title in Mac before? I know japanese works on MacOSX. Has it broken recently?

I have no direct experience, and maybe Shosh can amplify, but my understanding is that Hebrew titles work on Mac OSX but not on OS 8/9

>but my understanding >is that Hebrew titles work on Mac OSX but not on OS 8/9 Correct. this is true for tab labels as well as the window title bar itself

*** Bug 119881 has been marked as a duplicate of this bug. ***

IMHO this bug should have an higher severity. It realy should be fixed for mozilla 1.0 (and netscape 7.0) in order to make them usable for non english speakers. I don't see how you are going to convince web developers and web users to switch to mozilla/netscape when they do not support all the features available at IE (windows platforms ofcourse). More important then the title itself (I guess many people can work with ??? as title if they must) is the feeling that even 1.0 is not a finished product, and a commercial grade software, when it has such a main stream bug. I am using RC3 on XP. More sites for refference: hebrew - http://www.haaretz.co.il/ arabic - http://www.naseej.com.sa/StaticPages/main/about/about1_1_e.html

I agree too; those ? ? ? are really ugly! Those two URL in comment #76 can't be found anymore, but I've found another one related to developping Unicode App for 9x platform. timeless, is this the "microsoft unicode wrapper" that you talked about in comment #103 ? http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnmag01/html/MSLU.asp But if time is too short to use the MSLU before Mozilla 1.0 release, I would suggest that Mozilla display just the URL in the title bar, just like how IE does in Win9x platforms. For Unicode platforms, maybe Mozilla can do the same for the time being?

Using MS layer for Unicode is not a viable option at the moment as Roy mentioned before. That requires converting Mozilla (for MS-Windows) to a Unicode application, which is not so easy. As Roy and I wrote, selective use of W API calls is much more viable solution than that. My attempt to do that was partially successful (in that my build with that patch is able to display characters outside the repertoire of the present 'code page' in the title bar), but it still dumps core in some cases. Now that Mozilla 1.0 has been released, I guess it's time to work on this again.

I had little time to play with your idea and I did little clean up. Unfortunately, I am getting link error for from nsWindowFactory saying missing WindowClass() WindowPopupClass(). I am hoping someone can shed some light on this error.

I had little time to play with your idea and I did little clean up. Unfortunately, I am getting link error from nsWindowFactory saying missing WindowClass() WindowPopupClass(). I am hoping someone can shed some light on this error.

it needed to have -DMOZ_UNICODE in /src/build/makefile.win

yap. the patch works. I can display Korean title in WindowsXP-SC with no violation errors. However, we need more testing.

Is this going to be a windows only fix? Should I re-open Bug 119881 (a mac bug, marked right now as a dup of this one)?

Build 2002060704 / Win2000 : question marks instead of Russian characters still. Didn't the patch made it to the trunk?

>question marks instead of Russian characters still. >Didn't the patch made it to the trunk? No, no patch was checked into the trunk. I tested my patch on Win2000-Ja and it displayed russian, korean, french, hebrew and arabic correctly. (note. to display bidi text in correct order, we needed to enable bidi from windows control panel. However, moz is parity with IE.) >119881 If 119881 is mac specific, then please reopen 119881.

OK, will try it on 20020608 tomorrow.

Define MOZ_UNICODE as environment variable so that we can co-exist with the current moz-win32 build. To build moz-unicode: - define MOZ_UNICODE environment variable by SET MOZ_UNICODE=1 - re-build widget module after clobbering the module

Same as above except to include nsWindowAPI.h

shhh, I guess I am little excited. Here is the latest.

Roy, I've just tried your patch (not the newest one but attachment 86631 [details] [diff] [review]. Is there any difference between them other than Makefile tweaking for co-existence of MOZ_UNICODE build and non-MOZ_UNICODE build? ), I got exactly the same access violation (according to the stack trace in VC++ 6.0) at http://www.hani.co.kr, which is among sites where I got access violation with my patch. (see my comment 105) I applied your patch to the trunk (fetched afresh off the cvs last night). I had to hand-edit it a bit because the trunk and Mozilla 1.0 branch are different. Have you treid http://www.hani.co.kr and http://www.lemonde.fr with your patch applied? My test environment is Korean locale + English Windows 2k.

Re comment #122: Yeah, I still remember that you had insisted several times that Mozilla isn't a Unicode application. Mmmmm ...... I didn't think more about this, but now that I rethink about it, I have a question: do you mean that Mozilla wasn't programmed as a Unicode application at the very beginning ? Or do you mean some component aren't Unicode compliant? I'm also think if it isn't possible to make a general call or a general "bug" to every component maintainer to check their codes. Eg, file a "bug" for every component as a reminder to its maintainer so as to check if his/her code is Unicode compliant, and file one more "bug" and mark it as dependent to all others.

> do you mean that Mozilla wasn't programmed as a Unicode application at the > very beginning ? Have you ever looked at my or Roy's patch? In Win32 programming, a 'Unicode application' is an application written and compiled with 'UNICODE' defined. When it's defined, all generic Win32 API calls (which are NOT explicitly specified as A API calls) default to W API calls. W API calls are not available on Win9x/ME by default, but MS Layer for Unicode translates them to locale-dependent/codepage-dependent A API calls under(and only under) Win 9x/ME. Therefore, a single Unicode application can be written which works both under Win9x/ME and Win NT/2k/XP. However, this does not work for Mozilla at the moment. As I and Roy mentioned **multiple times** __here__, on numerous places in Mozilla source, it's assumed that generic Win32 API calls are A API calls. Sometimes A and W API calls have exactly the same arguments (in terms of the number and type of arguments, return type, etc). Unfortunately, often times, they're **different** in return type and type of arguments. (see Roy's patch and look into nsWindowAPI.h file and you'll see what I mean) Because it's assumed that all generic(without 'A' or 'W' attached at the end) Win32 API calls are A API calls in Mozilla source, just defining 'UNICODE' and turning all Win32 API calls into W API calls break Mozilla in a number of places. Fixing all these cases would take many man-hours and is not feasible at least for 1.0 branch. Consequently, MS Layer for Unicode is of no use for Mozilla(1.0 branch) at the moment. > Or do you mean some component aren't Unicode compliant? Nope. There might be some such components, but that's nothing to do with the issue at hand.

Sorry for spamming... > do you mean that Mozilla wasn't programmed as a Unicode application at the > very beginning ? See also comment 104.

>Have you treid http://www.hani.co.kr and http://www.lemonde.fr Not yet. I haven't got any access violations from my previous test sites. However, I'll try your test sites today. I am also in the process of having Unicode build column in the tinderbox (initially in the SeaMonkey-Ports page). This way we can slowly migrate Win32 build to Unicode-Hybrid build and monitor any build breakages. (Note Unicode-Hybrid build. You can see from my patch that I am designing the code so that the browser will be a true UNICODE app in WinNT base OSs where as it will act as ANSI app in Win9x base OSs)

Here is the approach we are basing on. Design a Single Unicode App that Runs on Both Windows 98 and Windows 2000 http://www.microsoft.com/msj/defaultframe.asp?page=/msj/0499/multilangunicode/multilangunicode.htm&nav=/msj/0499/newnav.htm

>>Have you treid http://www.hani.co.kr and http://www.lemonde.fr >Not yet. I haven't got any access violations from my previous test I've tested them on my build ( which was made from mozilla 1.0 branch as of last Saturday plus your patch) and unfortunately I got the same access violation as I got with my patch. (I should not forget to mention that it works on sites like http://cn.yahoo.com). This could well be due to something peculiar in my environment (Win2k English version with the system locale set to Korean). Anyway, it'd be nice to see the result of your test under different environments (WinXP SC, etc)

>>Have you treid http://www.hani.co.kr and http://www.lemonde.fr Hmmm No access violations on both sites. I am testing them in native Win2K-Ja and native WinXP-SC.

>>Have you treid http://www.hani.co.kr and http://www.lemonde.fr >Hmmm No access violations on both sites. >I am testing them in native Win2K-Ja and native WinXP-SC. Thank you for trying it. I made another build this time using Mozilla 1.0 source tar ball with your patch applied. Your patch got applied without any offset this time while it got applied with 4~50 lines offset against Mozilla 1.0 branch I pulled off from the cvs server (has there been some changes in files like nsWindow.cpp/h since 1.0 release? Lxr doesn't help me track down these changes, if any, on 1.0 branch). Anyway, the result is the same as before. I tried it this time under Enlgish locale and English MS Win 2k. Another site I got access violation is http://www.ohmynews.com Do you have any EN-US Win2k around to try this under? BTW, when I tried to build Mozilla with your patch and MOZ_UNICODE set to 1, 'webshell' module(?) didn't get compiled complaining about WindowPopupClass not resolved. Therefore I had to build Mozilla without MOZ_UNICODE set to 1, clobber widget module and rebuild widget module only with MOZ_UNICODE set. Do you think the way I built mozilla has to do with my problem? I can't think of any other way until webshell module(?) is tweaked to get compiled with MOZ_UNICODE set. Perhaps the best way to get this tested as extensively as possible is to make a binary build available for people under many different environments to try. I guess that'll be automatically the case when you're done with adding Unicode-hybrid build column to tinder-box, right?

> BTW, when I tried to build Mozilla with your patch and MOZ_UNICODE set to 1, > 'webshell' module(?) didn't get compiled complaining about WindowPopupClass > not resolved. Therefore I had to build Mozilla without MOZ_UNICODE set to 1, > clobber widget module and rebuild widget module only with MOZ_UNICODE set. This was solved by adding to webshell/tests/viewer/windows/makefile.win the following lines (right after DEFINES line) !ifdef MOZ_UNICODE DEFINES= $(DEFINES) -DMOZ_UNICODE !endif

bug 150131 is for Linux

>Do you have any EN-US Win2k around to try this under? No; but I built an optimized build for our iQA to test under other OS. I'll ask them to play with it today.

> bug 150131 is for Linux. As I wrote there, this bug was fixed a long time ago for Linux. See comment 39 to comment 59 and attachment 41935 [details] [diff] [review]. I think there's not much more Mozilla can do under Linux and it's up to the window manager a user choose to interpret _NET_WM_NAME window property (UTF8_STRING) properly.

I solved http://bugzilla.mozilla.org/show_bug.cgi?id=150958 and http://bugzilla.mozilla.org/show_bug.cgi?id=104027 on my Linux box for ISO-8859-1 characters. The only thing I changed was a locale. From: LANG=POSIX LC_CTYPE="POSIX" LC_NUMERIC="POSIX" LC_TIME="POSIX" LC_COLLATE="POSIX" LC_MONETARY="POSIX" LC_MESSAGES="POSIX" LC_ALL= To: LANG=POSIX LC_CTYPE=fr_CH LC_NUMERIC="POSIX" LC_TIME="POSIX" LC_COLLATE="POSIX" LC_MONETARY="POSIX" LC_MESSAGES="POSIX" LC_ALL= Et voil&#224;! AFAIK nothing to do with my window manager. BTW, I use quite old programs, fvwm 2.2.4 and XFree86 3.3.6 (Debian 2.2). But for non-ISO-8859-1 characters (Korean, Hebrew), I still get '?' in the window title. But everything else (including tabs title) is OK, very fine job!

>Do you have any EN-US Win2k around to try this under? iQA tried my OPT build on EN-US W2K and worked fine:) I asked him to try Korean locale. He/I will post the result later.

Marc, on non NETWM-compliant window managers (like FVWM which you're using), the title bar would only display languages which your locale can handle. If you wish to display all languages without switching to a NETWM-compliant window manager (such as KWin, Metacity...), you might try using a locale which has a UTF-8 codeset.

>>Do you have any EN-US Win2k around to try this under? >iQA tried my OPT build on EN-US W2K and worked fine:) >I asked him to try Korean locale. He/I will post the result later. Thank you for trying that. I guess running it under KO locale won't result in any difference although it'd be nice to see it confirmed. Is there any way for me to try your build on my Win2k EN-US? I also built an OPT build, but the result is the same. Something must be wrong on my build environment.(I'm using VC++6.0 with sp5.) Trying your build and finding that it works fine on my box would be a strong indication that my build environment is to blame.

I am not certain who I should request for review. shanjian: can you review my patch?

Thank you for sending me your build. > Is there any way for me to try your build on my Win2k EN-US? >I also built an OPT build, but the result is the same. Something >must be wrong on my build environment.(I'm using VC++6.0 with >sp5.) Trying your build and finding that it works fine on my box >would be a strong indication that my build environment is to blame. I've just tried your build on my Win2k box. To my great surprise, it also gave me access violation. It seems like it's not my build environment but something else on my Win2k box that is to blame. Anyway, it appears to be my own problem (with little to do with the patch). I just don't know what.....

At the top of attachment 78132 [details], I have the following lines: > ffff025d() > NPSWF32! 0534bb7a() > NPSWF32! 0534ae46() > USER32! 77e139a3() > USER32! 77e15c95() > NTDLL! 77fa032f() > nsView::SetDimensions(const nsRect & {...}, int 1) line 532 I should have noticed immediately that the access violation had occurred not in Mozilla code but in NPSWF32. NPSWF32.dll is for flash plugin. (it's beyond me why/how I missed that...) I had 5.0 r41 plugin from macromedia. After upgrading it to 6.0, the problem has gone for good. Had I realized this back in early April (when I wrote comment 105), this fix might have made it in 1.0. Anyway, Roy has come up with a cleaner patch, which is good.

joe: Can you review my patch? It's #ifdef MOZ_UNICODE so it has no impact on the tree; but once I got the unicode tinderbox column running, I want to be ready.

*** Bug 152839 has been marked as a duplicate of this bug. ***

*** Bug 152788 has been marked as a duplicate of this bug. ***

I talked with roy about this bug over the phone. If we can intergrate "Microsoft Layer for Unicode" before this patch, the code should be much simpler. Roy will file a bug and we will see what needs to be done.

just to recap what shanjian and I are suggesting. - make this patch much simpler by calling pure Unicode APIs only (thus no A API calls on Win9x platforms) - incorporate 'Unicode Layer' library (bug 118013) The reasons are as follow - easy to maintain - no performance hit on NT base OSs (ie. no if (nsToolkit::mUseWideApi) needed) moz may perform better on NT base OSs Overhead - "uniocde layer" will be executed in Win9x base OSs. Action items: - revise the patch to only include Wide APIs - add "Unicode Layer" library to the build sytem.

shanjian: can you review?

Comment on attachment 88673 [details] [diff] [review] use of W-APIs only >+ nsAutoString title; >+ title.Assign(mTitle); How about nsAutoString title(mTitle); >+ nsString dirBuffer; >+ dirBuffer = initialDir; Same thing here nsString dirBuffer(initialDir); >+ TranslateMessage(&msg); >+ DispatchMessageW(&msg); Is there a W version of TranslateMessage?

un-cc'ing me since i am not qualified to review windows-specific patches.

TranslateMessage() doesn't have W version. shanjian: please review.

Comment on attachment 89307 [details] [diff] [review] taking shanjian's recommendation r=shanjian,

kin: can you super review?

Comment on attachment 89307 [details] [diff] [review] taking shanjian's recommendation --- Can mDefault ever be empty when this call is made? I'm worried that get() might return an empty string, and I don't know how the calls that will make use of this will react to an empty "" string.: + wcsncpy(fileBuffer, mDefault.get(), MAX_PATH); --- Are we guaranteed that mLastUsedUnicodedirectory will not be empty too at this point? Once again I'm worried about the initialDir and dirBuffer calls that do a .get() below this, being unfamiliar with some of the calls that will use the result of .get(), I'm not sure they will handle a null or empty string. + if(initialDir.IsEmpty()) { + // Allocate copy of last used dir. + initialDir = mLastUsedUnicodeDirectory; + } + --- Is this guaranteed to never exceed MAX_PATH? Where are we getting MAX_PATH from? + result = ::SHGetPathFromIDListW(list, (LPWSTR)fileBuffer); --- Fix indention in the 2 places that have this change: +#ifdef MOZ_UNICODE + file->InitWithPath(mUnicodeFile); +#else file->InitWithNativePath(mFile); +#endif --- Do we want to rename |_retval| to follow the arg naming conventions used throughout the source base, and to make them more meaningful? NS_IMETHOD Show(PRInt16 *_retval); +#ifdef MOZ_UNICODE + NS_IMETHOD ShowW(PRInt16 *_retval); +#endif --- Have you built with MOZ_UNICODE defined and undefined? --- Is MOZ_UNICODE defined by default by some config file? And do all the win32 platforms everyone builds on (98,2k,XP,NT) support all the 'W' methods?

*** Bug 154881 has been marked as a duplicate of this bug. ***

>--- Can mDefault ever be empty when this call is made? >I'm worried that get() might return an empty string, >and I don't know how the calls that will make use >of this will react to an empty "" string.: Yes, it is possible to be an empty string. I tested with a dummy line wcsncpy(fileBuffer, L"", MAX_PATH); and it copied an empty string to fileBuffer successfully. >--- Are we guaranteed that mLastUsedUnicodedirectory >will not be empty too at this point? No, I guess it is possible to be empty. However, I am mimicing nsFilePicker::ShowW()from the already exiting nsFilePicker::Show(). I test run the ::SHBrowseForFolderW()with empty dirBuffer and worked fine after modifing the patch a little for dirBuffer and title to be PRUniChar. >--- Is this guaranteed to never exceed MAX_PATH? >Where are we getting MAX_PATH from? >+ result = ::SHGetPathFromIDListW(list, (LPWSTR)fileBuffer); MAX_PATH is defined in windef.h and utilized thru out Windows APIs. ( #define MAX_PATH 260 ) SHGetPathFromIDList() is described in http://msdn.microsoft.com/library/default.asp?url=/library/en-us/shellcc/platform/Shell/reference/functions/shgetpathfromidlist.asp which says fileBuffer: Address of a buffer to receive the file system path. This buffer must be at least MAX_PATH characters in size. It works. I am simply copying the line from http://lxr.mozilla.org/seamonkey/source/widget/src/windows/nsFilePicker.cpp#129 and calling W API instead. I block tested the code in the debug mode and it returns Path correctly. >--- Fix indention in the 2 places that have this change: Done! >--- Do we want to rename |_retval| to follow the arg naming >conventions used throughout the source base, and to make them more >meaningful? I chanaged to aReturnVal. >--- Have you built with MOZ_UNICODE defined and undefined? Yes, I built it for the second time. I'll triple check again before I check into the trunk once I got /sr from you. >--- Is MOZ_UNICODE defined by default by some config file? MOZ_UNICODE not defined by default. I made it to kick in when MOZ_UNICODE environment variable is defined just like other environment variables. (eg. c:>set MOZ_UNICODE=1) >And do all the win32 platforms everyone builds on (98,2k,XP,NT) >support all the 'W' methods? Yes, 'W' methods are defined in the Win32 include files in the standard MSVC++ 6. New Patch to follow........

kin: please super review?

*** Bug 154903 has been marked as a duplicate of this bug. ***

Comment on attachment 89631 [details] [diff] [review] adding kin's recommendations sr=kin@netscape.com with the removal of the first wcsncpy() call below, like we discussed. + PRUnichar fileBuffer[MAX_PATH+1]; + wcsncpy(fileBuffer, L"", MAX_PATH); + wcsncpy(fileBuffer, mDefault.get(), MAX_PATH);

checked into the trunk

Problem is still persists (bild 2002070310 by yokoyama@netscape.com) in the trunk For example see www.design.ru

>Problem is still persists (bild 2002070310 by yokoyama@netscape.com) in the trunk Which build are you testing with? Plain trunk build doesn't fix this bug just yet. (thus I didn't close this bug) If you want to test, you need to do the following after pull the source: 1) set envrionmen variable MOZ_UNICODE (eg. c:>set MOZ_UNICODE=1) 2) build the mozilla I've tested your site with my Unicode enabled build and works great on my Win2k-En. (no more question marks)

Soooo, when should we expect to have these question marks eliminated in the binaries supplied under "Latest Builds" directory?

> Soooo, when should we expect to have these question marks eliminated in the > binaries supplied under "Latest Builds" directory? When bug 118013 is resolved and Mozilla becomes a 'Unicode application' In the meantime, you don't have to be bothered by '?' in the title bar if you're running Mozilla under Win2k and the only language you're interested in is Russian. You can just set the system locale of your Win2k to Russian and reboot and all Russian titles should be rendered all right.

And this is exactly the way I don't want to solve the problem, because then I am limited to this only selection.

*** Bug 149908 has been marked as a duplicate of this bug. ***

Bug 157687 reports same problem with windows titlebar in Mail component, and I confirm that. Should we change Product from "Browser" to something more broad?

*** Bug 157687 has been marked as a duplicate of this bug. ***

Yes, it's everywhere, from browser to mailer as well as composer, but I've just check the "Product" field and there's no "General" selection. Anyway, if this bug is corrected for one, wouldn't it be corrected for all?

>Anyway, if this bug is corrected for one, wouldn't it be corrected for all? yes, we tested for browser, composer and mail/news. They worked well.

nsbeta1+ for m1.2final

move this bug from m1.2beta to m1.2alpha

kin: can you review? Webshell/test app failed to link.

Instead of this: #ifdef MOZ_UNICODE virtual LPCWSTR WindowClassW(); virtual LPCWSTR WindowPopupClassW(); + virtual LPCTSTR WindowClass(); + virtual LPCTSTR WindowPopupClass(); #else virtual LPCTSTR WindowClass(); virtual LPCTSTR WindowPopupClass(); #endif do this: #ifdef MOZ_UNICODE virtual LPCWSTR WindowClassW(); virtual LPCWSTR WindowPopupClassW(); #endif virtual LPCTSTR WindowClass(); virtual LPCTSTR WindowPopupClass(); Even though you added these new versions of WindowClass and WindowPopupClass() that call through to your W versions, I'm assuming the other changes you made previously to call the OS W functions are still needed right?

Uhhhh, I'm so nuthead. I was thinking of my patch last night. I should have been more careful. >to call the OS W functions are still needed right? Yes, definitely. This is to fix the linker error from //webshell/test. shanjian: can you review? kin: can you super review?

Comment on attachment 93168 [details] [diff] [review] Revised r=shanjian

Comment on attachment 93168 [details] [diff] [review] Revised sr=kin@netscape.com

Comment on attachment 93168 [details] [diff] [review] Revised a=asa (on behalf of drivers) for checkin to 1.1

I want to see Greek in the title bar! Here is another test case for you guys: http://library.ucy.ac.cy/electronic_services/dbase_interbl3_main_gr.htm P.S It works in IE6. My OS is Win2000 SP3 and i'm using Mozilla 1.1b

*** Bug 161248 has been marked as a duplicate of this bug. ***

Not only in titlebar (see my screenshots)

Comment on attachment 96031 [details] Problems with charset Western ISO-8859-1 this attach is a .zip file when I click in Download the attachment instead. http://bugzilla.mozilla.org/attachment.cgi?id=96031&action=view Mozilla 1.1b make download from a file named "attachment.cgi.exe" I need rename it for "attachment.zip" but when I donwloaded it with IE 5 it create a file "attachment.cgi.zip" I am using Mozilla 1.1b Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.1b) Gecko/20020721 build 2002072104

> Problems with charset Western ISO-8859-1 > Not only in titlebar (see my screenshots) No, your problem has NOTHING to do with this bug. Please, make sure that your VIEW|Character Encoding is set to either ISO-8859-1 or ISO-8859-15 when you view the page. The following is off-topic : > Mozilla 1.1b make download from a file named > "attachment.cgi.exe" > I need rename it for > "attachment.zip" Mozilla is 100% compliant to HTTP spec in this regard. > but when I donwloaded it with IE 5 > it create a file > "attachment.cgi.zip" MS IE is NOT compliant to HTTP spec. Its 'arrogant' mime-type peeking IGNORES C-T header field sent by http servers. All right this time, it got it right. However, often times, it gets it wrong !! When you attached your zip file, you should have specified MIME type "application/zip". Apparently, you specified 'application/octet-stream', instead. Moreover, please use 'jpeg' or 'png' instead of 'bmp'.

Comment on attachment 96031 [details] Problems with charset Western ISO-8859-1 Nothing to do with this bug.

Please can someone give any information why the patch did not make into 1.1?

104934 fixes this bug for Windows NT base systems. I believe other platforms has the same bug. This bug is for XP. Marking as fixed

I reopened bug 109599 for Linux.

Titlebar still do not show non-English characters: XP Pro SP1 and build 2002092308 here. I tried the Greek page given at comment #192 and Greek characters were replaced by ?'s. Same thing happening with Cyrillic (koi8-r) pages.

Titlebar still do not show non-English characters: XP Pro SP1 and build 2002092308 here. I tried the Greek page given at comment #192 and Greek characters were replaced by ?'s. Same thing happening with Cyrillic (koi8-r) pages.

drunclear: sorry about this. The patch was rolled back last night for a scroll-bar problem caused by Unicode conversion (104934). I have a patch to fix the scrollbar and you should be on the way to verify this by tomorrow. Regards.

I tested several urls in character coding koi8-r, iso-8859-2, windows-1251, windows-1253, etc... and they all worked for me with XP Pro SP1, build 2002092408. Congratulations! You've fixed a 3 year old bug.

I also have to say: Congratulations ! At last this wicked bug is removed and Mozilla can raise its head one more time against IE :) But ...... there's still a problem in Mailer window's title bar. Should I file a separate bug, or should I send a test case to continue this bug?

>there's still a problem in Mailer window's title bar. Last time I checked, the mail title bar didn't have problem either. By the way, this bug will reappear for few days until 104934 get's resolved. 104934 cause few regressions and need to be dealt with. (170855, 171208, 171228, 170969) two steps forward, one step back..........

Be assured, if the email is well formed, there's no problem. By well-formed, I mean there's MIME encoding like =?big5?B?Rn...... in Subject field. The problem comes from web-based emails and other mailers (possibly old mailers). There're two cases: 1) The MIME encoding is wrong: the subject is =?iso-8859-1?Q?=B9.... instead of =?big5?B?..... (IMO, this happens in Win98) 2) There's no MIME encoding: the subject is something like «Ü¦³·N«ä In both cases, if I set the View to the right encoding, Subject field will display the string correctly, but not the title bar. Do you need test case?

I'd appreciate if you could provide a test case. :)

confirm it fixed under OS/2, Mozilla 1.1 release

At the moment I put this test-case, there's indeed a regression, but the bug isn't due to the regression. Only try it after the regression has passed. The test case contains three test-case mails. For the first two, even if you've changed the encoding, Mailer title still couldn't display correctly. The third one would display correctly without human intervention.

This bug is NOT fixed. I still observe the problem (as described in bug 137143 which is a dupe of this one) on build 2002101608/NT4. Please clarify; otherwise I'm going to reopen it.

MOZ_UNICODE is turn OFF for now. We had some regressions and I was busy on other project. Please leave this bug as closed. 104934 will be the bug we need to fix now.

*** Bug 174539 has been marked as a duplicate of this bug. ***

*** Bug 175673 has been marked as a duplicate of this bug. ***

Changed QA contact to ylong@netscape.com.

This has been verified in Roy's windows unicode build.