*Symptom: What's related (related links) sidebar renders everything in a font for ISO-8859-1 so that non-Western-European characters are not rendered correctly in 'related links' sidebar. The default encoding (set in Edit|Preference|Navigator |Language) doesn't seem to affect how 'related links' sidebar is rendered. * To reproduce: go to any popular non-English (or non-Western -European) sites likely to have related links in non-Western-European and see how related links are rendered in 'related links' sidebar. Change the default encoding in Preference dialog to the encoding of the site and see if there's any difference. * Suggestion: Given that there are so many sites with no encoding information or incorrect encoding information, it's very hard to get right the encoding of links in 'related links' sidebar. However, as a zeroth order approximation, Mozilla may render everything in 'related links' sidebar as if they're in the default encoding set in Preference dialog. If a user sets her/his default encoding to a particular encoding, sites (s)he visits are more likely to be in that encoding than any other encodings and so are related links to sites (s)he visits. However, for some users, only a small portion of sites visited (let alone being the majority which might be the case for some users) may be indeed in the default encoding. For them, the zeroth order approximation doesn't work very well. A better approach would be use the encoding of the page currently being viewed to render links in 'related links' sidebar. There's no guarantee that related links to the site are in the same encoding as the site, but it's pretty likely that they are. Maybe, in case the encoding of related links can be explicitly determined (from http header or meta tag), that should be honored and two approximations mentioned above can be used only as fallbacks when the encoding cannot be determined explicitly (or the encoding is cached in memory from the previous visit). There's a problem, though, with depending on the encoding info. provided via http header or meta tag because there are a lot of sites that get it wrong.

The url for a site with non-English related links is added.

QA contact to blee since he is familiar with international issues involved in the Related links. I thought the related links (NS6) is tagged with UTF-8 encoding so that the problem has to do with the server database not bein correct.

This needs to be fixed for the next release -- added the nsbeta1 keyword. This may be a server-side problem. In the first What's Related release, the server only converted Latin1 and Japanese encodings to UTF-8. I don't know if the WR vendor fixed this for other encodings. So the first step would be to verify that the client is receiving good data from the WR server.

Adding tpringle, kmurray and lbaliman to cc: list.

Assinging to TPringle for resolution from Alexa. May need help from Bob Jung. Adding Bobj to cc: list.

nav triage team: Marking nsbeta1+

nav triage team: Resetting priority so that this bug gets retriaged.

Changing QA Contact to andreasb.

Removing nsbeta1+ from status whiteboard, need to figure what to do in general with what's related.

Changing QA contact to jonrubin@netscape.com.

Jon : is this still a problem in NS 6.01? Which uses the Netscape WR tab rather than the Alexa tab.

Is Alexa sending back info correctly converted to UTF-8? In Alexa's original implementation, it only did so for ISO-Latin1 and Japanese charset encodings. I don't know if Alexa ever fixed its server to convert other charsets (e.g., Korean charsets) to UTF-8. It appears from external usage, that Mozilla only displays ISO-Latin1 WR titles. For pages with non-ISO-Latin1 titles (even Japanese), the WR sidebar displays the URL. Is this because that is what Alexa is returning? Or is the browser doing something? Netscape 4.x displayed Japanese titles in the WR dropdown. But it appears that the WR info returned to 4.x is different than what is returned to Mozilla. When I try both, I get a different list of related URLs for the same URL. Are they pointing to the same WR server/URL or is Alexa sniffing the browser?

Ccing Matt and Myron - do you guys know the answer to this?

Does this affect 6.01 as vishy asked. We are checking in that code instead of using the Alexa tab. If it doesn't this bug is only for mozilla and not netscape

Vishy, 6.01 appears to be fine. Japanese characters to display correctly.

I checked Korean as well. I can see Korean characters in 6.01, but I cannot verify as to whether they make any sense. But Japanese is definitely displaying properly.

In NS 6.0, Korean characters look *mostly* fine, but Korean characters in some sites are treated as if they're ISO-8859-1. I guess this is due to the fact that they're regarded as *non-Korean* sites (and as a result the conversion to Unicode was not done properly) when the DB entries for them were made. For instance, try <http://www.ohmynews.com> and there are two entries in 'What's related' with Korean characters properly displayed and 5 entries with Korean characters garbled (rendered as though they're ISO-8859-1).

marking as nsbeta1- per i18n triage.

Todd - Who's got the answer to this one? This is a server side issue, correct.

Matt, do you think Shawkat would know?

Assigning TM = M0.9.2 | P3. Linda - Can you work with Todd on this one?

Todd, this was first reported quite a while ago, so I checked W/R results today (05.25.01) and I am seeing corrupted (German) characters in the W/R results. Please let me know how you want to proceed and what I can do to help.

Lynn, I think I see What's Related extended characters working in DE 6.1b, but not FR 6.1b. Would you please confirm? Teruko, would you please check JA 6.1b?

It depends what you're looking at. The default language for the browser is currently set wrong on FR, so you won't see the right information.

Changing milestone, reassigning to vishy.

I see a lot of sites are still broken, even netscape one' for example http://www.atour.co.jp/golf/index2.html http://home.netscape.com/zh/tw/ http://home.netscape.com/zh/cn/ http://home.netscape.com/ko/ http://www.edu.cn/ etc. This is a server side issue. I think we should first run the top 100 intl QA sites against this bug (See how many of the what's related links for those top 100 intl sites are borken) . It is very sad that this kind of problem still happen after years of intergrating "What's related" service into the client.

I quickly walked through the JA top 100 sites and at least the following sites are broken for "What's related": http://www.rakuten.co.jp http://www.cool.ne.jp http://www.tok2.com http://www.suntory.co.jp http://www.otd.co.jp http://www.fujitv.co.jp http://www.melma.com http://www.alpha-net.ne.jp

-> samir for investigation with help from Frank.

mass change, switching qa contact from jonrubin to ruixu.

Marking nsbranch- as it was decided in the August bug triage that we wouldn't have eenough time in eMojo to fix this. Let's revisit for MachV.

I just tried these again: > http://www.rakuten.co.jp > http://www.cool.ne.jp > http://www.tok2.com > http://www.suntory.co.jp > http://www.otd.co.jp > http://www.fujitv.co.jp > http://www.melma.com > http://www.alpha-net.ne.jp and all but fujitv returned Japanese results in the What's Related sidebar panel. fujitv reported no related links at all.

removed keyword nsbranch since it now has nsbranch-, per pdt mtg.

Mass-moving lower-priority 0.9.5 bugs off to 0.9.6 to make way for remaining 0.9.4/eMojo bugs, and MachV planning, performance and feature work. If you disagree with any of these targets, please let me know.

Moving to mozilla0.9.7.

-> mozilla0.9.9

Sidebar triage team: commercial client invesitgation to be done by Sujay with help from i18n QA. (The mozilla What's Related content is entirely web-based.)

As per Sujay's email, filed bug 12290 in bugscape for commercial build.

Currently the Alexa server seems smart enough to return just the URL as the link if the <title> of the linked page contains non-western characters. Please re-open if this is not the case.