Sin Hang KIn Reporter
	Description • 25 years ago

The character is displayed on IE5 as nothing. I believe this confirm to the 
name of the character. However, Word2000 display it as a vertical bar also. 
Which I think is apropriate. 

The composer should display this character as a short vertical bar, however, 
the browser should not display anything at all. 

The character is zero-width non joiner. Which I assume is used to mark some non-
join break point but do not distrub the reader with anything : a zero width 
character.

	Sean Richardson
	Comment 1 • 25 years ago

See http://bugzilla.mozilla.org/showattachment.cgi?attach_id=2616 for a 
testcase. The 2000-04-17-08-M16 nightly binary on WinNT displays ‌
as "?". This machine has the most recent TT MS core fonts installed.
Confirming that it is not displayed zero-width :-(.

Setting dependency for bug 17962, "Display all HTML 4 character entities in 
browser correctly", M20.

	Richard Zach
	Comment 2 • 25 years ago

-> Erik

	Erik van der Poel
	Comment 3 • 25 years ago

Hi Mike, I believe the zero width non-joiner is intended to be used with Arabic
(and possibly other scripts). Does your team intend to address this?

	Mike Kaply [:mkaply] Assignee
	Comment 4 • 25 years ago

Here are some remarks from Mati in Israel:

The ZWNJ (Zero-Width Non-Joiner) is an artificial character invented by 
Microsoft (I think) and adopted by Unicode to prevent adjacent letters to join 
in scripts like Arabic, where letter shapes are affected by their neighbors.  A 
typical usage would be to insert ZWNJs between the Arabic letters corresponding 
to IBM, since acronyms should be displayed in Isolated shapes.

Other than affecting the shape of letters, the ZWNJ should not be displayed, at 
least in a browser.  It seems that some editors choose to visualize ZWNJs (e.g. 
as vertical bars) to facilitate editing (it is quite awkward to edit text with 
invisible characters!).

By the way, all the above applies as well to ZWJ (Zero-Width Joiner), except 
maybe how it is visualized in editors.

So it should be displayed as a vertical bar in the composer.

Should the entity code be changed to display a space in browser and bar in 
composer?

	Mike Kaply [:mkaply] Assignee
	Comment 5 • 25 years ago

More info from Maha:

	the &zwnj is behaving correctly in the browser on Arabic NT platform 
(don't know about the US platform). 
it is a zero width character and performs its function correctly (showing the 
characters as isolated) with out any specific display to the zwnj it self. In 
the composer it is also not displayed. 

NT gives an option of displaying or not displaying such zero width character in 
notepad (try the right mouse button)
as well as inserting them.

We can only measure to that behaviour and keep in mind that at least Windows 95 
would be a different story.

There is no specific requirement for Arabic, although it would be a nice to have 
to be able to insert them.

	Mike Kaply [:mkaply] Assignee
	Comment 6 • 25 years ago

I am the person report the bug on Mozilla about the display of the ‌. From 
the conversions of the bug-report, it seems you know the real meaning of this 
code.

I am a Chinese and new to unicode. I would like to ask you a question about the 
use of this character.

I intended to use this char for my analysis of Asian text. Since Asian text do 
not have nature word boundary, and there seems not a perfect word boundary 
separator available. I would like to make an semi-auto process to do the word 
boundary by inserting some word break marks into the text.

Comments from the original opener of the bug:

In a look of the unicode table, I found the ‌ I find out it have zero 
width, and it have a function on non-joiner. Which I guess that it is suitable 
for the task I want. I would like to ask if such an use will break other things? 
Will normal search engines recognize this char as word boundary break? Will this 
conflict with the Arabic applications?

	Mike Kaply [:mkaply] Assignee
	Comment 7 • 25 years ago

Comments from Egypt:

I think that your choice of ZWNJ for delimiting words is not optimal.  The 
Unicode Standard 3.0 on page 316 states that "U+200C ZERO WIDTH NON-JOINER and 
U+200D ZERO WIDTH JOINER have no effect on word boundaries..."

The character which seems more appropriate for your purpose is U+200B ZERO WIDTH 
SPACE, about which Unicode says (on page 315): "The U+200B ZERO WIDTH SPACE 
indicates a word boundary, except that it has no width.  Zero-width space 
characters are intended to be used in languages that have no visible word 
spacing to represent word breaks, such as in Thai or Japanese."

To Mike Kaply: although the above may solve Kent's problem, there is still a 
Bidi-related problem about &zwnj being represented as vertical bar in Mozilla's 
browser.  I leave it to you to update the bug report if you see fit.

	Hixie (not reading bugmail)
	Comment 8 • 24 years ago

This now WORKSFORME on Windows 2000 commerical build 6.0.17.2000080104.
Should this be closed, or are there remaining issues?

	Hixie (not reading bugmail)
	Comment 9 • 24 years ago

Eli, can you verifiy that this is WORKSFORME on all three tier 1 platforms too?
Cheers! The testcase is here:
   http://bugzilla.mozilla.org/showattachment.cgi?attach_id=2616

	Sean Richardson
	Comment 11 • 24 years ago

Viewing the attached testcase with 2000-08-02-08-M18 on WinNT (U.S. Version),
‌, ‍ ‎ and ‏ all display as "?", both as named and numeric
entity references. Richard Zach saw the same on Linux with today's build,
see -- Additional Comments From Richard Zach 2000-08-02 11:58 -- in bug 17962.

The testcase also includes U+200B ZERO WIDTH SPACE, ​ -- it appears from
the empty table cell that Mozilla does not understand that character at all.

	Hixie (not reading bugmail)
	Comment 12 • 24 years ago

A zero width space is not content, is it? Hence there is no content to show in
the cell. Sounds right to me... I get no cell for:

?<!ENTITY zwnj  CDATA "&#8204;" -- zero width non-joiner, U+200C NEW RFC 2070 -->
??<!ENTITY zwj   CDATA "&#8205;" -- zero width joiner, U+200D NEW RFC 2070 -->
??<!ENTITY lrm   CDATA "&#8206;" -- left-to-right mark, U+200E NEW RFC 2070 -->
??<!ENTITY rlm   CDATA "&#8207;" -- right-to-left mark, U+200F NEW RFC 2070 -->

...all of which are, IIRC, sort of non-characters.

The attached testcase works for me, modulo bugs in the testcase (an extraneous
semicolon in one case, for example). Note that "zwsp" is not valid, and this
bug is on about zwnj not zwsp.

If a character is displayed as "?" on some platforms, it is probably because
there are no fonts with that character in it. Try downloading (if you are on
Windows) the latest Microsoft Web Core Fonts (or whatever they call them).

	Eli Goldberg
	Comment 13 • 24 years ago

[handing back to ian]

	Sean Richardson
	Comment 15 • 24 years ago

Testing 2000-08-07-14-M18 with Bitstream Cyberbase, Times New Roman 2.55, and 
Times New Roman 2.82 (current), the same results were seen with each: in text
‌, ‍ ‎ and ‏ all display as a thin vertical bar (this is the 
same as originally reported), while alone in a table cell they all appear as 
"nothing".

Ideally, those characters would be displayed as an invisible, zero-width
glyph no matter what font (or font version) was chosen, since they require
special handling anyway. But for FCS it would probably suffice to have them
display (and work) properly when a Middle-eastern font is active ... after all,
the impact will negligible for display of other languages.

Mike?

	tenthumbs
	Comment 16 • 24 years ago

If you make the font size bug enough in a table, then you will see something
for these entities; at least you do on Linux with Bitstream Cyberbase.

	Mike Kaply [:mkaply] Assignee
	Comment 17 • 24 years ago

This character does not display as nothing on IE on an English system.

Do we know what the right thing to do here is?

	Lina Kemmel
	Comment 18 • 24 years ago

Zero-width "bidi" characters (like '‌') wouldn't be displayed in "bidi" 
text (containing at least one strong Light-To-Left character - spacing or zero-
width - or Right-To-Left directional attribute)on mozilla, even on systems 
without bidi support. The reason is that we remove bidi control characters 
before displaying. Cannot we do the same for non-bidi text?

	Lina Kemmel
	Comment 20 • 24 years ago

Can anyone give a test to the patch?

Summary of the changes:

1. Added BIDI control characters to the definitions of the stuff to be 
discarded.
2. Removed the method StripBidiControlCharacters() from nsBidiPresUtils.

Question:

Before the mentioned changes, characters classified as 'IS_DISCARDED' were only 
single-byte ones. Newly joined to them BIDI controls are double-bytes. Is that 
OK?

	Lina Kemmel
	Comment 21 • 24 years ago

The patch is buggy: it doesn't work, if word length returned by 
nsTextTransformer::GetNextWord exceeds content length of a text frame (this may 
happen, when a text frame is split during BIDI processing). I'll fix the bug 
and submit a new patch. 

	Lina Kemmel
	Comment 23 • 24 years ago

Well, the patch seems to work, and it also blocks bug 88588 (since it strips 
BIDI control characters before both text drawing and measurement). However, it 
caused many changes to nsTextFrame. So far it covers only *BIDI* zero-width 
characters (but it would be very easy to add whatever we might want). And 
finally, we have to separate BIDI controls (and maybe other not-single-byte 
characters, if any) from other discardable stuff.

	Christopher Hoess (gone)
	Comment 24 • 23 years ago

Is this a working patch (i.e., ripe for reviewing?)  If so, you should add the 
"review" keyword, and email the BIDI module owner (mkaply) asking for review, 
cc'ing to reviewers@mozilla.org.  (This sounds more in his line than ftang's).

	Mike Kaply [:mkaply] Assignee
	Comment 25 • 23 years ago

Lina, can you give a quick explanation of your changes?

I am going to send it to a layout person for review.

	Lina Kemmel
	Comment 26 • 23 years ago

Hello all,

Here is a brief overview of the changes.

1. The idea was to add Bidi control characters to the existing list of 
discardable characters (nsTextTranformer.cpp). These characters, being omitted 
when obtaining next or previous word for presentation, are not included in a 
paint buffer, and are not counted when measuring or displaying a text.

2. But this very small change discovered a bug, which may occur if *any* (not 
only Bidi) discardable character is present.

For example,

Given a word abcDEF
(capitalized is Right-to-Left, '|' is a character to be taken off).

Suppose for Bidi purposes this word, initially represented by 1 text frame, is 
split into 2 text frames - "abc|" and "DEF". And suppose that for measuring the 
text content of the 1st frame - "abc|" - we're calling 
nsTextTransformer::GetNextWord(..). GetNextWord() removes the character '|' and 
returns word length that equals to 6 (while the desired length is 3). The 
caller is not aware of how many characters were discarded. He can only retrieve 
the frame content length (4) and ensure that the word length doesn't exceed it: 
then he sets the word length to 4 - which is also incorrect.
Working around this problem, before getting next or previous word, we set a 
stopper, such that word can't go beyond it (see nsTextFrame.cpp; 
nsTextTransformer.cpp).

3. And finally, StripBidiControlCharacters(..) in nsBidiPresUtils became 
unnecessary, and was removed.

	Lina Kemmel
	Comment 27 • 23 years ago

Sorry about the typo - the example should look as:
"Given a word abc|DEF"

	Marc Attinasi
	Comment 28 • 23 years ago

Comment on attachment 41932 [details] [diff] [review]
Working patch

There is some really bad indentation - in the new code it is not so bad, but see the BIDI blocks after
@@ -4038,12 +4064,18 @@

[s]r=attinasi

	Lina Kemmel
	Comment 29 • 23 years ago

This bad indentation is outside BIDI code.

	Alistair Vining
	Comment 30 • 23 years ago

Is there room in this bug for 'vertical tab (U+000B) should not display as box'
( http://www.fontfont.de/fffstuff/f_central.html ) or should I file another bug?

Affects the whole site, ignored by NS4.x and IE, may apply to other chars from
U+0000 - U+001F.

	Simon Montagu :smontagu
	Comment 31 • 23 years ago

Alistair, I think that's a separate issue (and there may be a bug on it already,
though I couldn't find one).

	Simon Montagu :smontagu
	Comment 32 • 23 years ago

Lina, I regret to inform you that the patch is bitrotted by the checkins on bug
98546

	Alistair Vining
	Comment 34 • 23 years ago

Thanks Simon, vertical tab issue is now bug 106311.

	Mike Kaply [:mkaply] Assignee
	Comment 35 • 23 years ago

Strange. On IE 5.5, zwnj and zwj are displayed.

I will get this checked fix checked in tomorrow night.

are we going to worry about the editor case?

	Mike Kaply [:mkaply] Assignee
	Comment 36 • 23 years ago

Fix checked in