Closed Bug 137742 Opened 23 years ago Closed 22 years ago

Multiple To and CC headers are not sent when using Long email addresses

Categories

(MailNews Core :: MIME, defect)

defect
Not set
critical

Tracking

(Not tracked)

VERIFIED FIXED
mozilla1.0.1

People

(Reporter: ludovic, Assigned: bugzilla)

References

Details

(Whiteboard: [adt2 RTM] [ETA 06/25])

Attachments

(1 file, 1 obsolete file)

From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020415 BuildID: 2002041513 When long email addresses are used (Ludovic Dubost <ludovic@netvalue.com>) instead of simple email addresses (ludovic@netvalue.com) with multiple To and/or CC recipient, then the second one of the To and the CC are not added to the headers.. However the email is sent to all people (so people that are not in the To or CC receive the email like in a BCC).. Example emails will follow.. The bug apparently appeared in Mozilla 0.9.8 and is reproducible in 1.0rc1 on Linux and Windows... The problem does not appear in Netscape 6.2.1 The email in the sent folder is correctly formated... Reproducible: Always Steps to Reproduce: 1. Write an email 2. Add To and CC looking like this: To: Ludovic Dubost <ludovic@netvalue.com> To: Ludovic Dubost <ludo@netvalue.com> CC: Ludovic Dubost <ldubost@pobox.com> CC: Ludovic Dubost <ludovic.dubost@netvalue.com> 3. Send the email.. Actual Results: Message received looks like this: Return-Path: <ludovic@netvalue.com> Received: from mail-fr.netvalue.fr ([192.168.1.18]) by mail.netvalue.fr (Netscape Messaging Server 3.6) with ESMTP id AAA4A33; Tue, 16 Apr 2002 14:04:02 +0200 Received: from netvalue.com ([192.168.1.102]) by mail-fr.netvalue.fr (Netscape Messaging Server 4.01) with ESMTP id GUNTIP00.OBV; Tue, 16 Apr 2002 14:04:01 +0200 Message-ID: <3CBC1331.5010606@netvalue.com> Date: Tue, 16 Apr 2002 14:04:01 +0200 From: "Ludovic Dubost" <ludovic@netvalue.com> Organization: Netvalue ( http://www.netvalue.com ) User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020415 X-Accept-Language: fr, en MIME-Version: 1.0 To: Ludovic Dubost <ludovic@netvalue.com> CC: Ludovic Dubost <ldubost@pobox.com> Subject: Test email to show missing headers Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Test email to show missing header Expected Results: Something like: Message-ID: <3CBC1331.5010606@netvalue.com> Date: Tue, 16 Apr 2002 14:04:01 +0200 From: Ludovic Dubost <ludovic@netvalue.com> Organization: Netvalue ( http://www.netvalue.com ) User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0rc1) Gecko/20020415 X-Accept-Language: fr, en MIME-Version: 1.0 To: Ludovic Dubost <ludovic@netvalue.com>, Ludovic Dubost <ludo@netvalue.com> CC: Ludovic Dubost <ldubost@pobox.com>, Ludovic Dubost <ludovic.dubost@netvalue.com> Subject: Test email to show missing headers Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Test email to show missing header All 4 people have actually received the email.. Look at the To and CC headers in the results: To: Ludovic Dubost <ludovic@netvalue.com> CC: Ludovic Dubost <ldubost@pobox.com> It should be: To: Ludovic Dubost <ludovic@netvalue.com>, Ludovic Dubost <ludo@netvalue.com> CC: Ludovic Dubost <ldubost@pobox.com>, Ludovic Dubost <ludovic.dubost@netvalue.com> This problem is a corporate stopper as it defeats all filtering and email exchanged inside the company.. This is actually a problem for me and I will need to downgrade my version of Mozilla if there is not a solution to this problem... I consider this is loss of data as people don't receive the email that they should...
Correction: The bug does not appear in Mozilla 0.9.8 and neither in 0.9.6
I believe this bug should be fixed for 1.0.. So I'm adding the mozilla1.0 keyword..
Keywords: mozilla1.0
rc1 is not out yet. you're running builds from the rc1 branch. I tried reproducing this by sending an email adressed to 4 email addressed: 2 TO and 2 CC and it came ok. Are you sure this is not a MTA problem? Can you send it in other clients and receive it properly formatter?
It only happens with certain versions of Mozilla.. It does not happen with Mozilla <= 0.9.8 with the same MTA.. The problem also happens using RC1.. It could be a combined pb.. Mozilla + certain MTA.. Our MTA is a Netscape Messaging 3.. When you are trying to reproduce are you sure you are putting email addresses and a full name (Ludovic Dubost <ludovic@netvalue.com>) .. Because the problem only happens in this case.. not if the email address only contains a standard email (ludovic@netvalue.com)
I confirm the problem. Happens using Mozilla >= 0.9.9 with MTA = Netscape Messaging Server 3.66 Must be something related to To: and CC: fields composition which changed between 0.9.8 and 0.9.9. Seems only to happen with long names with spaces in them like John Doe <john_doe@doecompany.com> but not with long names without spaces like Doe <john_doe@doecompany.com> I'll check that it doesn't happen if you use another MTA and that reverting to 0.9.8 will solve the problem. I suggest the bug is reopened so that I can be in the 1.0.0 scope...
confirming based on Thierry's comment. please provide the information you mentioned when you can
Status: UNCONFIRMED → NEW
Ever confirmed: true
Confirming this to be a regression : version 0.9.8 is working correctly. A few of my test cases : Sending with Mozilla >= 0.9.9 an email with : To: John Doe <jdoe@tc.com> To: Jane Doe <jane@dot.com> -> message will be received by John and Jane -> headers will just show John -> every email client will just show John -> some will show an incomplete To: line like "To: John Doe <jdoe@tc.com>," -> INCORRECT BEHAVIOUR Sending with Mozilla = 0.9.8 the same email : -> message will be received by John and Jane -> headers will show John and Jane -> CORRECT BEHAVIOUR Sending with Mozilla >= 0.9.9 an email with : To: John Doe <jdoe@tc.com> To: Jane <jane@dot.com> -> message will be received by John and Jane -> headers will show John and Jane (!) -> CORRECT BEHAVIOUR So the problem seems to be conditionned by : - using multiple To: or Cc: addresses - using long names with spaces in them - using the Netscape Messaging Server MTA (I have to check that one) I will check that behaviour is correct with Exim instead of NMS, but I think it will be : must be the reason why you don't manage to reproduce when you're not using NMS. Something must have changed in the way To: or Cc: addresses are submitted between 0.9.8 and 0.9.9 that confuses NMS (but sending is correct, it's just the headers that are incorrect...)
Informative network captures about differences between 0.9.8 and 1.0RC1 : (1) Moz 0.9.8 writes the multiple To: header in the DATA like this : To: John Doe <jdoe@tc.com>,[0D][0A][09]Jane Doe <jane@dot.com>[0D][0A] and it works properly with NMS delivery (2) Moz 1.0RC1 (and 0.9.9) writes it this way : To: John Doe <jdoe@tc.com>, Jane Doe[0D][0A] <jane@dot.com>[0D][0A] and it doesn't work properly. (3) One-word names are written OK using 1.0RC1 : To: John Doe <jdoe@tc.com>, Jane <jane@dot.com>[0D][0A] and it works properly. I think the 0.9.8 line is buggy but tolerated. It must have been corrected in 0.9.9 but there is still a problem with multiple-words in names (see trace (2) above) which confuses NMS and might confuse others. I think it should be corrected before the 1.0 final release since other mail servers than NMS might be affected by the buggy To: line...
I confirm this is not a NMS-specific bug. The "To:" and "Cc:" (and probably others like "Bcc:") headers are written incorrectly since Moz 0.9.9 : - when you use multiple addresses - when one of the names used (but not the first) is a multiple-word name like "Jane Doe <jane@dot.com>" The part about NMS is that it rewrites the headers before delivering the message, and the parsing of the wrongly-written headers fails, but this is because there is a bug in Mozilla, not in NMS. Reproduced on Linux and Win32.
QA Contact: gayatri → esther
I originally tried it with the conditions specified running qmail as the mta and I did not see this problem. So as you said, some maybe more tolerant than others.
More info : The problem is not due to having multiple words in the display name but is due to having a long name. More precisely, Mozilla 0.9.9 and over can cut the To: header line between the display name and the address : To: John Doe <jdoe@tc.com> To: JohnnieHasAVeryVeryLongName <johnnie@dot.com> will be cut in two lines between JohnnieHasAVeryVeryLongName and <johnnie@dot.com> (inserting space) : To: John Doe <jdoe@tc.com>, JohnnieHasAVeryVeryLongName <johnnie@dot.com> whereas with 0.9.8 and before, it wrote the display name and address on the same line and will cut between addresses (inserting tab). To: John Doe <jdoe@tc.com>, JohnnieHasAVeryVeryLongName <johnnie@dot.com> The two behaviours should be accepted by the mail server, but NMS 3.X doesn't like it and rewrites the headers (why?), dropping addresses from the incomplete one to the end. So this could be considered as an NMS bug, but there is no way to work around it and Mozilla is the only client I know of with this behaviour. Should this be corrected in Mozilla ? Are there any other mail servers having problems with the Moz 1.0RC1-style recipient headers ?
A last note on this issue : I think we should change the title of this bug to : "New recipient headers folding confuses some mail servers" because the problem can be reduced to where exactly you should fold the recipient header lines (like To: or Cc:) in case of long header fields. From RFC #822 : Note: While the standard permits folding wherever linear- white-space is permitted, it is recommended that struc- tured fields, such as those containing addresses, limit folding to higher-level syntactic breaks. For address fields, it is recommended that such folding occur between addresses, after the separating comma. Old Netcape Messaging Server versions (like 3.X) expect the "recommended" way of folding long header lines, and maybe other mail servers as well. Mozilla 0.9.8 was implementing the "recommended" way. Header folding in Mozilla 1.0RC1 is standard-compliant strictly-speaking, but not the recommended way.
From our testing the problem also occurs using Netscape Messaging Server 4.x Does annybody know why the header folding has been changed ? I think this should clearly be fixed because I'm not sure that even if patches are created for the servers, corporation would not necessasrly deploy them easily.. This can dramatically reduce the adoption of Mozilla in corporations.. and this is not what we all want...
This is a regression by a fix for #73403. Change platform and OS to ALL.
OS: Linux → All
Hardware: PC → All
Change component to MIME, cc ducarroz, putterman. So the current known problem is that address may be removed by NMS 3.X?
Component: Mail Back End → MIME
Yes.. but only in the headers.. the email are actually sent to all people..
>Yes.. but only in the headers.. the email are actually sent to all people.. so there may be a problem when reply all cc ducarroz, putterman
Yes.. the typical scenario of what happens is: - somebody sends an email to multiple people in the company.. With the LDAP server + rewritting of emails addresses by the server, the email comes in with long names in the addresses - reply to all using Mozilla 1.0 misses some addresses in the headers - recepient receive the email with missing headers.. Filters might not put the email in the right folder and reply to all will not include some users... After a few weeks of usage this caused several communication problems for me, and I had to revert to Mozilla 0.9.8 to avoid the problem...
Just tried with NMS 4.15 MTA and works fine with following To: line: To: Takayuki Tei <taka@netscape.com>, Johnnie Has A Very Very Long Name Johnnie Has A Very Very Long Name <takayukitei@netscape.com>
I think this has to be evaluated by mail group, adding nsbeta1.
Keywords: nsbeta1
Remark on comment #19 : Since the problem depends on where exactly the line folding occurs, it might work with short names, not work with long names, work with very long names, etc... But it is also possible that header parsing has been rewritten in NMS 4.15. To correct this one, I think there is two possibilities : you can either implement the RFC822 "recommended" way of folding or revert to the pre-0.9.9 recipient header encoding... Both should be compliant with older versions of NMS. Like Ludovic, I had to delay Mozilla deployment in my company due to header information lossage.
reassigning to ducarroz. Jean-Francois, can you look at this? Marking adt1 until more is known about this.
Assignee: mscott → ducarroz
Keywords: nsbeta1nsbeta1+
Whiteboard: [adt1]
>Since the problem depends on where exactly the line folding occurs, it might >work with short names, not work with long names, work with very long names, etc... Taka, is that also be applied to MIME encoded case which tends to be a long name? Is that possible to enable the old folding behavior by option?
> But it is also possible that header parsing has been rewritten in NMS 4.15. Which patch version are you using? There have been bunch of header rewrtting problem in earlier version. You should upgrade to latest patch. > Both should be compliant with older versions of NMS. We have released patch to collect these problem on server. Do you know any other server implementation which doesn't work correctly with folded structure headers? > Like Ludovic, I had to delay Mozilla deployment in my company due to header > information lossage. We recommend you to upgrade to latest patch of NMS.
> Taka, is that also be applied to MIME encoded case which tends to be a long name? > Is that possible to enable the old folding behavior by option? RFC 2047 "8 Examples" shows folded To: field body with multiple recipients. RFC 822 "3.1.1. LONG HEADER FIELDS" also shows folded To: field body with multiple recipients. Since RFC 2047 is designed on top of RFC 822, it's impossible to go back to old broken MIME encoding style without seeing bugs such as #73403.
In comment #24: > We have released patch to collect these problem on server. I meant "correct" not "collect".
Attached patch patch from taka (obsolete) (deleted) — Splinter Review
explanation about the patch copied from e-mail: Although the attached patch should fix the bug 137742, this bug is fundamentally caused by a NMS3.x bug. RFC 822 explicitly shows how long headers can be folded, and RFC 2047 is designed on top of that folding logic. This patch works only when given structured field body is in ASCII, therefore, those MTA which cannot unfold folded To:/Cc: field body can be screwed up when user reply to multiple receipients with RFC 2047 encoded names.
On comment #24 : > Which patch version are you using? There have been bunch of header rewrtting > problem in earlier version. You should upgrade to latest patch. I'm using NMS version 3.62. If there is a patch which could correct the bug appliable to this version, I would be interested to download and install it (I cannot find anymore NMS 3.X patches on IPlanet site). However, if the only way to get rid of this is to reinstall a complete NMS (or other mail server) I will have to delay this (we have plans to change the mail server for Q4 2002) and delay Mozilla deployment (planned for Q2 2002) accordingly. > Do you know any other server implementation which doesn't work correctly > with folded structure headers? No. I tested with sendmail and exim without any problem. On comment #27 : > Although the attached patch should fix the bug 137742, this bug is > fundamentally caused by a NMS3.x bug. I agree completely. For Mozilla, this is more a compatibility feature than a bug. But if NMS 3.X is the only mail server having problems, Mozilla is the only mail client I know of that triggers the problem... I'm available and ready to test any win32 build to validate the problem is gone. Just tell me in which nightly build it will appear. And thanks to you all for supporting all these very-specific demands and coping with my bad English !
I have very simlar problem when I sent message to both NNTP and SMTP. It sometimes fails. I cannot even see sent messages in Sent folder. (I use 1.0rc1/Linux and didn't have problem with 0.9.9) SMTP server is smtp.mail.yahoo.co.jp NNTP server is news.php.net I don't send much mails, so I cannot tell if I have the same problem or not. It seems it's closely related.
There is a known bug 137742 which also causes message delivery problem. Can you give us a couple of examples you couldn't send message as expected?
*** Bug 141703 has been marked as a duplicate of this bug. ***
Status: NEW → ASSIGNED
Target Milestone: --- → mozilla1.0.1
Blocks: 143047
sofar, no luck, I am not able to reproduce this problem using various version of Mozilla/Netscape, even most recent builds (branch and trunk)
The only problem I see here is that when we reformat the headers to comply with RFC2047, we can potentially fold an address in the middle between the long name and the email address. Then some MTA might tried to reformat the headers and cut some of the recipients by accident (bug?). I beleive this problem has been introduced on February 20, 2002. Taka proposed patch will fix that problem. Reassign to taka,
Assignee: ducarroz → taka
Status: ASSIGNED → NEW
Comment on attachment 81004 [details] [diff] [review] patch from taka R=ducarroz
Attachment #81004 - Flags: review+
Comment on attachment 81004 [details] [diff] [review] patch from taka Please make len a PRInt32 instead of a raw int. We should be using the NSPR types for integers unless the code is platform specific.
Whiteboard: [adt1] → [adt1 RTM]
reassign to myself as I have some free cycle...
Assignee: taka → ducarroz
Attached patch Proposed fix, v2 (deleted) — Splinter Review
Replaced int by PRInt32.
Attachment #81004 - Attachment is obsolete: true
accepting
Status: NEW → ASSIGNED
Comment on attachment 87115 [details] [diff] [review] Proposed fix, v2 sr=bienvenu
Attachment #87115 - Flags: superreview+
Comment on attachment 87115 [details] [diff] [review] Proposed fix, v2 R=ducarroz
Attachment #87115 - Flags: review+
Fix checked in the trunk
Status: ASSIGNED → RESOLVED
Closed: 22 years ago
Resolution: --- → FIXED
Esther, can you verify this on the trunk?
Correction appears in 2002061208/Win32. Thanks to everyone...
I'll give it a try, but I need to first reproduce it to be able to verify it. It will take time.
I spent a gread deal of time trying to reproduce this and couldn't. I took: linux mozilla build 0.9.9 linux mozilla build rv:1.Orc1 with Gecko string 20020417 (couldn't find one with 20020415) linux commercial branch 20020415 win32 commercial branch 20020610 All test conducted using Netscape Messaging Server 4.15 and testing addresses with really long pretty names with spaces (50+characters), multiple recipients, mixing addressing using To: and CC:, I could not reproduce this. Since reporter has tested this and commments that "correction appears in 2002061208/Win32", I will take a trunk build with the fix and do a spot check to see if basic sending to mutiple recipients is still working.
You need to test against early version of NMS3.x since this symptom appeared due to a bug in NMS3.x, not Mozilla.
I've checked it with build 2002061221 and the problem does not occur on my computer anymore with a NMS 3.x Great news.. that will probably allow deployment of the next release of mozilla in our company..
Lowering impact since it only happens on one server.
Whiteboard: [adt1 RTM] → [adt2 RTM]
esther - can you pls verify this fix on the branch? thanks!
Keywords: verifyme
Esther will not verify that this works with Netscape Messaging Server 3.66 since we do not actively support that server. However, Esther will verify that this does not break anything with the servers we do test with, i.e. NMS 6.0, NMS 6.1 and NMS 4.15.
Whiteboard: [adt2 RTM] → [adt2 RTM] [ETA 06/25]
Has Esther verified that this fix does not cause any regressions, with servers we currently support?
I have not done formal testing ye. I'm testing other regressions that have come up in the last 2 days that are now fixed on the trunk and waiting for verification. But I have been using the trunk builds for these regressions for daily sending to email address of the same type mentioned in the original scenario, there doesn't seem to be any problems. Since the reporter is saying it's fixed with the trunk build, if this hasn't gone into the branch yet I think that it could. I am confused by Jaime's question in comment #49 about verifying it on the branch? Is the fix in the branch or trunk or both now?
In the past week 6-24 through today 7-1, I have used the trunk builds (this bug was fixed in trunk on 6-12) and branch builds (I don't see that it was ever checked into the branch yet) with our NS server 4.15 & 6.0. I have seen no new problems with multiple addressees in messages with long names or short names. Since we never saw the problem on these servers all I can say is the fix doesn't appear to have broken anything in the addressing field. If this was checked into the branch after 6-27-2002 then I will say this is verified for both trunk and branch. If it hasn't been checked into branch, then I will say verified on trunk.
This fix has not been checked in the branch has it has never been approved by ADT.
adding adt1.0.1-. Let's get this in the next release.
Keywords: adt1.0.1adt1.0.1-
I tested this scenario again with latest trunk builds using the 4.15 & 6.0 messaging servers, this works fine. The fix that was checked in for a 4.33 messaging server bug, so I just verified the 4.15 and 6.0 servers still work. Since we will be basing future releases off the current trunk build I will verify this as fixed.
Status: RESOLVED → VERIFIED
Did this make it into Netscape 7.02? I'm seeing this problem with Netscape 7.02 and Netscape Messenger 3.6X.
Product: MailNews → Core
Product: Core → MailNews Core
Keywords: verifyme
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: