Mozilla 2002031708 / Mac OS 9.2.2: Some email messages recived disaplay in the mail pane as being from 1/1/1970. However, the headers seem to have the correct date in them. I am attaching the headers of 3 sample emails. I should note that other emails do display proper dates

I get this on build id 2002032008 on linux.

*** Bug 132514 has been marked as a duplicate of this bug. ***

I'd posted a duplicate of this bug, so I write here ! This bug seems to happen with long titles. More than 20 or 25 characters in it. Is this a clue ?! I hope so :-)

Shoshannah, could you please attach to this bug report a mail that has a correct date.The messages that you've attached do show 12/31/69 as date. We have several cases like that with malparsed headers. My guess for the emails in this bug report being incorrectly sorted is a missing Date attribute or an incorrect one; i see missing commas and Day in some server stamps. Thanks.

after some experimenting with the attached message , here is what i found: 1. if i edit the message source and swap the header with date (placing header after the date) then the message shows in my Inbox with the correct date (03/19/2002; - if i edit the page source and substitute the 1255 encoded header for another one (like iso-2022 ja) without changing the place of a Date tag (still before the date) the message shows up in the thread as 03-19-2002; - changing the mime 1255 header to "abc" will show the message in the thread as 03/19/2002; i am wondering whether this particular header is not missing any byte promting the mail server not to parse it correctly before the Date stamp. Cc'ing Naoki and Simon. Simon, could you please look if the header is properly encoded? Thanks.

The encoding of the headers of the attachments in here (and attachment 75394 [details] in the dupe) all seem correct to me.

Still having it, kind of random ! :-( It seems to be related with accent in the title (é,è and others). Maybe a clue ?!

I 'm seeing this bug for a couple of weeks (currently on Win2K, 2002041103). Needless to say, it's quite annoying because, in a long message list, it's difficult to find those messages. Imo, 1.0 should not be released with this regression. As far as I can remember, I see this bug on some messages with non-ascii (ISO-8859-7 encoded) subjects. Date formatting in the message source seems to be correct. Attaching such a message (slightly modified to remove personal info) that might help you as another testcase. Platform->All, OS->All.

Nominating for mozilla 1.0. This bug is highly visible for intl users.

From comment #8, it seems like the MIME encoded subject header may be confusing the parsing of the next line which is the date header. Possibly the subject line is malformed and we are not gracefully recovering for the next line, or we have a problem with our MIME decoding of the subject header which has a side affect on the next line... Someone should get in a debugger and see what is going on as we parse these 2 headers.

This is caused by bug 21267. It removes 'RE' from MIME encoded subject, decode it first then encode again. It assumes that the re-encoded length is smaller than the original length. But like the examples attached here, it may be encoded differently and the result could be longer than the original. It should not copy the re-encoded string if the new length is larger than the original length. Reassign to nhotta.

Nominating for nsbeta1. It always happen when the subject is encoded differently from the Mozilla's encode type.

This prevents the data corruption. In future, we need a capability to specify encoding method to MIME encoder instead of always use our pre-defined method.

when will this bug happen 1. the subject in message header is MIME encoded, and 2. the MIME "encoding" is B for single bytes charset or Q for Multibyte charset. 3. there are a "Re" inside the Subject field The same buggy code could also possible also will cause crash (altought we have not see an incident of that but that is also possilble) or damage other record. Impact users- mainly Hebrew (1.9M), Greek (1.6M) The patch potential fix other mail crash or data damage issues. let's do some research to see the buggy code really damange the data or cause some crash.

I looked at nsParseMailMessageState which is one of the callers of NS_MsgStripRE, it maintains pointers of original headers instead of copies. "RE" stripping is done by changing the pointer but not altering the original header. But the patch of bug 21267 modifies the header if MIME encoded which is wrong even when the result is smaller than the original. So far, only known problem is this bug. cc to putterman, ducarroz, bienvenu, taka to share the info.

Comment on attachment 80051 [details] [diff] [review] Changed to use 'B' encoding for arabic, greek, hebrew. Added an assertion, so we can catch more cases. sr=bienvenu

I have observed something similar, with much worse effects. Something similar happens in newsgroups in my college. It _might_ be related to Netscape 6.2 being installed in the computer-rooms. Anyway. The effect is there for everybody to be seen: news-server: news.fh-wedel.de newsgroup : fhw.talk In said newsgroup look for a thread "ab der TU müßt man sein...." and another thread "Re: ab der TU müßt man sein....". The "Re"-thread is aktually nothing more than lots of anwsers to the main-thread, which mozilla seems to be unable to sort into the correct location (in threaded-view). The awnsers in that thread are readable by mozilla, but I cannot view the source of them (<ctrl>-u) and the sender and date in the thread-pane are completely mangled.

Comment on attachment 80051 [details] [diff] [review] Changed to use 'B' encoding for arabic, greek, hebrew. Added an assertion, so we can catch more cases. R=ducarroz

*** Bug 138222 has been marked as a duplicate of this bug. ***

Bug 138222 contains examples of ISO-8859-1 and ISO-8859-2. My guess is that MIME decoder now changed to encode entire subject which may produce a larger result. Taka, is that correct?

You don't have to compromise with sorting issue. NS_MsgStripRE() is used at 3 places only. So, why don't you add an argument (PRBool *) to NS_MsgStripRE() to let caller know whether it has to free returned string rather than sticking to original length and using strcpy()?

adt2 -> adt1, because it also impact Latin1 and other users

Please change the summary to something that contains 'news' and 'broken threads', for example 'Some email/news messages display incorrectly with the date of 1/1/1970 and in broken threads'. It will make it easier to find for news users since bug 138222 is now resolved.

I was chatting with bobj and come up with an idea. It is possible to skip "RE:" without encoding the string again. It needs to decode MIME encode but without charset conversion then we can search for "RE:", remember the positions and copy the rest of the string over. This way, the result is always shorter than the original and no modification to the subject except for removing "RE:". Probably, we want that functionality in the MIME decoder/encoder interface. I think this would be not a small change, so not for moz1.0. I will try taka's suggestion for now. Changed summary to include 'news'.

Taking severity and dataloss keyword from duped bug 138222. Not sure about mail3 (I leave the mail[0-9] keyword for someone else). The summary could need a cleanup, maybe: mail/news: broken threads, lost articles, date of 1/1/1970 pi

nhotta@netscape.com wrote: >It is possible to skip "RE:" without encoding the string again. NS_MsgStripRE() must return "Hello, world." for the all of following input (it's in raw ISO-2022-JP for viewing purpose): Re: Hello, world. \x1b(BRe: Hello, world. \x1b(JRe: Hello, world. Although I don't know actual implementation works this way, they are all comformance to RFC 1468.

Filed bug 139317 for adding "Re:" strip support to nsIMimeConverter.

Comment on attachment 80478 [details] [diff] [review] Added out argument for NS_MsgStripRE to avoid altering the const argument. R=ducarroz

*** Bug 136461 has been marked as a duplicate of this bug. ***

Comment on attachment 80478 [details] [diff] [review] Added out argument for NS_MsgStripRE to avoid altering the const argument. sr=bienvenu

adding adt1.0.0+. Please check this into the branch as soon as possible and add the fixed1.0.0 keyword.

checked in to the trunk, asked branch check in approval

Marina, could you take care of this? Thanks.

in the 2002-04-23 trunk build the original mail from Shoshannah is displayed correctly (3/19/2002) but bug# 136461 is still present:the headers in newsgroups are still diplsaying as 12/31/69 or 01/01/70

It's showing fine using my debug build. Please make sure to remove old .msf file and try again. Please try data in bug 138222 also.

after removing old msf files for all intl newsgroups i don't see this bug any more, gone! fixed! Will verify other dups as well. ready to get checked into the branch

@marina i tried with deleting the msf -files too.. several times. the phenomen returned and returned. Now I have the prob in about 10 groups ... before trying to find out, whats the problem, only 2 groups showed me the 1970s. (098: few postings, 099: more postings, RC1: you can say: 1970s with each request for new postings) I hope, you're right and it's fixed... let's see ;-)

fixed in the trunk

frank.1977@gmx.de, after you delete all msf in your profile, restart the application and you shouldn't see this bug. One of the newsgroups i tried was de.test on news.mcom.com. I don't see it there any more. If your problem will persist please point me to the newsgroup you are having problem with. Thanks.

I think the safe way to verify is testing with a new profile.

Following comment #46, I deleted .msf file. Can see it again, my build should be a little older :-) "Only" using 2002042306... So, let's wait for a new and younger one :-)

2002042306 should contain the fix. Please test with a new profile.

I tried to download the "latest" - both, exe and zip are corrupt. - deinstalled rc1 - reinstalled RC1.. closed all instances and fast-start-icon - deleted all msf - startet mozilla - the "old 1970" are still there, when download up the indicated "here are waiting xxx postings for you" in the tree.. there will be downloaded new 1970s I will watch, what will happen in the other groups with the older ones. If there will be new ones, I'll tell you. new profile: won't help. others in a german newsgroup tried this, if I would correctly remember.. I'll test it tomorrow I noted two new bugs today.. I will look up, if I can find numbers (the "new mail"-symbol in the tray will not disappear, when reading in the folders .. when closing the last tab on the right side via the "X", the new last one won't have a right (shadow-)borderline... ) If I dont find sth I'll open new bugs

Please note this was checked in to the "trunk". The fix is not included in the branch (including RC1).

To avoid spamming :-) In answer to comment #49 : I donwloaded 2002042306 from latest-1.0.0 folder In answer to comment #51 : for now, I am only download branch build, not trunk ones. I had this bug for one month or so. I could live with it some more days. Just hopping this patch would be included soon in branch builds. Anyway, thanks for fixing it.

I downloaded rv:0.9.9+ Gecko/20020423 from /2002-04-23-06-trunk/ you told me It works properly with my old profile and with a new profile. thx the file (zip, exe that will be shown in /latest/ , /latest-trunk/, /latest-1.0.0/ is corrupt

Comment on attachment 80478 [details] [diff] [review] Added out argument for NS_MsgStripRE to avoid altering the const argument. a=rjesup@wgate.com for branch checkin

fixed1.0.0, checked in to the branch

*** Bug 142501 has been marked as a duplicate of this bug. ***

*** Bug 133847 has been marked as a duplicate of this bug. ***

*** Bug 142501 has been marked as a duplicate of this bug. ***

verified with 2002-05-29-1.0.0 build