User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; rv:1.9.1.1) Gecko/20090730 Iceweasel/3.5.1 (Debian-3.5.1-1) Build Identifier: Mozilla/5.0 (X11; U; Linux i686 (x86_64); ru; rv:1.9.3a1pre) Gecko/20090829 Shredder/3.1a1pre Subject, which is not MIME encoded, but contains non-ASCII characters is not displayed correctly in Message list: non-ASCII characters are replaced by question mark "?". In message part of window the topic line is displayed correctly. It looks like application doesn't check "Content-type" header when is building message list, but, when message is opened, all headers are recognized correctly. It doesn't match standards, but it exists. Reproducible: Always Steps to Reproduce: Subject: some 8-bit encoded string Content-type: text/html; charset=windows-1251 Actual Results: Question marks in topic in message list Expected Results: Display correct subject

Same problem as Bug 468351? > Build Identifier: Mozilla/5.0 (X11; U; Linux i686 (x86_64); ru; rv:1.9.3a1pre) > Gecko/20090829 Shredder/3.1a1pre Bug 468351 is already fixed(fixed on 2009-07-28, so landed around 7/28). Patch for Bug 468351 was landed on 1.9.1 branch only?

Bug 468351 looks similar, but, as i understand there was no talk about "Content-type" header. When i tried to open attached to this bug example message Thunderbird crashed. Anyway, as i said, the subject is displayed correctly in message part of window and when message is opened. When message is opened in another tab, application window and tab names contains "?"

Checkin date of patch for Bug 468351... > Mon Aug 31 00:19:26 2009 +0200 (at Mon Aug 31 00:19:26 2009 +0200) See Bug 468351 Comment #43. Vladimir, can you produce problem with newest build the patch is landed on?

I've checked Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.3a1pre) Gecko/20090901 Shredder/3.1a1pre. The problem still exists, but i've found one big note: Bug appears only with IMAP accounts, not with POP.

(In reply to comment #5) > big note: > Bug appears only with IMAP accounts, not with POP. Is it for same mail data in local mail folder(POP3) and in IMAP folder? If for different mail, charset may be relevant. Can you attach mail? (save as .eml, selecting "mail file" as file type and putting ".eml" in file name explicitly at file picker dialog. See Bug 508597, Bug 509379.)

Sample message. When it is opened from local file system it is displayed correctly, but when it is received from IMAP server it contains "?" in subject.

I've made additional tests with messages and found that the same problem appears with other headers in message list (fields from,to...), which are not mime-encoded. The subject of messages with non-mime headers, received using POP3 protocol is displayed correct, but "from" field is decoded using wrong codepage. In message part of window (or when message is opened in new tab) the headers are shown correct. Note: For fields other than "subject" it's not a big problem, because bad-written messages come from users of bad localized or very old software (it is very rare case), or from bots/notifiers/subscribes which is written with some simplification or for English-speaking users (when codepage problem don't appears at all). After the second source of "bad" messages is localized, usually only subject and content of message is translated, sender name is left in English or contains only e-mail address.

(In reply to comment #6) > (In reply to comment #5) > > big note: > > Bug appears only with IMAP accounts, not with POP. > > Is it for same mail data in local mail folder(POP3) and in IMAP folder? It is actual for same message (and same account), received different way. I've checked Gmail (pop3 and imap) account and account on local Zimbra server (also pop3 and imap).

(In reply to comment #7) > Sample message Checked with next build. Problem was reproduced. > Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4pre) Gecko/20090902 Shredder/3.0b4pre (1) Save the mail in local mail folder file(add "From - ..." line), rebuild-index. => Subject is displayed as expected in Subject column. (2) Copy the mail to a Gmail IMAP folder. => Subject is displayed as expected in Subject column. If copy from local to IMAP, subject of local mail looks to be copied to .msf of IMAP folder, instead of one in mail header returned by server. (3) Rebuild-index of the Gmail IMAP folder (or delete xxx.msf and restart of Tb). => Subject is displayed as "?...? ????" in Subject column. Problem is reproduced by attached mail. Confirming. When IMAP, mail data is obtained by two steps. (non multipart mail case) > 6 UID fetch 2 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)]) > 7 UID fetch 2 (UID RFC822.SIZE BODY.PEEK[]) Subject is obtained at first "6 UID fetch 2 (... FLAGS BODY.PEEK[HEADER.FIELDS ...". Care for "raw binary without encoding" of IMAP should probably be done at different place from POP3 case.

Who converted to "???????? ????" was Gmail IMAP. What can IMAP client do? (1) FETCH BODY[HEADER.FIELD ... > 00000045 4.71495581 [1012] 2752[1f6ac40]: 28f2800:imap.gmail.com:S-NS:CreateNewLineFromSocket: * 1 FETCH (UID 74 RFC822.SIZE 1782 FLAGS (\Seen) BODY[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type received)] {1157} >(snip) > 28f2800:imap.gmail.com:S-NS:CreateNewLineFromSocket: Subject: ???????? ???? > 00000097 4.71979713 [1012] 2752[1f6ac40]: 28f2800:imap.gmail.com:S-NS:CreateNewLineFromSocket: 18 OK Success (2) FETCH BODY.PEEK[] (viewed as windows-1251) > 00000107 4.77043772 [1012] 2752[1f6ac40]: 28f2800:imap.gmail.com:S-NS:SendData: 19 UID fetch 74 (UID RFC822.SIZE BODY.PEEK[]) >(snip) > 00000160 4.96851540 [1012] 2752[1f6ac40]: 28f2800:imap.gmail.com:S-NS:CreateNewLineFromSocket: Subject: ЃEроткий теЃEЋт > 00000181 4.99838066 [1012] 2752[1f6ac40]: 28f2800:imap.gmail.com:S-NS:CreateNewLineFromSocket: 19 OK Success

(In reply to comment #12) > Who converted to "???????? ????" was Gmail IMAP. > What can IMAP client do? But other clients deal with it somehow... At least Windows Live Mail and The Bat display these messages correctly. Isn't it possible to use body's Content-Type option to select encoding for message list? According to my mailbox it looks that almost all messages which have non-standard encoded headers have same encoding of body and headers.

same problem, think i'll try to take a closer look on it

Bug of mailer which mail sender uses. Subject header should be correctly encoded. So, INVALID. Possible Tb change for torelance with such wrong header via IMAP: As IMAP server usually returns as-is data(Gmail IMAP does do so) to BODY.PEEK[HEADER(no FIELD qualifier), use of BODY.PEEK[HEADER instead of BODY.PEEK[HEADER.FIELDS is probably a workaround in this case, because Tb already has quirks(Bug 468351 for Tb3) for such malformed/wrong header of POP3 mail(raw binary of charset of mail body is written in header). Because download of "Received: header"(which usually shares largets part of mal headers) is required to support "Received column", change from BODY.PEEK[HEADER.FIELDS to BODY.PEEK[HEADER is reasonable solution for both issues, except in slow dial up connetion environment. Following is request by Tb when custom filter for Content-Type, Received is added. > UID fetch 1:4 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Received I believe simpleest BODY.PEEK[HEADER is better if many fileds are required in request.

Will this bug be fixed on the next release?

Note that the bug is also present in Mozilla Firefox 7.0.1, going go your Gmail inbox -> Select problematic email -> droplist menu -> show original. please check as well with your gmail account.

Maybe my comment an attachment on Bug 254519 are more related to this bug. https://bugzilla.mozilla.org/show_bug.cgi?id=254519#c193 https://bugzilla.mozilla.org/attachment.cgi?id=589494

(In reply to Christian Reischl from comment #21) > Maybe my comment an attachment on Bug 254519 are more related to this bug. This is about the subject field and not the from field. They aren't necessary related, especially as this is on more about the server.

Please, guys, fix this bug. And why it's just "enhancement"? Bunch of question marks in the mail list instead of readable text is clearly "normal - some loss of functionality under specific circumstances". It works in GMail webmail, it works in POP3, why can't it work in IMAP?

(In reply to Serg Iv from comment #23) It would qualify "normal" if the program failed to show the standards-complying data. Suppose it's set in a community to communicate using English only. But some person comes and ignores this rule, and uses _broken_ mumbo-jumbo. You _may_ try to learn his tongue, and that would _enhance_ your ability to communicate with that selfish person. But completely ignoring such misbehaviour is just another acceptable option. That said, I want to note that I am also affected by this, and I have voted for this issue. So please don't take it as a call to ignore this problem. But this is clearly just an enhancement (that would colaterally support these non-standard headers).

Well said, Mike. But please do not forget we are talking about _importance_ of the bug. Not about "how much this sh*t broke the standards". And from the point of view of the average (russian for inst.) person this bug is quite important. Not just an enhancement that will not be implemented in a million years. For now I have to move all messages with broken subject to a local folder and delete msf-files afterwards. Not very convenient. BWT I do not understand why it's so hard to fix this bug. Does TB know how to decode the mail subject? Yes it does, 'cos I can see subj in the mail header without any problem. Next step - just show exactly the same line of text in the mail list. That's all.

For me, it seems to "fix" the problem by the following way: 1. Drop the account from TB (I use Icedeove 10.0.11 from Debian). 2. Re-create the account with download of messages turned ON ("Синхронизация и хранение" -> "Хранить сообщения для этой учётной записи на этом компьютере" in localized version) in account's settings. All and only new messages should be now shown correctly in the message list.

(In reply to ap from comment #26) Thank you for the useful tip. However, it is not a fix, it's just a workaround. As TB now stores all mails locally, it is able to see the real bytes of the message and guess its charset. TB could do a better job without requiring to download everything just to be able to see the subject correctly. And if developers don't want to make these modification to save efficiency, they could create a context menu option to "download correct headers" so that users could decide if they need to fix the garbled strings. Or maybe this could be implemented as a plugin?

I've always had the "Keep messages for this account on this computer" option enabled in Daily 22.0a1, and i'm still experiencing this issue with e-mail messages that have subject field without MIME encoding. That is, daily doesn't grok the subject field encoding from the Content-Type header field. What am i doing wrong?