Open Bug 750630 Opened 13 years ago Updated 4 years ago

TB 12: Message Body not loaded when using "Fetch Headers Only", if message is moved/copied to a folder by message filter upon download. If "after classification" is requested in message filter rule for "Move to folder", problem doesn't occur.

Categories

(MailNews Core :: Networking: POP, defect)

defect
Not set
critical

Tracking

(Not tracked)

People

(Reporter: standard8, Unassigned)

References

Details

(Keywords: dataloss, regression, Whiteboard: [Workaround = "Apply Filter when: Checking Mail(after classification)])

A fix for this was attempted in bug 748865, but the reporter stated that it didn't fix the issue. +++ This bug was initially created as a clone of Bug #748865 +++ User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:12.0) Gecko/20100101 Firefox/12.0 Build ID: 20120420145725 Steps to reproduce: After installing and openingThunderbird 12 I downloaded new mails from the server via POP3. I'm using the option "Fetch Headers Only" and some Filter Rules with sub-folders in the Inbox. Messages in sub-folders of the Inbox doesn't load by clicking "Here" to download fully. Instead there is a blank page and the Header with buttons is also disappeared Actual results: After clicking "Here" to download full message the panel window (not double-clicked to an e-mail) gets completely blank. If I double click a mail for extra window, every click on "Here" for download the full message Thunderbird loads instead the Header of the opened mail , so that there are twice or more copies of the same mail, but also with blank window. This is reproducible for every sub-folder: * Create a sub-folder in inbox. * Create a filter to do the mail in the sub-folder by income or manual filter start. * Load a message with "fetch Headers only" on. * Message now should be in a sub-folder, if not use the filter manual. * Try to download the full message by clicking "Here" to download * Message shouldn't be download and instead there is a blank window. No Problem when mail is in Inbox-Folder or put manually in a sub-folder Problem disapear by unchecking "Fetch Headers Only". (Message loads correctly. Additional information: popstate.dat marked not correctly downloaded mails with blank message window with a "k". The sub-folder document (not the .msf-file) not showing message text. the sub-folder .msf either not showing message text. Expected results: Normally clicking "Here" to download the full message downloads the full message.
trashmailings: when you're opening the message, are you using the preview window, a tab or the separate window to view the message?
normally i'm using the preview window. Here the message is only blank if i want to load the body. But for testing purpose i used also tab. In this case the message is blank AND the Header of in tab opened message was duplicated. Last Option - new window - i have not tested yet. If wished i upgrade from 11.0.1 to 12.0.1 for testing.
RETR command is normally issued by Tb when "Click Here" is clicked, even when this bug occurs. If a mail remained in Inbox after message filter execution, and if the mail is manually copied to other local mail folder(including folder/subfolder of other account) after download, I couldn't observe this bug. I could observe this bug in Tb 12.0.1 only on "mail copied or moved by message filter upon mail download from POP3 server". [Steps to reproduce] - POP3 account Leave Messages on Server(never delete from server) Fetch Headers Only mail-1 : not moved by message filter upon mail download mail-2 to mail-N : copied to Folder1, moved to Folder2 by message filter - Delete popstate.dat, Restart Tb, Get Msgs - Manually copy mail-1 in Inbox to Folder1, Folder2 Quick observation with Tb 12.0.1 on Win-XP. No problem on mail-1 in Inbox and Folder1, Folder2. Problem occurs on mail-2 to mail-N in Folder1 and Folder2. Mail data in mail folder file(Folder1 instead of Folder1.msf if Folder1) is not updated when "Click Here" is clicked and RETR is issued. X-Mozilla-Status: of header only version is not updated, and nothing is added to mail folder file for RETR'ed mail data, even though mail's status in .msf file is updated. Blank mail-2 to N disappers if Compact Folder is executed after "Click Here".
Problem still occurs with Thunderbird 13.0: If I try to download the full mail whose Header is fetched in a sub-folder, the mail gets blank. If I use "reapair .msf" i get back the dialog to download the full message. So I think Thunderbird doesn't write in the .msf folder or something like that. But maybe my message folder is corrupted in some way, but why does Thunderbird till Version 11 never had have any problems?
Same with WinXP in TB 12, 13.
Summary: TB 12: Message Body not loaded when using "Fetch Headers Only" if message is in a sub-folder → TB 12: Message Body not loaded when using "Fetch Headers Only", if message is moved/copied to a folder by message filter upon download
Assignee: dbienvenu → nobody
I have not been able to duplicate this (Thunderbird 17 beta on Windows 7)
Same issue with TB v.17.0.6 in OsX Snw Leopard (but the same in some ubuntu machine)
When "filter move upon download from POP3 server", Tb uses "Highest MsgKey value assigned by Mork" as messageKey instead of traditional messageKey=Offset in file". This produces status of messageKey!=messaeOffset. This is always possible after "pluggable message store" support which was landed on Tb 12, because the "pluggable message store" support cleanly isolated messageKey and messageOffset. "Click Here" code perhaps asumes messageKey==messageOffset. In contrast to above, other features, such as ordinal copy/move of mail, copy/move by manual filter run, Compact of local mail folder, still assigns messageKey==messageOffset. messageKey value of a mail is shown in "Order Received" column. Upon filter move of new mail, Tb assigns messageKey like next. When file size=N, mail_1 = first downloaded mail in one connection (mail size = SZ_1) messegeKey = N messageOffset = N mail_2 = second downloaded mail in one connection (mail size = SZ_2) messegeKey = N + 1 messageOffset = N + SZ_1 | mail_M = M-th downloaded mail in one connection (mail size = SZ_M) messegeKey = N + (M-1) messageOffset = N + SZ_1 + ... + SZ_(M-1) So, when mails are sorted by "Order Received" column, consecutive Order Received column value can be observed, if multiple mails are downloaded and moved by single download from POP3 server. To bug opener and problem reporters: (Q1) Do you see such "consecutive Order Received column value" in filter move target folder? (Q2) If yes, (messaeKey!=messageOffset occurs) does this bug occur after Compact? Note: Compact requests "actually deleted mails". Do "move some mails to other folder, then move back if required" before execute Compact, please. (Q3) If no, (No messaeKey!=messageOffset. all is messaeKey==messageOffset) can you reproduce this bug?
FYI. Following is link location of "Click Here" of "fetch header only" mail. > mailbox://userName@hostname/MoveTargetFolder?number=123456&messageid=007MB_MAIL.000001.000001&uidl=0000052b4c354af7 MoveTargetFolder part is mail folder name number= is messageKey value Phenomenon looks for me; Because mailbox://userName@hostname and UIDL is correct, RETR is issued correctly. If number==messaageKey==messageOffset, Tb can find "fetch header only" mail in the mail folder, and retrieved mail data is appended to mail folder file, and partial mail is normally deleted. However, if number==messaageKey!=messageOffset, Tb perhaps fails to find "fetch header only" mail in the mail folder, and nothing is appened to mail folder file, then MsgDBHdr of phantom mail is generated in the mail folder.
Correction of procedure. If you already experienced this bug in a local mail folder, garbled msgDBHdr may be already generated. Compact in such stuation is slightly dangerous. "Repair Folder" of Folder Properties/General also assigns messageKey==messageOffset, and garbled msgDBHdr such as phantom mail(no actual mail data in folder file) is cleared by "Repair Folder". Do "Repair Folder" instead of "Compact" before start of your test, please.
Problem was not "messagekey!=messageOffset" issue. This bug's problem was still observed even after forcing messageKey==messageOffset by Repair Folder, Compact, "delete .msf" etc. Funny phenomenon was observed. (1) 4 "fetch header only" mails. mail-11, mail-12 is kept in Inbox(not moved by message filter). mail-21, mail-22 is moved to FolderX by message filter. In FolderX, messageKey!=messageOffset. Note: If "Click Here" of mail-21, mail-22 in FolderX is executed, this bug occurs, and RETR is issued, nothing is appened to FolderX file, phantom msDBHdr is created with mesageKey==previousKey+1, instead of messageKey==messageOffset. (2) Copy mail-11, mail-12 in Inbox to TestFolder. Copy mail-21, mail-22 in FolderX to TestFolder. By copy, messageKey==messageOffset for all mails. (3) At TestFolder, "Click Here" of mail-11, mail-12(from Inbox). Full mail is downloadd to TestFolder with messageKey==messageOffset, (call mail-11-Full, mail-12-Full) and old partial mail is removed from Thread Pane. (4) At TestFolder, "Click Here" of mail-21, mail-22(from FolderX). Full mail is downloadd to FolderX!!! instead of TestFolder, with messageKey!=messageOffset, (call mail-21-Full, mail-22-Full) and old partial mail is removed from Thread Pane of TestFolder. (5) At TestFolder, "Repair Folder" => Following mails are shown mail-11 from Inbox (old partial mail is not deleted) mail-12 from Inbox (old partial mail is not deleted) mail-21 from FolderX (old partial mail is not deleted) mail-22 from FolderX (old partial mail is not deleted) mail-11-Full mail-12-Full Why mail-21/mail-22 is downloaded to FolderX instead of TestFolder? Where is the original folder name of "FolderX" kept? "Undo Move or Delete" like one is internally invoked?
Who moved mail-21 and mail-22 to FolderX was "message filter". When partial mail which is moved to FolderX by message filter and manually copied to TestFolder, following looks to occur. (i) At TestFolder, Click Here => downloaded to Inbox (ii) Because newly downloaded mail, message filter("Getting New Mail") is applied. (iii) Message filter has rule for this mail, so mail is moved to FolderX. If "Click Here" of partial mail is executed at "move target folder", following looks to occur. (a) At FolderX, Click Here => downloaded to Inbox (b) Because newly downloaded mail, message filter("Getting New Mail") is applied. (c) Message filter has rule for this mail, so moves to FolderX. (d) Contention between "delete of old partial mail by (a)" and "move mail from Inbox to FolderX by (c)" occurs. Questions. Why mail-21 and mail-22 is downloaded to Inbox instead of TestFolder in above (i) and (a)? Why message filter for "Getting New Mail" is applied on "download of full mail data for partial mail"? When "partially downloaded mail" is held in move target folder of message filter, "move from Inbox to target folder by message filter" fails?
"Downloaded to Inbox" was not mail-21/mail-22 only phenomenon. It was "Click Here always downloads to Inbox" and "message filter is always applied". Phenomenon was; 1. "Click Here" downloads full mail to Inbox. This is same as ordinal "Getting New Mail". => Message filter of "Getting New Mail" is applied to full mail. 2. If message filter doesn't have "Move to folder" for the mail, the downloaded mail is moved to messae folder wehere partial mail was held(TestFolder in above test) by "Click Here". 3. If message filter has "Move to folder" action for the mail, the downloaded mail is moved to "move target folder of message filter"(FolderX) by message filter. So, "Click Here" fails to move mail from Inbox to TestFolder where partial mail was held(TestFolder) => Phantom mail(msDBHdr only, no actual mail data in folder file) is generatd. 4. In both cases, msgDBHdr of old partial mail is normally removed. However, EXPUNGE bit in X-Mozilla-Status: header is not physically written. => Partial mail reappears by "Repair Folder". This bug was worked around by setting "Apply Filter when: Checking Mail(after classification)" to message filter rule for "Move to folder". i.e. Essentially same problem as bug 762318.
Whiteboard: [Workaround = "Apply Filter when: Checking Mail(after classification)]
OS: Mac OS X → All
Hardware: x86_64 → All
FYI. "EXPUNGE bit in X-Mozilla-Status: header is not physically written" part is possibly Bug 840418.
Summary: TB 12: Message Body not loaded when using "Fetch Headers Only", if message is moved/copied to a folder by message filter upon download → TB 12: Message Body not loaded when using "Fetch Headers Only", if message is moved/copied to a folder by message filter upon download, and if "after classification" is not requested in message filter rule for "Move to folder"
Following is perhaps reason why "Apply Filter when: Checking Mail(after classification) of message filter rule for Move to FolderX" can be a workaround of this bug. (1-1) Junk filter call for full mail in Inbox takes a while, (1-2) so "Filter rule application then Move to FolderX" is postponed. (2) Before "filter move" is actually invoked on the full mail in Inbox, "Move full mail from Inbox to FolderX where partial mail is held" normally completes. (3) Because full mail in Inbox is already deleted from Inbox, filter rule is not applied to the full mail, or "Move full mail in Inbox to FolderX by filter rule" silently fails. So, if Junk filter considers partial mail as Non-Junk but considers full mail as Junk, and if Junk move is enabled, workaround of "Apply Filter when: Checking Mail(after classification) of message filter rule for Move to FolderX" may produce new and different problem.
Summary: TB 12: Message Body not loaded when using "Fetch Headers Only", if message is moved/copied to a folder by message filter upon download, and if "after classification" is not requested in message filter rule for "Move to folder" → TB 12: Message Body not loaded when using "Fetch Headers Only", if message is moved/copied to a folder by message filter upon download. If "after classification" is requested in message filter rule for "Move to folder", problem doesn't occur.
Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.
I've got this bug with the 60.3.0 update today. I never had this issue with previous versions.

I had this issue in past and avoided it, but for another issue I just now checked it again and I can confirm, that it is still present in 60.7.0 32Bit on Windows 7.

  • only headers are downloaded from POP3 account
  • message is moved to another folder by filters
  • in this folder click the link to download the rest of the message

What happens:

  • by selecting the message in the list it is marked as read
  • when the download has finished, the message is marked unread again
  • the message body is empty, no content, no link to download the rest, only a white background
  • pressing CTRL-U also shows nothing, only an empty source window
  • before the download CTRL-U correctly showed the message header.

After a folder repair the message in question has the download the rest link again.
Moving the header only message back to the inbox will download the message and the filter moves it back to the destination folder again.
Then the message is there with its whole content.

You need to log in before you can comment on or make changes to this bug.