User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/67.0.3396.99 Chrome/67.0.3396.99 Safari/537.36 Steps to reproduce: I want to migrate an IMAP folder to another IMAP server. Client A Thunderbird stands on the same mailbox e.g. "Box = INBOX / DelTest" Client B, the migration tool reads the first mail, writes it to the other server and then flags the mail as deleted. Then with the next messages the same, until all messages are copied and all are marked as deleted. When thunderbird is set to "When deleting a message" -> "Move to this folder" (no folder selected) "Trash" is used. Every time the migration tool marks a mail as deleted, thunderbird will stop displaying that mail. When the migration tool is done, Thunderbird makes an "expunge". That does not happen if - Thunderbird is set as: "When deleting a message" -> "Mark as deleted". or - The migration tool makes a break of about 600 msec after each "delete". - Thunderbird is closed or displays another IMAP folder. The Migration tool is my own program. Therefore, I know what it does and could also insert this break. I used Dovecot's debugging options to see what happens. Whatever is not practical, the Thunderbird displays messages marked as deleted, not simply crossed out, if it is not configured to "Mark as deleted". That has nothing to do with how he should behave himself when deleting a message! I spent a day to get to these results and I really hope that it helps improve thunderbird. Actual results: Thunderbird makes an "expunge". All mails are irretrievably lost. Expected results: Nothing

(In reply to michel from comment #0) > Steps to reproduce: Have some questions... > > I want to migrate an IMAP folder to another IMAP server. > Client A Thunderbird stands on the same mailbox e.g. "Box = INBOX / DelTest" Which server is tb connected to here. Old, new or both?. Also, don't know what "stands on" means. I assume it means you are selected on Inbox's subfolder called DelTest. > Client B, the migration tool reads the first mail, writes it to the other > server and then flags the mail as deleted. Then with the next messages the > same, until all messages are copied and all are marked as deleted. What is tb doing during this process. Is it selected on the folder where messages are getting deleted or or where they are being put? > > When thunderbird is set to "When deleting a message" -> "Move to this > folder" (no folder selected) "Trash" is used. > Every time the migration tool marks a mail as deleted, thunderbird will stop > displaying that mail. Stop displaying I assume here means tb is selected on the folder that is having emails deleted and as they delete they are removed from the list of emails in tb? > When the migration tool is done, Thunderbird makes an "expunge". > How do you know tb is sending an IMAP expunge? Wireshark, tcpdump, tb imap logging(*)? I could be wrong, but I don't think tb ever sends imap expunge command unless the user does the right-click option "compact folder". (*)Ok, I just saw below about dovecot debugging option that I am not familiar with. Does it indicated tb is sending expunge? > That does not happen if > - Thunderbird is set as: "When deleting a message" -> "Mark as deleted". > or > - The migration tool makes a break of about 600 msec after each "delete". So tb is "watching" the folder and as messages get deleted by the tool client, tb is doing expunges. But if the tool works slowly tb doesn't do expunges? Is IDLE enabled? IDLE is enabled via the "allow immediate server notifications when message arrives" on the server settings page. IDLE will also cause messages marked as \DELETED by another client to be removed from tb's list but it does cause an expunge to be sent by tb. However, if the tool is doing the expunge, IDLE will report back to tb that the message was expunged. But it sounds like you are sure the tool is not expunging, it's just marking each message with \DELETED flag. > > - Thunderbird is closed or displays another IMAP folder. So if tb is out of the picture, nothing gets expunged. You mention the other delete options. One marks the message \deleted and crosses it out. The other just marks it as \deleted and it disappears from the list but, of course, is still in the folder. There is a pref in tb Config Editor that allows expunge on delete that is false by default. However, tb is not doing the delete but the tool is doing the deleted so don't think that is it. Also, not sure if dovecot has an "expunge on delete option" that might be set and is causing the problem. Anyhow, I will need an IMAP log that shows tb's reaction to the tool deleting messages. If you could provide that it would help the most. Google for "thunderbird logging" for instructions and attach the IMAP log at the "attach file" link above. Post back here if you have or need more info. Thanks.

Sorry, typo in previous comment, fixed here (does -> DOES NOT): >Is IDLE enabled? IDLE is enabled via the "allow immediate server notifications when message arrives" on the server settings page. IDLE will also cause messages marked as \DELETED by another client to be removed from tb's list but it DOES NOT cause an expunge to be sent by tb. However, if the tool is doing the expunge, IDLE will report back to tb that the message was expunged. But it sounds like you are sure the tool is not expunging, it's just marking each message with \DELETED flag.

(In reply to gene smith from comment #1) > (In reply to michel from comment #0) > > Steps to reproduce: > > Have some questions... > > > > > I want to migrate an IMAP folder to another IMAP server. > > Client A Thunderbird stands on the same mailbox e.g. "Box = INBOX / DelTest" > > Which server is tb connected to here. Old, new or both?. Also, don't know > what "stands on" means. I assume it means you are selected on Inbox's > subfolder called DelTest. > > > Client B, the migration tool reads the first mail, writes it to the other > > server and then flags the mail as deleted. Then with the next messages the > > same, until all messages are copied and all are marked as deleted. > > What is tb doing during this process. Is it selected on the folder where > messages are getting deleted or or where they are being put? Yes ,Thunderbird shows the folder where the messages are marked as deleted. Thunderbird is configured to move deleted mail to the Recycle Bin. Since Thunderbird does not show any more mails marked as deleted in this case, the mails marked as deleted will disappear from this folder. > > > > > When thunderbird is set to "When deleting a message" -> "Move to this > > folder" (no folder selected) "Trash" is used. > > Every time the migration tool marks a mail as deleted, thunderbird will stop > > displaying that mail. > > Stop displaying I assume here means tb is selected on the folder that is > having emails deleted and as they delete they are removed from the list of > emails in tb? > > > When the migration tool is done, Thunderbird makes an "expunge". > > > > How do you know tb is sending an IMAP expunge? Wireshark, tcpdump, tb imap > logging(*)? I could be wrong, but I don't think tb ever sends imap expunge > command unless the user does the right-click option "compact folder". (*)Ok, > I just saw below about dovecot debugging option that I am not familiar with. > Does it indicated tb is sending expunge? > > > That does not happen if > > - Thunderbird is set as: "When deleting a message" -> "Mark as deleted". > > or > > - The migration tool makes a break of about 600 msec after each "delete". > > So tb is "watching" the folder and as messages get deleted by the tool > client, tb is doing expunges. But if the tool works slowly tb doesn't do > expunges? Yes! > > Is IDLE enabled? IDLE is enabled via the "allow immediate server > notifications when message arrives" on the server settings page. IDLE will > also cause messages marked as \DELETED by another client to be removed from > tb's list but it does cause an expunge to be sent by tb. However, if the > tool is doing the expunge, IDLE will report back to tb that the message was > expunged. But it sounds like you are sure the tool is not expunging, it's > just marking each message with \DELETED flag. > IDLE is enabled in Thunderbird ! > > > > - Thunderbird is closed or displays another IMAP folder. > > So if tb is out of the picture, nothing gets expunged. > > You mention the other delete options. One marks the message \deleted and > crosses it out. The other just marks it as \deleted and it disappears from > the list but, of course, is still in the folder. > > There is a pref in tb Config Editor that allows expunge on delete that is > false by default. However, tb is not doing the delete but the tool is doing > the deleted so don't think that is it. Also, not sure if dovecot has an > "expunge on delete option" that might be set and is causing the problem. > > Anyhow, I will need an IMAP log that shows tb's reaction to the tool > deleting messages. If you could provide that it would help the most. Google > for "thunderbird logging" for instructions and attach the IMAP log at the > "attach file" link above. > > Post back here if you have or need more info. Thanks. I created the Thunderbird logfile under linux. A search with grep , resulted in 2092 "EXPUNGE" commands at the end of the logfile. grep EXPUNGE /tmp/imap.log | wc -l 2092 I upload the logfile as ZIP.

Thanks for the log. Do you have "Clean up (Expunge) Inbox on exit" set by any chance? Not sure if this affects also Inbox subfolders like your BulkTest but I will check. The end of the log looks like tb is shutting down. Yes, tb is definitely commanding an expunge on Inbox/BulkTest.

Search for expunge with Config Editor and make sure the 4 items that appear are all default (not bold). This provides the possibility to auto-expunge when you have >= 20 message (or other threshold you set) with \deleted flag set.

It appears there is a surprising (to me) "feature/bug" that causes a folder expunge whenever the number of \deleted messages exceeds or equals mail.imap.expunge_threshold_number when mail.imap.expunge_option is 0 and you are not "showing" deleted messages. A workaround is to set mail.imap.expunge_threshold_number larger than how many messages you ever expect in the largest folder, e.g., 10000000. See https://dxr.mozilla.org/comm-central/rev/2a29ee0adb310b54a6a2df72034953fed8f2b043/comm/mailnews/imap/src/nsImapProtocol.cpp#4190 A fix might be to allow mail.imap.expunge_threshold_number = 0 to completely disable auto-expunge. When you do "mark as deleted" the emails are shown so no auto-expunge occurs. Not sure why "slow" deletes by your helper client prevents expunge but I suspect that numDeleted maybe never reaches the threshold. Try setting the threshold to a huge value and see if that helps.

Another problem is that when tb shuts down, the selected folders are sent the imap close and then logout. Imap close also does an expunge on the selected folder. You can see in your log near the bottom where close and then logout occur. I don't understand the purpose of the imap close command. The only function is has is to silently expunge any messages flagged with \deleted and set the connection state back from selected to authenticated, per the RFC. Tb seems to only do imap close when "move deleted messages to trash" is used. Therefore, even if the threshold in comment 7 is increased, when you shutdown and startup tb I'm afraid you will still see that your messages are expunged. I think the only solution to this is to never do imap close ...??? Note: imap close is not the same as "closing" the tcp connection to the server. That occurs after imap logout.