Closed Bug 720158 Opened 13 years ago Closed 2 years ago

Thunderbird hang copying many messages over IMAP, while moving many mails from IMAP folder to IMAP folder of different server by Drag&Drop of mails

Categories

(Thunderbird :: Folder and Message Lists, defect)

10 Branch
x86
macOS
defect
Not set
critical

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: hub, Unassigned)

References

(Depends on 4 open bugs)

Details

(Keywords: hang, Whiteboard: [bulkoperations][dupme?])

Attachments

(2 files)

Attached file sample (deleted) —
I was copying 593 messages over IMAP from a remote server to a local (to the network) Courier-IMAP. Thunderbird hang at 225 of 593. Attaching a "sample"
Attached file second sample (deleted) —
this is the second sample a few minutes later.
I use Thunderbird 10 from the beta channel (updated this morning) on MacOS X 10.6.x
The remote server is Dreamhost. (FWIW).
It might have been that the folder hadn't finished loading / synching locally. As when I restarted Thunderbird it was resynching said folder from the remote IMAP.
> Bug summary: Thunderbird hang copying over IMAP What phenomenon do you call by "hang"? When status bar update stopped at "copied 225 of 593", Tb became unresponsive to any action? Or simply Tb stopped copy of 593 mails after copy 225 mails at Tb window(or a tab) where copy is requested?. How did you requeste copy of 593 mails? Copy To? Drag&Drop of mails? When copy stopped at 225 mails, was selection of 593 mails(highlight of mails) cleared? How many mails was actually copied to local mail folder from IMAP folder? Bug 538375 is for "bulk move of many mails from local folder to IMAP folder(opposite to your case) stops at mid of bulk move". In this case, it doesn't look network error happened while moving(uploading to IMAP server). Bug 716461 is for test result by emulation of "IMAP connection error while uploading" by pulling off of LAN cable. "stop copying at mid of bulk copy" is seen in these bug too. Is external symptom of your case similar one to these bugs? (In reply to Hub Figuiere [:hub] from comment #4) > It might have been that the folder hadn't finished loading / synching > locally. As when I restarted Thunderbird it was resynching said folder from > the remote IMAP. IIRC, bulk copy(bulk move too) from IMAP folder to local folder doesn't copy mail data in offline-store file. Bulk copy/move fetches mail data from server and write it to local mail folder. If new mail check, auto-sync, msgPurge etc. of this folder is invoked while copying multiple mails, it may interfere bulk mail copy. Can you get NSPR log for imap, msgcopyservice etc.? (see bug 402793 comment #28) > Win example : SET NSPR_LOG_MODULES=timestamp,imap:5,MsgCopyService:5,IMAPOFFLINE:5 Because many data is logged, get minimized log y next, please. (1-1) stop automatic new mail check of any folder, set offline-use=off of as many folders as possible(if possible) to stop auto-sync. (1-2) Restart Tb with logging enabled, and execute sae "copy 593 mails from IMAP folder to local folder". (1-3) Problem is reproduced? (2-1) If not reproduced, enable automatic new mail check(enabled for Inbox), and enable automatic new mail check of the copy source IMAP folder, se offline-use=on for the copy source IMAP folder. (2-2) Restart Tb with logging enabled, and execute sae "copy 593 mails from IMAP folder to local folder". (2-3) Problem is reproduced? In any case, as seen in Bug 538375, log file becomes huge, if activity on irrelevant IMAP folders, unwanted activities occurs while getting log. Get NSPR log with "copy of small number of mails" for short while first, and check log file content, and reduce logs for irrelevant IMAP folders as many as possible by setting change, please. Even after it, IMAP log is large, because mail data is also logged. imap:4 may be a way to avoid mail data log. Please try it.
@hub does this match to bug 296453 for you? unfortunately a trace without thunderbird/mozilla symbols isn't too helpful
Severity: normal → critical
Component: General → Folder and Message Lists
QA Contact: general → folders-message-lists
Summary: Thunderbird hang copying over IMAP → Thunderbird hang copying many messages over IMAP
Whiteboard: [dupeme]
(In reply to Wayne Mery (:wsmwk) from comment #6) > @hub does this match to bug 296453 for you? FYI. That bug and friends of that bug are for problem in mass delete and mass delete step of bulk move. This bug is for problem during bulk copy. Bug 538375 / Bug 716461 refers to move too, but error occurs in copy step of bulk move if move case in these bug, so bug 296453, bug 538378 is currently irrevant.
(In reply to WADA from comment #5) > > Bug summary: Thunderbird hang copying over IMAP > > What phenomenon do you call by "hang"? > When status bar update stopped at "copied 225 of 593", Tb became > unresponsive to any action? Or simply Tb stopped copy of 593 mails after > copy 225 mails at Tb window(or a tab) where copy is requested?. It was unresponsive. "Force Quit" was the only resolution. > How did you requeste copy of 593 mails? Copy To? Drag&Drop of mails? Select All, drag and drop. > When copy stopped at 225 mails, was selection of 593 mails(highlight of > mails) cleared? > How many mails was actually copied to local mail folder from IMAP folder? Not local mail folder. IMAP folder on the local network.... (the destination server is in my closet) 225 messages (maybe a few more) were copied.
(In reply to Wayne Mery (:wsmwk) from comment #6) > @hub does this match to bug 296453 for you? Not really > unfortunately a trace without thunderbird/mozilla symbols isn't too helpful Unfortunately this is a recurring problem where it is hard to actually get the symbols, etc.
(In reply to Hub Figuiere [:hub] from comment #8) > Not local mail folder. IMAP folder on the local network.... Oh, sorry, I missed "network) Courier-IMAP" in next line after "local". Bulk copy from IMAP folder to IMAP folder of different server case, so something wrong can happen on both "fetch from IMAP server" and "append to different IMAP server", and it can happen at same time. > (In reply to WADA from comment #5) > > > Bug summary: Thunderbird hang copying over IMAP > > What phenomenon do you call by "hang"? > It was unresponsive. "Force Quit" was the only resolution. > Select All, drag and drop. "Bulk move" case. Ctrl+Drag&Drop of mail == Copy, Drag&Drop of mail == Move. (Bug 713814 exists, so Ctrl+Drag&Drop becomes Move, but it's Tb 9 only problem) (Bug 713814 : Ctrl/Control+Drag moves a message instead of copying it) If problem in "mass delete step", I think all 593 mails should be copied to local Courier-IMAP server because append is used(copy to different IMAP server) for each mail, and NNN of "copied NNN of 593" should be near to 593, although it can be problem in "mass delete step" in next case. MULTIAPPEND is used, and something wrong happened in local Courier-IMAP server after server returned OK response to MULTIAPPEND. So I believe problem in "copy step of bulk move" instead of problem in "delete step of bulk move". Questions to surely rule out problem in "mass delete step of bulk move" which is relevant to bug 296453 and his friends. Drag&Drop of mails(move)? Ctrl+Drag&Drop of mails(copy)? MULTIAPPEND is supportd by your Courier-IMAP server and is used by Tb? (check IMAP log, please) When status bar update stopped at "copied 225 of 593" and Tb became unresponsive, was phenomenon like large Virtual and Real memory usage by Tb and/or CPU hog observed? Can similar phenomenon be observed by "Pull off LAN cable while copying"? (emulation of network error while accessing IMAP server, like bug 716461) (1) Copy same 593 mails (execute Copy instead of Move please, to surely rule out delete related issue) (2) When some mails are copied, pull off LAN cable. (3) Check Tb's status if possible. File/Offline/Work Offline. Checked status? Observe status bar, thread pane display. (4) Open new Tb Window if possible(at account's context menu, Open). Check number of mails in copy target folder. (5) Plug in LAN cable again. Tb perhaps goes back to Online mode automatically.
Because "Move from IMAP folder to IMAP folder on different server" and it is executed while any other activity, such as new mail check, new mail notification via IDLE, auto-sync, msgPurge, auto-compact, can occur at same time, many issues may be relevant. Setting dependency to bugs which is related to IMAP/copy/append/bulk mail copy/bulk mail move, for ease of tracking and problem analysis.
Depends on: 716461, 624082, 538375, 693353
Putting mass delete related bug in See Also because "bulk mail move" case, for ease of tracking.
FYI. Checked Tb's behaviour in "copy step of bulk mail move" when offline-use=on with Tb 9.0.1. - Copy source folder : Gmail IMAP folder, offline-use=on, already auto-sync'ed - Copy target folder : Yahoo! IMAP folder, offline-use=on, no mail in it 1. Select 1000 mails at Gmail IMAP folder, 2. Drag&Drop to Yahoo! IMAP folder, 3. When 10 mails are copied, Pull off Lan cable. In IMAP log, there was no log for fetch from Gmail IMAP. Log for append to Yahoo! IMAP folder only. So, as for "Copy step of bulk mail move + Pull off LAN cable while copying", it can be said that there is no difference from "Copy from local mail folder to IMAP folder + Pull off LAN cable while copying"(== Bug 716461), except that Copy source folder is affected by Offline mode due to "Pull off LAN cable" when Gmail IMAP folder case. In above test, selection of mails at "Gmail IMAP folder(move is rquested at here) was cleared after "Pull off LAN cable"(Tb goes Offline mode). This is perhaps a result of "entering Work Offline mode" and is a difference from actual local mail folder case.
Keywords: hang
Quick summary of this bug. This bug is for following case. - Move many mails by Drag&Drop of selected messages - Move between an IMAP folder-A on IMAP server-A to IMAP folder-B on other IMAP server-B - "hang" in bug summary == "unresponsive". "Force Quit" was the only resolution. - The "unresponsive" looks to happen while "copying". i.e. While (a) "fetch from Folder-A on IMAP-A" and (b) "append to Folder-B on IMAP-B". - CPU usage, Memory usage or increase while "unresponsive" is not reported. If Offline-Use=On, and if auto-sync is active, auto-sync may interfere (i) and (ii). But "Offline-Use=On/Off" nor "auto-sync=enabled/disabled" is not provided yet. If MULTIAPPEND is supported by server and if Tb utilized MULTIAPPEND in (ii), it may affect on problem. But "MULTIAPPEND=supported/not-supported" is not provided yet. Because IMAP process is done at main task and IMAP access is synchronous, if N times of ("fetch of many mails from Folder-A of IMAP-A" + "append of many mails to Folder-B of IMAP-B") is executed for an "Move many many mails between an IMAP folder-A on IMAP server-A to IMAP folder-B on other IMAP server-B", "unresponsive for a while" itself is ordinal phenomenon. It's by current design. It can be called as "restriction in current Tb". However, "what happens if wait for long time sufficiently" is not provided yet. If problem depends on "number of mails", data like next is usually needed. Move 10 mails : OK. Took nn_10 seconds Move 100 mails : OK. Took nn_100 seconds Move 1000 mails : OK. Took nn_1000 minutes <= "Time to take move" is O( number of mails )? "Time to take move" is O( number of mails **2 )? <= Something bad started to occur Move 10000 mails : BAD. Killed after MM minute Move 20000 mails : BAD Killed after MM minute This kind of data is not available yet.
Summary: Thunderbird hang copying many messages over IMAP → Thunderbird hang copying many messages over IMAP, while moving many mails from IMAP folder to IMAP folder of different server by Drag&Drop of mails
oh nice, 5 years later still not fixed ^^ i tried to move 900 emails into a different IMAP folder on the same server and firefox freezes instantly. after some time it asks if i want to continue the script or stop it. chrome://messenger/content/folderPane.js:2031 after restarting the dialog asking to stop the script didnt appear and firefox was just frozen.

(In reply to crosbow from comment #17)

oh nice, 5 years later still not fixed ^^

You mean tb, not ff, right? Been testing doing same thing with >7000 emails in folder and don't see what you describe.

Whiteboard: [dupeme] → [bulkoperations][dupme?]

Does this still reproduce for you when using version 102 ?

Flags: needinfo?(hub)
Flags: needinfo?(crosbow)

Problem was solved, sorry. The reason was the transferred emails were copied/migrated over to different servers a lot of times. Due to that reason some mail client in between "appended all mails" together somehow. So file 1 had one email inside, the second file the first one and the second one, the third file all 3 files together. And that went up to 30000.

This faulty files made Thunderbird hang up when trying to move them. The way I fixed it: I exported the mails (even as big as they were) and then imported them into Roundcube to my IMAP server. In the process Roundcube cleaned the files and Thunderbird was running fine again.

Weird process, but at least I got still all my mails from the last 20 years.

Regards,
Dennis

(In reply to kontakt from comment #20)

Problem was solved, sorry. The reason was the transferred emails were copied/migrated over to different servers a lot of times. Due to that reason some mail client in between "appended all mails" together somehow. So file 1 had one email inside, the second file the first one and the second one, the third file all 3 files together. And that went up to 30000.

This faulty files made Thunderbird hang up when trying to move them. The way I fixed it: I exported the mails (even as big as they were) and then imported them into Roundcube to my IMAP server. In the process Roundcube cleaned the files and Thunderbird was running fine again.

Weird process, but at least I got still all my mails from the last 20 years.

Regards,
Dennis

(In my case at least, didnt see that this isnt even my Bug Thread)

I haven't seen this in a long time.

Flags: needinfo?(hub)

Thanks for the updates folks.
Even if it still exists, is adequately covered in other bugs, for example bug 538375 and bug 716461

Status: NEW → RESOLVED
Closed: 2 years ago
Flags: needinfo?(crosbow)
Resolution: --- → WORKSFORME
Depends on: 1828372

Removing dep on bug 1828372. (Not sure why I marked it as dependent.)

No longer depends on: 1828372
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: