From Bugzilla Helper: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:0.9.2+) Gecko/20010711 BuildID: 2001071104 I upgraded to 2001071104 build and when I click on one of the POP3 account folder, it pops up a dialog showing: "Do you wish to compact all local and offline folders because the total wasted space in all accounts exceeds the purge threshhold?" I answered "Ok" and all the CPU time are grabed by Mozilla. I waited for quite a long time and Mozilla disappear from Windows "Alt-tab (task switching)" windows but remains only in task manager BTW,What is the wasted space and purge thershold? Reproducible: Always Steps to Reproduce: As described in description Actual Results: Mozilla Hang Expected Results: Mozilla completed the compaction using normal CPU time.

reassign to Navin.

During the hang, can you still use the menus, toolbar etc. in the other Mozilla windows?

I can't. The Mozilla lose control at all, cannot refresh the screen, browser windows, mail windows all no response.

Do you have folders configured for offline use ?

POP3 Account, has no "offline" options. IMAP Account has not such problem

I cannot reproduce this problem.

nsEnterprise+ eMojo bugs should be in the .9.4 milestone so I'm tweaking the milestone here. Although this may be a WFM if we can't figure out how to reproduce this in house.

Reporter, Are you still seeing this problem on the recent builds? If so can you tell us more information about your pop3 account? How many folders you have. Do you have filters set up to any of your local folders? How many messages do you have etc. I will try to set up my test account to fairly match yours and see if I can reproduce this problem. Can you also give us your system information too. thanks.

If we can't find a reproduceable test case for this, then we won't be fixing this for the branch. So leaving off the nsBranch+ status right now. Leaving the nsEnterprise nomination alone.

Triaging nsenterprise-.

moving to 0.9.6

rubbish@dr.com, are you still seeing this? If so, could you provide Sheela with the information she asked about? moving to 0.9.7 since it sounds like we won't be able to fix this until we can reproduce it.

Moving out. If we can get a reproduceable case then we can move back in.

I confirm this bug on 2002060308 Note this is a huge compacting as I have deleted a lot of messages Mozilla hangs and I have no control. It appears both when asking for compacting by the menu and when Mozilla proposes "compact folders" after launch. Note: I use Windows XP.

I see this on Linux, however, Mozilla stays responsive although sluggish (possibly because Linux kernel has better resource management?). When using a file manager I enter the disk directory containing the compacted folder's file, I can see that there's a temporary file nstmp there which grows slowly: [olo@tuxia ~/.mozilla/winda/4oali5gr.slt/Mail/Local Folders-1/Admin.sbd 18:39:33]$ll razem 92372 -rw-r--r-- 1 olo olo 52886332 lut 20 17:10 NMAIL Message Notifications -rw-r--r-- 1 olo olo 3573388 lut 20 18:20 NMAIL Message Notifications.msf -rw-r--r-- 1 olo olo 38123978 lut 20 18:39 nstmp -rw-r--r-- 1 olo olo 0 lut 20 17:47 nstmp.msf [olo@tuxia ~/.mozilla/winda/4oali5gr.slt/Mail/Local Folders-1/Admin.sbd 18:39:34]$ll razem 92376 -rw-r--r-- 1 olo olo 52886332 lut 20 17:10 NMAIL Message Notifications -rw-r--r-- 1 olo olo 3573388 lut 20 18:20 NMAIL Message Notifications.msf -rw-r--r-- 1 olo olo 38127466 lut 20 18:39 nstmp -rw-r--r-- 1 olo olo 0 lut 20 17:47 nstmp.msf eventually, the compacting finishes (but after a veeeery long time), the temporary file seems to be renamed to replace the old folder file. So to summarize, Mozilla actually doesn't hang, it's only that compacting is extremely slow (rebuilding speed in my case was roughly 6 KB/sec on a 50 MB folder! it took hours!). Dureing the compacting Mozilla users all available CPU power, so it looks like the compacting algorithm is some kind of bogosort ;)

BTW this is the same folder for which building summary takes long time too, see bug 172786. It seems strange, because when the folder is smaller (below 20 MB) all those operations are relatively fast, but when the folder grows, the execution time of compacting and building summary file increases exponentially!

See this URL for a testcase (tar-gzipped folder that gives me all those headaches, 3.8 MB in size when compressed): http://olo.office.altkom.com.pl/domowa/qa/mozilla/2003_01_24_Large_Mail_Folder_loading/

BTW I've placed more testcases, all are compressed archives containing versions of the big folder I have trouble with. The latest ones will be compressed with bzip2 instead of gzip - now they only take 1.9MB.

I haven't specificly test this bug for recent build but I did remember 1 or 2 months ago, I need to move some mail to a new folder so that I compact the original folder

mass re-assign.

Have you tried this with a 1.5 Mozilla or Thunderbird build? Compact has been sped up a great deal.

actually, this folder seems to have a very large number of messages in the same thread. That makes reparsing and compacting the folder very slow.

When you try 1.5b/final or thunderbird, you should set the following pref: user_pref("mail.thread_without_re", false); either by using about:config, or shutting down mozilla and editing prefs.js by hand. This will cause messages with the same subject but no Re: or references to be put in separate threads, but it will also speed up reparsing and compact enormously.

*** Bug 191890 has been marked as a duplicate of this bug. ***

changing summary - it's not hang; it's just very slow...

This is an ugly but simple fix for a difficult problem. Basically, when we add a message to a thread, we have to run through the thread to see if the new message is a parent of an existing message in the thread, and adjust things accordingly. If you thread by subject, and you have a large folder with messages w/ all the same subject, this code can take a really long time. So the pragmatic thing is to say that for threads with more than 1000 messages, it's simply not worth dealing with the case where the parent comes in after the child. Threads with more than 1000 messages are pretty unwieldy anyway.

Comment on attachment 190755 [details] [diff] [review] proposed fix how about a comment describing why we are skipping threads with more than 1,000 children

Comment on attachment 190755 [details] [diff] [review] proposed fix I've basically added the bugzilla comment to the code.

Comment on attachment 190755 [details] [diff] [review] proposed fix I assume that since this is a -w patch the indentation will be fixed on checkin. A comment referencing this bug would be good also.

added a ref to this bug in comment. The indentation was fine - I just had some whitespace cleanups in that file that I didn't want to complicate the diff with.