User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20060504 Minefield/3.0a1 Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20060504 Minefield/3.0a1 Note: I'm not sure the "large number of folders" -- I have over 200 -- is relevant, but it looks like it. Running Thunderbird, after a large number of POP3 messages have been downloaded and filtered, the "virtual memory" used for thunderbird grows from around 400 to as much as 800Meg. Since I have only a 'paltry' 256Meg of real memory, this makes the system thrash until it is very difficult to get any useful work done. An additional bad effect is that it takes about 10 MINUTES to shut Thunderbird down. Watching the Gnome System Monitor, it appears that a load of virtual pages have to be swapped in to release the many many open files. This also shows that the ".msf" files are the majority of the open files; they get opened and, apparently, never closed. Reproducible: Always Steps to Reproduce: 1. Create a profile with many folders & subfolders, and a lot of filtering into them; 2. Get New Mail with a large number of pending messages; 3. Monitor the memory usage and number of open files. Actual Results: As described in Detail. Expected Results: Some ceiling on memory usage should be applied, and files should be aggressively released when at or above the ceiling. Close the least-recently-used files even if they get opened again a minute later. I reviewed several bugs that "memory footprint" finds, notably 315691, 216535, 335288. They appear to be to specific to match this. This is a long-standing problem. It got slightly better not long ago, then got much worse. Sorry, I can't really pin down the dates. FYI: My system = UBUNTU "Breezy" with 2.6.10 kernel, Gnome 2.12.1; Real memory 256Meg. Fast CPU but it spends all its time in IO-Wait. I'm taking the liberty of calling this "Major" because it makes the program nearly un-useable.

Under W2K Thunderbird memory usage slowing increases over time, memory usage acellerates during folder compaction. Thunderbird usually hangs on exit with almost 100% utilization and has to be manually killed via task manager. I have almost 100 folders on an IMAP server, 16 RSS subscriptions and 15 local folders

I'm running version 1.5.0.8 (20061025) of Thunderbird under Win XP Professional (using IMAP) and often encounter this issue. It's been my experience that Thunderbird ramps up used memory when it is deleting a large batch of emails--say about 100 emails and up, but this is only a rough guesstimate. The number of folders I have is relatively small, about 20, but I do receive a lot of automated emails. I have many of my folders set up to delete messages after so long, and it's usually when Thunderbird goes about deleting them that I run into this issue.

1) Gee, guys, one would think the above two comments constituted confirmation. 2) Things are getting worse, rather than better. I'm running the mail client that was available with Lightning because I cannot use T-bird until Bug#389271 is addressed. Anyway, I quit the program after the download and it went into the "close files" process. This peaked at a virtual footprint of 2.2Gb! Physically, usage peaked at around 188/256Mb.

Some possibilities for you all to try, and then report results in the bug (and cite version being used): - use TB 1.5.0.13 which fixes bug 387403 - turn off GDS (google desktop search) if you installed it - turn off RSS automatic check for new articles (bump to high value) - delete <foldername>.msf files with thunderbird shut down, and restart (some of the above are from bug 346361) if the above don't help, start in _safe mode_, with v2 or executable/rpm from http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk/

Wayne, sorry! This either didn't get mailed to me or it got lost in the ether. CURRENTLY: 3.0a1pre (2008021503) - back on the trunk. (In reply to comment #4) > Some possibilities for you all to try, and then report results in the bug (and > cite version being used): > - use TB 1.5.0.13 which fixes bug 387403 Should be in 3.0a1pre, yes? > - turn off GDS (google desktop search) if you installed it Nope! > - turn off RSS automatic check for new articles (bump to high value) Nope! > - delete <foldername>.msf files with thunderbird shut down, and restart > (some of the above are from bug 346361) Yeah, I'll try that soon. > > if the above don't help, start in _safe mode_, with v2 or executable/rpm from > http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/latest-trunk/ > I am 99+44/100% sure I know the root cause. If I watch the System Monitor list of open files I see that around 120 ".msf" files are kept open until the program quits. The "peak" memory use as the program is trying to exit is apparently coming as the program tries to close all those files and they get mapped back in. I have somewhere around 200 filters and the like number of folders. That may make me a bit of an "edge case," but it's hard to do much with anything less.

Results from comment 5, using current trunk? Can you narrow down what's causing this at the FRONT end? For example does this happen when filters are not used? Is adaptive junk mail processing enabled? moving to backend for now.

Yep, two problems here: 1. We don't close mork databases. 2. Mork stores the entire database in memory. Should I mark this as a dupe of the other "uncontrollable growth in memory usage" bug?

(In reply to comment #7) > Yep, two problems here: > 1. We don't close mork databases. > 2. Mork stores the entire database in memory. > > Should I mark this as a dupe of the other "uncontrollable growth in memory > usage" bug? > As long as the two comments above get written into the other bug, that's fine. Please, which other "uncontrollable growth in memory usage" bug(s)?

Joshua, I think you should do the dupe. (comment 7)

In reply to comment #8) > (In reply to comment #7) > > Yep, two problems here: > > 1. We don't close mork databases. > > 2. Mork stores the entire database in memory. > > > > Should I mark this as a dupe of the other "uncontrollable growth in memory > > usage" bug? > > > As long as the two comments above get written into the other bug, that's fine. > Please, which other "uncontrollable growth in memory usage" bug(s)? I no longer remember numbers of the key bugs. Some but perhaps not all will be in https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=anywordssubstr&short_desc=msf+memory+index+rebuild+clos&product=MailNews+Core&product=Thunderbird&product=Toolkit&resolution=FIXED&chfieldfrom=250d&chfieldto=Now&chfield=resolution&chfieldvalue=fixed&field0-0-0=assigned_to&type0-0-0=anywordssubstr&value0-0-0=kent+bienvenu David A, can you retest with ftp://ftp.mozilla.org/pub/thunderbird/nightly/latest-comm-1.9.1/

(In reply to comment #8) > (In reply to comment #7) > > Yep, two problems here: > > 1. We don't close mork databases. > > 2. Mork stores the entire database in memory. > > > > Should I mark this as a dupe of the other "uncontrollable growth in memory > > usage" bug? > > > As long as the two comments above get written into the other bug, that's fine. > Please, which other "uncontrollable growth in memory usage" bug(s)? I think I had been intending to do bug 185634.

I will retest as asked, it will take a few days as I'm doing stuff I get paid for, like 6 or 7 days out of 7 right now. My normal pattern is to put up each Friday's NIGHTLY. The following rough observations are from that experience over time. The situation gets better, then it gets bad again. Right now, thunderbird-bin is using 608MiB virtual. Since I only have 256 physical, I'm thrashing a whole lot of the time. Yes, the problem is Mork! (Or it sure seems that way). Using the "System-Monitor" application, I can watch the list of open files on this process. In my case, they number in the hundreds. As described in the initial text, this causes big trouble when the program tries to shut down. It seems each file's buffers have to be paged in and flushed in order to close the file. It's time to go brew a pot of tea while waiting to get on with something else. The really irritating thing here (aside from the age of this problem) is that I am, or was when I had a functioning brain, a master of database related programming. So, having a hammer, every problem becomes a nail. I think I could help more constructively here than anywhere else, if I only had a brain.

The plan as I understand it is that database files are in fact supposed to be closed when they are not needed (except for those of type INBOX) - but I, like David Cobb, was able to see large numbers of .msf files being closed when I exit Thunderbird (in my case, 40 .msf files were closed at exit.) There were also some closes of .msf files occurring during normal operation, yet the number still open at the end seems unduly large. I don't experience the out-of-control memory though, but my memory even on my "slow" machine is 1.5 MB. Right now, the closing of the database files is directly related to the destruction of the database object, which only occurs when all references to that object have been removed. But since anyone doing anything needs a reference to the database, it is very easy to hold onto a reference, preventing the database closing from occurring. Hence they stack up at the end. I really think that we need a major change here, that allows the database to be closed independently from the destruction of the database object. That closing could then be managed, say in a timed fashion, to happen reliably. It would also eliminate certain bugs that occur because the database must be closed before certain operations. Whether this will ever happen, I don't know. Not before TB3, and I suspect TB.next will try to switch away from Mork, as Firefox already has.

(In reply to comment #13) > Right now, the closing of the database files is directly related to the > destruction of the database object, which only occurs when all references to > that object have been removed. But since anyone doing anything needs a > reference to the database, it is very easy to hold onto a reference, preventing > the database closing from occurring. Hence they stack up at the end. Another option would be to make folders hold only weak references to database objects. This would imply that only databases that are currently being used are open, assuming that the nsIMsgDBFolderInfo doesn't also play a part in holding the database open.

actually, that won't help - there are various cycles between the db and things it owns that are only cleared when we call nsIMsgFolder::setMsgDatabase(nsnull). I still think the more likely way out is to clean up cached folder db's in the background, on idle.

as I said on IRC, it would be nice to combine this with the concept of inactive db's, ones that haven't been used in a while. It's a little tricky because the db hands out objects like msg hdrs and enumerators, whose use should also indicate that a db is active. But you could get a pretty close approximation by just instrumenting GetMsgHdrForKey/ID and the enumerators' GetNext() methods.

D.Cobb, please test nightly and report results. thanks

Considering "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.3a1pre) Gecko/20091005 Shredder/3.1a1pre - Build ID: 20091005030552" Right this moment, Virtual Memory = 432Mib, Resident = 101Mib. On recent nightlies, I have seen the Virtual peak at >800Mib. (Generally, I download a new nightly once per week, so qualify "recent" that much). I can't provide numbers, but the overall feel is that one week there will be a real improvement -- at least after a period of relative inactivity the memory will drop to something reasonable. The next week that progress gets lost and we're back in the region where T-bird just wants to own my machine. Exacerbating this, the "principle of locality" that underlies paged memory design doesn't seem to apply to T-bird -- thrashing becomes so horrible none of the running processes make progress.

(nobody much cares about the last point these days except compilers, so moving on) It sounds like your biggest occurrences of this problem are "mostly" gone. What types of accounts do you have? pop, imap, rss, news [1]? Do you manipulate 100s of messages at the same time, eg move, junk? [2] Do you have filters that kick messages to several folders? open bugs: [1] bug 185634 Mozilla News consumes unbearable amounts of memory in big newsgroups. [2] bug 369255 Thunderbird is using insane amounts of memory during normal usage (leak in junk filter?) bug 485368 excessive memory usage, CPU loop, possible failure, on bulk message move or delete

Re: Comment #19. I've spent a long career in this (programming) business. Don't expect me to just smile about the fact that nobody much cares about efficiency or the impact on other processes these days!! Also, I've spent a long enough career that I'm now entitled to be old and cranky. What kind of accounts? See the original description!! POP. Filters? Over 200. For Pete's sake, have the courtesy to read the description of the problem before you write something like this. Bug #185634: I don't have any netnews subscriptions active, this is all filtering. Bug #369255: Maybe related, but see the Comments #5 thru #16. We pretty much KNOW the cause of this problem -- we just don't know an easy way to solve it. The long-term solution will happen when we replace all Mork use by mozStorage (?). Bug #485368: Probably related. I suppose the moves caused by filtering might be considered "bulk." Is this "mostly fixed?" I don't think so, because it doesn't STAY fixed. YES, something good* seems to be happening with this build (20091013033821), although it is still a negative impact on my operations -- I don't think my mail program should dominate my machine the way Thunderbird is doing. QUESTION: Would it be possible to insert an NSPR_LOG message when you open or close files? * Something good: at this moment, I've been running about 90 minutes; the "virtual memory" is 348Mib, with a resident working-set of 48Mib. And there is a fair chance that what looks like thrashing has another explanation. It takes place especially during the "Collect mail and filter it" operation.

(In reply to comment #20) > Re: Comment #19. > > I've spent a long career in this (programming) business. Don't expect me to > just smile about the fact that nobody much cares about efficiency or the impact > on other processes these days!! I understand your frustration. Hope we're not "shooting the messenger". Given your experience you know that performance issues generally take a back seat to such things as features and crashes. :) Including for example effects at the low end scale of the installed memory. Plus you're in a long line of bugs some of can be termed "more important" [1]. That said, despite the passage of years and the fact that there are still too few people working on too many bugs (help is always appreciated), I hope you are pleased that there is recent progress in bugs in areas directly related to yours have been getting attention by both volunteers and devs as seen here, and more clearly in the query I provided in comment 10. > What kind of accounts? See the original description!! POP. Filters? > Over 200. For Pete's sake, have the courtesy to read the description of > the problem before you write something like this. My question "What types of accounts do you have? pop, imap, rss, news" is entirely relevant and was not for lack of reading (which is your assumption, not fact). It is smart to narrow the scope and be *totally* clear that none of the bugs mentioned are remotely involved. It's important because multiple causes of memory issues have been found, and db closure is affected in multiple spots in the code. Many examples have been fixed since your previous documented test results of 2008-02-20, even since 2009-06-25, but also more have been found. gloda indexing beginning with beta 4 may complicate the memory picture. You can eliminate that complicating factor by disabling indexing in advanced options. > Bug #369255: Maybe related, but see the Comments #5 thru #16. We pretty much > KNOW the cause of this problem -- we just don't know an easy way to solve it. > The long-term solution will happen when we replace all Mork use by mozStorage > (?). True, if/when it happens. But who would want to count on that happening within the next two years? > Bug #485368: Probably related. I suppose the moves caused by filtering might be > considered "bulk." filtering in this case <> bulk > Is this "mostly fixed?" I don't think so, because it doesn't STAY fixed. A poor choice of words on my part. I mean "mostly fixed" in the sense that your comments seem to indicate the biggest case of memory usage has reduced or is gone. > * Something good: at this moment, I've been running about 90 minutes; the > "virtual memory" is 348Mib, with a resident working-set of 48Mib. And there is > a fair chance that what looks like thrashing has another explanation. It takes > place especially during the "Collect mail and filter it" operation. >"I have somewhere around 200 filters and the like number of folders. That may make me a bit of an "edge case," but it's hard to do much with anything less." Do you filter to most of those 200 folders? Thanks for the info. Odds still are great that this is a dupe of a bug or idea already in play [2]. And bienvenu and others are working on many of these. With your help, and perhaps through further organization of performance related bugs, we may find a match. Either way, we'll move forward. [1] * perf bugs 140+ -- https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&product=MailNews+Core&product=Thunderbird&resolution=---&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&chfieldto=Now&field1-0-0=short_desc&type1-0-0=nowordssubstr&value1-0-0=crash+assert&field1-1-0=component&type1-1-0=nowordssubstr&value1-1-0=build+install+ldap&field2-0-0=short_desc&type2-0-0=anywordssubstr&value2-0-0=msf+memory+rebuild+slow&field2-0-1=keywords&type2-0-1=substring&value2-0-1=perf * critical + major bugs (not including unconfirmed) 636 -- https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&product=MailNews+Core&product=Thunderbird&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&bug_severity=blocker&bug_severity=critical&bug_severity=major&chfieldto=Now [2] * a not too refined query of perf/memory issues - 39 bugs - https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=anywords&short_desc=msf+memory+db&product=MailNews+Core&product=Thunderbird&resolution=---&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&chfieldfrom=250d&chfieldto=Now&field1-0-0=short_desc&type1-0-0=notsubstring&value1-0-0=new+message&field1-1-0=short_desc&type1-1-0=nowordssubstr&value1-1-0=crash+assert&field1-2-0=keywords&type1-2-0=nowordssubstr&value1-2-0=dataloss * slightly more refined - 19 bugs - https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=anywords&short_desc=msf+memory+db+cach&long_desc_type=anywords&long_desc=cache+db+memory&product=MailNews+Core&product=Thunderbird&resolution=---&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&chfieldfrom=250d&chfieldto=Now&field1-0-0=short_desc&type1-0-0=notsubstring&value1-0-0=new+message&field1-1-0=short_desc&type1-1-0=nowordssubstr&value1-1-0=crash+assert&field1-2-0=keywords&type1-2-0=nowordssubstr&value1-2-0=dataloss

(In reply to comment #21) Wayne: For today, please accept my apologies for being a touch intemperate. As I suggested, old and cranky. :-D. I'll respond more fully after I go off and earn some money. [tomorrow, I hope] Also, HOW CAN I HELP MORE EFFECTIVELY. I would be happy to build instrumented versions, but I have yet to get a clean build. I get gcc errors that you don't. I can, however, collect logs and the like, if that is helpful.

let's revisit this point ... can you reproduce in safe mode? https://support.mozilla.com/en-US/kb/Safe+Mode

David, please see https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems#Diagnosis: and post your results. This will be very helpful

Sorry for being so slow to respond. I will get to this this week [he said, hopefully]. One thing I can say immediately is that I'm keeping gloda disabled at present -- I hardly ever do full-text searching, and I know it's a memory problem. Unsurprisingly, I've had some small success ameliorating this by combing through any folder with over 2000 messages, sorting threads into dated sub-folders, much like "Archives."

I have a log, but it is 30MiB so I cannot upload it to bugzilla. Is there somewhere I can put it? Also, perhaps someone can suggest a setting for NSPR_LOG_MODULES that will focus on this particular issue? This log is "all:7,timestamp". I ran in -safe-mode. The virtual memory went up to about 480MiB, during the filtering of new mail; physical memory got to 150MiB. I especially note the "mmm open DB's" in the log. I see some DB's being closed, so the number will remain unchanged for two or three reports. But then it goes up again. At a maximum of 140, it remains pretty constant.

(In reply to comment #26) > I especially note the "mmm open DB's" in the log. > At a maximum of 140, it remains pretty constant. Problem of bug 540214? Can you check log with next parameter? ("mmm open DB's" is logged) > NSPR_LOG_MODULES=timestamp,MSGDB:5,imap:5

Shows growing number of open DB's. Also, some DB's open which represent empty folders used to organize a hierarchy I find useful.

hmm, 131 open dbs. now that we have log, no need to speculate about dupme until it's analyzed.

David A. Cobb, was the log obtained with -safe-mode? If no, is there any difference between "-safe-mode" and no "-safe-mode"? Every 1 to several seconds, NNN of "NNN open dbs" increases by 1. What action inhibits close of MsgDB of local mail folder? auto-compact? message-purge by retention policy? RSS feed access? "during the filtering of new mail" is key factor? What is particurality of David A Cobb's environment? Why problem continues since 2006-05-13 till now in his environment? Interpretation of digits displayed as "real memory size used by Tb" and "virtual memory size used by Tb" is proper?

David, can you screen shot taskmgr in safe mode? (In reply to comment #30) > What is particurality of David A Cobb's environment? > Why problem continues since 2006-05-13 till now in his environment? indeed these two questions make the bug report particularly interesting, especially if the ORIGINAL reason of the comment 0 still exists. Unfortunately with the long passage of time and no detailed analysis of the original issue, I'm not sure we can really know that what we see now is the original issue. Note: of the 3 bugs cited in comment 0, one is WFM, and two were duped to bugs that are fixed, namely Bug 525646 (in v3) and Bug 266679 (in v2)

Replying to Comment#30: The log was made in safe-mode, and I haven't observed any improvement in the memory load with safe-mode. This really doesn't seem to be extension related. Auto-compact is on, but it asks me whether this is a good time, and didn't do that during this run. I don't have any RSS feeds. I still suspect Comment#7 points to the right problem. What's peculiar about my environment is probably the sheer number of folders and filters. The best way I can imagine to get the "right" number is by attaching my "msgFilterRules.dat". For anyone else to test this will require a script to create all that folder structure -- I haven't written that yet, my first thought would be "awk," but only *-nix systems are likely to have an awk. Would Python be better? Personal to Wayne: thanks, man. Sometimes I miss me too. Back later.

In all their dubious glory. I hope they're not too big.

honestly, 200 folders is small change. I'm not sure how to test comment 7 theory, but perhaps 1. rename msgFilterRules.dat 2. start thunderbird 3. click a few of the largest folders and see how much memory changes.

p.s. you might test with 3.1 instead of v3.0, so that you have all the latest patches - for example this rather narrow, special patch recently landed Bug 549794 Too many .msf file open v3.1 beta ftp://ftp.mozilla.org/pub/thunderbird/nightly/latest-comm-1.9.2/

David, What is your check for new messages intervals set for with RSS, pop and imap? Any results yet with version 3.1? (comment 35)

David, also check if mail.check_all_imap_folders_for_new config option is set to true

At the moment, running: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.3a5pre) Gecko/20100426 Shredder/3.2a1pre - Build ID: 20100426030633 Re Comment#37: I don't have any imap folders. Nor any prefs starting "mail.check" Re Comment@36: Only incoming mail is via Unix-Movemail, checked on start and every 79 minutes thereafter. No RSS. Wayne and David, Bug#549794 looks like it depends on some subscription state(?) it doesn't look like it's applicable (see the previous sentence). ON THE POSITIVE SIDE, there seems to have been some major improvement since the 2010-04-22 nightly. At one time, I left the machine alone and went out for a few hours. When I checked, Thunderbird was using 430MiB _virtual_, but only 24MiB resident. That's the smallest footprint I can recall ever seeing.

so you are pop-only. thanks for the update please close the bug WORKSFORME if after a few days your memory symptoms don't appear. Bonus points if you do a regression search to find when the problem to resolved.

(In reply to comment #38) > ... > ON THE POSITIVE SIDE, there seems to have been some major improvement since the > 2010-04-22 nightly. At one time, I left the machine alone and went out for a > few hours. When I checked, Thunderbird was using 430MiB _virtual_, but only > 24MiB resident. That's the smallest footprint I can recall ever seeing. taking this off the radar per comment 38. If you still see the problem or have identified a narrow date range of when the problem stopped, please comment.