I've instrumented it with DMD (https://wiki.mozilla.org/Performance/MemShrink/DMD#Everything_other_than_B2G-device_builds) with the help of florian. Here are two logs of a Thunderbird session with DMD enabled.

what is curious to me, is when did this start? because in the last two years we haven't had clustered reports of high memory usage except for the mime parsing incident and maybe one other thing - roughly Bug 810637 and Bug 784286.

(In reply to Wayne Mery (:wsmwk) from comment #2) > what is curious to me, is when did this start? > > because in the last two years we haven't had clustered reports of high > memory usage except for the mime parsing incident and maybe one other thing > - roughly Bug 810637 and Bug 784286. NO idea - it could just be https://bugzilla.mozilla.org/show_bug.cgi?id=844148 . I noticed it after a few weeks of my machine being unusable - and then I noticed how much memory it was consuming. Ain't sure it's a leak as at some point memory stabilizises.

(In reply to Wayne Mery (:wsmwk) from comment #2) > what is curious to me, is when did this start? I wouldn't say for sure that it's a regression unless we get other people reporting memory issues that run builds with DMD and give us logs with absurd amounts of memory spent with the same stack. Another possibility is that for some reason one of Ludovic's folders got corrupted (maybe because of another bug that was temporarily on nightlies) and that this bug started to manifest itself only after the corruption in the profile.

(In reply to Florian Quèze [:florian] [:flo] from comment #4) > (In reply to Wayne Mery (:wsmwk) from comment #2) > > what is curious to me, is when did this start? > > I wouldn't say for sure that it's a regression unless we get other people > reporting memory issues that run builds with DMD and give us logs with > absurd amounts of memory spent with the same stack. unclear. There's been some recent reports, but very few, and most do not have sufficient information. https://bugzilla.mozilla.org/buglist.cgi?list_id=5898201&bug_severity=blocker&bug_severity=critical&bug_severity=major&o9=allwordssubstr&o2=anywordssubstr&f12=short_desc&chfieldfrom=2012-12-01&o12=nowordssubstr&v2=memory%20leak&f10=CP&f1=OP&o3=substring&f0=OP&f8=short_desc&v3=mlk&resolution=---&classification=Client%20Software&classification=Components&f9=short_desc&j7=OR&f4=CP&chfieldto=Now&query_format=advanced&j1=OR&f3=keywords&f2=short_desc&v12=test&f11=CP&f5=CP&f6=OP&product=MailNews%20Core&product=Thunderbird&f7=OP&o8=substring

ludo, can you try an older build, to settle the question of regression/comment 4? Also, have you looked at the logging at https://wiki.mozilla.org/MailNews:Logging#Other_Useful_NSPR_Logging_Options and the several items described at https://wiki.mozilla.org/Performance:Leak_Tools#leak-gauge ? regarding bug 844148, I'd be (pleasantly) surprised if that is your problem.

(In reply to Wayne Mery (:wsmwk) from comment #6) > regarding bug 844148, I'd be (pleasantly) surprised if that is your problem. Meaning, I don't think that's what it is. ludo works with same type of folder/messages as justdave. And also Jake Maul [:jakem] - which is very interesting because jake documents in bug 757291 comment 25 that as of 18.0a2 his problem is largely gone. hmmm. justdave's issues over the years are well documented but not always well understood. But bug 589310 has much info. Might be same problem here. Noteworthy is justdave successfully uses postbox, which predates bug 589310 (circa 3.1). So how might bug 589310 relate to postbox? Possible scenarios include: 1. justdave's core perf issue with thunderbird wasn't fully documented in bug 589310 2. postbox fixed the problem 3. postbox never had the problem A noteworthy point is bug 589310 comment 37. 59 db's are open, which of course should not be happening. And ludo probably is not seeing that. But thether that was the core of justdave's perf problem is unknown since he's moved on to postbox. So more info is needed from ludo :) as one or more of the following - about:memory with more detail [1] - testcase folder data (is there no trusted sole it can be shared with?) - regression range, *if* this is a regression (which is still an unknown) - does problem happen with thunderbird with profile running only this folder using -no-remote - disconnected from network, left running started - as above, but connected and actively syncing And to restate a curious item - how do we explain comment 3, "at some point memory stabilizises."? [1] about:memory but unfortunately it doesn't yet tell us *which* part of memory usage related to the folder is causing the high memory. (ref bug 480843 blockers)

>(In reply to Wayne Mery (:wsmwk) from comment #6) >> ludo, can you try an older build, to settle the question of >> regression/comment 4? > >Can't really I don't postbox works - but that's 4+ year old imap code. Ludo there's a typo somewhere, so I don't understand your comment. Is there a technical reason you can't try 3.0 or 3.1 for example?

(In reply to Wayne Mery (:wsmwk) from comment #8) > Ludo there's a typo somewhere, so I don't understand your comment. > Is there a technical reason you can't try 3.0 or 3.1 for example? None - just lacking time ...

fair enough. ludo, Are the subjects of these messages mostly all the same? Are messages threaded or unthreaded in the folder? Do you have custom headers defined that would be picked up from messages? What is typical size of .msf?

(In reply to Wayne Mery (:wsmwk) from comment #10) > fair enough. > > ludo, > Are the subjects of these messages mostly all the same? yes they come up often looking alike. > Are messages threaded or unthreaded in the folder? default I never open that folder - In fact I've recreated a profile in order to make my Thunderbird usable. > Do you have custom headers defined that would be picked up from messages? no. > What is typical size of .msf? didn't check.

ludo, it never got to the point of crashing OOM, right?

Nope it never crashed in my case. Just made the machine unusable

reproduces with current trunk?

I've changed profile so I don't see it anymore.

(In reply to Ludovic Hirlimann [:Usul] from comment #15) > I've changed profile so I don't see it anymore. Thanks. We don't have anyone else reporting in this bug, and bug 787751 and bug 796989 are still open, so closing. (rough summary in comment 7)