Open Bug 1260698 Opened 8 years ago Updated 10 months ago

Compact folders - shows wrong estimated and compacted size. Compacts too often. totalExpungedBytes number is wrong.

Categories

(MailNews Core :: Backend, defect, P3)

Product:
MailNews Core
Component:
Backend
Type:
defect

Tracking

(thunderbird_esr52 affected, thunderbird_esr60- affected, thunderbird60 wontfix, thunderbird61 wontfix, thunderbird62 affected, thunderbird63 affected)

Tracking Flags:
Tracking Status
thunderbird_esr52 --- affected
thunderbird_esr60 - affected
thunderbird60 --- wontfix
thunderbird61 --- wontfix
thunderbird62 --- affected
thunderbird63 --- affected

People

(Reporter: direwolf, Unassigned, NeedInfo)

References

(
URL
)

Details

(Keywords: regression, regressionwindow-wanted, ux-trust)

User Story

https://www.reddit.com/r/Thunderbird/comments/pbaac0/thunderbird_asking_me_to_compact_and_telling_me/

All tagged support requests: https://support.mozilla.org/en-US/questions/thunderbird?tagged=bug1260698&order=votes&show=all

https://support.mozilla.org/en-US/questions/1228592
https://support.mozilla.org/en-US/questions/1226501
https://support.mozilla.org/en-US/questions/1225973
https://support.mozilla.org/en-US/questions/1225165
https://support.mozilla.org/en-US/questions/1225419
https://support.mozilla.org/en-US/questions/1226979
https://support.mozilla.org/en-US/questions/1225638
https://support.mozilla.org/en-US/questions/1225133

https://support.mozilla.org/en-US/questions/1231444
https://support.mozilla.org/en-US/questions/1231702
https://support.mozilla.org/en-US/questions/1234040
https://support.mozilla.org/en-US/questions/1234499

https://support.mozilla.org/en-US/questions/1241019
https://support.mozilla.org/en-US/questions/1243833#answer-1198405
https://support.mozilla.org/en-US/questions/1247192
https://support.mozilla.org/en-US/questions/1253800
https://support.mozilla.org/en-US/questions/1254925
https://support.mozilla.org/en-US/questions/1259888
https://support.mozilla.org/en-US/questions/1259990

Attachments

(3 files, 1 obsolete file)

Screenshots.png
(deleted), image/png
Details
compactbug.png
(deleted), image/png
Details
debugging patch
(deleted), text/plain
Details
Attached image Screenshots.png (deleted) — 
Steps to reproduce:
1. When emptying trash or archiving a letter - "compact folders" dialog appears.
2. The "compact folders" dialog, that appears, shows the estimated size that will be saved by compacting.
3. At the end of the compacting process the amount of saved space is displayed in the status-bar (at the bottom lest corner of the window).


Please note: I was unable to attach multiple screen-shots, so combined them all in one. All the screen-shots are from the same compacting.

Actual results:
At the step no.2 "estimated size" is too high - 80GB or more (in my cases). See 1 in the attached image.
At the step no.3 "approx saved space" is too high - 80GB or more (in my cases). See 2 in the attached image.

Expected results:
At the step no.2 should show more accurate "estimated size".
At the step no.3 should show more accurate "approx saved space" (289MB from the this compacting - see 3 in the attached image).

User agent - Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0
Priority: -- → P4
Hardware: Unspecified → All
Version: 38 Branch → 45 Branch
I encountered this bug after jumping to the beta channel, oddly I've never seen such compact prompt before although it's been around for years.

At first I thought it's just in wrong unit, but it doesn't match the actual figure even if the GB is replaced with MB. Actually I'm not even sure whether FormatFileSize [0] should be blamed, or if totalExpungedBytes [1] is wrong at the first place. Just thinking aloud while searching the codebase, as I haven't got time to setup an environment. Maybe someone will dig further if this issue particularly bugs them :-|

[0]: https://dxr.mozilla.org/comm-beta/rev/b6abecf3a8e13e56fc8d9ccb539e10d40017eaf3/mailnews/base/util/nsMsgUtils.cpp#484
[1]: https://dxr.mozilla.org/comm-beta/rev/b6abecf3a8e13e56fc8d9ccb539e10d40017eaf3/mailnews/base/util/nsMsgDBFolder.cpp#1915
I upgraded to Thunderbird 52.9.0 (32-bit release channel) this morning.  I received the "Compact All?" prompt, and clicked Yes.  When it finished, the message at the bottom of the Thunderbird window read, "Done compacting (approx. 6853 GB saved)."  That's an impressive amount of saved disk space on a 500 GB hard drive.

Environment:  Windows 7 Professional SP1

Also reference Bug 1434864.  I read through that duplicate bug before posting here.
Status: UNCONFIRMED → NEW
Component: Mail Window Front End → Backend
Ever confirmed: true
Product: Thunderbird → MailNews Core
Version: 45 Branch → 45
Attached image Thunderbird compact bug French.png (obsolete) (deleted) — 
171779 GB to be saved by compacting. So it's not just 171779 MB, but could it be 171779 KB?
(In reply to Thomas Bertels from comment #6)
> Created attachment 8990502 [details]
> Thunderbird compact bug French.png
> 
> 171779 GB to be saved by compacting. So it's not just 171779 MB, but could
> it be 171779 KB?

Thomas, Is this happening to you?  If so, you might be able to estimate it from a couple of your largest folders that have the most deleted messages.
Flags: needinfo?(tbertels+bugzilla)
OS: Linux → All
Kirill, 
Were you using Thunderbird PRIOR to version 38, and not seeing this problem?
Flags: needinfo?(direwolf)
Priority: P4 → --
Version: 45 → 38
Attached image compactbug.png (deleted) — 
Before compacting: 2879 MB
After compacting:   851 MB
Net gain:          2028 MB

Estimation before compacting: 99938 GB
Estimation after compacting: 124215 GB

Difference between Estimation after and Net gain: 62720 times bigger.

So it's not just a unit problem.
Attachment #8990502 - Attachment is obsolete: true
Flags: needinfo?(tbertels+bugzilla)
We should just remove the estimation... Easy workaround and not a big lost for users.
I, too, see this problem.  On Thunderbird 52.9.1 (64-bit) on two different Macs, when selecting a folder, I often see the Compact Folders dialog which promises to save an outrageous amount of space (16654 GB in one case). I've thus far been too scared to agree to a compaction that promises to free up 16TB. ;)

If the compaction savings can't be estimated correctly (but compaction works correctly otherwise), I think removing the estimation would be fine.  However, I am curious why I would even see the Compact Folders dialog box at all, since my preferences are configured to "Compact all folders when it will save over 20 MB in total".  Like 61.1p57 above, I assume this would make compaction automatic.
TB uses the estimated size of the saving to decide whether to start compaction or even show the dialog about it.
We need to determine whether it gets this number really wrong, or it counts it correctly, but only the display in the message is wrong.
In my local profile the displayed number is always correct (no weird numbers).

1.Does everybody who sees this have any IMAP accounts in the list?
2.Can somebody see the problem who only has POP3/Local Folder accounts in the list?
3.Is anybody willing to run a specially prepared nightly version of TB63 which will output debugging information to determine at which place the error appears (as described above).
I use IMAP exclusively and experience this issue. It's definitely not just the display that is wrong because normally I see this message every few weeks or months, now I get it every day once the broken value crosses the 20MB boundary. So the calculation that leads to the decision whether this dialog should be shown is already faulty.
Is this a one time occurrence after which the numbers get into 'normal' or are the messages regularly wrong on that same TB profile (mail accounts)?

It seems the reports are from all our 3 platforms so at least this shouldn't be some compiler/OS dependent variable size overflow.
Just to make sure, are all your accounts using the "mbox" store type? You can check that in "Account settings-><account>->Server settings->message store type" .
aceman, as I said I get this message at least once a day on the affected machine, if I dismiss it the bogus value raises every time. It started at around 1GB, then 2GB, then 3...

I will check about the mbox type tomorrow (it's my work machine).
Once you are on the account, please also see the folder where you use to delete messages, use right click->Properties and see "Size on disk", whether it matches the file size of that folder that you can see via the file manager. The file backing that folder is shown in "Location" field of that same dialog.
IF you are on Windows 64bit, you may try the debugging build from
https://queue.taskcluster.net/v1/task/ZEZIPwXvRjWi0I2hxrOqVw/runs/0/artifacts/public/build/target.zip which will show you in the compaction dialog each individual folder's raw number of bytes that TB thinks will be saved there. You may use it to spot the folder that contributes with this insane amount you reported.

You can just unzip the archive anywhere and run TB from it. It will not affect your existing TB installation, but WILL use your existing profile with your accounts. This is a version of TB 63, if you have any addons, those may not work in it for now.
I tried that build. My results are:

Do you wish to compact all local and offline folders to save disk space? This will save about 43,1 GB.
inbox1@example.org - 2018: 5152
inbox1@example.org - Inbox: 9697884
inbox1@example.org - Sent: 15429
inbox2@example.org - Inbox: 43799506216
inbox2@example.org - Folder 1: 2317003950
inbox2@example.org - Folder 2: 50376908
inbox2@example.org - Folder 3: 50320066
inbox2@example.org - Folder 4: 12595
inbox2@example.org - Drafts: 2464516
inbox2@example.org - Sent: 43012001
inbox2@example.org - Folder 5: 1273263 

None of these numbers reflect reality in any way. For example the file for "Folder 1: 2317003950" is about 8MB on disk, and certainly not 2.1GB.
And yes, both inboxes are mbox.
Are only the folders Inbox and Folder 1 of inbox2@example.org problematic? Or are also the other folders off?
So I assume the inbox2@example.org is an IMAP account. For folders Inbox and Folder 1, do you have "Select this folder for offline use" set in the Properties of those folders?
I don't know which ones are problematic - the dialog doesn't tell me after all. But since inbox2 is much bigger than inbox1, I noticed there that after folder compacting all those thousands of messages were downloaded again. inbox1 is pretty new but even there the estimated inbox size seems ways off. Both accounts are IMAP accounts by the way, not just inbox2, and I did not specifically chose offline use for either of those two accounts.
The dialog can't tell you, it's up to you to determine which folders are problematic, for which the reported size is impossible due to their size on disk. Or maybe I could add that crude comparison.

Please see, if after you compacted those folders, if the reported size "to be freed" changes.

May that reported size represent the size of the folder on server?

Can you also check, if those folders have the corresponding file called e.g. Inbox.msf alongside the Inbox file and it is not empty?
> The dialog can't tell you, it's up to you to determine which folders are problematic, for which the reported size is impossible due to their size on disk. Or maybe I could add that crude comparison.

I think that pretty much all the 6-figure and larger sizes are way off. I can check again tomorrow (together with the other stuff).

> Please see, if after you compacted those folders, if the reported size "to be freed" changes.

I'm pretty sure it would. As described above, the size increased by a gigabyte or so everytime the prompt was shown. The first time it was several hundred GB, then after compacting, it started again at 1GB or so, going up a GB each time.
(In reply to Johannes from comment #24)
> all. But since inbox2 is much bigger than inbox1, I noticed there that after
> folder compacting all those thousands of messages were downloaded again.

If that folders isn't set for offline use, what thousands of messages are downloaded again?

Also please report the results of process in comment 19, whether TB displays the total folder size correctly.
> If that folders isn't set for offline use, what thousands of messages are downloaded again?

To be precise, it probably just downloads all the headlines again, not the message bodies.
New build to try: https://queue.taskcluster.net/v1/task/cb1-v7VWRoSPGcTrQbVEzQ/runs/0/artifacts/public/build/target.zip, it should point out which folders are problematic.
Do you wish to compact all local and offline folders to save disk space? This will save about 45,7 GB.
inbox1@example.org - 2018: 5152
inbox1@example.org - Inbox: 10087691! more than the size on disk: 7142272
inbox1@example.org - Sent: 15429
[censored] - Inbox: 46329502464! more than the size on disk: 41099881
inbox2@example.org - Folder 1: 2544923598! more than the size on disk: 21445922
inbox2@example.org - Folder 2: 53336492! more than the size on disk: 200532
inbox2@example.org - Folder 3: 53760610! more than the size on disk: 154879
inbox2@example.org - Folder 4: 12595! more than the size on disk: 0
inbox2@example.org - Drafts: 2474288! more than the size on disk: 0
inbox2@example.org - Sent: 47064029! more than the size on disk: 13031952
inbox2@example.org - Folder 5: 1742057


All folders are selected for offline use (so I suppose this is the default).

> May that reported size represent the size of the folder on server?

My inbox is about 50MB on the IMAP server.

> Can you also check, if those folders have the corresponding file called e.g. Inbox.msf alongside the Inbox file and it is not empty?

They are not empty.

> Once you are on the account, please also see the folder where you use to delete messages, use right click->Properties and see "Size on disk", whether it matches the file size of that folder that you can see via the file manager. The file backing that folder is shown in "Location" field of that same dialog.

All these sizes are way off. Before comacting, Folder 1 is about 8MB on disk, and after compacting, it is about 2MB on disk. But both times it is reported as 20MB on disk inside Thunderbird.
Aceman, what is next step?
Severity: normal → major
tracking-thunderbird_esr60: --- → ?
User Story: (updated)
Flags: needinfo?(direwolf) → needinfo?(acelists)
Summary: Compact folders - shows wrong estimated and compacted size → Compact folders - shows wrong estimated and compacted size. Compacts too often.
Could some of these user be affected by Bug 765514 - message synchronization does not clean up old messages on imap (StringProperty_pendingRemoval is correctly set, but expungedBytes may wrap or be reset, Compact of offline-store file may fail if offline-store file after Compact exceeds 4GB)
Parse error; looks like you cut off in mid sentence there Wayne.  Is there any more testing needed?  I could download the test build if needed.
(In reply to Wayne Mery (:wsmwk) from comment #35)
> Could some of these user be affected by Bug 765514 - message synchronization
> does not clean up old messages on imap (StringProperty_pendingRemoval is
> correctly set, but expungedBytes may wrap or be reset, Compact of
> offline-store file may fail if offline-store file after Compact exceeds 4GB)

I'm not sure, but we strived to fix all 4GB folder issues. But it is true that the bug here looks like it may be caused by some integer overflow (in an insufficiently large variable) as those reported sizes are unusably high.
Flags: needinfo?(acelists)
For what it's worth, I built Thunderbird from source yesterday and poked around a bit.  In my case, I frequently saw the compact dialog with the inaccurate estimate because I always declined compaction.  I ran into a wall when I discovered that the bad expungedBytes values were being read from each folder's .mdb database.  (I confirmed the bad values were present in the mdb files.)  So in my case, the root cause may have been a one-off event that happened weeks ago (on version <= 52.9.1), making it hard to debug.

Chances are this is unrelated, but I am often seeing a problem (on yesterday's tip) where quitting will stall for a minute or two after the following log line before finally crashing:

[40353, Main Thread] WARNING: some msg dbs left open: '!m_dbCache.Length()', file /Users/simmons/thunderbird/source/comm/mailnews/db/msgdb/src/nsMsgDatabase.cpp, line 81

It reminded me that I occasionally experienced some similar stalls on release builds (version <= 52.9.1) where I had to manually find and kill the Thunderbird process.  I wonder if this could be leading to corrupt databases which in turn leads to the bad expungedBytes values.
Can this database be deleted and regenerated to get rid of the problem?
(In reply to David Simmons from comment #38)
> For what it's worth, I built Thunderbird from source yesterday and poked
> around a bit.  In my case, I frequently saw the compact dialog with the
> inaccurate estimate because I always declined compaction.  I ran into a wall
> when I discovered that the bad expungedBytes values were being read from
> each folder's .mdb database.  (I confirmed the bad values were present in
> the mdb files.)  So in my case, the root cause may have been a one-off event
> that happened weeks ago (on version <= 52.9.1), making it hard to debug.

Yes, that is the question here, how those bad values get into the .msf file.
 
> [40353, Main Thread] WARNING: some msg dbs left open: '!m_dbCache.Length()',
> file
> /Users/simmons/thunderbird/source/comm/mailnews/db/msgdb/src/nsMsgDatabase.
> cpp, line 81

Yes, we also see this occasionally and do not know what causes it. We haven't yet seen a case this would somehow corrupt the file and cause it to include wrong values (like the bug here).
 
> It reminded me that I occasionally experienced some similar stalls on
> release builds (version <= 52.9.1) where I had to manually find and kill the
> Thunderbird process.  I wonder if this could be leading to corrupt databases
> which in turn leads to the bad expungedBytes values.

Hopefully not, as the msf files are a database that should provide atomicity. So either the new expungedBytes value is updated or the previous value will be seen. Invalid values should not come up.
(In reply to Phillip Susi from comment #40)
> Can this database be deleted and regenerated to get rid of the problem?

You can delete the msf file for the particular folder. But please backup it first in case you need to restore it.
Sometimes it contains the information about tags on messages or which messages are unread. The folder file (mbox) should also contain the information (and .msf is just a cache of it), but there may be bugs when that is not true.
User Story: (updated)

Continues to be reported by users.

I know it doesn't solve the issue, but can we add the crude sanity check you suggested and, as a permanent fixture, throw something in the error console with the filename?

There is also Bug 1521073 to consider - CPU use on OSX rises to 70% after Inbox compact

User Story: (updated)
Flags: needinfo?(acelists)

Can we add a debug/sanity check at the point where messages are deleted or moved?

URL: https://support.mozilla.org/en-US/que...
User Story: (updated)
Keywords: regression, regressionwindow-wanted

More like at all places where we touch the expungedBytes variable. I could do that in the debugging patch we carry here.

Blocks: 1597338

Does anybody know of a workaround to make it stop asking me to compact folders every freaking day?

I know I had seen this error ages ago but I don't remember what I did that made it go away (or it went away on its own). Then it hase resurfaced recently after I have deleted tons of messages.

Uncheck the option? Tools > Options, Advanced, Network & Diskspace, uncheck "Compact all folders ...".

No longer blocks: 1597338

Well, ok, that might do as long as I remember to compact manually from time to time.

What I meant was whether there is a workaround to force whatever corrupted information seems to be stored somewhere to be recomputed, so that it will start behaving correctly again with the compact-when-above-threshold option enabled, at least for some time until it gets screwed up again.

I am affected by the same issue and would like to help resolving it, as it has bothered me for a long time now.

Searching the source code, I found that the size of space that can be freed up is tracked via the property m_expungedBytes of nsDBFolderInfo objects (see https://dxr.mozilla.org/comm-central/source/comm/mailnews/db/msgdb/src/nsDBFolderInfo.cpp). Whenever messages are deleted, this counter is increased. The compacting method nsMsgDBFolder::HandleAutoCompactEvent (https://dxr.mozilla.org/comm-central/source/comm/mailnews/base/util/nsMsgDBFolder.cpp) just reads and adds up this variable for different folders. If summing up works (which we may assume since otherwise everyone would have this issue), we therefore need to investigate where m_expungedBytes is changed in a potentially wrong way.

To this end, it would be great if there were an option to track the state of this variable and all calls to the ChangeExpungedBytes method (https://dxr.mozilla.org/comm-central/source/comm/mailnews/db/msgdb/src/nsDBFolderInfo.cpp). I think this would require a custom build of Tb. Unfortunately I do not have the hardware to build Tb myself (and I do not really know c++). Is there a way to get around this issue?

My current hypothesis is that the issue may be related to the setting to download new emails only, since I am basically never deleting messages on my own. However, I have set Tb to download messages newer than 1 year only. That is, older emails are deleted. This happens in ApplyRetentionSettings (https://dxr.mozilla.org/comm-central/comm/mailnews/imap/src/nsImapMailFolder.cpp).

To test this hypothesis without going deep into the code, it would already be helpful if I could figure out how much space can supposedly be freed up in each individual folder. Then I could compare folders with this setting to folders without it. Furthermore, I know that Tb regularly checks if compacting folders makes sense. Is there a way to trigger this query without having to wait?

(In reply to Sam from comment #52)

I am affected by the same issue and would like to help resolving it, as it has bothered me for a long time now.
...
My current hypothesis is that the issue may be related to the setting to download new emails only, since I am basically never deleting messages on my own. However, I have set Tb to download messages newer than 1 year only. That is, older emails are deleted. This happens in ApplyRetentionSettings (https://dxr.mozilla.org/comm-central/comm/mailnews/imap/src/nsImapMailFolder.cpp).

To test this hypothesis without going deep into the code, it would already be helpful if I could figure out how much space can supposedly be freed up in each individual folder. Then I could compare folders with this setting to folders without it. Furthermore, I know that Tb regularly checks if compacting folders makes sense. Is there a way to trigger this query without having to wait?

Sam, does this occur in a profile that has only has an inbox with retention settings applied?

Flags: needinfo?(acelists) → needinfo?(samuel.fischer)

David ...

(In reply to David Simmons from comment #38)

For what it's worth, I built Thunderbird from source yesterday and poked
around a bit. In my case, I frequently saw the compact dialog with the
inaccurate estimate because I always declined compaction. I ran into a wall
when I discovered that the bad expungedBytes values were being read from
each folder's .mdb database. (I confirmed the bad values were present in
the mdb files.) So in my case, the root cause may have been a one-off event
that happened weeks ago (on version <= 52.9.1), making it hard to debug.

Can you explain a) in what way the value is bad - high/low/... ? b) what event?

Chances are this is unrelated, but I am often seeing a problem (on
yesterday's tip) where quitting will stall for a minute or two after the
following log line before finally crashing:

[40353, Main Thread] WARNING: some msg dbs left open: '!m_dbCache.Length()',
file
/Users/simmons/thunderbird/source/comm/mailnews/db/msgdb/src/nsMsgDatabase.
cpp, line 81

It reminded me that I occasionally experienced some similar stalls on
release builds (version <= 52.9.1) where I had to manually find and kill the
Thunderbird process. I wonder if this could be leading to corrupt databases
which in turn leads to the bad expungedBytes values.

So it hung? (not crash)

Do you still see either of these problems - the incorrect compact values, and hangs?

And does Sam's comment 52 seem relevant to your situation?

Flags: needinfo?(mozilla-bugzilla)

(In reply to Wayne Mery (:wsmwk) from comment #53)

Sam, does this occur in a profile that has only has an inbox with retention settings applied?

The issue occurs also in accounts for which I download all emails without considering their age. This speaks against my earlier hypothesis and puzzles me even more, as I typically do not delete any emails in from my accounts. I am happy to provide more information if I can.

Flags: needinfo?(samuel.fischer)

(In reply to :aceman from comment #47)

More like at all places where we touch the expungedBytes variable. I could do that in the debugging patch we carry here.

assert (or something) on expungedBytes > folder size ?

Flags: needinfo?(mozilla-bugzilla) → needinfo?(acelists)
Keywords: ux-trust
Summary: Compact folders - shows wrong estimated and compacted size. Compacts too often. → Compact folders - shows wrong estimated and compacted size. Compacts too often. totalExpungedBytes

(In reply to Sam from comment #52)

...
Is there a way to trigger this query without having to wait?

(I haven't tested this) An indirect approach is set the compact threshold to a low value (1MB), create a template message with large attachments, send it to yourself a few times, delete the messages you received. If no prompt, then restart thunderbird.

FWIW the purge code is at https://searchfox.org/comm-central/rev/dfafa758b061c4b4052899dd51d4304c066b249f/mailnews/base/src/nsMsgPurgeService.cpp#111

And lightweight description at https://bugzilla.mozilla.org/show_bug.cgi?id=990551#c9

Flags: needinfo?(samuel.fischer)

We have a new example at https://support.mozilla.org/en-US/questions/1328291

I assume the debugging info being referenced is at https://bugzilla.mozilla.org/show_bug.cgi?id=750569#c62

If the regression is from v38 era, then it may be from one of https://mzl.la/30kBnUp
If from v52 era, then maybe from https://mzl.la/30fk0V1

I typically run without offline store so I seldom run into the auto-compact feature. But I sometimes set folders to use offline store when debugging a problem, for example the recent bug 1702692. In that bug the user has a message in offline store that has a header that TB v78.* deems invalid. This causes TB to re-download the message from the server when the folder is accessed and the message is redundantly stored to offline store many times. This causes the mbox file holding the message to grow in size continuously. Unfortunately, expunging, compacting or even repairing the folder doesn't get rid of this redundant content and reduce its size. So when an auto-compact occurs, this redundant space may be calculated as potentially able to be compacted out but it never is, thus causing the number of compact-able bytes to increase over time.

This is just a possible cause or a contributing cause of what is observed in this bug. The fix for bug 1702692 is not yet released and is currently only in the TB daily build and it doesn't get rid of the redundant content in the offline storage file; it just makes it less likely that redundant content will be added due to a bad header received from the IMAP server. The only way I could find to get rid of the redundant content was to delete the mbox file and the corresponding .msf file for the folder and let tb re-download and rebuild the folder with the patch for bug 1702692 applied which is more tolerant of "bad" header lines when accessing the mbox file.

Priority: -- → P3
Attached file debugging patch (deleted) —

This is the debugging patch I have have pending for this bug. It looks like it was never attached here, but maybe we made some try builds with it in the past.

It probably does not apply cleanly today.

Looks like there are some cleanups that could be landed independently on finding the main issue.

Flags: needinfo?(acelists)
User Story: (updated)
Summary: Compact folders - shows wrong estimated and compacted size. Compacts too often. totalExpungedBytes → Compact folders - shows wrong estimated and compacted size. Compacts too often. totalExpungedBytes number is wrong.

My thunderbird is exhibiting symptoms that sound like this bug. Many times a day it says, "Thunderbird needs to do regular file maintenance to improve the performance of your mail folders. This will recover 89.7 megabytes of disk space without changing your messages..." It is always 89.7 MB, regardless of how often I compact my folders.
I have an inbox with a few subfolders and sub-sub-folders in at least one of those subfolders. I don't know where to find totalExpungedBytes and estimated/compacted size, so I can't tell you if they are incorrect or not.
I am running Thunderbird 102.0 on Windows 7.

I should add that if this doesn't seem to be the same as bug 1260698, I will be happy to submit a separate report. And I will be happy to run any tests that can help in fixing the bug. Unfortunately, I can't think of any way to recreate this bug from scratch.

I am embarrassed to confess that my problem was most likely the result of user error. I forgot to re-boot my computer after upgrading Thunderbird. I have re-booted and the problem has gone away.

(In reply to Stuart Soloway from comment #69)

I am embarrassed to confess that my problem was most likely the result of user error. I forgot to re-boot my computer after upgrading Thunderbird. I have re-booted and the problem has gone away.

Is this necessary? What OS? I have never thought a program issue was dependent on the OS rebooting.

In answer to Worcester12345: I believe that after I upgraded Thunderbird, a pop-up said that I should reboot my computer. I didn't. Then I saw the bug and reported it. Finally I remembered that popup and rebooted. The bug disappeared. I'm running a Windows7 desktop.

I seem to suffer from this bug. :-(

I've never seen it before, and I'm a very longtime user, using TB on many different OS, many machines.

It's on a newly set up Mac, latest macOS (ARM64) version, latest TB version.

It's suddenly saying that compacting folders would free 65.3 GB of disk (somehow reminds me of "65.535", i.e. 2^16 - 1?!). I don't dare to let TB clean up, scared that something bad could happen.

My profile folder is only 5 GB in size in total.

Could somebody pls guide me where to find a recent debug build that would output the size estimates for individual folders, so that we could find out what's wrong?

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: