Open Bug 1719072 Opened 3 years ago Updated 2 years ago

Compact should never run on (or near) startup

Categories

(MailNews Core :: Backend, defect)

Unspecified
All
defect

Tracking

(thunderbird_esr78 wontfix, thunderbird_esr91- wontfix, thunderbird_esr102 affected, thunderbird90 wontfix, thunderbird91- wontfix)

Tracking Status
thunderbird_esr78 --- wontfix
thunderbird_esr91 - wontfix
thunderbird_esr102 --- affected
thunderbird90 --- wontfix
thunderbird91 - wontfix

People

(Reporter: wsmwk, Unassigned)

References

(Blocks 3 open bugs)

Details

(Keywords: perf, reproducible, Whiteboard: [enterprise-relevance])

Steps to reproduce:

  • get to a point where the compact threshold has been reached, by constantly deferring compact by picking Cancel at the compact prompt
  • restart Thunderbird

Result: compact prompt occurs on startup

Expected results: Compact does not run on startup, or shortly thereafter (say within 15-20 minutes of startup)

I can consistently produce the compact prompt at startup.

We should want to do this regardless of whether the compact prompt is enabled or not, for the following reasons:

  1. For performance reasons, we should provide significant time for a user to do some productive work before the performance impact of compact kicks in - on top of the high IO and other resource usage of downloading many messages, and the resulting filtering and indexing of messages

  2. To avoid locking contention during this high IO period, which might cause filtering errors (although the code should already be preventing "most" filter errors) or causing incomplete compacting

  3. To avoid interrupting user work with compact prompt, for those users who have decided not to suppress the prompt

  4. Even if the compact threshold has not been met immediately on startup, a user can hit the threshold during startup because of message filtering, spam filtering, and manually deleting messages from Inbox.

I suspect this behavior might rarely affect the average user. But there is a higher probability of impact for users of high message traffic, and use filters. And moreso if messages tend to be large.

Can this be accomplished by changing the initial value of gtimeOfLastPurgeCheck to a negative value ??

Also, I wonder whether this compact behavior will occur when waking a laptop after it has been sleeping over an hour. (the compact can't happen more than once an hour - https://searchfox.org/comm-central/rev/4c5a16a4e9a7d5e873e2518c743e231d9385f385/mailnews/base/src/nsMsgDBFolder.cpp#1822 )

Flags: needinfo?(bugzilla2007)

Given the incompatibility of Compact action with some other concurring actions like Filters (see related/dependent bugs here), this bug has my full support from an UX perspective. As Wayne hinted, enterprise consumers with more and larger messages and much filtering are likely to be affected by this.

Flags: needinfo?(bugzilla2007)
Whiteboard: [enterprise-relevance]

A few background notes ...

  • The code which implements the compact at startup and the one hour interval is bug 17204.
    ** This was 20 years ago when message stores and disks were much smaller (and, yes, much slower), and it was also not normal to be running a laptop on battery. SSD, with its limited lifetime writes, was in its infancy.
    ** There was no real discussion of the wisdom of compact on startup in that bug report. Only a couple comments about running it at startup and about users who might not be online for long https://bugzilla.mozilla.org/show_bug.cgi?id=17204#c65 - where thunderbird is running for a very short time. There is no mention of other possible complications, of any type.
    ** The was no discussion about the one hour interval - so we don't know the reason for choosing it over a longer or shorter value. Is one hour still a suitable value? (perhaps a question for a different bug)

  • https://bugzilla.mozilla.org/show_bug.cgi?id=439089#c4 broadly describes how compacting works, at the time bug 439089 was implemented: "Every local and offline folder has an expunged-byte counter, and they update this value after every delete operation. When an imap folder is selected while TB is in offline state, UpdateFolder() method starts an all-folders compacting operation via AutoCompact() method. Basically, AutoCompact goes through all the accounts and their folders to collect total expunged byte count. If it is bigger than the threshold set in the prefs, it shows a dialog to ask whether the user wants to execute the compacting op or not. If yes, it starts compacting local/pop3 and offline folders using nsFolderCompactState class that controls the compacting state machine."

We're not going to track a non-critical, non-regression bug.

Wayne, we've just duplicated status-thunderbird91 and tracking-thunderbird91 to status-thunderbird_esr91 and tracking-thunderbird_esr91 to not have to query on two different flags. BTW, after comment #1 you added tracking-thunderbird91: --- → ? yourself.

The feature request i came to file, and found this bug, is that Thunderbird should compact when i am inactive. Instead it reliably begins to compact the moment i return to Thunderbird, like i may have had it open and unused for three hours, and the moment i go to use it i cannot because it begins compacting. So if moving compacting to shutdown as suggested in bug 401295 works, that's great. Based on the current behavior this proposal to delay from the start would not be enough, we would have to at least have some delay from when it detects the need for compaction (since interacting with Thunderbird seems to trigger detecting) or, ideally, have the check for and compact action take place explicitly when no action has happened for five or fifteen or thirty minutes.

Bug 286888 would be dependent on a solution like this one (comments in that thread raise the idea of an option to never compact at all… if the only benefit is disk space and not faster indexing somehow then i would like that option please!)

(In reply to Benjamin Melançon from comment #6)

Based on the current behavior this proposal to delay from the start would not be enough, we would have to at least have some delay from when it detects the need for compaction (since interacting with Thunderbird seems to trigger detecting) or, ideally, have the check for and compact action take place explicitly when no action has happened for five or fifteen or thirty minutes.

No, the idea behind this bug report is to delay initial compact by 5 minutes or more, regardless of user interraction. Except an explicity user compact request.

The feature request i came to file, and found this bug, is that Thunderbird should compact when i am inactive. Instead it reliably begins to compact the moment i return to Thunderbird, like i may have had it open and unused for three hours, and the moment i go to use it i cannot because it begins compacting. So if moving compacting to shutdown as suggested in bug 401295 works, that's great.

Bug 286888 would be dependent on a solution like this one (comments in that thread raise the idea of an option to never compact at all… if the only benefit is disk space and not faster indexing somehow then i would like that option please!)

Neither of those changes is likely to ever happen, because mbox should be on it's way out in version 114 next year, and compacting on shutdown brings it's own complications.

You need to log in before you can comment on or make changes to this bug.