Closed Bug 1349915 Opened 8 years ago Closed 6 years ago

Slow typing, Thunderbird continuously using lots of CPU, until Deleted and rebuilt the global database.

Categories

(MailNews Core :: Database, defect)

Unspecified
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: martingreybeard, Unassigned)

References

Details

(Keywords: perf, testcase-wanted)

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 Steps to reproduce: Thunderbird open looking at one of my (large) folders. `top` showed Thunderbird continuously using 50% CPU (so consuming half a core). Things I did to try and make the problem go away: - Move 2/3 of the contents of the folder into a sub-folder - Increased mail.db.idle_limit to 30000000 - Disabled indexing on the sub-folder - Deleted and rebuilt the global database (which involved restarting Tbird) The last step fixed the problem (and I foolishly didn't save the old DB so you could peer at it). I don't know if any of the other steps were a necessary pre-condition. Running 45.7.0 on 64-bit Ubuntu 14.04. Actual results: Much CPU was used Expected results: Little CPU should have been used.
(In reply to Martin Bonner from comment #0) > Thinks I did to try and make the problem go away: s/Thinks/Things/ (isn't there a way to edit the description?)
> (isn't there a way to edit the description?) no > - Move 2/3 of the contents of the folder into a sub-folder you don't say how many. you have an approximate number? > The last step [delete global database] fixed the problem (and I foolishly didn't save the old DB so you could peer at it). how big was it? > I don't know if any of the other steps were a necessary pre-condition. unlikely
Flags: needinfo?(martingreybeard)
Keywords: perf
There are (In reply to Wayne Mery (:wsmwk, NI for questions) from comment #2) > > - Move 2/3 of the contents of the folder into a sub-folder > you don't say how many. you have an approximate number? Curious: If I view the subfolder it says "Unread: 0, Total: 8848". Meanwhile, the left of the status bar says "Downloading 1401 of 7432 in 2016-H2". If I view the main folder it says "Unread: 5 Total 1339". Astute mathematicians may notice this is not quite the promised 2:1 ratio. (The folder holds wiki notications. I moved six-months worth, and left three months. It looks like my assumption of constant rate of wiki changes was an alternative fact.) > > > The last step [delete global database] fixed the problem (and I foolishly didn't save the old DB so you could peer at it). > how big was it? The backup last night shows 13.56 MB The current file is 7.5 MB > > > I don't know if any of the other steps were a necessary pre-condition. > unlikely
Flags: needinfo?(martingreybeard)
It *just* occurs to me, that I have slightly strange locale settings: martin.bonner:~$ locale LANG=en_US.UTF-8 LANGUAGE=en_US:en_GB:en LC_CTYPE="en_US.UTF-8" LC_NUMERIC=de_DE.UTF-8 LC_TIME=de_DE.UTF-8 LC_COLLATE="en_US.UTF-8" LC_MONETARY=de_DE.UTF-8 LC_MESSAGES="en_US.UTF-8" LC_PAPER=de_DE.UTF-8 LC_NAME=de_DE.UTF-8 LC_ADDRESS=de_DE.UTF-8 LC_TELEPHONE=de_DE.UTF-8 LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=de_DE.UTF-8 LC_ALL= (Note the mix of en_US, de_DE, and en_GB). Now, obviously, that shouldn't make any difference in theory. In practise ...
> The backup [of the gloda file] last night shows 13.56 MB That's tiny. In terms of size it certainly would not have caused a performance problem. Besides... > - Disabled indexing on the sub-folder if you did this (and had restarted Thunderbird) then no gloda indexing activity was happening and there would have been no activity to the gloda file. > Total: 8848 And I see now your BMO account is gmail. Did the problem you are having occur in the gmail account? If so, in Thunderbird, unsubscribe from the "All Mail" folder. That should help some. And in general, selecting then moving more than a few hundred messages at a time can get you in trouble.
Flags: needinfo?(martingreybeard)
OS: Unspecified → Linux
(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #5) > > - Disabled indexing on the sub-folder > if you did this (and had restarted Thunderbird) then no gloda indexing > activity was happening and there would have been no activity to the gloda > file. I certainly restarted Thunderbird at least once before rebuilding the gloda. On one occasion I had to kill the background Thunderbird process. On one occasion (*possibly* just before rebuilding the gloda) Thunderbird generated a crash report (I don't suppose there is any way to link that crash report to this bug report.) I don't know whether I restarted Thunderbird after disabling indexing. > > Total: 8848 > And I see now your BMO account is gmail. Did the problem you are having > occur in the gmail account? No. I use Thunderbird with IMAP to connect to outlook.office365.com for my work email (there's no *way* I'm going to use Outlook if I can avoid it). I access gmail through the webmail interface (it's good enough, and it keeps work and private email seperate). > If so, in Thunderbird, unsubscribe from the "All Mail" folder. That should > help some. > And in general, selecting then moving more than a few hundred messages at a > time can get you in trouble. The problem occurred before I started the move. Also, I don't mind moving a lot of emails taking a long time (although I'd rather it was no worse than O(n log n)), but it should eventually succeed.
Flags: needinfo?(martingreybeard)
Just to set expectations here: I didn't raise this issue because I am going to demand my money back if you don't produce a fix (my problem has gone away). I raised it because it provides one data-point, and if you get enough of them, you may be able to find the problem. (But feel free to ask more questions to improve the data-point.)
Hard to say without a look at the database (perhaps hard even if we had it). https://mzl.la/2rT0FJv some gloda/global performance issues, both close and open
Component: Untriaged → Database
Product: Thunderbird → MailNews Core
Summary: Thunderbird continuously using lots of CPU → Thunderbird continuously using lots of CPU, until Deleted and rebuilt the global database
Version: 45 Branch → 45
Martin, were there symptoms other than high CPU, such as slow menus, message navigation or display, etc?
Flags: needinfo?(martingreybeard)
Keywords: testcase-wanted
I first noticed it because typing was slow to respond (fractions of a second rather than minutes, but enough to be very annoying). Not sure if menus were slow too.
Flags: needinfo?(martingreybeard)
(In reply to Martin Bonner from comment #10) > I first noticed it because typing was slow to respond (fractions of a second > rather than minutes, but enough to be very annoying). Not sure if menus > were slow too. I see slow responding typing in Windows 7 (64 bit) also from time to time. It almost seems like it is indexing or searching, or doing some other function, then comes back to the typing function, and it catches up again for a while.
Martin, better with version 60 from https://www.thunderbird.net/en-US/channel/ ??
Flags: needinfo?(martingreybeard)
Summary: Thunderbird continuously using lots of CPU, until Deleted and rebuilt the global database → Slow typing, Thunderbird continuously using lots of CPU, until Deleted and rebuilt the global database.
Martin writes "I haven't seen this bug recently. Possibly because it has been fixed (or made less likely) by some change, but possibly because I have changed my working habits a bit (I now only get summary emails from Atlassian Confluence, rather than one per change). "
Status: UNCONFIRMED → RESOLVED
Closed: 6 years ago
Flags: needinfo?(martingreybeard)
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.