Closed
Bug 101315
Opened 23 years ago
Closed 23 years ago
summary file creation is very slow, speedup possible
Categories
(SeaMonkey :: MailNews: Message Display, defect, P1)
SeaMonkey
MailNews: Message Display
Tracking
(Not tracked)
VERIFIED
FIXED
mozilla0.9.7
People
(Reporter: stefan, Assigned: Bienvenu)
References
Details
(Keywords: perf)
Attachments
(2 files)
(deleted),
patch
|
sspitzer
:
superreview+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
sspitzer
:
superreview+
|
Details | Diff | Splinter Review |
build id 2001091903 not sure if this is a backend, database or frontend problem. I've noticed a possible great improvement in mailnews summary creation performance: after deleting a summary file on a folder with 4260 messages, rebuilding on my system took 270 seconds. repeating the procedure, but immediately selecting a different folder once the rebuild starts, and the rebuild takes only 90 seconds. looks like the endless repaints on the message list pane takes 2/3 of the time!
![]() |
||
Updated•23 years ago
|
Reporter | ||
Comment 1•23 years ago
|
||
I've been trying to do some informal benchmarking of how much time this could save. I don't know about anyone else, but I seem to drive mailnews quite hard. Got maybe 30 folders with 500-3000 messages in each. And the program seems to need to rebuild its indexes from scratch from all the time? what is the criteria for saying do a rebuild? my experience is that would make a big perceptual difference to how much time I spend thumb-twiddling waiting for mozilla to come back to me, over the course of a working day.
Comment 2•23 years ago
|
||
adding some more developers to this. Any thoughts on this and if you agree is there anything that can be done about the extra painting?
Comment 3•23 years ago
|
||
We build summary file way too often, and i have seen this because i use pop3 as my primary acct. I will try to log some bugs to track this problem.
QA Contact: esther → stephend
Assignee | ||
Comment 4•23 years ago
|
||
yes, we could avoid displaying any headers until the folder was reparsed. Navin is right; there are situations where we were leaving the db out of sync - one was mark all read, which has since been fixed. But we have to reparse the db once it gets out of sync or we'd be very vulnerable to data loss.
Assignee: sspitzer → bienvenu
Comment 5•23 years ago
|
||
Navin, were you able to log some bugs about reparsing the summary file too often? Those might be worth looking into since it'll improve folder loading perf in those cases.
Assignee | ||
Comment 6•23 years ago
|
||
Be aware that running 4.x and 6.x/mozilla against the same profile will guarantee lots of reparsing, and there's nothing we can do about that.
Comment 7•23 years ago
|
||
right, I don't care about the 6.x/4.x case. Navin's and your comments made it sound like we are getting out of sync in other cases that we might be able to prevent.
Comment 8•23 years ago
|
||
Adding myself to this bug. I too am hitting hard performance problems with mail using POP3. I have about 500 MB of personal mail at work, and another 500 MB at home. The constant rebuilding of index files is very painful. Some of my folders have 5000 to 50000 messages in them and it just takes forever ... When I do it in 4.x it is an order of magnitude faster, and the reindexing only appears when the db gets corrupt somehow, which is very rare. In recent builds of mozilla (at least 0.9.5 and 0.9.6 on os/2), my folders *always* get reindexed when they are first accessed in the mail application after the brower has started. If I just exit mozilla , restart it, and go back to the same folder, it reindexes it. This problem has forced me to stay with 4.61 for os/2 at home instead of mozilla 0.9.x . It was not always that bad in mozilla, it seems that the reindexing problem has been getting much more frequent over the past few releases. FYI, I am not experiencing that constant reindexing bug in the Netscape 6.2 release on Windows on my work machine. So it may be a post 0.9.4 regression. I also want to add that I'm not sharing profiles between 4.x / 6.x at all (either at home on os/2 or at work on windows). I always have a separate migrated profile for mozilla/netscape6. But it is only for testing and I always "leave message on server" in them, with the actual remove always occuring in the 4.x browsers, until I pick up the next build of mozilla/netscape6 and re-migrate ...
Updated•23 years ago
|
OS: Windows ME → All
Hardware: PC → All
Updated•23 years ago
|
Assignee | ||
Comment 10•23 years ago
|
||
this patch adds a method that clients can use to see if a db is valid. The sole client (patch upcoming) is for local mail to know if a folder is getting reparsed, but I can imagine other clients in the future. The other dbs (beside local mail) will probably want to check the version but I'll do that later.
Assignee | ||
Comment 11•23 years ago
|
||
Comment 12•23 years ago
|
||
Comment on attachment 60868 [details] [diff] [review] db part of fix sr=sspitzer
Attachment #60868 -
Flags: superreview+
Comment 13•23 years ago
|
||
Comment on attachment 60869 [details] [diff] [review] check if folder is being reparsed before sending folder loaded notification sr=sspitzer any reason not to put "rv = mDatabase->GetSummaryValid(&valid))" on the line above, it makes debugging easier.
Attachment #60869 -
Flags: superreview+
Comment 14•23 years ago
|
||
I don't see how this will speed up folder loading. it may solve the problem that I was trying to fix in some other bug using that parsingFolder boolean
Comment 15•23 years ago
|
||
s/folder loading/ building summary file/
Assignee | ||
Comment 16•23 years ago
|
||
It's complicated, Navin, but believe me, it cuts the time by 10x or more. Essentially, we no longer send notifications to the front end to adjust the counts on each message we reparse. This is in conjunction with the fix for bug 112761, which relies on us not sending a folder loaded notification before we're done parsing. Once the FE receives a folder loaded notification, it turns on all the count updating.
Assignee | ||
Comment 17•23 years ago
|
||
I believe this was slow because we were updating the msg counts, not the thread pane. The checked in patch speeds it up dramatically for me.
Status: NEW → RESOLVED
Closed: 23 years ago
Resolution: --- → FIXED
Comment 18•23 years ago
|
||
sorry for the delay, r=naving, but I was just trying to figure out how we are saving much because we are just preventing one NotifyFolderEvent from being triggered.
Assignee | ||
Comment 19•23 years ago
|
||
like I said, it's complicated, but if we call StartFolderLoading, it turns off notifications, until EndFolderLoading gets called, and sending a folderloaded notifiation makes EndFolderLoading get called.
Verified FIXED using build 2001-12-16-08 and a build from the 7th to compare behaviors. Setup: 8,027 message local folder, deleted the .msf for that folder and then clicked on the folder, timed how long it took to fully load the folder's contents and stop the meteors. 12-16 | 12-07 ------------------------ 20.80 seconds | 50.90 seconds This was on a really fast machine, so we cut building (and I'm assuming parsing too, since after building we won't have the message headers cached) the summary file by 59% on a Pentium 733 with 384 megs of RAM.
Status: RESOLVED → VERIFIED
Updated•20 years ago
|
Product: Browser → Seamonkey
You need to log in
before you can comment on or make changes to this bug.
Description
•