Opening an IMAP mail folder for the first time is very slow (amout one message per second). This makes it unusable

On NT with my p200 with debug 5.0 build, I can download a 6600 msg imap folder in about 1 minute 30 seconds. My release 4.5 build takes 1:55 seconds. I don't know why the mac should be so slow, but IMAP is pretty zippy on windows, and I hear Linux.

cc: pmock.

Then you must have some real threading badness on Mac, or something. Perhaps you're doing so many paints/reflows that it's very slow, due to a GFX bug or something?

No, because of the painful slowness of telling RDF about headers, and the subsequent tree control and layout thrashing, we stopped telling RDF about new headers last week. The only painting that happens is the progress bar, and we throttle that down the same way we do in 4.5 (only update it every 1/4 second). We could play with slowing it down even more for the mac. I suppose the mac threading could be bad, but I don't see how it could be 60 times worse.

On my pitiful IMac, with a debug build, I can download a 750 header folder in 30 seconds, about 20 headers a second. What kind of mac are you running on?

450MHz G3. It should be much faster.

Has it always been this slow in 5.0, or did it suddenly slow down, or is this the first you've tried it?

I know it was very slow a while back. I think it was faster when I last fetched my IMAP inbox (around a week ago). It seems to have slowed down again recently.

when was the last time you tried it?

Lisa, does anyone in QA see this on the mac?

I will check with Peter - he has a G3 :-)

Simon, do you have any kind of logging turned on? Do you notice anything unusual while headers are getting downloaded, like flashing scrollbars? Are news and local message download speedy?

I'll do some poking around when I have a build. Can you recommend some instrumentation points in the IMAP fetching code, like a choke point for fetching each message?

Yes, I see the problem on my G3/400 128MB ram, vm off. I used my own mail account folder to run my test. I have a IMAP folder that contains 6813 scopus messages. On today Mac seamonkey build 1999-09-22-09-m11, it took 7.43 minutes to open my scopus folder and download messages for the first time. On 4.7 Sept 20 RTM Mac build, it took only 1.36 minutes. I do not have IMAP logging turned on. I do see the horizontal scroll bar in the messenger pane flash in and out while downloading messages. I have to agree with Simon finding that fetching IMAP mail is extremely slow.

What if you resize the window so that the flashing doesn't happen? Does that help performance? Also, your download rate is about 15 headers a second, which is 15 times faster than Simon, so I don't think you're seeing his problem.

re instrumentation, mailnews/imap/src/nsImapMailFolder.cpp, NS_IMETHODIMP nsImapMailFolder::NormalEndHeaderParseStream() is where the rubber hits the UI thread for adding a header to the database. It gets called once for each header. I doubt this routine takes much time; I would expect any sort of painful race condition to occur in the imap protocol code, but that's kinda hard to pinpoint. You might set a breakpoint in nsMsgDatabase::Commit and make sure it's only getting hit at the end of loading headers - if for some freaky reason, we commited after every header on the mac, that would be expensive. For fun, you could comment out the progress code in nsImapProtocol.cpp - nsImapProtocol::ProgressEventFunctionUsingId and nsImapProtocol::PercentProgressUpdateEvent to see if all the time is getting spent there.

I just saw this with hangas on his machine, another G3. both news and imap were oftly slow. idea #1: Is poking the status / progress meter really slows us down? (do we have to cross any thread boundries to do this? perhaps the way that works on the mac is slow.) we should try it with all progress / status turned off. idea #2: filters. filters now get migrated. maybe the filter code is being executed

I like idea #1 - yes, we do cross thread boundaries. And we do it with dougt's proxy event stuff - maybe that's really slow on the Mac? That would be unfortunate. Or maybe the timer math that we do to avoid a lot of status and progress messages is broken on G3's? That would be ironic since it was written on the mac :-) I doubt it's the filter stuff - we cache the filters in memory while downloading headers.

If I resize the window, it does help performance. There is no horizontal scroll bar in the messenger pane to flash. It took only 2.49 minutes. :)

Cool, that's a new bug, and it's a browser/xpapps bug.

one more thing..and this really from left field: change your messenger start page to about:blank by default, its set to pref("mailnews.start_page.url", "http://messenger.netscape.com/bookmark/4_5/messengerstart.html"); that page is very layout unfriendly. they have a pixel for a background image. http://home.netscape.com/messenger/start/images/pixel.gif in 4.x, layout had optimizations for this. in 5.0, we don't. (ducarroz@netscape.com knows more) if for some reason, we had to repaint often, and if repainting is slow on the mac, this would slow us down. again, this is WAY left field.

Yeah, eliminating that goofy start page fixed it. Someone at Netcenter should be having their gonads removed for that start page, if you ask me.

I'll forward to Sol to see if he can get the start page background changed before our Beta.

until then, I'm going to change the mozilla start page to http://www.mozilla.org/mailnews or something. it's a commerical build problem, not a mozilla problem.

(Sol is going to get the mail start page changed. I got a msg from someone over in Netcenter on what EXACTLY needs to be changed. I am unsure of this. Can someone tell me so I can relay the that person? Thanks.)

That pages uses a 1-pixel GIF file as a background for the left-hand table cell, to get a background color. Rendering 1-pixel GIFs is extremely slow. They should simply use a BGCOLOR on the table cell instead.

so this is a layout or xpapps problem. Reassigning to jevering.

Bienvenu: don't do that. An bug exists for the slow rending of 1-pixel table background GIFs exists (bug 13375). This bug should be that the mail-news start page needs to be fixed, or changed.

Simon, putting up status messages shouldn't cause a relayout of the start page. That's a bug in layout, as far as I'm concerned. Do we need a third bug for that? Plus, how am I going to get the start page changed? That doesn't belong to me.

If you want to, reassign this to me. I've emailed Karen Fagen in Netcenter to get the start page changed. We could have this bug be to track that for the commercial builds.

When using the progress meter, a reflow happens, which exposes the fact that the start page we are using is slow to layout. Reassigning to Lisa to get the start page changed. I may be smoking crack, but I still think the progress meter shouldn't ever cause a reflow of the start page...

It's not just the progress meter. You're also adding rows to the tree view, aren't you? I see a lot of flashing in this area in my build (double-buffering off).

we can also change the default start page pref, to be something a little more friendly, until netcenter fixes their page. say the word, and I'll change it.

Just to note: the bug on the small gifs and the performance is tracked under bug http://bugzilla.mozilla.org/show_bug.cgi?id=13375 I'm wondering if we should change the default start page to something else for M11 in the commercial builds. Will check with sol and chofmann.

Per Netcenter, "The messenger start page is being revamped for 4.7 content and will be pushed 9/28" I'm going to assign this target milestone of M12 for now. So, I assume this bug will now track the removal of those items in the mail start page and any associated issues. The performance issue is covered under that separate bug report which I pasted the bug number for earlier. If I'm wrong and there is still some other bug here to track, let me know. I will open a new bug since this bug report has too many comments in it which are unrelated. Thanks.

Putting on [Perf] radar

Start page has been changed on Netcenter. Simon has confirmed that it's way faster to load on the Mac. Also, I viewed the page source. BGCOLOR is used now.

for bug #14284, I need the old page, before they fixed it. lisa, who was the netcenter person who fixed this? I'd like to get ahold of the old html.

Karen Fagen. She should know who fixed this. She was my contact at Netcenter.