User-Agent: Mozilla/5.0 (Windows; Windows NT 5.1; rv:2.0b3) Gecko/20100805 Firefox/4.0b3 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2 A lot of our users have relocated their Local folders account on Windows servers in order to have a administrative backup of their old messages. With Thunderbird 1.5 and 2.0, they were happy users but with TB3, they enter hell because the access to their messages is so slow that Thunderbird sometimes fails to display a folder content or even an attachment. GLODA is off on our TB configurations. My test mailbox is huge (1,6 Gb) and contains lot of folders. But some users experience the issue with a size of mailboxes of only 100 Mb. On the serverside (my test "server" is a PC Intel Celeron 325@2.5Ghz+512Mb of RAM + WinXP Pro SP3 with a shared folder and the client PC is a WinXP Pro SP3+2Gb of RAM), I personally view that a single request on an attachment can make up to 50% of CPU charge during few seconds. A quasi DoS attack ! The same action with a previous version (<3.0) is really imperceptible in the task manager CPU display of the server. With Process Explorer (a MS Sysinternals utility) on the client, I saw that TB3 opens only few index files and mail databases whereas older TB versions open all index files. Other issue (related or not), when I click on another folder than the current one, Thunderbird sometimes seems to do nothing (or wait something) for a long time (~20+ seconds) before to display the content of the folder. In this case, nothing is relevant on Process Explorer. Reproducible: Always Steps to Reproduce: 1.Share a folder on a Windows Server (\\server\mylocalfolders) 2. On the Windows client, connect a net drive to the shared folder (X: -> \\Server\mylocalfolders 3.With the file manager, copy the files of a complex Local Folders account on the shared folder 4.Start TB and modify the location of the Local folders account. Deactivate GLODA indexing engine. 5.Restart TB 6.Browse folders and messages, open attachments. Actual Results: Access to certains folders is slow, very slow and, in extreme cases, fails. Server is overloaded Expected Results: Access to all folders, messages and attachments is quick without overloading the server.

(In reply to comment #0) > Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.8) > Gecko/20100802 Thunderbird/3.1.2 > A lot of our users have relocated their Local folders account on Windows > servers in order to have a administrative backup of their old messages. > With Thunderbird 1.5 and 2.0, they were happy users but with TB3, > they enter hell because the access to their messages is so slow > that Thunderbird sometimes fails to display a folder content or even an attachment. Tb 3.0's slowness(Tb 3.0.4 or before) in file I/O(write) of mail folder file due to "write request per a mail data line" is already known and is already fixed problem. See bug 539389, which is fixed by Tb 3.0.5/Tb 3.1.0 or later. Problem of bug 539389 is easily be seen by Process Monitor log. Does problem what you reported to this bug still occur with Tb 3.0.5/Tb 3.1.0 or later? If so, what is difference of Process Monitor log between "Tb 3.0.0-3.0.4" and "Tb 3.0.5 or later/Tb 3.1.0 or later"? (What is difference of this bug from bug 539389?)

(In reply to comment #1) > Does problem what you reported to this bug still occur with Tb 3.0.5/Tb 3.1.0 > or later? Yes. I use a Thunderbird 3.1.2 for my tests. > If so, what is difference of Process Monitor log between "Tb 3.0.0-3.0.4" and > "Tb 3.0.5 or later/Tb 3.1.0 or later"? > (What is difference of this bug from bug 539389?) I'm going to study and compare bug 539389 to answer ASAP.

OK I read the bug 539389 and perform some additional tests with Thunderbird 1.5, 3.0.4 and 3.1.2. I'm very surprised by the results. Double-clicking on an attachment of about 2156 kb (MIME encoded, real 1621 kb) 3.0.4 .... 37 s 3.1.2 .... 10 s 1.5.0.2 .. less than 1 s ! On serverside, the CPU usage is generally long (seconds) and high (60 .. 100%) for versions 3.x.x ; and short (1 sec or less) and high (40 .. 100%) for 1.5 series. So the fix for bug 539389 has definitely permit to retrieve a user experience close to normal but yet far from 1.5 series performance. Concerning the high server cpu usage, it will deserve to be drastically reduced. I will try to have some results from a real Windows server (I test on a pseudo server under WinXP Pro). Concerning the issue related to the folder browsing, I think it comes from non indexed folders in my test mailbox, once indexed the folder browsing experience is better.

I perform others additional tests with Sysinternals Process Monitor. So I note something strange for me: - TB15 download 11x4096 bytes in a sequential manner and flush data in tmp file by 4x8192 bytes + 1x4 bytes - TB3 download 11x4096 bytes in a multi-offset sequential manner and flush data in tmp file by 1x32768 bytes I think multi-offset data download could produce high CPU usage on the server. -> See attachment 471122 [details]

Laurent do you think you could produce a patch to fix this ?

Laurent, can you describe specific things that are slow now, since your initial report seems to describe 3.0?

This Windows Server hosts mainly Local folders of certain users who use principally Thunderbird 1.5 and 2.0 series. Only one uses TB in version 3.0.6. The high cpu usage painted in orange represents this user double-clicking on a simple attachment. Size of attachment isn't enormous as our messaging server limits the message size at 4 mb. This graph confirms the same behavior I see on my test server.

(In reply to comment #6) > Laurent do you think you could produce a patch to fix this ? Sorry, I couldn't produce a patch because I'm not a developer (just an IT crew member in charge of studying Thunderbird) and my organization doesn't have any development resource with knowledge in Mozilla coding. :(

Hi David (In reply to comment #7) > Laurent, can you describe specific things that are slow now, since your initial > report seems to describe 3.0? My initial report refers to a version 3.1.2 not a 3.0. But my TB3 test users work with a 3.0.* (currently a version 3.0.6). After fixing bug 539389 in 3.0.5, the point of view of these users hadn't changed: "Access to Local folders is always very slow, a real nightmare" (workaround: they copy message into a folder of their mailbox to access quickly to an attachment) To this slowness feeling from my test users, I add the potential serverside issue I found. I have to say that bug is a real problem for us because we had planned to start deploying Thunderbird 3.*.* early in september. In fact the deployment is now postponed.

I can look into the opening of attachments issue...

(In reply to comment #11) > I can look into the opening of attachments issue... Yes, it would be very appreciated that you spend some time to have a look on this issue. Thanks. :)

Concerning the folder browsing issue I mentioned in comment #3, I now note that (with TB 3.1.3, but perhaps also with previous versions) when I click on folders, TB seems frozen (UI is totally unclikable) for seconds. On server side, I saw the index file related to the folder has a size of 0 byte. When TB returns to a normal usability state, the list of messages appears and index file size is > 0 byte. But when I open the error console, it contains this: Error : uncaught exception: [Exception... "Component returned failure code: 0x80550006 [nsIMsgFolder.msgDatabase]" nsresult: "0x80550006 (<unknown>)" location: "JS frame :: chrome://messenger/content/mailWidgets.xml :: parseFolder :: line 2141" data: no] Does it deserve to file a new bug or not ? Thanks.

Laurent, could you do the measurements in https://bugzilla.mozilla.org/show_bug.cgi?id=589798#c3 for 3.0.5? Is this bug open essentially because 3.1 takes 10 seconds vs 1.5 taking less than 1 second? Or does 3.0.5 take 37 seconds? 3.0.5 doesn't have all the fixes that 3.1.x does. Re comment 13, somehow your .msf file is either getting corrupted (e.g., timestamp is changing dramatically on the corresponding folder, by more than an hour, or the file size is wrong). Either will cause us to rebuld the .msf file. Do you have any particular steps that cause us to rebuild the summary file?

Laurent, what kind of attachment are you opening? This is from a local folder, not an imap folder configured for offline use, right? I see 32Kb reads from the local mail folder and 32kb writes to the tmp file, when opening a .png attachment from a local mailbox. I also don't know what you mean by multi-offset - I'm not seeing the systinternals process monitor say anything about multi-offset writes.

3.1.3 does seem to do 4K reads, but trunk builds are doing 32K reads. We're telling the pipe code to use 8 4k segments in both trunk and 3.1 code. I'm not sure why that would result in different read counts.

Laurent, in what areas do you seen improvement when using v3.1.5 now that Bug 599119 is fixed? and what areas are still bad? (and comment 15)

First of all, I must apology to not feed this bug since a month. So, since the version 3.1.3, the issue concerning a slow access to attachment was dramatically reduced. It's definitely not as speedy as in 1.5 or 2 series but ... A/ attachment viewing is now useful (at least in users' feeling) because attachment is now always opened, and in an acceptable time lapse; and B/ the attachment's opening now requires less CPU usage & time on the Windows server. Nonetheless, in the current version (3.1.5), it remains the issue concerning the self re-indexing process of a folder which always freezes TB during dozens of seconds without any relation with the size of the message database file. For example, I experienced the case a 37 kb file re-indexing which has needed almost 40 seconds and during this time, TB seemed frozen for the user. Concerning 3.0.x series, nothing has been fixed related to this present bug.

(In reply to comment #18) > First of all, I must apology to not feed this bug since a month. > > So, since the version 3.1.3, the issue concerning a slow access to attachment > was dramatically reduced. It's definitely not as speedy as in 1.5 or 2 series > but ... > > A/ attachment viewing is now useful (at least in users' feeling) because > attachment is now always opened, and in an acceptable time lapse; and > > B/ the attachment's opening now requires less CPU usage & time on the Windows > server. thanks for the feedback. adjusting severity to normal. do you see your attachment issue described in any of the bugs in this list? .. https://bugzilla.mozilla.org/buglist.cgi?type1-0-0=substring&keywords=perf&keywords_type=allwords&field0-0-0=short_desc&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&bug_severity=minor&type0-0-1=substring&field0-0-1=component&type1-0-1=allwordssubstr&resolution=---&classification=Client%20Software&classification=Components&query_format=advanced&value0-0-1=attach&type0-0-0=anywordssubstr&value0-0-0=attach&field1-0-0=short_desc&product=MailNews%20Core&product=Thunderbird&field1-0-1=short_desc also, please reply about comment 15 > Nonetheless, in the current version (3.1.5), it remains the issue concerning > the self re-indexing process of a folder which always freezes TB during dozens > of seconds without any relation with the size of the message database file. I suspect this is covered by other bug(s). bienvenu may be able to suggest one > Concerning 3.0.x series, nothing has been fixed related to this present bug. I wouldn't expect v3.0 to be changed. Main focus is currently on v3.1 and its successor

Please consider making the network and disk I/O sizes adjustable in the 'registry' (about:config settings)... Ideal size for my local net is about 16MB read and write. 32K is VERY slow -- 4K is even slower. The problem described happens when you talk to a local IMAP server (which also has small I/O sizes). Given Tbird3.0's default of downloading everyone's IMAP folders from their local network server to their 'home dir' (also on the local network server for the person in this bug), I can't imagine Tbird being usable. In my case, My local home dir is in my 'roaming dir', which meant that tBird3.0 downloaded almost 2GB of indexed IMAP folders onto my local home dir, which then added 30 minutes to my logout time (and about 3minutes to login). Took a while to catch the reason, but reverting to 2.0 solved the problem.

LA Walsh, Laurent, this is still a problem in *current* versions, eg version 8 and newer, correct?

is the .msf rebuild issue possibly Bug 232047?? Frequent need to rebuild summary file(.msf) when mail folder is located on a different OS's sharing hard disk or network drive(timestamp inconsistency) (In reply to L A Walsh from comment #20) > Please consider making the network and disk I/O sizes adjustable in the > 'registry' (about:config settings)... this is wada's Bug 558528 - Consider larger buffer size in file I/O by Mail&News. Use of Necko's buffer size, buffer size user can easily customize, . => severity=major based on comment 3 measurements (and assuming they are still the same)

(In reply to Wayne Mery (:wsmwk) from comment #21) > LA Walsh, Laurent, this is still a problem in *current* versions, eg version > 8 and newer, correct?

(In reply to Wayne Mery (:wsmwk) from comment #21) > LA Walsh, Laurent, this is still a problem in *current* versions, eg version > 8 and newer, correct? Concerning the issue with message bases on Windows shared folders, Thunderbird 17 doesn't experience any slow down. Concerning the re-indexing issue which froze Thunderbird 3.*, I currently haven't time to make an in depth test but our internal beta-testers don't experience this kind of freeze with the lastest Thunderbird version (17).

Laurent, thanks for the update. WFM per reporter