User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0 (Beta/Release) Build ID: 20131112160018 Steps to reproduce: Nothing, except upgrade from TB v 17 to v24. Actual results: I have to wait about 15 seconds when opening TB while it counts the number of e-mails. I'd like an option to opt in or opt out of this and I'll take the opt out. I have 1,355 e-mails saved for posterity in Local Folders and that count remains basically the same, why recount it every time? And I don't care how many there are, I only care that they're still there if I need to access them. Expected results: When introducing a new 'feature', it should be made optional, not simply forced on everyone because some end user or developer thinks that it's a 'good' feature. I have 6 accounts plus Local Folders. Make the opt in or opt out a config option on each 'account' including 'Local Folders'. JWC

What do you mean by "counting"? Could you show a screenshot of what you mean? Also, have you tried starting in Safe Mode? For what it's worth, I have a folder with over 6,500 messages on my Windows machine and starting up Thunderbird in that folder takes about a second. After startup, I can switch to/from that folder almost instantaneously. On Linux, I have a folder with about 20k messages and startup is pretty fast there too.

Counting is adding up the number of e-mails in a folder and displaying the sum to the right of the folder name. I'm not interested in how fast your machine is after you start up TB, my issue occurs when I start TB by clicking on the TB icon in my task bar and then click on the [Work Online] button, at that point, the 'main' window opens and sits there for exactly 19 seconds before I can begin to enter the passwords for my 6 accounts. Since I upgraded from v 17 to 24 without any of the intermediate versions, I don't know when this 'feature' was introduced. I reverted to v 17 and the problem wasn't there, but TB automatically upgraded to v 24 again and the problem is back. Of course the new counts were the only obvious change, to me anyway, so I'm blaming that as the cause. What else could be the culprit? Not the speed of my system or anything else that I can think of... Jim C. I have only SSDs in my system, TB 'files' are on a SATA 2 SSD, the programs are on a SATA 3 SSD. The Intel quad core cpu runs at 3.4GHz.

Based on your screenshot, I'm guessing that each of those folders has a bunch of subfolders, and Thunderbird is recursively fetching the unread counts for all of them. CCing some folks who might want to look into this.

That's a very good guess!. Yes, I read the e-mails then Mark as Unread so that I can see a total of what's in a specific, low level folder, by vendor or type or.... From this I assume that I could go through the folders and Mark them as Read and the sums wouldn't 'roll up' to the higher level folders, but, TB would still be looking through all the folders and that's what would likely take the extra 19 seconds, not the actual 'summing up', so I'd be no further ahead. If the 'roll up' of totals was optional, I could avoid 'waiting 19 seconds' most of the time but if I ever wanted the high levels totals I could 'opt in' for just that time, which would make sense to me. Tx Jim

If all you really want is a count of the messages in each folder, you might try installing the add-on "Extra Folder Columns", which gives you the total number of messages in the folder without having to mark all of them as unread: https://addons.mozilla.org/en-us/thunderbird/addon/extra-folder-columns/

That said, if we can't make this feature work with decent performance when there are a lot of unread messages in nested folders, it's also reasonable to add a configuration option to disable it.

Well I just learned that I need to submit comments and attachments as a two step process not one. I installed "Extra Folder Columns", it's very nice, but TB still takes 19 seconds from clicking [Work Online] to the first password prompt. That has to be related to the number of nested folders, particularly in Local Folders. I seldom access Local Folders so rather than count everything at the very beginning, could you count only when a folder containing e-mails is visible. See the attachment screen shot. The biggest individual nested folder is "@ auto confirm forms". If that's the only folder that I'm interested in, I wouldn't open the "big" folder and the nested folders within "big" would not need to be counted... etc. Counting 288 e-mails in "@ auto confirm forms" would be so quick that the time taken would not even be noticed. Tx Jim C

Yes, this was changed in bug 235956. There are also more people in bug 918962 wanting to disable the recursive count. As there are different reasons to not want it (performance or workflow) we can base it on a pref. Can we expose it in the Options dialog?

1) There are two types/kinds of folders: - those that automatically receive new e-mails, e.g. "Inbox", "Junk" or other folders named in a Rule, and - those that simply *store* e-mails which may be marked as read or unread, for future reference. 2) When an e-mail has been newly received into, moved from/into or deleted from a folder, or had it's state toggled between read and unread, the counts for those folders have always been and are currently adjusted/recalculated for *just* those folders, 'on-the-fly', without performing a recursive count on the *entire* database. 3) In general, it's not important to have the recursive count totals for child folders in the parent folder if the child folders are *not* currently displayed and especially if the counts seldom change. An exception would be when a new e-mail is automatically received and inserted into a sub folder, but I'd think that those folders wouldn't be 6 levels deep...... 4) The highest level folders(?) are the Local Folder and optionally one or more Account folder(s), each containing the standard Inbox, Sent, Trash... etc. folders, plus additional sub(child) folders added by users. Most e-mail is received into the Inbox (and possibly Junk), is 'read' and then deleted or moved elsewhere according to a user's wish to group 'like' e-mails in an organized and possibly complex folder structure. Within each 'final resting place' folder, a user may wish to 'count' individual e-mails types, such as Order Confirmations, while not counting subsequent Shipping Notifications. This has been easy (for me) to do by Marking the Order Confirmations as unread and leaving the Shipping Notifications as read. Thus, within a Vendor folder, a count of Total Orders would be shown as 'unread'. These totals need not be recalculated every time TB is started, but only when a change occurs. Therefore I suggest that: (A) the current, automatic '*Recursive* Count at Start Up' be eliminated and that an option titled [Recalculate Totals] be added to: 1) the left click menu for File, just below [Compact Folders] (this would be what is currently done at Start Up), and 2) the right click menu for an Account or Local Folders, located just above or below [Settings] and 3) the right click menu for a Folder, located beneath [Compact] and above [Mark Folder Read]. This option would initiate a Recursive Count for the entire database, a single account or a single folder only. This would not require modification of the Folder Properties records or any other Configuration record(s). (B) As each 'level' of sub folders is 'displayed', counts of e-mails in just that level would be automatically recalculated, but *not* recursively for lower level sub folders. (C) as e-mails are added to or deleted from a Folder, the counts are recalculated for that level and all higher levels (parents). But, I'll be happy with whatever solution you folks feel will be easy to implement as long as I don't have to wait 19 seconds... :) Jim C

Is anybody able to test this patch? When applied the experimental pref is visible on the Options->Advanced->Reading&Display tab. Please tell me if this fixes the perf problem or restores back the behaviour of TB17 (not accumulating unread msg count into the parent).

I'll be glad to do the test but [ul]I need to know how to obtain and apply the patch[/ul]. (I'll backup first of course... :) ) J

OK, I opened the patch attachment, did Select All and copy/pasted to a .txt file. What do I do now to include it in TB for testing? J

If you do can not build TB from source, you can find omni.ja file in the TB installation folder. Back it up and open it, it is a zip file. Find the files mentioned in the patch and do the changes described in the patch. line preceded with "-" (minus) are to be removed and lines preceded with "+" are to be added. Then delete startupCache folder in TB profile and start TB. If you do not succeed in this manual patching, we can produce you a build with this patch, however it will be from the current experimental TB28 so you may be more reluctant to try it out on your real data :)

I tried to use 7Zip to unpack omni.ja but was told it's not a valid archive file. I can try the TB28 safely, I keep a backup of my C: drive which is easily restored and I keep my TB profiles on a separate second drive which is also backed up to a third drive, again easily restored. I'll wait for your instructions re: TB28 J

Aryx, could you please make us a trybuild with this patch?

Pushed: https://tbpl.mozilla.org/?tree=Thunderbird-Try&showall=1&rev=4c01a30a1dd2 You will get orange X, see Thunderbird-Trunk.

Oh, orange X from other patch, not mine. Thanks. APraxis, please get the builds from here (I assume you are on Windows): http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/archaeopteryx@coole-files.de-4c01a30a1dd2/try-comm-central-win32/ You can even use the .zip archive, extract it anywhere and run from there (clicking thunderbird.exe). In that way you do not need to touch your normal installation of TB 24. Of course, backup your profile and email data.

Sorry, that didn't help. I still have the long delay between clicking [Work Online] and when the first password prompt appears. I backed up my c drive and data drive (has TB profile & database). I installed the file thunderbird-28.0a1.en-US.win32.installer.exe and it found my e-mail database automatically. When I responded that I wanted to run 'Daily' it opened TB, clicked [Work Online', no delay until I could enter passwords *but* there were *no* counts. I closed TB and re-opened it and that's when I saw the delay, and then the counts *were* displayed. I didn't change any config. My e-mail database wasn't harmed (not yet anyway... :) ) Is there anything that you'd like me to do? (be specific please, I'm not a good guesser). Jim

stopped and started TB and the delay was still there, haven't done a restart yet, will do now.

After rebooting, I opened TB and it was 'quick', no delay, in a sense like it was the first time that I tried just after the install. Closed and re-opened it and the delay was there, almost as though it's quick the first time it runs after booting (or installing) and then if it's closed and re-opened, it does the recursive counting. Unless there's omething else in config that I need to set.... please advise. Jim

It would be strange if you get different times at first TB run and any subsequent runs. That must be caused by something else. I am not sure why you toggle Online mode and why you expect immediate password prompt. Also when there is the delay see into tools->activity manager.

I just rebooted into the copy of the OS containing TB v 24.1.1 and the password prompts were up quickly, no delay. I closed and re-opened and the delay was back again. The Activity Manager log shows only that TB attempted to download e-mails for my 6 accounts, one had your e-mail and the rest had none. There were no 'error' messages. Of course in the V 24 config, there's no option to "Accumulate folder statistics....." So definitely strange.

Well, for starters, I strongly disagree with APraxis' claim that "When introducing a new 'feature', it should be made optional". Ideally, we should make this more performant (or better yet, figure out what's _actually taking such a long time, and fix that). Perhaps we could store number of unread messages somewhere, and then recalculate it on idle, or in parts, so that it doesn't block other processing? Having said that, I am _slightly_ surprised that we show an unread count on a folder that's not the number of unread messages in that folder, and I do notice that bug 235956 never got a ui-review… So I wouldn't be opposed to reverting that patch, and perhaps indicating some other way that it's the subfolders which have unread messages. (i.e. adding "(0+)" when there are only unread messages in subfolders, or "(3+)" when there are three unread messages in the folder, and more in subfolders.)

(In reply to Blake Winton (:bwinton) from comment #25) > Ideally, we should make this more performant (or better yet, figure out > what's _actually taking such a long time, and fix that). Perhaps we could > store number of unread messages somewhere, and then recalculate it on idle, > or in parts, so that it doesn't block other processing? Yes, it is yet unsure what is actually so slow in his setup. The patch didn't help. However, the patch solves the preference of some people that do not want the unread count to include sunfolders (regardless of speed issues). > So I wouldn't be opposed to > reverting that patch, and perhaps indicating some other way that it's the > subfolders which have unread messages. (i.e. adding "(0+)" when there are > only unread messages in subfolders, or "(3+)" when there are three unread > messages in the folder, and more in subfolders.) Exactly, I have made this distinction in my update to the Extra folder columns extension, but it is hiding somewhere in standard8's mailbox for a year now ;) But if we have the patch here already, I could add this distinction character too. The question is if the exposure of the pref in the Preferences it fine, or it will be just a pref in about:config.

(In reply to :aceman from comment #26) > (In reply to Blake Winton (:bwinton) from comment #25) > > Ideally, we should make this more performant (or better yet, figure out > > what's _actually taking such a long time, and fix that). Perhaps we could > > store number of unread messages somewhere, and then recalculate it on idle, > > or in parts, so that it doesn't block other processing? > Yes, it is yet unsure what is actually so slow in his setup. The patch > didn't help. > > However, the patch solves the preference of some people that do not want the > unread count to include subfolders (regardless of speed issues). I'll admit it. I'm a little uncomfortable making random changes to fix a bug that don't actually fix the bug. ;) > > So I wouldn't be opposed to > > reverting that patch, and perhaps indicating some other way that it's the > > subfolders which have unread messages. (i.e. adding "(0+)" when there are > > only unread messages in subfolders, or "(3+)" when there are three unread > > messages in the folder, and more in subfolders.) > Exactly, I have made this distinction in my update to the Extra folder > columns extension, but it is hiding somewhere in standard8's mailbox for a > year now ;) > > But if we have the patch here already, I could add this distinction > character too. The question is if the exposure of the pref in the > Preferences it fine, or it will be just a pref in about:config. It feels to me more like an about:config kinda thing, if we want to land it at all, which I'm not entirely convinced of. Thanks, Blake.

(In reply to Blake Winton (:bwinton) from comment #27) > > However, the patch solves the preference of some people that do not want the > > unread count to include subfolders (regardless of speed issues). > > I'll admit it. I'm a little uncomfortable making random changes to fix a > bug that don't actually fix the bug. ;) Well, it does fix the summary of this bug exactly :)

(In reply to :aceman from comment #28) > (In reply to Blake Winton (:bwinton) from comment #27) > > > However, the patch solves the preference of some people that do not want the > > > unread count to include subfolders (regardless of speed issues). > > > > I'll admit it. I'm a little uncomfortable making random changes to fix a > > bug that don't actually fix the bug. ;) > > Well, it does fix the summary of this bug exactly :) I'm 100% with Blake. bug 235956 is a performance regression which wouldn't have been accepted if we had perf tests as good as firefox. Plus on principle we try not to take new preferences. I haven't read all of the comments here but adding a pref isn't a useful fix if for no other reason we shouldn't make users flip prefs to fix issues that they most likely won't be able to diagnose in the first place - all they will know is things are slow. IMO the only acceptable approaches are either to 1. back out bug 235956 2. backend fix the performance issue Furthermore, I think it's plausible this is likely one source, perhaps even the main source, of numerous and persistent user reports of bad performance and unresponsive script during startup.

(In reply to Wayne Mery (:wsmwk) from comment #29) > IMO the only acceptable approaches are either to > 1. back out bug 235956 > 2. backend fix the performance issue > > Furthermore, I think it's plausible this is likely one source, perhaps even > the main source, of numerous and persistent user reports of bad performance > and unresponsive script during startup. And ASAP. I'll be happy to cite the user reports and file a bug report if APraxis prefers not to morph this bug report to one of the above recommendations.

Yes, I agree that we should just back that feature out. We can always re-land it once we have the kinks worked out (performance, not counting messages in Junk folders, not doing this in the "Favorites" folders view, etc).

Does "backing out" in the previous comment refer to comm-esr24 or in general? If the latter, it would be hard to perform further improvements and debugging of that code. The other option is to introduce a hidden pref as suggested by aceman and keep it off by default on esr and maybe beta, thus leaving it intact for nightly testers (or let them switch it on manually if so desired and leave the pref false on all channels). That should be acceptable practice. The question for keeping a pref beyond that phase should be whether or not there is sufficient user interest to toggle that feature once usability issues are fixed (i.e., other than a workaround for performance). I see that this may be debatable.

(In reply to rsx11m from comment #32) > Does "backing out" in the previous comment refer to comm-esr24 or in > general? If the latter, it would be hard to perform further improvements and > debugging of that code. Everywhere. It's not hard to improve that code even with it being backed out. The developer of the patch just needs to test it and get proper feedback from UX folks. At the very minimum, it shouldn't re-land until it performs well. > The other option is to introduce a hidden pref as suggested by aceman and > keep it off by default on esr and maybe beta, thus leaving it intact for > nightly testers (or let them switch it on manually if so desired and leave > the pref false on all channels). That should be acceptable practice. Preferences delenda est. This feature is nowhere near so complex that we need to land it (or leave it in) preffed off. Firefox features that land with a pref to toggle them are usually pretty big rewrites, not a one-line change. > The question for keeping a pref beyond that phase should be whether or not > there is sufficient user interest to toggle that feature once usability > issues are fixed (i.e., other than a workaround for performance). I see that > this may be debatable. Assuming we make this feature work correctly (and that goes beyond just performance issues), I don't think we'd want a pref (although we could make it easy to modify via an add-on). I for one have no problems with landing a change that not everyone likes provided it doesn't actually *break* anything and makes life easier for the majority of users. We shouldn't be so beholden to the past that we can't change any aspect of the interface without adding prefs to go back to the old way. That's a nightmare for maintenance (and support).

(In reply to Jim Porter (:squib) from comment #33) > (In reply to rsx11m from comment #32) > > Does "backing out" in the previous comment refer to comm-esr24 or in > > general? If the latter, it would be hard to perform further improvements and > > debugging of that code. > > Everywhere. It's not hard to improve that code even with it being backed > out. The developer of the patch just needs to test it and get proper > feedback from UX folks. At the very minimum, it shouldn't re-land until it > performs well. I was more thinking about the testing community, if broader feedback is desired, but points taken. And yes, actually looking at the attachment 745677 [details] [diff] [review] making that count available, it's definitely not a complex change if that's the only patch that will have to be reverted.

(In reply to Wayne Mery (:wsmwk) from comment #29) > bug 235956 is a performance regression which wouldn't have been accepted if > we had perf tests as good as firefox. Plus on principle we try not to take > new preferences. > > IMO the only acceptable approaches are either to > 1. back out bug 235956 > 2. backend fix the performance issue > > Furthermore, I think it's plausible this is likely one source, perhaps even > the main source, of numerous and persistent user reports of bad performance > and unresponsive script during startup. I do not agree here. My patch 8336376 here basically reverts bug 235956 if the pref is toggled, but the testers didn't report it being any perf improvement. Technically it must be, but it may be too small to be noticeable. So there is no proof YET that bug 235956 by itself causes any perf problem.

(In reply to :aceman from comment #35) > (In reply to Wayne Mery (:wsmwk) from comment #29) > > ... > > Furthermore, I think it's plausible this is likely one source, perhaps even > > the main source, of numerous and persistent user reports of bad performance > > and unresponsive script during startup. > > I do not agree here. My patch 8336376 here basically reverts bug 235956 if > the pref is toggled, but the testers didn't report it being any perf > improvement. Technically it must be, but it may be too small to be > noticeable. So there is no proof YET that bug 235956 by itself causes any > perf problem. Good point. I assumed the perf issue and it's cause were both confirmed. Sounds like a regression range hunt is needed. I am afk so can someone help apraxis in using the nightlies? The cited regression landed 2013-05-28

can someone help apraxis in using the nightlies? Hi, tell me how and I'll help Apraxis

Wayne Having read the links that you provided, I think that you want to know that: I have my TB profile on a local drive G (no, not C nor a network drive) but both are SATA3 SSDs I have 6 POP accounts, no IMAP accounts. I have a 'complex' structure for e-mails that I want to keep 'forever' I see a "Mozilla Thunderbird (not responding)" in the window header during the 19 seconds delay IF I try to click on an Inbox, if i do nothing that doesn't appear but still takes 19 seconds. Jim Clements The only log file that I see is the test pilot log which shows: 2014-01-06 07:41:42 TestPilot.Setup TRACE TestPilotSetup.globalStartup was called. 2014-01-06 07:41:42 TestPilot.Setup TRACE Setting interval for showing reminders... 2014-01-06 07:41:42 TestPilot.Setup TRACE Now requiring remote experiment loader: 2014-01-06 07:41:42 TestPilot.Setup TRACE Now instantiating remoteExperimentLoader: 2014-01-06 07:41:42 TestPilot.Loader TRACE About to instantiate jar store. 2014-01-06 07:41:42 TestPilot.Loader TRACE About to instantiate cuddlefish loader. 2014-01-06 07:41:42 TestPilot.Loader TRACE Done instantiating remoteExperimentLoader. 2014-01-06 07:41:42 TestPilot.Loader INFO Unloading everything to prepare to check for updates. 2014-01-06 07:41:42 TestPilot.Loader INFO Setting if-modified-since header to Fri, 11 May 2012 18:51:08 GMT 2014-01-06 07:41:42 TestPilot.Loader INFO Using binary mode to download jar file. 2014-01-06 07:41:48 TestPilot.Loader INFO Verifying SSL channel security info before download... 2014-01-06 07:41:48 TestPilot.Loader WARN Failing security check: Error: TypeError: cert.verifyForUsage is not a function 2014-01-06 07:41:48 TestPilot.Loader INFO Could not download index.json, using cached version. 2014-01-06 07:41:48 TestPilot.Setup INFO Getting updated experiments... Success? true 2014-01-06 07:41:48 TestPilot.Loader TRACE GetExperiments called. 2014-01-06 07:41:48 TestPilot.Setup TRACE I'm in the callback from checkForTasks. 2014-01-06 07:41:48 TestPilot.Setup TRACE Testpilot startup complete. 2014-01-06 07:44:42 TestPilot.Setup TRACE Doing reminder after startup... 2014-01-06 07:48:26 TestPilot.Setup TRACE Global shutdown. Unregistering everything. 2014-01-06 07:48:26 TestPilot.Setup TRACE Done unregistering everything.

(In reply to :aceman from comment #35) > (In reply to Wayne Mery (:wsmwk) from comment #29) > > bug 235956 is a performance regression which wouldn't have been accepted if > > we had perf tests as good as firefox. Plus on principle we try not to take > > new preferences. > > > > IMO the only acceptable approaches are either to > > 1. back out bug 235956 > > 2. backend fix the performance issue > > > > Furthermore, I think it's plausible this is likely one source, perhaps even > > the main source, of numerous and persistent user reports of bad performance > > and unresponsive script during startup. > > I do not agree here. My patch 8336376 here basically reverts bug 235956 if > the pref is toggled, but the testers didn't report it being any perf > improvement. Technically it must be, but it may be too small to be > noticeable. So there is no proof YET that bug 235956 by itself causes any > perf problem. Perhaps the Testers didn't have the same number of folders, sub folders and total e-mails that I (and others) have. Therefore they wouldn't experience the same issue. That's a bit like a 6 foot tall person complaining that they bump their head going through a Hobbit's doorway and all the Hobbits say "Gee, I went through the doorway several times and don't see any problem".... (meant to add a bit of humor, I hope it gets the point across though). You need some "real world" testers it seems, but no one has attempted to instruct me on how to get "nightlys". I'd think that shouldn't be too difficult and would save your time discussing whether there is or isn't a plausible explanation, maybe..... But you should probably sort out a schedule for someone like me to only test when required, i.e. after you've done your testing and think that you have a finished product and want confirmation from folks with that "real world" set of conditions. Or, you can ignore everything except for discussions and hope 'we' give up 'complaining'. J

APraxis, I think you already conducted the test in comment 18 and 19 so we assume the "Testers" group includes you, with your number of folders. Did anything new crop up to revert this assumption?

Hi Aceman, I guess that I've expected more opportunity to test a fix if one has actually been made and needs testing. My post today was due to an e-mail recording that someone had dropped from the cc list. If you think that a fix is ready for testing, please point me to the download and I'll be most happy to give it a spin. It just seems that this has been dragging on with no real progress. I also realize that you're now likely working on v 29 (was v 28 when I last tested), and I see that in itself is a problem. I just updated from v 24.1.1 to v 24.2.0, the problem is there, and the fix, if found, should be applied to 'current' versions and not just held until v 29(?) is ready.

(In reply to :aceman from comment #23) > I am not sure why *after* > you toggle Online mode *that* you expect *immediate* password prompt. I thought that was obvious when I started this thread, I don't want to wait 19 seconds while TB is *unresponsive* before it prompts for passwords and checks for new e-mails.

I confirm that v 24.3 has returned the program to 'normal' (by my definition). The pwd prompts are displayed instantly after selecting "Work ONline". Thanks Aceman,I'm going to change the Status to Resolved, Works for me Jim C.

Hah, it only worked the first time after updating to v 24.3, then, after shutting down the PC and rebooting and opening TB the delay is still there. What a disappointment! If this is going to be fixed, which version will contain the 'fix'? If it's v 28 or v 29 (or v30) that's a long way off, given that this started with v24.0 in Nov. 2013. I'd hope that it would be made available in v 24.4 (a few month's from now). J

There is no fix yet as seen in the discussion below.

I don't see any discussions *below*. Perhaps you meant *above*?

I hope that you folks follow some basic programming rules and therefore assume that you have set up in the following manner: 1) User clicks on the icon to open TB 2) TB begins by prompting for Work Online or Offline 3) User selects one of the two options, there needn't be an error check here. 4) TB opens the main window showing the accounts and folders opened when TB was closed previously. 5) TB has one line of code that calls a subroutine that counts mail in ALL folders, including folders that are not currently visible, which takes 19 seconds in my case. 6) TB has one line of code that calls a subroutine to check for e-mail for all accounts and if new mail is found, it adds to the counters for unread mail for the appropriate folder 7) TB goes into a 'wait state' waiting for the User to select 'something' to do and does 'that', e.g. 7a) calls a subroutine that expands either an account or folder (if the 'expand' icon is clicked) AND counts e-mails in that/those folder(s) but NOT in sub folders that are not displayed yet. 7b) display contents of a selected e-mail, 7c) write a new e-mail, etc. etc. etc. Item 5) is where the problem lies, it should call a subroutine that will then recursively run the same subroutine that is called by 7a) and count mail for folders that are about to be currently displayed, which would be very quick. (recursive counting is required where there were multiple levels of sub folders displayed when TB was last closed). Item 6) may need to be modified so that it adds to the unread count for a folder only IF the folder is currently displayed but need not if the (sub) folder is not currently displayed. Obviously as the expand icon is clicked to cause sub folders to be displayed, the mail for those folders would be counted. If sub folders are not currently displayed then no time is wasted counting the contents. The subroutine to count mail should only count the lowest level of (sub) folders that are about to be displayed by that expand. Of course moving mail from one folder to another has both folders displayed, even at different levels, but only those two folders would be recounted or updated whichever is easiest, i.e. one is decreased, the other increased. And it seems that currently step 5) is skipped when a version update to TB has just occurred, which is weird IMHO, but it's run after closing and opening TB. I would think that the above would not require a massive rewrite of code, it should be quite simple to implement in all versions of TB, including v 24.4. J

(In reply to APraxis from comment #47) > I would think that the above would not require a massive rewrite of code, it > should be quite simple to implement in all versions of TB, including v 24.4. Excellent! I look forward to ui-reviewing your patch! How long do you think it'll take you? Thanks, Blake.

(In reply to Blake Winton (:bwinton) from comment #48) > (In reply to APraxis from comment #47) > > I would think that the above would not require a massive rewrite of code, it > > should be quite simple to implement in all versions of TB, including v 24.4. > > Excellent! I look forward to ui-reviewing your patch! How long do you > think it'll take you? > > Thanks, > Blake. Approximately forever, I don't know the programming language that you use and won't invest the time to learn it, although I don't think that you really expected me to do that anyway. :) But for someone skilled in the art and currently an accredited dev, it should take about 10 minutes to 1 hour to develop the code for the first version and about 15 minutes for each of the remaining versions. In fact, if your code is properly documented, you could look for the reason that the 5) does not occur the first time TB is opened after a version update and implement that solution 'all the time' when opening TB. That might be a single line of code to disable leaving the rest intact as simply 'bloat'. And that wouldn't require extensive testing. Your welcome for the help, anytime, Jim

http://forums.mozillazine.org/viewtopic.php?f=39&t=2859041&start=15 may be another example. APraxis can you obtain and post a URL which captures the bad performance? Please see https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Reporting_a_Thunderbird_Performance_Problem_with_G

(In reply to Wayne Mery (:wsmwk) from comment #50) > http://forums.mozillazine.org/viewtopic.php?f=39&t=2859041&start=15 may be > another example. > > APraxis can you obtain and post a URL which captures the bad performance? > Please see > https://developer.mozilla.org/en-US/docs/Mozilla/Performance/ > Reporting_a_Thunderbird_Performance_Problem_with_G Sorry Wayne but I'm happy with V 17 and have no need or interest in installing a "Daily...". I did read up on the Geko thing and don't see how I could use it. To recap, I click on the TB icon to start the app, I click on the "Work online" button and then wait for 19-20 seconds before TB is "available" and assume it has to be "available" before I can open the Geko and click Start. That's too late since it would happen after the delay. Of course I could be wrong, but that would be because the instructions for Geko are incomplete in terms of timing. And, I'm not sure if I'd want to have the massive log file indicated in the Geko write-up. I've watched with interest and see that V 17 was the last one without the problem but I don't automatically update so when I did update it was to V24. Others have reported the same issue, immediately after an update TB is "available" almost instantly but subsequently has the "lag". Maybe those folks would be willing to do your Geko thing, but I'm not, not anymore anyway since this has dragged on for such a long time. J

I won't be profiling because I don't see the problem. So if you are not willing to profile then it will be helpful if you direct those users to this bug report and ask them to profile. TIA > I did read up on the Geko thing and don't see how I could use it. I think my own mother could follow this directions, and she's no geek. If there's something unclear, please let me know and I will personally fix the documentation. And I don't know where you get the impression the log file takes a lot of disk space - it just uses memory.

what can we make of http://people.mozilla.org/~bgirard/cleopatra/?1418080121212#report=6da9b2c8c28303a50f5293a1973e03ee3dcd1e90 which is from http://forums.mozillazine.org/viewtopic.php?p=13918433#p13918433

FYI. attachment 8527814 [details] attached to Bug 1103333 Commen #17 is modified Extra folder columns extension provided by :aceman. See Bug 1103333 Comment #25 for how to use it. modified Extra folder columns extension changes folder pane colimn layout : Unread, Total, Size is added. If Unread col is hidden, NumUnread is shown at right of folder name as done by Tb. If all colimns is hidden, similar display as Thunderbird is obtained. If pref extensions.extra-cols@jminta_gmail.com.sumSubfolders=true, and if folder is collapsed, show accumulated count. If accumulated count is shown, prefix of "*" is added to number to indicate "accumulated number". If pref extensions.extra-cols@jminta_gmail.com.sumSubfolders=false, don't show accumulated count any time. So, extensions.extra-cols@jminta_gmail.com.sumSubfolders=true/false makes accumulated count feature optional. "Extra folder column" has come back with changing "accumulated number" optional. :-) This version is for experimental purpose, but it works pretty well in my Tb 31.3.0 with not-so-many folders/not-so-deep folders.

FYI. modified Extra folder columns extension doesn't show accumulated number if View/Folder/Unified. i.e. Bug 917886 is fixed by it.

I provided the profile information and can reproduce the problem, so happy to test out this fix. Let me know if there is a download that I can use to test the fix for you. I'll compare with current version that I'm running (Thunderbird 31.3.0) where problem still exists, and will post results of my test. Can even use Geko to create and upload a profile from after the change if you'd like. Ping me if you have something for me to test.

In the profile that Wayne linked to in comment 53, what I'm seeing is a pretty decent chunk of time being spent calling mozilla::a11y::Accessible::InvalidateChildrenGroupInfo again and again. That sounds to me like we're rapidly updating the DOM again and again and again, when we should probably just do some calculation and update the DOM at the end. That's my read on it, anyhow.

Mike, I'll be happy to test an updated version to verify the change that you recommend solves the problem. I can reproduce the problem easily. Barry

Barry, Apraxis, What happens if you set hidden preference accessibility.force_disabled = 1 via tools | options | advanced | general | config editor ? Bug 1058423 indicates this will help with an accessibility related performance issue.

Hi Wayne, That worked nicely. I set to 1 and restarted Thunderbird. When I went to file an email and open the list box, it opened within 1-2 seconds, dramatically faster than before. Restarted Thunderbird and retried same and it worked withing 1-2 seconds. Edited the option back to default value of 0, restarted Thunderbird, and the delayed response is back. So.... your workaround works quite well. Is this something that I can just leave with a value of 1, without hurting anything? It would be nice to use this workaround until a fix is supplied. Barry

That preference change is OK as long as you don't need to use accessibility features.

Awesome! Thanks Wayne. Does that test help narrow down the problem to root cause so it can be fixed? Let me know if you need anything more. Please also let me know when it is fixed in Thunderbird and what version so I can pull it and test.

Cross-reference: Fixing Bug 1125367 might also fix this one.

Installed 31.4.0 as soon as it was available. Tested. Problem still occurs and the workaround still works

If the accessibility hack doesn't help, notice in bug 464973 there will be a pref to turn off counting of subfolders into the values (of unread messages) of the parent folder. APraxis, you can then turn it of and we will see if the counting was the culprit in this bug.

APraxis, also try this. In Options->Advanced->Config editor, find these 2 preferences with these values: mail.db.idle_limit 300000 mail.db.max_open 30 Are they at those value? Then, try to increase mail.db.max_open to 10000 or more than the number of your folders in total.

(In reply to :aceman from comment #66) > APraxis, also try this. In Options->Advanced->Config editor, find these 2 > preferences with these values: > mail.db.idle_limit 300000 > mail.db.max_open 30 > > Are they at those value? > Then, try to increase mail.db.max_open to 10000 or more than the number of > your folders in total. I'm still running V17 so in order to perform your test I'll have to install/upgrade to the current version of TB, which I'll do in a test environment (after backing up my current TB profile and DB). I'll post the results as soon as I can.

Backup of course - but you do not need to replace your v17 - just use the exe installer or zip to put executable in its own directory and leave v17 intact. If using the installer choose "custom", and uncheck the option to update the start menu. But I think we are now less interested in results with those settings, and more keen on results with the proposed fix which you can test using the build at https://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2015-03-14-03-02-37-comm-central/thunderbird-39.0a1.en-US.win32.installer.exe - same deal as above choose "custom", and uncheck the option to update the start menu.

I have already posted to bug 1135310 that the build https://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/2015-03-14-03-02-37-comm-central/thunderbird-39.0a1.en-US.win32.installer.exe was successful. There were a couple of e-mails from Wayne Mery which I thank him for. So on my system v 39.0a1 is good and Wayne said and I [quote] The patch that was added in this bug will move to the aurora (TB 38) branch in a few days, assuming there are no major issues discovered. Then in about two weeks, there should be a beta of TB 38 that will also have this patch. So assuming nothing changes, this code will be in aurora in about a week, in beta in about 2 weeks, and in release (esr38) in about 9 weeks. There is more risk to newer code of course, but also more fixes. Thanks for the report, BTW, that it helped you. [/quote] So I'll continue with v 17 until v 38 is released... Thanks guys, J

I think you have said in the past that the first time TB is started after an update, it does not have the slowness. But subsequent starts of TB do have it. Can we assume you have been running and restarting with the TB39 for enough sessions that this effect can be ruled out?

(In reply to :aceman from comment #70) > I think you have said in the past that the first time TB is started after an > update, it does not have the slowness. But subsequent starts of TB do have > it. > Can we assume you have been running and restarting with the TB39 for enough > sessions that this effect can be ruled out? Yes, I closed/re-opened 3 times to verify there's no slowness anymore, and there isn't. I still have v 39 installed (in a test environment which is actually a copy of my main C) and can check some more but I'd expect the same results, this is a good version, for me at least. I then switched back to my live system with v 17 and the database is OK and I didn't need to restore it so v 39 didn't screw it up (which is good :) ) J

APraxis, given that bug 1135310 fixes the slowness, is there still a need for enumeration to be optional?

And the accumulation of subfolder counts in to a collapsed parent is optional now. You can turn it off via the mail.folderpane.sumSubfolders pref.

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #72) > APraxis, given that bug 1135310 fixes the slowness, is there still a need > for enumeration to be optional? No Wayne, not from my point of view. Thanks for getting this fix in, J

(In reply to :aceman from comment #73) > And the accumulation of subfolder counts in to a collapsed parent is > optional now. You can turn it off via the mail.folderpane.sumSubfolders pref. But if Ace's fix also works then leave it in (or not) which ever is easiest at this point. J

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #72) > APraxis, given that bug 1135310 fixes the slowness, is there still a need > for enumeration to be optional? No, but if Ace's fix works then leave it in (or not) which ever is easiest at this point. J

The pref is there for bug 464973, it toggles counting subfolders AND the display of the * (star) characters before accumulated subfolder numbers. So it is there for the UI feature, but if somebody uses it to make the display faster for him, then even better. Even if no longer needed for the bug here. The pref can be toggled in the advanced config editor. Thanks for reporting back.

Thanks aceman, J

While it seems the proposed patch here was not accepted, we do have a way to disable counting of subfolders in the folder pane. It was implemented in bug 464973. The preference is called mail.folderpane.sumSubfolders and you can find it in Options->Advanced->General->Config editor. Set it to 'false' and restart TB. It should be available since TB38. APraxis, have you tried to set this pref?

(In reply to :aceman from comment #79) > While it seems the proposed patch here was not accepted, we do have a way to > disable counting of subfolders in the folder pane. It was implemented in bug > 464973. > The preference is called mail.folderpane.sumSubfolders and you can find it > in Options->Advanced->General->Config editor. Set it to 'false' and restart > TB. > It should be available since TB38. > > APraxis, have you tried to set this pref? NO, I hadn't but will do it now and report back.I need to upgrade to V 38 again.

(In reply to :aceman from comment #79) > While it seems the proposed patch here was not accepted, we do have a way to > disable counting of subfolders in the folder pane. It was implemented in bug > 464973. > The preference is called mail.folderpane.sumSubfolders and you can find it > in Options->Advanced->General->Config editor. Set it to 'false' and restart > TB. > It should be available since TB38. > > APraxis, have you tried to set this pref? I had to update to 38.5 but that setting didn't improve anything. I then updated to 38.6, and that setting didn't improve anything either. to recap (see screen shots): TB 01 changed config as suggested TB 02 clicked [work online] TB 03 the main window opens and shows the Menu and the list of accounts in the left pane for a few seconds, TB 04 the the window goes blank with a rotating circle for 13 seconds. TB 05 the window shows the Menu, and both left and right panes are populated and waits for TB 06 the password prompts to be completed (tere are 6 accounts/pwds). From that point everything is as fast as usual.

OK, in that case it seems your problem is not fully caused by the summing of unread/total msg counts across subfolders. So the bug here as filed is fixed by bug 1135310 and bug 464973. Can you agree that the performance has improved between TB24-38? But you say it is not yet at the level of TB17. In that case I'd ask if you could file a new bug please to allow for proper tracking of the issues. Also please try TB45 first (should be out in a few weeks now). If the problem is still there, please create the new bug and we can tell you how to run a profiler to identify the code that is causing the slowness.

(In reply to :aceman from comment #88) > OK, in that case it seems your problem is not fully caused by the summing of > unread/total msg counts across subfolders. So the bug here as filed is fixed > by bug 1135310 and bug 464973. > > Can you agree that the performance has improved between TB24-38? > > But you say it is not yet at the level of TB17. In that case I'd ask if you > could file a new bug please to allow for proper tracking of the issues. > > Also please try TB45 first (should be out in a few weeks now). If the > problem is still there, please create the new bug and we can tell you how to > run a profiler to identify the code that is causing the slowness. I just tried the daily build thunderbird-47.0a1.en-US.win32.installer and posted in bug 1135310 and the delay is still there during a 'not responding' 13 seconds. I posted attachments/screen shots there as well using V 47

Yes, TB47 is even better and you do not need to retry in 45. In that case you can try running the profiler on it according to https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Reporting_a_Thunderbird_Performance_Problem_with_G. But please open a new bug with the findings.

OK, my bad, I thought that installing V 47 daily would use the same icon, I didn't immediately see the link on my desktop (too many other windows open etc.) so the screen shots are from V 38.06. Once I saw and used the Daily link, everything is the way it was in V 17, so V 45 may also be OK aceman. Forgive me my ignorance in not knowing how to use the Daily. Will it update every night and should I be cautious in using it? By that I mean should I keep V 17 as my 'normal' TB and only run the daily if you folks ask me to?

(In reply to :aceman from comment #90) > Yes, TB47 is even better and you do not need to retry in 45. > In that case you can try running the profiler on it according to > https://developer.mozilla.org/en-US/docs/Mozilla/Performance/ > Reporting_a_Thunderbird_Performance_Problem_with_G. But please open a new > bug with the findings. You will want to use version 38 with the profiler. I've updated the documentation.

(In reply to :aceman from comment #90) > Yes, TB47 is even better and you do not need to retry in 45. > In that case you can try running the profiler on it according to > https://developer.mozilla.org/en-US/docs/Mozilla/Performance/ > Reporting_a_Thunderbird_Performance_Problem_with_G. But please open a new > bug with the findings. Do I need to run the profiler because V 47 doesn't have any problem, actually I'd like to keep it but I think I see your point, if I keep it and it changes daily, a new bug might open up and if so, yes, I'll submit a new bug.

(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #92) > (In reply to :aceman from comment #90) > > Yes, TB47 is even better and you do not need to retry in 45. > > In that case you can try running the profiler on it according to > > https://developer.mozilla.org/en-US/docs/Mozilla/Performance/ > > Reporting_a_Thunderbird_Performance_Problem_with_G. But please open a new > > bug with the findings. > > You will want to use version 38 with the profiler. I've updated the > documentation. I just clicked the link to profiling and was overwhelmed at the options. Is there one that I should run with V 38 or should I run many? Please suggest the specific one(s) to run and I will.

OK, the link now takes me to a page that looks much easier, so I'll follow it. I got caught by another change, the Daily managed to insert it's link into my normal taskbar icon so when I thought that I was opening V 38 it was Daily. I've created a shortcut to V 38 and will proceed with the testing.

Problem: when starting TB V38 the first screen doesn't have the Gecko active. I have to wait until after the 'not responding' screen which is what we're trying to capture. So I'll try activating recording, close V 38, open V 38 and run it until I've got to the password prompts which I'll not enter because I don't want to have to change them. Didn't work, Gecko starts too late.

If TB47 is working fine for you there is a very high chance TB45 will be fine too. In that case you do not need to play with the profiler for now. You may want to try TB45 beta from https://www.mozilla.org/en-US/thunderbird/channel/ to make sure. You could use Daily/nightly regularly and it will update every day. However, some days there are bugs and even breakages such that TB does not even start. It seems you already had your share of problems and had to switch versions so I would recommend you to stay on v45 stable. It will be released soon.

(In reply to :aceman from comment #97) > If TB47 is working fine for you there is a very high chance TB45 will be > fine too. In that case you do not need to play with the profiler for now. > > You may want to try TB45 beta from > https://www.mozilla.org/en-US/thunderbird/channel/ to make sure. > > You could use Daily/nightly regularly and it will update every day. However, > some days there are bugs and even breakages such that TB does not even > start. It seems you already had your share of problems and had to switch > versions so I would recommend you to stay on v45 stable. It will be released > soon. When testing and reporting in 939462 - I'd installed it, no problem in V 47 (and V 38) - closed & opened it, no problem, which was different than V 38 which had the delay when opened after installation - repeated close & open several times, no problem, so I stopped testing V 47 then - I installed V 38, tried the Gecko tracking and found it started too late, so quit. then after some time, - I opened V 47 but I don't remember if there was a delay but eventually received mail and now I get the 20 second delay. So, I've restored my system to use V 17.06 which eliminates both V 38 & V 47 in Program Files (x86) but there is a folder re: Gecko and Lightning in my TB profiles, both are disabled. So tomorrow, Mar. 5th about 10am here (-5GMT): - I'll receive all e-mail using V 17 - I'll install V 47 Daily once more alongside V 17.06, hopefully no mail is received, I can avoid that by cancelling password prompts - I'll close/open V 47 without receiving mail , as above, and repeat if there's no delay - I'll send mail to myself and receive it - I'll close/open V 47 to see if new mail causes the delay and I'll try anything else that you want to suggest. I'll also restore my system to a point before and try V 45 beta which you suggested above. My only concern is having the Lightening folder in my profile. (I can delete the Gecko). I considered creating a new profile which would obviously be empty, but that wouldn't resolve the delay when I have lots of saved mail, so I won't. I could delete all mail from my current profile (after backing it up) and may try this.

see bug 1253235 with my experience updating V17 to V52 and my process which eliminates the delay.