Torkil Svensgaard Reporter
	Description • 8 years ago

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36

Steps to reproduce:

Start the program. I have a rather large profile directory (~26 GB), due to large IMAP accounts, which is part of the problem. I'm currently on 47.0a1 x64 for Windows but the latest x32 bit version had the same issue. I also run Thunderbird on my Linux desktop at work and although it sometimes time out for a few seconds it doesn't stop responding in the same way, even though it has the exact same IMAP accounts configured.


Actual results:

After some seconds it turns "Not Responding" for long periods of time, like 10 minutes or more. 


Expected results:

Program should respond.

	Wayne Mery (:wsmwk)
	Comment 1 • 8 years ago

Start *Windows'* safe mode with networking enabled
- win10 http://windows.microsoft.com/en-us/windows-10/change-startup-settings-in-windows-10
- win8 http://windows.microsoft.com/en-us/windows-8/windows-startup-settings-safe-mode
- win7 http://windows.microsoft.com/en-us/windows/start-computer-safe-mode#start-computer-safe-mode=windows-7
- XP http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/boot_failsafe.mspx

Still In Windows safe mode, start thunderbird in safe mode
- https://support.mozilla.org/kb/safe-mode-thunderbird

Does problem go away?

	Torkil Svensgaard Reporter
	Comment 2 • 8 years ago

Still "Not Responding" after a short period of time.

	Wayne Mery (:wsmwk)
	Comment 3 • 8 years ago

Go back to 32 bit - 64 bit isn't likely to help you and it's not fully supported. 

Please next visit the steps at https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems and post your results.

Also, please define "a short period of time".
And, what is your largest Inbox (number of messages)?

	Torkil Svensgaard Reporter
	Comment 4 • 8 years ago

Left it running with log overnight

	Torkil Svensgaard Reporter
	Comment 5 • 8 years ago

Screen shot of error console

	Torkil Svensgaard Reporter
	Comment 6 • 8 years ago

Back on 32 bit.

Steps:

1) On newest version yes
2) Safe mode didn't help (had disabled extensions etc. already)
3) Windows safe mode with networking didn't help
4) Not subscribed to "All Mail"
5) No newsgroups
6) Disabled all indexing, didn't help
7) Default profile location (on SSD)
8) Log attached
9) Globat indexing disabled
10) Renamed global-messages-db.sqlite. Old was 3 GB, new is 1.200 KB, no change.
11) Several folders are in the 1-2 GB range. 
12) Changed check interval to 5 mins, no change
13) I only have IMAP folders
14) I have 552 contacts
15) Panacea.dat is 48K
16) Don't think it's a script causing it, but screen shot of error console attached
17) Not using particularly high CPU, steady 13% for thunderbird.exe on a core i7 920. Updater not running.
18) All on system SSD
19) No RSS
20) Core i7 with 12GB RAM, no hardware problems I know of (and no problems with the system otherwise). Two of my inboxes have ~30.000 messages, but that isn't a problem on Ubuntu.
21) Windows 7
22) No real match in references I think.

Short period of time = within one minute. Largest inbox is ~30.000 messages.

	Torkil Svensgaard Reporter
	Comment 7 • 8 years ago

Turned of checking for new messages at startup for all accounts and it worked fine until i manually checked for new messages = freeze within one minute.

	Kent James (:rkent)
	Comment 8 • 8 years ago

It's hard to make sense of a large log with no context. What would be useful is to correlate times in the log with the observed behavior in the bug, like "freeze occurred at 2016-04-14 22:38:19.292000 in the log."

	Torkil Svensgaard Reporter
	Comment 9 • 8 years ago

Ok, I'll attach new log in a minute, with the following chronology:

21:11:12 Started program, pressed get all messages and started typing a new message
21:12:35 Noticed program lockup while typing
21.16.30 Noticed program responded again

From the log:
2016-04-20 21:12:32.312000 UTC - 0[2611140]: nsMsgDatabase::Open(C:\Users\torkil\AppData\Roaming\Thunderbird\Profiles\2z29axja.default\ImapMail\drcmr.dk\Admin-1.sbd\root.msf, FALSE, 81288d0, FALSE)
2016-04-20 21:16:28.819000 UTC - 0[2611140]: 13 open DB's

That folder has a size of 2.16 GB from 190.283 messages and I'm now compacting it to see if that makes any difference.

	Tim Johnson
	Comment 11 • 8 years ago

I am seeing a similar issue of protracted periods of "Not Responding" behavior. I'm using 64 bit Win 10 (ver 1511) with latest updates, and TB 45.2.0. This has been happening for over half a year, and seems to be getting worse. 

I see it most often after I've deleted some messages, or requested the Junk or Trash folders to be emptied. I have automatic compacting on and set to compact with over 20 MB would be saved. When going into or out of the Not Responding period, I have observed a very brief flash of the TB window.

I have the following extensions: Dictionary Search, FoxClocks, Lightning, Provider for Google Calendar, Mail Merge, QuoteCollapse and StartupMaster. Except for Lightning, Provider for Google Calendar and StartupMaster, these extensions have been there for years. I have the calendar set to refresh every 30 minutes.

On my wife's 64 bit Win 7 computer, with the same extensions, we see this Not Responding behavior infrequently.

	Tim Johnson
	Comment 12 • 8 years ago

Oh yes, my e-mail accounts are all pop3.

	Tim Johnson
	Comment 13 • 8 years ago

I've been running TB in safe mode (extensions temporarily disabled). This issue still persists, although the duration of the hang is considerably shorter. I'll run like this for the rest of the day, then start adding back extensions to see if there is one which brings back the much longer hang.

	Kent James (:rkent)
	Comment 14 • 8 years ago

Lightning is known to cause this issue in some cases. In theory THunderbird has lots of activities that might cause a hang, but they are surprisingly rare in my experience.

Then again, when I travel I run Thunderbird on a Microsoft Surface tablet, and I expect and see a lot of hangs there.

I think that we realize that there are way too many places where Thunderbird does sync processing on the IU thread. IMAP uses its own thread, POP3 does not. It's amazing that there are not more reports like this.

Recommendation? Disable Lightning and see if that is the issue. If not, if you can find a reproducible case where you can cause a hang in Thunderbird, we might be able to devise a kludge to stop that case. But there are underlying architectural issues, which to some extent we share with Firefox. They have been working for about 5 years on this, and I believe are now slowly starting to role out e10s. So this is not a simple change to fix.

	Tim Johnson
	Comment 15 • 8 years ago

With all my extensions disabled, I had very few hands, and when I got them, they were very short.

I enabled all extensions except Lightning and Provider for Google Calendar and for the next day did not see a single hang. 

Yesterday, I enabled Lightning and Provider for Google Calendar as well, and after over a day of running, I have not had a single hang. Go figure. If the hangs come back I'll post again.

This does smell like a thread race condition with timeouts to keep the race from going indefinitely. Why it has seemed to settle out for me is hard to explain.

	Wayne Mery (:wsmwk)
	Comment 16 • 8 years ago

(In reply to Torkil Svensgaard from comment #9)
> Ok, I'll attach new log in a minute, with the following chronology:
> 
> 21:11:12 Started program, pressed get all messages and started typing a new
> message
> 21:12:35 Noticed program lockup while typing
> 21.16.30 Noticed program responded again
> 
> From the log:
> 2016-04-20 21:12:32.312000 UTC - 0[2611140]:
> nsMsgDatabase::Open(C:
> \Users\torkil\AppData\Roaming\Thunderbird\Profiles\2z29axja.
> default\ImapMail\drcmr.dk\Admin-1.sbd\root.msf, FALSE, 81288d0, FALSE)
> 2016-04-20 21:16:28.819000 UTC - 0[2611140]: 13 open DB's
> 
> That folder has a size of 2.16 GB from 190.283 messages and I'm now
> compacting it to see if that makes any difference.

4 minutes is a long time to load from SSD what is probably a ~300MB .msf index file. However, 190k is a lot of messages. 

Torkil,
1. Do many of the messages in that file have the same subject?
2. why would Thunderbird be opening that file?

	Wayne Mery (:wsmwk)
	Comment 17 • 8 years ago

Torkil,...


(In reply to Torkil Svensgaard from comment #5)
> Screen shot of error console

Thanks. unfortunately nothing illuminating there


What did you find after compacting the folder Admin-1.sbd\root?
And is that a folder of server log files?


In reply to Torkil Svensgaard from comment #10)
> Created attachment 8743533 [details]
> imap.log

2016-04-20 21:12:32.312000 UTC - 0[2611140]: nsMsgDatabase::Open(C:\Users\torkil\AppData\Roaming\Thunderbird\Profiles\2z29axja.default\ImapMail\drcmr.dk\Admin-1.sbd\root.msf, FALSE, 81288d0, FALSE)
2016-04-20 21:16:28.819000 UTC - 0[2611140]: 13 open DB's

There is a 4 minute gap between those two log lines.  It seems to me there should have nearly the same time stamp.  Does that 4 minute gap match your delay time?

	Torkil Svensgaard Reporter
	Comment 18 • 8 years ago

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #16)
> (In reply to Torkil Svensgaard from comment #9)
> > Ok, I'll attach new log in a minute, with the following chronology:
> > 
> > 21:11:12 Started program, pressed get all messages and started typing a new
> > message
> > 21:12:35 Noticed program lockup while typing
> > 21.16.30 Noticed program responded again
> > 
> > From the log:
> > 2016-04-20 21:12:32.312000 UTC - 0[2611140]:
> > nsMsgDatabase::Open(C:
> > \Users\torkil\AppData\Roaming\Thunderbird\Profiles\2z29axja.
> > default\ImapMail\drcmr.dk\Admin-1.sbd\root.msf, FALSE, 81288d0, FALSE)
> > 2016-04-20 21:16:28.819000 UTC - 0[2611140]: 13 open DB's
> > 
> > That folder has a size of 2.16 GB from 190.283 messages and I'm now
> > compacting it to see if that makes any difference.
> 
> 4 minutes is a long time to load from SSD what is probably a ~300MB .msf
> index file. However, 190k is a lot of messages. 
> 
> Torkil,
> 1. Do many of the messages in that file have the same subject?

Yes, lots of emails from root accounts from various machines, with identical subjects.

> 2. why would Thunderbird be opening that file?

Not sure, as I've un-ticked "When getting new messages for this account, always check this folder"

	Torkil Svensgaard Reporter
	Comment 19 • 8 years ago

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #17)
> What did you find after compacting the folder Admin-1.sbd\root?
> And is that a folder of server log files?

Problem went away, pretty much, after compacting, IIRC. It's various service emails from mailman, smartd etc.

> In reply to Torkil Svensgaard from comment #10)
> > Created attachment 8743533 [details]
> > imap.log
> 
> 2016-04-20 21:12:32.312000 UTC - 0[2611140]:
> nsMsgDatabase::Open(C:
> \Users\torkil\AppData\Roaming\Thunderbird\Profiles\2z29axja.
> default\ImapMail\drcmr.dk\Admin-1.sbd\root.msf, FALSE, 81288d0, FALSE)
> 2016-04-20 21:16:28.819000 UTC - 0[2611140]: 13 open DB's
> 
> There is a 4 minute gap between those two log lines.  It seems to me there
> should have nearly the same time stamp.  Does that 4 minute gap match your
> delay time?

I guess so, as I wrote this in another comment:

21:11:12 Started program, pressed get all messages and started typing a new message
21:12:35 Noticed program lockup while typing
21.16.30 Noticed program responded again

I've since archived a lot of emails to speed things up (yearly folders) so my setup is not the same anymore.

	Wayne Mery (:wsmwk)
	Comment 20 • 8 years ago

Thanks!  A couple more questions embeded below...


(In reply to Torkil Svensgaard from comment #18)
> (In reply to Wayne Mery (:wsmwk, NI for questions) from comment #16)
> > (In reply to Torkil Svensgaard from comment #9)
> > > Ok, I'll attach new log in a minute, with the following chronology:
> > > 
> > > 21:11:12 Started program, pressed get all messages and started typing a new
> > > message
> > > 21:12:35 Noticed program lockup while typing
> > > 21.16.30 Noticed program responded again
> > > 
> > > From the log:
> > > 2016-04-20 21:12:32.312000 UTC - 0[2611140]:
> > > nsMsgDatabase::Open(C:
> > > \Users\torkil\AppData\Roaming\Thunderbird\Profiles\2z29axja.
> > > default\ImapMail\drcmr.dk\Admin-1.sbd\root.msf, FALSE, 81288d0, FALSE)
> > > 2016-04-20 21:16:28.819000 UTC - 0[2611140]: 13 open DB's
> > > 
> > > That folder has a size of 2.16 GB from 190.283 messages and I'm now
> > > compacting it to see if that makes any difference.
> > 
> > 4 minutes is a long time to load from SSD what is probably a ~300MB .msf
> > index file. However, 190k is a lot of messages. 

> > Torkil,
> > 1. Do many of the messages in that file have the same subject?
> 
> Yes, lots of emails from root accounts from various machines, with identical
> subjects.

good info - identical subjects in the UI can be a performance issue. also good - is that unless you actually clicked the folder, it does not present a problem.

  
> > 2. why would Thunderbird be opening that file?
> 
> Not sure, as I've un-ticked "When getting new messages for this account,
> always check this folder"

Oh. Getting new mail in the folder with TB filters or "always check this folder" explains why it was opening, even though you didn't click on the folder.

Is/was that folder populated by a server-side filter?  How recently did you untick that option? 
Are you using Thunderbird filters now to categorize those mails?

And do you have "check this folder" set for any other folders? 


> I've since archived a lot of emails to speed things up (yearly folders) so my setup is not the same anymore.
> Problem went away, pretty much, after compacting.

You say "pretty much". Does that mean you still see some delays?  And how many messages in the folder now?


I'm still a bit stumped - you have SSD so opening even a 200mb msf index file shouldn't cause minutes of delay - because we can rule out dozens of possible causes using your comment 6. Unless "check this folder" was still enabled at the time.  

Not to mention, that you say the linux system didn't have this problem. So I'll put it out there even though you said you still had trouble in WIndows safe mode, that this kind of extreme/patholigical performance degradation is what we sometimes see when Windows-based antivirus programs are involved.

	Torkil Svensgaard Reporter
	Comment 21 • 8 years ago

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #20)
> Thanks!  A couple more questions embeded below...
> 
   
> > > 2. why would Thunderbird be opening that file?
> > 
> > Not sure, as I've un-ticked "When getting new messages for this account,
> > always check this folder"
> 
> Oh. Getting new mail in the folder with TB filters or "always check this
> folder" explains why it was opening, even though you didn't click on the
> folder.

I meant it was un-ticked at the time (and still is), so no idea why the folders was checked.
 
> Is/was that folder populated by a server-side filter?  How recently did you
> untick that option? 
> Are you using Thunderbird filters now to categorize those mails?

Server side by sieve on the IMAP server. I'm not using Thunderbird filters

> And do you have "check this folder" set for any other folders? 

No.
 
> > I've since archived a lot of emails to speed things up (yearly folders) so my setup is not the same anymore.
> > Problem went away, pretty much, after compacting.
> 
> You say "pretty much". Does that mean you still see some delays?  And how
> many messages in the folder now?

Still delays sometimes, but in the less than a minute range. The folder contains about 47.000 emails currently but, perhaps more important, size on disk is ~200 mb. As I wrote earlier, I think the folder was 2.16 gb before I compacted it, and afterwards it was much smaller.
 
> I'm still a bit stumped - you have SSD so opening even a 200mb msf index
> file shouldn't cause minutes of delay - because we can rule out dozens of
> possible causes using your comment 6. Unless "check this folder" was still
> enabled at the time.  

It wasn't

> Not to mention, that you say the linux system didn't have this problem. So
> I'll put it out there even though you said you still had trouble in WIndows
> safe mode, that this kind of extreme/patholigical performance degradation is
> what we sometimes see when Windows-based antivirus programs are involved.

The linux system still/also have small delays, but also in the less than a minute range, and subjectively much shorter delays than on Windows.

	Wayne Mery (:wsmwk)
	Comment 22 • 8 years ago

> The folder contains about 47.000 emails currently but, perhaps more important, size on disk is ~200 mb. As I wrote earlier, I think the folder was 2.16 gb before I compacted it, and afterwards it was much smaller.

So it got much smaller than 2.1gb after the compact, but before you archived messages out of it??

If so, then automatic compact was failing on that folder.

	Torkil Svensgaard Reporter
	Comment 23 • 8 years ago

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #22)
> > The folder contains about 47.000 emails currently but, perhaps more important, size on disk is ~200 mb. As I wrote earlier, I think the folder was 2.16 gb before I compacted it, and afterwards it was much smaller.
> 
> So it got much smaller than 2.1gb after the compact, but before you archived
> messages out of it??
> 
> If so, then automatic compact was failing on that folder.

Exactly. That sounds plausible.

	Wayne Mery (:wsmwk)
	Comment 24 • 8 years ago

> Still delays sometimes, but in the less than a minute range. The folder contains about 47.000 emails currently but, perhaps more important, size on disk is ~200 mb

If you keep the folder open in a separate tab (you don't have to keep it in the foreground) does it impact the delays you see?

	Torkil Svensgaard Reporter
	Comment 25 • 8 years ago

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #24)
> > Still delays sometimes, but in the less than a minute range. The folder contains about 47.000 emails currently but, perhaps more important, size on disk is ~200 mb
> 
> If you keep the folder open in a separate tab (you don't have to keep it in
> the foreground) does it impact the delays you see?

I've had it on a separate tab for a couple of days, seems about the same.

	Mike
	Comment 26 • 8 years ago

Have same problem as Tim Johnson (comment 11)  Windows Ver1607 64 bit
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 
TB handles mix of POP3, IMAP, RSS and NNTP accounts

Not responding states seem to last a minute or so before system comes back to life and repeat frequently, say every 4 or so minutes. Most often happens whilst composing a new mail message editor, responding to an email downloaded from pop3 server.  UI in complete non-responding state.  Despite so many good features, this is making Thunderbird almost unworkable, just about to throw in the towel.

Concurrent with issue, the error console has reported:
Error: uncaught exception: 2147746065
Source File: resource:///modules/activity/autosync.js
Line: 210

Kent James message 14 seems pertinent:
"I think that we realize that there are way too many places where Thunderbird does sync processing on the IU thread. IMAP uses its own thread, POP3 does not. It's amazing that there are not more reports like this."

	Wayne Mery (:wsmwk)
	Comment 27 • 8 years ago

Torkil,
Do you have account settings for delete set to IMAP delete model (Mark as Deleted)?

	Torkil Svensgaard Reporter
	Comment 28 • 8 years ago

It's set to move to a trash folder on my Windows, probably the same on my Linux at work

	Wayne Mery (:wsmwk)
	Comment 29 • 8 years ago

(In reply to Mike from comment #26)
> Have same problem as Tim Johnson (comment 11)  Windows Ver1607 64 bit
> Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
> Thunderbird/45.7.1 
> TB handles mix of POP3, IMAP, RSS and NNTP accounts

(In reply to Tim Johnson from comment #11)
> I am seeing a similar issue of protracted periods of "Not Responding"
> behavior. I'm using 64 bit Win 10 (ver 1511) with latest updates, and TB
> 45.2.0. This has been happening for over half a year, and seems to be
> getting worse. 
> 

You may be seeing similar symptoms, but unless you have very large folders it is unlikely the cause is the same.  There is a litany of possible causes - too many to list here, but listed at https://wiki.mozilla.org/Thunderbird:Testing:Memory_Usage_Problems  For example check your junk and trash folder sizes.

1. you must eliminate all other software as potential cause, per comment 1.
2. RSS is a possible problem - turn off RSS checking at startup and for intervals. If that helps, with RSS checking try the nightly build which has some changes https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/ (backup your Thunderbird profile first.

	Wayne Mery (:wsmwk)
	Comment 30 • 8 years ago

(In reply to Torkil Svensgaard from comment #28)
> It's set to move to a trash folder on my Windows, probably the same on my
> Linux at work

Cool. What's clear is the problem is related to that "root" folder and perhaps others like it. What isn't clear, is whether it was device-related, Thunderbird compact repeatedly failing or some mork-related issue (compress commit failing), or something else.

If you don't use global search against large folders like "root" you can save yourself some overhead by disabling global search indexing on those high volume folders (n.b. comment 6 item 10 where index size dropped by 2/3) by installing glodaquilla addon and set those specific folder properties to not index.

	Wayne Mery (:wsmwk)
	Comment 31 • 7 years ago

(In reply to Mike from comment #26)
> Have same problem as Tim Johnson (comment 11)  Windows Ver1607 64 bit
> Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
> Thunderbird/45.7.1 
> TB handles mix of POP3, IMAP, RSS and NNTP accounts
> 
> Not responding states seem to last a minute or so before system comes back
> to life and repeat frequently, say every 4 or so minutes. Most often happens
> whilst composing a new mail message editor, responding to an email
> downloaded from pop3 server.  UI in complete non-responding state.  Despite
> so many good features, this is making Thunderbird almost unworkable, just
> about to throw in the towel.
> 
> Concurrent with issue, the error console has reported:
> Error: uncaught exception: 2147746065
> Source File: resource:///modules/activity/autosync.js
> Line: 210
> 
> Kent James message 14 seems pertinent:
> "I think that we realize that there are way too many places where
> Thunderbird does sync processing on the IU thread. IMAP uses its own thread,
> POP3 does not. It's amazing that there are not more reports like this."

The autosync error may be ignored.

Mike, disable checking for new messages in RSS and NNTP. What are your results?   If that helps, please try beta from http://www.mozilla.org/en-US/thunderbird/channel/ with those still disabled, and then again with them enabled. What are your results?

	Mike
	Comment 32 • 7 years ago

(In reply to Wayne Mery (:wsmwk, NI for questions) from comment #31)
> (In reply to Mike from comment #26)
> > Have same problem as Tim Johnson (comment 11)  Windows Ver1607 64 bit
> > Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
> > Thunderbird/45.7.1 
> > TB handles mix of POP3, IMAP, RSS and NNTP accounts
> > 
> > Not responding states seem to last a minute or so before system comes back
> > to life and repeat frequently, say every 4 or so minutes. Most often happens
> > whilst composing a new mail message editor, responding to an email
> > downloaded from pop3 server.  UI in complete non-responding state.  Despite
> > so many good features, this is making Thunderbird almost unworkable, just
> > about to throw in the towel.
> > 
> > Concurrent with issue, the error console has reported:
> > Error: uncaught exception: 2147746065
> > Source File: resource:///modules/activity/autosync.js
> > Line: 210
> > 
> > Kent James message 14 seems pertinent:
> > "I think that we realize that there are way too many places where
> > Thunderbird does sync processing on the IU thread. IMAP uses its own thread,
> > POP3 does not. It's amazing that there are not more reports like this."
> 
> The autosync error may be ignored.
> 
> Mike, disable checking for new messages in RSS and NNTP. What are your
> results?   If that helps, please try beta from
> http://www.mozilla.org/en-US/thunderbird/channel/ with those still disabled,
> and then again with them enabled. What are your results?

Hi Wayne,  thanks for suggestion.  I have got further in finding out that the underlying cause of my problem is triggered following a send of email  (usually it is the second email in a session).  If I look in a TCP analyser, I see that a TCP connection from Thunderbird to the smtp server involved in the send remains in a CLOSE_WAIT state and the user interface to Thunderbird UI goes into a "not responding" state until this CLOSE_WAIT connection gets closed which seems to take 2 minutes which possibly relates to TCP standards on timeout time
https://blogs.technet.microsoft.com/janelewis/2010/03/09/explaining-close_wait/
http://www.tcpipguide.com/free/t_TCPConnectionTermination-3.htm

I am not sure what triggers the close wait issue between Thunderbird and the smtp server.  It does not seem to happen with every smtp send but if it is possible to remove the dependency on the Thunderbird UI on this TCP connection closure and or if the 120 sec timeout could be shortened down, that would be brilliant as the user could continue instead of need to stall for 2 minutes.
cheers Mike

	Wayne Mery (:wsmwk)
	Comment 33 • 6 years ago

Torkil, 

Do you still see this issue with version 52? And if yes, does it also happen with current beta from http://www.mozilla.org/en-US/thunderbird/channel/ ?

	Wayne Mery (:wsmwk)
	Comment 34 • 6 years ago

Mike, 

Do you still see this issue with version 52? And if yes, does it also happen with current beta from http://www.mozilla.org/en-US/thunderbird/channel/ ?

	Mike
	Comment 35 • 6 years ago

I haven't seen the problem for a week or so now, being stuck with TCP CLOSE_WAIT state.  
Currently on     
Name: Thunderbird
    Version: 52.6.0
    User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0

Great if it is fixed,

Mike

	Wayne Mery (:wsmwk)
	Comment 36 • 6 years ago

So, bug 1157256 reported in 2015, prior to this bug, and just fixed may have played a role in reporter's issue - Automatic compacting folders not working in online mode. IMAP users not prompted for compact."

However, compact may have simply removed the conditions - I suspect reporter's issue is more related to large folder particularly those we have seen reported by others who store large numberes of logfile messages in imap, like bug 589310 / bug 588952.  Plus other factors may be at play, such as anything in https://mzl.la/2JKrq86


(In reply to Torkil Svensgaard from comment #21)
> (In reply to Wayne Mery (:wsmwk, NI for questions) from comment #20)
>    
> > > > 2. why would Thunderbird be opening that file?
> > > 
> > > Not sure, as I've un-ticked "When getting new messages for this account,
> > > always check this folder"
> > 
> > Oh. Getting new mail in the folder with TB filters or "always check this
> > folder" explains why it was opening, even though you didn't click on the
> > folder.
> 
> I meant it was un-ticked at the time (and still is), so no idea why the
> folders was checked.

Do you (or did you) have hidden preference  mail.check_all_imap_folders_for_new  set to true?  (either on linux or Windows)

	Wayne Mery (:wsmwk)
	Comment 37 • 6 years ago

(In reply to Mike from comment #35)
> I haven't seen the problem for a week or so now, being stuck with TCP CLOSE_WAIT state.  

That's good news.  So going from 45 to 52 helped?  

If you see the problem again you might look at open smtp bug reports that mention TCP or CLOSE_WAIT https://mzl.la/2LxodgQ

	Wayne Mery (:wsmwk)
	Comment 38 • 6 years ago

(In reply to Torkil Svensgaard from comment #18)
> (In reply to Wayne Mery (:wsmwk, NI for questions) from comment #16)
> > (In reply to Torkil Svensgaard from comment #9)
> > >...
> > > That folder has a size of 2.16 GB from 190.283 messages and I'm now
> > > compacting it to see if that makes any difference.
> > 
> > 4 minutes is a long time to load from SSD what is probably a ~300MB .msf
> > index file. However, 190k is a lot of messages. 
> > 
> > Torkil,
> > 1. Do many of the messages in that file have the same subject?
> 
> Yes, lots of emails from root accounts from various machines, with identical
> subjects.

Which are 
Bug 226730 - Slow threading when all messages have the same subject, with mail.strict_threading
Bug 452221 - When many mails have the same subject (e.g. very long thread), time to take Shift+Delete (not sure if this is applicable)
Bug 452944 - TB uses 1.3 GB RAM with sort criteria Junk yes/no and subject and big inbox

	Torkil Svensgaard Reporter
	Comment 39 • 6 years ago

(In reply to Wayne Mery (:wsmwk) from comment #33)
> Torkil, 
> 
> Do you still see this issue with version 52? And if yes, does it also happen
> with current beta from http://www.mozilla.org/en-US/thunderbird/channel/ ?

Sorry for being tardy. Still seeing an issue with the same or worse symptoms. Currently on version 52.8.0.

Mvh.

Torkil

	Torkil Svensgaard Reporter
	Comment 40 • 6 years ago

(In reply to Wayne Mery (:wsmwk) from comment #36)
> Do you (or did you) have hidden preference 
> mail.check_all_imap_folders_for_new  set to true?  (either on linux or
> Windows)

Not at the moment, and I'm certain I've never enabled it myself, but I don't know if it could have been set automagically at some point.

Mvh.

Torkil

	Torkil Svensgaard Reporter
	Comment 41 • 6 years ago

(In reply to Wayne Mery (:wsmwk) from comment #30)
> If you don't use global search against large folders like "root" you can
> save yourself some overhead by disabling global search indexing on those
> high volume folders (n.b. comment 6 item 10 where index size dropped by 2/3)
> by installing glodaquilla addon and set those specific folder properties to
> not index.

Thanks, will try that. That folder currently have about ~130.000 emails, I guess that's probably still the culprit. The issue does not crop up consistently but once in a while.

Mvh.

Torkil

	Wayne Mery (:wsmwk)
	Comment 42 • 6 years ago

(In reply to Mike from comment #35)
> I haven't seen the problem for a week or so now, being stuck with TCP
> CLOSE_WAIT state.  
> Currently on     
> Name: Thunderbird
>     Version: 52.6.0
>     User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
> Thunderbird/52.6.0
> 
> Great if it is fixed,
> 
> Mike

Yes, that would be great.  Please let us know if the condition returns

	Wayne Mery (:wsmwk)
	Comment 43 • 6 years ago

(In reply to Torkil Svensgaard from comment #40)
> I'm certain I've never enabled mail.check_all_imap_folders_for_new  myself, but I don't
> know if it could have been set automagically at some point.

unlikely :)


(In reply to Torkil Svensgaard from comment #41)
> (In reply to Wayne Mery (:wsmwk) from comment #30)
> > ...
> > save yourself some overhead by disabling global search indexing on those
> > high volume folders (n.b. comment 6 item 10 where index size dropped by 2/3)
> > by installing glodaquilla addon and set those specific folder properties to
> > not index.
> 
> Thanks, will try that. That folder currently have about ~130.000 emails, I
> guess that's probably still the culprit. The issue does not crop up
> consistently but once in a while.

How is that process going?