User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

Just daily use of Thunderbird with ~ 10 accounts, most IMAP.

Actual results:

After some time of usage (4 hours + more) TB uses much CPU.
Adding addresses to To: fields becomes slower and slower, e.g. finding the address from the address book for the 10 th entry takes a minute. This behaviour gets worse and worse since 1 year.
From the task manager it looks like some threads block each other, see attachment.

I also regularly see crash on shutdown, see e.g. bug 1346060

(In reply to Ulf Zibis from comment #0)

...
Just daily use of Thunderbird with ~ 10 accounts, most IMAP.

how many pop?

Actual results:

After some time of usage (4 hours + more) TB uses much CPU.

No calendar add-on?
Other add-ons?
What about memory.

I also regularly see crash on shutdown, see e.g. bug 1346060

Do you use master password?

(In reply to Wayne Mery (:wsmwk) from comment #1)

how many pop?

1

No calendar add-on?

I use xul-ext-lightning from Ubuntu

What about memory.

~ 700 MB

Do you use master password?

Yes!

Other add-ons?

Yes:

    Duplicate Contacts Manager
    1.1
    true
    duplicateContactsManager@ddvo.net

    Enigmail
    2.0.10
    true
    {847b3a00-7ab1-11d4-8f02-006008948af5}

    Get/Send-Button
    5.3.1
    true
    {6e071e83-d8d6-8ca9-6129-eb8e8144596f}

    ImportExportTools
    3.3.2
    true
    {3ed8cc52-86fc-4613-9026-c1ef969da4c3}

    Lightning
    6.2.6.1
    true
    {e2fda1a4-762b-4020-b5ad-a41df1933103}

    Messaging Menu and Unity Launcher integration
    1.4
    true
    messagingmenu@mozilla.com

    ProfileSwitcher
    1.7.7
    true
    {fa8476cf-a98c-4e08-99b4-65a69cb4b7d4}

    Remove Duplicate Messages (Alternate)
    0.3.18b
    true
    {a300a000-5e21-4ee0-a115-9ec8f4eaa92b}

    Später Senden
    6.4.6
    true
    sendlater3@kamens.us

    URL Link
    2.05
    true
    {139a120b-c2ea-41d2-bf70-542d9f063dfd}

Deactivated:

    abcTajpu "a b c teip-u"
    1.8.5.1-signed.1-signed
    false
    {15a7ef52-8a77-426e-9e17-e21af257d7c8}

    Add-on Compatibility Reporter
    1.1.1-signed.1-signed
    false
    compatibility@addons.mozilla.org

    AttachmentExtractor
    1.3.5.1
    false
    {35834d20-efdb-4f78-ab77-9635fb4e56c4}

    CardBook
    35.7
    false
    cardbook@vigneau.philippe

    ChromEdit Plus
    2.9.10
    false
    chromeditplus@webdesigns.ms11.net

    CompactHeader
    2.1.6
    false
    {58D4392A-842E-11DE-B51A-C7B855D89593}

    Copy Pure Text
    2.0.1.1-signed.1-signed
    false
    copy-pure-text@kashiif-gmail.com

    DictionarySearch
    28.0.0.1-signed
    false
    {a0faa0a4-f1a7-4098-9a74-21efc3a92372}

    DOM Inspector
    2.0.16.1-signed
    false
    inspector@mozilla.org

    DOM Inspector Plus! [dm]
    2.0.17.1.5
    false
    inspector-dp@mozilla.org

    External Editor
    1.0.0
    false
    {11d1a081-b8c3-4002-af38-6ad3e899fdfa}

    FiltaQuilla
    1.4.1
    false
    filtaquilla@mesquilla.com

    Identity Chooser
    1.9.7
    false
    identitychooser@janek.org

    Image Zoom
    0.6.3.1-signed.1-signed
    false
    {1A2D0EC4-75F5-4c91-89C4-3656F6E44B68}

    InfoLister
    0.10.4a3
    false
    {3f0da09b-c1ab-40c5-8d7f-53f475ac3fe8}

    Is It Compatible?
    0.6.0
    false
    isitcompatible@eternicode.com

    LookOut
    1.2.13
    false
    lookout@aron.rubin

    Password Exporter
    1.3.4
    false
    {B17C1C5A-04B1-11DB-9804-B622A1EF5492}

    pasteCode
    0.71
    false
    pastecode@morton

    Saved Password Editor
    2.10.4
    false
    savedpasswordeditor@daniel.dawson

    THUNDER!Box
    0.9.1
    false
    thunderbox@sourceforge.net

    ThunderBrowse
    3.82
    false
    ThunderBrowse@thunderbrowse.com

    ThunderPlunger
    2.15b2
    false
    {bfec0750-aa6f-11db-abbd-0800200c9a66}

As a starting point, disable unity addon

(In reply to Wayne Mery (:wsmwk) from comment #3)

As a starting point, disable unity addon

Where you see a unity addon?
I don't have it in the list.

Maybe Wayne meant to disable the 'Messaging Menu and Unity Launcher integration' extension.

(In reply to Stefan Sitter [:ssitter] from comment #5)

Maybe Wayne meant to disable the 'Messaging Menu and Unity Launcher integration' extension.

Thanks, I've disabled it. Will see, if it helps.

(In reply to Wayne Mery (:wsmwk) from comment #3)

As a starting point, disable unity addon

Now I've tested this for nearly 3 weeks. It hasn't changed anything :-(

Additionally it gets worse and worse. Now the slowness happens right from start of TB.

Lightning and any other addons eliminated as possibility?
No activity in Activity manager?
Tried beta?

Possible cross matches
https://mzl.la/2Jlcuim
https://mzl.la/2XnTJ7S

Some time ago I tried without addons, it didn't help.
Disabling Lightning for some time is difficult to me, as I need it regularly.
Using beta, I don't see a sense, since I have this issue since several normal updates.

About cross matches. The described cases are always slight different, but I also see often:

• hang in composing every ~15 seconds
• hang when scrolling mail folders pane
• UI unresonsiveness ... UI gets gray on Ubuntu
• "check mails every" is mostly on 3 minutes.
  Mail: 8 GB (most is in local folders)
  ImapMail: 2 GB

Please run the performance profiler:

1. You must be using Thunderbird 68 or newer - betas from https://www.thunderbird.net/en-US/channel/ or current nightly build from https://archive.mozilla.org/pub/thunderbird/nightly/latest-comm-central/
2. Install profiler add-on into thunderbird 68 (or newer https://www.thunderbird.net/en-US/channel/ ) - get the add-on file from https://github.com/firefox-devtools/Gecko-Profiler-Addon/blob/master/gecko_profiler.xpi?raw=true and in Tools > add-ons click the gear to install add-on from file
3. Follow instructions at https://profiler.firefox.com/ (videos BASED ON FIREFOX at https://profiler.firefox.com/docs/#/./videos-intro )
4. Create a profiler URL and post it here.

Hi,
as I'm on Ubuntu, I would like to know, when TB 68 will probably released and/or where I can download TB 68 (the beta is version 69). This seems more secure for me in respect of my profile data, than using a beta version.

(In reply to Ulf Zibis from comment #12)

when TB 68 will probably released and/or where I can download TB 68 (the beta is version 69).

https://www.thunderbird.net/ or ask your distro

(In reply to Wayne Mery (:wsmwk) from comment #13)

(In reply to Ulf Zibis from comment #12)

when TB 68 will probably released and/or where I can download TB 68 (the beta is version 69).

https://www.thunderbird.net/ or ask your distro

Much thanks for the hint Wayne.
In my experience, it takes about a month, a new version lands in the Ubuntu distro. I Think, I will wait for this, as I'm afraid to get again corrupted Lightning calendar data with a non-distro version.

(In reply to Ulf Zibis from comment #14)

In my experience, it takes about a month, a new version lands in the Ubuntu distro.

Unfortunately they still haven't put it up. We can reopen this when a profile using 68 is possible.

(In reply to Wayne Mery (:wsmwk) from comment #11)

3. Follow instructions at https://profiler.firefox.com/ (videos BASED ON FIREFOX at https://profiler.firefox.com/docs/#/./videos-intro )
4. Create a profiler URL and post it here.

Hi,
as I started the gecko profiler, after less than 1 minute, Tb crashed.

On a 2nd try, I could start the profiler, clicked "Publish", but when I tried to copy the URL, TB again crashed.

3rd try:
https://share.firefox.dev/2SwqWJx

After some minutes:
https://share.firefox.dev/2I4Ntv0

Thanks, but so-called "details" reports are of no use - you can skip posting them. Please post your 5 most recent crash IDs. You can find them at Help > Troubleshooting

The most recent crash reports:

bp-36b66e5b-974f-44d7-ad52-4e0f30201007 07.10.20, 12:18
bp-bc098b7c-f4a4-4ba0-b301-88f450201007 07.10.20, 11:45
bp-92ec24a1-7fd2-4e18-a988-075700201007 07.10.20, 11:39
bp-1f70400f-56e3-42a7-adab-e32e10201007 07.10.20, 02:21
bp-a83446d2-04ab-42e0-8dc2-297710200923 23.09.20, 09:34
bp-baac4eaf-01ff-4a44-9d5b-0d8ec0200922 22.09.20, 09:05
bp-3722904d-7c97-4a23-b300-4d34d0200919 19.09.20, 10:33
bp-a737dec9-2288-42d5-ad13-b69460200913 14.09.20, 00:12
bp-8f6c3b19-7771-4e79-8625-c81670200906 07.09.20, 00:29

My best guess, which might not be worth much in this case, is something add-on related. Which may or may not be helped by going to version 78, which is where you'll need to be anyway if a fix is needed from Thunderbird. So check which addons are available on version 78 and let us know how it behaves.

FWIW, The profile I looked at is mostly mork activity.

Hi Wayne,
FYI, I didn't get any notification about your last post and even not for my last posts, I don't understand why.

Here is a new profiler info, when TB starts to slow down:
https://share.firefox.dev/372n9MA

Anyway, TB 78 is still not available in Ubuntu Repository. This may take some weeks.

Here is a new profiler info, when TB starts to slow down:
https://share.firefox.dev/2STiy6Z

https://share.firefox.dev/2SW9ZIu

https://profiler.firefox.com/public/4a4t6tmjk72tqgdcsay6c3t8zj839m69rjxf2c8/calltree/?globalTrackOrder=0&localTrackOrderByPid=42017-1-0~&thread=0&v=5 looks maybe like a bad folder

(In reply to Wayne Mery (:wsmwk) from comment #28)

https://profiler.firefox.com/public/4a4t6tmjk72tqgdcsay6c3t8zj839m69rjxf2c8/calltree/?globalTrackOrder=0&localTrackOrderByPid=42017-1-0~&thread=0&v=5 looks maybe like a bad folder

Very interesting, thanks for the hint.
But what does that mean: "bad folder"?
What can I do for that, and how to find the folder?

Another profiling: https://share.firefox.dev/35sTdXn

Please reproduce with Thunderbird started in safe mode, preferably with version 78

(In reply to Wayne Mery (:wsmwk) from comment #31)

Please reproduce with Thunderbird started in safe mode, preferably with version 78

I'm still waiting for V 78 on Ubuntu. Anyway, using safe mode would be difficult to me, as often I have to wait more than 1 day of continuous usage of TB to notice the problem to occur and I need some addons in my normal work.

Good point: bug 1655346 comment 10
I too often see this after moving a big amount of messages (1000 .. 5000) between IMAP folders. Mostly only restart helps.

Don't you have an answer on comment 29 ?

Another profiling after click on IMAP INBOX: https://share.firefox.dev/34KlXff
I saw the sand glass for some time, and noticed, that this IMAP Inbox was not populated for several days.

Same problem again. The IMAP INBOX stagnates since oct 26: https://share.firefox.dev/35SlpD7

Many show significant GC at the start, and heavy jank - not surprising. A couple show imap, specifically nsImapMailFolder::FindKeysToDelete

We're stuck at this point - really need to see what happens on version 78, because so much has changed since 68, and 68 will no longer be fixed. I suggest copying your production profile to a test profile, and run 78 on the test profile - with whatever addons will function on 78 (not all will of course). https://support.mozilla.org/en-US/kb/profiling-thunderbird-performance describes how to get a profile when using version 78.

In reply to comment 21 and 32:

Fortunately version 78 now is available from Ubuntu repository since some weeks, so since yesterday I'm on 78.
I did not see the high CPU usage but again after some hours of running, I saw the shutdown crash. I report it here, since I still suspect an interrelation, see:
Crash ID: bp-8ebffcf3-cc81-4c47-88e1-793c90210320

(In reply to Wayne Mery (:wsmwk) from comment #28)

https://profiler.firefox.com/public/4a4t6tmjk72tqgdcsay6c3t8zj839m69rjxf2c8/calltree/?globalTrackOrder=0&localTrackOrderByPid=42017-1-0~&thread=0&v=5 looks maybe like a bad folder

Actually, there is very high GC activity (garbage collection)

I too often see this after moving a big amount of messages (1000 .. 5000) between IMAP folders. Mostly only restart helps.

Not really interested in issues related to that - we know moving messages can be slow, so that is covered in other bug reports.

Great that you are now on version 78. Please capture a profile if you see slowness again.

I forgot to ask, do you have large numbers of messages that:

• are log files?
• have the same subject? (say, several hundred with the same subject)
• contain many URL, eg hundreds?

(In reply to Wayne Mery (:wsmwk) from comment #37)

Great that you are now on version 78. Please capture a profile if you see slowness again.

I again see high CPU usage: https://share.firefox.dev/3ffGxK6

(In reply to Wayne Mery (:wsmwk) from comment #38)

I forgot to ask, do you have large numbers of messages that:

• are log files?

I don't understand, what you mean by that.

• have the same subject? (say, several hundred with the same subject)

Hm, there are some with same prefixes like [MyGroup] from mailing lists, but not completely identical.

• contain many URL, eg hundreds?

I don't think so.

(In reply to Ulf Zibis from comment #39)

(In reply to Wayne Mery (:wsmwk) from comment #37)

Great that you are now on version 78. Please capture a profile if you see slowness again.

I again see high CPU usage: https://share.firefox.dev/3ffGxK6

About 8% is specifically in the sendlater add-on code. Additional activity may be a result of that. If performance is a consistent problem you might try running without sendlater to see if performance improves.

(In reply to Wayne Mery (:wsmwk) from comment #40)

About 8% is specifically in the sendlater add-on code. Additional activity may be a result of that. If performance is a consistent problem you might try running without sendlater to see if performance improves.

No Improvement after disabling SendLater add-on. While I tested this time, I see IMAP access blocking, see: Bug 1705351

Looking at the profile data in comment 39, it seems like it spends a lot of time (12%) in glibc realpath(), canonicalizing a filename. Here's another view of the same data:

https://profiler.firefox.com/public/rdf2a2bnfnqjg85m6b6a5naar473q5pa2g9hjy0/calltree/?globalTrackOrder=0&invertCallstack&localTrackOrderByPid=2257-0~&thread=0&v=5

I can't tell if it's

1. a single very slow call to realpath(), or
2. lots and lots of calls.

If it were 1, maybe there's some symlink oddness in the profile, requiring realpath() to grind hard (like a loop of symlinks or something)?
If it were 2, it would seem that folderURIToPath() in ext-mail.js is being seriously overused by something...

But I'm not sure 12% would account for the serious slowdown Ulf is running into...
In any case it'd be interesting to see a another (!) profile run over a slowdown, with SendLater disabled.

The crash stack in comment #36 looks like shutdown just getting fed up while waiting for a thread to shut down (I'd guess the IMAP thread;- ). So I'd say it's probably just a follow-on effect of whatever is causing the slowdown.

In any case it'd be interesting to see a another (!) profile run over a slowdown, with SendLater disabled.

Ulf, can you provide that?

(In reply to Wayne Mery (:wsmwk) from comment #43)

In any case it'd be interesting to see a another (!) profile run over a slowdown, with SendLater disabled.

Ulf, can you provide that?

https://share.firefox.dev/3wdMNH0

Another profile after I moved 5000 messages from INBOX to another folder on same IMAP server:
https://share.firefox.dev/3jVB6lA
I have 2 cores with 2.6 GHz an TB used alternately 100 % of one and then the other, changing every 10 seconds. It looked like on thread is always waiting for the other to finish.

Additionally after I had copied the profile link, TB was completely frozen, but consumed ~ 20 % CPU --> crash.

(In reply to Ulf Zibis from comment #45)

Another profile after I moved 5000 messages from INBOX to another folder on same IMAP server:
https://share.firefox.dev/3jVB6lA
I have 2 cores with 2.6 GHz an TB used alternately 100 % of one and then the other, changing every 10 seconds. It looked like on thread is always waiting for the other to finish.

13% in nsMsgDatabase::ClearHdrCache seems excessive.

Comment 45's profile looks quite different from comment 42's. And no folderURIToPath

Are you able to update to version 91?

(In reply to Ben Campbell from comment #42)

Looking at the profile data in comment 39, it seems like it spends a lot of time (12%) in glibc realpath(), canonicalizing a filename. Here's another view of the same data:

https://profiler.firefox.com/public/rdf2a2bnfnqjg85m6b6a5naar473q5pa2g9hjy0/calltree/?globalTrackOrder=0&invertCallstack&localTrackOrderByPid=2257-0~&thread=0&v=5

I can't tell if it's

1. a single very slow call to realpath(), or
2. lots and lots of calls.

If it were 1, maybe there's some symlink oddness in the profile, requiring realpath() to grind hard (like a loop of symlinks or something)?
If it were 2, it would seem that folderURIToPath() in ext-mail.js is being seriously overused by something...

But I'm not sure 12% would account for the serious slowdown Ulf is running into...
In any case it'd be interesting to see a another (!) profile run over a slowdown, with SendLater disabled.

Indeed, 18% was isDraftsFolder jar:file:///home/ich/.thunderbird/Profiles/i7jwyulg.default/extensions/sendlater3@kamens.us.xpi!/experiments/sl3u.js

(In reply to Wayne Mery (:wsmwk) from comment #47)

(In reply to Ulf Zibis from comment #45)

Another profile after I moved 5000 messages from INBOX to another folder on same IMAP server:
https://share.firefox.dev/3jVB6lA
Are you able to update to version 91?

I now have installed 91 via PPA https://launchpad.net/~mozillateam/+archive/ubuntu/ppa
Hopefully I'll see some progress soon.

Indeed, 18% was isDraftsFolder jar:file:///home/ich/.thunderbird/Profiles/i7jwyulg.default/extensions/sendlater3@kamens.us.xpi!/experiments/sl3u.js

So disabling SendLater had some effect, but didn't solve the general problem.

(In reply to Ulf Zibis from comment #0)

...
After some time of usage (4 hours + more) TB uses much CPU.
Adding addresses to To: fields becomes slower and slower, e.g. finding the address from the address book for the 10 th entry takes a minute. This behaviour gets worse and worse since 1 year.

(In reply to Ulf Zibis from comment #32)

(In reply to Wayne Mery (:wsmwk) from comment #31)

Please reproduce with Thunderbird started in safe mode, preferably with version 78

... using safe mode would be difficult to me, as often I have to wait more than 1 day of continuous usage of TB to notice the problem to occur and I need some addons in my normal work.

Do you still see the address autocomplete slow after four hours?

Periodically there is still some high CPU usage and slow responsiveness without doing much or knowing another cause, but since webrender is disabled (see bug 1772673 comment 15), it is much better.
Today I tried to reproduce the autocomplete case after running TB for 1 day. This time it was little slow for the first 3 addresses, but after (up to 20 entries), it worked fluently.

Thanks for the update.

Experiencing this issue again, see also: bug 1772673 comment 18

(In reply to Ulf Zibis from comment #52)

Experiencing this issue again, see also: bug 1772673 comment 18

Is the problem gone again, like bug 1772673 which was closed at the end of September?

Is this the Core Duo system?
With version 102?
Webrender disabled?

(In reply to Wayne Mery (:wsmwk) from comment #53)

Please see bug 1772673 comment #26. This issue now seems solved.