Open Bug 1342054 Opened 8 years ago Updated 4 years ago

Large number of tags causes very bad tags preference UI performance (from autotaging RSS feeds)

Categories

(Thunderbird :: Preferences, defect)

Product:
Thunderbird
Component:
Preferences
Version:
45 Branch
Platform:
Unspecified
All
Type:
defect
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

People

(Reporter: acarrico, Unassigned)

References

(
URL
)

Details

(Keywords: perf, reproducible)

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 Steps to reproduce: Name Thunderbird Version 45.6.0 User Agent Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 Profile Directory (Local drive) Application Build ID 20161231151914 This is an Arch Linux build, but I've also tried other recent Thunderbird builds from the Nix package manager with the same profile. Meta issue: UI is very slow, I've tried many of the Thunderbird troubleshooting ideas out there (safe mode doesn't help, other profiles are very fast, indexing is off). Specific issue: Preferences->Display->Tags has thousands (estimated) of tags, apparently from autotaging RSS feeds, which I have disabled. I've tried deleting these tags, but it isn't humanly possible to do so from the UI. Actual results: I can delete the tags one at a time with the button, but it would take days to do this by hand, especially with the sluggish UI. Thunderbird often uses 100% of one CPU, including while deleting tags in offline mode. I can't say if these tags are behind the UI sluggishness since there isn't an easy way to delete them. The <shift>-click or <control>-click tricks for highlighting many list elements doesn't work in the tags dialog. Expected results: I could delete tags quickly, ideally by selecting many and pushing the delete button. In a perfect world deleting all these unwanted/unused tags would sort out my UI sluggishness, but that remains to be scene. BTW I also noticed that the vast majority of the elements in the config editor are of the form: mailnews.tags.*.tag mailnews.tags.*.ordinal
Yes, this is a problem due to the internal tag design, from when there were 5 hardcoded tags and people wanted custom tags.. Each tag is 2 prefs and the filter dialog is very slow as well due to the menulist being built before drawing the window. Your best workaround is to close Tb and manually delete the lines in the [profile dir]/prefs.js file. The usefulness of feed tagging very highly depends on the individual feed.
Status: UNCONFIRMED → NEW
Component: Untriaged → Mail Window Front End
Ever confirmed: true
Ok. Wow, I deleted 90,000 tags from prefs.js using emacs. I think this may have finally solved months of Thunderbird UI performance troubleshooting. I'll reopen / file new bug if necessary in a few days, but let's tentatively call this one solved. THANK YOU. This issue might deserve an official blog post or something, because none of my searches have put a spotlight on this issue which can really make Thunderbird unusable. I'd mark this RESOLVED, but not sure which status to use, and also you might want to keep it open for a technical or social fix since Thunderbird clearly has real trouble with thousands of tags. Perhaps even just a strong warning about the bug in the documentation would be a good start.
It should be left open; the entire design needs to be rewritten to handle volume. In addition to the perpetual lost tags in imap problem and several others listed in other bugs.
Keywords: perf
Summary: 1000s of autotags, can't delete, related? whole UI very slow → Large number of tags causes very bad UI performance
For the record, in Bug 996690 the original patch omitted autotags from widgets..
Anthony, is tags preference UI the only area that seems slow to you? For example, getting new RSS is not slow?
Component: Mail Window Front End → Preferences
Flags: needinfo?(acarrico)
Summary: Large number of tags causes very bad UI performance → Large number of tags causes very bad tags preference UI performance (from autotaging RSS feeds)
The entire UI was slow with 90,000ish tags. Drop down menus, scrolling, reaction to clicks, etc. Not just the preferences, but the main application window.
Now that I'm back to the default tag set the UI behaves as expected.
... three months later, Thunderbird 52.1.1, arch linux: Things had been smooth, but the upgrade to 52 has had major issues, with random crashes and other weird behavior (see https://bugzilla.mozilla.org/show_bug.cgi?id=1362895#c3). I also noticed sluggish UI, so I checked the tags, and sure enough there were 121 new tags, apparently harvested from feeds. I deleted the tags. The global autotag preference is off, but tagging is happening, so I checked each feed manually. Some have DO have autotagging enabled, seemingly at random. Incidentally, when I clicked to disable autotagging for the first autotagged feed, Thunderbird crashed, but I think this is due to other problems (see the other bug).
Flags: needinfo?(acarrico)
(In reply to Anthony Carrico from comment #8) > ... three months later, Thunderbird 52.1.1, arch linux: > > ... I also noticed > sluggish UI, so I checked the tags, and sure enough there were 121 new tags, > apparently harvested from feeds. I deleted the tags. What parts of Thunderbird are sluggish?? > The global autotag preference is off, but tagging is happening, so I checked > each feed manually. Some have DO have autotagging enabled, seemingly at > random. Incidentally, when I clicked to disable autotagging for the first > autotagged feed, Thunderbird crashed, but I think this is due to other > problems (see the other bug). Please file a different bug for this, and also for the crashing
URL: https://www.reddit.com/r/Thunderbird/...
Severity: normal → major
Flags: needinfo?(acarrico)
(In reply to alta88 from comment #3) > It should be left open; the entire design needs to be rewritten to handle > volume. Ouch! Here is another report https://www.reddit.com/r/Thunderbird/comments/4jw6qv/is_it_save_to_remove_unnecessary_tags_from_prefsjs/ Is this auto-tagging of serious value considering the problems it causes? > In addition to the perpetual lost tags in imap problem and several others listed in other bugs. Which bugs are you talking about?
First, the defaults for each individual feed and the global setting is off. It is totally up to the user to enable it. If the global setting is enabled, all subsequent newly added feeds will thus be enabled. (That's what the OP did, it's not at all "seemingly at random"). The updated account settings UI in Tb54 makes it more clear the settings there are for new feeds and not global; all existing feeds are only affected by their own settings in Subscribe. Anyway, 121 tags will not make anything sluggish, so best look elsewhere. Also, "The usefulness of feed tagging very highly depends on the individual feed", so it is, and should be, up to the user to decide whether it should be on for a particular feed. A statement in the Learn More guide about being careful with autotagging of feeds is appropriate (rather than panic). I don't have a list of bugs handy, but surely you're aware that lost mail tags are a persistent complaint.. The root fix here is twofold: 1) Tags and their metadata should not be 2 prefs per tag, but stored in a json file. It would be reasonable to have a configurable max tags throttle. 2) Widgets should not attempt to create a menu or list with thousands of tags, the original design considered 5 tags. And the management tab in Preferences should be much smarter about loading/be async/yield, etc.
> I don't have a list of bugs handy, but surely you're aware that lost mail tags are a persistent complaint. There are many possible causes for lost tags. To which are you referring? Dumb imap servers? Loss after compact? etc. (the ones I have stated have little or nothing to do with RSS tags, so I don't know what you are talking about else I would not have asked)
I meant not only the IMAP case, but everything else to do with tag management, as part of a revamp of the whole thing. Feeds have nothing to do with it other than demonstrating how poor the implementation is. https://bugzilla.mozilla.org/buglist.cgi?list_id=13609711&v2=Feed%20Reader&short_desc=tag&f4=CP&product=MailNews%20Core&product=Thunderbird&short_desc_type=anywords&f1=OP&f0=OP&resolution=---&classification=Client%20Software&classification=Components&query_format=advanced&j1=OR&f3=CP
> Please file a different bug for this, and also for the crashing I did reference another bug above, where I detailed my problems and solutions for 45->52 issues. I noticed this little reprise of the tagging along the way, and so I added an update here where it is relevant.
Flags: needinfo?(acarrico)
(In reply to alta88 from comment #11) > First, the defaults for each individual feed and the global setting is off. > It is totally up to the user to enable it. If the global setting is enabled, > all subsequent newly added feeds will thus be enabled. (That's what the OP > did, it's not at all "seemingly at random"). While debugging the major 45->52 issues, I noticed new tags had been harvested. Since autotags had been a major source of trouble in the past, I walked the RSS tree and found a few "random" feeds that were still tagging. I made a note here so others will know to manually check all their feeds, and not to count on the global setting when addressing this tag bug for themselves. This process was very crashy for me, but I believe this was because of other 45->52 upgrade issues which I have solved and noted in the other bug report.

If the claim here is that all UI is sluggish with many tags, we need to test that (I can try).

So it seems we can distinguish which tags are autogenerated, via tag.ordinal.contains("~AUTOTAG").
Can we have two methods to enumerate all tags? One that returns these autotags and one that doesn't.
It may be that we reference the tag list in many places. You could just make the default enumeration function (getAllTags) return only the real tags so that all UI is happy. Only the single place that uses the autotags would call the new method that returns all tags (or just the autotags actually).

Another solution could be to make nsMsgTagService::GetAllTags faster by returning all tags from a cache, not always re-read all tag pref keys (thousands), sort them and then get the pref values and build the tags and return the list.

If it's not that nsMsgTagService::GetAllTags is slow, but the building of menulists with thousands of items is slow, then you could salvage the parts of bug 996690 and finish them here. Looks like bug 996690 comment 3 wasn't implemented yet.

Also, we could allow deleting multiple tags from the preferences dialog, if we actually want to show the autotags there. We could sort the autotags to the bottom of the list so that user could delete them in one go (select start and end of the range).

I reviewed all RSS feeds options and found that ONE, which has the autogeneration of tags option is ON. There where 54000+ auto tags created. After turning that option off my prefs.js shrinkeв down to 200kB from 4.5 mB size.

RESOLVED.

If you have an ideas how to improve it you are welcome.
Thanks a lot from Russia.
My best regards to developers.

:aceman notes in Comment 19:

Also, we could allow deleting multiple tags from the preferences dialog, if we actually want to show the autotags there. We could sort the autotags to the bottom of the list so that user could delete them in one go (select start and end range).

I like it. No need to edit prefs.js manually.

Keywords: reproducible
OS: Unspecified → All
You need to log in before you can comment on or make changes to this bug.