jwq Reporter
	Description • 13 years ago

Thunderbird 3.1.17 / Lightning 1.0b2 / Enigmail 1.1.2

 When more than N folders are selected for a saved search in a POP3 account, with one or the other of the above extensions installed, then when the saved search is selected (other folders are unaffected):

1. only the first message is displayed, and subsequent messages are not displayed, in the message pane

2. Thunderbird deletes a small number (1 to 8, approx.) msf files from the within the profile Mail sub-directories, typically those last in a traversal of the folder tree.

3. Errors are displayed in the Error Console, in the system Console.log file and through error dialogs. Javascript Error Console logged errors most commonly include the following:

     Error: Component returned failure code: 0x80550006 [nsIMsgFolder.msgDatabase] = <unknown>
     Source file: chrome://messenger/content/mailWidgets.xml
     Line: 2144

     Error: Component returned failure code: 0x80550007 [nsIMsgFolder.getStringProperty] = <unknown>
     Source file: chrome://messenger/content/folderPane.js
     Line: 2441

     [The latter error is caused by the missing msf files and is repeated many times]

and similar errors.

Also observed are address book errors on opening the Message Pane, such as the error dialog

     Unable to Load Address Book File

     Unable to load address book file abook.mab. It may 
     be read-only, or locked by another application.
     Please try again later.

or system Console.log errors

     yyyy-mm-dd hh:mm:ss.sss thunderbird-bin[480] couldn't write file '/Users/jwq/Library/Caches/com.apple.AddressBook/MetaData/xxxx-xxxxx-xxx:ABPerson.abcdp'

or

     Couldn't reopen AddressBook database. Home directory may not be ready or is missing.

and similar.

 Enigamil occasionally fails to load if the Saved Search folder is navigated to directly, with error dialog

     Enigmail: Error in accessing Enigmail service

     To permanently avoid this alert, either fix the problem or
     uninstall Enigmail. Click on the Help button for more
     details.

and Error Console messages similar to the above. In particular, sometimes message decrypt will fail in the Saved Search Message Pane and the following will appear in the Enigmail Console

     enigmail.js: Enigmail.execStart: Error - Failed to start PipeTransport

Messages outside of the Saved Search folder will open without problem.

 Similarly, Lightning malfunctions and reports errors to the Javascript Error Console and the system Console.log.

 Some errors indicate access problems (NS_ERROR_FILE_ACCESS_DENIED) or errors reading the mail DB (Error: DB error: unable to open database file) or (Error: DB error: library routine called out of sequence)

 As the number of searched mailboxes approaches the threshold N (actual number depends on the system) from below, the performance of Thunderbird / Enigmail / Lightning slows significantly.

	jwq Reporter
	Comment 1 • 13 years ago

WORKAROUND: All of these can be worked around by either reducing the number of mailboxes searched by the Saved Search below the threshold N, or by uninstalling the extensions!

	David :Bienvenu
	Comment 2 • 13 years ago

this might be a file handles issue

	jwq Reporter
	Comment 3 • 13 years ago

FWIW, on my system (Mac OS 10.4.11, 1 GB RAM) the number of searched mailboxes requried to show the symptoms is a somewhat modest N = 193, give or take.

	Wayne Mery (:wsmwk)
	Comment 4 • 13 years ago

(In reply to David :Bienvenu from comment #2)
> this might be a file handles issue

making this invalid?

	jwq Reporter
	Comment 5 • 13 years ago

 Following David's suggestion, I looked into file descriptor limits in Mac OS X. I found that there is a limit to the number of open file descriptors per process which defaults to 256.

 This limit can be increased for the current shell session process and the new settings are then inherited by all child processes of the shell session. So, executing the following in the Terminal (bash):

    $ ulimit -S -n 512
    $ Applications/Thunderbird.app/Contents/MacOS/thunderbird-bin

and selecting all 211 folders for my Saved Search, I found Thunderbird did not malfunction. Restoring the default limit 256 and restarting Thunderbird saw the same malfunctions originally reported.

 This isn't invalid, there are at least two (possibly three) bugs in Thunderbird which contribute to the observed malfunctions:

    1. The situation when the maximum open file descriptor limit is hit is
       not handled gracefully. It should be (I believe a "Maximum open 
       files limit reached" dialog is traditional).

    2. The open file descriptor limit could be raised at launch time if 
       it is possible the profile's folder structure could lead to the 
       limit being hit by a saved search.

    3. The code could (maybe) be rewritten so that Saved Search doesn't 
       hold open the file descriptors of all the msf for the selected 
       folders in the Saved Search.

Based on what I have observed, I strongly suspect that this bug affects the latest version under Mac OS X and, probably, Linux (depending on what 'ulimit -a' reports for the open files limit). Would someone who is able to run a later version care to test this hypothesis?

	David :Bienvenu
	Comment 6 • 13 years ago

(In reply to jwq from comment #5)
>     3. The code could (maybe) be rewritten so that Saved Search doesn't 
>        hold open the file descriptors of all the msf for the selected 
>        folders in the Saved Search.
That would break updating of open saved searches, so it's a non-starter. We'll need to rewrite the db code not to use a single db file per folder. That's going to take a long time.

	WADA:World Anti-bad-Duping Agency
	Comment 7 • 10 years ago

Search folder opens all search target folders, so it may exceeds number of file handle limit.
Setting dependency to bug 495911 for ease of problem analysis, tracking, searcj.

	Wayne Mery (:wsmwk)
	Comment 8 • 9 years ago

This should be somewhat improved in version 15 after 2012-04-26 via bug 723248 which attempts to impose a limit of 30 open folders.

However, that patch had a couple issues solved in version 38 via bug 1135310

	Wayne Mery (:wsmwk)
	Comment 9 • 6 years ago

jwq, do you still see this using a current version and similar profile?

I'm not suggesting we close this, because I am not aware that we have explicitly fixed anything to help this.  But I am curious to what extent you still see the issue.

	Wayne Mery (:wsmwk)
	Comment 10 • 6 years ago

Although you might not be seeing this from using bug 800279 comment 22