Open Bug 1801464 Opened 2 years ago Updated 1 year ago

Message filter fails to move incoming messages from POP inbox when condition exists on message body and maildir format is used

Categories

(Thunderbird :: Filters, defect)

Thunderbird 102
Desktop
Windows
defect

Tracking

(Not tracked)

UNCONFIRMED

People

(Reporter: ebretscher, Unassigned)

References

(Depends on 1 open bug)

Details

(Whiteboard: [filterfail])

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0

Steps to reproduce:

  1. Create filter against POP account Inbox:
    "Match all of the rules"
    Rule 1 - "Sender" "is" "me@ymail.com"
    Rule 2 - "Body" "doesn't contain" "some text"
    Action 1 - Move message to "Local Folders/Deleted"
  2. Send e-mail to POP account from me@ymail.com without "some text" in body
  3. Check POP account for new messages

Actual results:

The message stays in the inbox and an empty index entry is created in the target folder.

FILTER LOG
[19/11/2022, 10:59:32 pm] Applied filter "Test filter" to message from Me <me@ymail.com> - Test10 at 19/11/2022, 10:58:58 pm moved message id = 343bc029-b188-62f3-fe70-9b03390e3711@ymail.com to mailbox://nobody@Local%20Folders/Trash
[19/11/2022, 10:59:32 pm] Filter Action Failed: "Move failed" with error code=0x8055000f while attempting: Applied filter "Test filter" to message from Me <me@ymail.com> - Test10 at 19/11/2022, 10:58:58 pm moved message id = 343bc029-b188-62f3-fe70-9b03390e3711@ymail.com to mailbox://nobody@Local%20Folders/Trash
[19/11/2022, 10:59:32 pm] Filter Action Failed: "Failed applying the filter action" with error code=0x8055000f while attempting: Applied filter "Test filter" to message from Me <me@ymail.com> - Test10 at 19/11/2022, 10:58:58 pm moved message id = 343bc029-b188-62f3-fe70-9b03390e3711@ymail.com to mailbox://nobody@Local%20Folders/Trash
[19/11/2022, 10:59:32 pm] Message from filter "Test filter": Applying filter actions failed
[19/11/2022, 10:59:32 pm] Filter run failed

Expected results:

The message should have been moved to Local Folders/Deleted when it arrived.

Notes:
1/ The filter does work if run manually against the inbox afterwards.
2/ The apparent cause is the rule on the message body content, as removing it clears the error.

Found another issue (MozillaZine) with same error code: code=0x8055000f: Filter action failed - error code=0x8055000f.

To quote some support advice:

  • "It would help if you copied and pasted the contents of the msgFilterRules.dat file and the path for the profile and/or the folder."
  • Comment in old bug 1367438 from TB52 mentions "Looks like 0x8055000f is NS_MSG_ERROR_WRITING_MAIL_FOLDER" . One of the comments references https://bugzilla.mozilla.org/show_bug.cgi?id=931303#c43 which talks about something is deleting the *.msf files.
  • Check that your *.msf files for those folders still exist. They are supposed to be automatically re-created if they don't exist. You might try using "repair folder" in the folders properties in the folder pane." - Although I am not sure if repairing is the right approach for a pop3 account. I personally use IMAP.

Two things I personally would recommend:

  1. Check if all msf files for your folders exist.
  2. What also comes to mind: In 2022 for TB102, there has been a (partial?) rewrite of the pop3 implementation into Javascript and during this, they also changed the way folder names are stored. You might want to check how your actual folder names are stored (e.g. also in the msf files) and compare this to the folder name that is in your filterrules.dat

Example for change of folder names in TB102:
bug 1778064 comment 2

I have looked at a backup of the msgFilterRules.dat (2 month old)
Here is an extract for one filer

  • Old (I was using Thunderbid 91) :
    actionValue="mailbox://jalios08@pop.jalios.com/1.%20Dev/SVN"
  • New (Thunderbid 102 since yesterday, but I dont know if this actionValue was modified before ) :
    actionValue="mailbox://olivier.jaquemet%40jalios.com@mail.exchangeincloud.com/1.%20Dev/SVN"

I HAVE NOT changed the username/hostname during that time

Keep in mind that the filter works perfectly when it is run manually afterwards against the inbox folder, i.e. there isn't much wrong with the configuration. It may be a timing issue when new messages arrive between the download of the header vs message body. It wouldn't be able to parse the message body to test the contents and/or move the message if it hadn't been fully received yet for example.

msgFilterRules.dat contains plenty of rather personal data, but the relevant entry looks like:

name="Some mail segregation"
enabled="yes"
type="17"
action="Move to folder"
actionValue="mailbox://nobody@Local%20Folders/Misc/Category"
action="Stop execution"
condition="AND (from,is,noreply@website.com) AND (body,doesn't contain,some text)"

The POP account inbox has a .msf file indeed and the path is C:\Users\Me\AppData\Roaming\Thunderbird\Profiles\h45qnval.default-release\Mail\pop.mail.yahoo.com. The local folders have .msf files as well for each folder and subfolders. Everything is stored in the maildir format.

Whiteboard: [filterfail]

Danny, any theories?

Flags: needinfo?(dannyfox)

I don't -- except the obvious that something changed in 102.

But I'm wondering about the Action 1: Move message to "Local Folders/Deleted"
Is this folder really the TRASH folder (where the log says it goes)? Why not explicitly say so, or execute a DELETE instead of MOVE? (Maybe TRASH and DELETED are coded as synonyms in older versions and not in 102?)

Flags: needinfo?(dannyfox)

(In reply to Dan Pernokis from comment #5)

I don't -- except the obvious that something changed in 102.

But I'm wondering about the Action 1: Move message to "Local Folders/Deleted"
Is this folder really the TRASH folder (where the log says it goes)? Why not explicitly say so, or execute a DELETE instead of MOVE? (Maybe TRASH and DELETED are coded as synonyms in older versions and not in 102?)

Dan/Wayne,

Action 1 came from a test case I had built. It fails the same moving the message to a specific local folder, as in my Comment #3, in which case it has to be a MOVE. So it has nothing to do with the Deleted/Trash folders.

It is as if the rule was running before the body of the message had downloaded to the Inbox, and then evaluating the condition on the body fails. Execution of the actions follows with an entry getting created in the .msf file of the target folder for the message, but it is invalid because it doesn't point at any message (failed MOVE) and the message stays in the Inbox. Again, maybe because the message hasn't yet fully downloaded and therefore cannot be moved. It is purely speculative of course.

The issue seems to have been corrected in the latest releases, now 102.12.0. The message filter now seems to be moving messages without the earlier glitches when there is a test on the body content. It was a systematic problem before, but I haven't seen it for some time now.

Please consider closing the issue, if it is resolved.

I decided to stage a deliberate test before closing the matter, but unfortunately the filter still fails the same. It creates an entry in the index of the target folder and leaves the message in the inbox. It is an easy bug to confirm.

OS: Unspecified → Windows
Hardware: Unspecified → Desktop

Things you could try:

a) use "filter after junk classification"
b) use "move later" filter action from filtaquilla addon
c) Avoid placing filters that will likely trigger on the same message right next to each other. Place some other filters inbetween or in front of this filter to give Thunderbird time to finish earlier operations. This will hopefully avoid any race conditions, if there are any.
d) This is grasping at straws: Test on hardware with great internet connection and low latency. Maybe Thunderbird is faster than your internet connection?
e) Test with a new and clean installation of Thunderbird without using your old profile. Check if results are similar.

Some interesting suggestions, thanks. Here are some results:

a) Running the filter "after junk classification" eliminates the issue, for some reason! Less desirable, but a useful workaround.
b) Using "Move later" from the filtaquilla add-on does resolve the issue when the filter runs "before junk classification".
c) Not applicable. It is the only matching filter for the message, but I moved the test filter to the top of the list with "Stop filter execution" as last action: the problem persists.
d) Thunderbird will literally always be faster because of the latency at the remote e-mail server... see e)
e) I installed TB 102.12.0 in a new sandboxed environment and could NOT reproduce the problem, regardless of the junk classification setting. Same internet connection and mail server. HOWEVER... there was one difference between the two installations: the accounts (including the Local Folders where the messages are to be moved to by the filter) were in Mbox format on the test setup, while the production environment is in Maildir format. I re-installed TB 102.12.0, but this time selected Maildir storage before creating any accounts. The filter problem is back and behaves exactly like in the production environment.

The results a) and b) could suggest conflict/timing issues with junk mail classification when Maildir is enabled, but not with MBox.

This also explains why it might have been hard to reproduce the bug, the installation must also use Maildir.

Summary: Message filter fails to move incoming messages from POP inbox when condition exists on message body → Message filter fails to move incoming messages from POP inbox when condition exists on message body and maildir format is used
Depends on: 662056
You need to log in before you can comment on or make changes to this bug.