Open Bug 622307 Opened 14 years ago Updated 2 years ago

Manual filter with custom header not working ("Run Filters on Message"/"Run Filters on Selected Message" only problem. No problem if "Run Filters on Folder")

Categories

(MailNews Core :: Filters, defect)

1.9.1 Branch
x86
All
defect

Tracking

(Not tracked)

People

(Reporter: duerst, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [filter-mgmt])

Attachments

(1 file)

User-Agent: Opera/9.80 (Windows NT 6.0; U; en) Presto/2.7.62 Version/11.00 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.9) Gecko/20100722 Eudora/3.0.4 I am subscribed to a lot of mailing lists. I want to read the mails in my inbox and then file them by "Run Filters on Message". This works for standard, built-in headers such as "Subject", "From",... But it does not work for custom headers such as "List-Id", "Sender", "List-Subscribe", and so on. (The custom headers work when they are invoked automatically during mail checking/download.) Reproducible: Always Steps to Reproduce: 1. Create custom header, e.g. List-Id 2. Create filter, e.g. List-Id contains i-d-announce.ietf.org, set to "Run Manually" 3. Invoke filter manually with "Run Filters on Message" on appropriate message. Actual Results: Nothing happens. Expected Results: Filter action (e.g. add a star, file in a folder,...) should be applied.
Can you enable the filter log and tell us what it contains when you run the filters manually ?
Component: General → Filters
Product: Thunderbird → MailNews Core
QA Contact: general → filters
Version: unspecified → 1.9.1 Branch
For a manual filter on a custom header field such as List-Id, where the filter actually doesn't get applied, there is also no entry in the filter log.
(In reply to comment #2) > For a manual filter on a custom header field such as List-Id, where the filter > actually doesn't get applied, there is also no entry in the filter log. These are working for me. Does starting Thunderbird in -safe-mode helps (see http://support.mozillamessaging.com/en-US/kb/Safe+Mode) ?
Same here. I set up a filter on the custom header "List-Id", and set it to move the message to the list folder in the same IMAP account. Manually applying the filter doesn't do a thing. Other custom headers (List-Post etc.) don't work, either. Also no output in filterlog.html. With the predefined filter headers, everything workds. System: Ubuntu 10.04 Lucid Lynx Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7
Same problem here. A service called Boxbe can add the following headers to a message: Boxbe-Signature Boxbe-Spam-Score I tried creating a rule that checks for specific values of Boxbe-Spam-Score (since you cannot add a type to a customer header and get the "greater than" and "less than" options). A test rule looking for ";" (no quotes) in Boxbe-Signature also failed to find any messages even though a semi-colon is always present in the header value. An "exists" test to check if a header exists would be useful (e.g. If "Boxbe-Spam-Score" header exists, move message to "Waiting List" folder.) I am running Windows 7 64-bit. Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9
It appears I'm having the same problem with TB 5.0. I updated from TB 3.1.x the other day. After this filtering for the custom header "Delivered-To" stopped working. The filter is set to be applied upon downloading new messages and upon manual execution, but neither works. I didn't touch the filter after the update and in TB 3.x everything was fine. Unfortunately it seems I can't get any logs, as the filter simply is not applied. Curiously all the other filters, including the ones for other custom headers, still seem to work. The filter looks like this: --- snip --- name="vertretung-someone" enabled="yes" type="17" action="Move to folder" actionValue="imap://doerges@imap.host/INBOX/Vertretung/someone" condition="AND (\"delivered-to\",contains,someone@pre)" --- snap --- I'd be happy to provide more information.
(In reply to Till Dörges from comment #6) > It appears I'm having the same problem with TB 5.0. I updated from TB 3.1.x > the other day. After this filtering for the custom header "Delivered-To" > stopped working. Any news on this? Is the Delivered-To issue perhaps a different bug?
OK, I can see this on TB7, linux. I tried it with "X-Envelope-To". - Invoking "Run Filters on Message" on the message that should be filtered DOES NOT work. (this is the original report). - Invoking "Run Filters on Folder" DOES work. - Opening "Message Filters..." selecting the filter with custom headers and clicking "Run now" DOES NOT work. Till, which of these commands are you invoking that don't work for you? If your filter doesn't work even when automatically downloading messages or when invoking "Run Filters on Folder" then it probably isn't this bug.
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Windows Vista → All
(In reply to aceman from comment #8) > Till, which of these commands are you invoking that don't work for you? If > your filter doesn't work even when automatically downloading messages or > when invoking "Run Filters on Folder" then it probably isn't this bug. It appears as if things changed for the better. :-) I'm now running TB 7.0.1. Quick tests for my particular filter show this: - Extras -> Filter on Folder: Does work - Extras -> Filter on Message: Does work - Extras -> Filter -> Run now: Does work - Filter autom. when downloading: Does work I did not test TB6 and TB7.
this affects management of filters, to the extent that one is not able to test a filter that might be wanted in automatic mode
Whiteboard: [filter-mgmt]
Martin, what about you? Is it any better in TB 8?
Depends on: 697522
Sorry, I'm using Eudora OSE, which seems to be stuck at TB 3.0.4.
To all problem reporters in this bug: Local mail folder(POP3 or pseudo account of "Local Folders")? Or IMAP mail folder? If IMAP mail folder, is offline-use=Off folder only problem? (Folder Properties/Synchronization) If IMAP folder & offline-use=Off folder only problem, dup of bug 184490?
(In reply to WADA from comment #13) > Local mail folder(POP3 or pseudo account of "Local Folders")? POP3 in my case.
For (In reply to Martin Dürst from comment #14) > POP3 in my case. IMAP relevant phenomenon such as bug 184490 can be ruled out. (In reply to Martin Dürst from comment #0) > But it does not work for custom headers such as "List-Id", "Sender", "List-Subscribe", and so on. > (The custom headers work when they are invoked automatically during > mail checking/download.) For "Sender": dup of 525022. Is next by you applicable to Sender case? > (The custom headers work when they are invoked automatically during > mail checking/download.) For "List-ID": bug 564189 is for "multiple line List-ID" case. IIRC, this is next; (a) When locally held mail(local mail folder or IMAP folder of offline-use=On), filter doesn't process multiple lines well. Searches first line of folded header only. (b) When IMAP and filtered upon download time, header is passed to filter with unfolded format, then whole header is searched. If next by you is applicable to your "List-ID" & POP3 case, (b) is perhaps applicable to POP3 too. > (The custom headers work when they are invoked automatically during > mail checking/download.) Folded List-ID: header case? (multiple line List-ID) Is "multiple LIST-ID: headers" case involved in your case? For "List-Subscribe", and so on. A variant of bug 564189? (List-Subscribe etc. version of bug 564189 on List-ID)
For "multiple same message headers" case: bug 310189 exists for multiple Subject: case(malformed mail). IIRC, this problem is applicable to any message header for which multiple headers is permitted, except Received: header. If Received: header, special care is done to it and multiple Received: headers are searched by Tb's filter/search code.
Bug 678322 was for "multiple same headers" case. See bug 678322 comment #11 for special treatment on Received: header.
Failed to linkify. For "Sender": dup of bug 525022.
Bug 435694 is for customized header of "Status" case.
My comment 8 is on POP3 (or News, must check). I'll check if it is a problem with duplicate headers. However, that would still not explain why would "Run filters on folder" and "run filters on message" behave differently.
(In reply to WADA from comment #15) > (In reply to Martin Dürst from comment #0) > > But it does not work for custom headers such as "List-Id", "Sender", "List-Subscribe", and so on. > > (The custom headers work when they are invoked automatically during > > mail checking/download.) > > For "Sender": > dup of 525022. > Is next by you applicable to Sender case? I tried again with "Sender", and that worked. But "List-Id" still isn't working. > > (The custom headers work when they are invoked automatically during > > mail checking/download.) > > For "List-ID": > bug 564189 is for "multiple line List-ID" case. > IIRC, this is next; > (a) When locally held mail(local mail folder or IMAP folder of > offline-use=On), > filter doesn't process multiple lines well. > Searches first line of folded header only. None of the headers I tried to match against were/are multiline.
(In reply to Martin Dürst from comment #21) > I tried again with "Sender", and that worked. But "List-Id" still isn't > working. Actually, I have to retract that. It worked for one or two mails, but then stopped. I suspect that this may have been due to the fact that the old filter (before I switched to using Sender) was still active for a while. But that's just a wild guess.
(In reply to Martin Dürst from comment #21) > But "List-Id" still isn't working. > None of the headers I tried to match against were/are multiline. I still couldn't reproduce problem on single line List-ID: header using Tb 8.0 on Win, with "Run Filters on Folder". (Bug 435694 for Status, bug 525022 for Sender, were already worksforme if "Run Filters on Folder".) Create customized header of "List-Id". Filter rule in msgFilterRules.dat > name="List-Id(customized) contains: abcdefg" > enabled="yes" > type="17" > action="AddTag" > actionValue="$label3" > condition="AND (\"list-id\",contains,abcdefg)" Test mail's header. (Local mail folder. Single line header. No other string of "abcdefg" in mail) > List-ID: abcdefg Select mail folder which contains the test mail, and do "Run Filters on Folder". Message filter log. > Applied filter "List-Id(customized) contains: abcdefg" to message > from x x <x1@x.x.x> - abc-def at 2011/02/23 03:22:31 tagged If this mail is selected at thread pane and is shown in message pane, color of mail at thread pane is changed just after filter run, but "tags Personal" is not automatically shown at message header pane. To see "tags Personal" at message header pane, "select other mail, then select the mail again" is required. If "Run Filters on Message"(single message is selected) and "Run Filters on Selected Messages"(multiple messages are selected), any filter rule of customized List-Id, customized Status, customized Sender, didn't fire even though single line header as you and :aceman say, although filter rule with standard criteria like "subject contains -, add Tag" fired as expected. Adding "Run Filters on Message" in bug summary. To Martin Dürst(bug opener) and :aceman, can you see problem on single line header with "Run Filters on Folder"?
Summary: Manual filter with custom header not working → Manual filter with custom header not working ("Run Filters on Message"/"Run Filters on Selected Message" only problem. No problem if "Run Filters on Folder")
Attached file testcase (mbox file) (deleted) —
Testcase: 1. Save this file into the mail directory so that it appears as a new folder after TB start. 2. Create this filter (or add it manually into the msgfilterrules.dat file): version="9" logging="yes" name="X-Envelope-To contains: webp1@domain.dummy.tld" enabled="yes" type="17" action="Mark flagged" condition="AND (\"x-envelope-to\",contains,webp1@domain.dummy.tld)" 3. Select both messages in this folder. 4. Run "Run filters on Folder" - WORKS 5. Run "Run filters on Selected Messages" - DOES NOT WORK 6. Open Filters, choose this filter, selected the proper folder and press Run Now - WORKS. This is on Win XP, Today's TB 11. WADA, can you reproduce it with these files?
If I create this filter (standard header with one occurrence: name="Subject contains: should" enabled="yes" type="17" action="Mark flagged" condition="AND (subject,contains,should)" ..the filter works in all three ways of invocation.
Dupe of bug 184490, at least as far as imap is concerned, no?
Not according to comment 15.
Blocks: 716551
I'm seeing the same behavior as the original bug report. I'm using TB 17.0.7.
I am running 24.6.0 and experience same problem as original post with the exception I cannot get it to apply AT ALL when comparing custom headers Example I have a mail in my IMAP account 'Inbox' that contains in the headers: X-hMailServer-ExternalAccount: decdotuk I have a filter in Thunderbird that has a CUSTOM HEADER set up to read 'X-hMailServer-ExternalAccount' and then checks this 'Customize' contains decdotuk (where customise is the above field selected) action: apply a STAR (or anything actually) Result when running 'on folder', or running 'manually', or running 'on message': NOTHING!
According to my tests as described here http://superuser.com/questions/904628/thunderbird-31-6-does-not-filter-by-reply-to-email-address not even the "No problem if "Run Filters on Folder" part of this appears to be broken as well. We really need this to work. Nowadays many SPAM mails have a "Reply-To" set which we'd like to filter for. There are 2 solutions: 1. Fix this bug fairly quickly OR 2. Offer a "Reply-To" in the dropdown box of the non-custom headers (preferred). NOTE: Entries with multiple headers "To, From, CC, or BCC" don't make that much sense since every filter can be defined with AT LEAST ONE filter expression to match...
I am experiencing this in Thunderbird 38.2.0.
Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.
I have this problem also, with TB 38.3.
The problem went away for me after I deleted and recreated the .msf file for the destination file for Cactus Spam. Cactus Spam was not moving spam into that file, because somehow the .msf file had been corrupted.
Ok, I have just done a complete re-test and confirm that it seems to be working now. To be clear, the (previously failing) test was: 1, Receive an email that may have custom headers in it. (eg, X-customheader-set) 2, set up a message filter to look for and action on that customer header (X-customheader-set) 3, Run Manually the new filter The result was that the filter didnt detect the headers. HOWEVER.... after reading this and the other bug 184490 https://bugzilla.mozilla.org/show_bug.cgi?id=184490 I see that Thunderbird needs to have the CUSTOM HEADER already defined in the Message Filter Setup (and therefore begins to record the occurrence) of the header prior to the messages being received. It is because the customer header is being set up AFTER the message arrives that it is not being found. SO THE SOLUTION IS: After you set up a new CUSTOMER HEADER (in Message Filter setup) to search for in the Filters List, you need to do a 'Folder Repair' (which rebuilds the MSF file for the folder, as Dan Meek mentions above) on the folders concerned. This then forces Thunderbird to record the existence of the custom header in the MSF file and you will then see that the messages are actioned accordingly to the new filter. Also, any NEW messages received AFTER THE CREATION of your customer header will be actioned on automatically (according to your filter definition) So, to set up a NEW filter for Custom Headers on existing messages, the process should be: 1, Receive an email that may have custom headers in it. (eg, X-customheader-set) 2, set up a message filter to look for and action on that customer header (X-customheader-set) 3, do FOLDER REBUILD on the folders that you will be running this filter against. 4, (Manually Run the new filter if you need to - depending on your Filter definition) 5, Any NEW messages with X-customheader-set will be actioned accordingly as per your Filter For me there is now no problem regarding this 'issue'. Using TB 31.6
Just tried the same on latest 38.3.0 and the results are the same. Issue fixed (for me) if you follow the advice in the last post https://bugzilla.mozilla.org/show_bug.cgi?id=622307#c35
For me these are not working even if "Run Filters on Folder" or the new "Run Filters on Message" are used. I do not regard this as solved yet.
I too still have a problem -- "Reply-To", which I added by using the "Custom" button, does not trigger filters. The "To" header works. I'm using Seamonkey-2.46
(In reply to kwutzke from comment #30) > not even the "No problem if "Run Filters on Folder" part of this appears to > be broken as well. (In reply to John David Galt from comment #37) > For me these are not working even if "Run Filters on Folder" or the new "Run > Filters on Message" are used. I do not regard this as solved yet. This bug is for following only, as reported, and as clearly stated in Bug Summary. - Custom header such as List-Id - Run Filters on Folder : *WORKS PRETTY WELL* - Run Filters on Message/Run Filters on Selected Message : Somehow doesn't work Please don't add comment for different problem, even if phenomenon, conditions etc. are similar for you, and even if same workaround can be applied. Following is a known different issue. - IMAP, custom header, Offline-Use=Off folder(entire message data is not downloaded) - When new message header fetch, custom header is retrieved by filter definition => Filter on new message works using fetched custom header => However, (X) data of custom header is not saved - When Run Filters on Folder after new message header fetch => Data of custom header is not available, because "header fetch" is not invoked by Run Filters on Folder, and because of (X). => IIUC, filter fails too in Run Filters on Message/Run Filters on Selected Message, but I'm not sure. To bypass issue of (X), the custom header should be defined as "Header data saved in msgDatabase(.msf file)" by user. This workaround may be applicable to this bug, but I don't know. To comment posters in this bug for different problem(s) : Please search Bugzilla.Mozilla.Org well for your problem by yourself.

I used to be affected by this bug but it no longer occurs for me in recent versions of Thunderbird. I'm currently running 60.8.0 (64-bit) on openSUSE Tumbleweed. I have message filters that filter on a custom field (List-Id) and set to run manually. The messages get filtered properly when using the Run Filters on Selected Message command. This is with an IMAP account.

Can anyone reproduce the problem with a recent version of Thunderbird or can this bug be closed?

Same problem with TB 68.5.0 32bits / W10
Sorry
It's a realy important feature for me, espescially to track spams.

Same problem with TB 68.5.0 32 & 64bits / W10
Same with TB 68.4.1 64bits / Ubuntu 19.10
Sorry
It's a realy important feature for me, espescially to track spams.

I can confirm that I have had problems getting custom filter headers to work for a very long time. I'm currently on Thunderbird 68.7.0 64-bit on Windows 10.

Custom Filter Headers such as Reply-To and List-ID and Mailing-list seem all get ignored.

I also confirm that custom headers are not working for me. Windows 10 - 68.9.0 (32-bit)

I was attempting to filter on a "Sender" field

Sender: noreply@salesforce.com
Blocks: 847831
Severity: normal → S3

12 year old bug... still an issue on Thunderbird 102.5. I am trying to build a custom filter based on the Reply-To field, but it doesn't work anywhere (even if I run filters on folder, it is not working).

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: