Right now if you get a spam message and the message gets filtered into a folder we won't run spam detection on the messages until the user selects the target folder. This means we show in the UI that there's new messages in the target folder (even if it's spam and we'd know it's spam if we'd check) and we fire off notifications about the new message(s). Seems like the user experience would be much better if we'd simply run a spam detection pass on the target folder after a filter has moved messages into the target folder, and then only once that's done, we'd notify the user about the new messages if there really are new non-spam messages in the folder.

I think for imap we're doing this right now on the trunk and 1.8 branch, if showPreviewText is turned on, which it is, by default...for pop3, we're not doing this yet, and we should.

Is this by any chance the same as Bug 200788?

Perhaps this is a duplicate, but has this bug been unfixed really for 3 years??? This bug has annoyed me everyday since I switched to Thunderbird 1.0.7. Every single time Thunderbird reports new mail, I check it, only to have it junk-filter every message after I selected the individual-folder for each of my accounts. Please, I want to reclaim my inbox! Filter junk, then report if I have mail, in that order. :) This bug makes new-mail notification pointless. Even if this is a duplicate, it needs to be duplicated, reopened, and reassigned. 2003?!?!?!?? I waited for the 1.5 release and didn't switch to outlook only because it seemed impossible that this wouldn't be fixed. The previous bug report utterly failed it purpose and us users. Please assign this bug to someone else. :) This made the 1.5 release a real let-down.

My mistake. We're not doing this for IMAP quite yet either. We do update all the target folders of filter moves after each move completes, which will cause us to analyze the new messages for junks status, but we fire biff when all the updates are complete, i.e., the headers have been downloaded and mail filters applied, but before junk mail analysis has completed on those folders. The target folders will get analyzed and messages moved to the junk folder (if you've set it up that way) as part of the get new mail process, but biff will still fire in some situations where incoming mail is all junk. It's an improvement, but still not right. For POP3, we're not even analyzing the filter move target folders until the user opens the folder. That's easy enough to change, but delaying the biff notification until that process is complete for all folders is going to be a challenge because there is no notification of that event.

Your comment was possibly the most helpful thing I've read about this so far! Thank you. I would like to draw your attention, to what was otherwise something of an afterthought though: "For POP3, we're not even analyzing the filter move target folders until the user opens the folder." This _IS_ the bug we are talking about. :) There are obviously two things going on here : 1 - an issue about WHEN junk mail gets analyzed on POP3 filter move target folder (easy fix) 2 - if biff notifies of new mail. (difficult due to lack of 'filter moved' event) While yes, I did include emphasis on #2 in my comment, fixing #1 "last year" is 90% of what I want. I don't pay that much attention to biff notices, but when opening the client, I shouldn't see (n) next to my folders indicating new messages, then find out they were all junk. Can this be fixed tonight? What can I do to help? I'll order you pizza, or whatever you need to do so that junk is filtered intuitively after mail is recieved. We can worry about the notification (biff) later, but that event does need to get created at some point. Thanks so much! Your response really puts me at ease that exact issues of causality are well understood and can therefore be addressed, at least in the case of #1 (the biggest priority in my book).

I can code something up tonight, but it'll need some testing...

this is the patch I'm testing right now - I had to tweak my original stab at this because m_filterTargetFolders was getting multiple entries per folder, if a folder had multiple messages filtered into it. I'll run with this for a bit and see if it behaves well. One regression this will cause is that we'll leave the db's open for folders that have messages filtered into them, if we analzye any of the new messages for junk status. Not sure how I'll deal with that...

Comment on attachment 214440 [details] [diff] [review] apply junk mail controls to folders that have pop3 mail filtered into them this is working well enough for me and is probably worth getting some testing on the trunk.

OK, the fix to make spam detection run on local folders that have messages filtered into them has been checked into the trunk and the 1.8 branch. The new mail alert will still fire even if all messages are marked as junk, but spam will be detected and dealt with immediately after the new mail is all finished downloading.

Thanks for finally fixing this, it really rendered the junk filter useless. Just one thing, if the "new mail"-notification still is shown, then I still get annoyed by spam and the filter still isn't 100% working. Say I get a spam every 30mins, then I'd have to check my mail every time only to see there are no new messages. IMO still a very unsatisfying situation. Are there any plans to change that?

I think the recent fix would be an improvement but it's _STILL_ not in the production build where it should have been part of an update months ago. At this point enough time as elapsed to expect the overall design flaw to have been fixed. The design flaw being the order and way spam filtering is performed. By now it is reasonable to expect that code to have been changed. Either move the spam filtering ahead of user filters, or create the event for filter completed and implement it. Finally, just change how biff is called. make an event for spam filter-completed. Okay, that'll take a couple hours of coding, but it's been months. Call biff from there. The current bug fix should have distributed to 1.5x updates long ago. Thanks guys. Also, I'm talking about the Windows build, but I'm sure the problem is the same across platforms.

1.5.0.x builds are mainly for security fixes and regressions.

David, you said this patch was checked in -- should this bug be marked fixed, or are you waiting for something else? I'm not going to be able to verify the fix as I get very little junk making it thru to Thunderbird.

Pinging David Bienvenu: is this bug FIXED or not? (comment 13)

yes, it's fixed, except that we will still fire a new mail notification even if all filtered mail turns out to be junk. I suppose we can open a new bug for that.

(In reply to comment #15) > yes, it's fixed, except that we will still fire a new mail notification even > if all filtered mail turns out to be junk. I suppose we can open a new bug > for that. Bug 233930 already exists for that problem. (See also bug 233929.)

*** Bug 200788 has been marked as a duplicate of this bug. ***

Moving this where it can be found.

fixed1.8.1 per comment 9 and bonsai

Sorry to drag some comments into an old bug but this feature is really hurting me! I subscribe to several mailing lists which are moderated and generally contain no spam but the spam filter still checks them. THis means when I am checking my mail remotely a folder with several thousand mails in it have to all be checked for spam which slows the whole thing down needlessly. Also recently my spam filter started being a bit overzealous and trashed loads of messages from legitimate mailing lists. I really would like a way to disable Junk checks of IMAP folders. I'd say turn them all off by default as 99% of the time the reason for server side filters is to filter specific contacts and notifications etc. The PENDING case listed in this bug is one folder where you DO want checks, but how many others really need it?? Perhaps my case isn't the norm, but I really think this is a lacking feature right now. A simple tick box in the folder properties would suffice to turn this on or off, and perhaps a global setting for "Check all folders for SPAM" could be available for IMAP accounts? What do you think? Perhaps there is a new bug for this already? If not please reply here and I can open one.

Colin, if your filters are moving messages to these folders, they can also mark the messages as nonjunk, and the spam filter won't run on them.

(In reply to comment #21) > Colin, if your filters are moving messages to these folders, they can also mark > the messages as nonjunk, and the spam filter won't run on them. A cunning plan, unfortunately thwarted by the fact that I am using /server side/ filters, not client side. Admittedly this was a fact I neglected to mention above. Basically I have several server side sieve scripts that direct various mailing list mails into their own folders and it is these folder I want Thunderbird's junk filter to ignore. If the server side could add a header in that says "I am not junk" then a lot of spammers would use this ;) I notice there is the option to "trust SpamAssassin" headers, but I'd imagine this is only in one direction (e.g. it believes it when it says it IS junk). Cheers for the idea tho'.

Yes, that's why I said "If" :-( The option to trust SpamAssassin headers can work both ways (good and bad), but it only works on the Inbox, because it runs with the normal incoming mail filters. So, having a per-folder setting to disable junk processing is probably the easiest way to go, though one could argue that trusting spam assassin headers should work on all folders, not just the Inbox. There is an open bug about the per-folder setting...

(In reply to comment #23) > There is an open bug about the > per-folder setting... I thought there might be but couldn't find it in my search - that's why I found this one ;) A little more digging and I found this: https://bugzilla.mozilla.org/show_bug.cgi?id=189970 /me is off to use some votes....

(In reply to comment #21) > Colin, if your filters are moving messages to these folders, they can also mark > the messages as nonjunk, and the spam filter won't run on them. David, if we add "Set Junk Status to NOT JUNK", won't that constantly "train" the junk filter? I would prefer that the junk filter completely ignore these messages... I don't want them used to train the junk filter. Is that possible? Or, does the "mark as NOT JUNK" really just mean "tell the junk filter to skip this message"? Thanks for continuing to follow up on this bug.

No, if the junk status is set on a message by a filter, the junk filter does not get involved at all - the "good" message is NOT run through the junk filter. It's NOT the same as pressing the "this is junk" button.

David, I had always assumed that setting the junk status would alter the training data. Thanks for the clarification.

(In reply to comment #7) > One regression this will cause is that we'll leave the db's open for folders > that have messages filtered into them, if we analyze any of the new messages > for junk status. Not sure how I'll deal with that... David, has the regression resolved? If not, "leave db's open for folders" possibly one of reasons of other issues such as "junk not moved", claim of too many file handle for a ".msf" file. Should we test and provide trace data for the "leave db's open for folders"? Note: I recently got "Process Monitor" in additon to "DebugView". And I have an mail account to which only spam's arrive(100 per a day). So I now can test with real junks and get mail folder file/directory related activity of Tb on MS Win. See http://www.microsoft.com/technet/sysinternals/default.mspx for tools on Win.

Hi ! Sorry to comment here again but I still have this problem in my TB 2.0.0.4 :-( I have an IMAP account and I want to get all messages locally. So I have messages filters defined for that IMAP account to move the messages to an other account (i got messages per POP before with this). All messages are correctly filtered to the folders in the other account, but no junk processing is done. Any idea?

I'm also still having this issue in TB 2.0.0.9. I'd love to see this resolved, as I use filtering extensively and running junk mail controls on each folder individually gets very tiring!

Excuse the double-post. I upgraded to 2.0.0.12 today and the issue is still present.

Erin and Ingo, this WFM on both current trunk builds, as well as 2.0.0.9 I have enabled adaptive filters on all accounts - including both servers and Local Folders (haven't tested to see if that matters though.) So either you are seeing something different, or there is some issue with your local setup. The patch for this bug definitely went into the branch that TB 2 is based on, and it fixed a problem. I suggest that you file a new bug giving a more clear description of what you are seeing if it is still an issue.

Hi Kent, for my case there already is a new bug 389098 with this issue for IMAP. It would be great if you could check this. It is confirmed. And maybe bug 389096 has to do with this directly ...

I have figured out what my problem is, and it seems to be another bug. I have tested this and confirmed that this is the case in my 2.0.0.12 TB: Even if the "Place a copy in" checkbox under Preferences -> Account -> Copies & Folders is unchecked (ie, no sent mail saved), junk mail controls do not run on the selected folder, even though it is greyed out, indicating that it is not a sent mail folder. Steps to reproduce: 1) With the "Place a copy in" checkbox checked, select the "Other" radio button and choose a folder which accumulates lots of spam. 2) Uncheck the "Place a copy in" checkbox. The "Other" radio button will still be selected, but it and the folder drop down box will be greyed out. 3) Even though no sent mail is being saved in the folder, junk mail controls will not run on that folder.

I can confirm this behavior both on TB 2.0.0.9, as well as a trunk build based on 2008-03-27 source. Good catch, Erin. Can you please file another bug for this, cc me, and then I'll confirm it? Thanks.

New bug is bug 426950.