Open Bug 1273818 Opened 8 years ago Updated 2 years ago

All messages in a Junk/Spam folder should automatically be marked/treated as SPAM with JUNK flag set, for content sanitation and easy UNJUNK, regardless of origin (e.g. from server side filtering)

Categories

(Thunderbird :: Filters, enhancement)

Product:
Thunderbird
Component:
Filters
Version:
45 Branch
Type:
enhancement

Tracking

(Not tracked)

People

(Reporter: kalle, Unassigned)

References

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:46.0) Gecko/20100101 Firefox/46.0 Build ID: 20160502172042 Steps to reproduce: Use server-side spam classification. Actual results: Messages downloaded into the designated Junk folder are not automatically flagged as junk but are displayed as normal. Expected results: Messages downloaded into the designated Junk folder should be recognized as junk by Thunderbird. This is particularly relevant when reviewing possible false positives: * Since right now the messages are not considered spam, they are displayed without content sanitization. * The "Not Junk" button is not displayed.
There never has been content sanitazation for junk mail. And Thunderbird is never going to automatically mark messages as junk that got into a folder from server side filtering. Even messages processed by "trust junk mail headers set by" don't get a junk designation when moved to the junk folder. (which is similar to your server side filtering) You can still mark such a message in Thunderbird as not junk - it WILL train Thunderbird junk processing, but there's no way it's going to affect your server side filtering. Junk designation is only given you you mark a message as junk, or an Inbox message was processed using automatic bayes filtering. So I'm going to say we wouldn't attempt to provide what you are asking.
Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago
Component: Untriaged → Filters
Resolution: --- → WONTFIX
(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #1) > There never has been content sanitazation for junk mail. What? All HTML messages identified as junk have been sanitized for display purposes for as long as I can remember. The setting in question is mail.spam.display.sanitize, which is enabled by default. IIUC, the feature dates back for more than ten years. Pick any HTML email tagged as junk, open it, and observe how, e.g., images are not loaded. > You can still mark such a message in Thunderbird as not junk - it WILL train > Thunderbird junk processing, but there's no way it's going to affect your > server side filtering. This is irrelevant and not what I was asking. > Junk designation is only given you you mark a message as junk, or an Inbox > message was processed using automatic bayes filtering. And I would argue that this makes no sense. If a message was moved to the junk folder by *anyone*, I would expect the client to treat that as the equivalent of manually marking the message as junk. Folders are free, and archiving legitimate messages into the Junk folder doesn't strike me as a canonical usage pattern. So, unless there is a specific rationale for ignoring the folder-based classification decision, I believe the issue merits a more thoughtful review.
(In reply to bintoro from comment #2) > (In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #1) > > > There never has been content sanitazation for junk mail. > > What? All HTML messages identified as junk have been sanitized for display > purposes for as long as I can remember. The setting in question is > mail.spam.display.sanitize, which is enabled by default. IIUC, the feature > dates back for more than ten years. > > Pick any HTML email tagged as junk, open it, and observe how, e.g., images > are not loaded. Thanks for pointing that out. I was thinking of the phishing feature, and what's aware Junk does something similar. > > You can still mark such a message in Thunderbird as not junk - it WILL train > > Thunderbird junk processing, but there's no way it's going to affect your > > server side filtering. > > This is irrelevant and not what I was asking. Then let's get to what you _really_ want. Based on what you have written, your end goal any message in the Junk folder should be subjected to mail.spam.display.sanitize. I suspect that might be possible with a simple tweak of the existing code. Which is quite different from your suggested solution of actually having Thunderbird mark messages as Junk. Aceman, what do you think?
Flags: needinfo?(acelists)
(In reply to Wayne Mery (:wsmwk, use Needinfo for questions) from comment #3) > Based on what you have written, > your end goal any message in the Junk folder [with the special folder flag] should be subjected to > mail.spam.display.sanitize. I suspect that might be possible with a simple > tweak of the existing code. > > Aceman, what do you think? Aceman, my suggestion would have the additional benefit of "protecting" messages that were moved to standard Junk folder by "trust junk mail headers set by" (they don't get the junk flag moved to the Junk folder)
Right, I wasn't aware that "Trust junk mail headers" doesn't cause the junk flag to be set either -- I would have expected it to. The main thing, in my view, is the sanitization issue. Obviously, a workaround is to manually flag the message before reviewing it, but most users probably don't think to do that. The lack of the "Not Junk" button is no more than a minor inconvenience. Anyone who gets a lot of false positives should be performing that action on the server anyway.
I think this would get some more thought. If the user moves the messages to the Junk folder on the server and we know the Junk folder is special (via the flag), we could use that information in some way. Also this may come up more often as also in Outlook it seems to me messages in Junk folder are junk. Just moving it out of the Junk folder they stop being junk. Only in TB there is a second level, that the junk flag can be sent on a message regardless in which folder it is. There can even be messages in Junk folder, that do not have the Junk flag (the bug here) and message outside Junk, that have the Junk flag. User may not understand this schema right away.
Status: RESOLVED → REOPENED
Ever confirmed: true
Flags: needinfo?(acelists)
OS: Unspecified → All
Hardware: Unspecified → All
Resolution: WONTFIX → ---
(In reply to :aceman from comment #6) > I think this would get some more thought. Yes I agree. These days I would encourage users to primarily reply on server-based junk filtering, yet then you may need to go to the Junk folder to review messages that have been marked as Junk. If sanitizing of Junk makes sense at all, then we should also do that on server-moved messages.
Summary: Messages in the Junk folder should be automatically marked as spam → Messages in the Junk folder that don't have Junk flag should be automatically marked as/treated as spam
Severity: normal → enhancement
Status: REOPENED → NEW
Summary: Messages in the Junk folder that don't have Junk flag should be automatically marked as/treated as spam → Messages in the Junk folder that don't have Junk flag should be automatically marked as/treated as spam (for example from server side filtering)
Blocks: 1482296
  • I'd consider this a bug. A message in SPAM/JUNK folder, should always be marked JUNK (if only to allow easy UNJUNK), regardless of how it got there.
  • I am wondering why training Thunderbird does not remove false positive messages from SPAM folder even though it was the server who put them there. Would this be possible?

I have been bitten by this and I find it quite confusing and annoying.
Server (I guess) has filtered false positives into server's SPAM folder, which I am reviewing on IMAP.
There's no reason for a message to be in Junk folder unless someone/something considered this message JUNK.
But surprisingly, the message is NOT marked JUNK, and consequently, there's no UNJUNK button - leaving users stranded with no obvious way to unjunk the message (it's possible via second-level context menu or main menus, but nothing in primary UI). The other "workaround", first marking the message as JUNK and then UNJUNK, is neither intuitive nor recommended, as it will add a lot of wrong training data and then (hopefully) remove the same data on UNJUNK.

Summary: Messages in the Junk folder that don't have Junk flag should be automatically marked as/treated as spam (for example from server side filtering) → All messages in a Junk/Spam folder should automatically be marked/treated as SPAM with JUNK flag set, for content sanitation and easy UNJUNK, regardless of origin (e.g. from server side filtering)

Workaround for IMAP accounts:

  1. Install the Addon "FiltaQuilla"
  2. Open FiltaQuilla preferences
  3. Add search term "folder name" See here for documentation
  4. Create a filter in Thunderbird
    • condition: Folder name - is - Junk/Spam
    • action: Set junk status to - junk

Alternatively, Filtaquilla offers filteraction: "train as junk".

I personally do not use the option to mark as junk. I just use the action "mark as read" and I do have one "search folder" from ctrl + shift + F search dialogue that searches for e-mails within selected folders (I selected all my junk/spam/deleted/trash folders) across all my e-mail accounts and I check this folder every few days and go through the spam manually to account for e-mails that may erroneously have landed in the spam folder. I then manually select all messages I deem junk and mark them as such. Most of the time I can use CTRL + A (Select ALL) for that. "Marking as read" prevents these annoying alerts popping up on my desktop for all new mail, even though most of it is junk anyway.

Severity: normal → S3

(In reply to Thomas D. (:thomas8) from comment #9)

  • I'd consider this a bug. A message in SPAM/JUNK folder, should always be marked JUNK (if only to allow easy UNJUNK), regardless of how it got there.
    I would agree, but there is also the need for the user to be able to identify how it got there.

Experience in support leads me to think most folk with spam filtering trouble have issues with a server side filter, generally one without an optout setting. The current situation does allow the user to see a junk flag not set for those emails. I also appreciate that not setting it is also problematical, so I am thinking it should be set, but in varying colours with tooltips to identify the source/score. In effect, having a junk status has to be supported with additional information not just a boolean (True/False).

You need to log in before you can comment on or make changes to this bug.