Get a phishing detector going for Thunderbird. I'm sure it can be improved quite a bit but this starts to catch some of the more obvious scams.

When the user clicks on a URL that we think is a phishing URL, he now gets prompted before we open it. Handles two cases so far. Hopefully we can add more as we figure out how: 1) The host name of the actual URL is an IP address 2) the link text is a URL whose host name does not match the host name of the actual URL. I added support for a silentMode so later on we can hopefully walk an existing message DOM and call into this routine on each link element in the DOM. This would allow us to insert an email scam warning bar in the message window down the road.

Comment on attachment 171936 [details] [diff] [review] first cut at a phishing detector This patch just gets the ball rolling and is better than nothing. Hopefully we'll continue to improve it by adding more checks. Read my comments up above for more details about the patch.

Maybe add some graphic of warning nature, and/or a very short timer like one when installing Fx extension (not too long/annoying)

(In reply to comment #3) > (From update of attachment 171936 [details] [diff] [review] [edit]) > This patch just gets the ball rolling and is better than nothing. Hopefully > we'll continue to improve it by adding more checks. > > Read my comments up above for more details about the patch. > FYI - I'm working on porting this to seamonkey.

Maybe you can do some lookups with the URI's against the SURBL database http://www.surbl.org/. If you consider using SURBL, then you might want to consider a local whitelist of domains not to query for

Comment on attachment 171936 [details] [diff] [review] first cut at a phishing detector (hmm, phishingDetector.js does not mention an original author... shouldn't it?) + var hrefURL = Components.classes["@mozilla.org/network/standard-url;1"]. + createInstance(Components.interfaces.nsIURI); + + hrefURL.spec = href; is there a reason to avoid nsIIOService::newURI? there are other types of URIs than standard-url...

I think that showing both URLs in the first dialog isn't a good idea. Users only skim read the detailed text of warning boxes like this one and there's a risk that they'll just see "www.mybank.com", think "that's what I was expecting", and click "Yes". It also makes the message longer than it needs to be. If we just said "Thunderbird thinks this link is suspicous. This link is to the site www.phishersrus.com, not to the site shown in the e-mail." users are more likely to have that jarring "phishersrus - who on earth are they?" moment, which will hopefully make them pay enough attention to work out that they probably want to click "No". Also we might try to go for more user-oriented terms than "host" ("computer", "site"?) and URL (link?). All that said this is a great idea.

Hrm... I think some careful testing of this stuff should be done against legit emails. In particular, URLs containing an IP address are something that spamassassin scores on, but I've actually had to lower the score for it (on the mailserver I run at work) due to false positives. Some legitimate newsletters and solicited marketing emails seem to use numeric IPs. Why they do that I have no idea, but checking for that alone will have false positives, which in turn will desensitise people to the warnings. I guess we can see how that goes...

(In reply to comment #1) > When the user clicks on a URL that we think is a phishing URL, he now gets > prompted before we open it. Handles two cases so far. Hopefully we can add more > as we figure out how: > > 1) The host name of the actual URL is an IP address > 2) the link text is a URL whose host name does not match the host name of the > actual URL. > Phishers have recently taken to using spyware / malware to override DNS by adding entries to users' HOSTS files (see http://news.com.com/Phishers+learn+new+tricks/2100-7349_3-5544193.html), so another useful check would be to see if the URL domain has an entry in the user's HOSTS file. For performance you could read and cache the HOSTS file at startup and optionally refresh it periodically.

Thanks, Scott, for implementing this. Once I implemented bug 116242 (if ever), that would be useful to determine, if there's a URL in the link text, even if there's additional text before and after it or if it's abbreviated. > Phishers have recently taken to using spyware / malware to override DNS by > adding entries to users' HOSTS files If that happened, the war is lost. I agree with comment 8. Another counter-measure would be to simply completely remove suspicious links (the <a>, not the text).

mscott: You may want to check out: http://reportphish.org/ data is available to researchers... there might be some information available through them (now or in the future) that might help build good testcases. Getting a good list of testcases could help take this tool to the next level.

(In reply to comment #1) > When the user clicks on a URL that we think is a phishing URL, he now gets > prompted before we open it. Handles two cases so far. Hopefully we can add more > as we figure out how: > > 1) The host name of the actual URL is an IP address I also believe a local whitelist mechanism will be necessary. If the IP is for a site that the user visits often (and they are always warned with a prompt), mozilla will soon start receiving complaints (if there is no clear way to turn off prompts for that site).

You may like to look at this: http://www.eudora.com/email/features/scamwatch.html .. to see how Eudora already handles this. The main addition to the Thunderbird proposal is the use of a popup warning when the mouse is over a suspect link. This way the user is warned before they even click the link that it may be bogus. I don't think the top-level domain check they use is good, especially for non-US users (there are real sites like ebay.com.au, yahoo.co.uk etc.. ) Something I would like to see is altering the colour of a suspect link to something non-default (eg. bright red), so that they stand out, especially compared to other non-suspect URLs in the same message. This is because many scam emails I see now have several legit links in them (usually to the home page or privacy sections of a website), but the "click here to login" one is bogus. If bogus links were a different colour, then dodgy links would become obvious instantly (without even moving the mouse over them, let alone clicking on them), and you might even get into the habit of instantly recognizing a phishing email by seeing red links in it as soon as you opened the message (html colouring of links aside of course).

What about working together with http://www.antiphishing.org/ Looks like they have a database of phishing mails

+ if (!aSilentMode) // allow the user to over ride the decision + isPhishingURL = confirmSuspiciousURL(phishingType, hrefURL, linkTextURL.value); I believe that should be: + if (!aSilentMode && isPhishingURL) // allow the user to over ride the decision Otherwise, linkTextURL.value isn't set, and you get a JS error: JavaScript strict warning: chrome://messenger/content/phishingDetector.js, line 71: reference to undefined property linkTextURL.value There's no need to call the confirmation function if we don't think it's a phishing URL.

Scott, this is a great idea. My apologies for the unsolicited yap, but I really want to make a couple of UI suggestions that build on comment 4, comment 8 and comment 14. When the user mouses over or tabs onto a could-be-phishing link, the link should change from www.imaphisher.com to !!! www.imaphisher.com where !!! is a small (character height) warning sign. The icon should be platform-specific. For Windows, see the warning sign under http://msdn.microsoft.com/library/en-us/dnwue/html/ch09f.asp. Sounds like a job for CSS. This is complementary to the warning bar you're thinking about for the message window, because they would be seen at different times by the user. When displaying the message box itself, use the same warning sign, not the question mark sign that's used now. The question mark is misleading, because it downplays the gravity of the message, and contrary to platform UI guidelines (link above), which advise that the question mark is just bad in general and shouldn't be used at all. Finally, here's a suggestion for the text itself. Thunderbird thinks this link is suspicious! [Note emphasis through exclamation and blank line following this sentence.] It may be trying to impersonate the web page you want to visit. [Note "impersonate", an active verb.] Are you sure you want to go to www.imaphisher.com? [Use only the hostname, not the full URL. Highlight the URL using a warning color like bright yellow.]

Will this work in Mozilla Mail (Suite)?

It's a great idea. I think the following jargon will cause many users' eyes to glaze over: URL, host, numerical ip address. They won't understand and we'll lose their attention. A quick rewrite in end-user languange: Thunderbird thinks this link is suspcious. The actual addresss, real.domain.tld, is different than the one displayed in the e-mail, fake.domain.tld. Are you sure you want to visit this link? Thunderbird thinks this link is suspcious. The link's address, 123.123.123.123, uses numbers. Most legitimate links use names. Are you sure you want to visit this link?" More users will understand it. Sophisticated users will know we're talking about URLs, hosts and numerical IPs. (I'm aware that this sentence isn't precise: "The link's address, 123.123.123.123, uses numbers", but unsophisticated users won't notice, and sophisticated users will understand anyway.)

(In reply to comment #2) > Created an attachment (id=171937) [edit] > screen shot of the 2 dialogs > Just a small suggestion: Wouldn't it be better to have an exclamation mark icon instead of a question mark baloon icon on the dialog? Or wouldn't that be correct if you look at the purpose of the dialog? (I don't know what the official dialog icon policy is)

(In reply to comment #15) > What about working together with http://www.antiphishing.org/ > Looks like they have a database of phishing mails Another site that provides this sort of feature is http://toolbar.netcraft.com/ They don't seem to have an open API, but I imagine the MoFo could work with them to get access to this "black list" of phishing sites.

Ref: Comment #10 If this means filtering against my HOSTS file, please don't! I use it as a local DNS table to speed access to obscure domains. Any domain that I see in my status bar with "Looking up" for more than a second or two gets added.

By populating the hosts file you are perpetuating the problem. The more you access the domain, the more it will be cached on your ISPs DNS servers. [ Sorry for the spam. ]

By populating the hosts file you are perpetuating the problem. The more you access the domain, the more it will be cached on your ISPs DNS servers. [ Sorry for the spam. ]

I agree with #11. Just mark the link dead and provide no ability to click through at all. If the user insists on doing it, make them work (perhaps a menu option, but a cut&paste of the text would be fine too). As soon as you get aggressive with link sanity, people will start sending out more responsible links. IMHO. (I also agree that the message needs to be pretty darn basic, and blue is the wrong colour. Go for bright flashing red as JPM says: http://www.financialcryptography.com/mt/archives/000312.html in the comment) Excellent stuff, Scott!

i suggest adding a "Help..." or "more info..." button with links to pages explaining the phenomenom of phishing in more detail to average users. keep in mind that phisinhg mails often look very official, and users might be confused that "Thunderbird thinks" they're suspicious. besides that, great work!

scammers frequently obscure the IP address by using a dword for the host name, hex encoding the IP address or octal encoding the IP address. They even do a mixture of both (i.e. hex.oct.oct.hex) This patch addresses that case. Test cases for this patch: “dword” – http://3515261219/ “octal” – http://0321.0206.0241.0043/ “hexadecimal” – http://0xD1.0x86.0xA1.0x23/ or http://0xD186A123/ "mixed format" - http://0321.0x86.161.0043

Ref: comment #23 (repeated in #24) I am using the Hosts file the way it was initially intended. In company internal networks, it is common for some destinations have no domain names, only IP addresses. For referring to those destinations symbolically, users create their own names in their local Hosts files. Putting bogus entries in Hosts files to block access to undesirable domains is a consequence of how the files are used, not their real purpose. Thus, any fix that uses the local Hosts file as a blacklist would be contrary to the purpose of that file and would have the effect of making the product unusable in large organizational networks. By the way, my daily refresh of my Hosts file to get any changes to IP addresses does send the domain name to or through my ISP's DNS.

(Crossposted by accident on https://bugzilla.mozilla.org/show_bug.cgi?id=254913#c16) Some IMAP Anchors seem to trigger the phishing warning when using an internal IP address (192.168.0.1) as IMAP server address. The anchor link was: imap://valentijn@192.168.0.1:143/fetch%3EUID%3E.INBOX.Trash%3E5099#32816 And triggered the anti-phishing dialog reporting that the link was directed to 192.168.0.1 (our internal mail server). I think this warning should not be triggered when the target address is the same as the IMAP server.

this additional change fixes comment #29 where non http urls are incorrectly getting identified as phishing urls (i.e. imap urls....)

(In reply to comment #30) > only check http and https urls Would it be useful to include ftp in this check to prevent spoofing of download links?

(In reply to comment #31) > (In reply to comment #30) > > only check http and https urls > > Would it be useful to include ftp in this check to prevent spoofing of download > links? > Not only FTP, also IRC and others? Why not check ALL url's except the internal ones?

*** Bug 282362 has been marked as a duplicate of this bug. ***

I've just received a bugzilla email (attached), that's triggered the 'scam' bar, and I can't see any reason why it has, although coincidentally its the homograph bug...: all link text matches link targets no ip address links

(In reply to comment #34) > Created an attachment (id=174809) [edit] > bugzilla email > > I've just received a bugzilla email (attached), that's triggered the 'scam' > bar, and I can't see any reason why it has, although coincidentally its the > homograph bug...: > > all link text matches link targets > no ip address links I've just received another one. It looks like the presence of any number in the domain triggers the warning, not just an IP address.

Good catch dougweb. If our calculated IP address for the url is 0.0.0.0 then we should treat it as an invalid IPv4 address and as such not an email scam.

Here's a potential issue: What about the many users with access points, and other products that are capable of sending email's with links to do things (like view logs, or turn on and off). Perhaps blackhole IP blocks should be whitelisted? Off hand I can think of a few : 192.168.x.x 10.10.x.x 191.8.x.x I belive there are some others.

Should IPs with the correctly linked DNS name also be blocked? For example: nslookup www.google.com results in: 216.239.57.99, 216.239.39.99, 216.239.37.99 For example, if a link points to 216.239.37.99 but the linked text says it's www.google.com, this link is certainly not a scam link.

Is there a repository of phishing scam email messages somewhere that are being referenced when deciding which criteria will be used for this detector, or is it just being assumed that email messages with IP addresses in them are phishing scams and that phishing scams do not contain FTP links without any reference to real world data?

and I also use: http://www.millersmiles.co.uk/

I had a look at those sites, but I didn't see any statistics, or any way of retrieving any statistics very easily. Are the detection methods in Thunderbird being developed based on an objective measurement of how phishing scams work, or just "eyeballing" a few messages and guessing? I ask because the detection so far has only insisted on pestering me about legitimate links and messages, which I find annoying. Although I would love to remove this feature, I know that won't happen, so I'm interested in making it as accurate as possible.

Why not check to see if the href, the status bar text, and the link text match (if the link text purports to be a url? For instance, if we have an href of http://1.2.3.4 and status bar text that reads http://secure.ebay.com/, then we have a problem. However, if we have an href that reads http://badguysareus.com and the status bar reads http://badguysareus.com, and the link text also reads http://badguysareus.com, then we're fine. I don't understand why Thunderbird wants to be so presumptious to assume that it knows better than I what constitutes a scam. This sort of thing is *exactly* what makes Microsoft products so annoying to use. They assume you're an idiot, then they make design decisions based on that assumption, and they give you no opportunity to override their decision. In effect, they're saying, "you're an idiot and we know what's best for you ... and you will not be permitted to decide otherwise." It's fine if Thunderbird wants to make some attempt at warning clueless users about phishing scams, but *please* let me turn it off at least.

(In reply to comment #43) > Why not check to see if the href, the status bar text, and the link text match > (if the link text purports to be a url? For instance, if we have an href of > http://1.2.3.4 and status bar text that reads http://secure.ebay.com/, then we > have a problem. That's exactly what it currently does. Of course you'll be able to turn it off. I just haven't added that yet. Maybe you should be sticking to release builds if you aren't interested in waiting for features to get fully implemented in nightly builds.

(In reply to comment #44) > Of course you'll be able to turn it off. I just haven't added that yet. Maybe > you should be sticking to release builds if you aren't interested in waiting for > features to get fully implemented in nightly builds. That's OK. I don't mind partially implemented features. I just became concerned when I started seeing this. Knowing that it will be possible to disable it alleviates my concerns. I was only concerned because in the past there have been unilateral decisions made by the Mozilla team that were both unpopular and unwanted. I was concerned that this would be the case here as well.

I've also noticed that any email with a <form> triggers the warning bar - that's definately not right...

Scott, the phishing detector doesn't address email messages like the one Seth opened the other "detect phishing" bug for -- attachment 155939 [details] (bug 254913). In that case, the link is to a part of the email itself (not an anchor, but a MIME part). Internally, it's a cid: link; when hovered, it shows as a mailbox: link. I don't know if there's ever a legitimate reason to have a link to a part, particularly when that part is not shown as an attachment (multipart/related) -- but see bug 241572. Technically, that message is not phishing so much as a trojan, but it's still a scam. See also bug 276991.

Since when are programmers in charge of deciding whether there is a reason for me to have email configured in any particular manner? If a message doesn't violate a specification, leave it alone. I'm a user, not a moron. Make Tardbird for the morons. Leave me something to use please.

*** Bug 285665 has been marked as a duplicate of this bug. ***

*** Bug 288817 has been marked as a duplicate of this bug. ***

I vote for a whitelist as part of this feature I like this feature, for people who don't know how to take care of themselves (a lot of people, frighteningly enough). However, I'm getting advertising email that I *every time* have to click "Not a Scam" and then "Show Images" for. Yes, I like some advertising emails every now and then; they are fun. If it works this way then I think many users will get used with mindlessly clicking those buttons. Once that happens, what good are they? For myself - I am forced to click *a lot*. My vote is for a whitelist that would be effective for all emails from that same domain/sender/something. Perhaps a question when you click "Not a Scam" if you want to apply that as a preference from now on, for that domain/sender/something. For the corporate users perhaps a special preferences file somewhere that they could keep up-to-date. Great idea though!

Never mind Comment 51 - if I add the company in my address book, everything works fine.

*** Bug 254913 has been marked as a duplicate of this bug. ***