Richweb Mail Content Filter: MailScanner

What is MailScanner and how does it work?

MailScanner is a mail content filtering system that Richweb is using to replace our legacy, failing MailFoundry-based system. Like MailFoundry, MailScanner examines each incoming message and will prevent viruses and spam from making it into your Inbox. MailScanner has several more powerful content analysis features and dangerous content blocklists that make it more effective than the MailFoundry at catching both spam and dangerous phishing emails. Phishing emails are very troublesome as they can trick even technically adept computer users into giving away financial, corporate, and personal information to attackers which will use and abuse this information. The MailFoundry was simply allowing too much bad/dangerous content to get through its filters.


What other features or differences should I know about with MailScanner?

MailFoundry excels at catching computer generated spam from templates, where the basic message is the same, with only a name or weblink within the email being different. Spammers and Phishers have caught up to this technique and are generating ever shorter messages, sometimes with just a single link. Its very hard for the MailFoundry to block these emails without also blocking legitimate email. MailScanner is smarter as it has blacklists/blocklists of Spam and Phishing domains in its databases that are updated regularly. If an email contains a link to a known phishing domain, it is blocked, regardless of whether the message or message template as been seen before. MailScanner can also disarm or make safe an email and send it along to your Inbox. See below for more details.


What is this {Cleaned} tag in the subject line of some messages ?

MailScanner disarms or cleans dangerous HTML tags and commands that can cause your computer to become infected with spyware or trojanware that can steal your personal information. MailScanner can find html that is unsafe (where the CLAIMED web link destination does not match the ACTUAL weblink destination). MailScanner removes the links, but if the rest of the message is deemed safe, and not spam, it will send it on to your Inbox with the {Cleaned} header in the subject to let you know that the message has been made much safer. This is a good thing. Disarming or Cleaning a message is important because every day new vulnerabilities and bugs are discovered in web browsers and email clients, typically Microsoft Outlook and IE. As attackers attempt to create more and more clever ways of tricking you AND your computer, MailScanner puts a stop to the basic tactic of bait and switch web links RIGHT AT THE SOURCE - the html. If you have an email that comes from a mailing list or company that is {Cleaned} you can forward the email back to the owner of the list or company that sent the email and ask that they fix the emails so that they are safer. In particular, email messages that have hidden IFRAMEs are not a good idea, as attackers use these techniques to trick you and your browser.


Does the MailScanner have that 10 to 15 minute delay in getting an email sometimes?

No, it does not. Since the MailFoundry needed time to detect spam signatures from new spams that are constantly generated, the MailFoundry box likes to hold all incoming email from a new source (sender) for up to 15 minutes while it waits to see if that sender or that message template appears is identified as spam by the MailFoundry team. This is of course irritating AND it does not always work! If you happen to have a directed attack of nasty spam messages at a certain user or few users, of if your domain happens to be at the TOP of a spammer list of thousands of domains that are about to get hit, then you may be out of luck with the MailFoundry! If the spammer is able to configure dns settings and buy IP transit from a legitimate host that is not currently blacklisted, then the spam will make it through to your Inbox. MailScanner is smarter about being able to actually look at the content of the message (words AND links, picture, etc) and not just the structure or template. Thus MailScanner is a stronger defense in some of these hard to handle situations like a targeted attack or a large domain that gets a lot of spam from many different sources.


Why are some domains hit so much harder than others with spam?

Domains that get a lot more spam have usually been around longer, and in almost
all cases one (or more) users on that domain has clicked one or more link(s) in spam mails, or bought stuff advertised in spam. Spammers track EVERY single message that they send, and they know who you are when you click a spam-vertised link. Your domain is then marked as having willing recipients that WANT spam, and spammers spend a lot more effort spamming your domain; they figure they have more to gain looking for repeat business than going after brand new domains!


Where is the quarantine report for MailScanner?

MailScanner does not provide a report. MailScanner makes every attempt to disarm or fix messages and send them on to you in a safe state. If MailScanner blocks a message, it is very certain that the message is spam and it takes a system admin (at Richweb) to release the email. Most messages that are spam are detected as high scoring spam (what people tend to describe as "obvious" spam). These high scoring spams are discarded. What we discovered is that most users dont even look at the MailFoundry reports, and for busy mailboxes the reports are so long anyway that its a waste of time having to wade through the reports.

MailScanner supports whitelisting of email senders and email domains. If you have a sender that you think is being rejected, send the email address to noc At richweb dot-com and we will take a look and whitelist the sender if it appears that the message is not making it through.


MailScanner info (intended for system admins)

I can't seem to get an email from a certain sender and it's not in the quarantine report!

What is likely happening is that the person that is sending you the message is sending from a computer system or network or company that has gotten blacklisted. This happens when an internet address (IP address) is either not setup to be able to originate (send email properly), or an infected PC has sent so much spam from that internet network address that the system is considered to no longer be a legitimate source of valid business or personal email.

What you need to do is get the email administrator of the sending email domain involved. Richweb can in some cases whitelist (permanently allow) the domain to send email. In other cases the administrator of the sending domain simply needs to correct the technical problems with their configuration and policy. In all cases to dig into the problem Richweb needs the exact information below:

A. Sending email domain
B. Sending IP address (if possible) of the mail server that transmits the emails (i..e. the mail server public IP or NAT - NOT the ip address of the laptop sending the email). If the ip address you are given starts with 192.168, 10.x, or 172.16 thru 172.31, then you have been given the internal ip address, which of course is NOT useful in researching the problem. We need the public (routable) IP address.

You should also check the ip address yourself first in a DNS black or blocklist tool such as:
http://www.dnsbl.info/

If your (or the organization of the person trying to send you email) mail server domain name OR ip address is on this list as blocked, then you can expect moderate to severe mail delivery problems with most if not all email domains. Step one in solving this problem is addressing the underlying cause of getting blocked - someone is stealing your network resources to send spam.

Richweb is happy to assist; of course we have to charge a consultation fee with sending domains that are not properly setup. Typical problems we see are: missing reverse DNS, bad SMTP HELO name, using a dynamic ip, shared host on a site with a poor reputation (i.e. a hoster that hosts spammers).

Refer to this page for additional Richweb helpful information about DNS and email troubleshooting:

http://www.richweb.com/mail_blocked

Mailscanner Bulk File Download Manager

The problem with email and large file attachments:

Most Email systems will not accept email attachments larger than 25 to 40 Megabytes (MB). Many email systems place strict limits/restrictions on maximum number of attachments, attachment types, and attachment content (zip files for example).

Richweb's MailScanner product provides a companion FTP manager solution that allows domain admins to create FTP dropboxes as well as web based download accounts that can be used to transit files for short and long term projects. You can create accounts for customers, vendors, projects, etc, and each account can only access files within its assigned folder.


Features and Benefits

1. The FTP Manager is built into the same console that you use to manage your MailScanner settings.

2. There is also a web account (download only) capability built into the system now. Each and every ftp user account can be accessed over http so users that need to download files and cant operate an ftp client can be given web links for download.

3. You can set an ftp user up with a root account (dir of /) and that account will have full web download and ftp upload/download access to your whole domain.

4. You can have multiple root accounts if you like. In fact you can point
multiple accounts at the exact same home dir folder if you like. Example - 5
different vendors all need their own access to download the same files in a
projectxyx/bids/ folder.

5. FTP Users can be inactivated but not deleted to temporarily remove access.

6. Passwords can be easily managed (changed/reset) by domain admins.

7. Each domain you control gets its own private FTP space.

8. FTP storage capacity and bandwidth can be purchased on an incremental basis.

Please contact Richweb for more information
http://www.richweb.com/contact


Firewall Settings for Richweb MailScanner

The following ip addresses need to be open for inbound SMTP for the mailscanner to work properly:

208.73.136.12
208.73.136.26
208.73.136.50
208.73.136.51
208.73.136.52
63.90.9.6

You can use 208.73.136.0/26 (208.73.136.0 255.255.255.192) in your firewall acl if you prefer.

The MailScanner machines are clustered and have the ability to fail over between them so you need to ensure that all of the above IP addresses are allowed.

From inside the MailScanner you can perform an SMTP connection test to ensure that your firewall is allowing smtp inbound properly:

[Run mydomain.com SMTP Connection Test]

If you get a message like this:
WARNING: Cannot connect to SMTP service at a.b.c.d; timeout: 15 sec.

that means that the MailScanner could not connect and you need to verify your firewall ACLs.

If you have changed the inbound SMTP ip address that your firewall conduits through to your mail server contact Richweb and we will update the SMTP transport database.

I need to report a spam, or have Richweb look at a troublesome email. What should I do?

Mail System Administrators have access to the Web based management tool:

Login to the MailScanner Manager at this URL:
https://vsmx1.richweb.com/

Use your assigned username and password.

Click: Tools

Under the Section/Menu:
Spam/Not Spam Databases

Click "Spam Messages not caught as spam"

MAKE SURE that you upload the message with FULL HEADERS and FULL MESSAGE CONTENT.




End users should have their mail system admin look at the message and follow the process above, OR if that is not possible, use this information below to send the message to us for further investigation.

Outlook 2007/2003/XP

1. Outlook does not have a bounce feature. Instead, you will need to copy the full headers and paste them into the email you are bouncing.

2. Double-click on a message so that it is in its own window. Click on View > Options. In the Internet headers: box, you should see the raw message header.

3. Highlight the message headers using the mouse (click and hold the left mouse button at the start of the headers and drag the cursor over the message or click anywhere in the box and enter Ctrl-A to select all the text).

4. Once the headers are highlighted, right click on the highlighted text, and choose Copy from the menu. This will copy the text to the clipboard.

5. Ensure you still have the spam message selected. Click the Forward button and then paste the headers you copied above the text of the email you are bouncing (forwarding in this case).

6. Enter reportspam@richweb.com and click Send button to send your message with the headers.





Thunderbird and Mozilla Mail

1. Current versions of Thunderbird and Mozilla Mail do not have a true bounce feature. You will need to forward the message with the full headers.

2. Select the message you wish to bounce/report as spam. On the Thunderbird menu bar, click View > Headers > All. You may notice that the original message will display more information than it did previously.

3. Click the Forward button and enter reportspam@richweb.com as the recipient.

4. Once you have bounced your message, you may want to return your settings to their previous state. To do so, click View > Headers > Normal.





Richweb SmarterMail

1. The normal mail display is HTML format.

2. Click on the “headers” from the view section.

3. Cut and paste all of the headers and the body of the message into a new email that you will send to reportspam@richweb.com.





It appears that all of our emails are now coming in plain text instead of html?

MailScanner is disarming or cleaning what it thinks are unsafe emails. Most html newsletters and such are programmed by marketing companies that do not practice secure methods for html in email.

Richweb can disable this feature, though it does somewhat increase your chances of getting a trojan horse on your computer by clicking a web link off of a dangerous email. Even if this feature is turned off, MailScanner still will be able to detect Phishing attacks thanks to its ClamAV + anti-phishing engine as well as its advanced spam and dangerous URL detection.

So the increased risk with disabling this feature is light to moderate. Still if you have users that tend to get themselves into a lot of trouble, or if your desktop Anti-virus is not working well and you are having Windows pcs get damaged and need to be rebuilt, you might want to leave this feature on as your users are probably clicking links in emails and surfing to places where the malware is downloaded direct from the web.

You may also want to look at a squid proxy + open dns setup or a commercial offering like WebSense/SurfControl or Barracuda Web Filter or NetGear.

Here is a link to additional information:
http://www.richweb.com/opendns_proxy_cache

Why Forwarding your Email is a BAD idea

Many people take advantage of “email forwarding” – the ability to easily forward email from your domain onto for example your hotmail or gmail or ISP home address.

Sounds a perfectly good thing to do, and what harm can it cause? Actually forwarding is a big problem that causes headaches for the sender of the email, the email provider that does the forwarding, the email server that accepts the forward AND the recipient! In short it can cost a LOT of time and money for all involved!

Lets understand why:

Lets say your name is Julie, and you have the domain test.com. You setup an email forwarder for julie@test.com to forward to your julietoo@hotmail.com, and all your email arrives very conveniently for you at Hotmail for you to read, and process in the normal way.

The email service provider that runs test.com for you though has a problem - you probably expect that ANYTHING sent to julie@test.com is forwarded on – including all the spam that you’ve been getting lately. Lets say you get 10 emails a day on average. For most email addresses and/or domains that have been use for more than a year, you may have 10 SPAMs coming in for every legitimate email. This means that the test.com email server is going to actually have to forward 100 additional SPAMs a day to hotmail.

Of course the hotmail Mail Firewall sees this behavior (100 SPAMs a day from the same sending machine) and quickly blacklists (refuses ALL messages from) the test.com email server. Not only is the email server that runs test.com seen as a SPAMMER, test.com is now seen as a SPAM SOURCE. This means that the reputation of both your domain and your service provider is damaged.

Additionally, if you have setup a catch-all email address - i.e. @test.com so that sales@, info@, jules@, etc all work and go to your hotmail account via a forward you have an even bigger problem. If a SPAMMER tries a dictionary attack against test.com - sending hundreds or thousands of emails to made up addresses @test.com then the test.com email service provider will be forwarding ALL of those messages on to hotmail, which will have the server blacklisted within minutes.

Suddenly you stop getting ANY email into your Hotmail account that you expect from your forwarded account. Who do you call ? Well, you will be lucky if you can actually get anyone from a large ISP (Verizon/Comcast/Embarq, etc) or large mail provider (hotmail, gmail, yahoo) to talk to. And even if you could you would get the no problem here, must be on the other end response, because as far as that provider is concerned, all they are doing is saving you the headache of getting an additonal 110 SPAMs a day (your 100 SPAMs plus the 10 legit emails). Remember, when you deal with large companies that process millions of emails an hour, its impossible for them to really care or worry much about a few legit emails that get blocked. Blocking the massive SPAM inflow is much more important, because if their customers get thousands of SPAMs each day, they would simply not use and/or pay for their service.

So, next you call the provider of test.com to investigate the problem on their side. The answer you will get is: "no problem here, we see that hotmail.com is blocking our attempts to send email". The provider may or may not be able to get hotmail.com to take action and fix this. More often than not, this is very time consuming for the providers to track down a human on the opposite side that is able to fix the problem. So email remains broken, or in a state of flux (sometimes works, sometimes does not, depending on whether hotmail removes the blacklist after a period or not, and depending on how much SPAM comes through the auto forward).

This is clearly not an ideal situation, and it gets worse. Some domains create SPF (Sender Permitted From) records to deal with forged emails. If the SENDER of an email has an SPF record, and the RECIPIENT of and email uses a forwarding account, things go haywire so to speak. More often than not the RECIPIENT side email firewalls will block the message unless the FORWARDING service provider has all of its mail servers added to the SPF record. This is difficult to do, and if this information changes, email breaks.

Finally, to avoid the forwarding of SPAM mess discussed above. most providers (if they have any clue at all) will fully SPAM filter all email BEFORE its forwarded, so they avoid getting blacklisted for forwarding SPAM. This means that an email will take the following path:

SENDER :: FORWARDER_FIREWALL :: FORWARDER :: RECIP_FIREWALL :: RECIPIENT

Either of the 2 firewalls - FORWARDER or RECIPIENT can possibly reject a message due to it matching:

1. SPAM or SPAM-like content (often the case if you forward off color jokes, or other chain letter type email)

2. VIRUS or SPYWARE

3. DANGEROUS file names or file contents (like a "cool" screensaver you found)

4. LARGE FILE ATTACHMENTS (multiple photos for example)

Each of the firewalls will have different policies (support FORWARDING firewall allows 20 MB attachments, but RECIPIENT firewall only allows 5 MB attachments because its a FREE ACCOUNT!)

Troubleshooting where the email was blocked wastes the time and resources of each provider (FORWARDING and RECIPIENT) neither of which will be sure where the problem really is unless they investigate manually, which generates zero profits, only costs for the providers.

Many web hosts are now banning email forwarding to third party email accounts, removing the capability all together. And the result for these hosts is a serious decrease in spam complaints against their servers. Richweb does not ban email forwarding just yet, but it is inevitable that for most providers that forwarding email externally is just too much trouble, and the benefits to everyone by turning it off, far outweigh any benefits of having this so called “feature”.