Commercial solutions win, spam loses

Proofpoint Protection Server

Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft Free IT resource Attend the SOA Executive Forum: Breaking SOA Bottlenecks SOAExecForum.com/may2007 Sponsored by InfoWorld DOWNLOAD PDF Click here to download InfoWorld's special report: Spam shootout

The Proofpoint Protection Server is a gateway that runs on Linux (Red Hat 8 or 9) or Solaris. Enterprises using Solaris or Linux and sendmail will find it a comfortable, easy fit. Fortunately, companies using Exchange, Notes, or other e-mail platforms can rely on Proofpoint to get things running. Proofpoint will even install its server on a system you send to it at no additional cost.

I installed the software on Red Hat Linux 9, with help from one of Proofpoint's systems engineers. She talked me through getting the Linux system configured properly, getting sendmail set up, and installing and configuring the Protection Server, which includes the MySQL database server for storing quarantined e-mail.

Configuration is simple, and delegation is straightforward -- although not as granular as it is in Postini. Multiple administrators can be created, and each has a limited set of seven areas to which they either do or don't have access. Rather than the two categories the others use in their reports, "spam" and "not spam," Proofpoint has three: "definitely spam," with a score of 80 to 100; "probably spam," with a score of 50 to 80; and "definitely not spam," with a score of 0 to 50. The qualifying scores can be changed for each category, and the action taken on the message can be different for each. For example, you could opt to delete messages that fall into the "definite spam" category and quarantine those in the "probable spam" category. Content filtering is also easy to set up, with a dictionary of undesirable terms included.

As often as administrators like, clients are sent a digest via e-mail that allows them to view quarantined e-mail, sorted by likelihood that it is spam. Users can release e-mails from quarantine and can whitelist senders directly from the e-mail client.

Proofpoint was second only to Brightmail in accuracy, catching more than 94 percent of spam. It also had no critical false positives, although its ability to recognize legitimate mass mailings fell slightly short of the three other commercial products.

SpamAssassin

You get what you pay for. SpamAssassin is an open source gateway that is included with Red Hat Linux 9, and can be downloaded free from spamassassin.org. However, it took more than 10 times as long to install and configure SpamAssassin as it did any of the other products. I achieved a much lower level of performance to boot -- roughly 63 percent accuracy in identifying spam, with a relatively high number of false positives.

I installed SpamAssassin Version 2.44 along with Red Hat Linux 9. Installing Red Hat 9 is easy, and the SpamAssassin package is included with the mail server installation. But just because the software is installed does not mean it will work -- filtering criteria must be added manually, and until that's done nothing is filtered out. Getting the various configuration files edited properly so that the whole package worked was not simple. Documentation was difficult to find, and not always easy to follow.

There are blacklists available that you can subscribe to, and some are updated regularly, but these are noncommercial lists with no guarantees. The whitelist is not difficult to add to, but there is no mechanism for end-users to add to the whitelist or to automatically notify the administrator to add senders. Filtering rules are relatively basic, and although there is a Bayesian filter available, it is not part of the distribution -- and I wasn't able to get it working for this review.

SpamAssassin is the perfect example of first-generation techniques becoming outmoded by advances in spamming technology. It looks for keywords in the subject or body of e-mails, but is frustrated by words not in the dictionary, such as "V!agra," or words that contain invisible HTML characters. It might be possible to get SpamAssassin to perform at a level similar to the other products reviewed here, but it would take a lot of work in addition to constant maintenance and research by the administrator.

Choosing Your Weapon

All of the commercial products worked well in my tests, and all should prove satisfactory in a corporate environment. After initial setup and a few weeks of tuning the whitelists for your organization's mail stream, false positives should drop to very near zero, and any of these anti-spam solutions should disappear into the background, requiring little attention.

The two services, FrontBridge and Postini, receive higher marks for setup and management -- they are easier to install and administer than the gateway solutions. The biggest drawback to services (e-mail won't get through if the service goes down) shouldn't be a factor with either of these solutions, as both have multiple datacenters and excellent reliability records.

FrontBridge offers a good feature set and is very easy to use, with excellent reports and fine anti-spam performance. Postini has the broadest feature set of any of the products I looked at, including the greatest range of controls over filter settings by user, group, or domain, and by types of spam filtered. Its controls over content filtering on inbound and outbound messages are also the most complete, an advantage for managers concerned about liability for e-mail content.

The gateways may make some paranoid (read: experienced) administrators more comfortable because they're nearly impossible to bypass by targeted spam attacks and they're completely under local control. They all require subscriptions or maintenance fees to keep working, so there isn't much difference from a service in that respect. Brightmail offers the broadest platform support among the gateways, nice integration with Exchange, and great accuracy in identifying spam -- the highest in the test. It was also the most appliancelike in installation and setup -- a real "set and forget" system.

Proofpoint is a good choice for Linux or Solaris shops, providing extremely high accuracy, great support, and excellent manageability through a Web interface. The next version, due in early December, will make filtering options by users and groups available, as well as provide additional reporting tools and management features.

Considering the price per user, per year, and given the time they'll save your users and administrators in dealing with spam, they're all bargains.