Commercial solutions win, spam loses

Brightmail, FrontBridge, Postini, and Proofpoint overwhelm open source in accuracy, flexibility, and ease

As anyone with an e-mail inbox knows, the spam problem isn't going away. According to a major anti-spam vendor, spam has increased from 8 percent of all e-mail traffic in 2001 to 50 percent in July 2003. Other estimates show that figure as high as 70 percent of all traffic. Two classes of products can help slay spam in the enterprise environment: gateways and services. Both allow you to block spam for all network users at a single, centrally managed point before it hits your mail server.

For this review, I looked at two services and three gateway products. Services filter spam before it arrives at your network, reducing the volume of traffic on your Internet connection. Services also typically offer multiple datacenters for redundancy, high volume, and fast response. Setup requires merely changing the MX (mail exchange) record for your domain. But a service is not under a local administrator's control, so if the service goes down, mail may not get through.

Gateways are harder for spammers to circumvent by sending e-mail to the real mail server's IP address; they offer local control of the anti-spam technology; and they allow mail to continue to arrive if the anti-spam gateway goes down. But a gateway gives the local administrator yet another system to maintain, and the total traffic through your Internet connection remains the same because spam isn't filtered until it reaches your network.

The five products I tested: Brightmail Anti-Spam Enterprise Edition Version 5.1, FrontBridge TrueProtect E-mail Security Suite, Postini Perimeter Manager Enterprise Edition, Proofpoint Protection Server 1.2.1, and SpamAssassin 2.44, an open source spam filter included with Red Hat Linux 9.

In contrast to the commercial products, SpamAssassin represents an older, first-generation anti-spam solution, and its age showed in my tests. It filtered only 62 percent of spam, whereas the other products produced great results, blocking 90 percent to 96 percent of all the spam they encountered with few, if any, legitimate messages blocked.

Differentiating between spam and legitimate messages can be difficult. Newsletters, press releases, and other marketing materials from companies you have a relationship with can be very similar to spam in content. These all present challenges to the filters. The e-mail I used for testing was real e-mail containing many messages that stressed the filters.

I looked at two categories of mail incorrectly identified as spam: false positives that were not critical, such as newsletters and marketing information; and false positives that were critical, such as personal e-mail from colleagues. Each product was tested with a different stream of mail, so the number of messages received varied, but all received enough messages to assess their capabilities.

The critical issue is not that the filter may have misidentified a few e-mails, but how easily those messages can be found and added to a whitelist so that future e-mails from the same source are not stopped. All the products except Brightmail and SpamAssassin allow end-users to add senders to the domain whitelist themselves. Brightmail allows users to forward misidentified e-mails to the administrator, who can choose to add the sender to the whitelist. SpamAssassin allows only the administrator to add to the whitelist, with no direct access for users.

All the products allow the administrator to blacklist known spammers and choose among a variety of responses to messages identified as spam -- adding an identifier to the subject line, adding a message header, deleting the message, or quarantining it. Delegation of specific administrative functions is possible with all the products except SpamAssassin, although the granularity of delegation varies among the four. Spam settings can be set by enterprise (multiple domains) or domain, and Postini also allows individual groups or users within a domain to have different rules.

And all the products but SpamAssassin use dynamic updates to keep up with the evolving technologies spammers use to circumvent less sophisticated filters. The default update cycle may be every few minutes or once per week, depending on the product. Keeping the filters up to date requires a subscription or maintenance fee.

Finally, in addition to stopping spam, all four commercial products provide content-filtering features, allowing the administrator to block incoming or outgoing e-mail that contains proprietary data, audio or video files, executables, sexually explicit words, or racial slurs. They also provide protection against DoS attacks and directory harvesting attacks.

In my testing, the performance of the newer products was more than acceptable in every case. Per-user, per-year pricing should not be an obstacle, even for the most expensive product. Choosing the right product will depend on your network topology, your philosophy regarding outsourcing, requirements for administrative control and reporting, traffic loads, and your operating system and mail server platform.

Click for larger view.

Brightmail Anti-Spam Enterprise

This gateway product constantly interacts with Brightmail's datacenter to keep filtering rules current. The gateway polls Brightmail's datacenter every few minutes and downloads new rule sets when they're available, in much the same way anti-virus applications do.

Brightmail's software can be installed on Linux, Solaris, or Windows, and features an easy to use GUI installer on all three platforms. I installed the gateway on a Windows 2000 server with Exchange Server 2000 and enabled Brightmail's Exchange spam folder agent in less than 10 minutes. The software automatically contacted the Brightmail site and downloaded the latest set of rules. No additional configuration or tuning was necessary. Brightmail caught the highest percentage of spam and had the lowest false-positive rate of any of the products tested.

Brightmail is the only product that does not allow end-users to add senders to the whitelist. On the other hand, Brightmail includes a spam folder agent for both Exchange and Lotus Domino -- all mail identified as spam can be sent to the end-user's spam folder, and an Outlook agent allows users to forward e-mail to the administrator, indicating "spam" or "not spam" with one click.

This makes scanning and recovery of false positives very simple and straightforward. Alternatively, mail identified as spam can be tagged as such in the header or subject line, and spam can be sent to a central spam mailbox, saved to disk, delivered normally, or simply deleted. You can configure different policies for different domains.

Brightmail offers extensive reporting features, a wide variety of standard reports as well as custom reports. An optional anti-virus capability powered by Symantec is available at additional cost, with virus definitions and engine updates delivered by Brightmail.

FrontBridge TrueProtect

FrontBridge is a hosted service that incorporates four layers of e-mail filtering -- custom blacklists, proprietary fingerprinting, adaptive rules-based scoring, and real-time attack prevention, which blocks illegitimate and potentially damaging e-mail based on a sender's IP address. FrontBridge was recently selected by Sprint as its anti-spam solution.

Installing FrontBridge consists of merely changing the MX record for your e-mail server to point to the FrontBridge mail processor. FrontBridge processes all your e-mail, incoming and outgoing, and forwards the good stuff to your mail server or its outbound destination. There is no impact on your local network configuration, and overall Internet traffic is reduced because spam never reaches your network. FrontBridge claims never to have had a service outage and guarantees 99.99 percent uptime. With eight datacenters worldwide, the company seems to have the infrastructure to make such a guarantee. FrontBridge offers additional services beyond anti-spam, including anti-virus, content filtering, policy enforcement (such as who can send and receive which file types), and disaster recovery, which involves holding all e-mail for as long as five days if your network is unreachable.

Configuring accounts and other administrative tasks is done through an HTTPS log-in to FrontBridge's Web site. Setting up accounts is simple: An automated user enrollment feature allows all the accounts in a domain to be added without having to build an access control list. Administrative tasks, such as modifying filter rules or anti-virus settings or adding and deleting users, can be set by domain so that each of several domains or sub domains can each be maintained by different administrators.

Reporting is excellent, and reports can be easily exported to Excel for analysis. By default, a digest of filtered spam is delivered weekly to all users as an HTML e-mail. Users can retrieve any e-mail that has been quarantined, and can whitelist the sender with a single click. End-users can also log in to the Web site at any time and view all filtered messages with the same options to deliver the message or whitelist the sender.

FrontBridge caught 90 percent of the spam in the test, ranking below Brightmail, Proofpoint, and Postini in accuracy. But it misidentified no critical e-mail, and only 1 percent of noncritical messages, proving more adept than all but Brightmail at avoiding false positives.

Postini Perimeter Manager

Click for larger view.

Postini's anti-spam service processes about 150 million messages per day. Although it started as a service for ISPs, it has recently moved into the enterprise space and provides a broad, sophisticated array of services. It is the only product I tested that includes anti-virus scanning in the base price.

Setting up the service is simple, requiring the same MX record change as FrontBridge's service. Adding users is automated and very easy -- each user receives a message the first time that spam is blocked from their account, letting them know how to access quarantined e-mail and retrieve, delete, or whitelist mail. All administrative tasks can be accomplished through the Postini Web site, and management tasks can be delegated in a very granular manner. Managing multiple domains is easy. Reporting is flexible in the criteria reported, but long-term tracking is not available in the standard corporate edition -- only daily and weekly reports are made available.

Response to spam is unusually flexible, and can be set by individual, group, or domain. Administrators can allow users to add senders to the whitelist, retrieve messages from quarantine, and even change filter settings -- or they can lock things down so that end-users can do nothing without an administrator. The spam filters have separate settings from lenient to strict for a variety of categories, including bulk e-mail, special offers, get-rich-quick messages, and adult content.

The Standard Edition includes spam filtering, inbound server monitoring, connection management, delivery management, detailed reporting, inbound attachment management, inbound virus blocking, and inbound content management. The Enterprise Edition adds outbound server monitoring, outbound virus blocking, outbound attachment management, outbound content management, and disaster-recovery service. It can also check outbound e-mail for policy violations concerning language, recipients, and attachments.

Postini is very flexible and feature-rich, and it caught nearly 94 percent of spam in my tests, edged out only by Brightmail and Proofpoint. It lagged slightly in avoiding false positives, but the differences here could easily be overcome by whitelist tuning.

Proofpoint Protection Server

The Proofpoint Protection Server is a gateway that runs on Linux (Red Hat 8 or 9) or Solaris. Enterprises using Solaris or Linux and sendmail will find it a comfortable, easy fit. Fortunately, companies using Exchange, Notes, or other e-mail platforms can rely on Proofpoint to get things running. Proofpoint will even install its server on a system you send to it at no additional cost.

I installed the software on Red Hat Linux 9, with help from one of Proofpoint's systems engineers. She talked me through getting the Linux system configured properly, getting sendmail set up, and installing and configuring the Protection Server, which includes the MySQL database server for storing quarantined e-mail.

Configuration is simple, and delegation is straightforward -- although not as granular as it is in Postini. Multiple administrators can be created, and each has a limited set of seven areas to which they either do or don't have access. Rather than the two categories the others use in their reports, "spam" and "not spam," Proofpoint has three: "definitely spam," with a score of 80 to 100; "probably spam," with a score of 50 to 80; and "definitely not spam," with a score of 0 to 50. The qualifying scores can be changed for each category, and the action taken on the message can be different for each. For example, you could opt to delete messages that fall into the "definite spam" category and quarantine those in the "probable spam" category. Content filtering is also easy to set up, with a dictionary of undesirable terms included.

1 2 Page 1