Prefilters put spammers in the crosshairs

CipherTrust, Mirapoint, Symantec, and Tumbleweed appliances bring IP blocking, TCP/IP throttling, and other network-level tricks to the battle against spam

Anti-spam filtering technologies have been perfected to the point where you can expect to see better than 95 percent accuracy, with no more than a couple of false positives out of every 10,000 messages. Despite this amazing progress, enterprises are still under attack. The grim truth is that filtering even 100 percent of incoming spam doesn't necessarily solve the spam problem for large organizations.

The reason is that a high volume of spam, even when it's caught, can be extraordinarily expensive for larger organizations that are finding that they need to add more mail servers, and more spam filters, to handle the load. Considering that spam can amount to between 80 percent and 95 percent of all incoming e-mail, a large enterprise could substantially reduce the number of mail servers and filters it manages and maintains if most of that spam would just go away.

Too much to ask? Each of the four vendors discussed here -- CipherTrust, Mirapoint, Symantec, and Tumbleweed Communications -- use different approaches to rejecting spam before it enters the corporate network. Although the methods differ, each solution is designed to complement a traditional anti-spam solution. All can dramatically ease the burden on your message filters and mail servers, creating more processing headroom for legitimate messages and ultimately reducing the number of mail systems you need to deploy and maintain.

Rather than filtering e-mail based on its content, these appliances use the sender's IP address, the recipient's address, and other factors to identify messages at the TCP/IP or SMTP protocol level. They then block all traffic from the IP addresses of known spammers, limit the number of connections or messages per minute from the IP address of a likely spammer, or allow all messages from addresses with a clean reputation.

CipherTrust IronMail Connection Control uses a reputation database the company calls TrustedSource to rate IP addresses of e-mail senders, for either sending no spam, sending lots of spam, or sending some spam, based on recent activity monitored by CipherTrust's global network of spam collectors. Connection Control then either rejects connection attempts from known spammers for a designated period or accepts their connections, allowing them to pass only a few messages an hour.

Mirapoint MailHurdle sends an SMTP retry message to the originating server, taking advantage of the fact that real e-mail servers will readily resend but most spam engines aren't equipped to retry. The recipient address can also be verified to ensure that the message was sent to a real user and not a random address. This retry and verification helps to stop directory harvest attacks, which attempt to identify all the valid addresses in a domain to sell to other spammers. If a message is addressed to invalid recipients, it is ignored; if it is addressed to both valid and invalid recipients, indicating that the sender is fishing for valid addresses, the connection is throttled, and the IP is marked as a suspicious sender. 

2005_3300.xml
Click for larger view.

Symantec uses a technique similar to that of CipherTrust, relying on the Brightmail reputation service and then throttling connections from IP addresses with bad reputations. The SMS (Symantec Mail Security) 8100 series allows known spammers only a few simultaneous connections -- or only one connection at a time -- and accepts only a few messages or even just one per hour, greatly restricting the ability of spammers to push their messages through the gateway.

Tumbleweed MailGate Edge takes a different approach altogether, focusing on authentication of the sender and recipient. MailGate Edge blocks or throttles bandwidth on mail that is sent to invalid users, using your e-mail server's directory -- Active Directory, Exchange 5.5, LDAP -- to identify valid users. It also identifies senders whose e-mail addresses don't match the IP address from which the e-mail was sent, using a reverse DNS lookup to verify that the domain in the header of the message is truly associated with the source IP address. In addition, MailGate Edge looks for nonstandard SMTP communications, blocking e-mail sent by some spam robots, aka spambots. To protect against DoS and directory harvest attacks, it watches for messages with lots of invalid users or for spikes in connection attempts or message volume.

These appliances can reduce network traffic and the load on the anti-spam filter or e-mail server by 50 percent to 90 percent, eliminating the need to upgrade mail server hardware or deploy additional servers. In addition, they can all protect your system against directory harvest, DoS, and DDoS attacks on e-mail servers, by identifying illegitimate mail flows and blocking the addresses from which they come. They also have the benefit of making spambots work harder to push messages through your gateway, encouraging spammers to remove you from their list of targets.  

All these methods work well for now. Time will tell whether spammers will figure out how to circumvent them. In some cases, evasion may prove possible but too expensive. For instance, if a spambot has to retry hundreds of thousands of connections an hour to push messages past Mirapoint's MailHurdle, the hardware resources necessary may become prohibitively expensive. Nevertheless, in all likelihood, these network-level spam defenses will have to continue to evolve, as spammers become increasingly more sophisticated.

The methods these four appliances use may not be terribly useful to small organizations. If you have only one lightly stressed mail server, you don't need to cut e-mail volume. Organizations with large volumes of e-mail, however, can gain dramatic benefits. Because the domains I use to test anti-spam products receive only about 10,000 messages a week, I contacted customers of each of the four vendors to get a feel for how the products perform in a high-volume production environment.

In addition, I brought the Mirapoint, Symantec, and Tumbleweed units into my lab to examine the administrative interface, management features, ease of installation, and functionality. The latest version of the CipherTrust software, IronMail 6.0, was not available in time for my test.

CipherTrust IronMail Connection Control

Connection Control is a feature of CipherTrust's IronMail e-mail security appliances, arriving with Version 4.5. I was not able to try CipherTrust's newest edition in my lab, but my experience with IronMail 4.0, which I reviewed in February 2004, suggests that Version 6.0 will be easy to install and configure. (IronMail 6.0 and a new pre-filtering appliance, called IronMail Edge, should be available by the time you read this.) As opposed to the Symantec and Tumbleweed appliances reviewed here, the IronMail appliance -- and the Mirapoint RazorGate appliance -- includes a full anti-spam filtering engine and e-mail server, as well as the Connection Control pre-filtering technology.

CipherTrust's TrustedSource is a reputation system that rates millions of IP addresses, using data garnered from more than 1,400 customers who have CipherTrust's anti-spam filtering systems installed. Connection Control uses the reputation of the sender to determine whether to reject connections from an address for a set period of time or to control the bandwidth available for that specific connection so that it might take an hour for a single message to make it through. Most spammers can be consistently associated with relatively few addresses, and the IronMail system, which starts by restricting the bandwidth available to unknown and therefore suspicious senders, catches up with new spammers quickly through rapid updates to the database.

2005_3300.xml
Click for larger view.

To get an idea of IronMail's real-world benefits, I spoke with Franklin Warlick, the Messaging Systems Administrator of Cox Communications. He manages the Cox.com corporate e-mail system, which has 35,000 mailboxes. This is Cox's internal e-mail system, not its ISP network. Warlick says that out of 50 million messages received per month, about 95 percent are spam.

Cox's mail system consists of six IronMail appliances filtering out spam and forwarding the rest of the e-mail to two Microsoft Exchange servers. When Connection Control was installed, the six appliances were running at close to 100 percent utilization, and a purchase order had been submitted for six more. After Connection Control was installed, the load on the six appliances was reduced to less than 50 percent utilization and has stayed at that level.

The six IronMail appliances are currently blocking close to a million connections per day. IP addresses identified as spammers are added at the rate of about 10,000 unique addresses per day. The addresses are automatically aged off the system after four days, and they're usually added back almost immediately.

Warlick says his goal is zero false positives, and based on his tracking system, IronMail seems to have achieved this, logging zero user complaints about legitimate mail not showing up. He says the accuracy of TrustedSource also allows Connection Control to do a very good job of keeping illegitimate mail out of the filters.

Mirapoint MailHurdle

MailHurdle is a component of Mirapoint's RazorGate appliances. I tested the Mirapoint RazorGate 100 on-site at California State Polytechnic University (Cal Poly) in San Luis Obispo, and my review was published in April 2005. To summarize, at Cal Poly, MailHurdle has been stopping 60 percent or more of spam before it is delivered, thus reducing the traffic strain on the organization's network, mail server, and anti-spam filter. Similar to the CipherTrust IronMail system, the Mirapoint RazorGate is a complete anti-spam solution, combining the MailHurdle pre-filter and filtering based on message content.

MailHurdle employs a variety of SMTP protocol-level techniques to verify both the sender and recipient of each message. To verify the recipient, MailHurdle uses a directory-based lookup of the recipient address and blocks messages not addressed to real users. This procedure defends against spammers' directory harvest attacks, as well as bulk e-mails sent to random lists of users.

To verify the sender, MailHurdle looks at the originating IP address and the From and To addresses in the message. If a recipient has not previously received a message from the sender's IP address, MailHurdle sends a temporary failure message to the originating mail server using a standard SMTP failure code. If the originating mail server is a bulk e-mail system, it will typically ignore the failure message, and the message will not be resent. (Viruses using raw SMTP to send messages will also not retry.) If the message is legitimate, the originating mail server will quickly retry, at which point the message is allowed through the MailHurdle gateway. (According to Mirapoint, there is one old version of Novell GroupWise that may not properly retry unless patched, but every other e-mail server the company has found supports this SMTP command.)

On the downside, if spammers begin incorporating the SMTP retry mechanism, they can bypass MailHurdle's primary blocking technique. Whether this is economically feasible for them comes down to whether they can support retries without adding too much processing overhead to their spambots. Typically, MailHurdle will cause legitimate mail to be delayed as long as five minutes -- the default retry interval. This compares favorably to the CipherTrust and Symantec systems, which could delay the delivery of legitimate mail by as long as an hour.

Cal Poly deployed its two RazorGate 450 appliances in a clustered system in May. Since then, MailHurdle reduced the number of messages that the filters had to handle from more than 400,000 per day to just over 100,000 per day. The user verification feature also reduced loads on the servers; by dropping e-mails addressed to nonexistent users, the queues of outgoing "user unknown" messages were eliminated. Of the messages that did reach the filters, 97 percent of the spam was caught; university mail admins reported no critical false positives and 0.01 percent noncritical false positives.

The RazorGate appliance's management features are good, and they are backed by effective reporting tools. The solution supports LDAP for user lookups and verification. Management of anti-spam functionality can be as granular as you like. For example, the administrator can set one policy for the whole domain and allow or forbid user access to quarantine, whitelist, blacklist, and other anti-spam criteria.

The RazorGate also supports policy enforcement on incoming and outgoing messages, looking for inappropriate language, proprietary information, or attached files that meet certain patterns. You can block mail that matches a specific pattern or save a copy for inspection.

Symantec Mail Security 8160

Symantec has two series of e-mail security appliances, the SMS 8100 series and the SMS 8200 series. The 8200 series features the Brightmail anti-spam filtering software that has done very well in our previous reviews. The 8100 is a new appliance based on "anti-spam router" technology acquired from TurnTide. The 8160 appliance combines the Brightmail reputation service with proprietary TurnTide algorithms to rate senders. It then uses the quality of service capability of TCP/IP to throttle connections so that senders at IP addresses with bad reputations can have only a few connections -- or even just one -- to the appliance and can send data only at very slow data rates, which might get one message through in an hour. When bandwidth is throttled down to a point where one e-mail transaction takes an hour, Symantec claims that 50 percent to 80 percent of all spambots will give up on the transaction, timing out after a minute or two without retrying.

If a sender can successfully spoof a reputable IP address (not the IP address in the header of the message but the actual IP address of the sending system) -- something that is very difficult to do -- then a spam, directory harvest, or DoS attack could conceivably get through, at least during the 10-minute window between updates of the reputation database. As with CipherTrust's IronMail, this is where Symantec's multilayer protection, in the form of the SMS 8200 series appliance and its Brightmail message filter, comes into play.

The SMS 8160 has three categories of senders: clean, mixed, and spammers. Mixed senders are defined as those who send 24 percent to 75 percent spam, and spammers are defined as those sending more than 75 percent spam during the past 30 days. In reality, the line between spammers and mixed senders tends to be clear-cut: Symantec says that 89 percent of all spam senders send 91 percent to 100 percent spam and little or no good mail at all.

Setup and administration of the SMS 8160 exemplifies the drop-in ease that one hopes for in an appliance. To understand the SMS 8160's impact in a production environment, I spoke with Carl Shivers, CIO of Aristotle.net, an ISP with more than 40,000 members.

When the SMS 8160 was installed, it produced a dramatic drop in messages -- from about a million a day to approximately 120,000. The statistics on messages delivered are quite interesting: You see the spikes you'd expect in legitimate messages, from a few to 20 messages per second, but spam delivery rates remain constant at about one message per second, regardless of time of day.

This contrasts markedly with Aristotle.net's pre-8160 statistics, when volumes of spam could spike up to hundreds of messages per second, saturating all available bandwidth and causing legitimate messages to be dropped. Overall e-mail bandwidth utilization has not only decreased since the installation of the SMS 8160 but is more predictable and even.

Tumbleweed MailGate Edge

Similar to the Symantec Mail Security 8160, the Tumbleweed MG1300 is a single-purpose appliance that doesn't filter spam but simply reduces what Tumbleweed calls Dark Traffic -- e-mail from invalid senders to invalid recipients, or with malformed SMTP requests that indicate a nonstandard mail server likely to be a spambot. MailGate Edge either blocks connections or allows them; it doesn't throttle message flows.

Installation of the MG1300 is simple; an LCD display on the front of the appliance makes initial TCP/IP configuration easy without the need for a serial terminal. An installation wizard speeds you through network setup and DNS setup.

To get a sense of real-world performance, I spoke with Scott Rose, senior infrastructure architect at Finisar. Four to six months ago, Rose replaced Posttix relays with the MG1300 MailGate Edge. The MG1300 initially blocked 26 percent to 28 percent of incoming mail due to invalid recipients or malformed SMTP. It also stopped a number of DoS and directory harvest attacks by detecting messages sent to both valid and invalid recipients and by detecting spikes in the number of connection attempts from the same IP address.

Because MailGate Edge's techniques are geared toward detecting threatening misbehavior, the volume of spam it blocks will depend on the number of directory harvest attacks you experience. Tumbleweed says the percentage of mail blocked at customer sites ranges from 20 percent to almost 80 percent at some large corporations and ISPs. Overall rates may not match those obtained by CipherTrust and Symantec, but this is also the only product here that doesn't require an ongoing subscription.

The Tumbleweed system uses Active Directory for internal verification of valid users and uses LDAP to edge routers for sender address verification. Finisar is also running MailGate anti-spam filtering appliances, which forward mail to an Exchange 2000 server on the back end. In the past 30 days, 100,000 of a total of 400,000 incoming messages were blocked due to invalid recipients. 

For now, the approaches used by these four appliances are quite effective. No matter which solution you adopt, the numbers you will get for spam rejected before it hits your network may vary dramatically, depending on how many users you have and the sorts of spam and SMTP-related attacks you typically get. But there's no question that you can substantially reduce the volume of messages that have to be filtered by your anti-spam appliance or e-mail server by deploying one of these boxes -- and at a much lower cost than adding another e-mail server.

None of these technologies is a complete anti-spam solution; they're all meant to be used in conjunction with message filtering. Smaller companies that are not experiencing directory harvest attacks or are not running their filtering engines at full utilization don't need them yet. Although it doesn't pay to underestimate the guile of spammers and their ability to find ways to circumvent barriers, these solutions should continue to work well for at least the next couple of years, and the vendors are already looking to additional techniques to improve their products.

From CIO: 8 Free Online Courses to Grow Your Tech Skills
Join the discussion
Be the first to comment on this article. Our Commenting Policies