The best way to can spam

It’s a security risk, a productivity drain, and just plain annoying. Put a lid on unwanted e-mail with a multi-layered strategy

If people also used their digital IDs to encrypt messages, IT would find itself in a bit of a quandary. Plain-text e-mail leaks confidential data like a sieve, so if users and their correspondents in other organizations make routine use of encryption, it will help seal the holes. The downside is that encrypted mail becomes opaque to another weapon in the spam detector’s arsenal: content analysis.

Click for larger view.

Inspecting the content

Anti-spam vendors collect and analyze vast databases of spam, boiling messages down to fingerprints (“fuzzy checksums”). These can be compared to corresponding fingerprints derived from incoming mail to weed out unwanted messages. These vendors also maintain vocabularies of objectionable terms and use Bayesian and other kinds of content classifiers to watch for common spam terms, but these approaches require a lot of feedback and training.

Bayesian filtering, which computes a score for each message based on statistical analysis, is proving both a popular and effective spam-fighting technology (infoworld.com/29). This does require users to make an initial investment in classifying their messages, and that’s something IT is rightly reluctant to impose. When trained, a client-side Bayesian filter adapts to the ever-changing stream of mail in a very easy and natural way: the Delete key becomes a more powerful Classify-as-Spam key.

Some users love this ability to define spam in a personal and unique way; others would rather not have to think about the problem. But until Bayesian filtering or related techniques are part of the standard enterprise desktop — as will begin to happen when, for example, Outlook 2003 rolls out — many companies are reluctant to embrace end-user deployment.

“Most users can’t even figure out how to use Outlook rules, so it’s a struggle to make the case for an individualized solution,” says Andrés Kohn, director of marketing at Cupertino, Calif.-based Proofpoint.

IT would rather do things in a centralized way, and that’s the right instinct — but client-side technologies can be powerfully complementary to the gateway technologies. The combination works well when users want to be more engaged in the anti-spam processes and enterprises support those efforts. Overall, it makes sense to combine the two. If you can pick only one, start with a centralized gateway solution — it will keep users slightly removed from the processes, making both users and IT departments happy.

Rethinking e-mail’s purpose

Anti-spam vendors want to deliver centralized turnkey solutions, and customers want to have them. The devil, however, is in the details. Because the damage done by a single missed message could be severe, anti-spam systems quarantine caught messages so that users can review them and, if necessary, release false positives. Of course, when the volume of junk falls off dramatically, users don’t want to root around in the haystack looking for a few needles.