The best way to can spam

These approaches deal with organizational identity, and so they cut a wide swath. Ideally, they should be complemented with finer distinctions based on the identity of individual senders. One possible addition is the challenge/response protocol offered by EarthLink and other e-mail providers. In this scheme, an unknown sender is challenged to read digits embedded in an image on a Web page; if successful, the sender is then exempt from future challenges. It’s highly effective, but it’s inappropriate and rude in a customer-service-oriented business setting.

Free IT resource Virtualization Insights from Top Experts - Learn how virtualization gets real! Sponsored by Dell Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft DOWNLOAD REPORT    Looking for this InfoWorld Special Report in one complete, printable format? Download it for free. >> Special Report: Canning spam

Another candidate is the S/MIME (Secure MIME) digital signature. Although all the major e-mail clients have supported that scheme for years, most enterprises opt not to deploy the client certificates that would enable digital signing and encryption.

Were such policy implemented now, it would simplify spam prevention in one way but complicate it in another: A digital signature, in and of itself, wouldn’t mean that a message was good, because spammers would sign their messages, too. But it would enable the anti-spam engines to look up the spammers’ signatures in online databases in a more granular way than using DNSBLs.

If people also used their digital IDs to encrypt messages, IT would find itself in a bit of a quandary. Plain-text e-mail leaks confidential data like a sieve, so if users and their correspondents in other organizations make routine use of encryption, it will help seal the holes. The downside is that encrypted mail becomes opaque to another weapon in the spam detector’s arsenal: content analysis.

Click for larger view.

Inspecting the content

Anti-spam vendors collect and analyze vast databases of spam, boiling messages down to fingerprints (“fuzzy checksums”). These can be compared to corresponding fingerprints derived from incoming mail to weed out unwanted messages. These vendors also maintain vocabularies of objectionable terms and use Bayesian and other kinds of content classifiers to watch for common spam terms, but these approaches require a lot of feedback and training.

Bayesian filtering, which computes a score for each message based on statistical analysis, is proving both a popular and effective spam-fighting technology (infoworld.com/29). This does require users to make an initial investment in classifying their messages, and that’s something IT is rightly reluctant to impose. When trained, a client-side Bayesian filter adapts to the ever-changing stream of mail in a very easy and natural way: the Delete key becomes a more powerful Classify-as-Spam key.

Some users love this ability to define spam in a personal and unique way; others would rather not have to think about the problem. But until Bayesian filtering or related techniques are part of the standard enterprise desktop — as will begin to happen when, for example, Outlook 2003 rolls out — many companies are reluctant to embrace end-user deployment.

“Most users can’t even figure out how to use Outlook rules, so it’s a struggle to make the case for an individualized solution,” says Andrés Kohn, director of marketing at Cupertino, Calif.-based Proofpoint.

IT would rather do things in a centralized way, and that’s the right instinct — but client-side technologies can be powerfully complementary to the gateway technologies. The combination works well when users want to be more engaged in the anti-spam processes and enterprises support those efforts. Overall, it makes sense to combine the two. If you can pick only one, start with a centralized gateway solution — it will keep users slightly removed from the processes, making both users and IT departments happy.

Rethinking e-mail’s purpose

Anti-spam vendors want to deliver centralized turnkey solutions, and customers want to have them. The devil, however, is in the details. Because the damage done by a single missed message could be severe, anti-spam systems quarantine caught messages so that users can review them and, if necessary, release false positives. Of course, when the volume of junk falls off dramatically, users don’t want to root around in the haystack looking for a few needles.

When the only tool you have is a hammer, every problem looks like a nail. But e-mail is no longer the only communication tool at your disposal. RSS is an easier and better way to run a subscription service. IM trumps e-mail for intensive real-time collaboration. These methods are useful in their own right, but the spam epidemic makes them even more compelling.

Even as our supply of anti-spam weapons grows, it makes sense to find ways to reduce the demand for them. Reliable, direct, and spontaneous communication, from anyone to anyone, is the special magic of e-mail — what makes it such a precious resource. Reserving e-mail for these purposes will make spam detection easier and will help us conserve that resource.