Shades of greylisting

In many cases, greylisting works very well and can significantly reduce spam volumes, but it's not for everyone.

I've talked about greylisting many times in the past when detailing some of my anti-spam measures. For those that don't know what greylisting is, the elevator pitch is that it produces a temporary SMTP error to incoming mail relays, refusing the mail on the first attempt. If the relay attempts to send the same message again (as identified by the sender/recipient/relay tuple), it is accepted, and that tuple is then not blocked again for some long period of time. The upshot of this is that most spammers blast mail out without worrying about queueing failed requests. They get what they get, and the overhead of dealing with SMTP errors isn't worth the effort. For deeper reading, take a look at Evan Harris' greylisting information.

For a number of years now, I've been using a Frankenstein version of Evan's Sendmail::Milter-based relaydelay code. Since 2004, I've run that code with minimal modifications on mail relays of every shape and size. Matt Prigge and I even wrote a whole management Web UI that supported searching, whitelisting, and blacklisting. For my own massive spam problem, I took the 0.04 release and proceeded to rewrite it substantially, adding firewall integration that will cause pf to drop all packets from abusive hosts, and a significant number of other features that I found necessary over the years. That particular greylisting implementation is still running full-steam and is basically the only reason that e-mail is still viable for that domain, which is a three-letter domain plagued by dictionary spammers and sees a 99.99 percent spam volume.

However, both relaydelay and the code it depends on are growing long in the tooth. Sendmail::Milter hasn't been updated since 2001, and refuses to compile properly on 64-bit hosts. Relaydelay itself is showing its age and could use some new features. Thus, when I was rebuilding a mail relay recently, I decided to go with milter-greylist instead of relaydelay.

Milter-greylist produces the same results as relaydelay, but uses a completely different method. It's completely written in C and functions as a sendmail milter like relaydelay. Instead of using a MySQL database to store all the tuples, milter-greylist keeps all that info in RAM, which makes for far faster lookups but can take some tweaking on large volume servers. To retain the tuple database across service restarts and reboots, milter-greylist writes out the internal database to a flat file periodically.

One of the major reasons why I switched this particular relay over to milter-greylist was peering. The relay I was rebuilding is actually a secondary relay, with the primary running relaydelay. Once I'd configured milter-greylist on the secondary relay, I did the same on the primary, and set up the peering between them, so that they share the same databases. This feature is particularly handy.

Overall, milter-greylist works quite well and uses a fairly straightforward configuration syntax. It would be terribly handy if the config supported includes, since that would make scripted blacklist/whitelist additions much simpler. One of the best features of relaydelay was that the whitelist and blacklists were actually just rows in the database. Thus, they could be added/deleted/updated on the fly with instant results, and no need to restart the daemon process. I've used this to write simple perl and PHP scripts that automatically modify rows in the database depending on outside influences, such as the presence of a particular email in an IMAP subfolder. Thus, if I drag a spam into a specific folder, within a few minutes, the originating relay for that email has been blacklisted by that code. Very handy.

Milter-greylist uses a fixed configuration file for whitelist/blacklist features, and while it's very easy to work with, it's far more difficult to wrap into a management layer. It might also be useful for milter-greylist to accept configuration requests during runtime. That way, blacklist/whitelist/acl entries can be added/deleted/modified on the fly, and the various tables and acl entries could be dumped to file like the database. Something akin to pfctl would be dandy. Oh, and the permissions on the database dumpfile should be a configurable parameter.

The lack of any Web UI element makes it difficult to delegate day-to-day monitoring to those who don't have administrative access to the relays, but I did throw together a very simple milter-greylist monitoring/searching UI written in PHP, which I'll release at some point in the near future when I get a few bugs ironed out and make it a little more robust.

There are other open source greylisting tools out there for just about every MTA, and even some commercial variants exist now. In many cases greylisting works very well and can significantly reduce spam volumes, but it's not for everyone, especially if you can't handle the chance that an important email may be delayed for a few minutes at best. Otherwise, it's definitely worth the effort.