When good recommendations go bad

Some guideline documents just age. They include settings long ago recommended, but since disproved or no longer needed. Some include unintentional errors that, once printed, seem to live on without anyone questioning their veracity. One government list I saw recommended blocking 20 specific file extensions on incoming e-mail. Ignoring for the moment that the list really should be “deny by default, allow by exception," the 20 file extensions included one that doesn't exist. I did months of research (in my spare time), only to learn that the file extension was mentioned in the source code for one worm that was popular for a month nearly a decade ago. The problem was that the extension never existed; it was a worm writer’s typo. But that didn’t stop it from being codified and promoted as a “best practice.”

Savvy technical people might try to ignore bad advice, but more and more auditors are demanding that we follow “best practice” guidelines. I can’t blame the auditors; they're just doing what they're told. Most of them aren’t that technically savvy, but auditors follow guidelines like they were a religion's holy book.

In discussing this problem with my friend Susan Bradley, she correctly pointed out that you don’t get in trouble for following the mandated guidelines, but you will have some “ 'splaining to do, Lucy” if you deviate. Great point.

Fixing guidelines

As guidelines and gold standards become more a part of our mandated life (and for many enterprises, that’s a good thing), we need to fix those that are broken. Here are my ideas:

First, every recommendation made in a guideline should be thoroughly tested and challenged before it’s put into the official document. Sounds like a no-brainer, right? But there's so much proof to the contrary -- because many recommendations don’t work.

Second, every recommendation document should discuss what the recommendation will do to your system, why it’s good, and what legitimate things it could possibly break.

Third, every guideline should include a clause that says something like, “This document is intended to be used solely as a general recommendation. There are many legitimate reasons for deviations from these official recommendations, including and not limited to the catastrophic interruption of legitimate services in some environments. All readers should test the implementation of any of the included settings before implementing in a production environment and deviate where appropriate.” The best-practice document in your hands may not be the best practice for your environment.

Last, every guideline document should include a well-documented avenue for challenging assertions. There needs to be an easy way to get rid of the bad advice. Each recommendation document should include a paragraph detailing the official process, and it should include e-mail addresses for sending challenges. Just as important, the person on the other end of that contact information must reply within a reasonable amount of time and give the sender an official response, either accepting the challenge for further research or denying the sender’s attestation with a reason for doing so.

Is it too much to ask that our official and mandated security guidelines be technically accurate?