I've been involved with multiple projects with file encryption lately, and even though I've been assisting with data encryption projects for years, I'm still learning something new every day. They say if you don't learn something new each day, then the day is wasted. Me, I'd settle for not looking like a goon in front of the client.

[ RogerGrimes's column is now a blog! Get the latest IT security news from the Security Adviser blog. ]

If you're embarking on a file encryption project, here are some ideas, questions, and caveats to be aware of.

Is file encryption right for you?
Many clients I deal with end up complaining about some of the issues inherent with file encryption. Issues such as plaintext remnants of encrypted files, files saved outside the expected encryption zones, and files that were locked during encryption and therefore left unencrypted. If these things bother you, consider volume or full disk encryption instead. The trend is heavily favoring these methods of data protection over file-by-file encryption.

Key archiving, key archiving, key archiving
You can define the long-term success of your project by the success of your data recovery events. People will forget passphrases and lose keys. They will corrupt encrypted volumes. Your CEO will lose access to her data when traveling. Plan ahead and make sure your data recovery methodology is flawless and accessible. It all begins by ensuring that crypto keys are automatically archived, by default, every time. Don't leave it up to end-users or allow gaps to creep into the process. And test, test, test before beginning deployment. It takes only one high-visibility data loss to mark your data encryption project as a failure.

Where is your data?
Survey where your data is stored and where it is transmitted. How can you begin a data protection plan if you don't know where your data is? Not only is it on hard drives (servers, workstations, and laptops), but USB keys, CD-ROMs, tape backups, and so on. Decide where you need to encrypt and pick the appropriate solutions.

What about data in transit?
Most data encryption programs protect information at rest. How do you protect the data crossing your network or WAN? Is it encrypted? Most of the projects I've been involved with have addressed data at rest scenarios (hard drives, USB keys, and so on), while completely neglecting transmitted data. That's OK -- you have to start somewhere, but don't overlook the second issue. In most cases, you'll need additional solutions to protect data in transit, although you can often rely on the old standby standards of SSL/TSL and IPSec.

Are your apps compatible?
This may surprise some readers, but are you sure your applications don't mind the encryption? Most encryption implementations are seamless and work in the background, but they all require specialized disk drivers and API calls. Many legacy programs may not use the expected disk calls, bypassing the encryption routines and corrupting data. After you've encrypted your data, you should thoroughly test all programs for interaction problems. And, oh yeah, make sure your data backup programs are compatible.

What can you encrypt?
Unless you are using volume or full disk encryption, it is unlikely that you can encrypt all files. Decide what you want to encrypt, and then test. You'll often be surprised at what can't be encrypted because of normal OS operations or application problems.

Double-checking the encryption
In every project I've seen, there's always something you told the program to encrypt that it didn't. Trust, but verify. And after you encrypt something, use a disk sector editor and look for accidental plaintext remnants of files that the encryption solution may have left behind. I like to place very unique text, such as "frogsturnintotadpoles," in a text file, encrypt it, and then search for the string. You'll be surprised how often plaintext remnants are left in the open. Address accordingly.

Performance hit?
All encryption incurs a performance hit. The questions are, How much? And how much is acceptable? In one project, the encryption process went great, but the decryption process choked on very large files, such as e-mail archives. Before you choose a solution, test and make sure the encryption/decryption process doesn't negatively impact performance. A 2 to 5 percent performance hit isn't unusual.

Employees will have to stay out of programs while encryption is happening. Open files cannot normally be encrypted. Consider rebooting the computer just before the encryption process is run for the first time to ensure that all files are closed. After the reboot, think about encrypting the largest files first (such as Outlook .ost and .pst files) and allowing your end-users into a limited set of applications to minimize operational interruptions.

If possible, test the encryption process at below-normal, normal, and high priorities. In Windows, you can change a program's CPU priority through Task Manager or by using the Start.exe program at the command line. Use below-normal priority if you want end-users to be able to access their computers and programs while the encryption process is taking place. Of course, be mindful that this will result in open files being locked and not encrypted. On the other end of the spectrum, you can tell the encryption process to be as fast as possible and keep users completely out until the entire process is finished, ensuring maximum speed and fewer locked files.

Do you have the available disk space?
Many encryption programs require a huge amount of disk overhead during the encrypting/decrypting process, unless the process is all done in memory. For example, encrypting a 100MB file might consume as much as 200MB of available disk space. You normally don't need double the available disk space for all the files collectively, but double the space of the largest single file/object being encrypted. After the object is encrypted or decrypted, the temp files are removed and the disk space is freed up.

Create a data protection policy
Write your employee policy first, before starting out with any encryption project, and get management's approval. That goes without saying, right? The key is defining end-user expectations and penalties for purposefully avoiding company-mandated encryption policies. You need management's backing on this, or you will go to a ton of trouble and expense only to have users undermine your efforts on a daily basis.

Ongoing auditing and verification
Encryption and employees aren't perfect. Develop an ongoing process for verifying that expected files are encrypted.

I could go on, but my hands are getting tired. Send me your additional data encryption hints and recommendations, and I'll put them on the blog for others to learn from.