Windows encryption programs open to kernel hack

Many popular Windows encryption programs that hide files inside mounted volumes could be fatally compromised by a new type of attack uncovered by a German researcher.

According to a paper published by Bern Roellgen, who also works for encryption software outfit PMC Ciphers, such OTFE (on-the-fly-encryption) programs typically pass the password and file path information in the clear to a device driver through a Windows programming function called DevicelOControl.

[ Learn how to secure your systems with Roger Grimes' Security Adviser blog and newsletter, both from InfoWorld. ]

Although it is impossible for a malicious program to get hold of this data directly -- a competently written encryption program will overwrite memory locations caching this data -- it could be retrieved if the attacker has found a way to compromise the Windows kernel itself.

Dubbed, the Mount IOCTL (input/output control) Attack by Roellgen, an attacker would need to substitute a modified version of the DevicelOControl function that is part of the kernel with one able to log I/O control codes in order to find the one used by an encryption driver. Once found, the plaintext passphrase used to encrypt and decrypt a mounted volume would be vulnerable.

As simple as it sounds, how easy would such an attack be in real-world conditions? The key elements are the ability in the first instance to burrow into the Windows kernel without being detected in the manner of a super-rootkit, and then find the probably unique control code used by the encryption program, neither of which would be easy, but are at least theoretically possible.

"As this kind of attack has so far been unknown, it is very likely that all disk encryption products which mount virtual volumes are affected," said Roellgen by e-mail.

"Instead of patenting the countermeasure it is probably better to spread the news as good as possible and to give other programmers the chance to strengthen their software. To be honest, I would not trust any disk encryption software that hands out keying material so easily to the OS anymore."

Roellgen's solution to the issue is to use a Diffie-Helman key exchange setup between the driver and the encryption application.

Roellgen has a record for finding vulnerabilities in encryption technologies. Last October, he published details of way to "see" image files inside encrypted backup files, while earlier in the year his company, PMC Ciphers, invented a novel method for defeating keyloggers with a high degree of certainty.

Techworld  is an InfoWorld affiliate.