Defeating hackers and malware with disorder

Security company Vigilant leverages the laws of probability to detect maliciousness in data and on machines

Entropy -- the measure of disorder or randomness -- isn't always desirable in the world of IT security. Kinda, sorta patching your IT systems sometimes, for example, would be a bad thing. At times, though, entropy can be a powerful tool, as in the case of well-chosen passwords that are difficult to crack. A fast-growing SIEM (security incident event managment) company called Vigilant is using entropy in an innovative way that warrants a closer look: Its anomaly-detection service identifies malicious threats based on entropy.

First, a quick primer: Entropy, often measured in bits, is the technical measurement of the randomness of the next piece of data in a string. If you see a sequence of letters with a clear pattern, such as "ABABABABA," you would logically predict the next letter in the series will be B. Because the answer is fairly certain, the entropy would be 0. If you're flipping a coin, the predicted outcome, heads or tails, is considered to be 1 bit of entropy. If a native English speaker is shown a sequence of standard English text and is asked to predict the next letter, he or she could guess it with great accuracy. English text is considered to have an entropy of 0.6 to 1.5 bits.

[ InfoWorld's Malware Deep Dive special report tells you how to identify and stop online attacks. Download it today! | Roger A. Grimes offers a guided tour of the latest threats in InfoWorld's Shop Talk video, "Fighting today's malware." | Stay up to date on the latest security developments with InfoWorld's Security Central newsletter. ]

One more example: A truly random 8-character password, using every possible character on a standard keyboard, could have 52 bits of entropy. Unfortunately, most people use common words as part of their passwords, so most have only 18 bits of entropy. Thus, a password cracker doesn't need to work through every permutation, just the most likely ones in the range of 218, which is far easier than 252. (Read Appendix A and Table A.1 of NIST Special Publication 800-63 for all the nerdy calculations and details.)

In the realm of IT security, antispam services have long used entropy and its link to anomaly detection to sniff out unwanted messages. If the service detects a single email account sending out messages to thousands of seemingly random and unconnected addresses, the provider will usually examine the message for other indicators that it's spam. Entropy and its link in anomaly detection isn't new, but Vigilant has extended it further to fighting malware.

1 2 Page 1
From CIO: 8 Free Online Courses to Grow Your Tech Skills