Researchers turn keyboard clicks into text

Researchers at the University of California, Berkeley, have found a way to turn the clicks and clacks of typing on a computer keyboard into a startlingly accurate transcript of what exactly is being typed.

In a paper released last week, the researchers explained how they developed software that could analyze the sound of someone typing on a keyboard for just 10 minutes and then piece together as much as 96 percent of what had been typed.

The technique works because of the simple fact that the sound of someone striking an "a" key is different from the sound of striking the "t," according to Doug Tygar, a professor of computer science at Berkeley. "Think of a Congo drum. If you hit a Congo drum on different parts of the skin, it makes a different tone," he said. "That's an analogy for what's happening here, because there's a plate underneath the keyboard [that is] being struck in different locations."

Once the different tones had been identified, Tygar and his team used techniques from a field of research called statistical learning theory to map them into similar categories and arrive at some early guesses at what the text might be. They then applied a number of spelling and grammar correction tools to this text to refine those guesses. This process ultimately converts the keyboard sounds into readable text.

Statistical learning, also called machine learning, provides a way for computers to make sense out of complex pieces of data. It has been a hot area for computer science research over the last 10 years, forming the basis for products such as spam detectors and speech recognition systems, Tygar said.

Because the Berkeley researcher's technique is based on the sound of the key and not the timing of the keystrokes, both touch and hunt-and-peck typists can have their keystrokes decoded using this technique, he said.

The idea of snooping in on keyboards has been around since the beginning of the Cold War, when Soviet spies bugged IBM Selectric typewriters in the American embassy in Moscow, but the Berkeley researchers are breaking new ground in using these techniques with computer keyboards, said Bruce Schneier, chief technology officer at Counterpane Internet Security and the author of Applied Cryptography.

"In security, the devil is in the details, and these guys did the details," he said.

Some details remain unsolved, however. The researchers did not use certain commonly used keys such as "shift" and "backspace" in their study, and they only looked at text that was typed in English. Still, neither Schneier nor Tygar believe that these details will prevent the techniques from ultimately working in uncontrolled environments.

In fact, Schneier believes it is only a matter of time before criminals begin using similar techniques. "Somebody else will use it," he said. "And if you believe the NSA (National Security Agency) hasn't done this already, you're naive."

Tygar agrees that the techniques described in his paper are relatively easy to use (his team used open source spell checkers and a $10 PC microphone, for example). And for that reason, the Berkeley team has decided not to release the source code they used in their study. "I don't think it' very hard for people to put this together, but I don't want to make it easy for people, either."

So what should computer users make of this new security threat?