Last Wednesday, researchers at Matousec.com outlined how attackers could exploit the kernel driver hooks that most security software use to reroute Windows system calls through their software to check for potential malicious code before it's able to execute.
[ Find out how to block the viruses, worms, and other malware that threaten your business, with hands-on advice from InfoWorld's expert contributors in InfoWorld's "Malware Deep Dive" PDF guide. ]
Calling the technique an "argument-switch attack," a Matousec-written paper spelled out in relatively specific terms how an attacker could swap out benign code for malicious code between the moments when the security software issues a green light and the code actually executes.
"This is definitely very serious," said Alfred Huger, vice president of engineering at Immunet, a Palo Alto, Calif.-based anti-virus company. "Probably any security product running on Windows XP can be exploited this way." Huger added that Immunet's desktop client is not vulnerable to the argument-switch attacks because the company's software uses a different method to hook into the Windows kernel.
According to Matousec, nearly three-dozen Windows desktop security titles, including ones from Symantec, McAfee, Trend Micro, BitDefender, Sophos and others, can be exploited using the argument-switch tactic. Matousec said it had tested the technique on Windows XP SP3 and Vista SP1 on 32-bit machines.
Some security vendors agreed with Huger. "It's a serious issue and Matousec's technical findings are correct," said Mikko Hypponen, chief research officer at Finnish firm F-Secure, in an email.
"Matousec's research is absolutely important and significant in the short term," echoed Rik Ferguson, a senior security advisor at Trend Micro, in a blog post earlier Monday.
Other anti-virus companies downplayed the threat, however. "Based on our initial review of the public documentation, we believe this is a complicated attack with several mitigating factors that make it unlikely to be a viable, real world, widespread attack scenario," a McAfee spokesman said in an email reply to a request for comment. "The attack would require some level of existing access to the target computer, as the attack described by Matousec does not on its own bypass security software or allow malware to run."
Kaspersky Lab had a similar reaction. "[We] have analyzed the published material and concluded that the issue is only linked to certain features of [our] products," Kaspersky said in an emailed statement. "Kaspersky Lab products implement not only [kernel] hooks, but a wide range of technologies, including secure sandboxing and other methods of restricting suspicious kernel mode activity."
Huger confirmed that attackers would have to drop malware of some sort on the targeted machine in order to utilize the argument-switch strategy, and that there are "lots of easier ways to game anti-virus" than Matousec's technique.
"But that doesn't lesson the impact," Huger argued. "Actually, it would be really tricky to stop this, and gives attackers a strong opportunity to get around disk-based security."
Huger's greatest fear is that others take Matousec's findings, weaponize the argument-switch attack, and add it to one of the numerous underground exploit kits. "If someone packages this into an easy-to-use library, I think it'll be in play pretty quickly, with widespread adoption," said Huger. "Why wouldn't it?"
Several researchers with anti-virus companies, including Huger, noted that security software isn't defenseless against attempts to use argument-switch, in large part because attackers would still need to plant malware on a machine, and on-demand scanning would theoretically block any malicious downloads, at least of known threats.
"Any malware that we detect by our anti-virus will still be blocked, just like it always was," said F-Secure's Hypponen. "So the issue only affects new, unknown malware that we do not yet have a detection signature for."
Huger expects that attacks using argument-switch will target 32-bit Windows XP machines, both because that operating system continues to dominate the Windows ecosystem, and because it lacks the PatchGuard kernel protection that Microsoft added to 64-bit versions of XP in 2005, then later to 64-bit editions of Vista and Windows 7 .
"They may not be exclusive to Windows XP, but they'll be more prevalent under XP," Huger said.
Microsoft faced resistance from several anti-virus companies, notably Symantec and McAfee, before the release of Windows Vista. They complained that PatchGuard would prevent them from delivering key functions in their Vista-compatible products, including behavior-based virus detection, host-based intrusion prevention and software tamper protection. Microsoft relented and eventually made security application programming interfaces (API) available to allow vendors to do what they needed without accessing the kernel.
Those APIs first appeared in Windows Vista SP1 in 2008.
Matousec claimed that 64-bit versions of Windows boasting PatchGuard could be vulnerable in some instances. "[This] will work against all user mode hooks and it will also work against the kernel mode hooks if they are installed, for example, after disabling PatchGuard," Matousec's paper stated.
Microsoft did not immediately reply to a request for comment on Matousec's claim.
Other problems security vendors face in blocking argument-switch attacks could arise if or when they release updates, argued Huger. "Kernel driver programming is pretty tricky," he said. "Redeployment [of updates] will complicate things. Any vendor nervy enough to put out new kernel drivers will have to do a pretty significant gut check. If something goes wrong, millions of machines could be blue-screened."
Huger pointed to the recent fiasco with a faulty McAfee signature update that crashed thousands of PCs running the company's security software as an example. "Enterprises would be very reticent to update because of the risk," he said.
Gregg Keizer covers Microsoft, security issues, Apple, Web browsers and general technology breaking news for Computerworld . Follow Gregg on Twitter at @gkeizer or subscribe to Gregg's RSS feed . His email address is email@example.com .
Read more about security hardware and software in Computerworld's Security Hardware and Software Knowledge Center.
This story, "New attack tactic sidesteps Windows security software" was originally published by Computerworld .