Trend Micro takes a different approach. Using experience gained in its security labs, the company developed a complete library -- the RCM (Rootkit Common Module) -- to replace the Windows APIs, says Geoff Grindrod, solution product manager at Trend Micro. According to Grindrod, the library includes double encryption to avoid spoofing, and its proxy for API calls is constructed as a special kernel module. With the RCM, the system sees hidden processes, hidden registry keys, and hidden files. As the RCM has matured, it has been integrated into more and more Trend products and is now a core component of anti-spyware and other Trend Micro products, Grindrod says.
Discovering rootkits, however, is only half the battle, as excising them can result in its own set of problems.
"Rootkits are so imbedded in the operating system," Mandiant's Butler says. "Plus, we're seeing firmware attacks and survivable rootkits installing themselves in the BIOS. Removing rootkits can also make the system unstable while it's running."
Admins should be aware of the implications of rootkit removal before lunging headlong into the endeavor, says Ron O'Brien, senior security analyst at Sophos, one of the first security vendors to offer a rootkit removal tool.
"Rootkits are not 'bad,' but they have developed a reputation for being bad," O'Brien says. "They are really just a form of hidden files" that may have legitimate uses. Ripping rootkits out before establishing their purpose can prove detrimental to overall system health, he adds.
Coping with an evolving threat
Despite advances in prevention and removal, Steve Manzuik, senior manager of security engineering and research at Juniper, sees no end in sight to the rootkit threat. In fact, Manzuik believes that rootkit.com, Joanna Rutkowska's work on the Windows kernel, and Microsoft's resource protections for 64-bit Windows Vista are "making it more difficult for both attackers and vendors."
Manzuik sees that current approaches to rootkit discovery and removal are beginning to fail despite improvements in Windows security. Factor in the lag time before Vista protections are widely deployed, and you have a perfect breeding ground for rootkit innovation. For example, Manzuik points out that some rootkits can now bypass the security sandbox. They detect they are in the sandbox and lay low, effectively tricking the system into thinking they are legitimate apps.
MANDIANT's Butler, however, believes that Vista protections will have an impact. Not only will the protections make it more difficult for rootkit authors to break in, Butler says, but it will also require "another separate effort to conceal themselves and maintain their presence."
Manzuik and Butler do, however, agree on the importance of strict user access policies. Both view rootkits as further evidence against giving users admin-level access to systems -- especially at smaller organizations, where the practice is often promoted as a cost-cutting necessity.
"The culture in smaller companies is that they will only call the IT guys if they can't figure it out themselves, which leads to most users having admin rights on machines," Manzuik says. Any organization employing this policy -- regardless of its size -- will be compromised, Manzuik says.
Because of this, Manzuik believes policy should figure foremost as a means for protecting systems against rootkits: "Without buying special technology, [most organizations] can deal with the majority of the threats with proper security policy and management."