Battle looms over securing virtualized systems

VMware's vSphere security works but has risks; Cisco, Check Point,and others have their own virtual security plans

There's growing consensus that traditional approaches to network security -- the firewall and intrusion-prevention appliances, the host-based antivirus software -- simply do not work well in virtualized environments for which they were never designed.

With virtualization becoming the foundation for corporate users and cloud service providers, many security vendors, including Check Point, McAfee, Trend Micro and Symantec, are adapting their products to maximize performance for the main virtualization platforms from VMware, Microsoft, and Citrix. VMware, as the market leader, carries a lot of clout, and the security architecture now proposed by VMware, called vShield, could radically transform how security services will be delivered in the VMware vSphere environment.

[ Doing storage virtualization right is not so simple. InfoWorld's expert contributors show you how to get it right in this "Storage Virtualization Deep Dive" PDF guide. ]

REVIEW: New security tools protect virtual machines

But despite its advantages, some analysts are warning vSphere carries risk.

VMware is ahead of Microsoft and Citrix in putting forward a security architecture aimed at maximizing functionality and performance for its virtual machine (VM) platform, says Gartner analyst Neil MacDonald. But unlike Citrix, which is pursuing something similar with a more open-source approach under the XenAcess initiative, he says VMware's strategy is totally proprietary and carries the risk of vendor lock-in.

"They don't want to make it easy for people to switch," says MacDonald about VMware and vShield. "It prevents the hypervisor from being commoditized. Any vendor wants you to stay with their platform, and VMware has very large market share."

"With the lock-in, you get these specialized functions," says MacDonald. But anyone adopting vShield, whether service provider or enterprise, for VM-based security "should do it with eyes wide open" by weighing the potential benefits and drawbacks, he advises.

In VMware's vShield, security services are delivered to VM-based applications through a specialized "security virtual machine" capable of introspection into VMs via an agentless approach that can be supported by third-party security vendors. However, Dean Coza, VMware's director of product management in security, acknowledges VMware is hand-picking the select group of vendors allowed to use the vShield APIs. And VMware is taking on the firewalling and management role through vShield products that are already out for vSphere.

Coza says physical firewalls don't work well in a VM environment and VMware is supplanting them with VMware's software-based firewalls, including vShield App, the hypervisor-based application-aware firewall for the virtual data center, and the built-in application firewalls called vShield Zones.

This is the foundation for what's known as "logical policy management" so it becomes possible to immediately apply firewall rules whenever a new VM drops in or is moved, says Coza, even with three-tier applications logically separated into virtual-machine containers.

In contrast, he says, a "Cisco PIX has 5,000 rules on it, you have to tread very carefully to transfer those rules; it can take two hours." Coza says one of the main problem is knowing where the workloads actually are in a virtualized environment when they can move around rapidly.

Cisco did not want to discuss how useful physical firewall appliances can be in a virtualized environment or any role Cisco might play in the vShield program, but Rajneesh Chopra, Cisco senior product manager, says Cisco has taken steps to design products specifically for vSphere, including the Virtual Security Gateway for Cisco Nexus 1000V Series Switches, intended to provide trusted access, firewalling, filtering and security policies for VMware VMotion events and more. Chopra says Cisco's goal is to "maintain a consistent policy and definitive enforcement" in both virtualized and non-virtualized environments. He adds Cisco's relationship with VMware from an architecture perspective "runs broad and deep."

With vShield Manager, security policy can be applied immediately, not just for firewalling, but decisions can also be made on how to do antivirus scanning, event logging, intrusion prevention, e-discovery, vulnerability management, file-integrity checking and data-loss prevention.

VMware so far has not figured out exactly how to bring encryption into this architecture but is working on it, Coza acknowledges. One goal with vShield is to adhere to the recent guidelines from the National Institute of Standards and Technology for use of virtualization.

Coza notes that VMware also wants to be able to provide an "application discovery manager" that can "sniff traffic" to discover sensitive data "so you can write business context around these containers" and design automated security procedures customized for data restricted under the Payment Card Industry guidelines, for example.

VMware's vShield Manager can act as middleware to accept instructions from management consoles of third-party security vendors, or conversely, send information to them, once integration with third-party security products is completed. But VMware is not interested in working with potentially hundreds of security vendors and making the vShield APIs available to them. Rather, VMware wants to work with a select group in a more controlled way than it did with its earlier security APIs, VMsafe, which will eventually be phased out; vendors expect that to happen by the end of 2013.

The selected vendors today working with VMware include Sourcefire, HP TippingPoint and Trend Micro, Coza says, with more vShield third-party vendors expected to be showcased at the upcoming VM World Conference this summer. He says vShield has been adopted or is under evaluation with several service providers, including Terremark, Savvis and AT&T.

Ty Smallwood, information services security officer at Medical Center of Central Georgia, the second largest hospital in the state with about 4,600 employees, says his hospital is a big VMware shop. The lock-in argument does not take the upper hand for the hospital, especially as one of the main security vendors it has relied on for a long time, Trend Micro, is supporting vShield through its Deep Security product.

The hospital is moving forward with its desktop virtualization project based on VMware's View as a migration path from stand-alone Windows-based desktop computers. Tests are showing that running antivirus scans and other services in the VMware environment does cause so-called "A/V storms" that can severely impact performance.

So far, the tests with Trend's vShield-based Deep Security, which does A/V scanning, shows the agentless approach eliminates that problem, although there have been some compatibility issues that have had to be resolved. For instance, VMware just issued some new endpoint drivers for vShield, which seem to have impacted how Trend Micro's software behaves, says Smallwood.

Deep Security also provides a virtual-patch capability, which the hospital wants to have to be able to scan and mitigate vulnerability risks without having to reboot, which helps the hospital avoid service disruptions. The vShield approach has "huge appeal," says Smallwood.

But even some vendors eyeing support for vShield have reservations.

Kim Singletary, McAfee's director of solutions marketing for virtualization, says McAfee's approach so far in creating antivirus scanning products specialized for virtualized environments has been "hypervisor-agnostic." McAfee's antivirus scanning software called McAfee Management for Optimized Virtualized Environments (MOVE), for example, is intended for use on either Citrix Xen, ESX, vSphere or Microsoft Hyper-V. While this has seen steady adoption among customers, McAfee is aware that a lot of them are waiting to see how McAfee and its ePolicy Orchestrator management console will integrate with vShield.

While McAfee couldn't provide complete details yet, Tyler Carter, product marketing senior group manager for network security at McAfee, says the security firm is working with VMware and seeking to determine how this integration could be done.

VMware's vShield elicits some scorn from arch-rival Microsoft.

"VMware is taking some technologies and trying to wrap it around virtualization," says Jeff Woolsey, principal program manager, lead for Windows Server virtualization at Microsoft. "It makes sense if you're a virtualization vendor. Virtualization is hot." But he adds Microsoft's view is broader because it's looking at security and management that will support both virtualized and non-virtualized environments -- at least those of Microsoft.

He says Microsoft doesn't have the kind of over-arching security framework like vShield, nor are there plans at present to introduce one, but it has already made technologies for integrating with Hyper-V, such as its virtual hard-disk format, openly available to vendors without royalties. He said it's being used by firms like McAfee that want to be hypervisor-agnostic. "Customers don't want to buy a technology that only works with one virtualization vendor," Woolsey says. He says Microsoft is offering free tools and configuration advice for Hyper-V.

He says the VMsafe APIs, known to become phased out, have been a failure with few vendors using them, and he predicts vShield may well suffer the same fate. Microsoft also argues against VMware's idea that the agentless approach is the best way to go for security in virtualization.

"Their whole point is you don't need agents anymore. That's a pipe dream," Woolsey says. Without the agent, you sacrifice getting a lot of information from the VM guest, he says.

Not surprisingly, VMware's push to be the software-based firewall for vSphere is disconcerting to traditional firewall vendors that have spent years building up product expertise, such as deep-packet inspection.

Check Point Software Technologies last fall introduced Security Gateway Virtual Edition, which includes a firewall, VPN, and intrusion prevention for use with VMware's ESX, ESXi and vSphere. But it's based on the older VMsafe APIs, not vShield. VMware's vShield firewalling concepts provide a good foundation but "there's a certain specialization in what we do," says Oded Gonda, vice president of network security at Check Point, adding customers are invested in equipment they'd like to extend into virtualized environments.

Read more about wide area network in Network World's Wide Area Network section.

This story, "Battle looms over securing virtualized systems" was originally published by Network World.

Copyright © 2011 IDG Communications, Inc.