Emerging IEEE Ethernet standards could ease data center headaches

Management of increasingly virtualized data centers is addressed in IEEE draft standards

Cisco, HP, and others are waging an epic battle to gain control of the data center, but at the same time they are joining forces to push through new Ethernet standards that could greatly ease management of those increasingly virtualized IT nerve centers.

The IEEE 802.1Qbg and 802.1Qbh specifications are designed to address serious management issues raised by the explosion of virtual machines in data centers that traditionally have been the purview of physical servers and switches. In a nutshell, the emerging standards would offload significant amounts of policy, security, and management processing from virtual switches on network interface cards (NIC) and blade servers and put it back onto physical Ethernet switches connecting storage and compute resources.

[ Keep up with the latest in virtualization with David Marshall's Virtualization Report, and read InfoWorld's Virtualization newsletter. |

The IEEE draft standards boast a feature called Virtual Ethernet Port Aggregation (VEPA), an extension to physical and virtual switching designed to eliminate the large number of switching elements that need to be managed in a data center. Adoption of the specs would make management easier for server and network administrators by requiring fewer elements to manage, and fewer instances of element characteristics -- such as switch address tables, security and service attribute policies, and configurations -- to manage.


"There needed to be a way to communicate between the hypervisor and the network," says Jon Oltsik, an analyst at Enterprise Systems Group. "When you start thinking about the complexities associated with running dozens of VMs on a physical server the sophistication of data center switching has to be there."

But adding this intelligence to the hypervisor or host would add a significant amount of network processing overhead to the server, Oltsik says. It would also duplicate the task of managing media access control address tables, aligning policies and filters to ports and/or VMs and so forth.

"If switches already have all this intelligence in them, why would we want to do this in a different place?" Oltsik notes.

10G Ethernet shakes net design to core

VEPA does its part by allowing a physical end station to collaborate with an external switch to provide bridging support between multiple virtual end stations and VMs, and external networks. This would alleviate the need for virtual switches on blade servers to store and process every feature -- such as security, policy and access control lists (ACLs) -- resident on the external data center switch.

Diving into IEEE draft standard details
Together, the 802.1Qbg and bh specifications are designed to extend the capabilities of switches and end station NICs in a virtual data center, especially with the proliferation and movement of VMs. Citing data from Gartner, officials involved in the IEEE's work on bg and bh say 50 percent of all data center workloads will be virtualized by 2012.

Some of the other vendors involved in the bg and bh work include 3Com, Blade Network Technologies, Brocade, Dell, Extreme Networks, IBM, Intel, Juniper Networks and QLogic. While not the first IEEE specifications to address virtual data centers, bg and bh are amendments to the IEEE 802.1Q specification for virtual LANs and are under the purview of the organization's 802.1 Data Center Bridging and Interworking task groups.

The bg and bh standards are expected to be ratified around mid-2011, according to those involved in the IEEE effort, but pre-standard products could emerge late this year. Specifically, bg addresses edge virtual bridging: an environment where a physical end station contains multiple virtual end stations participating in a bridged LAN. VEPA allows an external bridge -- or switch -- to perform inter-VM hairpin forwarding of frames, something standard 802.1Q bridges or switches are not designed to do.

"On a bridge, if the port it needs to send a frame on is the same it came in on, normally a switch will drop that packet," says Paul Congdon, CTO at HP ProCurve, vice chair of the IEEE 802.1 group and a VEPA author. "But VEPA enables a hairpin mode to allow the frame to be forwarded out the port it came in on. It allows it to turn around and go back."

VEPA does not modify the Ethernet frame format but only the forwarding behavior of switches, Congdon says. But VEPA by itself was limited in its capabilties. So HP combined its VEPA proposal with a Cisco's VN-Tag proposal for server/switch forwarding, management and administration to support the ability to run multiple virtual switches and multiple VEPAs simultaneously on the endpoint.

This required a channeling scheme for bg, which is based on the VN-Tag specification created by Cisco and VMware to have a policy follow a VM as it moves. This multichannel capability attaches a tag to the frame that identifies which VM the frame came in on.

But another extension was required to allow users to deploy remote switches -- instead of those adjacent to the server rack -- as the policy controlling switches for the virtual environment. This is where 802.1Qbh comes in: it allows edge virtual bridges to replicate frames over multiple virtual channels to a group of remote ports. This will enable users to cascade ports for flexible network design, and make more efficient use of bandwidth for multicast, broadcast and unicast frames.

The port extension capability of bh lets administrators choose the switch they want to delegate policies, ACLs, filters, QoS and other parameters to VMs. Port extenders will reside in the back of a blade rack or on individual blades and act as a line card of the controlling switch, says Joe Pelissier, technical lead at Cisco.

"It greatly reduces the number of things you have to manage and simplifies management because the controlling switch is doing all of the work," Pelissier says.

What's still missing from bg and bh is a discovery protocol for autoconfiguration, Pelissier says. Some in the 802.1 group are leaning towards using the existing Logical Link Discovery Protocol (LLDP), while others, including Cisco and HP, are inclined to define a new protocol for the task.

"LLDP is limited in amount of data it can carry and how quickly it can carry that data," Pelissier says. "We need something that carries data in the range of 10s to 100s of kilobytes and be able to send the data faster rather than one 1,500 byte frame a second. LLDP doesn't have fragmentation capability either. We want to have the capability to split the data among multiple frames."

Cisco, HP say they're in synch
Cisco and HP are leading proponents of the IEEE effort despite the fact that Cisco is charging hard into HP's traditional server territory while HP is ramping up its networking efforts in an attempt to gain control of data centers that have been turned on their heads by virtualization technology.

Cisco and HP say their VEPA and VN-Tag/multichannel and port extension proposals are complementary despite reports that they are competing techniques to accomplish the same thing: reducing the number of managed data center elements and defining a clear line of demarcation between NIC, server and switch administrators when monitoring VM communications.

"This isn't the battle it's been made out to be," Pelissier says.

Though Congdon acknowledges he initially proposed VEPA as an alternative to Cisco's VN-Tag technique, the two together present "a nice layered architecture that builds upon one another where virtual switches and VEPA form the lowest layer of implementation, and you can move all the way to more complex solutions such as Cisco's VN-Tag."

And the proposals seem to have broad industry support.

"We do believe this is the right way to go," says Dhritiman Dasgupta, senior manager of data center marketing at Juniper. "This is putting networking where it belongs, which is on networking devices. The network needs to know what's going on."

This story, "Emerging IEEE Ethernet standards could ease data center headaches" was originally published by Network World.

Copyright © 2010 IDG Communications, Inc.

InfoWorld Technology of the Year Awards 2023. Now open for entries!