Even though VMware participated in the creation of the new SPEC virtualization benchmark, that didn't stop the virtualization giant from also working on a new version of its own proprietary benchmark offering. VMmark 2.0, currently in beta, is being called a next-generation multihost virtualization benchmark that models not only application performance in a virtualized environment but also the effects of common virtual infrastructure operations.
So while SPECvirt_sc2010 and VMmark 1.x are both good at measuring the performance of a single virtualization host, VMmark 2.0 is being designed to benchmark the entire virtual data center. This implies measuring complex operations like manual (or user-initiated) vMotion, automated (or DRS-initiated) vMotion, Storage vMotion, as well as virtual machine cloning and deployment.
In moving from a single- to a multihost virtualization benchmark, VMware is constructing VMmark 2.0 with the following properties in mind:
- Multihost to model realistic data center deployments
- Virtualization infrastructure workloads to more accurately capture overall platform performance
- Heavier workloads than VMmark 1.x to reflect heavier customer usage patterns enabled by the increased capabilities of the virtualization and hardware layers
- Multitier workloads driving both virtual machine-to-virtual machine and external network traffic
- Workload burstiness to ensure robust performance under variable high loads
Organizations are at different places in their virtualization maturity, so benchmarking a single server still has a place in today's virtualized data center. But VMware's idea of looking at the data center as a whole definitely has its place as well. More mature virtualization shops will find this new benchmark inviting, as will infrastructure-as-a-service cloud hosting providers.
So in 2010, VMmark and SPEC virtualization benchmarks both have their place. But for the cost conscious, there is one other difference between these two virtualization benchmarking solutions: VMware VMmark is free, while the SPECvirt_sc2010 will cost you $3,000, though discounts are available for qualified nonprofit and educational institutions.
This story, "VMware introduces VMmark 2.0 beta," was originally published at InfoWorld.com. Follow the latest developments in virtualization and cloud computing at InfoWorld.com.