That's where TPC's price/performance measurement becomes important. It essentially tells you how much performance you get at a given workload per dollar spent buying the system. The intent is to prevent hardware vendors from gaming the system by simply reporting astronomical performance figures. Sure, a fully loaded server from Vendor X might yield three times better performance on a particular workload compared to a bare-bones model from Vendor Y -- but Vendor Y's server comes in at a quarter of the price. Factoring in those costs gives a data center operator more information to work with.
Notably, the TPC benchmarks also take into account the type of server OS and database being used, which is highly relevant. Some workloads perform better on certain platforms than on others.
MPG for the datacenter
That brings us to the newly introduced TPC-Energy spec. TPC-Energy adds yet another important piece of information to the equation: which system delivers the highest level of performance, measuring in transactions per second, using the least amount of energy.
TPC-Energy and other TPC specs aren't without their drawbacks. To their credit, they're highly detailed, which means you can get price and performance information about specific systems running a particular OS and a particular database, which is great if vendors report results for setups that match what you need for your own data center. That's correct: TPC test results are reported by vendors, as setting up these types of tests in-house, comparing the endless combinations of hardware and software, is prohibitive for all but the largest organizations.
Thus, vendors crunch the numbers themselves, adhering to the TPC benchmark standards. A vendor can then decide whether to submit their results to the TPC, which are audited and posted for public viewing at tpc.org. You're at the mercy of vendors to report numbers applicable to your data center's situation. If you see that IBM reports data on a system that matches your needs but, say, HP doesn't, does it mean that HP hasn't conducted the test -- or that HP has conducted it but its numbers fell short of IBM's?