In other cases, the Micro just melted down. On the Eclipse test, the Micro was occasionally about five times slower than the Medium, but it was often eight to 10 times slower. Once on the Eclipse test and several times on other tests, the Micro failed completely. The lack of RAM left an unstable machine. (Note that these several failures don't include the dozens of failures of the lemon, which crashed much more consistently than the other Micro instances I tested.)
These experiments, while still basic, show that packaging performance in the cloud is much trickier than with stand-alone machines. Not every machine will behave the same way. The effect of other users on the hardware and the network can and will distort any measurement.
To make matters worse, cloud companies are in a strange predicament. If they have spare cycles sitting around, it would be a shame to waste them. Yet if they give them out, the customers might get accustomed to better performance. No one notices when the machines run faster, but everyone starts getting angry if the machines slow down.
Such expectations make it harder for the companies to roll out newer hardware. If new chips that are 10 percent faster appear, those racks will run 10 percent faster. Everyone will be happy until they get stuck with an instance on the older racks.
Cloud users need to adjust their expectations or at least relax the error bars around the expectations. The cloud doesn't deliver performance with the same precision as the dedicated hardware sitting on your desk. It's not necessarily a good idea to demand it either, because the cloud company's only solution to a demand for precision is to eliminate any kind of bursting altogether. They would need to limit performance to the lowest common denominator.
In other words, the '70s may not have been the best years for Detroit's consistency, but they still turned out some great cars. The Camaros, Mustangs, Trans-Ams, and Corvettes often ran quite well. The assembly lines weren't perfect, but they were good most of the time -- until they produced a lemon. Like owners of those old muscle cars, drivers of cloud machines should keep a close watch on performance.
This article, "Benchmarking Amazon EC2: The wacky world of cloud performance," was originally published at InfoWorld.com. Follow the latest developments in cloud computing at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.
Having trouble installing and setting up Win10? You aren’t alone. Here are many of the most common...
It's all about knowing how to build an open source community -- plus experience running applications in...
Win7 Update scans got you fuming? Here’s how to make the most of Microsoft’s 'magic' speed-up patch
Sponsored by Hewlett Packard Enterprise
Your iPad can largely function like a laptop with two of the three main office productivity suites from...
Using SHA-1 for digital certificates and cryptographic keys hasn't been safe for years. With the...
The internet has your number—among many other deets. Prevent identity theft and doxxing by erasing...
Kubernetes allows you to deploy cloud-native applications anywhere and manage them exactly as you like...