In other cases, the Micro just melted down. On the Eclipse test, the Micro was occasionally about five times slower than the Medium, but it was often eight to 10 times slower. Once on the Eclipse test and several times on other tests, the Micro failed completely. The lack of RAM left an unstable machine. (Note that these several failures don't include the dozens of failures of the lemon, which crashed much more consistently than the other Micro instances I tested.)
These experiments, while still basic, show that packaging performance in the cloud is much trickier than with stand-alone machines. Not every machine will behave the same way. The effect of other users on the hardware and the network can and will distort any measurement.
To make matters worse, cloud companies are in a strange predicament. If they have spare cycles sitting around, it would be a shame to waste them. Yet if they give them out, the customers might get accustomed to better performance. No one notices when the machines run faster, but everyone starts getting angry if the machines slow down.
Such expectations make it harder for the companies to roll out newer hardware. If new chips that are 10 percent faster appear, those racks will run 10 percent faster. Everyone will be happy until they get stuck with an instance on the older racks.
Cloud users need to adjust their expectations or at least relax the error bars around the expectations. The cloud doesn't deliver performance with the same precision as the dedicated hardware sitting on your desk. It's not necessarily a good idea to demand it either, because the cloud company's only solution to a demand for precision is to eliminate any kind of bursting altogether. They would need to limit performance to the lowest common denominator.
In other words, the '70s may not have been the best years for Detroit's consistency, but they still turned out some great cars. The Camaros, Mustangs, Trans-Ams, and Corvettes often ran quite well. The assembly lines weren't perfect, but they were good most of the time -- until they produced a lemon. Like owners of those old muscle cars, drivers of cloud machines should keep a close watch on performance.
This article, "Benchmarking Amazon EC2: The wacky world of cloud performance," was originally published at InfoWorld.com. Follow the latest developments in cloud computing at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.
You may still be better off sticking with Win7 or Win8.1, given the wide range of ongoing Win10...
Now that we're down to the wire, many upgraders report that the installer hangs. If this happens to...
Based on a technique created by a German blogger, here's how to stop wasting hours checking for Windows...
PowerShell is a valuable tool for automating Windows admin tasks, including laborious security chores
Are your assets bankable in 2017? Hiring managers say they'll seek out these skills most in the New...
Don’t delay, and don’t let empire-builders stand in the way of adopting devops
The free service for document-sharing with live code can be used for building machine learning models