Ultimate cloud speed tests: Amazon vs. Google vs. Windows Azure
A diverse set of real-world Java benchmarks shows Google is fastest, Azure is slowest, and Amazon is priciest
By Peter Wayner
Contributing writer, InfoWorld |
Some of the benchmarks in the collection will be very familiar to server users. The Tomcat test, for instance, starts up the popular Web server and asks it to assemble some Web pages. The Luindex and Lusearch tests will put Lucene, the common indexing and search tool, through its paces. Another test, Avrora, will simulate some microcontrollers. Although this task may be useful only for chip designers, it still tests the raw CPU capacity of the machine.
I ran the 14 DaCapo tests on three different Linux machine configurations on each cloud, using the default JVM. The instances aren't perfect "apples to apples" matches, but they are roughly comparable in terms of size and price. The configurations and cost per hour are broken out in the table below.
Cloud machines under test
| Virtual CPUs or cores | RAM | Cost per hour | |
|---|---|---|---|
| Amazon m1.medium | 1 | 3.75GB | 12 cents |
| Amazon c3.large | 2 | 3.75GB | 15 cents |
| Amazon m3.2xlarge | 8 | 30.00GB | 90 cents |
| Google n1-standard1 | 1 | 3.75GB | 10.4 cents |
| Google n1-highcpu-2 | 2 | 1.80GB | 13.1 cents |
| Google n1-standard-8 | 8 | 30.00GB | 82.9 cents |
| Windows Azure Small VM | 1 | 1.75GB | 6 cents |
| Windows Azure Medium VM | 2 | 3.50GB | 12 cents |
| Windows Azure Extra Large VM | 8 | 14.00GB | 48 cents |
I gathered two sets of numbers for each machine. The first set shows the amount of time the instance took to run the benchmark from a dead stop. It fired up the JVM, loaded the code, and started to work. This isn't a bad simulation because many servers start up Java code from command lines in scripts.
To add another dimension, the second set reports the times using the "converge" option. This runs the benchmark repeatedly until consistent results appear. This sometimes happens after just a few runs, but in a few cases, the results failed to converge after 20 iterations. This option often resulted in dramatically faster times, but sometimes it only produced marginally faster times.