Racing JavaScript: Safari vs. Firefox

Apple's Safari 4 beta trumps Firefox 3 in JavaScript performance, according to WebKit's own SunSpider benchmarks

Apple's "fastest browser" boast (see my Safari 4 beta preview) is proved by SunSpider JavaScript Benchmark tests against Firefox 3.0.5, both running on OS X Leopard 10.5.6. The SunSpider JavaScript Benchmark suite lives at http://www2.webkit.org/perf/sunspider-0.9/sunspider.html, where you can test any browser you like. Note that the benchmark resides on WebKit's site.

You can see the results from my SunSpider run by using a couple of really ugly URLs. The first one (Firefox results) brings up the results page with Firefox 3 results without running the benchmark. To add the column of results from Safari 4 beta, you can cut and paste the second URL (Safari 4 results) into the field labeled "To compare to another run..." and press Enter while inside that field. The hyperlink "Run Again" at the top of the results page will run the SunSpider benchmarks in the browser you're running at the time.

The test platform is a 2.53GHz MacBook Pro with 4GB of RAM running Firefox 3.0.5 and Safari 4 beta, both Universal binaries in x86 mode.

The distilled SunSpider results number is handy, but like most benchmarks, it does a poor job of telling the real story. Individual test results range from 1.17x to 16.4x. Users' mileage will vary according to the application, but I'll offer some clues to the more meaningful results.

Incidentally, margin of error indicates that the results of multiple runs didn't average cleanly. This may indicate that certain benchmarks ran too fast for JavaScript's timing functions to measure accurately, or it might reflect a background task kicking in during the test (an unavoidable situation that I tried to mitigate by closing all other apps before testing).

In my opinion, the most AJAX-relevant broad categories are access, controlflow, date, regexp (regular expression), and string. Below that level of detail requires a bit of thought, but it may be worth the effort. Understand that the specificity of the benchmarks may reflect uncommon use cases. For example, not many Web apps perform raytracing in script. However, hand-coded object motion brings elements common to 3D math into play. Now, modern JavaScript lets you code motion as "move from point A to point B." However, drawing on a canvas can be highly dependent on pixel and vector manipulation that calls complex math into play.

The seemingly esoteric math and crypto tests aren't directly relevant when you're doing an AJAX carousel, but remember that HTML 5's local database support means that you'll be massaging retrieved table data, and you may want to store local table data in encrypted or obfuscated form. These are good tests of string translation.

Recursion, a function calling itself, is the sort of thing you might use to hand-parse XML or DOM or split text into tokens, cases that are better covered by JavaScript built-ins. Recursion really pounds on the stack, something that was a focus of WebKit Squirrelfish/Safari Nitro optimization. WebKit folks are especially proud of this. Recursion tests may be criticized as a bit of showing off, but it's something of a worst case. Simpler control flow tests would likely fall into the category of executing too rapidly to measure.

In a few cases, Nitro's speed makes it possible to tackle classes of tasks that programmers aren't crazy about doing in JavaScript at all, with regular expressions, date math, and binary trees being fair examples.

The tests also show common tasks in need of attention. Input validation, binary trees (oddly slow given such solid recursion results), and scientific/statistical math are marked "significant" in comparative results, but something less than 2x doesn't really rock my world. Standards preclude pulling these sorts of things into JavaScript built-ins to leverage native code. The WebKit team could do these as enhancements, but Safari-only sites won't play in Peoria.

Overall, it's WebKit's pace of change, its race to implement punishing new standards like CSS 3 and HTML 5 in a complete and accurate fashion, that makes Safari 4 beta and the WebKit browsers so remarkable. WebKit's implementation of the JavaScript language is built for incremental and non-disruptive improvement.

Test

Safari 4 speed

Firefox 3.0.5 result

Safari 4 beta result

 3d:

(group average)

1.50x as fast

393.8ms +/- 2.0%

263.2ms +/- 9.0%

 

cube:

1.17x as fast

144.6ms +/- 2.0%

123.6ms +/- 17.8%

 

morph:

1.63x as fast

122.6ms +/- 4.1%

75.4ms +/- 5.7%

 

raytrace:

1.97x as fast

126.6ms +/- 6.5%

64.2ms +/- 4.4%

         

 access:

(group average)

3.67x as fast

457.8ms +/- 8.8%

124.8ms +/- 2.6%

 

binary-trees:

4.31x as fast 

56.0ms +/- 5.2%

13.0ms +/- 6.8%

 

fannkuch:

10.3x as fast

174.6ms +/- 2.7%

17.0ms +/- 7.3%

 

nbody:  

1.95x as fast

169.0ms +/- 18.9%

86.8ms +/- 2.8%

 

nsieve: 

7.28x as fast 

58.2ms +/- 6.3%

8.0ms +/- 0.0%

         

 bitops:  

(group average)

5.88x as fast

310.4ms +/- 1.3%

52.8ms +/- 4.8%

 

3bit-bits-in-byte:

16.4x as fast  

52.4ms +/- 6.4%

3.2ms +/- 17.4%

 

bits-in-byte:

9.88x as fast 

79.0ms +/- 3.7%

8.0ms +/- 15.5%

 

bitwise-and: 

8.66x as fast 

81.4ms +/- 4.9%

9.4ms +/- 7.2%

 

nsieve-bits: 

3.03x as fast 

97.6ms +/- 3.2%

32.2ms +/- 6.9%

         

 controlflow: 

(group average)

8.65x as fast 

39.8ms +/- 4.1%

4.6ms +/- 24.2%

 

recursive: 

8.65x as fast 

39.8ms +/- 4.1%

4.6ms +/- 24.2%

         

 crypto:  

(group average)

2.66x as fast

189.6ms +/- 1.5%

71.2ms +/- 2.9%

 

aes:  

4.26x as fast 

75.0ms +/- 2.9%

17.6ms +/- 8.1%

 

md5:  

2.04x as fast 

56.2ms +/- 3.3%

27.6ms +/- 5.1%

 

sha1:  

2.25x as fast 

58.4ms +/- 5.3%

26.0ms +/- 9.6%

         

 date:  

(group average)

4.12x as fast

393.8ms +/- 2.7%

95.6ms +/- 2.0%

 

format-tofte:

6.18x as fast

236.0ms +/- 4.3%

38.2ms +/- 3.6%

 

format-xparb:

2.75x as fast

157.8ms +/- 5.9%

57.4ms +/- 3.6%

         

 math:  

(group average)

1.74x as fast

340.6ms +/- 3.6%

195.8ms +/- 2.4%

 

cordic: 

1.73x as fast

137.2ms +/- 3.9%

79.2ms +/- 3.4%

 

partial-sums:

1.69x as fast

142.8ms +/- 8.5%

84.6ms +/- 3.2%

 

spectral-norm:

1.89x as fast 

60.6ms +/- 3.1%

32.0ms +/- 2.7%

         

 regexp:  

(group average)

7.12x as fast

277.8ms +/- 1.7%

39.0ms +/- 6.4%

 

dna:  

7.12x as fast

277.8ms +/- 1.7%

39.0ms +/- 6.4%

         

 string:  

(group average)

2.43x as fast

804.0ms +/- 1.0%

330.6ms +/- 1.6%

 

base64: 

3.07x as fast 

93.4ms +/- 5.0%

30.4ms +/- 6.2%

 

fasta:  

4.76x as fast

210.6ms +/- 4.9%

44.2ms +/- 7.3%

 

tagcloud:  

1.49x as fast

165.0ms +/- 3.3%

110.8ms +/- 3.7%

 

unpack-code: 

3.13x as fast

220.0ms +/- 2.1%

70.2ms +/- 2.9%

 

validate-input:

1.53x as fast

115.0ms +/- 4.5%

75.0ms +/- 6.8%

TOTAL: 

 

2.72x as fast

3207.6ms +/- 1.8%

1177.6ms +/- 2.5%

Mobile Security Insider: iOS vs. Android vs. BlackBerry vs. Windows Phone
Recommended
Join the discussion
Be the first to comment on this article. Our Commenting Policies