Back in June, Google launched Sputnik, a suite of tools that runs over 5,000 tests to check a web browser’s JavaScript conformance. Last week, they made the tool a lot easier for anyone to use, with a version that works in the web browser. The results are interesting.
Notably, both the Opera and Safari web browsers beat Google’s own Chrome browser in the test. As you can see in the picture above, Opera is the clear leader, with only 78 failures (the closer to the center, the less errors). Safari came in second with 159 errors, with Chrome in third with 218 errors. Firefox is close behind with 259 errors, while Internet Explorer is the outlier with 463 errors.
These tests were run on Windows machines, with the latest released version of each browser. Using the web tool on my Mac, though, shows similar results (at least for Opera, Chrome, Safari, and Firefox — there is no IE for Mac anymore).
While much of the focus on JavaScript is about speed (that’s what the SunSpider test measures, for example), Sputnik is interesting because it focuses on conformity, making it more like the Acid3 test, which tests web standards compliance. Chrome, Safari, and Opera have all passed Acid3, with Firefox getting very close (94/100 for Firefox 3.6). IE, meanwhile, again lags behind with just 20/100 for IE8. And even the new IE9 preview only scores 55/100.
Speaking of IE9, I tried to run the Sputnik tool in the preview build of the new browser on Windows 7. Unfortunately, it completely shut down several times after getting up to about 50 failures after only a few hundred of the 5,000+ tests — not a good sign. But again, it’s just a very early preview release of the browser, and early SunSpider results for the browser have been good.