Google Launches Octane: A New JavaScript Benchmark Based On Popular Apps And Libraries

Google’s Chrome team just launched Octane, a new JavaScript benchmark suite that aims to measure real-world performance. Unlike most other benchmarks, which tend to use artificial tests developed to stress a certain feature, Octane is based upon a set of well-known web applications and libraries. This means, says Google, “a high score in the new benchmark directly translates to better and smoother performance in similar web applications.”

At its core, Octane is an extension of Google’s existing V8 benchmark suite and includes all of its predecessor’s tests. In addition, Octane also runs tests based on Mozilla’s PDF reader, a open-source GameBoy emulator, a 2D physics engine and a port of the 3D Bullet Engine. The CodeLoad test also uses code derived from Closure and jQuery to measure how quickly a JavaScript engine can load and execute commonly used JavaScript libraries. The idea here, says Google, is to avoid making Octane a micro-benchmark (like the popular SunSpider benchmark) that only tests very specific JavaScript features.

Here are the details of all the new tests, courtesy of Google:

  • Box2DWeb runs a JavaScript port of a popular 2D physics engine that is behind many well-known simulations and web games.
  • Mandreel puts a JavaScript port of the 3D Bullet Engine to the test with a twist: The original C++ source code for the engine is translated to JavaScript by Onan Games’ Mandreel compiler, which is also used in countless web-based games.
  • Pdf.js is based on Mozilla’s PDF reader and shows how Javascript applications can replace complex native browser plug-ins. It measures how fast the browser decodes a sample PDF document.
  • GB Emulator is derived from an open source emulator of a famous game console running a 3D demo.
  • CodeLoad measures how quickly a JavaScript engine can bootstrap commonly used JavaScript libraries and start executing code in them. The source for this test is derived from open source libraries (Closure, jQuery).

The source code of Octane itself is available for download, too.

With Kraken, Mozilla also offers a benchmark that aims to measure real-world browser performance. In my own informal tests with Octane, the current release version of Chrome generally beat the latest stable release of Firefox by a wide margin (the same, by the way, is also true for the Kraken test).