IE9, FF4 beta in real-world benchmarkSeptember 16, 2010
Most browser benchmarks are isolated, artificial tests that can be gamed by browser vendors optimizing those specific cases. With only those benchmarks to go on, we were skeptical that IE9’s beta would actually outperform other modern browsers in real-world applications.
To separate hype from reality, we built our first browser benchmarking tool based in Lucidchart itself.
Numbers and analysis are after the break.
The big picture
Surprisingly, Firefox 4.0 Beta 6 came in behind all other browsers except for IE8–even coming in behind its predecessor, Firefox 3.6. While we expect its performance to improve before release, it was surprising to see this result. It should be noted that no Firefox 4.0 Beta 6 test run came in faster than any test run by another modern browser.
IE9 Beta 1 performed surprisingly well (with a few caveats; see the notes below). In fact, it performs well enough that Lucidchart will finally be able to remove the obnoxious warning to IE users that they’re getting an inferior experience in Lucidchart.
These tests were conducted on a Windows 7 x64 laptop running a Core i7 Q720 CPU with 4GB of DDR3 RAM.
For each browser tested, we closed and relaunched the browser before each trial, and the window size was consistent across all trials. The document for the benchmark was very similar to our standard business process template, of moderate complexity. We ran four trials for each browser, and the results shown here are averaged across those trials.
Some final notes on Firefox 4 Beta 6
In several trials on Firefox 4 Beta 6, performance absolutely tanked due to the browser doing some unknown background work. These trials were discarded for the purpose of this data, but it should be noted that Firefox 4 may have inconsistent performance for certain types of modern web applications, like Lucidchart.
Some final notes on IE9 Beta 1
When we first attempted the benchmark in IE9 beta 1, we got results only about twice as fast as in IE8, and many times slower than all the other browsers tested. This actually led us to fix a minor flaw in our rendering algorithm that had us performing many more DOM operations that we needed to during rendering. Removing that flaw let IE9 catch up to most of the other browsers (Chrome excluded).
Run the benchmark yourself
This benchmark can be run by anyone in Lucidchart. First, sign up for a free account here.
Then create a document that you’d like to benchmark. Right-click the block you’d like to have dragged around the page by the benchmark, and click “Bring to Front.” The benchmark uses the topmost block for its test.
Then click the tools menu, and select “Benchmark Performance.”This entry was posted in Tech Talk. Bookmark the permalink.