After a lengthy legal face-off, Microsoft and European antitrust officials recently agreed on the implementation of a so-called ballot screen that will give European Windows users a chance to download rivals’ browsers – including Chrome, Safari, Firefox and Opera – as possible alternatives to Redmond’s own Internet Explorer (see screenshot above or go here).
Under the terms of the deal, Microsoft has agreed to provide a “ballot screen” to most European customers that will offer links to downloads of browsers offered by the company’s fiercest competitors when it comes to the Web browsing space, starting next week. The browser choice screen was designed to give all listed browsers a random order upon each new visit; antitrust regulators saw this as the right path to take to make European consumers more aware of alternative browsers to IE without favoring one over the other.
But how random is the presentation of the browser on that ballot screen, really?
That’s exactly what the good people behind Slovakian tech news site DSL.sk set out to discover, based on the current implementation and code found on www.browserchoice.eu. Their findings were quite interesting, as they seem to suggest that the selection isn’t really that random as one would imagine, and that Microsoft is not doing itself any favors at all, when in fact it may even be giving Google’s Chrome browser a bit of an edge.
It took me some creative Google Translating to figure out how the team got to its conclusion, but finally a Skype chat with one of the reporters at DSL.sk cleared things up for me.
The page on www.browserchoice.eu is static, running nothing but Javascript. The guys at DSL.sk basically automatically loaded that page tens of thousands of times, and they kept score of which browsers were shown in which order for each of those instances. And not only did they test this sufficiently on this page, but DSL.sk did the same for the core Javascript algorithm that triggers the random ordering.
The test were run using Internet Explorer 8 on a Windows 7 machine, because the ballot screen will pop up in IE for users who install the relevant Windows Update and have set Microsoft’s browser as default.
More than once out of every four hits, the page would show Google Chrome on the far left, and Internet Explorer would only make it to the first spot in 13,8% of page loads (scoring well below all four other browsers). In fact, in over 50% of all page hits, Internet Explorer would come out to the far right spot of the five browser choices shown on the screen.
Here’s a table with the stats – the titles are in Slovakian but are simply indicating the order of the browser and its average position in the right column:
What’s most apparent is that Google Chrome scored ‘best’ out of five for all 3 first spots on the browser choice screen, and that Internet Explorer appeared on the far right way more than rival browsers. We should note that this doesn’t necessarily mean there’s a conspiracy going on – perhaps tens of thousands of hits are simply not enough to produce relevant results, or the results are skewed for a different reason. DSL, for one, claims the test results are quite stable and don’t seem to alter much when the number of loads keeps on increasing.
Be that as it may, it’s also worth noting that the ‘first spot’ doesn’t necessarily mean it’s also the ‘best spot’ – eye movement research could well conclude that the middle, far right or any other of the spots is actually the most beneficial one.
For what it’s worth, the DSL team says they had to make two minor changes to the code in order to run their tests mimicking the real behavior of the page as closely as possible, so theoretically the results could end up being more random than they appear based on the results cited above when the browser choice screen actually goes live.
Also, different browsers produced different results, although it didn’t matter much whether IE6, IE7 or IE8 was used for testing. Tests were also run in Firefox, baring completely different results, although there was never an equal distribution between browsers whatsoever, so even then the ‘randomness’ can be questioned.
Do you think the selection on the browser choice screen will end up being completely random, or will more exhaustive research ultimately show that there’s a consistent pattern of browser selection happening here?
(Hat tip to Patrik Hornik)
