The digital world is all a-Twitter this morning with the news that Alexa, the much maligned ranking system, is to change its methodology. In an announcement on its site, Alexa explains exactly what this will mean (well, almost):
We now aggregate data from multiple sources to give you a better indication of website popularity among the entire population of Internet users.
Up until now Alexa has based its ranking system on sites visited by users who have downloaded the Alexa Toolbar. This meant that its rankings weren’t really worth the paper they could be printed out on to, as it inherently skewed rankings towards sites likely to be visited by Alexa users, most of whom tend to be involved in tech, and particularly SEO.
For instance Matt Cutts once pointed out that according to Alexa data his blog was only slightly less popular than Ask.com. Despite this Alexa data is frequently used by people who simply don’t understand how meaningless the data is (other than for trending purposes). At one company I worked at the board actually decided that Alexa rankings would be a good KPI by which to measure success, not realising that by doing so they encouraged everyone in the business to download the toolbar, thereby skewing the data even further.
So the news that they will now be using multiple data sources is a good thing right? Well, it might be if they actually disclosed what these new sources were. Until they do one can only assume that they will be as statistically water-tight as the ones they have used up till now. With that in mind I’ve been trying to contact Mystic Meg as I’m sure that she’d be able to provide them with the kind of water-tight data they’ve so far lacked.
As of posting this Meg could not be reached for comment; she’s probably too busy downloading the Alexa toolbar and reading Matt Cutts’ blog.
It’d be more interesting if Alexa actually let us know what these “other” sources are, as it is i’ll still not recomend using Alexa to anyone – apart from if they want to spoof it.
Exactly. Maybe Mystic Meg doesn’t want people to know about her weekend job..
There is some controversy over how representative Alexa’s user base is of typical Internet behavior. If Alexa’s user base is a fair statistical sample of the internet user population, Alexa’s ranking should be quite accurate.
But that’s kind of my point – I simply don’t believe that Alexa’s user-base is in any way representative of the general internet audience