Today Matt Marshall at Siliconbeat wrote in his blog that webanalytics is back to square one. He points out that Hitwise, Comscore and Alexa show different numbers. Here the graphs that he was writing about:
I. Comscore - Unique visitors and total pageviews
II. Hitwise- All sites market share in Computer & Internet - Net communities and chat based on visits
III. Alexa - daily page views by million
What Matt forgot to say is that all three companies above collect their data in completely different ways. Here an approach to explain the different types of data collection of the three vendors:
Alexa: Alexa data is collected via the Alexa browser toolbar, which is obviously more often installed on tech interested users, than on average Joe’s computer. This means that more tech relevant users on your site increase the chance that your Alexa rank increases. Also software programmes like Alexabooster can easily “boost” your rank . Alexa only gives you an idea or a trend, but will never be accurate.
Comscore: Comscore is based on 2 million participants, which allows them to capture a broad view of surfing and buying behavior. Problem here is pretty much the same. Accuracy won’t be 100% for tech related websites.
Hitwise: Hitwise collects logfile data directly from the ISP networks (network-centric) and does not have a user-centric or site centric approach. They also combine this rich ISP data with a worldwide opt-in panel to overlay demographic, lifestyle and transactional behavior across the thousands of websites that are reported on every day.
Conclusion:
I totally agree with Matt, that the numbers and techniques are not technicially mature, but webanalytics is definitely not at square one. The webanalytics industry is relatively new and norms and regulations have to be (and will be) found.
August 11th, 2006 at 8:38 pm
Excellent insightful objective comparison. Now, would you have any tips on how to explain these technically nuanced differences to those non-techies, as often vendors and executives alike would rely on Alexa ranking, when in fact, those numbers are only valid as a comparative trend to other websites, and not very valid for websites not in the top 150,000 trafficked?
August 12th, 2006 at 2:53 pm
[...] Stagnating, doubling — tomato, tomahto, right? To his credit, Marshall goes out of his way to note that while Hitwise is a “respected” traffic analysis firm, numbers are all over the map — and he links to the other Marshall’s critique of the field. The simple fact is that Hitwise, Comscore, Nielsen and Alexa all use different methodologies (a good description here) and as a result they are not just talking about apples and oranges, they are talking about apples and oranges and plums and peaches. [...]
August 13th, 2006 at 6:38 pm
[...] Auch ganz beeindruckend ist die Tatsache, dass der Serverpark seit der Übernahme durch Yahoo von 20 auf 100 Maschinen erweitert wurde. Einen genaueren Blick in die verschiedenen Zahlen und zählweisen kann man bei webanalytics werfen. [...]
August 26th, 2006 at 2:51 pm
[...] Webanalyticsbook added some graphs and a good summary of the main companies that guess traffic. Alexa: Alexa data is collected via the Alexa browser toolbar, which is obviously more often installed on tech interested users, than on average Joe’s computer. This means that more tech relevant users on your site increase the chance that your Alexa rank increases. Also software programmes like Alexabooster can easily “boost” your rank . Alexa only gives you an idea or a trend, but will never be accurate. [...]
September 8th, 2006 at 10:52 am
[...] As mentioned before Alexa has the problem, that it collects traffic with the Alexa toolbar, which is usually used by webmasters or technical interested users. This obviously caused technical related websites to rank higher than non-technical websites (Just google NYTimes vs. Digg.com). [...]
September 24th, 2006 at 10:44 pm
I was involved with another web analytics company you don’t mention (funny, since I believe it’s larger than the three you do mention). The biggest problem is that even with huge sample sizes, for sites below the top few thousand, you’re still dealing with a very small numbers of accesses among your sample, so it’s impossible to accurately extrapolate to the whole population with any confidence, even if you’re statistically tweaking the results based on demographics as Comscore does.
December 17th, 2006 at 11:41 am
[...] Since big controversial discsussions about numbers (Comscore vs. Hitwise vs. Alexa vs. everybody else) I believe the highest demanded jobs are webanalysts. My highlight happened last week, when a friend of mine referred me to these numbers: [...]
February 8th, 2007 at 12:55 pm
[...] Matt Marshall at Venturebeat seems not to stop beating up the webanalytics industry. Last time he spoke about the industry during the famous Digg vs. Deliciou.us “fight” and I responded. [...]