Web analytics faux pas

Next Story

Facebook Is Not Only The World's Largest Social Network, It Is Also The Fastest Growing

This is a guest post by Ed Freyfogle, co-founder of property search engine Nestoria.

It’s often claimed that the advantage of the internet versus traditional media is that
everything can be measured. Rarely mentioned though is the pain experienced by industry
insiders working in the fog of confusion caused by such a hyper-analytical environment.
In this rapidly evolving new medium fundamental misunderstandings are surprisingly common.

A few of the most striking, and common, examples:
  • Mistaking users for customers – Some people use the terms ‘users’ and ‘customers’ interchangeably. That’s nonsense. Customers are people who pay you. Users are people who use your service. There can certainly be overlap (ex: Amazon), but for most media sites customers means advertisers. Don’t misunderstand, I’m not saying users aren’t important. You should certainly focus on user experience, but mixing customers with users shows miscomprehension of the media business model. If the terms are interchanged, alarm bells should start ringing.
  • Using the term ‘hits’ – Before last.fm there were things called record companies. They produced hits. I have no idea what is meant when I hear someone say a website has a lot of “hits”. And evidently nor do they.
  • Belief in pageviews – Conceptually pageviews seem straightforward: a measure of the number of times users view your content. Sadly it’s not that simple. The interweb is swarming with bots querying servers for content that no human will ever see. Secondly, in the age of AJAX and widgets it’s hard to quantify what exactly a pageview is. Most importantly though, users are spending more time online, generating (or “consuming” as hip media exec likes to say) more and more pageviews. Facebook and Bebo are prime examples. As supply of pageview inventory rises, value (CPM) drops. None of this is news – see last year’s discussion re: Death of the Pageview – but it’s amazing how people still cling desperately to the pageview metric.
  • Rampant use of averages – any number worth knowing the average of is worth turning into a histogram. You don’t have ‘average’ users. Most likely your site is visited by power users, recurring visitors, and occasional passer-by’s. Using averages is a fast path to shaky conclusions.
  • Blind faith – There’s nothing worse than ‘true believers’ – those who never question the numbers put forth by their analytics system. Software has bugs, can be badly configured, garbage in still leads to garbage out, and different tools measure different things. As an example, Google Analytics often disagrees with Google Adwords. Tools just give you data, data needs to be analysed and understood to become information.

Things I haven’t covered: confusing correlation and causation, those who endlessly argue why comScore and Hitwise are all wrong, and (my personal favorite) flippantly tossing ‘it could be seasonality’ into any discussion of analytics. In short, the true potential of the mass of data the internet generates lies not in its quantity but in its thoughtful application.

BTW, in the three minutes it took to read this post, my RSS reader generated a few thousand pageviews.

(Photo from maisonbisson @ flickr)

  • http://www.vinayis.in Vinay

    “Not everything that can be counted counts, and not everything that counts can be counted” – Albert Einstein

    ^^ the quote above is so relevant to the post :-) ? Ain’t it? Good article.. thumbs up!


  • http://www.broadstuff.com alan p

    Nice post, Ed.

    I think it’s true to say that no-one has a clear view yet, hence the Measurementcamp initiative Will McInness kicked off, and no doubt many other similar things.

    The other main issue with metrics is that people cleave to what can be measured, which is not the same as what should be measured.

  • http://www.afterworkdeals.com Burak

    Hi Ed, I agree with your observations that jargon and statistics can be and are often used to confuse matters or hide/spin the truth. You point out a lot of abuses, however can you suggest any ways forward? Should we get the ISO to form a set of standards that we could all then adhere to?

  • http://edtorres.blogspot.com Ed Torres


    I wonder how many “hits” and “pageviews” you will get with our comments as well. :D

  • http://www.nestoria.co.uk Ed Freyfogle

    @Burak – the only solution I see is rigorous discipline.

    @alan p – very true about confusing what data are available with what data are important. A common mental trap.

  • http://www.webanalyticsbook.com/webanalytics-general/most-common-mistakes-in-web-analytics/ Most Common Mistakes In Web Analytics | Webanalyticsbook - Web Analytics

    […] UK has a funny post about “Web Analytics Faux Pas” from guest author Ed […]

  • http://www.SethHolladay.com/links/links-for-2008-08-13-deliciouscom/ Seth Holladay » Links » links for 2008-08-13 [delicious.com]

    […] TechCrunch UK » Blog Archive » Web analytics faux pas (tags: webanalytics analytics) […]

  • http://www.123.com J

    Hi Ed,

    Your comment ‘it could be seasonality’ – isn’t this a generalisation. As a tourism website specific to an area or town would have increased activity during the better parts of the year?

  • http://relearn.ie Eamon

    Great writing. And what about stat addiction? Its like checking your email constantly, opening 25 browsers tabs or counting your linked/facebook friends. Very soon it just becomes compulsive and you’re looking for a quick hit (quick hits).

    People should be sober and disciplined about their Google Analytics time. Put on a tie before you login. Time yourself. Give it up for lent.

  • http://www.blog-nestoria.es/en-techcrunch/ En Techcrunch en El Blog de Nestoria - el Buscador de Casas y Pisos más facil de usar

    […] años gracias a las Olimpiadas, puede que estéis interesados en leer en árticulo [en inglés] que Ed escribió la pasada semana en Techcrunch UK sobre los malentendidos que surgen en el mundo online en torno a cuestiones analíticas. Para ello, […]

  • http://blog.nestoria.co.uk/2008/11/04/nestoria-at-global-connect-in-nyc-in-january/ Nestoria at Global Connect in NYC in January at Nestoria Blog

    […] about. As an example I recently had the chance to guest post on TechCrunch about some of common web analytics pitfalls and misconceptions. I have no doubt the Global Connect panel will be interesting. Like all businesses the property […]

  • Cryresult

    Link Sky,commitment well know elsewhere exactly whom representative primary certainly award radio okay next engineering change search more quickly after category receive start cost steal tape prefer element perform official working give gentleman legal very another start consist control complex relationship violence organization piece audience comment whilst son king how soldier letter customer extend art individual meet commission ought growing anyone roll before fuel telephone which difficulty field cell suppose chemical will protect sound want wish sale advice evidence hole comment talk stick surface support complete

  • http://businessintelligence.me wearecloud

    I liked this post Ed, many valid points. it helped us when were researching for a similar post – 5 Web Analytics Misconceptions. if you are interested in reading it you can access it here : http://tumblr.com/xkzi4l33f

  • http://grasshopper.com/blog/2011/08/what%e2%80%99s-the-deal-with-web-analytics/ What’s The Deal with Web Analytics? | Grasshopper Blog

    […] But just like Uncle Ben said, with great power comes great responsibility. The incredible volume of data can make it easy for some to run away with misleading conclusions, as Ed Freyfogle warns in a post for TechCrunch Europe. […]

blog comments powered by Disqus