what your web analytics data gets wrong about users

Just how accurate are your site’s user figures, and what should you look at for accurate observations?

Google AnalyticsGoogle Analytics just celebrated its 10th birthday. The web has been popular for roughly two decades now. You’d think that by now, marketers would have this tracking thing down, right?

Of course, in 2015, marketers can’t even tell you how many people visited a web site.

It’s embarrassing, really.

To understand the disconnect, and what it means for your web analytics data, it pays to learn how we used to track things, how we’re tracking things now, and what all of your tools can do to give you actionable data.


50 First Dates

We had a period, very early on, when visiting web sites was essentially like having a conversation with Guy Pearce in Memento, or Drew Barrymore in 50 First Dates.

The sites were unable to form new memories – you’d have to introduce yourself every 30 seconds or so.

That meant that if you were at a checkout process, and you hit back or close your browser, you’re out of luck. You’d start over – the sites did not remember who you are or what you’ve done so far.

That’s essentially what first party cookies are for. Sites started dropping them so there are little reminders about your machine (versus other machines) and what actions you’ve taken on it.

First Party Cookies

Starbucks Reward Card

Cookies are good, but they are more like a rewards card at Starbucks than your social security card.

The tracking does not follow you around.

You visit a site, and boom – you get a stamp for that particular site. That one site can “remember” your machine as a unique machine, even if you visit 3 times in a month. (In web analytics parlance, that’s 3 visits, and 1 unique visitor.)

The thing with first party cookies is, only that site can remember you.

Your visit to site A will not be known to site B, the partner site. You can’t be served remarketing ads for GoPro accessories after you’ve purchased a camera for your kayaking trip.

Third-Party Cookies

Gopro And Gopole

Pretty soon, marketers started dabbling with third party cookies.

That is, sites dropped cookies for partner networks – this still isn’t like a universal ID for your browser, (that would be like a social security card for the web), but it’s closer.

Now, sites that partner up under a network will be able to detect that you’ve been to one of the other sites, if you’re using the same browser and the same device.

So if you bought a GoPro on one site, and the retailer has the right partner networks, you’ll get offered accessories on another retail site, or maybe discounts on popular kayaking and skiing deals on hotels and resorts.

This again presupposes that if you used Chrome on your laptop, you’re not using Internet Explorer when you go to the partner site, or your mobile phone on destination sites.

Cookies and Web Behavior

So, with all that in place, and 2 decades after the web became popular, why don’t sites know how many users there are every month?

Let’s say you have a user named Jane.

Jane visited Smart Recipes 3 times this month on Chrome, on a laptop. That’s one unique count for Jane that month.

But she also used Internet Explorer and Firefox on that laptop. The cookies are per browser, so now that’s 3 unique visits she’s registered. The unique count is already not great, but still sort of useful.

However, Jane also used Safari on her iPhone to check on another recipe, and then Chrome on that same device to check something else. That’s 5 uniques Jane has registered.

Because Jane’s laptop eliminates cookies at set intervals, the laptop cookies have gotten erased in the middle of the month. So when Jane used Chrome and IE on her laptop again, she then registered 7 unique visits.

Now, imagine that scenario for thousands of users.

The Fallacy of Unique Visitors

Your unique visitor count isn’t actually a count of unique visitors – it’s the arbitrary combination of one cookie, per browser, per device. Your Janes may be getting tracked as 1 unique visitor, or 7, or 3, and you have no way of telling which is which, unless you make your visitors log in.

So what should your user stats look like?

As Google and Facebook evolve, we may eventually get to an always signed in scenario across browsers and devices, and then we’ll be able to get better unique user counts.

Until then, there’s plenty to think about for user reports:

  • Unique visitors is a dirty statistic: Number of people may be the gold standard, but you shouldn’t ever report it without caveats to management, because current data is too inaccurate.
  • Page views cannot replace unique visitors: Page views are far more accurate, but also far less useful – they should not be tracked as a measure for site health, in place of uniques. (Users can hit 10 pages while being confused, adding to the page count, while actually being bad for site experience.)
  • Visits do part of the job of uniques: Visits are both accurate and useful – use visits and tasks for your reporting, AND limit the use of unique visitors when visits will do.
  • Downloads for certain things can proxy for top-level uniques: Near the top of your sales funnel, you should use downloads for early stage leads to monitor “prospect” growth.
  • Leads and conversions rock: Of course, the bottom of your funnel is filled with names inside a CRM and individual sales – those are tied to people, and pretty darned accurate.

Data about TV use, from Nielsen, is based on just 50 thousand out of over 116 million TV households. It doesn’t matter how good the math is – that is very dirty data. Still, in 2013, that was good enough for 78 billion dollars in ad spend.

Web data, by contrast, is much, much cleaner.

What it’s not is perfect.

So we, as a community, shouldn’t over-report its accuracy, especially when it comes to counts of people.

Unique people would be nice, and we might get there some day. Until then, combinations of visits, downloads, leads, and conversions should give us all plenty to work with, without presenting the data as about unique visitors.