A cautionary tale of web numbers

When you sell online advertising or a campaign, you can provide this information and then retrospectively provide a detailed and in-depth analysis of how it worked. You can track it across the social web, proudly and confidently declaring where it went and how it came back to reward you.

Conversely, if things don't go well, the numbers can help you locate the problem. Did people simply not click through? Or did they click through and then bounce? Because everything is digital, the numbers are already there.

Same warnings as all statistics

As incredibly powerful as all this is, it comes with the same warnings as all statistics. They require a certain amount of expertise at both ends of the spectrum. The creation of statistics is far more mysterious and non-standard than people think.

Whether purposeful or not, there are all sorts of different ways to make statistics suit a certain point. Even in rigorously scientific disciplines, there are ongoing arguments over how statistics should be done. In web analytics and statistics, where there is no peer review system, there is often no way of telling how a certain figure was calculated.

As simple a thing as the difference between using number of visits, number of visitors, number of page views, number of unique visitors or unique page views... all these lead to quite different results. Once you get to growth estimation, traffic variation over the time of year and a host of other variables affect your end figures. Most of the people who work with these stats online are not statisticians.

With the belief that the numbers don't lie, comes the assumption that they tell the truth. The truth, however, is always more complicated than a few numbers and, unfortunately, people have a tendency to try conglomerate their analytics into as few numbers as possible. This inevitably has the effect of oversimplifying the analytics.

Not quite as useful

As an example: without Klout, and its sub scores, the number of Twitter followers is only half of a story. Without knowing clickthrough rates and, more importantly, if any of the traffic ends up doing what you want it to (whether purchasing or subscribing etc), the number of eyeballs on your advertising is not quite as useful.

If you don't know the location and demographic of the traffic, you might end up advertising to people that have no possibility of becoming customers. Having a banner ad on a popular site is great, but if your ad is one in hundreds of others, then you probably are not getting your money's worth.

As important as the hard stats are, the explanations and sub-statistics can often tell you more. The numbers only make sense when they are looked at in their specific context.

There's an old tongue-and-cheek quip that 80% of statistics are made up on the spot. More than people expect, the numbers that are given can simply be inaccurate. A specific and very important example of this is public-facing web analytics. If we look at scores such as Alexa, Compete and even Afrigator, they present rankings and traffic stats to the public. This is particularly useful for advertisers, considering that these stats are otherwise exclusively in the possession of the web-space owner.

Stats can be very misleading

The trouble is that these stats can be very misleading. Alexa, as an example, uses a very simple sample-based approach. Web-users can install an Alexa toolbar on their browser which feeds information about their browsing habits back to Alexa. This community has formed Alexa's sample for the Internet population, which it uses to form its rankings and estimations.

The trouble, as any social scientist will tell you, is that such a sample is grossly misrepresentative of the real population. Those in the Alexa community are far more likely to be a specific kind of web-user than the average and thus will exhibit very different browsing habits. Similarly, on smaller (especially South African) sites, your Alexa ranking can be dramatically increased by convincing a few friends to install the toolbar and browse around your site.

Because of the relatively small sample it uses to make its rankings, what is in reality an insignificant amount of traffic will register as an influx of new users. The bigger the site is, the more accurate Alexa is.

Unfortunately, only a few South African blogs get big enough for the Alexa ranking to be a good metric. To emphasise this, I ask a very simple question. Do you know anyone who uses the Alexa toolbar?

Whole host of other problems

There are a whole host of other problems, including the toolbar not being compatible with Google Chrome, as well as it sometimes being recognised as spyware. This is not meant to condemn Alexa. Compete also runs off its own community (two million users).

Another method (that of Afrigator and Amatomu) puts third-party code into your website. Many people aren't happy with including this in their code. Quite a few Afrigator users, as an example, that have the Afrigator badge and tracking code installed found they recently caused issues. It also takes your own analytics out of your hands, and puts them under the aggregator's thumb.

Technically, Google Analytics does the same thing, but people are already used to the idea of Google having a backdoor key to almost every aspect of their online lives.

If you are comfortable with being a part of Afrigator, it is open about how it ranks its sites and its backend access does mean that its rankings are mostly accurate. It does, however, suffer from a few technical issues and its lack of updating doesn't inspire confidence.

The unfortunate truth is that it is difficult to get stats from others' websites, and public-facing alternatives always suffer from inaccuracy. Understandably, projects such as Alexa aim to find an independent source of traffic stats, some way to verify claimed traffic. The problem is often that advertisers and sponsors request these statistics, such as Alexa rankings, without a real understanding of what they actually mean.

Your lifeblood

For those who sell space on the web, advertising based on your traffic is your lifeblood. Especially for bloggers, who often get no remuneration for their efforts otherwise. If a high Alexa ranking is the key to hooking sponsorship and advertising, it is effectively incentivising the abuse of the system, a system which is not robust enough to handle even the simplest exploitation.

Of course, getting the stats from the person concerned suffers a similar risk. More easily than fudging their Alexa ranking, they can simply lie to you. But there are ways to deal with this. You can show potential advertisers your analytics openly. There are WordPress plug-ins for this very reason.

But the greater answer is to be careful and smart, on both sides.

Use a multitude of sources. Look at comments on a blog, or page views, Tweet and Facebook shares. Use the Alexa rank in conjunction with other stats. Look at its mentions on other sites, and even Google Trends.

To gauge a website's impact and popularity, you need more than a few numbers. The better you understand where you are advertising and its demographic, the more you can make sure that it will get the best results for your brand.

In the end, this is the power of online advertising. Rather than throwing the biggest net you can find into the ocean and dragging everything in, find that sweet fishing spot where you already know there's a catch you want.