The Wardman Wire

This is getting tedious, and slightly annoying.

Early in July I had a little go at Joy Lo Dico for suggesting that differences of 1 or 2% between website “unique user” statistics of national newspapers were of significance, when in reality any differences of less than 5% or maybe even 10% in Unique Users are smaller than the likely amount of noise on the figure.

Last weekend we had a repeat of exactly the same embarrassing mistake in exactly the same place - the Media Column of the Independent on Sunday .

On this occasion we had a case of major statistics abuse in an article about the superb satirical website the Daily Mash by Professor Tim Luckhurst.

(Illustration Credit: Wellington Grey )

“Alexa” - Ouch!

From the Professor:

The result was www.the-dailymash.co.uk , launched in April 2007 and now Britain’s most successful satirical website, according to statistics compiled by the web information company Alexa.

The entire internet industry knows that Alexa statistics are generally not worth the paper they aren’t written on. For example, last summer myself and a couple of other people ramped the Liberal Democrat party website from position 377,557 to 195,782 in the space of a week in Alexa by visiting it once a day each while running the Alexa toolbar. To be fair Alexa have made some improvements to their methodology, but it is not anything resembling a rigorous research tool.

Even when you get into the relatively high ranking positions of (say) sites in the top 50,000 which have larger samples of visitors - The Daily Mash is nearly there at 64,778 - there is still a major bias to the IT and Geek segments with the user base, i.e., to the sort of people who run extra add-ins in their web browsers.

What on earth are Alexa statistics doing in an article in a national newspaper?

“Page Impressions” - Ooops.

The Independent on Sunday said:

Last month it recorded 350,000 individual visits and nearly two million page impressions.

That’s an interesting concept: a newspaper reporting on “Page Impressions” as a measure of success for a website .

If we go back 18 months, several things were done in January 2007 to improve newspaper web statistics. One of them was that the newspaper industry abandoned “Page Impressions” as a primary measure of web statistics because they were agreed to be unreliable . “Visits” are not that much better - even ABCe measured “Visits”, since they are simply an aggregation of series of page impressions that have less than half an hour between them. These days the Audit Bureau of Circulations (Electronic) requires that newspapers cannot report Page Impressons alone without Unique Users. As it says in the ABCe Feesheet for 2007:

Unique Users is the industry-agreed mandated minimum metric.

Why do newspapers not apply the same standard to the websites they cover in their articles - for consumption by the public, that they apply to their own websites - for consumption by their advertisers?

Missing “Unique Users” - Hmmm.

I could understand the non-appearance of Unique Users figures if The Daily Mash refused to supply them, but the numbers were published only a few weeks ago.

If you move your mouse two inches, and type the obscure phrase “daily mash unique users” into your Google search box, it is right there in 2 of the first 3 entries: 175,000 unique users in June 2008. That’s impressive for a site started in April 2007: The Wardman Wire started at about the same time and we are only up to 50-55,000 unique users each month, measured from our log files.

Why is the figure for Daily Mash Unique Users not in the article?

Surveyed Demographics - Hurrah!

From Tim Luckhurst:

An online readership survey reveals that 65 per cent of users earn more than 30,000 UKP a year and 22 per cent bring in more than £70,000 UKP. Seventy-three per cent are university graduates and 22 per cent have higher degrees. Most are regular readers of the Guardian, Independent or Times.

No problems there. Online readership surveys are the standard way of getting these numbers.

Conquering America - Holy Hallucinations!

And finally we had, concerning The Onion - a much larger satirical site in the USA that partially inspired The Daily Mash:

The Onion had better look out. Although the Daily Mash gets 81 per cent of its hits from UK users, 19 per cent are American, and awareness is growing fast by word of mouth.

This is bonkers. It is also a fantastic illustration why you should read reputable blogs not dodgy newspapers for some of your analysis (!), or at least why you should read specialists as a sanity check on generalists.

For all that the Daily Mash is a fantastic site, figures from the compete.com metrics service - which measures USA-based traffic and is far more reliable than Alexa - suggest that reality does not match the claim.

On these numbers Last month, The Onion had 1,055,678 Unique Users in the USA; The Daily Mash has 6,678. So to challenge The Onion, The Daily Mash needs an increase in reach of .. er .. er .. wait for it:

15,708%

With the most vivid imagination in the world, and a huge admiration for the Daily Mash, I don’t see that happening any time soon.

Wrapping Up

Please - can we have at least some commonsense and consistency in newspapers where web statistics are concerned. Sweat the detail - please.

Finally, eagle-eyed readers may have noticed that the numbers 175,000 unique users (published Daily Mash figure), 19% of hits from American users, and 6,678 unique users in America don’t look entirely consistent.

You would be right. Services such as compete.com can easily differ from a report done by analysing the server logs of a website by a factor of 2 or 3 - it’s just par for the course.

That’s why I go on about the need for standard processes and a healthy scepticism about web statistics like a broken record. Why not join me in the “sceptical about statistics” club?