Quantcast

How do Different Statistics Systems Compare: Statistics Sunday

WardmanWayIcon3This post compares my traffic figures on the Wardman Wire for September using four different analysis systems I have in place.

These figures are taken monthly over the life of the Wardman Wire from March 2007, and range from 2-3000 unique visitors per month up to around 10,000 to 15,000 per month.

I do the analysis and conclusions at the top of the article. The detailed graphs and data tables are at the bottom. I will upload the full spreadsheet with the detailed data over at Poliblog Perspective .

Analysis

  1. I did some comparatives on 4 of the 5 stats systems I have in place, as described below (server log files, two Wordpress statistical plugins, and the Gostats web-based statistical system).
  2. I measured “Unique Users”, “Visits” and “Page Impressions” (defined below).
  3. The figures are taken, as I have mentioned, on reasonably large data sets of up to 9,000 to 18,000 unique visitors (depending which set of figures you believe) over the first 6 months of the life of the Wardman Wire.
  4. The figures varied by the following factors (i.e., between the highest figure and the lowest).
    - Page Impressions: x14
    - User sessions: x6
    - Unique visitors: x2
  5. The one caveat I will add is that the x14 and x8 are more likely to be
    x5 and x3 as my Gostats code is right at the bottom of the right hand sidebar, and so is often missed if visitors leave the page before the whole things has loaded.

Conclusions

  1. Worrying about absolute figures beyond an accuracy of perhaps +100/-50% in web statistics is time wasted - the figures are not accurate enough.
  2. However, provided you stick with the same measurement processes, trends can be meaningful.
  3. I can pull some useful information out of these figures, but only because I know what I posted throughout the period and over recent months, and understand what has been happening.
  4. For blogs, Unique Users is probably the most robust figure, but even that has severe limitations - for example many people in a large organisation may look like a single user.
  5. For day to day monitoring I choose the system I use by convenience rather than accuracy - since I want to understand the trends in 30 seconds.
  6. For a more careful month-end analysis I usually use server log files as the information is more comprehensive, so I can look at the real detail if necessary.

Summary - Measurements

I have done comparisons across three metrics:

Unique Visitors

The number of different computers that have been detected visiting the blog.

Visits / User Sessions

The number of times the blog has been visited. A “visit” is deemed to finish if the visitor does not return within a 30 minute period.

Page Impressions / Page Views

The total number of times a web page has been viewed in the period.

A note: If anyone starts talking about “hits” without a precise definition, you should regard that person with the same level of respect as a snake-oil salesman, or a politician with a dodgy bar-chart.

Summary - Analysis Systems

I have compared four ways of measuring my statistics:

  1. Go-Stats: A third party web-based statistics system.
  2. WP-Slim stat: A statistical plugin for Wordpress.
  3. WP-Slimstat2: A different version (WP-Slimstat-Ex) of the same plugin. I will explain the differences in a separate article if requested.
  4. Logs: The raw server logs analysed using the Deep Log Analyzer package.

Charts

Unique Visitors Comparison

Click on image for full detail.

20071014-wardman-wire-2007-unique-visitors-comparison

User Sessions Comparison

Click on image for full detail.

Note wp-slimstat does not record “visits”.

20071014-wardman-wire-2007-visitor-sessions-comparison

Page Impressions Comparison

Click on image for full detail.

20071014-wardman-wire-2007-page-impressions-comparison

Table of Figures

Click on image for full detail.

20071014-wardman-wire-2007-stats

Wrapping-Up

I will be posting the spreadsheet with the full data over at Poliblog Perspective later.

Tags: , , , , , , [tags], , , , , , [/tags]

About the Author

admin

Matt is an internet consultant, commentator, freelance writer and Project Manager based in the UK. He is available for hire. Matt edits the Wardman Wire, and writes at Poligeeks, Total Politics, and occasionally in several other places.

RSS feed | Trackback URI

4 Comments »

Comment by admin
2007-10-14 18:49:34

I’ll add one extra comment.

The “July bulge” in all these figures relates to this article (interview with Daniel Radcliffe on his 18th birthday), following which the about 5000-7000 people came and listened to it in the next 3 days. There are just under 300 comments.

My analysis is here.

Such is blog-life.

 
Comment by mutleythedog Subscribed to comments via email
2007-10-16 13:31:17

The variation in the degree of difference is really striking - the explanation of the July bulge is clear - but do you have any thoughts why the difference between the logs and slimstat e.g. on unique visitors varies so much from month to month?

I use sitemeter - which is just for trends - does it make difference that its at the bottom of the page?

 
Comment by admin
2007-10-16 14:10:00

Hey Mutts

I’ll move the Gostats code to the top for a bit. On this blog I reckon it misses at least half and perhaps 80% of the page impressions. This page is really much too “heavy”. I think the position on the page has a very major effect - as I commented.

See this article from back in April:

Comparing stats systems

Also, if you check Iain Dale’s source code, you’ll see he has his right at the top. I reckon it still undercounts, but hey…

As to the trends over the months - I’ll get round to writing an article soon over at the Poliblog Perspective blog, but it won’t be for a few days as it will be a 2 hour job to do the analysis.

BTW you may enjoy joining The Truth Laid Bear - ranks your blog on the evolutionary chain. I’m a slimy mollusc.

 
Name (required)
E-mail (required - never shown publicly)
URI
Subscribe to comments via email
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> in your comment.

Comments will be sent to the moderation queue.

Trackback responses to this post