A case study in dodgy statistics: National Secular Society Follies I
From the National Secular Society newsletter last week, an item about religion among “influentials” in London under the headline:
The London Evening Standard has conducted a survey among 700 “influentials” – that is to say businessmen, innovators, politicians etc – in the capital to find out their opinions about the impact of the Muslim community on London. The first finding is that 38% of the respondents have no religion; 47% say they are Christian (quite a drop on the 72% claiming to be Christians in the census); 1% said Buddhist; 1% Hindu; 6% Jewish 1% Muslim; 0% Sikhs; 1% other; 1% don’t know and 4% prefer not to say.
You can see an article about the survey here, including the graphic which gave the NSS their figures.
That (the text in bold) looks to me likely a rather dodgy statistical comparison, especially for advocates of an “evidence-based” tradition.
The NSS Claim
I see three problems with the claim in bold above.
1 - The NSS make a False Statistical Comparison
A comparison is being made between 2 completely different sets of data, which have been answers to 2 completely different question. The Census question was “what is your religion?“. The Evening Standard question was “which religion do you belong to?“.
You can take whatever view you wish about what the question should be, but organisations which make regular claims to be “evidence-based” should not be making false comparisons, and drawing conclusions on that basis.
2 - The NSS compare data from two non-comparable samples
Aside from the actual questions, how on earth can a credible comparison be made between a poll of “700 influentials” in London (whoever they are) and a nationally conducted census?
I could do an equally valid survey of 700 different London influentials and get a completely different result.
3 - The Evening Standard survey is not in an objective context anyway
No context is provided to help us evaluate the claim. In fact, if we dig a bit deeper, we find that the survey quoted is part of an Evening Standard exercise called “Is Islam Good for London?”.
Hardly an objective context in 2007 for a question about religion.
Conclusion
Basically, the comparison made is entirely specious. Not impressive for an “evidence-based” organisation.
The National Secular Society is a campaigning organisation, which is fine. They are campaigning for a particular ideological version of secularism - which is also fine.
But they should abandon any pretence, claim or aspiration to be fair and objective in their reporting.
I thought I would also put the case to the survey authors.
What do the Authors of the Survey say?
I thought I would run my questions past YouGovStone, who designed the original survey. This is what Oliver Rowe, their Business Development Director said:
Thanks for your email.
As you point out, there are some key differences between the responses from the YouGovStone panel of influentials and the census:
- the YouGovStone panel are senior and influential people from politics, business, media and the arts and therefore are very different from the broad UK population
- the questions asked on this survey are not the same as the census
- data collection methodologies are completely differentTherefore while people may wish to compare the survey results with census data, the results are not comparable and it would be a mistake to do so.
Regards
Oliver
Oliver Rowe
Business Development Director
About that Census
Since it came out the NSS have been whinging complaining about how the census generates “Figures On Religion Ludicrously Exaggerated“. Apparently asking people “what is your religion” is unclear, while an answer of “none” to that question scares off the non-religious.
3. The “none” option was presented in an unnecessarily bald way which put non-religious people off ticking it.
Poor diddums.
Quite why they are quoting statistics that they have previously excoriated is an interesting question. I suspect it has to do with the NSS’s distinctly “Blairite” approach to the presentation of facts.
Wrapping-up
As usual, when talking about the NSS, we need to remember, we need to remember that their particular version of “secularism” (a word which itself has problems of definition) as advocated by the NSS is both aggressive and (very) explicitly atheist.
So any attempt by the NSS to claim or imply that those filling in a survey saying they have “no religion” support its own position is invalid.
Those supporting a secular state are not the same as the much smaller group who support a French style purging of religion from public life.
[tags]yougovstone, nss, national secular society, nss follies, secularism, religion[/tags]











‘They’ say “lies, damned lies and statistics” and ‘they’ are not far wrong.
There are many dodgy sets of stats being regularly bandied about on almost any subject you care to mention, especially concerning obesity and alcohol.
They are reported un questioned by the MSM, who appear not to understand the basics about stats, to a public that are often hard pressed to calculate a percentage.
Well taken apart, Matt.
Yes, I enjoyed your analysis.
Spot on, the NSS deserve a reputation for misinterpreting statistics.