On scientific ignorance in politics

In this month’s issue of Nature, William J. Sutherland, David Spiegelhalter, and Mark Burgman – three scientists in the fields of Zoology, Mathematical Sciences, and Biosecurity Risk Analysis, respectively – address the tricky problem of the interpretation of scientific research by interested non-specialists, specifically politicians.  In their comment article, they argue that, rather than expecting politicians to learn the science, or scientists to get involved in politics, the ‘imperfect nature of science’ should be better understood by all.  They propose 20 concepts that the non-specialist reader should be aware of when judging the implications of, or the actions that should result from, a particular piece of scientific research.

Knowledge of these concepts can be put to good use when analysing research in any field, not just science.  Indeed, many of the real examples the authors use to illustrate each concept come from areas beyond basic science e.g. airport security policy, the risk of subprime mortgages and the reliability of school league tables.  We at Research The Headlines would agree with the relevance of many of these concepts and have been putting that knowledge into practice; to pick one example, the concept the authors call ‘Bias is rife’ was one of a number of analytical tests used by Karen Lorimer in her post ‘Thinking beyond ‘risk’ narrative when it comes to young people and sexual behaviours’.  Karen pointed out that the results of a survey carried out by a website that helps students meet up for sex can hardly be said to have drawn its data from a broad cross-section of the student population, or to have produced results that can be applied to students in general (despite how it was subsequently reported in the media).  If you are interested in using these approaches to evaluate research for yourself, the article can be found here.  We would also recommend visiting the website of Sense About Science – a UK charity with an ongoing campaign to address the public understanding of science and evidence.

So what may make a difference in politics?

Critical analysis of research is a skill that should be honed by all politicians and policymakers who need to make informed decisions and to justify them.  The authors of the Nature article argue that for most policies under consideration by government, there is ‘rarely’ an experiment that has already addressed all the relevant issues with unequivocal results.  That being said, we note that many people are pushing for policymakers to address part of this information gap via custom-designed, randomised controlled trials of public policy.  For example, the Behavioural Insights Team of the Cabinet Office, in collaboration with Dr Ben Goldacre of Badscience and Prof David Torgerson of the University of York published in June of last year a paper setting out how this can be done.  Their paper can be freely downloaded here.

The hope is, therefore, that with improved analysis of the available published research and by addressing the gaps in the literature via well-designed trials, future governments can make better-informed policy judgements.

William J. Sutherland, David Spiegelhalter, and Mark A. Burgman, (2013) “Twenty tips for interpreting scientific claims,”. Nature, 503, p335, 21st November 2013