HT08: No study stands alone

Here at Research the Headlines, we frequently write about the findings of a single study that have gripped the media. This attention comes about because the findings differ from what the general public commonly think or because they refute a large body of previous research. Often the findings have been the subject of a lot of media focus because they have implications for the health of the general public. It is therefore particularly important to know how to appraise such media coverage.

Part 8: No study stands alone – research findings must be put in the context of the wider body of research.

While a single study can provide evidence for a claim, it should be viewed as one piece of evidence within the broader context of the research literature in that area. The findings may have been influenced by factors specific to the characteristics of the sample under study which can be determined by a particular age, ethnic or social group.

We previously wrote about headlines written in relation to a breastfeeding study which included breast milk is ‘no better for a baby than bottled milk and it INCREASES the risk of asthma’ and ‘Hold the guilt! New study finds benefits of breastfeeding dramatically overstated’. In this piece we cautioned against disregarding decades of research in favour of the findings of one study. There were other issues too, such as overzealous and indeed inaccurate headlines. But we also emphasised the need to interpret the findings more broadly in relation to the existing literature on the topic.

We emphasised the need to put the findings of a study in the context of the broader literature in a post about the rise and fall of fish oils. Indeed here the media suggested overturning all we knew about fish oils based on the findings of a single study running with headlines such as “The great superfood U-turn”. The study focused on cognitive decline alone and so taking the findings in isolation of studies that look at other health benefits of eating fish leads to a biased interpretation.

Individual studies can be viewed like pieces of a jigsaw – each provide specific critical evidence for or against some claims but should be put together with other pieces to make sense out of them. It is particularly important to bear this in mind when a single report suggests a miracle cure or something of that nature. You need to ask yourself: ‘does this fit with the wider literature and, if not, why not?’

This How to “Research the Headlines” was brought to you by Sinead Rhodes. Access all of the How to “Research the Headlines” guides here.