HT09: Is this media article consistent with others?

Through our How to “Research the Headlines” series, we’ve provided some simple suggestions to assist with your critical consumption of research reported in the media. Most of the time, we’ve focused on specific things to look out for when reading any particular media article. Given that many of us get our news online, it is of course easier than ever to access multiple reports about the same topic. If more than one media outlet has covered a story, how might the consistency of reporting, or indeed inconsistency, help us to better appraise that research?

Part 9: Is this media article consistent with others?

When a piece of published research is reported in the media, no matter what newspaper or website is covering it, it should always be based on the same primary source. That is, the articles should all rely on the original research paper to provide the necessary information for the story. We might therefore expect that different media reports of the same research should have a degree of consistency. In many cases that’s indeed true. But while consistency might seem intuitively advantageous, there are occasions where identical or near-identical reporting might actually be something to be aware of when digesting the story. Below are a few things, though it’s far from an exhaustive list, to look out for when comparing two or more articles about the same study.

Two media reports about the same research come to similar conclusions, but they are definitely different articles

This might be what we would expect in terms of how research should be reported. The reporter named at the top of the article has read the original research paper, understood it, and has then produced their summary of it. Two different reporters might highlight slightly different aspects, or focus on different areas, but overall we might expect that they get to a similar point.

If they’ve spoken to the researchers, they might have slightly different, though similar, quotes, and they might have brought in different independent experts to also comment on the piece (you can read more about quotes from the researchers and others in earlier How to “Research the Headlines” posts).

This kind of consistency is possible when the research being reported has a clear, understandable message that has been well-described in the paper. Given the nature of research, that isn’t always the case unfortunately, and reporters can often have a difficult task of making something coherent from the published paper.

Two media reports on the same research come to quite different conclusions

But they’re based on the same piece of research, so why might that be? Well, it might be that the research reported handled a number of things within a single paper, so it might not be inconsistency per se, but just that the reporters have described somewhat non-overlapping parts of the research.

Or it might be that the research has not been clearly articulated, or is confusing in its conclusions. It wouldn’t then be surprising that different readers might take something different from it.

Another possibility is to do with the news outlet itself. One of our earlier How to “Research the Headlines” posts looked at the ways in which bias of a particular paper or website might flavour their take on the research. It’s always worth bearing in mind that wherever we go for our news is likely to have bias at one level or another. Sometimes inconsistencies in reporting actually help us to identify this.

Two media reports on the same research are very similar

This might sound like the first category above, but what we mean here is when two reports not only come to the same conclusion, but they say it with very, very similar words. That might suggest that the articles have been based in whole or in part on a press release describing the research. That can be fine, as press releases often do a good job summarising the paper minus the jargon, and is usually compiled by the research team, the university where the research was conducted, or by the journal where the research has been published.

However, uncritically using the press release can sometimes lead to problems if the press release is itself a little overly optimistic in its appraisal of the study (or is in fact a downright exaggeration). At Research the Headlines, we’ve discussed cases where reports in the media have been based on somewhat problematic press releases (our most recent example considered a study linking overnight phone charging and obesity). It can lead to stories where important issues with the study are not properly discussed, or indeed raised at all. This is where researchers have a really important role to play. A press release is a really useful tool, but it is also open to misuse, whether intentional or not. As those writing a press release have a vested interest in getting their research widely cited, using them as the sole source of a media report is best avoided.

Two media reports on the same research are identical

This is related to the point above, but as suggested, the two reports are not just similar, but word-for-word copies. Again, these words will usually have come from the press release (though there are cases where one media outlet might just copy from another!), so if that’s a fair and accurate reflection of the study, minimal harm done. The caveats about press releases above apply.

It’s understandable why such articles appear – it’s an easy way to fill some space (and of course, using press releases word-for-word is not exclusive to reporting research). The problem it hides, however, is the reliance on that technique rather than having specialised reporters providing a full and proper appraisal of the research. As researchers, we need to be aware of this when compiling our press releases, and ensure we carefully choose our words (and ensure they’re not lost in edits by university of journal press offices).

So, next time some research being reported in the media takes your interest, spend a few minutes looking at other newspapers and websites to see how consistently it has been covered. Very consistent or inconsistent reporting might both affect how you ultimately view that research.

This How to “Research the Headlines” was brought to you by Alan Gow. Access all of the How to “Research the Headlines” guides here.