Hyping evidence harms trust in research

When we talk about research in the media, it’s often assumed we’re focussing primarily on the way journalists or the media might misrepresent solidly conducted and robust research. While that does happen, and we’ve talked about that in various posts at Research the Headlines, it’s important to remember the translation of new research from “lab to headline” is a process that involves a number of people. Inaccuracies can creep in at different stages and those can ultimately affect the way people interpret that evidence, or indeed trust the whole process. That’s just one of the topics considered in a new report from the Academy of Medical Sciences released last month.The report, “Enhancing the Use of Scientific Evidence to Judge the Potential Benefits and Harms of Medicines”, provides some really interesting figures that might be a bit of surprise to many people (it mainly focuses on information about the use of medicines, but is equally applicable to almost any area of research that could impact on the decisions people make in their day to day lives). One of the findings reported was that:

“only about one-third (37%) of the public said they trusted evidence derived from medical research, but around two-thirds (65%) trusted the experiences of friends and family.”

That suggests quite a gap between what we hope are well-conducted studies given the best possible evidence and how people actually choose to access health-relevant information.

What might we do about it? Well, as researchers we need to remember it is our duty to remain involved in the messaging of our research. We need to be clear about not ‘over-egging the pudding’, but also in ensuring exaggerations don’t creep in later in the pipeline, from the press office for example. That means we need to retain final approval of those press releases, and importantly be available to discuss our new research with the media to try and ensure the clearest message is communicated. Of course, press offices have a key role to play too, in that they might sometimes need to reign in that over-enthusiastic researcher who’s extrapolating a bit too far from what their study can actually show! As with the whole process, good lines of communication are key. To read more about the pitfalls of press releases, have a look at one of our previous posts: Lies, damned lies and statistics (about press releases)

For members of the public, it can sometimes be difficult to know how to interpret new research being reported, for example, how do we know if it’s trustworthy? As we discussed in our How to “Research the Headlines” series, it’s important to remember that “No study stands alone”, so always look out for references in a media report to how the new findings fit (or don’t) with what is generally known about a topic. And look out for independent experts being quoted who often add a few caveats (that’s not to say the researcher themselves didn’t highlight these limitations, it’s just quite common in media reports to have those coming from someone else).

Another key message, in fact one of our easiest tips to follow, is “Don’t stop at the headline”. That can often be where simplifications to provide a catchy summary creep in. For example, imagine if a new research study found a compound commonly found in red wine, let’s just call it Factor X, was associated with improved memory performance in mice. It wouldn’t be unusual to see that translated as “Drink red wine to avoid Alzheimer’s”! Only by getting into the meat of the report will you hopefully be able to spot some of those things. The researchers again have a role to play in keeping those caveats clear, but we need journalists and the media to work with us to get those messages across.

One possible route that the report suggests is a traffic light system for press releases to accompany any new research. Just like the system used by many supermarkets on our food, the idea would be that red-amber-green ratings would help journalists and the public more easily assess how solid a piece of new research might be. Groups like NHS Choices, the Science Media Centre and Sense About Science are great sources of information as new stories break or in providing tips on how to better interpret those stories, and hopefully our How to “Research the Headlines” series will prove useful too.

The Academy of Medical Sciences report goes into many more issues, but if you’re looking for short summaries, then do consider checking out the videos that accompany the report or the short overview as a starting point.