Our blog’s typical pieces cover how science headlines in news outlets tend to favour sensationalism over accuracy in the reporting of scientific studies. This time instead, I am going to take the opposite approach and cover a scientific paper about media reporting which received relatively little media coverage. This study, conducted by a team at the University of Bordeaux, looked at more than 5000 articles about risk factors for different categories of diseases and how their results were covered in the media. The study analysed the extent of media coverage distinguishing between the initial results versus following follow up studies. The key message of this French study is not really surprising, yet quite revealing, even for us at Research the Headlines who are used to deal with this phenomenon.
The analysis found that media coverage increased with the prestige of the journal that published the article, especially when it was accompanied by glossy press releases. The most consistent trend was the high coverage for studies that reported positive associations for risk and protection for diseases. In contrast, lack of associations for different conditions received basically zero interest. This was true also for follow-up analyses which failed to replicate the positive associations reported in the original studies, and which were only rarely covered in the media. In fact, of the 156 studies reported by newspapers that initially described positive associations, only about 50% received support in subsequent studies.
This means that news outlets selectively choose to write up stories about curing diseases with diet and life style and to tell us about catastrophic effects of eating bacon and burned toast, but they are silent when the same effects find pretty much no support in independent studies. The recommendation of this paper is that journalists should make a bigger effort in reporting null findings particularly when they invalidate their original stories. This study tells us that, in about 50% of cases, reports about biomedical research are wrong. Before getting too excited about new research findings then, and perhaps before running stories about research, journalists could wait for independent validations or at the very least note that these are not available at present.
Quite disappointingly, in line with its own findings, this research paper, that describes pretty much negative results (i.e. lack of coverage for null effects), did not receive much media attention. According to Altmetric this study received good Twitter attention (>350 shares) but was covered by only four news outlets.
But journalists are not the only culprits. Scientists should be more cautious when preparing their press releases and aim to indicate the level of reliability of their findings. In fact, the final conclusion of the study urges scientists to take responsibility for communicating their own work. In the words of the authors:
they [scientists] are responsible for the accuracy of the press releases covering their work … This is not only a moral duty but also in the interest of science. Indeed, contrary to common beliefs, reception studies show that scientists are viewed by the public as more trustworthy when news coverage of health research acknowledges its uncertainty, especially if this is attributed to scientific authors.
“Poor replication validity of biomedical association studies reported by newspapers”, Estelle Dumas-Mallet, Andy Smith, Thomas Boraud and François Gonon. PLoS One, Published on February 21, 2017 http://dx.doi.org/10.1371/journal.pone.0172650
When we started Research the Headlines all the way back in 2013, we aimed to explore the relationship between academic research and the media, to help the general public know when the media was doing a good (or bad) job of portraying the latest discoveries about the world around us.
In 2017, we now face a new challenge – the so-called “fake news” phenomenon. We’ve seen the fragmentation of news media allow for extreme bias in media outlets. The speed at which the news cycle now operates allows websites to break a story based on false accounts, photoshopped images or bad data, and never be corrected. Worst of all, the way we consume news means that it can be very hard for us to see unbiased journalism, as we have carved social media bubbles for ourselves.
The Research the Headlines team can only address so many news stories, as our time and resources are limited. That doesn’t mean there is no hope – a piece of fake news is doomed to fail if the reader is vigilant.
We wrote a how-to series to help readers decide whether a news story did a good job depicting the research it was reporting. If you need to decide if a story is fake news, you can use the same techniques here:
1: Don’t Stop at The Headline. It is common to see headlines that have little bearing on the actual news story. A study indicated that as much as 59% of shared news stories on social media had not been read by the person doing the sharing. Make a point not to share a story unless you’ve read the entire article.
2: What Was Actually Said? At RtH, we’re usually interested in what researchers say. More generally, if a story is about a person or organisation, we should be sure that they actually said what is being claimed in the article. Remember quotes can be taken badly out of context. If they don’t reproduce the original statement in full, can you find it somewhere else?
3: Are Independent Experts Featured? A piece of fake news is very likely to avoid asking “the other side” for comment. On the other hand, some outlets pay slavish attention to “balance”, by asking for comment from both sides. The word “independent” becomes crucial here.
Take for example a climate change story, where climate scientists and climate change deniers share a platform. Experts who deny climate change can have a conflict of interest which prevents them from being independent. Contrary to some opinions, climate scientists are not espousing climate change as a means to gather grant funding. Climate scientists would still have a job even if humans were not causing climate change, and their independence is guaranteed by how they are funded.
4: Do they link to the original material? It is very easy to distort the narrative (or tell plain lies) if the original material is hidden from the reader. If you see a story that strongly provokes an emotional response in you, your first response should be to go to the source of the story. A news outlet that hides the material (or worse, corrupts it), is an outlet to avoid.
5: Do they make unsupported causal links? Every researcher knows that “correlation does not equal causation”. Correlation, an association between two things, is only the first step in identifying a cause. On subjects that are highly emotive (like the supposed “link” between immigration and terrorism), ask yourself – is the relationship real? Is the conclusion they draw from the correlation supported by the evidence? Is there another possible reason for the correlation? How would we test their conclusions?
6: Are they using statistics appropriately? The old adage “lies, damned lies and statistics” still holds today. If used properly and dispassionately, statistics is a powerful tool for revealing the truth of our circumstances, and a crucial component of the scientific method.
Used improperly (or malignantly), statistics is a powerful weapon of propaganda. Whether statistics are being used badly is difficult to diagnose, and often requires formal training. Our how-to gives some examples of how risk is often distorted in the media. If your mathematical skills aren’t up to it, the only other defence against bad statistics is to see if they are repeated in other articles (see 9). You can also ask a scientist, even if they aren’t experts in the specific field, whether the statistics is valid. There are lots of scientists on Twitter (try the #ActualLivingScientists hashtag), and most are happy to answer genuine questions.
7: Is the article or outlet biased? As far as the UK print media is concerned, the answer to the latter question sadly is almost always “yes”. The lack of British-made fake news outlets is apparently due to the deep polarisation of our long standing institutions. Our How-to tells us about some of the different indicators of bias. It’s also worth remembering that all humans suffer from cognitive bias – when we make judgements about the news (and other people), we are making these judgements subject to various forms of illogical thinking.
When you read a story, think: “How much is this simply confirming my own biases on the topic? How would I have responded if the article had said the opposite?”
8: How does the story compare with previous stories on this topic? When it comes to research, each new study needs to be taken in the context of the field it comes from. One study, with a result that shatters the consensus of the past, must initially be treated with skepticism until its conclusions can be confirmed by further work.
The same is true for news stories. Do the facts in the story fit with the facts given previously? A ringing example is Donald Trump’s statement that he was present in Scotland before the Brexit vote, when in fact he arrived the following day. In our post-truth world, even the President of the United States is happy to attempt to revise history. Thankfully, we have the internet to stop him (and anyone else) from doing so.
9: Is this article consistent with other articles? A really great litmus test of an article is to see its story told from a different viewpoint. In the UK, it can be as simple as reading the Express followed by the Mirror. Try reading the same story in an outlet you would normally avoid. How does the story change? What words does each side use? Who do they ask for comment? Who do they ignore?
10: How much of the article is opinion, or exaggeration? With the increasing blurred boundaries between news stories and writer’s columns, an article can have the sheen of news, but the raw opinion of the writer. When reading the story, ask yourself: “How many of these statements can be verified by the facts in the article (or links in the article)? What sort of language is being used? Does it seem like the author is advancing an argument rather than reporting the facts?”
The use of e-cigarettes has exploded in the last ten years, with some 2.2m people using e-cigarettes or vapourisers. This shift towards “vaping” is expected by some to have health benefits over using tobacco cigarettes, as many of the carcinogens in tobacco and the cigarette’s filter papers are not present in the aerosol mix that vaping provides the user.
A good deal of research confirms the view that switching from tobacco to e-cigarettes does carry net benefits, as we explored on RtH all the way back in 2014.
But vaping is still a new phenomenon. The dangers of tobacco took decades to identify, and required researchers to see the full life cycle of patients who smoke to find links between tobacco smoking and increased risks of lung cancer, cardiovascular disease and many other ailments.
A new study claims to provide some of the first evidence that vaping does possess its own health risks, particularly to the heart. But how can they show this after only 10 years?
What the Researchers Did
The team, led by David Geffen out of the School of Medicine at the University of California Los Angeles, were aware of the limitations of the current data. To link vaping to possible damage to the heart, they looked for symptoms that typically lead to increased “cardiovascular risk”, i.e. the chances that a patient would develop cardiovascular diseases later in life.
They looked at two classes of symptom – “oxidative stress” (essentially an imbalance between oxidising free radicals in the body, and anti-oxidants), and a change in “cardiac autonomic balance” (a measure of the stability of the heart rate).
Both of these phenomena are common in tobacco smokers, and are indicators for later myocardial infarction (heart attack). Generally, high oxidative stress also increases the risk of cancer.
The researchers reason that if these phenomena are associated with vaping, then vaping will also increase the risk of cardiovascular problems like heart attacks later in life.
They therefore studied a sample of 23 participants who were non smokers, but habitual vapers. The participants were otherwise healthy, and ranged from 21 to 45 years old. They were compared against a control sample of 19 participants who neither vaped nor smoked.
The subjects had electrocardiograms taken to measure their cardiac autonomic balance, and blood samples drawn to measure oxidative stress. In both cases, increased ecigarette use was correlated with increased exacerbation of these symptoms.
The authors admit that their study relies on self reporting of both smoking and vaping habits. They had asked the participants to refrain from vaping a day before the test, but were able to detect cotinine in the blood plasma in several participants (indicating recent tobacco use), and these participants were eliminated from the test. Previous studies show that smoking tobacco the day before or on the day of measurement can have significant effects.
It also isn’t clear how much vaping is needed to create this increased risk. In smokers, the risk leaps dramatically beyond some threshold number of cigarettes per day, although light use of tobacco is still dangerous. The fact that vapers can control the nicotine flow depending on the equipment they use makes this harder to measure.
Finally, it’s worth noting that this effect is only measured in a sample of 23 (relatively young) people. The statistical significance of some measures is quite weak as a result, so this work needs following up with a much larger study over a longer time period, with a wider sample of people at all ages.
What did the Media say?
I’m pleased to report that the media coverage of this story (at least in the British papers) is on the whole faithful to the original study.
This Metro article gives a fair account of the study, and even gives some handy information on terms like oxidative stress. Unfortunately, it does not link to the article, and the authors are not reached for comment (although we do applaud their publishing of commentary from the British Heart Foundation).
The Mirror reaches the authors for comment, and also asks the UK Vaping Industry Association (UKVIA) for their thoughts. They rightly note that tobacco and vaping are not directly compared in the study (although this is not crucial for measuring vaping’s negative effects). Further, the UKVIA claim that the results are comparable to those achieved by drinking a cup of coffee, or watching a scary film. It’s interesting to note the cardiovascular risk from a single cup of coffee per day is reduced by drinking slightly more coffee. It’s unclear at this stage whether vaping will show a similar relationship.
The Sun’s coverage of this story is studded with quotes from a range of scientists (some in the study, some not). Their piece is informative and wide-ranging – my only concern is the lack of links to the journal article.
The Bottom Line
This study is not the ironclad evidence we are used to for tobacco smoking risks. That will come once habitual vaping has existed in human populations over an entire generation. Most evidence suggests that vaping is significantly safer than tobacco smoking, and that switching to vaping poses benefits relative to smoking.
That doesn’t mean vaping is safe. The vaping industry lacks the regulation given to tobacco smoking. Studies like this are the first indications that there are possible health risks from vaping, and we will need more research to confirm them.
Like smoking, vaping is a lifestyle choice, just like alcohol and caffeine. We have to make sensible choices about the risks associated with our diet and general intake.
We know the risks of tobacco, and overconsumption of alcohol. The benefits and risks of vaping are becoming clearer as studies like this emerge. This information is crucial for helping us regulate vaping products, and finally for customers to make informed choices about what they choose to buy.
Volcanoes are fascinating and people are drawn to them. My classes this week have involved discussing with the students the biased nature of the geological record and how important it is that we understand how the nature of reporting volcanic eruptions has changed with time. We also touched on the possibility that in recent times more eruptions have been recorded as our technology has improved and today we have a more mobile population. The in’s and out of this discussion is a debate within itself. What we concluded was that the record was biased and we had to take care to accurately report eruptions.
At the same time, on my Facebook timeline, I could see colleagues becoming annoyed about some of the UK media reporting of the Colima eruption, in Mexico. Colima volcano has been erupting periodically for years (http://ciiv.ucol.mx/activity_of_volcan_de_colima.php) and is of interest to many researchers in the UK.
In the last week, there has been some interesting articles popping up in the UK press notably, The Express and The Mirror. These reports were being shared on my Facebook timeline not for the eruption, but for the quality of reporting of the UK media. So much so, the Centre de Intercambio e Investigación en Vulcanología (CIIV) has even commented on their Facebook profile about the poor quality of reporting. Many of the items discussed below are also commented by the CIIV on their Facebook page and also by visitors to their page.
The article in the Mirror leads with a headline ‘Massive volcanic eruption near Mexico City…’ but further down the article the location has changed to near Mexico’s west coast. A simple google search would easily clarify the location for the author and would indicate Colima is around 470 km from Mexico City. Certainly large eruptions can affect cities 470 km away, as we witnessed in 2010 with the Eyjafjallajökull eruption affecting much of northern Europe. However, the location of the volcano is normally well known today with our modern technology, even when located in the most remote parts of the world.
The Express article at least reports a few details, such as the height of the eruption plume although they still believe Colima is relatively close to Mexico City. The article goes onto state, “If ash does fall, residents have been advised to cover their bodies in water so they do not become contaminated” which is also highlighted in the headline. I suspect this is not the advice that has been issued. The reason why? Ash is extremely fine grained volcanic glass. So think of a bag of flour that is composed of small glass shards. Inhalation of such material is bad, which is why during eruptions you will see pictures of people covering their mouths and noses. However, if you mix ash with water (rain, river, moisture in your breath for example), the result with sufficient water turns the ash into a sludge like cement mixture, which if you inhale can lead to dire consequences.
If we continue through the Mirror article they go on to say:
“There is no confirmation at present whether an evacuation is underway, but the scale of the blast suggests this is imminent.”
It is uncertain where this information has arisen from and if we were to just consider the UK reporting of this volcanic eruption, it would appear that the area around Colima is in panic. According to the Facebook page of CIIV:
“Life continues as usual but with a close eye on the volcano to see if things increase further.”
The volcanic activity at Colima does seem to have increased in the last month, but for the time being (at the time of writing) life is continuing as normal to the local inhabitants, perhaps with a closer eye on their volcano, 470 km away from Mexico City.
It hasn’t been all bad reporting this week. There is a lovely piece in The Telegraph about the Smithsonian Volcano Database, which is part of the Global Volcanism program. According to the Global Volcanism Program Facebook feed, they had no input into this article highlighting the good quality of reporting that can be achieved.
The Smithsonian Volcano Database holds a catalogue of all known volcanic eruptions in the Holocene (last 11,500 years). Globally we believe over 1500 years have erupted, but within historical times (written history) humans have only witnessed eruptions from about 500 volcanoes. It was this catalogue of eruptions that I was discussing with my students, as it is a brilliant resource in our lectures this week. The catalogue is biased to eruptions that occur on land, and many of those that occur underwater are not recorded. The further back we go in time the more uncertain some of the records become. The piece even talks about the UK’s volcanoes, although none on our mainland are active any longer such as Arthur’s Seat in Edinburgh (see photo), but there are active volcanoes in the British Oversea territories.
Photo of Salisbury Crags part of the volcanic complex of Arthur’s Seat in Holyrood Park, Edinburgh.
The Smithsonian Volcano database and the information in it, is fundamental if we are to understand trends in volcanic behaviour over time. For anyone interested in volcanoes I would highly recommend this article and to go and have a look at the Smithsonian Volcano Database.
The emergence of extremely drug resistant bacteria could prove to be the first of many.
In the last week, two publications have appeared in the press describing the worrying emergence of bacteria that are resistant to many of the drugs we normally use to treat such infections. A lot has been written in the last few years regarding the threat of antimicrobial resistant infections, but it is clear, the more we look for these infections, the more we find them and the more dynamic the organisms causing them appear to be. Drug resistant infections are a global problem and the movement of people around the world can lead to the dissemination of drug resistant bacteria and both of these reports highlight this as a problem.
The first report was In the USA Centre for Disease Control and Prevention in their ‘Morbidity and Mortality Weekly Report’, which was described widely in the press (Daily Mail, The Independent and the BBC Website).
The report indicates that the patient contracted a strain of a bacterium called Klebsiella pneumoniae which resulted in sepsis and death. The bacterium that caused the infection was resistant to all available antibiotics used to treat systemic infections in the USA (26 different drugs), although it was sensitive to an antibiotic called fosfomycin – this drug is not licensed in the USA for delivery via injection into the bloodstream (intravenous delivery). In several countries in Europe (including the UK) fosfomycin is licensed for intravenous use in cases such as this. This is important because we are seeing increasing numbers of drug resistant infections and this is one of the first cases for Klebsiella where no drug options were open to the medical staff. The report also highlights that appropriate infection control measures, such as testing the drug susceptibility of the bacteria causing the infection, isolation of patients and good knowledge of previous medical history, are vital for slowing the spread of drug resistant infections.
The second report in the Daily Mail described a study published in the US journal Proceedings of the National Academy of Sciences that took an innovate approach to trying to understand the spread of a group of bacteria known as carbapenem resistant Enterobacteriaceae (CRE). The researchers, led by Bill Hanage at the Harvard University Chan School of Public Health, found that CRE are much more widespread, are much more diverse than previously thought and can be transmitted from person-to-person without causing disease – something often referred to as asymptomatic carriage. The researchers used very detailed analysis of the genomes of many bacteria isolated from the Boston area and also in California to come to their conclusions. Significantly, the majority of studies normally focus on treating infections and profiling the bacteria that caused those infections. This study however showed that these bacteria are in the community and tried to characterise the bacteria that are being passed from person to person, without causing disease. This is important as many bacteria are opportunistic, they cause disease in patients that may have other underlying medical conditions. If these opportunistic bacterial infections are drug resistant, then this can present a significant challenge to doctors and how they treat the infection (See the case discussed above). Worryingly, the researchers also identified new kinds of drug resistance in these bacteria, suggesting that there are more antimicrobial resistant bacteria and ways to be resistant out there and yet to be discovered.
Both of these studies were accurately reported in the press and highlight the real danger that antimicrobial resistant infections pose on a global scale. The silent spreading of drug resistant bacteria in the community may also be a bigger problem than previously appreciated suggesting that we need to increase surveillance and be vigilant during outbreaks.
We have previously discussed television soap opera coverage of sensitive issues, such as mental health that was addressed in EastEnders last year. In recent days, another soap, Coronation Street, has been covering the highly sensitive issue of late miscarriage.
Long standing actress Kym Marsh, who plays Michelle was seen on Wednesday night’s episode giving birth to a baby who was not breathing at 23 weeks. The programme is to be commended for the evidence-based approach they took to covering this important and rarely discussed issue.
The programme producers researched the issue of late miscarriage by connecting with Sands, the national stillbirth and neonatal death charity, that was established by bereaved parents in 1978. A bereavement support and awareness specialist, Erica Stewart, at Sands, was quoted in the media:
We’re pleased to have been approached by the researchers and writers at Coronation Street for advice and help to ensure that this heart-breaking storyline, that will see character Michelle have a late miscarriage at 23 weeks, is portrayed truthfully and sensitively.
Accurate media coverage of late miscarriage is extremely important. Up to 24 weeks, the loss of a baby is referred to as a miscarriage. Many people experiencing a late miscarriage, and giving birth to a baby as we see in the programme, are unaware of this. It is only from 24 weeks that the loss is classed as a stillbirth. Coronation Street chose to cover the stage of pregnancy that is closest to this timeline raising important awareness about pregnancy loss at this stage.
Miscarriage is extremely common and unfortunately the topic is still very much taboo. Both parents involved in the coverage, Kym Marsh and Simon Gregson, have themselves had personal experience of miscarriage and have spoken in the media about this ‘silenced’ issue. Because of this silence, most women who experience a miscarriage, unless it is a repeated experience, have very little knowledge or awareness of the emotional, physical and procedural issues involved.
Sands are quoted in the media as stating that
We hope that with a TV drama as popular as Coronation Street covering this heart breaking experience, it will help to lift the taboo, and raise awareness of all the issues that surround the death of a baby.
This has clearly been achieved, with coverage that refers to Sands and in many instances provides accurate facts about miscarriage and stillbirth evident in a range of media outlets, including The Telegraph, the BBC, and the Daily Mail.
Children’s use of technology, from iPads to playstations, television and internet exposure, is frequently discussed in the media. Here at Research the Headlines we have discussed several examples of such media coverage – an example from September 2015 is reblogged below. It was very welcoming then, to see a letter published in the Guardian last Friday signed by a group of psychologists (which includes 3 of the experts we interviewed in our Talking Headlines series, Dorothy Bishop, Suzi Gage and Kevin Mitchell) and other child development experts raising concern about how screen time guidelines ‘need to be built on evidence and not hype’. This letter was a response to a previous letter published in the Guardian that raised concerns about screen time without drawing on evidence.
John Curtice is Senior Research Fellow at NatCen, Professor of Politics at Strathclyde University, and Chief Commentator on the What UK Thinks: EU and What Scotland Thinks websites. He has been a regular contributor to the British Social Attitudes Report series since 1986 and an editor since 1994. He has also been a Co-Director of the Scottish Social Attitudes survey since its foundation in 1999, and his analyses of Scottish public opinion in the run up to the independence referendum were frequently featured throughout the campaigns. John is a regular media commentator on both British and Scottish politics. He is also President of the British Polling Council. Read more…
By most measures, the last 12 months have been epoch-making. From the UK’s decision to Brexit, to the USA’s decision to elect Donald Trump as President, we have seen again and again the importance of the media in shaping and driving public opinion, and the waning importance of fact in our political discourse.
At Research the Headlines, we try to remain as unbiased as possible when evaluating the media’s skill in translating academic research into news stories, and we will continue into 2017. This year, we addressed topics such as ADHD, Tourette’s syndrome, dogs in shelters, how we measure the damage humanity is causing to the natural world, teenage suicide, dementia, asteroids and earthquakes.
Dr Phillip Williamson is an associate fellow at the University of East Anglia, employed by the Natural Environment Research Council (NERC). His science coordination work includes programmes on ocean acidification (completed), shelf sea biogeochemistry (ongoing) and greenhouse gas removal (planned), all co-funded by NERC and UK government departments. Misleading reporting on ocean acidification has led to Dr Williamson to become a strong proponent of countering poor reporting and misinformation in the media, as shown by his recent column in Nature.