HT06: How to assess the risk?

Part 6: How to Assess the Risk?

Some of the most common forms of media story that catch our eyes at Research the Headlines are those that report studies of risk, particularly in terms of lifestyle choices that have been shown to increase the likelihood of developing disease.  Unfortunately this is also where the media can often get it wrong in a way that causes unnecessary concern to many people.  In the chapter entitled “Bad Stats” of his excellent book Bad Science, Dr Ben Goldacre states:

Newspapers like big numbers and eye-catching headlines. They need miracle cures and hidden scares, and small percentage shifts in risk will never be enough for them to sell readers to advertisers…

In this latest part of our How to “Research the Headlines” guide we look at how risks are often reported in the media and why these can be misleading.

To give some examples of the kind of stories we’re talking about here are a number of excerpts and headlines culled from recent Guardian and Daily Mail stories (that should give some balance right?):

Guardian: Individuals who used their mobiles for more than 15 hours each month over five years on average had between two and three times greater risk of developing glioma and meningioma tumours compared with people who rarely used their phones, they found.

Daily Mail: Three bacon rashers a day ‘raises breast cancer risk for young women’: Frequently eating red meat can raise chance of developing disease by more than 20%.

Guardian: US researchers found that a higher intake of red meat was associated with a 22% increased risk of breast cancer overall, with each additional serving per day of red meat associated with a 13% increase in risk.

Guardian: US scientists who tracked the progress of around 186,000 women aged 50 to 71 found that those who smoked were 19% more likely to develop breast cancer than those who had never smoked. Women who once smoked but then kicked the habit were still 7% more at risk.

The main problem with reporting these relative changes in risk is that they give the reader no idea of what the risks might be without the particular lifestyle factor involved.  While the idea that using a mobile phone for too long makes you two or three times more likely to develop a certain form of cancer sounds shocking, it’s very unhelpful if you don’t know what the chances are of developing a brain tumour is in the first place.  In their best practice guidelines for reporting on health stories, the Science Media Centre suggest that:

On health risks, include the absolute risk whenever it is available in the press release or the research paper – i.e. if ‘cupcakes double cancer risk’ state the outright risk of that cancer, with and without cupcakes.

Imagine that eating too many cupcakes increased the chance of someone suffering a certain type of cancer by 50%. That sounds like cupcake consumption is a major risk factor, without the context to judge that properly. If the risk of developing the cancer (without eating cupcakes) is only 0.2% for example, that means the increased risk is now 0.3% (a 50% increase on the no-cupcake 0.2% risk).  While we would still understand that there is an increased risk, stating the absolute value is generally much less sensational.  Dr. Goldacre would recommend going one step further by not talking about percentages (which many readers and writers have trouble understanding properly) and instead use the “natural frequency”.  This means stating the increase in terms of the extra number of events per 100 or 1000 people.  So, if on average for every 1000 people, two will suffer from cancer, but for those who eat a cupcake every day there will instead be three, then it would be better for the journalist to say “Cupcakes can lead to an extra one case of cancer for every 1000 people.”

Unfortunately, although Ben Goldacre writes for the Guardian, they still seem to be guilty of using relative risk where the absolute risk or, even better, the natural frequency would be more appropriate, informative and less likely to worry people.  If the absolute risk is not not mentioned in the article (an example of better practice of this from the Daily Mail) then it is often very difficult to find without going back to the original source, and so we need to continue to tell the newspapers to up their game.

This How to “Research the Headlines” was brought to you by Stewart Smith. Access all of the How to “Research the Headlines” guides here.