Can eating chocolate help you lose weight?

Of course not…

Back at the end of March last year, a wide selection of news outlets, including in the UK the BBC, Daily Mail, Daily Express and Daily Star, all reported a new study published by German researchers which suggested that consumption of good quality chocolate as part of a controlled diet could aid weight loss.

The study was published by Johannes Bohannon, research director of the Institute of Diet and Health, in the International Archives of Medicine. The trouble was that the whole exercise was intended to show up serious issues with the conduct and reporting of nutritional studies, in addition to problems with the peer review process (or lack of it) in certain open access journals.

Journalist John Bohannon (the real person behind Johannes Bohannon) describes what he and his collaborators did, and what the result was in this fascinating article for io9. While it was a real study involving a small number of volunteers, the results demonstrated a key problem with the reporting of statistical significance (p-values) when the number of participants in a study is very low. The io9 article does such a good job of describing what went on that there’s really no need to go into in great detail here. Suffice it to say, the study design and subsequent analysis had very specific problems planted within it that a rigorous review process (or subsequently a good journalist reading that study) should have spotted. Do read the full io9 article though, as it gives a number of pointers to bear in mind for the next time you’re reading about the latest dietary quick-fix.

Since the study was revealed to be seriously flawed (as it was designed to be), some sources that originally reported the story have published corrections, while most have ignored it. The International Archives of Medicine have tried to get out of a sticky position by insisting that the paper wasn’t actually published at all! Apparently it was published temporarily “by mistake” while still being in the review process. This despite allegedly collecting the publication fees from the authors and sending emails stating how “outstanding” the manuscript was. The paper has disappeared from the journal’s website but as you might expect, that doesn’t mean it’s gone for good. You can read the original research here.