Half of medical reporting 'is subject to spin'

Behind the Headlines

Wednesday September 19 2012

How accurate is the news you read in your morning paper?

A study that you probably won’t be reading in your daily paper or favourite news website anytime soon casts serious doubts on the reliability of mainstream medical and health journalism.

The study found that 51% of news items reporting on medical trials – specifically on randomised controlled trials (RCTs), which are seen as the gold standard in judging whether a treatment is effective or safe – were subject to “spin”.

 

What is meant by 'spin'?

To spin information is to distort the true picture to fulfil an agenda, often by presenting information in way that creates a positive or favourable impression.

The researchers defined spin for the purposes of the study as “specific reporting strategies (intentional or unintentional) emphasising the beneficial effect of the experimental treatment”.

Examples of medical spin cited by the researchers are:

  • Reporting positive effects that were not statistically significant – so that the effects could have been the result of chance.
  • Focusing on an outcome that the trial was not designed to study – for example, a trial that aimed, without success, to use acupuncture to treat hot flushes found incidentally that the treatment produced a slight improvement in sex drive. So the trial was spun with headlines such as “Acupuncture perks up sex drive”.
  • Focusing on inappropriate sub-groups – for example, a trial of a new type 2 diabetes drug might be a total failure in the population at large but show a slight improvement in women in their twenties. This can be spun as an important breakthrough. However, type 2 diabetes is rare in women in their twenties, so the new drug would not actually be of great use.
  • Ignoring safety data – we need to be sure that the potential benefits of treatment outweigh the risks but research summaries and press releases routinely omit mention of risks, side effects and so forth, and thus give an overly positive impression of results.

 

Where did the research come from?

The study was carried out by researchers working for a number of French institutions, including the Centre d’Epidemiologie Clinique, Beaujon University Hospital in Clichy and the Faculte de Medecine in Paris.

The study was published in the peer-reviewed journal PLoS Medicine.

No direct funding was received for this study. The salaries of the authors were paid by their respective institutions during the period of writing.

 

What did the researchers do?

The researchers used a news database called EurekAlert! to look for press releases relating to RCTs that were published over a four-month period.

They then checked another news database called LexisNexis to see what mainstream media coverage was generated by the press releases. They then went back to the original research summaries (abstracts) on which the press releases were based.

All three sources of information were then assessed by a panel of experts for the presence of spin.

 

What were the results?

According to the subjective judgement of that panel:

  • 41% of abstracts contained spin
  • 46% of press releases contained spin
  • 51% of news items contained spin

 

How does spin occur?

Based on the results, the researchers believe – though cannot prove – that there are three levels of spin at work.

First, at the abstract (summary) level. Leaving aside any deliberate spinning, many researchers may just unconsciously “sex-up” their report abstracts to present them in the best possible light.

If, for example, you were involved in a project that may have lasted several years, and were asked for a brief summary of your findings, it is likely that you would focus more on the positives than the negatives.

Second, at the press release level. Press officers for universities, research institutes or medical journals are under pressure to generate media coverage. And a lively, positive “breakthrough” will get more coverage than results that are dull and inconclusive.

Third, at the journalism level. Many journalists claim (with some justification) that they are over-worked and under-resourced so they simply read the press release (and some might read the abstract) before writing the story. The full study on which the press release is based is rarely read.

 

Why this is important?

It is estimated that 90% of the public get information on developments in medicine and healthcare from the mainstream media. So the quality and reliability (or lack of it) of medical and health journalism is vitally important in determining whether we get an accurate idea of medical advances.

At best, unreliable medical journalism can lead people to waste time and money on treatments for which there is no evidence of them being effective. At worst, it can kill.

For example, the unfounded link between the MMR vaccine and autism became a “health scare” perpetuated by large sections of the mainstream media from the late 1990s. Despite the lack of credible evidence to back up the link, frightened parents justifiably avoided letting their children have the MMR jab. Official statistics show that this led to a sharp rise in measles cases. While in most cases measles is simply unpleasant, in a small number of cases it can be fatal.

Between 1998 and 2008 there were 15 measles-related deaths reported to the Health Protection Agency in England and Wales. All of these deaths may have been prevented by MMR vaccination.

 

Things to consider

When you read a news report about a medical study, you may find it useful to consider:

  • Was the research in humans? Headlines that talk of a “miracle cure” often relate to research conducted on, say, mice – and the results may not apply to people.
  • How many people did the study involve? Small studies involving just a handful of people are more likely than large studies to reach conclusions that could simply be the result of chance.
  • Did the study actually assess what’s in the headline? As mentioned, a headline saying acupuncture boosts your sex-life was actually based on a study into whether acupuncture could treat hot flushes.
  • Who paid for the study? While most commercially funded studies are reliable, it is always worth checking if there could be any potential conflict of interest, for example where a company funds research into its own products.

Read more advice about how to read health news.

Conclusion

The study paints a picture of spinning at multiple levels, with around half of medical news stories being subject to deliberate or unconscious spin at some point.

Some researchers distort their abstracts which are then turned into inaccurate, “sexed up” press releases. The releases are then used to generate news stories for journalists who, in general, don’t read the original research.

Researchers often complain that journalists misrepresent their work, but if they are spinning the information that goes into the abstracts, then they are partially culpable for any misrepresentation.

Given the levels of spin found by this study, readers need to be wary of medical news stories and approach them in a sceptical frame of mind.

Edited by NHS Choices. Follow Behind the Headlines on twitter.

Links to the science

Yavchitz A, Boutron I, Bafeta A, et al. Misrepresentation of Randomized Controlled Trials in Press Releases and News Coverage: A Cohort Study. PLoS Medicine. Published online September 11 2012

Ratings

How helpful is this page?

Average rating

Based on 25 ratings

All ratings

Add your rating

Comments

The 3 comments posted are personal views. Any information they give has not been checked and may not be accurate.

Grinder206 said on 30 November 2013

While I agree that journalists spin content (and most content is based on press releases), this article has a small error. The truth is that in huge samples, with many thousands of people, some correlations will be significant due to chance. You have to look at effect size as well as significance.

Report this content as offensive or unsuitable

jimg57 said on 10 May 2013

User363614 - you misinterpret. DoH/NICE guidelines use the original full research as published, and only after carefully assessing the research for reliability and combining it with a wide range of information from other sources. It's the media that mislead, whether intentionally or not, by giving us the selective and highly polished version of the facts.
The original research itself may be wrong, of course - that's the nature of science - but until another piece of research is published to prove or disprove it, it's the best evidence we have.
[Note - I have no connection with NICE or DoH guideline production]

Report this content as offensive or unsuitable

User363614 said on 20 November 2012

So the evidence base for DoH/NICE guidelines is founded on spin.

Report this content as offensive or unsuitable

Analysis by NHS Choices

Edited by NHS Choices

What is Behind the Headlines?

What is Behind the Headlines?

We give you the facts without the fiction. Professor Sir Muir Gray, founder of Behind the Headlines, explains more...

Clinical trials and medical research

Clinical trials explained, including how to take part in one and why medical research is important

Ask the doctor

Tips on how to make the most of your doctor's appointment, including a checklist of the most basic questions to ask