Academic hype 'distorting' health news

Behind the Headlines

Wednesday December 10 2014

There is a pressure to generate publicity for academic research

Much scientific research is ignored by mainstream media

"Science and health news hype: where does it come from?," The Guardian asks. A new study suggests a lot of the hype comes from academics themselves, or at least their press offices, as many press releases contain exaggerations.

Researchers looked back at all health-related press releases issued by 20 major UK universities during 2011.

They found many spurious health news reports were based on misleading press releases – normally part written, or at least approved by, the scientists themselves. For example, 36% of press releases they studied made exaggerated claims about human health from research actually carried out on animals. 

But somewhat ironically, the study found press releases containing exaggerated claims were actually less likely to generate news coverage.

So the study asks who's to blame – journalists for not bothering to read the actual studies they are reporting on, or academic press releases for hyping results? Or possibly a 24/7 media culture in which the amount of content produced is seen as more important than the quality?

It would seem misrepresentations can occur at all levels. While there are many dedicated journalists and press officers who strive for transparency and accuracy, a minority are letting the side down.

 

What to look for when you're reading a health news story

There are a number of things you should always bear in mind when reading a new claim in a health news story. These include:

  • whether the study actually involved humans – claims based on animal studies, such as rats, do not automatically have human implications
  • how big the study was – a study involving 20 people has far less weight than one involving 20,000 people
  • whether real-world outcomes were assessed – a study that shows a new drug can lower blood pressure by a few mmHgs is not the same as proving a drug can prevent heart attacks
  • if cause and effect is proven – correlation is not the same as causation; for example, people who own expensive cars are less likely to develop lung cancer, but this doesn't prove that owning an expensive car prevents lung cancer (the true relationship is that rich people are less likely to smoke than poor people)

 

Read more about how to read health news

Where did the story come from?

The study was carried out by researchers from Cardiff and Swansea Universities in the UK and the universities of New South Wales and Wollongong in Australia.

It was funded by the British Psychological Society, the Experimental Psychology Society, the Wales Institute of Cognitive Neuroscience, the Wellcome Trust, the Economic and Social Research Council, the Biotechnology and Biological Sciences Research Council, and Cardiff University.

The study was published in the peer-reviewed British Medical Journal on an open access basis, so it is free to read online or download as a PDF (1.5Mb).

Not surprisingly, the study was not covered widely by most of the papers, especially those whose content is often dominated by health news.

While nobody is covered in glory by this research, journalists come out of it slightly better, as the researchers found hype invented by reporters was relatively uncommon.

But some journalists would seem to be guilty of recycling press releases rather than carrying out any independent reporting (or as it is known in the trade, "Churnalism").

The Guardian did publish a blog by the scientists who conducted the study, and The Independent provided an accurate summary of the study's findings.

And the BMJ put out a press release – on the exaggerations made in press releases. 

 

What kind of research was this?

This was a retrospective observational study, which looked at the content of all press releases about health-related science issued in 2011 by 20 major UK universities, together with the peer-reviewed journals they originated from and the printed news stories that followed.

It aimed to identify how often news stories contain claims or advice that go beyond those in the journal articles or if they try to identify the likely source – whether press releases or the news stories themselves.

The scientists point out health-related news has the widespread potential to influence health-related behaviour, but studies they are based on are often misreported.

It is often unclear whether inaccuracies and exaggerations originate in the news stories themselves or in the press releases issued by the academic institutions producing the research.

They also point out how journalists are increasingly expected to produce more copy in less time. This means press releases have become increasingly important, and the information they provide often makes up the core of the story.

Previous research, such as a study we covered in 2012, has suggested press releases can be a source of misinformation.

 

What did the research involve?

Using publicly accessible information from 20 leading research universities, the researchers identified all press releases based on published studies with possible relevance to human health, which had been issued in 2011 – they found 462 press releases.

For each press release, they sourced the original study and all relevant print or online news stories from the national press (not including broadcast news) – they found 668 news stories.

They coded each journal article, press release and news story.

They focused on three different types of exaggeration:

  • advice to readers to change their behaviour because of the study
  • claims that one thing causes another, but made from observational data only – they used a seven-point scale to rate the strength of such statements
  • inferring there was a relevance to humans from findings in animals beyond (or different to) that stated in the associated peer-reviewed paper

For each category of exaggeration, both news stories and press releases were coded for the strength of their statements.

Taking the peer-reviewed study as a baseline, the researchers then asked to what extent exaggerated statements in news stories were present in each press release.

For example, if a journal article reported an association between eating biscuits and cancer risk and the news story claimed the biscuits caused cancer – a common type of exaggeration – they looked at what the press release said, too.

Or if a news story claimed a treatment for humans but the actual study was on rodents – another common problem – they examined statements in the press release.

They also searched press releases and news stories for any caveats or qualifications to the claims being made.

They analysed their results using standard statistical methods.

 

What were the basic results?

Below are the study's main findings:

Exaggerated advice

Forty per cent of the press releases contained more direct or explicit advice than the journal article (95% confidence interval [CI] 33% to 46%).

Exaggerated causal claims

Thirty-three per cent of the claims in press releases were more "strongly deterministic" than those present in the associated journal article (95% CI 26% to 40%).

Exaggerated claims from animal or cell research

Thirty-six per cent of press releases exhibited inflated inference to humans compared with the journal article (95% CI 28% to 46%).

They also found when press releases contained exaggerations, it was more likely that news stories would, too (58% for advice, 81% for causal claims and 86% for inference to humans).

But when press releases did not contain exaggeration, rates of exaggeration in news stories were only 17%, 18% and 10% respectively.

Exaggeration was not significantly associated with increased news coverage compared with press releases, which were more accurate. So it would seem that not only is exaggeration "bending the truth", it is also ineffective.

 

How did the researchers interpret the results?

The researchers say it is common to blame media outlets and their journalists for exaggerated or sensationalised news stories about health – but their main finding was most exaggeration in health news is already present in academic press releases.

The blame, they say, "lies mainly with the increasing culture of university competition and self promotion, interacting with the increasing pressures on journalists to do more with less time."

The scientific community has the ability to improve this situation, they conclude. Press releases could be a primary target to improve the accuracy of science news, with potential benefit for public health.

In an accompanying editorial, Ben Goldacre, Research Fellow at the London School of Hygiene and Tropical Medicine and author of the book Bad Science, argues academics should be made accountable for exaggerations made about their own work in press releases.

 

Conclusion

As the authors point out, this was a retrospective observational study, so it cannot prove exaggeration in press releases accompanying health studies causes exaggeration in news stories.

To find out more, they are now planning a randomised trial on how different styles of press release influence the accuracy of science news stories.

However, it does chime with anecdotal evidence on the exaggerations in press releases that are then taken up by the media. It can only be a good thing if, as a result of this and future research, scientists themselves take more responsibility for the accuracy of press releases related to their studies. 

There is always the danger of creating a "boy who cried wolf" scenario. Readers may become so mistrustful of what they perceive as hype and exaggeration in health news that they ignore valid, evidence-based advice, which could lead to real harms. 

Analysis by Bazian. Edited by NHS Choices. Follow Behind the Headlines on Twitter. Join the Healthy Evidence forum.

Analysis by Bazian

Edited by NHS Choices

Links to the headlines

Science and health news hype: where does it come from? The Guardian, December 10 2014

Bad science reporting blamed on exaggerations in university press releases. The Independent, December 10 2014

Links to the science

Sumner P, Vivian-Griffiths S, Boivin J, et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ. Published online December 10 2014

Further reading

Goldacre B. Preventing bad reporting on health research. BMJ. Published online December 10 2014

Ratings

How helpful is this page?

Average rating

Based on 4 ratings

All ratings

3  ratings
1  ratings
0  ratings
0  ratings
0  ratings

Add your rating

What is Behind the Headlines?

What is Behind the Headlines?

We give you the facts without the fiction. Professor Sir Muir Gray, founder of Behind the Headlines, explains more...

Follow us on Twitter

Join more than 160,000 who follow @NHSChoices for the latest and best health news and lifestyle advice

Your NHS Health Check

Millions of people have already had their free 'midlife MOT'. Find out why this health check-up is so important

News: 'Half of all health news is spun'

Read about a September 2012 study, which found researchers, press officers and journalists are all guilty of being overoptimistic about scientific results

Our fun guide to getting your research in the news

If you're a struggling scientist looking for much-needed publicity, read our tongue-in-cheek guide to the media game

Volunteering

Find out how to volunteer, why volunteering can benefit your health, and read people's stories about volunteering