Echinacea cold study claims analysed

Behind the Headlines

Wednesday October 10 2012

Echinacea is a plant found in North America

Echinacea “can prevent colds”, reports The Daily Telegraph, while the Daily Mail reports that the “largest ever clinical study into Echinacea finds herbal remedy can protect against colds”.

These headlines were based on a study which found that giving healthy adults three doses of the herbal remedy Echinacea daily for four months reduced the combined number and duration of cold episodes by an average of 26% compared to placebo.

What was not widely reported in the news was that the study also reported finding no significant difference between the groups when they looked at the number of colds each group caught. So, the difference seems to have been related to how long a cold lasted, rather than the frequency of cold.

This randomised control trial was well designed and had a good sample size (755 participants), however, there are a number of oddities in the reporting of the study findings that cast a shadow of doubt over the results, such as:

  • no declaration of funding and only partial disclosure of conflict of interests
  • no results table
  • limited reporting of unpleasant side effects
  • no estimates of error around the results reported
  • selective reporting of results
  • the applicability of the results to the general population

Many of these basic problems would usually be picked up by the peer review process or journal editors. The lack of such quality standards may leave the reporters and editors a little red-faced. This news story should stand as a warning to journalists of the dangers of taking research at face value without bringing any critical faculties to bear.

In conclusion, based on this study alone, it is not clear whether taking Echinacea prevents cold episodes, though it does suggest it may reduce their duration. Further research is needed to confirm or refute these findings and also to see if they also apply to people with long-term health conditions such as asthma.

 

Echinacea – not suitable for the under 12s

The Medicines and Healthcare products Regulatory Agency (MHRA), the UK drugs regulator, recently advised that Echinacea should not be given to children under 12 due to the risk of severe allergic reactions, including rashes and swelling of the mouth and tongue.

For more on this topic, see the recent Behind the Headlines article - Echinacea allergy warning for children under 12

Where did the story come from?

The study was carried out by researchers from the Common Cold Centre in Cardiff University. The study was published in the open access, peer-reviewed medical journal Evidence-Based Complementary and Alternative Medicine.

No funding source was reported in the published research article, but three of the five authors declared no conflict of interest. Information on the other two authors’ potential conflicts of interest was absent.

The lack of funding information and incomplete declaration of conflicts of interest by all the study authors is unusual. The standard practice in all good science or medical journals is to clearly state all funding sources and conflicts of interest, or how the funder was involved in the research design or write-up. This article falls short of this standard, and this should arouse a healthy scepticism in readers.

Interestingly, both the Mail and the Telegraph reported that the study was part funded by A. Vogel, a Swiss manufacturer of herbal remedies, including Echinacea products (such as Echinacea toothpaste). This could not be confirmed from the research article alone, although the Echinacea used in the research was provided by this company, and the study is highlighted on the A. Vogel website’s blog.

The media reporting was generally focused on the finding that “taking three daily doses of the common remedy for four months reduced the number of colds and duration of the illness by an average of 26%”. This finding is the result of combining the number of colds and their duration into one variable.

The media chose not to report the finding that the number of colds alone was not significantly different between the two groups, which is also informative.

Finally, the media did not highlight the numerous and significant limitations of the study discussed below.

 

What kind of research was this?

This was a was a randomised, double blind, placebo-controlled clinical trial designed to assess the safety and benefit of Echinacea purpurea (Echinacea) extract in the prevention of common colds.

Common colds are caused by a range of viruses that result in the familiar symptoms of a runny nose, cough, and sore throat and sometimes complaints of headache and fever. The authors report that the common cold is the most prevalent disease in Western civilisation, with substantial related healthcare costs, so a medication to reduce this disease burden would be welcomed.

 

What did the research involve?

A total of 755 healthy subjects were randomly assigned to receive an extract of Echinacea purpurea (a flowering plant found in North America, supposed to have immune-boosting properties), or a placebo for a period of four months.

The Echinacea administered was a commercially available product called “Echinaforce drops”, supplied by A. Vogel Bioforce. The placebo drops were similar in shape, colour, consistency, odour and flavour. Participants and study investigators were ‘blinded’ to which treatment was given to which participant.

Participants took three doses of 0.9ml of the drops each day for four months in an attempt to prevent colds. This corresponded to 2,400mg of Echinacea extract per day. During a cold (cold episode), participants were instructed to increase the dose to five doses of 0.9ml per day (4,000mg per day). Each dose was diluted in water and held in the mouth for 10 seconds, a method described by researchers as having “maximum local anti-viral effects”, although why this would be the case is unclear.

Throughout the investigation period participants were required to keep a diary to record adverse events by answering questions, such as “did you have any unusual or unexpected symptoms today?”.
Researchers considered whether any reported adverse effects could be related to the study medication, documenting this as “unlikely”, “possible”, “probable/likely”. Participants were also asked to record and rate cold-related issues and the use of any medications in a diary.

When participants got a cold they were asked to collect nasal secretions using home kits which were then screened for viruses.

The statistical analysis employed by the researchers was basic and potentially incomplete.

No summary table of the comparisons that the researchers tested was provided in the write-up.

This lack of clarity in reporting makes it harder for the reader to see exactly what statistical comparisons have been tested, and which were significant.

The researchers combined individual measures of the number of colds caught (cold episodes) with cold duration (episode days) to create a single variable of “cumulative events” (cold episodes and episode days combined).

Combining the results in this manner falls far from the standards of transparency you would expect to see from a well-conducted randomised controlled trial.

 

What were the basic results?

Of the 755 participants randomised, 673 (89%) completed the study. The researchers aimed to  examine two main topics, safety and effectiveness. Unfortunately, due to the way the findings were reported (and the lack of a clear results table) the exact comparison and findings of this study were difficult to pin down.

Safety

There were no significant differences in the number of adverse events reported in the Echinacea group compared to the placebo group. As mentioned, very little information was given on adverse events, in both the Echinacea and placebo group.

This is somewhat surprising as the researchers write that one of the main reasons they are conducting the study is to establish whether Echinacea has ‘a good safety profile’.

There were also no significant differences between the two groups on key blood and biochemical measures.

Effectiveness

The authors report that the placebo group had 188 cold episodes, lasting 850 episode days, compared to the Echinacea group which had a total of 149 episodes, lasting 672 episode days. They report that the combined “cumulative events” variable (number of episodes and their duration) was 26% lower in participants given Echinacea, compared to placebo, and that this was statistically significant, but without giving a 95% confidence interval level  – the usual measure of statistical validity. The researchers also found that there was a 59% reduction in recurring cold infections in the Echinacea group compared with placebo (again, no 95% confidence interval level was reported). Not reporting confidence intervals for results is unusual for a randomised controlled trial.

The authors found that significantly more people (52% more, without 95% confidence interval) in the placebo group used aspirin, paracetamol, or ibuprofen to treat their cold while in the study, than in the Echinacea group. There were 58 cold episodes treated with pain medication in the Echinacea group compared to 88 in the placebo group.

The authors then did a subgroup analysis on those that had followed the study medication protocol for the entire four months of the treatment (84 people who took all doses in the Echinacea group; the number in the placebo group not reported). Those who had deviated in any way from the medication, or who had dropped out, were excluded. This sub-analysis showed that the Echinacea group had 53% fewer cold episode days, which was statistically significant (no 95% confidence level reported). The results of a comparison of cold episodes, the other key variable, were not reported.

 

How did the researchers interpret the results?

The authors concluded that “prophylactic intake of E. purpurea over a four-month period appeared to provide a positive risk to benefit ratio”.

 

Conclusion

This study appears to show that giving healthy adults Echinacea every day for four months may result in an average 26% reduction in the combined number and duration of cold episodes compared to placebo over the same period. Combining outcomes in this way is a potentially useful way of reporting outcomes. However, in some cases it is used to “fish” for statistically significant findings when the individual outcomes fail to reach significance on their own. Indeed, when this combined variable was ‘disaggregated’ the researchers found no significant difference between the numbers of colds that occurred in the two groups.

This study appeared to be well designed, and may have collected important results. However, the way in which it was written up meant that it was difficult to assess the findings. These basic problems should really be picked up by the peer review process or journal editors.

The following issues with the study's reporting muddied the water of interpreting these findings fully and clearly:

No declaration of funding and only partial disclosure of conflict of interest

The lack of funding information and incomplete declaration of conflicts of interest by all the study authors is highly unusual. The standard practice in all good science or medical journals is to clearly state all funding sources and conflicts of interest. This article falls short of this standard. While it is not unusual for commercial companies to fund, or provide drugs, for research purposes (which is not in itself a bad thing), it is unusual when this is not declared in the publication.

No results table

Unconventionally, this study did not contain a results table showing which statistical comparisons were made, for example, the number of colds in the placebo group versus the control group (and whether this was statistically significant). Describing only a few significant results in the results section makes it unclear whether other comparisons were performed and whether they were left out because they turned out to be non-significant. The discussion section of the article makes suggestions of non-significant findings that were not mentioned in the results section. The study also failed to list any adverse effects or side effects in a meaningful way.

No estimates of error around measures of effect reported

For the results that were reported, there were no 95% confidence intervals. The research does include the probability (p values), which confirms the significance of these calculations, but the 95% confidence intervals would be valuable. These can demonstrate, for example, if risk reductions only just reached significance, or, if the confidence intervals were wide, it would mean we could have less confidence in the accuracy of the estimated effect of Echinacea.

Selective reporting of results

In their discussion section, the authors state that statistically significant findings between the Echinacea group and placebo group were only found for “cumulative cold episode days” and on the use of pain medications to treat cold episodes. They also mention (in the discussion only) that the number of cold episodes alone was not statistically significant between the Echinacea and placebo group. Only raising important non-significant findings in the discussion without first mentioning them in the results section is a further unconventional practice. It also raises questions over whether the authors have reported only significant findings in the results section, which would give a biased view of their findings.

Statistical analysis

The authors highlight that the Echinacea group had a higher susceptibility to colds than those in the placebo group at the start of the study (measured by assessing the number of colds in the past). They also report that participants in the Echinacea group reported less frequent use of common pain medications. They point out that adjusting for these covariates would most likely have resulted in an even higher beneficial effect of Echinacea.

The researchers employed the double-blind randomised control trial method as this is rightly seen as the gold standard in judging whether a treatment is both effective and safe. However, they left out many of details you would expect to see in the reporting of a well-conducted randomised control trial – details that would ultimately add credibility to the results.

It could well be the case that Echinacea may have a role in the prevention or treatment of colds. But, based on the results of this study, it is very hard to state that with any certainty.

Analysis by Bazian. Edited by NHS Choices. Follow Behind the Headlines on twitter.

Links to the headlines

Largest ever clinical study into echinacea finds herbal remedy can protect against colds. Daily Mail, October 10 2012

Echinacea 'can prevent colds'. The Daily Telegraph, October 10 2012

Links to the science

Jawad M, Schoop R, Suter A, Klein P, Eccles R. Safety and Efficacy Profile of Echinacea purpurea to Prevent Common Cold Episodes: A Randomized, Double-Blind, Placebo-Controlled Trial. Evidence-Based Complementary and Alternative Medicine. Published online September 16 2012 

Ratings

How helpful is this page?

Average rating

Based on 19 ratings

All ratings

Add your rating

Comments

The 3 comments posted are personal views. Any information they give has not been checked and may not be accurate.

User492629 said on 15 February 2014

I dont think it is fair that there has been a lot of scare mongering recently about herbal medicine including echinacea. I think the big pharmaceutical companies see it as a huge threat to their profits. Any type of medicine can have side effects in certain people but one thing I will say is that echinacea is very gentle and effective for colds and many people worldwide swear by its usefullness. Our health system needs to be as open minded as possible and I would like to see more herbal and homeopathic medicines being used on the NHS who I think have very fixed ideas. I especially think growing children deserve gentle non addictive remedies . It saddens me that young kids are being prescribed lethal drugs like ritalin which can cause sudden death What is our health system thinking!!! Its drugs like this which should be banned for under 12s

Report this content as offensive or unsuitable

Dr_Bill said on 18 October 2012

User492632 did you actually read the article? As someone with knowledge of clinical studies, especially of how rigorously they must be applied, I can only say that this study looks 'dodgy'.

For you to say 'definitely does help with colds and flu' is beyond belief. During the the last few years when you have been using the concoction how do you know how many colds & flu you would have had if you hadn't taken it?

One other point to make is that antibiotics will have NO effect on colds or flu and no Doctor would prescribe them for these ailments.

What amazes me is this belief that the drug companies do not value natural remedies. This is nonsense, they would gladly use a natural remedy if it had any beneficial effect and was proven to be safe.

The on thing we can 'definitely' say is that this study has not shown any benefit of echinacea for the treatment or prevention of colds. There are far too many 'holes' in this publication to ake any positive statement.

Report this content as offensive or unsuitable

User492629 said on 12 October 2012

Echinacea definitely does help with colds and should be available on the nhs. This may stop the overprescribing of antibiotics. I use it every winter for my family. and take it with honey and lemon at the first sign of a cold.
I cannot understand it being banned for children under twelve when doctors are prescribing dangerous drugs like ritalin.
I think natural safe remedies are the best by far, after all in the beginning all medicines derived from plants.

Report this content as offensive or unsuitable

Analysis by Bazian

Edited by NHS Choices

Useful links

NHS Choices links

Cold or flu?

Do you know the difference between a cold and flu? An expert explains the conditions, treatments and remedies available.

Media last reviewed: 11/07/2013

Next review due: 11/07/2015

What is Behind the Headlines?

What is Behind the Headlines?

We give you the facts without the fiction. Professor Sir Muir Gray, founder of Behind the Headlines, explains more...

Cold comfort

Find out how to look after yourself if you get a cough, cold or flu, and when you need to see a doctor

Winter health

Tips and advice on how to stay healthy and well through the cold, dark days of winter