“Unpublished data shows breast cancer drug ‘third less effective’” is the headline in The Guardian today. The newspaper goes on to say that “unpublished data from trials of the breast cancer drug Herceptin show that it may be up to a third less effective than has been claimed”.
Other newspapers also carry the story and they are based on a comment by the New Zealand Pharmaceutical Management Agency (PHARMAC) about their investigations into the trials which provide evidence for the use of this drug to treat breast cancer. PHARMAC has a similar role to NICE in the UK: assessing the evidence behind treatments so that purchasing and funding decisions can be made. In the UK, it is recommended that herceptin is given sequentially (i.e. after chemotherapy) for women with early breast cancer that is HER2 positive. However, PHARMAC have suggested that the as yet unpublished results from one of the trials that contributed evidence to the decision to recommend this particular treatment regimen, may change the picture.
Importantly, the PHARMAC comment is not disputing that herceptin is an effective treatment – as may be implied by the newspaper headlines. PHARMAC is suggesting that using herceptin in one particular way may be less effective than is currently believed. They call on the group that conducted the study in question – the North Central Cancer Treatment Group – to publish fully the results of this aspect of the trial to help in answering any outstanding questions.
Where did the story come from?
Drs Scott Metcalfe, Carl Burgess, George Laking, Jackie Evans, Susan Wells and Steffan Crausaz were the authors of this comment. Three are from PHARMAC (Pharmaceutical Management Agency) in New Zealand, the others are from other academic and medical institutions in the country and provide advice to PHARMAC. The comment was published in the peer-reviewed medical journal: The Lancet .
What kind of scientific study was this?
This was a comment from PHARMAC based on their investigations into the main studies of herceptin. To make decisions about funding treatment for women with breast cancer, researchers at PHARMAC and their advisers looked at the studies that have been done using the drug. Of particular interest are studies that investigate whether it is better to use herceptin at the same time as existing chemotherapy or after it (i.e. sequentially).
What were the results of the study?
While investigating which treatment pattern to fund, the group have discovered that one study – by the North Central Cancer Treatment Group (NCCTG) – may have only reported part of their results. PHARMAC says that “data from the 985 women given 12-month sequential [herceptin] are missing”, while data from the other groups in that trial – women given herceptin alongside their chemotherapy and women in the control group – have been fully published in a peer-reviewed journal. Data from the “sequential” arm was only reported at a conference in 2005.
Other studies – the Herceptin Adjuvant Trial (HERA) and PACS-04 – conclude that using herceptin after chemotherapy (i.e. sequentially) is better than using it at the same time. However, the PHARMAC group say that when they combined these known results with the results they got from the conference presentation for the “sequential” arm of the NCCTG study, they found that the effect of herceptin was reduced by about one third.
What interpretations did the researchers draw from these results?
The PHARMAC group concludes that the “selective release of data from the NCCTG” has far-reaching implications for women with breast cancer (early breast cancer of the HER2 positive type) and not including these data in the overall assessment of the sequential use of herceptin has led to the use of the drug seeming more effective than it is. PHARMAC calls on the researchers who carried out the NCCTG trial to publish the results of the part of the study that looked at the sequential use of herceptin.
What does the NHS Knowledge Service make of this study?
Major doubts still exist as to the optimum sequence and duration of herceptin treatment for early breast cancer. Sequential use of herceptin (i.e. after chemotherapy) is the advised treatment regimen in the UK. This investigation by PHARMAC into the evidence that underpins this treatment recommendation raises important issues for the scientific community about how crucial it is to publish trial data fully. They conclude that adding currently unpublished data (from one part of the NCCTG study) to what is known about the effects of sequential treatment suggests that herceptin used in this way may be up to a third less effective than previously thought.
Sometimes researchers do not release their results until a certain period of follow-up is completed or a certain number of events have occurred that will ensure that their results have enough statistical power (i.e. that there are enough units of measurement to confidently say whether a true difference exists or not). This is the case with this part of the NCCTG trial. However, PHARMAC says that the results should be released and can be combined with results of other similar trials to address the power issue (e.g. through meta-analysis).
- Most importantly, the headlines in the newspapers may suggest that there is doubt as to whether herceptin is good at all. This is not the case. What is being disputed here is the way that herceptin is used: for women with early breast cancer is it better to use herceptin after chemotherapy or at the same time as chemotherapy? In the UK, herceptin is usually given after chemotherapy. Other trials that have studied this sequential pattern of treatment have concluded that using the drug in this way is effective.
- Herceptin is also licensed for use in women with advanced breast cancer where it is given either in combination with chemotherapy (in women who haven’t had chemotherapy before) or after chemotherapy (in women who have had at least two chemotherapy treatments). The appropriateness of these treatment patterns are not being disputed by PHARMAC.
Herceptin is effective in women with a particular kind of breast cancer (the type called HER2 positive). The research into the optimal way to use herceptin, particularly in women with early breast cancer, is ongoing and there is debate about the best treatment pattern. This comment highlights the problems of publication bias and once the full results of the trials concerned are added to the pooled evidence, purchasing and funding groups will be better informed.
Sir Muir Gray adds...
Everyone, researchers and editors prefer to publish positive results and ignore negative results. This is called positive publication bias.
The consequence of this is that new treatments may be thought to be more effective than they actually are.
Analysis by Bazian
Edited by NHS Website
Links to the headlines
The Guardian, 16 May 2008
The Daily Telegraph, 16 May 2008
Links to the science
Lancet 2008; 371:1646-1648