Not only do unproven treatments need to be tested, but the tests also need to be fair.
Without a fair test, the findings from any research may not mean much. Even worse, an unfair test could give healthcare professionals the wrong idea and people may be given treatment that does not work, or they may not be given treatment that could be helpful.
This page explains:
Making comparisons
If someone who is ill takes a treatment and then gets better, it could be due to a natural recovery that would have happened anyway.
To tell if the treatment has worked, it needs to be compared to another treatment or a placebo (see below). The two results have to be different enough to indicate a difference has not occurred by chance.
Comparing a treatment with a placebo
The treatment may be compared with a placebo (a dummy treatment), such as a sugar pill, that looks the same as the treatment.
If there are fewer symptoms or other problems after a treatment than after a placebo, this suggests that the treatment works.
Comparing a treatment with a standard treatment
Where a treatment is already known to be effective from previous research, it is usually not considered right (ethical) to compare the new treatment with a placebo. The new treatment usually needs to be compared with a standard treatment that is already known to be helpful.
This makes it possible to determine whether the new treatment works better than the treatment already being used. New treatments are as likely to be worse as they are to be better than existing treatments.
Placebo effect
The placebo effect is the phenomenon of someone’s symptoms improving when they have only been given a dummy treatment, or even after they have just seen a doctor.
Sometimes, a doctor's or other healthcare professional’s reassurance and their confident way of communicating with people who are feeling ill helps some people to feel better. The placebo effect is a largely mysterious and fascinating effect that can be quite powerful.
If you think and believe you are going to get better, you're much more likely to. However, this doesn't work in all situations and for all conditions.
Dummy treatments may be given to people in clinical trials. A placebo medicine looks the same as the medicine being studied, so you don't know which one you're taking. Some people may feel better after taking the placebo medicine because they think they are being given real medication. This is the placebo effect.
Placebos are particularly powerful in conditions where symptoms are important. For example, people feel pain differently and respond better to treatments they think are going to work. In extreme circumstances, some people who are in severe pain respond to a placebo apparently as well as they would to a powerful painkiller.
Placebos do not work for all conditions. High blood pressure can be lowered by active medicines, but placebos have no detectable effect. Similarly, placebo treatments do not lower blood cholesterol, but statin medicines do.
Sham treatments that work
Researchers have designed ways of creating placebos for complementary medicine treatments, such as acupuncture. It's possible to carry out sham acupuncture where needles are inserted to a different depth and in different places than those used in real Chinese acupuncture. In recent trials, both types of acupuncture appeared to be better than doing nothing.
Studies have also carried out placebo surgery on people with knee pain. The placebo treatment often has good results.
Control groups
Participants in a clinical trial will usually be put into one of two groups:
- a group in which they are given the unevaluated treatment being assessed
- a group in which they are given an existing standard treatment or a placebo if no proven standard treatment exists (known as the control group)
The aim is to compare what happens in these groups. Participants are randomly assigned to one of these groups (see randomisation, below).
While treatments are different in the two groups, as many other conditions as possible stay the same. For example, both groups should have people of a similar age, with a similar proportion of men and women, who are in similar overall health.
Randomisation
The best way to get similar groups is to allocate individuals to one of the groups in the trial in an unpredictable, random way. This increases the likelihood that the two groups will be similar. This process is called randomisation.
In most trials, a computer rather than a doctor will randomly decide which group each patient will be allocated to. This allocation will be concealed until after each eligible patient has been accepted for the trial.
These precautions mean that people who decide whether a patient is eligible to participate in the trial cannot influence which treatment a patient is allocated to receive. This protects the study from conscious or unconscious bias, which would make the test unreliable.
Blinding
Many trials are set up so that no one knows who has been allocated to receive which treatment. This is known as blinding and helps reduce the effects of bias when comparing the outcomes of the treatments.
Many people feel better if they think they're getting a better new treatment, even if the treatment is ineffective and their underlying health problem has not really changed at all.
When both the medical staff organising treatment and those taking part in the trial do not know who is receiving which treatment, it is called a double-blind trial.
Blinding is easier when testing medicines, but more difficult when testing other types of treatments or methods of caring for people. For example, it may be impossible to blind a trial that is comparing two types of surgery.
Why blinding is important
Some clinical trials measure hard outcomes such as survival, so outcome measurement is unlikely to be biased.
However, most trials measure outcomes that are more open to biased assessment. For example, patients and researchers may have to make some sort of judgement about how bad symptoms are.
If either researchers or participants know - or think they know - who is receiving which treatment (including placebos), that knowledge may influence what they report.
Participants who think that they are taking an active treatment may not want to let down the researcher, and may exaggerate benefits and minimise side effects. Researchers may allow their hopes about a new treatment to unconsciously influence their recording of symptoms.
The result of these biases is often to overestimate how effective a treatment is. To reduce these possible sources of bias, many trials are double-blind.
Size of trials
For a trial to be a fair test, the number of people taking part needs to be large enough.
For example, in a small trial of 20 people with 10 people taking each treatment, seven people may improve on the new treatment and five on the standard treatment.
Most of us would not think of that as a fair test because, while the new treatment may be better, the finding could easily have occurred by chance.
If the trial was bigger, with 700 out of 1,000 people improving on the new treatment and 500 out of 1,000 on the standard treatment, the result means researchers can be very confident that the new treatment was better.
The degree of this confidence can be estimated. Researchers can provide a range, called a confidence interval, to tell you how certain they are of the result.
Researchers can also test how "statistically significant" a result is. This can help identify where differences between treatments are unlikely to be due to chance.
