A recent study looking at acupuncture for the prevention of migraine attacks demonstrates all of the problems with acupuncture and acupuncture research that we have touched on over the years at SBM. Migraine is one indication for which there seems to be some support among mainstream practitioners. In fact the American Headache Society recently recommended acupuncture for migraines. Yet, the evidence is simply not there to support this recommendation, which, in my opinion, is a failure to understand a science-based assessment of the clinical evidence.
The recent study, like many acupuncture studies, was problematic, and was also negative. It showed that acupuncture does not work for migraines, but of course also contains the seeds of denial for those who want to believe in acupuncture. From the abstract:
METHODS:
We performed a multicentre, single-blind randomized controlled trial. In total, 480 patients with migraine were randomly assigned to one of four groups (Shaoyang-specific acupuncture, Shaoyang-nonspecific acupuncture, Yangming-specific acupuncture or sham acupuncture [control]). All groups received 20 treatments, which included electrical stimulation, over a period of four weeks. The primary outcome was the number of days with a migraine experienced during weeks 5-8 after randomization. Our secondary outcomes included the frequency of migraine attack, migraine intensity and migraine-specific quality of life.
RESULTS:
Compared with patients in the control group, patients in the acupuncture groups reported fewer days with a migraine during weeks 5-8, however the differences between treatments were not significant (p > 0.05). There was a significant reduction in the number of days with a migraine during weeks 13-16 in all acupuncture groups compared with control (Shaoyang-specific acupuncture v. control: difference -1.06 [95% confidence interval (CI) -1.77 to -0.5], p = 0.003; Shaoyang-nonspecific acupuncture v. control: difference -1.22 [95% CI -1.92 to -0.52], p < 0.001; Yangming-specific acupuncture v. control: difference -0.91 [95% CI -1.61 to -0.21], p = 0.011). We found that there was a significant, but not clinically relevant, benefit for almost all secondary outcomes in the three acupuncture groups compared with the control group. We found no relevant differences between the three acupuncture groups.
There are multiple methodological problems with this study. The first is that it is single-blind, which means the acupuncturists knew what type of acupuncture they were giving, including sham acupuncture, which was their control. Previous studies have shown that the perceived empathy of the practitioner affects the outcome, and so unblinded acupuncturists who are giving “real” acupuncture may just be a bit more enthusiastic than those who know they are giving sham acupuncture. This would tend to result in a false positive outcome.
Second, the acupuncture treatments used electrical stimulation. This is very problematic because it mixes modalities – acupuncture and transcutaneous electrical stimulation – therefore we cannot say which component is responsible for the outcome. Mixing variables is sloppy research, and I wonder why, with three acupuncture arms, they did not include one without electrical stimulation to control for that variable.
Also, the number of treatments given was very aggressive, and not practical. Some have noted that perhaps this is a difference between Chinese and Western application of acupuncture, and that in the US doing 20 treatments over 4 weeks is expensive and not practical. This is a good way to maximize placebo effects, however.
Despite all of these shortcomings, all of which would bias the study in the direction of being positive, the study was negative. For the primary outcome measure there was no statistically significant difference between any of the acupuncture groups and the sham acupuncture group. Once again we see that it does not matter where you stick the needles. Since acupuncture is the practice of stimulating acupuncture points with needles, we can conclude that acupuncture does not work for migraines.
Some of the secondary outcome measures were significantly different, but clinically irrelevant. You always have to be cautious about secondary outcome measures. It is easy to look at many secondary outcomes and choose the ones (even subconsciously) that show the biggest effect. Also, researchers often forget to control for multiple comparisons when doing the statistics. Simmons et al recently pointed out that exploiting researcher “degrees of freedom” can easily result in a false positive result. What this means is that researchers make many decisions about how to look at the data and which data points to look at, and can innocently bias those decisions in the direction of seeing a positive outcome. The degrees of freedom are greater when you have lots of secondary outcome measures to choose from. Picking a primary outcome at the beginning of a study, however, is more constraining.
The study also did not include a placebo acupuncture group – one in which there is no needle penetration, or tooth picks are used instead of needles. Studies using blinded placebo acupuncture generally show no difference in outcome, leading us to conclude that not only does it not matter where you stick the needles, it doesn’t matter if you stick the needles. In short – the needles don’t matter. The training of the acupuncturists doesn’t matter. All of the cultural beliefs about meridians, acupuncture points, and chi do not matter. All that matters is the kind and amount of attention given by the acupuncturist to the patient. In other words – acupuncture is entirely a non-specific placebo intervention. It is the irrelevant magical ritual surrounding the attention from an empathic practitioner. This study is further confirmation of that basic conclusion.
The most recent Cochrane review of acupuncture for migraine reflects the above difficulties, and is also another great example of the difference between evidence-based medicine and science-based medicine. The authors conclude:
There is no evidence for an effect of ‘true’ acupuncture over sham interventions, though this is difficult to interpret, as exact point location could be of limited importance. Available studies suggest that acupuncture is at least as effective as, or possibly more effective than, prophylactic drug treatment, and has fewer adverse effects. Acupuncture should be considered a treatment option for patients willing to undergo this treatment.
The authors find it difficult to interpret the fact that there is no evidence for a difference beetween “true” acupuncture and sham acupuncture. Let me help them. The key point that acupuncture supporters miss when reviewing the evidence is the difference between blinded and unblinded comparisons. In general blinded comparisons are much more reliable than unblinded comparisons, in any area of science. In medicine, and with acupuncture specifically, we have evidence for a large placebo effect, which is strongly affected by practitioner empathy and the attention that goes with the ritual of acupuncture.
What the evidence essentially shows is that blinded comparisons with acupuncture are negative, and unblinded comparisons are positive. This is not hard to interpret at all – it means acupuncture does not work. Whenever we see a phenomenon disappear with the application of proper blinding, we conclude the phenomenon is not real. A phenomenon that depends upon not being blinded is an illusion. Acupuncture is the N-rays of medicine.
Until acupuncture researchers can show a consistent, statistically significant, and clinically relevant effect from blinded comparisons of acupuncture to sham or placebo acupuncture, there is no reason to think that there is any underlying reality to acupuncture as a specific intervention. The best conclusion we can make today, based on all available evidence, is that acupuncture adds nothing to the non-specific effects of the attention from the practitioner. Acupuncture does not work for migraine, and recommendations to use acupuncture are misguided and unscientific.
I generally do not see this confusion in non-CAM areas of medicine. CAM, it appears, has an anti-science field around it that confuses researchers and prevents them from properly interpreting negative outcomes.
