A recent study published in thebmj nicely illustrates many of the reasons I find the evidence for the effectiveness of acupuncture for migraines (or anything else, really) very unconvincing. Acupuncture is the practice of sticking needles into acupuncture points for various symptoms and disease modification. While often touted as an “ancient” therapy, what currently passes for acupuncture is a very modern (20th century) reinvention. But still, it is based on the prescientific notions of life force (chi) occurring in two different flavors (yin and yang) and flowing through meridians in the body. There is no scientific basis for any of this.
In fact, not only is there no evidence supporting the existence of acupuncture points, a 2019 review by acupuncturists nicely demonstrated their lack of existence in practical terms. They reviewed studies of qualified acupuncturists and found that the location of acupuncture points were, “significantly inaccurate and imprecise”. Not only do different acupuncturists not agree on where the alleged points are, they don’t agree on which points to use for which indications. From a scientific perspective, therefore, acupuncture points have no validity.
Further, after thousands of studies, research has failed to adequately demonstrate a clinical effect beyond placebo from acupuncture for any indication. In short, the theoretical basis for acupuncture is non-existent and the clinical evidence is unconvincing. All this must be considered when a new study claims to show that acupuncture works, as in the current study for migraines. The authors conclude:
Twenty sessions of manual acupuncture was superior to sham acupuncture and usual care for the prophylaxis of episodic migraine without aura.
But how can this be true when acupuncture points objectively don’t exist? If we take a deep look into the study the answer emerges. The first red flag, which may sound like bias but it is based on published research, is that the study was conducted entirely in China. A systematic review in 1999, with a follow up in 2014, showed that 99.8% of acupuncture studies coming out of China reported positive outcomes. Even for a treatment that works, this is statistically implausible without significant researcher/publication bias.
The existence of chi and the effectiveness of acupuncture are strong cultural beliefs in China. Also, being poked with needles has a high placebo effect, especially in a culture that equates needles with healing. This is why protocols and blinding need to be air tight in any clinical acupuncture study. The current study – not so much.
The next huge, even fatal (especially given the documented bias outlined above), problem with this study is that it is single-blind only. The patients were blinded to their group allocation, but not the acupuncturists. Blinding was not really assessed – they only asked subjects if they thought a needle has been inserted, which was not different between the two needle groups. They were not asked if they had real or sham acupuncture. We also know from prior studies that the attitude and the emotional connection of the acupuncturist has the strongest effect on outcome than any other variable. So when the acupuncturist knows they are giving “fake” acupuncture, that is likely to be conveyed to the subject. At the very least, there is likely to be less of a placebo effect from the interaction.
In short, single-blinded acupuncture studies are all but worthless.
The results of the study show a small but statistically significant reduction in migraine days and migraine attacks from weeks 16-20 of the study. These are all subjective reports, by patients with a strong cultural belief in the treatment from practitioners who knew if they were giving “real” or “fake” acupuncture. In all honesty, my immediate question was – what about use of rescue medication? That is a good way to tell the difference between reporting reduced pain and actually experiencing reduced pain. Remember – studies like this do not measure pain, they ask patients how much pain they feel. If patients know what the desired answer is, that can bias their reporting. One way to address this issue is to look at slightly more objective measures, such as the use of pain medication and other measures of quality of life. These were secondary outcomes in this trial:
The total scores on the Pittsburgh Sleep Quality Index and Migraine Disability Assessment Score were significantly lower in the manual acupuncture group than in the usual care group at week 20. However, we found no significant difference in these scores between manual acupuncture and sham acupuncture. We also found no significant difference in the mean dose of rescue medication or in Beck Anxiety Inventory and Beck Depression Inventory II scores among the three groups at week 20.
No difference in MIDAS, which attempts to measure the effect of migraines on quality of life. And, most importantly in my opinion, no reduction in rescue medication. So while subjects reported fewer migraines, they took the same amount of pain medication to treat migraines. That brief paragraph is the only mention of this in the study, and the implications of this are completely ignored in the discussion and conclusion.
In the end we have a study with high risk of bias showing a small reduction in subjective reports but no reduction in more objective measures of migraine. In total, this is extremely unconvincing, especially given that by all accounts the acupuncture points they say made a difference in this study don’t actually exist. The authors do acknowledge:
However, our findings are not completely consistent with those of several randomised clinical trials, [they reference five trials – SN] which found no significant differences between true acupuncture and sham acupuncture in reducing migraine headaches. The inconsistency might be driven predominantly by differences in treatments received by the control group.
Or the difference might be due to the lack of proper blinding in this study, which plausibly lead to a higher placebo effect in the treatment group without a reduction in actual migraine disability or medication used.