The draft of the latest recommendations from the National Institute for Health Care Excellence (NICE) from the UK includes a recommendation to consider acupuncture for chronic pain (within some limits that primarily have to do with cost-effectiveness). They write:
Consider a course of acupuncture or dry needling, within a traditional Chinese or Western acupuncture system, for people aged 16 years and over to manage chronic primary pain,
Here is their justification for this recommendation:
Many studies (27 in total) showed that acupuncture reduced pain and improved quality of life in the short term (3 months) compared with usual care or sham acupuncture. There was not enough evidence to determine longer term benefits. The committee acknowledged the difficulty in blinding for sham procedures, but agreed that the benefit compared with a sham procedure indicated a specific treatment effect of acupuncture. There was a wide variation among the studies in the type and intensity of the intervention used, and the studies were from many different countries. The committee agreed that the type of acupuncture or dry needling should depend on the individual needs of the person with pain.
This recommendation and justification represent, in my opinion, not only a failure of NICE but a great example of the inadequacy of evidence-based medicine (EBM) in dealing with topics such as acupuncture, and the need for science-based medicine (SBM). A review of their supporting evidence (published separately) reveals the disconnect.
They summarize the evidence:
Very low quality evidence from 13 studies with 1230 participants showed a clinically important benefit of acupuncture compared to sham acupuncture at ≤3 months. Low quality evidence from 2 studies with 159 participants showed a clinically important benefit of acupuncture compared to sham acupuncture at ≤3 months.
Their recommendation for acupuncture benefits ≤3 months is based, by their own admission, mostly on very low quality evidence and some low quality evidence. What does this mean? Mostly that the trials are not properly controlled or blinded. Given that acupuncture has a high placebo effect, this renders the results mostly worthless. Even within a pure EBM framework, this is an extremely weak justification for a recommendation.
Edzard Ernst, also writing critically of this recommendation, points out that the most extensive review of the literature to date came to a different conclusion – that the evidence for acupuncture in various chronic pain conditions reveals weak evidence of a transient and small effect only. So again, even taking a pure EBM approach, the evidence is insufficient to conclude that acupuncture has a “specific effect” as NICE claims.
But now let’s take a more thorough SBM approach. What are the NICE guidelines and other reviews leaving out? First – what does it mean that acupuncture has a “specific effect”? It means that acupuncture points are real, and that sticking needles into acupuncture points (which vary depending on the style of acupuncture) has an effect other than just a theatrical placebo. This conclusion, however, is demonstrably false. There is no scientific or objective basis to conclude that acupuncture points have any basis in reality. In short – they don’t exist. Even when acupuncturists review their own literature they conclude that you cannot localize specific acupuncture points.
Further, the last century of research, since the modern reformulation of acupuncture into something a little less barbaric and medieval, has failed to find any physiological, neurological, biochemical, or anatomical correlate to alleged acupuncture points. This is an argument from absence of evidence, but given the extensive research it is very telling.
If acupuncture points don’t exist, either theoretically or empirically, then how can there be a specific effect from interacting with them? Short answer – there can’t be.
Experts reviewing acupuncture who are not familiar with the principles of SBM or the characteristics of pseudoscience also tend to miss the fact that acupuncture studies tend to mix their variables, in the hopes of passing off non-acupuncture effects as if they are specific effects of acupuncture. The most common example is using “electroacupuncture” in pain studies. This means, essentially, doing electrical stimulation – which is a treatment for pain (although with modest evidence itself) and selling it as acupuncture. This would be exactly like injecting morphine through acupuncture needles and calling it acupuncture.
Another glaring problem with the acupuncture literature is that much of it comes from China. Independent reviews have found that acupuncture studies coming out of China are 100% positive. This would be statistically near impossible even if acupuncture had a real and robust effect. Given that, at best, studies show a weak and tiny effect, the 100% positive results from China is solid evidence of systematic fraud and/or publication bias. The fact that you can be jailed in China for criticizing Traditional Chinese Medicine might have something to do with this.
There is also a clear pattern in the literature, but you need to go beyond counting up studies to see it. The better controlled and blinded a study of acupuncture, the more likely it is to be negative (outside of China, that is). There is a clear negative relationship between quality and effect size, with the effect decreasing to zero for the best studies.
If we put everything together, in a holistic SBM review of the evidence for acupuncture, we can only conclude that it does not work – there is no specific effect. What we see is exactly the kind of noise that we expect, and see in other areas, when there is no real effect at work. The data, for example, is virtually identical in pattern as for things like homeopathy or ESP. You can never seem to nail down a specific effect with high-quality evidence. Combine this with the very low prior probability (due, for example, to the non-existence of acupuncture points) and we can only come to one clear scientific conclusion. Acupuncture is a placebo-only intervention; acupuncture doesn’t work.
There is another important layer here as well, that NICE fails to consider in their review (including their cost-effectiveness review), and that is the downstream effects of recommending pseudoscience typically administered by pseudoscientists. Acupuncturists don’t just use acupuncture for pain, they use it for hundreds of indications, all unproven, up to and including serious illnesses like cancer (not all acupuncturists, of course, but many do). By recommending acupuncture for pain, based upon very weak evidence of a clinically tiny effect in the face of a lack of a plausible mechanism, you are throwing patients to the wolves. To think otherwise is extremely naïve – and that is exactly what I find when I deal with health care experts who are not steeped in SBM or medical pseudoscience.
The public comment period for the NICE recommendations is open until Sept. 14. Hopefully they will get the feedback they need.
