About a year ago the editors of Anesthesia & Analgesia solicited a written debate on whether or not acupuncture is effective or simply an elaborate placebo. Four experienced acupuncture researchers agreed to write the pro-acupuncture article, Wang, Harris, Lin and Gan. They asked David Colquhoun to write the con position, and David asked me to write it with him (which, of course, I enthusiastically agreed to do).
The article is fortunately published in open access, and so I can reprint it here (full article is below). What I think David and I convincingly demonstrated is that, according to the usual standards of medicine, acupuncture does not work.
Let me explain what I mean by that. Clinical research can never prove that an intervention has an effect size of zero. Rather, clinical research assumes the null hypothesis, that the treatment does not work, and the burden of proof lies with demonstrating adequate evidence to reject the null hypothesis. So, when being technical, researchers will conclude that a negative study “fails to reject the null hypothesis.”
Further, negative studies do not demonstrate an effect size of zero, but rather that any possible effect is likely to be smaller than the power of existing research to detect. The greater the number and power of such studies, however, the closer this remaining possible effect size gets to zero. At some point the remaining possible effect becomes clinically insignificant.
In other words, clinical research may not be able to detect the difference between zero effect and a tiny effect, but at some point it becomes irrelevant.
What David and I have convincingly argued, in my opinion, is that after decades of research and more than 3000 trials, acupuncture researchers have failed to reject the null hypothesis, and any remaining possible specific effect from acupuncture is so tiny as to be clinically insignificant.
In layman’s terms, acupuncture does not work – for anything.
This has profound clinical, ethical, scientific, and practical implications. In my opinion humanity should not waste another penny, another moment, another patient – any further resources on this dead end. We should consider this a lesson learned, cut our losses, and move on.
I suspect, however, human nature being what it is, that this will not happen anytime soon.
________________
Full article reprinted from Anesthesia & Analgesia
Acupuncture Is Theatrical Placebo
- David Colquhoun, PhD* and
- Steven P. Novella, MD†
Introduction
Pain is a big problem. If you read about pain management centers, you might think it had been solved. It has not. And when no effective treatment exists for a medical problem, it leads to a tendency to clutch at straws. Research has shown that acupuncture is little more than such a straw.
Although it is commonly claimed that acupuncture has been around for thousands of years, it has not always been popular, even in China. For almost 1000 years, it was in decline, and in 1822, Emperor Dao Guang issued an imperial edict stating that acupuncture and moxibustion should be banned forever from the Imperial Medical Academy.1
Acupuncture continued as a minor fringe activity in the 1950s. After the Chinese Civil War, the Chinese Communist Party ridiculed Traditional Chinese Medicine, including acupuncture, as superstitious. Chairman Mao Zedong later revived Traditional Chinese Medicine as part of the Great Proletarian Cultural Revolution of 1966.2 The revival was a convenient response to the dearth of medically trained people in postwar China and a useful way to increase Chinese nationalism. It is said that Chairman Mao himself preferred Western medicine. His personal physician quotes him as saying “Even though I believe we should promote Chinese medicine, I personally do not believe in it. I do not take Chinese medicine.”3
The political, or perhaps commercial, bias seems to still exist. It has been reported (by authors who are sympathetic to alternative medicine) that “all trials [of acupuncture] originating in China, Japan, Hong Kong, and Taiwan were positive.”4
Acupuncture was essentially defunct in the West until President Nixon visited China in 1972. Its revival in the West was largely a result of a single anecdote promulgated by journalist James Reston in the New York Times5 after he had acupuncture in Beijing for postoperative pain in 1971. Despite his eminence as a political journalist, Reston had no scientific background and evidently did not appreciate the post hoc ergo propter hoc fallacy, or the idea of regression to the mean.
After Reston’s report, acupuncture quickly became popular in the West. Stories circulated that patients in China had open heart surgery using only acupuncture.6The Medical Research Council (UK) sent a delegation, which included Alan Hodgkin, to China in 1972 to investigate these claims, about which they were skeptical. The claims were repeated in 2006 in a British Broadcasting Corporation TV program, but Simon Singh (author of Fermat’s Last Theorem) discovered that the patient had been given a combination of 3 very powerful sedatives (midazolam, droperidol, fentanyl) and large volumes of local anesthetic injected into the chest. The acupuncture needles were purely cosmetic.
Curiously, given that its alleged principles are as bizarre as those on any other sort of prescientific medicine, acupuncture seemed to gain somewhat more plausibility than other forms of alternative medicine. As a result, more research has been done on acupuncture than on just about any other fringe practice.
The outcome of this research, we propose, is that the benefits of acupuncture are likely nonexistent, or at best are too small and too transient to be of any clinical significance. It seems that acupuncture is little or no more than a theatrical placebo. The evidence for this conclusion will now be discussed.
THREE THINGS THAT ARE NOT RELEVANT TO THE ARGUMENT
We see no point in discussing surrogate outcomes, such as functional magnetic resonance imaging studies or endorphine release studies, until such time as it has been shown that patients get a useful degree of relief. It is now clear that they do not.
We also see little point in invoking individual studies. Inconsistency is a prominent characteristic of acupuncture research: the heterogeneity of results poses a problem for meta-analysis. Consequently, it is very easy to pick trials that show any outcome whatsoever. Therefore, we shall consider only meta-analyses.
The argument that acupuncture is somehow more holistic, or more patient-centered, than medicine seems to us to be a red herring. All good doctors are empathetic and patient-centered. The idea that empathy is restricted to those who practice unscientific medicine seems both condescending to doctors, and it verges on an admission that empathy is all that alternative treatments have to offer.
There is now unanimity that the benefits, if any, of acupuncture for analgesia, are too small to be helpful to patients.
Large multicenter clinical trials conducted in Germany7–10 and the United States11 consistently revealed that verum (or true) acupuncture and sham acupuncture treatments are no different in decreasing pain levels across multiple chronic pain disorders: migraine, tension headache, low back pain, and osteoarthritis of the knee.
If, indeed, sham acupuncture is no different from real acupuncture, the apparent improvement that may be seen after acupuncture is merely a placebo effect. Furthermore, it shows that the idea of meridians is purely imaginary. All that remains to be discussed is whether or not the placebo effect is big enough to be useful, and whether it is ethical to prescribe placebos.
Some meta-analyses have found that there may be a small difference between sham and real acupuncture. Madsen et al.12 looked at 13 trials with 3025 patients, in which acupuncture was used to treat a variety of painful conditions. There was a small difference between “real” and sham acupuncture (it did not matter which sort of sham was used), and a somewhat bigger difference between the acupuncture group and the no-acupuncture group. The crucial result was that even this bigger difference corresponded to only a 10-point improvement on a 100-point pain scale. A consensus report13 concluded that a change of this sort should be described as a “minimal” change or “little change.” It is not big enough for the patient to notice much effect.
The acupuncture and no-acupuncture groups were, of course, neither blind to the patients nor blind to the practitioner giving the treatment. It is not possible to say whether the observed difference is a real physiological action or whether it is a placebo effect of a rather dramatic intervention. Though it would be interesting to know this, it matters not a jot, because the effect just is not big enough to produce any tangible benefit.
Publication bias is likely to be an even greater problem for alternative medicine than it is for real medicine, so it is particularly interesting that the result just described has been confirmed by authors who practice, or sympathize with, acupuncture. Vickers et al.14 did a meta-analysis for 29 randomized controlled trials, with 17,922 patients. The patients were being treated for a variety of chronic pain conditions. The results were very similar to those of Madsen et al.12Real acupuncture was better than sham but by a tiny amount that lacked any clinical significance. Again there was a somewhat larger difference in the nonblind comparison of acupuncture and no-acupuncture, but again it was so small that patients would barely notice it.
Comparison of these 2 meta-analyses shows how important it is to read the results, not just the summaries. Although the outcomes were similar for both, the spin on the results in the abstracts (and consequently the tone of media reports) was very different.
An even more extreme example occurred in the CACTUS trial of acupuncture for “frequent attenders” with medically unexplained symptoms.15 In this case, the results showed very little difference even between acupuncture and no-acupuncture groups, despite the lack of blinding and lack of proper controls. But, by ignoring the problems of multiple comparisons, the authors were able to pick out a few results that were statistically significant, though trivial in size. Despite this unusually negative outcome, the result was trumpeted as a success for acupuncture. Not only the authors, but also their university’s public relations department and even the journal editor issued highly misleading statements. This gave rise to a flood of letters to the British Journal of General Practice16 and much criticism on the Internet.17
From the intellectual point of view, it would be interesting to know whether the small difference between real and sham acupuncture found in some recent studies is a genuine effect of acupuncture or whether it is a result of the fact that the practitioners are never blinded, or of publication bias. However, that knowledge is irrelevant for patients. All that matters for them is whether or not they get a useful degree of relief. It seems that they do not.
There is now unanimity between acupuncturists and nonacupuncturists that any benefits that may exist are too small to provide any noticeable benefit to patients. That being the case, it is hard to see why acupuncture is still used. Certainly, such an accumulation of negative results would result in the withdrawal of any conventional treatment.
SPECIFIC CONDITIONS
Acupuncture should, ideally, be tested separately for effectiveness for each individual condition for which it has been proposed (like so many other forms of alternative medicine, that is a very large number). Good quality trials have not been done for all of them, but results suggest strongly that it is unlikely that acupuncture works for rheumatoid arthritis, stopping smoking, irritable bowel syndrome, or for losing weight. There is also no good reason to think it works for addictions, asthma, chronic pain, depression, insomnia, neck pain, shoulder pain or frozen shoulder, osteoarthritis of the knee, sciatica, stroke or tinnitus, and many other conditions.18
In 2009, the United Kingdom’s National Institute for Clinical Excellence did recommend acupuncture for back pain.19 This exercise in clutching at straws caused something of a furore.20 In the light of National Institute for Clinical Excellence’s judgment, the Oxford Centre for Evidence-Based Medicine updated its analysis of acupuncture for back pain. Their verdict21 was
“Clinical bottom line. Acupuncture is no better than a toothpick for treating back pain.”
The article by Artus et al.22 is of particular interest for the problem of back pain. Their Figure 2 shows that there is a modest improvement in pain scores after treatment, but much the same effect, with the same time course is found regardless of what treatment is given, and even with no treatment at all. They say
“we found evidence that these responses seem to follow a common trend of early rapid improvement in symptoms that slows down and reaches a plateau 6 months after the start of treatment, although the size of response varied widely. We found a similar pattern of improvement in symptoms following any treatment, regardless of whether it was index, active comparator, usual care, or placebo treatment.”
It seems that most of what is being seen is regression to the mean, which is very likely to be the main reason why acupuncture sometimes appears to work when it does not.
Although the article by Wang et al.23 is written to defend the continued use of acupuncture, the only condition for which they claim that there is any reasonably strong evidence is for postoperative nausea and vomiting (PONV). It would certainly be odd if a treatment that had been advocated for such a wide variety of conditions turned out to work only for PONV. Nevertheless, let us look at the evidence.
The main papers that are cited to support the efficacy of acupuncture in alleviation of PONV are all from the same author: Lee and Done (1999)24 and 2 Cochrane reviews, Lee and Done (2004),25 updated in Lee and Fan (2009).26 We need to deal only with the latest updated meta-analysis26.
Although the authors conclude “P6 acupoint stimulation prevented PONV,” closer examination shows that this conclusion is very far from certain. Even taken at face value, a relative risk of 0.7 cannot be described as “prevention.” The trials that were included were not all tests of acupuncture but included several other more or less bizarre treatments (“acupuncture, electroacupuncture, transcutaneous nerve stimulation, laser stimulation, capsicum plaster, an acu-stimulation device, and acupressure”). The number needed to treat varied from a disastrous 34 to 5 for patients with control rates of PONV of 10% and 70%, respectively.
The meta-analysis showed, on average, similar effectiveness for acupuncture and antiemetic drugs. The problem is that the effectiveness of drugs is itself in doubt because an update to the Cochrane review has been delayed27 by the discovery of major fraud by a Japanese anesthetist, Yoshitaka Fujii.28 It has been suggested that metoclopramide barely works at all.29,30
Of the 40 trials (4858 participants) that were included in Lee and Fan,26 only 4 trials reported adequate allocation concealment. Ninety percent of trials were open to bias from this source. Twelve trials did not report all outcomes. The opportunities for bias are obvious. The authors themselves describe all estimates as being of “moderate quality” which is defined thus: “Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.” That being the case, perhaps the conclusion should have been “more research needed.” The conclusions of Lee and Fan26 are nowhere near as secure as the abstract implies. In fact, almost all trials of alternative medicines seem to end up with the conclusion that more research is needed. After more than 3000 trials, that is dubious.
CONCLUSIONS
It is clear from meta-analyses that results of acupuncture trials are variable and inconsistent, even for single conditions. After thousands of trials of acupuncture and hundreds of systematic reviews,18 arguments continue unabated. In 2011, Pain published an editorial31 that summed up the present situation well.
“Is there really any need for more studies? Ernst et al.18 point out that the positive studies conclude that acupuncture relieves pain in some conditions but not in other very similar conditions. What would you think if a new pain pill was shown to relieve musculoskeletal pain in the arms but not in the legs? The most parsimonious explanation is that the positive studies are false positives. In his seminal article on why most published research findings are false, Ioannidis32 points out that when a popular but ineffective treatment is studied, false positive results are common for multiple reasons, including bias and low prior probability.”
Since it has proved impossible to find consistent evidence after more than 3000 trials, it is time to give up. It seems very unlikely that the money that it would cost to do another 3000 trials would be well-spent.
A small excess of positive results after thousands of trials is most consistent with an inactive intervention. The small excess is predicted by poor study design and publication bias. Furthermore, Simmons et al.33 demonstrated that exploitation of “undisclosed flexibility in data collection and analysis” can produce statistically positive results even from a completely nonexistent effect. They say this is “… not driven by a willingness to deceive but by the self-serving interpretation of ambiguity, which enables us to convince ourselves that whichever decisions produced the most publishable outcome must have also been the most appropriate.”
With acupuncture, in particular, there is documented profound bias among proponents.4 Existing studies are also contaminated by variables other than acupuncture, such as the frequent inclusion of “electroacupuncture” which is essentially transdermal electrical nerve stimulation masquerading as acupuncture.
The best controlled studies show a clear pattern, with acupuncture the outcome does not depend on needle location or even needle insertion. Since these variables are those that define acupuncture, the only sensible conclusion is that acupuncture does not work. Everything else is the expected noise of clinical trials, and this noise seems particularly high with acupuncture research. The most parsimonious conclusion is that with acupuncture there is no signal, only noise.
The interests of medicine would be best-served if we emulated the Chinese Emperor Dao Guang and issued an edict stating that acupuncture and moxibustion should no longer be used in clinical practice.
No doubt acupuncture will continue to exist on the “High Streets” where they can be tolerated as a voluntary self-imposed tax on the gullible (as long as they do not make unjustified claims).
DISCLOSURES
Name: David Colquhoun, PhD.
Contribution: Professor Colquhoun coauthored the manuscript.
Attestation: Professor Colquhoun approved the final manuscript.
Conflicts of Interest: Professor Colquhoun has no financial conflicts of interest. Professor Colquhoun writes the “DC’s Improbable Science” blog (http://dcscience.net/), devoted to scientific fraud and medical quackery and education policy. Many postings to Dr. Colquhoun’s blog address the lack of scientific evidence for alternative medicine.
Name: Steven P. Novella, MD.
Contribution: Professor Novella coauthored the manuscript.
Attestation: Professor Novella approved the final manuscript.
Conflicts of Interest: Professor Novella has no financial conflicts of interest. Professor Novella maintains the “Neurologica blog”, described as “your daily fix of neuroscience, skepticism, and critical thinking.” The Neurologica blog occasionally addresses the lack of scientific evidence for alternative medicine.
This manuscript was handled by: Steven L. Shafer, MD.
Footnotes
-
Accepted for publication February 1, 2013.
-
Funding: None.
-
Conflict of Interest: See Disclosures at the end of the article.
-
Reprints will not be available from the authors.
- Copyright © 2013 International Anesthesia Research Society
REFERENCES
1. Acupuncture Centre. . About Acupuncture. Available at: http://www.acupuncturecentre.org/aboutacupuncture.html. Accessed March 30, 2013
2. Atwood K. “Acupuncture Anesthesia”: a Proclamation from Chairman Mao (Part IV). Available at: https://sciencebasedmedicine.org/index.php/acupuncture-anesthesia-a-proclamation-from-chairman-mao-part-iv/. Accessed September 2, 2012
3. Li Z Private Life of Chairman Mao: The Memoirs of Mao’s Personal Physician. 1996 New York: Random House
4. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19:159–66 Available at: http://bit.ly/WqVGWN. Accessed September 2, 2012
5. Reston J. Now, About My Operation in Peking; Now, Let Me Tell You About My Appendectomy in Peking … The New York Times. 1971 Available at: http://select.nytimes.com/gst/abstract.html?res=FB0D11FA395C1A7493C4AB178CD85F458785F9. Accessed March 30, 2013
6. Atwood K. “Acupuncture anesthesia”: a proclamation from chairman Mao (part I). Available at: https://sciencebasedmedicine.org/index.php/acupuncture-anesthesia-a-proclamation-of-chairman-mao-part-i/. Accessed September 2, 2012
7. Linde K, Streng A, Jürgens S, Hoppe A, Brinkhaus B, Witt C, Wagenpfeil S, Pfaffenrath V, Hammes MG, Weidenhammer W, Willich SN, Melchart D. Acupuncture for patients with migraine: a randomized controlled trial. JAMA. 2005;293:2118–25
8. Melchart D, Streng A, Hoppe A, Brinkhaus B, Witt C, Wagenpfeil S, Pfaffenrath V, Hammes M, Hummelsberger J, Irnich D, Weidenhammer W, Willich SN, Linde K. Acupuncture in patients with tension-type headache: randomised controlled trial. BMJ. 2005;331:376–82
9. Haake M, Müller HH, Schade-Brittinger C, Basler HD, Schäfer H, Maier C, Endres HG, Trampisch HJ, Molsberger A. German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups. Arch Intern Med. 2007;167:1892–8
10. Witt C, Brinkhaus B, Jena S, Linde K, Streng A, Wagenpfeil S, Hummelsberger J, Walther HU, Melchart D, Willich SN. Acupuncture in patients with osteoarthritis of the knee: a randomised trial. Lancet. 2005;366:136–43
11. Cherkin DC, Sherman KJ, Avins AL, Erro JH, Ichikawa L, Barlow WE, Delaney K, Hawkes R, Hamilton L, Pressman A, Khalsa PS, Deyo RA. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Arch Intern Med. 2009;169:858–66
12. Madsen MV, Gøtzsche PC, Hróbjartsson A. Acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups. BMJ. 2009;338:a3115
13. Dworkin RH, Turk DC, McDermott MP, Peirce-Sandner S, Burke LB, Cowan P, Farrar JT, Hertz S, Raja SN, Rappaport BA, Rauschkolb C, Sampaio C. Interpreting the clinical importance of group differences in chronic pain clinical trials: IMMPACT recommendations. Pain. 2009;146:238–44
14. Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, Sherman KJ, Witt CM, Linde K. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012;172:1444–53
15. Paterson C, Taylor RS, Griffiths P, Britten N, Rugg S, Bridges J, McCallum B, Kite G. Acupuncture for ‘frequent attenders’ with medically unexplained symptoms: a randomised controlled trial (CACTUS study). Br J Gen Pract. 2011;61:e295–e305
16. . Letters in response to Acupuncture for ‘frequent attenders’ with medically unexplained symptoms. Br J Gen Pract. 2011;61 Available at: http://www.ingentaconnect.com/content/rcgp/bjgp/2011/00000061/00000589. Accessed March 30, 2013
17. Colquhoun D. Acupuncturists show that acupuncture doesn’t work, but conclude the opposite: journal fails. 2011 Available at: http://www.dcscience.net/?p=4439. Accessed September 2, 2012
18. Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. Pain. 2011;152:755–64
19. Colquhoun D. NICE falls for Bait and Switch by acupuncturists and chiropractors: it has let down the public and itself. 2009 Available at: http://www.dcscience.net/?p=1516. Accessed September 2, 2012
20. Colquhoun D. The NICE fiasco, part 3. Too many vested interests, not enough honesty. 2009 Available at: http://www.dcscience.net/?p=1593. Accessed September 2, 2012
21. Bandolier. . Acupuncture for back pain—2009 update. Available at: http://www.medicine.ox.ac.uk/bandolier/booth/painpag/Chronrev/Other/acuback.html. Accessed March 30, 2013
22. Artus M, van der Windt DA, Jordan KP, Hay EM. Low back pain symptoms show a similar pattern of improvement following a wide range of primary care treatments: a systematic review of randomized clinical trials. Rheumatology (Oxford). 2010;49:2346–56
23. Wang S-M, Harris RE., Lin Y-C, Gan TJ. Acupuncture in 21st century anesthesia: is there a needle in the haystack? Anesth Analg. 2013;116:1356–9
24. Lee A, Done ML. The use of nonpharmacologic techniques to prevent postoperative nausea and vomiting: a meta-analysis. Anesth Analg. 1999;88:1362–9
25. Lee A, Done ML. Stimulation of the wrist acupuncture point P6 for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev. 2004:CD003281
26. Lee A, Fan LT. Stimulation of the wrist acupuncture point P6 for preventing postoperative nausea and vomiting. Cochrane Database Syst Rev. 2009:CD003281
27. Carlisle JB. A meta-analysis of prevention of postoperative nausea and vomiting: randomised controlled trials by Fujii etal. compared with other authors. Anaesthesia. 2012;67:1076–90
28. Sumikawa K. The results of investigation into Dr.Yoshitaka Fujii’s papers. Report of the Japanese Society of Anesthesiologists Special Investigation Committee. http://www.anesth.or.jp/english/pdf/news20120629.pdf
29. Bandolier. . Metoclopramide is ineffective in preventing postoperative nausea and vomiting. Available at: http://www.medicine.ox.ac.uk/bandolier/band71/b71-8.html. Accessed March 30, 2013
30. Henzi I, Walder B, Tramèr MR. Metoclopramide in the prevention of postoperative nausea and vomiting: a quantitative systematic review of randomized, placebo-controlled studies. Br J Anaesth. 1999;83:761–71
31. Hall H. Acupuncture’s claims punctured: not proven effective for pain, not harmless. Pain. 2011;152:711–2
32. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2:e124
33. Simmons JP, Leif DN, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22:1359–66
