Even more interesting to me than the question of whether or not acupuncture is effective for any particular symptom is the meta-question of how acupuncture proponents have managed to promote a treatment with systematically terrible scientific data. A new study provides a fresh example of this, which I will discuss below.
I think the behavior of acupuncturists reflects the fact that there are subcultures within science, where each community has its own standards, culture, and typical practices. You see this reflected in how they conduct their research and support their claims. Chiropractors, for example, have what is in my opinion a very unscientific culture. Their treatments are not science-based; science is an afterthought cherry-picked to support what is ultimately their philosophy.
The culture of acupuncture
The world of acupuncture has its own culture as well. Within this world there are special, very permissive rules of science that allow acupuncture to work for almost anything. One trend is to look for anything that happens locally in the skin when you stick a needle into it and then declare that a “mechanism for acupuncture.” The rules of the acupuncture culture also allow for a shifting definition of what acupuncture actually is, allowing the definition to conform to whatever the evidence shows. It’s a neat and subtle trick that allows acupuncture proponents to completely subvert the purpose of science.
The most common definition of acupuncture given, even in the very studies that then violate that definition, is the practice of sticking acupuncture needles into acupuncture points. The definition may also include eliciting the “de qi,” which is a sensation perceived when the needle achieves the proper depth or is manipulated. The site “evidencebasedacupuncture.org,” for example, gives this definition:
Acupuncture is the method of treatment based on influencing the body by inserting needles in the specific points of human body, called acupoints.
The problem (for acupuncturists) with this definition is that carefully controlled scientific studies consistently show that it does not matter where you stick the needles or even if you insert needles (as opposed to just poking the skin with dull needles, or retracting needles, or even tooth picks). To further support this conclusion, the perceived effectiveness of acupuncture does not depend on the degree of training or experience of the acupuncturist (so whatever they are learning has no effect), but only upon how warm and nice they are to the patient. In short, acupuncture is an elaborate placebo.
One strategy to get around this pesky scientific evidence is to redefine acupuncture as needed. Depending on the results of any particular study, sham acupuncture (at the “wrong” positions) or placebo acupuncture (without needle penetration) may count as acupuncture.
Further, it is common practice now to rebrand transcutaneous electrical nerve stimulation (TENS) as acupuncture. TENS is a treatment that involves, as the name suggests, providing small non-painful current of electricity to modulate pain and other subjective symptoms. TENS is probably effective for pain (I could not find any studies with hot flashes, the topic of the next section).
What the acupuncture community has done is magically transform TENS into acupuncture and call it electroacupuncture, by delivering TENS through needles that they call acupuncture needles (which are just thin needles). Again, acupuncture points do not matter – because they do not exist.
Acupuncture for hot flashes
The new study falls squarely into the bait-and-switch strategy that is now very popular among acupuncture proponents. They looked at women breast cancer survivors with hot flashes and compare four groups: “electroacupuncture” (EA), sham acupuncture (SA), gabapentin, and placebo pill (PP). They found:
By week 8, SA produced significantly greater reduction in HFCS [hot flash composite score] than did PP (-2.39; 95% CI, -4.60 to -0.17). Among all treatment groups, the mean reduction in HFCS was greatest in the EA group, followed by SA, GP, and PP (-7.4 v -5.9 v -5.2 v -3.4; P = < .001). The pill groups had more treatment-related adverse events than did the acupuncture groups: GP (39.3%), PP (20.0%), EA (16.7%), and SA (3.1%), with P = .005. By week 24, HFCS reduction was greatest in the EA group, followed by SA, PP, and GP (-8.5 v -6.1 v -4.6 v -2.8; P = .002).
The p-values are for the primary outcome, which was SA vs. PP. I thought that was an odd choice for primary outcome – essentially a competition between two placebos. In the end what this study shows is that sticking needles into someone has a larger placebo effect than taking a pill. This is something we already knew. Overall, the more invasive the procedure the larger the placebo effect. I should also point out that you cannot blind allocation to acupuncture vs pill – subjects know if they are getting stuck or taking a pill.
The comparison between EA and SA was not significant. So yet again, whenever you control for the variables that actually define acupuncture (acupoints and needle insertion) there is no significant effect.
A 2013 systematic review of acupuncture for hot flashes shows essentially the same thing:
We found insufficient evidence to determine whether acupuncture is effective for controlling menopausal vasomotor symptoms. When we compared acupuncture with sham acupuncture, there was no evidence of a significant difference in their effect on menopausal vasomotor symptoms. When we compared acupuncture with no treatment there appeared to be a benefit from acupuncture, but acupuncture appeared to be less effective than HT (hormone therapy). These findings should be treated with great caution as the evidence was low or very low quality and the studies comparing acupuncture versus no treatment or HT were not controlled with sham acupuncture or placebo HT. Data on adverse effects were lacking.
That conclusion is very common in the acupuncture literature. You could almost cut and paste that in to the conclusion of any systematic review of acupuncture and just replace the name of the condition being treated. There is no difference between acupuncture and sham acupuncture. When comparing acupuncture to no acupuncture there is a difference, because that comparison is unblinded and needle insertion has a known placebo effect. Overall the data quality is poor with high heterogeneity.
Again, what is most interesting is why the authors decided to spend the resources on the current study given the status of the literature.
Conclusion: Why indeed?
Medical science progresses by designing and performing more and more rigorous studies, controlling for specific variables, until we can say with a high degree of confidence what the risks and benefits are for a specific intervention in a specific subset of patients. The operative word there is “specific.”
The acupuncture literature does include the occasional well-designed study that follows this trend. The Archives of Internal Medicine acupuncture study in back pain published in 2009 was one such study – excellent control groups isolating specific variables. But then the authors blew it by completely misinterpreting their own data, confusing sham and placebo acupuncture for acupuncture.
More commonly we see a study similar to the current one – not designed to actually answer the question of whether or not there is any specific efficacy to acupuncture. We already have a pile of low quality studies, and now we have one more to throw onto the pile.
It seems almost that these types of studies are designed not to ask if acupuncture works, but to show that it works. The studies seem rigged to be positive – of course there is going to be a difference in an unblinded comparison between needle insertion and taking a pill.
There is a role for preliminary studies in medicine. They tell us where to focus our resources and how to design safer and better trials. They don’t answer the question of whether or not a treatment works.
There are now thousands of acupuncture studies looking at every indication you can imagine (which stretches credulity that there is any common underlying mechanism). We are well past the time for preliminary studies. Despite thousands of studies, there isn’t a single indication for which real acupuncture has been shown to work to a high degree of confidence. At this point I would say that acupuncture should be abandoned as a scientific concept. It is a failed hypothesis that has added no real knowledge to our understanding of health and disease.
If, however, you are going to spend the resources to do an acupuncture study, make sure it is rigorous enough to add new information, and isn’t just another preliminary study to throw on the pile and get another round of misleading headlines about how “acupuncture” works. Of course a cynical person might suspect that this is the real goal of these studies.
