Last week I discussed a clinical trial comparing standardized acupuncture, individualized acupuncture, placebo-acupuncture, and usual care. In that discussion I emphasized the comparison between the three acupuncture groups, which did not show any difference in outcome. These results are consistent with the overall acupuncture literature, which shows in the better controlled trials that it does not matter where you stick the needles or even if you stick them through the skin. Therefore the scientific evidence fails to reject the null hypothesis (that acupuncture does not work). This did not stop the press from declaring, almost uniformly, that acupuncture works for back pain, contributing to the public misunderstanding of clinical science.
This week I am going to focus on the other aspect of the trial – the one the researchers and the press chose to focus on – the comparison of the two real and one placebo acupuncture arms to “usual care.” This too was misrepresented by the press, encouraged by the overinterpretation of the evidence by the researchers.
In the comments to Part I of this discussion David Gorski correctly pointed out that the study in fact did not even constitute a comparison of acupuncture to standard medical treatment. He is absolutely correct, and the many reasons for this are worth explaining in detail. Understanding the technology of clinical trials is central to science-based medicine, including all of their pitfalls and limitations. For practical and logistical reasons there is almost never a perfect clinical trial, but mischief only ensues when limitations are not understood, leading to a misinterpretation (and almost always an overinterpretation in the direction of the researcher’s bias) of the evidence.
The primary reason why this study did not constitute a direct comparison of standard medical care to acupuncture for chronic lower back pain is (again, as David pointed out) because this was not a variable among the treatment groups. In the methods to this study it is explained that patients were chosen for having chronic low back pain, among other entry criteria. All patients in the study were allowed to continue their “usual” care – which means whatever they were doing to treat their back pain prior to entry into the trial.
Although it is not explicitly stated in the methods, the acupuncture groups also continued whatever care they were receiving for back pain. There is no indication that they were made to stop. Further, one of the secondary outcomes tracked in the trial was the use of anti-inflammatory pain medications. The study reports:
The use of medications for back pain in the past week (mostly nonsteroidal anti-inflammatory drugs) was similar across groups at baseline (ie, 62% to 65%), but by 8 weeks, it had decreased to 47% in the real and simulated acupuncture groups vs 59% in the usual care group (P=.01). This difference persisted at 26 and 52 weeks.
So clearly those receiving acupuncture were allowed to continue to take pain medication – their usual care. Therefore what the trial compared was usual care + a new intervention vs usual care. The group not receiving any new intervention was not blinded at all to that fact. The author’s acknowledge that a weakness of their trial was the “exclusion of a medical attention control group.”
The press, and the majority of the public (which I estimate based upon my copious feedback on this issue and from reading blog and article comments) seems to have completely missed the distinction between “medical attention” (standard care) as a control, and “usual care”, which essentially means doing nothing at all.
Given the unblinded nature of this comparison it is to be expected that there would be a significant placebo effect associated with introducing a novel treatment for subjective symptoms, especially one that involves personal attention and direct physical contact. Therefore this study only demonstrated a previously documented placebo effect from adding such a new intervention. This data cannot be used to conclude that acupuncture or placebo-acupuncture had any specific physiological effects. Any attempt to draw such a conclusion from this data, or to compare acupuncture to standard medical treatment, is misleading.
It also must be pointed out that study participants were included because they had chronic low back pain (more than three months) of a certain severity or greater despite their usual care. Therefore these subjects constitute a group that has already failed usual care, and it is not surprising that the usual care group showed little improvement.
Further, this means that this study was not a prospective study of usual care, or any specific medical intervention.
The study authors also introduced another variable into the mix. They report:
All participants received a self-care book with information on managing flare-ups, exercise, and lifestyle modifications.
So really the comparison was between real or fake acupuncture + usual care + self-care education compared to usual care + self-care education.The data on education for back pain indicates that it is as effective as medical interventions for acute and subacute low back pain, and effective but not as effective as medical intervention for chronic low back pain. While its overall effectiveness is still unclear, the contribution of this intervention in this trial should not be overlooked. It is likely responsible for some of the improvement reported in all the groups.
It is also plausible that the impact of exercise and lifestyle education on symptoms differed between the intervention groups (real or fake acupuncture) and the non-intervention group (usual care). While there was no difference reported in the percentage of subjects who read more than two-thirds of the self-education booklet, motivation to comply with exercise and lifestyle changes was not measured. Therefore one of the nonspecific effects of being in an intervention group could have been increased motivation to comply with exercise and lifestyle changes.
Conclusion
In part I of my analysis of this study I demonstrated that the results fail to reject the null hypothesis of a lack of effect from acupuncture for chronic low back pain – which is scientific jargon for “acupuncture does not work.” This depends upon a meaningful definition of “acupuncture” (scientific questions depend upon unambiguous definitions). Acupuncture is placing needles at specified acupuncture points through the skin to a specific depth. This study, and previous evidence, clearly shows that these two variables (placing needles through the skin at acupuncture points) have no clinical effects.
Acupuncture proponents have engaged in misdirection from this unavoidable conclusion of their own research. They have done this largely by playing with the definition of “acupuncture.” One such method of misdirection (not involved in this latest study) is to mix acupuncture with another intervention, such as electrical stimulation, and then ascribe effects which are likely due to the other intervention to acupuncture. (Even calling it “acupuncture” is misleading.)
In this and other placebo-controlled acupuncture studies, proponents have attempted to expand the definition of acupuncture to include placebo or fake acupuncture. This allows them to use nonspecific and placebo effects from the ritual of receiving a novel intervention from a caring practitioner to promote the notion that “acupuncture works.”
In this study we also see the confusion of unblinded comparison to no intervention (other than usual care and self-education) with standard medical treatment. While the study authors did not make this false comparison in the paper itself, their misinterpretation of the implications of their data and the promotion of the study results to the media has resulted in widespread misreporting of this study.
In fact this study was not designed to compare acupuncture to standard medical care. Such a study would involve randomizing patients to a new medical intervention they have not already been receiving without effect and acupuncture vs placebo acupuncture. In order to be reliable such a study should also be blinded as much as possible.
The inclusion of a usual care arm in this study was appropriate, however. This is standard practice and serves the purpose of demonstrating that the study design and power were adequate to detect an effect. In other words, the presence of a usual care group helps calibrate the study so that the difference between the acupuncture and placebo groups can be properly interpreted. If there were no difference between the usual care and the real or placebo acupuncture groups then one might conclude that the study was simply not able to detect a difference between acupuncture and placebo-acupuncture.
Therefore, ironically, the fact that the acupuncture and placebo acupuncture groups performed better than usual care allows us to more strongly interpret the lack of difference between acupuncture and placebo-acupuncture as being due to a lack of effectiveness of acupuncture – because if an effect existed this study was powerful enough to detect it.
There remains no compelling evidence for any of the claims made for acupuncture. When the variables specific to acupuncture are properly isolated there is consistently no demonstrable effect. This study adds to previous studies to allow the confident conclusion at this point that acupuncture is a failed medical hypothesis. There is also a complete lack of scientific support for the underlying claims of acupuncture – for the presence of “chi” or life energy that flows through meridians that can be manipulated to influence health and illness.
