Acupuncture and Allergic Rhinitis: Another Opportunity for Intellectual Sterility

You need to keep an open mind.

A common suggestion offered to naysayers of nonsense.

The usual retort concerns not letting one’s brain fall out.

Evaluating SCAM’s is less about having an open mind and more about having standards, a conceptual framework that is used to interpret and analyze new information. One of the benefits of writing and reading topics covered by science-based medicine (SBM) is it has clarified and sharpened the ideas by which I understand the world. Those concepts were nicely summed up by Steve Novella at Neurologica, and I reproduce them here, slightly modified. They should be on stone tablets, not quite commandants, but strong suggestions. The 8 strong suggestions somehow doesn’t cut it however.

1. Respect for knowledge and truth – SBM values reality and what is true. We therefore endeavor to be as reality-based as possible in our beliefs and opinions. This means subjecting all claims to a valid process of evaluation.
2. Methodological Naturalism – SBM believes that the world is knowable because it follows certain rules, or laws of nature. The only legitimate methods for knowing anything empirical about the universe follows this naturalistic assumption. In other words – within the realm of the empirical, you don’t get to invoke magic or the supernatural.
3. Promotion of Science – Science is the only set of methods for investigating and understanding the natural world. Science is therefore a powerful tool, and one of the best developments of human civilization. We therefore endeavor to promote the role of science in our society, public understanding of the findings and methods of science, and high quality science education. This includes protecting the integrity of science and education from ideological intrusion or anti-scientific attacks. This also includes promoting high quality science, which requires examining the process, culture, and institutions of science for flaws, biases, weaknesses, and fraud.
4. Promotion of Reason and Critical Thinking – Science works hand-in-hand with logic and philosophy, and therefore SBM also promotes understanding of these fields and the promotion of critical thinking skills.
5. Science vs. Pseudoscience – SBM seeks to identify and elucidate the borders between legitimate science and pseudoscience, to expose pseudoscience for what it is, and to promote knowledge of how to tell the difference.
6. Ideological Freedom/Free Inquiry – Science and reason can only flourish in a secular society in which no ideology (religious or otherwise) is imposed upon individuals or the process of science or free inquiry.
7. Neuropsychological Humility – Being a functional SBM proponent requires knowledge of all the various ways in which we deceive ourselves, the limits and flaws in human perception and memory, the inherent biases and fallacies in cognition, and the methods that can help mitigate all these flaws and biases.
8. Consumer Protection – SBM endeavors to protect themselves and others from fraud and deception by exposing fraud and educating the public and policy-makers to recognize deceptive or misleading claims or practices.

An excellent list of the mindset I strive to achieve, with variable success. I have always been reasonably well versed in methodological naturalism. Science in general (physics, chemistry, etc.), and the basic sciences of medicine (physiology, anatomy, biochemistry, etc.) have always been the starting point of my understanding of the world and the practice of medicine. The breadth and depth of understanding of the natural world these disciplines provide is amazing.

My time in science-based medicine has reinforced the validity of methodological naturalism, already strong, but added components that had never been addressed in my formal education. One is the importance of prior probability. In the teaching program at my hospital prior probability is usually invoked in the context of determining if a test result is likely to a true positive. In the real world of medical practice prior plausibility for therapeutic interventions is rarely a consideration. In my 30 years I have seen a lot medical interventions come and go, but none have been attempted or abandoned because of a priori implausibility.  Mine has not been a practice where magic is attempted.  Good ideas for therapy, based on interfering with known pathophysiology, sometimes are not effective.

With an understanding of how the natural world functions, a consideration of prior plausibility, combined with knowledge of the multitudinous ways in which we can fool ourselves and others, allows for near-instant pattern recognition that a medical intervention is nonsense, that it cannot be effective. Like treating a stuffy nose.

One of my favorite goofy therapies is the wet sock treatment, beloved by naturopaths and, along with homeopathy, evidence that naturopathic practice is divorced from any reality I know:

1. Take a pair of cotton socks and soak them completely with cold/ice water. Be sure to wring the socks out thoroughly so they do not drip. (May place the wet, wrung out thin socks in the freezer as well. Hang them on a freezer shelf so they are easy to remove).
2. Warm your feet first. This is very important as the treatment will not be as effective and could be harmful if your feet are not warmed first. Place and soak your lower legs into the bucket of hot water which you have prepared (be sure the water is quite hot (an indicator for the proper temperature is one may slowly place their lower legs into the bucket as they tolerate the heat).
3. Soak your lower legs in the bucket for approximately 3 to 5 minutes or taking a warm bath beforehand can accomplish warming.
4. Dry off feet with a dry towel.
5. Remove socks from the ice water bucket. Wring out well. – OR have someone remove your socks from the freezer and bring them to you.
6. Place cold wet socks on feet.
7. Cover with thick wool socks.
8. Go directly to bed, lay down and watch a movie, or read a book. Avoid getting chilled.
9. Keep the socks on overnight. Place the second set of pajamas next to the bed.
10. During the night, you may wake up with your whole body wet from sweat. If so, change into the dry pajamas, but leave on the socks. You will find that the wet cotton socks will be dry in the morning.
11. Do nightly if your symptoms have not yet improved
12. Stop the Wet Sock Treatment if your symptoms have abated or gone altogether

How does wet sock treatment work?

This treatment acts to reflexively increase the circulation and decrease congestion in the upper respiratory passages, head, and throat. It has a sedating action and many patients report that they sleep much better during the treatment. This treatment is also effective for pain relief and increases the healing response during acute infections.

While nonsense, practitioners swear by it. Readers of this blog are well aware of the multitudinous potential biases with assigning efficacy to wet sock treatment. At a fundamental level there is no difference between wet sock therapy, homeopathy, acupuncture or any other SCAM for the treatment of a stuffy nose. When contrasted with known reality they can’t work, and when neuropsychological humility is factored in, it is highly likely that any effects that are noted can be accounted for by unrecognized bias on the part of patient and practitioner.

The big difference is that acupuncture is a form of magic that is likely to get published in formerly reliable journals complete with positive spin: Acupuncture in Patients With Seasonal Allergic Rhinitis. A Randomized Trial. 

Allow me to evaluate the study with

…ad hominem remarks, anonymous criticism, phony expertise and the use of opinion to contradict data, (as a) self-proclaimed sceptic.

with a

… a near complete absence of substantive scientific critique, (A) lack of any reasoned debate… and intellectual sterility.

There multiple flaws, evidently ignored or missed by the editors of the Annals, that render the optimistic conclusions of the paper untenable.

The first is the basic protocol. I have read it multiple times and upon each reading it appears more ludicrous than the time before.

The study takes place during allergy season, springtime for needling and Germany, Deutschland is happy and gay!

Patients were randomized to get real (I will avoid putting real in quotes, like ‘fresh’ fish) or sham acupuncture, 12 treatments over 8 weeks. Their symptoms were followed by three surveys:

Rhinitis Quality of Life Questionnaire (RQLQ), which has 28 questions in 7 domains (activity limitation, sleep problems, nose symptoms, eye symptoms, other symptoms, practical problems, and emotional function) ranked from 0 (no impairment) to 6 (severe impairment) (12). Medication need was measured using an RM score (RMS), comprising the weekly sum of daily assessments (cetirizine, 10 mg/d, or equivalent [1 point]; cetirizine, 20 mg/d, or equivalent [2 points]; and any form of systemic steroids for SAR [3 points]) (daily range, 0 to 3; weekly range, 0 to 21) (13). Secondary outcomes included proportion of responders, defined as patients with a decrease in RQLQ score (mean scores at weeks 7 and 8) of at least 0.5 point compared with baseline; change in symptoms, assessed using a VAS (0 to 100 mm) for overall SAR symptom severity and for nasal, eye, pharyngeal, and common symptoms; and health-related quality of life, assessed with a German version of the Short Form-36 Health Survey (SF-36).

Two weeks ago I have the pleasure of listening to Dr. Ben Goldacre discuss his new book and during the talk he covered the utility of using multiple questionnaires in assessing antidepressant efficacy. He listed a dozen questionnaires in a frenetic manner that should win the title for worlds fastest talker, then he pointed out that if you ask enough questions, some of them will show significant positive effects.

They evaluated the effect of the acupuncture at baseline, at weeks 7 and 8 of the first year, at weeks 15 and 16 (two months after the acupuncture) of the first year, and at weeks 7 and 8 of the second year. A year later.  That is where ludicrous wanders into the study.

The results, from the abstract:

Compared with sham acupuncture and with RM, acupuncture was associated with improvement in RQLQ score (sham vs. acupuncture mean difference, 0.5 point [97.5% CI, 0.2 to 0.8 point; P < 0.001]; RM vs. acupuncture mean difference, 0.7 point [97.5% CI, 0.4 to 1.0 point; P < 0.001]) and RMS (sham vs. acupuncture mean difference, 1.1 points [97.5% CI, 0.4 to 1.9 points; P < 0.001]; RM vs. acupuncture mean difference, 1.5 points [97.5% CI, 0.8 to 2.2 points; P < 0.001]). There were no differences after 16 weeks in the first year. After the 8-week follow-up phase in the second year, small improvements favoring real acupuncture over the sham procedure were noted (RQLQ mean difference, 0.3 point [95% CI, 0.03 to 0.6 point; P < 0.032]; RMS mean difference, 1.0 point [95% CI, 0.2 to 1.9 points; P < 0.018]).

As they note:

After the 8-week follow-up phase in the second year, small improvements favoring real acupuncture over the sham procedure were noted.

suggesting efficacy of the intervention up to a year later. That is one powerful intervention if it can have an effect on outcomes up to a year later.

From my research the RQLQ has to change by more than 0.5 to be considered minimally significant.  So the response as measured by the RQLG is just at the level of normal variation in the score. Hardly impressive.

The hypothetical mechanism by which acupuncture is alleged to work?

… studies have shown a potential effect of acupuncture on atopic diseases via different mechanisms, including changes of the endogenous opioid peptides in the central nervous system; reduction of prostaglandin E2 levels in the brain and serum; suppression of IgE production and modulation of Th1/Th2 cell response; central influence of acupuncture stimulation with specific activation of brain regions, including the influence of neu- ronal structures containing encephalin or beta-endorphin; and reduction of allergen-induced basophil activation.

Outside of vaccines and some biologics such as Rituxan, I cannot think of any medical intervention whose affect upon disease physiology persists longer than a few days after cessation. To continue to alter physiology a year later would make acupuncture one of the most powerful and longest-lasting medical interventions ever discovered.  Especially if applied to the usually transient effects noted above.

Or, as is more likely, the differences are all noise and bias.

And is there the potential for noise and bias? As my in-laws would say, oh yeah, you betcha.

First, they

…developed the trial interventions in a consensus process using a Delphi approach with experienced acupuncture experts.

Which means? It means there is no standard treatment for allergic rhinitis with acupuncture. Everyone did it a different way and the Delphi approach is a method of achieving a consensus when there is no standard. And even then there was variability in how many needles each patient received and where they were placed.

I went looking on the Googles for information concerning the validity of the acupuncture points used in the trial and I will admit to defeat. Most of what I found concerned acupressure points, which were mostly around the nose.  If they used well defined points for allergic rhinitis, I cannot find them.  Which is not surprising as traditional Chinese medicine is not concerned with the underlying pathophysiology of a disease like allergic rhinitis.  TCM practitioners look at the tongue and feel the pulse, among other techniques, to determine a therapeutic intervention.  If acupuncture actually did something to underlying pathophysiology, and there is no reason to suspect it would, it would be like choosing an antibiotic based on the reading of goat entrails.  And I sometimes suspect that is how antibiotics are chosen.

The sham acupuncture patients

 …were needled in at least 5 of 7 predefined non acupuncture points bilaterally, with only superficial insertion of needles (maximum 20 mm in length). Needle type and diameter were not defined (10). De qi and manual stimulation of the needles were avoided.

So in essence what they did was come with a quasi-arbitrary quasi-random form of needling and compared it to sham acupuncture, a different quasi-arbitrary quasi-random form of needling.

We all know it is skull, ear and oral acupuncture that is best for ENT problems. No wait, it’s laser acupuncture.  That’s the ticket. Lasers for allergic sinusitis. And of course we all know that acupuncture works better for rhinitis if done during the dog-days.  Such is the state of published acupuncture for rhinitis.  No consistency.

Given the thousands of acupuncture points, the lack of precision of defining these points and the various acupuncture schools (Chinese, German, Japanese, etc) how do they know they were not in another school’s acupuncture point? The only part of the human body that is routinely free of acupuncture points in the genitalia. I doubt they were needling there. Maybe the sham acupuncture was in active sites that were inadvertently altering Japanese or ear qi to make the rhinitis worse.

The sham group was not the same in that they had fewer needles on average: 10 needles (range 4 − 14) compared to the real acupuncture’s 16,  (range 9 to 25).  The sham group also had no manipulation, the needles were shallow and they were not irritated to generate de qi.

The variation in the number of needles also suggests acupuncture is a therapy practitioners just make up as they go along.  In the SCAM world that is referred as individualization. Since bigger placebo and more placebo will have increased effect compared to smaller and lesser placebo, one would suspect a better effect from the real acupuncture.

The sham acupuncture was less of an intervention: fewer needles, less penetration and no manipulation. It was an inferior substitute for real acupuncture, not a valid placebo, which they admit in the study protocol.

Patients were told:

In this study different types of acupuncture will be compared. One type is similar to the acupuncture treatment used in China. The other type does not follow these principles, but as also been associated with positive outcomes in clinical trials.

I suspect that a quick search of the interwebs by patients could suggest if they were getting needled in valid acupuncture points, although not if they were the right valid points.

There is probably sufficient qualitative differences in the two intervention  to account for any differences in comparative efficacy and it is likely that unblinding occurred. Did they check for unblinding?

They did ask the patients:

How confident do you feel that acupuncture can alleviate your complaint? How confident would you be in recommending acupuncture to a friend suffering from similar complaints? How logical does this treatment seem to you? How successful do you think this treatment will be in alleviating other complaints?

The acupuncture groups scored higher in the “How confident do you feel that acupuncture can alleviate your complaint?” question and they note:

Although some degree of unblinding might have influenced the overall result (33), a major bias seems unlikely because we informed patients that 2 types of acupuncture treatment were being compared without mentioning such terms as “placebo” or “sham,” because similar strategies of informed consent have been used in most previous acupuncture trials (34–38), and because post hoc analyses suggested that differences in study outcomes could not be explained by patient beliefs about treatment or other baseline differences.

So they assessed belief in acupuncture but in an impressive manifestation of don’t ask, don’t tell, they did not inquire of the patients what they thought they had received. A key question that was artfully avoided.

And given the differences in number of needles and the techniques used, although practitioners “were instructed to deliver both in the same context and with the same behaviors.” I am skeptical they were successful. Plenty of opportunity for the Clever Hans effect where the patient would note unconscious behaviors of the acupuncturist that suggests they are, or are not, receiving the real deal.

So there is ample opportunity for noise and bias to be the reason the acupuncture group was barely better than the sham group.  And the differences were trivial at best.

When you look at the overlapping error bars you would conclude that acupuncture = sham acupuncture = doesn’t work. If the graphs were for angina treated with mammary artery ligation verus sham surgery, you would not advise the surgery for the patient nor suggest, as they do in accompanying editorial for acupuncture, that mammary artery ligation is effective.

This article is why I have long given up in the Annals. The editors seem to have turned off the part of the brain that allows for critical thinking.

In the summary they say:

Acupuncture led to statistically significant improvements in disease-specific quality of life and antihistamine use measures after 8 weeks of treatment compared with sham acupuncture and with RM alone, but the improvements may not be clinically significant.

May? The study is flawed in a way to suggest any effect is bias and that acupuncture has yet to find a clinical indication with clear cut efficacy. Which one would expect since it is based on nonsense. The editorial? They say:

With this study lending compelling support to the effectiveness of real world acupuncture for SAR…

Can you say cognitive disconnect? While the patient summary says:

Acupuncture seemed to improve symptoms for people with seasonal allergic rhinitis, but the effects were modest and did not last much beyond treatment. The improvement might have been caused in part by the volunteers’ preexisting beliefs about the treatment.

I do not mind bad studies. Well, I do. But I realize that what appears to be a good idea for a protocol at the beginning may turn out to have issues when carried out. Or the researchers cannot see the flaws of their design and outcomes because they are too enamored of the intervention.

But that is why there are editors of journals and peers to review the paper. Are the editors paying any attention to their content? The patient summary contradicts the editorial contradicts the paper. A careful and critical reading of the paper gives compelling evidence that acupuncture does nothing, in line with all the other well-designed acupuncture studies. The cognitive disconnect between the editorial and conclusion and the data presented in the paper is a wonder to behold. And most people will read only the spin in the abstract and the editorial, not noticing the fatal flaws, and erroneously conclude that acupuncture has efficacy.

Yet another reason why I am glad I let my Annals subscription lapse years ago. The editors of the Annals really need to keep a copy of the 8 strong suggestions on the wall.

I also wonder if this is an example of editorial Dunning-Kruger. Most of us spend our time in biomedical medicine. Pathophysiology leads to disease and we try to alter the disease by altering the pathophysiology. We implicitly evaluate papers with that approach in mind. The assumption is the therapy is doing something.

SCAM papers are not biomedical papers. Acupuncture does nothing as delivered that would be expected to alter the course of rhinitis or any other disease.

Acupuncture is a participatory passion play whose function is not to alter physiology but is a method of psychological manipulation to alter perception.  It is a ritual based on fantasy.  That is not to say rituals do not have powerful and long lasting effects.  The ritual of marriage objectively adds a ring to a finger, but there are many changes in perception of ones position in society after the ritual.

The researchers and editors behave as if the study is representative of an actual physical intervention, when all the good studies of acupuncture strongly suggest that all the benefits are subjective and dependent solely on the belief that acupuncture will work, as in this case.

Somehow I suspect the editors and reviewers at the Annals had the same approach to the paper that they would for an antibiotic being used to treat cellulitis, as if a real physiologic process is being modified. You cannot read a SCAM paper that way.

Properly evaluating a SCAM paper is a learned skill and it has taken a long time for me to recognize that when evaluating results whose efficacy is unlikely and whose success is subjective, you need to look carefully for differences in treatment groups where potential bias, often subtle, could account for treatment differences that are often just above random noise.  Because nothing is actually be altered in the patient, as occurs in this study.

Or just revert to the ad hominem, point at the paper and say neener neener, you’re ugly.  Works for me.

An ideal acupuncture study would have 4 groups: one that receives real acupuncture and is told they are receiving either real or sham and a group that receives sham acupuncture and is told they are receiving real or sham. There would be a subjective endpoint and an objective endpoint and I predict in advance the results:

The group that is told they are receiving real acupuncture will have more subjective benefit regardless of whether they receive real or sham when compared to those who are told they are receiving sham regardless of whether they are receiving real or sham. No group will have improvement in the objective endpoint.

That sums up the effects of acupuncture.  It is like all SCAM: the beer goggles of medicine;  it changes nothing but the symptoms appear better for a short period of time.

And acupuncture for allergic rhinitis?  There are enough flaws in this study to make any effect unlikely to be due to anything but noise and bias.  Coors Light at that.