Andrew Weil, MD, pops up quite frequently on SBM, most recently in this entry by Harriet Hall, so I will not spend much space introducing him. An excellent biography and critique of Dr. Weil was written by Arnold Relman, former Editor of the New England Journal of Medicine. It is over a decade old, but contemporary to some of the events described in this post, and still quite relevant.
Suffice it to say that Dr. Weil is one of the most successful and well recognized popularizers of alternative medicine. He has authored or coauthored dozens of books. His website sells everything from baby pacifiers to vitamins to breakfast sausages, packaged bearing his name and/or visage. He is an altmed rockstar. He has been a key player in the branding of alternative medicine. In particular, been an advocate of “integration” of traditional and alternative medicine. He has created and exported residency training programs, and more recently proposed board certification in integrative medicine.
I recently read a book entitled On Being Certain: Believing you are right, even when you’re not, by Robert Burton, nicely-reviewed and recommended by Harriet Hall. In his book Dr. Burton excerpted an interview with Dr. Andrew Weil, pointing out Dr. Weil’s profound certainty about the effectiveness of a particular alternative treatment in spite of contradictory evidence. Dr. Hall also discussed this section of the book in her review. I found the excerpts fascinating and decided to delve more deeply into the interview. I also found another interview with Dr. Weil relevant to his ideas about evidence.
An interview with Dr. Weil (among others) was incorporated as supplemental material to a PBS Frontline episode called: The Alternative Fix. The program was pretty well done, and prominently featured the skeptical views of emeritus SBM editor Wallace Samson, MD. This was the interview quoted by Robert Burton.
Early in the interview, while explaining the appeal of alternative medicine, he brandishes a common mantra of of the “alternative” community:
I also think that a lot of people seek out alternative practitioners in frustration. If their first choice were available, it would be to go to a medically trained person, an MD who was open-minded and had knowledge of things beyond conventional medicine and could advise them about how to use them.”
The implication, of course, is that those who accept the claims of alternative medicine possess the virtue of open-mindedness. This is demonstrably false, and Dr. Weil is about to prove it. Credulousness is not synonymous with open-mindedness. Later in the interview he relates a clinical observation:
I wrote up my experience with an old osteopath in Tucson, who was a master of method called cranial therapy. He would take a kid, one treatment of this very noninvasive, inexpensive method and they would never get another ear infection. I saw this again and again.
I have recommended in my writings on my website that kids with ear infections should go to osteopaths and get this method done.
Cranial manipulation fails by all science-based criteria. Think of it as chiropractic manipulation of the bones of the skull, only with less credibility. It is based on assumptions that have no basis in scientifically established anatomy or physiology. It also fails by evidence-based criteria. The British Columbia Office of Health Technology did a systematic review of craniosacral therapy, and found no evidence of efficacy.
Dr. Weil laments the fact that researchers and mainstream medicine ignored his observations:
After something like twenty years of trying to get the research community interested in this, we finally set up some tests of doing this with kids with recurrent ear infections.
Despite the lack of a plausibility and absence of supporting clinical data, the Office of Alternative Medicine, and it offspring, the National Center for Complementary and Alternative Medicine, came to his rescue and funded a randomized clinical trial of cranial manipulation (full text). The study was designed to investigate two therapies for the prevention of relapse in recurrent otitis media (middle ear infection) in children. In addition to cranial manipulation, they also studied the popular herbal supplement Echinacea pupurea. The methodology utilized a double randomized (factorial) study design. This is a legitimate study design that can answer 2 questions with one trial. 90 children with recurrent otitis media were enrolled. The study was blinded, to the extent that patients and their families were not informed of their randomized assignment. Half were randomized to cranial therapy. The control group received osteopathic examination without manipulation. The same 90 children were independently randomized to receive Echinacea or placebo.
Before getting to the results of the study, I will provide some excerpts from an article about Dr. Weil written at the time the grant was awarded.
Weil would not specify a success rate that he considers optimal for the study, but said that if the results were not positive, he “would be reluctant'” to continue using these methods in his clinic….
If he does not get the positive results he anticipates, however, Weil said the structure of the study would probably be more to blame than the treatments themselves.
“Either the testing was wrong or we had the wrong practitioners,” Weil said. “The first question would be whether we designed the experiment the right way.”
He is so certain in the effectiveness of cranial therapy that he is preemptively impugning the quality of a study that would dare fail to confirm his faith in cranial therapy. This theme will be replayed, shortly.
The results of the study:
Interestingly, patients who received Echinacea actually did worse than patients randomized to placebo. Unsurprisingly, cranial therapy did not improve the outcomes of children with recurrent otitis media.
There is a very curious aspect to this study. In the paper Dr. Weil is acknowledged as one of 2 principle investigators (PI) on the study, but neither he nor his co-PI is listed among the authors. In general, the principle investigator(s) is the driving force and team leader for a research program. He writes (or at least supervises the writing of) the proposal for the research grant. If the grant is found deserving, the funding is in awarded in the name of the principle investigator(s). The PI should be the prime mover in the design and execution of the study, as well as the analysis and presentation of the data. It is quite irregular for a PI not to be an author on a publication of the primary results of a research study. According to Dr Weil, he worked for 20 years to accomplish this study. We can only speculate why he chose not to appear as an author on a paper.
One has to give a great deal of credit to Dr. Weil for his successful navigation of the scientific method. He made an observation: Children who receive cranial manipulation seem to have fewer episodes of otitis media.
He formulated a hypothesis: Cranial manipulation may be effective in reducing recurrences in children with otitis media.
He designed a study to test his hypothesis. The study was funded and performed. The results were published. All kidding aside, this was not a trivial accomplishment. Dr. Weil comments on the study in the Frontline intervwiew.
He concludes:
We were unable in those tests to prove that this [cranial manipulation] had an effect.
Gathering evidence and reporting data are critical activities, but the real reward of research is seeing new results become integrated into patient care. In the light of the results of a randomized clinical trial, how have Dr. Weil’s opinions about the benefits of cranial therapy for prevention of recurrent otitis media changed?
The problem is, I’m sure there’s an effect there. We couldn’t capture it in the way we set up the experiment.
Dr. Weil may have some legitimate grounds to be critical of the study: Was it adequately powered? Were study treatments done properly? Were treatment group assignments maintained? Were endpoints captured properly? It is the responsibility of an investigator to consider and discuss all of these things in critiquing his own study.
However…
Considering that the only evidence for the benefit of cranial therapy is an unpublished, self-selected, anecdotal series; when faced with the results of a negative randomized clinical trial, an objective, open minded person must at least concede the possibility that the study failed to document an effect because… it doesn’t work! Dr. Weil, however, is undaunted: “I’m sure there is an effect there.” He leaves no room for any other conclusion. He anecdotal experience outweighs the data from a randomized trial. The science must be wrong.
Now, armed with the data he awaited for 20 years, he must have been relieved to have some evidence to inform his advice regarding treatment of otitis media in children. In response to his own study, he has concluded:
…I can see no harm in recommending to people that they try cranial therapy from a qualified osteopathic physician, even though we still have not yet been able to verify this in a randomized controlled trial.…
I checked his website, and indeed, found this entry (accessed 8/28/12).
He also advises putting warm oils into the ear canal, and opines:
Yes, despite his own study suggesting that echinacea was harmful in otitis media, he is still promoting its efficacy on his website.
Proponents of science-based medicine are often accused of being closed minded for eschewing implausible and unsubstantiated treatments. It is implicitly or explicitly stated that altmed practioners and promoters are open minded. Neither statement is necessarily true, or false. The fact is, one cannot define whether an individual is open minded or close minded based on a snapshot of an opinion on any given subject at any particular time. One must analyze how they came to reach their opinion and how they adjust their opinions base on new information. Does their opinion change when presented with credible, contradictory evidence?
In this instance, Dr. Weil has demonstrated unwillingness to change his opinion or his behavior based on new information. Even when the evidence, his own evidence, betrayed his hypothesis, he was unwilling to acknowledge even the possibility that he might be wrong. He has continued to promote an implausible treatment which failed to demonstrate benefit in a randomized trial, and even continues to recommend a treatment that was associated with outcomes worse than placebo.