Over the past two plus years of the existence of Science-Based Medicine (SBM) we have been highly critical of the National Center for Complementary and Alternative Medicine (NCCAM) – going so far as to call for it to be abolished. We are collectively concerned that the NCCAM primarily serves as a means for promoting unscientific medicine, and any useful research it funds can be handled by other centers at the NIH.
So we were a bit surprised when the current director of the NCCAM, Josephine Briggs, contacted us directly and asked for a face-to-face meeting to discuss our concerns.
That meeting took place this past Friday, April 2nd. David Gorski, Kimball Atwood and I met with Dr. Briggs, Deputy Director Dr. John Killen, Karin Lohman PhD (Director, Office of Policy, Planning, and Evaluation) and Christy Thomsen (Director, Office of Communications and Public Liaison).
Dr. Briggs very graciously began the meeting by telling us that she and her staff have been reading SBM and they find our arguments to be cogent and serious. She shares many of our concerns, and feels that we are an important voice and are having an impact. She then essentially turned it over to us to discuss our primary concerns regarding the NCCAM.
We were prepared for this.
I first pointed out that many of our concerns deal with issues that are outside the purview of the NCCAM director (such as regulation) and therefore we would not bring them up but would rather stick to constructive feedback and concrete ways in which the NCCAM can better serve its mandate. These issues broke down as follows:
Left to right: David Gorski, John Killen, Steven Novella, Kimball Atwood, Josephine Briggs, Christy Thomsen, Karin Lohman
Ethical Concerns
The NCCAM faces particularly complex ethical issues in funding some clinical trials because of the very nature of the topics it is tasked to research – those with low plausibility that have been bypassed by mainstream research. The ethical guidelines of clinical research dictate that before subjecting a person to an experimental treatment, there is sufficient evidence for safety and plausibility of benefit from pre-clinical and animal studies. It might therefore be considered unethical to subject people to experimental treatments that are highly implausible.
The particular study that is most concerning is the TACT trial, and Kimball reviews the specific details of concern here. TACT is a trial of chelation therapy for heart disease. The concern is that chelation therapy is not a benign treatment and there already has been sufficient evidence to conclude that it does not work. Further study is therefore unethical.
Dr. Briggs acknowledged our concerns, but pointed out two things. First, this study came into being before her tenure at NCCAM (she became director on January 24th 2008). Second, the TACT trial has been turned over to NHLBI (National Heart Lung and Blood Institute), who now sponsors the trial, while the NCCAM still partly funds and collaborates on the trial.
What this means is that Dr. Briggs was able to decline to comment on the TACT trial on the grounds that it falls under the aegis of the NHLBI. This effectively cut off discussion on this topic, which is unfortunate.
This does bring up another issue – the NCCAM funds many studies along with other centers at the NIH, and (as with TACT) they intend to allow centers with the proper expertise to take the lead. NHLBI does heart studies, so they took over TACT. This is reasonable, but does have the consequences of effectively increasing the amount of research funding the NCCAM controls, and also provides cover (intended or not) for controversial studies like TACT.
Kimball intends to follow up with the NHLBI regarding TACT and will likely give us an update.
TACT aside, the ethical concerns remain and this is an issue we will have to follow with future studies.
Types of Studies funded by NCCAM
Another core issue we discussed is the fact that the NCCAM funds many studies that are designed to promote CAM in general or specific CAM modalities rather than study whether or not they are effective. Studying how CAM is used, or barriers to CAM acceptance – prior to demonstrating that any particular CAM modality actually works, is putting the cart before the horse.
But there is a more subtle and insidious problem. So-called pragmatic studies are trials that either compare different treatments or follow outcomes for one treatment in real-world practice. They are often not rigorously blinded nor are variables controlled. They are typically “intention to treat” trials where everyone is followed, regardless of whether or not they complied with the treatment.
Pragmatic studies are a very useful way of tracking real world outcomes. It may be true that aspirin reduces strokes and heart attacks, but what happens when a typical primary care doctor prescribes aspirin? Are their patients compliant? Do they run into side effects or other problems that cause them to stop taking the medication? What do primary docs have to do to improve compliance and minimize side effects? All good questions.
But such studies are simply not designed to answer the question – does aspirin work for the reduction of heart attacks and strokes. Efficacy trials are needed for that.
What we have observed in the CAM world, however, is that pragmatic trials are being performed on treatments that have no proven efficacy, and the outcomes are being misinterpreted and presented as evidence for efficacy. For some modalities, such as acupuncture, this is a very deliberate strategy and is being done in response to well-controlled efficacy trials that are negative.
We would therefore like to see NCCAM focus on efficacy trials, especially for treatments that do not already have proven efficacy. Pragmatic studies of unproven therapies are inappropriate and are ripe for abuse.
Dr. Briggs response on this issue was equivocal – she defended the utility of pragmatic studies but also acknowledged our concerns. We ran into the same problem in that, any examples of such behavior more than 2 years old were before Dr. Briggs time. So we will have to simply monitor things going forward.
Never Say Never
Related to the issue of what kinds of studies the NCCAM should fund is the following question – are there any treatment modalities that have been sufficiently shown to be both implausible and lacking in efficacy that the NCCAM should close the door on future research. When is enough enough?
We used our favorite example – homeopathy, which is especially pertinent following the report of the House of Commons Science and Technology Committee in the UK, who concluded that homeopathy is worthless, cannot possibly work, and should be abandoned in all ways.
It seems to us that the NCCAM (at least so far) has never closed the door on any modality, no matter how implausible and no matter how much evidence for lack of efficacy there is. This seems, if nothing else, like a waste of taxpayer money.
Dr. Briggs response was that in the last two years (under her directorship) the NCCAM has not funded any studies of homeopathy, which is true. However, they still accept applications for homeopathic research, but none have made it through the review process and been awarded funding.
This is a tricky issue. Dr. Briggs pointed out that it is not the job of the NCCAM to make final pronouncements about any treatment or medical claim. This is fair enough – but depends on context. The NCCAM is responsible for informing the public about so-called CAM modalities, and that should include a fair assessment of the science. If the science says a treatment is worthless, the NCCAM should not be afraid to say so.
Further, the NCCAM does determine what studies the NCCAM funds. The NCCAM accepts applications for research into homeopathy, but have not funded any in several years. What does this mean? Will they consider funding homeopathy research, and if so they are basically saying that they do not close the door on any medical modality, no matter how implausible or damned by negative evidence.
If they will not consider funding homeopathy, then why are they accepting grant applications for homeopathy research? This could be construed and disingenuous – perhaps a way to not fund homeopathy research without having to say they will not fund homeopathy research.
This leads directly to our final core point of concern.
NCCAM Information
The final major topic of discussion was the information that the NCCAM provides on its website, newsletter, and press releases. In my opinion this is the easiest problem for the NCCAM to address, and one that is completely and solely within their purview – the information they themselves publish.
We were armed with the latest NCCAM newsletter, in which Dr. Briggs is quoted as saying that “Science must be neutral.” Of course, we agree. But in the same newsletter there is article discussing the evidence for acupuncture and pain showing a model of chi and meridians – mystical life force and the lines through which they allegedly flow.
There is also an interview with a member of the NCCAM national advisory board, Xiaoming Tian, a Chinese Medical Doctor. In the article he states that he uses acupuncture to treat a variety of ailments (1. Chronic and acute pain, 2. Osteoarthritis, 3. Fibromyalgia, 4. Sports injuries, 5. Sciatica and neuralgia, 6. Automobile-accident injuries, 7. Autoimmune diseases, 8. Allergies and asthma, 9. Depression, anxiety, and stress, 10. Bell’s palsy and paralysis, 11. Skin rashes and eczema, 12. Side effects of chemotherapy and radiation therapy for cancer.)
The pattern of information is consistent – NCCAM staff talk about a strict adherence to evidence-based medicine and science being neutral, but interspersed with this is an uncritical presentation of ancient superstition as if it were science, and endorsement of treatments that are not backed by science, and in fact have been shown not to work.
It is my interpretation of the evidence that acupuncture has not been shown to work for any indication (as I have written before, the studies show it does not matter where you stick the needles or if you stick the needles, and any benefit appears to be due to placebo effects, artifact, and the non-specific effects of the ritual surrounding acupuncture – none of which constitute acupuncture itself). But I will acknowledge that there can be some reasonable disagreement about whether or not acupuncture is useful for some symptomatic treatment, like pain. The problem is that wishy-washy evidence for symptomatic benefit is then used to support the use of acupuncture for serious medical conditions, like nerve injury. It’s a classic bait and switch.
All of this confirms our worst fears about NCCAM – that its very existence, and the generally positive and uncritical information it provides to the public, is used to promote and endorse unscientific medical modalities.
In fact, it is not enough to be “neutral”, which could easily fall into the trap of false balance (balancing legitimate scientific evidence and analysis with pseudoscientific promotion). The neutrality of science means letting the chips fall where they may – fairly and honestly reporting the state of the evidence without pulling any punches, like the HCSTC did regarding homeopathy.
But it is my experience that the worst thing that the NCCAM will say about a treatment is that there is not “yet” evidence to support its use. The “yet” is often used, but when not it is implied. Almost invariably the lack of evidence leads to the conclusion that “more research is needed.” What we don’t hear is that there is evidence for lack of efficacy, or a recommendation to not use a modality or to abandon further research.
Given that the CAM community is actively exploiting the existence of the NCCAM as an imprimatur of legitimacy, the NCCAM needs to take special care to avoid such exploitation. Meanwhile, it seems that they go out of their way to encourage such exploitation (although it seems just out of naivete) or at least make it easy.
We pointed out that we do not expect the NCCAM to engage in “debunking” (that’s our job). But we do expect that they are fair and do not give a free pass or special treatment to a modality because it’s CAM. That is the double standard we are frequently complaining about.
On a side note, Dr. Briggs did agree that anti-vaccine sentiments are common in the world of CAM and that the NCCAM can do more to combat this. Information countering anti-vaccine propaganda would be a welcome addition to the NCCAM site.
Conclusion
We greatly appreciate Dr. Briggs giving us the opportunity to voice our concerns to her and her staff directly. The meeting was overall very pleasant and constructive. We hope this will lead to an ongoing dialogue and as a result we can help the NCCAM evolve into a more science-based institution. Dr. Briggs did clearly voice her intention to make NCCAM a more rigorous scientific institution, in line with other centers at the NIH.
The one concrete result of the meeting was an offer to have experts from SBM review NCCAM material before it is published. We, of course, agreed to offer our services.
There continue to be very important issues and questions that are at a “higher level” than the NCCAM itself – such as the optimal regulation of medical products and practices, and also whether or not the public is best served by having a center of funding at the NIH which is organized around such a nebulous concept as CAM, rather than a disease or biological system. SBM will continue to address these issues head on.
But we are also happy to work with the NCCAM, and Dr. Briggs does profess her intention to move the NCCAM in a more rigorous scientific direction. We will see.
Addendum:
In response to my comment that NCCAM fails to condemn ineffective treatments, the following entry on the NCCAM site was pointed out: http://nccam.nih.gov/health/silver/
In which the NCCAM definitely states that colloidal silver does not work and is not safe. The wording of this entry (noting what the FDA states about colloidal silver) reminded my that Dr. Briggs did specifically mention that the NCCAM information is and will be in line with FDA positions on specific products.
To further clarify my statement – the NCCAM and even some CAM promoters in my experience will at times condemn specific products when there is evidence of harm, such as with colloidal silver. But this does not extend to treatment modalities, like homeopathy, acupuncture, or therapeutic touch, nor to mere lack of efficacy.
Further the threshold for negative conclusions about CAM modalities seems to follow a double standard, otherwise chelation for heart disease would never have made it past a review board.
