Paul Offit has published a thoughtful essay in the most recent Journal of the American Medical Association (JAMA) in which he argues against funding research into complementary and alternative therapies (CAM). Offit is a leading critic of the anti-vaccine movement and has written popular books discrediting many of their claims, such as disproved claim for a connection between some vaccines or ingredients and risk of developing autism. In his article he mirrors points we have made here at SBM many times in the past.
Offit makes several salient points – the first being that the track record of research into CAM, mostly funded by the NCCAM, is pretty dismal.
“NCCAM officials have spent $375,000 to find that inhaling lemon and lavender scents does not promote wound healing; $750,000 to find that prayer does not cure AIDS or hasten recovery from breast-reconstruction surgery; $390,000 to find that ancient Indian remedies do not control type 2 diabetes; $700,000 to find that magnets do not treat arthritis, carpal tunnel syndrome, or migraine headaches; and $406,000 to find that coffee enemas do not cure pancreatic cancer.”
The reason for the poor track record is fairly simple to identify – by definition CAM includes treatments that are scientifically implausible, which means there is a low prior probability that they will work. If the treatments were scientifically plausible then they wouldn’t be alternative.
CAM proponents argue that the treatments they advocate, like acupuncture, certain herbal remedies, and homeopathy, are not implausible, they are just neglected by mainstream medicine because they don’t fit into the narrow (and profitable) paradigm of “Western” medicine. This argument, however, is demonstrably wrong. Homeopathy is rejected by the mainstream because our current understanding of physics, chemistry, and biology tell us that it is impossible for homeopathic potions with extreme dilutions to have any physiological effect.
I admit that one benefit of all the research that the NCCAM has funded is to test these two versions of reality. Are CAM modalities scientifically implausible or are they just not being given a fair shake by modern medicine? If the former then research into CAM modalities should be largely negative, if the latter than we should regularly be finding CAM diamonds in the rough. Well, after 1.6 billion dollars of research any score of studies the NCCAM has left behind it a trail of negative studies, such as those listed above. This strongly supports the SBM view that “alternatives” to science-based treatments are not science-based.
CAM research, therefore, is like playing the lottery – the chance of winning is so small it is not significantly different from zero, but if you get really lucky perhaps you may hit upon something. As a society we have to decide if this is a worthy investment of our limited research dollars. To continue this analogy, financial advisers often recommend a range of investments from conservative to risky, but I don’t think they would consider buying lottery tickets part of a sound financial plan.
The first major criticism against doing research into CAM, therefore, is that it is simply a waste of resources – not just research money but all the components of the infrastructure of research. This includes access to sick patients. Patients who enter into a clinical study of a low-probability CAM modality may therefore not be available to enter into a study of a more plausible treatment.
That most of the studies funded by NCCAM are negative raises another point discussed by Offit – the value of negative studies. Offit acknowledges that negative studies in medicine can be very valuable. It is important to know what doesn’t work, especially if it is a treatment that is already being used. In fact I and others at SBM have argued that journal editors need to make more room in their journals (or at least in the online versions) for negative studies (and also for replications, but that is a separate issue), in order to limit the “publication bias” toward positive studies.
This, I feel, is the one legitimate argument for doing CAM research – the value of solid scientific evidence for lack of efficacy of a treatment that is being promoted, even on the fringe. But Offit raises a very important point – does this negative evidence have any effect on the practice of those who are promoting or using CAM modalities? The answer is largely, no. He gives as examples many supplements, like gingko biloba or echinacea, that continue to have robust sales even after large rigorous studies found they do not work.
There is some wiggle room in the data to spin it in more than one way. NCCAM director Josephine Briggs has argued that negative studies from the NCCAM have decreased the sales of specific herbal remedies, like echinacea. This is true, but the decrease was modest and temporary. Sales figures indicate that echinacea remains a popular herb and overall herbal product sales continue to increase (over $5 billion dollars in the US in 2009). The relative popularity of specific herbs is affected by large published studies that gain some media attention – but not dramatically.
The real problem is that regulation of herbs and supplements are not adequately tied to scientific evidence. Aggressive marketing can therefore have a greater long term effect on the popularity of a CAM treatment than a published study that the public soon forgets. Also, the media often does a poor job of putting published scientific studies into a proper context. For every large rigorous negative study, there are many small, preliminary, and positive studies. In the media, therefore, the positive will tend to get more headlines and more overall attention, even though their relative scientific value is much lower. The media just reports – “a study showed.”
There are also other issues of concern with CAM research. One is ethics – the ethics of biomedical human research is such that we owe it to people we experiment on to maximize the probability that the experimental treatment will be safe and effective. That is why there is the need for a great deal of pre-clinical and preliminary clinical studies before going to a large human trial. Researchers also have to justify their treatment with sound science that indicates it is plausible, that it is likely to be of benefit to the study subjects.
The entire notion of plausibility, however, was thrown out by the very concept of CAM. Offit argues that many CAM modalities “border on mysticism.” I would argue that many of them are mysticism or thinly veiled versions of faith healing. Not only is there no known mechanism, but there is no known way they can possibly work. It is one thing to not know what receptor is the target of interest for a specific effect of a drug, it is another to violate basic concepts of physics and chemistry.
In other words, some CAM modalities are the equivalent of magic. Is it ethical, therefore, to study magic therapies on human subjects? Does this violate the ethical requirement of informed consent? Even worse, in some cases there is already adequate evidence for lack of efficacy (such as the chelation trial that Kimball Atwood has been criticizing).
Conclusion
There is much to criticize in the funding of medical research into highly implausible treatments – they are a waste of resources with little probability of resulting in effective treatments, while negative evidence is useful, users of unscientific treatments don’t listen very much to the evidence, and there are significant ethical concerns.
I propose that as a society we strike a bargain with the proponents of so-called CAM. We will fund and conduct research into CAM modalities where it is reasonably ethical to do so, but in exchange treatments for which there is evidence of lack of efficacy will be abandoned. Further, regulations for health products (like herbal remedies) will better reflect the scientific evidence. I would prefer that evidence of safety and efficacy would be required before marketing, but failing that the FDA should have the power to remove a product from the market after research shows that it is ineffective (right now the FDA must meet a difficult burden of proof of harm to do so).
So, if the evidence shows that homeopathy does not work, the homeopathic industry will vanish. If a rigorous study shows that chiropractic manipulation does not work for asthma then chiropractors will condemn the practice and stop doing it.
However, if scientific evidence does not significantly affect practice or product sales, then why should we pay for it? Further, if CAM continues to be disconnected from scientific evidence, why should it enjoy any legitimacy?
