Over the last couple of days I have been engaged at NeuroLogica in a discussion with a fellow blogger, Marya Zilberberg who blogs at Healthcare, etc. Since the topic of discussion is science-based medicine I thought it appropriate to reproduce my two posts here, which contain links to her posts.
A Post-Modernist Response to Science-Based Medicine
I receive frequent commentary on my public writing, which is great. The feature that most distinguishes blogs is that they are conversations. So I am glad to see that science-based medicine (a term I coined) is getting targeted for criticism in other blogs. One blogger, Marya Zilberberg at Healthcare, etc., has written a series of posts responding to what she thinks is our position at Science-based medicine. What she has done, however, is make many of the logical fallacies typically committed in defense of unscientific medical modalities and framed them as one giant straw man.
She is partly responding to this article of mine on SBM (What’s the harm) in which I make the point that medicine is a risk vs benefit game. Ethical responsible medical practice involves interventions where there is at least the probability of doing more benefit than harm with proper informed consent, so the patient knows what those chances are. Using scientifically dubious treatments, where there is little or no chance of benefit, especially when they are overhyped, is therefore unethical. And further, the “harm” side of the equation needs to include all forms of harm, not just direct physical harm.
Zilberberg’s response is the typical tu quoque logical fallacy — well, science-based medicine is not all it’s cracked up to be either, so there. She writes:
Now, let’s get on to “proof” in science-based medicine. As you well know, while we do have evidence for efficacy and safety of some modalities, many are grandfathered without any science. Even those that are shown to have acceptable efficacy and safety profiles as mandated by the FDA, are arguably (and many do argue) not all that. There is an important concept in clinical science of heterogeneous response to treatment, HTE, which I have addressed extensively on my blog. I did not make it up, it is very real, and it is this phenomenon that makes it difficult to predict how an individual will respond to a particular intervention. This confounds much of what we think is God’s own word on what is supposed to work in allopathic medicine.
This is also the fallacy of the perfect solution — since science-based medicine is not perfect, there is no legitimate basis for criticism of any modality. This is also premised on the false dichotomy of “allopathic medicine” (a derogatory term only used, in my experience, by defenders of dubious medicine) vs “alternative” medicine (which I will refer to as CAM for short). I and others at SBM have been clear that we eschew this false dichotomy. There is only medicine with varying degrees of plausibility and evidence — there is a continuum, and we advocate always using the best that is available. We also think there should be a minimum standard, a fuzzy line of plausibility and evidence below which treatments should only be given with proper informed consent as part of an approved clinical trial. And a further line below which even research is unethical because there is no plausible potential for benefit.
These principles are, in fact, already part of ethical medicine. We did not invent these concepts. It is, rather, the proponents of CAM who wish to do away with this ethical standard — to create a false dichotomy in order to establish a double standard. We are not trying to create a new standard, just to do away with the double standard of CAM.
She refers to the heterogeneous response to treatment, again as if realization of this basic fact is not already part of science-based medicine. I, in fact, explain this to patients all the time. Our knowledge of treatments is based upon statistics, but we can never know ahead of time how an individual patient will respond. What’s the alternative? Until we get better at predicting individual response (which really will just be another application of statistics), this is the best we can do. That is why you monitor the individual response to any treatment, and act accordingly. This is basic medical student stuff, but Zilberberg acts as if this is a big revelation for science-based medicine.
We at SBM advocate for the highest scientific standards of medicine, and apply that across the board — including with pharmaceutical companies, surgeons, and anything else that is labeled as “mainstream.” Again — we do not make false categories and distinctions. It is all medicine.
The reference to “God’s own word” is an obvious allusion to the bad-old-days of paternalistic medicine (dead and buried for decades now), or the TV caricature of a doctor with a God complex. This is a typical ploy — portray any attempt at defending a scientific standard in medicine as paternalistic arrogance. In fact, Zilberberg dedicates an entire blog post to this fallacy. She writes:
First of all, it is my belief that all interventions should be approached with equanimity, if not equipoise. Although I am quite dubious that either healing crystals or Reiki can produce actual results, I do not want to confuse the absence of any evidence to this effect with the evidence of absence of the effect. Although I am not that interested in allocating resources to studying these fields, it would be paternalistic of me to bar their further investigation. So the society can decide what it wants to do with them, and in the meantime every individual can make her/his own choice whether to spend their money on them.
This is clearly where we differ. I do think, from reading her writing, that Zilberberg means well and is sincere in her positions (unlike some I criticize who I feel are just trying to sell something). But notice the logical contortions in her position — she wants “equipoise” with regard to all interventions, and would not dare dictate how research money is spent. I should point out that there is a range of opinions on SBM when it comes to regulation — so we are not a united front on this score. We range from libertarians who think that we should educate the public and professionals, but are against laws that would restrict access to unscientific modalities. Essentially, people have the right to make stupid decisions. Others believe that there needs to be a minimum safety net against fraud and quackery, and in fact the public wants there to be one and believes there already is one. I don’t want to get bogged down in this debate on this blog entry — I am just pointing out that Zilberberg’s premise is overly simplistic and paints with too broad a brush.
But the real point here is that she is taking an almost post-modernist position that we need to approach all claims in medicine with “equipoise.” She says that society can decide how research money is spent, even if she would not personally research an implausible topic. Depending upon how you slice it, this is not necessarily far off from my position. If people want to raise money to research an implausible question they should go right ahead. I never proposed banning implausible research. My position, rather, is that we should not waste limited public/government research resources on highly implausible modalities.
I would also add, however, that once you start doing research on humans there is a host of ethics that also come into play. In human research it is the accepted ethical standard that subjects should at least have a chance of benefiting from the treatments being studied, or at least there should be a greater chance of benefit than harm. I don’t see how this ethical standard can be met with homeopathy, for example, where there is essentially zero chance of benefit. At some point you pass a line of infinitessimal plausibility where the ethics become problematic.
Zilberberg then makes the “absence of evidence vs evidence of absence” mistake — really an oversimplification of this concept to the point of being wrong. While it is true that the absence of evidence if not necessarily evidence of absence — it can be, depending upon how thoroughly you have looked. If I search my house for a specific item and don’t find it, that is pretty good evidence that it’s not there. It is not “proof” of absence, but it is evidence. With many of the modalities that Zilberberg admits she is personally dubious about there is evidence of absence of an effect. This evidence comes in two forms — all of the science that tells us the modalities are highly implausible, and often there is clinical evidence of lack of an effect. To pretend otherwise is dishonest — it is hiding from the facts out of political correctness.
Further, our patients do not want equipoise from us. They want our informed opinion. When patients ask me if they should take a homeopathic remedy I don’t give them a wishy washy answer. I give them my informed opinion, and they are grateful to have it. In the comments to her blog a commenter speculates about my bedside manner, assuming, essentially, that I must be a paternalistic ass. This is the typical cardboard caricature I encounter, and it has no relationship to reality. It is possible to give patients useful information without being judgmental. To give them informed consent (how do you do this, by the way, without giving them information?) but understand that they will make up their own minds. Patients are in charge of their own health care, and our job as clinicians, more than ever, is to give them the information and perspective they need to make good decisions. This does not demand “equipoise”, but evidence and perspective. In my opinion equipoise in the face of ridiculously implausible claims and evidence of lack of efficacy is a disservice to patients and a violation of trust.
Ironically, Zilberberg concludes:
Bottom line, we need to appreciate that none of the science is all that straightforward. Let us not dumb down the arguments and create false dichotomies. If we do, no one wins.
Does she actually read science-based medicine? I am left to wonder — since we regularly argue for the complexity of the science of medicine. I want people to understand how complex the relationship is, so they are not shocked every time conflicting studies come out. Medical science is a messy business, and it is challenging often to infer what the best approach is. I want the profession and the public to have a much more nuanced understanding of medical science, and for the media to do a better job of representing it.
This is especially true since we do not have a paternalistic system. Patients are partners in their own health care, and therefore it helps me do my job when they understand the science that underpins medicine.
Zilberberg’s position is anti-science, although perhaps not deliberately so. It is anti-science in a post-modernist sense. She points out all the limitations of science, as if that means we cannot come to any meaningful decision, and therefore must treat all claims as equal. But all claims are not equal. Even the best are imperfect, but we can still apply science and evidence to make informed decisions about the probability of risk vs benefit. And there are some claims that are so against science and evidence (like homeopathy) that any stance other than rejection is a violation, in my opinion, of medical ethics and the trust that society places in medical professionals.
In Zilberberg’s world, however, any such judgments are the equivalent of pronouncing that these treatments over here in pile A are deemed “scientific” (as if by the word of God) and are accepted. And these over here in pile B are deemed “nonsense” and are to be ridiculed. But the false dichotomy is in her mind, not in science-based medicine. We are the ones railing against the false dichotomy — that of CAM which seeks to create a double standard. All we advocate is one consistent standard of science and evidence when evaluating all medical claims, and the rational application of science to the practice of medicine.
One final note — I would much prefer to have a conversation with the critics of science-based medicine that does not constantly involve defending SBM and myself from false accusations of arrogance and paternalism. I think it says a lot about their intellectual position when that is constantly the best they have.
Dr. Zilberberg Responds
Dr. Zilberberg responded to my original post and significantly modified her tone, to her credit. (She was simultaneously responding to Orac’s analysis of her posts as well.) Here is my analysis of that post.
The Tone Thing
I will address her main points below, but first my final thoughts on the “tone” thing. While she admits fault in setting the “confrontational tone,” I don’t think she quite gets what Orac and I were objecting to. I actually don’t mind a confrontational approach — as long as it is substantive (that’s the way science works — if you have a point to make, bring it on). We were objecting to her mischaracterizing our position and making ad hominem attacks in place of substantive criticism — essentially using the “arrogant” gambit with which we are all too familiar. Her readers obviously picked up on this, and piled one, accusing us of being bullies and thanking her for slapping us down. We objected to her logical fallacies, not her tone.
Interestingly Zilberberg’s initial response was dismissive, and she reiterated the charge of paternalism and arrogance, writing: “If the shoe fits?” At least now she seems to realize that if we are going to have a productive discussion, focusing on ad hominem attacks will be counterproductive.
Incidentally, having written about medicine for years I have definitely seen a strong pattern. When I criticize the logic and factual premises of another person’s argument I am frequently accused of being mean by people who then attack me personally. It seems many people do not understand the difference between a strong but substantive criticism and a personal attack. Zilberberg was falling into this category, but has significantly (if incompletely) backed off from that with her latest post.
One more minor point — “allopathic” is derogatory and does not apply to modern medicine (it was coined by Samuel Hahneman to refer to the poisons that passed for medicine in his time, and was definitely meant to be a criticism). I would suggest she drop this term rather than defend it.
Evidence in Medicine
Zilberberg then launches into a meaty discussion of what her position actually is. She observes that perhaps we are not that far off in our positions, which I think is true. There is a meaningful difference in spin — the final conclusions drawn from the analysis, but her analysis of the role of evidence in medicine is reasonable. But again, to clarify, Orac and I were not objecting to the point that evidence in medicine is messy and complex. We were objecting to the accusation that we do not understand this, and that we are promoting an overly simplistic and cheerleading approach to science in medicine. This left me with the impression that Zilberberg has not read deeply into the Science-Based Medicine website, or at least has failed to grasp what it is we are actually saying.
If she had she would have seen post after post in which SBM authors were pointing out all of the complexities and deficiencies of evidence in medicine that she and others might also point to. That is core to the point of SBM — evidence is complex. She might, in fact, have read my series of posts on evidence in medicine. We do spend a great deal of time pointing this out in the context of so-called CAM, because CAM proponents are the ones who most profoundly take a simplistic approach to the evidence. They engage in black-and-white thinking, display intolerance of ambiguity, and frequently advocate for the reliance on very problematic low-grade evidence to support their claims. But we also consistently apply the same standards to surgery and the pharmaceutical industry, and anything “mainstream.’
Zilberberg reviews the relative roles of experimental evidence vs observational evidence. Her analysis is reasonable, but I think she overstates the utility of observational data a bit (and she admits to a fondness for this type of data). The bottom line is that each type of evidence (basic science, observational, and experimental — and even anecdotal) has its own strengths and weaknesses, and the best result comes from analyzing all kinds of scientific evidence and looking for a consensus of evidence. That is, in fact, OUR criticism of evidence-based medicine -over reliance on randomized controlled clinical trials and undervaluing other forms of scientific evidence. That is why we advocate for “science”-based medicine, and not just “evidence”-based medicine.
Each type of evidence, in fact, is abused. We criticize the inappropriate extrapolation from basic science to clinical claims, assuming causation from observational correlation, failure to realize the limits of clinical trials, and the use of pragmatic studies as if they were evidence for efficacy.
Zilberberg also clarifies her position by saying that she feels there is good scientific evidence for some of medicine, but it seems she differs from my position in how evidence-based modern medicine actually is.
We can argue endlessly about this question — how much of modern medicine is based upon solid evidence — each pointing to limited examples and essentially giving our bias. But there are some facts we can point to. Zilberberg writes:
While it is true that the oft-cited 5-20% number representing the proportion of medical treatments having solid evidence behind them is very likely outdated, the kind of evidence we are talking about is a different matter.
The “5-20% number” is not outdated — it’s a myth. Actually, I had previously heard 15% as the low end, but I guess that number keeps dropping. I wrote previously about this myth here. The 15% number was based upon an extremely small survey of primary care practices in the north of England — in 1961. That’s almost 50 years ago. The number was never very relevant, and now it’s a joke.
More recent surveys of medical practice come to very different numbers. Bob Imrie reviewed the published evidence:
Thus, published results show an average of 37.02% of interventions are supported by RCT (median = 38%). They show an average of 76% of interventions are supported by some form of compelling evidence (median = 78%).
Of course, where you draw the line for “supported by compelling evidence” will determine what the percentage figure is. But the bottom line is that the 15% figure is basically an urban legend, and “5%” is nothing short of propaganda. More reasonable estimates range much higher.
And — the point of EBM and SBM is that we can and should do better. We also need to do better in adhering to EBM guidelines where they exist, and in utilizing continuing medical education and other mechanisms of quality control to improve adherence to the evidence where it does exist.
The difference in spin is not subtle. We can look at the evidence and say: modern medicine has a culture of science, endeavors to be scientific, and basically the system works but the process is complex and messy and there are multiple ways in which we can do better. Meanwhile someone else can look at the same data and conclude: modern medicine is broken, it is based upon arrogance, authority, and greed, and we can just throw up our hands and conclude that any treatment is as likely to be of value as any other, no matter how silly it may seem scientifically.
My position is essentially the former. Zilberberg came off originally as being close to the latter (and judging by the comments, many readers took her position to be supportive of the latter), but now has clarified that she is somewhere in the middle.
CAM
Zilberberg also clarifies her position on CAM. She had previously written that she advocates a position of “equipoise” towards clinical claims. Even though she might not use certain modalities herself, she sees no basis to condemn the use of them by others. I characterized this position as political correctness gone wild — to the point of practical post-modernism. Now she writes:
My belief is that all modalities that may impact what happens to public’s health need to be evaluated for safety, not question. I think we both agree, since there is really no reason to think that something like homeopathy has anything that can help, by the same token we do not believe that it have anything that can hurt. Same with healing crystals, reiki and prayer. So, if a person wants to engage in these activities, and they are perfectly safe physically, be my guest. Other modalities, such as chiropractic, acupuncture, herbalism and the like, definitely need to be evaluated more stringently, as there is reason to think that they may cause harm.
This is a common position to take. Val Jones at SBM coined the term “shruggie” to refer to this position — in essence, if there is no direct harm, then who cares what people do. First, as I discussed very recently on SBM, there are many types of harm from unscientific medical modalities other than direct physical harm. So I do not find this position tenable for that reason alone.
Further, context is everything. There are actually a variety of positions that authors at SBM take when it comes to regulating medical practice. We all generally believe that medical professionals should not engage in nor promote unscientific methods. In fact, we should oppose their adoption and promotion, we should oppose their inclusion in universities and mainstream hospitals, and spending public funds on researching extremely implausible or already disproven modalities. That seems to be a point of difference between myself and Zilberberg.
I personally do not oppose individuals doing whatever they want when it comes to their own health. If you want to chew on tree bark (a vivid example given to me by someone else), go right ahead. What I object to is someone selling the tree bark and claiming that it cures cancer based upon nothing but legend and anecdote, and scaring their customers away from proven therapies in order to make the sale. I object to distortions of logic and science in order to confuse the public so as to better market worthless or harmful products. And I object to medical professionals looking the other way out of misguided political correctness, or simply a naivete as to the significant harm that is done.
SBM has a huge consumer protection mission, and it puzzles frustrates me that this mission is so often and so thoroughly misrepresented. This misrepresentation is deliberate — part of the “health freedom” movement — and seeks to portray all health care consumer protection activity as arrogant elitism and protectionism. This is identical to the intelligent design movement’s representation of all attempts at quality control in education as arrogant elitism.
What I don’t understand is Zilberberg’s apparent position that, while she knows homeopathy is utterly worthless, a physician should refrain from telling her patients exactly that.
Vaccine Skepticism
Zilberberg goes on to argue that she is not anti-vaccine, as she has been accused (not by me). I have no reason not to accept her word on this, and it is good that she has clarified her position.
But I do think she is displaying a lack of appreciation for the nature of the anti-vaccine movement. As an example, if one publicly expresses doubt about an aspect of currently accepted Darwinian evolution it would be nice if they understand the many ways in which the scientific discourse is exploited by creationists, so that they don’t accidentally give succor to an anti-scientific movement.
Likewise, any public discussion about vaccines, while it should be candid and completely honest, should ideally be done with an adequate familiarity with the anti-vaccine movement’s propaganda so that one’s words and positions are not easily exploited. In fact, while expressing skepticism about a particular vaccine or vaccine program, I would recommend specifically clarifying one’s position to distance themselves from the extremists. Otherwise you are inviting misinterpretation.
Conclusion
The take home message from this exchange is that, in my opinion, accusations of using harsh tone or of arrogance are an ad hominem distraction from the real issue — what is the optimal relationship between the practice of medicine and the underlying science of medicine.
Zilberberg engaged fully in this distraction, but is now slowly backing away (but not enough, in my opinion). I think this was largely due to the fact that she has been taken in by the very active and sophisticated propaganda campaigns of CAM proponents. She seems to have bought into their rhetoric, and did not read carefully enough into our writing at SBM to see through it.
We are approaching 1000 blog posts at SBM. I don’t expect critics to read every post, but a tiny bit of scholarly due diligence would be nice, before essentially buying into the lies and distortions of our critics.
We at SBM write frequently about the complexity and limitations of the science of medicine. That is our mantra — a nuanced and sophisticated approach to evidence is needed. But at the end of the day, some treatments are better than others. We can accept and reject practices based upon plausibility and evidence, even while there is a vast gray zone in the middle where we just don’t know yet.
It is misleading and ironic in the extreme to criticize promoters of SBM for taking a simplistic approach to evidence. That is the opposite of the truth. Meanwhile, promoters of all sorts of so-called CAM do take a simplistic and highly distorted approach to evidence, display an intolerance of uncertainty, systematically misrepresent the evidence to their clients and the public, think in stark black-and-white terms, engage in bait-and-switch deceptions, distort the positions of their critics, rely upon low grade evidence and logical fallacies for their claims, and then hide behind political correctness, post-modernism, distractions about “health care freedom”, special pleading (science can’t test my claims), and accusations of arrogance and paternalism.
All of this behavior is carefully documented in the pages of Science-Based Medicine. Would-be critics of SBM should try reading some of them before launching into misguided criticism of what is ultimately a straw man of our actual positions.
I take Zilberberg at her word that she is interested in genuine discussion, and she has at least moved in that direction. I recommend she step back, read some more of SBM and see what we actually have to say about science and medicine.