One sign of good journalism is that it questions the prevailing narrative. Common knowledge is mostly wrong because it tends to drift downstream towards simplistic narratives that are emotionally appealing. That is why whenever a topic about which you have deep knowledge is portrayed in the popular culture it always seems simplistic to the point of being wrong, full of tropes and common misconceptions.
An investigative journalist, however, should dig beneath the surface. They should talk to experts, especially those who have reason to doubt the narrative, and find out what the consensus of expert opinion is. They could also try to explain the narrative – why do people believe X when experts say Y?
You will find none of this evidence for actual journalism in a recent New York Times piece by Jane Brody in the “Personal Health” section. What you will find is a completely credulous retelling of the standard “alternative medicine” narrative, complete with the most common tropes and logical fallacies, and without the slightest hint of skeptical journalism.
The article is based largely on the book, The Other Side of Impossible, by Susannah Meadows, which is also a simplistic bow to the standard CAM narrative.
Brody writes:
The families’ ventures into a realm that some would call quackery were typically inspired by love, desperation and hope, and were fueled by irrepressible grit and determination to find solutions to debilitating health problems that defied the best that conventional medicine could offer.
That is the narrative in a nutshell – framing it in stark emotional terms suitable for an after school special (do they still have those?). While I agree that patients who suffer from difficult medical problems often face them with grit and hope, there is a lot more nuance to this reality than that greeting card summary allows.
Brody could have spoken with someone who actually treats patients to get a more complete perspective. First, most patients who seek alternative treatments do not do so because they are unsatisfied with mainstream medicine. That is the common assumption, but it’s wrong. The most common reason offered is that they heard good things and thought it worth pursuing. People specifically reference positive reporting in the media as the reason they were open to pursuing unconventional treatments, so articles like Brody’s are as much the source as the result of this feedback loop.
Some patients do seek dubious treatments out of desperation, but that is not a good reason to do anything. Desperation as a motivator does not usually lead to careful calculations of risk vs benefit.
Further, many patients in the same situation do not resort to quackery. They focus their resources on maximizing their quality of life, on interfacing effectively with their evidence-based practitioners, and on living their life. This shows how important narrative framing is. You could easily portray grit as stubbornness, and determination as foolishness.
The core of the problem with Brody’s reporting, however, is that it focuses on extremely cherry picked anecdotal evidence. She relates stories from the book about patients or their families who defied the medical establishment, pursued unusual treatments, and are doing great (or hope to be doing great in the future, heads I win, tails you lose).
Here is a perfect summary of this highly problematic approach:
“With nothing to lose — if it helped one child, maybe it will help ours,” Ms. Meadows said. “In terms of hope, an example of one is very important.”
Quite often those who think they have nothing to lose will learn that they actually do. Faced with a serious illness you might think that your life could not get worse, but that is mostly either a lack of imagination or an unwillingness to face the possibility of even worse outcomes.
This is just another way of stating the “What’s the harm” trope. Any doctor can tell you the potential harm of pursuing dubious treatments. There is sometimes direct physical harm. There is the opportunity cost of not pursuing more likely treatments, or just living your life. There is emotional harm from false hope and betrayal. There is harm from sowing mistrust in actual professionals trying to give you evidence-based advice. And there is the potential for massive financial harm, not only for the patient but their families.
We have already explored at length the problems with anecdotal evidence. Meadows’ casual dismissal of these problems by stating that “an example of one” has meaning in terms of giving hope is itself a harmful deception. You could probably write a book filled with stories of people who played the lottery as the solution to their financial situation and won. You could cherry pick the winners, and then explore what they did to choose their winning numbers. They were hopeful, perhaps even desperate, but the universe delivered. Don’t talk to financial experts about the benefits of investing, or talk to people who wasted far too much of their limited income on lottery tickets. Don’t look at data about the fact that state lotteries used to finance programs is really a regressive tax that disproportionately hits the poorest citizens, luring them in with a statistically negligible hope. Focus on the hope and the winners.
Of course, only the most dramatic examples are going to make it into such a book. They are by definition not representative. There are also all the other problems with anecdotal evidence. We don’t really know if the original diagnosis was accurate. We don’t know which of the many treatments used were actually effective. We don’t know that the outcome wasn’t just the natural course of the illness or a spontaneous remission. There is also a lot of subjective judgement involved.
I see patients frequently who credit a worthless treatment for feeling better, and yet at the same time they are in my office complaining that their symptoms are unbearable and as bad as they have ever been. They don’t seem to see the contradiction in these two positions. In order to tell if a treatment is actually working we need either objective outcomes measures and/or blinded evaluation (preferably both).
Finally Brody appeals to the fallacy of emerging breakthroughs. The problem with emerging breakthroughs is that they have not happened yet. In a way you are trying to predict the future, and such predictions have a dismal history. Most new ideas, new targets for therapeutics, new theories of disease, will turn out to be wrong or incomplete, or will not lead to an effective treatment. Even when we are onto something, the complex homeostatic systems in the body prove to be far more complex and not so simple as to yield to easy interventions.
But new ideas in medicine sound cutting edge and sexy, and it is not difficult to extrapolate from such theories to semi-plausible sounding interventions. This gives the patina of science to what is essentially just guesswork. We now know that most of these guesses will be wrong, sometimes even harmful. Further, as science progresses and we pick all the low hanging fruit, new solutions are getting more and more complex.
The power of narrative
It is extremely disappointing that a NYT reporter pandered to such a simplistic and harmful narrative without the slightest bit of skepticism. There is, of course, a competing narrative, one informed by science and experience and known to most experts. That narrative is that brave and determined patients facing a serious illness do so with grit, but also an acceptance of reality. They make the best of the situation, without caving to desperation and fear. They do everything they can within logic and reason, and listen to the advice of objective experts.
They resist the siren song of conspiracy theories, false hope, simplistic advice, and the dubious claims of quacks who just want to empty their wallets. It may take real bravery to not bankrupt your family, who would never dream of saying no, just so you can play the healthcare lottery rather than face your situation with dignity.
In this narrative dubious treatments are “alternative” for a reason, because they don’t make sense, or they are highly speculative, or we already know that they don’t work. They often defy our basic understanding of how things work. In order to embrace such treatments you have to reject a lot of science, which means rejecting mainstream medicine. This leads to a dark place of conspiracies and malfeasance, where the only people who can be trusted are gurus selling you magic beans.
The health care community collectively carefully considers risk vs benefit to do the best for each patient. Of course many individual practitioners fall below this ideal, but there is a community standard based on carefully-thought-out reality. Going outside this system means that you are deviating from an optimal risk vs benefit approach. This further means that statistically your outcomes are going to be worse. You can hope you are the one person who wins the lottery, but I would not bet on it.
Another part of this narrative are the practitioners who are offering the dubious treatments. They are not science-based, or ethical. They do not follow due diligence, have proper humility, adhere to a common standard, or serve their patients’ best interests. Some of them are straight-up con artists, many are just misguided, but in the end it doesn’t really matter to the patient.
Their business of selling fake and dubious treatments is dependent on first selling a narrative in which reality is turned upside down. Brody and the NYT have just sold that narrative to their readers, and quacks everywhere are thankful.