The only U.S. media watchdog devoted exclusively to health news, HealthNewsReview.org, will shut down at the end of the year for lack of funding, according to founder and publisher Gary Schwitzer, a 45-year journalism veteran. As a big fan, loyal subscriber, and occasional critic of HealthNewsReview.org (HNR), I consider its anticipated demise a huge loss to the science-based medicine community.
Unlike shoestring-budget, volunteer-run blogs like Science-Based Medicine, which has done its fair share of media criticism, HealthNewsReview.org had the money and staff to maintain a laser-like focus exclusively on healthcare journalism and, starting in 2015, healthcare press releases. (We’ll get to some examples of their excellent reporting, and one boo-boo, in a moment.) At present, in addition to paid editorial staff, HNR has about 50 free-lancers, half of whom work pro bono. Its institutional home is the University of Minnesota’s School of Public Health. HNR’s main funder, the Laura and John Arnold Foundation, won’t renew its $1.6 million, two-year grant, which runs out this year. Schwitzer has spoken with at least 25 potential donors, but, so far, no takers. Schwitzer says he will still post occasionally and keep the website up for two years.
HNR believes all good news stories about health care interventions must address 10 factors, including cost, harms, quality of the evidence, conflicts of interest, alternatives, and whether the story commits “disease-mongering.” Since its founding in 2006, the non-profit organization has reviewed about 2,500 health news stories based on its 10-criteria checklist. HNR also evaluated nearly 500 healthcare press releases based on similar criteria, offering writers a pre-publication review service for their news releases by a team of clinicians and journalists. (Cynically, I can’t help but wonder whether some PR flaks really want to know if their work product meets HNR’s criteria before publishing.) In addition, HNR published more than 2,600 blog posts on a myriad of healthcare topics, as well as “toolkits” on medical subjects too often mangled by the media and misunderstood by the public, like medical tests, number needed to treat, surrogate outcomes, and non-inferiority trials.
Regular readers of SBM will not be surprised to learn that, when measured against HNR’s criteria, the media often fell short in healthcare reporting. In 2017, HNR analyzed the performance of over 11-years-worth of stories, reviewed on the five criteria they deem most important. The results show that, more often than not, the news stories were inadequate in their discussion of key elements in assessing a treatment or other medical intervention: 69% did not adequately discuss cost of an intervention; 66%, how big (or small) the benefits were; and 63%, the potential harms. The quality of the evidence was not evaluated in 61% of the stories and alternative options weren’t discussed in over half. Press releases fared even worse, with a whopping 92% failing to adequately discuss cost.
Most of HNR’s reviews are of news stories about “conventional” medicine. For example, Newsweek recently got a 3-star (out of 5) score for its reporting on development of a saliva test to identify men who are at the greatest risk of developing prostate cancer. HNR found the story lacking for failing to adequately discuss cost, benefits, harms, and the quality of the evidence, and for failing to consult sources independent of the study for comment. A recent 1-star review titled “NPR (yet again) writes uncritically about ketamine for mental illness” (ouch!), included these comments
The story relies largely on a single anecdote of a person who says ketamine “helped me get my life back.” The expert sources use only vague descriptions of ketamine as “transformational” and that patients in one trial “got dramatically better.”
And:
The only two experts quoted in the story are research collaborators who have long advocated for expanded use of ketamine treatment. One of them, Demitri Papolos, has applied for a patent on a ketamine delivery device.
HNR takes on SBM’s favorite subjects
Anecdotal evidence, nebulous claims of benefit, undisclosed financial interests – they could have been talking about media coverage of alternative/complementary/integrative medicine. In fact, HNR has reviewed a number of news stories about CAM and “natural” remedies, including several subjects that have drawn our ire here at SBM over credulous reporting. HNR warned reporters in a blog post that, when writing stories about CAM:
journalists still need to scrutinize the findings as closely as they would any other study.
Exactly!
Last year, HNR called out credulous reporting in HealthDay on a study of acupuncture in the ER. (David Gorski covered the same study here on SBM, including his own criticism credulous media coverage). As HNR pointed out:
the story makes the claim that “acupuncture is a safe and effective alternative to pain medications for some emergency room patients, a new study reports.” None of the data presented in the story proves it is a safe and effective alternative . . . Instead, the data just shows that none of the pain interventions worked very well.
HealthDay was also criticized for failing to consult outside sources, instead quoting only the lead investigator of the study. (Or, as David more colorfully put it, allowing him to “pimp his paper’s findings.”)
HNR ’s review of a University of Maryland press release (which garnered a measly 1-star score) was the first to question University researchers’ highly implausible claim that chocolate milk improves concussion symptoms. This led to widespread media coverage challenging the University’s remarkable assertion and, after dogged pursuit of further information by HNR and much stonewalling by the University, ultimately, a committee investigation.
On advice of the committee, the University took down the press release from its websites, returned some $200,000 in funds donated by (surprise!) dairy companies to the lab that conducted the study, and implemented over a dozen committee recommendations to “bring the university’s procedures in line with accepted norms,” including revision of its conflict of interest policy.
That is the sort of result we could use a lot more of in scrutiny of dubious healthcare claims.
Also awarded a 1-star review was a University of Iowa press release touting a compound made from components of oregano and thyme as a “cure” for cachexia, a wasting disease affecting patients with cancer, AIDS and other serious diseases. The University failed to disclose the study’s funding source, as well as the fact that it had sought a patent on the compound, that the researcher quoted in the release stood to benefit financially from its commercialization, and that the study was unpublished. That it was a mouse study was not revealed until well into the release. The University’s failure to adequately respond to HNR’s questions about its oversight of research and conflict of interest policy led to a drubbing in further HNR posts, including this stinging rebuke from bioethicist Arthur Caplan, PhD:
It is utterly irresponsible to report promise based on unpublished research or work not presented at peer reviewed meetings. The industry and conflict-of-interest aspects only make a bad situation worse. You ought not be attaching a public university’s name to hype and to fluffing up the interest of patients who are easily misled by their illness to hear promise as cure.
HNR did its own takedown of one of SBM’s favorite fans of quackery, Dr. Oz. One of HNR’s review team, Ranit Mishori, MD, Professor of Family Medicine at Georgetown University School of Medicine, wrote a blog post about the celebration (Oprah included) of Dr. Oz’s 1,500th show earlier this year.
I felt less jubilant: In the decade that The Dr. Oz Show has been on the air, it hasn’t been unusual for me to encounter patients who ask about topics, treatments and suggestions mentioned on the program and another popular medical show, “The Doctors.” The advice my patients tell me they’ve heard on these shows often does not square with what I know about the medical evidence.
No kidding! We’ve noticed the same thing here at SBM, many times. (Although, with all due respect, Dr. Mishori might want to check out the quackery issues festering in her own backyard.)
In 2013, Dr. Mishori and colleagues enlisted medical students to review a month’s worth of The Dr. Oz Show and The Doctors, another extremely CAM-friendly show. (This was a different project than the study of The Dr. Oz Show and The Doctors published in the BMJ, although similar in result.) In addition to finding that these shows rarely discussed risks or costs of recommended treatments, and that more than half of the shows had content linked directly to advertisements:
About 78% of statements made on the Dr. Oz show did not align with evidence-based medical guidelines, society recommendations, or authority statements. For The Doctors, this was about 80%. . . . For both shows, about half of the literature supporting the claims made on the show television was statistically insignificant or required extensive extrapolation. No literature support could be found for about a third of the claims. Of the supporting evidence we could find, about a third had Oxford EBM classification 3b or lower (the lower end of what’s considered good quality evidence). [Link added.]
Other SBM favorites taken on by HNR include stem cell and microbiome hype, “natural” flu remedies, overblown mice studies, false equivalency in reporting on vaccination “manufactroversy,” and questionable claims for meditation and “mindfulness.” In fact, HNF has referenced SBM posts favorably in its reviews: David Gorski’s post on “bad luck” and cancer in a summary of noteworthy healthcare news, and his post on the ketogenic diet in a discussion of breast cancer misinformation in the media. A June blog post (“Feeling the drip, drip, drip of credulous first-person reporting about unproven interventions”) calling out a Houston Chronicle story on IV vitamin infusions quoted Scott Gavura’s assessment of IV vitamins as “health care theater” and linked to his post’s review of the evidence.
Unfortunately, in one instance, HNR’s mention of a well-loved (and, sadly, former) SBM blogger was not so favorable. To their credit, upon the error of their ways being pointed out, they quickly made amends.
In 2016, HNR gave USA Today 4 out of 5 stars for story about dry needling, which also discussed acupuncture, noting that it correctly reported the lack of evidence of effectiveness and the risks, although it failed to discuss quantified benefits, such as how much pain relief people report in the studies, or for how long. The review also criticized USA Today for quoting experts with “a potential conflict of interest” who had “a predisposition for or against the procedure.” And who might that be? Among others:
Mark Crislip [who] writes for Science-Based Medicine — as the story notes ‘a website critical of alternative medicine’ — and generally is quite skeptical of alternative therapies like dry needling.
That prompted a counter-post from me on the Society for Science-Based Medicine website, “Health News Review: Wrong on Crislip and acupuncture:”
Since when is skepticism of alternative therapies a potential conflict of interest? Let’s take out our dictionaries and give this a closer reading. “Alternative medicine” is by definition lacking in evidence of effectiveness. If a treatment has adequate evidence of safety and effectiveness, it’s called “medicine.” It is also, by definition, used in lieu of “conventional medicine.” “Skeptical” means “having an attitude of doubt” or “doubtful.” A “conflict of interest” occurs when one’s personal interests “might benefit from his or her official actions or influence.” So, Dr. Crislip’s being doubtful about treatments with no evidence of effectiveness used in place of treatments with evidence of safety and effectiveness potentially benefits his personal interests? How?
Giving Health News Review’s reviewers the benefit of the doubt here, I have to suspect that they aren’t actually that sloppy. Surely they are aware that none of us who write for Science-Based Medicine has ever gotten a penny for what we do. And I know they are too smart to fall for the Big Pharma Shill gambit. What I imagine they meant was that Dr. Crislip and the rest of us at Science-Based Medicine are somehow biased against alternative medicine. They’re wrong there too. . . . The fact that we often, though not exclusively, critique alternative/complementary/integrative health care doesn’t mean we’re biased. It simply reflects the many ways in which that particular category of health care misinforms the consumer and fails to stand up to scientific scrutiny.
But that wasn’t HNR’s only mistake:
Health News Review’s second error (actually, three errors in one) claims there is enough evidence for acupuncture that some states have created “medical licensure programs for its practice.” First, there is no good evidence that acupuncture is effective for anything. Cite whatever source you are using for this statement and we’ll tell you why it’s faulty. (You do have a source, right?) Second, it’s more than “some states” if by “medical licensure programs” you mean licensing acupuncturists as health care practitioners. Kansas just became the 46th state to pass an acupuncture practice act. Third, and more importantly, please disabuse yourself of the notion that scientific evidence has anything whatsoever to do with state licensing of alternative health care practitioners, acupuncturists or otherwise. It doesn’t, any more than states’ requiring the teaching of creationism has anything to do with science. (I recommend you go to the Science-Based Medicine website and review my posts on Legislative Alchemy.)
True to their commitment to evidence-based healthcare reporting, HNR amended its review of the USA Today’s story:
Editor’s note: The notion that the evidence on acupuncture has persuaded states to create medical licensure programs has been contested by the Society for Science-Based Medicine in this post. They argue that evidence is not a factor in state licensing of such practitioners. We have also amended our comments under the “Independent Sources” criterion below in response to their feedback. . . . Mark Crislip’s name appeared on this list [“most of the sources have a potential conflict of interest in some way”] due to an editing error. We do not believe that being skeptical of alternative therapies constitutes a conflict of interest.
HNR will be hard to replace
I closed that critical SFSBM post with the comment:
I have been, and remain, a fan of Health News Review, and still look forward to its weekly emails.
I am not the only one to lament HNR’s pending demise. Julia Belluz, senior health care correspondent for Vox told Michael Schulson of the Undark blog that “it really will leave a big void.” Belluz said “she and her colleagues ‘really took those reviews to heart,’” describing HNR as “an important watchdog,” adding, “there’s nothing else that’s really like that.”
Ivan Oransky, the editor of Retraction Watch and the current president of the Association of Health Care Journalists, told Schulson that he uses the 10-point checklist in journalism classes he teaches at NYU and distributed it to staff when he was with Reuters:
“It’s really a lighthouse warning health care reporters about the shallow waters,” Oransky said. “I think it’ll be a big loss if this really means the end for HealthNewsReview.”
HealthNewsReview.org inspired the creation of healthcare media watchdogs in other countries, including the German HealthNewsReview website, medien-doktor.de, whose team said:
At a time, when the health care industry is a leading economic sector in industrialized countries while news organizations struggle to remain profitable, and fighting fake news of all sorts, an independent institution like the one Gary Schwitzer has built appears more necessary than ever.
Indeed.
The only consolation I can offer HNF writers is that, if they enjoy ferreting out credulous healthcare news stories without pay, with “job benefits” like being called Pharma shills, harassment at one’s real job, accusations of “abandoning patients” for criticizing so-called integrative medicine, threatening emails, vicious attacks online and in other media by quackery supporters, and being sued (unsuccessfully) for defamation, SBM always welcomes fresh faces.
