Et tu, BMJ? Again? Seriously?
It’s been nearly three years since I last asked: What the heck happened to The BMJ? As my frustration with this medical journal led me to sit down to write this post, I did think about entitling it the same thing but thought better of it, given that I’ve already used that title twice. Better to call what The BMJ just published last week what it is: Sensationalism custom-made for use (or, no doubt as the journalist would characterize it, misuse)by antivaxxers to spin a deceptive yarn in which during the early months after the rollout of COVID-19 vaccines in the US begun in December 2020, the CDC and FDA supposedly engaged in what is portrayed, in essence, as a coverup or suppression of reports of adverse reactions due to COVID-19 vaccines—or just gross negligence. This narrative comes in the form of an article by David Willman entitled, US officials knew covid vaccine safety system was flawed—but turned aside the FDA doctor who alerted them. It gets even better, because the tagline claims that “US officials embraced an algorithm for detecting signals of harm from mRNA covid-19 vaccines that they knew was broken and suppressed efforts to fix it.”
Sounds pretty damning, doesn’t it? A close reading of the article, however, makes culpability much less clear and involves convoluted arguments over two different statistical models used for detecting an increase in specific adverse events after vaccination in a passive reporting system. Personally, too, I was rather puzzled by the seeming amazement that the vast majority of reports to the Vaccine Adverse Events Reporting System (VAERS) database in 2021 were about mRNA-based COVID-19 vaccines, for example:
I bet those of you who have been paying attention to the antivaccine narratives surrounding the rollout of the mRNA-based COVID-19 vaccines can even guess before reading the article which specific adverse event this article is primarily about. Yes, it’s (mostly) about myocarditis in young people, predominantly young men, associated with COVID-19 vaccination and how, supposedly, the US utterly failed in pharmacovigilance because it didn’t identify the safety signals as soon as some other countries (e.g., Israel) did. There’s actually an interesting story here if one wasn’t wedded to what comes across as an intentional coverup, which is the narrative that antivaxxers are running with based on this story
As a number of people have pointed out, the observation that the vast majority of reports to VAERS in 2021 were regarding COVID-19 vaccines should be no surprise. Before COVID-19 vaccines, few people knew of VAERS, which is a passive reporting system to which anyone can report any problem that occurs after any vaccination. In brief, VAERS is intended to function as an early warning system, a canary in the coal mine, so to speak, that can generate hypotheses regarding specific adverse events that occur after vaccination with specific vaccines. These hypotheses then need to be tested using more rigorous systems, as VAERS is prone to a lot of false positive and is easily misused by antivax “researchers.” We’ve discussed VAERS many times before and how misusing VAERS has long been a favorite strategy of antivaxxers to attribute all manner of maladies to vaccination. I’ve discussed the shortcomings of VAERS many, many times, both here and at my not-so-super-secret other blog, going back 20 years; so I’ll just refer you to a post I did as 2021 came to a close when I described how the misuse of VAERS by antivaxxers was continuing apace, and how an old antivax technique was turbocharged and weaponized within two months of the rollout of COVID-19 vaccines.
Beginning with the rollout of the COVID vaccines in December 2020, for the first time ever the CDC and FDA really publicized VAERS and used its bully pulpit to encourage the public to report adverse events. Of course reporting of adverse events after COVID-19 vaccines skyrocketed, far surpassing any other reports, in particular given that millions and millions of doses were being administered as fast as the vaccines could be produced. They even created the V-Safe system, which periodically texted vaccinees for months after they were vaccinated to ask them if they were having any symptoms or problems. As Jeffrey Morris pointed out, “extraordinary public, medical and media scrutiny” produced “substantial stimulated reporting to VAERS” and healthcare professionals “had specific reporting requirements for adverse events following COVID vaccination.” He further observed that “the raw fact that COVID vaccines generated vastly more VAERS reports than previous vaccines does not by itself indicate that they caused vastly more adverse events,” the important thing being “whether particular events occurred disproportionately relative to vaccination exposure and appropriate comparators.” This massive increase in reporting in 2021 is an important factor in evaluating the two statistical methods Willman discusses in his article.
Speaking of the journalist, David Willman, under whose byline this story appears, is actually a legitimate investigative journalist. Just look at his Wikipedia entry if you don’t believe me. I can guess why he might have found a story about the alleged failure of the CDC to do appropriate analyses looking for safety signals for myocarditis attractive. Specifically, he won a Pulitzer Prize in Investigative Reporting in 2001 for his LA Times series reporting on seven FDA-approved drugs later withdrawn over safety issues and deaths. (Ironically, one of his articles notes that the too-fast approval of these drugs came about as a result of policies implemented to speed the approval of drugs and treatments for HIV/AIDS.)
However, as I read his BMJ article, I couldn’t help but notice my skeptical antennae starting to twitching fiercely. I was sensing the presence of another, unnamed, player in this story. It turns out that my skeptical antennae didn’t let me down (at least not this time), because, right at the end of the article, in the Competing Interests section I read:
This account is adapted from an upcoming book about covid-19 and the vaccines, written by David Willman and Peter Doshi. Doshi is senior editor, The BMJ Investigations Unit.
Bingo! Why, I wondered, wasn’t Peter Doshi a co-author of this investigation? More importantly, for those of you not familiar with Peter Doshi and why he is important to The BMJ’s heel turn on vaccines over the last decade, let me step back and briefly discuss it. If you’re familiar with Doshi’s antivax antics, his role at The BMJ as a senior editor, and the antivax narratives he’s pushed, using the imprimatur of a respected medical journal to lend them credibility, you can safely skip the next section and move onto my discussion of Willman’s article itself. However, you might be interested in revisiting this story, because it’s a doozy that makes me wonder how a legitimate journalist like Willman not only collaborated with Doshi, but co-authored a whole book with him on COVID-19 vaccines! Given Doshi’s track record, I fear that I know exactly what this book will say, aside from this new “revelation,” which basically just seems to be a chapter from the upcoming book modified to fit in The BMJ’s format.
Peter Doshi: What happened to The BMJ
Whenever I lament what has happened to The BMJ, I always feel obligated to point out that The BMJ used to be almost unambiguously a good guy when it came to vaccines. After all, it was The BMJ that published investigative journalist Brian Deer’s articles recounting the strong evidence indicating that Wakefield’s case series published in The Lancet that linked the measles-mumps-rubella (MMR) vaccine to autism was based on fraud. That study launched the modern iteration of the antivaccine movement and is, as I like to say, the study that launched a thousand quackeries directed at autistic children to “cure” them of their “vaccine-induced autism.” The headline of Deer’s accompanying commentary after his account of how the “case against the MMR was fixed” even referred to Wakefield’s study as “Piltdown medicine“, after the infamous Piltdown Man fraud. No one played a bigger role in finally exposing Wakefield for the fraud that he was (and still is) than Brian Deer and The BMJ.
Fast forward to 2010, when our very own Mark Crislip noticed that Peter Doshi and Tom Jefferson published a dubious article (that he deconstructed as only he can) casting doubt on whether Tamiflu works. Interestingly, a few years later, in 2014, Peter Doshi became an associate editor at The BMJ when he was an assistant professor in the Department of Pharmaceutical Health Services Research (PHSR) at the University of Maryland School of Pharmacy. I’ve been unable to pin down exactly when Doshi was promoted to senior editor, but it was no later than 2021, which is when I first started noticing articles under his byline showing up that promoted antivax narratives.
As for me, I first noticed Doshi in 2009, when I noted that, as the H1N1 pandemic of 2009-2010 was bearing down on us, he had been scheduled to speak at an antivaccine conference hosted by the National Vaccine Information Center (NVIC). Indeed, he had even been contacted about the nature of the conference and replied with the lamest of lame gambits, namely saying, “my speaking there does not imply endorsement.” After that, he was nothing if not consistent. While all the while continuing to claim that he’s “not antivaccine, he parroted more than a few antivaccine talking points himself while trying to portray himself as an authority on influenza and the flu vaccine. Through it all, when challenged, he’s played the “victim” card that people who are borderline antivaccine or antivaccine love to play, claiming that they are “just asking questions” and that anyone who “questions” vaccines is labeled, in a knee-jerk—and most unfair—fashion, “antivaccine.” In fact, Doshi’s borderline antivaccine stylings go back further than 2009, when a senior pseudonymous epidemiologist whose blog I used to read religiously back then chastised him for an “unhelpful” commentary in which Doshi had claimed that estimates for yearly influenza deaths were “grossly inflated.” The year was 2006.
It was during COVID-19, though, that Doshi let his antivax freak flag fly high. In February 2021, less than two months after the Pfizer COVID-19 vaccine had started being distributed first to healthcare workers and those in high risk occupations, he published an article on The BMJ’s blog entitled, Pfizer and Moderna’s “95% effective” vaccines—we need more details and the raw data. In it, he pulled some statistical prestidigitation so transparent that even I could spot it, all to cast doubt on the validity of the randomized clinical trial showing the Pfizer and Moderna vaccines to be highly effective, while using his bogus “analysis” to suggest that the real efficacy of the vaccines was far lower. The utter deceptiveness of his analysis led me to wonder why The BMJ was still employing him and Skeptical Raptor to note drily that “BMJ is not a hotbed of anti-vaccine pseudoscience, except for the presence of Peter Doshi.” Indeed, Doshi is not a vaccinologist or infectious disease expert, nor is he an expert in clinical trials, something very apparent to anyone who is an expert who read his post and its misuse and misrepresentation of protocol violations. In fact, Doshi received his BA in anthropology from Brown University, an MA in East Asian studies from Harvard University, and a Ph.D. in history, anthropology, and science, technology, and society from MIT. Yet, somehow, he is senior editor at The BMJ.
In his original article, based on specious reasoning Doshi estimated that the true efficacy of the vaccines was not over 90%, but was rather only 19%. I later pointed out how Doshi’s original “reanalysis” of the clinical trial data had given birth to what I call a “slasher myth” about COVID-19 vaccines, a term I coined for medical misinformation that no matter how many times it appears to have been killed, always reappears, just as slashers who appear to die at the end of one movie are always shown to have survived somehow so that they can kill again in the sequel.
Let’s just reiterate that The BMJ hired Doshi despite his long history of playing footsie with the antivaccine movement since at least 2009, amplifying antivaccine conspiracy theories, downplaying the severity of influenza and thus feeding antivaccine narratives, using sleight-of-hand to downplay the effectiveness of flu vaccines, and generally playing the role of a false skeptic with respect to vaccines, as well as having signed a petition in 2006 “questioning” whether HIV causes AIDS. It continues to employ him even after he’s fallen for a conspiracy theory that the Vaccine Adverse Events Reporting System (VAERS) database was being made inaccessible to suppress report. That’s not all, though, he’s also served as an expert witness for the plaintiffs in antivaccine leader Robert F. Kennedy Jr.’s lawsuit against the University of California’s influenza vaccine mandates.
Finally, seeing that Doshi’s role above is listed as “senior editor BMJ Investigations,” another gross abuse of science and journalism published by The BMJ makes a lot more sense to me now. I’m referring, of course, to the November 2021 “exposé” of Ventavia, one of the companies contracted by Pfizer to run its clinical trial of its COVID-19 vaccine, alleging shoddy working conditions and unblinding of clinical trial subjects. (Hard to believe that that was nearly five years ago!) As I discussed in depth at the time, it was all either exaggerated or outright nonsense, wrapped in a big bright conspiracy theory bow, the sort of reporting far more suited for Alex Jones’ Infowars than for a legitimate news outlet or medical journal. None of this is a surprise coming from Paul Thacker, who is a conspiracy theorist and hack. Assuming Doshi was in charge of BMJ Investigations back then, it makes a whole heck of a lot of sense that The BMJ let someone like Thacker appear in its pages.
The charges
With that background and the (barely acknowledged and likely ghost) co-author Peter Doshi being involved in writing this article by David Willman, let’s take a look at what is actually charged. The depressing thing, as I mentioned above, is that there is a story here. The problem is that it is spun to make the motives of the officials at the CDC and FDA involved look as nefarious and negligent as possible, even though there is no evidence that that was the case.
to its choice of algorithm to use to monitor data from the Vaccine Adverse Events Reporting System (VAERS) database. Worse, if you believe the narrative, the FDA officials refused to alter the algorithm. More specifically, the article seeks to damn the CDC for not using two algorithms to data mine VAERS. Like most compelling exposé narratives, the story features a whistleblower who repeatedly brought up concerns but was either ignored or muzzled. Noting that at the time of the rollout of COVID-19 vaccine distribution in December 2020, the CDC had announced that it was going to be on the lookout for any harmful reactions attributable to the vaccine, Willman lays the groundwork:
Millions of people were to be vaccinated each week, and the CDC would rely on statistical “data mining” of the VAERS database to quickly identify any safety signals in need of further probing.
Two well recognised, separate algorithms were to be used. The first involved calculating “proportional reporting ratios” (PRRs), which sought to identify types of reactions reported to VAERS disproportionately or more frequently than background rates. The second data mining technique, called “empirical bayesian,” was to be provided by the Food and Drug Administration (FDA), which operated VAERS jointly with the CDC, and the two agencies planned to share and discuss results.1
I can’t help but note that I wanted to read the link referenced as #1, and I clicked on the link. It was a dead link. To me, that indicates some sloppiness here. So I consulted the almighty Wayback Machine at Archive.org, and was informed that the Internet Archive services were temporarily offline. Oh, well. It’s late Sunday night, and so I’ll check back tomorrow. In any case, this strategy seemed at least not unreasonable, and I noted that the two methods had their strengths and weaknesses, which I’ll get into more later. Suffice to say for now that both methods address the same question of disproportionality, specifically: Is a particular adverse event reported disproportionately more often for the vaccine under study relative to other vaccines, which are used as a reference? I’ll get into that more shortly.
First, what happened next? According to Willman:
But in a little noticed September 2022 letter, CDC director Rochelle Walensky said that the agency did not perform PRR analyses for signals until 2022. This was more than a year into the vaccine rollout and was intended, she wrote, “to corroborate” the FDA’s approach. Instead of using the PRR and empirical bayesian methods jointly during the pandemic, Walensky explained that both the CDC and FDA instead “chose to rely” on empirical bayesian, which she characterised as “a more robust technique.”
The BMJ, however, has found that the FDA’s algorithm was mathematically compromised by the deluge of covid vaccine adverse event reports—resulting in a near total loss of signal detection sensitivity for the new mRNA vaccines made by Pfizer and Moderna. Top FDA officials were warned internally of this problem in early 2021, and the official who headed its pharmacovigilance division for vaccines commented more than two years later to his CDC and FDA colleagues: “We were aware of this [data mining] limitation before and during the pandemic.” Government documents also show that CDC officials were informed during rollout of the vaccines—on multiple occasions.
Yet instead of fixing the safety deficiency—or shutting down the unreliable signal detection system—officials continued to rely on the FDA’s broken algorithm. The officials also cited the absence of signals when assuring clinicians and the public of the vaccines’ safety.
This sounds pretty damning too, doesn’t it? It gets even worse (or is made to sound even worse):
Yet instead of fixing the safety deficiency—or shutting down the unreliable signal detection system—officials continued to rely on the FDA’s broken algorithm. The officials also cited the absence of signals when assuring clinicians and the public of the vaccines’ safety.
At the FDA, managers went a step further by silencing the career drug safety officer who had repeatedly brought the deficiency to their attention and had offered an updated algorithm intended to fix it. As a result, Americans received unfounded assurances of the safety of the most widely used covid vaccines and were denied prompt warnings of potential harms, The BMJ’s investigation found.
There we have it, the necessary element of any good story like this, the “whistleblower” whose concerns were downplayed, ignored, or blown off and, even better, who suffered career consequences for telling The Truth. In this case, the whistleblower was a career FDA official:
The concern had made its way to Marks, FDA’s top official overseeing the covid vaccines, in early 2021. And it came from a career medical officer in the FDA’s centre for drugs, not vaccines: Ana Szarfman, a doctor with a PhD in microbiology and immunology. She worked for decades on drug safety data mining at the FDA and had a lead role in modernising the agency’s approach.67
Szarfman had built a years long professional association with an accomplished statistician, William DuMouchel, who devised the particular bayesian data mining algorithm that the FDA began using more than 15 years before the pandemic.6
So what was Szarfman’s concern? Here’s where the flood of reports deluging VAERS during 2021 comes in:
The BMJ can reveal that the FDA’s algorithm failed to signal a potential relationship between mRNA covid vaccination and myocarditis, pericarditis, Bell’s palsy, tinnitus, and other reported disorders owing to a flaw in the detection methodology—a flaw that senior officials declined to rectify.
The problem arose because, in the first year of the rollout, almost all the reports coming into VAERS—more than 90% of 1.1 million—were for the mRNA covid shots, dwarfing vaccines for all other diseases.4 This meant that analysing the safety of one vaccine, for example Pfizer’s product, involved comparing it to other vaccines, which were overwhelmingly limited to Moderna’s product. But if both mRNA vaccines elevated the risk of an adverse event like myocarditis in roughly equal amounts, the “observed” frequency and “expected” frequency would be similar, resulting in no automated alert.5 The statistical phenomenon is known as “masking.”
Here’s where I first started thinking: This isn’t a problem with the statistical method chosen. This sounds like a problem with the control group chosen. Why on earth would one include any of the COVID-19 vaccines in the control group of reports of adverse events in VAERS? Notice the framing here, namely that the PRR was an “unreliable” method compared to the Bayesian system, which is utter hogwash. Even I could tell that, but I had to rely on someone someone who knows a lot more what he’s talking about when it comes to statistics than I do, namely Jeffrey Morris, who posted this analysis on Twitter.
First, I think it’s useful to quote Dr. Morris on the relative strengths/weaknesses of the two methods PRR vs. Bayesian:
The empirical-Bayes method is not some fundamentally different or defective type of analysis. It builds on the same observed-versus-expected reporting framework, but uses shrinkage to stabilize estimates, particularly when counts are small, and provides a posterior distribution for the underlying reporting ratio that can be examined statistically. That shrinkage is generally a statistical advantage. My white paper discusses this relationship in detail.
An important difference is how a potential signal is flagged. The commonly used screened PRR rule is essentially an ad hoc threshold: PRR > 2, ChiSq>4, with at least 3 reports. In contrast, FDA used the more stringent empirical-Bayes criterion EB05 > 2: the lower 5th percentile of the posterior reporting-ratio distribution must exceed 2, which is equivalent to requiring greater than 95% posterior probability that the underlying reporting ratio exceeds twofold. Not surprisingly, those criteria have different operating characteristics. EB05 > 2 is substantially more conservative: it should produce fewer false-positive alerts, particularly for sparse data, but at the cost of lower sensitivity and potentially more false negatives.
Here’s a link to the white paper, if you want to go into the nitty-gritty of the details. To boil it down, the PRR method has a higher sensitivity with a lower specificity, meaning that it will detect more events above what is expected, but at the expense of specificity, meaning that a lot of those events will turn out not to have been due to the vaccines. The empirical Bayesian method, on the other hand, is more stringent. It will detect fewer events, but the events detected will be more likely to be “true” adverse reactions actually caused by the vaccine, which is why Dr. Morris emphasizes:
The much larger number of events flagged by the classical PRR rule therefore does not by itself demonstrate that PRR was the better method or that FDA’s Bayesian methodology was defective. To establish that, one would want to compare the sensitivity, specificity, false-positive rate, and false-negative rate of the competing rules under realistic reporting conditions. The article emphasizes the greater number of PRR alerts but does not provide that type of operating-characteristic analysis. FDA itself describes data-mining signals as hypotheses for investigation, not causal findings.
According to Willman’s report, in a meeting held on February 29, 2020 Szarfman included a presentation comparing her proposed algorithm based on using PRR to the algorithm based on empiric Bayesian methods being used, and, unsurprisingly, a lot more safety signals were flagged, as shown in this figure included in the Willman article:
The heat map looks pretty convincing, doesn’t it? But what does it really mean? Any method that is more sensitive will produce more “hits” than a less sensitive method. Every algorithm of this sort has to balance sensitivity and specificity. Too much sensitivity, and there will be so many signals, many of which won’t pan out, that investigating them all will become impractical. Too much specificity, and, even though the signals that are found will have a much higher proportion of true positives, a lot of potentially important signals will be missed. How much sensitivity and specificity are needed relative to one another is a legitimate scientific question about which there can be legitimate scientific disagreement, but that’s not how this argument is portrayed here. The spin here is that Szarfman was good and right, and that Peter Marks and the FDA were wrong and negligent, if not outright villains for dismissing the brave whistleblower. I do note here, that she appears to have been given a respectful hearing, being invited to present her findings to Peter Marks, who was then the director of the Center for Biologics Evaluation and Research (CBER), the FDA center responsible for approving vaccines.
Moreover, as Dr. Vincent Iannelli points out, Szarfman found a bunch of signals for adverse events that COVID-19 vaccines don’t cause, in particular sudden death. According to Willman:
But by July 2021, Szarfman again sounded an alarm over a “strong signal” for “death and sudden death” that she said DuMouchel had identified using the updated algorithm. She shared the findings with a colleague, Richard Forshee, a deputy director for biostatistics and pharmacovigilance, who worked under Peter Marks. Forshee expressed his doubts about the analysis to Szarfman and wrote to Marks: “Ana Szarfman called me . . . She said that she and Bill DuMouchel had found an increased risk of mortality following covid-19 vaccination using data mining methods.
“I am very concerned that whatever association they think they have identified is spurious based on the way the covid-19 vaccination program prioritized individuals and the required and stimulated reporting we are seeing with the covid-19 vaccines. . . . Please let me know how you would like us to proceed.”
Eventually, Szarfman’s activities got the attention of Peter Marks again, leading him to write to the Director of the Center for Drug Evaluation and Research (CDER):
On 15 September 2021, Marks wrote to his counterpart in the FDA’s Center for Drug Evaluation and Research (CDER): “One of the CDER statisticians, Ana Szarfman, has decided on her own to do vaccine analyses using VAERS as part of her work at FDA. She is, however, not doing this in collaboration with our [vaccine centre] statisticians, and quite to the contrary, has been asked to cease and desist, because the strategy that she is using could create erroneous conflicts that feed in to anti-vaccination rhetoric. This is creating an issue . . . This issue came up previously during the pandemic and . . . it seemed to go away, but it is now back. Can we catch up about this sometime?”
Basically, it sounds as though Marks had finally become fed up, whether rightly or wrongly.
Now here’s the thing. As Dr. Morris points out (and so do I), it isn’t the empirical Bayesian algorithm that was the problem. It was the selection of the control group to include those vaccinated with one of the COVID-19 vaccines, which was not an inherent part of an empirical Bayesian analysis. The problem of masking was far more of an issue than the choice of specific algorithm. As Dr. Morris explains:
Second, there was a separate and legitimate problem with the particular reference distribution FDA used in its empirical-Bayes implementation. This is where the masking issue enters. FDA’s brand-specific analyses estimated expected reporting frequencies from the broader VAERS database. Because Pfizer and Moderna together accounted for an enormous fraction of COVID-era reports, an adverse event elevated similarly for both vaccines could raise the expected frequency against which each individual vaccine was compared. That can attenuate or eliminate a disproportionality signal. The published masking analysis demonstrates that this phenomenon can occur, and the newly released internal records indicate that FDA officials were aware of the issue.
But that is a problem with the construction of the comparison group, not evidence that empirical Bayes itself is an invalid method. Indeed, a conventional PRR calculated with the same poorly chosen reference set can suffer the same masking problem; the methodological literature demonstrates masking for both PRR and standard empirical-Bayes disproportionality methods.
And, left out of Willman’s report:
And importantly, CDC’s eventual PRR analysis did not use that same comparison. For its principal PRR analysis in 2022, CDC pooled Pfizer and Moderna together as the mRNA COVID-vaccine exposure and compared their adverse-event reporting proportions with those for non-COVID vaccines, using reports extending back to 2009. Thus Pfizer and Moderna could not mask one another in that pooled analysis. The dramatic difference between the number of FDA EB alerts and CDC PRR alerts therefore cannot cleanly be interpreted as “Bayesian method bad, PRR good”: both the flagging criterion and the reference population changed.
And, most importantly, Willman doesn’t mention at all that one of the biggest problems with using VAERS during a pandemic situation in which, for the first time ever, the public is being strongly encouraged to report adverse events to VAERS. These are two different situations: VAERS during “regular times,” in which relatively few people even know of the database and vaccination occurs at a reasonably constant rate from year to year, compared to VAERS during a pandemic and the rollout of a vaccine against the pandemic-causing organism, SARS-CoV-2, in which the public is being urged to report any suspected adverse event after vaccination to VAERS and there is even a new system, V-Safe, set up to remind people to report by sending text messages to people at specified time intervals ofter they receive a COVID-19 vaccine to ask them if they are experiencing any symptoms that might be attributable to the vaccine.
As Dr. Morris notes, VAERS is “susceptible to stimulated reporting, media attention, reporting requirements, and differences in the populations receiving different vaccines.” Indeed, I first wrote about this phenomenon over two decades ago, when a study was published that showed how vaccine litigation had affected reporting to VAERS. In brief, lawyers representing families suing pharmaceutical companies for their children’s autism based on the belief that vaccines had caused it. These attorneys encouraged parents to report their child’s autism to VAERS as an adverse event, thus creating “evidence” that vaccines cause autism to be used in legal actions against vaccine manufacturers and in Vaccine Court. None of this is new. The FDA and CDC knew that stimulated reporting could be a problem.
Moreover, it is likely not because of the choice of algorithm to data mine VAERS that Israel recognized the safety signal for myocarditis associated with COVID-19 vaccination. As Dr. Iannelli points out, there was a big difference in how Israel and the US approached their COVID-19 vaccination efforts. First, myocarditis after COVID-19 vaccination is an adverse event seen more often in the young, particularly young men. Second, it’s most often seen after the second dose of the vaccine. Dr. Iannelli notes that in the early months of the rollout, in the US it was high risk people (e.g., the elderly and those with chronic diseases, as well as those in high risk jobs, like healthcare) who were prioritized to be vaccinated, while in Israel vaccinated all of its citizens, with no restriction on age. He further notes that it wasn’t until March and April 2021 that older teens and young adults were receiving the vaccines, concluding:
And since myocarditis was mainly found after they got their second dose, it shouldn’t be a surprise that we didn’t see a safety signal until May.
A safety signal that was detected.
A safety signal that we were all told about.
Indeed.
Putting it all in context in a way The BMJ did not
I don’t mean to say that Willman’s report is entirely misguided. Clearly, he has documented problems at the FDA and the CDC in terms of vaccine safety monitoring. It’s just that, from my reading and consideration of Dr. Morris’ points, the main problem with the algorithm used to screen VAERS for adverse events related to COVID-19 vaccines was almost certainly not the problem that Willman emphasized, the choice of an empirical Bayesian algorithm versus calculating PRR. It was the control group initially chosen while doing the Bayesian analysis, and that is a valid point. Unfortunately, Willman flubs it completely by framing the problem almost entirely as a battle between PRR and empiric Bayesian methods, with the ‘correct” method being obvious (in retrospect), rather than as an issue with choice of control group during a massive vaccine rollout and a publicity campaign designed to promote stimulated reporting. Yes, Willman does mention the stimulated reporting, but mainly in passing and in a quoted email to Peter Marks from Richard Forshee.
Another frustrating thing about this article is that it does not really acknowledge the function of VAERS, falling seemingly inadvertently into line with the antivax view of VAERS as the be-all and end-all of vaccine safety monitoring, when in fact VAERS is an early warning system whose signals cannot be taken at face value and must be investigated using more rigorous methodology. Just as frustrating is its failure even to acknowledge that any algorithm chosen to analyze VAERS reports would involve deciding upon how much of a trade-off between sensitivity and specificity would be ideal, a decision about which reasonable scientists can have reasonable scientific and public health policy disagreements, as there are pros and cons to every approach.
Finally, Willman is not wrong to report that the CDC and FDA were in a bit of the “circle the wagons” mode in response to misuse of their data by the antivaccine movement (some examples of which we here on SBM wrote about at the time). Many of us engaged in science communication at the time were frustrated by the communications from the CDC and the FDA, although I can understand and sympathize with Marks’ apparent fear that Szarfman’s report would be weaponized by the antivaccine movement, a concern that Willman doesn’t seem to take seriously. The sad thing is, Willman’s reporting could have been valuable if it hadn’t been spun as a battle of a single brave whistleblower and a dogmatically—even negligent—FDA officials, rather than as a difficult choice of methodology by the FDA and CDC to screen for adverse events.
The article concludes:
But while reflecting on the events, DuMouchel said that he could not explain the government officials’ resistance. “It’s really hard to know what they really had in their mind,” he said. “Or whether to believe or to accept their excuses for not wanting to switch to a new method . . . I think that they were wrong.”
Approached by The BMJ, Szarfman insisted that she had not sought to undermine public support for the covid vaccines. She recalled her efforts with what seemed to be resigned frustration. “Very few people understand the statistics,” she said. “That’s the problem.”
Over lunch near his residence in Miami, Florida, DuMouchel voiced regret that FDA officials had rebuffed Szarfman’s attempts to overhaul the deficient approach with VAERS. “If they had paid attention to Ana, they would have done better.”
Maybe. Maybe not. An equally plausible case could be made that if they had “listened to Ana,” there would have been so many false positive signals that the CDC and FDA could not have dealt with them and that it would have undermined confidence in vaccines, Szarfman’s and DuMouchel’s insistence that such was not their goal notwithstanding. Unfortunately, as valuable as it might have been to learn about this controversy at the FDA and CDC during the early months of the rollout of the vaccines, right now, given the way Willman is reporting it and with the knowledge that this article is most likely just a lightly edited version of a chapter in his book with Peter Doshi, I have to wonder what the larger aim is. Given The BMJ’s recent history with articles by Peter Doshi and Paul Thacker, I wonder if its editors even care anymore that they are willingly lending the prestige of their journal to antivax narratives, like the one claiming that the CDC and FDA “covered up” vaccine safety signals in order to keep the vaccines flowing.