Peer review fail: Vaccine publishes antivax propaganda disguised as “reanalyses” of Pfizer and Moderna COVID-19 vaccine clinical trial data

While contemplating the lessons thus far from the COVID-19 pandemic, I cannot help but emphasize how starkly the pandemic has revealed the shortcomings in scientific publishing. Before the pandemic, I expected to see poorly designed studies designed to stoke fear about vaccines by exaggerating adverse reactions (or even making them up) only in bottom-feeding and predatory journals and generally viewed it as a rare to nonexistent occurrence to see such papers published in reputable journals. Obviously, the publication of Andrew Wakefield’s fraudulent case series linking the MMR vaccines to “autistic enterocolitis” published in The Lancet in 1998 was one huge exception, but in general reputable journals did not traffic in what I like to refer to antivaccine disinformation disguised as legitimate scientific studies. Since the pandemic, there has been one exception, The BMJ, which has published enough dodgy vaccine articles that I once asked, “What the heck happened to The BMJ?” My post was in response to a deceptive “exposé” by Paul Thacker published in The BMJ about Ventavia Research Group, a Texas contract research organization (CRO) subcontracted by Pfizer to run three out of its 153 clinical trial sites for its COVID-19 vaccine trial in 2020. Thacker’s report was based on a single “whistleblower” briefly employed at Ventavia but with little to no actual evidence of alleged data falsification, patient unblinding, and worse, with Thacker attempting to generalize these problems (again, without evidence) to the entire Pfizer clinical trial. In my post, I also listed a number of other examples of The BMJ‘s failures with respect to reports and studies on vaccines, a pattern that, unfortunately, did predate the pandemic and has been traceable to one of its editors, Peter Doshi, who has a long history of antivaccine-adjacent and outright antivaccine stylings going back at least to the H1N1 pandemic in 2009. Indeed, one of his “reanalyses” resulted in what I like to call the “slasher lie” about the Pfizer vaccine that it is only 12-19% effective against COVID-19.

Last week, Peter Doshi managed to expand his penchant for deceptive “reanalyses” to Vaccine, one of the foremost journals about vaccine science, in a study titled “Serious adverse events of special interest following mRNA COVID-19 vaccination in randomized trials in adults“. The article had previously made a huge splash in antivaccine social media in June as a preprint, where its PlumX Metrics showed (as of yesterday) over 584,000 abstract views, over 109,000 downloads, and over 29,000 Tweets about it, with its download rank being 7th for the SSRN preprint site. Now that the paper, whose flaws and misleading nature were discussed in great detail by many scientists (which I will discuss in a moment), as well as by our very own Dr. Jonathan Howard and Dr. Harriet Hall, has somehow inexplicably gone from preprint to peer-reviewed publication in Vaccine, the onslaught started again. Here are some examples:

Does this one look familiar? It’s Dr. Joseph Ladapo, the Florida Surgeon General who was previously a member of the COVID-19 minimizing, “natural immunity” touting, hydroxychloroquine– and ivermectin-pushing, antivax quacks known as America’s Frontline Doctors before being tapped by Governor Ron DeSantis to be his Surgeon General.

Here are some more. Here’s the study’s first author:

Who in response to criticism said:

Seriously, peer review isn’t some magical talisman that makes a bad paper “science”. I like to paraphrase Winston Churchill about democracy by saying about peer review that it has been said that peer review is the worst method of deciding what should be published in the scientific literature except for all those other forms that have been tried from time to time.

In particular, they cited this “finding” from the paper, which reinforces an antivax narrative that the vaccine is more dangerous than the disease:

The study was also misrepresented as being an “independent” randomized controlled clinical trial when in fact it was merely a “reanalysis” of the trial using different definitions of adverse events:

Then, of course, there were the conspiracy theories about a “coverup”:

I rather regret not writing about this study before here at SBM, back when it was a preprint (although Jonathan and Harriet did). Its having managed to go from preprint to actual publication in a usually-reputable peer-reviewed vaccine journal provides me the opportunity to rectify that oversight, examine how the manuscript changed from preprint to publication, and discuss from my perspective exactly why this paper is a huge flaming dumpster fire. Finally, I’ll ask how a highly eminent statistician (Sander Greenland), one whose work on Bayesian statistics both retired SBM contributor Kimball Atwood and I have cited before in talks and blog posts, sullied his name by associating itself with this exercise in revisionist science. It turns out that there’s history there dating back 14 years that I had totally forgotten about. Worse, I strongly suspect that there’s no way Vaccine would have published this dreck had Prof. Greenland not put his name on it.

Strap in, this is a long one even for me (although not my longest ever by far).

The Vaccine article: from preprint to peer-reviewed article

Let’s follow the path from preprint to peer-reviewed article in Vaccine. The first thing I wondered when I read the preprint for the first time is more of a meta issue, specifically: Why was this study even done in the first place? In the introduction to the preprint, Fraiman et al. wrote:

We sought to investigate the association between FDA-authorized mRNA COVID-19 vaccines and serious adverse events identified by the Brighton Collaboration, using data from the phase III randomized, placebo-controlled clinical trials on which authorization was based. We then use the results to illustrate the need for formal harm-benefit analyses of the vaccines that are stratified according to risk of serious COVID-19 outcomes, as well as contextualize the findings against post-authorization observational data.

What is the Brighton Collaborative? (I’m actually surprised that I hadn’t heard of it before.) It’s a group dedicated to vaccine safety and improving the scientific rigor of vaccine science. Learning that, to be honest, made me a bit annoyed at some criticisms of the study that the serious adverse events (SAEs) had been pulled out of someone’s nether regions. That the Brighton Collaborative is involved doesn’t absolve Doshi for his chicanery in this paper, but we should be careful regarding criticisms we make.

Back to my meta question, though: Why does this study exist? I’ll preface my answer by pointing out a simple observation. It’s been 21 months since the randomized clinical trial (RCT) results for the Pfizer and Moderna vaccines were first reported. Both of them involved only ~43K and ~30K participants, respectively. Next, I will point out that even large randomized clinical trials used to approve drugs and vaccines miss less common adverse events (AEs), including serious AEs (SAEs). That’s why we do postmarketing surveillance studies, particularly for vaccines. Less common AEs sometimes don’t show up until after a vaccine is rolled out and distribution goes from a population of tens of thousands to administration to millions, tens of millions, hundreds of millions, and even billions, as has happened with the Pfizer and Moderna COVID-19 mRNA vaccines over the last 21 months. In other words, if you are truly interested in the actual real-world safety and efficacy of COVID-19 vaccines right here, right now, in September 2022, then the original RCT data are not the best data to use to estimate rates of adverse events. After all, well over 12 billion doses have been administered since then, and numerous countries have safety and efficacy data. Say what you will about Peter Doshi’s “reanalysis” of the clinical trial data in January 2021 that falsely concluded that the Pfizer vaccine had only demonstrated 19% efficacy, in January 2021 the randomized clinical trial data for the vaccine was all that there was. It made sense to look at those data then. Today? Not nearly as much.

So, given that background, why reanalyze the original RCT results from Pfizer and Moderna at all now? There’s one reason, and one reason only, that scientists might want to reanalyze data from a completed and long ago published clinical trial, and that’s if they suspect some sort of serious flaw in the RCT design or how the RCT was carried out. They might even suspect outright fraud. Doshi and his colleagues don’t explicitly say this, but if you know Doshi’s history you’ll understand that the real reason he undertook this analysis was almost certainly because he thought that the RCTs for the Pfizer and Moderna vaccine didn’t show what they claimed to show and were analyzed in such a way to exaggerate efficacy and hide adverse events. It’s not as though he’s made a secret of this belief, given that he’s a senior editor at The BMJ who’s been allowed to use the journal as his own soapbox to argue just that going back to January 2021.

Interestingly, this paragraph underwent some…changes…in the final version published in Vaccine:

We sought to investigate the association between FDA-authorized mRNA COVID-19 vaccines and serious adverse events identified by the Brighton Collaboration, using data from the phase III randomized, placebo-controlled clinical trials on which authorization was based. We consider these trial data against findings from post-authorization observational safety data. Our study was not designed to evaluate the overall harm-benefit of vaccination programs so far. To put our safety results in context, we conducted a simple comparison of harms with benefits to illustrate the need for formal harm-benefit analyses of the vaccines that are stratified according to risk of serious COVID-19 outcomes. Our analysis is restricted to the randomized trial data, and does not consider data on post-authorization vaccination program impact. It does however show the need for public release of participant level trial datasets.

I note that in the first paragraph, a sentence was added touting the Brighton Collaborative as a “a global authority on the topic of vaccine safety”. In any event, it’s tempting for me to claim that because a paragraph was revised from preprint to paper in order to tone it down, that means I can dismiss everything in the preprint. After all, “lab leak” conspiracy theorists did exactly the same thing recently based on the removal of a single word from a preprint to finished paper. I will, however, refrain and note again that most likely peer reviewers demanded that Fraiman et al. tone it down.

There’s another “tell” in the Methods section regarding what this study is really about:

Pfizer and Moderna each submitted the results of one phase III randomized trial in support of the FDA’s emergency use authorization of their vaccines in adults. Two reviewers (PD and RK) searched journal publications and trial data on the FDA’s and Health Canada’s websites to locate serious adverse event results tables for these trials. The Pfizer and Moderna trials are expected to follow participants for two years. Within weeks of the emergency authorization, however, the sponsors began a process of unblinding all participants who elected to be unblinded. In addition, those who received placebo were offered the vaccine. These self-selection processes may have introduced nonrandom differences between vaccinated and unvaccinated participants, thus rendering the post-authorization data less reliable. Therefore, to preserve randomization, we used the interim datasets that were the basis for emergency authorization in December 2020, approximately 4 months after trials commenced.

The definition of a serious adverse event (SAE) was provided in each trial’s study protocol and included in the supplemental material of the trial’s publication. [2], [3], [4] Pfizer and Moderna used nearly identical definitions, consistent with regulatory expectations. An SAE was defined as an adverse event that results in any of the following conditions: death; life-threatening at the time of the event; inpatient hospitalization or prolongation of existing hospitalization; persistent or significant disability/incapacity; a congenital anomaly/birth defect; medically important event, based on medical judgment.

Here Doshi is echoing a common antivax talking point, in which it is claimed that the unblinding was carried out to hide AEs and much lower efficacy than reported based on the data used to obtain EUAs for the vaccines. Of course, the question of whether or not to unblind a clinical trial is a complex issue and depends on the intersection of bioethics and science. In the case of COVID-19 vaccines, after efficacy and safety were demonstrated in the first analyses, it became unethical to leave the control groups of those studies unprotected against COVID-19, which was surging around the world and causing mass illness, disability, and death. I won’t go into more detail, as I discussed the issue of unblinding the trials in detail over a year ago. Moreover, multiple assessments of the trials have concluded that both were rigorously conducted.

To accomplish the task of making the vaccines look bad, instead of looking at all AEs, as the papers and reports analyzing the data from the clinical trials did, Doshi and his colleagues decided to focus on “serious adverse events of special interest” (AESIs). The first version of this list was published early in the pandemic based on five reports from China and has undergone a total of four updates, the most recent of which was published last September. These SAEs were determined in the following manner described by Fraiman et al.:

We used an AESI list derived from the work of Brighton Collaboration’s Safety Platform for Emergency vACcines (SPEAC) Project. This project created an AESI list which categorizes AESIs into three categories: those included because they are seen with COVID-19, those with a proven or theoretical association with vaccines in general, and those with proven or theoretical associations with specific vaccine platforms. The first version was produced in March 2020 based on experience from China. Following the second update (May 2020), the WHO Global Advisory Committee on Vaccine Safety (GACVS) adopted the list, and Brighton commenced a systematic review process “to ensure an ongoing understanding of the full spectrum of COVID-19 disease and modification of the AESI list accordingly.” [7] This resulted in three additional AESIs being added to the list in December 2020. The subsequent (and most recent fourth) update did not result in any additional AESIs being added to the list. [1].

We matched SAEs recorded in the trial against an expanded list of AESIs created by combining Brighton’s SPEAC COVID-19 AESI list with a list of 29 clinical diagnoses Brighton identified as “known to have been reported but not in sufficient numbers to merit inclusion on the AESI list.” [7] Sensitivity analysis was used to determine whether use of the original versus expanded list altered our results.

So right away when I first read the preprint, I wondered how these diagnoses were being combined, mixed, and matched. The new version of the paper doesn’t alleviate my questions. If you look at the Brighton Collaborative document, you’ll see a lot of unremarkable standard AEs and SAEs, but you’ll also find ones that require interpretation. Here is the table defining its AESIs in Brighton’s Safety Platform for Emergency Vaccines (SPEAC):

The AESIs included because they have a theoretical or proven association with specific vaccine platforms are interesting, mainly because none of them are associated with the mRNA platform, but rather platforms that existed before the mRNA-based COVID-19 vaccines were released. Also note how the AESIs are (mostly) listed as broad categories, rather than specific diagnoses. Exceptions include, of course, myocarditis, which is associated with COVID-19 and has been associated in safety data with COVID-19 vaccines, but mapping the AEs in the clinical trials to these categories requires some subjectivity. There are more than just what’s listed above, delineated in a number of charts under each organ system. For example, colitis is listed in Annex 6, which encompasses the gastrointestinal system.

Indeed, here’s a passage that suggested the subjectivity to me:

We matched SAEs recorded in the trial against an expanded list of AESIs created by combining Brighton’s SPEAC COVID-19 AESI list with a list of 29 clinical diagnoses Brighton identified as “known to have been reported but not in sufficient numbers to merit inclusion on the AESI list.” [7] Sensitivity analysis was used to determine whether use of the original versus expanded list altered our results.

I note that in the original preprint, this passage read rather differently in that it used an example that really stood out as subjective:

We matched SAEs recorded in the trial against an expanded list of AESIs created by combining Brighton’s SPEAC COVID-19 AESI list with a list of 29 clinical diagnoses Brighton identified as “known to have been reported but not in sufficient numbers to merit inclusion on the AESI list.” Sensitivity analysis was used to determine whether the original versus expanded list had an effect on identifying a safety concern. For SAEs that described symptoms, not diagnoses, the clinician reviewers independently judged whether each SAE type was likely to have been caused by an AESI. For example, the SAE “abdominal pain” is a symptom based diagnosis, which was judged as fitting within the SPEAC clinical diagnosis of “colitis/enteritis.” Disagreements were resolved through consensus; in two cases, consensus could not be reached and were resolved by the judgment of a third clinician reviewer (PW) to create a majority opinion. For each included SAE, we recorded the corresponding Brighton Collaboration AESI category and organ.

Here’s a hint: Not all abdominal pain is due to colitis (inflammation of the colon) or enteritis (inflammation of the intestines). True, these are common causes, but there are so very many others. Similarly, what mapped to myocarditis and pericarditis? There are lots of causes of chest pain other than myocarditis and pericarditis. Also, in the supplemental data (which are in a Word document that can download and look at for yourself using a link at the end of the paper given that the lists are rather long to reproduce here), there are two lists, a list of “Included SAE types (matching AESI list)” and a list of “Excluded SAE types (not matching AESI list).”

Admittedly, many of the SAEs in the excluded list do make sense given that they include fractures, gunshot wounds, head injuries, and the like, but a number do not, such as viral pharyngitis, volvulus, vomiting, and others. When the preprint was published, Dr. Susan Oliver has posted a video discussing the problems with it, which I include here:

Dr. Oliver didn’t really discuss the meta problems with it and Doshi’s history, but she did note many of the same things that I did, in particular the odd choices of what was and wasn’t included as SAEs. For example, Doshi included diarrhea, but not vomiting (or, as the surgeon in me can’t help but note, intestinal perforation or volvulus, the latter a known complication of a certain vaccine); hyperglycemia (high blood sugar) but not hypoglycemia (low blood sugar); gastrointestinal hemorrhage but not duodenal ulcer hemorrhage (which is a form of gastrointestinal hemorrhage); and coronary artery disease but not atherosclerosis (which causes coronary artery disease). It’s all very curious. Perhaps the most important issue is that “events related to COVID-19” were excluded, which on the surface makes sense, but, given that COVID-19 cases were much more common in the placebo controlled group, automatically biases the results for the remaining SAEs to the vaccine-group.

That’s not all, though. Instead of comparing the number of people who had SAEs, they did this:

In their review of SAEs supporting the authorization of the Pfizer and Moderna vaccines, the FDA concluded that SAEs were, for Pfizer, “balanced between treatment groups,” [15] and for Moderna, were “without meaningful imbalances between study arms.” [16] In contrast to the FDA analysis, we found an excess risk of SAEs in the Pfizer trial. Our analysis of Moderna was compatible with FDA’s analysis, finding no meaningful SAE imbalance between groups.

The difference in findings for the Pfizer trial, between our SAE analysis and the FDA’s, may in part be explained by the fact that the FDA analyzed the total number of participants experiencing any SAE, whereas our analysis was based on the total number of SAE events. Given that approximately twice as many individuals in the vaccine group than in the placebo group experienced multiple SAEs (there were 24 more events than participants in the vaccine group, compared to 13 in the placebo group), FDA’s analysis of only the incidence of participants experiencing any SAE would not reflect the observed excess of multiple SAEs in the vaccine group.

To put it briefly, they compared number of SAEs, not the number of patients who suffered an SAE. This sort of analysis is guaranteed to double count SAES—at least!—because some of the SAEs or groups of SAES will be linked. For example, as Dr. Oliver originally pointed out, abdominal pain often goes along with diarrhea, to which I would add that colitis or enterocolitis can lead to gastrointestinal hemorrhage. Moreover, formal reporting systems for clinical trial AEs require that all AEs be entered, even when they are related, which is why analyses are usually done at the patient-level, as in “number of patients who suffered this AE”, rather than in total AEs reported in each group independent of the number of patients. I’d be willing to bet that if the same statistical analysis were done using per-patient-level data rather than SAE-level data the statistical significance would likely disappear.

Let’s look at the new version of the “money table,” which summarized the results and was being shared from the preprint on social media. I’ll cite David Grimes’ Tweet about it:

At this point, I can’t resist mentioning that defenders of the preprint claimed that this paper wasn’t “p-hacking” because no p-values were used. Personally, I’ll give that one to them and call the paper an exercise in data-dredging instead. (Of course, p-hacking is just a subset of the broader category of data dredging.) Or one can call it inflation bias if you like. It’s all the same thing in concept, namely when researchers try out several statistical analyses and/or data eligibility specifications and then selectively report those that produce significant results—or in this case seemingly significant results. In any event, whether you call it p-hacking, data dredging, or whatever else you might want to call it, it has been a widespread problem even under normal circumstances in the scientific literature, but a less known aspect of it is that it can be weaponized in the service of portraying vaccines, in this case the Pfizer and Moderna mRNA-based COVID-19 vaccines, as more dangerous than they are and the RCTs used to garner their EUAs in December 2020 as flawed and not showing the “true extent” of serious AEs attributable to them.

Fraiman et al. also included this doozy of a comparison in both the preprint and the final paper. Here’s the version from the final paper:

In the Moderna trial, the excess risk of serious AESIs (15.1 per 10,000 participants) was higher than the risk reduction for COVID-19 hospitalization relative to the placebo group (6.4 per 10,000 participants). [3] In the Pfizer trial, the excess risk of serious AESIs (10.1 per 10,000) was higher than the risk reduction for COVID-19 hospitalization relative to the placebo group (2.3 per 10,000 participants).

I notice that the wording was changed from “surpassed” to “was higher than,” but it’s basically the same paragraph in the preprint and the final paper. So what’s the problem? Think of it this way. You can only hospitalize patients, not SAEs (or AEs). An individual patient in the vaccine group could suffer more than one AE, but a patient in the placebo control group could only be hospitalized once (in the context of the limited timeframe of the clinical trial) for COVID-19. Here Doshi is comparing apples and oranges in order to make it look as though the vaccines were more dangerous than actually getting COVID-19, which is a ridiculous contention given what we know. Moreover, in clinical trials in general a lot of the “serious adverse events” are not serious enough to warrant hospitalization. In fact, according to the standard terminology used to rate SAEs in clinical trials grade 3 events and above (on a five-point scale) are rated severe. If you look at the list of specific AEs, you’ll see that some grade 3 AEs require hospitalization; some don’t. Grade 3 is defined as an AE that:

• Is severe or medically significant but not immediately life-threatening; OR
• Requires hospitalization or prolongation of hospitalization indicated; OR
• Limits self care/activities of daily living (ADL)

For completeness, I’ll mention that grade 4 AEs are by definition life-threatening events that require urgent intervention and that grade 5 events are by definition AEs that result in death. Any rigorous evaluation would compare hospitalizations due to AEs in control versus hospitalizations due to AEs in the vaccine group, not AEs (regardless of whether they are AESIs or just AEs). Again, the comparison was deeply intellectually dishonest then in the preprint and remains so now.

There’s another issue here as well. The rate of hospitalizations in the placebo control group would be expected to be highly dependent on the level of COVID-19 that was circulating in the populations tested during the time period in which the clinical trial was carried out, as these trials were not challenge trials, in which subjects are intentionally exposed to the virus. As a result, most people in the placebo and vaccine groups were not exposed to COVID-19, because these trials were carried out in the summer and early fall of 2020, before the really big winter surge hit.

Here’s a graph of COVID-19 cases in the US in 2020-21:

COVID-19 cases in 2020-1.

Enrollment for the Moderna trial ended on October 23, 2020; for the Pfizer trial, November 14, 2020. Note that this was before the winter surge took off. Had the trial started a few months later and ended in, for example, February 2021 or later, you can bet that the rates of hospitalization for COVID-19 would have been much higher in the placebo control group.

The bottom line is that this study is deeply misleading based on what sure looks like data dredging combined with misleading comparisons, further combined with a low enough risk of COVID-19 in the two populations to allow for a low rate of hospitalization when normalized to the entire population in the control group.

This paper is nothing less than antivax misinformation disguised as a “reanalysis” of the original Pfizer and Moderna trials. Worse, it doesn’t even show what it claims to have shown, that the trials were somehow designed and written in such a way as to hide lots of adverse events. As such, it’s no different at its core than Peter Doshi’s back-of-the-envelope “reanalysis” of the Pfizer trial in which he claimed that the vaccine was not more than 90% effective but rather only 19% effective. It just has fancier statistics and a much longer discussion of limitations, including this section, a beefed-up version of what is undoubtedly pure Peter Doshi, given how it echoes what he’s been demanding since January 2021, when he falsely claimed that the vaccines were only 19% effective:

We emphasize that our investigation is preliminary, to point to the need for more involved analysis. The risks of serious AESIs in the trials represent only group averages. SAEs are unlikely to be distributed equally across the demographic subgroups enrolled in the trial, and the risks may be substantially less in some groups compared to others. Thus, knowing the actual demographics of those who experienced an increase in serious AESI in the vaccine group is necessary for a proper harm-benefit analysis. In addition, clinical studies are needed to see if particular SAEs can be linked to particular vaccine ingredients as opposed to unavoidable consequences of exposure to spike protein, as future vaccines could then be modified accordingly or sensitivities can be tested for in advance. In parallel, a systematic review and meta-analysis using individual participant data should be undertaken to address questions of harm-benefit in various demographic subgroups, particularly in those at low risk of serious complications from COVID-19. Finally, there is a pressing need for comparison of SAEs and harm-benefit for different vaccine types; some initial work has already begun in this direction. [47].

Full transparency of the COVID-19 vaccine clinical trial data is needed to properly evaluate these questions. Unfortunately, as we approach 2 years after release of COVID-19 vaccines, participant level data remain inaccessible. [45], [46].

Again, given the billions of doses of these vaccines administered thus far, the only reason to sift through subject-level data in nearly three year old clinical trials of the Pfizer and Moderna vaccines, each of which only included only tens of thousands of participants, is if you’re looking for dirt and fraud, not to do an objective evaluation of whether the vaccines are safe and effective. After all, conditions have changed; the rise of the Delta and Omicron variants led to the vaccines becoming significantly less effective against infection and transmission, even as they remain potent at preventing hospitalization and death. If you really want to know how safe and effective the vaccines are now (and have been over time), the original clinical trials are far from the best dataset to use, and this “reanalysis” failed to find even a signal that there might have been a severe design defect or fraud in the original clinical trials. I’m all for data transparency and have no objections to releasing patient level data, but Doshi knows that this is not a trivial thing to do, as all the data have to be carefully de-identified and curated.

The authors

Whenever reviewing a paper of this sort, I always find it useful to look at the authors, even at the risk of being accused of ad hominem. Of course, it’s not an ad hominem if you discuss the persons making an argument briefly and then deconstruct the argument they make based on science, data, and reasoning. After all, track record matters. Interestingly, this group is, to say the least, a mixed bunch, ranging from authors associated with antivaxxers to highly respected academics: Joseph Fraiman, Juan Erviti, Mark Jones, Sander Greenland, Patrick Whelan, Robert M. Kaplan, and Peter Doshi.

I’ll start with the corresponding author, Peter Doshi, because regular readers will know that we’ve written about his antivaccine messaging before. In brief, Doshi is a senior editor at The BMJ, which inexplicably hired him several years ago despite his long history of playing footsie with the antivaccine movement since at least 2009, amplifying antivaccine conspiracy theories, downplaying the severity of influenza and thus feeding antivaccine narratives, using sleight-of-hand to downplay the effectiveness of flu vaccines, and generally playing the role of a false skeptic with respect to vaccines, as well as having signed a petition in 2006 “questioning” whether HIV causes AIDS. It continues to employ him even after he fell for a conspiracy theory that the Vaccine Adverse Events Reporting System (VAERS) database was being made inaccessible to suppress reporting of serious adverse events. That’s not all, though, he’s also served as an expert witness for the plaintiffs in antivaccine leader Robert F. Kennedy Jr.’s lawsuit against the University of California’s influenza vaccine mandates.

Since the pandemic, Doshi has only gotten worse. For example, he has used his title as a BMJ editor when taking part in a “roundtable” organized by Sen. Ron Johnson to go dumpster diving in VAERS to find “vaccine injuries” due to COVID-19 vaccines, whether the injuries were caused by them or not. In his testimony, Doshi denied that COVID-19 at the time (November 2021) was a “pandemic of the unvaccinated”. He even cited cherry-picked tables to claim that the vaccine wasn’t saving lives in what was basically an updated rehash of the nonsense he had peddled a few months earlier in which he claimed that there was “no biodistribution data” for COVID-19 vaccines and made a number of other negative false claims about the vaccines (also deconstructed by Dr. Hilda Bastian). In a truly risible moment, he even cited the Merriam-Webster definition of “antivaxxer” as opposed to those supposedly opposed to vaccine mandates to argue that he and his fellow COVID-19 contrarians were “not antivaccine” and that large numbers of people would qualify as “antivaccine”. He even parroted the antivaccine talking point that mRNA vaccines are not really vaccines and therefore shouldn’t be mandated like vaccines.

But what about the rest? Two other names that immediately leapt out at me are Joseph Fraiman and Patrick Whelan. Dr. Fraiman is an emergency medicine doctor in New Orleans who has appeared at the Urgency of Normal summit held by Gov. DeSantis and Dr. Ladapo and been associated with COVID-19 minimization since the beginning of the pandemic. In particular, recently he has been arguing that children should not be vaccinated against COVID-19 using arguments identical to those used by antivaxxers to argue against childhood vaccines since…time immemorial: “The question is, if you have a child who is at risk or has co-risk factors for COVID-19, that’s a discussion with your pediatrician, but if you have a healthy child, the chances of that child dying are incredibly low, essentially close to zero if not actually zero”. He’s also made statements claiming that “unvaccinated are more educated on the vaccine than most people who have gotten it”, and that experts cannot disprove concerns made by anti-vaccine advocates. (He’s also recently embraced the Great Barrington Declaration despite its increasing irrelevance.) You get the idea.

As for Patrick Whelan, he’s a pediatric rheumatologist at UCLA who made a splash in December 2020 by testifying to the FDA about his fears that the Pfizer mRNA-based COVID-19 vaccine could cause microvascular injury to the brain, heart, liver and kidneys in ways not assessed in safety trials and has been featured on antivaccine websites. He’s also warned against vaccinating those who have had COVID-19 because “sky high” antibodies after vaccination in people who were previously infected may contribute to adverse events. He, too, appeared at Sen. Ron Johnson’s antivaccine panel to fearmonger about spike protein from COVID-19 vaccines. Of late, Dr. Whelan appears to have been more quiet, with more recent searches turning up his name mainly in association with his December 2020 testimony and things he said in 2021. This paper apparently represents a return to vaccine “skepticism”.

I’ll be honest, I wasn’t able to find much about Mark Jones and Juan Erviti. Jones is an associate professor of health sciences and medicine at the Institute for Evidence-Based Healthcare, as well as a biostatistician at Bond University in Australia. Searches for his name related to COVID-19 turned up little except references to the preprint and Vaccine study, an article on cancer overdiagnosis, with little on COVID-19 other than articles studying healthcare workers’ responses to the pandemic and a letter asking why more studies of COVID-19 vaccine efficacy haven’t been carried out in Australia. As for Erviti, he’s published with Doshi before in The BMJ expressing skepticism over monitoring of COVID-19 vaccine safety and efficacy, as well as a BMJ Rapid Response to an article demanding patient-level data from the Pfizer and Moderna vaccine trials.

Then there are Sander Greenland and Robert M. Kaplan, who will feature in the next section.

The real reason why Vaccine published this paper?

The two authors of this paper who seem beyond reproach are Sander Greenland and Robert M. Kaplan. Greenland, as I mentioned above, is a retired emeritus professor and a veritable god of statistics, someone whom I have regularly cited before. Kaplan is a Distinguished Emeritus Professor of Health Services and Medicine at UCLA, where he led the UCLA/RAND US Agency for Health Care Research and Quality (AHRQ) health services training program and the UCLA/RAND CDC Prevention Research Center. Currently, he is an adjunct professor of medicine and primary care and population health at Stanford University. He’s had a long and storied career, having published over 560 articles and chapters and served as Chief Science Officer at the AHRQ and Associate Director of the National Institutes of Health, where he led the behavioral and social sciences programs. He was also Professor and Chair of the Department of Family and Preventive Medicine at the University of California, San Diego and is past president of several professional societies, including the American Psychological Association Division of Health Psychology, Section J of the American Association for the Advancement of Science (Pacific), the International Society for Quality of Life Research, the Society for Behavioral Medicine, and the Academy of Behavioral Medicine Research. He was also elected to the National Academy of Medicine in 2005.

Seeing these two makes me wonder: Why on earth did they sign on to this awfulness? For Sander Greenland, there could well be an answer. I can’t believe I either didn’t know or remember this, but this isn’t Greenland’s first antivax rodeo. Does anyone remember the Autism Omnibus hearings? Back in the mid-2000s, the Vaccine Court decided to take an approach to the flood of claims attributing autism to “vaccine injury” in which complainants would choose a few cases that they considered to be the very best, the most convincing, as evidence of a link between vaccines and autism, dividing them into two groups: causation due to mercury in the thimerosal preservative that had been used in many childhood vaccines before 2002 and other causation (e.g., MMR as a cause of autism). These cases then served as “test cases” to determine whether the ~5,000 other cases could go forward. Let’s just say that the complainants’ cases were all based on bad science and featured many of the usual suspects as expert witnesses arguing for vaccine causation. However, it turns out that one of those expert witnesses was not one of the usual suspects. As Steve Salzberg described at the time of a ruling on the test cases for a thimerosal-autism link, it was Sander Greenland:

It’s interesting that Vowell found that even if the “exquisitely small” amounts of mercury in vaccines had an effect, they wouldn’t cause autism. It was also somewhat sad to see how a well-known statistician, UCLA professor Sander Greenland, appearing in support of the thimerosal-autism link, embarrassed himself by presenting testimony that “largely represented an opinion based on a set of assumptions,” according to the ruling. Greenland’s arguments relied entirely on the existence of “clearly regressive autism,” but the Special Master pointed out that Greenland “was not qualified to opine on its existence.” Ouch.

Apparently, if the comments after Salzberg’s post are to be believed, Greenland later tried to claim that he had not testified in favor of a link between thimerosal-containing vaccines and autism, but he most definitely did. I have the receipts in terms of the transcript of his testimony.

As for Kaplan, I still remain puzzled why he put his name on this paper. It’s a blot on his reputation. On the other hand, in July 2021, Kaplan was arguing that vaccines were getting more credit than they deserved for the decline of COVID-19 cases being observed at the time, touting—you guessed it!—”natural immunity”:

As coronavirus infections decline in the U.S., it seems appropriate to celebrate the triumph of vaccines over viruses. But how much of the credit do vaccines deserve? Less than you might expect.

And later:

But one trend appears to be clear. As in previous pandemics, the rapid fall in new cases preceded the widespread distribution of vaccines. Although vaccines deserve much credit for declining rates of Covid-19, the protection provided by natural infection has been underappreciated. Emerging evidence shows that previously infected people have effective and durable immunity that rivals or exceeds the benefits of vaccines.

Sound familiar? I also note that this was before the rise of the Omicron variants, which have shown that “natural immunity” after infection is anything but “durable”. Still, you get the idea. Kaplan also wrote an op-ed for The Wall Street Journal, a hotbed of COVID-19 minimization, in which he stated that the “idea that recent, deliberate misinformation campaigns created hesitancy to the Covid-19 vaccine appears itself to be misinformation”. So apparently Kaplan was predisposed to Doshi’s narrative about the Pfizer and Moderna clinical trials because he thinks that “natural immunity” to COVID=19 is being ignored by public health officials. I could be wrong, but at least that’s the way it appears to me.

Defenders of Fraiman et al. will, of course, vigorously deny that they are antivaccine (as do all the authors themselves) and frequently issue a rejoinder along the lines of accusing anyone who questions anything about the safety and efficacy of any vaccine “antivaccine”. My rejoinder to such a rejoinder would be to suggest that maybe—just maybe—when you coauthor a study that leads to headlines on antivax sites like “Landmark first peer-reviewed study on Pfizer and Moderna covid vaccines confirms ‘excess risk’ of adverse side effects“, “New Study of Pfizer and Moderna Data Suggests Vaccine Harm Outweighs Benefit” (on The Epoch Times, yet!), or “Vaccinated at Higher Risk of Serious Adverse Events: Reanalysis of Original Trial Data” (also The Epoch Times), it should give you serious pause about what has been published. Again, maybe—just maybe—when your study inspires a hard core antivaxxer like Dr. Paul Alexander to publicize on his Substack with a rabid-sounding title like “BOOM! Florida Surgeon General Dr. Joe Ladapo SCHLONGING big pharma Pfizer & Moderna, all of them; new Doshi study in VACCINE journal shows catastrophic adverse events after mRNA injections” (and using the study as the basis for a whole lot of other invective against vaccines), you should reconsider where your career has brought you. (As an aside, I must admit that I’ve never seen “schlong” used as a verb, much less in all caps, in this manner. Clearly, I am more sheltered than I had thought.) Maybe—just maybe—when your study has inspired over 16,000 Twitter interactions across over 2,400 Tweets and 14,000 re-Tweets in just five days, the vast majority of which are using your study to amplify antivaccine conspiracy theories, you might ask yourself if your study is actually antivax propaganda more than a study.

A few more examples, because I can’t resist:

As I perused mentions of this study on Twitter, I also noted that quite a few of the Tweets were just links to the study Tweeted in response to pro-vaccine Tweets.

Don’t get me wrong, though. Contrary to how antivaxxers will characterize what I just wrote, I’m not saying that vaccines are sacrosanct in any way. It is certainly true that, if a scientist participates in a scientifically sound study that comes to a conclusion that antivaxxers happen like because it calls the safety or efficacy of a vaccine into question, absolutely the scientist should stand by the results. Unfortunately, that is not what we have here—far from it! In fact, what we appear to have here is not a group of scientists innocently wondering if there were more adverse events (of special interest!) in the vaccinated group in the Pfizer and Moderna studies, but rather a group of scientists that included several who almost certainly started with the conclusion that the vaccines must be more harmful than the disease and looked for a manner to “reanalyze” the clinical trial data to come to that conclusion, even if they had to compare apples and oranges (AEs and hospitalizations) and redefine AES as AESIs in order to achieve the desired result. Even then they failed, as David Grimes points out.

I’ll conclude this post with a little speculation—and I openly admit that it is speculation, as I have no “inside knowledge”—about why this paper, as horrible as it is, as heavily (and correctly) criticized as the preprint was in June, ended up in Vaccine with only relatively minor changes in the context of its findings. I strongly suspect that it’s the same reason that John Ioannidis was able to get a much worse paper published rather easily in which he used a satirical index of scientific influence in order to portray critics of the Great Barrington Declaration as far more influential on social media than they are for actual science. It’s the name. When someone like Ioannidis submits a manuscript as the most published living scientist at the moment, someone whose reputation is downright intimidating, rare is the peer reviewer or editor who will just say: Reject! Although Sander Greenland and Robert Kaplan are not quite at the level of John Ioannidis, they are very eminent in their fields, with Greenland being a legend. Again, I cannot demonstrate it, but it’s not unreasonable to suspect that those names dazzled the reviewers and editors, who changed what should have been a rejection to a conditional acceptance in which the authors had to change a few things and add a lot more caveats but left their basic analysis more or less intact. Thus was born antivax propaganda disguised as a “reanalysis” of clinical trial data.

I end with a plea to the most eminent physicians and scientists among us: Learn to understand how exercises like this one are weaponized as disinformation and refuse to lend your eminent names to them. When you sign on to a manuscript like this, you become part of the problem and provide antivaxxers with the powerful ammunition of your reputation, no matter how much you might try to tell yourself that you’re just doing a scientific study and are not antivaccine.