The State of Florida spreads antivaccine disinformation disguised as an epidemiological “study”

We here at Science-Based Medicine (SBM) have been combatting antivaccine misinformation and disinformation for nearly 15 years. In our early days—and even our not-so-early days—such pseudoscientific misinformation was disseminated primarily by antivax quacks and the people who believed them. Think Andrew Wakefield, Robert “Dr. Bob” Sears, Paul Thomas, Sherri Tenpenny, Kelly Brogan, and the like, as well as propagandists like Del Bigtree, Mike Adams, and Robert F. Kennedy, Jr. Unsurprisingly, when the COVID-19 pandemic hit, it didn’t take long for these antivax propagandists to pivot first to COVID-19 minimization and then, when COVID-19 vaccines were introduced, to repurposing old antivax tropes to spread fear, uncertainty, and doubt (FUD) about the new COVID vaccines. They were also soon joined by a new generation of antivax physicians and propagandists, as evidenced by the rise of the hydroxychloroquine-promoting America’s Frontline Doctors in 2020 during the first summer of the pandemic. Among that group of disinformation-peddling (now grifting) physicians was a man named Dr. Joseph Ladapo.

Unfortunately for the people of Florida—and, to a somewhat lesser extent, the people of the US in general—about a year ago Dr. Ladapo and his rejection of public health interventions such as masking and vaccines caught the eye of Florida’s COVID-19 minimizing and antivax Governor Ron DeSantis, who appointed him as Florida’s Surgeon General, the official in charge of the Florida Department of Health, in essence the entire medical and public health bureaucracy of the state. Unsurprisingly, Dr. Ladapo immediately started doing Gov. DeSantis’ bidding with respect to the pandemic—it is, after all, what he was hired to do—and promoting the message of “rejecting fear” as a public health strategy, while implementing the Great Barrington Declaration-like policies preferred by Gov. DeSantis that basically stopped trying to slow or mitigate the spread of COVID-19 and let children be infected in schools, even hosting a widely publicized “Urgency of Normal” roundtable with Gov. DeSantis and outright antivaxxers like Dr. Robert “inventor of mRNA vaccines” Malone, to promote “don’t worry, be happy” COVID-19 minimizing and denying policies. Mainstream public health officials like Dr. Ashish Jha, have noted:

Equally troubling for his critics was Dr. Ladapo’s failure to reject more fringe views on virus treatments, including the drugs hydroxychloroquine and ivermectin. He joined Mr. DeSantis in clamoring for the federal government to supply some monoclonal antibody treatments even after they had been deemed ineffective against the Omicron variant, which dominated caseloads.

“To say he’s out of the mainstream would be an understatement,” said Dr. Ashish K. Jha, dean of the Brown University School of Public Health. “His views are not only very unorthodox — they don’t make any sense.”

Indeed they don’t, so much so that one of his supervisors at UCLA stated in an evaluation requested by the Florida Senate as part of a background check that Ladapo’s hands-off approach toward managing COVID-19 made his colleagues feel uncomfortable, with the responses to questions asked on the evaluation being—shall we say?—less than glowing:

Question: “Would you recommend the applicant for employment as a surgeon general of Florida and confidence in his ability, honesty and integrity to perform related duties?” Answer: “No. In my opinion the people of Florida would be better served by a surgeon general who grounds his policy decisions and recommendations in the best scientific evidence rather than opinions.”

Then, there was a vague prompt called “personal relations (rapport with co-workers, supervisor).” The complete response: “I cannot answer… because Dr. Ladapo’s opinions, published in a number of popular media outlets, were contrary to the best scientific evidence available about the Covid-19 pandemic and caused concern among a large number of his research and clinical colleagues and subordinates who felt that his opinions violated the Hippocratic Oath that physicians do no harm. This situation created stress and acrimony among his co-workers and supervisors during the last year and a half of his employment. It is important to note that during this time at UCLA, he met all of the contractual obligations for the position that he was hired to perform, which is the underpinning of my otherwise satisfactory evaluation.”

That last bit was certainly the proverbial damning with faint praise. However, I fear that that evaluation actually made Dr. Ladapo more attractive to Gov. DeSantis and the Florida Senate. Be that as it may, unfortunately, when you hire an antivaxxer—and, yes, it’s no longer debatable to me that Dr. Ladapo has become an antivaxxer—to be in charge of your state’s public health policy, bad things happen, bad things like a study being weaponized as antivax disinformation. That’s exactly why Gov. DeSantis wanted Dr. Ladapo, to cover his predetermined policy choices with a veneer of seemingly plausible science, all to justify this conclusion:

The State Surgeon General now recommends against the COVID-19 mRNA vaccines for males ages 18-39 years old.

Individuals and health care providers should also be aware that this analysis¹ found:

• Males over the age of 60 had a 10% increased risk of cardiac-related death within 28 days of mRNA vaccination.
• Non-mRNA vaccines were not found to have these increased risks among any population.

Floridians are encouraged to discuss all the potential benefits and risks of receiving mRNA COVID-19 vaccines with their health care provider. The risk associated with mRNA vaccination should be weighed against the risk associated with COVID-19 infection.
The Department continues to stand by its Guidance for Pediatric COVID-19 Vaccines issued March 2022, which recommends against use in healthy children and adolescents 5 years old to 17 years old. This now includes recommendations against COVID-19 vaccination among infants and children under 5 years old, which has since been issued under Emergency Use Authorization.

Notice how the part about non-mRNA-based vaccines not showing this supposed effect is glossed over. If the intent of this press release and “guidance” were anything other than antivax, then you’d see a section about how all adults should be vaccinated against COVID-19 but with the caveat that young men aged 18-39 should receive vaccines that aren’t based on mRNA technology. Funny, but I didn’t see that anywhere. Maybe I missed something?

As you will see, this study doesn’t even show that mRNA-based COVID-19 vaccines are more risky than getting COVID-19.

Dr. Ladapo issues a press release

This brings us to something I had never seen before, a state warning people not to get vaccinated based on a poor quality study that was clearly willfully misinterpreted. On Friday, Florida issued an official press release that portrayed a vaccine as more dangerous than the disease, at least for some people, State Surgeon General Dr. Joseph A. Ladapo Issues New Mrna COVID-19 Vaccine Guidance:

Today, State Surgeon General Dr. Joseph A. Ladapo has announced new guidance regarding mRNA vaccines. The Florida Department of Health (Department) conducted an analysis through a self-controlled case series, which is a technique originally developed to evaluate vaccine safety.

This analysis found that there is an 84% increase in the relative incidence of cardiac-related death among males 18-39 years old within 28 days following mRNA vaccination. With a high level of global immunity to COVID-19, the benefit of vaccination is likely outweighed by this abnormally high risk of cardiac-related death among men in this age group. Non-mRNA vaccines were not found to have these increased risks.

As such, the State Surgeon General recommends against males aged 18 to 39 from receiving mRNA COVID-19 vaccines. Those with preexisting cardiac conditions, such as myocarditis and pericarditis, should take particular caution when making this decision.

“Studying the safety and efficacy of any medications, including vaccines, is an important component of public health,” said Surgeon General Dr. Joseph Ladapo. “Far less attention has been paid to safety and the concerns of many individuals have been dismissed – these are important findings that should be communicated to Floridians.”

The analysis can be found here. The guidance can be found here.

Let’s dig in. I will start with the study, which doesn’t really show what Dr. Ladapo claims it shows, and then move on to the guidance, which clearly willfully extrapolates far beyond what this nothingburger of a study can even be claimed to show. Finally, we’ll see how news sources and antivaccine media have amplified this “study” so that its repercussions go far beyond the State of Florida.

A bad, bad study

Before I take on the study, I have to wonder what the authors of the BMJ commentary about self-controlled case series cited by Dr. Ladapo (Irene Petersen, Ian Douglas, and Heather Whitaker), feel about the antivaxxers running Florida’s public health bureaucracy co-opting their work to spin COVID-19 vaccines as more dangerous than the disease for men under 40, and after I discuss the study itself I’ll get to how their commentary actually includes points that show major flaws in Dr. Ladapo’s “study”.

As for the study itself, it’s a really odd beast in several ways. First, no authors are listed. In a way, I can sort of understand that. If I were forced by my boss to produce a study like this one, I wouldn’t want my name associated with it either. More importantly, this study was not peer-reviewed. Rather, it was just published by the Florida Department of Health on its COVID-19 website. I mean, I could publish a “study” here on SBM or my not-so-secret other blog and claim it’s a good study. No one would believe me just based on my word—nor should anyone. To be honest, for me to take this as a serious effort, other than antivax propaganda designed to come to a predetermined conclusion, I would have at least liked to see an authors’ list and a statement that this study had been submitted to a peer-reviewed journal for consideration of publication. Unsurprisingly, I saw neither thing.

But enough of my editorializing. Let’s get to the study and—I hope—my ample justification for my opinions of the study. Its stated objective is “To evaluate the risks of all-cause and cardiac-related mortality following COVID-19 vaccination”. Its methods were stated thusly:

The self-controlled case series (SCCS) method adapted to evaluate death as the outcome was used.^1,2 The SCCS method, originally developed to assess vaccine safety, utilizes within-person comparisons to estimate the temporal association between a transient exposure and an acute event.¹ The SCCS method estimates relative incidence (RI) by comparing incidence during a defined high-risk period following exposure with incidence during a control period (i.e., all time in the follow-up period that is not the risk period).^1–4 A major strength of the SCCS method is that fixed-time confounders, such as health related risk-factors, are controlled for.^1,3

The primary analysis utilized the SCCS method developed for single exposures that cannot be repeated.^1,3,4 Since mRNA vaccinations require a multidose schedule, a simple modification was employed, where the last vaccination preceding death was used as the single exposure.² In this method, the within-individual comparison is between the immediate post-exposure period and later post-exposure periods.³

Now here’s the key part, where the rubber hits the road and the deficiencies of this methodology become apparent. Note that the study was concluded on June 1, 2022:

For the primary analysis, Florida residents aged 18 years or older who died within 25-weeks of COVID-19 vaccination since the start of the vaccination roll-out (December 15, 2020) were included.

Individuals were excluded if they (1) had a documented COVID-19 infection, (2) experienced a COVID-19 associated death, (3) received a booster, or (4) received their last COVID-19 vaccination after December 8, 2021 (to ensure each individual had the 25-week follow-up period to experience the event of interest).

I found it rather interesting that the study specifically excluded residents who experienced COVID-19 infection or death. Why, one wonders, might that be? (One wonders.) After all, death from COVID-19 is the very outcome that the vaccines were intended to prevent, and excluding them from the analysis is a rather telling sign. At the very least, if you truly wanted to compare outcomes, it would have been more appropriate to do the analysis both including and excluding COVID-19-related outcomes, in order to compare death rates among those who did and did not get COVID-19 among the vaccinated and unvaccinated. Again, I wonder why the authors didn’t do that. Actually, I’m pretty sure that I know.

I think I also know why the primary outcomes were chosen this way:

The exposure of interest was the 28-day risk period following COVID-19 vaccination.

Two outcomes were assessed. Natural all-cause deaths (i.e., excluding homicides, suicides, and accidents) and cardiac-related deaths. Cardiac-related deaths were included if their death record contained an ICD-10 code of I30-I52. For the primary analysis, only participants that experienced the exposure and outcome were included in this study.

The range of ICD codes listed above are cardiac-related, with ICD30 being pericarditis and I52 being “other heart disorders in diseases classified elsewhere.” You can look at the whole list here if you wish. It includes endocarditis, valve disorders, myocarditis, cardiomyopathy, cardiac arrest, conduction system disorders, atrial fibrillation and flutter, heart failure, other cardiac arrhythmias, endocarditis, and more. Interestingly, they left out ICD code I5A, which is non-ischemic myocardial injury (nontraumatic). I had to wonder why, given that, if you believe that COVID-19 vaccines are causing an epidemic of cardiac death (as Dr. Ladapo clearly does and had this study designed to “show”), then I’d think that such deaths might pop up under this category. Be that as it may, the authors included pretty much every cardiac diagnosis, even ones that aren’t really suspected to be related to COVID-19 vaccines. Remember, there is evidence that the mRNA-based vaccines can be associated with an increased risk of myocarditis. You can even posit possible associations in which the vaccines might be associated with arrhythmias. However, it’s a real stretch to associate them with mitral, tricuspid, or any valve disease.

As I discovered after I had written this section, Dr. Kristen Panthagani pointed it out better in that she included a chart:

In particular, her point here is most appreciated:

Exactly. The authors included basically a wastebasket set of cardiac diagnoses in their analysis, including ones without a plausible biologic link to vaccination (e.g., valvular disease) and ones that are often included in death certificates as the final cause of death, given that the terminal event in anyone dying is, ultimately, the heart stopping its beating.

Moving on, I appreciate Dr. Deepti Gurdasani’s discussion of the study on Twitter, where she was kind enough to provide a diagram at the top of her thread discussing its key flaws:

I appreciate the chart, as it conveys the information in a more compact form than I could have in my writeup, and not just because of my tendency towards logorrhea.

Follow-up began on the day of their last COVID-19 vaccination. Participants were not censored upon death, rather, they were followed for the entire 25-week follow-up period.^1–4

Or, as Dr. Gurdasani put it, first mentioning what I mentioned above, the issue of why the authors excluded the very cause of death that the vaccines were designed to prevent:

Indeed, I find the wording of some of the findings particularly…bizarre:

In the 28 days following vaccination, no increase in risk was observed for all-cause deaths. A statistically significant decrease was observed for participants 60 years or older in the 28 days following vaccination (RI = 0.97, 95% CI = 0.94 – 0.99).

Why not just start out by saying that there was a statistically significant decrease in 28-day mortality among those aged 60 and older? The answer is rather obvious. The authors almost certainly wished to deemphasize that finding by stating it after another finding that found no benefit in “all-cause mortality” (which wasn’t really “all-cause” mortality given that it didn’t include COVID-19 as one of the causes) from vaccination against COVID-19.

The authors also admit a huge limitation in their discussion:

This study cannot determine the causative nature of a participant’s death. We used death certificate data and not medical records. COVID testing status was unknown for those who did not die of/with COVID. Cardiac-related deaths were ascertained if an ACME code of I3-I52 were on their death certificate, thus, the underlying cause of death may not be cardiac-related.

In other words, even the anonymous authors themselves couldn’t say for sure whether all their “cardiac” deaths were, in fact, cardiac deaths.

The authors also (sort of) acknowledge Dr. Gurdasani’s criticism when they write:

While this method has been used to assess risk of death following COVID-19 vaccination,² it violates the assumption that an event does not affect subsequent exposure (for mRNA vaccines), which may introduce bias.⁶ Further, it does not consider the multidose vaccination schedule required for mRNA vaccination.

In fairness, not everyone was so harsh on the study design as I (and the doctors whom I cited above) have been. For example, Prof. Jeffrey Morris essentially agrees with the criticism that not censoring deaths that occur in the first 28-day period from consideration in the second reporting period violates the assumptions underlying the SCCS methodology, while conceding that the design “could” appropriately adjust for this bias:

He also cited another study that used a similar methodology carried out in the UK and published as a preprint in March that looked at the risk of death and cardiac-related death over 12 weeks among individuals aged 12-29. The preprint found:

• No evidence of an association between COVID-19 vaccination and an increased risk of death in young people;
• That SARS-CoV-2 infection was associated with substantially higher risk of cardiac-related and all-cause death.

One also notes the differences in the design of the UK study compared to the Florida “study”. First, the UK study did not exclude COVID-19 and found a six-fold increased risk of cardiac death within six weeks of infection with SARS-CoV-2, leading the authors to conclude:

Although there is a risk of myocarditis or myopericarditis with COVID-19, there is no evidence of increased risk of cardiac or all-cause mortality following COVID-19 vaccination in young people aged 12 to 29. Given the increased risk of mortality following SARS-CoV-2 infection in this group, the risk-benefit analysis favours COVID-19 vaccination for this age group.

And:

Whilst COVID-19 vaccination has been linked to an increased risk of myocarditis and other cardiac events in young people, our study shows that there is no evidence of increased risk of death due to cardiac events, which suggest that cases of myocarditis or myopericarditis due to the COVID-19 vaccination are unlikely to be fatal. This provides reassurance that the benefits of COVID-19 vaccines outweigh the risks even in young people.

There’s the big problem. The authors of the Florida study could have done such an analysis and then used it to estimate a risk-benefit ratio for vaccination versus the disease; they chose not to. Why didn’t they do that analysis, an analysis that most would consider to be very important? Again, I think you know the answer to that question. Likely such an analysis would have come to the same conclusion as that of the UK investigators, that the overall balance strongly favors COVID-19 vaccination in young people.) However, that is not the message that Dr. Ladapo wanted to convey. In fact, I strongly suspect that they probably did do such an analysis but left it out of the paper. Whatever the case, were I a peer reviewer for this article, I would refuse to endorse publishing this study unless the authors did an analysis that didn’t exclude COVID-19 cases. Failure to have done so provides a deceptive result.

In case you’re wondering how it should be done, Dr. Panthagani provided a reference to a study published in JAMA. In marked contrast to Florida’s “study,” this study looked at specific diagnoses and confirmed them using a review of medical records. The study also showed how to appropriately exclude COVID-19 cases. But wait! Didn’t the Florida authors do that? Yes, but not quite. While it is appropriate to exclude those deaths with COVID-19 listed as a cause on the death certificate, they didn’t bother to look at COVID-19 status:

Dr. Panthagani also echoes my (and many other critics’) observation that whenever you perform a risk assessment of vaccination, it is very important also to compare the risks of the vaccine versus the risks of the disease vaccinated against. Again, very tellingly, the anonymous authors of this Florida study did not do this. Similarly, they did not do a number of standard sensitivity analyses to test the robustness of their conclusions when changes are made to their assumptions and chosen time periods. I wonder why.

Dr. Ladapo’s interpretation of the study versus a more charitable view of the study

One thing that is also important to note about this study is that, for an epidemiological study of uncommon adverse events from vaccines, the sample size is quite small. Among the age group targeted (aged 18-39) there were a grand total of 20 deaths, which means that if even a tiny number of them were misclassified, the “statistical significance” of the anonymous authors’ result goes away. This brings me to Kyle Sheldrick’s analysis of the study. He’s less harsh on it than Prof. Morris, Drs. Panthagani, Gurdasani, or I have been.

Dr. Sheldrick notes:

STROBE stands for “strengthening the reporting of observational studies in epidemiology” and comes from an “international, collaborative initiative of epidemiologists, methodologists, statisticians, researchers and journal editors involved in the conduct and dissemination of observational studies, with the common aim of STrengthening the Reporting of OBservational studies in Epidemiology”. He’s right, too. If you click on one of the checklists in the link above, it’s easy to see that a lot of elements important in STROBE were not listed.

Dr. Sheldrick takes the study results (mostly) at face value and accepts that perhaps 9 additional cardiac deaths in men aged 18-39 could be attributable to COVID-19 vaccination. That’s why he attacks the spin on this study more than the study itself. I’ll admit that I’m not as convinced as he is to give the benefit of the doubt to these anonymous authors, but I can see the utility in doing so and then showing why their results still don’t justify the recommendation not to vaccinate men under 40.

He does this by making some reasonable (and conservative) assumptions about how much COVID-19 vaccination decreases the death rate from COVID-19 in this particular population:

Disagreeing with Dr. Sheldrake I’m not convinced, based on its methodological flaws, that this study accurately shows that there were nine additional deaths among the 1.8 million men in Florida aged 18-39 who underwent COVID-19 vaccination. After all, it wouldn’t take many misattributed deaths in that group at all to make that apparent increase in cardiac mortality no longer statistically significant, and the lack of any attempt to verify cause of death with medical records or to determine COVID-19 testing status are major potential sources of bias. Still, it is definitely justifiable from a science communication standpoint of Dr. Sheldrake to point out how this study, even if you accept its results as accurate, does not provide any evidence to support Dr. Ladapo’s conclusion that men under age 40 should not receive mRNA-based COVID-19 vaccines. Indeed, his conclusion goes along with all the criticism that I cited above and made myself of the anonymous authors for not doing an analysis that included COVID-19 cases and tried to determine a risk-benefit ratio for the vaccine. Again, the anonymous authors didn’t report that. (Note again that I said “didn’t report that,” as I strongly suspect that they probably did do that analysis but that Dr. Ladapo held it back, and, yes, I am suggesting that Dr. Ladapo is less than honest.)

Gideon Meyerowitz-Katz also took the study at face value. He even noted that he didn’t really care what the authors might have left out:

He then back-calculated an estimate of non-cardiac mortality:

I’m going to disagree a bit here. Even if this was a typo and the true range of ICD codes was I3-I52, that would still include a lot of diagnoses with no biologically plausible link to vaccination. Still:

I have to be honest. Meyerowtiz-Katz’s analysis makes it even clearer to me that both this study was deceptively phrased to make it sound as though the COVID-19 vaccines are dangerous, and the press releases even more so. Basically, as Meyerowitz-Katz and Sheldrake show, even if you take this analysis at face value, it does not show what Dr. Ladapo says it shows.

It’s not surprising that the usual suspects are amplifying this “study” as slam-dunk evidence that mRNA-based COVID-19 vaccines are dangerous and do more harm than good. I surveyed some of the usual suspects, and here’s just a little of what I found:

• Steve Kirsch: Florida study shows mRNA COVID vaccines kill kids. (No, it doesn’t. Also,, you might be amused by Mr. Kirsch’s attack on yours truly the other day.)
• Paul Alexander: Dr. Joe Ladapo, Florida’s Surgeon General (thanks Governor DeSantis), recommends against COVID mRNA vaccine in males 18-39 years, citing increased risk of myocarditis etc. Excellent move!
• The Daily Sceptic (a denialist website): Florida Recommends Against mRNA Vaccination for Males Aged 18-39 Due to 84% Increase in Heart Death Risk

I could go on, but Twitter is worse, as you can demonstrate for yourself by just typing the URL for the Florida press release into its search box.

Unfortunately, thanks to Gov. Ron DeSantis, the entire state medical and public health apparatus of Florida has become a tool for spreading his COVID-19 minimizing antivax propaganda, with Dr. Ladapo having eagerly signed on to be his willing accomplice in spreading misinformation. It’s a hell of a thing when an actual state public health apparatus publishes something that’s not just wrong, but intentionally misleading and dangerous, such as this study with anonymous authors. It has succeeded in its purpose, to serve as what seems to lay people like scientific justification for antivaccine messaging.