Dear Mr. Knudsen,
I recently read your article on Sensible Medicine titled Death and Isolation During a Pandemic. You wrote movingly of your grandfather’s death and reflected on the tragedy of people dying alone at the start of the pandemic. As someone who lived through it all, I couldn’t agree more. Anyone who worked in the hospital at that time remembers patients suffering without their loved ones and aching goodbyes via IPads. I’m sure you’ve read about it. It happened to an overwhelming number of families who will be forever scarred. I’ll never forget one father who screamed at us over the phone that we didn’t try hard enough to save his 35-year-old son. Had he been there, he would have at least had the comfort of knowing we did everything possible. His son had clots throughout his body, and there was nothing to do. How horrible that he believes his son might still be alive had we only tried a bit harder.
However, your article contained this problematic section. You wrote:
Now, the empirical question, does limiting visitors reduce the transmission of SARS-CoV-2 and thus lower the incidence of COVID-19? The answer to this question is important and can provide a framework for thinking about the risk-benefit calculus of allowing visitors. Unfortunately, we don’t have an answer or any high quality data supporting this question, as shown in this systematic analysis. As discussed previously, there are a host of possible upsides to this policy; however, it is unclear if they are true.
Even with reasonable predictions, we live in a world that is messy and random, and thus require rigorously designed studies to uncover one form of the truth. I imagine that a cluster randomized controlled trial could have provided an answer. This would consist of randomizing hospitals into two groups: ones that allows visitors and others that restrict them. Then, over time, outcomes would be tracked in each group to measure the impact of the policy. I specifically left outcomes vague because the pertinent outcomes are debatable, but I would argue that a good starting point is COVID-19 hospital admissions. Another relevant outcome may be the number of positive COVID-19 cases in the surrounding community – although this would be more difficult to directly link to the policy. The broader point being that there are ways to uncover answers. There are smart people in positions of power with resources that can design and conduct such studies. Ultimately, they failed us, and we are left without answers and policies that caused significant harm.
Really, what would have been the point of your study?
It might not seem so, but there’s a lot to unpack here.
We both agree that randomized controlled trials (RCTs) are the “gold standard” in medicine, so how could I object to your call for a rigorous RCT to answer an important question? To answer that, I am going to do something that the doctors behind Sensible Medicine obviously didn’t do- I am going to respect you and take you seriously.
First of all, who exactly are “they“? Whom exactly did you have in mind when you wrote, “There are smart people in positions of power with resources that can design and conduct such studies“? You seem certain that someone had the time, resources, and authority to run your imagined RCT, but just chose not to for some stupid reason. Incredibly, you also think “they” should have had the foresight to run your imagined RCT. As morgues overflowed with victims of a new virus that lacked treatments and a vaccine, you think “they” should have been marshaling their resources to conduct an RCT of hospital visitation policies, as if there were no bigger fish to fry. Of course, your imagined RCT would have sounded utterly absurd at the start of the pandemic, which is why nobody actually suggested anything like it at the time.
You see, unless you know something I don’t, there isn’t a well-funded, elite RCT Strike Force that’s been training for a pandemic- though when a real virus arrived, they sat on the sidelines doing nothing for no particular reason. You seem to think that a large group of people with relevant medical expertise were really lazy. Who exactly do you think spent the pandemic twiddling their thumbs?
Maybe there should be an RCT Strike Force, but the cold truth is that no government or institution anywhere could have instantly whipped up dozens and dozens of rigorous RCTs to answer every pandemic conundrum and quandary, as you seem to imagine. In the real-world, politicians meddled with scientific agencies, and the CDC couldn’t even get basic tests right. Much of the federal response was in shambles from the pandemic’s earliest days. You should read about it. According to one recent article:
Early on, the government went off the rails.
In February 2020, the focus was on containment, with measures such as the travel ban on China and repatriation of Americans, including those stranded on cruise ships. Emergency mobilization efforts “languished,” the report noted. There was “confusion and friction about who was in charge of what problems.” The government’s “crisis action plan” amounted to little more than jargon. “There was little in it about what people would actually do.”
On Feb. 24, 2020, President Donald Trump tweeted from India, “The Coronavirus is very much under control in the USA … Stock market starting to look very good to me!” But according to the report, that same day, the White House task force concluded “containment was failing.” It was time to shift to mitigation. The next day, a high-ranking Centers for Disease Control and Prevention official, Nancy Messonnier, announced that community spread in the United States was inevitable. The stock market dived.
“President Trump was furious,” the report recalls. He kept downplaying the danger. “It’s going to disappear,” he said on Feb. 27. “Everything is really under control,” he said Feb. 29.
It was not. The authors of the report show, in detail, how federal crisis management “splintered by the third week of March.” HHS Secretary Alex Azar had placed the assistant secretary for preparedness and response, Robert Kadlec, in charge of the HHS effort — but at the same time, Vice President Mike Pence’s staff kicked him off the White House task force. The head of the Food and Drug Administration was not even on the task force for the first month. The CDC was “fractured into too many missions.” While some officials recognized the urgency of a crash program of testing and masks, “Kadlec had no money, no real emergency fund.”
“By late April, as a frightened and bewildered country became more and more confused about continuing business and school closures, and after some brow-raising comments at a White House briefing in which he discussed treating the virus with light, heat, or disinfectant, Trump essentially detached himself from his own government,” the report says. “He moved toward questioning and challenging what other government officials were doing.”
“The administration abdicated its wartime responsibility to lead,” they add. “It left the battlefield, and the war strategy” to the states and localities. By April, the White House chief of staff concluded the task force was “useless and broken.”
This is whom you imagine had the power, resources, and competency to have carried out your imagined RCT. They are “they.”
Even if “they” had an elite RCT Strike Force at their disposal and bizarrely prioritized your imagined RCT at the start of the pandemic, it still faced many obstacles. I agree that COVID hospital admissions would be a reasonable outcome to track. But a woman who visits her sister at Hospital X, isn’t necessarily going to be admitted to Hospital X if she contracts COVID herself. At Bellevue, where I worked, nearly all of our patients were transferred from other overwhelmed hospitals. You should read about it. As such, your study would require that hospital visitors be followed and their hospital admissions tracked.
I am not an expert in this area, but it sure seems to me your imagined RCT would require informed consent for every hospital visitor. How would a grieving family feel if they lost another loved one and only later learned she was infected in a hospital that was part of a medical experiment to which she did not consent? Doesn’t she have the right to know the risks of your RCT, which include death, before she participates in it? At least one Twitter commentator had this to say about your proposed study:
Do you disagree? They ended their Twitter thread by saying, “The author of that blurb should have all their past contributions to field research reviewed for breaches of consent.”
Consenting every visitor at multiple hospitals while COVID raged would have been a Herculean task. Obtaining informed consent is a lengthy, formal process, not something done as people walk by you in the hallway. It’s not like asking people to sign a petition. Could you imagine actually trying to obtain informed consent from grieving families? What woman, on the verge of losing her husband of 50-years, would listen patiently as you explained the nature of your experiment? Much of this would have to be done via translator phones. Did you consider this? Many our patients did not speak English.
Tracking every visitor at multiple hospitals while COVID raged would also have been a Herculean task. How exactly do you think this should have been done? People would likely travel great distances to say goodbye to a loved one. Would grieving families be willing to share details of their fate to researchers over the telephone? It’s easy to imagine people would just want to be left alone if they got very sick with COVID as part of your experiment.Again, who exactly had the resources to do all this? Such expertise can’t be created out of thin air in a matter of weeks in the middle of a raging pandemic. There’s not an endless supply of clinical research coordinators. Why should these skilled professionals have dropped everything to run your imagined RCT? Thankfully, “they” chose to prioritize vaccine RCTs.
Many more questions and challenges remain:
- Would administrators have let their hospitals participate in your imagined RCT? I doubt it. An RCT of hospital visiting policies wouldn’t have been on their radar at the pandemic’s start. They were too busy trying to get PPE, staffing, and places to store the dead bodies.
- Would multiple hospital IRBs have approved a study that allowed large numbers of unvaccinated family members into unventilated rooms full of contagious patients? As morgues overflowed and forklifts were needed to move corpses into refrigerated trucks, would an IRB have actually accepted that there was true clinical equipoise and that your study needed to be run at that moment? I’m skeptical.
- I work at two hospitals, NYU and Bellevue, that are blocks apart in terms of their distance, but worlds apart in terms of the patients they treat and communities they serve. To control for these discrepancies, you’d have to enroll a large number of hospitals. How many would have to participate to get meaningful results? What would happen to your study if some dropped out?
- What if hospital staff objected to your study? We are packed like sardines on the elevators on a normal day. Your imagined RCT would have greatly increased our exposure to the virus at a time when no one was vaccinated. I wouldn’t have been thrilled to have been randomized to the “visitor’s allowed” hospital. We had enough sick healthcare workers as it was, and several of my co-workers died. You should read about them. Why didn’t you factor us into your imagined study?
- What if other family members objected to your study? It’s easy to imagine family members strongly disagreeing about whether to visit a dying relative. Imagine an elderly couple living with their two adult children. If dad gets hospitalized, his immunocompromised daughter may strongly feel the risk of anyone visiting is too high. Does she get a say in your RCT? What if her brother visits their dad against her wishes and brings the virus home. What effect might this have on the family? He too might regret his decision, especially if anyone suffers as a result. What if your RCT created great conflict in people and families? Did you think about this at all?
- What if the general public objected? Allowing hospital visitors would have meant more people on subways and buses, and therefore more risk for the essential workers who used them. You should read about their experience. Would these unvaccinated workers have had any input into your study?
- Many hospitals were deluged with COVID patients in the spring of 2020. You should read about it. It’s not unreasonable to imagine that large groups of mourners in a hospital room would have greatly increased the spread of COVID in the spring of 2020. What contingency measures would be in place if that happened? Did you even consider that possibility?
- Might patients and their families preferentially chose a hospital that was randomized to allow visitation? How would this affect the results?
- Would a trial conducted in the spring of 2020 have any relevance today? I don’t think so. Both the population and the virus itself have changed too much. The original vaccine RCTs became obsolete in less than a year. Why would your RCT age better?
- How long would your study have taken? In New York, one of the states with the strictest policies, hospital were mandated to allow visitors in June 2021. Do you really think your study would have been yielded actionable results before this? I doubt it.
- Would you, or anyone else, support visitation restrictions today if your study found that hospital visitors increased the spread of COVID? Even RCTs that yield an unambiguous scientific result rarely allow us all to agree what policy should flow from them. They almost never provide a crystal clear answer to complicated social and ethical questions, a point you recognize. But have you given any thought into how your imagined RCT might actually inform policy? Let’s pretend your study found that allowing visitors increased community COVID infections by 14.3% (or any other number). Should hospitals allow visitors or not? Would everyone agree with your answer? Are you certain that your imagined RCT wouldn’t interject uncertainty and polarization into the rare issue where there seems to be near uniform agreement today? I’m not sure exactly when the policy of restricting visitors should have ended, but like everyone else, I’m glad it’s gone. So, even if your study were perfectly and rapidly executed at the pandemic’s start, for how long would it have held relevance? Everybody made up their mind about this issue long ago, and everyone agrees on it today. So really, what would have been the point of your study?
Challenge 1
Your imagined RCT didn’t occur to anybody at the start of the pandemic and there’s no one who actually could have run it had the idea oddly popped into their head. To be blunt, I think it’s just another sad example of An Impossible, Unethical Study of Dubious Value That’s Not Meant to be Taken Seriously. Since you disagree, I have two challenges for you- one for now and one for later.
My immediate challenge is to write the protocol for your imagined RCT. I challenge you write up the exact methods, outcome measures, and power analysis for an RCT that actually would have gotten IRB approval as well as buy-in from the relevant stakeholders and that actually would have been feasible at the pandemic’s start and actionable in a useful time frame.
For reference, the protocol for just one of the pediatric vaccine RCTs was 460 pages of dense policies and procedures. You should read it. I don’t expect you to produce such a document, but it shows the enormous thought and effort that serious people put into designing an RCT, even a completely straightforward one, when they actually intend for it to be done in the real world. Moreover, RCTs are very expensive and our public health agencies haven’t been drowning in money. Quite the opposite, actually. You should read about it. Your trial sounds expensive. So in your proposal, please include an estimate of how much your study will cost, where funding should come from, and how this might have impacted funding for RCTs for vaccines and COVID treatments.
Of course, writing a protocol and securing funding is the easy part. Once those are done, the real work begins. Research staff has to be hired and trained, hospitals have to agree to sign-up, multiple IRB boards have to give their approval, subjects have to be enrolled, outcomes have to be measured, and the data has to be analyzed and published. Running a large RCT is a logistical marvel with many moving parts and opportunities for things to go wrong. I suggest you listen to people who actually tried this pandemic. You could learn a lot. One researcher said:
I think of what my clinical research team went through to enroll people in that trial, and I thought my nurses were going to die. One of them got covid and got sick. Imagine trying to do that on a daily basis, multiple patients, some of them facing intubation, none of them have their families.
Another said:
There was one day our system had 84 deaths. And then you’re going to ask me to potentially put them on a placebo? It’s just really heart wrenching, talking with families, if your patients are able to communicate — and you’re dealing with all these deaths.
It turns out that actually running an RCT is a lot harder that just imagining one, especially during a pandemic. How would your study have overcome these obstacles?
Not surprisingly, graveyards are full of dead RCTs and there’s research into why trials fail. You should read about it. This continued during the pandemic. An article from the Gates Foundation titled “Uninformative Research” is the Global Health Crisis You’ve Never Heard Of said:
According to a 2021 study of thousands of COVID-19 drug trials, only 5% of the “study arms” included enough people to generate a statistically valid result. Chaos is to be expected during a crisis like a pandemic, but the norm during “peacetime” is nearly as worrisome. A recent study of U.S. clinical trials related to heart disease, diabetes, and lung cancer found that only 26% ended informatively. To put it bluntly: Three out of four were a waste of time and money.
Even trials “they” ran failed miserably during the pandemic. Although it was much less complicated that your imagined RCT, a proposed RCT of hydroxychloroquine and azithromycin run by the NIH closed in a month after enrolling just 20 patients, or just 1% of their goal. I’m sure that study looked great on paper. Why would your study succeed when “they” couldn’t enroll more than 20 patients in the most basic RCT imaginable?
Let’s see what you can come with.
Challenge 2
My second challenge for you is more long term. I challenge you to avoid a lamentable career of merely writing blogs, imaging one impossible RCT after another and then whining that “they” didn’t do the trials you imagined. Unfortunately, throughout the pandemic, a small number of doctors, who’ve yet to make any contribution to an actual RCT, have falsely claimed that RCTs and only RCTs can answer even the most basic questions and that dozens of complicated RCTs could have easily been done if only “they” weren’t so stupid and incompetent. These doctors never promoted an actual RCT or encouraged their audience to enroll in one. Like you, they pretended the RCT Strike Force was real, and rather than do the hard work of advancing an actual RCT, they merely fantasized aloud what they would have done if they were in charge of it. It’s sad that these doctors are more famous and influential than the largely anonymous researchers who actually ran RCTs, don’t you think? (You should read this too).
Such idle fantasizing about impossible RCTs is a waste of time for a highly-trained medical doctor. It’s just not a valuable contribution. Do you really want your memoir to be titled “RCTs I Imagined and Things I Would Have Done“? I hope not.
Instead, I challenge you not to sit on the sidelines, but rather to become “they” and actually do the RCTs you imagine. You wrote that “there are ways to uncover answers”, and you made it all sound so easy. Soon, you should be the one uncovering those answers with RCTs.
In the meantime, I suggest you use your voice to educate the public that RCTs aren’t the only means by which doctors can learn about the world. There haven’t been any RCTs of the virus after all, and your statement that visitor restrictions caused “significant harm” wasn’t based on an RCT, was it? You just knew.
We can also make important decisions without an RCT to guide us. No RCT proved it would be safe to allow visitors at hospitals. We just used common sense and collectively agreed that the harm of COVID patients dying alone exceeded the unknown risk of allowing visitors. This didn’t seem to bother you. Yet, if we had clinical equipoise at the start of the pandemic don’t we still have it today? After all, your imagined RCT wasn’t actually done. So, would you support doing your RCT today, even though it would mean some families would be barred from visiting their dying loved one? I doubt it. People rarely demand rigorous RCTs to justify policies they support.
I also suggest you also use your voice to educate the public on the value of RCTs. Being part of an experiment is scary for a lot of people, yet medicine couldn’t advance without willing volunteers. Being a “guinea pig” is a derisive term, when it should be a compliment. Though it’s not been the main focus of my career, I’ve tried to enroll patients in many different studies over the years. It’s not always easy. As you’ll find out, many patients don’t trust “they“. Unfortunately, articles like yours give the public every reason to distrust “they“- the actual people who actually try to enroll patients in RCTs. How do you think “they” feel when a medical student degrades them as you did? Do you think it makes their job easier? Do articles like yours actually advance the cause of RCTs?
I don’t think so.
And this upsets me, because I believe strongly in RCTs. Those of us who recognize we can’t study every aspect of the pandemic via an RCT are not against them in principle, as some doctors imply. It would be wonderful if running an RCT was as easy as imagining one. Because I want RCTs to actually succeed in the real-world I’ve encouraged people to enroll in them. Here’s what I said in the very first article I wrote this pandemic:
I’ve always believed that people who participate in medical research are doing a noble thing. If no one volunteered for such research, medicine would never advance.
I doubt that alone convinced anyone to enter an RCT, but I suspect if more doctors spread positive messages about RCTs, instead of berating “they“, this would have had a positive impact in the real-world. My dedication is such that I volunteered to be in a vaccine-RCT. My contribution was a small one,- the study took place at my hospital and I got snacks- but at least it was a real one. I want my actions to match my words, and I don’t feel like a complete backseat driver when I say that more RCTs should have been done this pandemic. The UK’s RECOVERY program was a model for what should have been, and I actually think a well-funded, elite RCT Strike Force would be amazing.
Until that exists, however, your statement that there are “smart people in positions of power with resources that can design and conduct such studies” was gross misinformation. Endless calls for someone else to run impossible RTCs of everything are just a way for the commentariat class to do what they’ve done the entire pandemic; spread fear, uncertainty, and doubt about common sense measures to control a rapidly-spreading virus in a world with finite time and resources. Doctors imagine impossible studies not because they actually want them to get done, but rather to sow doubt in the public’s mind. They want people to think that we shouldn’t do anything until we know everything. None of this is new. Casting doubt on vaccines by calling for an impossible RCT of the entire vaccine schedule over the entire human lifespan looking at every outcome imaginable is one of the oldest anti-vaxxer tricks in the book. You should read about it.
Fantasies of impossible studies always end with some version of what you said: “they failed us, and we are left without answers and policies that caused significant harm.” In the real-world, during a raging pandemic with a brand new virus, “they” had to make many difficult decisions without the benefit of a rigorous RCT to guide them at every turn. Yet, doctors who merely imagine RCTs absurdly claim “they” have only themselves to blame for their predicament.
Unfortunately, this was all part of a larger trend where doctors with no real-world responsibility for anything made impossible things sound very easy in Tweets, editorials, and YouTube videos. They acted as if there were no difference between writing “protect the vulnerable” and actually protecting the vulnerable. As I wrote previously:
In the imaginary world, all it took to protect nursing homes and open schools was to say “protect nursing homes” and “open schools”. In the real world, this was impossible when the virus was allowed to spread freely.
When impossible things could not be done effortlessly in the real-world, “they” were always to blame. Is it any surprise that in the real-world, public health officials were routinely threatened and even attacked by angry, paranoid citizens? You should read about it.
There’s nothing inherently wrong with fantasizing how you would have handled the pandemic had you been given unlimited time, money, and power, as well as total public cooperation with all of your imagined plans- basically God-like powers. However, you should recognize that this is just play-acting, not a serious contribution. You see, none of us have to fantasize what we would have done during a pandemic. What you actually did the past three years is exactly what you would have done.
And something tells me that if you’re brave enough to devote your career to actually running RCTs, you’ll no longer be impressed by doctors who merely imagine RCTs they would have done, while moaning “they failed us” because in the real world, actually running an RCT is a lot harder than just imagining one.
Note: I also turned this into a video essay.