With COVID-19 cases continuing to rise, and no vaccine available anytime soon, investigations into repurposing existing drugs, to treat or prevent infections, continue to be a priority. Thankfully, the evidence base has improved dramatically since the early months of 2020 where there was almost nothing to guide treatment. Initial observations, starting in China and then Europe, led to case series reports and eventually, medical professionals started to form hypotheses about potentially effective therapies. Some of these approaches sounded plausible, and weak but promising observational studies started to emerge, showing correlations between specific treatments and better outcomes for patients. At the same time, randomized controlled trials (RCTs) were launched, based on the expectation that (1) these therapies might offer some benefit and (2) better evidence was needed. RCTs are experiments that are intended to reduce sources of bias, which can confound or distort the findings that come from observational studies. RCTs of COVID-19 treatments are now appearing regularly, as there has been adequate time to plan them, recruit patients, administer the therapy, collect data, and then evaluate and publish the results. And the conclusions of these trials show that some previously promising therapies did not actually seem to work. This 180° pivot in treatment advice after only a few months may seem bewildering – after all, if the medical advice isn’t consistent, why should you believe the newest evidence? But the findings of RCTs are exactly what we should have expected when we started treating COVID-19 based on anecdotes and research methods that were almost certain to have hidden biases. As more robust evidence emerges, the evidence-based approach to treating COVID-19 is changing. And it will continue to change. That is a good thing. One way to illustrate this evidence evolution is to examine two therapies that were both initially thought to be promising: Hydroxychloroquine and dexamethasone.
The importance of randomized controlled trials
Correlation does not equal causation. This is embedded in the minds of most health professionals during their training. But medical professionals are human, and humans are prone to biases. Carefully designed research can cut through biases and give us a better perspective on the value of different treatments. RCTs are sometimes called the “gold standard” of evidence. While RCTs cannot answer every question, they are a rigorous method of examining if there is a cause and effect relationship between a treatment and an outcome. One of the key feature is randomization: it distributes a population randomly into two (or more) groups, where we expect the only meaningful difference between the two groups is the treatment itself. A second important but not essential feature is blinding: where both the investigators and the participants are not aware of where the patient is randomized, to avoid any hidden biases in evaluating the effects. RCTs can be flawed (in terms of design, conduct and analysis), and they are time consuming and expensive to mount. However, a well-designed RCT provides a better view of a treatment’s “true” effect than a comparable observational study.
RCTs can and have shown that widely accepted treatments and practices are in fact useless. The example that I still think about today is a paper that was published while I was in pharmacy school, and its 30-year-old lessons are still relevant. The Cardiac Arrhythmia Suppression Trial (CAST) was a randomized controlled trial that was designed to test the effectiveness of drugs to treat a specific type of arrhythmia: heart rhythm dysfunction that had no symptoms, but could be detected with cardiac monitoring equipment. The occurrence of these premature beats was known to be a risk factor for subsequent sudden death, and medication existed that had been shown to reduce their frequency. Drug treatment of these arrhythmias was common, and many millions were spent every year on drug therapy. However, no evidence actually existed to prove that treating these arrhythmias actually reduced the risk of death. A randomized controlled trial, CAST, was established to presumably demonstrate the merits of this approach. 1,500 people were randomized to one of encainide, flecainide, or placebo. Both of these drugs were already in use to treat arrhythmia, so it was expected that they would show a mortality benefit. However, the trial was stopped early. While the drugs did reduce ventricular premature beats, they were also causing a significantly increased rate of deaths due to arrhythmia or cardiac arrest, as well as increasing the risk of death due to a second heart attack. The drugs were killing patients (8.3% mortality on the drugs vs. 3.5% on placebo), not helping them. Not only did CAST reinforce the importance of studying endpoints that matter to patients (like death) instead of surrogate indicators (like the presence of premature beats) but it reinforced the critical importance of rigorously testing therapies, prospectively, with RCTs.
How we know hydroxychloroquine doesn’t have any beneficial effect on COVID-19
David Gorski and other contributors have written numerous posts over the past several months regarding hydroxychloroquine from its anecdotal beginnings based on Chinese hypotheses to observational trials that were published in France. I refer you to those excellent posts if you’re interested in the politics, posturing, and bad arguments in favour of its use. I want to show you why there is now convincing evidence that hydroxychloroquine is ineffective, by looking past the (methodologically) weaker trial designs, and summarizing the six largest, most rigorously conducted trials. Five of these trials have been published, and one is available in pre-print only. (Thank you to Dr. Perry Wilson for the helpful list on Twitter.) Here are the findings and links so you can peruse the research yourself:
- In patients admitted to a hospital with mild-to-moderate COVID-19 and randomized to hydroxychloroquine or a placebo for 2-3 weeks, there was no difference in terms of improvement (measured by checking if the body had eliminated the virus) at the 28 day point. [Ref: BMJ 2020; 369:m1849]
- After exposure to someone with COVID-19, patients were randomized to hydroxychloroquine or placebo for five days. There was no difference in terms of the subsequent rate of infections. The hydroxychloroquine group experienced more side effects. [Ref: N Engl J Med 2020; 383:517-525]
- Non-hospitalized patients with confirmed COVID-19 and less than five days of symptoms were randomized to hydroxychloroquine or placebo for one week. There was no difference in terms of the reduction of viral load, the risk of hospitalization, or the time to resolution of symptoms. [Ref: Clinical Infectious Diseases 2020; ciaa1009]
- In non-hospitalized adults with confirmed COVID-19 or probable COVID-19, patients were randomized to hydroxychloroquine or placebo for five days. There was no difference in overall symptom severity over 14 days. The hydroxychloroquine group experienced more side effects. [Ref: Ann Int Med 2020; doi.org/10.7326/M20-4207]
- In hospitalized patients with suspected or confirmed COVID-19 who were receiving supplemental oxygen, patient were randomized to usual care, usual care plus hydroxychloroquine, or usual care plus hydroxychloroquine and azithromycin. There was no difference between the groups in terms of clinical status at day 15. [Ref: N Engl J Med 2020; DOI: 10.1056/NEJMoa2019014]
- In a preprint article (not yet peer-reviewed), hospitalized patients with COVID-19 were randomized to hydroxychloroquine for 10 days or until discharge, versus usual care. There was no difference with respect to 28 day mortality, but the hydroxychloroquine group had an increased duration of hospital stay and an increased risk of progressing to ventilation or death. [Ref: medRxiv 2020; doi 10.1101/2020.07.15.20151852]
Dr. F. Perry Wilson (@methodsmanmd) has done the world a great service by putting all the RCTs into the following table, where you can contrast with Didier Raoult’s paper (row 1, previously discussed by David Gorski), which was not an RCT:
The table is not every RCT, and more RCTs will undoubtedly emerge. But prospectively conducted rigorous trials give a consistent picture: hydroxychloroquine does not have any meaningful effects on the prevention or treatment of COVID-19 infections.
One good trial can change practice
In contrast with hydroxychloroquine, another cheap generic drug that has been around for years has also been studied for COVID-19, and the results don’t seem nearly as controversial. Dexamethasone is from the class of drugs known as corticosteroids – a type of drug that reduces inflammation. Corticosteroids mimic the chemical cortisol, a hormone produced by the body (corticosteroids are not similar to steroids like testosterone, which have different functions). Corticosteroids have a huge array of effects and that’s why you see them in non-prescription anti-itch creams (e.g., hydrocortisone) or used in the treatment of inflammatory diseases (e.g., prednisone). Along with a huge array of potentially beneficial and even life-saving effects (through reducing unwanted inflammation) there is also a huge list of side effects, which increase in severity with dose, type of corticosteroid, and duration of use.
Corticosteroids are regularly used in patients that are critically ill and in intensive care units. Their use has not been without controversy and it has not always been clear that the more immediate, beneficial effects correlate with longer-term outcomes, like survival. One of the earliest observations in patients with COVID-19 infections was the lung damage observed on X-ray. Given corticosteroids have the potential to reduce inflammatory damage, there was interest in testing them to see if they actually improved survival. These drugs were also used in the past to treat other illnesses related to COVID-19, like SARS and MERS, but the overall evidence from this use was unclear owing in part to different doses and treatment protocols. Dexamethasone is inexpensive and available in virtually every hospital and pharmacy.
The RECOVERY group (Randomised Evaluation of COVID-19 Therapy) designed an evaluation of dexamethasone in critically ill patients. (The same research group is responsible for the hydroxychloroquine paper that is still in preprint (#6) above.) It prospectively randomized hospitalized patients to receive dexamethasone or usual care. Patients received oral or intravenous dexamethasone (6mg daily) for ten days. Over 2,000 patients were recruited starting in mid-March, 2020. The primary outcome was 28-day mortality. Overall, 22.9% of patients that received dexamethasone died, compared to 25.7% of patients who received usual care. This result was statistically (and clinically) significant, and was even more pronounced in patients that received mechanical ventilation: 23.3% in the dexamethasone group versus 41.4% in the usual care group. Those who received oxygen without ventilation also benefited (23.3% vs. 26.2%). In the group that was receiving no ventilation support at randomization, there was no effect.
This large, rapidly conducted trial gave a clear answer to an important question. Dexamethasone will reduce the risk of death in some hospitalized patients by up to one-third. It clarifies early uncertainty about the risks and benefits of corticosteroids and has changed how COVID-19 will be treated.
Good evidence matters
In a time of pandemic, there is pressure to act. When COVID-19 arrived, medical professionals had to base treatment decisions on low-quality evidence, even anecdotes. There is tremendous pressure on researchers and medical professionals for clear, consistent answers, which is near-impossible when you start with almost nothing. Initial uncertainty is now being replaced by higher-quality, less biased evidence. Well-designed, rigorous research is giving us answers and changing medical practice – telling us that some therapies are ineffective, while some will save lives. RCTs are not perfect, and cannot answer every question we have about COVID-19. However, they contain safeguards in their design and when done well, can dispel accumulated dogma. We need more evidence on how to prevent and treat COVID-19 infections, and we need to be humble enough to accept that what we think may be true today, isn’t.