Shares

Given how fast events related to the COVID-19 pandemic are moving, I debated whether I should even write about this, even though it blew up on social media on Saturday. Upon reflection, though, it seemed to me a perfect example of the sorts of stories we’re going to be dealing with for the duration of this crisis, and it’s an excellent example of how responsible leadership should not behave with respect to news of potential new treatments for coronavirus disease. Although it didn’t begin with these two Tweets from President Trump on Saturday morning, it did blow up as a result:

It’s likely you’ve heard this, because whenever the President says or Tweets something, it’s news, no matter how irresponsible. Because I was working on a grant application most of the day on Saturday, I was sitting in front of my computer and, as I took the occasional social media break, saw this news explode on Twitter and Facebook over the course of hours. That led me to look into it and to discover just how thin a gruel the evidence supporting this “game changer” was and how irresponsible it was to tout these drugs as “game changers” based on a single preliminary and, to be honest, not very good study.

With the COVID-19 pandemic in the US entering the steep part of the sigmoidal curve and caseloads (and deaths) increasing exponentially, of course the public is desperately seeking some sign that there will be an effective treatment soon. Sadly (spoiler alert!), this is almost certainly not such a sign. Let’s look into why, but, first, let’s look at the two drugs and why it was suspected that they might be active against the virus causing COVID-19, SARS-CoV-2.

Hydroxychloroquine and azithromycin

Hydroxychloroquine (trade name Plaquenil) is a derivative of chloroquine (trade name Aralen), a common antimalarial drug. Indeed, some of you reading this might well have taken chloroquine as prophylaxis to prevent malaria while traveling to tropical regions where the disease is endemic. It is also used to treat amoebic liver abscesses when other drugs used for such infections are not working. These drugs also mildly suppress the immune system, which is why they are used as part of the treatment of some autoimmune disorders, such as lupus erythematosis or rheumatoid arthritis. One thing that should be understood is that these are not entirely benign drugs. They have a number of side effects and adverse reactions. In addition to more mild side effects, such as nausea, headache, loss of appetite, and diarrhea, there are two more severe potential side effects. One is that long term use of these drugs can damage the retina and lead to macular degeneration, which is why patients taking these drugs long term need regular ophthalmological examinations. They can also affect the heart by prolonging the QT interval and also lead to drug-induced torsade de pointes, a potentially lethal ventricular tachycardia.

The other drug in the combination, azithromycin (trade names Zithromax, Azithrocin, and others), is a common antibiotic, used to treat a number of infections, ranging from ear infections, to strep throat, pneumonia, and a number of sexually transmitted infections, including chlamydia and gonorrhea. It’s commonly prescribed as a “Z-Pak,” to be taken for five days, and it’s widely prescribed. It can also be used to treat malaria. It has few adverse side effects, but it shares one with hydroxychloroquine: QT-segment prolongation. Indeed, the FDA issued a warning in 2013 that azithromycine “can cause abnormal changes in the electrical activity of the heart that may lead to a potentially fatal irregular heart rhythm.” The warning further cautioned that people with certain pre-existing conditions are at particular risk, such as those with QT interval prolongation, low potassium or magnesium levels, a slower than normal heart rate, or those who use certain drugs to treat abnormal heart rhythms.

A number of doctors on Twitter were alarmed at the suggestion that two drugs that can affect heart rhythm be taken together without much stronger evidence that they were effective (more on the study in a moment):

Moreover, President Trump’s claim resulted on a run on the drugs and shortages at pharmacies, with some doctors writing scripts for their family and friends:

And even dentists are getting in on the action:

Elsewhere on Twitter:

Another concern was that it is easy to overdose on hydroxychloroqine, its therapeutic window (the difference between the lowest effective dose and doses that will cause toxicity, in this case cardiac toxicity) being narrow.

But why was it thought that these drugs might be active against SARS-CoV-2? First, it’s been known for a long time that chloroquine has antiviral activity against certain viruses (summarized in this commentary). The evidence, however, has been fairly conflicting, as described here:

The in vitro antiviral activity of chloroquine has been identified since the late 1960’s (Inglot, 1969; Miller and Lenard, 1981; Shimizu et al., 1972) and the growth of many different viruses can be inhibited in cell culture by both chloroquine and hydroxychloroquine, including the SARS coronavirus (Keyaerts et al., 2004). Some evidence for activity in mice has been found for a variety of viruses, including human coronavirus OC43 (Keyaerts et al., 2009), enterovirus EV-A71 (Tan et al., 2018), Zika virus (Li et al., 2017) and influenza A H5N1 (Yan et al., 2013). However, chloroquine did not prevent influenza infection in a randomized, double-blind, placebo-controlled clinical trial (Paton et al., 2011), and had no effect on dengue-infecteds [sic] patient in a randomized controlled trial in Vietnam (Tricou et al., 2010). Chloroquine was also active ex vivo but not in vivo in the case of ebolavirus in mice (Dowall et al., 2015; Falzarano et al., 2015), Nipah (Pallister et al., 2009) and influenza virus (Vigerust and McCullers, 2007) in ferrets.

More recently:

Recently, Wang and colleagues (Wang et al., 2020) evaluated in vitro five FDA-approved drugs and two broad spectrum antivirals against a clinical isolate of SARS-CoV-2. One of their conclusions was that “chloroquine (is) highly effective in the control of 2019-nCoV infection in vitro” and that its “safety track record suggests that it should be assessed in human patients suffering from the novel coronavirus disease”. At least 16 different trials for SARS-CoV-2 already registered in the Chinese Clinical Trial Registry (ChiCTR2000029939, ChiCTR2000029935, ChiCTR2000029899, ChiCTR2000029898, ChiCTR2000029868, ChiCTR2000029837, ChiCTR2000029826, ChiCTR2000029803, ChiCTR2000029762, ChiCTR2000029761, ChiCTR2000029760, ChiCTR2000029741, ChiCTR2000029740, ChiCTR2000029609, ChiCTR2000029559, ChiCTR2000029542) propose to use chloroquine or hydroxychloroquine in the treatment of COVID-19 (“Chinese Clinical Trial Register” (ChiCTR)). In a recent publication (Gao et al., 2020), Gao and colleagues indicate that, “according to the news briefing”, “results from more than 100 patients have demonstrated that chloroquine phosphate is superior to the control treatment in inhibiting the exacerbation of pneumonia, improving lung imaging findings, promoting a virus negative conversion, and shortening the disease course”.

I must say that the paper by Gao et al is really thin gruel. Its methods were not well described, and it simply asserted a positive result. I also note that the half-maximal effective concentration (EC50) for blocking viral replication in vitro was 1.13 μM and the half-cytotoxic concentration (CC50) was greater than 100 μM. These are high concentrations for a drug, especially the CC50. The same paper also found that remdesivir (a new antiviral drug) also had antiviral activity against SARS-CoV-2.

As for azithromycin, well, hang on for its rationale.

The study: Hydroxychloroquine and azithromycin as a treatment of COVID-19

So here is the study by Gautret et al., “Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open- label non-randomized clinical trial“. It was carried out at locations in France and Viet Nam and is a preprint that hasn’t yet gone through peer review (at least not as of yesterday). Interestingly, it was posted on March 17, which means it had been four days before Trump took notice of the study. It is also not a randomized study, and the primary endpoint was detectable presence of the virus, with reverse transcriptase polymerase chain reaction (RT-PCR, as SARS-CoV-2 is an RNA virus) used to determine if a specimen was positive or negative for SARS-CoV-2. Inclusion and exclusion criteria were simple. Patients could be included if they were >12 years old and had PCR-documented SARS-CoV-2 in a nasopharyngeal sample at admission, “whatever their clinical status.” Patients were excluded if they had an allergy to hydroxychloroquine or chloroquine or had “another known contraindication to treatment with the study drug, including retinopathy, G6PD deficiency and QT prolongation. Breastfeeding and pregnant patients were excluded based on their declaration and pregnancy test results when required.” The primary endpoint was viral load of SARS-CoV-2, as documented by qRT-PCR.

After that, there were three groups studied:

  • Patients from the main medical center, who received hydroxychloroquine (n=20).
  • “Control” patients from another center who received standard treatment (n=16).
  • Patients from the main center who received hyroxychloroquine and azithromycin (A) (n=6, but not really, as I will explain).

Before I discuss the problems with the study, I’ll cut to the chase and post a copy of the graph that’s been showing up all over social media, a graph showing the percentage of patients who were positive for the virus over time:

On the surface, it looks very promising, doesn’t it? The percentage of patients PCR-positive for SARS-CoV-2 is lower in the hydroxychloroquine-treated group and drops to zero by day 5. Of course, there are no error bars. Why might that be? Let’s go to the source:

We enrolled 36 out of 42 patients meeting the inclusion criteria in this study that had at least six days of follow-up at the time of the present analysis. A total of 26 patients received hydroxychloroquine and 16 were control patients. Six hydroxychloroquine-treated patients were lost in follow-up during the survey because of early cessation of treatment. Reasons are as follows: three patients were transferred to intensive care unit, including one transferred on day2 post-inclusion who was PCR-positive on day1, one transferred on day3 post-inclusion who was PCR-positive on days1-2 and one transferred on day4 post-inclusion who was PCR- positive on day1 and day3; one patient died on day3 post inclusion and was PCR-negative on day2; one patient decided to leave the hospital on day3 post-inclusion and was PCR-negative on days1-2; finally, one patient stopped the treatment on day3 post-inclusion because of nausea and was PCR-positive on days1-2-3. The results presented here are therefore those of 36 patients (20 hydroxychloroquine-treated patients and 16 control patients). None of the control patients was lost in follow-up.

So basically, an intent-to-treat analysis was not done, and patients who dropped out in the treatment group because they got sicker were excluded from the analysis. This is not how things are done. These patients were obviously sicker and could easily have had higher viral loads. Leaving them out of the final analysis was not justifiable. What is an intent-to-treat analysis and why is it important? Basically, it is a design in which the results of a trial are analyzed based on the initial treatment to which the patient was assigned, not the final actual treatment administered, requiring that subjects be included even if they do not adhere to the protocol. Intent-to-treat is designed to avoid misleading artifacts in the data that can come about from problems such as unequal dropout between groups (which this study definitely had) or crossover. Of course, intent-to-treat makes the analysis more difficult, because it’s not infrequent that there are missing data points, but there are statistical methods to deal with missing data.

Aside from not adhering to an intent-to-treat design, here’s where the study is truly revealed to be crap. This huge lapse was pointed out on Twitter by Gaetan Burgio, an Australian geneticist, who waded into the supplementary data and found this table:

And Rachael Dunlop, who did the same, and posted this to Facebook:

Read the Tweet and the Facebook post, but to make a long story short, researchers didn’t do PCR on a lot of the samples from the controls. They used incomplete data from even their own patients and didn’t do an intent-to-treat analysis and also had very small numbers. It’s also unclear to me what criteria were used to determine who got azithromycin. All the paper says is that it was given to six patients to suppress upper respiratory infections, noting simply that “depending on their clinical presentation, azithromycin was added to the treatment” and that it was added to “prevent bacterial super-infection.”

As for the PCR, I am unclear on how this assay was carried out. Because PCR amplifies nucleotide sequences exponentially, small differences in technique, temperature cycling settings, etc. can make a huge difference. I’ve done a lot of PCR. The primer sequences used are not included, and the only reference to PCR methods is a citation of a paper that hasn’t been published yet. This is basically all the method:

For all patients, 500 μL of the liquid collected from the nasopharyngeal swab were passed through 0.22-μm pore sized centrifugal filter (Merck millipore, Darmstadt, Germany), then were inoculated in wells of 96-well culture microplates, of which 4 wells contained Vero E6 cells (ATCC CRL-1586) in Minimum Essential Medium culture medium with 4% fetal calf serum and 1% glutamine. After centrifigation at 4,000 g, microplates were incubated at 37°C. Plates were observed daily for evidence of cytopathogenic effect. Presumptive detection of virus in supernatant was done using SU5000 SEM (Hitachi) then confirmed by specific RT- PCR.

This is a two-step assay. First, the swab contents were sterilized by filtration and inoculated on Vero E6 cells, which were examined daily for evidence of viral replication (cytopathic effect, meaning that cells were dying because the virus was replicating in them). Then the media was examined under scanning electron microscope for virus particles, only after which was PCR done. So wait a minute. This, of course, leads me to wonder why only PCR results were reported and whether the virus was seen in the media by electron microscope. Was PCR not done on some of the samples because no viral particles were seen by scanning electron microscope? Inquiring minds want to know! In any event, a procedure as tricky as trying to culture virus in cells, isolating RNA from the viral particles in the supernatant, and then doing RT-PCR has a lot of places where things can go wrong and/or where results can vary a lot depending on small differences in technique. Standardization is key, as is blinding of laboratory technicians doing the assays to the group that the patients came from. (Perhaps virologists can elaborate on the assay used to detect SARS-CoV-2 here.)

Worst of all, there is no evidence from this paper that these two drugs made any difference in the clinical outcomes of these patients. I understand that this was a pilot study and that it will require a much larger trial to determine if hydroxychloroquine has any effect on clinical outcomes, but the hype over this study is unconscionable. This is at best an uncertain result from a very preliminary study with very serious methodological shortcomings that looked only at viral positivity in nasopharyngeal aspirates. The authors justify their publication thusly:

For ethical reasons and because our first results are so significant and evident we decide to share our findings with the medical community, given the urgent need for an effective drug against SARS-CoV-2 in the current pandemic context.

We show here that hydroxychloroquine is efficient in clearing viral nasopharyngeal carriage of SARS-CoV-2 in COVID-19 patients in only three to six days, in most patients. A significant difference was observed between hydroxychloroquine-treated patients and controls starting even on day3 post-inclusion.

No, the ethical thing to do would have been to analyze the data properly and go through peer review. A number of journals are accelerating peer review of COVID-19-related papers, so that they can be published rapidly. This paper almost certainly would have failed peer review, given that it is not randomized, doesn’t do a proper intention-to-treat analysis, and has a whole lot of missing datapoints among the control subjects. As Dr. Burgio put it:

Exactly. Now one of the physicians involved, Didier Raoult, MD, PhD is advertising an “open label” trial for the prevention of COVID-19:

We’re an independent group of scientists and physicians working on an open-data clinical trial for prevention of COVID-19, through the use of hydroxychloroquine in combination with other therapeutic agents, for prevention of SARS-COV2 infection in high risk health care workers.

Unlike a typical commercial drug trial, our objective is to share trial data with the public* and health-care professionals as close to real-time as possible (with a reasonable level of data quality assurance).

Given the rapidly spreading coronavirus pandemic, we’re looking for every possible means to fast-track the effort.

It’s led by Gregory J. Rigano, Esq. Yes, you have that right, a lawyer. Dr. Raoult is listed only as one of two consulting physicians and scientists. The other? Dr. Chandra Duggirala, who is described as a “physician, clinical investigator, inventor and serial entrepreneur.” He’s also into antiaging research and has “founded a software company at the intersection of nutritional biology and A.I.” What could go wrong?

What’s depressing is that Dr. Raoult has a very good publication record and is viewed as one of France’s top scientists. He knows how it should be done. Why is he choosing to do it this way? A pandemic is not a good excuse to throw rigorous methodology out the window, particularly when a randomized trial using viral load as the endpoint could be completed very quickly. At least the French government is trying to replicate Dr. Raoult’s trial independent of him; so hopefully we will have some clarity soon. There are also at least three trials of hydroxychloroquine to treat or prevent COVID-19 on ClinicalTrials.gov (NCT04308668, NCT04304053, and NCT04315896).

Real world consequences and the future

Now that it has dawned on everyone that we have a real pandemic that could kill many millions of people worldwide, it’s understandable that people are desperate for effective treatment. We feel helpless, and understandably so, given a case fatality rate almost certainly considerably higher than that of influenza. Also, there is no vaccine (with no likelihood of one for at least a year) and no specific treatment other than supportive care. Even though most people will have mild disease, 20% or so will require hospitalization and 5-6% will wind up on a ventilator, some for extended periods of time, before recovering or dying. We are in a situation where hospitals could soon be overwhelmed with COVID-19 patients, as has already happened in Italy and Spain. The President’s words were deeply irresponsible, and he essentially single-handedly caused shortages of hydroxychloroquine, which is used to treat lupus and other autoimmune diseases. Worse, the dose used in the clinical trial reported was high (600 mg daily, with typical doses for lupus being 200-400 mg daily). If the drug is given to huge numbers of people without proper monitoring, there will be people who get sick or even die from it.

I got a lot of pushback on Twitter for complaining about this study over the weekend, with people asking “what’s the harm?” (death could be the harm in some cases) and asking why China and some other countries have added hydroxychloroquine to their recommended treatments. I couldn’t find the actual clinical trials published, only this announcement stating that the drug is “recommended for inclusion in the next version of the Guidelines for the Prevention, Diagnosis, and Treatment of Pneumonia Caused by COVID-19 issued by the National Health Commission of the People’s Republic of China” and a transcript in Chinese of a press conference held in February announcing the findings. Basically, other than this study and some in vitro studies, I can’t evaluate the evidence at this time. (If someone knows where to find the data, please let me know.)

The abstract of the Chinese expert consensus, translated into English, reads:

Here we found that treating the patients diagnosed as novel coronavirus pneumonia with chloroquine might improve the success rate of treatment, shorten hospital stay and improve patient outcome. In order to guide and regulate the use of chloroquine in patients with novel coronavirus pneumonia, the multicenter collaboration group of Department of Science and Technology of Guangdong Province and Health Commission of Guangdong Province for chloroquine in the treatment of novel coronavirus pneumonia developed this expert consensus after extensive discussion. It recommended chloroquine phosphate tablet, 500mg twice per day for 10 days for patients diagnosed as mild, moderate and severe cases of novel coronavirus pneumonia and without contraindications to chloroquine.

Again, without the evidence base used to come to this recommendation, I have a hard time judging whether the recommendation is scientifically justified. “Might improve the success rate of treatment” doesn’t sound like very definitive results.

Searching Pubmed, I found various articles reviewing the question of whether chloroquine or hydroxychloroquine are effective against COVID-19. This one, published two weeks ago, examined six articles (one narrative letter, one in-vitro study, one editorial, expert consensus paper, two national guideline documents) and 23 ongoing clinical trials in China and concluded that chloroquine “seems to be effective in limiting the replication of SARS-CoV-2 (virus causing COVID-19) in vitro,” going on to conclude in addition:

There is rationale, pre-clinical evidence of effectiveness and evidence of safety from long-time clinical use for other indications to justify clinical research on chloroquine in patients with COVID-19. However, clinical use should either adhere to the Monitored Emergency Use of Unregistered Interventions (MEURI) framework or be ethically approved as a trial as stated by the World Health Organization. Safety data and data from high-quality clinical trials are urgently needed.

Another recent review, by Dr. Raoult, notes that there have been mixed results using chloroquine to treat viral illnesses, despite its broad antiviral activity in cell culture and states, “Preliminary trials of chloroquine repurposing in the treatment of COVID-19 in China have been encouraging, leading to several new trials,” adding:

According to preliminary reports [50,51] from the Chinese authorities suggesting that approximately 100 infected patients treated with chloroquine experienced a more rapid decline in fever and improvement of lung computed tomography (CT) images and required a shorter time to recover compared with control groups, with no obvious serious adverse effects, the Chinese medical advisory board has suggested chloroquine inclusion in the SARS-CoV-2 treatment guidelines.

So, could hydroxychloroquine and chloroquine be effective drugs to treat and/or prevent COVID-19? Sure, it’s possible. There’s a plausible mechanism by which the drugs could inhibit viral replication (several, actually), plus in vitro evidence of antiviral activity. On the other hand, caution is definitely indicated, as Dr. Raoult notes:

The antiviral properties of chloroquine described in vitro have sometimes been confirmed during treatment of virus-infected patients but have not always been reproduced in clinical trials depending on the disease, the concentration of chloroquine used, the duration of treatment and the clinical team in charge of the trial.

The bottom line is that more data are desperately needed. Sadly, the French study doesn’t provide us with the data needed and has muddied the waters more than clarified whether using hydroxychloroquine to treat COVID-19 is an effective intervention. Fortunately, there are multiple trials testing hydroxychloroquine and other drugs with antiviral activity against COVID-19, including a “global megatrial” by The WHO.

Sadly, as the pandemic progresses, we’re going to see a lot more of this, preliminary studies hyped as “game changers.” As editor of SBM, I’m already getting emails from PR flacks touting this “expert” or that’s new treatment for COVID-19. Until there is an effective vaccine or treatment, this will be the new normal, with incremental advances being touted as “game changers” and desperate people wondering why there isn’t a treatment yet. Whenever you see a study like the French study touted, wait a while to see what the experts say about it, and don’t share it before you get a feel for whether it’s a good study or not. Don’t be like President Trump.

ADDENDUM 3/27/2020: I was made aware of this excellent post discussing the trial. It turns out that Didier Raoult’s group played even faster and looser with the data in this study than my post indicates. Worse, Raoult’s group has a documented history of data fabrication, likely due to his tyrannical manner of running the institute and his demand for results, where he publicly humiliates students, postdocs, and researchers who do not produce the results he wants. Indeed, the American Society for Microbiology even banned him from publishing in its journals for a year after data fabrication was detected in one of his group’s papers. Raoult also has a history of accusations of sexual misconduct with underlings, and he is feared in the French scientific community due to his propensity to use his power and influence to silence critics. After I posted this, a small randomized trial from China testing chloroquine to treat COVID-19 was published. It showed no benefit. That doesn’t mean that there might not be a benefit observed in a larger trial, but it sure does suggest that this is no “miracle drug,” because if its effects were so dramatic they should be seen in a small trial.

In terms of other science, Raoult is a climate science denier, having been quoted as saying, “La planète ne se réchauffe plus depuis 1998. Jusqu’à cette date, le réchauffement brutal était lié à un phénomène météorologique : El Niño.” (“The planet has not warmed since 1998. Until that date, the sudden warming was linked to a meteorological phenomenon: El Niño”). He also published a book, La vérité sur les vaccins (The Truth About Vaccines) that has a distinctly antivaccine vibe to it, given the title, the blurb, and what I’ve been able to find out about it. He’s also been quoted as saying:

I have never done randomized trials…The effect of randomized stuff, maybe it works on people who had a myocardial infarction, but putting that in infectious diseases, it doesn’t make sense.

That explains a lot about why his trial design was so incompetent.

Overall, Raoult strikes me as a “brave maverick” who might have been a great scientist in his prime but who has now become arrogant and dictatorial and has now come down with a serious case of the Dunning-Kruger effect. He has no expertise in clinical trials and in fact disputes the usefulness of randomized clinical trials in infectious disease. Anything he publishes on COVID-19 should be taken with a huge grain of salt.

Shares

Author

Posted by David Gorski

Dr. Gorski's full information can be found here, along with information for patients. David H. Gorski, MD, PhD, FACS is a surgical oncologist at the Barbara Ann Karmanos Cancer Institute specializing in breast cancer surgery, where he also serves as the American College of Surgeons Committee on Cancer Liaison Physician as well as an Associate Professor of Surgery and member of the faculty of the Graduate Program in Cancer Biology at Wayne State University. If you are a potential patient and found this page through a Google search, please check out Dr. Gorski's biographical information, disclaimers regarding his writings, and notice to patients here.

All Posts Website