Does Tamiflu have any meaningful effects on the prevention or treatment of influenza? Considering the drug’s been on the market for almost 15 years, and is widely used, you should expect this question has been answered after 15 flu seasons. Answering this question from a science-based perspective requires three steps: Consider prior probability, be systematic in the approach, and get all the data. It’s the third step that’s been (until now) impossible with Tamiflu: Some data was unpublished. In general, there’s good evidence to show that negative studies are less likely to be published than positive studies. Unless unpublished studies are included, systematic reviews are more likely to miss negative data, which means there’s the risk of bias in favor of an intervention.
The absence of a full data set on Tamiflu (oseltamivir) and the other neuraminidase inhibitor Relenza (zanamivir) became a rallying point for BMJ and the AllTrials campaign, which seeks to enhance the transparency and accessibility of clinical trials data by challenging trial investigators to make all evidence freely available. (Reforming and enhancing access to trial data was one of the most essential changes recommended by Ben Goldacre in his book, Bad Pharma.) In 2009, Tamiflu’s manufacturer, Hoffman-La Roche committed to making the Tamiflu data set available to investigators. Now after four years of back-and-forth between BMJ, investigators, and Roche, the full clinical trials data set has been made freely available. An updated systematic review was published today in BMJ (formerly The British Medical Journal), entitled “Oseltamivir for influenza in adults and children: systematic review of clinical study reports and summary of regulatory comments.” This will be a short post covering the highlights. As the entire study and accompanying data are freely available, I’ll await continued discussion in the comments.
What do we already know about influenza and the antiviral drugs?
Influenza is a significant cause of illness and death. While deaths directly attributed to the virus tend to be uncommon, influenza kills through its effects on other diseases, like lung and heart disease, and by causing secondary infections. It doesn’t always discriminate, either – every influenza season we see stories of otherwise healthy adults, pregnant women, and children who died due to influenza. Influenza is an unpredictable killer – seasonal variation and genetic variation make each year’s influenza season slightly different.
It’s been widely acknowledged that we have limited tools to prevent and treat influenza. Beyond the basic infection control processes (e.g., handwashing), there’s the vaccine, and then there’s antiviral drugs. The influenza vaccine can be effective (depending on the antigenic match) but even a well-matched vaccine still has only modest efficacy – especially compared to other vaccines, like smallpox (now eradicated), polio (on its way to being eradicated), and measles (potentially eradicable, but we’re not close yet). For decades, the main drug was amantadine, a drug with little persuasive evidence of efficacy. In the late 1990’s two new antiviral drugs were launched: oseltamivir, an oral capsule marketed as Tamiflu, and zanamivir, an inhaled powder, sold as Relenza. While these drugs were widely marketed when launched, there was very little good evidence to demonstrate that they had meaningful effects in the ambulatory (generally healthy) population that contracted influenza. Evidence suggested that it seemed to only shorten the duration of influenza slightly, though it seemed more useful at preventing its spread. There was also some observational data suggesting a possible mortality benefit. Meta-analysis by groups other than Cochrane suggested some benefit. Guidelines in the 2000’s reinforced the primary role of vaccination and tended to recommend the restriction and use of antiviral products to populations considered more “at-risk”: The hospitalized, those that could not be vaccinated, and those exposed to the virus, particularly in locations like nursing homes, where the risks and consequences of infection were much higher. Despite guidance that was, in some cases, quite limiting, the drugs developed a perception of efficacy, no doubt reinforced by effective marketing, extensive prescribing, and undoubtedly, patient demand.
Despite the modest efficacy of the new antivirals, most governments developed stockpiles of the medications. All public health agencies fear that we may someday see another influenza pandemic like 1919 (which killed millions worldwide), so it’s not surprising that there’s a supply maintained, even when the overall efficacy looked poor. There is evidence for the prevention of spread, and given the unknown virulence of the next virus to circulate, stockpiling is like an insurance policy of unknown value – it might help, and it might not, but if you don’t have any stockpiled, you won’t have anything at all in a pandemic situation. It’s important to reiterate (as we’ll come back to this point) that stockpiles are not for seasonal influenza, they are reserved for pandemics, where infectiousness and virulence may be much greater. Given most of the drug probably expires before it’s ever used, the value-for-money of this strategy isn’t clear and can really only be assessed in retrospect – and in the face of a new pandemic, it will be difficult to assess.
What new information does this analysis provide?
This new paper is written by Tom Jefferson, Mark Jones, Peter Doshi, Elizabeth A Spencer, Igho Onakpoya, and Carl J. Heneghan, several of whom contributed to the previous version of the same review. Some of these names may be familiar to regular readers of this blog. Tom Jefferson is a vocal critic of the influenza vaccine and his strict approach to data been criticized as “methodolatry” by some epidemiologists. Mark Crislip, this blog’s resident infectious disease specialist, agrees with many of Jefferson’s criticisms of the existing evidence but comes to a different conclusion regarding the vaccine’s effectiveness based on a different interpretation of the data, one that’s pretty consistent with the perspective of David Gorski. The other author who holds strong opinions about influenza is Peter Doshi, who is so stridently anti-vaccine he makes Tom Jefferson look pro-vaccine. Doshi believes that influenza itself is an example of “selling sickness”, and questions the morbidity and mortality estimates of the disease. It is perhaps not surprising, then, to see the authors describe influenza as “benign” in the text of this review. It all depends on the population you study. Lest you think I’m poisoning the well before getting into the data, it’s important to keep the authors’ perspectives on the disease itself in mind, when we discuss what the study results are telling us about the “real world” outside the randomized controlled trial.
The data set the authors used is described as complete, and they suggest that every trial of oseltamivir, both unpublished and published, has been included (83 trials). They included randomized controlled trials of oseltamivir that examined prevention, treatment, and post-exposure prophylaxis (another type of prevention). Patients had to be healthy, or have a chronic illness. Trials that studied the immune-compromised were excluded. The authors also excluded all trials for which they could not access unabridged clinical study reports, creating the strictest inclusion criteria I’ve ever seen in a systematic review. Clinical trial reports can run into the thousands of pages and are the medical equivalent of visiting a sausage factory – the final product looks nothing like the raw ingredients. They admit this was a challenge:
The main limitation of our study is our relative inexperience in dealing with large quantities of information and our lack of familiarity with certain trial documents such as blank case report forms. A further limitation of our review is that the methods we have developed to assess and summarise information from clinical study reports may not apply to non-industry trials (which may not be reported in clinical study reports).
Which is a pretty significant limitation, as non-industry-sponsored trials may have been excluded based on this strict criterion. With the new criteria and dataset, new findings and conclusions were drawn:
Treatment
- Tamiflu reduces symptoms in adults by about 2/3 of one day, from 7 days to 6.3 days (P<0.001)
- Tamiflu reduces symptoms in children by over one full day (P=0.001) but not in asthmatic children
- Tamiflu has no effects on rates of admission to hospital when used for treatment or prophylaxis (prevention)
- Tamiflu significantly reduced the risk of “investigator mediated” unverified pneumonia (risk ratio 0.55) (NNT=100)
- There was no reported difference for trials that reported pneumonia differently
- Tamiflu did not appear to influence the likelihood of pneumonia in pediatric patients
Prevention
- Tamiflu reduces the risk of symptomatic influenza in individuals by 55% (NNT=33)
- Tamiflu reduces the risk of symptomatic influenza in households by 80% (NNT=8)
- Tamiflu has no effects on rates of admission to hospital when used for prophylaxis (prevention)
- There was no effect on asymptomatic influenza transmission
Post-exposure prophylaxis
- There was only a single post-exposure trial included in the analysis. The manufacturer concluded the drug was effective; the authors disagree with the trial’s design. No statistics are provided.
Overall, these new findings provide convincing evidence that Tamiflu has a beneficial effect for treating influenza, but the net benefit is small. Moreover, the drug does not seem to prevent complications like hospitalizations. However the drug provides a meaningful benefit in the prevention of influenza, especially in households.
Harms
Like the efficacy data, the harms data are consistent with what’s already known about Tamiflu. When used for treatment of infection, the drug is generally well tolerated with the exception of nausea and vomiting, which are not uncommon (NNH 28 and 22, respectively). However, it reduces the risk of diarrhea (NNT 43) and “cardiac body system events” (NNT 148). There is no evidence to suggest that Tamiflu increase psychiatric side effects while on treatment.
When used for prevention of infection, Tamiflu can cause headache (NNH 32), nausea (NNH 25) and neuropsychiatric adverse effects (NNH 94).
There were insufficient deaths in the dataset to draw conclusions about effects.
What about real-world data?
Accompanying the Jefferson paper is a paper by Freemantle and colleagues, entitled “Oseltamivir: the real world data”, that goes beyond the dataset used in the Jefferson analysis, which was restricted to randomized controlled trials only. The authors note:
- RCTs tend to include healthier patients that may not reflect the population that receives a drug in the real world.
- While observational trials don’t randomize (and can be more easily prone to bias) they allow study in populations where RCTs may be ethically impossible (e.g., pregnancy). Some biases can be addressed in part by statistical techniques.
The authors updated a systematic review that was last completed in 2012 examining oseltamivir’s impact on mortality. The search strategy is well described and 18 studies were included:
- Prevention of death: Three observational studies evaluated the effectiveness of oseltamivir to prevent death. The authors noted design, analysis and reporting issues with the three studies. However all three studies found that oseltamivir reduces mortality, a finding the authors describe as “reasonably consistent evidence which points towards a benefit of oseltamivir in this setting.”
- Effects in pregnancy: Three observational studies have looked at maternal infant outcomes. No harms have been noted.
- Neuropsychiatric effects: Three observational studies examine psychiatric and neurological events. The effects are mixed to positive, with one study finding reduced psychiatric effects with treatment.
The authors conclude that Tamiflu has some overall positive effects but also argue that they’re inconclusive, given the quality issues in the data. They argue, justifiably, for more trials, both prospective and observational, in higher-risk populations, where there continues to be a lack of good-quality evidence. This conclusion is largely consistent with another, older, systematic review of observational studies completed in 2012 by Hsu, which cautiously suggested a net benefit in high-risk patients.
Conclusion
The headlines today may suggest Tamiflu is a scam, and there’s no doubt that the beneficial effects have been overstated, particularly in the general population where the risk of serious complications tend to be low. However, the data are more nuanced than the investigators themselves believe. Jefferson and associates argue their findings provide evidence that stockpiling of this drug should end. One question is how generalizable the current evidence base is to a pandemic situation, and how much we’re prepared to invest (to stockpile) given the limited beneficial effects shown as a treatment. The second question is the role of Tamiflu in the seriously ill, and in those at greater risk of flu-related complications and death. What the news coverage and the authors seem to be ignoring is that Tamiflu is demonstrably and meaningfully effective at preventing the spread of the infection. At the same time, there’s some real world evidence (albeit with limitations) that suggests that oseltamivir may provide a slight survival advantage in the very ill. Tamiflu is also well tolerated and has a generally good side effect profile. Overall, this new study reinforces what’s been argued by science-based medicine for years: Antivirals like Tamiflu don’t replace vaccination. It offers little benefit in the routine treatment of influenza in the otherwise-well individual, but may have a role in those with other medical conditions, or when used to prevent influenza’s spread.