There are two topics about which I know a fair amount. The first is Infectious Disease. I am expert in ID, Board Certified and certified bored, by the ABIM. The other, although to a lesser extent, is SCAMs.
When I read the literature on these topics, I do so with extensive knowledge and, in the case of ID, 30 years of clinical experience. The extensive knowledge, and, one hopes, understanding, has led me to read meta-analyses with a grain of salt substitute. They average meta-analysis and systematic review is good for gaining a general understanding of the topic within, as well as, and here is the key phrase, the limitations of the included studies.
And like all the published literature, when writing a meta-analysis, those with an axe to grind will grind it. Even, or perhaps especially, the Cochrane reviews.
Just because something is labelled as a systematic review does not mean it is any good. We have to be just as vigilant now as ever. Even a review with a Cochrane label does not make its true. Four out of 12 Cochrane reviews on acupuncture were wrong. Caveat lector rules, OK?
Randomized placebo-controlled trials can be definitive if done well. But all too often they are done on a well-defined population and how widely applicable the results are, is always a question. When I was a resident it seemed all the cardiovascular intervention studies were done on a VA population and we wondered how applicable outcomes on old, white, male, smokers were to other populations.
All clinical trials are part of a context of the medical literature and Xigris was a classic example of how a single study may not be definitive or even right. Even in a field as binary as ID, it is not always clear cut what the right intervention is from individual clinical trials and you have to try and synthesize a conclusion from the preponderance of the data.
As I have mentioned before, the theory behind meta-analyses is that if you gather all the cow pies into one big pile you get clinical gold. I think they are often better as an overview on a topic rather than as a tool for coming to a conclusion as to the benefit of a particular intervention, especially since meta-analysis do a poor job of predicting the results of a (perhaps) definitive (ha) clinical trial.
GIGO is perhaps the more common result of a meta-analysis. Reading the Cochrane review of late, I am less certain that their output is gold, but a larger pile of the original material.
Take oseltamivir (aka Tamiflu). Please.
As mentioned, I am an Infectious Disease doctor and this year was the second-worst influenza outbreak of my career. One of the hospitals in our system is a level-one trauma center and we offer ECMO, so we see those who are the most ill from influenza. I will say, on balance, I was glad we have oseltamivir during outbreaks.
How (or perhaps do) ID docs think? Infections are caused by germs. Duh. But not everyone thinks so. Germs come in a wide variety of forms: viruses, bacteria, fungi, parasites and more. I once did a rough count and, if you are splitter, there are about 1,200 common germs I need to know to do my job. See. ID isn’t that hard.
We treat these germs with antibiotics. Antibiotics usually interfere with one biochemical pathway or another to kill or disable the germ. Antiseptics, like alcohol, kill non-specifically. Some antibiotics are cidal (they kill germs) and some are static (they stop the germ from functioning, but once the antibiotics is removed they get back to business as usual). For many infections, but not all, static is good enough. Stop the bug from dividing and the host immune system, if there is one, will have time to catch up and take care of the infection.
Viruses are not alive (the debate as to what is alive can go on in comments), so they can’t be killed. It is part of reason antivirals are just OK as a therapeutic intervention. It would be nice if our drugs against viruses were as good as our drugs against S. pyogenes, eradicating them. But they don’t.
As a clinician what you expect with an antiviral is to take the edge off the illness. If you have genital herpes, you will get fewer attacks with less severity. For HIV it took the use of three antivirals to have a significant effect of the virus and they are still not a cure. And for influenza you would expect a mild amelioration of the disease given that the drugs are usually given late into a process of robust viral replication.
Of course not all viruses are the same and not all infections are the same. Outcomes depend on a complex integration of pathogen life cycle, pathogenicity, the host’s immune function and the ability of the medication to affect the organism. And sometimes, for some patients, the difference between doing well and doing poorly may be due to relatively small shifts in favor of the infection.
Oseltamivir is an OK drug for the treatment of influenza. By the time most patients get to a doctor, get the medication filled and then get steady state with drug, the virus has had several days to run rampant and I would not expect that oseltamivir would, in normal people, do much more than take the edge off. And, as the recent meta-analysis in the BMJ demonstrates, that is what oseltamivir does in relatively normal people in the outpatient setting.
As we have known for a while, oseltamivir shortens illness by about a day. In relatively normal outpatients. It may reduce pneumonia and it modestly reduces the risk of symptomatic flu in individuals and households. And there are side effects. There always are. The results of the meta-analysis confirm what was known. Oseltamivir is a drug of modest efficacy in treating influenza in a mostly normal outpatient population.
The authors did a tremendous amount of work and the basic results are sound. Then the editorializing slips in, ground to the fineness of the atomic axe.
Two of the authors have a strong bias against the treatment and prevention of influenza by the current methodologies. I have discussed Jefferson’s semi-coherent arguments concerning influenza in another post and it is safe to say that Doshi and I are not on the same planet about influenza since he considers that:
For the vast majority, influenza is unpleasant but self-limiting.
It is a characteristic of those who are against the current treatments and prevention of infections to directly, or indirectly, minimize those who suffer and die from infections. They usually use the word ‘only’ in front of morbidity and mortality statistics.
It is certainly true that I see the minority of influenza patients: those trying to die. The 18,000 who died in 2009 in the US of H1N1 were a minority of the 60.8 million cases. As a physician I tend to side with John Donne in my approach to the ill and dying:
No man is an island,
Entire of itself,
Every man is a piece of the continent,
A part of the main.
If a clod be washed away by the sea,
Europe is the less.
As well as if a promontory were.
As well as if a manor of thy friend’s
Or of thine own were:
Any man’s death diminishes me,
Because I am involved in mankind,
And therefore never send to know for whom the bell tolls;
It tolls for thee.
If the patient is sick enough to be hospitalized, prompt therapy with oseltamivir probably decreases the odds of needing ICU care and if you are in the ICU or pregnant, probably decreases your chances of dying. Note “decreases the chance”. The data is not great, but consistent: in hospitalized patients, oseltamivir decreases the chance of death. When thinking about treating populations during a pandemic, having sufficient medication available will be a good thing for those ill enough to require hospitalization.
The problem if you are deciding to stockpile drug for an infection like influenza is virus’s variability. This century we have had influenza with high mortality but low infectivity and low mortality and high infectivity. When, not if, we get a strain of influenza with both high infectivity and mortality, we will probably be thankful that we have oseltamivir and given the data it will probably prevent some from dying of the disease.
Of course, those in public health are screwed no matter what they do. If they prepare and the pandemic does not occur, they have wasted money. And if there is an event that is the ID-equivalent of Katrina, no preparation will ever be enough. The scariest thing in my career was the 2009 H1N1, when in Portland every bed in the ICU and every ventilator we had was being used. If another patient came in with flu and needed ICU support we had nothing to offer them. And, by total luck, we peaked exactly at our surge capacity. And, thanks to the ICU, ECMO and maybe oseltamivir, some people that should have died, did not.
Whether we should stockpile anti-influenza medications, spend the money and time, is an interesting political questions. Of course we never have honest conversations about health care in the US, so let’s make a decision based on a good study with flawed and misleading propaganda in the discussion and conclusion. Sounds about right for a county that got all wiggens about death panels.
However, I do not think the data on the efficacy for treating influenza in relatively healthy outpatient populations during influenza seasons of relatively low mortality really informs decisions as to the appropriateness of stockpiling drugs for a repeat of the pandemic of 1919 (an estimated 675,000 Americans among the dead) or if one of the bird influenzas becomes more contagious.
They allude to this exact issue in their conclusion:
Given that oseltamivir is now recommended as an essential medicine for the treatment of seriously ill patients or those in higher risk groups with pandemic influenza, the issues of mode of action, lack of sizeable benefits, and toxicity are of concern. This is made worse by the record and stated intentions of governments to distribute oseltamivir to healthy people to prevent complications and interrupt transmission on the basis of a published evidence base that has been affected by reporting bias, ghost authorship, and poor methods.
We believe these findings provide reason to question the stockpiling of oseltamivir, its inclusion on the WHO list of essential drugs, and its use in clinical practice as an anti-influenza drug.
Note: seriously ill patients or those in higher risk groups. Those not covered in this meta-analysis. Those who may die and, at least from observational studies, get benefit from oseltamivir. I like how they analyzes one patient population then tries to apply it to another population where the preponderance of data suggests benefit. And if you are dying of flu, those side effects are of a little less importance. I would take a bit of a headache and kidney dysfunction to avoid being buried 6 feet under.
Not that it stopped anyone from reporting the conclusions of the study without a modicum of critical thought. Most of the secondary reporting repeated the no stockpiling message and that oseltamivir therapy wasn’t good. The anti-Tamiflu propaganda component seems to have far exceeded the actual results of the paper.
I would lean towards the stockpile. I like to keep the bell from tolling. But to use the BMJ as evidence against stockpiling for a more virulent pandemic is, to my mind, inappropriate. Like I have often said, a meta-analysis is useful if you can use it to confirm your pre-existing bias and ignored if it does not.
That bias can lead to some really Food Babian discussion, as is evident in the BMJ article. They suggest in the conclusion that there are
the issues of mode of action
for oseltamivir. The paper says
Oseltamivir may have anti-inflammatory properties that make people with influenza-like illness feel better by shortening the duration of symptoms and reducing the occurrence of symptoms
and
Sufficient plasma concentrations of oseltamivir carboxylate from orally administered oseltamivir phosphate may act directly on the host’s endogenous neuraminidase to reduce (or suppress) the immune response. The potential hypothermic or antipyretic effect of free oseltamivir as a central nervous system depressant may also contribute to the apparent reduction of symptoms.
As Mr. Doo says, “Huh?” Both of these paragraphs are not referenced. Residents at my hospitals know I am enamored of the immunomodulatory effects of antibiotics. I had never heard of either of these effects with oseltamivir so I went looking.
In mice, oseltamivir given at 30 to 1,000 times the human dose causes hypothermia. In another mouse model oseltamivir led to a decrease in a variety of pulmonary cytokines. Color me unimpressed, and if you are going to use the mouse model, might as well include all the animal studies that show increased survival with oseltamivir. No wait. Only including statements that support your contention is the Food Babian way.
Oseltamivir is a neuraminidase inhibitor, serving as a competitive inhibitor of the activity of the viral neuraminidase (NA) enzyme upon sialic acid, found on glycoproteins on the surface of normal host cells. By blocking the activity of the enzyme, oseltamivir prevents new viral particles from being released by infected cells.
As a rule when you inhibit viral replication you give the immune system a chance to catch up and you slow down the disease. I can find no data that the immunomodulatory effects of oseltamivir are relevant in human disease at standard dosing and the data in humans and animals all point to the usual mechanism by which an antimicrobial works.
Not only is the press not bothering to read the article with more critical thinking than Food Babe, the Wikipedia editors fell for it hook, line and stinker [sic].
Oseltamivir might induce a low immune response with low levels of pro-inflammatory cytokines. This may reduce symptoms of influenza, but is not related to inhibition of virus replication. There is also a potential temperature lowering effect that can contribute to symptom reduction. The influenza virus specific mechanism of action proposed by the producers does not fit the clinical evidence. Evidence rather suggests a multi-system and central action.
Not. It is amazing how easily and rapidly cow pies get spread on the internet. I suppose a meta-analysis gets halfway around the web before the truth has a chance to get its shoes on.
As best I call tell, these two ideas are more for the purpose of FUD. It is of Food Babian proportions to suggest the primary mode of an antiviral is due to immunomodulatory effects instead of messing with viral replication, considering the nature of the data supporting the idea. All the data suggests benefit by slowing down the virus.
They continue
Oseltamivir relieves symptoms in otherwise healthy children but has no effect on children with asthma who have influenza-like illness, a population that should most benefit from its intake.
Should it? I don’t know. That is why we do studies. I would expect valacyclovir suppression to help with recurrent herpes meningitis. It doesn’t. It makes the disease worse. That doesn’t mean that valacyclovir is ineffective for other manifestations of the disease or in other patient populations.
An explanation for this finding is in the nature of the young asthmatic population, which is well cared for and used to a regular intake of powerful drugs and close follow-up. The incremental benefit of oseltamivir assumption is thus likely to be undetectable in such a population.
On the other hand, once lung inflammation started because influenza I would expect cough and SOB to be hard to control as the asthma kicked in, and inhaled steroids, known to increase the risk of pneumonia and TB, may suppress local inflammation to decrease the local immune response in favor of the infection and prolong illness.
The oseltamivir trials did not detect any influenza related mortality events, a reflection of the benign nature of influenza and influenza-like illness and perhaps trial design.
Or it could have been healthier populations with prior immunity from prior disease exposure and vaccinations. Outpatient antibiotic studies are usually done in patients with little risk for mortality and until the 2009 H1N1 flu seasons have been relatively mild, except to those who died. The mortality from flu leading up to the release of oseltamivir in 1996 were comparatively low. Test a drug during a benign flu season and you will not see mortality.
There was a lot of good information in the article, but the bias and FUD rendered it, like much of the Cochrane influenza reviews, more infomercial than a nuanced discussion of a complicated topic. And while I didn’t go through all the supplementary material, I did not see that they suggested oseltamivir is used in the manufacture of yoga mats. But not for a lack of trying.
There are other goofy Cochrane reviews that leave we wondering about their standards. They evidently operate on the assumption that every intervention, no matter how ridiculous, should be subject to a meta-analysis. When all you have is a hammer, everything is a nail. So we get acupuncture for mumps.
We could not reach any confident conclusions about the efficacy and safety of acupuncture based on one study. More high-quality research is needed.
No it isn’t.
Or Homeopathic Oscillococcinum(®) for preventing and treating influenza and influenza-like illness..
Our findings do not rule out the possibility that Oscillococcinum(®) could have a clinically useful treatment effect [although reality does ed.] but, given the low quality of the eligible studies, the evidence is not compelling.
They think a meta-analysis on magic is going to have compelling results. It is a tough job but someone has to see if magic is effective. But if you want the ultimate in goofy, you have to turn to “Vitamin C for preventing and treating the common cold” which contains what may be the most dumbass statement in the medical literature of 2014. Here goes:
Authors’ conclusions .The failure of vitamin C supplementation to reduce the incidence of colds in the general population indicates that routine vitamin C supplementation is not justified, yet vitamin C may be useful for people exposed to brief periods of severe physical exercise. Regular supplementation trials have shown that vitamin C reduces the duration of colds, but this was not replicated in the few therapeutic trials that have been carried out.
Fine so far. The literature shows vitamin C is of minimal benefit, and probably none, for cold treatment and prevention. As the main conclusions say
Thirty one comparisons examined the effect of regular vitamin C on common cold duration (9745 episodes). In adults the duration of colds was reduced by 8% (3% to 12%) and in children by 14% (7% to 21%). In children, 1 to 2 g/day vitamin C shortened colds by 18%. The severity of colds was also reduced by regular vitamin C administration.
Seven comparisons examined the effect of therapeutic vitamin C (3249 episodes). No consistent effect of vitamin C was seen on the duration or severity of colds in the therapeutic trials.
Here it comes. Ready? It is really muttonheaded. Do not say I did not warn you, because, as my favorite box of blinking lights likes to say, it has napalm levels of burning stupid.
Nevertheless, given the consistent effect of vitamin C on the duration and severity of colds in the regular supplementation studies, and the low cost and safety,
Here it comes, the apex of idiocy.
it may be worthwhile for common cold patients to test on an individual basis whether therapeutic vitamin C is beneficial for them.
A more pathetic example of what’s-the-harm shrugery I cannot imagine.
As if there is any way a patient can really know if vitamin C does anything. Cochrane, please, give me a framework I can suggest to my patients that they can apply to help determine if vitamin C is effective. I am waiting. Godot will arrive sooner.
The uselessness of personal experience in determining efficacy of medical interventions is why we do clinical trials. For crying out loud, I though it the raison d’être of the whole Cochrane Collaborative: relying on evidence instead of anecdotes. Wrong.
The biases alone render such an endeavor useless. It is why the words “In my experience” are the most dangerous in medicine for determining if an intervention is effective. And it is what they suggest. Why even bother with the analysis in the first place?
If there is a worse statement in all the published evidenced-based medicine, let me know.
There was a time when I was under the impression that the conclusions of a Cochrane review were of value. No more. Now I see one and a roll my eyes. What kind of Food Babian nonsense is this one going to have?