I guess I will be spending the rest the flu season writing about the nonsense that is promulgated about the flu vaccine and the disease. One of the more common laments about the flu vaccine is that it doesn’t work: I got the flu vaccine and still got the flu. Well maybe. Maybe not. It takes a few weeks to get protection, so the flu could have developed before the antibody response to the vaccine. The vaccine does not protect to the numerous other viral infections that circulate each winter, so perhaps you had an adenovirus but thought it was the flu. Then there is the evidence. Some readers of the blog are worried that the literature does not support the use of the vaccine.
My research for good studies on the efficasy [sic] of seasonal flu vaccines so far has left me wondering if I’ve somehow missed the good research. Tom Jefferson of the Cochrane Institute says that Most studies are of poor methodological quality and the impact of confounders is high. I agree. Please would you refer me to some of the best studies on the efficasy [sic] of seasonal flu vaccines. After a critical appraisal of the best studies you know of I’d like to submit the same for publication in the interest of science.
Why some readers think I am a research librarian, I do not know. It is not an uncommon request. As an aside, I have a full time job and a family to raise. Don’t be asking me to do your grunt work. It’s called Pubmed. Use it.
But the topic for this post concerns the efficacy of the flu vaccine. I am limiting myself to the use of the vaccine in adults.
Why the influenza vaccine kinda sucks
The flu vaccine is not out best vaccine for at least three reasons:
First, every year they have to make an educated guess which influenza strains will be circulating 9 months in the future. The better the guess, the better the protection the vaccine should provide. Some years they choose better than others. But often the match between the vaccine and the disease is not optimal, so vaccine efficacy can be decreased. The vaccine works best when there is a good antigentic match between the vaccine and circulating strain of influenza.
Second, response to the vaccine is not 100%. The older and more immunoincompetent are the least likely to develop a good antibody response to the vaccine. In a bit of medical irony, the more likely a patient is to need protection from the vaccine, the less likely they are to get a protective antibody response from the vaccine.
Third, vaccination rates are often suboptimal to get benefit in populations, i.e. herd immunity. The elderly will more likely benefit if they are not exposed to influenza at all rather than relying of vaccine mediated protection. It may be more important if those around them, say their health care provider or family, receives the vaccine and as a result does not pass flu on to more vulnerable people. But we rarely (never, ever, never) get vaccination rates at levels for herd immunity to kick in.
So it’s a suboptimal vaccine. And that’s a problem. One, because it will make it more difficult to prove efficacy in clinical studies and two, there is a sub group of anti vaccine goofs who seem to require that vaccines either be perfect, with 100% efficacy and 100% safe, or they are not worth taking.
The influenza vaccine is not 100% efficacious in preventing disease, but it is as close to 100% safe, and much safer than the disease.
Influenza vaccine studies
There are multiple kinds of studies one could do to prove efficacy, each with their own problems.
One kind of study would be direct inoculation of flu into volunteers who received either the vaccine or placebo. This is the simplest study to do (if you can find volunteers to get the flu) and lends biologic plausibility to the vaccine, but doesn’t reflect the real world.
Another study would be to to give vaccine or placebo to populations and see who develops influenza. These are not easy studies to do well. Do you diagnose flu as a flu-like illness, i.e. on clinical grounds, or do you do cultures or PCR or both to prove the diagnosis? Outside of prison or the military, it is hard to capture people through an entire flu season to get the appropriate specimens.
Or do you use other endpoints? Death? Antibiotic use? Hospitalization? Perhaps the flu will not be prevented but attenuated, so it is less likely to spread, and overall the secondary complications of influenza will decrease in a population.
Influenza and pneumonia are both associated with acute myocardial infarction. “Many observational studies in different settings with a range of methods reported consistent associations between influenza and acute myocardial infarction.” (PubMed) Influenza is a risk for developing acute bacterial pneumonia. Prevent influenza (PubMed) and you would prevent both primary and secondary myocardial infarctions.
The best proof of concept of this effect, where vaccination of one population leads to a decrease in disease in another population, has been with the conjugated pneumococcal vaccine used in children, the Prevnar. This vaccine covers the seven most common strains of invasive pneumococcus in children and uptake of this vaccine is high. Use of the vaccine has resulted in invasive disease plummeting in children, and as an side benefit, invasive disease with the same strains has also plummeted in the elderly (PubMed) .
Such an effect may be occurring when children are vaccinated against influenza. “Vaccination of approximately 20-25% of children, 1.5-18 years of age in the intervention communities resulted in an indirect protection of 8-18% against MAARI (medically attended acute respiratory illness in adults > or =35 years of age.” (PubMed)(PubMed).
In the end, almost all clinical studies have limitations. For a (mostly) good review of the problematic nature of the studies to date, read “Influenza vaccination: policy versus evidence.” I say mostly good as the conclusion from this paper is that the data is suboptimal and doesn’t support widespread use of the vaccine. I conclude, as shall be seen, that while the data may be suboptimal, it does support the use of the influenza vaccine. And, of course, I am the right one.
Sometimes you get unlucky doing clinical trials: plan a big placebo controlled trial with the influenza vaccine and that year you have both a vaccine mismatch and a mild flu season, and you end up with no good data (PubMed).
You have make do with the tools at your disposal. There are 1,207 references on pubmed using “‘influenza’ ‘vaccine’ ‘efficacy'” as search criteria. That’s a lot of papers, that show varying effects. Like much of medicine, some studies are better and some are worse and they have variable outcomes.
Direct inoculation
This type of study is the most compelling for proof of concept, but the least “real world”. Take a population of (one hopes) volunteers and give them either the vaccine or placebo, then squirt them in the nose with influenza. This has advantages and disadvantages, but at least you know exactly when there was an infection, with what strain, and you can measure antibody levels to see if the patients had what should be protective antibodies. Best of all, you can control the strain of influenza to get a best antigenic match between the vaccine and the pathogen. In the real world none of this is possible. Unless you are fool enough to walk into the room of a known influenza patient without a mask and inhale deeply.
There have been a variety direct infection studies with flu vaccine over the years:
Protection rate against artificial challenge with influenza A was 96% when vaccine and challenge viruses were homotypic. When the vaccine strain and challenges virus were heterotypic, protection ranged from 70-100%. Protection rate from infection during a homotypic epidemic was, retrospectively, 95%; while 50-87% protection from influenza illness was achieved during a heterotypic epidemic. In all instances, vaccinees experienced mild, mostly afebrile upper respiratory symptoms, unlike controls who had moderate to severe symptoms, often with fever. Infecting virus was shed more often by unvaccinated controls (PubMed).
Similar results are found for the live attenuated virus vaccine:
The efficacy of live attenuated cold-adapted (ca) reassortant influenza A H3N2 and H1N1 virus vaccines against experimental challenge with homologous wild-type virus 7 months after vaccination was compared with that of licensed inactivated virus vaccine in 106 seronegative (hemagglutination-inhibiting antibody titer less than or equal to 1:8) college students. The live attenuated virus vaccines induced as much resistance against illness as did the inactivated vaccine. Vaccine efficacy, measured by reduction in febrile or systemic illness in vaccines, compared with that in controls was 100% for ca H3N2 vaccine, 84% for inactivated H3N2 vaccine, 79% for ca H1N1 vaccine, and 67% for inactivated H1N1 vaccine. Less protection was conferred against upper respiratory tract illness; there was 50 and 77% protection in ca and inactivated H3N2 vaccines, respectively, but there was no protection in ca or inactivated H1N1 vaccinees. The duration, but not the magnitude, of H1N1 wild-type virus shedding in both ca and inactivated vaccinees was significantly reduced compared with controls. In contrast, a significant reduction in the duration and magnitude of H3N2 virus shedding was observed in ca vaccinees but not in inactivated vaccines. After wild-type virus challenge, live ca virus vaccinees demonstrated resistance at least as great 7 months postvaccination as did inactivated virus vaccinees (PubMed).
I find these types of studies compelling. They also make me optimistic for the H1N1 vaccine in that the match between the virus and the vaccine is excellent and we should get high levels of protection from the vaccine in those that can mount an antibody response.
These are not real world examples of course. In the real world the vaccine may not match perfectly, the vaccine may not be handled and administered correctly and the patient, for innumerable co-morbid conditions, may not be able to respond to the vaccine.
However, I am still, barely, young and no medical problems yet, and a health care worker who has daily contact with the old and infirm. The data above is enough for me to take the vaccine, if for no other reason that it may prevent me from passing it on to my patients.
Specific groups
This is the most most difficult study to do for the reasons mentioned above.
What group of people are in the study? What are the endpoints and how are they defined? And, for the study period, how good a match is the vaccine for the circulated strains of flu? All these and more make the studies for flu efficacy at best difficult to do.
NEJM this month had a nice study on the relative efficacy of killed vs. attenuated vaccine in a healthy adult population that was allowed to get seasonal flu naturally. The end point was culture or positive PCR:
A total of 1952 subjects were enrolled and received study vaccines in the fall of 2007. Influenza activity occurred from January through April 2008, with the circulation of influenza types A (H3N2) (about 90%) and B (about 9%). Absolute efficacy against both types of influenza, as measured by isolating the virus in culture, identifying it on real-time polymerase-chain-reaction assay, or both, was 68% (95% confidence interval [CI], 46 to 81) for the inactivated vaccine and 36% (95% CI, 0 to 59) for the live attenuated vaccine. In terms of relative efficacy, there was a 50% (95% CI, 20 to 69) reduction in laboratory-confirmed influenza among subjects who received inactivated vaccine as compared with those given live attenuated vaccine. The absolute efficacy against the influenza A virus was 72% (95% CI, 49 to 84) for the inactivated vaccine and 29% (95% CI, -14 to 55) for the live attenuated vaccine, with a relative efficacy of 60% (95% CI, 33 to 77) for the inactivated vaccine.
Perfect? No. 72% protection from the inactivated vaccine. Not bad. Other studies have demonstrated the the live attenuated vaccine efficacy rates of 50% or better. In the elderly, for example:
[A] randomized, double-blind, placebo-controlled study investigated the efficacy, safety, and immunogenicity of LAIV in community-dwelling ambulatory adults >/=60 years of age in South Africa in 2001. Nose and throat swabs were obtained for influenza virus culture based on the symptoms of influenza-like illness. A total of 3242 subjects were enrolled, with a mean age of 69.5 years. The efficacy of LAIV against influenza viruses antigenically similar to the vaccine was 42.3% (95% CI, 21.6-57.8%). Efficacy against A/H3N2 viruses was 52.5% (95% CI, 32.1-67.2%); vaccine efficacy was not observed against antigenically similar B strains (PubMed).
Of course, the weapon of choice in these debates are the meta analysis. In institutionalized elderly, the vaccine, when given to staff and patients, did not prevent influenza but did decrease pneumonia and all cause mortality (PubMed)(PubMed). “Staff vaccination had a significant effect on influenza-like illness (vaccine effectiveness [VE] 86%, 95% CI 40-97%) only when patients were vaccinated too.”
So, as a health care worker, you could conclude that a) vaccine doesn’t work it doesn’t prevent influenza. I don’t need it. Or b) the vaccine benefits my patients by preventing secondary complications and decreases their risk for death.
How about the general population? The most recent Cochrane meta analysis of the vaccine efficacy in 66,248 people (almost cardiology-levels of patient involvement) from 2004 (PubMed),
Inactivated parenteral vaccines were 30% effective (95% CI 17% to 41%) against influenza-like illness, and 80% (95% CI 56% to 91%) efficacious against influenza when the vaccine matched the circulating strain and circulation was high, but decreased to 50% (95% CI 27% to 65%) when it did not. Excluding the studies of the 1968 to 1969 pandemic, effectiveness was 15% (95% CI 9% to 22%) and efficacy was 73% (95% CI 53% to 84%). Vaccination had a modest effect on time off work, but there was insufficient evidence to draw conclusions on hospital admissions or complication rates. Inactivated vaccines caused local tenderness and soreness and erythema. Spray vaccines had more modest performance. Monovalent whole-virion vaccines matching circulating viruses had high efficacy (VE 93%, 95% CI 69% to 98%) and effectiveness (VE 66%, 95% CI 51% to 77%) against the 1968 to 1969 pandemic.
AUTHORS’ CONCLUSIONS: Influenza vaccines are effective in reducing cases of influenza, especially when the content predicts accurately circulating types and circulation is high. However, they are less effective in reducing cases of influenza-like illness and have a modest impact on working days lost. There is insufficient evidence to assess their impact on complications.
Again, those qualifiers. But the overall meta-analysis suggests the vaccine works for prevention of influenza, and the conclusion does bode well for vaccination against H1N1, which has matches the current strain and is in high circulation.
I will say as an aside that the earlier Cochrane review suggested that influenza cases were decreased by 6% from the vaccine. One commenter over at my Medscape blog, drdan23, suggested that a 6% decrease in disease was not worth it. 30,000 direct and indirect deaths from influenza in the US, maybe 500,000 worldwide. 6% of 30,000 is 1800. 6% of 500,000 is 30,000. Only 6% decrease in deaths? I was always wondering who would be sitting on those death panels that Palin was talking about. It’s the anti-flu-vaccine docs.
Since 2004, clinical trials testing the efficacy of the influenza vaccine have been drifting in:
A total of 2058 persons were vaccinated in October and November 2005. Studywide influenza activity was prolonged but of low intensity; type A (H3N2) virus was circulating, which was antigenically similar to the vaccine strain. The absolute efficacy of the inactivated vaccine was 16% (95% confidence interval [CI], -171% to 70%) for the virus identification end point (virus isolation in cell culture or identification through polymerase chain reaction) and 54% (95% CI, 4%-77%) for the primary end point (virus isolation or increase in serum antibody titer). The absolute efficacies of the live attenuated vaccine for these end points were 8% (95% CI, -194% to 67%) and 43% (95% CI, -15% to 71%), respectively.
CONCLUSIONS: With serologic end points included, efficacy was demonstrated for the inactivated vaccine in a year with low influenza attack rates (PubMed).
Or this:
We carried out a randomized, double-blind, placebo-controlled trial of inactivated and live attenuated influenza vaccines in healthy adults during the 2004-2005 influenza season and estimated both absolute and relative efficacies.
RESULTS: A total of 1247 persons were vaccinated between October and December 2004. Influenza activity in Michigan began in January 2005 with the circulation of an antigenically drifted type A (H3N2) virus, the A/California/07/2004-like strain, and of type B viruses from two lineages. The absolute efficacy of the inactivated vaccine against both types of virus was 77% (95% confidence interval [CI], 37 to 92) as measured by isolating the virus in cell culture, 75% (95% CI, 42 to 90) as measured by either isolating the virus in cell culture or identifying it through real-time polymerase chain reaction, and 67% (95% CI, 16 to 87) as measured by either isolating the virus or observing a rise in the serum antibody titer. The absolute efficacies of the live attenuated vaccine were 57% (95% CI, -3 to 82), 48% (95% CI, -7 to 74), and 30% (95% CI, -57 to 67), respectively. The difference in efficacy between the two vaccines appeared to be related mainly to reduced protection of the live attenuated vaccine against type B viruses.
CONCLUSIONS: In the 2004-2005 season, in which most circulating viruses were dissimilar to those included in the vaccine, the inactivated vaccine was efficacious in preventing laboratory-confirmed symptomatic illnesses from influenza in healthy adults. The live attenuated vaccine also prevented influenza illnesses but was less efficacious. (PubMed).
Or this:
INTRODUCTION: Influenza vaccine has been shown to be highly effective in temperate regions with well-defined seasonal influenza. Healthcare workers (HCWs) are advised to receive regular influenza vaccination to protect themselves and their patients. However, there are limited data on the efficacy of influenza vaccine in HCWs in the tropics.
MATERIALS AND METHODS: In this observational, investigator blinded cohort study, bi-monthly questionnaires recording influenza-like illness (ILI) episodes and medical leave were administered to 541 HCWs at the Singapore National University Hospital and KK Women’s and Children’s Hospital from 2004 to 2005. ILI was defined according to a standard symptom score.
RESULTS: Baseline characteristics were comparable in both the vaccinated and non-vaccinated groups. Overall, the relative risk of self-reported ILI in vaccinated HCWs was 1.13 [95% confidence interval (CI), 0.98-1.13; P=0.107]; medical leave taken was lower in the vaccinated group [mean 0.26+/-0.6 days per visit, compared with 0.30+/-0.5 days in the non-vaccinated group (P=0.40)]. Because of the reported Northern Hemisphere 2003/04 vaccine mismatch, we stratified the cohort and determined that the group which received a matched vaccine had a relative risk of ILI of 0.49 (95% CI, 0.37-0.66; P<0.001), achieving a vaccine efficacy of 51%. Mean medical leave decreased significantly in HCWs who received the matched vaccine, compared with those who did not receive vaccination (0.13+/-0.3 vs 0.30+/-0.5; P<0.001) and with HCWs vaccinated with mismatched strains (0.13+/-0.3 vs 0.39+/-0.9; P=0.01).
CONCLUSIONS: A well-matched influenza vaccine is effective in preventing ILI and reducing sickness absence in healthcare workers in tropical settings. Efforts need to be made to increase influenza vaccination rates and to improve the currently available vaccines (PubMed).
Or this:
BACKGROUND: We assessed the effects of an influenza season on patients with COPD. Data from 2,215 veterans in a multicenter, randomized, double-blind influenza vaccine efficacy study were analyzed for changes in spirometric and functional status, comparing patients with laboratory-documented influenza (LDI)-caused illness, non-LDI-caused respiratory illness, or no illness, and for association with influenza vaccination.
METHODS: Patients received either IM trivalent inactivated influenza virus vaccine (TIV) plus intranasal trivalent, live attenuated, cold-adapted influenza virus vaccine (TC) or TIV plus intranasal placebo (TP). We performed spirometry, measured the chronic lung disease severity index (CLDSI) score to assess functional status and well-being, and tested for influenza virus infection.
RESULTS: Worsening in FEV(1), percentage of predicted FEV(1), and CLDSI score (p < 0.001) was associated with acute respiratory illness in 585 illnesses including 94 LDI-caused illnesses. LDI-caused illness was more likely to be associated with worsening in FEV(1) and CLDSI score acutely than non-LDI-caused illness (p < 0.01). Logistic regression showed acute respiratory illness (odds ratio [OR], 1.78; 95% confidence limit [CL], 1.40 to 2.26) to be associated with worsening in CLDSI score, and receipt of TC (OR, 1.39; 95% CL, 1.10 to 1.74) and no illness (OR, 0.70; 95% CL, 0.53 to 0.91 for acute respiratory illness) to be associated with better CLDSI score at the end of the study. Hospitalization was more frequent in patients with acute respiratory illness (p < 0.0001).
CONCLUSIONS: Acute respiratory illness was associated with increased health-care utilization and obstruction to airflow, and worse functional status and well-being. At the end of the study, receipt of TC was associated with improvement and acute respiratory illness was associated with worsening in functional status and well-being (PubMed).
But wait. There’s more. If you call now, not only will you get the above studies, we will add, at no additional cost:
We identified six studies that assessed the incidence of influenza in vaccinated HIV-infected subjects. Four of these studies compared the incidence in vaccinated versus unvaccinated subjects. These involved a total of 646 HIV-infected subjects. In all the 4 studies, the incidence of influenza was lower in the vaccinated compared to unvaccinated subjects with RD ranging from -0.48 (95% CI: -0.63, -0.34) to -0.15 (95% CI: -0.25, 0.05); between 3 and 7 people would need to be vaccinated to prevent one case of influenza. Vaccine effectiveness ranged from 27% to 78%. A random effects model was used to obtain a summary RD of -0.27 (95%CI: -0.42, -0.11). There was no evidence of publication bias. CONCLUSION: Current evidence, though limited, suggests that influenza vaccines are moderately effective in reducing the incidence of influenza in HIV-infected individuals. With the threat of a global influenza pandemic, there is an urgent need to evaluate the effectiveness of influenza vaccines in trials with a larger number of representative HIV-infected persons (PubMed).
Now it may be file drawer effect and negative studies are not being published. The studies show benefits from the vaccine; the benefit is variable depending on the circulating strain, the vaccine match and the population vaccinated. But a benefit none the less.
Population studies
The most compelling population data comes from Ontario, Canada, where they have had a ongoing attempt to maximize the vaccination of the whole population against influenza (PubMed). The other provinces did not see fit to try and vaccinate everyone, continuing with targeted influenza vaccination.
This represents an interesting natural experiment. If the effects of the influenza vaccine are less in preventing disease but more in decreasing secondary endpoints like death, hospitalizations, or antibiotic usage, it may show up in population studies. There are numerous issues with this kind of study, but are “appropriate for assessing the public health impact of a population-wide intervention.”
During the period in the reference, Ontario experienced greater uptake of vaccine than any other Province:
Between the pre-UIIP 1996–1997 estimate to the mean post-UIIP vaccination rate, influenza vaccination rates for the household population aged ≥12 y increased 20 percentage points (18%–38%) for Ontario, compared to 11 percentage points (13%–24%) for other provinces (p < 0.001) (Table 2). For those <65 y, the vaccination rate increases were greater in Ontario than in other provinces, while for those ≥75 y, the increase was smaller in Ontario. For all age groups, Ontario always achieved higher vaccination rates than other provinces.”
And the results of all that vaccination:
“After UIIP introduction, influenza-associated mortality for the overall population decreased 74% in Ontario (RR = 0.26, 95% confidence interval [CI], 0.20–0.34) compared to 57% in other provinces (RR = 0.43, 95% CI, 0.37–0.50) (ratio of RRs = 0.61, p = 0.002) (Table 3). In age-specific analyses, larger mortality decreases in Ontario were found to be statistically significant only in those ≥85 y.
Not bad.
Overall, influenza-associated health care use decreased more in Ontario than other provinces for hospitalizations (RR = 0.25 versus 0.44, ratio of RRs = 0.58, p < 0.001), ED use (RR = 0.31 versus 0.70, ratio of RRs = 0.45, p < 0.001), and doctors’ office visits (RR = 0.21 versus 0.53, ratio of RRs = 0.41, p < 0.001). In age-specific analyses, greater decreases were consistently observed in Ontario than other provinces for age groups <65 y. For seniors, greater decreases were observed in Ontario than other provinces for hospitalizations among those aged 65–84 y and for ED use among those 65–74 y.
A subsequent paper (PubMed) demonstrated that “universal influenza immunization is associated with reduced influenza-associated antibiotic prescriptions.” So it may be the secondary effects of the vaccines that are more important than primarily preventing me from getting flu.
It’s the old rising tide lifts boats effect.
The study that needs to be done would be vaccinate everyone west of the Mississippi and no one to the east. Prevent travel between the two parts of the county. See who gets influenza. Now that would be an epidemiological study.
The study is in contrast to the Tom Jefferson paper that several, evidently anti vaccine folks, have sent me where it says:
A meta analysis of inactivated vaccines in elderly people showed a gradient from no effect against influenza or influenza-like illness to a large effect (up to 60%) in preventing all-cause mortality. These findings are both counterintuitive and implausible, as other causes of death are far more prevalent in elderly people even in the winter months. It is impossible for a vaccine that does not prevent influenza to prevent its complications, including admission to hospital.”
Again with the binary approach to the vaccine. A milder case of flu due to the vaccine will lead to fewer complications and deaths. It is only impossible if you do not recognize that the influenza vaccine, unfortunately, is not like the tetanus vaccine, but its effects are more likely a continuum due to partial effects on the person getting the vaccine and the hard to measure benefits of being less likely to spread the disease.
In my mind that is the true benefit of the influenza vaccines: decreasing the morbidity and mortality of populations. The benefit for populations is derived through vaccinating individuals. That requires a bit of altruism on the part of those receiving the vaccine, as they may be getting vaccinated more for the benefit of others than for themselves. However, at least in the US, a premium is currently placed on being a self centered narcissist; indirectly helping others, even for MDs and RNs, is apparently not on the to do list.
Do flu vaccines work?
Do flu vaccines work? It depends on what the meaning of is is. If you are simplistic and like binary answers, yes or no, then you can pick yes or pick no, and find studies to support your contention that the vaccine doesn’t work.
If you realize that medicine is subtle and nuanced, and often the answers are filled with qualifiers and uncertainty, that the practice of medicine is messy, I think the answer is that the flu vaccine is of benefit. And that the more people who get the vaccine, the greater the benefit for everyone. You do not know how much it pains me to quote Donald Rumsfeld, but he was partly right when he said “You go to war with the army you have, not the army you might want or wish to have at a later time.”
It is true in medicine as well. My army is the vaccine and the data used to support it. You can conclude that neither the vaccine nor the data is perfect, and decide the vaccine is not useful.
Or you can look at the preponderance of data, with all the flaws, nuance, subtleties and qualifiers, and conclude the flu vaccine is of benefit. The vaccine decreases the probability of morbidity and mortality. It is a good thing.