Publications

The case test-negative design for studies of the effectiveness of influenza vaccine in inpatient settings, Ivo M. Foppa, Jill M. Ferdinands, Sandra S. Chaves, Michael J. Haber, Sue B. Reynolds, Brendan Flannery and Alicia M. Fry, International Journal of Epidemiology, 45(6), 1 December 2016, 2052–2059. [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]Background: The test-negative design (TND) to evaluate influenza vaccine effectiveness is based on patients seeking care for acute respiratory infection, with those who test positive for influenza as cases and the test-negatives serving as controls. This design has not been validated for the inpatient setting where selection bias might be different from an outpatient setting. Methods: We derived mathematical expressions for vaccine effectiveness (VE) against laboratory-confirmed influenza hospitalizations and used numerical simulations to verify theoretical results exploring expected biases under various scenarios. We explored meaningful interpretations of VE estimates from inpatient TND studies. Results: VE estimates from inpatient TND studies capture the vaccine-mediated protec- tion of the source population against laboratory-confirmed influenza hospitalizations. If vaccination does not modify disease severity, these estimates are equivalent to VE against influenza virus infection. If chronic cardiopulmonary individuals are enrolled because of non-infectious exacerbation, biased VE estimates (too high) will result. If chronic cardiopulmonary disease status is adjusted for accurately, the VE estimates will be unbiased. If chronic cardiopulmonary illness cannot be adequately be characterized, excluding these individuals may provide unbiased VE estimates. Conclusions: The inpatient TND offers logistic advantages and can provide valid esti- mates of influenza VE. If highly vaccinated patients with respiratory exacerbation of chronic cardiopulmonary conditions are eligible for study inclusion, biased VE esti- mates will result unless this group is well characterized and the analysis can adequately adjust for it. Otherwise, such groups of subjects should be excluded from the analysis. [/creativ_toggle]

A comparison of the test-negative and the traditional case-control study designs for estimation of influenza vaccine effectiveness under nonrandom vaccination, Meng Shi, Qian An, Kylie E. C. Ainslie, Michael Haber and Walter A. Orenstein, BMC Infectious Diseases, 17(1), 8 December 2017, 757. [Manuscript] [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]Background: As annual influenza vaccination is recommended for all U.S. persons aged 6 months or older, it is unethical to conduct randomized clinical trials to estimate influenza vaccine effectiveness (VE). Observational studies are being increasingly used to estimate VE. We developed a probability model for comparing the bias and the precision of VE estimates from two case-control designs: the traditional case-control (TCC) design and the test-negative (TN) design. In both study designs, acute respiratory illness (ARI) patients seeking medical care testing positive for influenza infection are considered cases. In the TN design, ARI patients seeking medical care who test negative serve as controls, while in the TCC design, controls are randomly selected individuals from the community who did not contract an ARI. Methods: Our model assigns each study participant a covariate corresponding to the person’s health status. The probabilities of vaccination and of contracting influenza and non-influenza ARI depend on health status. Hence, our model allows non-random vaccination and confounding. In addition, the probability of seeking care for ARI may depend on vaccination and health status. We consider two outcomes of interest: symptomatic influenza (SI) and medically-attended influenza (MAI). Results: If vaccination does not affect the probability of non-influenza ARI, then VE estimates from TN studies usually have smaller bias than estimates from TCC studies. We also found that if vaccinated influenza ARI patients are less likely to seek medical care than unvaccinated patients because the vaccine reduces symptoms’ severity, then estimates of VE from both types of studies may be severely biased when the outcome of interest is SI. The bias is not present when the outcome of interest is MAI. Conclusions: The TN design produces valid estimates of VE if (a) vaccination does not affect the probabilities of non-influenza ARI and of seeking care against influenza ARI, and (b) the confounding effects resulting from non-random vaccination are similar for influenza and non-influenza ARI. Since the bias of VE estimates depends on the outcome against which the vaccine is supposed to protect, it is important to specify the outcome of interest when evaluating the bias.[/creativ_toggle]

Maximum likelihood estimation of influenza vaccine effectiveness against transmission from the household and from the community, Kylie E. C. Ainslie, Michael J. Haber, Ryan E. Malosh, Joshua G. Petrie and Arnold S. Monto, Statistics in Medicine,  37(6), 15 March 2018, 970-982 [Manuscript] [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]Influenza vaccination is recommended as the best way to protect against influenza infection and illness. Due to seasonal changes in influenza virus types and subtypes, a new vaccine must be produced, and vaccine effectiveness (VE) must be estimated, annually. Since 2010, influenza vaccination has been recommended universally in the United States, making randomized clinical trials unethical. Recent studies have used a monitored household cohort study design to determine separate VE estimates against influenza transmission from the household and community. We developed a probability model and accompanying maximum likelihood procedure to estimate vaccine-related protection against transmission of influenza from the household and the community. Using agent-based stochastic simulations, we validated that we can obtain maximum likelihood estimates of transmission parameters and VE close to their true values. Sensitivity analyses to examine the effect of deviations from our assumptions were conducted. We used our method to estimate transmission parameters and VE from data from a monitored household study in Michigan during the 2012-2013 influenza season and were able to detect a significant protective effect of influenza vaccination against community-acquired transmission.[/creativ_toggle]

On the bias of estimates of influenza vaccine effectiveness from test–negative studies, Kylie E.C. Ainslie, Meng Shi, Michael Haber and Walter A. Orenstein, Vaccine, 35(52), 19 December 2017, 7297-7301 [Manuscript] [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]Estimates of the effectiveness of influenza vaccines are commonly obtained from a test-negative design (TND) study, where cases and controls are patients seeking care for an acute respiratory illness who test positive and negative, respectively, for influenza infection. Vaccine effectiveness (VE) estimates from TND studies are usually interpreted as vaccine effectiveness against medically-attended influenza (MAI). However, it is also important to estimate VE against any influenza illness (symptomatic influenza (SI)) as individuals with SI are still a public health burden even if they do not seek medical care. We present a numerical method to evaluate the bias of TND-based estimates of influenza VE with respect to MAI and SI. We consider two sources of bias: (a) confounding bias due to a (possibly unobserved) covariate that is associated with both vaccination and the probability of the outcome of interest and (b) bias resulting from the effect of vaccination on the probability of seeking care. Our results indicate that (a) VE estimates may suffer from substantial confounding bias when a confounder has a different effect on the probabilities of influenza and non-influenza ARI, and (b) when vaccination reduces the probability of seeking care against influenza ARI, then estimates of VE against MAI may be unbiased while estimates of VE against SI may be have a substantial positive bias.[/creativ_toggle]

Estimating Direct and Indirect Protective Effect of Influenza Vaccination in the United States, Nimalan Arinaminpathy, Inkyu Kevin Kim, Paul Gargiullo, Michael J. Haber, Ivo M. Foppa, Manoj Gambhir, and Joseph Bresee, American Journal of Epidemiology,  186(1), 1 July 2017, 92–100 [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]With influenza vaccination rates in the United States recently exceeding 45% of the population, it is important to understand the impact that vaccination is having on influenza transmission. In this study, we used a Bayesian modeling approach, combined with a simple dynamical model of influenza transmission, to estimate this impact. The combined framework synthesized evidence from a range of data sources relating to influenza transmission and vaccination in the United States. We found that, for seasonal epidemics, the number of infections averted ranged from 9.6 million in the 2006–2007 season (95% credible interval (CI): 8.7, 10.9) to 37.2 million (95% CI: 34.1, 39.6) in the 2012–2013 season. Expressed in relative terms, the proportion averted ranged from 20.8% (95% CI: 16.8, 24.3) of potential infections in the 2011–2012 season to 47.5% (95% CI: 43.7, 50.8) in the 2008–2009 season. The percentage averted was only 1.04% (95% CI: 0.15, 3.2) for the 2009 H1N1 pandemic, owing to the late timing of the vaccination program in relation to the pandemic in the Northern hemisphere. In the future, further vaccination coverage, as well as improved influenza vaccines (especially those offering better protection in the elderly), could have an even stronger effect on annual influenza epidemics.[/creativ_toggle]

A dynamic model for evaluation of bias of estimates of influenza vaccine effectiveness from observational studies, Kylie E.C. Ainslie, Meng Shi, Michael J. Haber and Walter A. Orenstein [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]As influenza vaccination is now widely recommended, randomized clinical trials for estimating influenza vaccine effectiveness (VE) are no longer ethical, and observational studies based on patients with acute respiratory illness (ARI) remain the only option. We developed a dynamic probability model for the evaluation of bias of VE estimates from four commonly used observational study designs: active surveillance cohort (ASC), passive surveillance co- hort, test-negative (TN), and traditional case-control. The model includes two covariates (health status and health awareness), which may affect the probabilities of vaccination, developing ARI, and seeking medical care. We consider two outcomes of interest: symptomatic influenza (SI) and medically-attended influenza (MAI). Our results suggest that when the outcome of interest is SI, ASC studies produce unbiased estimates, except when health sta- tus influences the probability of influenza ARI. When vaccination affects the probability of non-influenza ARI, VE estimates against SI from TN studies may be severely biased (90% Interval: (-0.34, 0.22)), while VE estimates from both types of cohort studies are unbiased. However, TN estimates are unbiased against MAI when vaccination does not affect the probability of non-influenza ARI and health status has the same effect on the probability of influenza and non-influenza ARIs.[/creativ_toggle]

Bias of influenza vaccine effectiveness estimates from test-negative studies conducted during an influenza pandemic, Kylie E.C. Ainslie, Michael J. Haber and Walter A. Orenstein [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]Test-negative (TN) studies have become the most widely used study design for the estimation of influenza vaccine effectiveness (VE) and are easily incorporated into existing influenza surveillance networks. We seek to determine the bias of TN-based VE estimates during a pandemic using a dynamic probability model. The model is used to evaluate and compare the bias of VE estimates under various sources of bias when vaccination occurs over time, such as during a pandemic. The model includes two covariates (health status and health awareness), which may affect the probabilities of vaccination, developing influenza and non-influenza acute respiratory illness (ARI), and seeking medical care. Specifically, we evaluate the bias of VE estimates when (1) vaccination affects the probability of developing a non-influenza ARI; (2) vaccination affects the probability of seeking medical care; (3) a covariate (e.g. health status) is related to both the probabilities of vaccination and developing an ARI; and (4) a covariate (e.g. health awareness) is related to both the probabilities of vaccination and of seeking medical care.
We considered two outcomes against which the vaccine is supposed to protect: symptomatic influenza (SI) and medically-attended influenza (MAI). When vaccination occurs over time, we found that the effect of delayed onset of vaccination is unpredictable and that VE estimates from TN studies were biased regardless of the source of bias present. However, if vaccination does not affect the probability of developing a non-influenza ARI then, TN-based estimates of VE against MAI will suffer only small (<0.05) to moderate (≥0.05 and <0.10) bias.
[/creativ_toggle]

A comparison of the test-negative and traditional case-control study designs with respect to the bias of estimates of rotavirus vaccine effectiveness, Michael J. Haber, Benjamin A. Lopman, Jacqueline E. Tate, Meng Shi and Umesh D. Parashar, Vaccine, 36(33), 9 August 2018, 5071-5076 [creativ_toggle icon=”” heading=”Abstract” onload=”closed”]Estimation of the effectiveness of rotavirus vaccines via the test-negative control study design has gained popularity over the past few years. In this study design, children with severe diarrhea who test positive for rotavirus infection are considered as cases, while children who test negative serve as controls. We use a simple probability model to evaluate and compare the test-negative control and the traditional case- control designs with respect to the bias of resulting estimates of rotavirus vaccine effectiveness (VE). Comparisons are performed under two scenarios, corresponding to studies performed in high-income and low-income countries. We consider two potential sources of bias: (a) misclassification bias resulting from imperfect sensitivity and specificity of the test used to diagnose rotavirus infection, and (b) selection bias associated with possible effect of rotavirus vaccination on the probability of contracting severe non- rotavirus diarrhea.
Our results suggest that both sources of bias may produce VE estimates with substantial bias. Particularly, lack of perfect specificity is associated with severe negative bias. For example, if the speci- ficity of the diagnostic test is 90% then VE estimates from both types of case-control studies may under- estimate the true VE by more than 20%. If the vaccine protects children against non-rotavirus diarrhea then VE estimates from test-negative control studies may be close to zero even though the true VE is 50%. However, the sensitivity and specificity of the enzyme immunoassay test currently used to diagnose rotavirus infections are both over 99%, and there is no solid evidence that the existing rotavirus vaccines affect the rates of non-rotavirus diarrhea. We therefore conclude that the test-negative control study design is a convenient and reliable alternative for estimation of rotavirus VE.[/creativ_toggle]