Quasi-experimental methods for pharmacoepidemiology: difference-in-differences and synthetic control methods with case studies for vaccine evaluation.

Difference-in-differences and synthetic control methods have become common study designs for evaluating the effects of policy changes, including health policies. They also have potential for providing real-world effectiveness and safety evidence in pharmacoepidemiology. To effectively add to the toolkit of the field, however, designs-including both their benefits and drawbacks-must be well understood. Quasi-experimental designs provide an opportunity to estimate the average treatment effect on the treated without requiring the measurement of all possible confounding factors, and to assess population-level effects. This requires, however, other key assumptions, including the parallel trends or stable weighting assumptions, a lack of other concurrent events that could alter time trends, and an absence of contamination between exposed and unexposed units. The targeted estimands are also highly specific to the settings of the study, and combining across units or time periods can be challenging. Case studies are presented for three vaccine evaluation studies, showcasing some of these challenges and opportunities in a specific field of pharmacoepidemiology. These methods provide feasible and valuable sources of evidence in various pharmacoepidemiologic settings and can be improved through research to identify and weigh the advantages and disadvantages in those settings.

Anastomotic Stricture After Minimally Invasive Esophagectomy.

BACKGROUND: Despite improved outcomes, minimally invasive esophagectomy (MIE) continues to be associated with anastomotic strictures. Most resolve after a single dilation; however, some become refractory. Little is known about strictures after MIE in North America. METHODS: We performed a single-institution retrospective review of MIEs from 2015 to 2019. Primary outcomes were the proportion of patients requiring anastomotic dilation and the dilation rate per year. Univariate analyses of patients undergoing dilation by various risk factors were performed with nonparametric tests, and multivariate analyses of the dilation rate were conducted using generalized linear models. RESULTS: Of 391 included patients, 431 dilations were performed on 135 patients (34.5%, 3.2 dilations per patient who required at least 1 per patient). One complication occurred after dilation. Comorbidities, tumor histology, and tumor stage were not significantly associated with stricture. Three-field MIE was associated with a higher percentage of patients undergoing dilation (48.9% vs 27.1%, P < .001) and a higher rate of dilations (0.944 vs 0.441 dilations per year, P = .007) than 2-field MIE, and this association remained significant after controlling for covariates. When accounting for surgeon variability, this difference was no longer significant. Among patients with 1 or more dilations, those receiving dilation within 100 days of surgery needed more subsequent dilations (2.0 vs 0.6 dilations per year, P < .001). CONCLUSIONS: After controlling for multiple variables, a 3-field MIE approach was associated with a higher rate of repeat dilations in patients undergoing MIE. A shorter interval between esophagectomy and initial dilation is strongly associated with the need for repeated dilations.

Public Health Impacts of Vaccines for COVID-19 and Beyond: Opportunities to Overcome Technical and Regulatory Barriers for Randomized Trials.

The COVID-19 pandemic has revealed the importance of the population-scale effects of both diseases and interventions. Vaccines have had an enormous impact, greatly reducing the suffering caused by COVID-19. Clinical trials have focused on individual-level clinical benefits, however, so the broader effects of the vaccines on preventing infection and transmission, and their overall effect at the community level, remain unclear. These questions can be addressed through alternative designs for vaccine trials, including assessing different endpoints and randomizing at the cluster instead of individual level. Although these designs exist, various factors have limited their use as preauthorization pivotal trials. They face statistical, epidemiological, and logistical limitations as well as regulatory barriers and uncertainty. Addressing these hindrances through research, communication, and policy can improve the evidence base of vaccines, their strategic deployment, and population health, both in the COVID-19 pandemic and in future infectious disease outbreaks. (Am J Public Health. 2023;113(7):778-785. https://doi.org/10.2105/AJPH.2023.307302).

Comparative performance of between-population vaccine allocation strategies with applications for emerging pandemics.

Vaccine allocation decisions during emerging pandemics have proven to be challenging due to competing ethical, practical, and political considerations. Complicating decision making, policy makers need to consider vaccine allocation strategies that balance needs both within and between populations. When vaccine stockpiles are limited, doses should be allocated in locations to maximize their impact. Using a susceptible-exposed-infectious-recovered (SEIR) model we examine optimal vaccine allocation decisions across two populations considering the impact of characteristics of the population (e.g., size, underlying immunity, heterogeneous risk structure, interaction), vaccine (e.g., vaccine efficacy), pathogen (e.g., transmissibility), and delivery (e.g., varying speed and timing of rollout). Across a wide range of characteristics considered, we find that vaccine allocation proportional to population size (i.e., pro-rata allocation) performs either better or comparably to nonproportional allocation strategies in minimizing the cumulative number of infections. These results may argue in favor of sharing of vaccines between locations in the context of an epidemic caused by an emerging pathogen, where many epidemiologic characteristics may not be known.

The required size of cluster randomized trials of nonpharmaceutical interventions in epidemic settings.

To control the SARS-CoV-2 pandemic and future pathogen outbreaks requires an understanding of which nonpharmaceutical interventions are effective at reducing transmission. Observational studies, however, are subject to biases that could erroneously suggest an impact on transmission, even when there is no true effect. Cluster randomized trials permit valid hypothesis tests of the effect of interventions on community transmission. While such trials could be completed in a relatively short period of time, they might require large sample sizes to achieve adequate power. However, the sample sizes required for such tests in outbreak settings are largely undeveloped, leaving unanswered the question of whether these designs are practical. We develop approximate sample size formulae and simulation-based sample size methods for cluster randomized trials in infectious disease outbreaks. We highlight key relationships between characteristics of transmission and the enrolled communities and the required sample sizes, describe settings where trials powered to detect a meaningful true effect size may be feasible, and provide recommendations for investigators in planning such trials. The approximate formulae and simulation banks may be used by investigators to quickly assess the feasibility of a trial, followed by more detailed methods to more precisely size the trial. For example, we show that community-scale trials requiring 220 clusters with 100 tested individuals per cluster are powered to identify interventions that reduce transmission by 40% in one generation interval, using parameters identified for SARS-CoV-2 transmission. For more modest treatment effects, or when transmission is extremely overdispersed, however, much larger sample sizes are required.

Comparative performance of between-population vaccine allocation strategies with applications for emerging pandemics.

Vaccine allocation decisions during emerging pandemics have proven to be challenging due to competing ethical, practical, and political considerations. Complicating decision making, policy makers need to consider vaccine allocation strategies that balance needs both within and between populations. Due to limited vaccine stockpiles, vaccine doses should be allocated in locations where their impact will be maximized. Using a susceptible-exposed-infectious-recovered (SEIR) model we examine optimal vaccine allocation decisions across two populations considering the impact of population size, underlying immunity, continuous vaccine roll-out, heterogeneous population risk structure, and differences in disease transmissibility. We find that in the context of an emerging pathogen where many epidemiologic characteristics might not be known, equal vaccine allocation between populations performs optimally in most scenarios. In the specific case considering heterogeneous population risk structure, first targeting individuals at higher risk of transmission or death due to infection leads to equal resource allocation across populations.

Power and sample size calculations for cluster randomized trials with binary outcomes when intracluster correlation coefficients vary by treatment arm.

BACKGROUND/AIMS: Generalized estimating equations are commonly used to fit logistic regression models to clustered binary data from cluster randomized trials. A commonly used correlation structure assumes that the intracluster correlation coefficient does not vary by treatment arm or other covariates, but the consequences of this assumption are understudied. We aim to evaluate the effect of allowing variation of the intracluster correlation coefficient by treatment or other covariates on the efficiency of analysis and show how to account for such variation in sample size calculations. METHODS: We develop formulae for the asymptotic variance of the estimated difference in outcome between treatment arms obtained when the true exchangeable correlation structure depends on the treatment arm and the working correlation structure used in the generalized estimating equations analysis is: (i) correctly specified, (ii) independent, or (iii) exchangeable with no dependence on treatment arm. These formulae require a known distribution of cluster sizes; we also develop simplifications for the case when cluster sizes do not vary and approximations that can be used when the first two moments of the cluster size distribution are known. We then extend the results to settings with adjustment for a second binary cluster-level covariate. We provide formulae to calculate the required sample size for cluster randomized trials using these variances. RESULTS: We show that the asymptotic variance of the estimated difference in outcome between treatment arms using these three working correlation structures is the same if all clusters have the same size, and this asymptotic variance is approximately the same when intracluster correlation coefficient values are small. We illustrate these results using data from a recent cluster randomized trial for infectious disease prevention in which the clusters are groups of households and modest in size (mean 9.6 individuals), with intracluster correlation coefficient values of 0.078 in the control arm and 0.057 in an intervention arm. In this application, we found a negligible difference between the variances calculated using structures (i) and (iii) and only a small increase (typically <5%) for the independent correlation structure (ii), and hence minimal effect on power or sample size requirements. The impact may be larger in other applications if there is greater variation in the ICC between treatment arms or with an additional covariate. CONCLUSION: The common approach of fitting generalized estimating equations with an exchangeable working correlation structure with a common intracluster correlation coefficient across arms likely does not substantially reduce the power or efficiency of the analysis in the setting of a large number of small or modest-sized clusters, even if the intracluster correlation coefficient varies by treatment arm. Our formulae, however, allow formal evaluation of this and may identify situations in which variation in intracluster correlation coefficient by treatment arm or another binary covariate may have a more substantial impact on power and hence sample size requirements.

Estimating Vaccine Efficacy Against Transmission via Effect on Viral Load.

Determining policies to end the SARS-CoV-2 pandemic will require an understanding of the efficacy and effectiveness (hereafter, efficacy) of vaccines. Beyond the efficacy against severe disease and symptomatic and asymptomatic infection, understanding vaccine efficacy against virus transmission, including efficacy against transmission of different viral variants, will help model epidemic trajectory and determine appropriate control measures. Recent studies have proposed using random virologic testing in individual randomized controlled trials to improve estimation of vaccine efficacy against infection. We propose to further use the viral load measures from these tests to estimate efficacy against transmission. This estimation requires a model of the relationship between viral load and transmissibility and assumptions about the vaccine effect on transmission and the progress of the epidemic. We describe these key assumptions, potential violations of them, and solutions that can be implemented to mitigate these violations. Assessing these assumptions and implementing this random sampling, with viral load measures, will enable better estimation of the crucial measure of vaccine efficacy against transmission.

Joint Estimation of Generation Time and Incubation Period for Coronavirus Disease 2019.

BACKGROUND: Coronavirus disease 2019 (COVID-19) has caused a heavy disease burden globally. The impact of process and timing of data collection on the accuracy of estimation of key epidemiological distributions are unclear. Because infection times are typically unobserved, there are relatively few estimates of generation time distribution. METHODS: We developed a statistical framework to jointly estimate generation time and incubation period from human-to-human transmission pairs, accounting for sampling biases. We applied the framework on 80 laboratory-confirmed human-to-human transmission pairs in China. We further inferred the infectiousness profile, serial interval distribution, proportions of presymptomatic transmission, and basic reproduction number (R0) for COVID-19. RESULTS: The estimated mean incubation period was 4.8 days (95% confidence interval [CI], 4.1-5.6), and mean generation time was 5.7 days (95% CI, 4.8-6.5). The estimated R0 based on the estimated generation time was 2.2 (95% CI, 1.9-2.4). A simulation study suggested that our approach could provide unbiased estimates, insensitive to the width of exposure windows. CONCLUSIONS: Properly accounting for the timing and process of data collection is critical to have correct estimates of generation time and incubation period. R0 can be biased when it is derived based on serial interval as the proxy of generation time.

Estimating epidemiologic dynamics from cross-sectional viral load distributions.

Estimating an epidemic's trajectory is crucial for developing public health responses to infectious diseases, but case data used for such estimation are confounded by variable testing practices. We show that the population distribution of viral loads observed under random or symptom-based surveillance-in the form of cycle threshold (Ct) values obtained from reverse transcription quantitative polymerase chain reaction testing-changes during an epidemic. Thus, Ct values from even limited numbers of random samples can provide improved estimates of an epidemic's trajectory. Combining data from multiple such samples improves the precision and robustness of this estimation. We apply our methods to Ct values from surveillance conducted during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic in a variety of settings and offer alternative approaches for real-time estimates of epidemic trajectories for outbreak management and response.

Snowball Sampling Study Design for Serosurveys Early in Disease Outbreaks.

Serological surveys can provide evidence of cases that were not previously detected, depict the spectrum of disease severity, and estimate the proportion of asymptomatic infections. To capture these parameters, survey sample sizes may need to be very large, especially when the overall infection rate is still low. Therefore, we propose the use of "snowball sampling" to enrich serological surveys by testing contacts of infected persons identified in the early stages of an outbreak. For future emerging pandemics, this observational study sampling design can answer many key questions, such as estimation of the asymptomatic proportion of all infected cases, the probability of a given clinical presentation for a seropositive individual, or the association between characteristics of either the host or the infection and seropositivity among contacts of index individuals. We provide examples, in the context of the coronavirus disease 2019 (COVID-19) pandemic, of studies and analysis methods that use a snowball sample and perform a simulation study that demonstrates scenarios where snowball sampling can answer these questions more efficiently than other sampling schemes. We hope such study designs can be applied to provide valuable information to slow the present pandemic as it enters its next stage and in early stages of future pandemics.

Perfect as the enemy of good: tracing transmissions with low-sensitivity tests to mitigate SARS-CoV-2 outbreaks.

Throughout the COVID-19 pandemic, governments and individuals have attempted a wide variety of strategies to limit the damage of the pandemic on human lives, population health, and economies. Contact tracing has been a commonly used strategy, and various approaches have been proposed and attempted. We summarise some methods of contact tracing and testing, considering the resources demanded by each and how features of SARS-CoV-2 transmission affect their effectiveness. We also propose an approach focusing on tracing transmission events, which can be particularly effective when superspreading events play a large role in transmission. Accounting for the best available evidence on a pathogen and for the availability of resources can make control strategies more effective, even if they are not perfect.

How to detect and reduce potential sources of biases in studies of SARS-CoV-2 and COVID-19.

In response to the coronavirus disease (COVID-19) pandemic, public health scientists have produced a large and rapidly expanding body of literature that aims to answer critical questions, such as the proportion of the population in a geographic area that has been infected; the transmissibility of the virus and factors associated with high infectiousness or susceptibility to infection; which groups are the most at risk of infection, morbidity and mortality; and the degree to which antibodies confer protection to re-infection. Observational studies are subject to a number of different biases, including confounding, selection bias, and measurement error, that may threaten their validity or influence the interpretation of their results. To assist in the critical evaluation of a vast body of literature and contribute to future study design, we outline and propose solutions to biases that can occur across different categories of observational studies of COVID-19. We consider potential biases that could occur in five categories of studies: (1) cross-sectional seroprevalence, (2) longitudinal seroprotection, (3) risk factor studies to inform interventions, (4) studies to estimate the secondary attack rate, and (5) studies that use secondary attack rates to make inferences about infectiousness and susceptibility.

Estimating epidemiologic dynamics from cross-sectional viral load distributions.

Estimating an epidemic's trajectory is crucial for developing public health responses to infectious diseases, but incidence data used for such estimation are confounded by variable testing practices. We show instead that the population distribution of viral loads observed under random or symptom-based surveillance, in the form of cycle threshold (Ct) values, changes during an epidemic and that Ct values from even limited numbers of random samples can provide improved estimates of an epidemic's trajectory. Combining multiple such samples and the fraction positive improves the precision and robustness of such estimation. We apply our methods to Ct values from surveillance conducted during the SARS-CoV-2 pandemic in a variety of settings and demonstrate new approaches for real-time estimates of epidemic trajectories for outbreak management and response.

Potential Biases Arising From Epidemic Dynamics in Observational Seroprotection Studies.

The extent and duration of immunity following infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are critical outstanding questions about the epidemiology of this novel virus, and studies are needed to evaluate the effects of serostatus on reinfection. Understanding the potential sources of bias and methods for alleviating biases in these studies is important for informing their design and analysis. Confounding by individual-level risk factors in observational studies like these is relatively well appreciated. Here, we show how geographic structure and the underlying, natural dynamics of epidemics can also induce noncausal associations. We take the approach of simulating serological studies in the context of an uncontrolled or controlled epidemic, under different assumptions about whether prior infection does or does not protect an individual against subsequent infection, and using various designs and analytical approaches to analyze the simulated data. We find that in studies assessing whether seropositivity confers protection against future infection, comparing seropositive persons with seronegative persons with similar time-dependent patterns of exposure to infection by stratifying or matching on geographic location and time of enrollment is essential in order to prevent bias.

Decreasing Trends in Heavy Sugar-Sweetened Beverage Consumption in the United States, 2003 to 2016.

BACKGROUND: Although previous studies have documented declines in intake from sugar-sweetened beverages (SSB) in the United States, it is important to examine whether heavy SSB intake (≥500 kcal/day) is decreasing in parallel. Examining the intake patterns of heavy SSB consumers is imperative because these individuals face the greatest health risks and thus may benefit the most from targeted policy and programmatic efforts to reduce intake. OBJECTIVE: To provide the most recent national estimates for trends in heavy SSB intake among children and adults in the United States between 2003-2004 and 2015-2016, to examine whether these trends differ by sociodemographic characteristics, and to describe where SSB are acquired and consumed by the heaviest SSB consumers. DESIGN: Trend analyses of demographic and 24-hour dietary recall data in the 2003-2004 to 2015-2016 National Health and Nutrition Examination Survey. PARTICIPANTS/SETTING: Participants were 21,783 children (aged 2 to 19 years) and 32,355 adults (aged ≥20 years). MAIN OUTCOME MEASURES: Heavy SSB intake (≥500 kcal/day). STATISTICAL ANALYSIS: Survey-weighted logistic regression was used to estimate the proportion of heavy SSB consumers, overall and by age group, race/ethnicity, sex, and income status (lower income = <130% Federal Poverty Level). Proportions were used to summarize where SSB are most often acquired and consumed. RESULTS: Between 2003-2004 and 2015-2016, the prevalence of heavy SSB intake declined significantly among children (10.9% to 3.3%) and adults (12.7% to 9.1%). For children, these declines were observed across age group, sex, family income status, and most races/ethnicities. For adults, these significant declines were observed among 20- to 39-year olds, most races/ethnicities, and higher-income adults. However, there was a significant increase in heavy SSB intake among adults aged ≥60 years and no significant change among 40- to 59-year olds and non-Mexican Hispanic adults. The majority of energy intake from SSB consumed by heavy SSB drinkers was from products acquired from stores and was consumed at home. CONCLUSIONS: Heavy SSB intake is declining, but attention must be paid to certain subgroups with high intake for whom trends are not decreasing, particularly 40- to 59-year olds and non-Mexican Hispanic adults.

Statistical Properties of Stepped Wedge Cluster-Randomized Trials in Infectious Disease Outbreaks.

Randomized controlled trials are crucial for the evaluation of interventions such as vaccinations, but the design and analysis of these studies during infectious disease outbreaks is complicated by statistical, ethical, and logistical factors. Attempts to resolve these complexities have led to the proposal of a variety of trial designs, including individual randomization and several types of cluster randomization designs: parallel-arm, ring vaccination, and stepped wedge designs. Because of the strong time trends present in infectious disease incidence, however, methods generally used to analyze stepped wedge trials might not perform well in these settings. Using simulated outbreaks, we evaluated various designs and analysis methods, including recently proposed methods for analyzing stepped wedge trials, to determine the statistical properties of these methods. While new methods for analyzing stepped wedge trials can provide some improvement over previous methods, we find that they still lag behind parallel-arm cluster-randomized trials and individually randomized trials in achieving adequate power to detect intervention effects. We also find that these methods are highly sensitive to the weighting of effect estimates across time periods. Despite the value of new methods, stepped wedge trials still have statistical disadvantages compared with other trial designs in epidemic settings.

Potential biases arising from epidemic dynamics in observational seroprotection studies.

The extent and duration of immunity following SARS-CoV-2 infection are critical outstanding questions about the epidemiology of this novel virus, and studies are needed to evaluate the effects of serostatus on reinfection. Understanding the potential sources of bias and methods to alleviate biases in these studies is important for informing their design and analysis. Confounding by individual-level risk factors in observational studies like these is relatively well appreciated. Here, we show how geographic structure and the underlying, natural dynamics of epidemics can also induce noncausal associations. We take the approach of simulating serologic studies in the context of an uncontrolled or a controlled epidemic, under different assumptions about whether prior infection does or does not protect an individual against subsequent infection, and using various designs and analytic approaches to analyze the simulated data. We find that in studies assessing the efficacy of serostatus on future infection, comparing seropositive individuals to seronegative individuals with similar time-dependent patterns of exposure to infection, by stratifying or matching on geographic location and time of enrollment, is essential to prevent bias.

Statistical Properties of Stepped Wedge Cluster-Randomized Trials in Infectious Disease Outbreaks.

Randomized controlled trials are crucial for the evaluation of interventions such as vaccinations, but the design and analysis of these studies during infectious disease outbreaks is complicated by statistical, ethical, and logistical factors. Attempts to resolve these complexities have led to the proposal of a variety of trial designs, including individual randomization and several types of cluster randomization designs: parallel-arm, ring vaccination, and stepped wedge designs. Because of the strong time trends present in infectious disease incidence, however, methods generally used to analyze stepped wedge trials may not perform well in these settings. Using simulated outbreaks, we evaluate various designs and analysis methods, including recently proposed methods for analyzing stepped wedge trials, to determine the statistical properties of these methods. While new methods for analyzing stepped wedge trials can provide some improvement over previous methods, we find that they still lag behind parallel-arm cluster-randomized trials and individually-randomized trials in achieving adequate power to detect intervention effects. We also find that these methods are highly sensitive to the weighting of effect estimates across time periods. Despite the value of new methods, stepped wedge trials still have statistical disadvantages compared to other trial designs in epidemic settings.