Quasi-experimental methods for pharmacoepidemiology: difference-in-differences and synthetic control methods with case studies for vaccine evaluation.

Difference-in-differences and synthetic control methods have become common study designs for evaluating the effects of changes in policies, including health policies. They also have potential for providing real-world effectiveness and safety evidence in pharmacoepidemiology. To effectively add to the toolkit of the field, however, designs-including both their benefits and drawbacks-must be well understood. Quasi-experimental designs provide an opportunity to estimate the average treatment effect on the treated without requiring the measurement of all possible confounding factors, and to assess population-level effects. This requires, however, other key assumptions, including the parallel trends or stable weighting assumptions, a lack of other concurrent events that could alter time trends, and an absence of contamination between exposed and unexposed units. The targeted estimands are also highly specific to the settings of the study, and combining across units or time periods can be challenging. Case studies are presented for 3 vaccine evaluation studies, showcasing some of these challenges and opportunities in a specific field of pharmacoepidemiology. These methods provide feasible and valuable sources of evidence in various pharmacoepidemiologic settings and can be improved through research to identify and weigh the advantages and disadvantages in those settings. This article is part of a Special Collection on Pharmacoepidemiology.

Public Health Impacts of Vaccines for COVID-19 and Beyond: Opportunities to Overcome Technical and Regulatory Barriers for Randomized Trials.

The COVID-19 pandemic has revealed the importance of the population-scale effects of both diseases and interventions. Vaccines have had an enormous impact, greatly reducing the suffering caused by COVID-19. Clinical trials have focused on individual-level clinical benefits, however, so the broader effects of the vaccines on preventing infection and transmission, and their overall effect at the community level, remain unclear. These questions can be addressed through alternative designs for vaccine trials, including assessing different endpoints and randomizing at the cluster instead of individual level. Although these designs exist, various factors have limited their use as preauthorization pivotal trials. They face statistical, epidemiological, and logistical limitations as well as regulatory barriers and uncertainty. Addressing these hindrances through research, communication, and policy can improve the evidence base of vaccines, their strategic deployment, and population health, both in the COVID-19 pandemic and in future infectious disease outbreaks. (Am J Public Health. 2023;113(7):778-785. https://doi.org/10.2105/AJPH.2023.307302).

The required size of cluster randomized trials of nonpharmaceutical interventions in epidemic settings.

To control the SARS-CoV-2 pandemic and future pathogen outbreaks requires an understanding of which nonpharmaceutical interventions are effective at reducing transmission. Observational studies, however, are subject to biases that could erroneously suggest an impact on transmission, even when there is no true effect. Cluster randomized trials permit valid hypothesis tests of the effect of interventions on community transmission. While such trials could be completed in a relatively short period of time, they might require large sample sizes to achieve adequate power. However, the sample sizes required for such tests in outbreak settings are largely undeveloped, leaving unanswered the question of whether these designs are practical. We develop approximate sample size formulae and simulation-based sample size methods for cluster randomized trials in infectious disease outbreaks. We highlight key relationships between characteristics of transmission and the enrolled communities and the required sample sizes, describe settings where trials powered to detect a meaningful true effect size may be feasible, and provide recommendations for investigators in planning such trials. The approximate formulae and simulation banks may be used by investigators to quickly assess the feasibility of a trial, followed by more detailed methods to more precisely size the trial. For example, we show that community-scale trials requiring 220 clusters with 100 tested individuals per cluster are powered to identify interventions that reduce transmission by 40% in one generation interval, using parameters identified for SARS-CoV-2 transmission. For more modest treatment effects, or when transmission is extremely overdispersed, however, much larger sample sizes are required.

Power and sample size calculations for cluster randomized trials with binary outcomes when intracluster correlation coefficients vary by treatment arm.

BACKGROUND/AIMS: Generalized estimating equations are commonly used to fit logistic regression models to clustered binary data from cluster randomized trials. A commonly used correlation structure assumes that the intracluster correlation coefficient does not vary by treatment arm or other covariates, but the consequences of this assumption are understudied. We aim to evaluate the effect of allowing variation of the intracluster correlation coefficient by treatment or other covariates on the efficiency of analysis and show how to account for such variation in sample size calculations. METHODS: We develop formulae for the asymptotic variance of the estimated difference in outcome between treatment arms obtained when the true exchangeable correlation structure depends on the treatment arm and the working correlation structure used in the generalized estimating equations analysis is: (i) correctly specified, (ii) independent, or (iii) exchangeable with no dependence on treatment arm. These formulae require a known distribution of cluster sizes; we also develop simplifications for the case when cluster sizes do not vary and approximations that can be used when the first two moments of the cluster size distribution are known. We then extend the results to settings with adjustment for a second binary cluster-level covariate. We provide formulae to calculate the required sample size for cluster randomized trials using these variances. RESULTS: We show that the asymptotic variance of the estimated difference in outcome between treatment arms using these three working correlation structures is the same if all clusters have the same size, and this asymptotic variance is approximately the same when intracluster correlation coefficient values are small. We illustrate these results using data from a recent cluster randomized trial for infectious disease prevention in which the clusters are groups of households and modest in size (mean 9.6 individuals), with intracluster correlation coefficient values of 0.078 in the control arm and 0.057 in an intervention arm. In this application, we found a negligible difference between the variances calculated using structures (i) and (iii) and only a small increase (typically <5%) for the independent correlation structure (ii), and hence minimal effect on power or sample size requirements. The impact may be larger in other applications if there is greater variation in the ICC between treatment arms or with an additional covariate. CONCLUSION: The common approach of fitting generalized estimating equations with an exchangeable working correlation structure with a common intracluster correlation coefficient across arms likely does not substantially reduce the power or efficiency of the analysis in the setting of a large number of small or modest-sized clusters, even if the intracluster correlation coefficient varies by treatment arm. Our formulae, however, allow formal evaluation of this and may identify situations in which variation in intracluster correlation coefficient by treatment arm or another binary covariate may have a more substantial impact on power and hence sample size requirements.

Joint Estimation of Generation Time and Incubation Period for Coronavirus Disease 2019.

BACKGROUND: Coronavirus disease 2019 (COVID-19) has caused a heavy disease burden globally. The impact of process and timing of data collection on the accuracy of estimation of key epidemiological distributions are unclear. Because infection times are typically unobserved, there are relatively few estimates of generation time distribution. METHODS: We developed a statistical framework to jointly estimate generation time and incubation period from human-to-human transmission pairs, accounting for sampling biases. We applied the framework on 80 laboratory-confirmed human-to-human transmission pairs in China. We further inferred the infectiousness profile, serial interval distribution, proportions of presymptomatic transmission, and basic reproduction number (R0) for COVID-19. RESULTS: The estimated mean incubation period was 4.8 days (95% confidence interval [CI], 4.1-5.6), and mean generation time was 5.7 days (95% CI, 4.8-6.5). The estimated R0 based on the estimated generation time was 2.2 (95% CI, 1.9-2.4). A simulation study suggested that our approach could provide unbiased estimates, insensitive to the width of exposure windows. CONCLUSIONS: Properly accounting for the timing and process of data collection is critical to have correct estimates of generation time and incubation period. R0 can be biased when it is derived based on serial interval as the proxy of generation time.

Snowball Sampling Study Design for Serosurveys Early in Disease Outbreaks.

Serological surveys can provide evidence of cases that were not previously detected, depict the spectrum of disease severity, and estimate the proportion of asymptomatic infections. To capture these parameters, survey sample sizes may need to be very large, especially when the overall infection rate is still low. Therefore, we propose the use of "snowball sampling" to enrich serological surveys by testing contacts of infected persons identified in the early stages of an outbreak. For future emerging pandemics, this observational study sampling design can answer many key questions, such as estimation of the asymptomatic proportion of all infected cases, the probability of a given clinical presentation for a seropositive individual, or the association between characteristics of either the host or the infection and seropositivity among contacts of index individuals. We provide examples, in the context of the coronavirus disease 2019 (COVID-19) pandemic, of studies and analysis methods that use a snowball sample and perform a simulation study that demonstrates scenarios where snowball sampling can answer these questions more efficiently than other sampling schemes. We hope such study designs can be applied to provide valuable information to slow the present pandemic as it enters its next stage and in early stages of future pandemics.

Potential Biases Arising From Epidemic Dynamics in Observational Seroprotection Studies.

The extent and duration of immunity following infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are critical outstanding questions about the epidemiology of this novel virus, and studies are needed to evaluate the effects of serostatus on reinfection. Understanding the potential sources of bias and methods for alleviating biases in these studies is important for informing their design and analysis. Confounding by individual-level risk factors in observational studies like these is relatively well appreciated. Here, we show how geographic structure and the underlying, natural dynamics of epidemics can also induce noncausal associations. We take the approach of simulating serological studies in the context of an uncontrolled or controlled epidemic, under different assumptions about whether prior infection does or does not protect an individual against subsequent infection, and using various designs and analytical approaches to analyze the simulated data. We find that in studies assessing whether seropositivity confers protection against future infection, comparing seropositive persons with seronegative persons with similar time-dependent patterns of exposure to infection by stratifying or matching on geographic location and time of enrollment is essential in order to prevent bias.

Estimating Vaccine Efficacy Against Transmission via Effect on Viral Load.

Determining policies to end the SARS-CoV-2 pandemic will require an understanding of the efficacy and effectiveness (hereafter, efficacy) of vaccines. Beyond the efficacy against severe disease and symptomatic and asymptomatic infection, understanding vaccine efficacy against virus transmission, including efficacy against transmission of different viral variants, will help model epidemic trajectory and determine appropriate control measures. Recent studies have proposed using random virologic testing in individual randomized controlled trials to improve estimation of vaccine efficacy against infection. We propose to further use the viral load measures from these tests to estimate efficacy against transmission. This estimation requires a model of the relationship between viral load and transmissibility and assumptions about the vaccine effect on transmission and the progress of the epidemic. We describe these key assumptions, potential violations of them, and solutions that can be implemented to mitigate these violations. Assessing these assumptions and implementing this random sampling, with viral load measures, will enable better estimation of the crucial measure of vaccine efficacy against transmission.

How to detect and reduce potential sources of biases in studies of SARS-CoV-2 and COVID-19.

In response to the coronavirus disease (COVID-19) pandemic, public health scientists have produced a large and rapidly expanding body of literature that aims to answer critical questions, such as the proportion of the population in a geographic area that has been infected; the transmissibility of the virus and factors associated with high infectiousness or susceptibility to infection; which groups are the most at risk of infection, morbidity and mortality; and the degree to which antibodies confer protection to re-infection. Observational studies are subject to a number of different biases, including confounding, selection bias, and measurement error, that may threaten their validity or influence the interpretation of their results. To assist in the critical evaluation of a vast body of literature and contribute to future study design, we outline and propose solutions to biases that can occur across different categories of observational studies of COVID-19. We consider potential biases that could occur in five categories of studies: (1) cross-sectional seroprevalence, (2) longitudinal seroprotection, (3) risk factor studies to inform interventions, (4) studies to estimate the secondary attack rate, and (5) studies that use secondary attack rates to make inferences about infectiousness and susceptibility.

Statistical Properties of Stepped Wedge Cluster-Randomized Trials in Infectious Disease Outbreaks.

Randomized controlled trials are crucial for the evaluation of interventions such as vaccinations, but the design and analysis of these studies during infectious disease outbreaks is complicated by statistical, ethical, and logistical factors. Attempts to resolve these complexities have led to the proposal of a variety of trial designs, including individual randomization and several types of cluster randomization designs: parallel-arm, ring vaccination, and stepped wedge designs. Because of the strong time trends present in infectious disease incidence, however, methods generally used to analyze stepped wedge trials might not perform well in these settings. Using simulated outbreaks, we evaluated various designs and analysis methods, including recently proposed methods for analyzing stepped wedge trials, to determine the statistical properties of these methods. While new methods for analyzing stepped wedge trials can provide some improvement over previous methods, we find that they still lag behind parallel-arm cluster-randomized trials and individually randomized trials in achieving adequate power to detect intervention effects. We also find that these methods are highly sensitive to the weighting of effect estimates across time periods. Despite the value of new methods, stepped wedge trials still have statistical disadvantages compared with other trial designs in epidemic settings.

Sample size estimation for stratified individual and cluster randomized trials with binary outcomes.

Individual randomized trials (IRTs) and cluster randomized trials (CRTs) with binary outcomes arise in a variety of settings and are often analyzed by logistic regression (fitted using generalized estimating equations for CRTs). The effect of stratification on the required sample size is less well understood for trials with binary outcomes than for continuous outcomes. We propose easy-to-use methods for sample size estimation for stratified IRTs and CRTs and demonstrate the use of these methods for a tuberculosis prevention CRT currently being planned. For both IRTs and CRTs, we also identify the ratio of the sample size for a stratified trial vs a comparably powered unstratified trial, allowing investigators to evaluate how stratification will affect the required sample size when planning a trial. For CRTs, these can be used when the investigator has estimates of the within-stratum intracluster correlation coefficients (ICCs) or by assuming a common within-stratum ICC. Using these methods, we describe scenarios where stratification may have a practically important impact on the required sample size. We find that in the two-stratum case, for both IRTs and for CRTs with very small cluster sizes, there are unlikely to be plausible scenarios in which an important sample size reduction is achieved when the overall probability of a subject experiencing the event of interest is low. When the probability of events is not small, or when cluster sizes are large, however, there are scenarios where practically important reductions in sample size result from stratification.

Novel methods for the analysis of stepped wedge cluster randomized trials.

Stepped wedge cluster randomized trials (SW-CRTs) have become increasingly popular and are used for a variety of interventions and outcomes, often chosen for their feasibility advantages. SW-CRTs must account for time trends in the outcome because of the staggered rollout of the intervention. Robust inference procedures and nonparametric analysis methods have recently been proposed to handle such trends without requiring strong parametric modeling assumptions, but these are less powerful than model-based approaches. We propose several novel analysis methods that reduce reliance on modeling assumptions while preserving some of the increased power provided by the use of mixed effects models. In one method, we use the synthetic control approach to find the best matching clusters for a given intervention cluster. Another method makes use of within-cluster crossover information to construct an overall estimator. We also consider methods that combine these approaches to further improve power. We test these methods on simulated SW-CRTs, describing scenarios in which these methods have increased power compared with existing nonparametric methods while preserving nominal validity when mixed effects models are misspecified. We also demonstrate theoretical properties of these estimators with less restrictive assumptions than mixed effects models. Finally, we propose avenues for future research on the use of these methods; motivation for such research arises from their flexibility, which allows the identification of specific causal contrasts of interest, their robustness, and the potential for incorporating covariates to further increase power. Investigators conducting SW-CRTs might well consider such methods when common modeling assumptions may not hold.

When the Alpha is the Omega: P-Values, "Substantial Evidence," and the 0.05 Standard at FDA.

A prominent feature of statistical reasoning for nearly a century, the p-value plays an especially vital role in the clinical testing of new drugs. Over the last fifty years, the U.S. Food and Drug Administration (FDA) has relied on p-values and significance testing to demonstrate the efficacy of new drugs in the premarket approval process. This article seeks to illuminate the history of this statistic and explain how the statistical significance threshold of 0.05, commonly decried as an arbitrary cutoff, is a useful tool that came to be the cornerstone of FDA decision-making.

Comparative performance of between-population vaccine allocation strategies with applications for emerging pandemics.

Vaccine allocation decisions during emerging pandemics have proven to be challenging due to competing ethical, practical, and political considerations. Complicating decision making, policy makers need to consider vaccine allocation strategies that balance needs both within and between populations. When vaccine stockpiles are limited, doses should be allocated in locations to maximize their impact. Using a susceptible-exposed-infectious-recovered (SEIR) model we examine optimal vaccine allocation decisions across two populations considering the impact of characteristics of the population (e.g., size, underlying immunity, heterogeneous risk structure, interaction), vaccine (e.g., vaccine efficacy), pathogen (e.g., transmissibility), and delivery (e.g., varying speed and timing of rollout). Across a wide range of characteristics considered, we find that vaccine allocation proportional to population size (i.e., pro-rata allocation) performs either better or comparably to nonproportional allocation strategies in minimizing the cumulative number of infections. These results may argue in favor of sharing of vaccines between locations in the context of an epidemic caused by an emerging pathogen, where many epidemiologic characteristics may not be known.

Anastomotic Stricture After Minimally Invasive Esophagectomy.

BACKGROUND: Despite improved outcomes, minimally invasive esophagectomy (MIE) continues to be associated with anastomotic strictures. Most resolve after a single dilation; however, some become refractory. Little is known about strictures after MIE in North America. METHODS: We performed a single-institution retrospective review of MIEs from 2015 to 2019. Primary outcomes were the proportion of patients requiring anastomotic dilation and the dilation rate per year. Univariate analyses of patients undergoing dilation by various risk factors were performed with nonparametric tests, and multivariate analyses of the dilation rate were conducted using generalized linear models. RESULTS: Of 391 included patients, 431 dilations were performed on 135 patients (34.5%, 3.2 dilations per patient who required at least 1 per patient). One complication occurred after dilation. Comorbidities, tumor histology, and tumor stage were not significantly associated with stricture. Three-field MIE was associated with a higher percentage of patients undergoing dilation (48.9% vs 27.1%, P < .001) and a higher rate of dilations (0.944 vs 0.441 dilations per year, P = .007) than 2-field MIE, and this association remained significant after controlling for covariates. When accounting for surgeon variability, this difference was no longer significant. Among patients with 1 or more dilations, those receiving dilation within 100 days of surgery needed more subsequent dilations (2.0 vs 0.6 dilations per year, P < .001). CONCLUSIONS: After controlling for multiple variables, a 3-field MIE approach was associated with a higher rate of repeat dilations in patients undergoing MIE. A shorter interval between esophagectomy and initial dilation is strongly associated with the need for repeated dilations.

Comparative performance of between-population vaccine allocation strategies with applications for emerging pandemics.

Vaccine allocation decisions during emerging pandemics have proven to be challenging due to competing ethical, practical, and political considerations. Complicating decision making, policy makers need to consider vaccine allocation strategies that balance needs both within and between populations. Due to limited vaccine stockpiles, vaccine doses should be allocated in locations where their impact will be maximized. Using a susceptible-exposed-infectious-recovered (SEIR) model we examine optimal vaccine allocation decisions across two populations considering the impact of population size, underlying immunity, continuous vaccine roll-out, heterogeneous population risk structure, and differences in disease transmissibility. We find that in the context of an emerging pathogen where many epidemiologic characteristics might not be known, equal vaccine allocation between populations performs optimally in most scenarios. In the specific case considering heterogeneous population risk structure, first targeting individuals at higher risk of transmission or death due to infection leads to equal resource allocation across populations.

Perfect as the enemy of good: tracing transmissions with low-sensitivity tests to mitigate SARS-CoV-2 outbreaks.

Throughout the COVID-19 pandemic, governments and individuals have attempted a wide variety of strategies to limit the damage of the pandemic on human lives, population health, and economies. Contact tracing has been a commonly used strategy, and various approaches have been proposed and attempted. We summarise some methods of contact tracing and testing, considering the resources demanded by each and how features of SARS-CoV-2 transmission affect their effectiveness. We also propose an approach focusing on tracing transmission events, which can be particularly effective when superspreading events play a large role in transmission. Accounting for the best available evidence on a pathogen and for the availability of resources can make control strategies more effective, even if they are not perfect.

Estimating epidemiologic dynamics from cross-sectional viral load distributions.

Estimating an epidemic's trajectory is crucial for developing public health responses to infectious diseases, but case data used for such estimation are confounded by variable testing practices. We show that the population distribution of viral loads observed under random or symptom-based surveillance-in the form of cycle threshold (Ct) values obtained from reverse transcription quantitative polymerase chain reaction testing-changes during an epidemic. Thus, Ct values from even limited numbers of random samples can provide improved estimates of an epidemic's trajectory. Combining data from multiple such samples improves the precision and robustness of this estimation. We apply our methods to Ct values from surveillance conducted during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic in a variety of settings and offer alternative approaches for real-time estimates of epidemic trajectories for outbreak management and response.

Estimating epidemiologic dynamics from cross-sectional viral load distributions.

Estimating an epidemic's trajectory is crucial for developing public health responses to infectious diseases, but incidence data used for such estimation are confounded by variable testing practices. We show instead that the population distribution of viral loads observed under random or symptom-based surveillance, in the form of cycle threshold (Ct) values, changes during an epidemic and that Ct values from even limited numbers of random samples can provide improved estimates of an epidemic's trajectory. Combining multiple such samples and the fraction positive improves the precision and robustness of such estimation. We apply our methods to Ct values from surveillance conducted during the SARS-CoV-2 pandemic in a variety of settings and demonstrate new approaches for real-time estimates of epidemic trajectories for outbreak management and response.

Statistical Properties of Stepped Wedge Cluster-Randomized Trials in Infectious Disease Outbreaks.

Randomized controlled trials are crucial for the evaluation of interventions such as vaccinations, but the design and analysis of these studies during infectious disease outbreaks is complicated by statistical, ethical, and logistical factors. Attempts to resolve these complexities have led to the proposal of a variety of trial designs, including individual randomization and several types of cluster randomization designs: parallel-arm, ring vaccination, and stepped wedge designs. Because of the strong time trends present in infectious disease incidence, however, methods generally used to analyze stepped wedge trials may not perform well in these settings. Using simulated outbreaks, we evaluate various designs and analysis methods, including recently proposed methods for analyzing stepped wedge trials, to determine the statistical properties of these methods. While new methods for analyzing stepped wedge trials can provide some improvement over previous methods, we find that they still lag behind parallel-arm cluster-randomized trials and individually-randomized trials in achieving adequate power to detect intervention effects. We also find that these methods are highly sensitive to the weighting of effect estimates across time periods. Despite the value of new methods, stepped wedge trials still have statistical disadvantages compared to other trial designs in epidemic settings.