Quasi-experimental methods for pharmacoepidemiology: difference-in-differences and synthetic control methods with case studies for vaccine evaluation.

Difference-in-differences and synthetic control methods have become common study designs for evaluating the effects of changes in policies, including health policies. They also have potential for providing real-world effectiveness and safety evidence in pharmacoepidemiology. To effectively add to the toolkit of the field, however, designs-including both their benefits and drawbacks-must be well understood. Quasi-experimental designs provide an opportunity to estimate the average treatment effect on the treated without requiring the measurement of all possible confounding factors, and to assess population-level effects. This requires, however, other key assumptions, including the parallel trends or stable weighting assumptions, a lack of other concurrent events that could alter time trends, and an absence of contamination between exposed and unexposed units. The targeted estimands are also highly specific to the settings of the study, and combining across units or time periods can be challenging. Case studies are presented for 3 vaccine evaluation studies, showcasing some of these challenges and opportunities in a specific field of pharmacoepidemiology. These methods provide feasible and valuable sources of evidence in various pharmacoepidemiologic settings and can be improved through research to identify and weigh the advantages and disadvantages in those settings. This article is part of a Special Collection on Pharmacoepidemiology.

Designing Difference-in-Difference Studies with Staggered Treatment Adoption: Key Concepts and Practical Guidelines.

Difference-in-difference (DID) estimators are a valuable method for identifying causal effects in the public health researcher's toolkit. A growing methods literature points out potential problems with DID estimators when treatment is staggered in adoption and varies with time. Despite this, no practical guide exists for addressing these new critiques in public health research. We illustrate these new DID concepts with step-by-step examples, code, and a checklist. We draw insights by comparing the simple 2 × 2 DID design (single treatment group, single control group, two time periods) with more complex cases: additional treated groups, additional time periods of treatment, and treatment effects possibly varying over time. We outline newly uncovered threats to causal interpretation of DID estimates and the solutions the literature has proposed, relying on a decomposition that shows how the more complex DIDs are an average of simpler 2 × 2 DID subexperiments.

Residential relocation to assess impact of changes in the living environment on cardio-respiratory health: A narrative literature review with considerations for exposome research.

Residential relocation studies have become increasingly valuable tools for evaluating the effects of changing living environments on human health, but little is known about their application to multiple aspects of the living environment and the most appropriate methodology. This narrative review explores the utility of residential relocation as a natural experiment for studying the impact of changing urban exposures on cardio-metabolic health in high-income settings. It provides a comprehensive overview of the use of residential relocation studies, evaluates their methodological approaches, and synthesizes findings related to health behaviors and cardio-metabolic outcomes. Our search identified 43 relevant studies published between January 1995 and February 2023, from eight countries, predominantly the USA, Canada, and Australia. The majority of eligible studies were published between 2012 and 2021 and examined changes in various domains of the living environment, such as walkability, the built and social environments, but rarely combinations of exposures. Included studies displayed heterogeneity in design and outcomes, 25 involving only movers and 18 considering both movers and non-movers. To mitigate the issue of residential self-selection bias, most studies employed a "change-in-change" design and adjusted for baseline covariates but only a fraction of them accounted for time-varying confounding. Relocation causes simultaneous changes in various features of the living environment, which presents an opportunity for exposome research to establish causal relationships, using large datasets with increased statistical power and a wide range of health outcomes, behaviors and biomarkers. Residential relocation is not a random process. Thus, studies focusing on living environment characteristics need to carefully select time-varying covariates and reference group. Overall, this review informs future research by guiding choices in study design, data requirements, and statistical methodologies. Ultimately, it contributes to the advancement of the urban exposome field and enhances our understanding of the complex relationship between urban environments and human health.

The social life of natural experiments in epidemiology and public health.

Over the twentieth century, the concept of the natural experiment has become increasingly prominent across a variety of disciplines, albeit most consequentially in epidemiology and public health. Drawing on an analysis of the scientific and medical literature, we explore the social life of the natural experiment, tracing its changing use, meaning and uptake to better understand the work done by the concept. We demonstrate how the natural experiment became central to the identity of post-war epidemiology as the discipline professionalised, turned its attention to the prevention of chronic disease and took centre stage in the field of public health. We then turn to its growing significance amid the rise of evidence-based medicine, and the new meanings natural experiments came to take on in the context of concerns about policy and evidence. Finally, we turn to the newest iteration of the natural experiment in the COVID-19 era, which saw an explosion of studies drawing on the term, albeit in ways that reveal more about the underlying politics of health than the method itself. Throughout, we illustrate that the concept of the natural experiment has always been fundamentally social and political and tied to disciplinary claims-making about evidence and what should count as such.

The American Journal of Epidemiology and Social Epidemiology: Past and Future.

Historically, the American Journal of Epidemiology (AJE) has played an important role in the development of social epidemiology as a field. To mark the centennial of the Journal, we invited commentaries from 3 sets of authors in which we asked them to reflect on the role of the AJE in advancing our understanding of the social determinants of population health and health disparities. Drs. Lisa Berkman, Mauricio Avendano, and Emilie Courtin discuss the promises and pitfalls of using experimental data (from both randomized trials and natural policy experiments) to interrogate the causal connection between social determinants and population health outcomes. Drs. Julie Palmer, Yvette Crozier, and Lynn Rosenberg look back on the first 25 years of the Black Women's Health Study and the contributions of that study to our understanding of racial disparities in health. Drs. Luisa Borrell and Natalie Crawford review the AJE's contributions to the definition and use of race and ethnicity in epidemiologic research and make recommendations for strengthening diversity and inclusion in the profession. A common thread that emerges across the invited essays is the AJE's commitment to publishing articles that engage in critical self-reflection of the discipline.

Invited Commentary: Use of Epidemiologic Methods to Guide Comprehensive and Equitable Approaches to Policy.

Despite a dramatic reduction in the prevalence of commercial cigarette smoking in the United States, children are still commonly exposed to secondhand smoke (SHS), which is a cause of various pediatric health problems. Further, SHS exposure is patterned by race and class, exacting an inequitable toll on children from families with lesser social and economic advantage. In this issue of the Journal, Titus et al. (Am J Epidemiol. 2023;192(1):25-33) use natural experiment evaluation methods (difference-in-differences) to test whether the recently implemented US Department of Housing and Urban Development policy that forbade smoking in and around New York City Housing Authority buildings affected child respiratory health. The results from their work remind us that policies do not always impact outcomes as we might expect. Given that policy is one of the most potent tools for population health promotion, this work underlines the need for epidemiologists to engage in policy evaluation at all stages of the policy life cycle, in order to discover comprehensive approaches to policy development and implementation that prioritize equity and address structural racism.

A guide to regression discontinuity designs in medical applications.

We present a practical guide for the analysis of regression discontinuity (RD) designs in biomedical contexts. We begin by introducing key concepts, assumptions, and estimands within both the continuity-based framework and the local randomization framework. We then discuss modern estimation and inference methods within both frameworks, including approaches for bandwidth or local neighborhood selection, optimal treatment effect point estimation, and robust bias-corrected inference methods for uncertainty quantification. We also overview empirical falsification tests that can be used to support key assumptions. Our discussion focuses on two particular features that are relevant in biomedical research: (i) fuzzy RD designs, which often arise when therapeutic treatments are based on clinical guidelines, but patients with scores near the cutoff are treated contrary to the assignment rule; and (ii) RD designs with discrete scores, which are ubiquitous in biomedical applications. We illustrate our discussion with three empirical applications: the effect CD4 guidelines for anti-retroviral therapy on retention of HIV patients in South Africa, the effect of genetic guidelines for chemotherapy on breast cancer recurrence in the United States, and the effects of age-based patient cost-sharing on healthcare utilization in Taiwan. Complete replication materials employing publicly available data and statistical software in Python, R and Stata are provided, offering researchers all necessary tools to conduct an RD analysis.

Evaluating the impact of marketing interventions on sugar-free and sugar-sweetened soft drink sales and sugar purchases in a fast-food restaurant setting.

BACKGROUND: Beverages high in added sugar, such as sugar-sweetened soft drinks, continue to be associated with various health issues. This study examines the effects of a manufacturer-initiated multicomponent intervention on the sales of sugar-free (SFD) and sugar-sweetened (SSD) soft drinks and the amount of sugar people purchase from soft drinks in a fast-food restaurant setting. METHODS: A database of monthly sales data of soft drinks from January 2016 to December 2018 was obtained from three treatment and three control fast-food restaurants. A multicomponent intervention consisting of free coupons, point-of-purchase displays, a menu board, and two sugar-free replacements for sugar-sweetened soft drinks was introduced in August 2018 for five months in Western Sydney, Australia. A retrospective interrupted time series analysis was used to model the data and examine the effects of the interventions on SFD and SSD sales and their consequential impact on sugar purchases from soft drinks. The analyses were carried out for volume sales in litres and sugar in grams per millilitre of soft drinks sales. A comparison of these measures within the treatment site (pre- and post-intervention) and between sites (treatment and control) was conducted. RESULTS: The interventions had a statistically significant impact on SFDs but not SSDs. On average, SFD sales in the treatment site were 56.75% higher than in the control site. Although SSD sales were lower in the treatment site, the difference with the control site was not statistically significant. The net reduction of 6.34% in the amount of sugar purchased from soft drinks between sites during the experimental period was attributed to the interventions. CONCLUSIONS: The interventions significantly increased SFD sales and reduced sugar purchases in the short run. Aside from free coupons, the findings support the recommendation for fast food restaurants to nudge customers towards choosing SFDs through point-of-purchase displays and the replacement of popular SSDs with their SFD counterparts.

Evolution of Risk Aversion over Five Years after a Major Natural Disaster.

The impact of exposure to a major unanticipated natural disaster on the evolution of survivors' attitudes toward risk is examined, exploiting plausibly exogenous variation in exposure to the 2004 Indian Ocean tsunami in combination with rich population-representative longitudinal survey data spanning the five years after the tsunami. Respondents chose among pairs of hypothetical income streams. Those directly exposed to the tsunami made choices consistent with greater willingness to take on risk relative to those not directly exposed to the tsunami. These differences are short-lived: starting a year later, there is no evidence of differences in willingness to take on risk between the two groups. These conclusions hold for tsunami-related exposures measured at the individual and community level. Apparently, tsunami survivors were inclined to assume greater financial risk in the short-term while rebuilding their lives after the disaster.

The Influence of Front-of-Package Nutrition Labeling on Consumer Behavior and Product Reformulation.

Countries worldwide have implemented mandatory or voluntary front-of-package nutrition labeling systems. We provide a narrative review of (a) real-world evaluations of front-of-package nutrition labels that analyze objective sales data and (b) studies that objectively assess product reformulation in response to a front-of-package nutrition label implementation. We argue that there is sufficient scientific evidence to recommend that governments implement mandatory front-of-package nutrition labeling systems to improvepopulation health. We also present a conceptual framework to describe front-of-package label influence and provide recommendations for the optimal label design, emphasizing that labeling systems should be highly visible and salient, be simple and easy to understand, leverage automatic associations, and integrate informational and emotional messaging. The existing research suggests that Guideline Daily Amount labels should be avoided and that the Health Star Rating and Nutri-Score systems are promising but that systems with warning labels like the one in Chile are likely to produce the largest public health benefits.

Evaluating Natural Experiments that Impact the Diabetes Epidemic: an Introduction to the NEXT-D3 Network.

PURPOSE OF REVIEW: Diabetes is an ongoing public health issue in the USA, and, despite progress, recent reports suggest acute and chronic diabetes complications are increasing. RECENT FINDINGS: The Natural Experiments for Translation in Diabetes 3.0 (NEXT-D3) Network is a 5-year research collaboration involving six academic centers (Harvard University, Northwestern University, Oregon Health & Science University, Tulane University, University of California Los Angeles, and University of California San Francisco) and two funding agencies (Centers for Disease Control and Prevention and National Institutes of Health) to address the gaps leading to persisting diabetes burdens. The network builds on previously funded networks, expanding to include type 2 diabetes (T2D) prevention and an emphasis on health equity. NEXT-D3 researchers use rigorous natural experiment study designs to evaluate impacts of naturally occurring programs and policies, with a focus on diabetes-related outcomes. NEXT-D3 projects address whether and to what extent federal or state legislative policies and health plan innovations affect T2D risk and diabetes treatment and outcomes in the USA; real-world effects of increased access to health insurance under the Affordable Care Act; and the effectiveness of interventions that reduce barriers to medication access (e.g., decreased or eliminated cost sharing for cardiometabolic medications and new medications such as SGLT-2 inhibitors for Medicaid patients). Overarching goals include (1) expanding generalizable knowledge about policies and programs to manage or prevent T2D and educate decision-makers and organizations and (2) generating evidence to guide the development of health equity goals to reduce disparities in T2D-related risk factors, treatment, and complications.

Does Prolonged Education Causally Affect Dementia Risk When Adult Socioeconomic Status Is Not Altered? A Swedish Natural Experiment in 1.3 Million Individuals.

Intervening on modifiable risk factors to prevent dementia is of key importance, since progress-modifying treatments are not currently available. Education is inversely associated with dementia risk, but causality and mechanistic pathways remain unclear. We aimed to examine the causality of this relationship in Sweden using, as a natural experiment, data on a compulsory schooling reform that extended primary education by 1 year for 70% of the population between 1936 and 1949. The reform introduced substantial exogenous variation in education that was unrelated to pupils' characteristics. We followed 18 birth cohorts (n = 1,341,842) from 1985 to 2016 (up to ages 79-96 years) for a dementia diagnosis in the National Inpatient and Cause of Death registers and fitted Cox survival models with stratified baseline hazards at the school-district level, chronological age as the time scale, and cohort indicators. Analyses indicated very small or negligible causal effects of education on dementia risk (main hazard ratio = 1.01, 95% confidence interval: 0.98, 1.04). Multiple sensitivity checks considering only compliers, the pre-/post- design, differences in health-care-seeking behavior, and the impact of exposure misclassification left the results essentially unaltered. The reform had limited effects on further adult socioeconomic outcomes, such as income. Our findings suggest that without mediation through adult socioeconomic position, education cannot be uncritically considered a modifiable risk factor for dementia.

Conceptualising natural and quasi experiments in public health.

BACKGROUND: Natural or quasi experiments are appealing for public health research because they enable the evaluation of events or interventions that are difficult or impossible to manipulate experimentally, such as many policy and health system reforms. However, there remains ambiguity in the literature about their definition and how they differ from randomized controlled experiments and from other observational designs. We conceptualise natural experiments in the context of public health evaluations and align the study design to the Target Trial Framework. METHODS: A literature search was conducted, and key methodological papers were used to develop this work. Peer-reviewed papers were supplemented by grey literature. RESULTS: Natural experiment studies (NES) combine features of experiments and non-experiments. They differ from planned experiments, such as randomized controlled trials, in that exposure allocation is not controlled by researchers. They differ from other observational designs in that they evaluate the impact of events or process that leads to differences in exposure. As a result they are, in theory, less susceptible to bias than other observational study designs. Importantly, causal inference relies heavily on the assumption that exposure allocation can be considered 'as-if randomized'. The target trial framework provides a systematic basis for evaluating this assumption and the other design elements that underpin the causal claims that can be made from NES. CONCLUSIONS: NES should be considered a type of study design rather than a set of tools for analyses of non-randomized interventions. Alignment of NES to the Target Trial framework will clarify the strength of evidence underpinning claims about the effectiveness of public health interventions.

Linkages Between Air Pollution and the Health Burden From COVID-19: Methodological Challenges and Opportunities.

The coronavirus disease 2019 (COVID-19) pandemic revealed and exacerbated existing social and economic health disparities, and actionable epidemiologic evidence is needed to identify potential vulnerability factors to help inform targeted responses. In this commentary, methodological challenges and opportunities regarding the links between air pollution and COVID-19 are discussed with a focus on 2 factors: 1) the role of differential exposure to air pollution across populations as an explanation for spatiotemporal variability of the epidemic spread and resultant mortality; and 2) the indirect impacts of interventions to control COVID-19 person-to-person spread treated as natural experiments on air pollution and population health. I first discuss the potential mechanisms between exposure to air pollution and COVID-19 and the opportunity to clearly formulate causal questions of interest through the target trial framework. Then, I discuss challenges regarding the use of quasiexperimental designs that capitalize on the differential timing of COVID-19 policies including the selection of control groups and potential violations of the common shock assumption. Finally, I discuss environmental justice implications of this many-headed beast of a crisis.

Postdisaster Changes in Social Capital and Mental Health: A Natural Experiment From the 2016 Kumamoto Earthquake.

Levels of social capital can change after a natural disaster; thus far, no study has examined how changes in social capital affect the mental health of disaster victims. In this study, we examined how predisaster social capital and its changes after a disaster were associated with the onset of mental disorders. In October 2013, we mailed a questionnaire to participants in the Japan Gerontological Evaluation Study living in Mifune, a town in Kumamoto Prefecture, Japan, and measured predisaster social capital. In April 2016, the Kumamoto earthquake struck the region. Three years after the baseline survey, postdisaster social capital and symptoms of mental disorders were measured using the Screening Questionnaire for Disaster Mental Health (n = 828). Multiple Poisson regression indicated that a 1-standard-deviation change in predisaster social cohesion at the community level reduced the risk of depression among women (relative risk = 0.44, 95% confidence interval: 0.24, 0.78); a decline in social capital after the disaster elevated the risk (relative risk = 2.44, 95% confidence interval: 1.33, 4.47). In contrast to social cohesion, high levels of social participation at the community level were positively associated with the risk of depression among women. Policy-makers should pay attention to sex differences and types of social capital when leveraging social capital for recovery from disasters.

Evaluating a Key Instrumental Variable Assumption Using Randomization Tests.

Instrumental variable (IV) analyses are becoming common in health services research and epidemiology. Most IV analyses use naturally occurring instruments, such as distance to a hospital. In these analyses, investigators must assume that the instrument is as-if randomly assigned. This assumption cannot be tested directly, but it can be falsified. Most IV falsification tests compare relative prevalence or bias in observed covariates between the instrument and exposure. These tests require investigators to make covariate-by-covariate judgments about the validity of the IV design. Often, only some covariates are well-balanced, making it unclear whether as-if randomization can be assumed for the instrument. We propose an alternative falsification test that compares IV balance or bias with the balance or bias that would have been produced under randomization. A key advantage of our test is that it allows for global balance measures as well as easily interpretable graphical comparisons. Furthermore, our test does not rely on parametric assumptions and can be used to validly assess whether the instrument is significantly closer to being as-if randomized than the exposure. We demonstrate our approach using data from (SPOT)light, a prospective cohort study carried out in 48 National Health Service hospitals in the United Kingdom between November 1, 2010, and December 31, 2011. This study used bed availability in the intensive care unit as an instrument for admission to the intensive care unit.

Accounting for Confounding in Observational Studies.

The goal of this review is to enable clinical psychology researchers to more rigorously test competing hypotheses when studying risk factors in observational studies. We argue that there is a critical need for researchers to leverage recent advances in epidemiology/biostatistics related to causal inference and to use innovative approaches to address a key limitation of observational research: the need to account for confounding. We first review theoretical issues related to the study of causation, how causal diagrams can facilitate the identification and testing of competing hypotheses, and the current limitations of observational research in the field. We then describe two broad approaches that help account for confounding: analytic approaches that account for measured traits and designs that account for unmeasured factors. We provide descriptions of several such approaches and highlight their strengths and limitations, particularly as they relate to the etiology and treatment of behavioral health problems.

Using natural experiments to improve public health evidence: a review of context and utility for obesity prevention.

BACKGROUND: Natural experiments are increasingly valued as a way to assess the health impact of health and non-health interventions when planned controlled experimental research designs may be infeasible or inappropriate to implement. This study sought to investigate the value of natural experiments by exploring how they have been used in practice. The study focused on obesity prevention research as one complex programme area for applying natural experiment studies. METHODS: A literature search sought obesity prevention research from January 1997 to December 2017 and identified 46 population health studies that self-described as a natural experiment. RESULTS: The majority of studies identified were published in the last 5 years, illustrating a more recent adoption of such opportunities. The majority of studies were evaluations of the impact of policies (n = 19), such as assessing changes to food labelling, food advertising or taxation on diet and obesity outcomes, or were built environment interventions (n = 17), such as the impact of built infrastructure on physical activity or access to healthy food. Research designs included quasi-experimental, pre-experimental and non-experimental methods. Few studies applied rigorous research designs to establish stronger causal inference, such as multiple pre/post measures, time series designs or comparison of change against an unexposed group. In general, researchers employed techniques to enhance the study utility but often were limited in the use of more rigorous study designs by ethical considerations and/or the particular context of the intervention. CONCLUSION: Greater recognition of the utility and versatility of natural experiments in generating evidence for complex health issues like obesity prevention is needed. This review suggests that natural experiments may be underutilised as an approach for providing evidence of the effects of interventions, particularly for evaluating health outcomes of interventions when unexpected opportunities to gather evidence arise.

Do natural experiments have an important future in the study of mental disorders?

There is an enormous interest in identifying the causes of psychiatric disorders but there are considerable challenges in identifying which risks are genuinely causal. Traditionally risk factors have been inferred from observational designs. However, association with psychiatric outcome does not equate to causation. There are a number of threats that clinicians and researchers face in making causal inferences from traditional observational designs because adversities or exposures are not randomly allocated to individuals. Natural experiments provide an alternative strategy to randomized controlled trials as they take advantage of situations whereby links between exposure and other variables are separated by naturally occurring events or situations. In this review, we describe a growing range of different types of natural experiment and highlight that there is a greater confidence about findings where there is a convergence of findings across different designs. For example, exposure to hostile parenting is consistently found to be associated with conduct problems using different natural experiment designs providing support for this being a causal risk factor. Different genetically informative designs have repeatedly found that exposure to negative life events and being bullied are linked to later depression. However, for exposure to prenatal cigarette smoking, while findings from natural experiment designs are consistent with a causal effect on offspring lower birth weight, they do not support the hypothesis that intra-uterine cigarette smoking has a causal effect on attention-deficit/hyperactivity disorder and conduct problems and emerging findings highlight caution about inferring causal effects on bipolar disorder and schizophrenia.

A systematic review of the effect of infrastructural interventions to promote cycling: strengthening causal inference from observational data.

BACKGROUND: Previous reviews have suggested that infrastructural interventions can be effective in promoting cycling. Given inherent methodological complexities in the evaluation of such changes, it is important to understand whether study results obtained depend on the study design and methods used, and to describe the implications of the methods used for causality. The aims of this systematic review were to summarize the effects obtained in studies that used a wide range of study designs to assess the effects of infrastructural interventions on cycling and physical activity, and whether the effects varied by study design, data collection methods, or statistical approaches. METHODS: Six databases were searched for studies that evaluated infrastructural interventions to promote cycling in adult populations, such as the opening of cycling lanes, or the expansion of a city-wide cycling network. Controlled and uncontrolled studies that presented data before and after the intervention were included. No language or date restrictions were applied. Data was extracted for any outcome presented (e.g. bikes counted on the new infrastructure, making a bike trip, cycling frequency, cycling duration), and for any purpose of cycling (e.g. total cycling, recreational cycling, cycling for commuting). Data for physical activity outcomes and equity effects was extracted, and quality assessment was conducted following previous methodologies and the UK Medical Research Council guidance on natural experiments. The PROGRESS-Plus framework was used to describe the impact on subgroups of the population. Studies were categorized by outcome, i.e. changes in cycling behavior, or usage of the cycling infrastructure. The relative change was calculated to derive a common outcome across various metrics and cycling purposes. The median relative change was presented to evaluate whether effects differed by methodological aspects. RESULTS: The review included 31 studies and all were conducted within urban areas in high-income countries. Most of the evaluations found changes in favor of the intervention, showing that the number of cyclists using the facilities increased (median relative change compared to baseline: 62%; range: 4 to 438%), and to a lesser extent that cycling behavior increased (median relative change compared to baseline: 22%; range: - 21 to 262%). Studies that tested for statistical significance and studies that used subjective measurement methods (such as surveys and direct observations of cyclists) found larger changes than those that did not perform statistical tests, and those that used objective measurement methods (such as GPS and accelerometers, and automatic counting stations). Seven studies provided information on changes of physical activity behaviors, and findings were mixed. Three studies tested for equity effects following the opening of cycling infrastructure. CONCLUSIONS: Study findings of natural experiments evaluating infrastructural interventions to promote cycling depended on the methods used and the approach to analysis. Studies measuring cycling behavior were more likely to assess actual behavioral change that is most relevant for population health, as compared to studies that measured the use of cycling infrastructure. Triangulation of methods is warranted to overcome potential issues that one may encounter when evaluating environmental changes within the built environment. TRIAL REGISTRATION: The protocol of this study was registered at PROSPERO (CRD42018091079).