Quasi-experimental methods for pharmacoepidemiology: difference-in-differences and synthetic control methods with case studies for vaccine evaluation.

Difference-in-differences and synthetic control methods have become common study designs for evaluating the effects of changes in policies, including health policies. They also have potential for providing real-world effectiveness and safety evidence in pharmacoepidemiology. To effectively add to the toolkit of the field, however, designs-including both their benefits and drawbacks-must be well understood. Quasi-experimental designs provide an opportunity to estimate the average treatment effect on the treated without requiring the measurement of all possible confounding factors, and to assess population-level effects. This requires, however, other key assumptions, including the parallel trends or stable weighting assumptions, a lack of other concurrent events that could alter time trends, and an absence of contamination between exposed and unexposed units. The targeted estimands are also highly specific to the settings of the study, and combining across units or time periods can be challenging. Case studies are presented for 3 vaccine evaluation studies, showcasing some of these challenges and opportunities in a specific field of pharmacoepidemiology. These methods provide feasible and valuable sources of evidence in various pharmacoepidemiologic settings and can be improved through research to identify and weigh the advantages and disadvantages in those settings. This article is part of a Special Collection on Pharmacoepidemiology.

The Impact of Minimum Wage Policy on Alcohol Use Disorder: A Quasi-Experimental Study in South Korea.

South Korea's 2018 minimum wage hike was examined for its impact on potential alcohol use disorders among affected individuals, using data from the Korea Welfare Panel Study (2015-2019). The study sample was restricted to workers aged 19-64 employed over the study years. The treatment group was identified as those below minimum wages, and the control group as those earning more than minimum wages in 2016-2017 (n=3,117 control, n=578 treatment). Using outcomes derived from the Alcohol Use Disorders Identification Test, our results from difference-in-differences models showed that the 2018 wage hike was linked to a 1.9% increase in the 'high risk' of alcohol use disorder and a 3.6% rise in hazardous consumption in the treatment group. Notably, the effects were more pronounced among men and those aged 50-64. Additionally, we confirmed that the spillover effects extended to workers earning up to 20% above the minimum wage. This study underscores the unintended substance use risk of minimum wage policies in the East Asian context. As wage policies are implemented, integrated public health campaigns targeting at-risk groups are required.

Employment outcomes of adolescent and young adult cancer survivors and their partners: A Dutch population-based study.

BACKGROUND: The aim of this population-based registry study was to examine the impact of cancer on employment outcomes in adolescent and young adult (AYA) survivors and their partners and associated sociodemographic and clinical characteristics. METHODS: A total of 2456 AYA cancer patients, diagnosed in 2013 and aged 18 through 39 years old, were selected from the Netherlands Cancer Registry and linked to employment data from Statistics Netherlands, from which 1252 partners of AYAs could be identified. For both patients and their partners, a control group with same age, migration background, and sex was selected. The impact (i.e., causal effect) was estimated by implementing a doubly robust difference-in-differences method, from 3 years before to 5 years after cancer diagnosis. RESULTS: Patients suffered a reduced employment probability (3.8 percentage points) and number of hours worked when employed (3.8%). This effect was larger for females, and individuals with a migration background, high tumor stage, or diagnosed with a central nervous system tumor/hematologic malignancy. In regard to employment, no significant effect could be found for the patients' partners, although a 5.5 percentage-point increase in employment probability was found in partners who were either unemployed or worked fewer than 400 hours. CONCLUSIONS: A cancer diagnosis significantly affects employment outcomes of AYA patients with cancer. Patients at risk should have access to services such as job counseling to help them return into society in the best possible way. No objective impact on partners' employment outcomes was found; however, subjective well-being was not taken into account. PLAIN LANGUAGE SUMMARY: This study estimated the causal effect of a cancer diagnosis on employment outcomes. Adolescent and young adult cancer survivors face a reduction in both employment probability and the number of hours worked when employed. Partners that were unemployed or worked the least number of hours a year before diagnosis had a 5.5 percentage-point increased employment probability, but for other partners effects are small.

Designing Difference-in-Difference Studies with Staggered Treatment Adoption: Key Concepts and Practical Guidelines.

Difference-in-difference (DID) estimators are a valuable method for identifying causal effects in the public health researcher's toolkit. A growing methods literature points out potential problems with DID estimators when treatment is staggered in adoption and varies with time. Despite this, no practical guide exists for addressing these new critiques in public health research. We illustrate these new DID concepts with step-by-step examples, code, and a checklist. We draw insights by comparing the simple 2 × 2 DID design (single treatment group, single control group, two time periods) with more complex cases: additional treated groups, additional time periods of treatment, and treatment effects possibly varying over time. We outline newly uncovered threats to causal interpretation of DID estimates and the solutions the literature has proposed, relying on a decomposition that shows how the more complex DIDs are an average of simpler 2 × 2 DID subexperiments.

Changes in Medication Utilization and Adherence Associated with Homeless Adults' Entry into Permanent Supportive Housing.

BACKGROUND: Permanent supportive housing (PSH) programs, which have grown over the last decade, have been associated with changes in health care utilization and spending. However, little is known about the impact of such programs on use of prescription drugs critical for managing chronic diseases prevalent among those with unstable housing. OBJECTIVE: To evaluate the effects of PSH on medication utilization and adherence among Medicaid enrollees in Pennsylvania. DESIGN: Difference-in-differences study comparing medication utilization and adherence between PSH participants and a matched comparison cohort from 7 to 18 months before PSH entry to 12 months post PSH entry. SUBJECTS: Pennsylvania Medicaid enrollees (n = 1375) who entered PSH during 2011-2016, and a propensity-matched comparison cohort of 5405 enrollees experiencing housing instability who did not receive PSH but received other housing services indicative of episodic or chronic homelessness (e.g., emergency shelter stays). MAIN MEASURES: Proportion with prescription fill, mean proportion of days covered (PDC), and percent adherent (PDC ≥ 80%) for antidepressants, antipsychotics, anti-asthmatics, and diabetes medications. KEY RESULTS: The PSH cohort saw a 4.77% (95% CI 2.87% to 6.67%) relative increase in the proportion filling any prescription, compared to the comparison cohort. Percent adherent among antidepressant users in the PSH cohort rose 7.41% (95% CI 0.26% to 14.57%) compared to the comparison cohort. While utilization increased in the other medication classes among the PSH cohort, differences from the comparison cohort were not statistically significant. CONCLUSIONS: PSH participation is associated with increases in filling prescription medications overall and improved adherence to antidepressant medications. These results can inform state and federal policy to increase PSH placement among Medicaid enrollees experiencing homelessness.

The societal impact of individual placement and support implementation on employment outcomes for young adults receiving temporary health-related welfare benefits: a difference-in-differences study.

BACKGROUND: Individual placement and support (IPS) is an evidence-based practice that helps individuals with mental illness gain and retain employment. IPS was implemented for young adults at a municipality level through a cross-sectoral collaboration between specialist mental healthcare, primary mental healthcare, and the government funded employment service (NAV). We investigated whether IPS implementation had a causal effect on employment outcomes for all young adults in receipt of a temporary health-related rehabilitation (work assessment allowance, WAA) welfare benefit, measured at the societal level compared to municipalities that did not implement IPS. METHOD: We used a difference in differences design to estimate the effects of IPS implementation on the outcome of workdays per year using longitudinal registry data. We estimate the average effect of being exposed to IPS implementation during four-years of implementation compared to ten control municipalities without IPS for all WAA recipients. RESULTS: We found a significant, positive, causal effect on societal level employment outcomes of 5.6 (p = 0.001, 95% CI 2.7-8.4) increased workdays per year per individual, equivalent to 12.7 years of increased work in the municipality where IPS was implemented compared to municipalities without IPS. Three years after initial exposure to IPS implementation individuals worked, on average, 10.5 more days per year equating to 23.8 years of increased work. CONCLUSIONS: Implementing IPS as a cross sectoral collaboration at a municipality level has a significant, positive, causal, societal impact on employment outcomes for all young adults in receipt of a temporary health-related rehabilitation welfare benefit.

Do later school start times improve adolescents' sleep and substance use? A quasi-experimental study.

OBJECTIVE: A later school start time policy has been recommended as a solution to adolescents' sleep deprivation. We estimated the impacts of later school start times on adolescents' sleep and substance use by leveraging a quasi-experiment in which school start time was delayed in some regions in South Korea. METHODS: A later school start time policy was implemented in 2014 and 2015, which delayed school start times by approximately 30-90 minutes. We applied difference-in-differences and event-study designs to longitudinal data on a nationally representative cohort of adolescents from 2010 to 2015, which annually tracked sleep and substance use of 1133 adolescents from grade 7 through grade 12. RESULTS: The adoption of a later school start time policy was initially associated with a 19-minute increase in sleep duration (95% CI, 5.52 to 32.04), driven by a delayed wake time and consistent bedtime. The policy was also associated with statistically significant reductions in monthly smoking and drinking frequencies. However, approximately a year after implementation, the observed increase in sleep duration shrank to 7  minutes (95% CI, -12.60 to 25.86) and became statistically nonsignificant. Similarly, the observed reduction in smoking and drinking was attenuated a year after. CONCLUSIONS: Our findings suggest that policies that increase sleep in adolescents may have positive effects on health behaviors, but additional efforts may be required to sustain positive impacts over time. Physicians and education and health policymakers should consider the long-term effects of later school start times on adolescent health and well-being.

Carbon Abatement and Leakage in China's Regional Carbon Emission Trading.

Emission trading schemes (ETS) are increasingly becoming a popular policy instrument to balance carbon abatement and economic growth. As a globally unified carbon pricing system has not yet been established, whether regionally operated ETSs cause carbon leakage remains a major concern. Taking China's regional pilot ETSs as a quasi-natural experiment, the study uses the spatial difference-in-differences method to examine how regional ETSs affect carbon emissions in and outside cities of policy implementation. Our analysis finds that China's regional ETS policy contributes to a 6.1% reduction in urban CO2 emissions and a 6.6% decline in emissions intensity in regulated cities, causing carbon leakages that increase CO2 emissions in neighboring cities by 1.7% on average. Our finding further suggests that regional ETSs mitigate local CO2 emissions through outsourcing production, improving energy efficiency and decarbonizing energy structure, whereas the outsourcing of industrial production drives up CO2 emissions in adjacent cities. Moreover, the performances of regional ETSs vary largely by socioeconomic context and mechanism design. China's regional ETSs reduce CO2 emissions more effectively in central and industrial cities but with more severe carbon leakage, while rigorous compliance mechanisms and active market trading help deepen carbon abatement and alleviate carbon leakage.

The impact of the expanded health insurance coverage policy on healthcare spending: evidence from Korea.

BACKGROUND: South Korea's National Health Insurance (NHI) system pursues universal health coverage, but it has not been able to alleviate patients' financial burden owing to limited coverage and a high proportion of out-of-pocket expenses. In 2017, the government announced a plan to strengthen universality by providing coverage for all unincluded services, expanding coverage, and alleviating household financial burden. We aimed to evaluate the effect of "Moon Care" with a focus on changes in health expenditures following policy implementation, and to provide empirical evidence for future policies to strengthen the NHI system's universality. METHODS: Using data from the 2016 and 2018 Korea Health Panel (KHP), we established a treatment group affected by the policy and an unaffected control group; we ensured homogeneity between the groups using propensity score matching (PSM). Subsequently, we examined changes in NHI payments, non-payments, and out-of-pocket payments (OOP); we performed difference-in-differences (DID) analysis to evaluate the policy's effect. RESULTS: Following policy implementation, the control group had a higher increase than the treatment group in all categories of health expenditures, including NHI payments, non-payments, and OOP. We noted significant decreases in all three categories of health expenditures when comparing the differences before and after policy implementation, as well as between the treatment and control groups. However, we witnessed a significant decrease in the interaction term, which confirms the policy's effect, but only for non-payments. CONCLUSIONS: We observed the policy's intervention effect over time as a decrease in non-payments, on the effectivity of remunerating covered medical services. However, the policy did not work for NHI payments and OOP, suggesting that it failed to control the creation of new non-covered services as noncovered services were converted into covered ones. Thus, it is crucial to discuss the financial spending of health insurance regarding the inclusion of non-covered services in the NHI benefits package.

Impact of New York City Cigarette Floor Price Policy on Reducing Smoking Disparities.

INTRODUCTION: In 2017, New York City (NYC) passed a minimum floor price law (MFPL) to raise the minimum price of a pack of cigarettes to $13.00. Evaluation of the MFPL in NYC is limited and has yet to examine its potential as a pro-equity policy. METHODS: Data (n = 20,241; pre-policy n =15,037, post-policy n =5,204) were obtained from the New York State Adult Tobacco Survey, a quarterly repeated cross-sectional survey. Using the Difference-in-Differences approach, we compared changes in reported cigarette prices, cigarette consumption, and smoking status among NYC residents before and after policy implementation to changes in the same outcomes among residents in the rest of the state (ROS) over the same period. RESULTS: For some smokers, cigarette price increased in NYC for post-policy period; moreover, prices increased more in NYC than in ROS. NYC smokers who reported higher income, more education, or White or "Other" race, reported a bigger price increase than their ROS counterparts. Cigarette consumption decreased more in the post-policy period for people in the ROS, in general and among certain groups. Everyday smoking status decreased similarly in both NYC and ROS, whereas someday smoking status decreased primarily in the ROS during the analysis period. CONCLUSIONS: Cigarette prices in NYC increased after the 2017 MFPL; these increases were greater than those occurring elsewhere in the state, suggesting the policy might be a factor in the change. However, the increases were concentrated among relatively higher priced purchases, and groups with lower smoking prevalence. Changes in smoking status and cigarette consumption did not correspond to study hypotheses. IMPLICATIONS: This study provides an empirical analysis of a real-world policy in tobacco control. It examines the potential of the MFPL in New York City as a pro-equity policy. Findings extend the current MFPL literature and suggest that they may be able to raise cigarette prices for some purchases, but also may have a limited impact on smoking behaviors.

Drink then drive away: The effects of lowering the blood alcohol concentration in Utah.

In March of 2017 Utah announced its intent to lower the legal blood alcohol content (BAC) for driving from 0.08 to 0.05 g/dL. However, this change did not take effect until 2019. We employ a difference-in- differences strategy on Utah counties using neighboring states as controls to test whether this policy change significantly affected the number of traffic accidents or the severity of those accidents. Results show the policy appears to temporarily decrease the total number of accidents, limited primarily to property damage- only accidents. We believe these results may be partially explained by drivers who, after the policy is enacted, avoid reporting property damage-only accidents if possible. Using insurance claims data, we show there is no corresponding fall in insurance claims or payouts suggesting that the fall in total accidents likely comes from under-reporting.

Air pollution, viral spread and health outcomes evidence from strikes in France.

To evidence the impact of air pollution on the health of urban populations, several studies use natural experiments that shift commuting from public transport to cars (or vice-versa). However, as public transport use declines, reduced interpersonal contact may lead to slower virus spread and thus lower respiratory morbidity. Using a difference-in-differences strategy, we show that respiratory hospitalisations are both positively affected by air pollution and negatively affected by viral spread following partial unavailability of public transport due to strikes in the ten most populated French cities during the period 2010-2015. Our results are in line with studies in other countries that have found a significant increase in urgent respiratory hospitalisations following a public transport strike, most likely due to car pollution, but we also find a detectable interaction with viral spread, which should not be overlooked when interpreting these studies.

Effects of rural hospital closures on nurse staffing levels and health care utilization at nearby hospitals.

Our study examines the causal effect of rural hospital closures on nearby hospitals' nurse staffing levels and health care utilization. We use data from the 2014-2019 American Hospital Association Survey on nurse staffing level outcomes including licensed practical or vocational nurses (LPNs), registered nurses (RNs), and advanced practice nurses (APNs); and health care utilization outcomes, including inpatient and outpatient surgical operations and emergency department (ED) visits. Using propensity score matching and difference-in-differences (DID) methods, we find that rural hospital closures lead to an average increase of 37.3% in the number of nurses in nearby rural hospitals during the 4 years following the closure. This increase is found across all categories of nurses, including LPNs, RNs, and APNs. We also find a substantial increase in the provision of inpatient and outpatient surgical operations but there is no change in ED visits. We do not find any effects for nearby urban hospitals. Our study suggests that a large proportion of the nursing workforce relocates to nearby hospitals after a rural hospital closure, which mitigates the negative consequences of such closures and allows these nearby hospitals to provide a larger volume of highly profitable services.

Effects of essential caregiver policies on COVID-19 and non-COVID-19 deaths in nursing homes.

Federal authorities banned nursing home visitation in the early days of the coronavirus disease 2019 (COVID-19) pandemic. However, there was growing concern that physical isolation may have unintended harms on nursing home residents. Thus, nursing homes and policymakers faced a tradeoff between minimizing COVID-19 outbreaks and limiting the unintended harms. Between June 2020 and January 2021, 17 states implemented Essential Caregiver policies (ECPs) allowing nursing home visitation by designated family members or friends under controlled circumstances. Using the Nursing Home COVID-19 Public File and other relevant data, we analyze the effects of ECPs on deaths among nursing home residents. We exploit variation in the existence of ECPs across states and over time, finding that these policies effectively reduce both non-COVID-19 and COVID-19 deaths, resulting in a decrease in total deaths. These effects are larger for states that implemented policies mandatorily or without restrictions, indicating a dose-response relationship. These policies reduce non-COVID-19 deaths in facilities with higher quality or staffing levels, while reducing COVID-19 deaths in facilities with lower quality or staffing levels. Our findings support the use and expansion of ECPs to balance resident safety and the need for social interaction and informal care during future pandemics.

Efficiency and productivity gains of robotic surgery: The case of the English National Health Service.

This paper examines the effect of new medical technology (robotic surgery) on efficiency gains and productivity changes for surgical treatment in patients with prostate cancer from the perspective of a public health sector organization. In particular, we consider three interrelated surgical technologies within the English National Health System: robotic, laparoscopic and open radical prostatectomy. Robotic and laparoscopic techniques are minimally invasive procedures with similar clinical benefits. While the clinical benefits in adopting robotic surgery over laparoscopic intervention are unproven, it requires a high initial investment cost and carries high on-going maintenance costs. Using data from Hospital Episode Statistics for the period 2000-2018, we observe growing volumes of prostatectomies over time, mostly driven by an increase in robotic-assisted surgeries, and further analyze whether hospital providers that adopted a robot see improved measures of throughput. We then quantify changes in total factor and labor productivity arising from the use of this technology. We examine the impact of robotic adoption on efficiency gains employing a staggered difference-in-difference estimator and find evidence of a 50% reduction in length of stay (LoS), 49% decrease in post-LoS and 44% and 46% decrease in postoperative visits after 1 year and 2 years, respectively. Productivity analysis shows the growth in radical prostatectomy volume is sustained with a relatively stable number of urology surgeons. The robotic technique increases total production at the hospital level between 21% and 26%, coupled with a 29% improvement in labor productivity. These benefits lend some, but not overwhelming support for the large-scale hospital investments in such costly technology.

How do physicians respond to new medical research?

What happens when the findings of a prominent medical study are overturned? Using a medical trial on breech births, we estimate the effect of the reversal of such a medical study on physician choices and infant health outcomes. Using the United States Birth Certificate Records from 1995 to 2010, we employ a difference-in-differences estimator for C-sections, low Apgar, and low birth weight measures. We find that the reversal of a multi-site, high profile, randomized control trial on the appropriate delivery of term breech births, the Term Breech Trial, led to a 15%-23% decline in C-sections for such births at a time when the overall trend in C-sections was rising. We find our largest estimated effects amongst traditionally disadvantaged groups (i.e., non-white, and minimal education). However, we do not find that such a change in practice had significant impacts on infant health. Contrary to prior studies, we find that physicians updated their beliefs quickly, and do indeed adjust to new medical research, particularly young physicians, prior to mandatory policy or professional guidelines.

Negative emotions increase unhealthy eating: Evidence from the Wuhan lockdown during COVID-19.

This paper studies how negative emotions like stress, anxiety, and boredom can affect unhealthy food consumption. Using the Wuhan lockdown as an external shock, we examine the changes in food consumption in a city that was not in lockdown. We applied the difference-in-differences method to a large scanner dataset from a retail monopoly in China. Our findings reveal that negative emotions induced by the pandemic lockdown significantly elevated consumer spending on unhealthy food items such as crisps, sugary beverages, regular soda, and low-alcohol beverages. Notably, the effect of unhealthy food consumption was more pronounced among younger and wealthier demographics. Triggering factors, like information about confirmed new deaths and infections as well as proximity to local hospitals, were found to strongly influence the consumption of unhealthy foods. Overall, the lockdown's impact extended beyond short-term increases in snack consumption to substantial increases in overall dietary and nutritional intake.

Impacts of Norway's extended free choice reform on waiting times and hospital visits.

Norway's extended free choice (EFC) reform extends the patient's choice of publicly funded hospitals for treatment to authorized private institutions (EFC providers). We study the effects of the reform on waiting times, number of visits, and patients' Charlson Comorbidity Index scores in public hospitals. We use a difference-in-differences model to compare changes over time for public hospitals with and without EFC providers in the catchment area. Focusing on five prevalent somatic services, we find that the EFC reform did not exert pressure on public hospitals to stimulate shorter waiting times and more visits. Moreover, we do not find that the sum of public and private visits increased. When we compare patient comorbidity between public hospitals and EFC providers, we find that for non-invasive diagnostic services, patient comorbidity is lower in EFC providers. For surgical services, we detect no difference in patient comorbidities between public and EFC providers.

The impact of integrated care on health care utilization and costs in a socially deprived urban area in Germany: A difference-in-differences approach within an event-study framework.

We investigated the impact of an integrated care initiative in a socially deprived urban area in Germany. Using administrative data, we empirically assessed the causal effect of its two sub-interventions, which differed by the extent to which their instruments targeted the supply and demand side of healthcare provision. We addressed confounding using propensity score matching via the Super Learner machine learning algorithm. For our baseline model, we used a two-way fixed-effects difference-in-differences approach to identify causal effects. We then employed difference-in-differences analyses within an event-study framework to explore the heterogeneity of treatment effects over time, allowing us to disentangle the effects of the sub-interventions and improve causal interpretation and generalizability. The initiative led to a significant increase in hospital and emergency admissions and non-hospital outpatient visits, as well as inpatient, non-hospital outpatient, and total costs. Increased utilization may indicate that the intervention improved access to care or identified unmet need.

Unlocking the double-dividend: Evaluating the impact of SO2 emissions trading scheme on firm's environmental and economic performance.

The optimal design of environmental instruments demands a balance between environmental enhancement and economic growth. Utilizing microdata from the China Environmental Statistics Database and the China Industrial Firm Database, this study employs the difference-in-differences (DD) methodology to explore the dual effects of the SO2 Emissions Trading Scheme (ETS) on the environmental and economic performance of micro-firms. The findings suggest that: (1) The SO2 ETS not only induces emission reduction effects among firms in pilot areas but also improves their industrial added value. (2) The SO2 ETS exhibits heterogeneous impacts across firms of diverse ownership, export status, and size. (3) While the SO2 ETS prompts firms to advance technologically, boosting desulfurization capacities and subsequently enhancing total factor productivity, it also inadvertently results in companies offsetting some environmental compliance costs by curtailing employee wages.