Designing Difference-in-Difference Studies with Staggered Treatment Adoption: Key Concepts and Practical Guidelines.

Difference-in-difference (DID) estimators are a valuable method for identifying causal effects in the public health researcher's toolkit. A growing methods literature points out potential problems with DID estimators when treatment is staggered in adoption and varies with time. Despite this, no practical guide exists for addressing these new critiques in public health research. We illustrate these new DID concepts with step-by-step examples, code, and a checklist. We draw insights by comparing the simple 2 × 2 DID design (single treatment group, single control group, two time periods) with more complex cases: additional treated groups, additional time periods of treatment, and treatment effects possibly varying over time. We outline newly uncovered threats to causal interpretation of DID estimates and the solutions the literature has proposed, relying on a decomposition that shows how the more complex DIDs are an average of simpler 2 × 2 DID subexperiments.

Disease awareness and healthcare utilization in rural South Africa: a comparative analysis of HIV and diabetes in the HAALSI cohort.

BACKGROUND: Studies from rural South Africa indicate that people living with HIV (PLHIV) may have better health outcomes than those without, potentially due to the frequent healthcare visits necessitated by infection. Here, we examined the association between HIV status and healthcare utilization, using diabetes as an illustrative comparator of another high-burden, healthcare-intensive disease. METHODS: Our exposure of interest was awareness of positive disease status for both HIV and diabetes. We identified 742 individuals who were HIV-positive and aware of their status and 305 who had diabetes and were aware of their status. HIV-positive status was further grouped by viral suppression. For each disease, we estimated the association with (1) other comorbid, chronic conditions, (2) health facility visits, (3) household-level healthcare expenditure, and (4) per-visit healthcare expenditure. We used log-binomial regression models to estimate prevalence ratios for co-morbid chronic conditions. Linear regression models were used for all other outcomes. RESULTS: Virally suppressed PLHIV had decreased prevalence of chronic conditions, increased public clinic visits [ß = 0.59, 95% CI: 0.5, 0.7], and reduced per-visit private clinic spending [ß = -60, 95% CI: -83, -6] compared to those without HIV. No differences were observed in hospitalizations and per-visit spending at hospitals and public clinics between virally suppressed PLHIV and non-PLHIV. Conversely, diabetic individuals had increased prevalence of chronic conditions, increased visits across facility types, increased household-level expenditures (ß = 88 R, 95% CI: 29, 154), per-visit hospital spending (ß = 54 R, 95% CI: 7, 155), and per-visit public clinic spending (ß = 31 R, 95% CI: 2, 74) compared to those without diabetes. CONCLUSIONS: Our results suggest that older adult PLHIV may visit public clinics more often than their HIV-negative counterparts but spend similarly on a per-visit basis. This provides preliminary evidence that the positive health outcomes observed among PLHIV in rural South Africa may be explained by different healthcare engagement patterns. Through our illustrative comparison between PLHIV and diabetics, we show that shifting disease burdens towards chronic and historically underfunded diseases, like diabetes, may be changing the landscape of health expenditure inequities.

Evidence from internet search data shows information-seeking responses to news of local COVID-19 cases.

The COVID-19 outbreak is a global pandemic with community circulation in many countries, including the United States, with confirmed cases in all states. The course of this pandemic will be shaped by how governments enact timely policies and disseminate information and by how the public reacts to policies and information. Here, we examine information-seeking responses to the first COVID-19 case public announcement in a state. Using an event study framework for all US states, we show that such news increases collective attention to the crisis right away. However, the elevated level of attention is short-lived, even though the initial announcements are followed by increasingly strong policy measures. Specifically, searches for "coronavirus" increased by about 36% (95% CI: 27 to 44%) on the day immediately after the first case announcement but decreased back to the baseline level in less than a week or two. We find that people respond to the first report of COVID-19 in their state by immediately seeking information about COVID-19, as measured by searches for coronavirus, coronavirus symptoms, and hand sanitizer. On the other hand, searches for information regarding community-level policies (e.g., quarantine, school closures, testing) or personal health strategies (e.g., masks, grocery delivery, over-the-counter medications) do not appear to be immediately triggered by first reports. These results are representative of the study period being relatively early in the epidemic, and more-elaborate policy responses were not yet part of the public discourse. Further analysis should track evolving patterns of responses to subsequent flows of public information.

Effects of Pension Eligibility Expansion on Men's Cognitive Function: Findings from Rural South Africa.

Two-thirds of people living with Alzheimer's disease and related dementias (ADRD) live in low- and middle-income countries, and this figure is expected to rise as these populations are rapidly aging. Since evidence demonstrates links between socioeconomic status and slower rates of cognitive decline, protecting older adults' cognitive function in resource-limited countries that lack the infrastructure to cope with ADRD is crucial to reduce the burden it places on these populations and their health systems. While cash transfers are a promising intervention to promote healthy cognitive aging, factors such as household wealth and level of education often confound the ability to make causal inferences on the impact of cash transfers and cognitive function. This study uses a quasi-experimental design, leveraging an exogenous expansion to the Old Age Pension for men in South Africa, to approximate causal associations with cognitive function. We found evidence that there is a potential benefit of cash transfers at an earlier age for older individuals. As such, transfers such as pensions or other forms of direct basic income transfers may hold promise as potential interventions to promote healthy cognitive aging.

Determinants of Disparities in Early COVID-19 Job Losses.

This study examines the sociodemographic divide in early labor market responses to the U.S. COVID-19 epidemic and associated policies, benchmarked against two previous recessions. Monthly Current Population Survey (CPS) data show greater declines in employment in April and May 2020 (relative to February) for Hispanic individuals, younger workers, and those with a high school diploma or some college. Between April and May, the demographic subgroups considered regained some employment. Reemployment in May was broadly proportional to the employment drop that occurred through April, except for Black individuals, who experienced a smaller rebound. Compared to the 2001 recession and the Great Recession, employment losses in the early COVID-19 recession were smaller for groups with low or high (vs. medium) education. We show that job loss was greater in occupations that require more interpersonal contact and that cannot be performed remotely, and that pre-COVID-19 sorting of workers into occupations and industries along demographic lines can explain a sizable portion of the demographic gaps in new unemployment. For example, while women suffered more job losses than men, their disproportionate pre-epidemic sorting into occupations compatible with remote work shielded them from even larger employment losses. However, substantial gaps in employment losses across groups cannot be explained by socioeconomic differences. We consider policy lessons and future research needs regarding the early labor market implications of the COVID-19 crisis.

Impacts of state COVID-19 reopening policy on human mobility and mixing behavior.

This study quantifies the effect of the 2020 state COVID economic activity reopening policies on daily mobility and mixing behavior, adding to the economic literature on individual responses to public health policy that addresses public contagion risks. We harness cellular device signal data and the timing of reopening plans to provide an assessment of the extent to which human mobility and physical proximity in the United States respond to the reversal of state closure policies. We observe substantial increases in mixing activities, 13.56% at 4 days and 48.65% at 4 weeks, following reopening events. Echoing a theme from the literature on the 2020 closures, mobility outside the home increased on average prior to these state actions. Furthermore, the largest increases in mobility occurred in states that were early adopters of closure measures and hard-hit by the pandemic, suggesting that psychological fatigue is an important barrier to implementation of closure policies extending for prolonged periods of time.

Does the built environment influence the effectiveness of behavioral weight management interventions?

Outcomes of behavioral lifestyle interventions for promoting weight loss vary widely across participants. The effectiveness of a weight management intervention may depend on a person's environmental context. This study compared short- and longer-term effects of a structured nationwide weight management program for people living in neighborhoods with different levels of walkability and different access to recreational places (parks, fitness facilities). Drawing on the health production model, we tested competing hypotheses for whether treatment effects of the program complement environmental supports or substitute for environmental constraints. We studied the US Department of Veterans Affairs (VA) MOVE! weight management program using VA electronic heath record data (2009-2014) and a difference-in-differences design with an inverse propensity score matched comparison group. A total of 114,256 program participants and 498,494 non-participants comprised the sample. Built environment features were measured within one-mile of each person's home. We estimated program effects on body mass index (BMI) for subgroups with different built environments at 6-, 12-, 18-, and 24-month follow-up using linear regressions with person and year fixed effects. At 6 months, the program reduced BMI by 0.4-0.6 kg/m2 among men and 0.3-0.5 kg/m2 among women. The effect diminished at 12, 18, and 24 months. The program effect did not vary significantly across subgroups with different walkability, park access, or fitness facility access. The MOVE! program was not sensitive to environmental context. Results did not lend support to either hypothesis that the MOVE! program complements or substitutes for a person's built environment to affect weight management outcomes.

Recruiting and retaining dental labor in federal facilities: Harder than pulling teeth?

The U.S. Veterans Administration (VA) is a large publicly financed health system that has long struggled with provider shortages. Shortages may arise at the VA because it offers different compensation than private sector employment options or because of differences in the way that labor is supplied to public versus private employers. In the mid-2000s, the VA adopted a more generous and flexible pay schedule for its dentists. We exploit this salary schedule change to study the impact of a positive wage shock on dental labor supplied to the VA, within a difference-in-differences framework. We find limited effects on VA separation and new hire rates overall-though early career dentists appear more sensitive to the wage change. More generous pay has its clearest effects on employment type for VA dentists, reducing the likelihood of being part-time by roughly 10%.

Economic downturns and nurse attachment to federal employment.

The impact of the financial crisis has been uneven-with differences across industries and occupations. Jobs linked to health care appear better insulated, with nurses specifically showing labor force gains during the recent recession. What is not known is how important public sector employment opportunities are for these national nursing trends. Observing the universe of nurses working for one of the largest (and publicly operated) health care employers, we show that worsening economic conditions lead to stronger job attachment. Relatedly, older nurses also seem more willing to delay retirement and instead transition to part-time positions during a downturn.

Designing Difference in Difference Studies: Best Practices for Public Health Policy Research.

The difference in difference (DID) design is a quasi-experimental research design that researchers often use to study causal relationships in public health settings where randomized controlled trials (RCTs) are infeasible or unethical. However, causal inference poses many challenges in DID designs. In this article, we review key features of DID designs with an emphasis on public health policy research. Contemporary researchers should take an active approach to the design of DID studies, seeking to construct comparison groups, sensitivity analyses, and robustness checks that help validate the method's assumptions. We explain the key assumptions of the design and discuss analytic tactics, supplementary analysis, and approaches to statistical inference that are often important in applied research. The DID design is not a perfect substitute for randomized experiments, but it often represents a feasible way to learn about casual relationships. We conclude by noting that combining elements from multiple quasi-experimental techniques may be important in the next wave of innovations to the DID approach.

Estimate of incidence and cost of recreational waterborne illness on United States surface waters.

BACKGROUND: Activities such as swimming, paddling, motor-boating, and fishing are relatively common on US surface waters. Water recreators have a higher rate of acute gastrointestinal illness, along with other illnesses including respiratory, ear, eye, and skin symptoms, compared to non-water recreators. The quantity and costs of such illnesses are unknown on a national scale. METHODS: Recreational waterborne illness incidence and severity were estimated using data from prospective cohort studies of water recreation, reports of recreational waterborne disease outbreaks, and national water recreation statistics. Costs associated with medication use, healthcare provider visits, emergency department (ED) visits, hospitalizations, lost productivity, long-term sequelae, and mortality were aggregated. RESULTS: An estimated 4 billion surface water recreation events occur annually, resulting in an estimated 90 million illnesses nationwide and costs of $2.2- $3.7 billion annually (central 90% of values). Illnesses of moderate severity (visit to a health care provider or ED) were responsible for over 65% of the economic burden (central 90% of values: $1.4- $2.4 billion); severe illnesses (result in hospitalization or death) were responsible for approximately 8% of the total economic burden (central 90% of values: $108- $614 million). CONCLUSION: Recreational waterborne illnesses are associated with a substantial economic burden. These findings may be useful in cost-benefit analysis for water quality improvement and other risk reduction initiatives.

Neighborhood Resources to Support Healthy Diets and Physical Activity Among US Military Veterans.

INTRODUCTION: Among the nearly 21 million military veterans living in the United States, 64.0% of women and 76.1% of men are overweight or obese, higher rates than in the civilian population (56.9% of women and 69.9% of men). Attributes of the residential environment are linked to obesity. The objective of this study was to characterize the residential environments of the US veteran population with respect to availability of food and recreational venues. METHODS: We used American Community Survey data to determine the concentration of veterans (the percentage of veterans among the adult population) in all continental US census tracts in 2013, and we used proprietary data to construct measures of availability of food and recreational venues per census tract. Using descriptive statistics and ordinary least-squares regression, we examined associations between the concentration of veterans per census tract and those residential environmental features. RESULTS: In census tracts with high concentrations of veterans, residents had, on average, 0.5 (interquartile range, 0-0.8) supermarkets within a 1-mile radius, while residents in census tracts with low concentrations of veterans had 3.2 (interquartile range, 0.6-3.7) supermarkets. Patterns were similar for grocery and convenience stores, fast food restaurants, parks, and commercial fitness facilities. In adjusted analyses controlling for census-tract-level covariates, veteran concentration remained strongly negatively associated with availability of those food and recreational venues. In nonmetropolitan tracts, adjusted associations were greatly attenuated and even positive. CONCLUSION: Where veterans live is strongly associated with availability of food outlets providing healthy (and unhealthy) foods and with recreational venues, raising questions about the contributions of veterans' residential environments to their high obesity rates. Additional research is needed to address those questions.

Genetic ancestry as an effect modifier of naltrexone in smoking cessation among African Americans: an analysis of a randomized controlled trial.

OBJECTIVES: To determine whether there were differential quit rates between African Americans (AA) and European Americans with the experimental treatment naltrexone, and examine the role of genetic ancestry on these outcomes among AAs. METHODS: Data from a previous randomized trial of 315 smokers to naltrexone versus placebo were reanalyzed using West African (WA) genetic ancestry to define subpopulations. Logistic regression models were used to estimate treatment effects on early and end of treatment quit rates, by race and WA ancestry. RESULTS: Among European Americans (n=136), naltrexone significantly increased quit rates at 4 weeks (62 vs. 43%, P=0.03) with directional, but not statistically significant effects at 12 weeks (30 vs. 18%, P=0.12). In contrast, among the AAs (n=95), quit rates did not differ between naltrexone and placebo groups at either interval (4 weeks: 43 vs. 32%, P=0.27; 12 weeks: 22 vs. 18%, P=0.60). A median split was conducted in AAs for WA ancestry. Among AAs with low WA ancestry, quit rates were significantly higher with naltrexone compared with placebo (60 vs. 27%, P=0.03). There was no advantage in quit rates with naltrexone for the high WA ancestry group. CONCLUSION: Naltrexone efficacy for smoking cessation varies across AA individuals with different levels of WA ancestry. These results suggest that genetic background may partially explain racial differences in drug response.

Child support grant expansion and cognitive function among women in rural South Africa: Findings from a natural experiment in the HAALSI cohort.

BACKGROUND: Cash transfers are a promising but understudied intervention that may protect cognitive function in adults. Although South Africa has a rapidly ageing population, little is known about the nature of association between cash transfers and cognitive function in this setting. OBJECTIVES: We leveraged age-eligibility expansions to South Africa's Child Support Grant (CSG) to investigate the association between duration of CSG eligibility and cognitive function of biological mothers of child beneficiaries in South Africa. METHODS: We analysed 2014/2015 baseline data from 944 women, aged 40-59 years with at least one CSG-eligible child, enrolled in the population-representative HAALSI cohort in Agincourt, South Africa. Duration of CSG eligibility for each mother was calculated based on the birth dates of all their children and the CSG age-eligibility expansion years (2003-2012). Cognitive function was measured using a cognitive battery administered at the HAALSI baseline interview. Linear regression was used to estimate the association between duration of CSG eligibility, dichotomized as low (≤10 years) and high (>10 years) eligibility, and cognitive function z-scores of the mothers. RESULTS: High vs. low duration of CSG eligibility, was associated with higher cognitive function z-scores in the full sample [ß: 0.15 SD units; 95% CI: 0.04, 0.26; p-value = 0.01]. In mothers with one to four lifetime children, but not five or more, high vs. low duration of CSG eligibility, was associated with higher cognitive function z-scores [ß: 0.19 SD units; 95% CI: 0.05, 0.34, p-value = 0.02]. CONCLUSION: Government cash transfers given to support raising children may confer substantial protective effects on the subsequent cognitive function of mothers. Further studies are needed to understand how parity may influence this relationship. Our findings bring evidence to policymakers for designing income supplementation programmes to promote healthy cognitive ageing in low-income settings.

The validity of causal claims with repeated measures designs: A within-study comparison evaluation of differences-in-differences and the comparative interrupted time series.

Modern policies are commonly evaluated not with randomized experiments but with repeated measures designs like difference-in-differences (DID) and the comparative interrupted time series (CITS). The key benefit of these designs is that they control for unobserved confounders that are fixed over time. However, DID and CITS designs only result in unbiased impact estimates when the model assumptions are consistent with the data at hand. In this paper, we empirically test whether the assumptions of repeated measures designs are met in field settings. Using a within-study comparison design, we compare experimental estimates of the impact of patient-directed care on medical expenditures to non-experimental DID and CITS estimates for the same target population and outcome. Our data come from a multi-site experiment that includes participants receiving Medicaid in Arkansas, Florida, and New Jersey. We present summary measures of repeated measures bias across three states, four comparison groups, two model specifications, and two outcomes. We find that, on average, bias resulting from repeated measures designs are very close to zero (less than 0.01 standard deviations; SDs). Further, we find that comparison groups which have pre-treatment trends that are visibly parallel to the treatment group result in less bias than those with visibly divergent trends. However, CITS models that control for baseline trends produced slightly more bias and were less precise than DID models that only control for baseline means. Overall, we offer optimistic evidence in favor of repeated measures designs when randomization is not feasible.

Child Support Grant expansion and cognitive function among women in rural South Africa: findings from a natural experiment in HAALSI cohort.

Cash transfers are a promising but understudied intervention that may protect cognitive function in adults by promoting their cognitive reserve. South Africa has a rapidly ageing population, however, less is known about the nature of association between cash transfers and cognitive function in this setting. We leveraged natural experiments from Child Support Grant (CSG) age-eligibility expansions to investigate the association between duration of CSG eligibility and cognitive function among biological mothers of child beneficiaries in South Africa. We analysed 2014/2015 baseline data from 944 women, aged 40 - 59 years with at least one CSG-eligible child, enrolled in the HAALSI cohort in Agincourt, South Africa. Duration of CSG eligibility for each mother was calculated based on the birth dates of all their children and the CSG age eligibility expansion years. Cognitive function was measured using a cognitive battery administered to the mothers at baseline interview. Linear regression was used to estimate the association between duration of CSG eligibility, dichotomized as low (≤10 years) and high (>10 years) eligibility, and cognitive function z-scores of the mothers. Our study finds that high duration of CSG eligibility, compared to low, was associated with higher cognitive function z-scores in the full sample [ß: 0.15 SD; 95% CI: 0.04, 0.26; p-value = 0.01]. In mothers with one to four lifetime children, but not five or more, high duration of CSG eligibility, compared to low, was associated with higher cognitive function z-scores [ß: 0.19 SD; 95% CI: 0.05, 0.34, p-value = 0.02]. Government cash transfers given to support raising children may confer substantial protective effect on cognitive function of mothers in their mid-life. Further studies are needed to understand how parity may influence this relationship. Our findings bring evidence to policymakers for designing income supplementation programmes to promote healthy cognitive ageing in low-income settings.

Effects of social distancing policy on labor market outcomes.

US workers receive unemployment benefits if they lose their job, but not for reduced working hours. In alignment with the benefits incentives, we find that the labor market responded to COVID-19 and related closure-policies mostly on the extensive (12 pp outright job loss) margin. Exploiting timing variation in state closure-policies, difference-in-differences (DiD) estimates show, between March 12 and April 12, 2020, employment rate fell by 1.7 pp for every 10 extra days of state stay-at-home orders (SAH), with little effect on hours worked/earnings among those employed. Forty percentage of the unemployment was due to a nationwide shock, rest due to social-distancing policies, particularly among "non-essential" workers.

Active living-oriented zoning codes and cardiometabolic conditions across the lifespan.

Environments that make it easier for people to incorporate physical activity into their daily life may help to reduce high rates of cardiometabolic conditions. Local zoning codes are a policy and planning tool to create more walkable and bikeable environments. This study evaluated relationships between active living-oriented zoning code environments and cardiometabolic conditions (body mass index, hyperlipidemia, hypertension). The study used county identifiers to link electronic health record and other administrative data for a sample of patients utilizing primary care services between 2012 and 2016 with county-aggregated zoning code data and built environment data. The analytic sample included 7,441,991 patients living in 292 counties in 44 states. Latent class analysis was used to summarize municipal- and unincorporated county-level data on seven zoning provisions (e.g., sidewalks, trails, street connectivity, mixed land use), resulting in classes that differed in strength of the zoning provisions. Based on the probability of class membership, counties were categorized as one of four classes. Linear and logistic regression models estimated cross-sectional associations with each cardiometabolic condition. Models were fit separately for youth (aged 5-19), adults (aged 20-59), and older adults (aged 60+). Little evidence was found that body mass index in youth, adults, or older adults or the odds of hyperlipidemia or hypertension in adults or older adults differed according to the strength of active living-oriented zoning. More research is needed to identify the health impacts of zoning codes and whether alterations to these codes would improve population health over the long term.

Effects of large gatherings on the COVID-19 epidemic: Evidence from professional and college sports.

We compare COVID-19 case loads and mortality across counties that hosted more versus fewer NHL hockey games, NBA basketball games, and NCAA basketball games during the early months of 2020, before any large outbreaks were identified. We find that hosting one additional NHL/NBA game in March 2020 leads to an additional 7520 cases and 658 deaths. Similarly, we find that hosting an additional NCAA Division 1 men's basketball game in March 2020 results in an additional 34 deaths. Back-of-the-envelope calculations suggest that the per-game fatality costs were 200-300 times greater than per-game spending.

Vaccinations Against COVID-19 May Have Averted Up To 140,000 Deaths In The United States.

COVID-19 vaccination campaigns continue in the United States, with the expectation that vaccines will slow transmission of the virus, save lives, and enable a return to normal life in due course. However, the extent to which faster vaccine administration has affected COVID-19-related deaths is unknown. We assessed the association between US state-level vaccination rates and COVID-19 deaths during the first five months of vaccine availability. We estimated that by May 9, 2021, the US vaccination campaign was associated with a reduction of 139,393 COVID-19 deaths. The association varied in different states. In New York, for example, vaccinations led to an estimated 11.7 fewer COVID-19 deaths per 10,000, whereas Hawaii observed the smallest reduction, with an estimated 1.1 fewer deaths per 10,000. Overall, our analysis suggests that the early COVID-19 vaccination campaign was associated with reductions in COVID-19 deaths. As of May 9, 2021, reductions in COVID-19 deaths associated with vaccines had translated to value of statistical life benefit ranging between $625 billion and $1.4 trillion.