Estimating generalized propensity scores with survey and attrition weighted data.

Prior work in causal inference has shown that using survey sampling weights in the propensity score estimation stage and the outcome model stage for binary treatments can result in a more robust estimator of the effect of the binary treatment being analyzed. However, to date, extending this work to continuous treatments and exposures has not been explored nor has consideration been given for how to handle attrition weights in the propensity score model. Nonetheless, generalized propensity score (GPS) analyses are being used for estimating continuous treatment effects on outcomes when researchers have observational data, and those data sets often have survey or attrition weights that need to be accounted for in the analysis. Here, we extend prior work and show with analytic results that using survey sampling or attrition weights in the GPS estimation stage and the outcome model stage for continuous treatments can result in a more robust estimator than one that does not. Simulation study results show that, although using weights in both estimation stages is sufficient for robust estimation, it is not necessary and unbiased estimation is possible in some cases under various approaches to using weights in estimation. Analysts do not know if the conditions of our simulation studies hold, so use of weights in both estimation stages might provide insurance for reducing potential bias. We discuss the implications of our results in the context of an empirical example.

Geographic and Demographic Differences in the Proportion of Individuals Living in Households With a Firearm, 1990-2018.

Importance: Measures of the proportion of individuals living in households with a firearm (HFR), over time, across states, and by demographic groups are needed to evaluate disparities in firearm violence and the effects of firearm policies. Objective: To estimate HFR across states, years, and demographic groups in the US. Design, Setting, and Participants: In this survey study, substate HFR totals from 1990 to 2018 were estimated using bayesian multilevel regression with poststratification to analyze survey data on HFR from the Behavioral Risk Factor Surveillance System and the General Social Survey. HFR was estimated for 16 substate demographic groups defined by gender, race, marital status, and urbanicity in each state and year. Exposures: Survey responses indicating household firearm ownership were analyzed and compared with a common proxy for firearm ownership, the fraction of suicides completed with a firearm (FSS). Main Outcome and Measure: HFR, FSS, and their correlations and differences. Results: Among US adults in 2018, HFR was significantly higher among married, nonurban, non-Hispanic White and American Indian male individuals (65.0%; 95% credible interval [CI], 61.5%-68.7%) compared with their unmarried, urban, female counterparts from other racial and ethnic groups (7.3%; 95% CIs, 6.0%-9.2%). Marginal HFR rates for larger demographic groups also revealed important differences, with racial minority groups and urban dwellers having less than half the HFR of either White and American Indian (39.5%; 95% CI, 37.4%-42.9% vs 17.2%; 95% CI, 15.5%-19.9%) or nonurban populations (46.0%; 95% CI, 43.8%-49.5% vs 23.1%; 95% CI, 21.3%-26.2%). Population growth among groups less likely to own firearms, rather than changes in ownership within demographic groups, explains 30% of the 7 percentage point decline in HFR nationally from 1990 to 2018. Comparing HFR estimates with FSS revealed the expected high overall correlation across states (r = 0.84), but scaled FSS differed from HFR by as many as 20 percentage points for some states and demographic groups. Conclusions and Relevance: This survey study of HFR providing detailed, publicly available HFR estimates highlights key disparities among individuals in households with firearms across states and demographic groups; it also identifies potential biases in the use of FSS as a proxy for firearm ownership rates. These findings are essential for researchers, policymakers, and public health experts looking to address geographic and demographic disparities in firearm violence.

MACRA Palliative Care Quality Measure Development-Testing Summary Report: Measure Name: Receiving Desired Help for Pain.

Palliative care has expanded rapidly in the past 20 years, especially in the ambulatory (office) setting, and there is growing consensus regarding the need to systematically measure and incentivize high-quality care. The Centers for Medicare & Medicaid Services entered a cooperative agreement with the American Academy of Hospice and Palliative Medicine (AAHPM) as part of the Medicare Access and CHIP Reauthorization Act of 2015 to develop two patient-reported measures of ambulatory palliative care experience: Feeling Heard and Understood and Receiving Desired Help for Pain. Under contract to AAHPM, RAND Health Care researchers developed and tested both measures over a three-year project period. Researcher efforts included identifying, developing, testing, and validating appropriate patient-reported data elements for each measure; developing and fielding a survey instrument to collect necessary data in a national beta field test with 44 ambulatory palliative care programs; and collecting and analyzing data about measure reliability and validity to establish measure performance and final specifications. Further, the authors elicited provider and program perspectives on the use and value of the performance measures and their implementation and elicited the perspectives of patients from racial and ethnic minorities to understand their experience of ambulatory palliative care and optimal approaches to measurement. In this study, the authors present results from their test of the Receiving Desired Help for Pain performance measure, which they demonstrate to be a reliable and valid measure that is ready for use in quality improvement and quality payment programs.

MACRA Palliative Care Quality Measure Development-Testing Summary Report: Measure Name: Feeling Heard and Understood.

Palliative care has expanded rapidly in the past 20 years, especially in the ambulatory (office) setting, and there is growing consensus regarding the need to systematically measure and incentivize high-quality care. The Centers for Medicare & Medicaid Services entered a cooperative agreement with the American Academy of Hospice and Palliative Medicine (AAHPM) as part of the Medicare Access and CHIP Reauthorization Act of 2015 to develop two patient-reported measures of ambulatory palliative care experience: Feeling Heard and Understood and Receiving Desired Help for Pain. Under contract to AAHPM, RAND Health Care researchers developed and tested both measures over a three-year project period. Researcher efforts included identifying, developing, testing, and validating appropriate patient-reported data elements for each measure; developing and fielding a survey instrument to collect necessary data in a national beta field test with 44 ambulatory palliative care programs; and collecting and analyzing data about measure reliability and validity to establish measure performance and final specifications. Further, the authors elicited provider and program perspectives on the use and value of the performance measures and their implementation and elicited the perspectives of patients from racial and ethnic minorities to understand their experience of ambulatory palliative care and optimal approaches to measurement. In this study, the authors present results from their test of the Feeling Heard and Understood performance measure, which they demonstrate to be a reliable and valid measure that is ready for use in quality improvement and quality payment programs.

Comparative effectiveness of individual versus family-based substance use treatment on adolescent self-injurious thoughts and behaviors.

INTRODUCTION: Self-injurious thoughts and behaviors (SITB) are of increasing concern among adolescents, especially those who use substances. Some evidence suggests that existing evidence-based substance use treatments (EBTs) could impact not only their intended substance use targets but also SITB. However, which types of substance use treatments may have the greatest impact on youth SITB is not yet clear. Based on prior literature showing that family support and connection may buffer youth from SITB, we initially hypothesized that family-based EBTs would show greater improvement in SITB compared to those receiving individually focused EBTs and that the size of the effects would be small given the comparison between two active, evidence-based interventions, and base rates of SITB. METHODS: In a sample of 2893 youth in substance use treatment, we compared the effectiveness of individually and family-based EBTs in reducing SITBs. The study used entropy balancing and regression modeling to balance the groups on pre-treatment characteristics and examine change in outcomes over a one-year follow-up period. RESULTS: Both groups improved in self-injury and suicide attempts over the one-year study period, but only youth in individual treatment improved in suicidal ideation. However, the study found no significant difference between the changes over time in the two groups for any outcome. As expected, effect sizes were small and power was constrained in this study given the rarity of the outcomes, but effect sizes are similar to those observed with substance use outcomes. CONCLUSIONS: The results provide important exploratory evidence on the potential relative effectiveness of these two treatments for SITBs. This study supports prior findings that EBTs for youth substance use may help to improve SITB and suggests that different treatment formats (individual or family-based) could result in different benefits for SITB outcomes.

Higher Moments for Optimal Balance Weighting in Causal Estimation.

We expand upon a simulation study that compared three promising methods for estimating weights for assessing the average treatment effect on the treated for binary treatments: generalized boosted models, covariate-balancing propensity scores, and entropy balance. The original study showed that generalized boosted models can outperform covariate-balancing propensity scores, and entropy balance when there are likely to be nonlinear associations in both the treatment assignment and outcome models and when the other two models are fine-tuned to obtain balance only on first-order moments. We explore the potential benefit of using higher-order moments in the balancing conditions for covariate-balancing propensity scores and entropy balance. Our findings showcase that these two models should, by default, include higher-order moments and focusing only on first moments can result in substantial bias in estimated treatment effect estimates from both models that could be avoided using higher moments.

Nonparametric Estimation of Population Average Dose-Response Curves using Entropy Balancing Weights for Continuous Exposures.

Weighted estimators are commonly used for estimating exposure effects in observational settings to establish causal relations. These estimators have a long history of development when the exposure of interest is binary and where the weights are typically functions of an estimated propensity score. Recent developments in optimization-based estimators for constructing weights in binary exposure settings, such as those based on entropy balancing, have shown more promise in estimating treatment effects than those methods that focus on the direct estimation of the propensity score using likelihood-based methods. This paper explores recent developments of entropy balancing methods to continuous exposure settings and the estimation of population dose-response curves using nonparametric estimation combined with entropy balancing weights, focusing on factors that would be important to applied researchers in medical or health services research. The methods developed here are applied to data from a study assessing the effect of non-randomized components of an evidence-based substance use treatment program on emotional and substance use clinical outcomes.

Expanding outcomes when considering the relative effectiveness of two evidence-based outpatient treatment programs for adolescents.

The current study seeks to advance understanding about how to address substance use and co-occurring mental health problems in adolescents. Specifically, we compared the effectiveness of two evidence-based treatment programs (Motivational Enhancement Treatment/Cognitive Behavior Therapy, 5 Sessions [MET/CBT5] and Adolescent Community Reinforcement Approach [A-CRA]) for both substance use and mental health outcomes (i.e., crossover effects). We used statistical methods designed to approximate randomized controlled trials when comparing nonequivalent groups using observational study data. Our methods also included an assessment of the potential impact of omitted variables. We found that after applying balancing weighting to ensure similarity of the baseline samples (given the nonrandomized study design), both groups significantly improved on the two substance use outcomes (days abstinent and percent of youth in recovery) and on the two mental health outcomes (post-traumatic stress disorder (PTSD) symptoms and general emotional problems). Youth in A-CRA were significantly more likely to be in recovery at the 3-month follow-up compared to youth in MET/CBT5, but the size of this effect was very small. Youth receiving MET/CBT5 appeared to show significantly more improvement in the two mental health measures compared to youth in A-CRA, though these effect sizes were also very small. The findings indicate that adolescents with co-occurring substance use and mental health problems improve on both substance use and mental health outcomes with both treatments even though they are not specifically targeting mental health problems.

Optimally Balanced Gaussian Process Propensity Scores for Estimating Treatment Effects.

Propensity scores are commonly employed in observational study settings where the goal is to estimate average treatment effects. This paper introduces a flexible propensity score modeling approach, where the probability of treatment is modeled through a Gaussian process framework. To evaluate the effectiveness of the estimated propensity score, a metric of covariate imbalance is developed that quantifies the discrepancy between the distributions of covariates in the treated and control groups. It is demonstrated that this metric is ultimately a function of the hyperparameters of the covariance matrix of the Gaussian process and therefore it is possible to select the hyperparameters to optimize the metric and minimize overall covariate imbalance. The effectiveness of the GP method is compared in a simulation against other methods of estimating the propensity score and the method is applied to data from Dehejia and Wahba (1999) to demonstrate benchmark performance within a relevant policy application.

Across continents and demographics, unpredictable maternal signals are associated with children's cognitive function.

BACKGROUND: Early life experiences have persisting influence on brain function throughout life. Maternal signals constitute a primary source of early life experiences, and their quantity and quality during sensitive developmental periods exert enduring effects on cognitive function and emotional and social behaviors. Here we examined if, in addition to established qualitative dimensions of maternal behavior during her interactions with her infant and child, patterns of maternal signals may contribute to the maturation of children's executive functions. We focused primarily on effortful control, a potent predictor of mental health outcomes later in life. METHODS: In two independent prospective cohorts in Turku, Finland (N = 135), and Irvine, CA, USA (N = 192) that differed significantly in race/ethnicity and sociodemographic parameters, we assessed whether infant exposure to unpredictable patterns of maternal-derived sensory signals portended poor effortful control. OUTCOMES: In both the Irvine and Turku cohorts, unpredictable sequences of maternal behavior during infancy were associated with worse effortful control at one year of age. Longitudinal analyses demonstrated that this association persisted for as long as each cohort was assessed-until two years of age in the Turku cohort and to 9.5 years in the Irvine cohort. The relation of unpredictable maternal signals during infancy and the measures of executive function persisted after adjusting for covariates. INTERPRETATIONS: The consistency of our findings across two cohorts from different demographic backgrounds substantiated the finding that patterns, and specifically unpredictable sequences, of maternal behaviors may influence the development of executive functions which may be associated with vulnerability to subsequent psychopathology. FUND: This research was supported by the National Institutes of Health (NIH) awards P50MH096889, HD051852, NS041298, HD02413, HD050662, HD065823, and by the FinnBrain funders: Academy of Finland (129839, 134950, 253270, 286829, 287908, 308176, 308252), Jane and Aatos Erkko Foundation, Signe and Ane Gyllenberg Foundation, Yrjö Jahnsson Foundation, and State Research Grants (P3498, P3654).

Exposure to unpredictable maternal sensory signals influences cognitive development across species.

Maternal care is a critical determinant of child development. However, our understanding of processes and mechanisms by which maternal behavior influences the developing human brain remains limited. Animal research has illustrated that patterns of sensory information is important in shaping neural circuits during development. Here we examined the relation between degree of predictability of maternal sensory signals early in life and subsequent cognitive function in both humans (n = 128 mother/infant dyads) and rats (n = 12 dams; 28 adolescents). Behaviors of mothers interacting with their offspring were observed in both species, and an entropy rate was calculated as a quantitative measure of degree of predictability of transitions among maternal sensory signals (visual, auditory, and tactile). Human cognitive function was assessed at age 2 y with the Bayley Scales of Infant Development and at age 6.5 y with a hippocampus-dependent delayed-recall task. Rat hippocampus-dependent spatial memory was evaluated on postnatal days 49-60. Early life exposure to unpredictable sensory signals portended poor cognitive performance in both species. The present study provides evidence that predictability of maternal sensory signals early in life impacts cognitive function in both rats and humans. The parallel between experimental animal and observational human data lends support to the argument that predictability of maternal sensory signals causally influences cognitive development.