A rank-based approach to evaluate a surrogate marker in a small sample setting.

In clinical studies of chronic diseases, the effectiveness of an intervention is often assessed using "high cost" outcomes that require long-term patient follow-up and/or are invasive to obtain. While much progress has been made in the development of statistical methods to identify surrogate markers, that is, measurements that could replace such costly outcomes, they are generally not applicable to studies with a small sample size. These methods either rely on nonparametric smoothing which requires a relatively large sample size or rely on strict model assumptions that are unlikely to hold in practice and empirically difficult to verify with a small sample size. In this paper, we develop a novel rank-based nonparametric approach to evaluate a surrogate marker in a small sample size setting. The method developed in this paper is motivated by a small study of children with nonalcoholic fatty liver disease (NAFLD), a diagnosis for a range of liver conditions in individuals without significant history of alcohol intake. Specifically, we examine whether change in alanine aminotransferase (ALT; measured in blood) is a surrogate marker for change in NAFLD activity score (obtained by biopsy) in a trial, which compared Vitamin E ($n=50$) versus placebo ($n=46$) among children with NAFLD.

Survey Protocols, Response Rates, and Representation of Underserved Patients: A Randomized Clinical Trial.

Importance: Surveys often underrepresent certain patients, such as underserved patients. Methods that improve their response rates (RRs) would help patient surveys better represent their experiences and assess equity and equity-targeted quality improvement efforts. Objective: To estimate the effect of adding an initial web mode to existing Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey protocols and extending the fielding period on RR and representativeness of underserved patient groups. Design, Setting, and Participants: This randomized clinical trial included 36 001 patients discharged from 46 US hospitals from May through December 2021. Data analysis was performed from May 2022 to September 2023. Exposures: Patients were randomized to 1 of 6 survey protocols: 3 standard HCAHPS protocols (mail only, phone only, mail-phone) plus 3 web-enhanced protocols (web-mail, web-phone, web-mail-phone). Main Outcomes and Measures: RR and number of respondents per 100 survey attempts (yield) were calculated and compared for each of the 6 survey protocols, overall, and by patient age, service line, sex, and race and ethnicity. Results: A total of 34 335 patients (median age range, 55-59 years; 59.3% female individuals and 40.7% male individuals) were eligible and included in the study. Of the respondents, 6.9% were Asian American or Native Hawaiian or Other Pacific Islander, 0.7% were American Indian or Alaska Native, 11.5% were Black, 17.4% were Hispanic, 61.0% were White, and 2.6% were multiracial. Of the 6 protocols, RRs were highest in web-mail-phone (36.5%), intermediate for the 3 two-mode survey protocols (mail-phone, web-mail, web-phone, 30.3%-31.1%), and lowest for the 2 single-mode protocols (mail only, phone only, 22.1%-24.3%). Web-mail-phone resulted in the highest yield for 3 racial and ethnic groups (Black, Hispanic, and White patients) and second highest for another (multiracial patients). Otherwise, the highest or second highest yield was almost always a 2-mode protocol. Mail only was the lowest-yield protocol for Black, Hispanic, and multiracial patients and phone-only was the lowest-yield protocol for White patients; these 2 protocols tied for lowest-yield for Asian American or Native Hawaiian or Other Pacific Islander patients. Gains from multimode approaches were often 2 to 3 times as large for Asian American or Native Hawaiian or Other Pacific Islander, Black, Hispanic, and multiracial patients as for White patients. Web-mail-phone had the highest RR for 6 of 8 age groups and 4 of 5 combinations of service line and sex. Conclusions and Relevance: In this randomized clinical trial, web-first multimode survey protocols significantly improved the RR and representativeness of patient surveys. The best-performing protocol based on RR and representativeness was web-mail-phone. Web-phone performed well for young and diverse patient populations, and web-mail for older and less diverse patient populations. The US Centers for Medicare & Medicaid Services will allow hospitals to use the web-mail, web-phone, and web-mail-phone protocols for HCAHPS administration beginning in 2025.

Statistical Methods to Evaluate Surrogate Markers.

BACKGROUND: There is tremendous interest in evaluating surrogate markers given their potential to decrease study time, costs, and patient burden. OBJECTIVES: The purpose of this statistical workshop article is to describe and illustrate how to evaluate a surrogate marker of interest using the proportion of treatment effect (PTE) explained as a measure of the quality of the surrogate marker for: (1) a setting with a general fully observed primary outcome (eg, biopsy score); and (2) a setting with a time-to-event primary outcome which may be censored due to study termination or early drop out (eg, time to diabetes). METHODS: The methods are motivated by 2 randomized trials, one among children with nonalcoholic fatty liver disease where the primary outcome was a change in biopsy score (general outcome) and another study among adults at high risk for Type 2 diabetes where the primary outcome was time to diabetes (time-to-event outcome). The methods are illustrated using the Rsurrogate package with a detailed R code provided. RESULTS: In the biopsy score outcome setting, the estimated PTE of the examined surrogate marker was 0.182 (95% confidence interval [CI]: 0.121, 0.240), that is, the surrogate explained only 18.2% of the treatment effect on the biopsy score. In the diabetes setting, the estimated PTE of the surrogate marker was 0.596 (95% CI: 0.404, 0.760), that is, the surrogate explained 59.6% of the treatment effect on diabetes incidence. CONCLUSIONS: This statistical workshop provides tools that will support future researchers in the evaluation of surrogate markers.

Flexible evaluation of surrogate markers with Bayesian model averaging.

When long-term follow up is required for a primary endpoint in a randomized clinical trial, a valid surrogate marker can help to estimate the treatment effect and accelerate the decision process. Several model-based methods have been developed to evaluate the proportion of the treatment effect that is explained by the treatment effect on the surrogate marker. More recently, a nonparametric approach has been proposed allowing for more flexibility by avoiding the restrictive parametric model assumptions required in the model-based methods. While the model-based approaches suffer from potential mis-specification of the models, the nonparametric method fails to give desirable estimates when the sample size is small, or when the range of the data does not follow certain conditions. In this paper, we propose a Bayesian model averaging approach to estimate the proportion of treatment effect explained by the surrogate marker. Our procedure offers a compromise between the model-based approach and the nonparametric approach by introducing model flexibility via averaging over several candidate models and maintains the strength of parametric models with respect to inference. We compare our approach with previous model-based methods and the nonparametric method. Simulation studies demonstrate the advantage of our method when surrogate supports are inconsistent and sample sizes are small. We illustrate our method using data from the Diabetes Prevention Program study to examine hemoglobin A1c as a surrogate marker for fasting glucose.

Lower socioeconomic status may help explain racial disparities in asthma and atopic dermatitis prevalence: A mediation analysis.

BACKGROUND: Racial disparities in atopic disease (atopic dermatitis [AD], asthma, and allergies) prevalence are well documented. Despite strong associations between race and socioeconomic deprivation in the United States, and socioeconomic status (SES) and atopic diseases, the extent to which SES explains these disparities is not fully understood. OBJECTIVE: We sought to identify racial disparities in childhood atopic disease prevalence and determine what proportion of those disparities is mediated by SES. METHODS: This study used the National Health Interview Survey (2011-2018) to investigate AD, asthma, and respiratory allergy prevalence in Black and White children and the extent to which measures of SES explain any identified disparities. RESULTS: By race, prevalences were as follows: AD, White 11.8% (95% CI: 11.4%, 12.2%) and Black 17.4% (95% CI: 16.6%, 18.3%); asthma prevalence, White 7.4% (95% CI: 7.0%, 7.7%) and Black 14.3% (95% CI: 13.5%, 15.0%); respiratory allergy, White 11.4% (95% CI: 11.0%, 11.9%) and Black 10.9% (95% CI: 10.3%, 11.6%). The percentage of the disparity between racial groups and disease prevalence explained by a multivariable measure of SES was 25% (95% CI: 15%, 36%) for Black versus White children with AD and 47% (95% CI: 40%, 54%) for Black versus White children with asthma. CONCLUSIONS: In a nationally representative US population, Black children had higher prevalence of AD and asthma than White children did and similar prevalence of respiratory allergy; a multivariable SES measure explained a proportion of the association between Black versus White race and AD and a much larger proportion for asthma.

Can earlier biomarker measurements explain a treatment effect on diabetes incidence? A robust comparison of five surrogate markers.

INTRODUCTION: We measured and compared five individual surrogate markers-change from baseline to 1 year after randomization in hemoglobin A1c (HbA1c), fasting glucose, 2-hour postchallenge glucose, triglyceride-glucose index (TyG) index, and homeostatic model assessment of insulin resistance (HOMA-IR)-in terms of their ability to explain a treatment effect on reducing the risk of type 2 diabetes mellitus at 2, 3, and 4 years after treatment initiation. RESEARCH DESIGN AND METHODS: Study participants were from the Diabetes Prevention Program study, randomly assigned to either a lifestyle intervention (n=1023) or placebo (n=1030). The surrogate markers were measured at baseline and 1 year, and diabetes incidence was examined at 2, 3, and 4 years postrandomization. Surrogacy was evaluated using a robust model-free estimate of the proportion of treatment effect explained (PTE) by the surrogate marker. RESULTS: Across all time points, change in fasting glucose and HOMA-IR explained higher proportions of the treatment effect than 2-hour glucose, TyG index, or HbA1c. For example, at 2 years, glucose explained the highest (80.1%) proportion of the treatment effect, followed by HOMA-IR (77.7%), 2-hour glucose (76.2%), and HbA1c (74.6%); the TyG index explained the smallest (70.3%) proportion. CONCLUSIONS: These data suggest that, of the five examined surrogate markers, glucose and HOMA-IR were the superior surrogate markers in terms of PTE, compared with 2-hour glucose, HbA1c, and TyG index.

Exploring Which Patients Use Their Closest Emergency Departments Using Geocoded Data.

BACKGROUND: Each year, roughly 20% of U.S. adults visit an emergency department (ED), but little is known about patients' choice of ED. OBJECTIVES: Examine the discretion patients have to choose among EDs, characteristics associated with ED choice, and relationship between ED choice and self-reported care experiences of ED patients. METHODS: We surveyed adult patients discharged to the community (DTC) in January-March 2018 from 16 geographically dispersed hospital-based EDs, geocoded patient and hospital-based ED addresses within 100 miles of patient addresses, and calculated travel distances. We examined the likelihood of visiting the closest ED based on patient and ED characteristics. Linear regression models examined the association of choosing the closest ED with seven measures of patient experience of care (scaled 0-100), adjusting for patient characteristics. RESULTS: 43.6% of 4647 responding patients visited the ED nearest their home (on average, 5.7 miles away). Patients who chose a farther ED had more urgent conditions, were more educated, and were less likely to be non-Hispanic White. They were significantly more likely to have visited an ED in a higher-rated, metropolitan, network hospital with major teaching status, a cardiac intensive care unit, and a certified trauma center. Patients who chose a farther ED were more likely to recommend that ED, with "medium-to-large" differences in scores (+4.3% more selected "definitely yes", p < 0.05). CONCLUSIONS: Fewer than half of patients visited the closest ED. Patients who chose a farther ED tended to seek higher-rated hospitals and report more favorable experiences.

Improving the Quality of Written Discharge Instructions: A Multisite Collaborative Project.

BACKGROUND AND OBJECTIVES: Written discharge instructions help to bridge hospital-to-home transitions for patients and families, though substantial variation in discharge instruction quality exists. We aimed to assess the association between participation in an Institute for Healthcare Improvement Virtual Breakthrough Series collaborative and the quality of pediatric written discharge instructions across 8 US hospitals. METHODS: We conducted a multicenter, interrupted time-series analysis of a medical records-based quality measure focused on written discharge instruction content (0-100 scale, higher scores reflect better quality). Data were from random samples of pediatric patients (N = 5739) discharged from participating hospitals between September 2015 and August 2016, and between December 2017 and January 2020. These periods consisted of 3 phases: 1. a 14-month precollaborative phase; 2. a 12-month quality improvement collaborative phase when hospitals implemented multiple rapid cycle tests of change and shared improvement strategies; and 3. a 12-month postcollaborative phase. Interrupted time-series models assessed the association between study phase and measure performance over time, stratified by baseline hospital performance, adjusting for seasonality and hospital fixed effects. RESULTS: Among hospitals with high baseline performance, measure scores increased during the quality improvement collaborative phase beyond the expected precollaborative trend (+0.7 points/month; 95% confidence interval, 0.4-1.0; P < .001). Among hospitals with low baseline performance, measure scores increased but at a lower rate than the expected precollaborative trend (-0.5 points/month; 95% confidence interval, -0.8 to -0.2; P < .01). CONCLUSIONS: Participation in this 8-hospital Institute for Healthcare Improvement Virtual Breakthrough Series collaborative was associated with improvement in the quality of written discharge instructions beyond precollaborative trends only for hospitals with high baseline performance.

Association of Hospice Profit Status With Family Caregivers' Reported Care Experiences.

Importance: Expansive growth in the US hospice market has been driven almost exclusively by an increase in for-profit hospices. Prior research found that, in contrast to not-for-profit hospices, for-profit hospices focus on delivering care to patients in nursing homes, provide fewer nursing visits, and use less skilled staff. However, prior studies have not reported on the associations of these differences in care patterns with hospice care quality. Patient- and family-centeredness is a core element of hospice care quality that is measured through surveys of care experiences. Objective: To examine whether differences in profit status are associated with family caregivers' reports of hospice care experiences and assess factors that may be associated with observed differences in care experiences by profit status. Design, Setting, and Participants: Consumer Assessment of Healthcare Providers and Systems (CAHPS) Hospice Survey data from 653 208 caregiver respondents, reflecting care received from 3107 hospices between April 2017 and March 2019, were used for a cross-sectional examination of hospice care experiences by profit status. Data analysis was performed from January 2020 to November 2022. Main Outcomes and Measures: Outcomes were case-mix-adjusted and mode-adjusted top-box scores for 8 measures of hospice care experiences, including communication, timely care, symptom management, and emotional and religious support, as well as a summary score averaging across measures. Linear regression examined the association between profit status and hospice-level scores, adjusting for other organizational and structural hospice characteristics. Results: There were 906 not-for-profit and 1761 for-profit hospices with mean (SD) time in operation of 25.7 (7.8) years and 13.8 (8.0) years, respectively. Mean (SD) decedent age at death was 82.8 (2.3) years, similar for not-for-profit and for-profit hospices. The mean proportion of patients who were Black, Hispanic, and White was 4.9%, 0.9%, and 91.4% for not-for-profit hospices and 9.0%, 2.2%, and 85.4% for for-profit hospices, respectively. Family caregivers reported worse care experiences at for-profit hospices than at not-for-profit hospices for all measures. Significant differences in average hospice performance by profit status remained after adjusting for hospice characteristics. However, for-profit hospice performance varied, with 548 of 1761 (31.1%) for-profit hospices scoring 3 or more points below the national hospice average of overall performance and 386 of 1761 (21.9%) scoring 3 or more points above the average. In contrast, only 113 of 906 (12.5%) not-for-profit hospices scored 3 or more points below the average, and 305 of 906 (33.7%) scored 3 or more points above the average. Conclusions and Relevance: In this cross-sectional study of CAHPS Hospice Survey data, caregivers of patients receiving hospice care reported substantially worse care experiences in for-profit than in not-for-profit hospices; however, there was variation in reported experiences among both types of hospices. Public reporting of hospice quality is important.

Using a surrogate with heterogeneous utility to test for a treatment effect.

The primary benefit of identifying a valid surrogate marker is the ability to use it in a future trial to test for a treatment effect with shorter follow-up time or less cost. However, previous work has demonstrated potential heterogeneity in the utility of a surrogate marker. When such heterogeneity exists, existing methods that use the surrogate to test for a treatment effect while ignoring this heterogeneity may lead to inaccurate conclusions about the treatment effect, particularly when the patient population in the new study has a different mix of characteristics than the study used to evaluate the utility of the surrogate marker. In this article, we develop a novel test for a treatment effect using surrogate marker information that accounts for heterogeneity in the utility of the surrogate. We compare our testing procedure to a test that uses primary outcome information (gold standard) and a test that uses surrogate marker information, but ignores heterogeneity. We demonstrate the validity of our approach and derive the asymptotic properties of our estimator and variance estimates. Simulation studies examine the finite sample properties of our testing procedure and demonstrate when our proposed approach can outperform the testing approach that ignores heterogeneity. We illustrate our methods using data from an AIDS clinical trial to test for a treatment effect using CD4 count as a surrogate marker for RNA.

Doubly robust evaluation of high-dimensional surrogate markers.

When evaluating the effectiveness of a treatment, policy, or intervention, the desired measure of efficacy may be expensive to collect, not routinely available, or may take a long time to occur. In these cases, it is sometimes possible to identify a surrogate outcome that can more easily, quickly, or cheaply capture the effect of interest. Theory and methods for evaluating the strength of surrogate markers have been well studied in the context of a single surrogate marker measured in the course of a randomized clinical study. However, methods are lacking for quantifying the utility of surrogate markers when the dimension of the surrogate grows. We propose a robust and efficient method for evaluating a set of surrogate markers that may be high-dimensional. Our method does not require treatment to be randomized and may be used in observational studies. Our approach draws on a connection between quantifying the utility of a surrogate marker and the most fundamental tools of causal inference-namely, methods for robust estimation of the average treatment effect. This connection facilitates the use of modern methods for estimating treatment effects, using machine learning to estimate nuisance functions and relaxing the dependence on model specification. We demonstrate that our proposed approach performs well, demonstrate connections between our approach and certain mediation effects, and illustrate it by evaluating whether gene expression can be used as a surrogate for immune activation in an Ebola study.

Testing for heterogeneity in the utility of a surrogate marker.

In studies that require long-term and/or costly follow-up of participants to evaluate a treatment, there is often interest in identifying and using a surrogate marker to evaluate the treatment effect. While several statistical methods have been proposed to evaluate potential surrogate markers, available methods generally do not account for or address the potential for a surrogate to vary in utility or strength by patient characteristics. Previous work examining surrogate markers has indicated that there may be such heterogeneity, that is, that a surrogate marker may be useful (with respect to capturing the treatment effect on the primary outcome) for some subgroups, but not for others. This heterogeneity is important to understand, particularly if the surrogate is to be used in a future trial to replace the primary outcome. In this paper, we propose an approach and estimation procedures to measure the surrogate strength as a function of a baseline covariate W and thus examine potential heterogeneity in the utility of the surrogate marker with respect to W. Within a potential outcome framework, we quantify the surrogate strength/utility using the proportion of treatment effect on the primary outcome that is explained by the treatment effect on the surrogate. We propose testing procedures to test for evidence of heterogeneity, examine finite sample performance of these methods via simulation, and illustrate the methods using AIDS clinical trial data.

Robust approach to combining multiple markers to improve surrogacy.

Identifying effective and valid surrogate markers to make inference about a treatment effect on long-term outcomes is an important step in improving the efficiency of clinical trials. Replacing a long-term outcome with short-term and/or cheaper surrogate markers can potentially shorten study duration and reduce trial costs. There is sizable statistical literature on methods to quantify the effectiveness of a single surrogate marker. Both parametric and nonparametric approaches have been well developed for different outcome types. However, when there are multiple markers available, methods for combining markers to construct a composite marker with improved surrogacy remain limited. In this paper, building on top of the optimal transformation framework of Wang et al. (2020), we propose a novel calibrated model fusion approach to optimally combine multiple markers to improve surrogacy. Specifically, we obtain two initial estimates of optimal composite scores of the markers based on two sets of models with one set approximating the underlying data distribution and the other directly approximating the optimal transformation function. We then estimate an optimal calibrated combination of the two estimated scores which ensures both validity of the final combined score and optimality with respect to the proportion of treatment effect explained by the final combined score. This approach is unique in that it identifies an optimal combination of the multiple surrogates without strictly relying on parametric assumptions while borrowing modeling strategies to avoid fully nonparametric estimation which is subject to the curse of dimensionality. Our identified optimal transformation can also be used to directly quantify the surrogacy of this identified combined score. Theoretical properties of the proposed estimators are derived, and the finite sample performance of the proposed method is evaluated through simulation studies. We further illustrate the proposed method using data from the Diabetes Prevention Program study.

Collaborative to Increase Lethal Means Counseling for Caregivers of Youth With Suicidality.

BACKGROUND: The number of youth presenting to hospitals with suicidality and/or self-harm has increased substantially in recent years. We implemented a multihospital quality improvement (QI) collaborative from February 1, 2018 to January 31, 2019, aiming for an absolute increase in hospitals' mean rate of caregiver lethal means counseling (LMC) of 10 percentage points (from a baseline mean performance of 68% to 78%) by the end of the collaborative, and to evaluate the effectiveness of the collaborative on LMC, adjusting for secular trends. METHODS: This 8 hospital collaborative used a structured process of alternating learning sessions and action periods to improve LMC across hospitals. Electronic medical record documentation of caregiver LMC was evaluated during 3 phases: precollaborative, active QI collaborative, and postcollaborative. We used statistical process control to evaluate changes in LMC monthly. Following collaborative completion, interrupted time series analyses were used to evaluate changes in the level and trend and slope of LMC, adjusting for covariates. RESULTS: In the study, 4208 children and adolescents were included-1314 (31.2%) precollaborative, 1335 (31.7%) during the active QI collaborative, and 1559 (37.0%) postcollaborative. Statistical process control analyses demonstrated that LMC increased from a hospital-level mean of 68% precollaborative to 75% (February 2018) and then 86% (October 2018) during the collaborative. In interrupted time series analyses, there were no significant differences in LMC during and following the collaborative beyond those expected based on pre-collaborative trends. CONCLUSIONS: LMC increased during the collaborative, but the increase did not exceed expected trends. Interventions developed by participating hospitals may be beneficial to others aiming to improve LMC for caregivers of hospitalized youth with suicidality.

Variation in Hospice Experiences by Care Setting for Patients With Dementia.

OBJECTIVES: Use of hospice care among patients with dementia has been steadily increasing. Our objectives were to characterize quality of hospice care experiences among decedents who had a primary diagnosis of dementia and their caregivers and investigate differences across settings of hospice care. DESIGN: We analyzed Consumer Assessment of Healthcare Providers and Systems (CAHPS) Hospice Survey data from caregiver respondents whose family members received hospice care. SETTING AND PARTICIPANTS: Data from 96,845 caregiver respondents whose family members had a primary diagnosis of dementia and died in 2017 or 2018 while receiving hospice care in 2829 hospices. METHODS: We calculated quality measure scores overall and stratified by setting, adjusting for mode of survey administration and differences in case mix, and examined variability in hospice-level scores among decedents with dementia. RESULTS: Mean quality measure scores ranged from 69.0 (Getting Hospice Care Training) to 90.9 (Getting Emotional Support). Measure scores varied significantly across settings, with caregivers of decedents who received care in a nursing home (NH), acute care hospital (ACH), or assisted living facility (ALF) consistently reporting poorer quality of care. Hospice-level scores varied substantially, with a wide range between the 10th and 90th percentiles of hospice performance (eg, 25 points). CONCLUSIONS AND IMPLICATIONS: There are important opportunities to improve hospice care for patients with dementia and their caregivers, particularly with respect to caregiver training, symptom management, and across all dimensions within the NH, ACH, and ALF settings. Variability in care experiences across hospices, as well as long lengths of stay for those with dementia, highlight the importance of informed and timely hospice referral.

National travel distances for emergency care.

BACKGROUND: Most emergency department (ED) patients arrive by their own transport and, for various reasons, may not choose the nearest ED. How far patients travel for ED treatment may reflect both patients' access to care and severity of illness. In this study, we aimed to examine the travel distance and travel time between a patient's home and ED they visited and investigate how these distances/times vary by patient and hospital characteristics. METHODS: We randomly sampled and collected data from 14,812 patients discharged to the community (DTC) between January and March 2016 from 50 hospital-based EDs nationwide. We geocoded and calculated the distance and travel time between patient and hospital-based ED addresses, examined the travel distances/ times between patients' home and the ED they visited, and used mixed-effects regression models to investigate how these distances/times vary by patient and hospital characteristics. RESULTS: Patients travelled an average of 8.0 (SD = 10.9) miles and 17.3 (SD = 18.0) driving minutes to the ED. Patients travelled significantly farther to avoid EDs in lower performing hospitals (p < 0.01) and in the West (p < 0.05) and Midwest (p < 0.05). Patients travelled farther when visiting EDs in rural areas. Younger patients travelled farther than older patients. CONCLUSIONS: Understanding how far patients are willing to travel is indicative of whether patient populations have adequate access to ED services. By showing that patients travel farther to avoid a low-performing hospital, we provide evidence that DTC patients likely do exercise some choice among EDs, indicating some market incentives for higher-quality care, even for some ED admissions. Understanding these issues will help policymakers better define access to ED care and assist in directing quality improvement efforts. To our knowledge, our study is the most comprehensive nationwide characterization of patient travel for ED treatment to date.

Development and Testing of an Emergency Department Quality Measure for Pediatric Suicidal Ideation and Self-Harm.

OBJECTIVE: To develop and test a new quality measure assessing timeliness of follow-up mental health care for youth presenting to the emergency department (ED) with suicidal ideation or self-harm. METHODS: Based on a conceptual framework, evidence review, and a modified Delphi process, we developed a quality measure assessing whether youth 5 to 17 years old evaluated for suicidal ideation or self-harm in the ED and discharged to home had a follow-up mental health care visit within 7 days. The measure was tested in 4 geographically dispersed states (California, Pennsylvania, South Carolina, Tennessee) using Medicaid administrative data. We examined measure feasibility of implementation, variation, reliability, and validity. To test validity, adjusted regression models examined associations between quality measure scores and subsequent all-cause and same-cause hospital readmissions/ED return visits. RESULTS: Overall, there were 16,486 eligible ED visits between September 1, 2014 and July 31, 2016; 53.5% of eligible ED visits had an associated mental health care follow-up visit within 7 days. Measure scores varied by state, ranging from 26.3% to 66.5%, and by youth characteristics: visits by youth who were non-White, male, and living in an urban area were significantly less likely to be associated with a follow-up visit within 7 days. Better quality measure performance was not associated with decreased reutilization. CONCLUSIONS: This new ED quality measure may be useful for monitoring and improving the quality of care for this vulnerable population; however, future work is needed to establish the measure's predictive validity using more prevalent outcomes such as recurrence of suicidal ideation or deliberate self-harm.