Disparities in Youth and Family Experiences of System-of-Care Principles by Level of Youth Need.

The extent to which mental health services for youths embody system-of-care (SOC) principles is an important quality indicator. This study tested whether youth and family experiences of SOC principles varied depending on youths' level of need after adjusting for sociodemographic and treatment factors. The relationship to caregiver-reported clinical outcomes was also examined. Using administrative data and cross-sectional surveys from a stratified random sample of 1124 caregivers of youths ages 5-20 within a statewide system, adjusted analyses indicated caregivers of youths with the most intensive needs were significantly less likely to report receiving care that embodied SOC principles, with deficits on six of nine items. Youths whose services embodied SOC principles experienced significantly greater improvement in caregiver-reported functioning even after adjusting for level of need. Results highlight disparities in SOC principles for youths with intensive needs and the need for policy and intervention development to improve care for this population.

In Search of the Common Elements of Clinical Supervision: A Systematic Review.

The importance of clinical supervision for supporting effective implementation of evidence-based treatments (EBTs) is widely accepted; however, very little is known about which supervision practice elements contribute to implementation effectiveness. This systematic review aimed to generate a taxonomy of empirically-supported supervision practice elements that have been used in treatment trials and shown to independently predict improved EBT implementation. Supervision practice elements were identified using a two-phase, empirically-validated distillation process. In Phase I, a systematic review identified supervision protocols that had evidence of effectiveness based on (a) inclusion in one or more EBT trials, and (b) independent association with improved EBT implementation in one or more secondary studies. In Phase II, a hybrid deductive-inductive coding process was applied to the supervision protocols to characterize the nature and frequency of supervision practice elements across EBTs. Twenty-one of the 876 identified articles assessed the associations of supervision protocols with implementation or clinical outcomes, representing 13 separate studies. Coding and distillation of the supervision protocols resulted in a taxonomy of 21 supervision practice elements. The most frequently used elements were: reviewing supervisees' practice (92%; n = 12), clinical suggestions (85%; n = 11), behavioral rehearsal (77%; n = 10), elicitation (77%; n = 10), and fidelity assessment (77%; n = 10). This review identified supervision practice elements that could be targets for future research testing which elements are necessary and sufficient to support effective EBT implementation. Discrepancies between supervision practice elements observed in trials as compared to routine practice highlights the importance of research addressing supervision-focused implementation strategies.

Organization-level variation in therapists' attitudes toward and use of measurement-based care.

PURPOSE: Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation. METHODS: Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes. RESULTS: Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC. CONCLUSIONS: There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Community stakeholder preferences for evidence-based practice implementation strategies in behavioral health: a best-worst scaling choice experiment.

BACKGROUND: Community behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling. METHODS: A total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles. RESULTS: On average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics. CONCLUSIONS: The presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.

The Organizational Financial Context of Publicly-Funded Mental Health Clinics: Development and Preliminary Psychometric Evaluation of the Agency Financial Status Scales.

Funding is a major barrier to implementation of evidence-based practices (EBPs) in publicly-funded community mental health clinics (CMHCs). Understanding how best to deploy implementation strategies that address this barrier requires greater clarity on the financial context within agencies. We developed the Agency Financial Status Scales (AFSS) to assess employee perceptions of the level of three hypothesized and theoretical funding related constructs in organizations: (a) perceptions of financial health, (b) financial attitudes toward EBPs, and (c) strategic financial climate. This investigation serves as a preliminary evaluation of this measure. Participants were 239 therapists and 40 supervisors from 25 publicly-funded CMHCs providing outpatient mental health services for young people. Confirmatory factor analysis was used to investigate the latent trait structure of the items. Internal consistency, interrater agreement, concordance between therapists and supervisors, and convergent validity were also examined. A two-factor model measuring perceptions of financial health and strategic financial climate best fit the data. For both of these scales, alpha reliability was acceptable and agreement statistics provided moderate support for aggregation at the organizational level. Analyses supported the convergent validity of the scales. The development and preliminary evaluation of the AFSS is an important first step in understanding the financial context of publicly-funded CMHCs. Though findings from this investigation are promising, additional development and testing are needed to develop a more thorough understanding of the constructs and to improve the validity and reliability of this measure.

Applying NUDGE to Inform Design of EBP Implementation Strategies in Community Mental Health Settings.

We demonstrate the application of NUDGE (Narrow, Understand, Discover, Generate, Evaluate), a behavioral economics approach to systematically identifying behavioral barriers that impede behavior enactment, to the challenge of evidence-based practice (EBP) use in community behavioral health. Drawing on 65 clinician responses to a system-wide crowdsourcing challenge about EBP underutilization, we applied NUDGE to discover, synthesize and validate specific behavioral barriers to EBP utilization that directly inform the design of tailored implementation strategies. To our knowledge, this is the first study to apply behavioral economic insights to clinician-proposed solutions to implementation challenges in order to design implementation strategies. The study demonstrates the successful application of NUDGE to implementation strategy design and provides novel targets for intervention.

Identifying the organizational innovation-specific capacity needed for exposure therapy.

BACKGROUND: Current approaches to increasing the rates of clinician use of exposure therapy for anxiety disorders in community settings are limited. Research underscores the importance of addressing contextual variables to facilitate clinician use of evidence-based practices; however, no studies have identified the innovation-specific organizational capacity necessary to implement exposure therapy. Such work is critical to ensure that treatment-seeking individuals with anxiety receive effective care. METHODS: We used a two-step process to identify the innovation-specific organizational capacity necessary to deliver exposure. First, 24 leaders of specialty anxiety clinics in the United States (50% female, mean [M]age = 47.7 years) completed a survey about the organizational innovation-specific capacity (e.g., policies and procedures) they employ to support their providers in delivering exposure therapy. Second, 19 community clinicians (79% female, M age = 42.9 years) reported on the extent to which these characteristics were present in their settings. RESULTS: In Step 1, specialty clinic leaders unanimously endorsed six organizational characteristics as essential and five as important within the areas of organizational policies, supervisory support, and peer clinician support. These characteristics were present in more than 90% of specialty clinics. In Step 2, therapists in community clinics reported these characteristics were minimally present in their organizations. CONCLUSIONS: Specialty clinic leaders exhibited consensus on the innovation-specific organizational capacity necessary to implement exposure therapy. Identified characteristics were largely absent from community clinics. Developing fiscal, policy, or organizational strategies that enhance the organizational capacity within community settings may improve the patients' access to effective treatment for anxiety disorders.

Annual Research Review: The state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field.

BACKGROUND: Scientists have developed evidence-based interventions that improve the symptoms and functioning of youth with psychiatric disorders; however, these interventions are rarely used in community settings. Eliminating this research-to-practice gap is the purview of implementation science, the discipline devoted to the study of methods to promote the use of evidence-based practices in routine care. METHODS: We review studies that have tested factors associated with implementation in child psychology and psychiatry, explore applications of social science theories to implementation, and conclude with recommendations to advance implementation science through the development and testing of novel, multilevel, causal theories. RESULTS: During its brief history, implementation science in child psychology and psychiatry has documented the implementation gap in routine care, tested training approaches and found them to be insufficient for behavior change, explored the relationships between variables and implementation outcomes, and initiated randomized controlled trials to test implementation strategies. This research has identified targets related to implementation (e.g., clinician motivation, organizational culture) and demonstrated the feasibility of activating these targets through implementation strategies. However, the dominant methodological approach has been atheoretical and predictive, relying heavily on a set of variables from heuristic frameworks. CONCLUSIONS: Optimizing the implementation of effective treatments in community care for youth with psychiatric disorders is a defining challenge of our time. This review proposes a new direction focused on developing and testing integrated causal theories. We recommend implementation scientists: (a) move from observational studies of implementation barriers and facilitators to trials that include causal theory; (b) identify a core set of implementation determinants; (c) conduct trials of implementation strategies with clear targets, mechanisms, and outcomes; (d) ensure that behaviors that are core to EBPs are clearly defined; and (e) agree upon standard measures. This agenda will help fulfill the promise of evidence-based practice for improving youth behavioral health.

Interactions Between Clinician and Organizational Characteristics to Predict Cognitive-Behavioral and Psychodynamic Therapy Use.

Conceptual models of implementation posit contextual factors and their associations with evidence-based practice (EBP) use at multiple levels and suggest these factors exhibit complex cross-level interactions. Little empirical work has examined these interactions, which is critical to advancing causal implementation theory and optimizing implementation strategy design. Mixed effects regression examined cross-level interactions between clinician (knowledge, attitudes) and organizational characteristics (culture, climate) to predict cognitive-behavioral and psychodynamic therapy use with youth (N = 247 clinicians across 28 agencies). Results indicated several interactions, highlighting the importance of attending to interactions between variables at multiple levels to advance multilevel implementation theory and strategies.

Therapist Financial Strain and Turnover: Interactions with System-Level Implementation of Evidence-Based Practices.

Therapist turnover is a major problem in community mental health. Financial strain, which is composed of cognitive, emotional, and behavioral responses to the experience of economic hardship, is an understudied antecedent of therapist turnover given the tumultuous financial environment in community mental health. We prospectively examined the relationship between therapist financial strain and turnover in 247 therapists in 28 community mental health agencies. We expected greater therapist financial strain to predict higher turnover and participation in a system-funded evidence-based practice (EBP) training initiative to alleviate this effect. Controlling for covariates, financial strain predicted therapist turnover (OR 1.12, p = .045), but not for therapists who participated in an EBP training initiative. Reducing financial strain and/or promoting EBP implementation may be levers to reduce turnover.

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate.

Organizational culture and climate are important determinants of behavioral health service delivery for youth. The Organizational Social Context measure is a well validated assessment of organizational culture and climate that has been developed and extensively used in public sector behavioral health service settings. The degree of concordance between administrators and clinicians in their reports of organizational culture and climate may have implications for research design, inferences, and organizational intervention. However, the extent to which administrators' and clinicians' reports demonstrate concordance is just beginning to garner attention in public behavioral health settings in the United States. We investigated the concordance between 73 administrators (i.e., supervisors, clinical directors, and executive directors) and 247 clinicians in 28 child-serving programs in a public behavioral health system. Findings suggest that administrators, compared to clinicians, reported more positive cultures and climates. Organizational size moderated this relationship such that administrators in small programs (<466 youth clients served annually) provided more congruent reports of culture and climate in contrast to administrators in large programs (≥466 youth clients served annually) who reported more positive cultures and climates than clinicians. We propose a research agenda that examines the effect of concordance between administrators and clinicians on organizational outcomes in public behavioral health service settings.

Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians' EBP Adoption Through Improved Organizational Culture and Capacity.

The development of efficient and scalable implementation strategies in mental health is restricted by poor understanding of the change mechanisms that increase clinicians' evidence-based practice (EBP) adoption. This study tests the cross-level change mechanisms that link an empirically-supported organizational strategy for supporting implementation (labeled ARC for Availability, Responsiveness, and Continuity) to mental health clinicians' EBP adoption and use. Four hundred seventy-five mental health clinicians in 14 children's mental health agencies were randomly assigned to the ARC intervention or a control condition. Measures of organizational culture, clinicians' intentions to adopt EBPs, and job-related EBP barriers were collected before, during, and upon completion of the three-year ARC intervention. EBP adoption and use were assessed at 12-month follow-up. Multilevel mediation analyses tested changes in organizational culture, clinicians' intentions to adopt EBPs, and job-related EBP barriers as linking mechanisms explaining the effects of ARC on clinicians' EBP adoption and use. ARC increased clinicians' EBP adoption (OR = 3.19, p = .003) and use (81 vs. 56 %, d = .79, p = .003) at 12-month follow-up. These effects were mediated by improvement in organizational proficiency culture leading to increased clinician intentions to adopt EBPs and by reduced job-related EBP barriers. A combined mediation analysis indicated the organizational culture-EBP intentions mechanism was the primary carrier of ARC's effects on clinicians' EBP adoption and use. ARC increases clinicians' EBP adoption and use by creating proficient organizational cultures that increase clinicians' intentions to adopt EBPs.

Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice.

A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.

Assessing and changing organizational social contexts for effective mental health services.

Culture and climate are critical dimensions of a mental health service organization's social context that affect the quality and outcomes of the services it provides and the implementation of innovations such as evidence-based treatments (EBTs). We describe a measure of culture and climate labeled Organizational Social Context (OSC), which has been associated with innovation, service quality, and outcomes in national samples and randomized controlled trials (RCTs) of mental health and social service organizations. The article also describes an empirically supported organizational intervention model labeled Availability, Responsiveness, and Continuity (ARC), which has improved organizational social context, innovation, and effectiveness in five RCTs. Finally, the article outlines a research agenda for developing more efficient and scalable organizational strategies to improve mental health services by identifying the mechanisms that link organizational interventions and social context to individual-level service provider intentions and behaviors associated with innovation and effectiveness.

The organizational social context of mental health medicaid waiver programs with family support services: implications for research and practice.

Peer family support specialists (FSS) are parents with practical experience in navigating children's mental health care systems who provide support, advocacy, and guidance to the families of children who need mental health services. Their experience and training differ from those of formally trained mental health clinicians, creating potential conflicts in priorities and values between FSS and clinicians. We hypothesized that these differences could negatively affect the organizational cultures and climates of mental health clinics that employ both FSS and mental health clinicians, and lower the job satisfaction and organizational commitment of FSS. The Organizational Social Context measure was administered on site to 209 FSS and clinicians in 21 mental health programs in New York State. The study compared the organizational-level culture and climate profiles of mental health clinics that employ both FSS and formally trained clinicians to national norms for child mental health clinics, assessed individual-level job satisfaction and organizational commitment as a function of job (FSS vs. clinician) and other individual-level and organizational-level characteristics, and tested whether FSS and clinicians job attitudes were differentially associated with organizational culture and climate. The programs organizational culture and climate profiles were not significantly different from national norms. Individual-level job satisfaction and organizational commitment were unrelated to position (FSS vs. clinician) or other individual-level and organizational-level characteristics except for culture and climate. Both FSS' and clinicians' individual-level work attitudes were associated similarly with organizational culture and climate.

Calculating power for multilevel implementation trials in mental health: Meaningful effect sizes, intraclass correlation coefficients, and proportions of variance explained by covariates.

Background: Despite the ubiquity of multilevel sampling, design, and analysis in mental health implementation trials, few resources are available that provide reference values of design parameters (e.g., effect size, intraclass correlation coefficient [ICC], and proportion of variance explained by covariates [covariate R 2]) needed to accurately determine sample size. The aim of this study was to provide empirical reference values for these parameters by aggregating data on implementation and clinical outcomes from multilevel implementation trials, including cluster randomized trials and individually randomized repeated measures trials, in mental health. The compendium of design parameters presented here represents plausible values that implementation scientists can use to guide sample size calculations for future trials. Method: We searched NIH RePORTER for all federally funded, multilevel implementation trials addressing mental health populations and settings from 2010 to 2020. For all continuous and binary implementation and clinical outcomes included in eligible trials, we generated values of effect size, ICC, and covariate R2 at each level via secondary analysis of trial data or via extraction of estimates from analyses in published research reports. Effect sizes were calculated as Cohen d; ICCs were generated via one-way random effects ANOVAs; covariate R2 estimates were calculated using the reduction in variance approach. Results: Seventeen trials were eligible, reporting on 53 implementation and clinical outcomes and 81 contrasts between implementation conditions. Tables of effect size, ICC, and covariate R2 are provided to guide implementation researchers in power analyses for designing multilevel implementation trials in mental health settings, including two- and three-level cluster randomized designs and unit-randomized repeated-measures designs. Conclusions: Researchers can use the empirical reference values reported in this study to develop meaningful sample size determinations for multilevel implementation trials in mental health. Discussion focuses on the application of the reference values reported in this study.

To improve the planning and execution of implementation research in mental health settings, researchers need accurate estimates of several key metrics to help determine what sample size should be obtained at each level of a multi-level study (e.g., number of patients, doctors, and clinics). These metrics include the (1) effect size, which indicates how large of a difference in the primary outcome is expected between a treatment and control group, (2) intraclass correlation coefficient, which describes how similar two people in the same group might be, and (3) covariate R ², which indicates how much of the variability in an outcome is explained by a background variable, such as level of health at the start of a study. We collected data from mental health implementation trials conducted between 2010 and 2020. We extracted information about each of these metrics and aggregated the results for researchers to use in planning their own studies. Seventeen trials were eligible, and we were able to obtain statistical information on 53 different outcome variables from these studies. We provide a set of values which will assist in sample size calculations for future mental health implementation trials.

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial.

BACKGROUND: Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity. METHODS: Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses. RESULTS: LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045). CONCLUSIONS: LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care.

Background: Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects. Method: A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate. Results: Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience. Conclusion: This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.

Using treatments with known positive impact in community-based mental health programs is challenging. Many studies suggest leaders of these programs can help. Similarly, certain features of community-based programs can also be helpful. This case study of an outpatient mental health clinic provides rich descriptions of actions leaders took that shaped the environment in their program and helped improve the use of a treatment with known positive impact. This case study can serve as a practical guide for leaders to reference when aiming to improve the use of treatments with known impact in their own programs.

Implementation of a Secure Firearm Storage Program in Pediatric Primary Care: A Cluster Randomized Trial.

Importance: Increased secure firearm storage can reduce youth firearm injury and mortality, a leading cause of death for children and adolescents in the US. Despite the availability of evidence-based secure firearm storage programs and recommendations from the American Academy of Pediatrics, few pediatric clinicians report routinely implementing these programs. Objective: To compare the effectiveness of an electronic health record (EHR) documentation template (nudge) and the nudge plus facilitation (ie, clinic support to implement the program; nudge+) at promoting delivery of a brief evidence-based secure firearm storage program (SAFE Firearm) that includes counseling about secure firearm storage and free cable locks during all pediatric well visits. Design, Setting, and Participants: The Adolescent and Child Suicide Prevention in Routine Clinical Encounters (ASPIRE) unblinded parallel cluster randomized effectiveness-implementation trial was conducted from March 14, 2022, to March 20, 2023, to test the hypothesis that, relative to nudge, nudge+ would result in delivery of the firearm storage program to an additional 10% or more of the eligible population, and that this difference would be statistically significant. Thirty pediatric primary care clinics in 2 US health care systems (in Michigan and Colorado) were included, excluding clinics that were not the primary site for participating health care professionals and a subset selected at random due to resource limitations. All pediatric well visits at participating clinics for youth ages 5 to 17 years were analyzed. Interventions: Clinics were randomly assigned in a 1:1 ratio to receive either the nudge or nudge+. Main Outcomes and Measures: Patient-level outcomes were modeled to estimate the primary outcome, reach, which is a visit-level binary indicator of whether the parent received both components of the firearm storage program (counseling and lock), as documented by the clinician in the EHR. Secondary outcomes explored individual program component delivery. Results: A total of 47 307 well-child visits (median [IQR] age, 11.3 [8.1-14.4] years; 24 210 [51.2%] male and 23 091 [48.8%] female) among 46 597 children and 368 clinicians were eligible to receive the firearm storage program during the trial and were included in analyses. Using the intention-to-treat principle, a higher percentage of well-child visits received the firearm storage program in the nudge+ condition (49%; 95% CI, 37-61) compared to nudge (22%; 95% CI, 13-31). Conclusions and Relevance: In this study, the EHR strategy combined with facilitation (nudge+) was more effective at increasing delivery of an evidence-based secure firearm storage program compared to nudge alone. Trial Registration: ClinicalTrials.gov Identifier: NCT04844021.