Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial.

BACKGROUND: Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity. METHODS: Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses. RESULTS: LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045). CONCLUSIONS: LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care.

Background: Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects. Method: A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate. Results: Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience. Conclusion: This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.

Using treatments with known positive impact in community-based mental health programs is challenging. Many studies suggest leaders of these programs can help. Similarly, certain features of community-based programs can also be helpful. This case study of an outpatient mental health clinic provides rich descriptions of actions leaders took that shaped the environment in their program and helped improve the use of a treatment with known positive impact. This case study can serve as a practical guide for leaders to reference when aiming to improve the use of treatments with known impact in their own programs.

Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health.

OBJECTIVE: Measurement-based care (MBC), which collects session-by-session symptom data from patients and provides clinicians with feedback on treatment response, is a highly generalizable evidence-based practice with significant potential to improve the outcomes of mental health treatment in youth when implemented with fidelity; however, it is rarely used in community settings. This study tested whether an implementation strategy targeting organizational leadership and organizational implementation climate could improve MBC fidelity and clinical outcomes for youth in outpatient mental health clinics. METHOD: In a cluster randomized trial, 21 clinics were assigned to the Leadership and Organizational Change for Implementation strategy plus training and technical assistance in MBC (k = 11, n = 117) or training and technical assistance only (k = 10, n = 117). Primary outcomes of MBC fidelity (assessed via electronic metadata) and youth symptom improvement (assessed via caregiver-reported change on the Shortform Assessment for Children Total Problem Score) were collected for consecutively enrolled youths (ages 4-18 years) who initiated treatment in the 12 months following MBC training. Outcomes of each youth were assessed for 6 months following baseline. RESULTS: A total of 234 youths were enrolled and included in intent-to-treat analyses. At baseline, there were no significant differences by condition in clinic, clinician, or youth characteristics. Youths in clinics using the Leadership and Organizational Change for Implementation strategy experienced significantly higher MBC fidelity compared with youths in control clinics (23.1% vs 3.4%, p = .014), and exhibited significantly greater reductions in symptoms from baseline to 6 months (d = 0.31, 95% CI: 0.04-0.58, p = .023). CONCLUSION: Implementation strategies targeting organizational leadership and focused implementation climate can improve fidelity to evidence-based practices and clinical outcomes of youth mental health services. CLINICAL TRIAL REGISTRATION INFORMATION: Working to Implement and Sustain Digital Outcome Measures (WISDOM); https://clinicaltrials.gov/; NCT04096274.

Association of organizational culture and climate with variation in the clinical outcomes of collaborative care for maternal depression in community health centers.

Background: Organizational factors may help explain variation in the effectiveness of evidence-based clinical innovations through implementation and sustainment. This study tested the relationship between organizational culture and climate and variation in clinical outcomes of the Collaborative Care Model (CoCM) for treatment of maternal depression implemented in community health centers. Method: Organizational cultures and climates of 10 community health centers providing CoCM for depression among low-income women pregnant or parenting were assessed using the organizational social context (OSC) measure. Three-level hierarchical linear models tested whether variation in culture and climate predicted variation in improvement in depression symptoms from baseline to 6.5-month post-baseline for N = 468 women with care ±1 year of OSC assessment. Depression symptomology was measured using the Patient Health Questionnaire (PHQ-9). Results: After controlling for patient characteristics, case mix, center size, and implementation support, patients served by centers with more proficient cultures improved significantly more from baseline to 6.5-month post-baseline than patients in centers with less proficient cultures (mean improvement = 5.08 vs. 0.14, respectively, p = .020), resulting in a large adjusted effect size of dadj = 0.78. A similar effect was observed for patients served by centers with more functional climates (mean improvement = 5.25 vs. 1.12, p < .044, dadj = 0.65). Growth models indicated that patients from all centers recovered on average after 4 months of care. However, those with more proficient cultures remained stabilized whereas patients served by centers with less proficient cultures deteriorated by 6.5-month post-baseline. A similar pattern was observed for functional climate. Conclusions: Variation in clinical outcomes for women from historically underserved populations receiving Collaborative Care for maternal depression was associated with the organizational cultures and climates of community health centers. Implementation strategies targeting culture and climate may improve the implementation and effectiveness of integrated behavioral health care for depression.

While many implementation theories espouse the importance of organizational culture and climate for the successful implementation of evidence-based practices in primary care, there is little research that tests this hypothesis. Since there are interventions which can improve organizational culture and climate, having more evidence that these factors are associated with implementation would support efforts to modify these aspects of a community health center as a means of improving implementation. This study showed that the extent to which patients clinically benefitted from the evidence-based Collaborative Care Model for maternal depression was related to the prevailing culture and climate of community health centers where they received treatment. Specifically, women seen at centers in which the staff and providers indicated that their organizations prioritize responsiveness to patients' needs over competing organizational goals and maintain competence in up-to-date treatment models (referred to as proficient culture), and understand their role in the organization and receive the cooperation and support they need from colleagues and supervisors to perform their job well (functional climate) were associated with sustained improvements in depression symptoms. This benefit was independent of other factors already known to be associated with these outcomes. Implementation strategies that target organizational culture (i.e., priorities and expectations for staff) and climate (i.e., quality of working environment) may improve the clinical outcomes of integrated collaborative care models for depression and reduce the commonly seen variation in outcomes across health centers.

Eight characteristics of rigorous multilevel implementation research: a step-by-step guide.

BACKGROUND: Although healthcare is delivered in inherently multilevel contexts, implementation science has no widely endorsed methodological standards defining the characteristics of rigorous, multilevel implementation research. We identify and describe eight characteristics of high-quality, multilevel implementation research to encourage discussion, spur debate, and guide decision-making around study design and methodological issues. RECOMMENDATIONS: Implementation researchers who conduct rigorous multilevel implementation research demonstrate the following eight characteristics. First, they map and operationalize the specific multilevel context for defined populations and settings. Second, they define and state the level of each construct under study. Third, they describe how constructs relate to each other within and across levels. Fourth, they specify the temporal scope of each phenomenon at each relevant level. Fifth, they align measurement choices and construction of analytic variables with the levels of theories selected (and hypotheses generated, if applicable). Sixth, they use a sampling strategy consistent with the selected theories or research objectives and sufficiently large and variable to examine relationships at requisite levels. Seventh, they align analytic approaches with the chosen theories (and hypotheses, if applicable), ensuring that they account for measurement dependencies and nested data structures. Eighth, they ensure inferences are made at the appropriate level. To guide implementation researchers and encourage debate, we present the rationale for each characteristic, actionable recommendations for operationalizing the characteristics in implementation research, a range of examples, and references to make the characteristics more usable. Our recommendations apply to all types of multilevel implementation study designs and approaches, including randomized trials, quantitative and qualitative observational studies, and mixed methods. CONCLUSION: These eight characteristics provide benchmarks for evaluating the quality and replicability of multilevel implementation research and promote a common language and reference points. This, in turn, facilitates knowledge generation across diverse multilevel settings and ensures that implementation research is consistent with (and appropriately leverages) what has already been learned in allied multilevel sciences. When a shared and integrated description of what constitutes rigor is defined and broadly communicated, implementation science is better positioned to innovate both methodologically and theoretically.

Eliminating Monitor Overuse (EMO) type III effectiveness-deimplementation cluster-randomized trial: Statistical analysis plan.

Background: Deimplementing overused health interventions is essential to maximizing quality and value while minimizing harm, waste, and inefficiencies. Three national guidelines discourage continuous pulse oximetry (SpO2) monitoring in children who are not receiving supplemental oxygen, but the guideline-discordant practice remains prevalent, making it a prime target for deimplementation. This paper details the statistical analysis plan for the Eliminating Monitor Overuse (EMO) SpO2 trial, which compares the effect of two competing deimplementation strategies (unlearning only vs. unlearning plus substitution) on the sustainment of deimplementation of SpO2 monitoring in children with bronchiolitis who are in room air. Methods: The EMO Trial is a hybrid type 3 effectiveness-deimplementation trial with a longitudinal cluster-randomized design, conducted in Pediatric Research in Inpatient Settings Network hospitals. The primary outcome is deimplementation sustainment, analyzed as a longitudinal difference-in-differences comparison between study arms. This analysis will use generalized hierarchical mixed-effects models for longitudinal clustering outcomes. Secondary outcomes include the length of hospital stay and oxygen supplementation duration, modeled using linear mixed-effects regressions. Using the well-established counterfactual approach, we will also perform a mediation analysis of hospital-level mechanistic measures on the association between the deimplementation strategy and the sustainment outcome. Discussion: We anticipate that the EMO Trial will advance the science of deimplementation by providing new insights into the processes, mechanisms, and likelihood of sustained practice change using rigorously designed deimplementation strategies. This pre-specified statistical analysis plan will mitigate reporting bias and support data-driven approaches. Trial registration: ClinicalTrials.gov NCT05132322. Registered on 24 November 2021.

Disparities in Youth and Family Experiences of System-of-Care Principles by Level of Youth Need.

The extent to which mental health services for youths embody system-of-care (SOC) principles is an important quality indicator. This study tested whether youth and family experiences of SOC principles varied depending on youths' level of need after adjusting for sociodemographic and treatment factors. The relationship to caregiver-reported clinical outcomes was also examined. Using administrative data and cross-sectional surveys from a stratified random sample of 1124 caregivers of youths ages 5-20 within a statewide system, adjusted analyses indicated caregivers of youths with the most intensive needs were significantly less likely to report receiving care that embodied SOC principles, with deficits on six of nine items. Youths whose services embodied SOC principles experienced significantly greater improvement in caregiver-reported functioning even after adjusting for level of need. Results highlight disparities in SOC principles for youths with intensive needs and the need for policy and intervention development to improve care for this population.

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy.

BACKGROUND: Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning. METHODS: Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples. RESULTS: In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples. CONCLUSIONS: Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings. TRIAL REGISTRATION: ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.

A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy.

Clinician fidelity to cognitive behavioral therapy (CBT) is an important mechanism by which desired clinical outcomes are achieved and is an indicator of care quality. Despite its importance, there are few fidelity measurement methods that are efficient and have demonstrated reliability and validity. Using a randomized trial design, we compared three methods of assessing CBT adherence-a core component of fidelity-to direct observation, the gold standard. Clinicians recruited from 27 community mental health agencies (n = 126; M age = 37.69 years, SD = 12.84; 75.7% female) were randomized 1:1:1 to one of three fidelity conditions: self-report (n = 41), chart-stimulated recall (semistructured interviews with the chart available; n = 42), or behavioral rehearsal (simulated role-plays; n = 43). All participating clinicians completed fidelity assessments for up to three sessions with three different clients that were recruited from clinicians' caseloads (n = 288; M age = 13.39 years SD = 3.89; 41.7% female); sessions were also audio-recorded and coded for comparison to determine the most accurate method. All fidelity measures had parallel scales that yielded an adherence maximum score (i.e., the highest-rated intervention in a session), a mean of techniques observed, and a count total of observed techniques. Results of three-level mixed effects regression models indicated that behavioral rehearsal produced comparable scores to observation for all adherence scores (all ps > .01), indicating no difference between behavioral rehearsal and observation. Self-report and chart-stimulated recall overestimated adherence compared to observation (ps < .01). Overall, findings suggested that behavioral rehearsal indexed CBT adherence comparably to direct observation, the gold-standard, in pediatric populations. Behavioral rehearsal may at times be able to replace the need for resource-intensive direct observation in implementation research and practice.

Sustainable deimplementation of continuous pulse oximetry monitoring in children hospitalized with bronchiolitis: study protocol for the Eliminating Monitor Overuse (EMO) type III effectiveness-deimplementation cluster-randomized trial.

BACKGROUND: Methods of sustaining the deimplementation of overused medical practices (i.e., practices not supported by evidence) are understudied. In pediatric hospital medicine, continuous pulse oximetry monitoring of children with the common viral respiratory illness bronchiolitis is recommended only under specific circumstances. Three national guidelines discourage its use for children who are not receiving supplemental oxygen, but guideline-discordant practice (i.e., overuse) remains prevalent. A 6-hospital pilot of educational outreach with audit and feedback resulted in immediate reductions in overuse; however, the best strategies to optimize sustainment of deimplementation success are unknown. METHODS: The Eliminating Monitor Overuse (EMO) trial will compare two deimplementation strategies in a hybrid type III effectiveness-deimplementation trial. This longitudinal cluster-randomized design will be conducted in Pediatric Research in Inpatient Settings (PRIS) Network hospitals and will include baseline measurement, active deimplementation, and sustainment phases. After a baseline measurement period, 16-19 hospitals will be randomized to a deimplementation strategy that targets unlearning (educational outreach with audit and feedback), and the other 16-19 will be randomized to a strategy that targets unlearning and substitution (adding an EHR-integrated clinical pathway decision support tool). The primary outcome is the sustainment of deimplementation in bronchiolitis patients who are not receiving any supplemental oxygen, analyzed as a longitudinal difference-in-differences comparison of overuse rates across study arms. Secondary outcomes include equity of deimplementation and the fidelity to, and cost of, each deimplementation strategy. To understand how the deimplementation strategies work, we will test hypothesized mechanisms of routinization (clinicians developing new routines supporting practice change) and institutionalization (embedding of practice change into existing organizational systems). DISCUSSION: The EMO trial will advance the science of deimplementation by providing new insights into the processes, mechanisms, costs, and likelihood of sustained practice change using rigorously designed deimplementation strategies. The trial will also advance care for a high-incidence, costly pediatric lung disease. TRIAL REGISTRATION: ClinicalTrials.gov, NCT05132322 . Registered on November 10, 2021.

Required sample size to detect mediation in 3-level implementation studies.

BACKGROUND: Statistical tests of mediation are important for advancing implementation science; however, little research has examined the sample sizes needed to detect mediation in 3-level designs (e.g., organization, provider, patient) that are common in implementation research. Using a generalizable Monte Carlo simulation method, this paper examines the sample sizes required to detect mediation in 3-level designs under a range of conditions plausible for implementation studies. METHOD: Statistical power was estimated for 17,496 3-level mediation designs in which the independent variable (X) resided at the highest cluster level (e.g., organization), the mediator (M) resided at the intermediate nested level (e.g., provider), and the outcome (Y) resided at the lowest nested level (e.g., patient). Designs varied by sample size per level, intraclass correlation coefficients of M and Y, effect sizes of the two paths constituting the indirect (mediation) effect (i.e., X→M and M→Y), and size of the direct effect. Power estimates were generated for all designs using two statistical models-conventional linear multilevel modeling of manifest variables (MVM) and multilevel structural equation modeling (MSEM)-for both 1- and 2-sided hypothesis tests. RESULTS: For 2-sided tests, statistical power to detect mediation was sufficient (≥0.8) in only 463 designs (2.6%) estimated using MVM and 228 designs (1.3%) estimated using MSEM; the minimum number of highest-level units needed to achieve adequate power was 40; the minimum total sample size was 900 observations. For 1-sided tests, 808 designs (4.6%) estimated using MVM and 369 designs (2.1%) estimated using MSEM had adequate power; the minimum number of highest-level units was 20; the minimum total sample was 600. At least one large effect size for either the X→M or M→Y path was necessary to achieve adequate power across all conditions. CONCLUSIONS: While our analysis has important limitations, results suggest many of the 3-level mediation designs that can realistically be conducted in implementation research lack statistical power to detect mediation of highest-level independent variables unless effect sizes are large and 40 or more highest-level units are enrolled. We suggest strategies to increase statistical power for multilevel mediation designs and innovations to improve the feasibility of mediation tests in implementation research.

Testing a Theory of Implementation Leadership and Climate Across Autism Evidence-Based Interventions of Varying Complexity.

Prominent theories within the field of implementation science contend that organizational leaders can improve providers' fidelity to evidence-based practices (EBPs) by using focused implementation leadership behaviors that create an organizational climate for EBP implementation. However, this work has been criticized for overreliance on nonspecific, self-report fidelity measures and poor articulation of the boundary conditions that may attenuate leadership and climate's influence. This study tests the predictions of EBP implementation leadership and climate theory on observed fidelity to three school-based EBPs for autism that vary in complexity: pivotal response training (PRT), discrete trial training (DTT), and visual schedules (VS). Educators in kindergarten to third-grade autism support classrooms in 65 schools assessed their principals' EBP implementation leadership and school EBP implementation climate prior to the school year. Mid-school year, trained observers rated educator fidelity to all three interventions. Expert raters confirmed PRT was significantly more complex than DTT or VS using the Intervention Complexity Assessment Tool for Systematic Reviews. Linear regression analyses at the school level indicated principals' increased frequency of EBP implementation leadership predicted a higher school EBP implementation climate, which in turn predicted higher educator fidelity to PRT-however, there was no evidence of a relationship between implementation climate and fidelity to DTT or VS. Comparing principals whose EBP implementation leadership was ±1 SD from the mean, there was a significant indirect association of EBP implementation leadership with PRT fidelity through EBP implementation climate (d = 0.49, 95% CI [0.04, 0.93]). Strategies that target EBP implementation leadership and climate may support fidelity to complex behavioral interventions.

Organization-level variation in therapists' attitudes toward and use of measurement-based care.

PURPOSE: Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation. METHODS: Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes. RESULTS: Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC. CONCLUSIONS: There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: a lagged analysis.

BACKGROUND: Theory and empirical research suggest organizational climate for evidence-based practice (EBP) implementation may be an important and malleable target to improve clinician use of EBPs in healthcare; however, this work has been criticized for overreliance on self-report measures of implementation outcomes and cross-sectional designs. This study combines data from two studies spanning 7 years to test the hypothesis that higher levels of organizational EBP implementation climate prospectively predicts improved clinician adherence to an EBP, cognitive behavioral therapy (CBT), as rated by expert observers. METHODS: Biennial assessments of EBP implementation climate collected in 10 community mental health agencies in Philadelphia as part of a systemwide evaluation (time 1) were linked to subsequent observer ratings of clinician adherence to CBT in clinical encounters with 108 youth (time 2). Experts rated clinician adherence to CBT using the Therapy Process Observation Coding System which generated two primary outcomes (a) maximum CBT adherence per session (i.e., highest rated CBT intervention per session; depth of delivery) and (b) average CBT adherence per session (i.e., mean rating across all CBT interventions used; depth and breadth of delivery). RESULTS: On average, time 2 clinician adherence observations occurred 19.8 months (SD = 10.15) after time 1 organizational climate assessments. Adjusting for organization, clinician, and client covariates, a one standard deviation increase in organizational EBP implementation climate at time 1 predicted a 0.63-point increase in clinicians' maximum CBT adherence per session at time 2 (p = 0.000), representing a large effect size (d = 0.93; 95% CI = 0.63-1.24) when comparing organizations in the upper (k = 3) versus lower tertiles (k = 3) of EBP implementation climate. Higher levels of time 1 organizational EBP implementation climate also predicted higher time 2 average CBT adherence per session (b = 0.23, p < 0.001, d = 0.72). Length of time between assessments of climate and adherence did not moderate these relationships. CONCLUSIONS: Organizational EBP implementation climate is a promising predictor of clinicians' subsequent observed adherence to CBT. Implementation strategies that target this antecedent may improve the delivery of EBPs in healthcare settings.

In Search of the Common Elements of Clinical Supervision: A Systematic Review.

The importance of clinical supervision for supporting effective implementation of evidence-based treatments (EBTs) is widely accepted; however, very little is known about which supervision practice elements contribute to implementation effectiveness. This systematic review aimed to generate a taxonomy of empirically-supported supervision practice elements that have been used in treatment trials and shown to independently predict improved EBT implementation. Supervision practice elements were identified using a two-phase, empirically-validated distillation process. In Phase I, a systematic review identified supervision protocols that had evidence of effectiveness based on (a) inclusion in one or more EBT trials, and (b) independent association with improved EBT implementation in one or more secondary studies. In Phase II, a hybrid deductive-inductive coding process was applied to the supervision protocols to characterize the nature and frequency of supervision practice elements across EBTs. Twenty-one of the 876 identified articles assessed the associations of supervision protocols with implementation or clinical outcomes, representing 13 separate studies. Coding and distillation of the supervision protocols resulted in a taxonomy of 21 supervision practice elements. The most frequently used elements were: reviewing supervisees' practice (92%; n = 12), clinical suggestions (85%; n = 11), behavioral rehearsal (77%; n = 10), elicitation (77%; n = 10), and fidelity assessment (77%; n = 10). This review identified supervision practice elements that could be targets for future research testing which elements are necessary and sufficient to support effective EBT implementation. Discrepancies between supervision practice elements observed in trials as compared to routine practice highlights the importance of research addressing supervision-focused implementation strategies.

Analysis of Specialist and Patient Perspectives on Strategies to Improve Cardiovascular Disease Prevention Among Persons With Psoriatic Disease.

IMPORTANCE: Patients with psoriatic disease are at increased risk of cardiovascular disease (CVD), which is a leading cause of mortality in this population. However, many of these patients do not have an active relationship with a primary care physician, and there may be a role for specialist-led care in prevention of CVD. OBJECTIVE: To explore clinician and patient perspectives regarding strategies to improve CVD prevention via specialist-led care. DESIGN, SETTING, AND PARTICIPANTS: Using electronically collected surveys, a best-worst scaling experimental survey study was conducted among dermatologists through the National Psoriasis Foundation as well as the American Academy of Dermatology from October 27, 2020, to April 1, 2021, to rank the strategies according to their potential to improve CVD prevention among patients with psoriatic disease. Participants were asked about the feasibility of specialist-led screening through an electronically delivered survey from the National Psoriasis Foundation conducted between February 1 and April 21, 2021. Patients with psoriatic disease were asked about whether they would like the specialist to screen for CVD risk factors. In addition, patients reported their likelihood to engage in CVD risk screening and management behaviors in scenarios in which either the primary care physician or specialist was making the recommendations. MAIN OUTCOMES AND MEASURES: For the clinician surveys, the primary outcome was the ratio scaled preference score (range, 0-100; higher is more preferred), as well as whether they think calculating a 10-year CVD risk score and prescribing statins seems feasible. For the patient surveys, the primary outcome was the likelihood to check cholesterol level, incorporate diet and exercise, or use statin therapy depending on whether recommended by the specialist or primary care physician, whether they would like their specialist to educate them about CVD risk, and whether they would find it convenient to have their cholesterol level checked by their specialist. RESULTS: Among 183 dermatologists (102 [55.7%] women; mean [SD] age not collected), clinical decision support (preference score, 22.3; 95% CI, 20.7-24.0), patient education (preference score, 14.1; 95% CI, 12.5-15.7), and clinician education (preference score, 15.8; 95% CI, 14.3-17.3) were ranked as strategies likely to improve CVD prevention in patients with psoriatic disease. In addition, 69.3% (95% CI, 62.2%-76.0%) of dermatologists agreed or strongly agreed that checking lipid levels was feasible. Among 160 patients with psoriasis and 162 patients with psoriatic arthritis (226 [70.2%] women; mean [SD] age, 54 [13.3] years), patients reported they were as likely to engage in cardiovascular risk screening and management behaviors whether recommended by their primary care physician or their specialist. In addition, 60.0% (95% CI, 52.0%-67.7%) of patients with psoriasis and 75.3% (95% CI, 67.9%-81.7%) of those with psoriatic arthritis agreed that it would be convenient for them to have their cholesterol checked by their dermatologist/rheumatologist. CONCLUSIONS AND RELEVANCE: In this survey study, dermatologists and patients with psoriatic disease expressed positive perspectives about engaging in a specialist-led model of care to improve CVD prevention. Dermatologists appear to view several strategies as having potential to improve cardiovascular risk prevention.

Assessing Causal Pathways and Targets of Implementation Variability for EBP use (Project ACTIVE): a study protocol.

BACKGROUND: Advancing causal implementation theory is critical for designing tailored implementation strategies that target specific mechanisms associated with evidence-based practice (EBP) use. This study will test the generalizability of a conceptual model that integrates organizational constructs and behavioral theory to predict clinician use of cognitive-behavioral therapy (CBT) techniques in community mental health centers. CBT is a leading psychosocial EBP for psychiatric disorders that remains underused despite substantial efforts to increase its implementation. METHODS: We will leverage ongoing CBT implementation efforts in two large public health systems (Philadelphia and Texas) to recruit 300 mental health clinicians and 600 of their clients across 40 organizations. Our primary implementation outcomes of interest are clinician intentions to use CBT and direct observation of clinician use of CBT. As CBT comprises discrete components that vary in complexity and acceptability, we will measure clinician use of six discrete components of CBT. After finishing their CBT training, participating clinicians will complete measures of organizational and behavior change constructs delineated in the model. Clinicians also will be observed twice via audio recording delivering CBT with a client. Within 48 h of each observation, theorized moderators of the intention-behavior gap will be collected via survey. A subset of clinicians who report high intentions to use CBT but demonstrate low use will be purposively recruited to complete semi-structured interviews assessing reasons for the intention-behavior gap. Multilevel path analysis will test the extent to which intentions and determinants of intention predict the use of each discrete CBT component. We also will test the extent to which theorized determinants of intention that include psychological, organizational, and contextual factors explain variation in intention and moderate the association between intentions and CBT use. DISCUSSION: Project ACTIVE will advance implementation theory, currently in its infancy, by testing the generalizability of a promising causal model of implementation. These results will inform the development of implementation strategies targeting modifiable factors that explain substantial variance in intention and implementation that can be applied broadly across EBPs.

Efficacy of Probiotics as Prophylaxis for Urinary Tract Infections in Premenopausal Women: A Systematic Review and Meta-Analysis.

INTRODUCTION: Although antibiotic therapy has been the mainstay of prophylaxis and treatment of urinary tract infections (UTIs), antibacterial resistance has led to increased incidence of infections and healthcare spending in both community-acquired and nosocomial UTIs. This has led to an active exploration of alternative remedies for both the prophylaxis and treatment of UTIs, especially in women with recurrent urinary tract infections. Probiotic supplementation is one novel intervention that has been studied as a prophylactic measure in patients with UTIs. The current systematic review and meta-analysis was conducted to evaluate the efficacy of probiotics for prophylaxis in UTIs in premenopausal women. METHODS: Detailed search strategies for each electronic database were developed for PubMed, EMBASE, and Scopus to identify relevant literature published between 2001-2021. RevMan 5.3 statistical software was used to analyze data in studies. The random-effects model was used for pooling the data. The risk of bias and study quality were assessed using Cochrane Collaboration's tool for assessing risk of bias in included studies. The scope of focus for this review was premenopausal adult women with a history of one or more UTI. The intervention consisted of a probiotic regimen for which the goal was to enhance the defensive microflora of the urogenital tract. Studies comparing a probiotic regimen to a placebo regimen were included. These studies' primary outcome was the proportion of women with at least one symptomatic bacterial UTI in each group (i.e., UTI recurrence rate) in the 12-month period following probiotic intervention. This study extends the work of researchers who systematically investigated the scientific literature on probiotics in the prevention of urinary tract infections with a particular focus on premenopausal women. RESULTS: After screening, three parallel-group randomized-controlled trials (RCTs) were included. We estimated the overall pooled data of these three studies with a total of 284 participants to have met the predefined inclusion criteria and were therefore included in this review. The results demonstrated that probiotics did not have a significant effect in the prophylaxis of UTIs. (Risk Ratio (RR): 0.59 confidence interval (CI): 0.26, 1.33), Heterogeneity: Chi² = 6.63, df = 2 (p = 0.04); I² =70%, Test for overall effect: Z = 1.27 (p = 0.20).  Conclusions: Probiotics did not demonstrate a significant benefit in reducing UTI recurrence compared to placebo in premenopausal women. However, more conclusive data is needed to determine the effect that probiotics have on strengthening the urogenital microbial barrier against pathogenic bacteria and protecting against UTI recurrence.

Study protocol for a type III hybrid effectiveness-implementation trial of strategies to implement firearm safety promotion as a universal suicide prevention strategy in pediatric primary care.

BACKGROUND: Insights from behavioral economics, or how individuals' decisions and behaviors are shaped by finite cognitive resources (e.g., time, attention) and mental heuristics, have been underutilized in efforts to increase the use of evidence-based practices in implementation science. Using the example of firearm safety promotion in pediatric primary care, which addresses an evidence-to-practice gap in universal suicide prevention, we aim to determine: is a less costly and more scalable behavioral economic-informed implementation strategy (i.e., "Nudge") powerful enough to change clinician behavior or is a more intensive and expensive facilitation strategy needed to overcome implementation barriers? METHODS: The Adolescent and child Suicide Prevention in Routine clinical Encounters (ASPIRE) hybrid type III effectiveness-implementation trial uses a longitudinal cluster randomized design. We will test the comparative effectiveness of two implementation strategies to support clinicians' use of an evidence-based firearm safety practice, S.A.F.E. Firearm, in 32 pediatric practices across two health systems. All pediatric practices in the two health systems will receive S.A.F.E. Firearm materials, including training and cable locks. Half of the practices (k = 16) will be randomized to receive Nudge; the other half (k = 16) will be randomized to receive Nudge plus 1 year of facilitation to target additional practice and clinician implementation barriers (Nudge+). The primary implementation outcome is parent-reported clinician fidelity to the S.A.F.E Firearm program. Secondary implementation outcomes include reach and cost. To understand how the implementation strategies work, the primary mechanism to be tested is practice adaptive reserve, a self-report practice-level measure that includes relationship infrastructure, facilitative leadership, sense-making, teamwork, work environment, and culture of learning. DISCUSSION: The ASPIRE trial will integrate implementation science and behavioral economic approaches to advance our understanding of methods for implementing evidence-based firearm safety promotion practices in pediatric primary care. The study answers a question at the heart of many practice change efforts: which strategies are sufficient to support change, and why? Results of the trial will offer valuable insights into how best to implement evidence-based practices that address sensitive health matters in pediatric primary care. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04844021 . Registered 14 April 2021.