Randomized trial of the leadership and organizational change for implementation (LOCI) strategy in substance use treatment clinics.

INTRODUCTION: Evidence-based practice (EBP) implementation represents a strategic change that requires alignment of leadership and support throughout organizations. Leadership and Organizational Change for Implementation (LOCI) is a multifaceted implementation strategy that aims to improve implementation leadership and climate within organizations through iterative cycles of leadership and climate assessment and feedback, leadership training and coaching, and strategic planning with upper-level leaders. This study tested the effects of LOCI on transformational and implementation leadership, implementation climate, implementation citizenship behavior, and EBP reach. METHODS: A multiple cohort, cluster randomized trial tests the effect of LOCI in 60 clinics across nine behavioral health organizations in California and Arizona, USA. The study randomized clinics within organizations to either LOCI or a leadership training webinar control condition in three consecutive cohorts. Repeated web-based surveys of direct service providers (nLOCI = 201, nControl = 179) assessed leadership, implementation climate, and implementation citizenship over time. Multilevel autoregressive modeling was the primary statistical analysis such that providers (level-1) were nested within clinics (level-2). The study predicted between-condition differences at 4-, 8-, and 12-month follow-up assessments. Provider engagement in a fidelity monitoring process assessed reach of motivational interviewing (i.e., number of sessions recorded/submitted for fidelity coding). An independent sample t-test explored between condition differences in motivational interviewing reach. RESULTS: Results indicated between condition differences at 4 months for implementation leadership, implementation climate, and implementation citizenship behavior such that greater improvements were evidenced in the LOCI condition compared to the control condition. Reach of MI was significantly greater in the LOCI vs control condition such that LOCI providers were significantly more likely to engage in the fidelity monitoring process (chi-square (1, n = 370) = 5.59, p = .018). CONCLUSIONS: LOCI was developed based on organizational theories of strategic leadership and climate to affect organizational change processes that communicate that innovation implementation is expected, supported, and recognized as a value of the organization. The LOCI implementation strategy resulted in more positive hypothesized outcomes compared to the control condition. Organizational change strategies have utility for implementing health innovations in complex, multilevel contexts and for greater sustainment of facilitative leader behaviors, strategic implementation climate, and improved implementation outcomes. TRIAL REGISTRATION: This study is registered with Clinicaltrials.gov gov (NCT03042832, 2 February 2017; retrospectively registered).

Utility of the Leadership and Organizational Change for Implementation-Systems Level (LOCI-SL) strategy for a statewide substance use treatment implementation effort.

INTRODUCTION: Multi-level and cross-context implementation strategies are needed to support health systems, healthcare delivery organizations, and providers to adopt evidence-based practice (EBP) for substance use disorder (SUD) treatment. However, misalignment between state oversight agencies and healthcare organizations about which services to prioritize and which outcomes are reasonable to expect can hinder implementation success and widespread access to high-quality care. This study investigated the utility of the Leadership and Organizational Change for Implementation-System Level (LOCI-SL) strategy for supporting statewide EBP implementation for SUD treatment. METHODS: Nine community mental health centers (CMHCs) contracted by a state agency participated in a combined motivational-enhancement therapy and cognitive behavioral therapy (MET/CBT) implementation effort. Five of the CMHCs also received the LOCI-SL strategy to obtain ongoing implementation support. We conducted 21 individual interviews and three small group interviews with 30 participants across CMHCs and state health agencies to investigate the utility of LOCI-SL in supporting their EBP implementation efforts. Deductive thematic analysis was guided by the Exploration, Preparation, Implementation, Sustainment Framework. RESULTS: Five themes described CMHCs' LOCI-SL and broader contextual experiences implementing EBPs: (1) LOCI-SL supported executives in Preparation phase activities that holistically considered organizational needs and capacity to implement and sustain EBPs; (2) LOCI-SL facilitated trust and communication processes across Preparation, Implementation, and Sustainment phases to improve EBP uptake; (3) LOCI-SL increased CMHCs' use of implementation climate strengthening activities throughout the Implementation phase; (4) state contracts did not emphasize quality and thus were not sufficient bridging factors to enforce EBP fidelity during Implementation; and, (5) limited funding and low Medicaid reimbursement rates hindered EBP use throughout the Implementation and Sustainment phases. CONCLUSIONS: LOCI-SL was viewed as a favorable and useful implementation strategy for supporting statewide adoption of EBPs. However, outer context barriers, including limited financial investments in the treatment system, impeded implementation and sustainment efforts. While previous research suggests that contracts are viable alignment-promoting bridging factors, this study demonstrates the importance of articulating implementation outcome expectations to aid state-contracted organizations in achieving EBP implementation success. This study also highlights the need for multi-level implementation strategies to effectively align implementation expectations between outer- and inner-context entities.

Inter-organizational alignment and implementation outcomes in integrated mental healthcare for children and adolescents: a cross-sectional observational study.

BACKGROUND: Integrated care involves care provided by a team of professionals, often in non-traditional settings. A common example worldwide is integrated school-based mental health (SBMH), which involves externally employed clinicians providing care at schools. Integrated mental healthcare can improve the accessibility and efficiency of evidence-based practices (EBPs) for vulnerable populations suffering from fragmented traditional care. However, integration can complicate EBP implementation due to overlapping organizational contexts, diminishing the public health impact. Emerging literature suggests that EBP implementation may benefit from the similarities in the implementation context factors between the different organizations in integrated care, which we termed inter-organizational alignment (IOA). This study quantitatively explored whether and how IOA in general and implementation context factors are associated with implementation outcomes in integrated SBMH. METHODS: SBMH clinicians from community-based organizations (CBOs; nclinician = 27) and their proximal student-support school staff (nschool = 99) rated their schools and CBOs (clinician only) regarding general (organizational culture and molar climate) and implementation context factors (Implementation Climate and Leadership), and nine common implementation outcomes (e.g., treatment integrity, service access, acceptability). The levels of IOA were estimated by intra-class correlations (ICCs). We fitted multilevel models to estimate the standalone effects of context factors from CBOs and schools on implementation outcomes. We also estimated the 2-way interaction effects between CBO and school context factors (i.e., between-setting interdependence) on implementation outcomes. RESULTS: The IOA in general context factors exceeded those of implementation context factors. The standalone effects of implementation context factors on most implementation outcomes were larger than those of general context factors. Similarly, implementation context factors between CBOs and schools showed larger 2-way interaction effects on implementation outcomes than general context factors. CONCLUSIONS: This study preliminarily supported the importance of IOA in context factors for integrated SBMH. The findings shed light on how IOA in implementation and general context factors may be differentially associated with implementation outcomes across a broad array of integrated mental healthcare settings.

Helping Educational Leaders Mobilize Evidence (HELM): The iterative redesign of the Leadership and Organizational Change for Implementation (LOCI) intervention for use in schools.

Background: Few "intervention agnostic" strategies have been developed that can be applied to the broad array of evidence-based practices (EBPs) in schools. This paper describes two studies that reflect the initial iterative redesign phases of an effective leadership-focused implementation strategy-Leadership and Organizational Change for Implementation (LOCI)-to ensure its acceptability, feasibility, contextual appropriateness, and usability when used in elementary schools. Our redesigned strategy-Helping Educational Leaders Mobilize Evidence (HELM)-is designed to improve principals' use of strategic implementation leadership to support the adoption and high-fidelity delivery of a universal EBP to improve student outcomes. Method: In Study 1, focus groups were conducted (n = 6) with 54 district administrators, principals, and teachers. Stakeholders provided input on the appropriateness of original LOCI components to maximize relevance and utility in schools. Transcripts were coded using conventional content analysis. Key themes referencing low appropriateness were summarized to inform LOCI adaptations. We then held a National Expert Summit (Study 2) with 15 research and practice experts. Participants provided feedback via a nominal group process (NGP; n = 6 groups) and hackathon (n = 4 groups). The research team rated each NGP suggestion for how actionable, impactful/effective, and feasible it was. We also coded hackathon notes for novel ideas or alignment with LOCI components. Results: Study 1 suggestions included modifications to LOCI content and delivery. Study 2's NGP results revealed most recommendations to be actionable, impactful/effective, and feasible. Hackathon results surfaced two novel ideas (distributed leadership teams and leaders' knowledge to support educators EBP use) and several areas of alignment with LOCI components. Conclusion: Use of these iterative methods informed the redesign of LOCI and the development of HELM. Because it was collaboratively constructed, HELM has the potential to be an effective implementation strategy to support the use of universal EBP in schools.

Our research team designed a strategy (HELM) for school principals to improve the support they provide to staff to implement practices proven to work in research for improving student outcomes. We designed HELM by conducting focus groups with school district administrators, principals, and teachers. Participants were asked for their feedback on how to adapt an existing leadership strategy (LOCI) to the school context. After collecting this feedback, we held a meeting with 15 research and practice experts. During this meeting, the group of experts reviewed the focus group feedback and decided how to incorporate it into the design of the HELM strategy. We believe that collecting this feedback and involving research and practice experts in interpreting and integrating participant feedback into the HELM strategy will make HELM a more effective strategy for supporting school principals' in implementing supports in their schools.

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial.

BACKGROUND: Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity. METHODS: Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses. RESULTS: LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045). CONCLUSIONS: LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care.

Background: Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects. Method: A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate. Results: Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience. Conclusion: This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.

Using treatments with known positive impact in community-based mental health programs is challenging. Many studies suggest leaders of these programs can help. Similarly, certain features of community-based programs can also be helpful. This case study of an outpatient mental health clinic provides rich descriptions of actions leaders took that shaped the environment in their program and helped improve the use of a treatment with known positive impact. This case study can serve as a practical guide for leaders to reference when aiming to improve the use of treatments with known impact in their own programs.

Inter-Organizational Alignment and Implementation Outcomes in Integrated Mental Healthcare for Children and Adolescents: A Cross-Sectional Observational Study.

Background: Integrated care involves care provided by a team of professionals, often in non-traditional settings. A common example worldwide is integrated school-based mental health (SBMH), which involves externally employed clinicians providing care at schools. Integrated mental healthcare can improve the accessibility and efficiency of evidence-based practices (EBPs) for vulnerable populations suffering from fragmented traditional care. However, integration can complicate EBP implementation due to overlapping organizational contexts, diminishing the public health impact. Emerging literature suggests that EBP implementation may benefit from the similarities in the implementation context factors between the different organizations in integrated care, which we termed inter-organizational alignment (IOA). This study quantitatively explored whether and how IOAs in general and implementation context factors are associated with implementation outcomes in integrated SBMH. Methods: SBMH clinicians from community-based organizations (CBOs; nclinician=27) and their proximal student-support school staff (nschool=99) rated their schools and CBOs (clinician only) regarding general (organizational culture and molar climate) and implementation context factors (Implementation Climate and Leadership), and nine common implementation outcomes (e.g., treatment integrity, service access, acceptability). The levels of IOA were estimated by intra-class correlations (ICCs). We fitted multilevel models to estimate the standalone effects of context factors from CBOs and schools on implementation outcomes. We also estimated the 2-way interaction effects between CBO and school context factors (i.e., between-setting interdependence) on implementation outcomes. Results: The IOA in general context factors exceeded those of implementation context factors. The standalone effects of implementation context factors on most implementation outcomes were larger than those of general context factors. Similarly, implementation context factors between CBOs and schools showed larger 2-way interaction effects on implementation outcomes than general context factors. Conclusions: This study preliminarily supported the importance of IOA in context factors for integrated SBMH. The findings shed light on how IOA in implementation and general context factors may be differentially associated with implementation outcomes across a broad array of integrated mental healthcare settings.

Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health.

OBJECTIVE: Measurement-based care (MBC), which collects session-by-session symptom data from patients and provides clinicians with feedback on treatment response, is a highly generalizable evidence-based practice with significant potential to improve the outcomes of mental health treatment in youth when implemented with fidelity; however, it is rarely used in community settings. This study tested whether an implementation strategy targeting organizational leadership and organizational implementation climate could improve MBC fidelity and clinical outcomes for youth in outpatient mental health clinics. METHOD: In a cluster randomized trial, 21 clinics were assigned to the Leadership and Organizational Change for Implementation strategy plus training and technical assistance in MBC (k = 11, n = 117) or training and technical assistance only (k = 10, n = 117). Primary outcomes of MBC fidelity (assessed via electronic metadata) and youth symptom improvement (assessed via caregiver-reported change on the Shortform Assessment for Children Total Problem Score) were collected for consecutively enrolled youths (ages 4-18 years) who initiated treatment in the 12 months following MBC training. Outcomes of each youth were assessed for 6 months following baseline. RESULTS: A total of 234 youths were enrolled and included in intent-to-treat analyses. At baseline, there were no significant differences by condition in clinic, clinician, or youth characteristics. Youths in clinics using the Leadership and Organizational Change for Implementation strategy experienced significantly higher MBC fidelity compared with youths in control clinics (23.1% vs 3.4%, p = .014), and exhibited significantly greater reductions in symptoms from baseline to 6 months (d = 0.31, 95% CI: 0.04-0.58, p = .023). CONCLUSION: Implementation strategies targeting organizational leadership and focused implementation climate can improve fidelity to evidence-based practices and clinical outcomes of youth mental health services. CLINICAL TRIAL REGISTRATION INFORMATION: Working to Implement and Sustain Digital Outcome Measures (WISDOM); https://clinicaltrials.gov/; NCT04096274.

Eight characteristics of rigorous multilevel implementation research: a step-by-step guide.

BACKGROUND: Although healthcare is delivered in inherently multilevel contexts, implementation science has no widely endorsed methodological standards defining the characteristics of rigorous, multilevel implementation research. We identify and describe eight characteristics of high-quality, multilevel implementation research to encourage discussion, spur debate, and guide decision-making around study design and methodological issues. RECOMMENDATIONS: Implementation researchers who conduct rigorous multilevel implementation research demonstrate the following eight characteristics. First, they map and operationalize the specific multilevel context for defined populations and settings. Second, they define and state the level of each construct under study. Third, they describe how constructs relate to each other within and across levels. Fourth, they specify the temporal scope of each phenomenon at each relevant level. Fifth, they align measurement choices and construction of analytic variables with the levels of theories selected (and hypotheses generated, if applicable). Sixth, they use a sampling strategy consistent with the selected theories or research objectives and sufficiently large and variable to examine relationships at requisite levels. Seventh, they align analytic approaches with the chosen theories (and hypotheses, if applicable), ensuring that they account for measurement dependencies and nested data structures. Eighth, they ensure inferences are made at the appropriate level. To guide implementation researchers and encourage debate, we present the rationale for each characteristic, actionable recommendations for operationalizing the characteristics in implementation research, a range of examples, and references to make the characteristics more usable. Our recommendations apply to all types of multilevel implementation study designs and approaches, including randomized trials, quantitative and qualitative observational studies, and mixed methods. CONCLUSION: These eight characteristics provide benchmarks for evaluating the quality and replicability of multilevel implementation research and promote a common language and reference points. This, in turn, facilitates knowledge generation across diverse multilevel settings and ensures that implementation research is consistent with (and appropriately leverages) what has already been learned in allied multilevel sciences. When a shared and integrated description of what constitutes rigor is defined and broadly communicated, implementation science is better positioned to innovate both methodologically and theoretically.

The incremental association of implementation leadership and school personnel burnout beyond transformational leadership.

Successful implementation of school-wide interventions (i.e., delivered to all students by a wide array of school personnel) is key to promoting students' academic achievement and psychosocial development. Yet, the implementation of school-wide interventions is complex and can be psychologically taxing for implementing personnel. If evidence-based practice and program (EBP) implementation goes unsupported, implementation challenges might result in chronic stress among school personnel that leads to burnout. While generally effective leadership tends to decrease educator burnout, implementation-specific leadership may also decrease burnout through its strategic supports for EBP implementation. A series of linear regression and path models were used to examine the concurrent association between transformational (e.g., general) and implementation (e.g., strategic) leadership and burnout and its component parts (i.e., emotional exhaustion, depersonalization, personal accomplishment). In a sample of 338 school personnel, we found transformational and implementation leadership were each significantly associated with decreased burnout. However, transformational leadership was not significantly associated with any of the three burnout components, whereas implementation leadership was significantly associated with increased personal accomplishment. These results suggest both general and strategic forms of leadership are key supports for school personnel burnout and as such, leaders may benefit from training to improve each. Additional implications for schools and future directions to understand how best to support school personnel are discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Teacher attitudes toward evidence-based practices: Exploratory and confirmatory analyses of the school-adapted evidence-based practice attitude scale.

Background: The Evidence-Based Practice Attitudes Scale (EBPAS) is widely used in implementation research, but it has not been adapted and validated for use among general education teachers, who are most likely to deliver evidence-based prevention programs in schools, the most common setting where youth access social, emotional, and behavioral health services. Method: School-based stakeholders and a research team comprised of experts in the implementation of evidence-based practices in schools adapted the EBPAS for teachers (the S-EBPAS). The adapted instrument was administered to a representative sample (n = 441) of general education teachers (grades K-5) to assess the reliability and internal consistency via factor analyses. The S-EBPAS included two forms (i.e., EBP-agnostic and EBP-specific item referents), therefore, a multiple-group confirmatory factor analysis (CFA) was also performed to establish measurement invariance between the two forms. Results: After adaptation and refinement, a 9-item, 3-factor structure was confirmed, with the final model supporting three first-order factors that load onto a second-order factor capturing attitudes toward adopting evidence-based practices. Multiple-group CFA analyses of measurement invariance indicated there were no significant differences between the two forms. Conclusions: Overall, this study provides a brief, flexible instrument capturing attitudes toward adopting EBPs that has high reliability and internal consistency, which support its use among general education teachers in school settings implementing evidence-based practices. Plain Language Summary: The Evidence-Based Practice Attitudes Scale (EBPAS) is a popular instrument for measuring attitudes toward evidence-based practices (EBPs). This instrument provides valuable information during implementation initiatives, such as whether providers or front-line implementers have favorable attitudes toward a given practice. The EBPAS has been used in many different settings, such as in community-based mental health clinics, medical hospitals, and in child welfare. However, it's use in schools has been limited, and it has not yet been tested with general education teachers, who are key implementers of evidence-based practices in schools. In order to trust that the scores from an instrument are accurate, it needs to be evaluated when scaling it out to new populations and settings. One popular method to determine this is to use factor analysis, which was employed in this study. This study fills the identified gap by assessing the reliability (i.e., accuracy) and internal consistency of the EBPAS among a representative sample of general education teachers. Findings from this study indicate that the school-adapted EBPAS (S-EBPAS) is a brief, nine-item instrument that provides a reliable estimate of teachers' attitudes toward evidence-based practices. Our results also provide evidence that the S-EBPAS can be used to capture attitudes toward specific EBPs as well as attitudes toward EBP-agnostic. This study provides a flexible instrument that can be used by school-based implementation researchers, practitioners, and intermediaries at multiple phases of implementation projects, such as when exploring a new EBP to adopt.

Climate for evidence-based mental health care implementation in Germany: psychometric investigation of the Implementation Climate Scale (ICS).

Organizational implementation climate is an important construct in implementation research to describe to what extent implementation is expected, supported, and rewarded. Efforts in bridging the research-practice gap by implementing evidence-based practice (EBP) can benefit from consideration of implementation climate. The Implementation Climate Scale (ICS) is a psychometrically strong measure assessing employees' perceptions of the implementation climate. The present cross-sectional study aimed at providing a German translation and investigating its psychometric properties. The translation followed standard procedures for adapting psychometric instruments. German psychotherapists (N = 425) recruited online completed the ICS, the Evidence Based Practice Attitudes Scale (EBPAS-36D) and the Intention Scale for Providers (ISP). We conducted standard item and reliability analyses. Factorial validity was assessed by comparing an independent cluster model of Confirmatory Factorial Analysis (ICM-CFA), a Bifactor CFA, a Second-order CFA and an (Bifactor) Exploratory Structural Equation Model (ESEM). Measurement invariance was tested using multiple-group CFA and ESEM, convergent validity with correlation analysis between the ICS and the ISP subjective norms subscale (ISP-D-SN). The mean item difficulty was pi = .47, mean inter-item correlation r = .34, and mean item-total correlation ritc = .55. The total scale (ω = 0.91) and the subscales (ω = .79-.92) showed acceptable to high internal consistencies. The model fit indices were comparable and acceptable (Second-order CFA: RMSEA [90% CI] = .077 [.069; .085], SRMR = .078, CFI = .93). Multiple-group CFA and ESEM indicated scalar measurement invariance across gender and presence of a psychotherapy license. Psychotherapists in training reported higher educational support for EBP than licensed psychotherapists (T = 2.09, p = .037, d = 0.25). The expected high correlation between the ICS and the ISP-D-SN was found (r = .59, p < .001). Results for the German ICS confirm good psychometric properties including validity.

Perceptual differences in nursing implementation leadership and climate: a cross-sectional study.

BACKGROUND: The literature on perceptual differences between managers and staff regarding social dynamic factors (e.g., leadership, climate) in nursing settings is sparse. Addressing this gap in knowledge is critical for informing implementation efforts and improving patient and organizational outcomes. The purpose of this study was to test the perceptual differences regarding implementation leadership and implementation climate between nursing staff and their managers. METHODS: This study was a secondary analysis of cross-sectional survey data collected in 2016-2017. The setting included 22 adult medical-surgical units nested in 7 acute care hospitals in the Eastern and Midwestern United States. Participants were registered nurses (N = 261) and nurse managers (N = 22) who completed an electronic survey consisting of the Implementation Leadership Scale (ILS), the Implementation Climate Scale (ICS), and demographic items. Differences in perception were analyzed at the unit level using structural equation modeling to develop latent difference score models (LDS). We assessed associations of the LDSs with manager ILS and ICS scores, years of nursing experience, and years of experience working on the current unit. The association of ILS LDS with the observed nursing staff ICS scores was also analyzed. RESULTS: Higher manager scores on the ILS and ICS were associated with greater perceptual differences in implementation leadership and implementation climate. Greater years of experience as a nurse were associated with greater perceptual differences in ILS and ICS scores. Greater tenure on the unit was associated with smaller differences on the ILS knowledge domain. Greater perceptual differences regarding implementation leadership were associated with worse staff ratings of implementation climate. CONCLUSIONS: Although this study observed significant relationships among manager ILS and ICS scores, staff-manager perceptual differences, and staff ratings of implementation climate in nursing settings, it is still unclear why perceptual differences in implementation leadership and climate exist and how to address them. Future studies are warranted to test the effect of perceptual differences on implementation and patient outcomes.

The Interaction Between General and Strategic Leadership and Climate on Their Multilevel Associations with Implementer Attitudes Toward Universal Prevention Programs for Youth Mental Health: A Cross-Sectional Study.

Emerging literature has highlighted the importance of discerning general and strategic organizational context (OC) factors (e.g., leadership and climate) and their interaction effect on individual implementation behaviors (e.g., attitudes toward evidence-based practices; EBPs) in youth mental healthcare. This study aimed to examine how leadership and climate (general and strategic) are associated with implementer attitudes toward EBPs across the individual and organizational levels and their interaction effect in schools. A series of multilevel models (MLMs) were fitted on a diverse sample of schools actively implementing universal prevention programs for youth mental health (441 implementers from 52 schools). The organization-level aggregates and individual educators' perceptions of general and strategic leadership and climate, and their interaction terms, were entered as level-2 and level-1 predictors of four attitudinal dimensions (Requirement, Openness, Appeal, and Divergence) based on their level of measurement. At the organizational level, higher levels of strategic leadership and climate, but not their general counterparts, were consistently associated with more favorable attitudes in all four dimensions. At the individual/within-school level, higher levels of perceived general and strategic leadership and climate were associated with more favorable attitudes of Requirement and Openness. At the organizational/between-school level, general climate moderated the positive effect of strategic climate on implementers' perception of appeal and divergence of EBPs. Our findings indicate that leaders should make data-based decisions to allocate resources on strategic and/or general leadership and climate to foster favorable staff attitudes toward EBPs based on the level of measurement, implementation-specificity, and attitudinal dimensions.

Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS).

BACKGROUND: Strategic implementation leadership is a critical determinant of successful implementation, hypothesized to create a more supportive implementation climate conducive to the adoption and use of evidence-based practices. Implementation leadership behaviors may vary significantly across contexts, necessitating studies that examine the validity of established measurement tools in novel health service delivery sectors. The education sector is the most common site for delivering mental health services to children and adolescents in the USA, but research focused on implementation leadership in schools is in the early phases, and there is a need for adaptation and expansion of instruments in order to tailor to the school context. The current study adapted and validated the School Implementation Leadership Scale (SILS) (based on the Implementation Leadership Scale) in a sample of elementary school personnel from six school districts who were implementing one of two well-established prevention programs for supporting children's mental health. METHODS: Participants were 441 public school teachers from 52 elementary schools in the Midwest and West Coast of the USA. Participants completed a survey that contained: (1) an adapted and expanded version of the SILS with additional items generated for four existing subscales as well as three new subscales (communication, vision/mission, and availability), and (2) additional tools to evaluate convergent and divergent validity (i.e., measures of general/molar leadership and teaching attitudes). Data underwent (1) examination of item characteristic curves to reduce items and ensure a pragmatic instrument, (2) confirmatory factor analyses to establish structural validity, and (3) evaluation of convergent and divergent validity. RESULTS: Item reduction analyses resulted in seven subscales of three items each. Results indicated acceptable fit for a seven-factor structural model (CFI = .995, TLI = .99, RMSEA = .07, SRMR = 0.02). Second-order factor loadings were high (λ = .89 to .96), suggesting that the SILS subscales comprise a higher-order implementation leadership factor. All subscales demonstrated good inter-item reliability (α = .91-.96). Convergent and divergent validity results were generally as hypothesized, with moderate to high correlations between SILS subscales and general leadership, moderate correlations with teaching attitudes, and low correlations with school demographics. CONCLUSIONS: Overall, results provided strong structural, convergent, and divergent validity evidence for the 21-item, 7-factor SILS instrument. Implications for the measurement of implementation leadership in schools are discussed, as well as strategies to support leaders to enhance their strategic behaviors related to the implementation of mental health prevention programs (e.g., adaptation of existing leadership-focused implementation strategies).

"Change Doesn't Happen by Itself": A Thematic Analysis of First-Level Leaders' Experiences Participating in the Leadership and Organizational Change for Implementation (LOCI) Strategy.

The Leadership and Organizational Change for Implementation (LOCI) strategy is a multifaceted implementation strategy that aims to support successful evidence-based practice (EBP) implementation by fostering effective general leadership, implementation leadership, and implementation climate. How implementation strategies are experienced by participants is important for their utilization and effectiveness in supporting EBP implementation. The current study is the first in-depth qualitative study exploring first-level leaders' experiences of participating in the LOCI strategy. Data were collected as part of a trial where Norwegian child and adult mental health outpatient clinics implemented EBPs for posttraumatic stress disorder (PTSD). Eleven first-level leaders from adult and child clinics participated in semi-structured interviews after completing the LOCI strategy. Data were analyzed through reflexive thematic analysis. The analysis generated four themes related to leaders' experiences of participating in the LOCI strategy: (1) structuring the EBP implementation, (2) taking responsibility for the EBP implementation, (3) interacting with others about the EBP implementation, and (4) becoming aware of EBP implementation and their own leadership. Most participants experienced the LOCI strategy as beneficial for implementing EBPs for PTSD in their clinic. The strategy succeeded in raising awareness of leadership for EBP implementation, and simultaneously provided participants with tools and support for leading the implementation in their clinic. Two participants experienced LOCI as less beneficial than the others. Our results support the strategy's potential to engage and empower first-level leaders to get involved in implementation processes and point to important challenges for future research on implementation strategies.

A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate.

BACKGROUND: This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate. METHODS: A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate. RESULTS: After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI. CONCLUSIONS: The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation. TRIAL REGISTRATION: Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .

Construct validity of the school-implementation climate scale.

Background: Implementation climate is an organizational construct theorized to facilitate the adoption and delivery of evidence-based practices. Within schools, teachers often are tasked with implementing universal prevention programs. Therefore, they are ideal informants when assessing school implementation climate for initial and continuous implementation improvement efforts. The purpose of this study was to examine the construct validity (i.e., factor structure and convergent/divergent validity) of a school-adapted measure of strategic implementation climate called the School Implementation Climate Scale (SICS). Methods: Confirmatory factor analyses of SICS data, collected from 441 teachers in 52 schools, were used to compare uncorrelated and correlated first-order factor models and a second-order hierarchical model. Correlations with other school measures were examined to assess SICS convergent and divergent validities. Results: Results demonstrated acceptable internal consistency for each SICS subscale (αs > 0.80 for all subscales) and construct validity of the hypothesized factor structure of the SICS with three new scales. The hierarchical second-order factor structure with eight first-order factors was found to best model the SICS data. Correlations with other school measures were in the expected direction and magnitude. Conclusions: Results from this study provide psychometric evidence that supports the use of the SICS to inform the implementation research and practice in schools. Plain Language Summary: Schools are busy trying to implement various universal programs and systems to help support kids in their growth. Beginning and sustaining these efforts is quite challenging, and there is need for tools and ideas to help those implementation efforts. One concept is implementation climate, which is broadly the school staff's perception of the implementation support for a given practice. However, no measure currently exists to help schools assess their implementation climate. The goal of our study was to adapt a measure of implementation climate used in other settings to the school environment. We used feedback from educational experts to make changes and used various analyses to determine if the newly adapted measure was psychometrically sound. Findings suggest the new measure is usable to guide implementation efforts in schools.

Provider REport of Sustainment Scale (PRESS): development and validation of a brief measure of inner context sustainment.

BACKGROUND: Implementation scientists and practitioners often rely on frontline providers for reporting on implementation outcomes. Furthermore, measures of sustainment are few, and available sustainment measures are mainly setting or evidenced-based practice (EBP) specific, require organizational and system-level knowledge to complete, and often lack psychometric rigor. The aim of this study was to develop a brief, pragmatic, and generalizable measure for completion by frontline service providers of the implementation outcome, sustainment. METHODS: We utilized a Rasch measurement theory approach to scale the development and testing of psychometric parameters. Sustainment items were developed to be relevant for direct service providers to complete. In order to promote generalizability, data were collected and items were tested across four diverse psychosocial evidence-based practices (motivational interviewing [MI], SafeCare®, classroom pivotal response training [CPRT], and an individualized mental health intervention for children with autism spectrum disorder [AIM-HI]) and in four service settings (substance use disorder treatment, child welfare, education, and specialty mental health). Associations between the sustainment measure and sustainment leadership, sustainment climate, and attitudes towards the adoption and use of each of the EBPs were assessed to confirm construct validity. RESULTS: Three items for the Provider REport of Sustainment Scale (PRESS) were assessed for measuring the core component of sustainment: continued use of the EBP. Internal consistency reliability was high. The scale indicated fit to the Rasch measurement model with no response dependency, ordered thresholds, no differential item functioning, and supported unidimensionality. Additionally, construct validity evidence was provided based on the correlations with related variables. CONCLUSION: The PRESS measure is a brief, three-item measure of sustainment that is both pragmatic and useable across different EBPs, provider types, and settings. The PRESS captures frontline staffs' report of their clinic, team, or agency's continued use of an EBP. Future testing of the PRESS for concurrent and predictive validity is recommended.