Characterizing Shared and Unique Implementation Influences in Two Community Services Systems for Autism: Applying the EPIS Framework to Two Large-Scale Autism Intervention Community Effectiveness Trials.

The purpose of this study was to examine common and unique factors influencing implementation process for two evidence-based interventions for children with autism spectrum disorder (ASD) in mental health and education service contexts. This study prospectively collected qualitative data from intervention developers and research staff on the implementation process within the context of two separate ASD intervention effectiveness trials. Results reveal common and unique factors influencing implementation in both study contexts. Implementation leadership and provider attitudes and motivation emerge as key influences on implementation across systems. These findings provide promising targets for modular implementation interventions that can be leveraged within growing, large-scale translation efforts in usual care.

Validation of the Implementation Leadership Scale (ILS) with Supervisors' Self-Ratings.

Although often discussed, there is a lack of empirical research on the role of leadership in the management and delivery of health services. The implementation leadership scale (ILS) assesses the degree to which leaders are knowledgeable, proactive, perseverant, and supportive during evidence-based practice (EBP) implementation. The purpose of this study was to examine the psychometric properties of the ILS for leaders' self-ratings using a sample of mental health clinic supervisors (N = 119). Supervisors (i.e., leaders) completed surveys including self-ratings of their implementation leadership. Confirmatory factor analysis, reliability, and validity of the ILS were evaluated. The ILS factor structure was supported in the sample of supervisors. Results demonstrated internal consistency reliability and validity. Cronbach alpha's ranged from 0.92 to 0.96 for the ILS subscales and 0.95 for the ILS overall scale. The factor structure replication and reliability of the ILS in a sample of supervisors demonstrates its applicability with employees across organizational levels.

Contracting and Procurement for Evidence-Based Interventions in Public-Sector Human Services: A Case Study.

Sustainment of evidence-based interventions (EBIs) in human services depends on the inner context of community-based organizations (CBOs) that provide services and the outer context of their broader environment. Increasingly, public officials are experimenting with contracting models from for-profit industries to procure human services. In this case study, we conducted qualitative interviews with key government and CBO stakeholders to examine implementation of the Best Value-Performance Information Procurement System to contract for EBIs in a child welfare system. Findings suggest that stakeholder relationships may be compromised when procurement disregards local knowledge, communication, collaboration, and other factors supporting EBIs and public health initiatives.

Parent And Caregiver Active Participation Toolkit (PACT): Adaptation for a Home Visitation Program.

OBJECTIVES: Parent engagement poses a persistent challenge to home visitation (HV) programs. Previous work on parent engagement in HV has focused primarily on enrollment, attendance, and retention, with less attention on participation. The purpose of this study was to adapt an engagement toolkit originally developed for child mental health treatment settings, the Parent And Caregiver Active Participation Toolkit (PACT), and test the adapted toolkit in a HV program, SafeCare® (SC), with a focus on parent participation. METHODS: Toolkit adaptation was informed by interviews/focus groups with parents and home visitors. Next, home visitors (n = 6) were trained to use adapted PACT for SC as part of SC delivery to 18 parents. A comparison group included 24 parents who received SC one year prior to this study. Analyses compared PACT for SC participants to the comparison group on parent participation and home visitor fidelity to assignment of homework. Qualitative and quantitative data from parents, home visitors, and supervisors (n = 4) assessed the acceptability, utility, appropriateness, and feasibility of PACT for SC. RESULTS: Parents receiving PACT for SC had higher participation and reported greater home visitor fidelity to homework assignment than comparison parents. Parents found PACT for SC acceptable and useful as part of SC. Home visitors and supervisors identified some limitations in PACT for SC's utility but generally found it to be a positive, feasible addition to HV services. CONCLUSIONS: Results suggest that enhancing HV programs with an engagement toolkit may improve parents' participation in services and providers' assignment of homework between sessions.

The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health.

OBJECTIVES: Discrepancies, or perceptual distance, between leaders' self-ratings and followers' ratings of the leader are common but usually go unrecognized. Research on discrepancies is limited, but there is evidence that discrepancies are associated with organizational context. This study examined the association of leader-follower discrepancies in Implementation Leadership Scale (ILS) ratings of mental health clinic leaders and the association of those discrepancies with organizational climate for involvement and performance feedback. Both involvement and performance feedback are important for evidence-based practice (EBP) implementation in mental health. METHODS: A total of 593 individuals-supervisors (leaders, N=80) and clinical service providers (followers, N=513)-completed surveys that included ratings of implementation leadership and organizational climate. Polynomial regression and response surface analyses were conducted to examine the associations of discrepancies in leader-follower ILS ratings with organizational involvement climate and performance feedback climate, aspects of climate likely to support EBP implementation. RESULTS: Both involvement climate and performance feedback climate were highest where leaders rated themselves low on the ILS and their followers rated those leaders high on the ILS ("humble leaders"). CONCLUSIONS: Teams with "humble leaders" showed more positive organizational climate for involvement and for performance feedback, contextual factors important during EBP implementation and sustainment. Discrepancy in leader and follower ratings of implementation leadership should be a consideration in understanding and improving leadership and organizational climate for mental health services and for EBP implementation and sustainment in mental health and other allied health settings.

Validation of the Implementation Leadership Scale (ILS) in Substance use Disorder Treatment Organizations.

There have been recent calls for pragmatic measures to assess factors that influence evidence-based practice (EBP) implementation processes and outcomes. The Implementation Leadership Scale (ILS) is a brief and efficient measure that can be used for research or organizational development purposes to assess leader behaviors and actions that actively support effective EBP implementation. The ILS was developed and validated in mental health settings. This study validates the ILS factor structure with providers in alcohol and other drug (AOD) use treatment agencies. Participants were 323 service providers working in 72 workgroups from three AOD use treatment agencies. Confirmatory factor analyses and reliability analyses were conducted to examine the psychometric properties of the ILS. Convergent and discriminant validity were also assessed. Confirmatory factor analyses demonstrated good fit to the hypothesized first and second order factor structure. Internal consistency reliability was excellent. Convergent and discriminant validity was supported. The ILS psychometric characteristics, reliability, and validity were supported in AOD use treatment agencies. The ILS is a brief and pragmatic measure that can be used for research and practice to assess leadership for EBP implementation in AOD use treatment agencies.

Cross-Validation of the Implementation Leadership Scale (ILS) in Child Welfare Service Organizations.

The Implementation Leadership Scale (ILS) is a brief, pragmatic, and efficient measure that can be used for research or organizational development to assess leader behaviors and actions that actively support effective implementation of evidence-based practices (EBPs). The ILS was originally validated with mental health clinicians. This study validates the ILS factor structure with providers in community-based organizations (CBOs) providing child welfare services. Participants were 214 service providers working in 12 CBOs that provide child welfare services. All participants completed the ILS, reporting on their immediate supervisor. Confirmatory factor analyses were conducted to examine the factor structure of the ILS. Internal consistency reliability and measurement invariance were also examined. Confirmatory factor analyses showed acceptable fit to the hypothesized first- and second-order factor structure. Internal consistency reliability was strong and there was partial measurement invariance for the first-order factor structure when comparing child welfare and mental health samples. The results support the use of the ILS to assess leadership for implementation of EBPs in child welfare organizations.

The role of collaborations in sustaining an evidence-based intervention to reduce child neglect.

Child neglect is the most prevalent form of child maltreatment and represents 79.5% of open child-welfare cases. A recent study found the evidence-based intervention (EBI) SafeCare(®) (SC) to significantly reduce child neglect recidivism rates. To fully capitalize on the effectiveness of such EBIs, service systems must engage in successful implementation and sustainment; however, little is known regarding what factors influence EBI sustainment. Collaborations among stakeholders are suggested as a means for facilitating EBI implementation and sustainment. This study combines descriptive quantitative survey data with qualitative interview and focus group findings to examine the role of collaboration within the context of public-private partnerships in 11 child welfare systems implementing SC. Participants included administrators of government child welfare systems and community-based organizations, as well as supervisors, coaches, and home visitors of the SC program. Sites were classified as fully-, partially-, and non-sustaining based on implementation fidelity. One-way analysis of variance was used to examine differences in stakeholder reported Effective Collaboration scores across fully-sustaining, partially-sustaining, and non-sustaining sites. Qualitative transcripts were analyzed via open and focused coding to identify the commonality, diversity, and complexity of collaborations involved in implementing and sustaining SC. Fully-sustaining sites reported significantly greater levels of effective collaboration than non-sustaining sites. Key themes described by SC stakeholders included shared vision, building on existing relationships, academic support, problem solving and resource sharing, and maintaining collaborations over time. Both quantitative and qualitative results converge in highlighting the importance of effective collaboration in EBI sustainment in child welfare service systems.