Advancing a mission of translational intervention science: Comment on premature implementation.

Replies to comments made by Kenneth E. Freedland et al. (see record 2024-89430-002) on Rinad S. Beida, Lisa Saldana, and Rachel C. Shelton's original article (see record 2023-46817-001). In reading Freedland et al.'s (2024) commentary, it appears that their lens prioritizes internal validity and more explanatory and mechanistic work. While we also value these scientific goals and concur that the approaches they identify are clearly methodologically rigorous, we do not think the approaches will substantially reduce the unacceptable translation gap or address the fundamental issues of context. Our approach recognizes that there is tremendous value in cocreating solutions and interventions with patients, clinicians, and community members in the settings where we are seeking to promote health and address health inequities, and questions traditional assumptions and paradigms that scientists "know best" have effective solutions or should hold all of the power and knowledge (Brownson et al., 2022; Sanchez et al., 2023; Shelton, Adsul, & Oh, 2021; Shelton, Adsul, Oh, et al., 2021). We believe it is critical that we expand the pathways through which we advance intervention science in a meaningful and impactful way, and with more explicit attention to issues of context, equity, engagement, and external validity. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Using policy codesign to achieve multi-sector alignment in adolescent behavioral health: a study protocol.

BACKGROUND: Policymaking is quickly gaining focus in the field of implementation science as a potential opportunity for aligning cross-sector systems and introducing incentives to promote population health, including substance use disorders (SUD) and their prevention in adolescents. Policymakers are seen as holding the necessary levers for realigning service infrastructure to more rapidly and effectively address adolescent behavioral health across the continuum of need (prevention through crisis care, mental health, and SUD) and in multiple locations (schools, primary care, community settings). The difficulty of aligning policy intent, policy design, and successful policy implementation is a well-known challenge in the broader public policy and public administration literature that also affects local behavioral health policymaking. This study will examine a blended approach of coproduction and codesign (i.e., Policy Codesign), iteratively developed over multiple years to address problems in policy formation that often lead to poor implementation outcomes. The current study evaluates this scalable approach using reproducible measures to grow the knowledge base in this field of study. METHODS: This is a single-arm, longitudinal, staggered implementation study to examine the acceptability and short-term impacts of Policy Codesign in resolving critical challenges in behavioral health policy formation. The aims are to (1) examine the acceptability, feasibility, and reach of Policy Codesign within two geographically distinct counties in Washington state, USA; (2) examine the impact of Policy Codesign on multisector policy development within these counties using social network analysis; and (3) assess the perceived replicability of Policy Codesign among leaders and other staff of policy-oriented state behavioral health intermediary organizations across the USA. DISCUSSION: This study will assess the feasibility of a specific approach to collaborative policy development, Policy Codesign, in two diverse regions. Results will inform a subsequent multi-state study measuring the impact and effectiveness of this approach for achieving multi-sector and evidence informed policy development in adolescent SUD prevention and treatment.

Implementing a pharmacist-integrated collaborative model of medication treatment for opioid use disorder in primary care: study design and methodological considerations.

BACKGROUND: Pharmacists remain an underutilized resource in the treatment of opioid use disorder (OUD). Although studies have engaged pharmacists in dispensing medications for OUD (MOUD), few studies have evaluated collaborative care models in which pharmacists are an active, integrated part of a primary care team offering OUD care. METHODS: This study seeks to implement a pharmacist integrated MOUD clinical model (called PrIMO) and evaluate its feasibility, acceptability, and impact across four diverse primary care sites. The Consolidated Framework for Implementation Research is used as an organizing framework for study development and interpretation of findings. Implementation Facilitation is used to support PrIMO adoption. We assess the primary outcome, the feasibility of implementing PrIMO, using the Stages of Implementation Completion (SIC). We evaluate the acceptability and impact of the PrIMO model at the sites using mixed-methods and combine survey and interview data from providers, pharmacists, pharmacy technicians, administrators, and patients receiving MOUD at the primary care sites with patient electronic health record data. We hypothesize that it is feasible to launch delivery of the PrIMO model (reach SIC Stage 6), and that it is acceptable, will positively impact patient outcomes 1 year post model launch (e.g., increased MOUD treatment retention, medication regimen adherence, service utilization for co-morbid conditions, and decreased substance use), and will increase each site's capacity to care for patients with MOUD (e.g., increased number of patients, number of prescribers, and rate of patients per prescriber). DISCUSSION: This study will provide data on a pharmacist-integrated collaborative model of care for the treatment of OUD that may be feasible, acceptable to both site staff and patients and may favorably impact patients' access to MOUD and treatment outcomes. TRIAL REGISTRATION: The study was registered on Clinicaltrials.gov (NCT05310786) on April 5, 2022, https://www. CLINICALTRIALS: gov/study/NCT05310786?id=NCT05310786&rank=1.

Specifying cross-system collaboration strategies for implementation: a multi-site qualitative study with child welfare and behavioral health organizations.

BACKGROUND: Cross-system interventions that integrate health, behavioral health, and social services can improve client outcomes and expand community impact. Successful implementation of these interventions depends on the extent to which service partners can align frontline services and organizational operations. However, collaboration strategies linking multiple implementation contexts have received limited empirical attention. This study identifies, describes, and specifies multi-level collaboration strategies used during the implementation of Ohio Sobriety Treatment and Reducing Trauma (Ohio START), a cross-system intervention that integrates services across two systems (child welfare and evidence-based behavioral health services) for families that are affected by co-occurring child maltreatment and parental substance use disorders. METHODS: In phase 1, we used a multi-site qualitative design with 17 counties that implemented Ohio START. Qualitative data were gathered from 104 staff from child welfare agencies, behavioral health treatment organizations, and regional behavioral health boards involved in implementation via 48 small group interviews about collaborative approaches to implementation. To examine cross-system collaboration strategies, qualitative data were analyzed using an iterative template approach and content analysis. In phase 2, a 16-member expert panel met to validate and specify the cross-system collaboration strategies identified in the interviews. The panel was comprised of key child welfare and behavioral health partners and scholars. RESULTS: In phase 1, we identified seven cross-system collaboration strategies used for implementation. Three strategies were used to staff the program: (1) contract for expertise, (2) provide joint supervision, and (3) co-locate staff. Two strategies were used to promote service access: (4) referral protocols and (5) expedited access agreements. Two strategies were used to align case plans: (6) shared decision-making meetings, and (7) sharing data. In phase 2, expert panelists specified operational details of the cross-system collaboration strategies, and explained the processes by which strategies were perceived to improve implementation and service system outcomes. CONCLUSIONS: We identified a range of cross-system collaboration strategies that show promise for improving staffing, service access, and case planning. Leaders, supervisors, and frontline staff used these strategies during all phases of implementation. These findings lay the foundation for future experimental and quasi-experimental studies that test the effectiveness of cross-system collaboration strategies.

Swimming Against the Current: Addressing Community Needs and Establishing Partnerships for the Prevention of Opioid and Methamphetamine use Among Parents.

The opioid and methamphetamine crises in Oregon have a consequential impact on young families, as an increasing number of parents experience substance use disorder (SUD). As parental substance use escalates, the child welfare system (CWS) becomes overwrought with families who have complex needs. The burden placed on families and on the CWS indicates a need for prevention and treatment interventions for parental SUDs. In response to the worst statewide opioid and methamphetamine epidemics in the USA, a Hybrid Type 2 trial of PRE-FAIR-a prevention intervention for parents-is being implemented in five Oregon counties. Establishing strong partnerships within the communities helped identify the need to implement the evidence-based FAIR treatment model alongside PRE-FAIR. A strong focus on implementation includes meeting the needs of communities and establishing the infrastructure necessary for sustainment of the FAIR programs at the provider agencies. Despite implementation efforts to direct toward PRE-FAIR referrals, parents in need of treatment are being referred at a disproportionate rate, as are older parents who fall outside of study-eligibility. Both challenges have delayed the ability to enroll a sufficient number of participants for the prevention trial. This commentary describes the impact of the opioid and methamphetamine epidemics in Oregon as the crises relate to implementing prevention versus treatment interventions-highlighting the importance of addressing community needs and establishing strong partnerships, which has allowed creative strategies to increase PRE-FAIR recruitment.

Adapting the stages of implementation completion to an evidence-based implementation strategy: The development of the NIATx stages of implementation completion.

Background: Dissemination and implementation frameworks provide the scaffolding to explore the effectiveness of evidence-based practices (EBPs) targeting process of care and organizational outcomes. Few instruments, like the stages of implementation completion (SIC) examine implementation fidelity to EBP adoption and how organizations differ in their approach to implementation. Instruments to measure organizational competency in the utilization of implementation strategies are lacking. Method: An iterative process was utilized to adapt the SIC to the NIATx implementation strategies. The new instrument, NIATx-SIC, was applied in a randomized controlled trial involving 53 addiction treatment agencies in Washington state to improve agency co-occurring capacity. NIATx-SIC data were reported by state staff and external facilitators and through participating agency documentation. Proportion and duration scores for each stage and phase of the NIATx-SIC were calculated for each agency. Competency was assessed using the NIATx fidelity tool. Comparisons of proportion, duration, and NIATx activities completed were determined using independent sample t-tests by agency competency level. Results: The NIATx-SIC distinguished between agencies achieving competency (n = 23) and those not achieving competency (n = 26). Agencies achieving competency completed a greater proportion of implementation phase activities and had a significantly longer Stage 7 duration. These agencies participated in significantly more individual and group coaching calls, attended more in-person meetings, implemented more change projects, and spent approximately 64 more days, on average, engaging in all NIATx activities. Conclusions: Organizational participation in dissemination and implementation research requires a significant investment of staff resources. The inability of an organization to achieve competency when utilizing a set of implementation strategies waste an opportunity to institutionalize knowledge of how to apply implementation strategies to future change efforts. The NIATx-SIC provides evidence that competency is not an attribute of the organization but rather a result of the application of the NIATx implementation strategies to improve agency co-occurring capacity. Trial Registration: ClinicalTrials.gov, NCT03007940. Registered January 2, 2017, https://clinicaltrials.gov/ct2/show/NCT03007940.

Access to integrated services for persons with co-occurring substance use and mental health disorders is a long-standing behavioral health problem. Evidence-based practices (EBPs) that focus on patient needs are effective in improving care for persons with co-occurring disorders. The stages of implementation completion (SIC) is a measure that assesses the process that organizations go through when implementing a new EBP and can be used to compare differences between organizations in their fidelity to recommended processes. To implement, organizations use specified strategies to integrate EBP into the care process. These strategies require a significant investment of staff resources. When organizations struggle to achieve competency with a set of implementation strategies, resources are wasted impacting the ability to use the strategies in future change efforts. As such, it is critical to measure organizational efforts to achieve competency, but instruments to do so are lacking. The SIC was adapted for a proven implementation strategy, NIATx, to address this gap. The NIATx strategy provides outside support and coaching to facilitate the implementation of a new EBP. Results from this study indicated that the NIATx-SIC could distinguish between addiction treatment agencies that applied NIATx implementation strategies with competency, versus those that did not, in the context of a multilevel randomized control trial. Study results provide evidence for the utility of adapting the SIC to specific implementation strategies and the benefit that the NIATx-SIC could provide for similar studies involving the use of NIATx to implement EBPs.

The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs.

BACKGROUND: Most implementations fail before the corresponding services are ever delivered. Measuring implementation process fidelity may reveal when and why these attempts fail. This knowledge is necessary to support the achievement of positive implementation milestones, such as delivering services to clients (program start-up) and competency in treatment delivery. The present study evaluates the extent to which implementation process fidelity at different implementation stages predicts achievement of those milestones. METHODS: Implementation process fidelity data-as measured by the Stages of Implementation Completion (SIC)-from 1287 implementing sites across 27 evidence-informed programs were examined in mixed effects regression models with sites nested within programs. Implementation process fidelity, as measured by the proportion of implementation activities completed during the three stages of the SIC Pre-Implementation phase and overall Pre-Implementation (Phase 1) and Implementation (Phase 2) proportion scores, was assessed as a predictor of sites achieving program start-up (i.e., delivering services) and competency in program delivery. RESULTS: The predicted probability of start-up across all sites was low at 35% (95% CI [33%, 38%]). When considering the evidence-informed program being implemented, that probability was nearly twice as high (64%; 95% CI [42%, 82%]), and 57% of the total variance in program start-up was attributable to the program. Implementation process fidelity was positively and significantly associated with achievement of program start-up and competency. The magnitude of this relationship varied significantly across programs for Pre-Implementation Stage 1 (i.e., Engagement) only. Compared to other stages, completing more Pre-Implementation Stage 3 (Readiness Planning) activities resulted in the most rapid gains in probability of achieving program start-up. The predicted probability of achieving competency was very low unless sites had high scores in both Pre-Implementation and Implementation phases. CONCLUSIONS: Strong implementation process fidelity-as measured by SIC Pre-Implementation and Implementation phase proportion scores-was associated with sites' achievement of program start-up and competency in program delivery, with early implementation process fidelity being especially potent. These findings highlight the importance of a rigorous Pre-Implementation process.

Assessment of Adoption and Early Implementation Barriers and Facilitators of the Safe Environment for Every Kid (SEEK) Model.

OBJECTIVE: Although the use of interventions for screening for social determinants of health of families in pediatric primary care clinics has increased in the past decade, research on the barriers and facilitators of implementing such interventions has been limited. We explored barriers, facilitators, and the mechanisms clarifying their roles in the adoption and implementation of the Safe Environment for Every Kid (SEEK) model, an approach for strengthening families, promoting children's health and development, and preventing child maltreatment. METHODS: A total of 28 semistructured interviews were completed with 9 practice champions, 11 primary care professionals, 5 behavioral health professionals, and 3 nursing/administrative staff representing 12 pediatric primary care practices participating in a larger randomized control trial of implementing SEEK. RESULTS: We identified several barriers and facilitators in the stages of SEEK's adoption and early implementation. Barriers associated with outer and inner setting determinants and poor innovation-organization fit declined in importance over time, while facilitators associated with SEEK characteristics increased in importance based on participants' responses. Barriers and facilitators were linked by mechanisms of comparison and contrast of burdens and benefits, and problem-solving to address limited capacity with available resources. CONCLUSIONS: Any screening for and addressing social determinants of health demands greater attention to adoption and implementation mechanisms and the processes by which primary care professionals assess and utilize facilitators to address barriers. This occurs in a context defined by perceived burdens and benefits of innovation adoption and implementation, the capacity of the practice, and changes in perception with experiencing the innovation.

Study Protocol: Multi-level Determinants of Implementation and Sustainment in the Education Sector.

Evidence-based programs (EBPs) delivered in elementary schools show great promise in reducing risk for emotional and behavioral disorders (EBDs). However, efforts to sustain EBPs in school face barriers. Improving EBP sustainment thus represents a priority, but little research exists to inform the development of sustainment strategies. To address this gap, the Sustaining Evidenced-Based Innovations through Multi-level Implementation Constructs (SEISMIC) project will: (a) Determine if malleable individual, intervention, and organizational factors predict EBP treatment fidelity and modifications during implementation, sustainment, or both; (b) Assess the impact of EBP fidelity and modifications on child outcomes during implementation and sustainment; and (c) Explore the mechanisms through which individual, intervention, and organizational factors influence sustainment outcomes. This protocol paper describes SEISMIC, which builds upon a federally-funded RCT evaluating BEST in CLASS, a teacher-delivered program for K-3rd grade children at risk for EBDs. The sample will include 96 teachers, 384 children, and 12 elementary schools. A multi-level, interrupted time series design will be used to examine the relationship between baseline factors, treatment fidelity, modifications, and child outcomes, followed by a mixed-method approach to elucidate the mechanisms that influence sustainment outcomes. Findings will be used to create a strategy to improve EBP sustainment in schools.

Testing psychosocial interventions in the contexts they are meant to be delivered.

This article discusses psychosocial interventions in the contexts they are meant to be delivered. Prevention and intervention science often follow the linear pathway of preclinical or pre-intervention research-efficacy trials, effectiveness trials, and implementation studies-with the assessment of translation into public and population health impact occurring at the end. This linear translational pathway follows stages developed for ascertaining safe, efficacious, and effective dosages for biological compounds. This approach has created limitations in the need to rapidly deploy complex, multi-component, multilevel approaches to change behavior and improve health into widespread practice for diverse clinical and public health settings. While it is important to use efficacy trials when safety is yet to be established, when the risks are identified to be low, as is often the case for psychosocial interventions, we can go faster to achieve equitable population health impact. The authors recommend that clinical trialists engaged in intervention development incorporate two considerations in the next generation of prevention and intervention research. First, consider moving right to effectiveness or pragmatic trials, as the most valid test of an intervention is the est of that intervention in the context(s) in which it is intended. Second, when designing effectiveness studies, consider investigating questions related to both effectiveness (i.e., does the intervention improve clinical outcomes) and implementation (i.e., what supports are needed to deploy the intervention routinely in that context) to accelerate impact. As a matter of both ethics and equity, there is a need to expedite the research-to-practice pipeline at a pace faster than is made available through current approaches. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Barriers and facilitators to implementing evidence-based interventions in the context of a randomized clinical trial in the United States: a qualitative study.

BACKGROUND: Evidence-based interventions, which are typically supported by data from randomized controlled trials (RCTs), are highly valued by providers of human services like child welfare. However, implementing such interventions in the context of a randomized clinical trial is a complex process, as conducting an RCT adds extra tasks for providers and complicating factors for provider organizations. Utilizing the Exploration, Preparation, Implementation, and Sustainment Framework, this study examines factors that facilitate or impede success in the implementation of evidence-based interventions in the context of a largescale trial of SafeCare,® a child maltreatment intervention. METHODS: Qualitative data were obtained as part of a larger mixed-methods study involving a cluster randomized trial comparing SafeCare to usual services for caregivers within nine child welfare agencies across four states. Between May and October 2017, individual interviews were conducted with a purposive sample of 21 child welfare administrators and 24 supervisors, and 19 focus groups were conducted with 84 providers. Data were coded iteratively and grouped into themes. RESULTS: Several interconnected themes centered on facilitators and barriers to SafeCare implementation in the context of a randomized clinical trial. Facilitators included: (1) Benefits afforded through RCT participation; (2) Shared vision and sustained buy-in across system and organizational levels; and (3) Ongoing leadership support for SafeCare and the RCT. Barriers that hindered SafeCare were: (1) Insufficient preparation to incorporate SafeCare into services; (2) Perceived lack of fit, leading to mixed support for SafeCare and the RCT; and (3) Requirements of RCT participation at the provider level. CONCLUSIONS: These data yield insight into an array of stakeholder perspectives on the experience of implementing a new intervention in the context of a largescale trial. This research also sheds light on how the dynamics of conducting an RCT may affect efforts to implement interventions in complex and high-pressure contexts. Findings highlight the importance of aligning knowledge and expectations among researchers, administrators of organizations, and supervisors and providers. Researchers should work to alleviate the burdens of study involvement and promote buy-in among frontline staff not only for the program but also for the research itself.

Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem.

BACKGROUND: Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers. MAIN BODY: Ten mid-career extramurally funded US-based researchers completed a "pre-mortem" or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it - to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a "legitimate science," (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches. CONCLUSION: We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.

Bringing Evidence-Based Interventions into the Schools: An Examination of Organizational Factors and Implementation Outcomes.

Although schools are one of the largest providers of behavioral health services for youth, many barriers exist to the implementation of evidence-based interventions in schools. This study used the Stages of Implementation Completion (SIC) to examine school-based implementation outcomes for a computer-assisted cognitive behavioral therapy intervention for anxious youth. Organizational factors and predictors of program startup also were examined. Results indicated that the SIC detected implementation variability in schools and suggested that spending more time completing pre-implementation activities may better prepare schools for active implementation of program delivery. Furthermore, proficiency emerged as a potentially important organizational factor to examine in future school-based implementation research.

FAST: A Framework to Assess Speed of Translation of Health Innovations to Practice and Policy.

The 17-year time span between discovery and application of evidence in practice has become a unifying challenge for implementation science and translational science more broadly. Further, global pandemics and social crises demand timely implementation of rapidly accruing evidence to reduce morbidity and mortality. Yet speed remains an understudied metric in implementation science. Prevailing evaluations of implementation lack a temporal aspect, and current approaches have not yielded rapid implementation. In this paper, we address speed as an important conceptual and methodological gap in implementation science. We aim to untangle the complexities of studying implementation speed, offer a framework to assess speed of translation (FAST), and provide guidance to measure speed in evaluating implementation. To facilitate specification and reporting on metrics of speed, we encourage consideration of stakeholder perspectives (e.g., comparison of varying priorities), referents (e.g., speed in attaining outcomes, transitioning between implementation phases), and observation windows (e.g., time from intervention development to first patient treated) in its measurement. The FAST framework identifies factors that may influence speed of implementation and potential effects of implementation speed. We propose a research agenda to advance understanding of the pace of implementation, including identifying accelerators and inhibitors to speed.

Wait No Longer: Reducing Medication Wait-Times for Individuals with Co-Occurring Disorders.

Objective: Community addiction treatment agencies have utilized Network for the Improvement of Addiction Treatment (NIATx), a proven implementation strategy, to reduce appointment wait-times. However, its effectiveness at reducing medication access wait-times has not been explored. Thus, we conducted an exploratory analysis to evaluate the impact of the NIATx implementation strategies on reduced wait-times to addiction, psychotropic or both medications for individuals with co-occurring disorders (COD). Methods: In a cluster-randomized waitlist control group design, community addiction treatment agencies (n = 49) were randomized to receive the NIATx strategy (Cohort 1, n = 25) or to a Waitlist control (Cohort 2, n = 24). All agencies had a 12-month active intervention period. The primary outcome was the medication encounter wait-time. A univariate general linear model analysis utilizing a logarithmic (log10) transformation examined medication wait-times improvements. Results: The intent-to-treat analysis for psychotropic medications and both medications (reflecting integrated treatment) showed significant main effects for intervention and time, especially comparing Baseline and Year 1 to Year 2. Conversely, only the main effect for time was significant for addiction medications. Wait-time reductions in Cohort 1 agencies was delayed and occurred in the sustainment phase. Wait-times to a psychotropic, addiction, or both medications encounter declined by 3 days, 4.9 days, and 6.8 days, respectively. For Cohort 2 agencies, reduced wait-times were seen for psychotropic (3.4 days), addiction (6 days), and both medications (4.9 days) during their active implementation period. Same- or next-day medication access also improved. Conclusions: NIATx implementation strategies reduced medication encounter wait-times but timing of agency improvements varied. Despite a significant improvement, a three-week wait-time to receive integrated pharmacological interventions is clinically suboptimal for individuals with a COD in need of immediate intervention. Community addiction treatment agencies should identify barriers and implement changes to improve medication access so that their patients "wait no longer" to receive integrated treatment and medications for their COD.

Using economic evaluations in implementation science to increase transparency in costs and outcomes for organizational decision-makers.

BACKGROUND: Economic evaluations frequently are utilized to compare the value of different interventions in medicine and health in concrete terms. Implementation science also would benefit from the incorporation of economic evaluations, but such studies are rare in the literature. The National Cancer Institute has supported a special collection of articles focusing on economic evaluations in implementation science. Even when interventions are supported by substantial evidence, they are implemented infrequently in the field. Implementation costs are important determinants for whether organizational decision-makers choose to adopt an intervention and whether the implementation process is successful. Economic evaluations, such as cost-effectiveness analyses, can help organizational decision-makers choose between implementation approaches for evidence-based interventions by accounting for costs and succinctly presenting cost/benefit tradeoffs. MAIN TEXT: This manuscript presents a discussion of important considerations for incorporating economic evaluations into implementation science. First, the distinction between intervention and implementation costs is presented, along with an explanation of why the comprehensive representation of implementation costs is elusive. Then, the manuscript describes how economic evaluations in implementation science may differ from those in medicine and health intervention studies, especially in terms of determining the perspectives and outcomes of interest. Finally, referencing a scale-up trial of an evidence-based behavioral health intervention, concrete case examples of how cost data can be collected and used in economic evaluations targeting implementation, rather than clinical outcomes, are described. CONCLUSIONS: By gaining a greater understanding of the costs and economic impact associated with different implementation approaches, organizational decision-makers will have better transparency for future replication and scale-up. The use of economic evaluations can help to advance this understanding and provide researchers, purveyors or third-party intermediaries, and organizational decision-makers with essential information to facilitate implementation.

Rates of sustainment in the Universal Stages of Implementation Completion.

BACKGROUND: Sustainment is a desirable outcome of implementation, but its precise definition remains unclear, contributing to the difficulty of identifying a generalized rate of sustainment. Several studies and reviews on the topic differ on both definition and levels of analysis. Furthermore, methodological limitations might have influenced the results, including the unknown quality with which some interventions were delivered. The Universal Stages of Implementation Completion (UniSIC) is a standardized measurement tool that tracks the implementation process and milestone completion across a wide range of real-world implementations-this provides a unique opportunity to identify a generalized rate of sustainment. METHODS: UniSIC data was captured from the SIC website on 27 September 2020 and included data from all sites (n = 1778) that had been tracked to date. Data were restricted to sites that achieved competency in program delivery, and thus had a newly adopted program worthy of sustainment. Dates and indicator variables of implementation activities were combined to form two alternate definitions of sustainment: sustained (start-up) was achieved if sites continued to deliver services 2 years past their program start-up date; sustained (competent) was achieved if sites continued to deliver services 2 years past their competence and/or certification date. Of sites eligible for inclusion based on these definitions (N = 208), descriptive analyses were conducted to determine a rate of sustainment for all programs that successfully started a program. These definitions were also applied to a combined sample for a general rate of sustainment among all sites. Rates of competency among both a sample of sites that started up and a combined sample were also identified. RESULTS: The rate of competence was 58.5% and the rate of sustained (start-up) was 37.1%, while the rate of sustained (competent) was 25.1%. The rates of competence and sustainment among the combined samples were far lower: 15.6% for competence, 6.8% for sustained (start-up), and 4.4% for sustained (competent). CONCLUSIONS: These identified rates of sustainment are accurate initial estimates of sustainment of community-based practices, or in general. Future research on rates of sustainment should carefully define measures of sustainment and be transparent about the real-world conditions on which analyses are centered.

Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial.

OBJECTIVE: Integrated treatment services are the gold standard for addressing co-occurring mental and substance use disorders, yet they are not readily available. The Network for the Improvement of Addiction Treatment (NIATx) was hypothesized to be an effective strategy to implement and sustain integrated mental health and substance use care in addiction treatment programs. This study examined sustainment of integrated services for up to 2 years after the active implementation phase. METHODS: The effectiveness of NIATx strategies to implement and sustain integrated services was evaluated by using a cluster-randomized, waitlist control group design. Forty-nine addiction treatment organizations were randomly assigned to either NIATx1 (active implementation strategy) or NIATx2 (waitlist control). The Dual Diagnosis Capability in Addiction Treatment Index was used to evaluate organizations' capability to provide integrated care. The NIATx Stages of Implementation Completion scale was used to assess participation in and adherence to the NIATx implementation process. Linear mixed-effects modeling was used to evaluate changes from baseline to end of the sustainment period. RESULTS: Both cohorts sustained their capability to provide integrated treatment services. Both groups achieved successful implementation and sustained integrated services to a similar degree, regardless of sustainment year. Sustainment did not vary as a function of NIATx adherence. CONCLUSIONS: The delivery of integrated treatment services was sustained for 2 years after receipt of active implementation support. Future research should consider how contextual factors may predict, mediate, and moderate sustainment outcomes.