Implementation Science and Practice-Oriented Research: Convergence and Complementarity.

Implementation science is the scientific study of methods to promote the uptake of research findings and other evidence-based practices in routine care, with the goal of improving the quality and effectiveness of health services (Bauer et al., 2015). In addition to this common goal, practice-oriented psychotherapy research (and researchers) and implementation science (and scientists) share a common focus on the people and the places where treatment happens. Thus, there exists strong potential for combining these two approaches. In this article, we provide a primer on implementation science for psychotherapy researchers and highlight important areas and examples of convergence and complementarity between implementation science and practice-oriented psychotherapy research. Specifically, we (a) define and describe the core features of implementation science; (b) discuss similarities and areas of complementarity between implementation science and practice-oriented psychotherapy research; (c) discuss a case example that exemplifies the integration of implementation science and practice-oriented research; and (d) propose directions for future research and collaborations that leverage both implementation science and practice-oriented research.

A pilot randomized effectiveness trial of the unified protocol in trauma-exposed veterans.

BACKGROUND: The unified protocol (UP) is a promising transdisgnostic treatment for emotional disorders; limited data exists with trauma-exposed populations. This study compared effectiveness of the UP, presented centered therapy (PCT), and treatment as usual (TAU) in trauma-exposed veterans presenting to routine care. METHOD: Trauma-exposed veterans with one or more emotional disorder diagnoses participated in a pilot hybrid-1 effectiveness/preimplementation study. Thirty-seven male and female veterans were randomized to one of three conditions. RESULTS: Multilevel growth curve modeling demonstrated improvement over time across conditions with large effect sizes (range: -2.15 to -3.32), with the UP demonstrating the greatest change. The between group effect sizes for reductions in number of comorbid diagnoses were medium to small and statistically significant (TAU and UP, d = 0.49, p = .056; TAU and PCT d = 0.18, p = .166, UP and PCT d = 0.31, p = .229). Only the UP led to a decrease in the number of comorbid diagnoses (d = -0.71). Psychosocial functioning varied by group, with slight increases in impairment in PCT and TAU, and medium effect size reduction in the UP. Only the UP exhibited significant decreases in self-reported anxiety and depression. Between group differences for UP and PCT were medium to large and statistically significant for depression across two measures (d = -0.72 to d = -1.40). CONCLUSIONS: This represents the first trial examining effectiveness of the UP, PCT, and TAU in trauma-exposed veterans. Despite a small sample, large effect size differences demonstrated promising advantages for the UP. Trial Registration Number: NCT02944994.

The Effect of Therapeutic Alliance on Dropout in Cognitive Processing Therapy for Posttraumatic Stress Disorder.

A substantial number of individuals who undergo cognitive processing therapy (CPT) for posttraumatic stress disorder (PTSD) drop out before receiving a full course of treatment. Therapeutic alliance, defined as the working relationship between the therapist and client, is a dynamic process within therapy that may change over time. Research suggests that therapeutic alliance is associated with dropout in various treatments. However, no studies have yet examined the association between therapeutic alliance and dropout in CPT, and few studies have examined therapeutic alliance longitudinally over the course of treatment. Examining alliance in CPT through different methods may increase clinicians' understanding of how to tailor interventions to prevent treatment dropout. The present study examined the association between therapeutic alliance and treatment dropout among 169 participants in a randomized implementation effectiveness trial. In total, 33.1% of clients dropped out over the course of CPT, and nearly half of these individuals dropped out during the first six sessions. Continuous-time survival analysis results indicated that mean ratings of alliance significantly predicted treatment dropout, Wald χ2 (1, N = 167) = 4.08, Exp(ß) = .64, p = .043, whereas initial alliance, late alliance, and change in alliance over treatment did not. These findings suggest that overall therapeutic alliance is an important predictor of dropout from CPT.

Training During a Pandemic: Successes, Challenges, and Practical Guidance From a Virtual Facilitated Learning Collaborative Training Program for Written Exposure Therapy.

In response to COVID-19, continued workforce training is essential to ensure that evidence-based treatments are available on the frontline to meet communities' ongoing and emerging mental health needs. However, training during a pandemic imposes many new challenges. This paper describes a multisite training and implementation pilot program, facets of which allowed for continued training despite the onset of the COVID-19 pandemic and subsequent social distancing guidelines. This virtual facilitated learning collaborative in Written Exposure Therapy, an evidence-based treatment for posttraumatic stress disorder, included virtual workshop training, phone-based clinical consultation, implementation-focused video calls for program leadership, and program evaluation. Data are presented about program enrollees and patient impact following the onset of COVID-19-related social distancing restrictions. Challenges, successes, and practical guidance are discussed to inform the field regarding training strategies likely to be durable in an uncertain, dynamic healthcare landscape.

Correction to: The Association Between Clinician and Perceived Organizational Factors with Early Fidelity to Cognitive Processing Therapy for Posttraumatic Stress Disorder in a Randomized Controlled Implementation Trial.

The article "The Association Between Clinician and Perceived Organizational Factors with Early Fidelity to Cognitive Processing Therapy for Posttraumatic Stress Disorder in a Randomized Controlled Implementation Trial", written by Iris Sijercic, Jeanine E. M. Lane, Cassidy A. Gutner, Candice M. Monson and Shannon Wiltsey Stirman, was originally published electronically on the publisher's internet portal (currently SpringerLink) on 28 August 2019 with open access. With the author(s)' decision to step back from Open Choice, the copyright of the article changed on 30 August 2019 to © Springer Science+Business Media, LLC, part of Springer Nature 2019 and the article is forthwith distributed under the terms of copyright.

The Association Between Clinician and Perceived Organizational Factors with Early Fidelity to Cognitive Processing Therapy for Posttraumatic Stress Disorder in a Randomized Controlled Implementation Trial.

A common metric for determining implementation success is the measurement of clinician adherence to, and competence in, delivering a psychotherapy. The present study examined clinician and organizational factors as predictors of early adherence and competence among 78 clinicians delivering cognitive processing therapy (CPT), an evidence-based psychotherapy (EBP) for posttraumatic stress disorder, in a randomized controlled implementation trial. Results indicated that clinicians' willingness to adopt an EBP if required to do so was significantly associated with early adherence and competence in CPT delivery. Level of clinician education was significantly associated with early competence in delivering CPT. Organizational factors did not predict early adherence or competence. Implications of the findings are discussed.

Mixed Methods Analysis of Implementation of Interpersonal Psychotherapy (IPT) for Major Depressive Disorder in Prisons in a Hybrid Type I Randomized Trial.

This article describes a mixed methods evaluation of implementation of interpersonal psychotherapy (IPT) in the first fully-powered trial of any treatment for major depressive disorder in an incarcerated population. Assessments in this Hybrid Type I trial included surveys of prison providers and administrators (n = 71), measures of feasibility and acceptability to prison patients (n = 90), and a planned document review (n = 460) to assess potential determinants of implementation. Quantitative and qualitative results indicated that IPT was a good fit for prisoners, and that prisoners and providers were enthusiastic about IPT. Providers were open to feedback, open to learning evidence-based practices, and committed to helping their clients. Limited treatment staff and variable supervision and collegial support may pose implementation challenges. For widespread prison implementation, scalable models for ongoing IPT training and supervision are needed.

Intimate Partner Violence Screening Programs in the Veterans Health Administration: Informing Scale-up of Successful Practices.

OBJECTIVES: Screening women for intimate partner violence (IPV) is increasingly expected in primary care, consistent with clinical prevention guidelines (e.g., United States Preventive Services Task Force). Yet, little is known about real-world implementation of clinical practices or contextual factors impacting IPV screening program success. This study identified successful clinical practices, and barriers to and facilitators of IPV screening program implementation in the Veterans Health Administration (VHA). DESIGN: Descriptive, qualitative study of a purposeful sample of 11 Veterans Affairs Medical Centers (VAMCs) categorized as early and late adopters of IPV screening programs within women's health primary care clinics. VAMCs were categorized based on performance measures collected by VHA operations partners. PARTICIPANTS: Thirty-two administrators and clinician key informants (e.g., Women's Health Medical Directors, IPV Coordinators, and physicians) involved in IPV screening program implementation decisions from six early- and five late-adopting sites nationwide. MAIN MEASURES: Participants reported on IPV screening and response practices, and contextual factors impacting implementation, in individual 1-h semi-structured phone interviews. Transcripts were analyzed using rapid content analysis with key practices and issues synthesized in profile summaries. Themes were identified and iteratively revised, utilizing matrices to compare content across early- and late-adopting sites. KEY RESULTS: Five successful clinical practices were identified (use of two specific screening tools for primary IPV screening and secondary risk assessment, multilevel resource provision and community partnerships, co-location of mental health/social work, and patient-centered documentation). Multilevel barriers (time/resource constraints, competing priorities and mounting responsibilities in primary care, lack of policy, inadequate training, and discomfort addressing IPV) and facilitators (engaged IPV champions, internal and external supports, positive feedback regarding IPV screening practices, and current, national attention to violence against women) were identified. CONCLUSIONS: Findings advance national efforts by highlighting successful clinical practices for IPV screening programs and informing strategies useful for enhancing their implementation within and beyond the VHA, ultimately improving services and women's health.

When user-centered design meets implementation science: integrating provider perspectives in the development of an intimate partner violence intervention for women treated in the United States' largest integrated healthcare system.

BACKGROUND: Intimate partner violence (IPV) against women is a global health problem that is a substantial source of human suffering. Within the United States (US), women veterans are at high risk for experiencing IPV. There is an urgent need for feasible, acceptable, and patient-centered IPV counseling interventions for the growing number of women treated in the US's largest integrated healthcare system, the Veterans Health Administration (VHA). Implementation science and user-centered-design (UCD) can play an important role in accelerating the research-to-practice pipeline. Recovering from IPV through Strengths and Empowerment (RISE) is a flexible, patient-centered, modular-based program that holds promise as a brief counseling intervention for women veterans treated in VHA. We utilized a UCD approach to develop and refine RISE (prior to formal effectiveness evaluations) by soliciting early feedback from the providers where the intervention will ultimately be implemented. The current study reports on the feedback from VHA providers that was used to tailor and refine RISE. METHOD: We conducted and analyzed semi-structured, key-informant interviews with VHA providers working in clinics relevant to the delivery of IPV interventions (n = 23) at two large medical centers in the US. Participants' mean age was 42.6 years (SD = 11.6), they were predominately female (91.3%) and from a variety of relevant disciplines (39.1% psychologists, 21.7% social workers, 17.4% physicians, 8.7% registered nurses, 4.3% psychiatrists, 4.3% licensed marriage and family therapists, 4.3% peer specialists). We conducted rapid content analysis using a hybrid inductive-deductive approach. RESULTS: Providers perceived RISE as highly acceptable and feasible, noting strengths including RISE's structure, patient-centered agenda, and facilitation of provider comfort in addressing IPV. Researchers identified themes related to content and context modifications, including requests for additional safety check-ins, structure for goal-setting, and suggestions for how to develop and implement RISE-specific trainings. CONCLUSIONS: These findings have guided refinements to RISE prior to formal effectiveness testing in VHA. We discuss implications for the use of UCD in intervention development and refinement for interventions addressing IPV and other trauma in health care settings globally. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT03261700; Date of registration: 8/25/2017, date of enrollment of first participant in trial: 10/22/2018. Unique Protocol ID: IIR 16-062.

The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care.

There is strong interest in implementation science to address the gap between research and practice in public health. Research on the sustainability of evidence-based interventions has been growing rapidly. Sustainability has been defined as the continued use of program components at sufficient intensity for the sustained achievement of desirable program goals and population outcomes. This understudied area has been identified as one of the most significant translational research problems. Adding to this challenge is uncertainty regarding the extent to which intervention adaptation and evolution are necessary to address the needs of populations that differ from those in which interventions were originally tested or implemented. This review critically examines and discusses conceptual and methodological issues in studying sustainability, summarizes the multilevel factors that have been found to influence the sustainability of interventions in a range of public health and health care settings, and highlights key areas for future research.

A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy.

Despite the central role of training and consultation in the implementation of evidence-based psychological interventions (EBPIs), comprehensive reviews of research on training have highlighted serious gaps in knowledge regarding best practices. Consultation after initial didactic training appears to be of critical importance, but there has been very little research to determine optimal consultation format or interventions. This observational study compared two consultation formats that included review of session audio and feedback in the context of a program to train clinicians (n = 85) in community mental health clinics to deliver cognitive therapy (CT). A "gold standard" condition in which clinicians received individual feedback after expert consultants reviewed full sessions was compared to a group consultation format in which short segments of session audio were reviewed by a group of clinicians and an expert consultant. After adjusting for potential baseline differences between individuals in the two consultation conditions, few differences were found in terms of successful completion of the consultation phase or in terms of competence in CT at the end of consultation or after a 2 year follow-up. However, analyses did not support hypotheses regarding non-inferiority of the group consultation condition. While both groups largely maintained competence, clinicians in the group consultation condition demonstrated increases in competence over the follow-up period, while a sub-group of those in the individual condition experienced decreases. These findings, if replicated, have important implications for EBP implementation programs, as they suggest that observation and feedback is feasible in community mental health setting, and that employing this method in a group format is an effective and efficient consultation strategy that may enhance the implementation and sustainability of evidence-based psychotherapies.

System-Wide Implementation of Routine Outcome Monitoring and Measurement Feedback System in a National Network of Operational Stress Injury Clinics.

This manuscript reviews the development and implementation process of the Client Reported Outcome Monitoring Information System in the VA Canada Operational Stress Injury National Network, and reports on outcomes of an evaluation to assess usage, barriers, and facilitators to implementation. The majority of clinicians reported regular use of routine outcomes monitoring, although objective data suggested somewhat lower actual use. In general, clinicians endorsed all barriers and most facilitators as influencing their use of routine outcomes monitoring in a minor way. However, users and non-users differed in their endorsement of facilitators and barriers. Implications for research and implementation efforts are discussed.

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): a modified Delphi study.

BACKGROUND: Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources. METHOD: MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability and inclusion (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis. RESULTS: In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application. DISCUSSION: MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation.

Large language models (LLMs) such as Open AI's GPT-4 (which power ChatGPT) and Google's Gemini, built on artificial intelligence, hold immense potential to support, augment, or even eventually automate psychotherapy. Enthusiasm about such applications is mounting in the field as well as industry. These developments promise to address insufficient mental healthcare system capacity and scale individual access to personalized treatments. However, clinical psychology is an uncommonly high stakes application domain for AI systems, as responsible and evidence-based therapy requires nuanced expertise. This paper provides a roadmap for the ambitious yet responsible application of clinical LLMs in psychotherapy. First, a technical overview of clinical LLMs is presented. Second, the stages of integration of LLMs into psychotherapy are discussed while highlighting parallels to the development of autonomous vehicle technology. Third, potential applications of LLMs in clinical care, training, and research are discussed, highlighting areas of risk given the complex nature of psychotherapy. Fourth, recommendations for the responsible development and evaluation of clinical LLMs are provided, which include centering clinical science, involving robust interdisciplinary collaboration, and attending to issues like assessment, risk detection, transparency, and bias. Lastly, a vision is outlined for how LLMs might enable a new generation of studies of evidence-based interventions at scale, and how these studies may challenge assumptions about psychotherapy.

Clinicians' perspectives on cognitive therapy in community mental health settings: implications for training and implementation.

Policymakers are investing significant resources in large-scale training and implementation programs for evidence-based psychological treatments (EBPTs) in public mental health systems. However, relatively little research has been conducted to understand factors that may influence the success of efforts to implement EBPTs for adult consumers of mental health services. In a formative investigation during the development of a program to implement cognitive therapy (CT) in a community mental health system, we surveyed and interviewed clinicians and clinical administrators to identify potential influences on CT implementation within their agencies. Four primary themes were identified. Two related to attitudes towards CT: (1) ability to address client needs and issues that are perceived as most central to their presenting problems, and (2) reluctance to fully implement CT. Two themes were relevant to context: (1) agency-level barriers, specifically workload and productivity concerns and reactions to change, and (2) agency-level facilitators, specifically, treatment planning requirements and openness to training. These findings provide information that can be used to develop strategies to facilitate the implementation of CT interventions for clients being treated in public-sector settings.

An examination of individual and organizational theory in a pilot virtual facilitated learning collaborative to implement written exposure therapy.

Participant training alone is typically insufficient to ensure adoption of a new treatment. This study provides preliminary data on a pilot program that implemented written exposure therapy (WET) through a virtual facilitated learning collaborative (VFLC) that combined clinician training with implementation support. Ninety-five mental health clinicians from Veterans Health Administration clinics enrolled in the VFLC. Clinicians received 6 hr of training in WET plus weekly group consultation on training cases for 12-16 weeks. Clinic leaders participated in monthly group calls with facilitators to develop and track implementation plans, address barriers, and share resources. Clinicians completed the Perceived Characteristics of Intervention Scale and Implementation Climate Scale before training and after consultation, and the Intention Scale for providers after the program. Clinicians' positive perceptions of WET increased over time, t(85) = -8.80, p < .001, d = 0.95, and changes were moderated by how much reduction in symptoms their patients experienced, t(151) = -2.13, p = .035. The implementation climate for WET also improved over time, t(85) = -3.55, p < .001, d = 0.36. Consistent with the theory of planned behavior, clinicians' intention to continue using WET after training was influenced by their perceptions and attitudes about WET, subjective norms, and perceived behavioral control. The VFLC model was feasible and associated with increases in clinicians' and leaders' support for using WET. Patient outcomes predicted the amount of change in clinicians' perceptions about WET. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): A modified Delphi study.

Background: Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources. Method: MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis. Results: In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application. Discussion: MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: Study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial.

treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase - the Sustainment Phase - of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial - the Implementation Phase and Train-the-Trainer Phase - TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes. Methods: Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data. Discussion: The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change. Trial Registration: ClinicalTrials.gov identifier: NCT05956678. Registered on July 21, 2023. https://classic.clinicaltrials.gov/ct2/show/NCT05956678?term=NCT05956678&draw=2&rank=1.

A critical evaluation of the complex PTSD literature: implications for DSM-5.

Complex posttraumatic stress disorder (CPTSD) has been proposed as a diagnosis for capturing the diverse clusters of symptoms observed in survivors of prolonged trauma that are outside the current definition of PTSD. Introducing a new diagnosis requires a high standard of evidence, including a clear definition of the disorder, reliable and valid assessment measures, support for convergent and discriminant validity, and incremental validity with respect to implications for treatment planning and outcome. In this article, the extant literature on CPTSD is reviewed within the framework of construct validity to evaluate the proposed diagnosis on these criteria. Although the efforts in support of CPTSD have brought much needed attention to limitations in the trauma literature, we conclude that available evidence does not support a new diagnostic category at this time. Some directions for future research are suggested.