Application of the Expert Recommendations for Implementing Change (ERIC) compilation of strategies to health intervention implementation in low- and middle-income countries: a systematic review.

BACKGROUND: The Expert Recommendations for Implementing Change (ERIC) project developed a compilation of implementation strategies that are intended to standardize reporting and evaluation. Little is known about the application of ERIC in low- and middle-income countries (LMICs). We systematically reviewed the literature on the use and specification of ERIC strategies for health intervention implementation in LMICs to identify gaps and inform future research. METHODS: We searched peer-reviewed articles published through March 2023 in any language that (1) were conducted in an LMIC and (2) cited seminal ERIC articles or (3) mentioned ERIC in the title or abstract. Two co-authors independently screened all titles, abstracts, and full-text articles, then abstracted study, intervention, and implementation strategy characteristics of included studies. RESULTS: The final sample included 60 studies describing research from all world regions, with over 30% published in the final year of our review period. Most studies took place in healthcare settings (n = 52, 86.7%), while 11 (18.2%) took place in community settings and four (6.7%) at the policy level. Across studies, 548 distinct implementation strategies were identified with a median of six strategies (range 1-46 strategies) included in each study. Most studies (n = 32, 53.3%) explicitly matched implementation strategies used for the ERIC compilation. Among those that did, 64 (87.3%) of the 73 ERIC strategies were represented. Many of the strategies not cited included those that target systems- or policy-level barriers. Nearly 85% of strategies included some component of strategy specification, though most only included specification of their action (75.2%), actor (57.3%), and action target (60.8%). A minority of studies employed randomized trials or high-quality quasi-experimental designs; only one study evaluated implementation strategy effectiveness. CONCLUSIONS: While ERIC use in LMICs is rapidly growing, its application has not been consistent nor commonly used to test strategy effectiveness. Research in LMICs must better specify strategies and evaluate their impact on outcomes. Moreover, strategies that are tested need to be better specified, so they may be compared across contexts. Finally, strategies targeting policy-, systems-, and community-level determinants should be further explored. TRIAL REGISTRATION: PROSPERO, CRD42021268374.

Ten years of implementation outcomes research: a scoping review.

BACKGROUND: Proctor and colleagues' 2011 paper proposed a taxonomy of eight implementation outcomes and challenged the field to address a research agenda focused on conceptualization, measurement, and theory building. Ten years later, this paper maps the field's progress in implementation outcomes research. This scoping review describes how each implementation outcome has been studied, research designs and methods used, and the contexts and settings represented in the current literature. We also describe the role of implementation outcomes in relation to implementation strategies and other outcomes. METHODS: Arksey and O'Malley's framework for conducting scoping reviews guided our methods. Using forward citation tracing, we identified all literature citing the 2011 paper. We conducted our search in the Web of Science (WOS) database and added citation alerts sent to the first author from the publisher for a 6-month period coinciding with the WOS citation search. This produced 1346 titles and abstracts. Initial abstract screening yielded 480 manuscripts, and full-text review yielded 400 manuscripts that met inclusion criteria (empirical assessment of at least one implementation outcome). RESULTS: Slightly more than half (52.1%) of included manuscripts examined acceptability. Fidelity (39.3%), feasibility (38.6%), adoption (26.5%), and appropriateness (21.8%) were also commonly examined. Penetration (16.0%), sustainability (15.8%), and cost (7.8%) were less frequently examined. Thirty-two manuscripts examined implementation outcomes not included in the original taxonomy. Most studies took place in healthcare (45.8%) or behavioral health (22.5%) organizations. Two-thirds used observational designs. We found little evidence of progress in testing the relationships between implementation strategies and implementation outcomes, leaving us ill-prepared to know how to achieve implementation success. Moreover, few studies tested the impact of implementation outcomes on other important outcome types, such as service systems and improved individual or population health. CONCLUSIONS: Our review presents a comprehensive snapshot of the research questions being addressed by existing implementation outcomes literature and reveals the need for rigorous, analytic research and tests of strategies for attaining implementation outcomes in the next 10 years of outcomes research.

Factors influencing pediatric transfusion: A complex decision impacting quality of care.

BACKGROUND: The risks of red blood cell transfusion may outweigh the benefits for many patients in pediatric intensive care units (PICUs), but guidelines from the Transfusion and Anemia eXpertise Initiative (TAXI) have not been consistently adopted. We sought to identify factors that influenced transfusion decision-making in PICUs to explore potential barriers and facilitators to implementing the guidelines. STUDY DESIGN AND METHODS: A total of 50 ICU providers working in eight US ICUs of different types (non-cardiac PICUs, cardiovascular ICUs, combined units) and variable sizes (11-32 beds) completed semi-structured interviews. Providers included ICU attendings and trainees, nurse practitioners, nurses, and subspecialty physicians. Interviews examined factors that influenced transfusion decisions, transfusion practices, and provider beliefs. Qualitative analysis utilized a Framework Approach. Summarized data was compared between provider roles and units with consideration to identify patterns and unique informative statements. RESULTS: Providers cited clinical, physiologic, anatomic, and logistic factors they considered in making transfusion decisions. Improving oxygen carrying capacity, hemodynamics and perfusion, respiratory function, volume deficits, and correcting laboratory values were among the reasons given for transfusion. Other sought-after benefits included alleviating symptoms of anemia, improving ICU throughput, and decreasing blood waste. Providers in different roles approached transfusion decisions differently, with the largest differences noted between nurses and subspecialists as compared with other ICU providers. While ICU attendings most often made the decision to transfuse, all providers influenced the decision-making. DISCUSSION: Implementation of transfusion guidelines requires multi-professional approaches that emphasize the known risks of transfusion, its limited benefits, and highlight evidence around the safety and benefit of restrictive approaches.

Implementation of a Low-Cost, Multi-component, Web-Based Training for Trauma-Focused Cognitive-Behavioral Therapy.

Effective, interactive trainings in evidence-based practices remain expensive and largely inaccessible to most practicing clinicians. To address this need, the current study evaluated the impact of a low-cost, multi-component, web-based training for Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT) on clinicians' TF-CBT knowledge, strategy use, adherence and skill. Clinician members of a practice-based research network were recruited via email and randomized to either an immediate training group (N = 89 assigned) or waitlist control group (N = 74 assigned) that was offered access to the same training after six months, with half of each group further randomized to receive or not receive incentives for participation. Clinicians completed assessments at baseline, 6 months, and 12 months covering TF-CBT knowledge, strategy use, and for a subset of clinicians (n = 28), TF-CBT adherence and skill. Although significant differences in overall TF-CBT skillfulness and readiness were found, there were no significant differences between the training and waitlist control group on TF-CBT knowledge and strategy use at six months. However, there was considerable variability in the extent of training completed by clinicians. Subsequent post-hoc analyses indicated a significant, positive association between the extent of training completed by clinicians and clinician TF-CBT knowledge, strategy use, demonstrated adherence and skill across the three TF-CBT components, and overall TF-CBT readiness. We also explored whether incentives predicted training participation and found no differences in training activity participation between clinicians who were offered an incentive and those who were not. Findings highlight the limitations of self-paced web-based trainings. Implications for web-based trainings are discussed.

Scaling-up Child and Youth Mental Health Services: Assessing Coverage of a County-Wide Prevention and Early Intervention Initiative During One Fiscal Year.

PURPOSE: In the U.S., the percentage of youth in need of evidence-based mental health practices (EBPs) who receive them (i.e., coverage rate) is low. We know little about what influences coverage rates. In 2010, the Los Angeles County Department of Mental Health (LACDMH) launched a reimbursement-driven implementation of multiple EBPs in youth mental health care. This study examines two questions: (1) What was the coverage rate of EBPs delivered three years following initial implementation? (2) What factors are associated with the coverage rates? METHODS: To assess coverage rates of publicly insured youth, we used LACDMH administrative claims data from July 1, 2013 to June 30, 2014 and estimates of the size of the targeted eligible youth population from the 2014 American Community Survey (ACS). The unit of analysis was clinic service areas (n = 254). We used Geographic Information Systems and an OLS regression to assess community and clinic characteristics related to coverage. RESULTS: The county coverage rate was estimated at 17%, much higher than national estimates. The proportion of ethnic minorities, individuals who are foreign-born, adults with a college degree within a geographic area were negatively associated with clinic service area coverage rates. Having more therapists who speak a language other than English, providing care outside of clinics, and higher proportion of households without a car were associated with higher coverage rates. CONCLUSION: Heterogeneity in municipal mental health record type and availability makes it difficult to compare the LACDMH coverage rate with other efforts. However, the LACDMH initiative has higher coverage than published national rates. Having bilingual therapists and providing services outside the clinic was associated with higher coverage. Even with higher coverage, inequities persisted.

Six practical recommendations for improved implementation outcomes reporting.

BACKGROUND: Implementation outcomes research spans an exciting mix of fields, disciplines, and geographical space. Although the number of studies that cite the 2011 taxonomy has expanded considerably, the problem of harmony in describing outcomes persists. This paper revisits that problem by focusing on the clarity of reporting outcomes in studies that examine them. Published recommendations for improved reporting and specification have proven to be an important step in enhancing the rigor of implementation research. We articulate reporting problems in the current implementation outcomes literature and describe six practical recommendations that address them. RECOMMENDATIONS: Our first recommendation is to clearly state each implementation outcome and provide a definition that the study will consistently use. This includes providing an explanation if using the taxonomy in a new way or merging terms. Our second recommendation is to specify how each implementation outcome will be analyzed relative to other constructs. Our third recommendation is to specify "the thing" that each implementation outcome will be measured in relation to. This is especially important if you are concurrently studying interventions and strategies, or if you are studying interventions and strategies that have multiple components. Our fourth recommendation is to report who will provide data and the level at which data will be collected for each implementation outcome, and to report what kind of data will be collected and used to assess each implementation outcome. Our fifth recommendation is to state the number of time points and frequency at which each outcome will be measured. Our sixth recommendation is to state the unit of observation and the level of analysis for each implementation outcome. CONCLUSION: This paper advances implementation outcomes research in two ways. First, we illustrate elements of the 2011 research agenda with concrete examples drawn from a wide swath of current literature. Second, we provide six pragmatic recommendations for improved reporting. These recommendations are accompanied by an audit worksheet and a list of exemplar articles that researchers can use when designing, conducting, and assessing implementation outcomes studies.

Implementing evidence-based practices in nonspecialty mental health settings.

OBJECTIVE: Mental health services are often implemented in settings in which mental health is not the primary mission. Schools, primary care clinics, criminal justice and child welfare institutions, and senior centers have been increasingly adding mental health care to their compendium of services owing to the high rates of mental health needs in these settings. Despite numerous challenges to implementing mental health practices in settings where mental health care has not traditionally been a part of the service model, the demand for mental health services in these settings is growing. Implementation science offers a lens through which to understand unique challenges and potential solutions to implementing mental health services in these settings, with the ultimate goal of ensuring the implementation of evidence-based services. METHOD AND RESULTS: In this conceptual article, we discuss common overarching barriers to implementing evidence-based mental health interventions in nonspecialty mental health settings, such as workforce challenges, competing priorities, and financial considerations. Then, using examples from schools and primary care, we elucidate unique contextual considerations and implementation challenges in these settings. Finally, we articulate a research agenda for advancing implementation of evidence-based practices in nonspecialty mental health including highlighting promising implementation strategies (e.g., task shifting, adaptation) and the most relevant implementation outcomes to assess in these contexts (e.g., appropriateness, feasibility). CONCLUSIONS: Given that nonspecialty mental health settings deliver a large proportion of mental health care to traditionally underserved populations, there are important public health implications to advancing research in this area and ensuring implemented services are evidence-based. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy.

Numerous efforts are underway to train clinicians in evidence-based practices. Unfortunately, the field has few practical measures of therapist adherence and skill with which to judge the success of these training and implementation efforts. One possible assessment method is using behavioral rehearsal, or role-play, as an analogue for therapist in-session behavior. The current study describes aspects of reliability, validity and utility of a behavioral role-play assessment developed to evaluate therapist adherence and skill in implementing Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). TF-CBT role-play assessments were conducted with a sample of 43 therapists as part of a larger training study. The TF-CBT role-play assessments were independently coded for TF-CBT adherence and skill by a certified TF-CBT trainer and three clinical psychology doctoral students. Findings indicated good interrater reliability for the individual items (ICC: M = .71, SD = .15). Regarding utility, 67.19% (n = 43/64) of contacted therapists completed the role-play assessment, which took an average of 30 min (M = 31.42, SD = 5.65) to complete and 60 min (M = 62.84, SD = 11.31) to code. Therapists with a master's degree were more likely to complete the role-play assessment than those with other degrees but no other differences in demographic variables, practice characteristics, or TF-CBT knowledge or training were found between participants and nonparticipants. Role-play assessments may offer an alternative to observational coding for assessing therapist adherence and skill, particularly in contexts where session recordings are not feasible.

Moving beyond Aim Three: a need for a transdisciplinary approach to build capacity for economic evaluations in implementation science.

BACKGROUND: Understanding the costs and economic benefits of implementation has been identified by policymakers and researchers as critical to increase the uptake and sustainment of evidence-based practices, but this topic remains relatively understudied. Conducting team science with health economists has been proposed as a solution to increase economic evaluation in implementation science; however, these recommendations ignore the differences in goals and perspectives in these two fields. Our recent qualitative research identified that implementation researchers predominantly approach health economists to examine costs, whereas the majority of health economists expressed limited interest in conducting economic evaluations and a desire to be more integrated within implementation science initiatives. These interviews pointed to challenges in establishing fruitful partnerships when health economists are relegated to the "Third Aim" (i.e., lowest-priority research objective) in implementation science projects by their research partners. DISCUSSION: In this debate paper, we argue that implementation researchers and health economists need to focus on team science research principles to expand capacity to address pressing research questions that cut across the two fields. Specifically, we use the four-phase model of transdisciplinary research to outline the goals and processes needed to build capacity in this area (Hall et al., Transl Behav Med 2:415-30, 2012). The first phase focuses on the development of transdisciplinary research teams, including identifying appropriate partners (e.g., considering policy or public health researchers in addition to health economists) and building trust. The conceptual phase focuses on strategies to consider when developing joint research questions and methodology across fields. In the implementation phase, we outline the effective processes for conducting research projects, such as team learning. Finally, in the translation phase, we highlight how a transdisciplinary approach between health economists and implementation researchers can impact real-world practice and policy. The importance of investigating the economic impact of evidence-based practice implementation is widely recognized, but efforts have been limited due to the challenges in conducting team science across disciplines. Training in team science can help advance transdisciplinary efforts, which has the potential to increase the rigor and impact of economic evaluations in implementation science while expanding the roles taken by health economists.

Market viability: a neglected concept in implementation science.

This debate paper asserts that implementation science needs to incorporate a key concept from entrepreneurship-market demand-and demonstrates how assessing an innovation's potential market viability might advance the pace and success of innovation adoption and sustainment. We describe key concepts, language distinctions, and questions that entrepreneurs pose to implementation scientists-many of which implementation scientists appear ill-equipped to answer. The paper concludes with recommendations about how concepts from entrepreneurship, notably market viability assessment, can enhance the translation of research discoveries into real-world adoption, sustained use, and population health benefits. The paper further proposes activities that can advance implementation science's capacity to draw from the field of entrepreneurship, along with the data foundations required to assess and cultivate market demand.

A new lane for science.

A recent Science editorial on the social and political headwinds that have blunted, obfuscated, and confused public behavior in the United States' COVID-19 response cautioned both politicians who appoint themselves scientists and scientists-including virologists and epidemiologists-to stay in their lanes. The warning raises an important question: Should science add another lane?

Documenting Behavioral Health Needs in an Urban Setting.

Most local communities lack the capacity to conduct behavioral health needs assessments. The purpose of this paper is to describe a mixed-methods approach to estimate the behavioral health needs in St. Louis, MO. Data were drawn from multiple sources including local and state government prevalence estimates, medical records, and key informant interviews. The most prevalent behavioral conditions were depression, alcohol, and drug abuse. Priority populations were residents with co-occurring disorders, youth transitioning into the adult behavioral system, and homeless individuals with behavioral health needs. Treatment rates for behavioral health conditions were low, relative to identified needs. There are significant provider shortages and high staff turnover, which extend wait times, diminish the quality of care, and contribute to the use of emergency departments for behavioral health care. The data and methods described in this paper could be helpful to other municipalities that are looking to conduct behavioral health needs assessments.

Partnered innovation to implement timely and personalized care: A case study.

BACKGROUND: Understanding how to translate research discoveries into solutions for healthcare improvement is a priority of NIH-funded Clinical and Translational Science Awards (CTSA). This study, supported by one CTSA, aims to capture one process of shaping and implementing innovations to advance the timeliness and patient-centeredness of cardiovascular care. Specifically, we sought to understand a partnership between a private digital health startup company, a university innovation lab, and an academic health system's cardiology program pursuing this goal. FINDINGS: The collaboration proceeded through clear phases to address the questions and challenges: problem definition, exploration and formalization of the partnership, innovation co-creation and pilot test, and scale-up planning. Phases were punctuated by key decisions, such as forming the partnership, negotiating terms of the partnership, iterating form and features of the innovation, and exploring sufficiency of its value-add for scale-up and sustainment. Key implementation concepts were apparent, including implementation strategies (e.g., champions and iterative trialing) and the implementation outcomes of acceptability, sustainment, and scale-up. Participants identified potential risks of collaboration, reflected on their co-creation process, and the value of engaging stakeholders in innovation design. Findings may inform subsequent collaborations between innovators and translational researchers. METHODS: We conducted a case study to understand the partnership; characterize the questions they pursued, their decision points, information and data sources; and identify the challenges and risks. Data were collected through a series of four focus groups with members of each partnering organization. A transdisciplinary research team iteratively worked to condense and synthesize data from audio recorded transcripts into a case narrative.

Maintenance of Training Effects of Two Models for Implementing Evidence-Based Psychological Treatment.

OBJECTIVE: The authors compared maintenance of training outcomes for two approaches to training college therapists in interpersonal psychotherapy (IPT): train the trainer versus expert training. METHODS: A cluster-randomized trial was conducted in 24 college counseling centers. Therapists were recruited from enrolled centers, and the therapists enrolled students with depression and eating disorder symptoms. The therapists (N=184) provided data during baseline, posttraining (during the 12 months of expert consultation offered to the expert training group), and maintenance (approximately 7 months after the expert consultation ended). Outcomes were therapist fidelity (i.e., adherence and competence) and IPT knowledge. RESULTS: Both groups showed within-group improvement from baseline to the maintenance period for adherence, competence, and IPT knowledge; however, the train-the-trainer group had greater improvement over time in adherence and competence. CONCLUSIONS: Given that the effects of the train-the-trainer approach were better maintained, and this model's potential to train more therapists over time, the train-the-trainer approach may help increase dissemination of evidence-based treatments such as IPT.

Ten years of implementation outcome research: a scoping review protocol.

INTRODUCTION: A 2011 paper proposed a working taxonomy of implementation outcomes, their conceptual distinctions and a two-pronged research agenda on their role in implementation success. Since then, over 1100 papers citing the manuscript have been published. Our goal is to compare the field's progress to the originally proposed research agenda, and outline recommendations for the next 10 years. To accomplish this, we are conducting the proposed scoping review. METHODS AND ANALYSIS: Our approach is informed by Arksey and O'Malley's methodological framework for conducting scoping reviews. We will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews. We first aim to assess the degree to which each implementation outcome has been investigated in the literature, including healthcare settings, clinical populations and innovations represented. We next aim to describe the relationship between implementation strategies and outcomes. Our last aim is to identify studies that empirically assess relationships among implementation and/or service and client outcomes. We will use a forward citation tracing approach to identify all literature that cited the 2011 paper in the Web of Science (WOS) and will supplement this with citation alerts sent to the second author for a 6-month period coinciding with the WOS citation search. Our review will focus on empirical studies that are designed to assess at least one of the identified implementation outcomes in the 2011 taxonomy and are published in peer-reviewed journals. We will generate descriptive statistics from extracted data and organise results by these research aims. ETHICS AND DISSEMINATION: No human research participants will be involved in this review. We plan to share findings through a variety of means including peer-reviewed journal publications, national conference presentations, invited workshops and webinars, email listservs affiliated with our institutions and professional associations, and academic social media.

Measurement of unnecessary psychiatric readmissions in the context of care transition interventions: a scoping review.

OBJECTIVE: The objective of this study was to examine how published studies of inpatient to outpatient mental healthcare transition processes have approached measuring unnecessary psychiatric readmissions. DESIGN: Scoping review using Levac et al's enhancement to Arksey and O'Malley's framework for conducting scoping reviews. DATA SOURCES: Medline (Ovid), Embase (Ovid), PsycINFO, CINAHL, Cochrane and ISI Web of Science article databases were searched from 1 January 2009 through 28 February 2019. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included studies that (1) are about care transition processes associated with unnecessary psychiatric readmissions and (2) specify use of at least one readmission time interval (ie, the time period since previous discharge from inpatient care, within which a hospitalisation can be considered a readmission). DATA EXTRACTION AND SYNTHESIS: We assessed review findings through tabular and content analyses of the data extracted from included articles. RESULTS: Our database search yielded 3478 unique articles, 67 of which were included in our scoping review. The included articles varied widely in their reported readmission time intervals used. They provided limited details regarding which readmissions they considered unnecessary and which risks they accounted for in their measurement. There were no perceptible trends in associations between the variation in these findings and the included studies' characteristics (eg, target population, type of care transition intervention). CONCLUSIONS: The limited specification with which studies report their approach to unnecessary psychiatric readmissions measurement is a noteworthy gap identified by this scoping review, and one that can hinder both the replicability of conducted studies and adaptations of study methods by future investigations. Recommendations stemming from this review include (1) establishing a framework for reporting the measurement approach, (2) devising enhanced guidelines regarding which approaches to use in which circumstances and (3) examining how sensitive research findings are to the choice of the approach.