Using cognitive interviews to improve a measure of organizational readiness for implementation.

BACKGROUND: Organizational readiness is a key factor for successful implementation of evidence-based interventions (EBIs), but a valid and reliable measure to assess readiness across contexts and settings is needed. The R = MC2 heuristic posits that organizational readiness stems from an organization's motivation, capacity to implement a specific innovation, and its general capacity. This paper describes a process used to examine the face and content validity of items in a readiness survey developed to assess organizational readiness (based on R = MC2) among federally qualified health centers (FQHC) implementing colorectal cancer screening (CRCS) EBIs. METHODS: We conducted 20 cognitive interviews with FQHC staff (clinical and non-clinical) in South Carolina and Texas. Participants were provided a subset of items from the readiness survey to review. A semi-structured interview guide was developed to elicit feedback from participants using "think aloud" and probing techniques. Participants were recruited using a purposive sampling approach and interviews were conducted virtually using Zoom and WebEx. Participants were asked 1) about the relevancy of items, 2) how they interpreted the meaning of items or specific terms, 3) to identify items that were difficult to understand, and 4) how items could be improved. Interviews were transcribed verbatim and coded in ATLAS.ti. Findings were used to revise the readiness survey. RESULTS: Key recommendations included reducing the survey length and removing redundant or difficult to understand items. Additionally, participants recommended using consistent terms throughout (e.g., other units/teams vs. departments) the survey and changing pronouns (e.g., people, we) to be more specific (e.g., leadership, staff). Moreover, participants recommended specifying ambiguous terms (e.g., define what "better" means). CONCLUSION: Use of cognitive interviews allowed for an engaged process to refine an existing measure of readiness. The improved and finalized readiness survey can be used to support and improve implementation of CRCS EBIs in the clinic setting and thus reduce the cancer burden and cancer-related health disparities.

Improving Sexual Assault and Sexual Harassment Prevention from the Bottom-up: a Pilot of Getting To Outcomes in the US Military.

While the Department of Defense (DoD) has given increased attention and priority to preventing sexual assault and sexual harassment (SA/SH), it remains a problem. To build its prevention capacity, DoD piloted Getting To Outcomes® (GTO®) from 2019 to 2022 at 10 military installations. GTO is an evidence-based planning and implementation support that has been used in many civilian contexts but has only recently been adapted for military SA/SH. The purpose of this study was to describe GTO use, identify its benefits and challenges, and discuss lessons the GTO effort yielded for prevention more broadly using a framework of organizational and program-level capacities needed for successful prevention in the military context, called the Prevention Evaluation Framework (PEF). GTO was piloted with 10 military installations ("sites") representing all Military Services, plus the Coast Guard and National Guard. GTO is comprised of a written guide, training, and ongoing coaching. The pilot's goal was for each site to use GTO to implement a SA/SH prevention program twice. Participants from each site were interviewed and data was collected on GTO steps completed, whether GTO spurred new evaluation activities and collaborations, and the degree of leadership support for GTO. Most sites completed all GTO steps at least once. Interviews showed that DoD participants believe GTO improved prevention understanding, planning, and evaluation capacity; strengthened confidence in chosen programs; and helped sites tailor programs to the military context. Barriers were the complexity of GTO, DoD personnel turnover, and the disruption that the COVID pandemic caused in sexual assault prevention program delivery. Many respondents were unsure if they would continue all of GTO after the coaching ended, but many believed they would continue at least some parts. According to the PEF, the GTO pilot revealed several additional prevention system gaps (e.g., need for leadership support) and changes needed to GTO (e.g., stronger leader and champion engagement), to support quality prevention. The military and other large organizations will need to focus on these issues to ensure prevention implementation and evaluation are conducted with quality.

A Proactive, Systematic Approach to Building the Capacity of Technical Assistance Providers.

Technical assistance (TA) is a major capacity building strategy used by the government sector to promote health outcomes in the United States. However, there is minimal literature about how to develop TA provider capacities. This article describes a systematic and proactive approach for developing TA provider capacity, referred to as Technical Assistance for Technical Assistance Providers (TAFTAP), which draws on three implementation science frameworks (Interactive Systems Framework for Dissemination and Implementation, Getting To Outcomes, and R = MC2). We present an application of TAFTAP within a federal agency providing a readiness-informed TA approach to health departments of states, territories, and tribal areas implementing comprehensive tobacco prevention control programs. Pilot data suggest that TAFTAP is a promising approach for improving the quality of TA delivery. At the end of the 2-year project period, TAFTAP recipients provided generally positive qualitative feedback about the support they received. They chose to sustain the readiness-informed TA by incorporating it into a future funding announcement. Downstream state-level TA grantee recipients reported positive outcomes (e.g., accelerated progress, enjoying more one-on-one time with TA providers) from receiving the TA innovation from TAFTAP recipients. We suggest that funding agencies and training and TA centers consider this approach to bolster the capacity and motivation of TA providers for downstream benefit to health and human services staff and their clients. Practical steps for employing TAFTAP to advance health outcomes are included in this article.

Examining readiness for implementing practice changes in federally qualified health centers: A rapid qualitative study.

Implementing evidence-based interventions remains slow in federally qualified health centers (FQHCs). The purpose of this study is to qualitatively examine the R = MC2 (Readiness = motivation × innovation specific capacity × general capacity) heuristic subcomponents in the context of implementing general and colorectal cancer screening (CRCS)-related practice changes in FQHCs. We conducted 17 interviews with FQHC employees to examine (1) experiences with successful or unsuccessful practice change efforts, (2) using approaches to promote CRCS, and (3) opinions about R = MC2 subcomponents. We conducted a rapid qualitative analysis to examine the frequency, depth, and spontaneity of subcomponents. Priority, compatibility, observability (motivation), intra- and interorganizational relationships (innovation-specific capacity), and organizational structure and resource utilization (general capacity) emerged as highly relevant. For example, organizational structure was described as related to an organization's open communication during meetings to help with scheduling procedures. The results contribute to understanding organizational readiness in the FQHC setting and can be helpful when identifying and prioritizing barriers and facilitators that affect implementation.

Using readiness to understand implementation challenges in school mental health research.

Schools and research partners are increasingly implementing complex, multicomponent interventions and school-wide frameworks to better meet students' social, emotional, behavioral, and academic needs; however, in the research and real-world contexts, implementation is often fraught with many challenges and barriers to success. This study explores implementation barriers encountered during a randomized controlled trial testing effects of one complex intervention strategy-the Interconnected Systems Framework-from the lens of a practical model for conceptualizing organizational readiness-the Interactive Systems Framework for Dissemination and Implementation. Implementation of the Interconnected Systems Framework was explored via focus group and key informant interviews with school and mental health professionals, and research team members responsible for implementing the intervention in randomly assigned study schools. Results from inductive thematic analysis of verbatim transcripts identified three primary implementation challenges: staff turnover, inadequate leadership buy-in, and insufficient time for training/planning. Each challenge is explored from interview participants' perspectives and the extant literature, then connected to recommendations from implementation science to help others avoid similar challenges in their well-intentioned efforts to address the mounting concern for students' wellbeing.

R = MC2 readiness building process: A practical approach to support implementation in local, state, and national settings.

Effective implementation of evidence-based interventions is a persistent challenge across community settings. Organizational readiness - or, the motivation and collective capacity of an entity to adopt and sustain an innovation - is important to facilitate implementation. Drawing on the R = MC2 readiness framework, we developed a readiness building process to tailor support for implementation. The process is composed of the following stages: assessment, feedback and prioritization, and strategize. In this article, we describe the application of the readiness building process through three case examples representing interventions at different ecological levels: local, state, and national. The case examples illuminate challenges and practical considerations for using the readiness building process, including the significance of on-going leadership engagement and collaboration between support system and delivery system staff. To further the research and practice of implementation readiness, we suggest examining the impact of readiness building on implementation outcomes and developing an empirically-informed repository of change management strategies matched to readiness constructs.

Community coalition readiness for implementing something new: Using a Delphi methodology.

Organizational readiness for an innovation (a program, policy, or practice new to a setting) is important for successful implementation. The R = MC2 heuristic focuses on capacity and motivation as a framework for readiness; it comprises 3 components and 18 subcomponents. A Delphi study with community coalition leaders was undertaken to assess when each subcomponent is most significant in the life cycle of implementing an innovation in a coalition. Results include that motivation is viewed as most important in the early stages of implementation and leadership, climate, program champions, and interorganizational relationships are relevant throughout the lifespan of implementation. Results have implications for the future of assessing and building coalition readiness.

Research versus practice in quality improvement? Understanding how we can bridge the gap.

The gap between implementers and researchers of quality improvement (QI) has hampered the degree and speed of change needed to reduce avoidable suffering and harm in health care. Underlying causes of this gap include differences in goals and incentives, preferred methodologies, level and types of evidence prioritized and targeted audiences. The Salzburg Global Seminar on 'Better Health Care: How do we learn about improvement?' brought together researchers, policy makers, funders, implementers, evaluators from low-, middle- and high-income countries to explore how to increase the impact of QI. In this paper, we describe some of the reasons for this gap and offer suggestions to better bridge the chasm between researchers and implementers. Effectively bridging this gap can increase the generalizability of QI interventions, accelerate the spread of effective approaches while also strengthening the local work of implementers. Increasing the effectiveness of research and work in the field will support the knowledge translation needed to achieve quality Universal Health Coverage and the Sustainable Development Goals.

Long-Term Sustainability of Evidence-Based Prevention Interventions and Community Coalitions Survival: a Five and One-Half Year Follow-up Study.

This study examines (1) coalition survival, (2) prevalence of evidence-based prevention interventions (EBPIs) to reduce substance abuse implemented as part of the Tennessee Strategic Prevention Framework (SPF) State Incentive Grant (SIG), (3) EBPI sustainability, and (4) factors that predict EBPI sustainability. Secondary data were collected on 27 SPF SIG-funded coalitions and 88 EBPI and non-EBPI implementations. Primary data were collected by a telephone interview/web survey five and one-half years after the SPF SIG ended. Results from secondary data show that 25 of the 27 coalitions survived beyond the SPF SIG for one to five and one-half years; 19 coalitions (70%) were still active five and one-half years later. Further, 88 EBPIs and non-EBPIs were implemented by 27 county SPF SIG coalitions. Twenty-one (21) of 27 coalitions (78%) implemented one to three EBPIs, totaling 37 EBPI implementations. Based on primary survey data on 29 of the 37 EBPI implementations, 28 EBPIs (97%) were sustained between two and five and one-half years while 22 EBPI implementations (76%) were sustained for five and one-half years. When controlling for variability among coalitions (nesting of EBPIs in coalitions), increases in data resources (availability of five types of prevention data) was a strong predictor of length of EBPI sustainability. Positive change in extramural funding resources and level of expertise during SPF SIG implementation, as well as level of coalition formalization at the end of the SPF SIG predicted EBPI sustainability length. One intervention attribute (trialability) also predicted length of sustainability. Implications are discussed.

Technical Assistance to Enhance Prevention Capacity: a Research Synthesis of the Evidence Base.

Despite the availability of many evidence-based prevention interventions (EBIs), gaps exist in bringing these programs into widespread practice. Technical assistance (TA) is a strategy for enhancing the readiness of practitioners to implement EBIs. Although many millions of dollars are spent on TA each year, there is little consensus about what the essential features of TA are and how to provide TA with quality. A broad-based research synthesis methodology was used for analyzing the current evidence base for TA using three frames: (1) applying the Getting To Outcomes (GTO) model for categorizing evidence on TA that specifies tasks for planning, implementing, and evaluating TA; (2) understanding the relevance of a successful relationship between the TA provider and TA recipient; and (3) considering the extent to which TA fits the life cycle needs of the preventive intervention. Results indicated that an explicit model or organizing framework is rarely used to plan, implement, and/or evaluate TA; specific TA tasks performed vary widely across studies; TA is rarely delivered to recipients who are seeking to sustain innovations subsequent to adoption and implementation; however, there is systematic attention to relationships and relationship-building. Overall, this synthesis indicates that the extent to which TA is being delivered systematically is limited. We suggest that funders and other stakeholders develop and implement standards for TA quality in order to ensure that many of these limitations are addressed.

The Results of the "Positive Action for Today's Health" (PATH) Trial for Increasing Walking and Physical Activity in Underserved African-American Communities.

BACKGROUND: The "Positive Action for Today's Health" (PATH) trial tested an environmental intervention to increase walking in underserved communities. METHODS: Three matched communities were randomized to a police-patrolled walking plus social marketing, a police-patrolled walking-only, or a no-walking intervention. The 24-month intervention addressed safety and access for physical activity (PA) and utilized social marketing to enhance environmental supports for PA. African-Americans (N=434; 62% females; aged 51±16 years) provided accelerometry and psychosocial measures at baseline and 12, 18, and 24 months. Walking attendance and trail use were obtained over 24 months. RESULTS: There were no significant differences across communities over 24 months for moderate-to-vigorous PA. Walking attendance in the social marketing community showed an increase from 40 to 400 walkers per month at 9 months and sustained ~200 walkers per month through 24 months. No change in attendance was observed in the walking-only community. CONCLUSIONS: Findings support integrating social marketing strategies to increase walking in underserved African-Americans (ClinicalTrials.gov #NCT01025726).

Strengthening the Science and Practice of Implementation Support: Evaluating the Effectiveness of Training and Technical Assistance Centers.

Hundreds of millions of dollars are spent each year by U.S. federal agencies for training and technical assistance (TTA) to be delivered by training and technical assistance centers (TTACs) to "delivery system organizations" (e.g., federally qualified health centers, state departments of health, substance abuse treatment centers, schools, and healthcare organizations). TTACs are often requested to help delivery system organizations implement evidence-based interventions. Yet, counterintuitively, TTACs are rarely required to use evidence-based approaches when supporting delivery systems (in the use of evidence-based programs). In fact, evaluations of TTAC activities tend to be minimal; evaluation of technical assistance (if conducted at all) often emphasizes outputs (number of encounters), satisfaction, and self-reports of knowledge gained-more substantive outcomes are not evaluated. The gap between (a) the volume of TTA services being funded and provided and (b) the evaluation of those services is immense and has the potential to be costly. The basic question to be answered is: how effective are TTA services? This article introduces the special issue on Strengthening the Science and Practice of Implementation Support: Evaluating the Effectiveness of Training and Technical Assistance Centers. The special issue promotes 1) knowledge of the state of the art of evaluation of TTACs and 2) advances in what to evaluate in TTA. A major goal of the issue is to improve the science and practice of implementation support, particularly in the areas of TTA.

Developing a sustainability readiness strategy for health systems: Toolkit, interactive tools, and virtual support system.

While the literature strongly supports the need for sustainability of evidence-based interventions (EBIs), we present a review of the literature that indicates only three articles discuss a health-focused sustainability strategy. The aims of our sustainability readiness strategy (SRS) are to increase infrastructure capacity and EBI advocacy to impact the level of sustainability readiness. In this article, we describe the development of an evidence-informed promising practice sustainability readiness strategy (SRS) with three evidence-based components. This strategy: 1) is based on an adaptation of the Getting To Outcomes® (GTO) evidence-based implementation process, 2) includes a logic model with documented evidence of the connection between targeted readiness factors and sustainability outcomes, and 3) describes resources considered necessary to support implementation of the readiness strategy, namely a step-by-step Toolkit, Excel™ Tools, webinar coaching and evaluation guides, and a coaching and evaluation training guide. The national SRS survey results are presented. Lessons learned and future dissemination and implementation plans are described.

Formative process evaluation for implementing a social marketing intervention to increase walking among African Americans in the Positive Action for Today's Health trial.

OBJECTIVES: Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today's Health trial. METHODS: We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months. RESULTS: Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating. CONCLUSIONS: Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities.

Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement.

An individual or organization that sets out to implement an innovation (e.g., a new technology, program, or policy) generally requires support. In the Interactive Systems Framework for Dissemination and Implementation, a Support System should work with Delivery Systems (national, state and/or local entities such as health and human service organizations, community-based organizations, schools) to enhance their capacity for quality implementation of innovations. The literature on the Support System [corrected] has been underresearched and under-developedThis article begins to conceptualize theory, research, and action for an evidence-based system for innovation support (EBSIS). EBSIS describes key priorities for strengthening the science and practice of support. The major goal of EBSIS is to enhance the research and practice of support in order to build capacity in the Delivery System for implementing innovations with quality, and thereby, help the Delivery System achieve outcomes. EBSIS is guided by a logic model that includes four key support components: tools, training, technical assistance, and quality assurance/quality improvement. EBSIS uses the Getting To Outcomes approach to accountability to aid the identification and synthesis of concepts, tools, and evidence for support. We conclude with some discussion of the current status of EBSIS and possible next steps, including the development of collaborative researcher-practitioner-funder-consumer partnerships to accelerate accumulation of knowledge on the Support System.

The quality implementation framework: a synthesis of critical steps in the implementation process.

Implementation science is growing in importance among funders, researchers, and practitioners as an approach to bridging the gap between science and practice. We addressed three goals to contribute to the understanding of the complex and dynamic nature of implementation. Our first goal was to provide a conceptual overview of the process of implementation by synthesizing information from 25 implementation frameworks. The synthesis extends prior work by focusing on specific actions (i.e., the "how to") that can be employed to foster high quality implementation. The synthesis identified 14 critical steps that were used to construct the Quality Implementation Framework (QIF). These steps comprise four QIF phases: Initial Considerations Regarding the Host Setting, Creating a Structure for Implementation, Ongoing Structure Once Implementation Begins, and Improving Future Applications. Our second goal was to summarize research support for each of the 14 QIF steps and to offer suggestions to direct future research efforts. Our third goal was to outline practical implications of our findings for improving future implementation efforts in the world of practice. The QIF's critical steps can serve as a useful blueprint for future research and practice. Applying the collective guidance synthesized by the QIF to the Interactive Systems Framework for Dissemination and Implementation (ISF) emphasizes that accountability for quality implementation does not rest with the practitioner Delivery System alone. Instead, all three ISF systems are mutually accountable for quality implementation.

Practical implementation science: developing and piloting the quality implementation tool.

According to the Interactive Systems Framework for Dissemination and Implementation, implementation is a major mechanism and concern in bridging research and practice. The growing number of implementation frameworks need to be synthesized and translated so that the science and practice of quality implementation can be furthered. In this article, we: (1) use the synthesis of frameworks developed by Meyers et al. (Am J Commun Psychol, 2012) and translate the results into a practical implementation science tool to use for improving quality of implementation (i.e., the Quality Implementation Tool; QIT), and (2) present some of the benefits and limitations of the tool by describing how the QIT was implemented in two different pilot projects. We discuss how the QIT can be used to guide collaborative planning, monitoring, and evaluation of how an innovation is implemented.

Using a training-of-trainers approach and proactive technical assistance to bring evidence based programs to scale: an operationalization of the interactive systems framework's support system.

Bringing evidence based programs to scale was a major initial impetus for the development of the Interactive Systems Framework for Dissemination and Implementation (ISF). The ISF demonstrates the importance of the Support System in facilitating the uptake of innovations in the community (the Delivery System). Two strategies that members of the Support System commonly use are training-of-trainers (TOT) models and technical assistance (TA). In this article, we focus on the role of the Support System in bringing evidence-based programs (EBPs) to scale in the Delivery System using a case example, with special attention on two strategies employed by Support Systems-training-of-trainers (TOT) and proactive technical assistance. We then report on findings from a case example from the Promoting Science Based Approaches to Teen Pregnancy Prevention project that furthers our conceptualization of these strategies and the evidence base for them. We also report on the limitations in the literature regarding research on TOTs and proactive TA and provide suggestions for future research on TOTs and proactive TA that will enhance the science and practice of support in the ISF.

Advances in bridging research and practice: introduction to the second special issue on the interactive system framework for dissemination and implementation.

The need for new ways to bridge the gap between research and practice is clear; the use of evidence-based prevention programs and implementation with fidelity in practice are strikingly limited. The Interactive Systems Framework for Dissemination and Implementation (ISF) was created to help bridge research and practice by specifying the systems and processes required to support dissemination and implementation of evidence-based programs, processes, practices, and policies. The ISF identifies three key systems necessary for this process which include the Synthesis and Translation System, the Support System, and the Delivery System. The ISF was featured in a special issue of the American Journal of Community Psychology in 2008. This special issue extends that work by including both researchers who have applied an ISF lens to aspects of their current work and researchers who have proactively applied the ISF in a process that goes across the various systems of the ISF, i.e., Synthesis and Translation, Support, and Delivery. Content areas include: children's mental health, teen pregnancy prevention, HIV prevention, violence prevention, heart disease and stroke prevention, breast cancer prevention, and substance abuse prevention. In this introductory article, we provide a brief description of the history of the ISF and a summary of the articles in the special issue.

Developing a prevention synthesis and translation system to promote science-based approaches to teen pregnancy, HIV and STI prevention.

The Interactive Systems Framework for Dissemination and Implementation (ISF) is a multi-system framework that can guide research-to-practice efforts by building and supporting the work of three interacting systems: the Prevention Delivery, Support, and Synthesis and Translation Systems. The Synthesis and Translation system is vital to bridging science and practice, yet how to develop it and train support system partners to use it is under-researched. This article bridges this gap by offering a case example of the planning, development, and use of a synthesis and translation product called Promoting Science-based Approaches to Teen Pregnancy Prevention using Getting To Outcomes. The case presented documents the process used for developing the synthesis and translation product, reports on efforts to engage the Prevention Support system to use the product, and how we approached building interaction between the Synthesis and Translation System and the Support System partners. Practice-oriented evaluation data are also presented. Implications for practice, policy and research are discussed.