Understanding pathways from implementation to sustainment: a longitudinal, mixed methods analysis of promising practices implemented in the Veterans Health Administration.

BACKGROUND: The Veterans Health Administration (VHA) is the United States largest learning health system. The Diffusion of Excellence (DoE) program is a large-scale model of diffusion that identifies and diffuses evidence-informed practices across VHA. During the period of 2016-2021, 57 evidence-informed practices were implemented across 82 VHA facilities. This setting provides a unique opportunity to understand sustainment determinants and pathways. Our objective was to characterize the longitudinal pathways of practices as they transition from initial implementation to long-term sustainment at each facility. METHODS: A longitudinal, mixed-methods evaluation of 82 VHA facilities. Eighty-two facility representatives, chosen by leadership as points-of-contact for 57 DoE practices, were eligible for post-implementation interviews and annual sustainment surveys. Primary outcomes (implementation, sustainment), and secondary outcomes (institutionalization, effectiveness, anticipated sustainment) at four time-points were collected. We performed descriptive statistics and directed content analysis using Hailemariam et al.'s factors influencing sustainment. RESULTS: After approximately five years post-implementation (e.g., 2021 sustainment outcomes), of the 82 facilities, about one-third fully sustained their practice compared to one-third that did not fully sustain their practice because it was in a "liminal" stage (neither sustained nor discontinued) or permanently discontinued. The remaining one-third of facilities had missing 2021 sustainment outcomes. A higher percentage of facilities (70%) had inconsistent primary outcomes (changing over time) compared to facilities (30%) with consistent primary outcomes (same over time). Thirty-four percent of facilities with sustained practices reported resilience since they overcame implementation and sustainment barriers. Facilities with sustained practices reported more positive secondary outcomes compared to those that did not sustain their practice. Key factors facilitating practice sustainment included: demonstrating practice effectiveness/benefit, sufficient organizational leadership, sufficient workforce, and adaptation/alignment with local context. Key factors hindering practice sustainment included: insufficient workforce, not able to maintain practice fidelity/integrity, critical incidents related to the COVID-19 pandemic, organizational leadership did not support sustainment of practice, and no ongoing support. CONCLUSIONS: We identified diverse pathways from implementation to sustainment, and our data underscore that initial implementation outcomes may not determine long-term sustainment outcomes. This longitudinal evaluation contributes to understanding impacts of the DoE program, including return on investment, achieving learning health system goals, and insights into achieving high-quality healthcare in VHA.

Supporting Veteran's Administration Medical Center Directors' Decisions When Adopting Innovative Practices: Development and Implementation of the "QuickView" and "WishList" Tools.

Background Since 2015, the Veterans Health Administration (VHA) Diffusion of Excellence Program has supported spread of practices developed by frontline employees. Shark Tank-style competitions encourage "Sharks" nationwide (VHA medical center/regional directors) to bid for the opportunity to implement practices at their institutions. Methods The authors evaluated bidding strategies (2016-2020), developing the "QuickView" practice comparator to promote informed bidding. Program leaders distributed QuickView and revised versions in subsequent competitions. Our team utilized in-person observation, online chats after the competition, bidder interviews, and bid analysis to evaluate QuickView use. Bids were ranked based on demonstrated understanding of resources required for practice implementation. Results Sharks stated that QuickView supported preparation before the competition and suggested improvements. Our revised tool reported necessary staff time and incorporated a "WishList" from practice finalists detailing minimum requirements for successful implementation. Bids from later years reflected increased review of facilities' current states before the competition and increased understanding of the resources needed for implementation. Percentage of bids describing local need for the practice rose from 2016 to 2020: 4.7% (6/127); 62.1% (54/87); 78.3% (36/46); 80.6% (29/36); 89.7% (26/29). Percentage of bids committing specific resources rose following QuickView introduction: 81.1% (103/127) in 2016, 69.0% (60/87) in 2017, then 73.9% (34/46) in 2018, 88.9% (32/36) in 2019, and 89.7% (26/29) in 2020. Discussion In the years following QuickView/WishList implementation, bids reflected increased assessment before the competition of both local needs and available resources. Conclusion Selection of a new practice for implementation requires an understanding of local need, necessary resources, and fit. QuickView and WishList appear to support these determinations.

A qualitative evaluation of frontline clinician perspectives toward antibiotic stewardship programs.

OBJECTIVE: To examine the perspectives of caregivers that are not part of the antibiotic stewardship program (ASP) leadership team (eg, physicians, nurses, and clinical pharmacists), but who interact with ASPs in their role as frontline healthcare workers. DESIGN: Qualitative semistructured interviews. SETTING: The study was conducted in 2 large national healthcare systems including 7 hospitals in the Veterans' Health Administration and 4 hospitals in Intermountain Healthcare. PARTICIPANTS: We interviewed 157 participants. The current analysis includes 123 nonsteward clinicians: 47 physicians, 26 pharmacists, 29 nurses, and 21 hospital leaders. METHODS: Interviewers utilized a semistructured interview guide based on the Consolidated Framework for Implementation Research (CFIR), which was tailored to the participant's role in the hospital as it related to ASPs. Qualitative analysis was conducted using a codebook based on the CFIR. RESULTS: We identified 4 primary perspectives regarding ASPs. (1) Non-ASP pharmacists considered antibiotic stewardship activities to be a high priority despite the added burden to work duties: (2) Nurses acknowledged limited understanding of ASP activities or involvement with these programs; (3) Physicians criticized ASPs for their restrictions on clinical autonomy and questioned the ability of antibiotic stewards to make recommendations without the full clinical picture; And (4) hospital leaders expressed support for ASPs and recognized the unique challenges faced by non-ASP clinical staff. CONCLUSION: Further understanding these differing perspectives of ASP implementation will inform possible ways to improve ASP implementation across clinical roles.

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool.

BACKGROUND: There are challenges associated with measuring sustainment of evidence-informed practices (EIPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EIP being in use in the future while sustainment assesses the extent to which an EIP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors "Shark Tank" competitions, in which leaders bid on the opportunity to implement a practice with approximately 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and pilot a pragmatic survey tool to assess sustainment of DoE practices. METHODS: In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open-text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open-text box responses. RESULTS: A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1-4. CONCLUSIONS: The initial development and piloting of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold. If additional piloting illustrates the survey is useful, we plan to assess the reliability and validity of this measure for broader use in the field.

Diffusion of excellence: evaluating a system to identify, replicate, and spread promising innovative practices across the Veterans health administration.

Introduction: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program provides a system to identify, replicate, and spread promising practices across the largest integrated healthcare system in the United States. DoE identifies innovations that have been successfully implemented in the VHA through a Shark Tank style competition. VHA facility and regional directors bid resources needed to replicate promising practices. Winning facilities/regions receive external facilitation to aid in replication/implementation over the course of a year. DoE staff then support diffusion of successful practices across the nationwide VHA. Methods: Organized around the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) Framework, we summarize results of an ongoing long-term mixed-methods implementation evaluation of DoE. Data sources include: Shark Tank application and bid details, tracking practice adoptions through a Diffusion Marketplace, characteristics of VHA facilities, focus groups with Shark Tank bidders, structured observations of DoE events, surveys of DoE program participants, and semi-structured interviews of national VHA program office leaders, VHA healthcare system/facility executives, practice developers, implementation teams and facilitators. Results: In the first eight Shark Tanks (2016-2022), 3,280 Shark Tank applications were submitted; 88 were designated DoE Promising Practices (i.e., practices receive facilitated replication). DoE has effectively spread practices across the VHA, with 1,440 documented instances of adoption/replication of practices across the VHA. This includes 180 adoptions/replications in facilities located in rural areas. Leadership decisions to adopt innovations are often based on big picture considerations such as constituency support and linkage to organizational goals. DoE Promising Practices that have the greatest national spread have been successfully replicated at new sites during the facilitated replication process, have close partnerships with VHA national program offices, and tend to be less expensive to implement. Two indicators of sustainment indicate that 56 of the 88 Promising Practices are still being diffused across the VHA; 56% of facilities originally replicating the practices have sustained them, even up to 6 years after the first Shark Tank. Conclusion: DoE has developed a sustainable process for the identification, replication, and spread of promising practices as part of a learning health system committed to providing equitable access to high quality care.

The updated Consolidated Framework for Implementation Research based on user feedback.

BACKGROUND: Many implementation efforts fail, even with highly developed plans for execution, because contextual factors can be powerful forces working against implementation in the real world. The Consolidated Framework for Implementation Research (CFIR) is one of the most commonly used determinant frameworks to assess these contextual factors; however, it has been over 10 years since publication and there is a need for updates. The purpose of this project was to elicit feedback from experienced CFIR users to inform updates to the framework. METHODS: User feedback was obtained from two sources: (1) a literature review with a systematic search; and (2) a survey of authors who used the CFIR in a published study. Data were combined across both sources and reviewed to identify themes; a consensus approach was used to finalize all CFIR updates. The VA Ann Arbor Healthcare System IRB declared this study exempt from the requirements of 38 CFR 16 based on category 2. RESULTS: The systematic search yielded 376 articles that contained the CFIR in the title and/or abstract and 334 unique authors with contact information; 59 articles included feedback on the CFIR. Forty percent (n = 134/334) of authors completed the survey. The CFIR received positive ratings on most framework sensibility items (e.g., applicability, usability), but respondents also provided recommendations for changes. Overall, updates to the CFIR include revisions to existing domains and constructs as well as the addition, removal, or relocation of constructs. These changes address important critiques of the CFIR, including better centering innovation recipients and adding determinants to equity in implementation. CONCLUSION: The updates in the CFIR reflect feedback from a growing community of CFIR users. Although there are many updates, constructs can be mapped back to the original CFIR to ensure longitudinal consistency. We encourage users to continue critiquing the CFIR, facilitating the evolution of the framework as implementation science advances.

Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum.

BACKGROUND: The challenges of implementing evidence-based innovations (EBIs) are widely recognized among practitioners and researchers. Context, broadly defined as everything outside the EBI, includes the dynamic and diverse array of forces working for or against implementation efforts. The Consolidated Framework for Implementation Research (CFIR) is one of the most widely used frameworks to guide assessment of contextual determinants of implementation. The original 2009 article invited critique in recognition for the need for the framework to evolve. As implementation science has matured, gaps in the CFIR have been identified and updates are needed. Our team is developing the CFIR 2.0 based on a literature review and follow-up survey with authors. We propose an Outcomes Addendum to the CFIR to address recommendations from these sources to include outcomes in the framework. MAIN TEXT: We conducted a literature review and surveyed corresponding authors of included articles to identify recommendations for the CFIR. There were recommendations to add both implementation and innovation outcomes from these sources. Based on these recommendations, we make conceptual distinctions between (1) anticipated implementation outcomes and actual implementation outcomes, (2) implementation outcomes and innovation outcomes, and (3) CFIR-based implementation determinants and innovation determinants. CONCLUSION: An Outcomes Addendum to the CFIR is proposed. Our goal is to offer clear conceptual distinctions between types of outcomes for use with the CFIR, and perhaps other determinant implementation frameworks as well. These distinctions can help bring clarity as researchers consider which outcomes are most appropriate to evaluate in their research. We hope that sharing this in advance will generate feedback and debate about the merits of our proposed addendum.

Factors Influencing Fidelity to a Calorie Posting Policy in Public Hospitals: A Mixed Methods Study.

Background: Labelling menus with nutrition information has increasingly become an important obesity policy option. While much research to-date has focused on determining its effectiveness, few studies report the extent to which menu labelling is implemented as designed. The aim of this study was to explore factors influencing fidelity to a calorie posting policy in Irish acute public hospitals. Methods: A mixed methods sequential explanatory study design was employed, with a nested case study for the qualitative component. Quantitative data on implementation fidelity at hospitals were analysed first and informed case sampling in the follow-on qualitative phase. Maximum variation sampling was used to select four hospitals with high and low levels of implementation and variation in terms of geographic location, hospital size, complexity of care provided and hospital type. Data were collected using structured observations, unstructured non-participant observations and in-depth semi-structured interviews. The Consolidated Framework for Implementation Research guided qualitative data collection and analysis. Using framework analysis, factors influencing implementation were identified. A triangulation protocol was used to integrate fidelity findings from multiple sources. Data on influencing factors and fidelity were then combined using joint displays for within and cross-case analysis. Results: Quantitative fidelity data showed seven hospitals were categorised as low implementers and 28 hospitals were high implementers of the policy. Across the four hospitals selected as cases, qualitative analysis revealed factors influencing implementation and fidelity were multiple, and operated independently and in combination. Factors were related to the internal hospital environment (e.g., leadership support, access to knowledge and information, perceived importance of calorie posting implementation), external hospital environment (e.g., national policy, monitoring), features of the calorie posting policy (e.g., availability of supporting materials), and the implementation process (e.g., engaging relevant stakeholders). Integrated analysis of fidelity indicated a pattern of partial adherence to the calorie posting policy across the four hospitals. Across all hospitals, there was a consistent pattern of low adherence to calorie posting across all menu items on sale, low adherence to calorie information displayed per standard portion or per meal, low adherence to standardised recipes/portions, and inaccurate calorie information. Conclusion: Efforts to maximise fidelity require multi-level, multi-component strategies in order to reduce or mitigate barriers and to leverage facilitators. Future research should examine the relative importance of calorie posting determinants and the association between implementation strategies and shifts in fidelity to intervention core components.

Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR).

BACKGROUND: Qualitative approaches, alone or in mixed methods, are prominent within implementation science. However, traditional qualitative approaches are resource intensive, which has led to the development of rapid qualitative approaches. Published rapid approaches are often inductive in nature and rely on transcripts of interviews. We describe a deductive rapid analysis approach using the Consolidated Framework for Implementation Research (CFIR) that uses notes and audio recordings. This paper compares our rapid versus traditional deductive CFIR approach. METHODS: Semi-structured interviews were conducted for two cohorts of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE). The CFIR guided data collection and analysis. In cohort A, we used our traditional CFIR-based deductive analysis approach (directed content analysis), where two analysts completed independent in-depth manual coding of interview transcripts using qualitative software. In cohort B, we used our new rapid CFIR-based deductive analysis approach (directed content analysis), where the primary analyst wrote detailed notes during interviews and immediately "coded" notes into a MS Excel CFIR construct by facility matrix; a secondary analyst then listened to audio recordings and edited the matrix. We tracked time for our traditional and rapid deductive CFIR approaches using a spreadsheet and captured transcription costs from invoices. We retrospectively compared our approaches in terms of effectiveness and rigor. RESULTS: Cohorts A and B were similar in terms of the amount of data collected. However, our rapid deductive CFIR approach required 409.5 analyst hours compared to 683 h during the traditional deductive CFIR approach. The rapid deductive approach eliminated $7250 in transcription costs. The facility-level analysis phase provided the greatest savings: 14 h/facility for the traditional analysis versus 3.92 h/facility for the rapid analysis. Data interpretation required the same number of hours for both approaches. CONCLUSION: Our rapid deductive CFIR approach was less time intensive and eliminated transcription costs, yet effective in meeting evaluation objectives and establishing rigor. Researchers should consider the following when employing our approach: (1) team expertise in the CFIR and qualitative methods, (2) level of detail needed to meet project aims, (3) mode of data to analyze, and (4) advantages and disadvantages of using the CFIR.

Merging Implementation Practice and Science to Scale Up Promising Practices: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) Program.

INTRODUCTION: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program developed and manages a framework for identification, replication, and diffusion of promising practices throughout the nation's largest integrated health care system. DoE identifies promising practices through a "Shark Tank" competition with winning bidders receiving external implementation facilitation. DoE further supports diffusion of successful practices across the VHA. METHODS: This article presents results of a mixed methods implementation evaluation of DoE, focusing on program reach, program participation and decisions to adopt innovative practices, implementation processes, and practice sustainment. Data sources include practice adoption metrics, focus groups with bidders (two focus groups), observations of DoE events (seven events), surveys of stakeholders (five separate surveys), and semistructured interviews of facility directors, practice developers, implementation teams, and facilitators (133 participants). RESULTS: In the first four Shark Tank cohorts (2016-2018), 1,676 practices were submitted; 47 were designated Gold Status Practices (practices with facilitated implementation). Motivation for participation varied. Generally, staff led projects targeting problems they felt passionate about, facility directors focused on big-picture quality metrics and getting middle manager support, and frontline staff displayed variable motivation to implement new projects. Approximately half of facilitated implementation efforts were successful; barriers included insufficient infrastructure, staff, and resources. At the facility level, 73.3% of facilities originating or receiving facilitated implementation support have maintained the practice. VHA-wide, 834 decisions to adopt these practices were made. CONCLUSION: DoE has resulted in the identification of many candidate practices, promoted adoption of promising practices by facility directors, and supported practice implementation and diffusion across the VHA.

Survey of physician and pharmacist steward perceptions of their antibiotic stewardship programs.

Objective: To examine how individual steward characteristics (eg, steward role, sex, and specialized training) are associated with their views of antimicrobial stewardship program (ASP) implementation at their institution. Design: Descriptive survey from a mixed-methods study. Setting: Two large national healthcare systems; the Veterans' Health Administration (VA) (n = 134 hospitals) and Intermountain Healthcare (IHC; n = 20 hospitals). Participants: We sent the survey to 329 antibiotic stewards serving in 154 hospitals; 152 were physicians and 177 were pharmacists. In total, 118 pharmacists and 64 physicians from 126 hospitals responded. Methods: The survey was grounded in constructs of the Consolidated Framework for Implementation Research, and it assessed stewards' views on the development and implementation of antibiotic stewardship programs (ASPs) at their institutions We then examined differences in stewards' views by demographic factors. Results: Regardless of individual factors, stewards agreed that the ASP added value to their institution and was advantageous to patient care. Stewards also reported high levels of collegiality and self-efficacy. Stewards who had specialized training or those volunteered for the role were less likely to think that the ASP was implemented due to a mandate. Similarly volunteers and those with specialized training felt that they had authority in the antibiotic decisions made in their facility. Conclusions: Given the importance of ASPs, it may be beneficial for healthcare institutions to recruit and train individuals with a true interest in stewardship.

Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study.

BACKGROUND: One goal of health systems seeking to evolve into learning health systems is to accelerate the implementation and sustainment of evidence-based practices (EBPs). As part of this evolution, the Veterans Health Administration (VHA) developed the Innovation Ecosystem, which includes the Diffusion of Excellence (DoE), a program that identifies and diffuses Gold Status Practices (GSPs) across facilities. The DoE hosts an annual "Shark Tank" competition in which leaders bid on the opportunity to implement a GSP with 6 months of implementation support. Over 750 diverse practices were submitted in cohorts 2 and 3 of Shark Tank; 23 were designated GSPs and were implemented in 31 VA networks or facilities. As part of a national evaluation of the DoE, we identified factors contributing to GSP implementation and sustainment. METHODS: Our sequential mixed methods evaluation of cohorts 2 and 3 of Shark Tank included semi-structured interviews with at least one representative from 30/31 implementing teams (N = 78/105 people invited) and survey responses from 29/31 teams (N = 39/47 invited). Interviews focused on factors influencing implementation and future sustainment. Surveys focused on sustainment 1.5-2 years after implementation. The Consolidated Framework for Implementation Research (CFIR) informed data collection and directed content analysis. Ordinal scales were developed inductively to rank implementation and sustainment outcomes. RESULTS: Over 50% of teams (17/30) successfully implemented their GSP within the 6-month implementation period. Despite extensive implementation support, significant barriers related to centralized decision-making, staffing, and resources led to partial (n = 6) or no (n = 7) implementation for the remaining teams. While 12/17 initially successful implementation teams reported sustained use of their GSP, over half of the initially unsuccessful teams (n = 7/13) also reported sustained GSP use 1.5 years after the initial implementation period. When asked at 6 months, 18/27 teams with complete data accurately anticipated their future sustainability based on reported sustainment an average of 1.5 years later. CONCLUSIONS: Most teams implemented within 6 months and/or sustained their GSP 1.5 years later. High levels of implementation and sustainment across diverse practices and teams suggest that VHA's DoE is a successful large-scale model of diffusion. Team predictions about sustainability after the first 6 months of implementation provide a promising early assessment and point of intervention to increase sustainability.

Perspectives of Physician and Pharmacist Stewards on Successful Antibiotic Stewardship Program Implementation: A Qualitative Study.

BACKGROUND: Antibiotic stewardship programs (ASPs) are required at every hospital regardless of size. We conducted a qualitative study across different hospital settings to examine perspectives of physician and pharmacist stewards about the dynamics within their team and contextual factors that facilitate the success of their programs. METHODS: Semistructured interviews were conducted in March-November 2018 with 46 ASP stewards, 30 pharmacists, and 16 physicians, from 39 hospitals within 2 large hospital systems. RESULTS: We identified 5 major themes: antibiotic stewards were enthusiastic about their role, committed to the goals of stewardship for their patients and as a public-health imperative, and energized by successful interventions; responsibilities of pharmacist and physician stewards are markedly different, and pharmacy stewards performed the majority of the day-to-day stewardship work; collaborative teamwork is important to improving care, the pharmacists and physicians supported each other, and pharmacists believed that having a strong physician leader was essential; provider engagement strategies are a critical component of stewardship, and recommendations must be communicated in a collegial manner that did not judge the provider competence, preferably through face-to-face interactions; and hospital leadership support for ASP goals and for protected time for ASP activities is critical for success. CONCLUSIONS: The physician-pharmacist team is essential for ASPs; most have pharmacists leading and performing day-to-day activities with physician support. Collaborative, persuasive approaches for ASP interventions were the norm. Stewards were careful not to criticize or judge inappropriate antibiotic prescribing. Further research should examine whether this persuasive approach undercuts provider appreciation of stewardship as a public health mandate.

Barriers and facilitators to implementation of menu labelling interventions from a food service industry perspective: a mixed methods systematic review.

BACKGROUND: Eating outside the home contributes to poor dietary habits worldwide and is associated with increased body fat and weight gain. Evidence shows menu labelling is effective in promoting healthier food choices; however, implementation issues have arisen. The purpose of this systematic review was to synthesise the evidence on the perceived barriers and facilitators to implementation of menu labelling interventions from the perspective of the food service industry. METHODS: Peer-reviewed and grey literature were searched using databases, specialised search engines and public health organisation websites. Screening reference lists, citation chaining and contacting authors of all included studies were undertaken. Primary research studies relevant to direct supply-side stakeholders were eligible for inclusion. There were no restrictions on menu labelling scheme or format, study methods, publication year or language. At least two independent reviewers performed study selection, data extraction and quality appraisal. The results were synthesised using the 'best fit' framework synthesis approach, with reference to the Consolidated Framework for Implementation Research (CFIR). RESULTS: Seventeen studies met the eligibility criteria, with the majority rated as average quality (n = 10). The most frequently cited barriers were coded to the CFIR constructs 'Consumer Needs & Resources' (e.g. lack of customer demand for/interest in menu labelling, risk of overwhelmed/confused customers) and 'Compatibility' with organisation work processes (e.g. lack of standardised recipes, limited space on menus). Frequently cited facilitators were coded to the CFIR constructs 'Relative Advantage' of menu labelling (e.g. improved business image/reputation) and 'Consumer Needs & Resources' (e.g. customer demand for/interest in menu labelling, providing nutrition information to customers). An adapted framework consisting of a priori and new constructs was developed, which illustrates the relationships between domains. CONCLUSION: This review generates an adapted CFIR framework for understanding implementation of menu labelling interventions. It highlights that implementation is influenced by multiple interdependent factors, particularly related to the external and internal context of food businesses, and features of the menu labelling intervention. The findings can be used by researchers and practitioners to develop or select strategies to address barriers that impede implementation and to leverage facilitators that assist with implementation effort. TRIAL REGISTRATION: Systematic review registration: PROSPERO CRD42017083306.

Influence of an Implementation Support Intervention on Barriers and Facilitators to Delivery of a Substance Use Prevention Program.

Implementation support interventions have helped organizations implement programs with quality and obtain intended outcomes. For example, a recent randomized controlled trial called Preparing to Run Effective Programs (PREP) showed that an implementation support intervention called Getting To Outcomes (GTO) improved implementation of an evidence-based substance use prevention program (CHOICE) run in community-based settings. However, more information is needed on how these interventions affect organizational barriers and facilitators of implementation. This paper aims to identify differences in implementation facilitators and barriers in sites conducting a substance use prevention program with and without GTO. PREP is a cluster-randomized controlled trial testing GTO, a two-year implementation support intervention, in Boys & Girls Clubs. The trial compares 15 Boys & Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 Boys & Girls Club sites implementing CHOICE supported by GTO (intervention group). All sites received CHOICE training. Intervention sites also received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE, and then evaluate and make quality improvements to CHOICE implementation using feedback reports summarizing their data. Following the second year of CHOICE and GTO implementation, all sites participated in semi-structured interviews to identify barriers and facilitators to CHOICE implementation using the Consolidated Framework for Implementation Research (CFIR). This paper assesses the extent to which these facilitators and barriers differed between intervention and control group. Intervention sites had significantly higher average ratings than control sites for two constructs from the CFIR process domain: planning and reflecting and evaluating. At the same time, intervention sites had significantly lower ratings on the culture and available resources constructs. Findings suggest that strong planning, evaluation, and reflection-likely improved with GTO support-can facilitate implementation even in the face of perceptions of a less desirable implementation climate. These findings highlight that implementation support, such as GTO, is likely to help low-resourced community-based organizations improve program delivery through a focus on implementation processes. TRIAL REGISTRATION: This project is registered at ClinicalTrials.gov with number NCT02135991 (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was first registered May 12, 2014.

Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration.

BACKGROUND: It is challenging to conduct and quickly disseminate findings from in-depth qualitative analyses, which can impede timely implementation of interventions because of its time-consuming methods. To better understand tradeoffs between the need for actionable results and scientific rigor, we present our method for conducting a framework-guided rapid analysis (RA) and a comparison of these findings to an in-depth analysis of interview transcripts. METHODS: Set within the context of an evaluation of a successful academic detailing (AD) program for opioid prescribing in the Veterans Health Administration, we developed interview guides informed by the Consolidated Framework for Implementation Research (CFIR) and interviewed 10 academic detailers (clinical pharmacists) and 20 primary care providers to elicit detail about successful features of the program. For the RA, verbatim transcripts were summarized using a structured template (based on CFIR); summaries were subsequently consolidated into matrices by participant type to identify aspects of the program that worked well and ways to facilitate implementation elsewhere. For comparison purposes, we later conducted an in-depth analysis of the transcripts. We described our RA approach and qualitatively compared the RA and deductive in-depth analysis with respect to consistency of themes and resource intensity. RESULTS: Integrating the CFIR throughout the RA and in-depth analysis was helpful for providing structure and consistency across both analyses. Findings from the two analyses were consistent. The most frequently coded constructs from the in-depth analysis aligned well with themes from the RA, and the latter methods were sufficient and appropriate for addressing the primary evaluation goals. Our approach to RA was less resource-intensive than the in-depth analysis, allowing for timely dissemination of findings to our operations partner that could be integrated into ongoing implementation. CONCLUSIONS: In-depth analyses can be resource-intensive. If consistent with project needs (e.g., to quickly produce information to inform ongoing implementation or to comply with a policy mandate), it is reasonable to consider using RA, especially when faced with resource constraints. Our RA provided valid findings in a short timeframe, enabling identification of actionable suggestions for our operations partner.

Implementation findings from a hybrid III implementation-effectiveness trial of the Diabetes Prevention Program (DPP) in the Veterans Health Administration (VHA).

BACKGROUND: The Diabetes Prevention Program (DPP) is an effective lifestyle intervention to reduce incidence of type 2 diabetes. However, there are gaps in knowledge about how to implement DPP. The aim of this study was to evaluate implementation of DPP via assessment of a clinical demonstration in the Veterans Health Administration (VHA). METHODS: A 12-month pragmatic clinical trial compared weight outcomes between the Veterans Affairs Diabetes Prevention Program (VA-DPP) and the usual care MOVE!® weight management program (MOVE!). Eligible participants had a body mass index (BMI) ≥30 kg/m2 (or BMI ≥ 25 kg/m2 with one obesity-related condition), prediabetes (glycosylated hemoglobin (HbA1c) 5.7-6.5% or fasting plasma glucose (FPG) 100-125 mg/dL), lived within 60 min of their VA site, and had not participated in a weight management program within the last year. Established evaluation and implementation frameworks were used to guide the implementation evaluation. Implementation barriers and facilitators, delivery fidelity, participant satisfaction, and implementation costs were assessed. Using micro-costing methods, costs for assessment of eligibility and scheduling and maintaining adherence per participant, as well as cost of delivery per session, were also assessed. RESULTS: Several barriers and facilitators to Reach, Adoption, Implementation, Effectiveness and Maintenance were identified; barriers related to Reach were the largest challenge encountered by site teams. Fidelity was higher for VA-DPP delivery compared to MOVE! for five of seven domains assessed. Participant satisfaction was high in both programs, but higher in VA-DPP for most items. Based on micro-costing methods, cost of assessment for eligibility was $68/individual assessed, cost of scheduling and maintaining adherence was $328/participant, and cost of delivery was $101/session. CONCLUSIONS: Multi-faceted strategies are needed to reach targeted participants and successfully implement DPP. Costs for assessing patients for eligibility need to be carefully considered while still maximizing reach to the targeted population.

Implementation evaluation of the Telephone Lifestyle Coaching (TLC) program: organizational factors associated with successful implementation.

The Telephone Lifestyle Coaching (TLC) program provided telephone-based coaching for six lifestyle behaviors to 5321 Veterans at 24 Veterans Health Administration (VHA) medical facilities. The purpose of the study was to conduct an evaluation of the TLC program to identify factors associated with successful implementation. A mixed-methods study design was used. Quantitative measures of organizational readiness for implementation and facility complexity were used to purposively select a subset of facilities for in-depth evaluation. Context assessments were conducted using interview transcripts. The Consolidated Framework for Implementation Research (CFIR) was used to guide qualitative data collection and analysis. Factors most strongly correlated with referral rates included having a skilled implementation leader who used effective multi-component strategies to engage primary care clinicians as well as general clinic structures that supported implementation. Evaluation findings pointed to recommendations for local and national leaders to help anticipate and mitigate potential barriers to successful implementation.

Implementation and evaluation of the VA DPP clinical demonstration: protocol for a multi-site non-randomized hybrid effectiveness-implementation type III trial.

BACKGROUND: The Diabetes Prevention Program (DPP) study showed that lifestyle intervention resulted in a 58% reduction in incidence of type 2 diabetes among individuals with prediabetes. Additional large randomized controlled trials have confirmed these results, and long-term follow-up has shown sustained benefit 10-20 years after the interventions ended. Diabetes is a common and costly disease, especially among Veterans, and despite strong evidence supporting the feasibility of type 2 diabetes prevention, the DPP has not been widely implemented. The first aim of this study will evaluate implementation of the Veterans Affairs (VA) DPP in three VA medical centers. The second aim will assess weight and hemoglobin A1c (A1c) outcomes, and the third aim will determine the cost-effectiveness and budget impact of implementation of the VA DPP from a health system perspective. METHODS/DESIGN: This partnered multi-site non-randomized systematic assignment study will use a highly pragmatic hybrid effectiveness-implementation type III mixed methods study design. The implementation and administration of the VA DPP will be funded by clinical operations while the evaluation of the VA DPP will be funded by research grants. Seven hundred twenty eligible Veterans will be systematically assigned to the VA DPP clinical demonstration or the usual care VA MOVE!® weight management program. A multi-phase formative evaluation of the VA DPP implementation will be conducted. A theoretical program change model will be used to guide the implementation process and assess applicability and feasibility of the DPP for VA. The Consolidated Framework for Implementation Research (CFIR) will be used to guide qualitative data collection, analysis, and interpretation of barriers and facilitators to implementation. The RE-AIM framework will be used to assess Reach, Effectiveness, Adoption, Implementation, and Maintenance of the VA DPP. Twelve-month weight and A1c change will be evaluated for the VA DPP compared to the VA MOVE! PROGRAM: Mediation analyses will be conducted to identify whether program design differences impact outcomes. DISCUSSION: Findings from this pragmatic evaluation will be highly applicable to practitioners who are tasked with implementing the DPP in clinical settings. In addition, findings will determine the effectiveness and cost-effectiveness of the VA DPP in the Veteran population.