A Mixed Methods Study Exploring Patient Safety Culture at Four VHA Hospitals.

BACKGROUND: Patient safety culture (PSC) fosters an environment of trust where people are encouraged to share information to promote psychological safety. To measure PSC, the Veteran's Health Administration (VHA) developed a PSC survey consisting of 20 items administered to all VHA employees. The survey comprises four scales: (1) risk identification and Just Culture, (2) error transparency and mitigation, (3) supervisor communication and trust, and (4) team cohesion and engagement. Our objective was to compare the PSC survey data to qualitative data regarding high reliability organization (HRO) implementation from four purposively selected VHA hospitals to assess how it manifests and converges. METHODS: Qualitative data focused on understanding HRO implementation efforts were collected from key informants between 2019 and 2020 at 4 of the 18 VHA HRO implementation hospitals. To explore the extent and manifestation of each of the PSC scales among the 4 sites, we combined the qualitative data with the PSC survey data from each hospital using a joint display. RESULTS: Survey responses were significantly different between the 4 hospitals for all 4 PSC scales. Of the 20 PSC survey items, 12 (60.0%) significantly differed across the 4 hospitals. For example, we saw cross-hospital differences in the following survey items: "We are given feedback about changes put into place based on event reports" and "We take the time to identify and assess risks to patient safety." Qualitative data supported manifestations for 80.0% (16/20) of PSC individual survey items among hospitals. CONCLUSION: The authors found that the qualitative data manifestations were well aligned with the VHA PSC scales, but relationships were not always consistent between data sources. Further research is necessary to elucidate these relationships.

Improving access to first-line treatment for pediatric obesity: Lessons from the dissemination of SmartMoves.

OBJECTIVE: The increasing prevalence of and inequities in childhood obesity demand improved access to effective treatment. The SmartMoves curriculum used in Bright Bodies, a proven-effective, intensive health behavior and lifestyle treatment (IHBLT), was disseminated to ≥30 US sites from 2003 to 2018. We aimed to identify barriers to and facilitators of IHBLT implementation/sustainment. METHODS: We surveyed and interviewed key informants about experiences acquiring/implementing SmartMoves. In parallel, we analyzed and then integrated survey findings and themes from interviews using the constant comparative method. RESULTS: Participants from 16 sites (53%) completed surveys, and 12 participants at 10 sites completed interviews. The 11 sites (63%) that implemented SmartMoves varied in both use of training opportunities/materials and fidelity to program components. In interviews, demand for obesity programming, organizational priorities, and partnerships facilitated implementation. Seven sites discontinued SmartMoves prior to the COVID-19 pandemic. Funding insecurity and insufficient staffing emerged as dominant barriers to implementation/sustainment discussed by all interviewees, and some also noted participants' competing demands and the program's fit with population as challenges. CONCLUSIONS: System- and organizational-level barriers impeded sustainment of an evidence-based IHBLT program. Adequate funding could enable sufficient staffing and training to promote fidelity to the intervention's core functions and adaptation to fit local populations/context.

Effectiveness of a virtual quality improvement training program to improve reach of weight management programs within a large health system.

OBJECTIVE: To test effectiveness of the LEAP (Learn Engage Act Process) Program on engaging frontline Veteran Health Administration (VHA) medical center teams in continuous quality improvement (QI), a core capability for learning health systems. DATA SOURCES AND STUDY SETTING: Data sources included VHA electronic health record (EHR) data, surveys, and LEAP coaching field notes. STUDY DESIGN: A staggered difference-in-differences study was conducted. Fifty-five facilities participated in LEAP across eight randomly assigned clusters of 6-8 facilities per cluster over 2 years. Non-participating facilities were used as controls. A MOVE! weight management program team completed a Plan-Do-Study-Act cycle of change supported by learning curriculum, coaching, and virtual collaboratives in LEAP facilities. Primary outcome was program reach to Veterans. A mixed-effects model compared pre- versus post-LEAP periods for LEAP versus control facilities. LEAP adherence, satisfaction, and cost to deliver LEAP were evaluated. DATA COLLECTION/EXTRACTION METHODS: Thirty months of facility-level EHR MOVE! enrollment data were included in analyses. LEAP Satisfaction and QI skills were elicited via surveys at baseline and 6-month post-LEAP. PRINCIPAL FINDINGS: Fifty-five facilities were randomly assigned to eight time-period-based clusters to receive LEAP (71% completed LEAP) and 82 non-participating facilities were randomly assigned as controls. Reach in LEAP and control facilities was comparable in the 12-month pre-LEAP period (p = 0.07). Though LEAP facilities experienced slower decline in reach in the 12-month post-LEAP period compared with controls (p < 0.001), this is likely due to unexplained fluctuations in controls. For LEAP facilities, satisfaction was high (all mean ratings >4 on a 5-point scale), self-reported use of QI methods increased significantly (p-values <0.05) 6 months post-LEAP, and delivery cost was $4024 per facility-based team. CONCLUSION: Control facilities experienced declining reach in the 12-month post-LEAP period, but LEAP facilities did not, plus they reported higher engagement in QI, an essential capability for learning health systems.

Real-world impacts from a decade of Quality Enhancement Research Initiative-partnered projects to translate the Diabetes Prevention Program in the Veterans Health Administration.

OBJECTIVES: To describe the impacts of four Veterans Health Administration (VA) Quality Enhancement Research Initiative (QUERI) projects implementing an evidence-based lifestyle intervention known as the Diabetes Prevention Program (DPP). DATA SOURCES AND STUDY SETTING: 2012-2024 VA administrative and survey data. STUDY DESIGN: This is a summary of findings and impacts from four effectiveness-implementation projects focused on in-person and/or online DPP across VA sites. DATA COLLECTION/EXTRACTION METHODS: Patient demographics, participation data, and key findings and impacts were summarized across reports from the VA Diabetes-Mellitus Quality Enhancement Research Initiative (QUERI-DM) Diabetes Prevention Program (VA DPP) Trial, QUERI-DM Online DPP Trial, the Enhancing Mental and Physical Health of Women through Engagement and Retention (EMPOWER) QUERI DPP Project, and EMPOWER 2.0 QUERI Program. PRINCIPAL FINDINGS: Between 2012 and 2024, four VA QUERI studies enrolled 963 Veterans in DPP across 16 VA sites. All participants had overweight/obesity with one additional risk factor for type 2 diabetes (i.e., prediabetes, elevated risk score, or history of gestational diabetes) and 56% (N = 536) were women. In addition to enhancing the reach of and engagement in diabetes prevention services among Veterans, these projects resulted in three key impacts as follows: (1) informing the national redesign of VA MOVE! including recommendations to increase the number of MOVE! sessions and revise guidelines across 150+ VA sites, (2) enhancing the national evidence base to support online DPP delivery options with citations in national care guidelines outside VA, and (3) demonstrating the importance of gender-tailoring of preventive care services by and for women Veterans to enhance engagement in preventive services. CONCLUSIONS: Over the past decade, the evolution of VA QUERI DPP projects increased the reach of and engagement in diabetes prevention services among Veterans, including women Veterans who have been harder to engage in lifestyle change programs in VA, and resulted in three key impacts informing type 2 diabetes and obesity prevention efforts within and outside of VA.

User-Centered Framework for Implementation of Technology (UFIT): Development of an Integrated Framework for Designing Clinical Decision Support Tools Packaged With Tailored Implementation Strategies.

BACKGROUND: Electronic health record-based clinical decision support (CDS) tools can facilitate the adoption of evidence into practice. Yet, the impact of CDS beyond single-site implementation is often limited by dissemination and implementation barriers related to site- and user-specific variation in workflows and behaviors. The translation of evidence-based CDS from initial development to implementation in heterogeneous environments requires a framework that assures careful balancing of fidelity to core functional elements with adaptations to ensure compatibility with new contexts. OBJECTIVE: This study aims to develop and apply a framework to guide tailoring and implementing CDS across diverse clinical settings. METHODS: In preparation for a multisite trial implementing CDS for pediatric overweight or obesity in primary care, we developed the User-Centered Framework for Implementation of Technology (UFIT), a framework that integrates principles from user-centered design (UCD), human factors/ergonomics theories, and implementation science to guide both CDS adaptation and tailoring of related implementation strategies. Our transdisciplinary study team conducted semistructured interviews with pediatric primary care clinicians and a diverse group of stakeholders from 3 health systems in the northeastern, midwestern, and southeastern United States to inform and apply the framework for our formative evaluation. RESULTS: We conducted 41 qualitative interviews with primary care clinicians (n=21) and other stakeholders (n=20). Our workflow analysis found 3 primary ways in which clinicians interact with the electronic health record during primary care well-child visits identifying opportunities for decision support. Additionally, we identified differences in practice patterns across contexts necessitating a multiprong design approach to support a variety of workflows, user needs, preferences, and implementation strategies. CONCLUSIONS: UFIT integrates theories and guidance from UCD, human factors/ergonomics, and implementation science to promote fit with local contexts for optimal outcomes. The components of UFIT were used to guide the development of Improving Pediatric Obesity Practice Using Prompts, an integrated package comprising CDS for obesity or overweight treatment with tailored implementation strategies. TRIAL REGISTRATION: ClinicalTrials.gov NCT05627011; https://clinicaltrials.gov/study/NCT05627011.

Understanding pathways from implementation to sustainment: a longitudinal, mixed methods analysis of promising practices implemented in the Veterans Health Administration.

BACKGROUND: The Veterans Health Administration (VHA) is the United States largest learning health system. The Diffusion of Excellence (DoE) program is a large-scale model of diffusion that identifies and diffuses evidence-informed practices across VHA. During the period of 2016-2021, 57 evidence-informed practices were implemented across 82 VHA facilities. This setting provides a unique opportunity to understand sustainment determinants and pathways. Our objective was to characterize the longitudinal pathways of practices as they transition from initial implementation to long-term sustainment at each facility. METHODS: A longitudinal, mixed-methods evaluation of 82 VHA facilities. Eighty-two facility representatives, chosen by leadership as points-of-contact for 57 DoE practices, were eligible for post-implementation interviews and annual sustainment surveys. Primary outcomes (implementation, sustainment), and secondary outcomes (institutionalization, effectiveness, anticipated sustainment) at four time-points were collected. We performed descriptive statistics and directed content analysis using Hailemariam et al.'s factors influencing sustainment. RESULTS: After approximately five years post-implementation (e.g., 2021 sustainment outcomes), of the 82 facilities, about one-third fully sustained their practice compared to one-third that did not fully sustain their practice because it was in a "liminal" stage (neither sustained nor discontinued) or permanently discontinued. The remaining one-third of facilities had missing 2021 sustainment outcomes. A higher percentage of facilities (70%) had inconsistent primary outcomes (changing over time) compared to facilities (30%) with consistent primary outcomes (same over time). Thirty-four percent of facilities with sustained practices reported resilience since they overcame implementation and sustainment barriers. Facilities with sustained practices reported more positive secondary outcomes compared to those that did not sustain their practice. Key factors facilitating practice sustainment included: demonstrating practice effectiveness/benefit, sufficient organizational leadership, sufficient workforce, and adaptation/alignment with local context. Key factors hindering practice sustainment included: insufficient workforce, not able to maintain practice fidelity/integrity, critical incidents related to the COVID-19 pandemic, organizational leadership did not support sustainment of practice, and no ongoing support. CONCLUSIONS: We identified diverse pathways from implementation to sustainment, and our data underscore that initial implementation outcomes may not determine long-term sustainment outcomes. This longitudinal evaluation contributes to understanding impacts of the DoE program, including return on investment, achieving learning health system goals, and insights into achieving high-quality healthcare in VHA.

Supporting Veteran's Administration Medical Center Directors' Decisions When Adopting Innovative Practices: Development and Implementation of the "QuickView" and "WishList" Tools.

Background Since 2015, the Veterans Health Administration (VHA) Diffusion of Excellence Program has supported spread of practices developed by frontline employees. Shark Tank-style competitions encourage "Sharks" nationwide (VHA medical center/regional directors) to bid for the opportunity to implement practices at their institutions. Methods The authors evaluated bidding strategies (2016-2020), developing the "QuickView" practice comparator to promote informed bidding. Program leaders distributed QuickView and revised versions in subsequent competitions. Our team utilized in-person observation, online chats after the competition, bidder interviews, and bid analysis to evaluate QuickView use. Bids were ranked based on demonstrated understanding of resources required for practice implementation. Results Sharks stated that QuickView supported preparation before the competition and suggested improvements. Our revised tool reported necessary staff time and incorporated a "WishList" from practice finalists detailing minimum requirements for successful implementation. Bids from later years reflected increased review of facilities' current states before the competition and increased understanding of the resources needed for implementation. Percentage of bids describing local need for the practice rose from 2016 to 2020: 4.7% (6/127); 62.1% (54/87); 78.3% (36/46); 80.6% (29/36); 89.7% (26/29). Percentage of bids committing specific resources rose following QuickView introduction: 81.1% (103/127) in 2016, 69.0% (60/87) in 2017, then 73.9% (34/46) in 2018, 88.9% (32/36) in 2019, and 89.7% (26/29) in 2020. Discussion In the years following QuickView/WishList implementation, bids reflected increased assessment before the competition of both local needs and available resources. Conclusion Selection of a new practice for implementation requires an understanding of local need, necessary resources, and fit. QuickView and WishList appear to support these determinations.

Exploring implementation outcomes in the clinical trial context: a qualitative study of physician trial stakeholders.

INTRODUCTION: Cancer clinical trials can be considered evidence-based interventions with substantial benefits, but suffer from poor implementation leading to low enrollment and frequent failure. Applying implementation science approaches such as outcomes frameworks to the trial context could aid in contextualizing and evaluating trial improvement strategies. However, the acceptability and appropriateness of these adapted outcomes to trial stakeholders are unclear. For these reasons, we interviewed cancer clinical trial physician stakeholders to explore how they perceive and address clinical trial implementation outcomes. METHODS: We purposively selected 15 cancer clinical trial physician stakeholders from our institution representing different specialties, trial roles, and trial sponsor types. We performed semi-structured interviews to explore a previous adaptation of Proctor's Implementation Outcomes Framework to the clinical trial context. Emergent themes from each outcome were developed. RESULTS: The implementation outcomes were well understood and applicable (i.e., appropriate and acceptable) to clinical trial stakeholders. We describe cancer clinical trial physician stakeholder understanding of these outcomes and current application of these concepts. Trial feasibility and implementation cost were felt to be most critical to trial design and implementation. Trial penetration was most difficult to measure, primarily due to eligible patient identification. In general, we found that formal methods for trial improvement and trial implementation evaluation were poorly developed. Cancer clinical trial physician stakeholders referred to some design and implementation techniques used to improve trials, but these were infrequently formally evaluated or theory-based. CONCLUSION: Implementation outcomes adapted to the trial context were acceptable and appropriate to cancer clinical trial physician stakeholders. Use of these outcomes could facilitate the evaluation and design of clinical trial improvement interventions. Additionally, these outcomes highlight potential areas for the development of new tools, for example informatics solutions, to improve the evaluation and implementation of clinical trials.

A qualitative evaluation of frontline clinician perspectives toward antibiotic stewardship programs.

OBJECTIVE: To examine the perspectives of caregivers that are not part of the antibiotic stewardship program (ASP) leadership team (eg, physicians, nurses, and clinical pharmacists), but who interact with ASPs in their role as frontline healthcare workers. DESIGN: Qualitative semistructured interviews. SETTING: The study was conducted in 2 large national healthcare systems including 7 hospitals in the Veterans' Health Administration and 4 hospitals in Intermountain Healthcare. PARTICIPANTS: We interviewed 157 participants. The current analysis includes 123 nonsteward clinicians: 47 physicians, 26 pharmacists, 29 nurses, and 21 hospital leaders. METHODS: Interviewers utilized a semistructured interview guide based on the Consolidated Framework for Implementation Research (CFIR), which was tailored to the participant's role in the hospital as it related to ASPs. Qualitative analysis was conducted using a codebook based on the CFIR. RESULTS: We identified 4 primary perspectives regarding ASPs. (1) Non-ASP pharmacists considered antibiotic stewardship activities to be a high priority despite the added burden to work duties: (2) Nurses acknowledged limited understanding of ASP activities or involvement with these programs; (3) Physicians criticized ASPs for their restrictions on clinical autonomy and questioned the ability of antibiotic stewards to make recommendations without the full clinical picture; And (4) hospital leaders expressed support for ASPs and recognized the unique challenges faced by non-ASP clinical staff. CONCLUSION: Further understanding these differing perspectives of ASP implementation will inform possible ways to improve ASP implementation across clinical roles.

A pragmatic context assessment tool (pCAT): using a Think Aloud method to develop an assessment of contextual barriers to change.

BACKGROUND: The Consolidated Framework for Implementation Research (CFIR) is a determinant framework that can be used to guide context assessment prior to implementing change. Though a few quantitative measurement instruments have been developed based on the CFIR, most assessments using the CFIR have relied on qualitative methods. One challenge to measurement is to translate conceptual constructs which are often described using highly abstract, technical language into lay language that is clear, concise, and meaningful. The purpose of this paper is to document methods to develop a freely available pragmatic context assessment tool (pCAT). The pCAT is based on the CFIR and designed for frontline quality improvement teams as an abbreviated assessment of local facilitators and barriers in a clinical setting. METHODS: Twenty-seven interviews using the Think Aloud method (asking participants to verbalize thoughts as they respond to assessment questions) were conducted with frontline employees to improve a pilot version of the pCAT. Interviews were recorded and transcribed verbatim; the CFIR guided coding and analyses. RESULTS: Participants identified several areas where language in the pCAT needed to be modified, clarified, or allow more nuance to increase usefulness for frontline employees. Participants found it easier to respond to questions when they had a recent, specific project in mind. Potential barriers and facilitators tend to be unique to each specific improvement. Participants also identified missing concepts or that were conflated, leading to refinements that made the pCAT more understandable, accurate, and useful. CONCLUSIONS: The pCAT is designed to be practical, using everyday language familiar to frontline employees. The pCAT is short (14 items), freely available, does not require research expertise or experience. It is designed to draw on the knowledge of individuals most familiar with their own clinical context. The pCAT has been available online for approximately two years and has generated a relatively high level of interest indicating potential usefulness of the tool.

Diffusion of excellence: evaluating a system to identify, replicate, and spread promising innovative practices across the Veterans health administration.

Introduction: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program provides a system to identify, replicate, and spread promising practices across the largest integrated healthcare system in the United States. DoE identifies innovations that have been successfully implemented in the VHA through a Shark Tank style competition. VHA facility and regional directors bid resources needed to replicate promising practices. Winning facilities/regions receive external facilitation to aid in replication/implementation over the course of a year. DoE staff then support diffusion of successful practices across the nationwide VHA. Methods: Organized around the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) Framework, we summarize results of an ongoing long-term mixed-methods implementation evaluation of DoE. Data sources include: Shark Tank application and bid details, tracking practice adoptions through a Diffusion Marketplace, characteristics of VHA facilities, focus groups with Shark Tank bidders, structured observations of DoE events, surveys of DoE program participants, and semi-structured interviews of national VHA program office leaders, VHA healthcare system/facility executives, practice developers, implementation teams and facilitators. Results: In the first eight Shark Tanks (2016-2022), 3,280 Shark Tank applications were submitted; 88 were designated DoE Promising Practices (i.e., practices receive facilitated replication). DoE has effectively spread practices across the VHA, with 1,440 documented instances of adoption/replication of practices across the VHA. This includes 180 adoptions/replications in facilities located in rural areas. Leadership decisions to adopt innovations are often based on big picture considerations such as constituency support and linkage to organizational goals. DoE Promising Practices that have the greatest national spread have been successfully replicated at new sites during the facilitated replication process, have close partnerships with VHA national program offices, and tend to be less expensive to implement. Two indicators of sustainment indicate that 56 of the 88 Promising Practices are still being diffused across the VHA; 56% of facilities originally replicating the practices have sustained them, even up to 6 years after the first Shark Tank. Conclusion: DoE has developed a sustainable process for the identification, replication, and spread of promising practices as part of a learning health system committed to providing equitable access to high quality care.

Facility-level program components leading to population impact: a coincidence analysis of obesity treatment options within the Veterans Health Administration.

Obesity is a well-established risk factor for increased morbidity and mortality. Comprehensive lifestyle interventions, pharmacotherapy, and bariatric surgery are three effective treatment approaches for obesity. The Veterans Health Administration (VHA) offers all three domains but in different configurations across medical facilities. Study aim was to explore the relationship between configurations of three types of obesity treatments, context, and population impact across VHA using coincidence analysis. This was a cross-sectional analysis of survey data describing weight management treatment components linked with administrative data to compute population impact for each facility. Coincidence analysis was used to identify combinations of treatment components that led to higher population impact. Facilities with higher impact were in the top two quintiles for (1) reach to eligible patients and (2) weight outcomes. Sixty-nine facilities were included in the analyses. The final model explained 88% (29/33) of the higher-impact facilities with 91% consistency (29/32) and was comprised of five distinct pathways. Each of the five pathways depended on facility complexity-level plus factors from one or more of the three domains of weight management: comprehensive lifestyle interventions, pharmacotherapy, and/or bariatric surgery. Three pathways include components from multiple treatment domains. Combinations of conditions formed "recipes" that lead to higher population impact. Our coincidence analyses highlighted both the importance of local context and how combinations of specific conditions consistently and uniquely distinguished higher impact facilities from lower impact facilities for weight management.

Obesity can contribute to increased rates of ill health and earlier death. Proven treatments for obesity include programs that help people improve lifestyle behaviors (e.g., being physically active), medications, and/or bariatric surgery. In the Veterans Health Administration (VHA), all three types of treatments are offered, but not at every medical center­in practice, individual medical centers offer different combinations of treatment options to their patients. VHA medical centers also have a wide range of population impact. We identified high-impact medical centers (centers with the most patients participating in obesity treatment who would benefit from treatment AND that reported the most weight loss for their patients) and examined which treatment configurations led to better population-level outcomes (i.e., higher population impact). We used a novel analysis approach that allows us to compare combinations of treatment components, instead of analyzing them one-by-one. We found that optimal combinations are context-sensitive and depend on the type of center (e.g., large centers affiliated with a university vs. smaller rural centers). We list five different "recipes" of treatment combinations leading to higher population-level impact. This information can be used by clinical leaders to design treatment programs to maximize benefits for their patients.

The updated Consolidated Framework for Implementation Research based on user feedback.

BACKGROUND: Many implementation efforts fail, even with highly developed plans for execution, because contextual factors can be powerful forces working against implementation in the real world. The Consolidated Framework for Implementation Research (CFIR) is one of the most commonly used determinant frameworks to assess these contextual factors; however, it has been over 10 years since publication and there is a need for updates. The purpose of this project was to elicit feedback from experienced CFIR users to inform updates to the framework. METHODS: User feedback was obtained from two sources: (1) a literature review with a systematic search; and (2) a survey of authors who used the CFIR in a published study. Data were combined across both sources and reviewed to identify themes; a consensus approach was used to finalize all CFIR updates. The VA Ann Arbor Healthcare System IRB declared this study exempt from the requirements of 38 CFR 16 based on category 2. RESULTS: The systematic search yielded 376 articles that contained the CFIR in the title and/or abstract and 334 unique authors with contact information; 59 articles included feedback on the CFIR. Forty percent (n = 134/334) of authors completed the survey. The CFIR received positive ratings on most framework sensibility items (e.g., applicability, usability), but respondents also provided recommendations for changes. Overall, updates to the CFIR include revisions to existing domains and constructs as well as the addition, removal, or relocation of constructs. These changes address important critiques of the CFIR, including better centering innovation recipients and adding determinants to equity in implementation. CONCLUSION: The updates in the CFIR reflect feedback from a growing community of CFIR users. Although there are many updates, constructs can be mapped back to the original CFIR to ensure longitudinal consistency. We encourage users to continue critiquing the CFIR, facilitating the evolution of the framework as implementation science advances.

Applying implementation frameworks to the clinical trial context.

BACKGROUND: Clinical trials advance science, benefit society, and provide optimal care to individuals with some conditions, such as cancer. However, clinical trials often fail to reach their endpoints, and low participant enrollment remains a critical problem with trial conduct. In these ways, clinical trials can be considered beneficial evidence-based practices suffering from poor implementation. Prior approaches to improving trials have had difficulties with reproducibility and limited impact, perhaps due to the lack of an underlying trial improvement framework. For these reasons, we propose adapting implementation science frameworks to the clinical trial context to improve the implementation of clinical trials. MAIN TEXT: We adapted an outcomes framework (Proctor's Implementation Outcomes Framework) and a determinants framework (the Consolidated Framework for Implementation Research) to the trial context. We linked these frameworks to ERIC-based improvement strategies and present an inferential process model for identifying and selecting trial improvement strategies based on the Implementation Research Logic Model. We describe example applications of the framework components to the trial context and present a worked example of our model applied to a trial with poor enrollment. We then consider the implications of this approach on improving existing trials, the design of future trials, and assessing trial improvement interventions. Additionally, we consider the use of implementation science in the clinical trial context, and how clinical trials can be "test cases" for implementation research. CONCLUSIONS: Clinical trials can be considered beneficial evidence-based interventions suffering from poor implementation. Adapting implementation science approaches to the clinical trial context can provide frameworks for contextual assessment, outcome measurement, targeted interventions, and a shared vocabulary for clinical trial improvement. Additionally, exploring implementation frameworks in the trial context can advance the science of implementation through both "test cases" and providing fertile ground for implementation intervention design and testing.

Continuous quality improvement at the frontline: One interdisciplinary clinical team's four-year journey after completing a virtual learning program.

Background: The Veterans Health Administration (VHA) is the largest integrated health system in the U.S. and has identified the learning health system as a strategic priority. Clinicians and staff engaging in active learning through continuous quality improvement (QI) is a key pillar for learning system maturity. An interdisciplinary frontline team at a VHA medical center participated in the Learn. Engage. Act. Process. (LEAP) virtual coaching program to learn how to conduct multidisciplinary team-based QI cycles of change. These clinicians lead and deliver the MOVE! weight management program, an evidence-based comprehensive lifestyle intervention. The team worked to continuously improve patient weight loss by engaging in incremental learning cycles of change. The aim of this study is to tell the story of this team's learning experience and the resulting positive reinforcing loop with patient outcomes. Methods: This is a mixed methods case study description of one team that participated in the LEAP Program that provides hands-on QI learning for frontline teams with virtual coaching and a structured curriculum. Autoethnographic qualitative descriptions of team experiences over time illustrate this team's continued engagement in learning loops. Multilevel linear modeling was used to assess patient outcomes before vs after the team's participation in LEAP. Results: The team's participation in LEAP provided a set of fundamental QI skills and established a commitment to continual learning. Incremental improvements led to significant weight loss for patients who participated in MOVE! after the team completed LEAP (mean = 9.80 pounds; SD 10.43) compared to the pre-LEAP time period (mean = -6.83 pounds; SD 9.63). Conclusions: Despite competing priorities and time limitations, this team's experiences provide a positive vision of how team engagement in data-driven continuous learning is feasible at the frontline and can lead to higher job satisfaction and stronger teams. These types of team activities provide much-needed backbone to being a mature learning health system.

Planning and Reporting Effective Web-Based RAND/UCLA Appropriateness Method Panels: Literature Review and Preliminary Recommendations.

BACKGROUND: The RAND/UCLA Appropriateness Method (RAM), a variant of the Delphi Method, was developed to synthesize existing evidence and elicit the clinical judgement of medical experts on the appropriate treatment of specific clinical presentations. Technological advances now allow researchers to conduct expert panels on the internet, offering a cost-effective and convenient alternative to the traditional RAM. For example, the Department of Veterans Affairs recently used a web-based RAM to validate clinical recommendations for de-intensifying routine primary care services. A substantial literature describes and tests various aspects of the traditional RAM in health research; yet we know comparatively less about how researchers implement web-based expert panels. OBJECTIVE: The objectives of this study are twofold: (1) to understand how the web-based RAM process is currently used and reported in health research and (2) to provide preliminary reporting guidance for researchers to improve the transparency and reproducibility of reporting practices. METHODS: The PubMed database was searched to identify studies published between 2009 and 2019 that used a web-based RAM to measure the appropriateness of medical care. Methodological data from each article were abstracted. The following categories were assessed: composition and characteristics of the web-based expert panels, characteristics of panel procedures, results, and panel satisfaction and engagement. RESULTS: Of the 12 studies meeting the eligibility criteria and reviewed, only 42% (5/12) implemented the full RAM process with the remaining studies opting for a partial approach. Among those studies reporting, the median number of participants at first rating was 42. While 92% (11/12) of studies involved clinicians, 50% (6/12) involved multiple stakeholder types. Our review revealed that the studies failed to report on critical aspects of the RAM process. For example, no studies reported response rates with the denominator of previous rounds, 42% (5/12) did not provide panelists with feedback between rating periods, 50% (6/12) either did not have or did not report on the panel discussion period, and 25% (3/12) did not report on quality measures to assess aspects of the panel process (eg, satisfaction with the process). CONCLUSIONS: Conducting web-based RAM panels will continue to be an appealing option for researchers seeking a safe, efficient, and democratic process of expert agreement. Our literature review uncovered inconsistent reporting frameworks and insufficient detail to evaluate study outcomes. We provide preliminary recommendations for reporting that are both timely and important for producing replicable, high-quality findings. The need for reporting standards is especially critical given that more people may prefer to participate in web-based rather than in-person panels due to the ongoing COVID-19 pandemic.

Envisioning clinical trials as complex interventions.

Clinical trials are critical components of modern health care and infrastructure. Trials benefit society through scientific advancement and individual patients through trial participation. In fact, billions of dollars are spent annually in support of these benefits. Despite the massive investments, clinical trials often fail to accomplish their primary aims and trial enrollment rates remain low. Prior efforts to improve trial conduct and enrollment have had limited success, perhaps due to oversimplification of the complex, multilevel nature of trials. For these reasons, the authors propose applying implementation science to the clinical trials context. In this commentary, the authors posit clinical trials as complex, multilevel evidence-based interventions with significant societal and individual benefits yet with persistent gaps in implementation. An application of implementation science concepts to the clinical trials context as means to build common vocabulary and establish a platform for applying implementation science and practice to improve clinical trial conduct is introduced. Applying implementation science to the clinical trials context can augment improvement efforts and build capacity for better and more efficient evidence-based care for all patients and trial stakeholders throughout the clinical trials enterprise.

Maintaining Implementation through Dynamic Adaptations (MIDAS): protocol for a cluster-randomized trial of implementation strategies to optimize and sustain use of evidence-based practices in Veteran Health Administration (VHA) patients.

BACKGROUND: The adoption and sustainment of evidence-based practices (EBPs) is a challenge within many healthcare systems, especially in settings that have already strived but failed to achieve longer-term goals. The Veterans Affairs (VA) Maintaining Implementation through Dynamic Adaptations (MIDAS) Quality Enhancement Research Initiative (QUERI) program was funded as a series of trials to test multi-component implementation strategies to sustain optimal use of three EBPs: (1) a deprescribing approach intended to reduce potentially inappropriate polypharmacy; (2) appropriate dosing and drug selection of direct oral anticoagulants (DOACs); and (3) use of cognitive behavioral therapy as first-line treatment for insomnia before pharmacologic treatment. We describe the design and methods for a harmonized series of cluster-randomized control trials comparing two implementation strategies. METHODS: For each trial, we will recruit 8-12 clinics (24-36 total). All will have access to relevant clinical data to identify patients who may benefit from the target EBP at that clinic and provider. For each trial, clinics will be randomized to one of two implementation strategies to improve the use of the EBPs: (1) individual-level academic detailing (AD) or (2) AD plus the team-based Learn. Engage. Act. PROCESS: (LEAP) quality improvement (QI) learning program. The primary outcomes will be operationalized across the three trials as a patient-level dichotomous response (yes/no) indicating patients with potentially inappropriate medications (PIMs) among those who may benefit from the EBP. This outcome will be computed using month-by-month administrative data. Primary comparison between the two implementation strategies will be analyzed using generalized estimating equations (GEE) with clinic-level monthly (13 to 36 months) percent of PIMs as the dependent variable. Primary comparative endpoint will be at 18 months post-baseline. Each trial will also be analyzed independently. DISCUSSION: MIDAS QUERI trials will focus on fostering sustained use of EBPs that previously had targeted but incomplete implementation. Our implementation approaches are designed to engage frontline clinicians in a dynamic optimization process that integrates the use of actional clinical data and making incremental changes, designed to be feasible within busy clinical settings. TRIAL REGISTRATION: ClinicalTrials.gov: NCT05065502 . Registered October 4, 2021-retrospectively registered.

Balancing reality in embedded research and evaluation: Low vs high embeddedness.

Embedding research and evaluation into organizations is one way to generate "practice-based" evidence needed to accelerate implementation of evidence-based innovations within learning health systems. Organizations and researchers/evaluators vary greatly in how they structure and operationalize these collaborations. One key aspect is the degree of embeddedness: from low embeddedness where researchers/evaluators are located outside organizations (eg, outside evaluation consultants) to high embeddedness where researchers/evaluators are employed by organizations and thus more deeply involved in program evolution and operations. Pros and cons related to the degree of embeddedness (low vs high) must be balanced when developing these relationships. We reflect on this process within the context of an embedded, mixed-methods evaluation of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program. Considerations that must be balanced include: (a) low vs high alignment of goals; (b) low vs high involvement in strategic planning; (c) observing what is happening vs being integrally involved with programmatic activities; (d) reporting findings at the project's end vs providing iterative findings and recommendations that contribute to program evolution; and (e) adhering to predetermined aims vs adapting aims in response to evolving partner needs.