Supporting Veteran's Administration Medical Center Directors' Decisions When Adopting Innovative Practices: Development and Implementation of the "QuickView" and "WishList" Tools.

Background Since 2015, the Veterans Health Administration (VHA) Diffusion of Excellence Program has supported spread of practices developed by frontline employees. Shark Tank-style competitions encourage "Sharks" nationwide (VHA medical center/regional directors) to bid for the opportunity to implement practices at their institutions. Methods The authors evaluated bidding strategies (2016-2020), developing the "QuickView" practice comparator to promote informed bidding. Program leaders distributed QuickView and revised versions in subsequent competitions. Our team utilized in-person observation, online chats after the competition, bidder interviews, and bid analysis to evaluate QuickView use. Bids were ranked based on demonstrated understanding of resources required for practice implementation. Results Sharks stated that QuickView supported preparation before the competition and suggested improvements. Our revised tool reported necessary staff time and incorporated a "WishList" from practice finalists detailing minimum requirements for successful implementation. Bids from later years reflected increased review of facilities' current states before the competition and increased understanding of the resources needed for implementation. Percentage of bids describing local need for the practice rose from 2016 to 2020: 4.7% (6/127); 62.1% (54/87); 78.3% (36/46); 80.6% (29/36); 89.7% (26/29). Percentage of bids committing specific resources rose following QuickView introduction: 81.1% (103/127) in 2016, 69.0% (60/87) in 2017, then 73.9% (34/46) in 2018, 88.9% (32/36) in 2019, and 89.7% (26/29) in 2020. Discussion In the years following QuickView/WishList implementation, bids reflected increased assessment before the competition of both local needs and available resources. Conclusion Selection of a new practice for implementation requires an understanding of local need, necessary resources, and fit. QuickView and WishList appear to support these determinations.

Augmented Reality Demonstration Survey Results From a Veteran Affairs Medical Center.

Background: Augmented reality (AR) has a wide range of potential applications to enhance health care. Understanding how the introduction of a new technology may impact employees is essential for overall health care system success. Methods: Survey responses were obtained before and after a health care-focused interactive AR demonstration at a US Department of Veterans Affairs (VA) medical center. Data were assessed with descriptive statistics, Wilcoxon signed rank matched pairs test, pooled t test, and analysis of variance. Results: A total of 166 individuals participated in the demonstration and survey. Statistically significant improvements were seen after the use of the new AR technology in each of the categories assessed using a 5-point Likert scale. Scores for perceptions of institutional innovativeness increased from 3.4 to 4.5 (a 22% increase; P < .001); employee excitement about the VA increased from 3.7 to 4.3 (a 12% increase; P < .001); and employee likelihood to continue working at VA increased from 4.2 to 4.5 (a 6% increase; P < .001). Subgroup analysis demonstrated statistically significant differences by employee veteran status, VA tenure, and sex. Respondents felt strongly that this type of work will positively impact health care and that the VA should continue these efforts. Conclusions: An AR demonstration significantly increased employee excitement and intention to continue employment at the VA and provided valuable insights about the most impactful uses of AR in health care.

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool.

BACKGROUND: There are challenges associated with measuring sustainment of evidence-informed practices (EIPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EIP being in use in the future while sustainment assesses the extent to which an EIP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors "Shark Tank" competitions, in which leaders bid on the opportunity to implement a practice with approximately 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and pilot a pragmatic survey tool to assess sustainment of DoE practices. METHODS: In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open-text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open-text box responses. RESULTS: A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1-4. CONCLUSIONS: The initial development and piloting of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold. If additional piloting illustrates the survey is useful, we plan to assess the reliability and validity of this measure for broader use in the field.

How the VA is training the Next-Generation workforce for learning health systems.

Objectives: The U.S. Department of Veterans Affairs (VA) has been a national leader in Learning Health System (LHS) implementation due to its combined mission of research, education, clinical care, and emergency preparedness. We describe the current VA LHS training ecosystem within the Veterans Health Administration's Office of Academic Affiliations (OAA), Office of Research and Development (ORD), ORD's Health Services Research and Development (HSR&D) program, and Innovation Ecosystem (IE), including lessons learned regarding their sustainment. Methods: The VA LHS training ecosystem is based on the Learning Loop and HSR&D Quality Enhancement Research Initiative (QUERI) Roadmap, which describes VA learning opportunities, underlying infrastructures, and core competencies. Results: VA-focused LHS educational programs include data-to-knowledge initiatives in health sciences and analytics, for example, OAA/HSR&D health services and informatics research fellowships; knowledge-to-performance opportunities in implementation and quality improvement, for example, QUERI Learning Hubs and IEs' Diffusion of Excellence Initiative; and performance-to-data embedded opportunities, for example, IE's entrepreneur fellowship programs and QUERI's Advancing Diversity in Implementation Leadership. These training programs are supported by combined VA research and clinical operations investments in funding, informatics, governance, and processes. Lessons learned include ongoing alignment of research funding with operational priorities and capacity, relentless recruitment and retention of implementation, system, and information scientists especially from under-represented groups, sustainment of data infrastructures suitable for research and quality improvement, and ensuring sustainable funding opportunities for researchers to work on system-wide health care problems. Conclusions: There is an urgent need to expand training opportunities in LHSs, especially as health care is increasingly driven by multiple interested parties, impacted by persistent health disparities exacerbated by emerging public health threats, and rapid technology growth. With ongoing alignment of research and clinical goals, foundational support through research funding, underlying clinical operations infrastructures, and active engagement interested parties, VA's LHS training ecosystem promotes a more LHS-savvy, 21st century workforce.

Balancing reality in embedded research and evaluation: Low vs high embeddedness.

Embedding research and evaluation into organizations is one way to generate "practice-based" evidence needed to accelerate implementation of evidence-based innovations within learning health systems. Organizations and researchers/evaluators vary greatly in how they structure and operationalize these collaborations. One key aspect is the degree of embeddedness: from low embeddedness where researchers/evaluators are located outside organizations (eg, outside evaluation consultants) to high embeddedness where researchers/evaluators are employed by organizations and thus more deeply involved in program evolution and operations. Pros and cons related to the degree of embeddedness (low vs high) must be balanced when developing these relationships. We reflect on this process within the context of an embedded, mixed-methods evaluation of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program. Considerations that must be balanced include: (a) low vs high alignment of goals; (b) low vs high involvement in strategic planning; (c) observing what is happening vs being integrally involved with programmatic activities; (d) reporting findings at the project's end vs providing iterative findings and recommendations that contribute to program evolution; and (e) adhering to predetermined aims vs adapting aims in response to evolving partner needs.

The association between non-ventilator associated hospital acquired pneumonia and patient outcomes among U.S. Veterans.

BACKGROUND: Non-ventilator associated hospital acquired pneumonia (NV-HAP) affects approximately 1 in 100 hospitalized patients yet risk-adjusted outcomes associated with developing NV-HAP are unknown. METHODS: Retrospective cohort study with propensity score matched populations (NV-HAP vs no NV-HAP), using ICD-10 codes for bacterial pneumonia not present on admission. Outcomes included the patient level probability of NV-HAP developing among acute care non-transfer admissions in 133 Veterans Affairs hospitals and subsequent mortality, length of stay, inpatient sepsis, and 12-month costs. RESULTS: NV-HAP occurred in 0.6% of Veteran admissions. Among admissions that developed NV-HAP, the mean length of stay of 26.3 days (6.72 days among non-NV-HAP), 30-day mortality was 18.4% (4.5% among non-NV-HAP), 1-year mortality was 47.8% (21.4% among non-NV-HAP), and total median 12-month direct medical costs were $138,136.32 ($64,357.21 among non-NV-HAP). Inpatient sepsis occurred in approximately 20% of NV-HAP admissions (0.7% among non-NV-HAP). Data available at admission was insufficient to identify high and low risk patient groups. CONCLUSIONS: NV-HAP is associated with severely worse patient outcomes and increased costs of care up to 12 months post-episode. Since population risk stratification is not feasible, prevention efforts should be directed at the full population of hospitalized Veterans.

Merging Implementation Practice and Science to Scale Up Promising Practices: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) Program.

INTRODUCTION: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program developed and manages a framework for identification, replication, and diffusion of promising practices throughout the nation's largest integrated health care system. DoE identifies promising practices through a "Shark Tank" competition with winning bidders receiving external implementation facilitation. DoE further supports diffusion of successful practices across the VHA. METHODS: This article presents results of a mixed methods implementation evaluation of DoE, focusing on program reach, program participation and decisions to adopt innovative practices, implementation processes, and practice sustainment. Data sources include practice adoption metrics, focus groups with bidders (two focus groups), observations of DoE events (seven events), surveys of stakeholders (five separate surveys), and semistructured interviews of facility directors, practice developers, implementation teams, and facilitators (133 participants). RESULTS: In the first four Shark Tank cohorts (2016-2018), 1,676 practices were submitted; 47 were designated Gold Status Practices (practices with facilitated implementation). Motivation for participation varied. Generally, staff led projects targeting problems they felt passionate about, facility directors focused on big-picture quality metrics and getting middle manager support, and frontline staff displayed variable motivation to implement new projects. Approximately half of facilitated implementation efforts were successful; barriers included insufficient infrastructure, staff, and resources. At the facility level, 73.3% of facilities originating or receiving facilitated implementation support have maintained the practice. VHA-wide, 834 decisions to adopt these practices were made. CONCLUSION: DoE has resulted in the identification of many candidate practices, promoted adoption of promising practices by facility directors, and supported practice implementation and diffusion across the VHA.

Dynamic Diffusion Network: Advancing moral injury care and suicide prevention using an innovative model.

Healthcare providers across a wide variety of settings face a common challenge: the need to provide real time care for complex problems that are not adequately addressed by existing protocols. In response to these intervention gaps, frontline providers may utilize existing evidence to develop new approaches that are tailored to specific problems. It is imperative that such approaches undergo some form of evaluation, ensuring quality control while permitting ongoing adaptation and refinement. "Dynamic diffusion" is an innovative approach to intervention improvement and dissemination whereby care practices are delivered and continuously evaluated under real-world conditions as part of a structured network experience. This "dynamic diffusion network" (DDN) promotes cross-pollination of ideas and shared learning to generate relatively rapid improvements in care. The pilot Mental Health and Chaplaincy DDN was developed to advance suicide prevention efforts and moral injury care practices being conducted by 13 chaplain-mental health professional teams across the Veterans Health Administration. Lessons learned from the pilot DDN include the importance of the following: geographic and cultural diversity among innovation collaborators to ensure the broadest possible relevance of solutions; leadership support to facilitate engagement of frontline providers in quality improvement efforts; and participation in a community of practice to motivate providers and offer opportunities for direct collaboration and cross-pollination of ideas.

Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study.

BACKGROUND: One goal of health systems seeking to evolve into learning health systems is to accelerate the implementation and sustainment of evidence-based practices (EBPs). As part of this evolution, the Veterans Health Administration (VHA) developed the Innovation Ecosystem, which includes the Diffusion of Excellence (DoE), a program that identifies and diffuses Gold Status Practices (GSPs) across facilities. The DoE hosts an annual "Shark Tank" competition in which leaders bid on the opportunity to implement a GSP with 6 months of implementation support. Over 750 diverse practices were submitted in cohorts 2 and 3 of Shark Tank; 23 were designated GSPs and were implemented in 31 VA networks or facilities. As part of a national evaluation of the DoE, we identified factors contributing to GSP implementation and sustainment. METHODS: Our sequential mixed methods evaluation of cohorts 2 and 3 of Shark Tank included semi-structured interviews with at least one representative from 30/31 implementing teams (N = 78/105 people invited) and survey responses from 29/31 teams (N = 39/47 invited). Interviews focused on factors influencing implementation and future sustainment. Surveys focused on sustainment 1.5-2 years after implementation. The Consolidated Framework for Implementation Research (CFIR) informed data collection and directed content analysis. Ordinal scales were developed inductively to rank implementation and sustainment outcomes. RESULTS: Over 50% of teams (17/30) successfully implemented their GSP within the 6-month implementation period. Despite extensive implementation support, significant barriers related to centralized decision-making, staffing, and resources led to partial (n = 6) or no (n = 7) implementation for the remaining teams. While 12/17 initially successful implementation teams reported sustained use of their GSP, over half of the initially unsuccessful teams (n = 7/13) also reported sustained GSP use 1.5 years after the initial implementation period. When asked at 6 months, 18/27 teams with complete data accurately anticipated their future sustainability based on reported sustainment an average of 1.5 years later. CONCLUSIONS: Most teams implemented within 6 months and/or sustained their GSP 1.5 years later. High levels of implementation and sustainment across diverse practices and teams suggest that VHA's DoE is a successful large-scale model of diffusion. Team predictions about sustainability after the first 6 months of implementation provide a promising early assessment and point of intervention to increase sustainability.

A Handoffs Software Led to Fewer Errors of Omission and Better Provider Satisfaction: A Randomized Control Trial.

BACKGROUND: Computer-assisted communication is shown to prevent critical omissions ("errors") in the handoff process. OBJECTIVE: The aim of the study was to study this effect and related provider satisfaction, using a standardized software. METHODS: Fourteen internal medicine house officers staffed 6 days and 1 cross-covering teams were randomized to either the intervention group or control, employing usual handoff, so that handoff information was exchanged only between same-group subjects (daily, for 28 days). RESULTS: In the intervention group, fewer omissions (among those studied) occurred intravenous access (17 versus 422, P < 0.001), code status (1 versus 158, P < 0.001), diet/nothing per mouth (28 versus 477, P < 0.001), and deep venous thrombosis prophylaxis (17 versus 284, P < 0.001); duration to compose handoff was similar; and physicians perceived less workload adjusted for patient census and provider characteristics (P = 0.004) as well as better handoff quality (P < 0.001) and clarity (P < 0.001). CONCLUSIONS: The intervention was associated with fewer errors and superior provider satisfaction.

Diffusion of Excellence: Accelerating the Spread of Clinical Innovation and Best Practices across the Nation's Largest Health System.

The time it takes for clinical innovation and evidence-based practices to reach patients remains a major challenge for the health care sector. In 2015, the Veterans Health Administration (VHA) launched the Diffusion of Excellence Initiative aimed at aligning organizational resources with early-stage to midstage promising practices and innovations to replicate, scale, and eventually spread those with greatest potential for impact and positive outcomes. Using a 5-step systematic approach refined over time, frontline VHA staff have submitted more than 1676 practices since the initiative's inception, 47 of which have been selected as high-impact, Gold Status practices. These Gold Status practices have been replicated more than 412 times in Veterans Affairs hospitals across the country, improving care for more than 100,000 veterans and approximately $22.6 million in cost avoidance for the VHA. More importantly, practices such as Project HAPPEN (Hospital-Acquired Pneumonia Prevention by Engaging Nurses to complete oral care) and rapid availability of intranasal naloxone have saved veterans' lives. Several practices are now being implemented across the country, and the Diffusion of Excellence Initiative is playing a pivotal role as the VHA works to modernize its health care system. This initiative serves as a promising model for other health care systems seeking to accelerate the spread and adoption of clinical innovation and evidence-based practices.

How to Move a "Giant": 7 Lessons Learned for Making a Change in a Large Organization.

Bringing about change in a large organization can be a herculean task, similar to tales of old with the hero overcoming the slow and powerful giant. In this article, we discuss some of the lessons learned from our own journey: 1) Don't present your ideas in a whisper because you are afraid of feedback; share your project with a megaphone, because feedback can only result in a better product; 2) Be enthusiastic; every discussion about your practice is an opportunity to engage others; 3) Focus on those who are willing to participate in your vision; don't spend too much time and energy on naysayers; 4) Use the language of your audience. You do not need to become an expert in every subject matter, but you must be able to verbalize your project in a way that matches their terminology and priorities; 5) Streamline your facts and present enough information to remove the frightening unknown, but not so much that the details distract from the purpose; 6) Share your results, successes, and struggles; your authentic efforts will affect your audience far more than promises or predictions; and 7) Couple the fires of passion with the cold determination of commitment; the combination will help you persevere and achieve success.

Research Lifecycle to Increase the Substantial Real-world Impact of Research: Accelerating Innovations to Application.

BACKGROUND: US health care systems face a growing demand to incorporate innovations that improve patient outcomes at a lower cost. Funding agencies increasingly must demonstrate the impact of research investments on public health. The Learning Health System promotes continuous institutional innovation, yet specific processes to develop innovations for further research and implementation into real-world health care settings to maximize health impacts have not been specified. OBJECTIVE: We describe the Research Lifecycle and how it leverages institutional priorities to support the translation of research discoveries to clinical application, serving as a broader operational approach to enhance the Learning Health System. METHODS: Developed by the US Department of Veterans Affairs Office of Research and Development Research-to-Real-World Workgroup, the Research Lifecycle incorporates frameworks from product development, translational science, and implementation science methods. The Lifecycle is based on Workgroup recommendations to overcome barriers to more direct translation of innovations to clinical application and support practice implementation and sustainability. RESULTS: The Research Lifecycle posits 5 phases which support a seamless pathway from discovery to implementation: prioritization (leadership priority alignment), discovery (innovation development), validation (clinical, operational feasibility), scale-up and spread (implementation strategies, performance monitoring), and sustainability (business case, workforce training). An example of how the Research Lifecycle has been applied within a health system is provided. CONCLUSIONS: The Research Lifecycle aligns research and health system investments to maximize real-world practice impact via a feasible pathway, where priority-driven innovations are adapted for effective clinical use and supported through implementation strategies, leading to continuous improvement in real-world health care.

Psychological Safety and Support: Assessing Resident Perceptions of the Clinical Learning Environment.

BACKGROUND: Assessments of the clinical learning environment could allow early interventions to improve graduate medical education. To date, measurement tools for this have not been identified. OBJECTIVE: We established the concurrent validity of 2 instruments that assess cultural facets of the clinical learning environment by correlating them with external program evaluation data. METHODS: In 2017 we surveyed residents across 19 training programs on their perceptions of organizational support by using the Short Survey of Perceived Organizational Support (SPOS), and psychological safety by using the Psychological Safety Scale (PSS). Data were aggregated to the program level and correlated with results from the Accreditation Council for Graduate Medical Education (ACGME) Resident Survey. RESULTS: Of 511 residents, 322 (63%) completed the survey, and 496 of 516 residents (96%) completed the ACGME Resident Survey. Perceived organizational support correlated positively with overall program evaluation score (r = 0.75, P < .001); faculty (r = 0.72, P < .001); evaluation (r = 0.73, P < .001); educational content (r = 0.52, P = .022); and resources domains (r = 0.55, P = .014). Psychological safety had a positive correlation with overall program evaluation (r = 0.57, P = .011); faculty (r = 0.50, P = .028); and evaluation (r = 0.62, P < .005). CONCLUSIONS: The SPOS and PSS correlated with key ACGME Resident Survey domains. Programs showing greater support of residents were likely to show more positive ratings on program evaluation metrics. Teaching institutions may benefit from actively monitoring and improving aspects of their learning environment through internal assessments.