Understanding pathways from implementation to sustainment: a longitudinal, mixed methods analysis of promising practices implemented in the Veterans Health Administration.

BACKGROUND: The Veterans Health Administration (VHA) is the United States largest learning health system. The Diffusion of Excellence (DoE) program is a large-scale model of diffusion that identifies and diffuses evidence-informed practices across VHA. During the period of 2016-2021, 57 evidence-informed practices were implemented across 82 VHA facilities. This setting provides a unique opportunity to understand sustainment determinants and pathways. Our objective was to characterize the longitudinal pathways of practices as they transition from initial implementation to long-term sustainment at each facility. METHODS: A longitudinal, mixed-methods evaluation of 82 VHA facilities. Eighty-two facility representatives, chosen by leadership as points-of-contact for 57 DoE practices, were eligible for post-implementation interviews and annual sustainment surveys. Primary outcomes (implementation, sustainment), and secondary outcomes (institutionalization, effectiveness, anticipated sustainment) at four time-points were collected. We performed descriptive statistics and directed content analysis using Hailemariam et al.'s factors influencing sustainment. RESULTS: After approximately five years post-implementation (e.g., 2021 sustainment outcomes), of the 82 facilities, about one-third fully sustained their practice compared to one-third that did not fully sustain their practice because it was in a "liminal" stage (neither sustained nor discontinued) or permanently discontinued. The remaining one-third of facilities had missing 2021 sustainment outcomes. A higher percentage of facilities (70%) had inconsistent primary outcomes (changing over time) compared to facilities (30%) with consistent primary outcomes (same over time). Thirty-four percent of facilities with sustained practices reported resilience since they overcame implementation and sustainment barriers. Facilities with sustained practices reported more positive secondary outcomes compared to those that did not sustain their practice. Key factors facilitating practice sustainment included: demonstrating practice effectiveness/benefit, sufficient organizational leadership, sufficient workforce, and adaptation/alignment with local context. Key factors hindering practice sustainment included: insufficient workforce, not able to maintain practice fidelity/integrity, critical incidents related to the COVID-19 pandemic, organizational leadership did not support sustainment of practice, and no ongoing support. CONCLUSIONS: We identified diverse pathways from implementation to sustainment, and our data underscore that initial implementation outcomes may not determine long-term sustainment outcomes. This longitudinal evaluation contributes to understanding impacts of the DoE program, including return on investment, achieving learning health system goals, and insights into achieving high-quality healthcare in VHA.

Predicting Incident Adenocarcinoma of the Esophagus or Gastric Cardia Using Machine Learning of Electronic Health Records.

BACKGROUND & AIMS: Tools that can automatically predict incident esophageal adenocarcinoma (EAC) and gastric cardia adenocarcinoma (GCA) using electronic health records to guide screening decisions are needed. METHODS: The Veterans Health Administration (VHA) Corporate Data Warehouse was accessed to identify Veterans with 1 or more encounters between 2005 and 2018. Patients diagnosed with EAC (n = 8430) or GCA (n = 2965) were identified in the VHA Central Cancer Registry and compared with 10,256,887 controls. Predictors included demographic characteristics, prescriptions, laboratory results, and diagnoses between 1 and 5 years before the index date. The Kettles Esophageal and Cardia Adenocarcinoma predictioN (K-ECAN) tool was developed and internally validated using simple random sampling imputation and extreme gradient boosting, a machine learning method. Training was performed in 50% of the data, preliminary validation in 25% of the data, and final testing in 25% of the data. RESULTS: K-ECAN was well-calibrated and had better discrimination (area under the receiver operating characteristic curve [AuROC], 0.77) than previously validated models, such as the Nord-Trøndelag Health Study (AuROC, 0.68) and Kunzmann model (AuROC, 0.64), or published guidelines. Using only data from between 3 and 5 years before index diminished its accuracy slightly (AuROC, 0.75). Undersampling men to simulate a non-VHA population, AUCs of the Nord-Trøndelag Health Study and Kunzmann model improved, but K-ECAN was still the most accurate (AuROC, 0.85). Although gastroesophageal reflux disease was strongly associated with EAC, it contributed only a small proportion of gain in information for prediction. CONCLUSIONS: K-ECAN is a novel, internally validated tool predicting incident EAC and GCA using electronic health records data. Further work is needed to validate K-ECAN outside VHA and to assess how best to implement it within electronic health records.

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool.

BACKGROUND: There are challenges associated with measuring sustainment of evidence-informed practices (EIPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EIP being in use in the future while sustainment assesses the extent to which an EIP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors "Shark Tank" competitions, in which leaders bid on the opportunity to implement a practice with approximately 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and pilot a pragmatic survey tool to assess sustainment of DoE practices. METHODS: In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open-text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open-text box responses. RESULTS: A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1-4. CONCLUSIONS: The initial development and piloting of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold. If additional piloting illustrates the survey is useful, we plan to assess the reliability and validity of this measure for broader use in the field.

Yield of Repeat Endoscopy for Barrett's Esophagus After Normal Index Endoscopy.

INTRODUCTION: Guidelines suggest 1-time screening with esophagogastroduodenoscopy (EGD) for Barrett's esophagus (BE) in individuals at an increased risk of esophageal adenocarcinoma (EAC). We aimed to estimate the yield of repeat EGD performed at prolonged intervals after a normal index EGD. METHODS: We conducted a national retrospective analysis within the U S Veterans Health Administration, identifying patients with a normal index EGD between 2003 and 2009 who subsequently had a repeat EGD. We tabulated the proportion with a new diagnosis of BE, EAC, or esophagogastric junction adenocarcinoma (EGJAC) and conducted manual chart review of a sample. We fitted logistic regression models for the odds of a new diagnosis of BE/EAC/EGJAC. RESULTS: We identified 71,216 individuals who had a repeat EGD between 1 and 16 years after an index EGD without billing or cancer registry codes for BE/EAC/EGJAC. Of them, 4,088 had a new billing or cancer registry code for BE/EAC/EGJAC after the repeat EGD. On manual review of a stratified sample, most did not truly have new BE/EAC/EGJAC. A longer duration between EGD was associated with greater odds of a new diagnosis (adjusted odds ratio [aOR] for each 5 years 1.31; 95% confidence interval [CI] 1.19-1.44), particularly among those who were younger during the index EGD (ages 19-29 years: aOR 3.92; 95% CI 1.24-12.4; ages 60-69 years: aOR 1.19; 95% CI 1.01-1.40). DISCUSSION: The yield of repeat EGD for BE/EAC/EGJAC seems to increase with time after a normal index EGD, particularly for younger individuals. Prospective studies are warranted to confirm these findings.

Diffusion of excellence: evaluating a system to identify, replicate, and spread promising innovative practices across the Veterans health administration.

Introduction: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program provides a system to identify, replicate, and spread promising practices across the largest integrated healthcare system in the United States. DoE identifies innovations that have been successfully implemented in the VHA through a Shark Tank style competition. VHA facility and regional directors bid resources needed to replicate promising practices. Winning facilities/regions receive external facilitation to aid in replication/implementation over the course of a year. DoE staff then support diffusion of successful practices across the nationwide VHA. Methods: Organized around the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) Framework, we summarize results of an ongoing long-term mixed-methods implementation evaluation of DoE. Data sources include: Shark Tank application and bid details, tracking practice adoptions through a Diffusion Marketplace, characteristics of VHA facilities, focus groups with Shark Tank bidders, structured observations of DoE events, surveys of DoE program participants, and semi-structured interviews of national VHA program office leaders, VHA healthcare system/facility executives, practice developers, implementation teams and facilitators. Results: In the first eight Shark Tanks (2016-2022), 3,280 Shark Tank applications were submitted; 88 were designated DoE Promising Practices (i.e., practices receive facilitated replication). DoE has effectively spread practices across the VHA, with 1,440 documented instances of adoption/replication of practices across the VHA. This includes 180 adoptions/replications in facilities located in rural areas. Leadership decisions to adopt innovations are often based on big picture considerations such as constituency support and linkage to organizational goals. DoE Promising Practices that have the greatest national spread have been successfully replicated at new sites during the facilitated replication process, have close partnerships with VHA national program offices, and tend to be less expensive to implement. Two indicators of sustainment indicate that 56 of the 88 Promising Practices are still being diffused across the VHA; 56% of facilities originally replicating the practices have sustained them, even up to 6 years after the first Shark Tank. Conclusion: DoE has developed a sustainable process for the identification, replication, and spread of promising practices as part of a learning health system committed to providing equitable access to high quality care.

Environmental Contamination of Contact Precaution and Non-Contact Precaution Patient Rooms in Six Acute Care Facilities.

BACKGROUND: Environmental contamination is an important source of hospital multidrug-resistant organism (MDRO) transmission. Factors such as patient MDRO contact precautions (CP) status, patient proximity to surfaces, and unit type likely influence MDRO contamination and bacterial bioburden levels on patient room surfaces. Identifying factors associated with environmental contamination in patient rooms and on shared unit surfaces could help identify important environmental MDRO transmission routes. METHODS: Surfaces were sampled from MDRO CP and non-CP rooms, nursing stations, and mobile equipment in acute care, intensive care, and transplant units within 6 acute care hospitals using a convenience sampling approach blinded to cleaning events. Precaution rooms had patients with clinical or surveillance tests positive for methicillin-resistant Staphylococcus aureus, vancomycin-resistant enterococci, carbapenem-resistant Enterobacteriaceae or Acinetobacter within the previous 6 months, or Clostridioides difficile toxin within the past 30 days. Rooms not meeting this definition were considered non-CP rooms. Samples were cultured for the above MDROs and total bioburden. RESULTS: Overall, an estimated 13% of rooms were contaminated with at least 1 MDRO. MDROs were detected more frequently in CP rooms (32% of 209 room-sample events) than non-CP rooms (12% of 234 room-sample events). Surface bioburden did not differ significantly between CP and non-CP rooms or MDRO-positive and MDRO-negative rooms. CONCLUSIONS: CP room surfaces are contaminated more frequently than non-CP room surfaces; however, contamination of non-CP room surfaces is not uncommon and may be an important reservoir for ongoing MDRO transmission. MDRO contamination of non-CP rooms may indicate asymptomatic patient MDRO carriage, inadequate terminal cleaning, or cross-contamination of room surfaces via healthcare personnel hands.

Eliciting patient views on the allocation of limited healthcare resources: a deliberation on hepatitis C treatment in the Veterans Health Administration.

BACKGROUND: In response to the development of highly effective but expensive new medications, policymakers, payors, and health systems are considering novel and pragmatic ways to provide these medications to patients. One approach is to target these treatments to those most likely to benefit. However, to maximize the fairness of these policies, and the acceptance of their implementation, the values and beliefs of patients should be considered. The provision of treatments for chronic hepatitis C (CHC) in the resource-constrained context of the Veterans Health Administration (VHA) offered a real-world example of this situation, providing the opportunity to test the value of using Democratic Deliberation (DD) methods to solicit the informed opinions of laypeople on this complex issue. METHODS: We recruited Veterans (n = 30) from the VHA to attend a DD session. Following educational presentations from content experts, participants engaged in facilitated small group discussions to: 1) identify strategies to overcome CHC treatment barriers and 2) evaluate, vote on, and modify/improve two CHC treatment policies - "first come, first served" (FCFS) and "sickest first" (SF). We used transcripts and facilitators' notes to identify key themes from the small group discussions. Additionally, participants completed pre- and post-DD surveys. RESULTS: Most participants endorsed the SF policy over the FCFS policy, emphasizing the ethical and medical appropriateness of treating the sickest first. Concerns about SF centered on the difficulty of implementation (e.g., how is "sickest" determined?) and unfairness to other Veterans. Proposed modifications focused on: 1) the need to consider additional health factors, 2) taking behavior and lifestyle into account, 3) offering education and support, 4) improving access, and 5) facilitating better decision-making. CONCLUSIONS: DD offered a robust and useful method for addressing the allocation of the scarce resource of CHC treatment. Participants were able to develop a modified version of the SF policy and offered diverse recommendations to promote fairness and improve quality of care for Veterans. DD is an effective approach for incorporating patient preferences and gaining valuable insights for critical healthcare policy decisions in resource-limited environments.