RE-AIM for rural health innovations: perceptions of (mis) alignment between the RE-AIM framework and evaluation reporting in the Department of Veterans Affairs Enterprise-Wide Initiatives program.

Background: The Department of Veterans Affairs (VA) Office of Rural Health (ORH) supports national VA program offices' efforts to expand health care to rural Veterans through its Enterprise-Wide Initiatives (EWIs) program. In 2017, ORH selected Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM), an implementation science framework, to structure the EWI evaluation and reporting process. As part of its mandate to improve EWI program evaluation, the Center for the Evaluation of Enterprise-Wide Initiatives conducted a qualitative evaluation to better understand EWI team' perceptions of, and barriers and facilitators to, the EWI evaluation process. Methods: We conducted 43 semi-structured interviews with 48 team members (e.g., evaluators, program office leads, and field-based leads) representing 21 EWIs from April-December 2020. Questions focused on participants' experiences using strategies targeting each RE-AIM dimension. Interviews were inductively analyzed in MAXQDA. We also systematically reviewed 51 FY19-FY20 EWI annual reports to identify trends in misapplications of RE-AIM. Results: Participants had differing levels of experience with RE-AIM. While participants understood ORH's rationale for selecting a common framework to structure evaluations, the perceived misalignment between RE-AIM and EWIs' work emerged as an important theme. Concerns centered around 3 sub-themes: (1) (Mis)Alignment with RE-AIM Dimensions, (2) (Mis)Alignment between RE-AIM and the EWI, and (3) (Mis)Alignment with RE-AIM vs. other Theories, Models, or Frameworks. Participants described challenges differentiating between and operationalizing dimensions in unique contexts. Participants also had misconceptions about RE-AIM and its relevance to their work, e.g., that it was meant for established programs and did not capture aspects of initiative planning, adaptations, or sustainability. Less commonly, participants shared alternative models or frameworks to RE-AIM. Despite criticisms, many participants found RE-AIM useful, cited training as important to understanding its application, and identified additional training as a future need. Discussion: The selection of a shared implementation science framework can be beneficial, but also challenging when applied to diverse initiatives or contexts. Our findings suggest that establishing a common understanding, operationalizing framework dimensions for specific programs, and assessing training needs may better equip partners to integrate a shared framework into their evaluations.

Tracking implementation strategies in real-world settings: VA Office of Rural Health enterprise-wide initiative portfolio.

OBJECTIVE: To use a practical approach to examining the use of Expert Recommendations for Implementing Change (ERIC) strategies by Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) dimensions for rural health innovations using annual reports on a diverse array of initiatives. DATA SOURCES AND STUDY SETTING: The Veterans Affairs (VA) Office of Rural Health (ORH) funds initiatives designed to support the implementation and spread of innovations and evidence-based programs and practices to improve the health of rural Veterans. This study draws on the annual evaluation reports submitted for fiscal years 2020-2022 from 30 of these enterprise-wide initiatives (EWIs). STUDY DESIGN: Content analysis was guided by the RE-AIM framework conducted by the Center for the Evaluation of Enterprise-Wide Initiatives (CEEWI), a Quality Enhancement Research Initiative (QUERI)-ORH partnered evaluation initiative. DATA COLLECTION AND EXTRACTION METHODS: CEEWI analysts conducted a content analysis of EWI annual evaluation reports submitted to ORH. Analysis included cataloguing reported implementation strategies by Reach, Adoption, Implementation, and Maintenance (RE-AIM) dimensions (i.e., identifying strategies that were used to support each dimension) and labeling strategies using ERIC taxonomy. Descriptive statistics were conducted to summarize data. PRINCIPAL FINDINGS: A total of 875 implementation strategies were catalogued in 73 reports. Across these strategies, 66 unique ERIC strategies were reported. EWIs applied an average of 12 implementation strategies (range 3-22). The top three ERIC clusters across all 3 years were Develop stakeholder relationships (21%), Use evaluative/iterative strategies (20%), and Train/educate stakeholders (19%). Most strategies were reported within the Implementation dimension. Strategy use among EWIs meeting the rurality benchmark were also compared. CONCLUSIONS: Combining the dimensions from the RE-AIM framework and the ERIC strategies allows for understanding the use of implementation strategies across each RE-AIM dimension. This analysis will support ORH efforts to spread and sustain rural health innovations and evidence-based programs and practices through targeted implementation strategies.

Gaps in communication theory paradigms when conducting implementation science research: qualitative observations from interviews with administrators, implementors, and evaluators of rural health programs.

BACKGROUND: Communication is considered an inherent element of nearly every implementation strategy. Often it is seen as a means for imparting new information between stakeholders, representing a Transaction orientation to communication. From a Process orientation, communication is more than information-exchange and is acknowledged as being shaped by (and shaping) the individuals involved and their relationships with one another. As the field of Implementation Science (IS) works to strengthen theoretical integration, we encourage an interdisciplinary approach that engages communication theory to develop richer understanding of strategies and determinants of practice. METHODS: We interviewed 28 evaluators, 12 implementors, and 12 administrators from 21 Enterprise-Wide Initiatives funded by the Department of Veteran Affairs Office of Rural Health. Semi-structured interviews focused on experiences with implementation and evaluation strategies. We analyzed the interviews using thematic analysis identifying a range of IS constructs. Then we deductively classified those segments based on a Transaction or Process orientation to communication. RESULTS: We organized findings using the two IS constructs most commonly discussed in interviews: Collaboration and Leadership Buy-in. The majority of segments coded as Collaboration (n = 34, 74%) and Leadership Buy-in (n = 31, 70%) discussed communication from a Transaction orientation and referred to communication as synonymous with information exchange, which emphasizes the task over the relationships between the individuals performing the tasks. Conversely, when participants discussed Collaboration and Leadership Buy-in from a Process orientation, they acknowledged both constructs as the result of long-term efforts to develop positive relationships based on trust and respect, and emphasized the time costliness of such strategies. Our findings demonstrate that participants who discussed communication from a Process orientation recognized the nuance and complexity of interpersonal interactions, particularly in the context of IS. CONCLUSIONS: Efficient, reliable information exchange is a critical but often overemphasized element of implementation. Practitioners and researchers must recognize and incorporate the larger role of communication in IS. Two suggestions for engaging a Process orientation to communication are to: (a) use interview probes to learn how communication is enacted, and (b) use process-oriented communication theories to develop interventions and evaluation tools.

Perceptions of an automated benchmarking dashboard for antimicrobial stewardship programs among antimicrobial stewards within the veterans' health administration: a multicenter qualitative study.

Objective: To evaluate the impact of a multicenter, try automated dashboard on ASP activities and its acceptance among ASP leaders. Design: Frontline stewards were asked to participate in semi-structured interviews before and after implementation of a web-based ASP information dashboard providing risk-adjusted benchmarking, longitudinal trends, and analysis of antimicrobial usage patterns at each facility. Setting: The study was performed at Iowa City VA Health Care System. Participants: ASP team members from nine medical centers in the VA Midwest Health Care Network (VISN 23). Methods: Semi-structured interviews were conducted pre- and post-implementation, with interview guides informed by clinical experiences and the Consolidated Framework for Implementation Research (CFIR). Participants evaluated the dashboard's ease of use, applicability to ongoing ASP activities, perceived validity and reliability, and relative advantage over other ASP monitoring systems. Results: Compared to established stewardship data collection and reporting methods, participants found the dashboard more intuitive and accessible, allowing them to reduce dependence on other systems and staff to obtain and share data. Standardized and risk-adjusted rankings were largely accepted as a valuable benchmarking method; however, participants felt their facility's characteristics significantly influenced the rankings' validity. Participants recognized staffing, training, and uncertainty with using the dashboard as an intervention tool as barriers to consistent and comprehensive dashboard implementation. Conclusions: Participants generally accepted the dashboard's risk-adjusted metrics and appreciated its usability. While creating automated tools to rigorously benchmark antimicrobial use across hospitals can be helpful, the displayed metrics require further validation, and the longitudinal utility of the dashboard warrants additional study.

Brain Fog and Fatigue following COVID-19 Infection: An Exploratory Study of Patient Experiences of Long COVID.

Post-acute sequelae of SARS-CoV-2 (PASC) is a poorly understood condition with significant impact on quality of life. We aimed to better understand the lived experiences of patients with PASC, focusing on the impact of cognitive complaints ("brain fog") and fatigue on (1) daily activities, (2) work/employment, and (3) interpersonal relationships. We conducted semi-structured qualitative interviews with 15 patients of a Midwestern academic hospital's post-COVID-19 clinic. We audio-recorded, transcribed, and analyzed interviews thematically using a combined deductive-inductive approach and collected participants' characteristics from chart review. Participants frequently used descriptive and metaphorical language to describe symptoms that were relapsing-remitting and unpredictable. Fatigue and brain fog affected all domains and identified subthemes included symptoms' synergistic effects, difficulty with multitasking, lack of support, poor self-perception, and fear of loss of income and employment. Personal relationships were affected with change of responsibilities, difficulty parenting, social isolation, and guilt due to the burdens placed on family. Furthermore, underlying social stigma contributed to negative emotions, which significantly affected emotional and mental health. Our findings highlight PASC's negative impact on patients' daily lives. Providers can better support COVID-19 survivors during their recovery by identifying their needs in a sensitive and timely manner.

Community Treatment for Opioid Use Disorders in Urban and Rural Veterans.

PURPOSE: The recent opioid crisis is characterized by a relatively greater increase in opioid use disorder and related mortality in rural populations when compared with urban populations.1-5 As almost a quarter of our nation's veterans reside in rural settings, the United States Veterans Health Administration (VHA) is interested in the impact of this epidemic on rural veterans. This study aims to develop a comprehensive understanding of the trends in substance use disorders (SUD) in veterans seeking treatment from community, non-VHA providers. METHODS: Using Substance Abuse and Mental Health Services Administration (SAMHSA)'s Treatment Episode Data Set (TEDS), this study presents the prevalence of treatment for veterans seeking initial admission into publicly funded non-VHA SUD treatment centers for years 2011-2016. Comparisons were made for all SUD types. Multivariate trend analysis based on annual data from 2011 to 2016 compared urban and rural veterans for opioid use disorder treatment. FINDINGS: Both urban and rural veterans had comparable rates of treatment for SUD, though rural veterans had slightly higher rates of injectable (11.2% vs 8.7%; P < .001) and opiate drug use disorder admissions (20.7% vs 18.1%; P = .014). Both urban and rural showed an increase in admissions for opioid, heroin, and injectable drug use disorders between 2011 and 2016 (P < .001). CONCLUSIONS: Comprehensive understanding of veteran SUD and treatment should include national-level data on community non-VHA treatment. SAMHSA's TEDS for years 2011-2016 provides clinical information for more than 90,000 veterans and indicates continued increase in treatment seeking for opioid use disorders, particularly for rural veterans.

Racial/Ethnic Differences in Alcohol and Drug Use Outcomes Following Screening, Brief Intervention, and Referral to Treatment (SBIRT) in Federally Qualified Health Centers.

BACKGROUND: Substance use disorders (SUDs) pose a significant public health concern. Previous findings, while equivocal, demonstrate screening, brief intervention, and referral to treatment (SBIRT) is effective in reducing substance use and improving overall health. While race/ethnic and sex differences in SBIRT outcomes exist, racial/ethnic differences within sex groups remain unclear. The present study sought to quantify differences within race/ethnicity and sex in drug and alcohol use following SBIRT screenings. METHODS: Using health service data (N = 29,121) from a Midwestern state in four federally qualified health centers (FQHC) from 2012 to 2016, we assessed racial/ethnic and sex differences in the effect of SBIRT screening on alcohol and drug use between visits. We used McNemar's tests and multiple logistic regression to predict substance use at follow-up visits. RESULTS: We found a significant race/ethnicity by sex interaction predicting a positive alcohol prescreening (p < 0.001), precipitating a full alcohol screening, and subsequent hazardous drinking (p < 0.001) at full alcohol screening follow-up. Black males demonstrated the largest reduction in positive alcohol prescreenings at follow-up (9.24%). Patients identifying as White, Black, or Other demonstrated a reduction in hazardous drinking, though effect sizes were small and not clinically meaningful. No interactions in our drug outcome models were significant. CONCLUSION: SBIRT is useful in addressing health services equity among Black and male populations. Public health policy should support universal substance use screening and targeting interventions for underserved groups in clinical facilities likely to benefit the most. Resources should be directed to groups with the most pressing SUD treatment needs.

Use of a machine learning framework to predict substance use disorder treatment success.

There are several methods for building prediction models. The wealth of currently available modeling techniques usually forces the researcher to judge, a priori, what will likely be the best method. Super learning (SL) is a methodology that facilitates this decision by combining all identified prediction algorithms pertinent for a particular prediction problem. SL generates a final model that is at least as good as any of the other models considered for predicting the outcome. The overarching aim of this work is to introduce SL to analysts and practitioners. This work compares the performance of logistic regression, penalized regression, random forests, deep learning neural networks, and SL to predict successful substance use disorders (SUD) treatment. A nationwide database including 99,013 SUD treatment patients was used. All algorithms were evaluated using the area under the receiver operating characteristic curve (AUC) in a test sample that was not included in the training sample used to fit the prediction models. AUC for the models ranged between 0.793 and 0.820. SL was superior to all but one of the algorithms compared. An explanation of SL steps is provided. SL is the first step in targeted learning, an analytic framework that yields double robust effect estimation and inference with fewer assumptions than the usual parametric methods. Different aspects of SL depending on the context, its function within the targeted learning framework, and the benefits of this methodology in the addiction field are discussed.