RE-AIM for rural health innovations: perceptions of (mis) alignment between the RE-AIM framework and evaluation reporting in the Department of Veterans Affairs Enterprise-Wide Initiatives program.

Background: The Department of Veterans Affairs (VA) Office of Rural Health (ORH) supports national VA program offices' efforts to expand health care to rural Veterans through its Enterprise-Wide Initiatives (EWIs) program. In 2017, ORH selected Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM), an implementation science framework, to structure the EWI evaluation and reporting process. As part of its mandate to improve EWI program evaluation, the Center for the Evaluation of Enterprise-Wide Initiatives conducted a qualitative evaluation to better understand EWI team' perceptions of, and barriers and facilitators to, the EWI evaluation process. Methods: We conducted 43 semi-structured interviews with 48 team members (e.g., evaluators, program office leads, and field-based leads) representing 21 EWIs from April-December 2020. Questions focused on participants' experiences using strategies targeting each RE-AIM dimension. Interviews were inductively analyzed in MAXQDA. We also systematically reviewed 51 FY19-FY20 EWI annual reports to identify trends in misapplications of RE-AIM. Results: Participants had differing levels of experience with RE-AIM. While participants understood ORH's rationale for selecting a common framework to structure evaluations, the perceived misalignment between RE-AIM and EWIs' work emerged as an important theme. Concerns centered around 3 sub-themes: (1) (Mis)Alignment with RE-AIM Dimensions, (2) (Mis)Alignment between RE-AIM and the EWI, and (3) (Mis)Alignment with RE-AIM vs. other Theories, Models, or Frameworks. Participants described challenges differentiating between and operationalizing dimensions in unique contexts. Participants also had misconceptions about RE-AIM and its relevance to their work, e.g., that it was meant for established programs and did not capture aspects of initiative planning, adaptations, or sustainability. Less commonly, participants shared alternative models or frameworks to RE-AIM. Despite criticisms, many participants found RE-AIM useful, cited training as important to understanding its application, and identified additional training as a future need. Discussion: The selection of a shared implementation science framework can be beneficial, but also challenging when applied to diverse initiatives or contexts. Our findings suggest that establishing a common understanding, operationalizing framework dimensions for specific programs, and assessing training needs may better equip partners to integrate a shared framework into their evaluations.

Ankle- and Toe-Brachial Index for Peripheral Artery Disease Identification: Unlocking Clinical Data Through Novel Methods.

BACKGROUND: Despite its high prevalence and clinical impact, research on peripheral artery disease (PAD) remains limited due to poor accuracy of billing codes. Ankle-brachial index (ABI) and toe-brachial index can be used to identify PAD patients with high accuracy within electronic health records. METHODS: We developed a novel natural language processing (NLP) algorithm for extracting ABI and toe-brachial index values and laterality (right or left) from ABI reports. A random sample of 800 reports from 94 Veterans Affairs facilities during 2015 to 2017 was selected and annotated by clinical experts. We trained the NLP system using random forest models and optimized it through sequential iterations of 10-fold cross-validation and error analysis on 600 test reports and evaluated its final performance on a separate set of 200 reports. We also assessed the accuracy of NLP-extracted ABI and toe-brachial index values for identifying patients with PAD in a separate cohort undergoing ABI testing. RESULTS: The NLP system had an overall precision (positive predictive value) of 0.85, recall (sensitivity) of 0.93, and F1 measure (accuracy) of 0.89 to correctly identify ABI/toe-brachial index values and laterality. Among 261 patients with ABI testing (49% PAD), the NLP system achieved a positive predictive value of 92.3%, sensitivity of 83.1%, and specificity of 93.1% to identify PAD when compared with a structured chart review. The above findings were consistent in a range of sensitivity analysis. CONCLUSIONS: We successfully developed and validated an NLP system for identifying patients with PAD within the Veterans Affairs electronic health record. Our findings have broad implications for PAD research and quality improvement.

The STS case study: an analysis method for longitudinal qualitative research for implementation science.

BACKGROUND: Ethnographic approaches offer a method and a way of thinking about implementation. This manuscript applies a specific case study method to describe the impact of the longitudinal interplay between implementation stakeholders. Growing out of science and technology studies (STS) and drawing on the latent archaeological sensibilities implied by ethnographic methods, the STS case-study is a tool for implementors to use when a piece of material culture is an essential component of an innovation. METHODS: We conducted an ethnographic process evaluation of the clinical implementation of tele-critical care (Tele-CC) services in the Department of Veterans Affairs. We collected fieldnotes and conducted participant observation at virtual and in-person education and planning events (n = 101 h). At Go-Live and 6-months post-implementation, we conducted site visits to the Tele-CC hub and 3 partnered ICUs. We led semi-structured interviews with ICU staff at Go-Live (43 interviews with 65 participants) and with ICU and Tele-CC staff 6-months post-implementation (44 interviews with 67 participants). We used verification strategies, including methodological coherence, appropriate sampling, collecting and analyzing data concurrently, and thinking theoretically, to ensure the reliability and validity of our data collection and analysis process. RESULTS: The STS case-study helped us realize that we must think differently about how a Tele-CC clinician could be noticed moving from communal to intimate space. To understand how perceptions of surveillance impacted staff acceptance, we mapped the materials through which surveillance came to matter in the stories staff told about cameras, buttons, chimes, motors, curtains, and doorbells. CONCLUSIONS: STS case-studies contribute to the literature on longitudinal qualitive research (LQR) in implementation science, including pen portraits and periodic reflections. Anchored by the material, the heterogeneity of an STS case-study generates questions and encourages exploring differences. Begun early enough, the STS case-study method, like periodic reflections, can serve to iteratively inform data collection for researchers and implementors. The next step is to determine systematically how material culture can reveal implementation barriers and direct attention to potential solutions that address tacit, deeply rooted challenges to innovations in practice and technology.

Environmental service workers as potential designers of infection control policy in long-term care settings.

BACKGROUND: Long-term care facility residents are at higher risk of methicillin-resistant Staphylococcus aureus infection and colonization than the general population. In 2009, the Department of Veterans Affairs (VA) implemented the "methicillin-resistant S. aureus prevention initiative" in long-term care facilities (ie, Community Living Centers or "CLCs"). METHODS: Over 4 months, 40 semistructured interviews were conducted with staff in medicine, nursing, and environmental services at 5 geographically dispersed CLCs. Interviews addressed knowledge, attitudes, and beliefs concerning infection prevention and resident-centered care. A modified constant comparative approach was used for data analysis. RESULTS: In CLCs, staff work to prevent and control infections in spaces where residents live. Nurses and Environmental Service Workers daily balance infection prevention conventions with the CLC setting. Infection control team members, who are accustomed to working in acute care settings, struggle to reconcile the CLC context with infection prevention. DISCUSSION: The focus on the resident's room as the locus of care, and thus the main target of infection control, misses opportunities for addressing infection prevention in the spaces beyond the residents' rooms. CONCLUSIONS: Environmental Service Workers' daily work inside the rooms and within the wider facility produces a unique perspective that might help in the design of workable infection control policies in CLCs.

Utilized or Underutilized: A Qualitative Analysis of Building Coherence During Early Implementation of a Tele-Intensive Care Unit.

Background: Generating, reading, or interpreting data is a component of Telemedicine-Intensive Care Unit (Tele-ICU) utilization that has not been explored in the literature. Introduction: Using the idea of "coherence," a construct of Normalization Process Theory, we describe how intensive care unit (ICU) and Tele-ICU staff made sense of their shared work and how they made use of Tele-ICU together. Materials and Methods: We interviewed ICU and Tele-ICU staff involved in the implementation of Tele-ICU during site visits to a Tele-ICU hub and 3 ICUs, at preimplementation (43 interviews with 65 participants) and 6 months postimplementation (44 interviews with 67 participants). Data were analyzed using deductive coding techniques and lexical searches. Results: In the early implementation of Tele-ICU, ICU and Tele-ICU staff lacked consensus about how to share information and consequently how to make use of innovations in data tracking and interpretation offered by the Tele-ICU (e.g., acuity systems). Attempts to collaborate and create opportunities for utilization were supported by quality improvement (QI) initiatives. Discussion: Characterizing Tele-ICU utilization as an element of a QI process limited how ICU staff understood Tele-ICU as an innovation. It also did not promote an understanding of how the Tele-ICU used data and may therefore attenuate the larger promise of Tele-ICU as a potential tool for leveraging big data in critical care. Conclusions: Shared data practices lay the foundation for Tele-ICU program utilization but raise new questions about how the promise of big data can be operationalized for bedside ICU staff.

Implementation of the Smoking Treatment and Recovery (STAR) program: healthy cancer survivorship through integrated tobacco control.

PURPOSE: It is well-established that persistent tobacco use among patients with cancer results in numerous adverse outcomes. However, the assessment and treatment of tobacco use with evidence-based methods have been lacking in cancer care. Our cancer center has established its first tobacco treatment program, a multidisciplinary, evidence-based, clinical program for hematology/oncology patients. METHODS: We describe the development and implementation of the program, emphasizing lessons learned in treating nicotine addiction among patients who are at very high risk for continuing to use tobacco throughout the survivorship phase. RESULTS: We developed a system to assess tobacco use at each outpatient visit, from those recently diagnosed to long-term survivors. For patients who have smoked in the past month, the protocol offers standard behavioral and pharmacological treatments, delivered by tobacco treatment specialists and nurse practitioners over four in-person and/or telephone-based sessions. Partnerships with the Psychosocial Oncology and Cancer Survivorship Programs have provided integrated and comprehensive care for patients during and after their cancer treatment. CONCLUSIONS: The systematic efforts to reach and engage current smokers have laid the groundwork for maximizing the program's future effectiveness and impact. Our initial results demonstrate not only the complexities but also the feasibility of developing a new tobacco treatment program in the oncology setting. IMPLICATIONS FOR CANCER SURVIVORS: The implications for cancer survivors are the significant improvements in treatment outcomes that occur with tobacco abstinence.

An Interscholastic Network To Generate LexA Enhancer Trap Lines in Drosophila.

Binary expression systems like the LexA-LexAop system provide a powerful experimental tool kit to study gene and tissue function in developmental biology, neurobiology, and physiology. However, the number of well-defined LexA enhancer trap insertions remains limited. In this study, we present the molecular characterization and initial tissue expression analysis of nearly 100 novel StanEx LexA enhancer traps, derived from the StanEx1 index line. This includes 76 insertions into novel, distinct gene loci not previously associated with enhancer traps or targeted LexA constructs. Additionally, our studies revealed evidence for selective transposase-dependent replacement of a previously-undetected KP element on chromosome III within the StanEx1 genetic background during hybrid dysgenesis, suggesting a molecular basis for the over-representation of LexA insertions at the NK7.1 locus in our screen. Production and characterization of novel fly lines were performed by students and teachers in experiment-based genetics classes within a geographically diverse network of public and independent high schools. Thus, unique partnerships between secondary schools and university-based programs have produced and characterized novel genetic and molecular resources in Drosophila for open-source distribution, and provide paradigms for development of science education through experience-based pedagogy.

Social network members who engage in activities with older adults: Do they bring more social benefits than other members?

Active participation in social activities is important for the well-being of older adults. This study explored benefits of active social engagement by evaluating whether relationships that comprise active involvement (e.g., co-engagement in activities) bring more social benefits (i.e., social support, companionship, positive social influence) than other relationships that do not involve co-engagement. A total of 133 adults ages 60 years and older living in a rural Midwestern city in the United States were interviewed once and provided information on 1,740 social network members. Among 1,506 social relationships in which interactions occurred at least once a month, 52% involved engagement in social activities together and 35% involved eating together regularly. Results of the generalized linear mixed model showed that relationships involving co-engagement were significantly more likely to also convey social support (i.e., emotional, instrumental, informational), companionship, and social influence (encouragement for healthy behaviors) than relationships that do not involve co-engagement. Having more network members who provide companionship was associated with higher sense of environmental mastery, positive relations with others, and satisfaction with social network. Interventions may focus on maintaining and developing such social relationships and ensuring the presence of social settings in which co-engagement can occur. Future research may explore whether increasing co-engagement leads to enhanced sense of companionship and psychological well-being.

Understanding the Process of Prioritizing Fruit and Vegetable Purchases in Families With Low Incomes: "A Peach May Not Fill You Up as Much as Hamburger".

Fruits and vegetables (F&V) are an important component of a healthy diet, but few children are meeting the recommended number of servings. Children from families with limited resources may be least likely to meet the recommendation. This study was designed to understand the strategies and priorities of families with low income related to purchasing F&V. We conducted qualitative, in-depth telephone interviews with low-income parents of elementary school-aged children as part of a random sample of parents participating in a telephone survey who agreed to be contacted for an in-depth interview. Interview transcripts were coded based on predetermined codes that were informed by the research questions. F&V were not considered staples by parents and cost was one of the main concerns. Parents equated F&V with fresh F&V. Interventions encouraging F&V purchasing by families with low income need to find new ways to address the issue of cost, including advocating for F&V in all forms (fresh, frozen, canned, and dried).

The importance of matching the evaluation population to the intervention population: Using Medicaid data to reach hard-to-reach intervention populations.

Subject recruitment is a challenge for researchers and evaluators, particularly with populations that are traditionally hard to reach and involve in research, such as low-income and minority groups. However, when the evaluation sample does not reflect a program's intended audience, the discrepancy may lead to evaluation results that are not valid for that audience. We conducted evaluation activities for a state Supplemental Nutrition Assistance Program-Education (SNAP-Ed) intervention that promotes consumption of fruits and vegetables (F&V) in low-income areas. Previous program evaluations efforts had failed to obtain a sufficient proportion of individuals identified as low-income based on their participation in SNAP. We used state Medicaid data as a means of identifying low-income families to recruit for a telephone survey (n=311) and an in-depth qualitative interview (n=30) that we designed for the program being evaluated. We chose to focus on the dynamics of parent-child communication around F&V because we considered this previously unevaluated component of the intervention vital to understanding program effectiveness. Our results indicated that the Medicaid database provided an appropriate sample and that parents reported frequent F&V requests from their children. Parents also reported that they would positively respond to requests in many different settings, such as grocery stores (92.6%), restaurants (88.1%), and fast food restaurants (80.4%).