The STS case study: an analysis method for longitudinal qualitative research for implementation science.

BACKGROUND: Ethnographic approaches offer a method and a way of thinking about implementation. This manuscript applies a specific case study method to describe the impact of the longitudinal interplay between implementation stakeholders. Growing out of science and technology studies (STS) and drawing on the latent archaeological sensibilities implied by ethnographic methods, the STS case-study is a tool for implementors to use when a piece of material culture is an essential component of an innovation. METHODS: We conducted an ethnographic process evaluation of the clinical implementation of tele-critical care (Tele-CC) services in the Department of Veterans Affairs. We collected fieldnotes and conducted participant observation at virtual and in-person education and planning events (n = 101 h). At Go-Live and 6-months post-implementation, we conducted site visits to the Tele-CC hub and 3 partnered ICUs. We led semi-structured interviews with ICU staff at Go-Live (43 interviews with 65 participants) and with ICU and Tele-CC staff 6-months post-implementation (44 interviews with 67 participants). We used verification strategies, including methodological coherence, appropriate sampling, collecting and analyzing data concurrently, and thinking theoretically, to ensure the reliability and validity of our data collection and analysis process. RESULTS: The STS case-study helped us realize that we must think differently about how a Tele-CC clinician could be noticed moving from communal to intimate space. To understand how perceptions of surveillance impacted staff acceptance, we mapped the materials through which surveillance came to matter in the stories staff told about cameras, buttons, chimes, motors, curtains, and doorbells. CONCLUSIONS: STS case-studies contribute to the literature on longitudinal qualitive research (LQR) in implementation science, including pen portraits and periodic reflections. Anchored by the material, the heterogeneity of an STS case-study generates questions and encourages exploring differences. Begun early enough, the STS case-study method, like periodic reflections, can serve to iteratively inform data collection for researchers and implementors. The next step is to determine systematically how material culture can reveal implementation barriers and direct attention to potential solutions that address tacit, deeply rooted challenges to innovations in practice and technology.

Utilized or Underutilized: A Qualitative Analysis of Building Coherence During Early Implementation of a Tele-Intensive Care Unit.

Background: Generating, reading, or interpreting data is a component of Telemedicine-Intensive Care Unit (Tele-ICU) utilization that has not been explored in the literature. Introduction: Using the idea of "coherence," a construct of Normalization Process Theory, we describe how intensive care unit (ICU) and Tele-ICU staff made sense of their shared work and how they made use of Tele-ICU together. Materials and Methods: We interviewed ICU and Tele-ICU staff involved in the implementation of Tele-ICU during site visits to a Tele-ICU hub and 3 ICUs, at preimplementation (43 interviews with 65 participants) and 6 months postimplementation (44 interviews with 67 participants). Data were analyzed using deductive coding techniques and lexical searches. Results: In the early implementation of Tele-ICU, ICU and Tele-ICU staff lacked consensus about how to share information and consequently how to make use of innovations in data tracking and interpretation offered by the Tele-ICU (e.g., acuity systems). Attempts to collaborate and create opportunities for utilization were supported by quality improvement (QI) initiatives. Discussion: Characterizing Tele-ICU utilization as an element of a QI process limited how ICU staff understood Tele-ICU as an innovation. It also did not promote an understanding of how the Tele-ICU used data and may therefore attenuate the larger promise of Tele-ICU as a potential tool for leveraging big data in critical care. Conclusions: Shared data practices lay the foundation for Tele-ICU program utilization but raise new questions about how the promise of big data can be operationalized for bedside ICU staff.

RE-AIM for rural health innovations: perceptions of (mis) alignment between the RE-AIM framework and evaluation reporting in the Department of Veterans Affairs Enterprise-Wide Initiatives program.

Background: The Department of Veterans Affairs (VA) Office of Rural Health (ORH) supports national VA program offices' efforts to expand health care to rural Veterans through its Enterprise-Wide Initiatives (EWIs) program. In 2017, ORH selected Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM), an implementation science framework, to structure the EWI evaluation and reporting process. As part of its mandate to improve EWI program evaluation, the Center for the Evaluation of Enterprise-Wide Initiatives conducted a qualitative evaluation to better understand EWI team' perceptions of, and barriers and facilitators to, the EWI evaluation process. Methods: We conducted 43 semi-structured interviews with 48 team members (e.g., evaluators, program office leads, and field-based leads) representing 21 EWIs from April-December 2020. Questions focused on participants' experiences using strategies targeting each RE-AIM dimension. Interviews were inductively analyzed in MAXQDA. We also systematically reviewed 51 FY19-FY20 EWI annual reports to identify trends in misapplications of RE-AIM. Results: Participants had differing levels of experience with RE-AIM. While participants understood ORH's rationale for selecting a common framework to structure evaluations, the perceived misalignment between RE-AIM and EWIs' work emerged as an important theme. Concerns centered around 3 sub-themes: (1) (Mis)Alignment with RE-AIM Dimensions, (2) (Mis)Alignment between RE-AIM and the EWI, and (3) (Mis)Alignment with RE-AIM vs. other Theories, Models, or Frameworks. Participants described challenges differentiating between and operationalizing dimensions in unique contexts. Participants also had misconceptions about RE-AIM and its relevance to their work, e.g., that it was meant for established programs and did not capture aspects of initiative planning, adaptations, or sustainability. Less commonly, participants shared alternative models or frameworks to RE-AIM. Despite criticisms, many participants found RE-AIM useful, cited training as important to understanding its application, and identified additional training as a future need. Discussion: The selection of a shared implementation science framework can be beneficial, but also challenging when applied to diverse initiatives or contexts. Our findings suggest that establishing a common understanding, operationalizing framework dimensions for specific programs, and assessing training needs may better equip partners to integrate a shared framework into their evaluations.

Ankle- and Toe-Brachial Index for Peripheral Artery Disease Identification: Unlocking Clinical Data Through Novel Methods.

BACKGROUND: Despite its high prevalence and clinical impact, research on peripheral artery disease (PAD) remains limited due to poor accuracy of billing codes. Ankle-brachial index (ABI) and toe-brachial index can be used to identify PAD patients with high accuracy within electronic health records. METHODS: We developed a novel natural language processing (NLP) algorithm for extracting ABI and toe-brachial index values and laterality (right or left) from ABI reports. A random sample of 800 reports from 94 Veterans Affairs facilities during 2015 to 2017 was selected and annotated by clinical experts. We trained the NLP system using random forest models and optimized it through sequential iterations of 10-fold cross-validation and error analysis on 600 test reports and evaluated its final performance on a separate set of 200 reports. We also assessed the accuracy of NLP-extracted ABI and toe-brachial index values for identifying patients with PAD in a separate cohort undergoing ABI testing. RESULTS: The NLP system had an overall precision (positive predictive value) of 0.85, recall (sensitivity) of 0.93, and F1 measure (accuracy) of 0.89 to correctly identify ABI/toe-brachial index values and laterality. Among 261 patients with ABI testing (49% PAD), the NLP system achieved a positive predictive value of 92.3%, sensitivity of 83.1%, and specificity of 93.1% to identify PAD when compared with a structured chart review. The above findings were consistent in a range of sensitivity analysis. CONCLUSIONS: We successfully developed and validated an NLP system for identifying patients with PAD within the Veterans Affairs electronic health record. Our findings have broad implications for PAD research and quality improvement.

Environmental service workers as potential designers of infection control policy in long-term care settings.

BACKGROUND: Long-term care facility residents are at higher risk of methicillin-resistant Staphylococcus aureus infection and colonization than the general population. In 2009, the Department of Veterans Affairs (VA) implemented the "methicillin-resistant S. aureus prevention initiative" in long-term care facilities (ie, Community Living Centers or "CLCs"). METHODS: Over 4 months, 40 semistructured interviews were conducted with staff in medicine, nursing, and environmental services at 5 geographically dispersed CLCs. Interviews addressed knowledge, attitudes, and beliefs concerning infection prevention and resident-centered care. A modified constant comparative approach was used for data analysis. RESULTS: In CLCs, staff work to prevent and control infections in spaces where residents live. Nurses and Environmental Service Workers daily balance infection prevention conventions with the CLC setting. Infection control team members, who are accustomed to working in acute care settings, struggle to reconcile the CLC context with infection prevention. DISCUSSION: The focus on the resident's room as the locus of care, and thus the main target of infection control, misses opportunities for addressing infection prevention in the spaces beyond the residents' rooms. CONCLUSIONS: Environmental Service Workers' daily work inside the rooms and within the wider facility produces a unique perspective that might help in the design of workable infection control policies in CLCs.