Transitioning Cognitive Aids into Decision Support Platforms: Requirements and Design Guidelines.

Digital cognitive aids have the potential to serve as clinical decision support platforms, triggering alerts about process delays and recommending interventions. In this mixed-methods study, we examined how a digital checklist for pediatric trauma resuscitation could trigger decision support alerts and recommendations. We identified two criteria that cognitive aids must satisfy to support these alerts: (1) context information must be entered in a timely, accurate, and standardized manner, and (2) task status must be accurately documented. Using co-design sessions and near-live simulations, we created two checklist features to satisfy these criteria: a form for entering the pre-hospital information and a progress slider for documenting the progression of a multi-step task. We evaluated these two features in the wild, contributing guidelines for designing these features on cognitive aids to support alerts and recommendations in time- and safety-critical scenarios.

Real-time Context-Aware Multimodal Network for Activity and Activity-Stage Recognition from Team Communication in Dynamic Clinical Settings.

In clinical settings, most automatic recognition systems use visual or sensory data to recognize activities. These systems cannot recognize activities that rely on verbal assessment, lack visual cues, or do not use medical devices. We examined speech-based activity and activity-stage recognition in a clinical domain, making the following contributions. (1) We collected a high-quality dataset representing common activities and activity stages during actual trauma resuscitation events-the initial evaluation and treatment of critically injured patients. (2) We introduced a novel multimodal network based on audio signal and a set of keywords that does not require a high-performing automatic speech recognition (ASR) engine. (3) We designed novel contextual modules to capture dynamic dependencies in team conversations about activities and stages during a complex workflow. (4) We introduced a data augmentation method, which simulates team communication by combining selected utterances and their audio clips, and showed that this method contributed to performance improvement in our data-limited scenario. In offline experiments, our proposed context-aware multimodal model achieved F1-scores of 73.2±0.8% and 78.1±1.1% for activity and activity-stage recognition, respectively. In online experiments, the performance declined about 10% for both recognition types when using utterance-level segmentation of the ASR output. The performance declined about 15% when we omitted the utterance-level segmentation. Our experiments showed the feasibility of speech-based activity and activity-stage recognition during dynamic clinical events.

Discovering interpretable medical process models: A case study in trauma resuscitation.

Understanding the actual work (i.e., "work-as-done") rather than theorized work (i.e., "work-as-imagined") during complex medical processes is critical for developing approaches that improve patient outcomes. Although process mining has been used to discover process models from medical activity logs, it often omits critical steps or produces cluttered and unreadable models. In this paper, we introduce a TraceAlignment-based ProcessDiscovery method called TAD Miner to build interpretable process models for complex medical processes. TAD Miner creates simple linear process models using a threshold metric that optimizes the consensus sequence to represent the backbone process, and then identifies both concurrent activities and uncommon-but-critical activities to represent the side branches. TAD Miner also identifies the locations of repeated activities, an essential feature for representing medical treatment steps. We conducted a study using activity logs of 308 pediatric trauma resuscitations to develop and evaluate TAD Miner. TAD Miner was used to discover process models for five resuscitation goals, including establishing intravenous (IV) access, administering non-invasive oxygenation, performing back assessment, administering blood transfusion, and performing intubation. We quantitively evaluated the process models with several complexity and accuracy metrics, and performed qualitative evaluation with four medical experts to assess the accuracy and interpretability of the discovered models. Through these evaluations, we compared the performance of our method to that of two state-of-the-art process discovery algorithms: Inductive Miner and Split Miner. The process models discovered by TAD Miner had lower complexity and better interpretability than the state-of-the-art methods, and the fitness and precision of the models were comparable. We used the TAD process models to identify (1) the errors and (2)the best locations for the tentative steps in knowledge-driven expert models. The knowledge-driven models were revised based on the modifications suggested by the discovered models. The improved modeling using TAD Miner may enhance understanding of complex medical processes.

Factors Affecting Peripheral Intravenous Catheter Placement During Pediatric Trauma Resuscitation.

INTRODUCTION: Intravenous access is required for resuscitation of injured patients but may be delayed in children because of challenges associated with peripheral intravenous (PIV) catheter placement. Early identification of factors predisposing patients to difficult PIV placement can assist in deciding strategies for timely intravenous access. METHODS: We conducted a retrospective, video-based review of injured children and adolescents treated between April 2018 and May 2019. Patient demographic, physiological, injury, and resuscitation characteristics were obtained from the patient record, including age, race, weight, injury type, Injury Severity Score, initial systolic blood pressure, initial Glasgow Coma Score, intubation status, activation level, and presence of prearrival notification. Video review was used to determine the time to PIV placement, the number of attempts required, the purpose for additional access, and the reason for abandonment of PIV placement. Multivariable regressions were used to determine factors associated with successful placement. RESULTS: During the study period, 154 consented patients underwent attempts at PIV placement in the trauma bay. Placement was successful in 139 (90.3%) patients. Older patients (OR [odds ratio]: 0.9, 95% confidence interval [CI]: 0.9, 0.9) and patients who required the highest level activation response (OR: 0.0, 95% CI: 0.0, 0.3) were less likely to have an attempt at PIV placement abandoned. Children with nonblunt injuries (OR: 11.6, 95% CI: 1.3, 119.2) and pre-existing access (OR: 39.6, 95% CI: 7.0, 350.6) were more likely to have an attempt at PIV placement abandoned. Among patients with successful PIV placement, the time required for establishing PIV access was faster as age increased (-0.5 s, 95% CI: -1.1, -0.0). CONCLUSIONS: Younger age was associated with abandonment of PIV attempts and, when successful, increased time to placement. Strategies to improve successful PIV placement and alternate routes of access should be considered early to prevent treatment delays in younger children.

Analysis of Task Attributes Associated with Crisis Checklist Compliance in Pediatric Trauma Resuscitation.

Although checklists can improve overall team performance during medical crises, non-compliant checklist use poses risks to patient safety. We examined how task attributes affected checklist compliance by studying the use of a digital checklist during trauma resuscitation. We first determined task attributes and checklist compliance behaviors for 3,131 resuscitation tasks. Using statistical analyses and qualitative video review, we then identified barriers to accurately tracking task status, finding that certain task attributes were associated with non-compliant checklist behaviors. For example, tasks with multiple steps were more likely to be incorrectly recorded as completed when the task was not performed to completion. We discuss challenges in capturing and tracking the status of tasks with attributes that contribute to non-compliant checklist use. We also contribute a framework for understanding how tasks with certain attributes can be designed on checklists to improve compliance.

An Analysis of Speech during Life Saving Interventions to Inform the Design of a Computerized System for Delay Detection.

We describe an analysis of speech during time-critical, team-based medical work and its potential to indicate process delays. We analyzed speech intention and sentence types during 39 trauma resuscitations with delays in one of three major lifesaving interventions: intravenous/intraosseous (IV/IO) line insertion, cardiopulmonary and resuscitation (CPR), and intubation. We found a significant difference in patterns of speech during delays vs. speech during non-delayed work. The speech intention during CPR delays, however, differed from the other LSIs, suggesting that context of speech must be considered. These findings will inform the design of a clinical decision support system (CDSS) that will use multiple sensor modalities to alert medical teams to delays in real time. We conclude with design implications and challenges associated with speech-based activity recognition in complex medical processes.

Personal Protective Equipment Adherence of Pediatric Resuscitation Team Members During the COVID-19 Pandemic.

STUDY OBJECTIVE: During the COVID-19 pandemic, health care workers have had the highest risk of infection among essential workers. Although personal protective equipment (PPE) use is associated with lower infection rates, appropriate use of PPE has been variable among health care workers, even in settings with COVID-19 patients. We aimed to evaluate the patterns of PPE adherence during emergency department resuscitations that included aerosol-generating procedures. METHODS: We conducted a retrospective, video-based review of pediatric resuscitations involving one or more aerosol-generating procedures during the first 3 months of the COVID-19 pandemic in the United States (March to June 2020). Recommended adherence (complete, inadequate, absent) with 5 PPE items (headwear, eyewear, masks, gowns, gloves) and the duration of potential exposure were evaluated for individuals in the room after aerosol-generating procedure initiation. RESULTS: Among the 345 health care workers observed during 19 resuscitations, 306 (88.7%) were nonadherent (inadequate or absent adherence) with the recommended use of at least 1 PPE type at some time during the resuscitation, 23 (6.7%) of whom had no PPE. One hundred and forty health care workers (40.6%) altered or removed at least 1 type of PPE during the event. The aggregate time in the resuscitation room for health care workers across all events was 118.7 hours. During this time, providers had either absent or inadequate eyewear for 46.4 hours (39.1%) and absent or inadequate masks for 35.2 hours (29.7%). CONCLUSION: Full adherence with recommended PPE use was limited in a setting at increased risk for SARS-CoV-2 virus aerosolization. In addition to ensuring appropriate donning, approaches are needed for ensuring ongoing adherence with PPE recommendations during exposure.

Metrics of shock in pediatric trauma patients: A systematic search and review.

INTRODUCTION: Shock-index (SI) and systolic blood pressure (SBP) are metrics for identifying children and adults with hemodynamic instability following injury. The purpose of this systematic review was to assess the quality of these metrics as predictors of outcomes following pediatric injury. MATERIALS AND METHODS: We conducted a literature search in Pubmed, SCOPUS, and CINAHL to identify studies describing the association between shock metrics on the morbidity and mortality of injured children and adolescents. We used the data presented in the studies to calculate the sensitivity and specificity for each metric. This study was registered with Prospero, protocol CRD42020162971. RESULTS: Fifteen articles met the inclusion criteria. seven studies evaluated SI or SIPA score, an age-corrected version of SI, as predictors of outcomes following pediatric trauma, with one study comparing SIPA score and SBP and one study comparing SI and SBP. The remaining eight studies evaluated SBP as the primary indicator of shock. The median sensitivity for predicting mortality and need for blood transfusion was highest for SI, followed by SIPA, and then SBP. The median specificity for predicting these outcomes was highest for SBP, followed by SIPA, and then SI. CONCLUSIONS: Common conclusions were that high SIPA scores were more specific than SI and more sensitive than SBP. SIPA score had better discrimination for severely injured children compared to SI and SBP. An elevated SIPA was associated with a greater need for blood transfusion and higher in-hospital mortality. SIPA is specific enough to exclude most patients who do not require a blood transfusion.

Association of Ventilation during Initial Trauma Resuscitation for Traumatic Brain Injury and Post-Traumatic Outcomes: A Systematic Review.

OBJECTIVES: In the absence of evidence of acute cerebral herniation, normal ventilation is recommended for patients with traumatic brain injury (TBI). Despite this recommendation, ventilation strategies vary during the initial management of patients with TBI and may impact outcome. The goal of this systematic review was to define the best evidence-based practice of ventilation management during the initial resuscitation period. METHODS: A literature search of PubMed, CINAHL, and SCOPUS identified studies from 2009 through 2019 addressing the effects of ventilation during the initial post-trauma resuscitation on patient outcomes. RESULTS: The initial search yielded 899 articles, from which 13 were relevant and selected for full-text review. Six of the 13 articles met the inclusion criteria, all of which reported on patients with TBI. Either end-tidal carbon dioxide (ETCO2) or partial pressure carbon dioxide (PCO2) were the independent variables associated with mortality. Decreased rates of mortality were reported in patients with normal PCO2 or ETCO2. CONCLUSIONS: Normoventilation, as measured by ETCO2 or PCO2, is associated with decreased mortality in patients with TBI. Preventing hyperventilation or hypoventilation in patients with TBI during the early resuscitation phase could improve outcome after TBI.

Assessment of Non-Routine Events and Significant Physiological Disturbances during Emergency Department Evaluation after Pediatric Head Trauma.

Outcomes following pediatric traumatic brain injury (TBI) are dependent on initial injury severity and prevention of secondary injury. Hypoxia, hypotension, and hyperventilation following TBI are associated with increased mortality. The purpose of this study was to determine the association of non-routine events (NREs) during the initial resuscitation phase with these physiological disturbances. We conducted a video review of pediatric trauma resuscitations of patients with suspected TBI and Glasgow Coma Scale (GCS) scores <13. NREs were rated as "momentary" if task progression was delayed by <1 min and "moderate" if delayed by >1 min. Vital sign monitor data were used to identify periods of significant physiological disturbances. We calculated the association between the rate of overall and moderate NREs per case and the proportion of cases with abnormal vital signs using multi-variate linear regression, controlling for GCS score and need for intubation. Among 26 resuscitations, 604 NREs were identified with a median of 23 (interquartile range [IQR] 17-27.8, range 5-44) per case. Moderate delay NREs occurred in 19 resuscitations (n = 32, median 1 NRE/resuscitation, IQR 0.3-1, range 0-5). Oxygen desaturation and respiratory depression were associated with a greater rate of moderate NREs (p = 0.008, p < 0.001, respectively). We observed no association between duration of hypotension, desaturation, and respiratory depression and overall NRE rate. NREs are common in the initial resuscitation of children with moderate to severe TBI. Episodes of hypoxia and respiratory depression are associated with NREs that cause a moderate delay in task progression. Conformance with resuscitation guidelines is needed to prevent physiological events associated with adverse outcomes following pediatric TBI.

Understanding Paper-Based Documentation Practices in Medical Resuscitations to Inform the Design of Electronic Documentation Tools.

OBJECTIVE: Despite growing use of electronic health records, many resuscitation settings still use paper-based documentation. The fast-paced and safety-critical nature of trauma and medical resuscitation environments pose challenges for real-time documentation. This study aims to understand paper-based documentation practices and inform the design of efficient electronic documentation solutions for supporting safety-critical medical processes. METHODS: Data were collected through in situ observations of nurse documenters during resuscitation events and postevent interviews with nurses. These data were analyzed using frequency distribution and qualitative, open-coding techniques. Data analysis focused on the following 3 main documentation factors: temporal distribution of documentation, total number of filled out sections on the paper flow sheet across all resuscitations, and completeness of documentation per resuscitation. RESULTS: Findings from this study highlight the time-critical nature of these settings, showing that 74% of the documentation was completed within the first 15 minutes of the resuscitation. Some sections of the paper flow sheet were filled out more than others, and a few sections were left incomplete across all events. Interviews with nurses provided insight about documentation experiences in a fast-paced environment, including variable usage of flow sheet based on nurse experience level and patient scenarios, supplemental documentation mechanisms, and information needs and preferences. CONCLUSIONS: Several design implications are discussed to inform the design of effective electronic documentation systems. Design implications focus on layout structure, prepopulating items, section placement, and completion status of the flow sheet. Future plans for research focus on combining video review with in situ observations and conducting detailed interviews with nurses to better understand their documentation experiences and preferences.

Transitioning to Remote User-Centered Design Activities in the Emergency Medical Field During a Pandemic.

During the COVID-19 pandemic, we had to transition our user-centered research and design activities in the emergency medical domain of trauma resuscitation from in-person settings to online environments. This transition required that we replicate the in-person interactions remotely while maintaining the critical social connection and the exchange of ideas with medical providers. In this paper, we describe how we designed and conducted four user-centered design activities from our homes: participatory design workshops, near-live simulation sessions, usability evaluation sessions, and interviews and design walkthroughs. We discuss the differences we observed in our interactions with participants in remote sessions, as well as the differences in the interactions among the research team members. From this experience, we draw several lessons and outline the best practices for remotely conducting user-centered design activities that have been traditionally held in person.

User Needs and Challenges in Information Sharing between Pre-Hospital and Hospital Emergency Care Providers.

Effective communication between pre-hospital and hospital providers is a critical first step towards ensuring efficient patient care. Despite many efforts in improving the communication process, inefficiencies persist. It is critical to understand user needs, work practices, and existing barriers to inform technology design for supporting pre-hospital communication. However, existing research examining the ways in which patient information is collected and shared by pre-hospital providers in the field has been limited. We conducted a series of ethnographic studies with both prehospital and hospital care providers to examine 1) the types of information that are commonly collected and shared by the pre-hospital providers in the field; 2) the types of pre-hospital information that are needed by hospital-based teams for ensuring appropriate preparation; and 3) the challenges in the pre-hospital communication process. We conclude by discussing technology opportunities for facilitating real-time information sharing in the field.

Designing Interactive Alerts to Improve Recognition of Critical Events in Medical Emergencies.

Vital sign values during medical emergencies can help clinicians recognize and treat patients with life-threatening injuries. Identifying abnormal vital signs, however, is frequently delayed and the values may not be documented at all. In this mixed-methods study, we designed and evaluated a two-phased visual alert approach for a digital checklist in trauma resuscitation that informs users about undocumented vital signs. Using an interrupted time series analysis, we compared documentation in the periods before (two years) and after (four months) the introduction of the alerts. We found that introducing alerts led to an increase in documentation throughout the post-intervention period, with clinicians documenting vital signs earlier. Interviews with users and video review of cases showed that alerts were ineffective when clinicians engaged less with the checklist or set the checklist down to perform another activity. From these findings, we discuss approaches to designing alerts for dynamic team-based settings.

Exploring the Design of Streaming Data Interfaces for Emergency Medical Contexts.

In this late-breaking work, we describe the design of an interface for displaying streaming vital sign data on a digital checklist used in the emergency medical setting of pediatric trauma resuscitation. We used feedback from interviews and participatory design workshops with clinicians to develop two prototypes of the streaming vital sign interface. We evaluated these prototypes in design-walkthroughs, finding that clinicians preferred the design displaying trend graphs for all four vital signs at once. We discuss how streaming data interfaces on interactive mobile devices can be used to provide situational awareness while unobtrusively supporting different levels of clinical experience.

Classification strategies for non-routine events occurring in high-risk patient care settings: A scoping review.

INTRODUCTION: Non-routine events (NREs) are atypical or unusual occurrences in a pre-defined process. Although some NREs in high-risk clinical settings have no adverse effects on patient care, others can potentially cause serious patient harm. A unified strategy for identifying and describing NREs in these domains will facilitate the comparison of results between studies. METHODS: We conducted a literature search in PubMed, CINAHL, and EMBASE to identify studies related to NREs in high-risk domains and evaluated the methods used for event observation and description. We applied The Joint Commission on Accreditation of Healthcare Organization (JCAHO) taxonomy (cause, impact, domain, type, prevention, and mitigation) to the descriptions of NREs from the literature. RESULTS: We selected 25 articles that met inclusion criteria for review. Real-time documentation of NREs was more common than a retrospective video review. Thirteen studies used domain experts as observers and seven studies validated observations with interrater reliability. Using the JCAHO taxonomy, "cause" was the most frequently applied classification method, followed by "impact," "type," "domain," and "prevention and mitigation." CONCLUSIONS: NREs are frequent in high-risk medical settings. Strengths identified in several studies included the use of multiple observers with domain expertise and validation of the event ascertainment approach using interrater reliability. By applying the JCAHO taxonomy to the current literature, we provide an example of a structured approach that can be used for future analyses of NREs.

Assessment of Nonroutine Events During Intubation After Pediatric Trauma.

BACKGROUND: Intubation in the early postinjury phase can be a high-risk procedure associated with an increased risk of mortality when delayed. Nonroutine events (NREs) are workflow disruptions that can be latent safety threats in high-risk settings and may contribute to adverse outcomes. MATERIALS AND METHODS: We reviewed videos of intubations of injured children (age<17 y old) in the emergency department occurring between 2014 and 2018 to identify NREs occurring between the decision to intubate and successful intubation ("critical window"). RESULTS: Among 34 children requiring intubation, the indications included GCS≤8 (n = 20, 58.8%), cardiac arrest (n = 6, 17.6%), airway protection (n = 5, 14.7%), and respiratory failure (n = 3, 8.8%). The median duration of the "critical window" was 7.5 min (range 1.4-27.5 min), with a median of six NREs per case in this period (range 2-30). Most NREs (n = 159, 61.9%) delayed workflow, with 31 (12.1%) of these delays each lasting more than one minute. Eighty-seven NREs (33.9%) had a potential for harm but did not lead to direct patient harm. The most common NREs directly related to the intubation process were poor positioning for intubation (n = 23, 8.9%) and difficulty passing the endotracheal tube (n = 5, 1.9%), with most being attributed to the anesthesiologist performing the intubation (n = 51, range 0-7). CONCLUSIONS: Workflow disruptions related to nonroutine events were frequent during pediatric trauma intubation and were often associated with delays and potential for patient harm. Interventions for improving the efficiency and timeliness of the critical window should focus on adherence to intubation protocol and improving communication and teamwork related to tasks in this phase.

Understanding Digital Checklist Use Through Team Communication.

Introducing technology support in a complex, team-based work setting requires a study of teamwork effects on technology use. In this paper, we present our initial analysis of team communications in a trauma resuscitation setting, where we deployed a digital checklist to support trauma team leaders in guiding patient care. By analyzing speech transcripts, checklist interaction logs, and videos of 15 resuscitations, we identified several tensions that arose from the use of a checklist in a team-based process with multi-step tasks. The tensions included incorrect markings of in-progress tasks as completed, failure to mark completed tasks due to missed communications, failure to record planned tasks, and difficulties in recording dynamic values. From these findings, we discuss design implications for checklist design for dynamic, team-based activities.

Checklist Design Reconsidered: Understanding Checklist Compliance and Timing of Interactions.

We examine the association between user interactions with a checklist and task performance in a time-critical medical setting. By comparing 98 logs from a digital checklist for trauma resuscitation with activity logs generated by video review, we identified three non-compliant checklist use behaviors: failure to check items for completed tasks, falsely checking items when tasks were not performed, and inaccurately checking items for incomplete tasks. Using video review, we found that user perceptions of task completion were often misaligned with clinical practices that guided activity coding, thereby contributing to non-compliant check-offs. Our analysis of associations between different contexts and the timing of check-offs showed longer delays when (1) checklist users were absent during patient arrival, (2) patients had penetrating injuries, and (3) resuscitations were assigned to the highest acuity. We discuss opportunities for reconsidering checklist designs to reduce non-compliant checklist use.

Comparing the Effects of Paper and Digital Checklists on Team Performance in Time-Critical Work.

This mixed-methods study examines the effects of a tablet-based checklist system on team performance during a dynamic and safety-critical process of trauma resuscitation. We compared team performance from 47 resuscitations that used a paper checklist to that from 47 cases with a digital checklist to determine if digitizing a checklist led to improvements in task completion rates and in how fast the tasks were initiated for 18 most critical assessment and treatment tasks. We also compared if the checklist compliance increased with the digital design. We found that using the digital checklist led to more frequent completions of the initial airway assessment task but fewer completions of ear and lower extremities exams. We did not observe any significant differences in time to task performance, but found increased compliance with the checklist. Although improvements in team performance with the digital checklist were minor, our findings are important because they showed no adverse effects as a result of the digital checklist introduction. We conclude by discussing the takeaways and implications of these results for effective digitization of medical work.