Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms.

OBJECTIVE: To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications. MATERIALS AND METHODS: We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared. RESULTS: The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion. DISCUSSION: These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability. CONCLUSION: Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.

Eye tracking and audio sensors to evaluate surgeon's non-technical skills: An empirical study.

Non-Technical Skills (NTS) of medical teams are currently measured using subjective and resource-intensive ratings given by experts. This study explores if objective NTS assessment approaches with eye-tracking and audio sensors can measure teamwork and communication skills in surgery. Eight surgeons participated in a simulated two-phase surgical scenario developed to assess their NTS. Sensor-based audio, eye tracking and video data were collected and analyzed along with rating from the NOTSS scale. Different levels of communication were detected by the sensor data during the two phases of the simulated surgery. Sensor data detected leadership qualities among surgeons based on speech metrics, and eye tracking offered additional evidence about gaze patterns related to NTS. This objective approach to NTS measurement captured differences in communication in greater detail as opposed to a single collective rating obtained using current assessment tools.

Nontechnical Skills Assessment in Acute Care Trauma Simulations: A Mixed Methods Approach Using Eye Tracking and Behavioral Marker Systems.

INTRODUCTION: The condition of trauma patients and the urgent need for timely resuscitation present unique challenges to trauma teams. These difficulties are exacerbated for military trauma teams in combat environments. Consequently, there is a need for continued improvement of nontechnical skills (NTS) training for trauma teams. However, current approaches to NTS assessment rely on subjective ratings, which can introduce bias. Accordingly, objective methods of NTS evaluation are needed. Eye-tracking (ET) methods have been applied to studying communication, situation awareness, and leadership in many health care settings, and could be applied to studying physicians' NTS during trauma situations. In this study, we aimed to assess the relationship between trauma team leaders' objective gaze patterns and subjective expert NTS ratings during patient care simulations. MATERIALS AND METHODS: After Institutional Review Board approval, 9 trauma teams from first-year post-graduate general surgery and emergency medicine residents were recruited to participate in 1 of 2 trauma simulations (a difficult airway case and a multi-patient trauma). Each scenario lasted approximately 15 minutes. All team leaders wore a mobile ET system to evaluate gaze metrics-time to first fixation (TTFF), average fixation duration (AFD), and total percentage of the scenario (TPS) focused on Areas of Interest (AOI), which included patient, care team, diagnostic equipment, and patient care equipment. Trained faculty raters completed the Non-Technical Skills for Surgeons (NOTSS) assessment tool and the Trauma Non-Technical Skills (T-NOTECHS) scale. One-way analysis of variance, Kruskal-Wallis, and appropriate post-hoc pairwise comparison tests were run to assess differences between ET metrics across AOI groups. Spearman's Rho tests were used to assess correlations between ET and subjective NTS ratings. RESULTS: Compared to other NTS domains, trauma teams scored relatively poorly on communication across both T-NOTECHS (3.29$ \pm $0.61, maximum = 5) and NOTSS (2.87$ \pm $0.66, maximum = 4). We found significant differences in trauma team leaders' TTFF between teammates and the patient (Team: 1.56 vs Patient: 29.82 seconds, P < .001). TTFF on the diagnostic equipment was negatively correlated (P < .05) to multiple measures of subjective NTS assessments. There were no significant differences in AFD between AOIs, and AFD on teammates was positively correlated (P < .05) to communication and teamwork. There were significant differences in TPS across most AOI pairs (P < .05), and the average TPS fixated was highest on the patient (32%). Finally, there were several significant correlations between additional ET and NTS metrics. CONCLUSIONS: This study utilized a mixed methods approach to assess trauma team leaders' NTS in simulated acute care trauma simulations. Our results provide several objective insights into trauma team leaders' NTS behaviors during patient care simulations. Such objective insights provide a more comprehensive understanding of NTS behaviors and can be leveraged to guide NTS training of trauma physicians in the future. More studies are needed to apply these methods to capture NTS from a larger sample of teams in both simulated and real trauma environments.

A sensor-based framework for layout and workflow assessment in operating rooms.

Due to their large sizes and impediments to personnel workflows, integrating robotic technologies into the existing operating rooms (OR) is a challenge. In this study, we developed an ultra-wideband sensor-based human-machine-environment framework for layout and workflow assessments within the OR. In addition to providing best practices for use of the framework, we also demonstrated its effectiveness in understanding layout and workflow inefficiencies in 12 robotic-assisted surgeries (RAS) across 4 different surgical specialties. We found avoidable movements as the circulating nurse covers at least twice the distance of any other OR personnel before the patient cart (robot) is docked. OR areas of congestion and undesirable personnel-pair proximities across RAS phases that impose extra non-technical skill challenges were determined. Our findings highlight several implications for the added complexity of integrating robotic technologies into the OR, which can serve as drivers for objective evidence-based recommendations to combat RAS OR layout and workflow inefficiencies.

Quantifying the Impact of Infusion Alerts and Alarms on Nursing Workflows: A Retrospective Analysis.

BACKGROUND: Smart infusion pumps affect workflows as they add alerts and alarms in an information-rich clinical environment where alarm fatigue is already a major concern. An analytic approach is needed to quantify the impact of these alerts and alarms on nursing workflows and patient safety. OBJECTIVES: To analyze a detailed infusion dataset from a smart infusion pump system and identify contributing factors for infusion programming alerts, operational alarms, and alarm resolution times. METHODS: We analyzed detailed infusion pump data across four hospitals in a health system for up to 1 year. The prevalence of alerts and alarms was grouped by infusion type and a selected list of 32 high-alert medications (HAMs). Logistic regression was used to explore the relationship between a set of risk factors and the occurrence of alerts and alarms. We used nonparametric tests to explore the relationship between alarm resolution times and a subset of predictor variables. RESULTS: The study dataset included 745,641 unique infusions with a total of 3,231,300 infusion events. Overall, 28.7% of all unique infusions had at least one operational alarm, and 2.1% of all unique infusions had at least one programming alert. Alarms averaged two per infusion, whereas at least one alert happened in every 48 unique infusions. Eight percent of alarms took over 4 minutes to resolve. Intravenous fluid infusions had the highest rate of error-state occurrence. HAMs had 1.64 more odds for alerts than the rest of the infusions. On average, HAMs had a higher alert rate than maintenance fluids. CONCLUSION: Infusion pump alerts and alarms impact clinical care, as alerts and alarms by design interrupt clinical workflow. Our study showcases how hospital system leadership teams can leverage infusion pump informatics to prioritize quality improvement and patient safety initiatives pertaining to infusion practices.