User-Centered Evaluation of a Visual Annotation Tool for Rapid Assessment of Pediatric Weight Entry Errors.

Weight entry errors can cause significant patient harm in pediatrics due to pervasive weight-based dosing practices. While computerized algorithms can assist in error detection, they have not achieved high sensitivity and specificity to be further developed as a clinical decision support tool. To train an advanced algorithm, expert-annotated weight errors are essential but difficult to collect. In this study, we developed a visual annotation tool to gather large amounts of expertly annotated pediatric weight charts and conducted a formal user-centered evaluation. Key features of the tool included configurable grid sizes and annotation styles. The user feedback was collected through a structured survey and user clicks on the interface. The results show that the visual annotation tool has high usability (average SUS=86.4). Different combinations of the key features, however, did not significantly improve the annotation efficiency and duration. We have used this tool to collect expert annotations for algorithm development and benchmarking.

The Effects of Medication Alerts on Prescriber Response in a Pediatric Hospital.

OBJECTIVE: More than 70% of hospitals in the United States have electronic health records (EHRs). Clinical decision support (CDS) presents clinicians with electronic alerts during the course of patient care; however, alert fatigue can influence a provider's response to any EHR alert. The primary goal was to evaluate the effects of alert burden on user response to the alerts. METHODS: We performed a retrospective study of medication alerts over a 24-month period (1/2013-12/2014) in a large pediatric academic medical center. The institutional review board approved this study. The primary outcome measure was alert salience, a measure of whether or not the prescriber took any corrective action on the order that generated an alert. We estimated the ideal number of alerts to maximize salience. Salience rates were examined for providers at each training level, by day of week, and time of day through logistic regressions. RESULTS: While salience never exceeded 38%, 49 alerts/day were associated with maximal salience in our dataset. The time of day an order was placed was associated with alert salience (maximal salience 2am). The day of the week was also associated with alert salience (maximal salience on Wednesday). Provider role did not have an impact on salience. CONCLUSION: Alert burden plays a role in influencing provider response to medication alerts. An increased number of alerts a provider saw during a one-day period did not directly lead to decreased response to alerts. Given the multiple factors influencing the response to alerts, efforts focused solely on burden are not likely to be effective.