User-Centered Evaluation of a Visual Annotation Tool for Rapid Assessment of Pediatric Weight Entry Errors.

Weight entry errors can cause significant patient harm in pediatrics due to pervasive weight-based dosing practices. While computerized algorithms can assist in error detection, they have not achieved high sensitivity and specificity to be further developed as a clinical decision support tool. To train an advanced algorithm, expert-annotated weight errors are essential but difficult to collect. In this study, we developed a visual annotation tool to gather large amounts of expertly annotated pediatric weight charts and conducted a formal user-centered evaluation. Key features of the tool included configurable grid sizes and annotation styles. The user feedback was collected through a structured survey and user clicks on the interface. The results show that the visual annotation tool has high usability (average SUS=86.4). Different combinations of the key features, however, did not significantly improve the annotation efficiency and duration. We have used this tool to collect expert annotations for algorithm development and benchmarking.

An interactive online dashboard for tracking COVID-19 in U.S. counties, cities, and states in real time.

OBJECTIVE: The study sought to create an online resource that informs the public of coronavirus disease 2019 (COVID-19) outbreaks in their area. MATERIALS AND METHODS: This R Shiny application aggregates data from multiple resources that track COVID-19 and visualizes them through an interactive, online dashboard. RESULTS: The Web resource, called the COVID-19 Watcher, can be accessed online (https://covid19watcher.research.cchmc.org/). It displays COVID-19 data from every county and 188 metropolitan areas in the United States. Features include rankings of the worst-affected areas and auto-generating plots that depict temporal changes in testing capacity, cases, and deaths. DISCUSSION: The Centers for Disease Control and Prevention does not publish COVID-19 data for local municipalities, so it is critical that academic resources fill this void so the public can stay informed. The data used have limitations and likely underestimate the scale of the outbreak. CONCLUSIONS: The COVID-19 Watcher can provide the public with real-time updates of outbreaks in their area.

Feedback at the Point of Care to Decrease Medication Alert Rates in an Electronic Health Record.

Frequently overridden alerts in the electronic health record can highlight alerts that may need revision. This method is a way of fine-tuning clinical decision support. We evaluated the feasibility of a complementary, yet different method that directly involved pediatric emergency department (PED) providers in identifying additional medication alerts that were potentially incorrect or intrusive. We then evaluated the effect subsequent resulting modifications had on alert salience. METHODS: We performed a prospective, interventional study over 34 months (March 6, 2014, to December 31, 2016) in the PED. We implemented a passive alert feedback mechanism by enhancing the native electronic health record functionality on alert reviews. End-users flagged potentially incorrect/bothersome alerts for review by the study's team. The alerts were updated when clinically appropriate and trends of the impact were evaluated. RESULTS: More than 200 alerts were reported from both inside and outside the PED, suggesting an intuitive approach. On average, we processed 4 reviews per week from the PED, with attending physicians as major contributors. The general trend of the impact of these changes seems favorable. DISCUSSION: The implementation of the review mechanism for user-selected alerts was intuitive and sustainable and seems to be able to detect alerts that are bothersome to the end-users. The method should be run in parallel with the traditional data-driven approach to support capturing of inaccurate alerts. CONCLUSIONS: User-centered, context-specific alert feedback can be used for selecting suboptimal, interruptive medication alerts.

Development and Preliminary Evaluation of a Visual Annotation Tool to Rapidly Collect Expert-Annotated Weight Errors in Pediatric Growth Charts.

Patient weights can be entered incorrectly into electronic health record (EHR) systems. These weight errors can cause significant patient harm especially in pediatrics where weight-based dosing is pervasively used. Determining weight errors through manual chart reviews is impractical in busy clinics, and current EHR alerts are rudimentary. To address these issues, we seek to develop an advanced algorithm to detect weight errors using supervised machine learning techniques. The critical first step is to collect labelled weight errors for algorithm training. In this paper, we designed and preliminarily evaluated a visual annotation tool using Agile software development to achieve the goal of supporting the rapid collection of expert-annotated weight errors. The design was based on the fact that weight errors are infrequent and medical experts can easily spot potential errors. The results show positive user feedback and prepared us for the formal user-centered evaluation as the next step.