Automating detection of diagnostic error of infectious diseases using machine learning.

Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.

A flexible framework for visualizing and exploring patient misdiagnosis over time.

Diagnosis is a complex and ambiguous process and yet, it is the critical hinge point for all subsequent clinical reasoning and decision-making. Tracking the quality of the patient diagnostic process has the potential to provide valuable insights in improving the diagnostic accuracy and to reduce downstream errors but needs to be informative, timely, and efficient at scale. However, due to the rate at which healthcare data are captured on a daily basis, manually reviewing the diagnostic history of each patient would be a severely taxing process without efficient data reduction and representation. Application of data visualization and visual analytics to healthcare data is one promising approach for addressing these challenges. This paper presents a novel flexible visualization and analysis framework for exploring the patient diagnostic process over time (i.e., patient diagnosis paths). Our framework allows users to select a specific set of patients, events and/or conditions, filter data based on different attributes, and view further details on the selected patient cohort while providing an interactive view of the resulting patient diagnosis paths. A practical demonstration of our system is presented with a case study exploring infection-based patient diagnosis paths.

Interactive Visualization and Exploration of Patient Progression in a Hospital Setting.

As medical organizations increasingly adopt the use of electronic health records (EHRs), large volumes of clinical data are being captured on a daily basis. These data provide comprehensive information about patients and have the potential to improve a wide range of application domains in healthcare. Physicians and clinical researchers are interested in finding effective ways to understand this abundance of data. Use of visual analytics to explore healthcare data is one such research direction. Here, we present a visualization and analysis environment to understand patient progression over time. Through the use of optimized data structures and progressive visualization techniques, we allow users to interactively explore how patients and their progression change over time. Compared to existing techniques, our work provides additional flexibility in analyzing patient data and has the potential to be used in a real-time hospital setting. Finally, we demonstrate the utility of our approach using a publicly available intensive care unit (ICU) database.