Computer clinical decision support that automates personalized clinical care: a challenging but needed healthcare delivery strategy.

How to deliver best care in various clinical settings remains a vexing problem. All pertinent healthcare-related questions have not, cannot, and will not be addressable with costly time- and resource-consuming controlled clinical trials. At present, evidence-based guidelines can address only a small fraction of the types of care that clinicians deliver. Furthermore, underserved areas rarely can access state-of-the-art evidence-based guidelines in real-time, and often lack the wherewithal to implement advanced guidelines. Care providers in such settings frequently do not have sufficient training to undertake advanced guideline implementation. Nevertheless, in advanced modern healthcare delivery environments, use of eActions (validated clinical decision support systems) could help overcome the cognitive limitations of overburdened clinicians. Widespread use of eActions will require surmounting current healthcare technical and cultural barriers and installing clinical evidence/data curation systems. The authors expect that increased numbers of evidence-based guidelines will result from future comparative effectiveness clinical research carried out during routine healthcare delivery within learning healthcare systems.

CheXED: Comparison of a Deep Learning Model to a Clinical Decision Support System for Pneumonia in the Emergency Department.

PURPOSE: Patients with pneumonia often present to the emergency department (ED) and require prompt diagnosis and treatment. Clinical decision support systems for the diagnosis and management of pneumonia are commonly utilized in EDs to improve patient care. The purpose of this study is to investigate whether a deep learning model for detecting radiographic pneumonia and pleural effusions can improve functionality of a clinical decision support system (CDSS) for pneumonia management (ePNa) operating in 20 EDs. MATERIALS AND METHODS: In this retrospective cohort study, a dataset of 7434 prior chest radiographic studies from 6551 ED patients was used to develop and validate a deep learning model to identify radiographic pneumonia, pleural effusions, and evidence of multilobar pneumonia. Model performance was evaluated against 3 radiologists' adjudicated interpretation and compared with performance of the natural language processing of radiology reports used by ePNa. RESULTS: The deep learning model achieved an area under the receiver operating characteristic curve of 0.833 (95% confidence interval [CI]: 0.795, 0.868) for detecting radiographic pneumonia, 0.939 (95% CI: 0.911, 0.962) for detecting pleural effusions and 0.847 (95% CI: 0.800, 0.890) for identifying multilobar pneumonia. On all 3 tasks, the model achieved higher agreement with the adjudicated radiologist interpretation compared with ePNa. CONCLUSIONS: A deep learning model demonstrated higher agreement with radiologists than the ePNa CDSS in detecting radiographic pneumonia and related findings. Incorporating deep learning models into pneumonia CDSS could enhance diagnostic performance and improve pneumonia management.

Contemporary clinical decision support standards using Health Level Seven International Fast Healthcare Interoperability Resources.

OBJECTIVE: To facilitate the development of standards-based clinical decision support (CDS) systems, we review the current set of CDS standards that are based on Health Level Seven International Fast Healthcare Interoperability Resources (FHIR). Widespread adoption of these standards may help reduce healthcare variability, improve healthcare quality, and improve patient safety. TARGET AUDIENCE: This tutorial is designed for the broad informatics community, some of whom may be unfamiliar with the current, FHIR-based CDS standards. SCOPE: This tutorial covers the following standards: Arden Syntax (using FHIR as the data model), Clinical Quality Language, FHIR Clinical Reasoning, SMART on FHIR, and CDS Hooks. Detailed descriptions and selected examples are provided.

Enabling a learning healthcare system with automated computer protocols that produce replicable and personalized clinician actions.

Clinical decision-making is based on knowledge, expertise, and authority, with clinicians approving almost every intervention-the starting point for delivery of "All the right care, but only the right care," an unachieved healthcare quality improvement goal. Unaided clinicians suffer from human cognitive limitations and biases when decisions are based only on their training, expertise, and experience. Electronic health records (EHRs) could improve healthcare with robust decision-support tools that reduce unwarranted variation of clinician decisions and actions. Current EHRs, focused on results review, documentation, and accounting, are awkward, time-consuming, and contribute to clinician stress and burnout. Decision-support tools could reduce clinician burden and enable replicable clinician decisions and actions that personalize patient care. Most current clinical decision-support tools or aids lack detail and neither reduce burden nor enable replicable actions. Clinicians must provide subjective interpretation and missing logic, thus introducing personal biases and mindless, unwarranted, variation from evidence-based practice. Replicability occurs when different clinicians, with the same patient information and context, come to the same decision and action. We propose a feasible subset of therapeutic decision-support tools based on credible clinical outcome evidence: computer protocols leading to replicable clinician actions (eActions). eActions enable different clinicians to make consistent decisions and actions when faced with the same patient input data. eActions embrace good everyday decision-making informed by evidence, experience, EHR data, and individual patient status. eActions can reduce unwarranted variation, increase quality of clinical care and research, reduce EHR noise, and could enable a learning healthcare system.

The design and evaluation of a Bayesian system for detecting and characterizing outbreaks of influenza.

The prediction and characterization of outbreaks of infectious diseases such as influenza remains an open and important problem. This paper describes a framework for detecting and characterizing outbreaks of influenza and the results of testing it on data from ten outbreaks collected from two locations over five years. We model outbreaks with compartment models and explicitly model non-influenza influenza-like illnesses.

Implementation of Real-Time Electronic Clinical Decision Support for Emergency Department Patients with Pneumonia Across a Healthcare System.

A real-time electronic CDS for pneumonia (ePNa) identifies possible pneumonia patients, measures severity and antimicrobial resistance risk, and then recommends disposition, antibiotics, and microbiology studies. Use is voluntary, and clinicians may modify treatment recommendations. ePNa was associated with lower mortality in emergency department (ED) patients versus usual care (Annals EM 66:511). We adapted ePNa for the Cerner EHR, and implemented it across Intermountain Healthcare EDs (Utah, USA) throughout 2018. We introduced ePNa through didactic, interactive presentations to ED clinicians; follow-up visits identified barriers and facilitators to use. Email reminded clinicians and answered questions. Hospital admitting clinicians encouraged ePNa use to smooth care transitions. Audit-and-feedback measured utilization, showing variations from best practice when ePNa and associated electronic order sets were not used. Use was initially low, but gradually increased especially at larger hospitals. A user-friendly interface, frequent reminders, audit-and- feedback, a user survey, a nurse educator, and local physician champions are additive towards implementation success.

Promoting national and international standards to build interoperable clinical applications.

During the last decade, software supporting healthcare delivery has proliferated. This software can be divided into electronic medical record (EHR) systems and applications that treat EHRs as platforms. These collect, manage, and interpret medical data, thereby adding value to associated EHRs. To reduce the burden of developing for multiple EHR platforms, a group of standards has evolved that allow software written for one vendor's EHR to be introduced into settings supported by other vendors. The Health Services Platform Consortium (HSPC) is a collaborative effort to advocate for standards that will make healthcare applications truly interoperable. In this document, we discuss the approach adopted by the consortium and the standards central to this approach. We discriminate between interoperability standards that support the plug-and-play transfer of applications from one vendor's EHR to another and knowledge portability standards that allow knowledge artifacts used in one software environment to be introduced effectively in others.

Enhancing a Commercial EMR with an Open, Standards-Based Publish-Subscribe Infrastructure.

Intermountain Healthcare has designed and implemented a publish-subscribe (PubSub) infrastructure to support essential event processing workflows across our organization. A recent implementation of a commercial EMR highlighted the need to provide this capability on top of the EMR to support external applications and services that require access to triggering events within the EMR. A description of the PubSub architecture is presented. Use cases for health information exchange, public health reporting, and pulmonary embolism diagnosis that utilize PubSub are described, along with benefits of using the paradigm. Besides providing support for these external applications, the PubSub infrastructure allows additional event handling functionality not available in the commercial EMR. The open, standards-based nature of the design should allow other organizations to implement the system in their information systems environment.

A Bayesian system to detect and characterize overlapping outbreaks.

Outbreaks of infectious diseases such as influenza are a significant threat to human health. Because there are different strains of influenza which can cause independent outbreaks, and influenza can affect demographic groups at different rates and times, there is a need to recognize and characterize multiple outbreaks of influenza. This paper describes a Bayesian system that uses data from emergency department patient care reports to create epidemiological models of overlapping outbreaks of influenza. Clinical findings are extracted from patient care reports using natural language processing. These findings are analyzed by a case detection system to create disease likelihoods that are passed to a multiple outbreak detection system. We evaluated the system using real and simulated outbreaks. The results show that this approach can recognize and characterize overlapping outbreaks of influenza. We describe several extensions that appear promising.

Automating Construction of Machine Learning Models With Clinical Big Data: Proposal Rationale and Methods.

BACKGROUND: To improve health outcomes and cut health care costs, we often need to conduct prediction/classification using large clinical datasets (aka, clinical big data), for example, to identify high-risk patients for preventive interventions. Machine learning has been proposed as a key technology for doing this. Machine learning has won most data science competitions and could support many clinical activities, yet only 15% of hospitals use it for even limited purposes. Despite familiarity with data, health care researchers often lack machine learning expertise to directly use clinical big data, creating a hurdle in realizing value from their data. Health care researchers can work with data scientists with deep machine learning knowledge, but it takes time and effort for both parties to communicate effectively. Facing a shortage in the United States of data scientists and hiring competition from companies with deep pockets, health care systems have difficulty recruiting data scientists. Building and generalizing a machine learning model often requires hundreds to thousands of manual iterations by data scientists to select the following: (1) hyper-parameter values and complex algorithms that greatly affect model accuracy and (2) operators and periods for temporally aggregating clinical attributes (eg, whether a patient's weight kept rising in the past year). This process becomes infeasible with limited budgets. OBJECTIVE: This study's goal is to enable health care researchers to directly use clinical big data, make machine learning feasible with limited budgets and data scientist resources, and realize value from data. METHODS: This study will allow us to achieve the following: (1) finish developing the new software, Automated Machine Learning (Auto-ML), to automate model selection for machine learning with clinical big data and validate Auto-ML on seven benchmark modeling problems of clinical importance; (2) apply Auto-ML and novel methodology to two new modeling problems crucial for care management allocation and pilot one model with care managers; and (3) perform simulations to estimate the impact of adopting Auto-ML on US patient outcomes. RESULTS: We are currently writing Auto-ML's design document. We intend to finish our study by around the year 2022. CONCLUSIONS: Auto-ML will generalize to various clinical prediction/classification problems. With minimal help from data scientists, health care researchers can use Auto-ML to quickly build high-quality models. This will boost wider use of machine learning in health care and improve patient outcomes.

The effects of natural language processing on cross-institutional portability of influenza case detection for disease surveillance.

OBJECTIVES: This study evaluates the accuracy and portability of a natural language processing (NLP) tool for extracting clinical findings of influenza from clinical notes across two large healthcare systems. Effectiveness is evaluated on how well NLP supports downstream influenza case-detection for disease surveillance. METHODS: We independently developed two NLP parsers, one at Intermountain Healthcare (IH) in Utah and the other at University of Pittsburgh Medical Center (UPMC) using local clinical notes from emergency department (ED) encounters of influenza. We measured NLP parser performance for the presence and absence of 70 clinical findings indicative of influenza. We then developed Bayesian network models from NLP processed reports and tested their ability to discriminate among cases of (1) influenza, (2) non-influenza influenza-like illness (NI-ILI), and (3) 'other' diagnosis. RESULTS: On Intermountain Healthcare reports, recall and precision of the IH NLP parser were 0.71 and 0.75, respectively, and UPMC NLP parser, 0.67 and 0.79. On University of Pittsburgh Medical Center reports, recall and precision of the UPMC NLP parser were 0.73 and 0.80, respectively, and IH NLP parser, 0.53 and 0.80. Bayesian case-detection performance measured by AUROC for influenza versus non-influenza on Intermountain Healthcare cases was 0.93 (using IH NLP parser) and 0.93 (using UPMC NLP parser). Case-detection on University of Pittsburgh Medical Center cases was 0.95 (using UPMC NLP parser) and 0.83 (using IH NLP parser). For influenza versus NI-ILI on Intermountain Healthcare cases performance was 0.70 (using IH NLP parser) and 0.76 (using UPMC NLP parser). On University of Pisstburgh Medical Center cases, 0.76 (using UPMC NLP parser) and 0.65 (using IH NLP parser). CONCLUSION: In all but one instance (influenza versus NI-ILI using IH cases), local parsers were more effective at supporting case-detection although performances of non-local parsers were reasonable.

Using classification models for the generation of disease-specific medications from biomedical literature and clinical data repository.

OBJECTIVE: Mining disease-specific associations from existing knowledge resources can be useful for building disease-specific ontologies and supporting knowledge-based applications. Many association mining techniques have been exploited. However, the challenge remains when those extracted associations contained much noise. It is unreliable to determine the relevance of the association by simply setting up arbitrary cut-off points on multiple scores of relevance; and it would be expensive to ask human experts to manually review a large number of associations. We propose that machine-learning-based classification can be used to separate the signal from the noise, and to provide a feasible approach to create and maintain disease-specific vocabularies. METHOD: We initially focused on disease-medication associations for the purpose of simplicity. For a disease of interest, we extracted potentially treatment-related drug concepts from biomedical literature citations and from a local clinical data repository. Each concept was associated with multiple measures of relevance (i.e., features) such as frequency of occurrence. For the machine purpose of learning, we formed nine datasets for three diseases with each disease having two single-source datasets and one from the combination of previous two datasets. All the datasets were labeled using existing reference standards. Thereafter, we conducted two experiments: (1) to test if adding features from the clinical data repository would improve the performance of classification achieved using features from the biomedical literature only, and (2) to determine if classifier(s) trained with known medication-disease data sets would be generalizable to new disease(s). RESULTS: Simple logistic regression and LogitBoost were two classifiers identified as the preferred models separately for the biomedical-literature datasets and combined datasets. The performance of the classification using combined features provided significant improvement beyond that using biomedical-literature features alone (p-value<0.001). The performance of the classifier built from known diseases to predict associated concepts for new diseases showed no significant difference from the performance of the classifier built and tested using the new disease's dataset. CONCLUSION: It is feasible to use classification approaches to automatically predict the relevance of a concept to a disease of interest. It is useful to combine features from disparate sources for the task of classification. Classifiers built from known diseases were generalizable to new diseases.

A study of the transferability of influenza case detection systems between two large healthcare systems.

OBJECTIVES: This study evaluates the accuracy and transferability of Bayesian case detection systems (BCD) that use clinical notes from emergency department (ED) to detect influenza cases. METHODS: A BCD uses natural language processing (NLP) to infer the presence or absence of clinical findings from ED notes, which are fed into a Bayesain network classifier (BN) to infer patients' diagnoses. We developed BCDs at the University of Pittsburgh Medical Center (BCDUPMC) and Intermountain Healthcare in Utah (BCDIH). At each site, we manually built a rule-based NLP and trained a Bayesain network classifier from over 40,000 ED encounters between Jan. 2008 and May. 2010 using feature selection, machine learning, and expert debiasing approach. Transferability of a BCD in this study may be impacted by seven factors: development (source) institution, development parser, application (target) institution, application parser, NLP transfer, BN transfer, and classification task. We employed an ANOVA analysis to study their impacts on BCD performance. RESULTS: Both BCDs discriminated well between influenza and non-influenza on local test cases (AUCs > 0.92). When tested for transferability using the other institution's cases, BCDUPMC discriminations declined minimally (AUC decreased from 0.95 to 0.94, p<0.01), and BCDIH discriminations declined more (from 0.93 to 0.87, p<0.0001). We attributed the BCDIH decline to the lower recall of the IH parser on UPMC notes. The ANOVA analysis showed five significant factors: development parser, application institution, application parser, BN transfer, and classification task. CONCLUSION: We demonstrated high influenza case detection performance in two large healthcare systems in two geographically separated regions, providing evidentiary support for the use of automated case detection from routinely collected electronic clinical notes in national influenza surveillance. The transferability could be improved by training Bayesian network classifier locally and increasing the accuracy of the NLP parser.

Generating disease-pertinent treatment vocabularies from MEDLINE citations.

OBJECTIVE: Healthcare communities have identified a significant need for disease-specific information. Disease-specific ontologies are useful in assisting the retrieval of disease-relevant information from various sources. However, building these ontologies is labor intensive. Our goal is to develop a system for an automated generation of disease-pertinent concepts from a popular knowledge resource for the building of disease-specific ontologies. METHODS: A pipeline system was developed with an initial focus of generating disease-specific treatment vocabularies. It was comprised of the components of disease-specific citation retrieval, predication extraction, treatment predication extraction, treatment concept extraction, and relevance ranking. A semantic schema was developed to support the extraction of treatment predications and concepts. Four ranking approaches (i.e., occurrence, interest, degree centrality, and weighted degree centrality) were proposed to measure the relevance of treatment concepts to the disease of interest. We measured the performance of four ranks in terms of the mean precision at the top 100 concepts with five diseases, as well as the precision-recall curves against two reference vocabularies. The performance of the system was also compared to two baseline approaches. RESULTS: The pipeline system achieved a mean precision of 0.80 for the top 100 concepts with the ranking by interest. There were no significant different among the four ranks (p=0.53). However, the pipeline-based system had significantly better performance than the two baselines. CONCLUSIONS: The pipeline system can be useful for an automated generation of disease-relevant treatment concepts from the biomedical literature.

A method for the development of disease-specific reference standards vocabularies from textual biomedical literature resources.

OBJECTIVE: Disease-specific vocabularies are fundamental to many knowledge-based intelligent systems and applications like text annotation, cohort selection, disease diagnostic modeling, and therapy recommendation. Reference standards are critical in the development and validation of automated methods for disease-specific vocabularies. The goal of the present study is to design and test a generalizable method for the development of vocabulary reference standards from expert-curated, disease-specific biomedical literature resources. METHODS: We formed disease-specific corpora from literature resources like textbooks, evidence-based synthesized online sources, clinical practice guidelines, and journal articles. Medical experts annotated and adjudicated disease-specific terms in four classes (i.e., causes or risk factors, signs or symptoms, diagnostic tests or results, and treatment). Annotations were mapped to UMLS concepts. We assessed source variation, the contribution of each source to build disease-specific vocabularies, the saturation of the vocabularies with respect to the number of used sources, and the generalizability of the method with different diseases. RESULTS: The study resulted in 2588 string-unique annotations for heart failure in four classes, and 193 and 425 respectively for pulmonary embolism and rheumatoid arthritis in treatment class. Approximately 80% of the annotations were mapped to UMLS concepts. The agreement among heart failure sources ranged between 0.28 and 0.46. The contribution of these sources to the final vocabulary ranged between 18% and 49%. With the sources explored, the heart failure vocabulary reached near saturation in all four classes with the inclusion of minimal six sources (or between four to seven sources if only counting terms occurred in two or more sources). It took fewer sources to reach near saturation for the other two diseases in terms of the treatment class. CONCLUSIONS: We developed a method for the development of disease-specific reference vocabularies. Expert-curated biomedical literature resources are substantial for acquiring disease-specific medical knowledge. It is feasible to reach near saturation in a disease-specific vocabulary using a relatively small number of literature sources.

An in silico method to identify computer-based protocols worthy of clinical study: An insulin infusion protocol use case.

OBJECTIVE: Develop an efficient non-clinical method for identifying promising computer-based protocols for clinical study. An in silico comparison can provide information that informs the decision to proceed to a clinical trial. The authors compared two existing computer-based insulin infusion protocols: eProtocol-insulin from Utah, USA, and Glucosafe from Denmark. MATERIALS AND METHODS: The authors used eProtocol-insulin to manage intensive care unit (ICU) hyperglycemia with intravenous (IV) insulin from 2004 to 2010. Recommendations accepted by the bedside clinicians directly link the subsequent blood glucose values to eProtocol-insulin recommendations and provide a unique clinical database. The authors retrospectively compared in silico 18,984 eProtocol-insulin continuous IV insulin infusion rate recommendations from 408 ICU patients with those of Glucosafe, the candidate computer-based protocol. The subsequent blood glucose measurement value (low, on target, high) was used to identify if the insulin recommendation was too high, on target, or too low. RESULTS: Glucosafe consistently provided more favorable continuous IV insulin infusion rate recommendations than eProtocol-insulin for on target (64% of comparisons), low (80% of comparisons), or high (70% of comparisons) blood glucose. Aggregated eProtocol-insulin and Glucosafe continuous IV insulin infusion rates were clinically similar though statistically significantly different (Wilcoxon signed rank test P = .01). In contrast, when stratified by low, on target, or high subsequent blood glucose measurement, insulin infusion rates from eProtocol-insulin and Glucosafe were statistically significantly different (Wilcoxon signed rank test, P < .001), and clinically different. DISCUSSION: This in silico comparison appears to be an efficient nonclinical method for identifying promising computer-based protocols. CONCLUSION: Preclinical in silico comparison analytical framework allows rapid and inexpensive identification of computer-based protocol care strategies that justify expensive and burdensome clinical trials.

Impact of an Electronic Clinical Decision Support Tool for Emergency Department Patients With Pneumonia.

STUDY OBJECTIVE: Despite evidence that guideline adherence improves clinical outcomes, management of pneumonia patients varies in emergency departments (EDs). We study the effect of a real-time, ED, electronic clinical decision support tool that provides clinicians with guideline-recommended decision support for diagnosis, severity assessment, disposition, and antibiotic selection. METHODS: This was a prospective, controlled, quasi-experimental trial in 7 Intermountain Healthcare hospital EDs in Utah's urban corridor. We studied adults with International Classification of Diseases, Ninth Revision codes and radiographic evidence for pneumonia during 2 periods: baseline (December 2009 through November 2010) and post-tool deployment (December 2011 through November 2012). The tool was deployed at 4 intervention EDs in May 2011, leaving 3 as usual care controls. We compared 30-day, all-cause mortality adjusted for illness severity, using a mixed-effect, logistic regression model. RESULTS: The study population comprised 4,758 ED pneumonia patients; 14% had health care-associated pneumonia. Median age was 58 years, 53% were female patients, and 59% were admitted to the hospital. Physicians applied the tool for 62.6% of intervention ED study patients. There was no difference overall in severity-adjusted mortality between intervention and usual care EDs post-tool deployment (odds ratio [OR]=0.69; 95% confidence interval [CI] 0.41 to 1.16). Post hoc analysis showed that patients with community-acquired pneumonia experienced significantly lower mortality (OR=0.53; 95% CI 0.28 to 0.99), whereas mortality was unchanged among patients with health care-associated pneumonia (OR=1.12; 95% CI 0.45 to 2.8). Patient disposition from the ED postdeployment adhered more to tool recommendations. CONCLUSION: This study demonstrates the feasibility and potential benefit of real-time electronic clinical decision support for ED pneumonia patients.

Developing a section labeler for clinical documents.

Natural language processing (NLP) technologies provide an opportunity to extract key patient data from free text documents within the electronic health record (EHR). We are developing a series of components from which to construct NLP pipelines. These pipelines typically begin with a component whose goal is to label sections within medical documents with codes indicating the anticipated semantics of their content. This Clinical Section Labeler prepares the document for further, focused information extraction. Below we describe the evaluation of six algorithms designed for use in a Clinical Section Labeler. These algorithms are trained with N-gram-based feature sets extracted from document sections and the document types. In the evaluation, 6 different Bayesian models were trained and used to assign one of 27 different topics to each section. A tree-augmented Bayesian network using the document type and N-grams derived from section headers proved most accurate in assigning individual sections appropriate section topics.

Bias in recording of body mass index data in the electronic health record.

The relationship between patient disease status and the presence or absence of body mass index (BMI) data in the electronic health record (EHR) has not been characterized. We conducted a descriptive study of the completeness of BMI data for three patient cohorts. Cross-sectional descriptions of BMI presence rates per patient were compared between a cohort having at least one ICD-9-CM code for diabetes mellitus (DM) versus a cohort with no diagnosis constraints. Conversely, frequencies of encounter diagnoses were compared among subgroups having BMI recorded or not in both cohorts described and a third cohort having DM codes from a separate organization's EHR. The data demonstrate a correlation with presence of BMI and higher disease status. This effect may bias the cohort average BMIs, which appear higher than expected. When EHR BMI data are repurposed for research, biases in the selective recording of BMI may affect the results.