Using classification models for the generation of disease-specific medications from biomedical literature and clinical data repository.

OBJECTIVE: Mining disease-specific associations from existing knowledge resources can be useful for building disease-specific ontologies and supporting knowledge-based applications. Many association mining techniques have been exploited. However, the challenge remains when those extracted associations contained much noise. It is unreliable to determine the relevance of the association by simply setting up arbitrary cut-off points on multiple scores of relevance; and it would be expensive to ask human experts to manually review a large number of associations. We propose that machine-learning-based classification can be used to separate the signal from the noise, and to provide a feasible approach to create and maintain disease-specific vocabularies. METHOD: We initially focused on disease-medication associations for the purpose of simplicity. For a disease of interest, we extracted potentially treatment-related drug concepts from biomedical literature citations and from a local clinical data repository. Each concept was associated with multiple measures of relevance (i.e., features) such as frequency of occurrence. For the machine purpose of learning, we formed nine datasets for three diseases with each disease having two single-source datasets and one from the combination of previous two datasets. All the datasets were labeled using existing reference standards. Thereafter, we conducted two experiments: (1) to test if adding features from the clinical data repository would improve the performance of classification achieved using features from the biomedical literature only, and (2) to determine if classifier(s) trained with known medication-disease data sets would be generalizable to new disease(s). RESULTS: Simple logistic regression and LogitBoost were two classifiers identified as the preferred models separately for the biomedical-literature datasets and combined datasets. The performance of the classification using combined features provided significant improvement beyond that using biomedical-literature features alone (p-value<0.001). The performance of the classifier built from known diseases to predict associated concepts for new diseases showed no significant difference from the performance of the classifier built and tested using the new disease's dataset. CONCLUSION: It is feasible to use classification approaches to automatically predict the relevance of a concept to a disease of interest. It is useful to combine features from disparate sources for the task of classification. Classifiers built from known diseases were generalizable to new diseases.

Generating disease-pertinent treatment vocabularies from MEDLINE citations.

OBJECTIVE: Healthcare communities have identified a significant need for disease-specific information. Disease-specific ontologies are useful in assisting the retrieval of disease-relevant information from various sources. However, building these ontologies is labor intensive. Our goal is to develop a system for an automated generation of disease-pertinent concepts from a popular knowledge resource for the building of disease-specific ontologies. METHODS: A pipeline system was developed with an initial focus of generating disease-specific treatment vocabularies. It was comprised of the components of disease-specific citation retrieval, predication extraction, treatment predication extraction, treatment concept extraction, and relevance ranking. A semantic schema was developed to support the extraction of treatment predications and concepts. Four ranking approaches (i.e., occurrence, interest, degree centrality, and weighted degree centrality) were proposed to measure the relevance of treatment concepts to the disease of interest. We measured the performance of four ranks in terms of the mean precision at the top 100 concepts with five diseases, as well as the precision-recall curves against two reference vocabularies. The performance of the system was also compared to two baseline approaches. RESULTS: The pipeline system achieved a mean precision of 0.80 for the top 100 concepts with the ranking by interest. There were no significant different among the four ranks (p=0.53). However, the pipeline-based system had significantly better performance than the two baselines. CONCLUSIONS: The pipeline system can be useful for an automated generation of disease-relevant treatment concepts from the biomedical literature.

A Bayesian system to detect and characterize overlapping outbreaks.

Outbreaks of infectious diseases such as influenza are a significant threat to human health. Because there are different strains of influenza which can cause independent outbreaks, and influenza can affect demographic groups at different rates and times, there is a need to recognize and characterize multiple outbreaks of influenza. This paper describes a Bayesian system that uses data from emergency department patient care reports to create epidemiological models of overlapping outbreaks of influenza. Clinical findings are extracted from patient care reports using natural language processing. These findings are analyzed by a case detection system to create disease likelihoods that are passed to a multiple outbreak detection system. We evaluated the system using real and simulated outbreaks. The results show that this approach can recognize and characterize overlapping outbreaks of influenza. We describe several extensions that appear promising.

Impact of an Electronic Clinical Decision Support Tool for Emergency Department Patients With Pneumonia.

STUDY OBJECTIVE: Despite evidence that guideline adherence improves clinical outcomes, management of pneumonia patients varies in emergency departments (EDs). We study the effect of a real-time, ED, electronic clinical decision support tool that provides clinicians with guideline-recommended decision support for diagnosis, severity assessment, disposition, and antibiotic selection. METHODS: This was a prospective, controlled, quasi-experimental trial in 7 Intermountain Healthcare hospital EDs in Utah's urban corridor. We studied adults with International Classification of Diseases, Ninth Revision codes and radiographic evidence for pneumonia during 2 periods: baseline (December 2009 through November 2010) and post-tool deployment (December 2011 through November 2012). The tool was deployed at 4 intervention EDs in May 2011, leaving 3 as usual care controls. We compared 30-day, all-cause mortality adjusted for illness severity, using a mixed-effect, logistic regression model. RESULTS: The study population comprised 4,758 ED pneumonia patients; 14% had health care-associated pneumonia. Median age was 58 years, 53% were female patients, and 59% were admitted to the hospital. Physicians applied the tool for 62.6% of intervention ED study patients. There was no difference overall in severity-adjusted mortality between intervention and usual care EDs post-tool deployment (odds ratio [OR]=0.69; 95% confidence interval [CI] 0.41 to 1.16). Post hoc analysis showed that patients with community-acquired pneumonia experienced significantly lower mortality (OR=0.53; 95% CI 0.28 to 0.99), whereas mortality was unchanged among patients with health care-associated pneumonia (OR=1.12; 95% CI 0.45 to 2.8). Patient disposition from the ED postdeployment adhered more to tool recommendations. CONCLUSION: This study demonstrates the feasibility and potential benefit of real-time electronic clinical decision support for ED pneumonia patients.

Natural Language Processing to identify pneumonia from radiology reports.

PURPOSE: This study aimed to develop Natural Language Processing (NLP) approaches to supplement manual outcome validation, specifically to validate pneumonia cases from chest radiograph reports. METHODS: We trained one NLP system, ONYX, using radiograph reports from children and adults that were previously manually reviewed. We then assessed its validity on a test set of 5000 reports. We aimed to substantially decrease manual review, not replace it entirely, and so, we classified reports as follows: (1) consistent with pneumonia; (2) inconsistent with pneumonia; or (3) requiring manual review because of complex features. We developed processes tailored either to optimize accuracy or to minimize manual review. Using logistic regression, we jointly modeled sensitivity and specificity of ONYX in relation to patient age, comorbidity, and care setting. We estimated positive and negative predictive value (PPV and NPV) assuming pneumonia prevalence in the source data. RESULTS: Tailored for accuracy, ONYX identified 25% of reports as requiring manual review (34% of true pneumonias and 18% of non-pneumonias). For the remainder, ONYX's sensitivity was 92% (95% CI 90-93%), specificity 87% (86-88%), PPV 74% (72-76%), and NPV 96% (96-97%). Tailored to minimize manual review, ONYX classified 12% as needing manual review. For the remainder, ONYX had sensitivity 75% (72-77%), specificity 95% (94-96%), PPV 86% (83-88%), and NPV 91% (90-91%). CONCLUSIONS: For pneumonia validation, ONYX can replace almost 90% of manual review while maintaining low to moderate misclassification rates. It can be tailored for different outcomes and study needs and thus warrants exploration in other settings.

Building a robust, scalable and standards-driven infrastructure for secondary use of EHR data: the SHARPn project.

The Strategic Health IT Advanced Research Projects (SHARP) Program, established by the Office of the National Coordinator for Health Information Technology in 2010 supports research findings that remove barriers for increased adoption of health IT. The improvements envisioned by the SHARP Area 4 Consortium (SHARPn) will enable the use of the electronic health record (EHR) for secondary purposes, such as care process and outcomes improvement, biomedical research and epidemiologic monitoring of the nation's health. One of the primary informatics problem areas in this endeavor is the standardization of disparate health data from the nation's many health care organizations and providers. The SHARPn team is developing open source services and components to support the ubiquitous exchange, sharing and reuse or 'liquidity' of operational clinical data stored in electronic health records. One year into the design and development of the SHARPn framework, we demonstrated end to end data flow and a prototype SHARPn platform, using thousands of patient electronic records sourced from two large healthcare organizations: Mayo Clinic and Intermountain Healthcare. The platform was deployed to (1) receive source EHR data in several formats, (2) generate structured data from EHR narrative text, and (3) normalize the EHR data using common detailed clinical models and Consolidated Health Informatics standard terminologies, which were (4) accessed by a phenotyping service using normalized data specifications. The architecture of this prototype SHARPn platform is presented. The EHR data throughput demonstration showed success in normalizing native EHR data, both structured and narrative, from two independent organizations and EHR systems. Based on the demonstration, observed challenges for standardization of EHR data for interoperable secondary use are discussed.

CheXED: Comparison of a Deep Learning Model to a Clinical Decision Support System for Pneumonia in the Emergency Department.

PURPOSE: Patients with pneumonia often present to the emergency department (ED) and require prompt diagnosis and treatment. Clinical decision support systems for the diagnosis and management of pneumonia are commonly utilized in EDs to improve patient care. The purpose of this study is to investigate whether a deep learning model for detecting radiographic pneumonia and pleural effusions can improve functionality of a clinical decision support system (CDSS) for pneumonia management (ePNa) operating in 20 EDs. MATERIALS AND METHODS: In this retrospective cohort study, a dataset of 7434 prior chest radiographic studies from 6551 ED patients was used to develop and validate a deep learning model to identify radiographic pneumonia, pleural effusions, and evidence of multilobar pneumonia. Model performance was evaluated against 3 radiologists' adjudicated interpretation and compared with performance of the natural language processing of radiology reports used by ePNa. RESULTS: The deep learning model achieved an area under the receiver operating characteristic curve of 0.833 (95% confidence interval [CI]: 0.795, 0.868) for detecting radiographic pneumonia, 0.939 (95% CI: 0.911, 0.962) for detecting pleural effusions and 0.847 (95% CI: 0.800, 0.890) for identifying multilobar pneumonia. On all 3 tasks, the model achieved higher agreement with the adjudicated radiologist interpretation compared with ePNa. CONCLUSIONS: A deep learning model demonstrated higher agreement with radiologists than the ePNa CDSS in detecting radiographic pneumonia and related findings. Incorporating deep learning models into pneumonia CDSS could enhance diagnostic performance and improve pneumonia management.

Computer clinical decision support that automates personalized clinical care: a challenging but needed healthcare delivery strategy.

How to deliver best care in various clinical settings remains a vexing problem. All pertinent healthcare-related questions have not, cannot, and will not be addressable with costly time- and resource-consuming controlled clinical trials. At present, evidence-based guidelines can address only a small fraction of the types of care that clinicians deliver. Furthermore, underserved areas rarely can access state-of-the-art evidence-based guidelines in real-time, and often lack the wherewithal to implement advanced guidelines. Care providers in such settings frequently do not have sufficient training to undertake advanced guideline implementation. Nevertheless, in advanced modern healthcare delivery environments, use of eActions (validated clinical decision support systems) could help overcome the cognitive limitations of overburdened clinicians. Widespread use of eActions will require surmounting current healthcare technical and cultural barriers and installing clinical evidence/data curation systems. The authors expect that increased numbers of evidence-based guidelines will result from future comparative effectiveness clinical research carried out during routine healthcare delivery within learning healthcare systems.

Population-based family history-specific risks for colorectal cancer: a constellation approach.

BACKGROUND & AIMS: Colorectal cancer (CRC) risk estimates based on family history typically include only close relatives. We report familial relative risk (FRR) in probands with various combinations, or constellations, of affected relatives, extending to third-degree. METHODS: A population-based resource that includes a computerized genealogy linked to statewide cancer records was used to identify genetic relationships among CRC cases and their first-, second-, and third-degree relatives (FDRs, SDRs, and TDRs). FRRs were estimated by comparing the observed number of affected persons with a particular family history constellation to the expected number, based on cohort-specific CRC rates. RESULTS: A total of 2,327,327 persons included in > or =3 generation family histories were analyzed; 10,556 had a diagnosis of CRC. The FRR for CRC in persons with > or =1 affected FDR = 2.05 (95% CI, 1.96-2.14), consistent with published estimates. In the absence of a positive first-degree family history, considering both affected SDRs and TDRs, only 1 constellation had an FRR estimate that was significantly >1.0 (0 affected FDRs, 1 affected SDR, 2 affected TDRs; FRR = 1.33; 95% CI, 1.13-1.55). The FRR for persons with 1 affected FDR, 1 affected SDR, and 0 affected TDRs was 1.88 (95% CI, 1.59-2.20), increasing to FRR = 3.28 (95% CI, 2.44-4.31) for probands with 1 affected FDR, 1 affected SDR, and > or =3 affected TDRs. CONCLUSIONS: Increased numbers of affected FDRs influences risk much more than affected SDRs or TDRs. However, when combined with a positive first-degree family history, a positive second- and third-degree family history can significantly increase risk.

How well does family history predict who will get colorectal cancer? Implications for cancer screening and counseling.

PURPOSE: Using a large, retrospective cohort from the Utah Population Database, we assess how well family history predicts who will acquire colorectal cancer during a 20-year period. METHODS: Individuals were selected between ages 35 and 80 with no prior record of colorectal cancer diagnosis, as of the year 1985. Numbers of colorectal cancer-affected relatives and diagnosis ages were collected. Familial relative risk and absolute risk estimates were calculated. Colorectal cancer diagnoses in the cohort were counted between years 1986 and 2005. Cox regression and Harrell's C were used to measure the discriminatory power of resulting models. RESULTS: A total of 431,153 individuals were included with 5,334 colorectal cancer diagnoses. Familial relative risk ranged from 0.83 to 12.39 and 20-year absolute risk from 0.002 to 0.21. With familial relative risk as the only predictor, Harrell's C = 0.53 and with age only, Harrell's C = 0.66. Familial relative risk combined with age produced a Harrell's C = 0.67. CONCLUSION: Family history by itself is not a strong predictor of exactly who will acquire colorectal cancer within 20 years. However, stratification of risk using absolute risk probabilities may be more helpful in focusing screening on individuals who are more likely to develop the disease.

Comparison of compliance for colorectal cancer screening and surveillance by colonoscopy based on risk.

PURPOSE: To compare colonoscopy screening/surveillance rates by level of risk for colorectal cancer based on age, personal history of adenomatous polyps or colorectal cancer, or family history of colorectal cancer. METHODS: Participants were aged 30-90 years, were seen within 5 years at Intermountain Healthcare, and had family history in the Utah Population Database. Colonoscopy rates were measured for those with/without risk factors. RESULTS: Among those aged 60-69 years, 48.4% had colonoscopy in the last 10 years, with rates declining after age 70 years. Percentages of those having had a colonoscopy in the last 10 years generally increased by risk level from 38.5% in those with a familial relative risk <1.0 to 47.6% in those with a familial relative risk >3.0. Compared with those with no family history, the odds ratio for being screened according to guidelines was higher for those with one first-degree relative diagnosed with colorectal cancer ≥ 60 years or two affected second-degree relatives (1.54, 95% confidence interval: 1.46-1.61) than those with one affected first-degree relative diagnosed <60 years or ≥2 affected first-degree relatives (1.25, 95% confidence interval: 1.14-1.37). CONCLUSIONS: Compliance with colonoscopy guidelines was higher for those with familial risk but did not correspond with the degree of risk.

Contemporary clinical decision support standards using Health Level Seven International Fast Healthcare Interoperability Resources.

OBJECTIVE: To facilitate the development of standards-based clinical decision support (CDS) systems, we review the current set of CDS standards that are based on Health Level Seven International Fast Healthcare Interoperability Resources (FHIR). Widespread adoption of these standards may help reduce healthcare variability, improve healthcare quality, and improve patient safety. TARGET AUDIENCE: This tutorial is designed for the broad informatics community, some of whom may be unfamiliar with the current, FHIR-based CDS standards. SCOPE: This tutorial covers the following standards: Arden Syntax (using FHIR as the data model), Clinical Quality Language, FHIR Clinical Reasoning, SMART on FHIR, and CDS Hooks. Detailed descriptions and selected examples are provided.

Enabling a learning healthcare system with automated computer protocols that produce replicable and personalized clinician actions.

Clinical decision-making is based on knowledge, expertise, and authority, with clinicians approving almost every intervention-the starting point for delivery of "All the right care, but only the right care," an unachieved healthcare quality improvement goal. Unaided clinicians suffer from human cognitive limitations and biases when decisions are based only on their training, expertise, and experience. Electronic health records (EHRs) could improve healthcare with robust decision-support tools that reduce unwarranted variation of clinician decisions and actions. Current EHRs, focused on results review, documentation, and accounting, are awkward, time-consuming, and contribute to clinician stress and burnout. Decision-support tools could reduce clinician burden and enable replicable clinician decisions and actions that personalize patient care. Most current clinical decision-support tools or aids lack detail and neither reduce burden nor enable replicable actions. Clinicians must provide subjective interpretation and missing logic, thus introducing personal biases and mindless, unwarranted, variation from evidence-based practice. Replicability occurs when different clinicians, with the same patient information and context, come to the same decision and action. We propose a feasible subset of therapeutic decision-support tools based on credible clinical outcome evidence: computer protocols leading to replicable clinician actions (eActions). eActions enable different clinicians to make consistent decisions and actions when faced with the same patient input data. eActions embrace good everyday decision-making informed by evidence, experience, EHR data, and individual patient status. eActions can reduce unwarranted variation, increase quality of clinical care and research, reduce EHR noise, and could enable a learning healthcare system.

Deriving consumer-facing disease concepts for family health histories using multi-source sampling.

The family health history has long been recognized as an effective way of understanding individuals' susceptibility to familial disease; yet electronic tools to support the capture and use of these data have been characterized as inadequate. As part of an ongoing effort to build patient-facing tools for entering detailed family health histories, we have compiled a set of concepts specific to familial disease using multi-source sampling. These concepts were abstracted by analyzing family health history data patterns in our enterprise data warehouse, collection patterns of consumer personal health records, analyses from the local state health department, a healthcare data dictionary, and concepts derived from genetic-oriented consumer education materials. Collectively, these sources yielded a set of more than 500 unique disease concepts, represented by more than 2500 synonyms for supporting patients in entering coded family health histories. We expect that these concepts will be useful in providing meaningful data and education resources for patients and providers alike.

Classification models for the prediction of clinicians' information needs.

OBJECTIVE: Clinicians face numerous information needs during patient care activities and most of these needs are not met. Infobuttons are information retrieval tools that help clinicians to fulfill their information needs by providing links to on-line health information resources from within an electronic medical record (EMR) system. The aim of this study was to produce classification models based on medication infobutton usage data to predict the medication-related content topics (e.g., dose, adverse effects, drug interactions, patient education) that a clinician is most likely to choose while entering medication orders in a particular clinical context. DESIGN: We prepared a dataset with 3078 infobutton sessions and 26 attributes describing characteristics of the user, the medication, and the patient. In these sessions, users selected one out of eight content topics. Automatic attribute selection methods were then applied to the dataset to eliminate redundant and useless attributes. The reduced dataset was used to produce nine classification models from a set of state-of-the-art machine learning algorithms. Finally, the performance of the models was measured and compared. MEASUREMENTS: Area under the ROC curve (AUC) and agreement (kappa) between the content topics predicted by the models and those chosen by clinicians in each infobutton session. RESULTS: The performance of the models ranged from 0.49 to 0.56 (kappa). The AUC of the best model ranged from 0.73 to 0.99. The best performance was achieved when predicting choice of the adult dose, pediatric dose, patient education, and pregnancy category content topics. CONCLUSION: The results suggest that classification models based on infobutton usage data are a promising method for the prediction of content topics that a clinician would choose to answer patient care questions while using an EMR system.

A multivariate time series approach to modeling and forecasting demand in the emergency department.

STUDY OBJECTIVE: The goals of this investigation were to study the temporal relationships between the demands for key resources in the emergency department (ED) and the inpatient hospital, and to develop multivariate forecasting models. METHODS: Hourly data were collected from three diverse hospitals for the year 2006. Descriptive analysis and model fitting were carried out using graphical and multivariate time series methods. Multivariate models were compared to a univariate benchmark model in terms of their ability to provide out-of-sample forecasts of ED census and the demands for diagnostic resources. RESULTS: Descriptive analyses revealed little temporal interaction between the demand for inpatient resources and the demand for ED resources at the facilities considered. Multivariate models provided more accurate forecasts of ED census and of the demands for diagnostic resources. CONCLUSION: Our results suggest that multivariate time series models can be used to reliably forecast ED patient census; however, forecasts of the demands for diagnostic resources were not sufficiently reliable to be useful in the clinical setting.

The design and evaluation of a Bayesian system for detecting and characterizing outbreaks of influenza.

The prediction and characterization of outbreaks of infectious diseases such as influenza remains an open and important problem. This paper describes a framework for detecting and characterizing outbreaks of influenza and the results of testing it on data from ten outbreaks collected from two locations over five years. We model outbreaks with compartment models and explicitly model non-influenza influenza-like illnesses.

Implementation of Real-Time Electronic Clinical Decision Support for Emergency Department Patients with Pneumonia Across a Healthcare System.

A real-time electronic CDS for pneumonia (ePNa) identifies possible pneumonia patients, measures severity and antimicrobial resistance risk, and then recommends disposition, antibiotics, and microbiology studies. Use is voluntary, and clinicians may modify treatment recommendations. ePNa was associated with lower mortality in emergency department (ED) patients versus usual care (Annals EM 66:511). We adapted ePNa for the Cerner EHR, and implemented it across Intermountain Healthcare EDs (Utah, USA) throughout 2018. We introduced ePNa through didactic, interactive presentations to ED clinicians; follow-up visits identified barriers and facilitators to use. Email reminded clinicians and answered questions. Hospital admitting clinicians encouraged ePNa use to smooth care transitions. Audit-and-feedback measured utilization, showing variations from best practice when ePNa and associated electronic order sets were not used. Use was initially low, but gradually increased especially at larger hospitals. A user-friendly interface, frequent reminders, audit-and- feedback, a user survey, a nurse educator, and local physician champions are additive towards implementation success.

Effectiveness of topic-specific infobuttons: a randomized controlled trial.

OBJECTIVE: Infobuttons are decision support tools that provide links within electronic medical record systems to relevant content in online information resources. The aim of infobuttons is to help clinicians promptly meet their information needs. The objective of this study was to determine whether infobutton links that direct to specific content topics ("topic links") are more effective than links that point to general overview content ("nonspecific links"). DESIGN: Randomized controlled trial with a control and an intervention group. Clinicians in the control group had access to nonspecific links, while those in the intervention group had access to topic links. MEASUREMENTS: Infobutton session duration, number of infobutton sessions, session success rate, and the self-reported impact that the infobutton session produced on decision making. RESULTS: The analysis was performed on 90 subjects and 3,729 infobutton sessions. Subjects in the intervention group spent 17.4% less time seeking for information (35.5 seconds vs. 43 seconds, p = 0.008) than those in the control group. Subjects in the intervention group used infobuttons 20.5% (22 sessions vs. 17.5 sessions, p = 0.21) more often than in the control group, but the difference was not significant. The information seeking success rate was equally high in both groups (89.4% control vs. 87.2% intervention, p = 0.99). Subjects reported a high positive clinical impact (i.e., decision enhancement or knowledge update) in 62% of the sessions. Limitations The exclusion of users with a low frequency of infobutton use and the focus on medication-related information needs may limit the generalization of the results. The session outcomes measurement was based on clinicians' self-assessment and therefore prone to bias. CONCLUSION: The results support the hypothesis that topic links are more efficient than nonspecific links regarding the time seeking for information. It is unclear whether the statistical difference demonstrated will result in a clinically significant impact. However, the overall results confirm previous evidence that infobuttons are effective at helping clinicians to answer questions at the point of care and demonstrate a modest incremental change in the efficiency of information delivery for routine users of this tool.

Infobuttons and classification models: a method for the automatic selection of on-line information resources to fulfill clinicians' information needs.

OBJECTIVE: Infobuttons are decision support tools that offer links to information resources based on the context of the interaction between a clinician and an electronic medical record (EMR) system. The objective of this study was to explore machine learning and web usage mining methods to produce classification models for the prediction of information resources that might be relevant in a particular infobutton context. DESIGN: Classification models were developed and evaluated with an infobutton usage dataset. The performance of the models was measured and compared with a reference implementation in a series of experiments. MEASUREMENTS: Level of agreement (kappa) between the models and the resources that clinicians actually used in each infobutton session. RESULTS: The classification models performed significantly better than the reference implementation (p<.0001). The performance of these models tended to decrease over time, probably due to a phenomenon known as concept drift. However, the performance of the models remained stable when concept drift handling techniques were used. CONCLUSIONS: The results suggest that classification models are a promising method for the prediction of information resources that a clinician would use to answer patient care questions.