Association of Diagnostic Discrepancy with Length of Stay and Mortality in Congestive Heart Failure Patients Admitted to the Emergency Department.

The goal of this study was to analyze diagnostic discrepancies between emergency department (ED) and hospital discharge diagnoses in patients with congestive heart failure admitted to the ED. Using a synthetic dataset from the Department of Veterans Affairs, the patients' primary diagnoses were compared at two levels: diagnostic category and body system. With 12,621 patients and 24,235 admission cases, the study found a 58% mismatch rate at the category level, which was reduced to 30% at the body system level. Diagnostic categories associated with higher levels of mismatch included aplastic anemia, pneumonia, and bacterial infections. In contrast, diagnostic categories associated with lower levels of mismatch included alcohol-related disorders, COVID-19, cardiac dysrhythmias, and gastrointestinal hemorrhage. Further investigation revealed that diagnostic mismatches are associated with longer hospital stays and higher mortality rates. These findings highlight the importance of reducing diagnostic uncertainty, particularly in specific diagnostic categories and body systems, to improve patient care following ED admission.

Standard Vocabularies to Improve Machine Learning Model Transferability With Electronic Health Record Data: Retrospective Cohort Study Using Health Care-Associated Infection.

BACKGROUND: With the widespread adoption of electronic healthcare records (EHRs) by US hospitals, there is an opportunity to leverage this data for the development of predictive algorithms to improve clinical care. A key barrier in model development and implementation includes the external validation of model discrimination, which is rare and often results in worse performance. One reason why machine learning models are not externally generalizable is data heterogeneity. A potential solution to address the substantial data heterogeneity between health care systems is to use standard vocabularies to map EHR data elements. The advantage of these vocabularies is a hierarchical relationship between elements, which allows the aggregation of specific clinical features to more general grouped concepts. OBJECTIVE: This study aimed to evaluate grouping EHR data using standard vocabularies to improve the transferability of machine learning models for the detection of postoperative health care-associated infections across institutions with different EHR systems. METHODS: Patients who underwent surgery from the University of Utah Health and Intermountain Healthcare from July 2014 to August 2017 with complete follow-up data were included. The primary outcome was a health care-associated infection within 30 days of the procedure. EHR data from 0-30 days after the operation were mapped to standard vocabularies and grouped using the hierarchical relationships of the vocabularies. Model performance was measured using the area under the receiver operating characteristic curve (AUC) and F1-score in internal and external validations. To evaluate model transferability, a difference-in-difference metric was defined as the difference in performance drop between internal and external validations for the baseline and grouped models. RESULTS: A total of 5775 patients from the University of Utah and 15,434 patients from Intermountain Healthcare were included. The prevalence of selected outcomes was from 4.9% (761/15,434) to 5% (291/5775) for surgical site infections, from 0.8% (44/5775) to 1.1% (171/15,434) for pneumonia, from 2.6% (400/15,434) to 3% (175/5775) for sepsis, and from 0.8% (125/15,434) to 0.9% (50/5775) for urinary tract infections. In all outcomes, the grouping of data using standard vocabularies resulted in a reduced drop in AUC and F1-score in external validation compared to baseline features (all P<.001, except urinary tract infection AUC: P=.002). The difference-in-difference metrics ranged from 0.005 to 0.248 for AUC and from 0.075 to 0.216 for F1-score. CONCLUSIONS: We demonstrated that grouping machine learning model features based on standard vocabularies improved model transferability between data sets across 2 institutions. Improving model transferability using standard vocabularies has the potential to improve the generalization of clinical prediction models across the health care system.

An alert tool to promote lung protective ventilation for possible acute respiratory distress syndrome.

Objective: Computer-aided decision tools may speed recognition of acute respiratory distress syndrome (ARDS) and promote consistent, timely treatment using lung-protective ventilation (LPV). This study evaluated implementation and service (process) outcomes with deployment and use of a clinical decision support (CDS) synchronous alert tool associated with existing computerized ventilator protocols and targeted patients with possible ARDS not receiving LPV. Materials and Methods: We performed an explanatory mixed methods study from December 2019 to November 2020 to evaluate CDS alert implementation outcomes across 13 intensive care units (ICU) in an integrated healthcare system with >4000 mechanically ventilated patients annually. We utilized quantitative methods to measure service outcomes including CDS alert tool utilization, accuracy, and implementation effectiveness. Attitudes regarding the appropriateness and acceptability of the CDS tool were assessed via an electronic field survey of physicians and advanced practice providers. Results: Thirty-eight percent of study encounters had at least one episode of LPV nonadherence. Addition of LPV treatment detection logic prevented an estimated 1812 alert messages (41%) over use of disease detection logic alone. Forty-eight percent of alert recommendations were implemented within 2 h. Alert accuracy was estimated at 63% when compared to gold standard ARDS adjudication, with sensitivity of 85% and positive predictive value of 62%. Fifty-seven percent of survey respondents observed one or more benefits associated with the alert. Conclusion: Introduction of a CDS alert tool based upon ARDS risk factors and integrated with computerized ventilator protocol instructions increased visibility to gaps in LPV use and promoted increased adherence to LPV.

Ascertainment of Veterans With Metastatic Prostate Cancer in Electronic Health Records: Demonstrating the Case for Natural Language Processing.

PURPOSE: Prostate cancer (PCa) is among the leading causes of cancer deaths. While localized PCa has a 5-year survival rate approaching 100%, this rate drops to 31% for metastatic prostate cancer (mPCa). Thus, timely identification of mPCa is a crucial step toward measuring and improving access to innovations that reduce PCa mortality. Yet, methods to identify patients diagnosed with mPCa remain elusive. Cancer registries provide detailed data at diagnosis but are not updated throughout treatment. This study reports on the development and validation of a natural language processing (NLP) algorithm deployed on oncology, urology, and radiology clinical notes to identify patients with a diagnosis or history of mPCa in the Department of Veterans Affairs. PATIENTS AND METHODS: Using a broad set of diagnosis and histology codes, the Veterans Affairs Corporate Data Warehouse was queried to identify all Veterans with PCa. An NLP algorithm was developed to identify patients with any history or progression of mPCa. The NLP algorithm was prototyped and developed iteratively using patient notes, grouped into development, training, and validation subsets. RESULTS: A total of 1,144,610 Veterans were diagnosed with PCa between January 2000 and October 2020, among which 76,082 (6.6%) were identified by NLP as having mPCa at some point during their care. The NLP system performed with a specificity of 0.979 and sensitivity of 0.919. CONCLUSION: Clinical documentation of mPCa is highly reliable. NLP can be leveraged to improve PCa data. When compared to other methods, NLP identified a significantly greater number of patients. NLP can be used to augment cancer registry data, facilitate research inquiries, and identify patients who may benefit from innovations in mPCa treatment.

Natural language processing for the surveillance of postoperative venous thromboembolism.

BACKGROUND: The objective of this study was to develop a portal natural language processing approach to aid in the identification of postoperative venous thromboembolism events from free-text clinical notes. METHODS: We abstracted clinical notes from 25,494 operative events from 2 independent health care systems. A venous thromboembolism detected as part of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) was used as the reference standard. A natural language processing engine, easy clinical information extractor-pulmonary embolism/deep vein thrombosis (EasyCIE-PEDVT), was trained to detect pulmonary embolism and deep vein thrombosis from clinical notes. International Classification of Diseases (ICD) discharge diagnosis codes for venous thromboembolism were used as baseline comparators. The classification performance of EasyCIE-PEDVT was compared with International Classification of Diseases codes using sensitivity, specificity, area under the receiver operating characteristic curve, using an internal and external validation cohort. RESULTS: To detect pulmonary embolism, EasyCIE-PEDVT had a sensitivity of 0.714 and 0.815 in internal and external validation, respectively. To detect deep vein thrombosis, EasyCIE-PEDVT had a sensitivity of 0.846 and 0.849 in internal and external validation, respectively. EasyCIE-PEDVT had significantly higher discrimination for deep vein thrombosis compared with International Classification of Diseases codes in internal validation (area under the receiver operating characteristic curve: 0.920 vs 0.761; P < .001) and external validation (area under the receiver operating characteristic curve: 0.921 vs 0.794; P < .001). There was no significant difference in the discrimination for pulmonary embolism between EasyCIE-PEDVT and ICD codes. CONCLUSION: Accurate surveillance of postoperative venous thromboembolism may be achieved using natural language processing on clinical notes in 2 independent health care systems. These findings suggest natural language processing may augment manual chart abstraction for large registries such as NSQIP.

Portable Automated Surveillance of Surgical Site Infections Using Natural Language Processing: Development and Validation.

OBJECTIVES: We present the development and validation of a portable NLP approach for automated surveillance of SSIs. SUMMARY OF BACKGROUND DATA: The surveillance of SSIs is labor-intensive limiting the generalizability and scalability of surgical quality surveillance programs. METHODS: We abstracted patient clinical text notes after surgical procedures from 2 independent healthcare systems using different electronic healthcare records. An SSI detected as part of the American College of Surgeons' National Surgical Quality Improvement Program was used as the reference standard. We developed a rules-based NLP system (Easy Clinical Information Extractor [CIE]-SSI) for operative event-level detection of SSIs using an training cohort (4574 operative events) from 1 healthcare system and then conducted internal validation on a blind cohort from the same healthcare system (1850 operative events) and external validation on a blind cohort from the second healthcare system (15,360 operative events). EasyCIE-SSI performance was measured using sensitivity, specificity, and area under the receiver-operating-curve (AUC). RESULTS: The prevalence of SSI was 4% and 5% in the internal and external validation corpora. In internal validation, EasyCIE-SSI had a sensitivity, specificity, AUC of 94%, 88%, 0.912 for the detection of SSI, respectively. In external validation, EasyCIE-SSI had sensitivity, specificity, AUC of 79%, 92%, 0.852 for the detection of SSI, respectively. The sensitivity of EasyCIE-SSI decreased in clean, skin/subcutaneous, and outpatient procedures in the external validation compared to internal validation. CONCLUSION: Automated surveillance of SSIs can be achieved using NLP of clinical notes with high sensitivity and specificity.

A Bayesian approach for detecting a disease that is not being modeled.

Over the past decade, outbreaks of new or reemergent viruses such as severe acute respiratory syndrome (SARS) virus, Middle East respiratory syndrome (MERS) virus, and Zika have claimed thousands of lives and cost governments and healthcare systems billions of dollars. Because the appearance of new or transformed diseases is likely to continue, the detection and characterization of emergent diseases is an important problem. We describe a Bayesian statistical model that can detect and characterize previously unknown and unmodeled diseases from patient-care reports and evaluate its performance on historical data.

Driving pressure is not associated with mortality in mechanically ventilated patients without ARDS.

BACKGROUND: In patients with acute respiratory distress syndrome (ARDS), low tidal volume ventilation has been associated with reduced mortality. Driving pressure (tidal volume normalized to respiratory system compliance) may be an even stronger predictor of ARDS survival than tidal volume. We sought to study whether these associations hold true in acute respiratory failure patients without ARDS. METHODS: This is a retrospectively cohort analysis of mechanically ventilated adult patients admitted to ICUs from 12 hospitals over 2 years. We used natural language processing of chest radiograph reports and data from the electronic medical record to identify patients who had ARDS. We used multivariable logistic regression and generalized linear models to estimate associations between tidal volume, driving pressure, and respiratory system compliance with adjusted 30-day mortality using covariates of Acute Physiology Score (APS), Charlson Comorbidity Index (CCI), age, and PaO2/FiO2 ratio. RESULTS: We studied 2641 patients; 48% had ARDS (n = 1273). Patients with ARDS had higher mean APS (25 vs. 23, p < .001) but similar CCI (4 vs. 3, p = 0.6) scores. For non-ARDS patients, tidal volume was associated with increased adjusted mortality (OR 1.18 per 1 mL/kg PBW increase in tidal volume, CI 1.04 to 1.35, p = 0.010). We observed no association between driving pressure or respiratory compliance and mortality in patients without ARDS. In ARDS patients, both ΔP (OR1.1, CI 1.06-1.14, p < 0.001) and tidal volume (OR 1.17, CI 1.04-1.31, p = 0.007) were associated with mortality. CONCLUSIONS: In a large retrospective analysis of critically ill non-ARDS patients receiving mechanical ventilation, we found that tidal volume was associated with 30-day mortality, while driving pressure was not.

The design and evaluation of a Bayesian system for detecting and characterizing outbreaks of influenza.

The prediction and characterization of outbreaks of infectious diseases such as influenza remains an open and important problem. This paper describes a framework for detecting and characterizing outbreaks of influenza and the results of testing it on data from ten outbreaks collected from two locations over five years. We model outbreaks with compartment models and explicitly model non-influenza influenza-like illnesses.

Use of Computerized Provider Order Entry Events for Postoperative Complication Surveillance.

Importance: Conventional approaches for tracking postoperative adverse events requires manual medical record review, thus limiting the scalability of such efforts. Objective: To determine if a surveillance system using computerized provider order entry (CPOE) events for selected medications as well as laboratory, microbiologic, and radiologic orders can decrease the manual medical record review burden for surveillance of postoperative complications. Design, Setting, and Participants: This cohort study reviewed the medical records of 21 775 patients who underwent surgical procedures at a university-based tertiary referral center (University of Utah, Salt Lake City) from July 1, 2007, to August 31, 2017. Patients were included if their case was selected for review by a surgical clinical reviewer as part of the National Surgical Quality Improvement Program. Patients were excluded if they had incomplete follow-up data. Main Outcomes and Measures: Thirty-day postoperative occurrences of superficial surgical site infection, deep surgical site infection, organ space surgical site infection, urinary tract infection, pneumonia, sepsis, septic shock, deep vein thrombosis requiring therapy, and pulmonary embolism, as defined by the National Surgical Quality Improvement Program. A logistic regression model was developed for each postoperative complication using CPOE features as predictors on a development set, and performance was measured on a holdout internal validation set. The models were internally validated using bootstrapping with 10 000 replications to determine the sensitivity, specificity, positive predictive value, and negative predictive value of CPOE-based surveillance system. Results: The study included 21 775 patients who underwent surgical procedures. Among these patients, 11 855 (54.4%) were women and 9920 (45.6%) were men, with a mean (SD) age of 51.7 (16.8) years. Overall, the prevalence of postoperative complications was low, ranging from 0.2% (pulmonary embolism) to 2.6% (superficial surgical site infection). Use of CPOE events to detect patients who experienced at least 1 complication had a sensitivity of 74.8% (95% CI, 71.1%-78.4%), specificity of 86.8% (95% CI, 85.5%-88.3%), positive predictive value of 33.8% (95% CI, 31.2%-36.4%), negative predictive value of 97.5% (95% CI, 97.1%-97.8%), and area under the curve of 0.808 (95% CI, 0.791-0.824). The negative predictive value for individual complications ranged from 98.7% to 100%. Use of CPOE events to screen for adverse events was estimated to diminish the burden of manual medical record review by 55.4% to 90.3%. A CPOE-based surveillance system performed well for both inpatient and outpatient procedures. Conclusions and Relevance: A CPOE-based surveillance of postoperative complications has high negative predictive value, which demonstrates that this approach can augment the currently used, resource-intensive manual medical record review process.

Using Natural Language Processing to improve EHR Structured Data-based Surgical Site Infection Surveillance.

Surgical Site Infection surveillance in healthcare systems is labor intensive and plagued by underreporting as current methodology relies heavily on manual chart review. The rapid adoption of electronic health records (EHRs) has the potential to allow the secondary use of EHR data for quality surveillance programs. This study aims to investigate the effectiveness of integrating natural language processing (NLP) outputs with structured EHR data to build machine learning models for SSI identification using real-world clinical data. We examined a set of models using structured data with and without NLP document-level, mention-level, and keyword features. The top-performing model was based on a Random Forest classifier enhanced with NLP document-level features achieving a 0.58 sensitivity, 0.97 specificity, 0.54 PPV, 0.98 NPV, and 0.52 F0.5 score. We further interrogated the feature contributions, analyzed the errors, and discussed future directions.

Epidemiology and Clinical Features of Invasive Fungal Infection in a US Health Care Network.

BACKGROUND: A better understanding of the epidemiology and clinical features of invasive fungal infection (IFI) is integral to improving outcomes. We describe a novel case-finding methodology, reporting incidence, clinical features, and outcomes of IFI in a large US health care network. METHODS: All available records in the Intermountain Healthcare Enterprise Data Warehouse from 2006 to 2015 were queried for clinical data associated with IFI. The resulting data were overlaid in 124 different combinations to identify high-probability IFI cases. The cohort was manually reviewed, and exclusions were applied. European Organization for Research and Treatment of Cancer/Invasive Fungal Infections Cooperative Group and the National Institute of Allergy and Infectious Diseases Mycoses Study Group Consensus Group definitions were adapted to categorize IFI in a broad patient population. Linear regression was used to model variation in incidence over time. RESULTS: A total of 3374 IFI episodes occurred in 3154 patients. The mean incidence was 27.2 cases/100 000 patients per year, and there was a mean annual increase of 0.24 cases/100 000 patients (P = .21). Candidiasis was the most common (55%). Dimorphic fungi, primarily Coccidioides spp., comprised 25.1% of cases, followed by Aspergillus spp. (8.9%). The median age was 55 years, and pediatric cases accounted for 13%; 26.1% of patients were on immunosuppression, 14.9% had autoimmunity or immunodeficiency, 13.3% had active malignancy, and 5.9% were transplant recipients. Lymphopenia preceded IFI in 22.1% of patients. Hospital admission occurred in 76.2%. The median length of stay was 16 days. All-cause mortality was 17.0% at 42 days and 28.8% at 1 year. Forty-two-day mortality was highest in Aspergillus spp. (27.5%), 20.5% for Candida, and lowest for dimorphic fungi (7.5%). CONCLUSIONS: In this population, IFI was not uncommon, affected a broad spectrum of patients, and was associated with high crude mortality.

A Bayesian system to detect and characterize overlapping outbreaks.

Outbreaks of infectious diseases such as influenza are a significant threat to human health. Because there are different strains of influenza which can cause independent outbreaks, and influenza can affect demographic groups at different rates and times, there is a need to recognize and characterize multiple outbreaks of influenza. This paper describes a Bayesian system that uses data from emergency department patient care reports to create epidemiological models of overlapping outbreaks of influenza. Clinical findings are extracted from patient care reports using natural language processing. These findings are analyzed by a case detection system to create disease likelihoods that are passed to a multiple outbreak detection system. We evaluated the system using real and simulated outbreaks. The results show that this approach can recognize and characterize overlapping outbreaks of influenza. We describe several extensions that appear promising.

The effects of natural language processing on cross-institutional portability of influenza case detection for disease surveillance.

OBJECTIVES: This study evaluates the accuracy and portability of a natural language processing (NLP) tool for extracting clinical findings of influenza from clinical notes across two large healthcare systems. Effectiveness is evaluated on how well NLP supports downstream influenza case-detection for disease surveillance. METHODS: We independently developed two NLP parsers, one at Intermountain Healthcare (IH) in Utah and the other at University of Pittsburgh Medical Center (UPMC) using local clinical notes from emergency department (ED) encounters of influenza. We measured NLP parser performance for the presence and absence of 70 clinical findings indicative of influenza. We then developed Bayesian network models from NLP processed reports and tested their ability to discriminate among cases of (1) influenza, (2) non-influenza influenza-like illness (NI-ILI), and (3) 'other' diagnosis. RESULTS: On Intermountain Healthcare reports, recall and precision of the IH NLP parser were 0.71 and 0.75, respectively, and UPMC NLP parser, 0.67 and 0.79. On University of Pittsburgh Medical Center reports, recall and precision of the UPMC NLP parser were 0.73 and 0.80, respectively, and IH NLP parser, 0.53 and 0.80. Bayesian case-detection performance measured by AUROC for influenza versus non-influenza on Intermountain Healthcare cases was 0.93 (using IH NLP parser) and 0.93 (using UPMC NLP parser). Case-detection on University of Pittsburgh Medical Center cases was 0.95 (using UPMC NLP parser) and 0.83 (using IH NLP parser). For influenza versus NI-ILI on Intermountain Healthcare cases performance was 0.70 (using IH NLP parser) and 0.76 (using UPMC NLP parser). On University of Pisstburgh Medical Center cases, 0.76 (using UPMC NLP parser) and 0.65 (using IH NLP parser). CONCLUSION: In all but one instance (influenza versus NI-ILI using IH cases), local parsers were more effective at supporting case-detection although performances of non-local parsers were reasonable.

A study of the transferability of influenza case detection systems between two large healthcare systems.

OBJECTIVES: This study evaluates the accuracy and transferability of Bayesian case detection systems (BCD) that use clinical notes from emergency department (ED) to detect influenza cases. METHODS: A BCD uses natural language processing (NLP) to infer the presence or absence of clinical findings from ED notes, which are fed into a Bayesain network classifier (BN) to infer patients' diagnoses. We developed BCDs at the University of Pittsburgh Medical Center (BCDUPMC) and Intermountain Healthcare in Utah (BCDIH). At each site, we manually built a rule-based NLP and trained a Bayesain network classifier from over 40,000 ED encounters between Jan. 2008 and May. 2010 using feature selection, machine learning, and expert debiasing approach. Transferability of a BCD in this study may be impacted by seven factors: development (source) institution, development parser, application (target) institution, application parser, NLP transfer, BN transfer, and classification task. We employed an ANOVA analysis to study their impacts on BCD performance. RESULTS: Both BCDs discriminated well between influenza and non-influenza on local test cases (AUCs > 0.92). When tested for transferability using the other institution's cases, BCDUPMC discriminations declined minimally (AUC decreased from 0.95 to 0.94, p<0.01), and BCDIH discriminations declined more (from 0.93 to 0.87, p<0.0001). We attributed the BCDIH decline to the lower recall of the IH parser on UPMC notes. The ANOVA analysis showed five significant factors: development parser, application institution, application parser, BN transfer, and classification task. CONCLUSION: We demonstrated high influenza case detection performance in two large healthcare systems in two geographically separated regions, providing evidentiary support for the use of automated case detection from routinely collected electronic clinical notes in national influenza surveillance. The transferability could be improved by training Bayesian network classifier locally and increasing the accuracy of the NLP parser.

Impact of an Electronic Clinical Decision Support Tool for Emergency Department Patients With Pneumonia.

STUDY OBJECTIVE: Despite evidence that guideline adherence improves clinical outcomes, management of pneumonia patients varies in emergency departments (EDs). We study the effect of a real-time, ED, electronic clinical decision support tool that provides clinicians with guideline-recommended decision support for diagnosis, severity assessment, disposition, and antibiotic selection. METHODS: This was a prospective, controlled, quasi-experimental trial in 7 Intermountain Healthcare hospital EDs in Utah's urban corridor. We studied adults with International Classification of Diseases, Ninth Revision codes and radiographic evidence for pneumonia during 2 periods: baseline (December 2009 through November 2010) and post-tool deployment (December 2011 through November 2012). The tool was deployed at 4 intervention EDs in May 2011, leaving 3 as usual care controls. We compared 30-day, all-cause mortality adjusted for illness severity, using a mixed-effect, logistic regression model. RESULTS: The study population comprised 4,758 ED pneumonia patients; 14% had health care-associated pneumonia. Median age was 58 years, 53% were female patients, and 59% were admitted to the hospital. Physicians applied the tool for 62.6% of intervention ED study patients. There was no difference overall in severity-adjusted mortality between intervention and usual care EDs post-tool deployment (odds ratio [OR]=0.69; 95% confidence interval [CI] 0.41 to 1.16). Post hoc analysis showed that patients with community-acquired pneumonia experienced significantly lower mortality (OR=0.53; 95% CI 0.28 to 0.99), whereas mortality was unchanged among patients with health care-associated pneumonia (OR=1.12; 95% CI 0.45 to 2.8). Patient disposition from the ED postdeployment adhered more to tool recommendations. CONCLUSION: This study demonstrates the feasibility and potential benefit of real-time electronic clinical decision support for ED pneumonia patients.

Improving performance of natural language processing part-of-speech tagging on clinical narratives through domain adaptation.

OBJECTIVE: Natural language processing (NLP) tasks are commonly decomposed into subtasks, chained together to form processing pipelines. The residual error produced in these subtasks propagates, adversely affecting the end objectives. Limited availability of annotated clinical data remains a barrier to reaching state-of-the-art operating characteristics using statistically based NLP tools in the clinical domain. Here we explore the unique linguistic constructions of clinical texts and demonstrate the loss in operating characteristics when out-of-the-box part-of-speech (POS) tagging tools are applied to the clinical domain. We test a domain adaptation approach integrating a novel lexical-generation probability rule used in a transformation-based learner to boost POS performance on clinical narratives. METHODS: Two target corpora from independent healthcare institutions were constructed from high frequency clinical narratives. Four leading POS taggers with their out-of-the-box models trained from general English and biomedical abstracts were evaluated against these clinical corpora. A high performing domain adaptation method, Easy Adapt, was compared to our newly proposed method ClinAdapt. RESULTS: The evaluated POS taggers drop in accuracy by 8.5-15% when tested on clinical narratives. The highest performing tagger reports an accuracy of 88.6%. Domain adaptation with Easy Adapt reports accuracies of 88.3-91.0% on clinical texts. ClinAdapt reports 93.2-93.9%. CONCLUSIONS: ClinAdapt successfully boosts POS tagging performance through domain adaptation requiring a modest amount of annotated clinical data. Improving the performance of critical NLP subtasks is expected to reduce pipeline error propagation leading to better overall results on complex processing tasks.

An ontology-driven, diagnostic modeling system.

OBJECTIVES: To present a system that uses knowledge stored in a medical ontology to automate the development of diagnostic decision support systems. To illustrate its function through an example focused on the development of a tool for diagnosing pneumonia. MATERIALS AND METHODS: We developed a system that automates the creation of diagnostic decision-support applications. It relies on a medical ontology to direct the acquisition of clinic data from a clinical data warehouse and uses an automated analytic system to apply a sequence of machine learning algorithms that create applications for diagnostic screening. We refer to this system as the ontology-driven diagnostic modeling system (ODMS). We tested this system using samples of patient data collected in Salt Lake City emergency rooms and stored in Intermountain Healthcare's enterprise data warehouse. RESULTS: The system was used in the preliminary development steps of a tool to identify patients with pneumonia in the emergency department. This tool was compared with a manually created diagnostic tool derived from a curated dataset. The manually created tool is currently in clinical use. The automatically created tool had an area under the receiver operating characteristic curve of 0.920 (95% CI 0.916 to 0.924), compared with 0.944 (95% CI 0.942 to 0.947) for the manually created tool. DISCUSSION: Initial testing of the ODMS demonstrates promising accuracy for the highly automated results and illustrates the route to model improvement. CONCLUSIONS: The use of medical knowledge, embedded in ontologies, to direct the initial development of diagnostic computing systems appears feasible.