ABSTRACT
Background: Identifying regional wall motion abnormalities (RWMAs) is critical for diagnosing and risk stratifying patients with cardiovascular disease, particularly ischemic heart disease. We hypothesized that a deep neural network could accurately identify patients with regional wall motion abnormalities from a readily available standard 12-lead electrocardiogram (ECG). Methods: This observational, retrospective study included patients who were treated at Beth Israel Deaconess Medical Center and had an ECG and echocardiogram performed within 14 days of each other between 2008 and 2019. We trained a convolutional neural network to detect the presence of RWMAs, qualitative global right ventricular (RV) hypokinesis, and varying degrees of left ventricular dysfunction (left ventricular ejection fraction [LVEF] ≤50%, LVEF ≤40%, and LVEF ≤35%) identified by echocardiography, using ECG data alone. Patients were randomly split into development (80%) and test sets (20%). Model performance was assessed using area under the receiver operating characteristic curve (AUC). Cox proportional hazard models adjusted for age and sex were performed to estimate the risk of future acute coronary events. Results: The development set consisted of 19,837 patients (mean age 66.7±16.4; 46.7% female) and the test set comprised of 4,953 patients (mean age 67.5±15.8 years; 46.5% female). On the test dataset, the model accurately identified the presence of RWMA, RV hypokinesis, LVEF ≤50%, LVEF ≤40%, and LVEF ≤35% with AUCs of 0.87 (95% CI 0.858-0.882), 0.888 (95% CI 0.878-0.899), 0.923 (95% CI 0.914-0.933), 0.93 (95% CI 0.921-0.939), and 0.876 (95% CI 0.858-0.896), respectively. Among patients with normal biventricular function at the time of the index ECG, those classified as having RMWA by the model were 3 times the risk (age- and sex-adjusted hazard ratio, 2.8; 95% CI 1.9-3.9) for future acute coronary events compared to those classified as negative. Conclusions: We demonstrate that a deep neural network can help identify regional wall motion abnormalities and reduced LV function from a 12-lead ECG and could potentially be used as a screening tool for triaging patients who need either initial or repeat echocardiographic imaging.
ABSTRACT
OBJECTIVE: Leverage electronic health record (EHR) audit logs to develop a machine learning (ML) model that predicts which notes a clinician wants to review when seeing oncology patients. MATERIALS AND METHODS: We trained logistic regression models using note metadata and a Term Frequency Inverse Document Frequency (TF-IDF) text representation. We evaluated performance with precision, recall, F1, AUC, and a clinical qualitative assessment. RESULTS: The metadata only model achieved an AUC 0.930 and the metadata and TF-IDF model an AUC 0.937. Qualitative assessment revealed a need for better text representation and to further customize predictions for the user. DISCUSSION: Our model effectively surfaces the top 10 notes a clinician wants to review when seeing an oncology patient. Further studies can characterize different types of clinician users and better tailor the task for different care settings. CONCLUSION: EHR audit logs can provide important relevance data for training ML models that assist with note-writing in the oncology setting.
Subject(s)
Electronic Health Records , Machine Learning , Medical Oncology , Humans , Logistic Models , Metadata , Medical Audit , Proof of Concept StudyABSTRACT
Digital data collection during routine clinical practice is now ubiquitous within hospitals. The data contains valuable information on the care of patients and their response to treatments, offering exciting opportunities for research. Typically, data are stored within archival systems that are not intended to support research. These systems are often inaccessible to researchers and structured for optimal storage, rather than interpretability and analysis. Here we present MIMIC-IV, a publicly available database sourced from the electronic health record of the Beth Israel Deaconess Medical Center. Information available includes patient measurements, orders, diagnoses, procedures, treatments, and deidentified free-text clinical notes. MIMIC-IV is intended to support a wide array of research studies and educational material, helping to reduce barriers to conducting clinical research.
Subject(s)
Electronic Health Records , Humans , Databases, Factual , HospitalsABSTRACT
Importance: Alternative methods for hospital occupancy forecasting, essential information in hospital crisis planning, are necessary in a novel pandemic when traditional data sources such as disease testing are limited. Objective: To determine whether mandatory daily employee symptom attestation data can be used as syndromic surveillance to estimate COVID-19 hospitalizations in the communities where employees live. Design, Setting, and Participants: This cohort study was conducted from April 2, 2020, to November 4, 2020, at a large academic hospital network of 10 hospitals accounting for a total of 2384 beds and 136â¯000 discharges in New England. The participants included 6841 employees who worked on-site at hospital 1 and lived in the 10 hospitals' service areas. Exposure: Daily employee self-reported symptoms were collected using an automated text messaging system from a single hospital. Main Outcomes and Measures: Mean absolute error (MAE) and weighted mean absolute percentage error (MAPE) of 7-day forecasts of daily COVID-19 hospital census at each hospital. Results: Among 6841 employees living within the 10 hospitals' service areas, 5120 (74.8%) were female individuals and 3884 (56.8%) were White individuals; the mean (SD) age was 40.8 (13.6) years, and the mean (SD) time of service was 8.8 (10.4) years. The study model had a MAE of 6.9 patients with COVID-19 and a weighted MAPE of 1.5% for hospitalizations for the entire hospital network. The individual hospitals had an MAE that ranged from 0.9 to 4.5 patients (weighted MAPE ranged from 2.1% to 16.1%). For context, the mean network all-cause occupancy was 1286 during this period, so an error of 6.9 is only 0.5% of the network mean occupancy. Operationally, this level of error was negligible to the incident command center. At hospital 1, a doubling of the number of employees reporting symptoms (which corresponded to 4 additional employees reporting symptoms at the mean for hospital 1) was associated with a 5% increase in COVID-19 hospitalizations at hospital 1 in 7 days (regression coefficient, 0.05; 95% CI, 0.02-0.07; P < .001). Conclusions and Relevance: This cohort study found that a real-time employee health attestation tool used at a single hospital could be used to estimate subsequent hospitalizations in 7 days at hospitals throughout a larger hospital network in New England.
Subject(s)
COVID-19/epidemiology , Forecasting/methods , Hospitalization/trends , Personnel, Hospital/statistics & numerical data , Sentinel Surveillance , Adult , COVID-19/diagnosis , Cohort Studies , Female , Hospitals , Humans , Incidence , Male , Middle Aged , New England/epidemiology , SARS-CoV-2 , Symptom Assessment/statistics & numerical dataABSTRACT
OBJECTIVE: While the judicious use of antibiotics takes past microbiological culture results into consideration, this data's typical format in the electronic health record (EHR) may be unwieldy when incorporated into clinical decision-making. We hypothesize that a visual representation of sensitivities may aid in their comprehension. MATERIALS AND METHODS: A prospective parallel unblinded randomized controlled trial was undertaken at an academic urban tertiary care center. Providers managing emergency department (ED) patients receiving antibiotics and having previous culture sensitivity testing were included. Providers were randomly selected to use standard EHR functionality or a visual representation of patients' past culture data as they answered questions about previous sensitivities. Concordance between provider responses and past cultures was assessed using the kappa statistic. Providers were surveyed about their decision-making and the usability of the tool using Likert scales. RESULTS: 518 ED encounters were screened from 3/5/2018 to 9/30/18, with providers from 144 visits enrolled and analyzed in the intervention arm and 129 in the control arm. Providers using the visualization tool had a kappa of 0.69 (95% CI: 0.65-0.73) when asked about past culture results while the control group had a kappa of 0.16 (95% CI: 0.12-0.20). Providers using the tool expressed improved understanding of previous cultures and found the tool easy to use (P < .001). Secondary outcomes showed no differences in prescribing practices. CONCLUSION: A visual representation of culture sensitivities improves comprehension when compared to standard text-based representations.
Subject(s)
Comprehension , Electronic Health Records , Emergency Service, Hospital , Humans , Prospective Studies , Surveys and QuestionnairesABSTRACT
PURPOSE: To develop a machine learning model to classify the severity grades of pulmonary edema on chest radiographs. MATERIALS AND METHODS: In this retrospective study, 369 071 chest radiographs and associated radiology reports from 64 581 patients (mean age, 51.71 years; 54.51% women) from the MIMIC-CXR chest radiograph dataset were included. This dataset was split into patients with and without congestive heart failure (CHF). Pulmonary edema severity labels from the associated radiology reports were extracted from patients with CHF as four different ordinal levels: 0, no edema; 1, vascular congestion; 2, interstitial edema; and 3, alveolar edema. Deep learning models were developed using two approaches: a semisupervised model using a variational autoencoder and a pretrained supervised learning model using a dense neural network. Receiver operating characteristic curve analysis was performed on both models. RESULTS: The area under the receiver operating characteristic curve (AUC) for differentiating alveolar edema from no edema was 0.99 for the semisupervised model and 0.87 for the pretrained models. Performance of the algorithm was inversely related to the difficulty in categorizing milder states of pulmonary edema (shown as AUCs for semisupervised model and pretrained model, respectively): 2 versus 0, 0.88 and 0.81; 1 versus 0, 0.79 and 0.66; 3 versus 1, 0.93 and 0.82; 2 versus 1, 0.69 and 0.73; and 3 versus 2, 0.88 and 0.63. CONCLUSION: Deep learning models were trained on a large chest radiograph dataset and could grade the severity of pulmonary edema on chest radiographs with high performance.Supplemental material is available for this article.See also the commentary by Auffermann in this issue.© RSNA, 2021.
ABSTRACT
We propose and demonstrate a representation learning approach by maximizing the mutual information between local features of images and text. The goal of this approach is to learn useful image representations by taking advantage of the rich information contained in the free text that describes the findings in the image. Our method trains image and text encoders by encouraging the resulting representations to exhibit high local mutual information. We make use of recent advances in mutual information estimation with neural network discriminators. We argue that the sum of local mutual information is typically a lower bound on the global mutual information. Our experimental results in the downstream image classification tasks demonstrate the advantages of using local features for image-text representation learning. Our code is available at: https://github.com/RayRuizhiLiao/mutual_info_img_txt.
ABSTRACT
AIM: Pancreatic ductal adenocarcinoma (PDAC) is often diagnosed at a late, incurable stage. We sought to determine whether individuals at high risk of developing PDAC could be identified early using routinely collected data. METHODS: Electronic health record (EHR) databases from two independent hospitals in Boston, Massachusetts, providing inpatient, outpatient, and emergency care, from 1979 through 2017, were used with case-control matching. PDAC cases were selected using International Classification of Diseases 9/10 codes and validated with tumour registries. A data-driven feature selection approach was used to develop neural networks and L2-regularised logistic regression (LR) models on training data (594 cases, 100,787 controls) and compared with a published model based on hand-selected diagnoses ('baseline'). Model performance was validated on an external database (408 cases, 160,185 controls). Three prediction lead times (180, 270 and 365 days) were considered. RESULTS: The LR model had the best performance, with an area under the curve (AUC) of 0.71 (confidence interval [CI]: 0.67-0.76) for the training set, and AUC 0.68 (CI: 0.65-0.71) for the validation set, 365 days before diagnosis. Data-driven feature selection improved results over 'baseline' (AUC = 0.55; CI: 0.52-0.58). The LR model flags 2692 (CI 2592-2791) of 156,485 as high risk, 365 days in advance, identifying 25 (CI: 16-36) cancer patients. Risk stratification showed that the high-risk group presented a cancer rate 3 to 5 times the prevalence in our data set. CONCLUSION: A simple EHR model, based on diagnoses, can identify high-risk individuals for PDAC up to one year in advance. This inexpensive, systematic approach may serve as the first sieve for selection of individuals for PDAC screening programs.
Subject(s)
Adenocarcinoma/epidemiology , Carcinoma, Pancreatic Ductal/epidemiology , Electronic Health Records/standards , Female , Humans , Male , Reproducibility of Results , Research DesignABSTRACT
Emergency department triage is the first point in time when a patient's acuity level is determined. The time to assign a priority at triage is short and it is vital to accurately stratify patients at this stage, since under-triage can lead to increased morbidity, mortality and costs. Our aim was to present a model that can assist healthcare professionals in triage decision making, namely in the stratification of patients through the risk prediction of a composite critical outcome-mortality and cardiopulmonary arrest. Our study cohort consisted of 235826 adult patients triaged at a Portuguese Emergency Department from 2012 to 2016. Patients were assigned to emergent, very urgent or urgent priorities of the Manchester Triage System (MTS). Demographics, clinical variables routinely collected at triage and the patients' chief complaint were used. Logistic regression, random forests and extreme gradient boosting were developed using all available variables. The term frequency-inverse document frequency (TF-IDF) natural language processing weighting factor was applied to vectorize the chief complaint. Stratified random sampling was used to split the data into train (70%) and test (30%) data sets. Ten-fold cross validation was performed in train to optimize model hyper-parameters. The performance obtained with the best model was compared against the reference model-a regularized logistic regression trained using only triage priorities. Extreme gradient boosting exhibited good calibration properties and yielded areas under the receiver operating characteristic and precision-recall curves of 0.96 (95% CI 0.95-0.97) and 0.31 (95% CI 0.26-0.36), respectively. The predictors ranked with higher importance by this model were the Glasgow coma score, the patients' age, pulse oximetry and arrival mode. Compared to the reference, the extreme gradient boosting model using clinical variables and the chief complaint presented higher recall for patients assigned MTS-3 and can identify those who are at risk of the composite outcome.
Subject(s)
Forecasting/methods , Risk Assessment/methods , Triage/methods , Adult , Cohort Studies , Emergency Service, Hospital/trends , Female , Heart Arrest , Hospitalization , Humans , Logistic Models , Machine Learning , Male , Middle Aged , Natural Language Processing , Patient Acuity , Portugal , ROC Curve , Risk FactorsABSTRACT
The risk stratification of patients in the emergency department begins at triage. It is vital to stratify patients early based on their severity, since undertriage can lead to increased morbidity, mortality and costs. Our aim was to present a new approach to assist healthcare professionals at triage in the stratification of patients and in identifying those with higher risk of ICU admission. Adult patients assigned Manchester Triage System (MTS) or Emergency Severity Index (ESI) 1 to 3 from a Portuguese and a United States Emergency Departments were analyzed. Variables routinely collected at triage were used and natural language processing was applied to the patient chief complaint. Stratified random sampling was applied to split the data in train (70%) and test (30%) sets and 10-fold cross validation was performed for model training. Logistic regression, random forests, and a random undersampling boosting algorithm were used. We compared the performance obtained with the reference model-using only triage priorities-with the models using additional variables. For both hospitals, a logistic regression model achieved higher overall performance, yielding areas under the receiver operating characteristic and precision-recall curves of 0.91 (95% CI 0.90-0.92) and 0.30 (95% CI 0.27-0.33) for the United States hospital and of 0.85 (95% CI 0.83-0.86) and 0.06 (95% CI 0.05-0.07) for the Portuguese hospital. Heart rate, pulse oximetry, respiratory rate and systolic blood pressure were the most important predictors of ICU admission. Compared to the reference models, the models using clinical variables and the chief complaint presented higher recall for patients assigned MTS/ESI 3 and can identify patients assigned MTS/ESI 3 who are at risk for ICU admission.
Subject(s)
Patient Admission/statistics & numerical data , Triage/methods , Adult , Aged , Aged, 80 and over , Emergency Service, Hospital , Female , Humans , Intensive Care Units , Logistic Models , Machine Learning , Male , Middle Aged , Natural Language Processing , Portugal/epidemiology , Risk Assessment , United States/epidemiologyABSTRACT
Increasingly large electronic health records (EHRs) provide an opportunity to algorithmically learn medical knowledge. In one prominent example, a causal health knowledge graph could learn relationships between diseases and symptoms and then serve as a diagnostic tool to be refined with additional clinical input. Prior research has demonstrated the ability to construct such a graph from over 270,000 emergency department patient visits. In this work, we describe methods to evaluate a health knowledge graph for robustness. Moving beyond precision and recall, we analyze for which diseases and for which patients the graph is most accurate. We identify sample size and unmeasured confounders as major sources of error in the health knowledge graph. We introduce a method to leverage non-linear functions in building the causal graph to better understand existing model assumptions. Finally, to assess model generalizability, we extend to a larger set of complete patient visits within a hospital system. We conclude with a discussion on how to robustly extract medical knowledge from EHRs.
Subject(s)
Electronic Health Records , Pattern Recognition, Automated , Computational Biology , HumansABSTRACT
We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at: https://github.com/RayRuizhiLiao/joint_chestxray.
ABSTRACT
OBJECTIVE: Linking emergency medical services (EMS) electronic patient care reports (ePCRs) to emergency department (ED) records can provide clinicians access to vital information that can alter management. It can also create rich databases for research and quality improvement. Unfortunately, previous attempts at ePCR and ED record linkage have had limited success. In this study, we use supervised machine learning to derive and validate an automated record linkage algorithm between EMS ePCRs and ED records. MATERIALS AND METHODS: All consecutive ePCRs from a single EMS provider between June 2013 and June 2015 were included. A primary reviewer matched ePCRs to a list of ED patients to create a gold standard. Age, gender, last name, first name, social security number, and date of birth were extracted. Data were randomly split into 80% training and 20% test datasets. We derived missing indicators, identical indicators, edit distances, and percent differences. A multivariate logistic regression model was trained using 5-fold cross-validation, using label k-fold, L2 regularization, and class reweighting. RESULTS: A total of 14 032 ePCRs were included in the study. Interrater reliability between the primary and secondary reviewer had a kappa of 0.9. The algorithm had a sensitivity of 99.4%, a positive predictive value of 99.9%, and an area under the receiver-operating characteristic curve of 0.99 in both the training and test datasets. Date-of-birth match had the highest odds ratio of 16.9, followed by last name match (10.6). Social security number match had an odds ratio of 3.8. CONCLUSIONS: We were able to successfully derive and validate a record linkage algorithm from a single EMS ePCR provider to our hospital EMR.
Subject(s)
Emergency Medical Services , Emergency Service, Hospital , Medical Record Linkage/methods , Supervised Machine Learning , Algorithms , Female , Humans , Logistic Models , Male , Retrospective StudiesABSTRACT
Importance: Electronic health records allow teams of clinicians to simultaneously care for patients, but an unintended consequence is the potential for duplicate orders of tests and medications. Objective: To determine whether a simple visual aid is associated with a reduction in duplicate ordering of tests and medications. Design, Setting, and Participants: This cohort study used an interrupted time series model to analyze 184â¯694 consecutive patients who visited the emergency department (ED) of an academic hospital with 55â¯000 ED visits annually. Patient visits occurred 1 year before and after each intervention, as follows: for laboratory orders, from August 13, 2012, to August 13, 2014; for medication orders, from February 3, 2013, to February 3, 2015; and for radiology orders, from December 12, 2013, to December 12, 2015. Data were analyzed from April to September 2019. Exposure: If an order had previously been placed during the ED visit, a red highlight appeared around the checkbox of that order in the computerized provider order entry system. Main Outcomes and Measures: Number of unintentional duplicate laboratory, medication, and radiology orders. Results: A total of 184â¯694 patients (mean [SD] age, 51.6 [20.8] years; age range, 0-113.0 years; 99â¯735 [54.0%] women) who visited the ED were analyzed over the 3 overlapping study periods. After deployment of a noninterruptive nudge in electronic health records, there was an associated 49% decrease in the rate of unintentional duplicate orders for laboratory tests (incidence rate ratio, 0.51; 95% CI, 0.45-0.59), from 4485 to 2731 orders, and an associated 40% decrease in unintentional duplicate orders of radiology tests (incidence rate ratio, 0.60; 95% CI, 0.44-0.82), from 956 to 782 orders. There was not a statistically significant change in unintentional duplicate orders of medications (incidence rate ratio, 1.17; 95% CI, 0.52-2.61), which increased from 225 to 287 orders. The nudge eliminated an estimated 17â¯936 clicks in our electronic health record. Conclusions and Relevance: In this interrupted time series cohort study, passive visual cues that provided just-in-time decision support were associated with reductions in unintentional duplicate orders for laboratory and radiology tests but not in unintentional duplicate medication orders.
Subject(s)
Audiovisual Aids/statistics & numerical data , Emergency Service, Hospital/statistics & numerical data , Health Plan Implementation/statistics & numerical data , Health Services Misuse/prevention & control , Medical Order Entry Systems/statistics & numerical data , Adolescent , Adult , Aged , Aged, 80 and over , Child , Child, Preschool , Electronic Health Records , Female , Humans , Infant , Infant, Newborn , Interrupted Time Series Analysis , Male , Middle Aged , Young AdultABSTRACT
Chest radiography is an extremely powerful imaging modality, allowing for a detailed inspection of a patient's chest, but requires specialized training for proper interpretation. With the advent of high performance general purpose computer vision algorithms, the accurate automated analysis of chest radiographs is becoming increasingly of interest to researchers. Here we describe MIMIC-CXR, a large dataset of 227,835 imaging studies for 65,379 patients presenting to the Beth Israel Deaconess Medical Center Emergency Department between 2011-2016. Each imaging study can contain one or more images, usually a frontal view and a lateral view. A total of 377,110 images are available in the dataset. Studies are made available with a semi-structured free-text radiology report that describes the radiological findings of the images, written by a practicing radiologist contemporaneously during routine clinical care. All images and reports have been de-identified to protect patient privacy. The dataset is made freely available to facilitate and encourage a wide range of research in computer vision, natural language processing, and clinical data mining.
Subject(s)
Databases, Factual , Radiography, Thoracic , Algorithms , Data Mining , Humans , Image Interpretation, Computer-Assisted , Natural Language ProcessingABSTRACT
OBJECTIVES: To determine the effect of a domain-specific ontology and machine learning-driven user interfaces on the efficiency and quality of documentation of presenting problems (chief complaints) in the emergency department (ED). METHODS: As part of a quality improvement project, we simultaneously implemented three interventions: a domain-specific ontology, contextual autocomplete, and top five suggestions. Contextual autocomplete is a user interface that ranks concepts by their predicted probability which helps nurses enter data about a patient's presenting problems. Nurses were also given a list of top five suggestions to choose from. These presenting problems were represented using a consensus ontology mapped to SNOMED CT. Predicted probabilities were calculated using a previously derived model based on triage vital signs and a brief free text note. We evaluated the percentage and quality of structured data captured using a mixed methods retrospective before-and-after study design. RESULTS: A total of 279,231 consecutive patient encounters were analyzed. Structured data capture improved from 26.2% to 97.2% (pâ¯<â¯0.0001). During the post-implementation period, presenting problems were more complete (3.35 vs 3.66; pâ¯=â¯0.0004) and higher in overall quality (3.38 vs. 3.72; pâ¯=â¯0.0002), but showed no difference in precision (3.59 vs. 3.74; pâ¯=â¯0.1). Our system reduced the mean number of keystrokes required to document a presenting problem from 11.6 to 0.6 (pâ¯<â¯0.0001), a 95% improvement. DISCUSSION: We demonstrated a technique that captures structured data on nearly all patients. We estimate that our system reduces the number of man-hours required annually to type presenting problems at our institution from 92.5â¯h to 4.8â¯h. CONCLUSION: Implementation of a domain-specific ontology and machine learning-driven user interfaces resulted in improved structured data capture, ontology usage compliance, and data quality.
Subject(s)
Algorithms , Depressive Disorder, Major/diagnosis , Depressive Disorder, Major/therapy , Documentation/standards , Emergency Service, Hospital/standards , Forms and Records Control/methods , Machine Learning , Case-Control Studies , Decision Support Systems, Clinical , Documentation/methods , Female , Humans , Male , Quality Improvement , Retrospective Studies , User-Computer InterfaceABSTRACT
OBJECTIVE: Numerous attempts have been made to create a standardized "presenting problem" or "chief complaint" list to characterize the nature of an emergency department visit. Previous attempts have failed to gain widespread adoption as they were not freely shareable or did not contain the right level of specificity, structure, and clinical relevance to gain acceptance by the larger emergency medicine community. Using real-world data, we constructed a presenting problem list that addresses these challenges. MATERIALS AND METHODS: We prospectively captured the presenting problems for 180,424 consecutive emergency department patient visits at an urban, academic, Level I trauma center in the Boston metro area. No patients were excluded. We used a consensus process to iteratively derive our system using real-world data. We used the first 70% of consecutive visits to derive our ontology, followed by a 6-month washout period, and the remaining 30% for validation. All concepts were mapped to Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT). RESULTS: Our system consists of a polyhierarchical ontology containing 692 unique concepts, 2,118 synonyms, and 30,613 nonvisible descriptions to correct misspellings and nonstandard terminology. Our ontology successfully captured structured data for 95.9% of visits in our validation data set. DISCUSSION AND CONCLUSION: We present the HierArchical Presenting Problem ontologY (HaPPy). This ontology was empirically derived and then iteratively validated by an expert consensus panel. HaPPy contains 692 presenting problem concepts, each concept being mapped to SNOMED CT. This freely sharable ontology can help to facilitate presenting problem-based quality metrics, research, and patient care.