Ensembles of natural language processing systems for portable phenotyping solutions.

BACKGROUND: Manually curating standardized phenotypic concepts such as Human Phenotype Ontology (HPO) terms from narrative text in electronic health records (EHRs) is time consuming and error prone. Natural language processing (NLP) techniques can facilitate automated phenotype extraction and thus improve the efficiency of curating clinical phenotypes from clinical texts. While individual NLP systems can perform well for a single cohort, an ensemble-based method might shed light on increasing the portability of NLP pipelines across different cohorts. METHODS: We compared four NLP systems, MetaMapLite, MedLEE, ClinPhen and cTAKES, and four ensemble techniques, including intersection, union, majority-voting and machine learning, for extracting generic phenotypic concepts. We addressed two important research questions regarding automated phenotype recognition. First, we evaluated the performance of different approaches in identifying generic phenotypic concepts. Second, we compared the performance of different methods to identify patient-specific phenotypic concepts. To better quantify the effects caused by concept granularity differences on performance, we developed a novel evaluation metric that considered concept hierarchies and frequencies. Each of the approaches was evaluated on a gold standard set of clinical documents annotated by clinical experts. One dataset containing 1,609 concepts derived from 50 clinical notes from two different institutions was used in both evaluations, and an additional dataset of 608 concepts derived from 50 case report abstracts obtained from PubMed was used for evaluation of identifying generic phenotypic concepts only. RESULTS: For generic phenotypic concept recognition, the top three performers in the NYP/CUIMC dataset are union ensemble (F1, 0.634), training-based ensemble (F1, 0.632), and majority vote-based ensemble (F1, 0.622). In the Mayo dataset, the top three are majority vote-based ensemble (F1, 0.642), cTAKES (F1, 0.615), and MedLEE (F1, 0.559). In the PubMed dataset, the top three are majority vote-based ensemble (F1, 0.719), training-based (F1, 0.696) and MetaMapLite (F1, 0.694). For identifying patient specific phenotypes, the top three performers in the NYP/CUIMC dataset are majority vote-based ensemble (F1, 0.610), MedLEE (F1, 0.609), and training-based ensemble (F1, 0.585). In the Mayo dataset, the top three are majority vote-based ensemble (F1, 0.604), cTAKES (F1, 0.531) and MedLEE (F1, 0.527). CONCLUSIONS: Our study demonstrates that ensembles of natural language processing can improve both generic phenotypic concept recognition and patient specific phenotypic concept identification over individual systems. Among the individual NLP systems, each individual system performed best when they were applied in the dataset that they were primary designed for. However, combining multiple NLP systems to create an ensemble can generally improve the performance. Specifically, the ensemble can increase the results reproducibility across different cohorts and tasks, and thus provide a more portable phenotyping solution compared to individual NLP systems.

Risk of Alzheimer's Disease Among Senior Medicare Beneficiaries Treated With Androgen Deprivation Therapy for Prostate Cancer.

Purpose To assess the relative risk of Alzheimer's disease (AD) among patients with prostate cancer who received androgen deprivation therapy (ADT), after adjustment for other cancer therapies. Methods Data from demographics, survival, diagnoses codes, procedure codes, and other information about beneficiaries age 67 years or older in the Medicare claims database was assessed to determine the unadjusted and adjusted risks of AD and of dementia from ADT. The prespecified survival analysis method was competing risk regression. Results Of the 1.2 million fee-for-service Medicare beneficiaries who developed prostate cancer in 2001 to 2014, 35% received ADT. Of these, 109,815 (8.9%) and 223,765 (18.8%) developed AD and dementia, respectively, and 26% to 33% died without either outcome. Unadjusted rates of AD and all-cause mortality per 1,000 patient-years were higher among ADT recipients; the unadjusted rates of AD were 17.0 and 15.5 per 1,000 person-years in recipients and nonrecipients, respectively, and the unadjusted rates of all-cause mortality were 73.0 and 51.6 per 1,000 person-years, respectively. The unadjusted rates for dementia in ADT recipients versus nonrecipients were 38.5 and 32.9, respectively, and the unadjusted rates of mortality were 60.2 versus 40.4, respectively. However, after analysis was adjusted for other cancer therapies and other covariates, patients with ADT treatment had no increased risk of AD (subdistribution hazard ratio [SHR], 0.98; 95% CI, 0.97 to 0.99) and had only a miniscule (1%) risk of dementia (SHR, 1.01; 95% CI, 1.01 to 1.02); patients treated with ADT were more likely to die before progression to AD (SHR, 1.24; 95% CI, 1.23 to 1.24) or dementia (SHR, 1.26; 95% CI, 1.25 to 1.26). The risks of AD and dementia were not associated with duration of ADT (ie, no dose effect). Other secondary analyses confirmed these results. Conclusion These data suggest that ADT treatment has no hazard for AD and no meaningful hazard for dementia among men age 67 years or older who are enrolled in Medicare.