Cross-institution natural language processing for reliable clinical association studies: a methodological exploration.

OBJECTIVES: Natural language processing (NLP) of clinical notes in electronic medical records is increasingly used to extract otherwise sparsely available patient characteristics, to assess their association with relevant health outcomes. Manual data curation is resource intensive and NLP methods make these studies more feasible. However, the methodology of using NLP methods reliably in clinical research is understudied. The objective of this study is to investigate how NLP models could be used to extract study variables (specifically exposures) to reliably conduct exposure-outcome association studies. STUDY DESIGN AND SETTING: In a convenience sample of patients admitted to the intensive care unit of a US academic health system, multiple association studies are conducted, comparing the association estimates based on NLP-extracted vs. manually extracted exposure variables. The association studies varied in NLP model architecture (Bidirectional Encoder Decoder from Transformers, Long Short-Term Memory), training paradigm (training a new model, fine-tuning an existing external model), extracted exposures (employment status, living status, and substance use), health outcomes (having a do-not-resuscitate/intubate code, length of stay, and in-hospital mortality), missing data handling (multiple imputation vs. complete case analysis), and the application of measurement error correction (via regression calibration). RESULTS: The study was conducted on 1,174 participants (median [interquartile range] age, 61 [50, 73] years; 60.6% male). Additionally, up to 500 discharge reports of participants from the same health system and 2,528 reports of participants from an external health system were used to train the NLP models. Substantial differences were found between the associations based on NLP-extracted and manually extracted exposures under all settings. The error in association was only weakly correlated with the overall F1 score of the NLP models. CONCLUSION: Associations estimated using NLP-extracted exposures should be interpreted with caution. Further research is needed to set conditions for reliable use of NLP in medical association studies.

Topic modeling on clinical social work notes for exploring social determinants of health factors.

Objective: Existing research on social determinants of health (SDoH) predominantly focuses on physician notes and structured data within electronic medical records. This study posits that social work notes are an untapped, potentially rich source for SDoH information. We hypothesize that clinical notes recorded by social workers, whose role is to ameliorate social and economic factors, might provide a complementary information source of data on SDoH compared to physician notes, which primarily concentrate on medical diagnoses and treatments. We aimed to use word frequency analysis and topic modeling to identify prevalent terms and robust topics of discussion within a large cohort of social work notes including both outpatient and in-patient consultations. Materials and methods: We retrieved a diverse, deidentified corpus of 0.95 million clinical social work notes from 181 644 patients at the University of California, San Francisco. We conducted word frequency analysis related to ICD-10 chapters to identify prevalent terms within the notes. We then applied Latent Dirichlet Allocation (LDA) topic modeling analysis to characterize this corpus and identify potential topics of discussion, which was further stratified by note types and disease groups. Results: Word frequency analysis primarily identified medical-related terms associated with specific ICD10 chapters, though it also detected some subtle SDoH terms. In contrast, the LDA topic modeling analysis extracted 11 topics explicitly related to social determinants of health risk factors, such as financial status, abuse history, social support, risk of death, and mental health. The topic modeling approach effectively demonstrated variations between different types of social work notes and across patients with different types of diseases or conditions. Discussion: Our findings highlight LDA topic modeling's effectiveness in extracting SDoH-related themes and capturing variations in social work notes, demonstrating its potential for informing targeted interventions for at-risk populations. Conclusion: Social work notes offer a wealth of unique and valuable information on an individual's SDoH. These notes present consistent and meaningful topics of discussion that can be effectively analyzed and utilized to improve patient care and inform targeted interventions for at-risk populations.

Assessment of valvular function in over 47,000 people using deep learning-based flow measurements.

Valvular heart disease is associated with a high global burden of disease. Even mild aortic stenosis confers increased morbidity and mortality, prompting interest in understanding normal variation in valvular function at scale. We developed a deep learning model to study velocity-encoded magnetic resonance imaging in 47,223 UK Biobank participants. We calculated eight traits, including peak velocity, mean gradient, aortic valve area, forward stroke volume, mitral and aortic regurgitant volume, greatest average velocity, and ascending aortic diameter. We then computed sex-stratified reference ranges for these phenotypes in up to 31,909 healthy individuals. In healthy individuals, we found an annual decrement of 0.03cm 2 in the aortic valve area. Participants with mitral valve prolapse had a 1 standard deviation [SD] higher mitral regurgitant volume (P=9.6 × 10 -12 ), and those with aortic stenosis had a 4.5 SD-higher mean gradient (P=1.5 × 10 -431 ), validating the derived phenotypes' associations with clinical disease. Greater levels of ApoB, triglycerides, and Lp(a) assayed nearly 10 years prior to imaging were associated with higher gradients across the aortic valve. Metabolomic profiles revealed that increased glycoprotein acetyls were also associated with an increased aortic valve mean gradient (0.92 SD, P=2.1 x 10 -22 ). Finally, velocity-derived phenotypes were risk markers for aortic and mitral valve surgery even at thresholds below what is considered relevant disease currently. Using machine learning to quantify the rich phenotypic data of the UK Biobank, we report the largest assessment of valvular function and cardiovascular disease in the general population.

From Genes to Geography, from Cells to Community, from Biomolecules to Behaviors: The Importance of Social Determinants of Health.

Much scientific work over the past few decades has linked health outcomes and disease risk to genomics, to derive a better understanding of disease mechanisms at the genetic and molecular level. However, genomics alone does not quite capture the full picture of one's overall health. Modern computational biomedical research is moving in the direction of including social/environmental factors that ultimately affect quality of life and health outcomes at both the population and individual level. The future of studying disease now lies at the hands of the social determinants of health (SDOH) to answer pressing clinical questions and address healthcare disparities across population groups through its integration into electronic health records (EHRs). In this perspective article, we argue that the SDOH are the future of disease risk and health outcomes studies due to their vast coverage of a patient's overall health. SDOH data availability in EHRs has improved tremendously over the years with EHR toolkits, diagnosis codes, wearable devices, and census tract information to study disease risk. We discuss the availability of SDOH data, challenges in SDOH implementation, its future in real-world evidence studies, and the next steps to report study outcomes in an equitable and actionable way.

Accuracy of medical billing data against the electronic health record in the measurement of colorectal cancer screening rates.

OBJECTIVE: Medical billing data are an attractive source of secondary analysis because of their ease of use and potential to answer population-health questions with statistical power. Although these datasets have known susceptibilities to biases, the degree to which they can distort the assessment of quality measures such as colorectal cancer screening rates are not widely appreciated, nor are their causes and possible solutions. METHODS: Using a billing code database derived from our institution's electronic health records, we estimated the colorectal cancer screening rate of average-risk patients aged 50-74 years seen in primary care or gastroenterology clinic in 2016-2017. 200 records (150 unscreened, 50 screened) were sampled to quantify the accuracy against manual review. RESULTS: Out of 4611 patients, an analysis of billing data suggested a 61% screening rate, an estimate that matches the estimate by the Centers for Disease Control. Manual review revealed a positive predictive value of 96% (86%-100%), negative predictive value of 21% (15%-29%) and a corrected screening rate of 85% (81%-90%). Most false negatives occurred due to examinations performed outside the scope of the database-both within and outside of our institution-but 21% of false negatives fell within the database's scope. False positives occurred due to incomplete examinations and inadequate bowel preparation. Reasons for screening failure include ordered but incomplete examinations (48%), lack of or incorrect documentation by primary care (29%) including incorrect screening intervals (13%) and patients declining screening (13%). CONCLUSIONS: Billing databases are prone to substantial bias that may go undetected even in the presence of confirmatory external estimates. Caution is recommended when performing population-level inference from these data. We propose several solutions to improve the use of these data for the assessment of healthcare quality.

Assessment of Postdonation Outcomes in US Living Kidney Donors Using Publicly Available Data Sets.

Importance: There are limited resources providing postdonation conditions that can occur in living donors (LDs) of solid-organ transplant. Consequently, it is difficult to visualize and understand possible postdonation outcomes in LDs. Objective: To assemble an open access resource that is representative of the demographic characteristics in the US national registry, maintained by the Organ Procurement and Transplantation Network and administered by the United Network for Organ Sharing, but contains more follow-up information to help to examine postdonation outcomes in LDs. Design, Setting, and Participants: Cohort study in which the data for the resource and analyses stemmed from the transplant data set derived from 27 clinical studies from the ImmPort database, which is an open access repository for clinical studies. The studies included data collected from 1963 to 2016. Data from the United Network for Organ Sharing Organ Procurement and Transplantation Network national registry collected from October 1987 to March 2016 were used to determine representativeness. Data analysis took place from June 2016 to May 2018. Data from 20 ImmPort clinical studies (including clinical trials and observational studies) were curated, and a cohort of 11 263 LDs was studied, excluding deceased donors, LDs with 95% or more missing data, and studies without a complete data dictionary. The harmonization process involved the extraction of common features from each clinical study based on categories that included demographic characteristics as well as predonation and postdonation data. Main Outcomes and Measures: Thirty-six postdonation events were identified, represented, and analyzed via a trajectory network analysis. Results: The curated data contained 10 869 living kidney donors (median [interquartile range] age, 39 [31-48] years; 6175 [56.8%] women; and 9133 [86.6%] of European descent). A total of 9558 living kidney donors with postdonation data were analyzed. Overall, 1406 LDs (14.7%) had postdonation events. The 4 most common events were hypertension (806 [8.4%]), diabetes (190 [2.0%]), proteinuria (171 [1.8%]), and postoperative ileus (147 [1.5%]). Relatively few events (n = 269) occurred before the 2-year postdonation mark. Of the 1746 events that took place 2 years or more after donation, 1575 (90.2%) were nonsurgical; nonsurgical conditions tended to occur in the wide range of 2 to 40 years after donation (odds ratio, 38.3; 95% CI, 4.12-1956.9). Conclusions and Relevance: Most events that occurred more than 2 years after donation were nonsurgical and could occur up to 40 years after donation. Findings support the construction of a national registry for long-term monitoring of LDs and confirm the value of secondary reanalysis of clinical studies.

Assessment of a Deep Learning Model Based on Electronic Health Record Data to Forecast Clinical Outcomes in Patients With Rheumatoid Arthritis.

Importance: Knowing the future condition of a patient would enable a physician to customize current therapeutic options to prevent disease worsening, but predicting that future condition requires sophisticated modeling and information. If artificial intelligence models were capable of forecasting future patient outcomes, they could be used to aid practitioners and patients in prognosticating outcomes or simulating potential outcomes under different treatment scenarios. Objective: To assess the ability of an artificial intelligence system to prognosticate the state of disease activity of patients with rheumatoid arthritis (RA) at their next clinical visit. Design, Setting, and Participants: This prognostic study included 820 patients with RA from rheumatology clinics at 2 distinct health care systems with different electronic health record platforms: a university hospital (UH) and a public safety-net hospital (SNH). The UH and SNH had substantially different patient populations and treatment patterns. The UH has records on approximately 1 million total patients starting in January 2012. The UH data for this study were accessed on July 1, 2017. The SNH has records on 65 000 unique individuals starting in January 2013. The SNH data for the study were collected on February 27, 2018. Exposures: Structured data were extracted from the electronic health record, including exposures (medications), patient demographics, laboratories, and prior measures of disease activity. A longitudinal deep learning model was used to predict disease activity for patients with RA at their next rheumatology clinic visit and to evaluate interhospital performance and model interoperability strategies. Main Outcomes and Measures: Model performance was quantified using the area under the receiver operating characteristic curve (AUROC). Disease activity in RA was measured using a composite index score. Results: A total of 578 UH patients (mean [SD] age, 57 [15] years; 477 [82.5%] female; 296 [51.2%] white) and 242 SNH patients (mean [SD] age, 60 [15] years; 195 [80.6%] female; 30 [12.4%] white) were included in the study. Patients at the UH compared with those at the SNH were seen more frequently (median time between visits, 100 vs 180 days) and were more frequently prescribed higher-class medications (biologics) (364 [63.0%] vs 70 [28.9%]). At the UH, the model reached an AUROC of 0.91 (95% CI, 0.86-0.96) in a test cohort of 116 patients. The UH-trained model had an AUROC of 0.74 (95% CI, 0.65-0.83) in the SNH test cohort (n = 117) despite marked differences in the patient populations. In both settings, baseline prediction using each patients' most recent disease activity score had statistically random performance. Conclusions and Relevance: The findings suggest that building accurate models to forecast complex disease outcomes using electronic health record data is possible and these models can be shared across hospitals with diverse patient populations.

A systematic assessment of linking gene expression with genetic variants for prioritizing candidate targets.

Gene expression and disease-associated variants are often used to prioritize candidate genes for target validation. However, the success of these gene features alone or in combination in the discovery of therapeutic targets is uncertain. Here we evaluated the effectiveness of the differential expression (DE), the disease-associated single nucleotide polymorphisms (SNPs) and the combination of the two in recovering and predicting known therapeutic targets across 56 human diseases. We demonstrate that the performance of each feature varies across diseases and generally the features have more recovery power than predictive power. The combination of the two features, however, has significantly higher predictive power than each feature alone. Our study provides a systematic evaluation of two common gene features, DE and SNPs, for prioritization of candidate targets and identified an improved predictive power of coupling these two features.

Systematic evaluation of personal genome services for Japanese individuals.

Disease risk prediction (DRP) is one of the most important challenges in personal genome research. Although many direct-to-consumer genetic test (DTC) companies have begun to offer personal genome services for DRP, there is still no consensus on what constitutes a gold-standard service. Here, we systematically evaluated the distributions of DRPs from three DTC companies, that is, 23andMe, Navigenics and deCODEme, for 22 diseases using three Japanese samples. We systematically quantified and analyzed the differences between each DTC company's DRPs. Our independency test showed that the overall prediction results were correlated with each other, but not perfectly matched; less than onethird mismatching of the opposite direction occurred in eight diseases. Moreover, we found that the differences could mainly be attributed to four factors: (1) single nucleotide polymorphism (SNP) selection, (2) average risk estimation, (3) the disease risk calculation algorithm and (4) ethnicity adjustment. In particular, only 7.1% of SNPs over 22 diseases were reviewed by all three companies. Therefore, development of a universal core SNPs list for non-Caucasian samples will be important for achieving better prediction capacity for Japanese samples. This systematic methodology provides useful insights for improving the capacity of DRPs in future personal genome services.

Systematic functional regulatory assessment of disease-associated variants.

Genome-wide association studies have discovered many genetic loci associated with disease traits, but the functional molecular basis of these associations is often unresolved. Genome-wide regulatory and gene expression profiles measured across individuals and diseases reflect downstream effects of genetic variation and may allow for functional assessment of disease-associated loci. Here, we present a unique approach for systematic integration of genetic disease associations, transcription factor binding among individuals, and gene expression data to assess the functional consequences of variants associated with hundreds of human diseases. In an analysis of genome-wide binding profiles of NFκB, we find that disease-associated SNPs are enriched in NFκB binding regions overall, and specifically for inflammatory-mediated diseases, such as asthma, rheumatoid arthritis, and coronary artery disease. Using genome-wide variation in transcription factor-binding data, we find that NFκB binding is often correlated with disease-associated variants in a genotype-specific and allele-specific manner. Furthermore, we show that this binding variation is often related to expression of nearby genes, which are also found to have altered expression in independent profiling of the variant-associated disease condition. Thus, using this integrative approach, we provide a unique means to assign putative function to many disease-associated SNPs.

Systematic evaluation of environmental factors: persistent pollutants and nutrients correlated with serum lipid levels.

BACKGROUND: Both genetic and environmental factors contribute to triglyceride, low-density lipoprotein-cholesterol (LDL-C), and high-density lipoprotein-cholesterol (HDL-C) levels. Although genome-wide association studies are currently testing the genetic factors systematically, testing and reporting one or a few factors at a time can lead to fragmented literature for environmental chemical factors. We screened for correlation between environmental factors and lipid levels, utilizing four independent surveys with information on 188 environmental factors from the Centers of Disease Control, National Health and Nutrition Examination Survey, collected between 1999 and 2006. METHODS: We used linear regression to correlate each environmental chemical factor to triglycerides, LDL-C and HDL-C adjusting for age, age(2), sex, ethnicity, socio-economic status and body mass index. Final estimates were adjusted for waist circumference, diabetes status, blood pressure and survey. Multiple comparisons were controlled for by estimating the false discovery rate and significant findings were tentatively validated in an independent survey. RESULTS: We identified and validated 29, 9 and 17 environmental factors correlated with triglycerides, LDL-C and HDL-C levels, respectively. Findings include hydrocarbons and nicotine associated with lower HDL-C and vitamin E (γ-tocopherol) associated with unfavourable lipid levels. Higher triglycerides and lower HDL-C were correlated with higher levels of fat-soluble contaminants (e.g. polychlorinated biphenyls and dibenzofurans). Nutrients and vitamin markers (e.g. vitamins B, D and carotenes), were associated with favourable triglyceride and HDL-C levels. CONCLUSIONS: Our systematic association study has enabled us to postulate about broad environmental correlation to lipid levels. Although subject to confounding and reverse causality bias, these findings merit evaluation in additional cohorts.

Exploiting drug-disease relationships for computational drug repositioning.

Finding new uses for existing drugs, or drug repositioning, has been used as a strategy for decades to get drugs to more patients. As the ability to measure molecules in high-throughput ways has improved over the past decade, it is logical that such data might be useful for enabling drug repositioning through computational methods. Many computational predictions for new indications have been borne out in cellular model systems, though extensive animal model and clinical trial-based validation are still pending. In this review, we show that computational methods for drug repositioning can be classified in two axes: drug based, where discovery initiates from the chemical perspective, or disease based, where discovery initiates from the clinical perspective of disease or its pathology. Newer algorithms for computational drug repositioning will likely span these two axes, will take advantage of newer types of molecular measurements, and will certainly play a role in reducing the global burden of disease.

Clinical assessment incorporating a personal genome.

BACKGROUND: The cost of genomic information has fallen steeply, but the clinical translation of genetic risk estimates remains unclear. We aimed to undertake an integrated analysis of a complete human genome in a clinical context. METHODS: We assessed a patient with a family history of vascular disease and early sudden death. Clinical assessment included analysis of this patient's full genome sequence, risk prediction for coronary artery disease, screening for causes of sudden cardiac death, and genetic counselling. Genetic analysis included the development of novel methods for the integration of whole genome and clinical risk. Disease and risk analysis focused on prediction of genetic risk of variants associated with mendelian disease, recognised drug responses, and pathogenicity for novel variants. We queried disease-specific mutation databases and pharmacogenomics databases to identify genes and mutations with known associations with disease and drug response. We estimated post-test probabilities of disease by applying likelihood ratios derived from integration of multiple common variants to age-appropriate and sex-appropriate pre-test probabilities. We also accounted for gene-environment interactions and conditionally dependent risks. FINDINGS: Analysis of 2.6 million single nucleotide polymorphisms and 752 copy number variations showed increased genetic risk for myocardial infarction, type 2 diabetes, and some cancers. We discovered rare variants in three genes that are clinically associated with sudden cardiac death-TMEM43, DSP, and MYBPC3. A variant in LPA was consistent with a family history of coronary artery disease. The patient had a heterozygous null mutation in CYP2C19 suggesting probable clopidogrel resistance, several variants associated with a positive response to lipid-lowering therapy, and variants in CYP4F2 and VKORC1 that suggest he might have a low initial dosing requirement for warfarin. Many variants of uncertain importance were reported. INTERPRETATION: Although challenges remain, our results suggest that whole-genome sequencing can yield useful and clinically relevant information for individual patients. FUNDING: National Institute of General Medical Sciences; National Heart, Lung And Blood Institute; National Human Genome Research Institute; Howard Hughes Medical Institute; National Library of Medicine, Lucile Packard Foundation for Children's Health; Hewlett Packard Foundation; Breetwor Family Foundation.

Translational bioinformatics: coming of age.

The American Medical Informatics Association (AMIA) recently augmented the scope of its activities to encompass translational bioinformatics as a third major domain of informatics. The AMIA has defined translational bioinformatics as "... the development of storage, analytic, and interpretive methods to optimize the transformation of increasingly voluminous biomedical data into proactive, predictive, preventative, and participatory health." In this perspective, I will list eight reasons why this is an excellent time to be studying translational bioinformatics, including the significant increase in funding opportunities available for informatics from the United States National Institutes of Health, and the explosion of publicly-available data sets of molecular measurements. I end with the significant challenges we face in building a community of future investigators in Translational Bioinformatics.