Predicting Risk of Alzheimer's Diseases and Related Dementias with AI Foundation Model on Electronic Health Records.

Early identification of Alzheimer's disease (AD) and AD-related dementias (ADRD) has high clinical significance, both because of the potential to slow decline through initiating FDA-approved therapies and managing modifiable risk factors, and to help persons living with dementia and their families to plan before cognitive loss makes doing so challenging. However, substantial racial and ethnic disparities in early diagnosis currently lead to additional inequities in care, urging accurate and inclusive risk assessment programs. In this study, we trained an artificial intelligence foundation model to represent the electronic health records (EHR) data with a vast cohort of 1.2 million patients within a large health system. Building upon this foundation EHR model, we developed a predictive Transformer model, named TRADE, capable of identifying risks for AD/ADRD and mild cognitive impairment (MCI), by analyzing the past sequential visit records. Amongst individuals 65 and older, our model was able to generate risk predictions for various future timeframes. On the held-out validation set, our model achieved an area under the receiver operating characteristic (AUROC) of 0.772 (95% CI: 0.770, 0.773) for identifying the AD/ADRD/MCI risks in 1 year, and AUROC of 0.735 (95% CI: 0.734, 0.736) in 5 years. The positive predictive values (PPV) in 5 years among individuals with top 1% and 5% highest estimated risks were 39.2% and 27.8%, respectively. These results demonstrate significant improvements upon the current EHR-based AD/ADRD/MCI risk assessment models, paving the way for better prognosis and management of AD/ADRD/MCI at scale.

Evaluating Large Language Models in Extracting Cognitive Exam Dates and Scores.

Importance: Large language models (LLMs) are crucial for medical tasks. Ensuring their reliability is vital to avoid false results. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR. Objective: Evaluate ChatGPT and LlaMA-2 performance in extracting MMSE and CDR scores, including their associated dates. Methods: Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation. Results: For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date. Conclusions: In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

Deep learning integrates histopathology and proteogenomics at a pan-cancer level.

We introduce a pioneering approach that integrates pathology imaging with transcriptomics and proteomics to identify predictive histology features associated with critical clinical outcomes in cancer. We utilize 2,755 H&E-stained histopathological slides from 657 patients across 6 cancer types from CPTAC. Our models effectively recapitulate distinctions readily made by human pathologists: tumor vs. normal (AUROC = 0.995) and tissue-of-origin (AUROC = 0.979). We further investigate predictive power on tasks not normally performed from H&E alone, including TP53 prediction and pathologic stage. Importantly, we describe predictive morphologies not previously utilized in a clinical setting. The incorporation of transcriptomics and proteomics identifies pathway-level signatures and cellular processes driving predictive histology features. Model generalizability and interpretability is confirmed using TCGA. We propose a classification system for these tasks, and suggest potential clinical applications for this integrated human and machine learning approach. A publicly available web-based platform implements these models.

On gaps of clinical diagnosis of dementia subtypes: A study of Alzheimer's disease and Lewy body disease.

Introduction: Alzheimer's disease (AD) and Lewy body disease (LBD) are the two most common neurodegenerative dementias and can occur in combination (AD+LBD). Due to overlapping biomarkers and symptoms, clinical differentiation of these subtypes could be difficult. However, it is unclear how the magnitude of diagnostic uncertainty varies across dementia spectra and demographic variables. We aimed to compare clinical diagnosis and post-mortem autopsy-confirmed pathological results to assess the clinical subtype diagnosis quality across these factors. Methods: We studied data of 1,920 participants recorded by the National Alzheimer's Coordinating Center from 2005 to 2019. Selection criteria included autopsy-based neuropathological assessments for AD and LBD, and the initial visit with Clinical Dementia Rating (CDR) stage of normal, mild cognitive impairment, or mild dementia. Longitudinally, we analyzed the first visit at each subsequent CDR stage. This analysis included positive predictive values, specificity, sensitivity and false negative rates of clinical diagnosis, as well as disparities by sex, race, age, and education. If autopsy-confirmed AD and/or LBD was missed in the clinic, the alternative clinical diagnosis was analyzed. Findings: In our findings, clinical diagnosis of AD+LBD had poor sensitivities. Over 61% of participants with autopsy-confirmed AD+LBD were diagnosed clinically as AD. Clinical diagnosis of AD had a low sensitivity at the early dementia stage and low specificities at all stages. Among participants diagnosed as AD in the clinic, over 32% had concurrent LBD neuropathology at autopsy. Among participants diagnosed as LBD, 32% to 54% revealed concurrent autopsy-confirmed AD pathology. When three subtypes were missed by clinicians, "No cognitive impairment" and "primary progressive aphasia or behavioral variant frontotemporal dementia" were the leading primary etiologic clinical diagnoses. With increasing dementia stages, the clinical diagnosis accuracy of black participants became significantly worse than other races, and diagnosis quality significantly improved for males but not females. Discussion: These findings demonstrate that clinical diagnosis of AD, LBD, and AD+LBD are inaccurate and suffer from significant disparities on race and sex. They provide important implications for clinical management, anticipatory guidance, trial enrollment and applicability of potential therapies for AD, and promote research into better biomarker-based assessment of LBD pathology.

Deep multi-task learning and random forest for series classification by pulse sequence type and orientation.

PURPOSE: Increasingly complex MRI studies and variable series naming conventions reveal limitations of rule-based image routing, especially in health systems with multiple scanners and sites. Accurate methods to identify series based on image content would aid post-processing and PACS viewing. Recent deep/machine learning efforts classify 5-8 basic brain MR sequences. We present an ensemble model combining a convolutional neural network and a random forest classifier to differentiate 25 brain sequences and image orientation. METHODS: Series were grouped by descriptions into 25 sequences and 4 orientations. Dataset A, obtained from our institution, was divided into training (16,828 studies; 48,512 series; 112,028 images), validation (4746 studies; 16,612 series; 26,222 images) and test sets (6348 studies; 58,705 series; 3,314,018 images). Dataset B, obtained from a separate hospital, was used for out-of-domain external validation (1252 studies; 2150 series; 234,944 images). We developed an ensemble model combining a 2D convolutional neural network with a custom multi-task learning architecture and random forest classifier trained on DICOM metadata to classify sequence and orientation by series. RESULTS: The neural network, random forest, and ensemble achieved 95%, 97%, and 98% overall sequence accuracy on dataset A, and 98%, 99%, and 99% accuracy on dataset B, respectively. All models achieved > 99% orientation accuracy on both datasets. CONCLUSION: The ensemble model for series identification accommodates the complexity of brain MRI studies in state-of-the-art clinical practice. Expanding on previous work demonstrating proof-of-concept, our approach is more comprehensive with greater sequence diversity and orientation classification.

Generalizable deep learning model for early Alzheimer's disease detection from structural MRIs.

Early diagnosis of Alzheimer's disease plays a pivotal role in patient care and clinical trials. In this study, we have developed a new approach based on 3D deep convolutional neural networks to accurately differentiate mild Alzheimer's disease dementia from mild cognitive impairment and cognitively normal individuals using structural MRIs. For comparison, we have built a reference model based on the volumes and thickness of previously reported brain regions that are known to be implicated in disease progression. We validate both models on an internal held-out cohort from The Alzheimer's Disease Neuroimaging Initiative (ADNI) and on an external independent cohort from The National Alzheimer's Coordinating Center (NACC). The deep-learning model is accurate, achieved an area-under-the-curve (AUC) of 85.12 when distinguishing between cognitive normal subjects and subjects with either MCI or mild Alzheimer's dementia. In the more challenging task of detecting MCI, it achieves an AUC of 62.45. It is also significantly faster than the volume/thickness model in which the volumes and thickness need to be extracted beforehand. The model can also be used to forecast progression: subjects with mild cognitive impairment misclassified as having mild Alzheimer's disease dementia by the model were faster to progress to dementia over time. An analysis of the features learned by the proposed model shows that it relies on a wide range of regions associated with Alzheimer's disease. These findings suggest that deep neural networks can automatically learn to identify imaging biomarkers that are predictive of Alzheimer's disease, and leverage them to achieve accurate early detection of the disease.

Evaluating the Effect of a COVID-19 Predictive Model to Facilitate Discharge: A Randomized Controlled Trial.

BACKGROUND: We previously developed and validated a predictive model to help clinicians identify hospitalized adults with coronavirus disease 2019 (COVID-19) who may be ready for discharge given their low risk of adverse events. Whether this algorithm can prompt more timely discharge for stable patients in practice is unknown. OBJECTIVES: The aim of the study is to estimate the effect of displaying risk scores on length of stay (LOS). METHODS: We integrated model output into the electronic health record (EHR) at four hospitals in one health system by displaying a green/orange/red score indicating low/moderate/high-risk in a patient list column and a larger COVID-19 summary report visible for each patient. Display of the score was pseudo-randomized 1:1 into intervention and control arms using a patient identifier passed to the model execution code. Intervention effect was assessed by comparing LOS between intervention and control groups. Adverse safety outcomes of death, hospice, and re-presentation were tested separately and as a composite indicator. We tracked adoption and sustained use through daily counts of score displays. RESULTS: Enrolling 1,010 patients from May 15, 2020 to December 7, 2020, the trial found no detectable difference in LOS. The intervention had no impact on safety indicators of death, hospice or re-presentation after discharge. The scores were displayed consistently throughout the study period but the study lacks a causally linked process measure of provider actions based on the score. Secondary analysis revealed complex dynamics in LOS temporally, by primary symptom, and hospital location. CONCLUSION: An AI-based COVID-19 risk score displayed passively to clinicians during routine care of hospitalized adults with COVID-19 was safe but had no detectable impact on LOS. Health technology challenges such as insufficient adoption, nonuniform use, and provider trust compounded with temporal factors of the COVID-19 pandemic may have contributed to the null result. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT04570488.

Artificial intelligence and deep learning to map immune cell types in inflamed human tissue.

Biopsies of inflammatory tissue contain a complex network of interacting cells, orchestrating the immune or autoimmune response. While standard histological examination can identify relationships, it is clear that a great amount of data on each slide is not quantitated or categorized in standard microscopic examinations. To deal with the huge amount of data present in biopsy tissue in an unbiased and comprehensive way, we have developed a deep learning algorithm to identify immune cells in biopsies of inflammatory lesions. We focused on T follicular helper (Tfh) cell subsets and B cells in dermatomyositis biopsy images. We achieved strong performance on detection and classification of cells, including the rare Tfh cell subsets present in the tissue. This algorithm could be used to perform distance mapping between cell types in tissue, and could be easily adapted to other disease states.

Deep Learning and Pathomics Analyses Reveal Cell Nuclei as Important Features for Mutation Prediction of BRAF-Mutated Melanomas.

Image-based analysis as a method for mutation detection can be advantageous in settings when tumor tissue is limited or unavailable for direct testing. In this study, we utilize two distinct and complementary machine-learning methods of analyzing whole-slide images for predicting mutated BRAF. In the first method, whole-slide images of melanomas from 256 patients were used to train a deep convolutional neural network to develop a fully automated model that first selects for tumor-rich areas (area under the curve = 0.96) and then predicts for mutated BRAF (area under the curve = 0.71). Saliency mapping was performed and revealed that pixels corresponding to nuclei were the most relevant to network learning. In the second method, whole-slide images were analyzed using a pathomics pipeline that first annotates nuclei and then quantifies nuclear features, showing that mutated BRAF nuclei were significantly larger and rounder than BRAF‒wild-type nuclei. Finally, we developed a model that combines clinical information, deep learning, and pathomics that improves the predictive performance for mutated BRAF to an area under the curve of 0.89. Not only does this provide additional insights on how BRAF mutations affect tumor structural characteristics, but machine learning‒based analysis of whole-slide images also has the potential to be integrated into higher-order models for understanding tumor biology.

Predicting endometrial cancer subtypes and molecular features from histopathology images using multi-resolution deep learning models.

The determination of endometrial carcinoma histological subtypes, molecular subtypes, and mutation status is critical for the diagnostic process, and directly affects patients' prognosis and treatment. Sequencing, albeit slower and more expensive, can provide additional information on molecular subtypes and mutations that can be used to better select treatments. Here, we implement a customized multi-resolution deep convolutional neural network, Panoptes, that predicts not only the histological subtypes but also the molecular subtypes and 18 common gene mutations based on digitized H&E-stained pathological images. The model achieves high accuracy and generalizes well on independent datasets. Our results suggest that Panoptes, with further refinement, has the potential for clinical application to help pathologists determine molecular subtypes and mutations of endometrial carcinoma without sequencing.

An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department.

During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3661 patients, achieves an area under the receiver operating characteristic curve (AUC) of 0.786 (95% CI: 0.745-0.830) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients.

Association of Psychiatric Disorders With Mortality Among Patients With COVID-19.

Importance: To date, the association of psychiatric diagnoses with mortality in patients infected with coronavirus disease 2019 (COVID-19) has not been evaluated. Objective: To assess whether a diagnosis of a schizophrenia spectrum disorder, mood disorder, or anxiety disorder is associated with mortality in patients with COVID-19. Design, Setting, and Participants: This retrospective cohort study assessed 7348 consecutive adult patients for 45 days following laboratory-confirmed COVID-19 between March 3 and May 31, 2020, in a large academic medical system in New York. The final date of follow-up was July 15, 2020. Patients without available medical records before testing were excluded. Exposures: Patients were categorized based on the following International Statistical Classification of Diseases, Tenth Revision, Clinical Modification diagnoses before their testing date: (1) schizophrenia spectrum disorders, (2) mood disorders, and (3) anxiety disorders. Patients with these diagnoses were compared with a reference group without psychiatric disorders. Main Outcomes and Measures: Mortality, defined as death or discharge to hospice within 45 days following a positive severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) test result. Results: Of the 26 540 patients tested, 7348 tested positive for SARS-CoV-2 (mean [SD] age, 54 [18.6] years; 3891 [53.0%] women). Of eligible patients with positive test results, 75 patients (1.0%) had a history of a schizophrenia spectrum illness, 564 (7.7%) had a history of a mood disorder, and 360 (4.9%) had a history of an anxiety disorder. After adjusting for demographic and medical risk factors, a premorbid diagnosis of a schizophrenia spectrum disorder was significantly associated with mortality (odds ratio [OR], 2.67; 95% CI, 1.48-4.80). Diagnoses of mood disorders (OR, 1.14; 95% CI, 0.87-1.49) and anxiety disorders (OR, 0.96; 95% CI, 0.65-1.41) were not associated with mortality after adjustment. In comparison with other risk factors, a diagnosis of schizophrenia ranked behind only age in strength of an association with mortality. Conclusions and Relevance: In this cohort study of adults with SARS-CoV-2-positive test results in a large New York medical system, adults with a schizophrenia spectrum disorder diagnosis were associated with an increased risk for mortality, but those with mood and anxiety disorders were not associated with a risk of mortality. These results suggest that schizophrenia spectrum disorders may be a risk factor for mortality in patients with COVID-19.

A validated, real-time prediction model for favorable outcomes in hospitalized COVID-19 patients.

The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.

An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department.

During the coronavirus disease 2019 (COVID-19) pandemic, rapid and accurate triage of patients at the emergency department is critical to inform decision-making. We propose a data-driven approach for automatic prediction of deterioration risk using a deep neural network that learns from chest X-ray images and a gradient boosting model that learns from routine clinical variables. Our AI prognosis system, trained using data from 3,661 patients, achieves an area under the receiver operating characteristic curve (AUC) of 0.786 (95% CI: 0.745-0.830) when predicting deterioration within 96 hours. The deep neural network extracts informative areas of chest X-ray images to assist clinicians in interpreting the predictions and performs comparably to two radiologists in a reader study. In order to verify performance in a real clinical setting, we silently deployed a preliminary version of the deep neural network at New York University Langone Health during the first wave of the pandemic, which produced accurate predictions in real-time. In summary, our findings demonstrate the potential of the proposed system for assisting front-line physicians in the triage of COVID-19 patients.

Artificial Intelligence Explained for Nonexperts.

Artificial intelligence (AI) has made stunning progress in the last decade, made possible largely due to the advances in training deep neural networks with large data sets. Many of these solutions, initially developed for natural images, speech, or text, are now becoming successful in medical imaging. In this article we briefly summarize in an accessible way the current state of the field of AI. Furthermore, we highlight the most promising approaches and describe the current challenges that will need to be solved to enable broad deployment of AI in clinical practice.

Correction: Predicting childhood obesity using electronic health records and publicly available data.

[This corrects the article DOI: 10.1371/journal.pone.0215571.].

Predicting childhood obesity using electronic health records and publicly available data.

BACKGROUND: Because of the strong link between childhood obesity and adulthood obesity comorbidities, and the difficulty in decreasing body mass index (BMI) later in life, effective strategies are needed to address this condition in early childhood. The ability to predict obesity before age five could be a useful tool, allowing prevention strategies to focus on high risk children. The few existing prediction models for obesity in childhood have primarily employed data from longitudinal cohort studies, relying on difficult to collect data that are not readily available to all practitioners. Instead, we utilized real-world unaugmented electronic health record (EHR) data from the first two years of life to predict obesity status at age five, an approach not yet taken in pediatric obesity research. METHODS AND FINDINGS: We trained a variety of machine learning algorithms to perform both binary classification and regression. Following previous studies demonstrating different obesity determinants for boys and girls, we similarly developed separate models for both groups. In each of the separate models for boys and girls we found that weight for length z-score, BMI between 19 and 24 months, and the last BMI measure recorded before age two were the most important features for prediction. The best performing models were able to predict obesity with an Area Under the Receiver Operator Characteristic Curve (AUC) of 81.7% for girls and 76.1% for boys. CONCLUSIONS: We were able to predict obesity at age five using EHR data with an AUC comparable to cohort-based studies, reducing the need for investment in additional data collection. Our results suggest that machine learning approaches for predicting future childhood obesity using EHR data could improve the ability of clinicians and researchers to drive future policy, intervention design, and the decision-making process in a clinical setting.