Implications of the Use of Artificial Intelligence Predictive Models in Health Care Settings : A Simulation Study.

BACKGROUND: Substantial effort has been directed toward demonstrating uses of predictive models in health care. However, implementation of these models into clinical practice may influence patient outcomes, which in turn are captured in electronic health record data. As a result, deployed models may affect the predictive ability of current and future models. OBJECTIVE: To estimate changes in predictive model performance with use through 3 common scenarios: model retraining, sequentially implementing 1 model after another, and intervening in response to a model when 2 are simultaneously implemented. DESIGN: Simulation of model implementation and use in critical care settings at various levels of intervention effectiveness and clinician adherence. Models were either trained or retrained after simulated implementation. SETTING: Admissions to the intensive care unit (ICU) at Mount Sinai Health System (New York, New York) and Beth Israel Deaconess Medical Center (Boston, Massachusetts). PATIENTS: 130 000 critical care admissions across both health systems. INTERVENTION: Across 3 scenarios, interventions were simulated at varying levels of clinician adherence and effectiveness. MEASUREMENTS: Statistical measures of performance, including threshold-independent (area under the curve) and threshold-dependent measures. RESULTS: At fixed 90% sensitivity, in scenario 1 a mortality prediction model lost 9% to 39% specificity after retraining once and in scenario 2 a mortality prediction model lost 8% to 15% specificity when created after the implementation of an acute kidney injury (AKI) prediction model; in scenario 3, models for AKI and mortality prediction implemented simultaneously, each led to reduced effective accuracy of the other by 1% to 28%. LIMITATIONS: In real-world practice, the effectiveness of and adherence to model-based recommendations are rarely known in advance. Only binary classifiers for tabular ICU admissions data were simulated. CONCLUSION: In simulated ICU settings, a universally effective model-updating approach for maintaining model performance does not seem to exist. Model use may have to be recorded to maintain viability of predictive modeling. PRIMARY FUNDING SOURCE: National Center for Advancing Translational Sciences.

Functional Effects of Cardiomyocyte Injury in COVID-19.

COVID-19 affects multiple organs. Clinical data from the Mount Sinai Health System show that substantial numbers of COVID-19 patients without prior heart disease develop cardiac dysfunction. How COVID-19 patients develop cardiac disease is not known. We integrated cell biological and physiological analyses of human cardiomyocytes differentiated from human induced pluripotent stem cells (hiPSCs) infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in the presence of interleukins (ILs) with clinical findings related to laboratory values in COVID-19 patients to identify plausible mechanisms of cardiac disease in COVID-19 patients. We infected hiPSC-derived cardiomyocytes from healthy human subjects with SARS-CoV-2 in the absence and presence of IL-6 and IL-1ß. Infection resulted in increased numbers of multinucleated cells. Interleukin treatment and infection resulted in disorganization of myofibrils, extracellular release of troponin I, and reduced and erratic beating. Infection resulted in decreased expression of mRNA encoding key proteins of the cardiomyocyte contractile apparatus. Although interleukins did not increase the extent of infection, they increased the contractile dysfunction associated with viral infection of cardiomyocytes, resulting in cessation of beating. Clinical data from hospitalized patients from the Mount Sinai Health System show that a significant portion of COVID-19 patients without history of heart disease have elevated troponin and interleukin levels. A substantial subset of these patients showed reduced left ventricular function by echocardiography. Our laboratory observations, combined with the clinical data, indicate that direct effects on cardiomyocytes by interleukins and SARS-CoV-2 infection might underlie heart disease in COVID-19 patients. IMPORTANCE SARS-CoV-2 infects multiple organs, including the heart. Analyses of hospitalized patients show that a substantial number without prior indication of heart disease or comorbidities show significant injury to heart tissue, assessed by increased levels of troponin in blood. We studied the cell biological and physiological effects of virus infection of healthy human iPSC-derived cardiomyocytes in culture. Virus infection with interleukins disorganizes myofibrils, increases cell size and the numbers of multinucleated cells, and suppresses the expression of proteins of the contractile apparatus. Viral infection of cardiomyocytes in culture triggers release of troponin similar to elevation in levels of COVID-19 patients with heart disease. Viral infection in the presence of interleukins slows down and desynchronizes the beating of cardiomyocytes in culture. The cell-level physiological changes are similar to decreases in left ventricular ejection seen in imaging of patients' hearts. These observations suggest that direct injury to heart tissue by virus can be one underlying cause of heart disease in COVID-19.

GUÍA: a digital platform to facilitate result disclosure in genetic counseling.

PURPOSE: Use of genomic sequencing is increasing at a pace that requires technological solutions to effectively meet the needs of a growing patient population. We developed GUÍA, a web-based application, to enhance the delivery of genomic results and related clinical information to patients and families. METHODS: GUÍA development occurred in five overlapping phases: formative research, content development, stakeholder/community member input, user interface design, and web application development. Development was informed by formative qualitative research involving parents (N = 22) whose children underwent genomic testing. Participants enrolled in the NYCKidSeq pilot study (N = 18) completed structured feedback interviews post-result disclosure using GUÍA. Genetic specialists, researchers, patients, and community stakeholders provided their perspectives on GUÍA's design to ensure technical, cultural, and literacy appropriateness. RESULTS: NYCKidSeq participants responded positively to the use of GUÍA to deliver their children's results. All participants (N = 10) with previous experience with genetic testing felt GUÍA improved result disclosure, and 17 (94%) participants said the content was clear. CONCLUSION: GUÍA communicates complex genomic information in an understandable and personalized manner. Initial piloting demonstrated GUÍA's utility for families enrolled in the NYCKidSeq pilot study. Findings from the NYCKidSeq clinical trial will provide insight into GUÍA's effectiveness in communicating results among diverse, multilingual populations.

Machine Learning to Predict Mortality and Critical Events in a Cohort of Patients With COVID-19 in New York City: Model Development and Validation.

BACKGROUND: COVID-19 has infected millions of people worldwide and is responsible for several hundred thousand fatalities. The COVID-19 pandemic has necessitated thoughtful resource allocation and early identification of high-risk patients. However, effective methods to meet these needs are lacking. OBJECTIVE: The aims of this study were to analyze the electronic health records (EHRs) of patients who tested positive for COVID-19 and were admitted to hospitals in the Mount Sinai Health System in New York City; to develop machine learning models for making predictions about the hospital course of the patients over clinically meaningful time horizons based on patient characteristics at admission; and to assess the performance of these models at multiple hospitals and time points. METHODS: We used Extreme Gradient Boosting (XGBoost) and baseline comparator models to predict in-hospital mortality and critical events at time windows of 3, 5, 7, and 10 days from admission. Our study population included harmonized EHR data from five hospitals in New York City for 4098 COVID-19-positive patients admitted from March 15 to May 22, 2020. The models were first trained on patients from a single hospital (n=1514) before or on May 1, externally validated on patients from four other hospitals (n=2201) before or on May 1, and prospectively validated on all patients after May 1 (n=383). Finally, we established model interpretability to identify and rank variables that drive model predictions. RESULTS: Upon cross-validation, the XGBoost classifier outperformed baseline models, with an area under the receiver operating characteristic curve (AUC-ROC) for mortality of 0.89 at 3 days, 0.85 at 5 and 7 days, and 0.84 at 10 days. XGBoost also performed well for critical event prediction, with an AUC-ROC of 0.80 at 3 days, 0.79 at 5 days, 0.80 at 7 days, and 0.81 at 10 days. In external validation, XGBoost achieved an AUC-ROC of 0.88 at 3 days, 0.86 at 5 days, 0.86 at 7 days, and 0.84 at 10 days for mortality prediction. Similarly, the unimputed XGBoost model achieved an AUC-ROC of 0.78 at 3 days, 0.79 at 5 days, 0.80 at 7 days, and 0.81 at 10 days. Trends in performance on prospective validation sets were similar. At 7 days, acute kidney injury on admission, elevated LDH, tachypnea, and hyperglycemia were the strongest drivers of critical event prediction, while higher age, anion gap, and C-reactive protein were the strongest drivers of mortality prediction. CONCLUSIONS: We externally and prospectively trained and validated machine learning models for mortality and critical events for patients with COVID-19 at different time horizons. These models identified at-risk patients and uncovered underlying relationships that predicted outcomes.

Pharmacological risk factors associated with hospital readmission rates in a psychiatric cohort identified using prescriptome data mining.

BACKGROUND: Worldwide, over 14% of individuals hospitalized for psychiatric reasons have readmissions to hospitals within 30 days after discharge. Predicting patients at risk and leveraging accelerated interventions can reduce the rates of early readmission, a negative clinical outcome (i.e., a treatment failure) that affects the quality of life of patient. To implement individualized interventions, it is necessary to predict those individuals at highest risk for 30-day readmission. In this study, our aim was to conduct a data-driven investigation to find the pharmacological factors influencing 30-day all-cause, intra- and interdepartmental readmissions after an index psychiatric admission, using the compendium of prescription data (prescriptome) from electronic medical records (EMR). METHODS: The data scientists in the project received a deidentified database from the Mount Sinai Data Warehouse, which was used to perform all analyses. Data was stored in a secured MySQL database, normalized and indexed using a unique hexadecimal identifier associated with the data for psychiatric illness visits. We used Bayesian logistic regression models to evaluate the association of prescription data with 30-day readmission risk. We constructed individual models and compiled results after adjusting for covariates, including drug exposure, age, and gender. We also performed digital comorbidity survey using EMR data combined with the estimation of shared genetic architecture using genomic annotations to disease phenotypes. RESULTS: Using an automated, data-driven approach, we identified prescription medications, side effects (primary side effects), and drug-drug interaction-induced side effects (secondary side effects) associated with readmission risk in a cohort of 1275 patients using prescriptome analytics. In our study, we identified 28 drugs associated with risk for readmission among psychiatric patients. Based on prescription data, Pravastatin had the highest risk of readmission (OR = 13.10; 95% CI (2.82, 60.8)). We also identified enrichment of primary side effects (n = 4006) and secondary side effects (n = 36) induced by prescription drugs in the subset of readmitted patients (n = 89) compared to the non-readmitted subgroup (n = 1186). Digital comorbidity analyses and shared genetic analyses further reveals that cardiovascular disease and psychiatric conditions are comorbid and share functional gene modules (cardiomyopathy and anxiety disorder: shared genes (n = 37; P = 1.06815E-06)). CONCLUSIONS: Large scale prescriptome data is now available from EMRs and accessible for analytics that could improve healthcare outcomes. Such analyses could also drive hypothesis and data-driven research. In this study, we explored the utility of prescriptome data to identify factors driving readmission in a psychiatric cohort. Converging digital health data from EMRs and systems biology investigations reveal a subset of patient populations that have significant comorbidities with cardiovascular diseases are more likely to be readmitted. Further, the genetic architecture of psychiatric illness also suggests overlap with cardiovascular diseases. In summary, assessment of medications, side effects, and drug-drug interactions in a clinical setting as well as genomic information using a data mining approach could help to find factors that could help to lower readmission rates in patients with mental illness.

Big and disparate data: considerations for pediatric consortia.

PURPOSE OF REVIEW: Increasingly, there is a need for examining exposure disease associations in large, diverse datasets to understand the complex determinants of pediatric disease and disability. Recognizing that children's health research consortia will be important sources of big data, it is crucial for the pediatric research community to be knowledgeable about the challenges and opportunities that they will face. The present review will provide examples of existing children's health consortia, highlight recent pooled analyses conducted by children's health research consortia, address common challenges of pooled analyses, and provide recommendations to advance collective research efforts in pediatric research. RECENT FINDINGS: Formal consortia and other collective-science initiatives are increasingly being created to share individual data from a set of relevant epidemiological studies to address a common research topic under the concept that the joint effort of many individual groups can accomplish far more than working alone. There are practical challenges to the participation of investigators within consortia that need to be addressed in order for them to work. SUMMARY: Researchers who access consortia with data centers will be able to go far beyond their initial hypotheses and potentially accomplish research that was previously thought infeasible or too costly.

A novel method leveraging time series data to improve subphenotyping and application in critically ill patients with COVID-19.

Computational subphenotyping, a data-driven approach to understanding disease subtypes, is a prominent topic in medical research. Numerous ongoing studies are dedicated to developing advanced computational subphenotyping methods for cross-sectional data. However, the potential of time-series data has been underexplored until now. Here, we propose a Multivariate Levenshtein Distance (MLD) that can account for address correlation in multiple discrete features over time-series data. Our algorithm has two distinct components: it integrates an optimal threshold score to enhance the sensitivity in discriminating between pairs of instances, and the MLD itself. We have applied the proposed distance metrics on the k-means clustering algorithm to derive temporal subphenotypes from time-series data of biomarkers and treatment administrations from 1039 critically ill patients with COVID-19 and compare its effectiveness to standard methods. In conclusion, the Multivariate Levenshtein Distance metric is a novel method to quantify the distance from multiple discrete features over time-series data and demonstrates superior clustering performance among competing time-series distance metrics.

A foundational transformer leveraging full night, multichannel sleep study data accurately classifies sleep stages.

Study Objectives: To investigate whether a foundational transformer model using 8-hour, multichannel data from polysomnograms can outperform existing artificial intelligence (AI) methods for sleep stage classification. Methods: We utilized the Sleep Heart Health Study (SHHS) visits 1 and 2 for training and validation and the Multi-Ethnic Study of Atherosclerosis (MESA) for testing of our model. We trained a self-supervised foundational transformer (called PFTSleep) that encodes 8-hour long sleep studies at 125 Hz with 7 signals including brain, movement, cardiac, oxygen, and respiratory channels. These encodings are used as input for training of an additional model to classify sleep stages, without adjusting the weights of the foundational transformer. We compared our results to existing AI methods that did not utilize 8-hour data or the full set of signals but did report evaluation metrics for the SHHS dataset. Results: We trained and validated a model with 8,444 sleep studies with 7 signals including brain, movement, cardiac, oxygen, and respiratory channels and tested on an additional 2,055 studies. In total, we trained and tested 587,944 hours of sleep study signal data. Area under the precision recall curve (AUPRC) scores were 0.82, 0.40, 0.53, 0.75, and 0.82 and area under the receiving operating characteristics curve (AUROC) scores were 0.99, 0.95, 0.96, 0.98, and 0.99 for wake, N1, N2, N3, and REM, respectively, on the SHHS validation set. For MESA, the AUPRC scores were 0.56, 0.16, 0.40, 0.45, and 0.65 and AUROC scores were 0.94, 0.77, 0.87, 0.91, and 0.96, respectively. Our model was compared to the longest context window state-of-the-art model and showed increases in macro evaluation scores, notably sensitivity (3.7% increase) and multi-class REM (3.39% increase) and wake (0.97% increase) F1 scores. Conclusions: Utilizing full night, multi-channel PSG data encodings derived from a foundational transformer improve sleep stage classification over existing methods.

Local large language models for privacy-preserving accelerated review of historic echocardiogram reports.

OBJECTIVES: The study developed framework that leverages an open-source Large Language Model (LLM) to enable clinicians to ask plain-language questions about a patient's entire echocardiogram report history. This approach is intended to streamline the extraction of clinical insights from multiple echocardiogram reports, particularly in patients with complex cardiac diseases, thereby enhancing both patient care and research efficiency. MATERIALS AND METHODS: Data from over 10 years were collected, comprising echocardiogram reports from patients with more than 10 echocardiograms on file at the Mount Sinai Health System. These reports were converted into a single document per patient for analysis, broken down into snippets and relevant snippets were retrieved using text similarity measures. The LLaMA-2 70B model was employed for analyzing the text using a specially crafted prompt. The model's performance was evaluated against ground-truth answers created by faculty cardiologists. RESULTS: The study analyzed 432 reports from 37 patients for a total of 100 question-answer pairs. The LLM correctly answered 90% questions, with accuracies of 83% for temporality, 93% for severity assessment, 84% for intervention identification, and 100% for diagnosis retrieval. Errors mainly stemmed from the LLM's inherent limitations, such as misinterpreting numbers or hallucinations. CONCLUSION: The study demonstrates the feasibility and effectiveness of using a local, open-source LLM for querying and interpreting echocardiogram report data. This approach offers a significant improvement over traditional keyword-based searches, enabling more contextually relevant and semantically accurate responses; in turn showing promise in enhancing clinical decision-making and research by facilitating more efficient access to complex patient data.

Multimodal fusion learning for long QT syndrome pathogenic genotypes in a racially diverse population.

Congenital long QT syndrome (LQTS) diagnosis is complicated by limited genetic testing at scale, low prevalence, and normal QT corrected interval in patients with high-risk genotypes. We developed a deep learning approach combining electrocardiogram (ECG) waveform and electronic health record data to assess whether patients had pathogenic variants causing LQTS. We defined patients with high-risk genotypes as having ≥1 pathogenic variant in one of the LQTS-susceptibility genes. We trained the model using data from United Kingdom Biobank (UKBB) and then fine-tuned in a racially/ethnically diverse cohort using Mount Sinai BioMe Biobank. Following group-stratified 5-fold splitting, the fine-tuned model achieved area under the precision-recall curve of 0.29 (95% confidence interval [CI] 0.28-0.29) and area under the receiver operating curve of 0.83 (0.82-0.83) on independent testing data from BioMe. Multimodal fusion learning has promise to identify individuals with pathogenic genetic mutations to enable patient prioritization for further work up.

Derivation, external and clinical validation of a deep learning approach for detecting intracranial hypertension.

Increased intracranial pressure (ICP) ≥15 mmHg is associated with adverse neurological outcomes, but needs invasive intracranial monitoring. Using the publicly available MIMIC-III Waveform Database (2000-2013) from Boston, we developed an artificial intelligence-derived biomarker for elevated ICP (aICP) for adult patients. aICP uses routinely collected extracranial waveform data as input, reducing the need for invasive monitoring. We externally validated aICP with an independent dataset from the Mount Sinai Hospital (2020-2022) in New York City. The AUROC, accuracy, sensitivity, and specificity on the external validation dataset were 0.80 (95% CI, 0.80-0.80), 73.8% (95% CI, 72.0-75.6%), 73.5% (95% CI 72.5-74.5%), and 73.0% (95% CI, 72.0-74.0%), respectively. We also present an exploratory analysis showing aICP predictions are associated with clinical phenotypes. A ten-percentile increment was associated with brain malignancy (OR = 1.68; 95% CI, 1.09-2.60), intracerebral hemorrhage (OR = 1.18; 95% CI, 1.07-1.32), and craniotomy (OR = 1.43; 95% CI, 1.12-1.84; P < 0.05 for all).

Derivation, External Validation and Clinical Implications of a deep learning approach for intracranial pressure estimation using non-cranial waveform measurements.

Importance: Increased intracranial pressure (ICP) is associated with adverse neurological outcomes, but needs invasive monitoring. Objective: Development and validation of an AI approach for detecting increased ICP (aICP) using only non-invasive extracranial physiological waveform data. Design: Retrospective diagnostic study of AI-assisted detection of increased ICP. We developed an AI model using exclusively extracranial waveforms, externally validated it and assessed associations with clinical outcomes. Setting: MIMIC-III Waveform Database (2000-2013), a database derived from patients admitted to an ICU in an academic Boston hospital, was used for development of the aICP model, and to report association with neurologic outcomes. Data from Mount Sinai Hospital (2020-2022) in New York City was used for external validation. Participants: Patients were included if they were older than 18 years, and were monitored with electrocardiograms, arterial blood pressure, respiratory impedance plethysmography and pulse oximetry. Patients who additionally had intracranial pressure monitoring were used for development (N=157) and external validation (N=56). Patients without intracranial monitors were used for association with outcomes (N=1694). Exposures: Extracranial waveforms including electrocardiogram, arterial blood pressure, plethysmography and SpO2. Main Outcomes and Measures: Intracranial pressure > 15 mmHg. Measures were Area under receiver operating characteristic curves (AUROCs), sensitivity, specificity, and accuracy at threshold of 0.5. We calculated odds ratios and p-values for phenotype association. Results: The AUROC was 0.91 (95% CI, 0.90-0.91) on testing and 0.80 (95% CI, 0.80-0.80) on external validation. aICP had accuracy, sensitivity, and specificity of 73.8% (95% CI, 72.0%-75.6%), 99.5% (95% CI 99.3%-99.6%), and 76.9% (95% CI, 74.0-79.8%) on external validation. A ten-percentile increment was associated with stroke (OR=2.12; 95% CI, 1.27-3.13), brain malignancy (OR=1.68; 95% CI, 1.09-2.60), subdural hemorrhage (OR=1.66; 95% CI, 1.07-2.57), intracerebral hemorrhage (OR=1.18; 95% CI, 1.07-1.32), and procedures like percutaneous brain biopsy (OR=1.58; 95% CI, 1.15-2.18) and craniotomy (OR = 1.43; 95% CI, 1.12-1.84; P < 0.05 for all). Conclusions and Relevance: aICP provides accurate, non-invasive estimation of increased ICP, and is associated with neurological outcomes and neurosurgical procedures in patients without intracranial monitoring.

Development and Validation of a Policy Tree Approach for Optimizing Intravenous Fluids in Critically Ill Patients with Sepsis and Acute Kidney Injury.

Purpose: Intravenous fluids are mainstay of management of acute kidney injury (AKI) after sepsis but can cause fluid overload. Recent literature shows that restrictive fluid strategy may be beneficial in some patients with AKI, however, identifying these patients is challenging. We aimed to develop and validate a machine learning algorithm to identify patients who would benefit from a restrictive fluid strategy. Methods: We included patients with sepsis who developed AKI within 48 hours of ICU admission and defined restrictive fluid strategy as receiving <500mL fluids within 24 hours after AKI. Our primary outcome was early AKI reversal within 48 hours of AKI onset, and secondary outcomes included sustained AKI reversal and major adverse kidney events (MAKE) at discharge. We used a causal forest, a machine learning algorithm to estimate individual treatment effects and policy tree algorithm to identify patients who would benefit by restrictive fluid strategy. We developed the algorithm in MIMIC-IV and validated it in eICU database. Results: Among 2,091 patients in the external validation cohort, policy tree recommended restrictive fluids for 88.2%. Among these, patients who received restrictive fluids demonstrated significantly higher rate of early AKI reversal (48.2% vs 39.6%, p<0.001), sustained AKI reversal (36.7% vs 27.4%, p<0.001) and lower rates of MAKE by discharge (29.3% vs 35.1%, p=0.019). These results were consistent in adjusted analysis. Conclusion: Policy tree based on causal machine learning can identify septic patients with AKI who benefit from a restrictive fluid strategy. This approach needs to be validated in prospective trials.

Using sequence clustering to identify clinically relevant subphenotypes in patients with COVID-19 admitted to the intensive care unit.

OBJECTIVE: The novel coronavirus disease 2019 (COVID-19) has heterogenous clinical courses, indicating that there might be distinct subphenotypes in critically ill patients. Although prior research has identified these subphenotypes, the temporal pattern of multiple clinical features has not been considered in cluster models. We aimed to identify temporal subphenotypes in critically ill patients with COVID-19 using a novel sequence cluster analysis and associate them with clinically relevant outcomes. MATERIALS AND METHODS: We analyzed 1036 confirmed critically ill patients with laboratory-confirmed SARS-COV-2 infection admitted to the Mount Sinai Health System in New York city. The agglomerative hierarchical clustering method was used with Levenshtein distance and Ward's minimum variance linkage. RESULTS: We identified four subphenotypes. Subphenotype I (N = 233 [22.5%]) included patients with rapid respirations and a rapid heartbeat but less need for invasive interventions within the first 24 hours, along with a relatively good prognosis. Subphenotype II (N = 418 [40.3%]) represented patients with the least degree of ailments, relatively low mortality, and the highest probability of discharge from the hospital. Subphenotype III (N = 259 [25.0%]) represented patients who experienced clinical deterioration during the first 24 hours of intensive care unit admission, leading to poor outcomes. Subphenotype IV (N = 126 [12.2%]) represented an acute respiratory distress syndrome trajectory with an almost universal need for mechanical ventilation. CONCLUSION: We utilized the sequence cluster analysis to identify clinical subphenotypes in critically ill COVID-19 patients who had distinct temporal patterns and different clinical outcomes. This study points toward the utility of including temporal information in subphenotyping approaches.

Multiethnic Investigation of Risk and Immune Determinants of COVID-19 Outcomes.

Background: Disparate COVID-19 outcomes have been observed between Hispanic, non-Hispanic Black, and White patients. The underlying causes for these disparities are not fully understood. Methods: This was a retrospective study utilizing electronic medical record data from five hospitals within a single academic health system based in New York City. Multivariable logistic regression models were used to identify demographic, clinical, and lab values associated with in-hospital mortality. Results: A total of 3,086 adult patients with self-reported race/ethnicity information presenting to the emergency department and hospitalized with COVID-19 up to April 13, 2020, were included in this study. While older age (multivariable odds ratio (OR) 1.06, 95% CI 1.05-1.07) and baseline hypoxia (multivariable OR 2.71, 95% CI 2.17-3.36) were associated with increased mortality overall and across all races/ethnicities, non-Hispanic Black (median age 67, interquartile range (IQR) 58-76) and Hispanic (median age 63, IQR 50-74) patients were younger and had different comorbidity profiles as compared to non-Hispanic White patients (median age 73, IQR 62-84; p < 0.05 for both comparisons). Among inflammatory markers associated with COVID-19 mortality, there was a significant interaction between the non-Hispanic Black population and interleukin-1-beta (interaction p-value 0.04). Conclusions: This analysis of a multiethnic cohort highlights the need for inclusion and consideration of diverse populations in ongoing COVID-19 trials targeting inflammatory cytokines.

Autoencoders for sample size estimation for fully connected neural network classifiers.

Sample size estimation is a crucial step in experimental design but is understudied in the context of deep learning. Currently, estimating the quantity of labeled data needed to train a classifier to a desired performance, is largely based on prior experience with similar models and problems or on untested heuristics. In many supervised machine learning applications, data labeling can be expensive and time-consuming and would benefit from a more rigorous means of estimating labeling requirements. Here, we study the problem of estimating the minimum sample size of labeled training data necessary for training computer vision models as an exemplar for other deep learning problems. We consider the problem of identifying the minimal number of labeled data points to achieve a generalizable representation of the data, a minimum converging sample (MCS). We use autoencoder loss to estimate the MCS for fully connected neural network classifiers. At sample sizes smaller than the MCS estimate, fully connected networks fail to distinguish classes, and at sample sizes above the MCS estimate, generalizability strongly correlates with the loss function of the autoencoder. We provide an easily accessible, code-free, and dataset-agnostic tool to estimate sample sizes for fully connected networks. Taken together, our findings suggest that MCS and convergence estimation are promising methods to guide sample size estimates for data collection and labeling prior to training deep learning models in computer vision.

Multi-ethnic Investigation of Risk and Immune Determinants of COVID-19 Outcomes.

Background: Disparate COVID-19 outcomes have been observed between Hispanic, Non-Hispanic Black, and White patients. The underlying causes for these disparities are not fully understood. Methods: This was a retrospective study utilizing electronic medical record data from five hospitals within a single academic health system based in New York City. Multivariable logistic regression models were used to identify demographic, clinical, and lab values associated with in-hospital mortality. Results: 3,086 adult patients with self-reported race/ethnicity information presenting to the emergency department and hospitalized with COVID-19 up to April 13, 2020 were included in this study. While older age (multivariable OR 1.06, 95% CI 1.05-1.07) and baseline hypoxia (multivariable OR 2.71, 95% CI 2.17-3.36) were associated with increased mortality overall and across all races/ethnicities, Non-Hispanic Black (median age 67, IQR 58-76) and Hispanic (median age 63, IQR 50-74) patients were younger and had different comorbidity profiles compared to Non-Hispanic White patients (median age 73, IQR 62-84; p<0.05 for both comparisons). Among inflammatory markers associated with COVID-19 mortality, there was a significant interaction between the Non-Hispanic Black population and interleukin-1-beta (interaction p-value 0.04). Conclusions: This analysis of a multi-ethnic cohort highlights the need for inclusion and consideration of diverse popualtions in ongoing COVID-19 trials targeting inflammatory cytokines.

Automated Determination of Left Ventricular Function Using Electrocardiogram Data in Patients on Maintenance Hemodialysis.

BACKGROUND AND OBJECTIVES: Left ventricular ejection fraction is disrupted in patients on maintenance hemodialysis and can be estimated using deep learning models on electrocardiograms. Smaller sample sizes within this population may be mitigated using transfer learning. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: We identified patients on hemodialysis with transthoracic echocardiograms within 7 days of electrocardiogram using diagnostic/procedure codes. We developed four models: (1) trained from scratch in patients on hemodialysis, (2) pretrained on a publicly available set of natural images (ImageNet), (3) pretrained on all patients not on hemodialysis, and (4) pretrained on patients not on hemodialysis and fine-tuned on patients on hemodialysis. We assessed the ability of the models to classify left ventricular ejection fraction into clinically relevant categories of ≤40%, 41% to ≤50%, and >50%. We compared performance by area under the receiver operating characteristic curve. RESULTS: We extracted 705,075 electrocardiogram:echocardiogram pairs for 158,840 patients not on hemodialysis used for development of models 3 and 4 and n=18,626 electrocardiogram:echocardiogram pairs for 2168 patients on hemodialysis for models 1, 2, and 4. The transfer learning model achieved area under the receiver operating characteristic curves of 0.86, 0.63, and 0.83 in predicting left ventricular ejection fraction categories of ≤40% (n=461), 41%-50% (n=398), and >50% (n=1309), respectively. For the same tasks, model 1 achieved area under the receiver operating characteristic curves of 0.74, 0.55, and 0.71, respectively; model 2 achieved area under the receiver operating characteristic curves of 0.71, 0.55, and 0.69, respectively, and model 3 achieved area under the receiver operating characteristic curves of 0.80, 0.51, and 0.77, respectively. We found that predictions of left ventricular ejection fraction by the transfer learning model were associated with mortality in a Cox regression with an adjusted hazard ratio of 1.29 (95% confidence interval, 1.04 to 1.59). CONCLUSION: A deep learning model can determine left ventricular ejection fraction for patients on hemodialysis following pretraining on electrocardiograms of patients not on hemodialysis. Predictions of low ejection fraction from this model were associated with mortality over a 5-year follow-up period. PODCAST: This article contains a podcast at https://www.asn-online.org/media/podcast/CJASN/2022_06_06_CJN16481221.mp3.

Quantitative chest computed tomography combined with plasma cytokines predict outcomes in COVID-19 patients.

Despite extraordinary international efforts to dampen the spread and understand the mechanisms behind SARS-CoV-2 infections, accessible predictive biomarkers directly applicable in the clinic are yet to be discovered. Recent studies have revealed that diverse types of assays bear limited predictive power for COVID-19 outcomes. Here, we harness the predictive power of chest computed tomography (CT) in combination with plasma cytokines using a machine learning and k-fold cross-validation approach for predicting death during hospitalization and maximum severity degree in COVID-19 patients. Patients (n = 152) from the Mount Sinai Health System in New York with plasma cytokine assessment and a chest CT within five days from admission were included. Demographics, clinical, and laboratory variables, including plasma cytokines (IL-6, IL-8, and TNF-α), were collected from the electronic medical record. We found that CT quantitative alone was better at predicting severity (AUC 0.81) than death (AUC 0.70), while cytokine measurements alone better-predicted death (AUC 0.70) compared to severity (AUC 0.66). When combined, chest CT and plasma cytokines were good predictors of death (AUC 0.78) and maximum severity (AUC 0.82). Finally, we provide a simple scoring system (nomogram) using plasma IL-6, IL-8, TNF-α, ground-glass opacities (GGO) to aerated lung ratio and age as new metrics that may be used to monitor patients upon hospitalization and help physicians make critical decisions and considerations for patients at high risk of death for COVID-19.

Prediction of individual COVID-19 diagnosis using baseline demographics and lab data.

The global surge in COVID-19 cases underscores the need for fast, scalable, and reliable testing. Current COVID-19 diagnostic tests are limited by turnaround time, limited availability, or occasional false findings. Here, we developed a machine learning-based framework for predicting individual COVID-19 positive diagnosis relying only on readily-available baseline data, including patient demographics, comorbidities, and common lab values. Leveraging a cohort of 31,739 adults within an academic health system, we trained and tested multiple types of machine learning models, achieving an area under the curve of 0.75. Feature importance analyses highlighted serum calcium levels, temperature, age, lymphocyte count, smoking, hemoglobin levels, aspartate aminotransferase levels, and oxygen saturation as key predictors. Additionally, we developed a single decision tree model that provided an operable method for stratifying sub-populations. Overall, this study provides a proof-of-concept that COVID-19 diagnosis prediction models can be developed using only baseline data. The resulting prediction can complement existing tests to enhance screening and pandemic containment workflows.