Assessing calibration and bias of a deployed machine learning malnutrition prediction model within a large healthcare system.

Malnutrition is a frequently underdiagnosed condition leading to increased morbidity, mortality, and healthcare costs. The Mount Sinai Health System (MSHS) deployed a machine learning model (MUST-Plus) to detect malnutrition upon hospital admission. However, in diverse patient groups, a poorly calibrated model may lead to misdiagnosis, exacerbating health care disparities. We explored the model's calibration across different variables and methods to improve calibration. Data from adult patients admitted to five MSHS hospitals from January 1, 2021 - December 31, 2022, were analyzed. We compared MUST-Plus prediction to the registered dietitian's formal assessment. Hierarchical calibration was assessed and compared between the recalibration sample (N = 49,562) of patients admitted between January 1, 2021 - December 31, 2022, and the hold-out sample (N = 17,278) of patients admitted between January 1, 2023 - September 30, 2023. Statistical differences in calibration metrics were tested using bootstrapping with replacement. Before recalibration, the overall model calibration intercept was -1.17 (95% CI: -1.20, -1.14), slope was 1.37 (95% CI: 1.34, 1.40), and Brier score was 0.26 (95% CI: 0.25, 0.26). Both weak and moderate measures of calibration were significantly different between White and Black patients and between male and female patients. Logistic recalibration significantly improved calibration of the model across race and gender in the hold-out sample. The original MUST-Plus model showed significant differences in calibration between White vs. Black patients. It also overestimated malnutrition in females compared to males. Logistic recalibration effectively reduced miscalibration across all patient subgroups. Continual monitoring and timely recalibration can improve model accuracy.

Characterizing cell type specific transcriptional differences between the living and postmortem human brain.

Single-nucleus RNA sequencing (snRNA-seq) is often used to define gene expression patterns characteristic of brain cell types as well as to identify cell type specific gene expression signatures of neurological and mental illnesses in postmortem human brains. As methods to obtain brain tissue from living individuals emerge, it is essential to characterize gene expression differences associated with tissue originating from either living or postmortem subjects using snRNA-seq, and to assess whether and how such differences may impact snRNA-seq studies of brain tissue. To address this, human prefrontal cortex single nuclei gene expression was generated and compared between 31 samples from living individuals and 21 postmortem samples. The same cell types were consistently identified in living and postmortem nuclei, though for each cell type, a large proportion of genes were differentially expressed between samples from postmortem and living individuals. Notably, estimation of cell type proportions by cell type deconvolution of pseudo-bulk data was found to be more accurate in samples from living individuals. To allow for future integration of living and postmortem brain gene expression, a model was developed that quantifies from gene expression data the probability a human brain tissue sample was obtained postmortem. These probabilities are established as a means to statistically account for the gene expression differences between samples from living and postmortem individuals. Together, the results presented here provide a deep characterization of both differences between snRNA-seq derived from samples from living and postmortem individuals, as well as qualify and account for their effect on common analyses performed on this type of data.

Accurate prediction of neurologic changes in critically ill infants using pose AI.

Importance: Infant alertness and neurologic changes are assessed by exam, which can be intermittent and subjective. Reliable, continuous methods are needed. Objective: We hypothesized that our computer vision method to track movement, pose AI, could predict neurologic changes. Design: Retrospective observational study from 2021-2022. Setting: A level four urban neonatal intensive care unit (NICU). Participants: Infants with corrected age ≤1 year, comprising 115 patients with 4,705 hours of video data linked to electroencephalograms (EEG), including 46% female and 25.2% white non-Hispanic. Exposures: Pose AI prediction of anatomic landmark position and an XGBoost classifier trained on one-minute variance in pose. Main outcomes and measures: Outcomes were cerebral dysfunction, diagnosed from EEG readings by an epileptologist, and sedation, defined by the administration of sedative medications. Measures of algorithm performance were receiver operating characteristic-area under the curves (ROC-AUCs) on cross-validation and on two test datasets comprised of held-out infants and held-out video frames from infants used in training. Results: Infant pose was accurately predicted in cross-validation, held-out frames, and held-out infants (respective ROC-AUCs 0.94, 0.83, 0.89). Median movement increased with age and, after accounting for age, was lower with sedative medications and in infants with cerebral dysfunction (all P<5×10-3, 10,000 permutations). Sedation prediction had high performance on cross-validation, held-out frames, and held-out infants (ROC-AUCs 0.90, 0.91, 0.87), as did prediction of cerebral dysfunction (ROC-AUCs 0.91, 0.90, 0.76). Conclusions and Relevance: We used pose AI to predict sedation and cerebral dysfunction in 4,705 hours of video from a large, diverse cohort of infants. Pose AI may offer a scalable, minimally invasive method for neuro-telemetry in the NICU.

Divergent landscapes of A-to-I editing in postmortem and living human brain.

Adenosine-to-inosine (A-to-I) editing is a prevalent post-transcriptional RNA modification within the brain. Yet, most research has relied on postmortem samples, assuming it is an accurate representation of RNA biology in the living brain. We challenge this assumption by comparing A-to-I editing between postmortem and living prefrontal cortical tissues. Major differences were found, with over 70,000 A-to-I sites showing higher editing levels in postmortem tissues. Increased A-to-I editing in postmortem tissues is linked to higher ADAR1 and ADARB1 expression, is more pronounced in non-neuronal cells, and indicative of postmortem activation of inflammation and hypoxia. Higher A-to-I editing in living tissues marks sites that are evolutionarily preserved, synaptic, developmentally timed, and disrupted in neurological conditions. Common genetic variants were also found to differentially affect A-to-I editing levels in living versus postmortem tissues. Collectively, these discoveries illuminate the nuanced functions and intricate regulatory mechanisms of RNA editing within the human brain.

Machine Learning Quantification of Pulmonary Regurgitation Fraction from Echocardiography.

Assessment of pulmonary regurgitation (PR) guides treatment for patients with congenital heart disease. Quantitative assessment of PR fraction (PRF) by echocardiography is limited. Cardiac MRI (cMRI) is the reference-standard for PRF quantification. We created an algorithm to predict cMRI-quantified PRF from echocardiography using machine learning (ML). We retrospectively performed echocardiographic measurements paired to cMRI within 3 months in patients with ≥ mild PR from 2009 to 2022. Model inputs were vena contracta ratio, PR index, PR pressure half-time, main and branch pulmonary artery diastolic flow reversal (BPAFR), and transannular patch repair. A gradient boosted trees ML algorithm was trained using k-fold cross-validation to predict cMRI PRF by phase contrast imaging as a continuous number and at > mild (PRF ≥ 20%) and severe (PRF ≥ 40%) thresholds. Regression performance was evaluated with mean absolute error (MAE), and at clinical thresholds with area-under-the-receiver-operating-characteristic curve (AUROC). Prediction accuracy was compared to historical clinician accuracy. We externally validated prior reported studies for comparison. We included 243 subjects (median age 21 years, 58% repaired tetralogy of Fallot). The regression MAE = 7.0%. For prediction of > mild PR, AUROC = 0.96, but BPAFR alone outperformed the ML model (sensitivity 94%, specificity 97%). The ML model detection of severe PR had AUROC = 0.86, but in the subgroup with BPAFR, performance dropped (AUROC = 0.73). Accuracy between clinicians and the ML model was similar (70% vs. 69%). There was decrement in performance of prior reported algorithms on external validation in our dataset. A novel ML model for echocardiographic quantification of PRF outperforms prior studies and has comparable overall accuracy to clinicians. BPAFR is an excellent marker for > mild PRF, and has moderate capacity to detect severe PR, but more work is required to distinguish moderate from severe PR. Poor external validation of prior works highlights reproducibility challenges.

Meta-Analysis of Genome-Wide Association Studies Reveals Genetic Mechanisms of Supraventricular Arrhythmias.

BACKGROUND: Substantial data support a heritable basis for supraventricular tachycardias, but the genetic determinants and molecular mechanisms of these arrhythmias are poorly understood. We sought to identify genetic loci associated with atrioventricular nodal reentrant tachycardia (AVNRT) and atrioventricular accessory pathways or atrioventricular reciprocating tachycardia (AVAPs/AVRT). METHODS: We performed multiancestry meta-analyses of genome-wide association studies to identify genetic loci for AVNRT (4 studies) and AVAP/AVRT (7 studies). We assessed evidence supporting the potential causal effects of candidate genes by analyzing relations between associated variants and cardiac gene expression, performing transcriptome-wide analyses, and examining prior genome-wide association studies. RESULTS: Analyses comprised 2384 AVNRT cases and 106 489 referents, and 2811 AVAP/AVRT cases and 1,483 093 referents. We identified 2 significant loci for AVNRT, which implicates NKX2-5 and TTN as disease susceptibility genes. A transcriptome-wide association analysis supported an association between reduced predicted cardiac expression of NKX2-5 and AVNRT. We identified 3 significant loci for AVAP/AVRT, which implicates SCN5A, SCN10A, and TTN/CCDC141. Variant associations at several loci have been previously reported for cardiac phenotypes, including atrial fibrillation, stroke, Brugada syndrome, and electrocardiographic intervals. CONCLUSIONS: Our findings highlight gene regions associated with ion channel function (AVAP/AVRT), as well as cardiac development and the sarcomere (AVAP/AVRT and AVNRT) as important potential effectors of supraventricular tachycardia susceptibility.

Machine learning derived serum creatinine trajectories in acute kidney injury in critically ill patients with sepsis.

BACKGROUND: Current classification for acute kidney injury (AKI) in critically ill patients with sepsis relies only on its severity-measured by maximum creatinine which overlooks inherent complexities and longitudinal evaluation of this heterogenous syndrome. The role of classification of AKI based on early creatinine trajectories is unclear. METHODS: This retrospective study identified patients with Sepsis-3 who developed AKI within 48-h of intensive care unit admission using Medical Information Mart for Intensive Care-IV database. We used latent class mixed modelling to identify early creatinine trajectory-based classes of AKI in critically ill patients with sepsis. Our primary outcome was development of acute kidney disease (AKD). Secondary outcomes were composite of AKD or all-cause in-hospital mortality by day 7, and AKD or all-cause in-hospital mortality by hospital discharge. We used multivariable regression to assess impact of creatinine trajectory-based classification on outcomes, and eICU database for external validation. RESULTS: Among 4197 patients with AKI in critically ill patients with sepsis, we identified eight creatinine trajectory-based classes with distinct characteristics. Compared to the class with transient AKI, the class that showed severe AKI with mild improvement but persistence had highest adjusted risks for developing AKD (OR 5.16; 95% CI 2.87-9.24) and composite 7-day outcome (HR 4.51; 95% CI 2.69-7.56). The class that demonstrated late mild AKI with persistence and worsening had highest risks for developing composite hospital discharge outcome (HR 2.04; 95% CI 1.41-2.94). These associations were similar on external validation. CONCLUSIONS: These 8 classes of AKI in critically ill patients with sepsis, stratified by early creatinine trajectories, were good predictors for key outcomes in patients with AKI in critically ill patients with sepsis independent of their AKI staging.

Evaluating the accuracy of a state-of-the-art large language model for prediction of admissions from the emergency room.

BACKGROUND: Artificial intelligence (AI) and large language models (LLMs) can play a critical role in emergency room operations by augmenting decision-making about patient admission. However, there are no studies for LLMs using real-world data and scenarios, in comparison to and being informed by traditional supervised machine learning (ML) models. We evaluated the performance of GPT-4 for predicting patient admissions from emergency department (ED) visits. We compared performance to traditional ML models both naively and when informed by few-shot examples and/or numerical probabilities. METHODS: We conducted a retrospective study using electronic health records across 7 NYC hospitals. We trained Bio-Clinical-BERT and XGBoost (XGB) models on unstructured and structured data, respectively, and created an ensemble model reflecting ML performance. We then assessed GPT-4 capabilities in many scenarios: through Zero-shot, Few-shot with and without retrieval-augmented generation (RAG), and with and without ML numerical probabilities. RESULTS: The Ensemble ML model achieved an area under the receiver operating characteristic curve (AUC) of 0.88, an area under the precision-recall curve (AUPRC) of 0.72 and an accuracy of 82.9%. The naïve GPT-4's performance (0.79 AUC, 0.48 AUPRC, and 77.5% accuracy) showed substantial improvement when given limited, relevant data to learn from (ie, RAG) and underlying ML probabilities (0.87 AUC, 0.71 AUPRC, and 83.1% accuracy). Interestingly, RAG alone boosted performance to near peak levels (0.82 AUC, 0.56 AUPRC, and 81.3% accuracy). CONCLUSIONS: The naïve LLM had limited performance but showed significant improvement in predicting ED admissions when supplemented with real-world examples to learn from, particularly through RAG, and/or numerical probabilities from traditional ML models. Its peak performance, although slightly lower than the pure ML model, is noteworthy given its potential for providing reasoning behind predictions. Further refinement of LLMs with real-world data is necessary for successful integration as decision-support tools in care settings.

Multiomic foundations of human prefrontal cortex tissue function.

The prefrontal cortex (PFC) is a region of the brain that in humans is involved in the production of higher-order functions such as cognition, emotion, perception, and behavior. Neurotransmission in the PFC produces higher-order functions by integrating information from other areas of the brain. At the foundation of neurotransmission, and by extension at the foundation of higher-order brain functions, are an untold number of coordinated molecular processes involving the DNA sequence variants in the genome, RNA transcripts in the transcriptome, and proteins in the proteome. These "multiomic" foundations are poorly understood in humans, perhaps in part because most modern studies that characterize the molecular state of the human PFC use tissue obtained when neurotransmission and higher-order brain functions have ceased (i.e., the postmortem state). Here, analyses are presented on data generated for the Living Brain Project (LBP) to investigate whether PFC tissue from individuals with intact higher-order brain function has characteristic multiomic foundations. Two complementary strategies were employed towards this end. The first strategy was to identify in PFC samples obtained from living study participants a signature of RNA transcript expression associated with neurotransmission measured intracranially at the time of PFC sampling, in some cases while participants performed a task engaging higher-order brain functions. The second strategy was to perform multiomic comparisons between PFC samples obtained from individuals with intact higher-order brain function at the time of sampling (i.e., living study participants) and PFC samples obtained in the postmortem state. RNA transcript expression within multiple PFC cell types was associated with fluctuations of dopaminergic, serotonergic, and/or noradrenergic neurotransmission in the substantia nigra measured while participants played a computer game that engaged higher-order brain functions. A subset of these associations - termed the "transcriptional program associated with neurotransmission" (TPAWN) - were reproduced in analyses of brain RNA transcript expression and intracranial neurotransmission data obtained from a second LBP cohort and from a cohort in an independent study. RNA transcripts involved in TPAWN were found to be (1) enriched for RNA transcripts associated with measures of neurotransmission in rodent and cell models, (2) enriched for RNA transcripts encoded by evolutionarily constrained genes, (3) depleted of RNA transcripts regulated by common DNA sequence variants, and (4) enriched for RNA transcripts implicated in higher-order brain functions by human population genetic studies. In PFC excitatory neurons of living study participants, higher expression of the genes in TPAWN tracked with higher expression of RNA transcripts that in rodent PFC samples are markers of a class of excitatory neurons that connect the PFC to deep brain structures. TPAWN was further reproduced by RNA transcript expression patterns differentiating living PFC samples from postmortem PFC samples, and significant differences between living and postmortem PFC samples were additionally observed with respect to (1) the expression of most primary RNA transcripts, mature RNA transcripts, and proteins, (2) the splicing of most primary RNA transcripts into mature RNA transcripts, (3) the patterns of co-expression between RNA transcripts and proteins, and (4) the effects of some DNA sequence variants on RNA transcript and protein expression. Taken together, this report highlights that studies of brain tissue obtained in a safe and ethical manner from large cohorts of living individuals can help advance understanding of the multiomic foundations of brain function.

Integrated multiomics implicates dysregulation of ECM and cell adhesion pathways as drivers of severe COVID-associated kidney injury.

COVID-19 has been a significant public health concern for the last four years; however, little is known about the mechanisms that lead to severe COVID-associated kidney injury. In this multicenter study, we combined quantitative deep urinary proteomics and machine learning to predict severe acute outcomes in hospitalized COVID-19 patients. Using a 10-fold cross-validated random forest algorithm, we identified a set of urinary proteins that demonstrated predictive power for both discovery and validation set with 87% and 79% accuracy, respectively. These predictive urinary biomarkers were recapitulated in non-COVID acute kidney injury revealing overlapping injury mechanisms. We further combined orthogonal multiomics datasets to understand the mechanisms that drive severe COVID-associated kidney injury. Functional overlap and network analysis of urinary proteomics, plasma proteomics and urine sediment single-cell RNA sequencing showed that extracellular matrix and autophagy-associated pathways were uniquely impacted in severe COVID-19. Differentially abundant proteins associated with these pathways exhibited high expression in cells in the juxtamedullary nephron, endothelial cells, and podocytes, indicating that these kidney cell types could be potential targets. Further, single-cell transcriptomic analysis of kidney organoids infected with SARS-CoV-2 revealed dysregulation of extracellular matrix organization in multiple nephron segments, recapitulating the clinically observed fibrotic response across multiomics datasets. Ligand-receptor interaction analysis of the podocyte and tubule organoid clusters showed significant reduction and loss of interaction between integrins and basement membrane receptors in the infected kidney organoids. Collectively, these data suggest that extracellular matrix degradation and adhesion-associated mechanisms could be a main driver of COVID-associated kidney injury and severe outcomes.

Neuropsychiatric complications of coronavirus disease 2019: Mount Sinai Health System cohort study.

OBJECTIVE: To describe the frequency of neuropsychiatric complications among hospitalized patients with coronavirus disease 2019 (COVID-19) and their association with pre-existing comorbidities and clinical outcomes. METHODS: We retrospectively identified all patients hospitalized with COVID-19 within a large multicenter New York City health system between March 15, 2020 and May 17, 2021 and randomly selected a representative cohort for detailed chart review. Clinical data, including the occurrence of neuropsychiatric complications (categorized as either altered mental status [AMS] or other neuropsychiatric complications) and in-hospital mortality, were extracted using an electronic medical record database and individual chart review. Associations between neuropsychiatric complications, comorbidities, laboratory findings, and in-hospital mortality were assessed using multivariate logistic regression. RESULTS: Our study cohort consisted of 974 patients, the majority were admitted during the first wave of the pandemic. Patients were treated with anticoagulation (88.4%), glucocorticoids (24.8%), and remdesivir (10.5%); 18.6% experienced severe COVID-19 pneumonia (evidenced by ventilator requirement). Neuropsychiatric complications occurred in 58.8% of patients; 39.8% experienced AMS; and 19.0% experienced at least one other complication (seizures in 1.4%, ischemic stroke in 1.6%, hemorrhagic stroke in 1.0%) or symptom (headache in 11.4%, anxiety in 6.8%, ataxia in 6.3%). Higher odds of mortality, which occurred in 22.0%, were associated with AMS, ventilator support, increasing age, and higher serum inflammatory marker levels. Anticoagulant therapy was associated with lower odds of mortality and AMS. CONCLUSION: Neuropsychiatric complications of COVID-19, especially AMS, were common, varied, and associated with in-hospital mortality in a diverse multicenter cohort at an epicenter of the COVID-19 pandemic.

A Multimodal Video-Based AI Biomarker for Aortic Stenosis Development and Progression.

Importance: Aortic stenosis (AS) is a major public health challenge with a growing therapeutic landscape, but current biomarkers do not inform personalized screening and follow-up. A video-based artificial intelligence (AI) biomarker (Digital AS Severity index [DASSi]) can detect severe AS using single-view long-axis echocardiography without Doppler characterization. Objective: To deploy DASSi to patients with no AS or with mild or moderate AS at baseline to identify AS development and progression. Design, Setting, and Participants: This is a cohort study that examined 2 cohorts of patients without severe AS undergoing echocardiography in the Yale New Haven Health System (YNHHS; 2015-2021) and Cedars-Sinai Medical Center (CSMC; 2018-2019). A novel computational pipeline for the cross-modal translation of DASSi into cardiac magnetic resonance (CMR) imaging was further developed in the UK Biobank. Analyses were performed between August 2023 and February 2024. Exposure: DASSi (range, 0-1) derived from AI applied to echocardiography and CMR videos. Main Outcomes and Measures: Annualized change in peak aortic valve velocity (AV-Vmax) and late (>6 months) aortic valve replacement (AVR). Results: A total of 12 599 participants were included in the echocardiographic study (YNHHS: n = 8798; median [IQR] age, 71 [60-80] years; 4250 [48.3%] women; median [IQR] follow-up, 4.1 [2.4-5.4] years; and CSMC: n = 3801; median [IQR] age, 67 [54-78] years; 1685 [44.3%] women; median [IQR] follow-up, 3.4 [2.8-3.9] years). Higher baseline DASSi was associated with faster progression in AV-Vmax (per 0.1 DASSi increment: YNHHS, 0.033 m/s per year [95% CI, 0.028-0.038] among 5483 participants; CSMC, 0.082 m/s per year [95% CI, 0.053-0.111] among 1292 participants), with values of 0.2 or greater associated with a 4- to 5-fold higher AVR risk than values less than 0.2 (YNHHS: 715 events; adjusted hazard ratio [HR], 4.97 [95% CI, 2.71-5.82]; CSMC: 56 events; adjusted HR, 4.04 [95% CI, 0.92-17.70]), independent of age, sex, race, ethnicity, ejection fraction, and AV-Vmax. This was reproduced across 45 474 participants (median [IQR] age, 65 [59-71] years; 23 559 [51.8%] women; median [IQR] follow-up, 2.5 [1.6-3.9] years) undergoing CMR imaging in the UK Biobank (for participants with DASSi ≥0.2 vs those with DASSi <.02, adjusted HR, 11.38 [95% CI, 2.56-50.57]). Saliency maps and phenome-wide association studies supported associations with cardiac structure and function and traditional cardiovascular risk factors. Conclusions and Relevance: In this cohort study of patients without severe AS undergoing echocardiography or CMR imaging, a new AI-based video biomarker was independently associated with AS development and progression, enabling opportunistic risk stratification across cardiovascular imaging modalities as well as potential application on handheld devices.

A Novel Digital Twin Strategy to Examine the Implications of Randomized Control Trials for Real-World Populations.

Randomized clinical trials (RCTs) are essential to guide medical practice; however, their generalizability to a given population is often uncertain. We developed a statistically informed Generative Adversarial Network (GAN) model, RCT-Twin-GAN, that leverages relationships between covariates and outcomes and generates a digital twin of an RCT (RCT-Twin) conditioned on covariate distributions from a second patient population. We used RCT-Twin-GAN to reproduce treatment effect outcomes of the Systolic Blood Pressure Intervention Trial (SPRINT) and the Action to Control Cardiovascular Risk in Diabetes (ACCORD) Blood Pressure Trial, which tested the same intervention but had different treatment effect results. To demonstrate treatment effect estimates of each RCT conditioned on the other RCT patient population, we evaluated the cardiovascular event-free survival of SPRINT digital twins conditioned on the ACCORD cohort and vice versa (SPRINT-conditioned ACCORD twins). The conditioned digital twins were balanced by the intervention arm (mean absolute standardized mean difference (MASMD) of covariates between treatment arms 0.019 (SD 0.018), and the conditioned covariates of the SPRINT-Twin on ACCORD were more similar to ACCORD than a sprint (MASMD 0.0082 SD 0.016 vs. 0.46 SD 0.20). Most importantly, across iterations, SPRINT conditioned ACCORD-Twin datasets reproduced the overall non-significant effect size seen in ACCORD (5-year cardiovascular outcome hazard ratio (95% confidence interval) of 0.88 (0.73-1.06) in ACCORD vs median 0.87 (0.68-1.13) in the SPRINT conditioned ACCORD-Twin), while the ACCORD conditioned SPRINT-Twins reproduced the significant effect size seen in SPRINT (0.75 (0.64-0.89) vs median 0.79 (0.72-0.86)) in ACCORD conditioned SPRINT-Twin). Finally, we describe the translation of this approach to real-world populations by conditioning the trials on an electronic health record population. Therefore, RCT-Twin-GAN simulates the direct translation of RCT-derived treatment effects across various patient populations with varying covariate distributions.

Local large language models for privacy-preserving accelerated review of historic echocardiogram reports.

OBJECTIVES: The study developed framework that leverages an open-source Large Language Model (LLM) to enable clinicians to ask plain-language questions about a patient's entire echocardiogram report history. This approach is intended to streamline the extraction of clinical insights from multiple echocardiogram reports, particularly in patients with complex cardiac diseases, thereby enhancing both patient care and research efficiency. MATERIALS AND METHODS: Data from over 10 years were collected, comprising echocardiogram reports from patients with more than 10 echocardiograms on file at the Mount Sinai Health System. These reports were converted into a single document per patient for analysis, broken down into snippets and relevant snippets were retrieved using text similarity measures. The LLaMA-2 70B model was employed for analyzing the text using a specially crafted prompt. The model's performance was evaluated against ground-truth answers created by faculty cardiologists. RESULTS: The study analyzed 432 reports from 37 patients for a total of 100 question-answer pairs. The LLM correctly answered 90% questions, with accuracies of 83% for temporality, 93% for severity assessment, 84% for intervention identification, and 100% for diagnosis retrieval. Errors mainly stemmed from the LLM's inherent limitations, such as misinterpreting numbers or hallucinations. CONCLUSION: The study demonstrates the feasibility and effectiveness of using a local, open-source LLM for querying and interpreting echocardiogram report data. This approach offers a significant improvement over traditional keyword-based searches, enabling more contextually relevant and semantically accurate responses; in turn showing promise in enhancing clinical decision-making and research by facilitating more efficient access to complex patient data.

Remote Short Sessions of Heart Rate Variability Biofeedback Monitored With Wearable Technology: Open-Label Prospective Feasibility Study.

BACKGROUND: Heart rate variability (HRV) biofeedback is often performed with structured education, laboratory-based assessments, and practice sessions. It has been shown to improve psychological and physiological function across populations. However, a means to remotely use and monitor this approach would allow for wider use of this technique. Advancements in wearable and digital technology present an opportunity for the widespread application of this approach. OBJECTIVE: The primary aim of the study was to determine the feasibility of fully remote, self-administered short sessions of HRV-directed biofeedback in a diverse population of health care workers (HCWs). The secondary aim was to determine whether a fully remote, HRV-directed biofeedback intervention significantly alters longitudinal HRV over the intervention period, as monitored by wearable devices. The tertiary aim was to estimate the impact of this intervention on metrics of psychological well-being. METHODS: To determine whether remotely implemented short sessions of HRV biofeedback can improve autonomic metrics and psychological well-being, we enrolled HCWs across 7 hospitals in New York City in the United States. They downloaded our study app, watched brief educational videos about HRV biofeedback, and used a well-studied HRV biofeedback program remotely through their smartphone. HRV biofeedback sessions were used for 5 minutes per day for 5 weeks. HCWs were then followed for 12 weeks after the intervention period. Psychological measures were obtained over the study period, and they wore an Apple Watch for at least 7 weeks to monitor the circadian features of HRV. RESULTS: In total, 127 HCWs were enrolled in the study. Overall, only 21 (16.5%) were at least 50% compliant with the HRV biofeedback intervention, representing a small portion of the total sample. This demonstrates that this study design does not feasibly result in adequate rates of compliance with the intervention. Numerical improvement in psychological metrics was observed over the 17-week study period, although it did not reach statistical significance (all P>.05). Using a mixed effect cosinor model, the mean midline-estimating statistic of rhythm (MESOR) of the circadian pattern of the SD of the interbeat interval of normal sinus beats (SDNN), an HRV metric, was observed to increase over the first 4 weeks of the biofeedback intervention in HCWs who were at least 50% compliant. CONCLUSIONS: In conclusion, we found that using brief remote HRV biofeedback sessions and monitoring its physiological effect using wearable devices, in the manner that the study was conducted, was not feasible. This is considering the low compliance rates with the study intervention. We found that remote short sessions of HRV biofeedback demonstrate potential promise in improving autonomic nervous function and warrant further study. Wearable devices can monitor the physiological effects of psychological interventions.

Genome-first evaluation with exome sequence and clinical data uncovers underdiagnosed genetic disorders in a large healthcare system.

Population-based genomic screening may help diagnose individuals with disease-risk variants. Here, we perform a genome-first evaluation for nine disorders in 29,039 participants with linked exome sequences and electronic health records (EHRs). We identify 614 individuals with 303 pathogenic/likely pathogenic or predicted loss-of-function (P/LP/LoF) variants, yielding 644 observations; 487 observations (76%) lack a corresponding clinical diagnosis in the EHR. Upon further investigation, 75 clinically undiagnosed observations (15%) have evidence of symptomatic untreated disease, including familial hypercholesterolemia (3 of 6 [50%] undiagnosed observations with disease evidence) and breast cancer (23 of 106 [22%]). These genetic findings enable targeted phenotyping that reveals new diagnoses in previously undiagnosed individuals. Disease yield is greater with variants in penetrant genes for which disease is observed in carriers in an independent cohort. The prevalence of P/LP/LoF variants exceeds that of clinical diagnoses, and some clinically undiagnosed carriers are discovered to have disease. These results highlight the potential of population-based genomic screening.

Artificial Intelligence in Cardiovascular Care - Part 2: Applications: JACC Review Topic of the Week.

Recent Artificial Intelligence (AI) advancements in cardiovascular care offer potential enhancements in effective diagnosis, treatment, and outcomes. Over 600 Food and Drug Administration (FDA)-approved clinical AI algorithms now exist, with 10% focusing on cardiovascular applications, highlighting the growing opportunities for AI to augment care. This review discusses the latest advancements in the field of AI, with a particular focus on the utilization of multimodal inputs and the field of generative AI. Further discussions in this review involve an approach to understanding the larger context in which AI-augmented care may exist, and include a discussion of the need for rigorous evaluation, appropriate infrastructure for deployment, ethics and equity assessments, regulatory oversight, and viable business cases for deployment. Embracing this rapidly evolving technology while setting an appropriately high evaluation benchmark with careful and patient-centered implementation will be crucial for cardiology to leverage AI to enhance patient care and the provider experience.

Artificial Intelligence for Cardiovascular Care - Part 1: Advances: JACC Review Topic of the Week.

Recent AI advancements in cardiovascular care offer potential enhancements in diagnosis, treatment, and outcomes. Innovations to date focus on automating measurements, enhancing image quality, and detecting diseases using novel methods. Applications span wearables, electrocardiograms, echocardiography, angiography, genetics, and more. AI models detect diseases from electrocardiograms at accuracy not previously achieved by technology or human experts, including reduced ejection fraction, valvular heart disease, and other cardiomyopathies. However, AI's unique characteristics necessitates rigorous validation by addressing training methods, real-world efficacy, equity concerns, and long-term reliability. Despite an exponentially growing number of studies in cardiovascular AI, trials showing improvement in outcomes remain lacking. A number are currently underway. Embracing this rapidly evolving technology while setting a high evaluation benchmark will be crucial for cardiology to leverage AI to enhance patient care and the provider experience.

Artificial intelligence-guided detection of under-recognized cardiomyopathies on point-of-care cardiac ultrasound.

Background: Point-of-care ultrasonography (POCUS) enables access to cardiac imaging directly at the bedside but is limited by brief acquisition, variation in acquisition quality, and lack of advanced protocols. Objective: To develop and validate deep learning models for detecting underdiagnosed cardiomyopathies on cardiac POCUS, leveraging a novel acquisition quality-adapted modeling strategy. Methods: To develop the models, we identified transthoracic echocardiograms (TTEs) of patients across five hospitals in a large U.S. health system with transthyretin amyloid cardiomyopathy (ATTR-CM, confirmed by Tc99m-pyrophosphate imaging), hypertrophic cardiomyopathy (HCM, confirmed by cardiac magnetic resonance), and controls enriched for the presence of severe AS. In a sample of 290,245 TTE videos, we used novel augmentation approaches and a customized loss function to weigh image and view quality to train a multi-label, view agnostic video-based convolutional neural network (CNN) to discriminate the presence of ATTR-CM, HCM, and/or AS. Models were tested across 3,758 real-world POCUS videos from 1,879 studies in 1,330 independent emergency department (ED) patients from 2011 through 2023. Results: Our multi-label, view-agnostic classifier demonstrated state-of-the-art performance in discriminating ATTR-CM (AUROC 0.98 [95%CI: 0.96-0.99]) and HCM (AUROC 0.95 [95% CI: 0.94-0.96]) on standard TTE studies. Automated metrics of anatomical view correctness confirmed significantly lower quality in POCUS vs TTE videos (median view classifier confidence of 0.63 [IQR: 0.44-0.88] vs 0.93 [IQR: 0.69-1.00], p<0.001). When deployed to POCUS videos, our algorithm effectively discriminated ATTR-CM and HCM with AUROC of up to 0.94 (parasternal long-axis (PLAX)), and 0.85 (apical 4 chamber), corresponding to positive diagnostic odds ratios of 46.7 and 25.5, respectively. In total, 18/35 (51.4%) of ATTR-CM and 32/57 (41.1%) of HCM patients in the POCUS cohort had an AI-positive screen in the year before their eventual confirmatory imaging. Conclusions: We define and validate an AI framework that enables scalable, opportunistic screening of under-diagnosed cardiomyopathies using POCUS.

Large language models for generating medical examinations: systematic review.

BACKGROUND: Writing multiple choice questions (MCQs) for the purpose of medical exams is challenging. It requires extensive medical knowledge, time and effort from medical educators. This systematic review focuses on the application of large language models (LLMs) in generating medical MCQs. METHODS: The authors searched for studies published up to November 2023. Search terms focused on LLMs generated MCQs for medical examinations. Non-English, out of year range and studies not focusing on AI generated multiple-choice questions were excluded. MEDLINE was used as a search database. Risk of bias was evaluated using a tailored QUADAS-2 tool. RESULTS: Overall, eight studies published between April 2023 and October 2023 were included. Six studies used Chat-GPT 3.5, while two employed GPT 4. Five studies showed that LLMs can produce competent questions valid for medical exams. Three studies used LLMs to write medical questions but did not evaluate the validity of the questions. One study conducted a comparative analysis of different models. One other study compared LLM-generated questions with those written by humans. All studies presented faulty questions that were deemed inappropriate for medical exams. Some questions required additional modifications in order to qualify. CONCLUSIONS: LLMs can be used to write MCQs for medical examinations. However, their limitations cannot be ignored. Further study in this field is essential and more conclusive evidence is needed. Until then, LLMs may serve as a supplementary tool for writing medical examinations. 2 studies were at high risk of bias. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.