Vision-language foundation model for echocardiogram interpretation.

The development of robust artificial intelligence models for echocardiography has been limited by the availability of annotated clinical data. Here, to address this challenge and improve the performance of cardiac imaging models, we developed EchoCLIP, a vision-language foundation model for echocardiography, that learns the relationship between cardiac ultrasound images and the interpretations of expert cardiologists across a wide range of patients and indications for imaging. After training on 1,032,975 cardiac ultrasound videos and corresponding expert text, EchoCLIP performs well on a diverse range of benchmarks for cardiac image interpretation, despite not having been explicitly trained for individual interpretation tasks. EchoCLIP can assess cardiac function (mean absolute error of 7.1% when predicting left ventricular ejection fraction in an external validation dataset) and identify implanted intracardiac devices (area under the curve (AUC) of 0.84, 0.92 and 0.97 for pacemakers, percutaneous mitral valve repair and artificial aortic valves, respectively). We also developed a long-context variant (EchoCLIP-R) using a custom tokenizer based on common echocardiography concepts. EchoCLIP-R accurately identified unique patients across multiple videos (AUC of 0.86), identified clinical transitions such as heart transplants (AUC of 0.79) and cardiac surgery (AUC 0.77) and enabled robust image-to-text search (mean cross-modal retrieval rank in the top 1% of candidate text reports). These capabilities represent a substantial step toward understanding and applying foundation models in cardiovascular imaging for preliminary interpretation of echocardiographic findings.

Impact of Measurement Noise on Genetic Association Studies of Cardiac Function.

Recent research has effectively used quantitative traits from imaging to boost the capabilities of genome-wide association studies (GWAS), providing further understanding of disease biology and various traits. However, it's important to note that phenotyping inherently carries measurement error and noise that could influence subsequent genetic analyses. The study focused on left ventricular ejection fraction (LVEF), a vital yet potentially inaccurate quantitative measurement, to investigate how imprecision in phenotype measurement affects genetic studies. Several methods of acquiring LVEF, along with simulating measurement noise, were assessed for their effects on ensuing genetic analyses. The results showed that by introducing just 7.9% of measurement noise, all genetic associations in an LVEF GWAS with almost forty thousand individuals could be eliminated. Moreover, a 1% increase in mean absolute error (MAE) in LVEF had an effect equivalent to a 10% reduction in the sample size of the cohort on the power of GWAS. Therefore, enhancing the accuracy of phenotyping is crucial to maximize the effectiveness of genome-wide association studies.

Opportunistic Screening of Chronic Liver Disease with Deep Learning Enhanced Echocardiography.

Importance: Chronic liver disease affects more than 1.5 billion adults worldwide, however the majority of cases are asymptomatic and undiagnosed. Echocardiography is broadly performed and visualizes the liver; but this information is not leveraged. Objective: To develop and evaluate a deep learning algorithm on echocardiography videos to enable opportunistic screening for chronic liver disease. Design: Retrospective observational cohorts. Setting: Two large urban academic medical centers. Participants: Adult patients who received echocardiography and abdominal imaging (either abdominal ultrasound or abdominal magnetic resonance imaging) with ≤30 days between tests, between July 4, 2012, to June 4, 2022. Exposure: Deep learning model predictions from a deep-learning computer vision pipeline that identifies subcostal view echocardiogram videos and detects the presence of cirrhosis or steatotic liver disease (SLD). Main Outcome and Measures: Clinical diagnosis by paired abdominal ultrasound or magnetic resonance imaging (MRI). Results: A total of 1,596,640 echocardiogram videos (66,922 studies from 24,276 patients) from Cedars-Sinai Medical Center (CSMC) were used to develop EchoNet-Liver, an automated pipeline that identifies high quality subcostal images from echocardiogram studies and detects the presence of cirrhosis or SLD. In the held-out CSMC test cohort, EchoNet-Liver was able to detect the presence of cirrhosis with an AUC of 0.837 (0.789 - 0.880) and SLD with an AUC of 0.799 (0.758 - 0.837). In a separate test cohort with paired abdominal MRIs, cirrhosis was detected with an AUC of 0.704 (0.689-0.718) and SLD was detected with an AUC of 0.726 (0.659-0.790). In an external test cohort of 106 patients (n = 5,280 videos), the model detected cirrhosis with an AUC of 0.830 (0.738 - 0.909) and SLD with an AUC of 0.768 (0.652 - 0.875). Conclusions and Relevance: Deep learning assessment of clinical echocardiography enables opportunistic screening of SLD and cirrhosis. Application of this algorithm may identify patients who may benefit from further diagnostic testing and treatment for chronic liver disease. KEY POINTS: Question: Can a deep learning algorithm applied to echocardiography videos effectively identify chronic liver diseases including cirrhosis and steatotic liver disease (SLD)?Findings: This retrospective observational cohort study utilized 1,596,640 echocardiography videos from 66,922 studies of 24,276 patients. The deep learning model with a computer vision pipeline (EchoNet-Liver) demonstrated strong performance to detect cirrhosis and SLD. External validation at a geographically distinct site demonstrated similar discriminative ability.Meaning: The application of EchoNet-Liver to echocardiography could aid opportunistic screening of chronic liver diseases, providing a unique cost-effective angle to improve patient management.

Using Deep learning to Predict Cardiovascular Magnetic Resonance Findings from Echocardiography Videos.

Background: Echocardiography is the most common modality for assessing cardiac structure and function. While cardiac magnetic resonance (CMR) imaging is less accessible, CMR can provide unique tissue characterization including late gadolinium enhancement (LGE), T1 and T2 mapping, and extracellular volume (ECV) which are associated with tissue fibrosis, infiltration, and inflammation. While deep learning has been shown to uncover findings not recognized by clinicians, it is unknown whether CMR-based tissue characteristics can be derived from echocardiography videos using deep learning. We hypothesized that deep learning applied to echocardiography could predict CMR-based measurements. Methods: In a retrospective single-center study, adult patients with CMRs and echocardiography studies within 30 days were included. A video-based convolutional neural network was trained on echocardiography videos to predict CMR-derived labels including wall motion abnormality (WMA) presence, LGE presence, and abnormal T1, T2 or ECV across echocardiography views. The model performance was evaluated in a held-out test dataset not used for training. Results: The study population included 1,453 adult patients (mean age 56±18 years, 42% female) with 2,556 paired echocardiography studies occurring on average 2 days after CMR (interquartile range 2 days prior to 6 days after). The model had high predictive capability for presence of WMA (AUC 0.873 [95%CI 0.816-0.922]), however, the model was unable to reliably detect the presence of LGE (AUC 0.699 [0.613-0.780]), native T1 (AUC 0.614 [0.500-0.715]), T2 0.553 [0.420-0.692], or ECV 0.564 [0.455-0.691]). Conclusions: Deep learning applied to echocardiography accurately identified CMR-based WMA, but was unable to predict tissue characteristics, suggesting that signal for these tissue characteristics may not be present within ultrasound videos, and that the use of CMR for tissue characterization remains essential within cardiology. Clinical Perspective: Tissue characterization of the heart muscle is useful for clinical diagnosis and prognosis by identifying myocardial fibrosis, inflammation, and infiltration, and can be measured using cardiac MRI. While echocardiography is highly accessible and provides excellent functional information, its ability to provide tissue characterization information is limited at this time. Our study using a deep learning approach to predict cardiac MRI-based tissue characteristics from echocardiography showed limited ability to do so, suggesting that alternative approaches, including non-deep learning methods should be considered in future research.

GANcMRI: Cardiac magnetic resonance video generation and physiologic guidance using latent space prompting.

Generative artificial intelligence can be applied to medical imaging on tasks such as privacy-preserving image generation and superresolution and denoising of existing images. Few prior approaches have used cardiac magnetic resonance imaging (cMRI) as a modality given the complexity of videos (the addition of the temporal dimension) as well as the limited scale of publicly available datasets. We introduce GANcMRI, a generative adversarial network that can synthesize cMRI videos with physiological guidance based on latent space prompting. GANcMRI uses a StyleGAN framework to learn the latent space from individual video frames and leverages the timedependent trajectory between end-systolic and end-diastolic frames in the latent space to predict progression and generate motion over time. We proposed various methods for modeling latent time-dependent trajectories and found that our Frame-to-frame approach generates the best motion and video quality. GANcMRI generated high-quality cMRI image frames that are indistinguishable by cardiologists, however, artifacts in video generation allow cardiologists to still recognize the difference between real and generated videos. The generated cMRI videos can be prompted to apply physiologybased adjustments which produces clinically relevant phenotypes recognizable by cardiologists. GANcMRI has many potential applications such as data augmentation, education, anomaly detection, and preoperative planning.

Impact of Measurement Imprecision on Genetic Association Studies of Cardiac Function.

Background: Recent studies have leveraged quantitative traits from imaging to amplify the power of genome-wide association studies (GWAS) to gain further insights into the biology of diseases and traits. However, measurement imprecision is intrinsic to phenotyping and can impact downstream genetic analyses. Methods: Left ventricular ejection fraction (LVEF), an important but imprecise quantitative imaging measurement, was examined to assess the impact of precision of phenotype measurement on genetic studies. Multiple approaches to obtain LVEF, as well as simulated measurement noise, were evaluated with their impact on downstream genetic analyses. Results: Even within the same population, small changes in the measurement of LVEF drastically impacted downstream genetic analyses. Introducing measurement noise as little as 7.9% can eliminate all significant genetic associations in an GWAS with almost forty thousand individuals. An increase of 1% in mean absolute error (MAE) in LVEF had an equivalent impact on GWAS power as a decrease of 10% in the cohort sample size, suggesting optimizing phenotyping precision is a cost-effective way to improve power of genetic studies. Conclusions: Improving the precision of phenotyping is important for maximizing the yield of genome-wide association studies.

Deep learning-enabled analysis of medical images identifies cardiac sphericity as an early marker of cardiomyopathy and related outcomes.

BACKGROUND: Quantification of chamber size and systolic function is a fundamental component of cardiac imaging. However, the human heart is a complex structure with significant uncharacterized phenotypic variation beyond traditional metrics of size and function. Examining variation in cardiac shape can add to our ability to understand cardiovascular risk and pathophysiology. METHODS: We measured the left ventricle (LV) sphericity index (short axis length/long axis length) using deep learning-enabled image segmentation of cardiac magnetic resonance imaging data from the UK Biobank. Subjects with abnormal LV size or systolic function were excluded. The relationship between LV sphericity and cardiomyopathy was assessed using Cox analyses, genome-wide association studies, and two-sample Mendelian randomization. FINDINGS: In a cohort of 38,897 subjects, we show that a one standard deviation increase in sphericity index is associated with a 47% increased incidence of cardiomyopathy (hazard ratio [HR]: 1.47, 95% confidence interval [CI]: 1.10-1.98, p = 0.01) and a 20% increased incidence of atrial fibrillation (HR: 1.20, 95% CI: 1.11-1.28, p < 0.001), independent of clinical factors and traditional magnetic resonance imaging (MRI) measurements. We identify four loci associated with sphericity at genome-wide significance, and Mendelian randomization supports non-ischemic cardiomyopathy as causal for LV sphericity. CONCLUSIONS: Variation in LV sphericity in otherwise normal hearts predicts risk for cardiomyopathy and related outcomes and is caused by non-ischemic cardiomyopathy. FUNDING: This study was supported by grants K99-HL157421 (D.O.) and KL2TR003143 (S.L.C.) from the National Institutes of Health.