A review of self-supervised, generative, and few-shot deep learning methods for data-limited magnetic resonance imaging segmentation.

Magnetic resonance imaging (MRI) is a ubiquitous medical imaging technology with applications in disease diagnostics, intervention, and treatment planning. Accurate MRI segmentation is critical for diagnosing abnormalities, monitoring diseases, and deciding on a course of treatment. With the advent of advanced deep learning frameworks, fully automated and accurate MRI segmentation is advancing. Traditional supervised deep learning techniques have advanced tremendously, reaching clinical-level accuracy in the field of segmentation. However, these algorithms still require a large amount of annotated data, which is oftentimes unavailable or impractical. One way to circumvent this issue is to utilize algorithms that exploit a limited amount of labeled data. This paper aims to review such state-of-the-art algorithms that use a limited number of annotated samples. We explain the fundamental principles of self-supervised learning, generative models, few-shot learning, and semi-supervised learning and summarize their applications in cardiac, abdomen, and brain MRI segmentation. Throughout this review, we highlight algorithms that can be employed based on the quantity of annotated data available. We also present a comprehensive list of notable publicly available MRI segmentation datasets. To conclude, we discuss possible future directions of the field-including emerging algorithms, such as contrastive language-image pretraining, and potential combinations across the methods discussed-that can further increase the efficacy of image segmentation with limited labels.

Estimation of subvoxel fat infiltration in neurodegenerative muscle disorders using quantitative multi-T2 analysis.

MRI's T2 relaxation time is a valuable biomarker for neuromuscular disorders and muscle dystrophies. One of the hallmarks of these pathologies is the infiltration of adipose tissue and a loss of muscle volume. This leads to a mixture of two signal components, from fat and from water, to appear in each imaged voxel, each having a specific T2 relaxation time. In this proof-of-concept work, we present a technique that can separate the signals from water and from fat within each voxel, measure their separate T2 values, and calculate their relative fractions. The echo modulation curve (EMC) algorithm is a dictionary-based technique that offers accurate and reproducible mapping of T2 relaxation times. We present an extension of the EMC algorithm for estimating subvoxel fat and water fractions, alongside the T2 and proton-density values of each component. To facilitate data processing, calf and thigh anatomy were automatically segmented using a fully convolutional neural network and FSLeyes software. The preprocessing included creating two signal dictionaries, for water and for fat, using Bloch simulations of the prospective protocol. Postprocessing included voxelwise fitting for two components, by matching the experimental decay curve to a linear combination of the two simulated dictionaries. Subvoxel fat and water fractions and relaxation times were generated and used to calculate a new quantitative biomarker, termed viable muscle index, and reflecting disease severity. This biomarker indicates the fraction of remaining muscle out of the entire muscle region. The results were compared with those using the conventional Dixon technique, showing high agreement (R = 0.98, p < 0.001). It was concluded that the new extension of the EMC algorithm can be used to quantify abnormal fat infiltration as well as identify early inflammatory processes corresponding to elevation in the T2 value of the water (muscle) component. This new ability may improve the diagnostic accuracy of neuromuscular diseases, help stratification of patients according to disease severity, and offer an efficient tool for tracking disease progression.

Psychophysical Evaluation of Visual vs. Computer-Aided Detection of Brain Lesions on Magnetic Resonance Images.

BACKGROUND: Magnetic resonance imaging (MRI) diagnosis is usually performed by analyzing contrast-weighted images, where pathology is detected once it reached a certain visual threshold. Computer-aided diagnosis (CAD) has been proposed as a way for achieving higher sensitivity to early pathology. PURPOSE: To compare conventional (i.e., visual) MRI assessment of artificially generated multiple sclerosis (MS) lesions in the brain's white matter to CAD based on a deep neural network. STUDY TYPE: Prospective. POPULATION: A total of 25 neuroradiologists (15 males, age 39 ± 9, 9 ± 9.8 years of experience) independently assessed all synthetic lesions. FIELD STRENGTH/SEQUENCE: A 3.0 T, T2 -weighted multi-echo spin-echo (MESE) sequence. ASSESSMENT: MS lesions of varying severity levels were artificially generated in healthy volunteer MRI scans by manipulating T2 values. Radiologists and a neural network were tasked with detecting these lesions in a series of 48 MR images. Sixteen images presented healthy anatomy and the rest contained a single lesion at eight increasing severity levels (6%, 9%, 12%, 15%, 18%, 21%, 25%, and 30% elevation in T2 ). True positive (TP) rates, false positive (FP) rates, and odds ratios (ORs) were compared between radiological diagnosis and CAD across the range lesion severity levels. STATISTICAL TESTS: Diagnostic performance of the two approaches was compared using z-tests on TP rates, FP rates, and the logarithm of ORs across severity levels. A P-value <0.05 was considered statistically significant. RESULTS: ORs of identifying pathology were significantly higher for CAD vis-à-vis visual inspection for all lesions' severity levels. For a 6% change in T2 value (lowest severity), radiologists' TP and FP rates were not significantly different (P = 0.12), while the corresponding CAD results remained statistically significant. DATA CONCLUSION: CAD is capable of detecting the presence or absence of more subtle lesions with greater precision than the representative group of 25 radiologists chosen in this study. LEVEL OF EVIDENCE: 1 TECHNICAL EFFICACY: Stage 3.

Semiautomated segmentation of hepatocellular carcinoma tumors with MRI using convolutional neural networks.

OBJECTIVE: To assess the performance of convolutional neural networks (CNNs) for semiautomated segmentation of hepatocellular carcinoma (HCC) tumors on MRI. METHODS: This retrospective single-center study included 292 patients (237 M/55F, mean age 61 years) with pathologically confirmed HCC between 08/2015 and 06/2019 and who underwent MRI before surgery. The dataset was randomly divided into training (n = 195), validation (n = 66), and test sets (n = 31). Volumes of interest (VOIs) were manually placed on index lesions by 3 independent radiologists on different sequences (T2-weighted imaging [WI], T1WI pre-and post-contrast on arterial [AP], portal venous [PVP], delayed [DP, 3 min post-contrast] and hepatobiliary phases [HBP, when using gadoxetate], and diffusion-weighted imaging [DWI]). Manual segmentation was used as ground truth to train and validate a CNN-based pipeline. For semiautomated segmentation of tumors, we selected a random pixel inside the VOI, and the CNN provided two outputs: single slice and volumetric outputs. Segmentation performance and inter-observer agreement were analyzed using the 3D Dice similarity coefficient (DSC). RESULTS: A total of 261 HCCs were segmented on the training/validation sets, and 31 on the test set. The median lesion size was 3.0 cm (IQR 2.0-5.2 cm). Mean DSC (test set) varied depending on the MRI sequence with a range between 0.442 (ADC) and 0.778 (high b-value DWI) for single-slice segmentation; and between 0.305 (ADC) and 0.667 (T1WI pre) for volumetric-segmentation. Comparison between the two models showed better performance in single-slice segmentation, with statistical significance on T2WI, T1WI-PVP, DWI, and ADC. Inter-observer reproducibility of segmentation analysis showed a mean DSC of 0.71 in lesions between 1 and 2 cm, 0.85 in lesions between 2 and 5 cm, and 0.82 in lesions > 5 cm. CONCLUSION: CNN models have fair to good performance for semiautomated HCC segmentation, depending on the sequence and tumor size, with better performance for the single-slice approach. Refinement of volumetric approaches is needed in future studies. KEY POINTS: • Semiautomated single-slice and volumetric segmentation using convolutional neural networks (CNNs) models provided fair to good performance for hepatocellular carcinoma segmentation on MRI. • CNN models' performance for HCC segmentation accuracy depends on the MRI sequence and tumor size, with the best results on diffusion-weighted imaging and T1-weighted imaging pre-contrast, and for larger lesions.

Fully automated prediction of liver fibrosis using deep learning analysis of gadoxetic acid-enhanced MRI.

OBJECTIVES: To (1) develop a fully automated deep learning (DL) algorithm based on gadoxetic acid-enhanced hepatobiliary phase (HBP) MRI and (2) compare the diagnostic performance of DL vs. MR elastography (MRE) for noninvasive staging of liver fibrosis. METHODS: This single-center retrospective study included 355 patients (M/F 238/117, mean age 60 years; training, n = 178; validation, n = 123; test, n = 54) who underwent gadoxetic acid-enhanced abdominal MRI, including HBP and MRE, and pathological evaluation of the liver within 1 year of MRI. Cropped liver HBP images from a custom-written fully automated liver segmentation were used as input for DL. A transfer learning approach based on the ImageNet VGG16 model was used. Different DL models were built for the prediction of fibrosis stages F1-4, F2-4, F3-4, and F4. ROC analysis was performed to evaluate the performance of DL in training, validation, and test sets and of MRE liver stiffness in the test set. RESULTS: AUC values of DL were 0.99/0.70/0.77 (F1-4), 0.92/0.71/0.91 (F2-4), 0.91/0.78/0.90 (F3-4), and 0.98/0.83/0.85 (F4) for training/validation/test sets, respectively. The AUCs of MRE liver stiffness in the test set were 0.86 (F1-4), 0.87 (F2-4), 0.92 (F3-4), and 0.86 (F4). AUCs of MRE and DL were not significantly different for any of the fibrosis stages (p > 0.134). CONCLUSIONS: The fully automated DL models based on HBP gadoxetic acid MRI showed good-to-excellent diagnostic performance for staging of liver fibrosis, with similar diagnostic performance to MRE. After validation in independent sets, the DL algorithm may allow for noninvasive liver fibrosis assessment without the need for additional MRI hardware. KEY POINTS: • The developed deep learning algorithm, based on routine standard-of-care gadoxetic acid-enhanced MRI data, showed good-to-excellent diagnostic performance for noninvasive staging of liver fibrosis. • The diagnostic performance of the deep learning algorithm was equivalent to that of MR elastography in a separate test set.

A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises.

Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.

Convolutional Neural Networks for Radiologic Images: A Radiologist's Guide.

Deep learning has rapidly advanced in various fields within the past few years and has recently gained particular attention in the radiology community. This article provides an introduction to deep learning technology and presents the stages that are entailed in the design process of deep learning radiology research. In addition, the article details the results of a survey of the application of deep learning-specifically, the application of convolutional neural networks-to radiologic imaging that was focused on the following five major system organs: chest, breast, brain, musculoskeletal system, and abdomen and pelvis. The survey of the studies is followed by a discussion about current challenges and future trends and their potential implications for radiology. This article may be used as a guide for radiologists planning research in the field of radiologic image analysis using convolutional neural networks.

The fiber-density-coreset for redundancy reduction in huge fiber-sets.

State of the art Diffusion Weighted Magnetic Resonance Imaging (DW-MRI) protocols of white matter followed by advanced tractography techniques produce impressive reconstructions of White Matter (WM) pathways. These pathways often contain millions of trajectories (fibers). While for several applications the high number of fibers is essential, other applications (visualization, registration, some types of across-subject comparison) can achieve satisfying results using much smaller sets and may be overburdened by the computational load of the large fiber sets. In this paper we propose a novel, highly efficient algorithm for extracting a meaningful subset of fibers, which we term the Fiber-Density-Coreset (FDC). The reduced set is optimized to represent the main structures of the brain. FDC is based on an efficient geometric approximation paradigm named coresets, an optimization scheme showing much success in tasks requiring large computation time and/or memory. FDC was compared to two commonly used methods for selecting a reduced set of fibers: fiber-clustering and downsampling. The reduced sets were evaluated by several methods, including a novel structural comparison to the full sets called 3D indicator structure comparison (3D-ISC). The comparison was applied to High Angular Resolution Diffusion Imaging (HARDI) scans of 15 healthy individuals obtained from the Human Connectome Project. FDC produced the most satisfying subsets, consistently in all 15 subjects. It also displayed low memory usage and significantly lower running time than conventional fiber reduction schemes.

Longitudinal multiple sclerosis lesion segmentation: Resource and challenge.

In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time-points, and test data of fourteen subjects with a mean of 4.4 time-points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state-of-the-art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: (1) the sharing of a rich data set; (2) collaboration and comparison of the various avenues of research being pursued in the community; and (3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website2 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters.

Automated analysis of individual sperm cells using stain-free interferometric phase microscopy and machine learning.

Currently, the delicate process of selecting sperm cells to be used for in vitro fertilization (IVF) is still based on the subjective, qualitative analysis of experienced clinicians using non-quantitative optical microscopy techniques. In this work, a method was developed for the automated analysis of sperm cells based on the quantitative phase maps acquired through use of interferometric phase microscopy (IPM). Over 1,400 human sperm cells from 8 donors were imaged using IPM, and an algorithm was designed to digitally isolate sperm cell heads from the quantitative phase maps while taking into consideration both the cell 3D morphology and contents, as well as acquire features describing sperm head morphology. A subset of these features was used to train a support vector machine (SVM) classifier to automatically classify sperm of good and bad morphology. The SVM achieves an area under the receiver operating characteristic curve of 88.59% and an area under the precision-recall curve of 88.67%, as well as precisions of 90% or higher. We believe that our automatic analysis can become the basis for objective and automatic sperm cell selection in IVF. © 2017 International Society for Advancement of Cytometry.

Quantitative phase microscopy spatial signatures of cancer cells.

We present cytometric classification of live healthy and cancerous cells by using the spatial morphological and textural information found in the label-free quantitative phase images of the cells. We compare both healthy cells to primary tumor cells and primary tumor cells to metastatic cancer cells, where tumor biopsies and normal tissues were isolated from the same individuals. To mimic analysis of liquid biopsies by flow cytometry, the cells were imaged while unattached to the substrate. We used low-coherence off-axis interferometric phase microscopy setup, which allows a single-exposure acquisition mode, and thus is suitable for quantitative imaging of dynamic cells during flow. After acquisition, the optical path delay maps of the cells were extracted and then used to calculate 15 parameters derived from the cellular 3D morphology and texture. Upon analyzing tens of cells in each group, we found high statistical significance in the difference between the groups in most of the parameters calculated, with the same trends for all statistically significant parameters. Furthermore, a specially designed machine learning algorithm, implemented on the phase map extracted features, classified the correct cell type (healthy/cancer/metastatic) with 81-93% sensitivity and 81-99% specificity. The quantitative phase imaging approach for liquid biopsies presented in this paper could be the basis for advanced techniques of staging freshly isolated live cancer cells in imaging flow cytometers. © 2017 International Society for Advancement of Cytometry.

Whole brain fiber-based comparison (FBC)-A tool for diffusion tensor imaging-based cohort studies.

We present a novel method for fiber-based comparison of diffusion tensor imaging (DTI) scans of groups of subjects. The method entails initial preprocessing and fiber reconstruction by tractography of each brain in its native coordinate system. Several diffusion parameters are sampled along each fiber and used in subsequent comparisons. A spatial correspondence between subjects is established based on geometric similarity between fibers in a template set (several choices for template are explored), and fibers in all other subjects. Diffusion parameters between groups are compared statistically for each template fiber. Results are presented at single fiber resolution. As an initial exploratory step in neurological population studies this method points to the locations affected by the pathology of interest, without requiring a hypothesis. It does not make any grouping assumptions on the fibers and no manual intervention is needed. The framework was applied here to 18 healthy subjects and 23 amyotrophic lateral sclerosis (ALS) patients. The results are compatible with previous findings and with the tract based spatial statistics (TBSS) method. Hum Brain Mapp 37:477-490, 2016. © 2015 Wiley Periodicals, Inc.

Quantitative Prediction of Right Ventricular Size and Function From the ECG.

BACKGROUND: Right ventricular ejection fraction (RVEF) and end-diastolic volume (RVEDV) are not readily assessed through traditional modalities. Deep learning-enabled ECG analysis for estimation of right ventricular (RV) size or function is unexplored. METHODS AND RESULTS: We trained a deep learning-ECG model to predict RV dilation (RVEDV >120 mL/m2), RV dysfunction (RVEF ≤40%), and numerical RVEDV and RVEF from a 12-lead ECG paired with reference-standard cardiac magnetic resonance imaging volumetric measurements in UK Biobank (UKBB; n=42 938). We fine-tuned in a multicenter health system (MSHoriginal [Mount Sinai Hospital]; n=3019) with prospective validation over 4 months (MSHvalidation; n=115). We evaluated performance with area under the receiver operating characteristic curve for categorical and mean absolute error for continuous measures overall and in key subgroups. We assessed the association of RVEF prediction with transplant-free survival with Cox proportional hazards models. The prevalence of RV dysfunction for UKBB/MSHoriginal/MSHvalidation cohorts was 1.0%/18.0%/15.7%, respectively. RV dysfunction model area under the receiver operating characteristic curve for UKBB/MSHoriginal/MSHvalidation cohorts was 0.86/0.81/0.77, respectively. The prevalence of RV dilation for UKBB/MSHoriginal/MSHvalidation cohorts was 1.6%/10.6%/4.3%. RV dilation model area under the receiver operating characteristic curve for UKBB/MSHoriginal/MSHvalidation cohorts was 0.91/0.81/0.92, respectively. MSHoriginal mean absolute error was RVEF=7.8% and RVEDV=17.6 mL/m2. The performance of the RVEF model was similar in key subgroups including with and without left ventricular dysfunction. Over a median follow-up of 2.3 years, predicted RVEF was associated with adjusted transplant-free survival (hazard ratio, 1.40 for each 10% decrease; P=0.031). CONCLUSIONS: Deep learning-ECG analysis can identify significant cardiac magnetic resonance imaging RV dysfunction and dilation with good performance. Predicted RVEF is associated with clinical outcome.

Artificial intelligence in medicine: mitigating risks and maximizing benefits via quality assurance, quality control, and acceptance testing.

The adoption of artificial intelligence (AI) tools in medicine poses challenges to existing clinical workflows. This commentary discusses the necessity of context-specific quality assurance (QA), emphasizing the need for robust QA measures with quality control (QC) procedures that encompass (1) acceptance testing (AT) before clinical use, (2) continuous QC monitoring, and (3) adequate user training. The discussion also covers essential components of AT and QA, illustrated with real-world examples. We also highlight what we see as the shared responsibility of manufacturers or vendors, regulators, healthcare systems, medical physicists, and clinicians to enact appropriate testing and oversight to ensure a safe and equitable transformation of medicine through AI.

AI and machine learning in medical imaging: key points from development to translation.

Innovation in medical imaging artificial intelligence (AI)/machine learning (ML) demands extensive data collection, algorithmic advancements, and rigorous performance assessments encompassing aspects such as generalizability, uncertainty, bias, fairness, trustworthiness, and interpretability. Achieving widespread integration of AI/ML algorithms into diverse clinical tasks will demand a steadfast commitment to overcoming issues in model design, development, and performance assessment. The complexities of AI/ML clinical translation present substantial challenges, requiring engagement with relevant stakeholders, assessment of cost-effectiveness for user and patient benefit, timely dissemination of information relevant to robust functioning throughout the AI/ML lifecycle, consideration of regulatory compliance, and feedback loops for real-world performance evidence. This commentary addresses several hurdles for the development and adoption of AI/ML technologies in medical imaging. Comprehensive attention to these underlying and often subtle factors is critical not only for tackling the challenges but also for exploring novel opportunities for the advancement of AI in radiology.

Multimodal fusion models for pulmonary embolism mortality prediction.

Pulmonary embolism (PE) is a common, life threatening cardiovascular emergency. Risk stratification is one of the core principles of acute PE management and determines the choice of diagnostic and therapeutic strategies. In routine clinical practice, clinicians rely on the patient's electronic health record (EHR) to provide a context for their medical imaging interpretation. Most deep learning models for radiology applications only consider pixel-value information without the clinical context. Only a few integrate both clinical and imaging data. In this work, we develop and compare multimodal fusion models that can utilize multimodal data by combining both volumetric pixel data and clinical patient data for automatic risk stratification of PE. Our best performing model is an intermediate fusion model that incorporates both bilinear attention and TabNet, and can be trained in an end-to-end manner. The results show that multimodality boosts performance by up to 14% with an area under the curve (AUC) of 0.96 for assessing PE severity, with a sensitivity of 90% and specificity of 94%, thus pointing to the value of using multimodal data to automatically assess PE severity.

Discovery Viewer (DV): Web-Based Medical AI Model Development Platform and Deployment Hub.

The rapid rise of artificial intelligence (AI) in medicine in the last few years highlights the importance of developing bigger and better systems for data and model sharing. However, the presence of Protected Health Information (PHI) in medical data poses a challenge when it comes to sharing. One potential solution to mitigate the risk of PHI breaches is to exclusively share pre-trained models developed using private datasets. Despite the availability of these pre-trained networks, there remains a need for an adaptable environment to test and fine-tune specific models tailored for clinical tasks. This environment should be open for peer testing, feedback, and continuous model refinement, allowing dynamic model updates that are especially important in the medical field, where diseases and scanning techniques evolve rapidly. In this context, the Discovery Viewer (DV) platform was developed in-house at the Biomedical Engineering and Imaging Institute at Mount Sinai (BMEII) to facilitate the creation and distribution of cutting-edge medical AI models that remain accessible after their development. The all-in-one platform offers a unique environment for non-AI experts to learn, develop, and share their own deep learning (DL) concepts. This paper presents various use cases of the platform, with its primary goal being to demonstrate how DV holds the potential to empower individuals without expertise in AI to create high-performing DL models. We tasked three non-AI experts to develop different musculoskeletal AI projects that encompassed segmentation, regression, and classification tasks. In each project, 80% of the samples were provided with a subset of these samples annotated to aid the volunteers in understanding the expected annotation task. Subsequently, they were responsible for annotating the remaining samples and training their models through the platform's "Training Module". The resulting models were then tested on the separate 20% hold-off dataset to assess their performance. The classification model achieved an accuracy of 0.94, a sensitivity of 0.92, and a specificity of 1. The regression model yielded a mean absolute error of 14.27 pixels. And the segmentation model attained a Dice Score of 0.93, with a sensitivity of 0.9 and a specificity of 0.99. This initiative seeks to broaden the community of medical AI model developers and democratize the access of this technology to all stakeholders. The ultimate goal is to facilitate the transition of medical AI models from research to clinical settings.

A foundational vision transformer improves diagnostic performance for electrocardiograms.

The electrocardiogram (ECG) is a ubiquitous diagnostic modality. Convolutional neural networks (CNNs) applied towards ECG analysis require large sample sizes, and transfer learning approaches for biomedical problems may result in suboptimal performance when pre-training is done on natural images. We leveraged masked image modeling to create a vision-based transformer model, HeartBEiT, for electrocardiogram waveform analysis. We pre-trained this model on 8.5 million ECGs and then compared performance vs. standard CNN architectures for diagnosis of hypertrophic cardiomyopathy, low left ventricular ejection fraction and ST elevation myocardial infarction using differing training sample sizes and independent validation datasets. We find that HeartBEiT has significantly higher performance at lower sample sizes compared to other models. We also find that HeartBEiT improves explainability of diagnosis by highlighting biologically relevant regions of the EKG vs. standard CNNs. Domain specific pre-trained transformer models may exceed the classification performance of models trained on natural images especially in very low data regimes. The combination of the architecture and such pre-training allows for more accurate, granular explainability of model predictions.

Quantitative prediction of right ventricular and size and function from the electrocardiogram.

Background: Right ventricular ejection fraction (RVEF) and end-diastolic volume (RVEDV) are not readily assessed through traditional modalities. Deep-learning enabled 12-lead electrocardiogram analysis (DL-ECG) for estimation of RV size or function is unexplored. Methods: We trained a DL-ECG model to predict RV dilation (RVEDV>120 mL/m2), RV dysfunction (RVEF≤40%), and numerical RVEDV/RVEF from 12-lead ECG paired with reference-standard cardiac MRI (cMRI) volumetric measurements in UK biobank (UKBB; n=42,938). We fine-tuned in a multi-center health system (MSHoriginal; n=3,019) with prospective validation over 4 months (MSHvalidation; n=115). We evaluated performance using area under the receiver operating curve (AUROC) for categorical and mean absolute error (MAE) for continuous measures overall and in key subgroups. We assessed association of RVEF prediction with transplant-free survival with Cox proportional hazards models. Results: Prevalence of RV dysfunction for UKBB/MSHoriginal/MSHvalidation cohorts was 1.0%/18.0%/15.7%, respectively. RV dysfunction model AUROC for UKBB/MSHoriginal/MSHvalidation cohorts was 0.86/0.81/0.77, respectively. Prevalence of RV dilation for UKBB/MSHoriginal/MSHvalidation cohorts was 1.6%/10.6%/4.3%. RV dilation model AUROC for UKBB/MSHoriginal/MSHvalidation cohorts 0.91/0.81/0.92, respectively. MSHoriginal MAE was RVEF=7.8% and RVEDV=17.6 ml/m2. Performance was similar in key subgroups including with and without left ventricular dysfunction. Over median follow-up of 2.3 years, predicted RVEF was independently associated with composite outcome (HR 1.37 for each 10% decrease, p=0.046). Conclusions: DL-ECG analysis can accurately identify significant RV dysfunction and dilation both overall and in key subgroups. Predicted RVEF is independently associated with clinical outcome.

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer-aided diagnosis in medical imaging.

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both "traditional" machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support.