Artificial intelligence in medicine: mitigating risks and maximizing benefits via quality assurance, quality control, and acceptance testing.

The adoption of artificial intelligence (AI) tools in medicine poses challenges to existing clinical workflows. This commentary discusses the necessity of context-specific quality assurance (QA), emphasizing the need for robust QA measures with quality control (QC) procedures that encompass (1) acceptance testing (AT) before clinical use, (2) continuous QC monitoring, and (3) adequate user training. The discussion also covers essential components of AT and QA, illustrated with real-world examples. We also highlight what we see as the shared responsibility of manufacturers or vendors, regulators, healthcare systems, medical physicists, and clinicians to enact appropriate testing and oversight to ensure a safe and equitable transformation of medicine through AI.

A review of self-supervised, generative, and few-shot deep learning methods for data-limited magnetic resonance imaging segmentation.

Magnetic resonance imaging (MRI) is a ubiquitous medical imaging technology with applications in disease diagnostics, intervention, and treatment planning. Accurate MRI segmentation is critical for diagnosing abnormalities, monitoring diseases, and deciding on a course of treatment. With the advent of advanced deep learning frameworks, fully automated and accurate MRI segmentation is advancing. Traditional supervised deep learning techniques have advanced tremendously, reaching clinical-level accuracy in the field of segmentation. However, these algorithms still require a large amount of annotated data, which is oftentimes unavailable or impractical. One way to circumvent this issue is to utilize algorithms that exploit a limited amount of labeled data. This paper aims to review such state-of-the-art algorithms that use a limited number of annotated samples. We explain the fundamental principles of self-supervised learning, generative models, few-shot learning, and semi-supervised learning and summarize their applications in cardiac, abdomen, and brain MRI segmentation. Throughout this review, we highlight algorithms that can be employed based on the quantity of annotated data available. We also present a comprehensive list of notable publicly available MRI segmentation datasets. To conclude, we discuss possible future directions of the field-including emerging algorithms, such as contrastive language-image pretraining, and potential combinations across the methods discussed-that can further increase the efficacy of image segmentation with limited labels.

Quantitative Prediction of Right Ventricular Size and Function From the ECG.

BACKGROUND: Right ventricular ejection fraction (RVEF) and end-diastolic volume (RVEDV) are not readily assessed through traditional modalities. Deep learning-enabled ECG analysis for estimation of right ventricular (RV) size or function is unexplored. METHODS AND RESULTS: We trained a deep learning-ECG model to predict RV dilation (RVEDV >120 mL/m2), RV dysfunction (RVEF ≤40%), and numerical RVEDV and RVEF from a 12-lead ECG paired with reference-standard cardiac magnetic resonance imaging volumetric measurements in UK Biobank (UKBB; n=42 938). We fine-tuned in a multicenter health system (MSHoriginal [Mount Sinai Hospital]; n=3019) with prospective validation over 4 months (MSHvalidation; n=115). We evaluated performance with area under the receiver operating characteristic curve for categorical and mean absolute error for continuous measures overall and in key subgroups. We assessed the association of RVEF prediction with transplant-free survival with Cox proportional hazards models. The prevalence of RV dysfunction for UKBB/MSHoriginal/MSHvalidation cohorts was 1.0%/18.0%/15.7%, respectively. RV dysfunction model area under the receiver operating characteristic curve for UKBB/MSHoriginal/MSHvalidation cohorts was 0.86/0.81/0.77, respectively. The prevalence of RV dilation for UKBB/MSHoriginal/MSHvalidation cohorts was 1.6%/10.6%/4.3%. RV dilation model area under the receiver operating characteristic curve for UKBB/MSHoriginal/MSHvalidation cohorts was 0.91/0.81/0.92, respectively. MSHoriginal mean absolute error was RVEF=7.8% and RVEDV=17.6 mL/m2. The performance of the RVEF model was similar in key subgroups including with and without left ventricular dysfunction. Over a median follow-up of 2.3 years, predicted RVEF was associated with adjusted transplant-free survival (hazard ratio, 1.40 for each 10% decrease; P=0.031). CONCLUSIONS: Deep learning-ECG analysis can identify significant cardiac magnetic resonance imaging RV dysfunction and dilation with good performance. Predicted RVEF is associated with clinical outcome.

Discovery Viewer (DV): Web-Based Medical AI Model Development Platform and Deployment Hub.

The rapid rise of artificial intelligence (AI) in medicine in the last few years highlights the importance of developing bigger and better systems for data and model sharing. However, the presence of Protected Health Information (PHI) in medical data poses a challenge when it comes to sharing. One potential solution to mitigate the risk of PHI breaches is to exclusively share pre-trained models developed using private datasets. Despite the availability of these pre-trained networks, there remains a need for an adaptable environment to test and fine-tune specific models tailored for clinical tasks. This environment should be open for peer testing, feedback, and continuous model refinement, allowing dynamic model updates that are especially important in the medical field, where diseases and scanning techniques evolve rapidly. In this context, the Discovery Viewer (DV) platform was developed in-house at the Biomedical Engineering and Imaging Institute at Mount Sinai (BMEII) to facilitate the creation and distribution of cutting-edge medical AI models that remain accessible after their development. The all-in-one platform offers a unique environment for non-AI experts to learn, develop, and share their own deep learning (DL) concepts. This paper presents various use cases of the platform, with its primary goal being to demonstrate how DV holds the potential to empower individuals without expertise in AI to create high-performing DL models. We tasked three non-AI experts to develop different musculoskeletal AI projects that encompassed segmentation, regression, and classification tasks. In each project, 80% of the samples were provided with a subset of these samples annotated to aid the volunteers in understanding the expected annotation task. Subsequently, they were responsible for annotating the remaining samples and training their models through the platform's "Training Module". The resulting models were then tested on the separate 20% hold-off dataset to assess their performance. The classification model achieved an accuracy of 0.94, a sensitivity of 0.92, and a specificity of 1. The regression model yielded a mean absolute error of 14.27 pixels. And the segmentation model attained a Dice Score of 0.93, with a sensitivity of 0.9 and a specificity of 0.99. This initiative seeks to broaden the community of medical AI model developers and democratize the access of this technology to all stakeholders. The ultimate goal is to facilitate the transition of medical AI models from research to clinical settings.

A foundational vision transformer improves diagnostic performance for electrocardiograms.

The electrocardiogram (ECG) is a ubiquitous diagnostic modality. Convolutional neural networks (CNNs) applied towards ECG analysis require large sample sizes, and transfer learning approaches for biomedical problems may result in suboptimal performance when pre-training is done on natural images. We leveraged masked image modeling to create a vision-based transformer model, HeartBEiT, for electrocardiogram waveform analysis. We pre-trained this model on 8.5 million ECGs and then compared performance vs. standard CNN architectures for diagnosis of hypertrophic cardiomyopathy, low left ventricular ejection fraction and ST elevation myocardial infarction using differing training sample sizes and independent validation datasets. We find that HeartBEiT has significantly higher performance at lower sample sizes compared to other models. We also find that HeartBEiT improves explainability of diagnosis by highlighting biologically relevant regions of the EKG vs. standard CNNs. Domain specific pre-trained transformer models may exceed the classification performance of models trained on natural images especially in very low data regimes. The combination of the architecture and such pre-training allows for more accurate, granular explainability of model predictions.

Multimodal fusion models for pulmonary embolism mortality prediction.

Pulmonary embolism (PE) is a common, life threatening cardiovascular emergency. Risk stratification is one of the core principles of acute PE management and determines the choice of diagnostic and therapeutic strategies. In routine clinical practice, clinicians rely on the patient's electronic health record (EHR) to provide a context for their medical imaging interpretation. Most deep learning models for radiology applications only consider pixel-value information without the clinical context. Only a few integrate both clinical and imaging data. In this work, we develop and compare multimodal fusion models that can utilize multimodal data by combining both volumetric pixel data and clinical patient data for automatic risk stratification of PE. Our best performing model is an intermediate fusion model that incorporates both bilinear attention and TabNet, and can be trained in an end-to-end manner. The results show that multimodality boosts performance by up to 14% with an area under the curve (AUC) of 0.96 for assessing PE severity, with a sensitivity of 90% and specificity of 94%, thus pointing to the value of using multimodal data to automatically assess PE severity.

Quantitative prediction of right ventricular and size and function from the electrocardiogram.

Background: Right ventricular ejection fraction (RVEF) and end-diastolic volume (RVEDV) are not readily assessed through traditional modalities. Deep-learning enabled 12-lead electrocardiogram analysis (DL-ECG) for estimation of RV size or function is unexplored. Methods: We trained a DL-ECG model to predict RV dilation (RVEDV>120 mL/m2), RV dysfunction (RVEF≤40%), and numerical RVEDV/RVEF from 12-lead ECG paired with reference-standard cardiac MRI (cMRI) volumetric measurements in UK biobank (UKBB; n=42,938). We fine-tuned in a multi-center health system (MSHoriginal; n=3,019) with prospective validation over 4 months (MSHvalidation; n=115). We evaluated performance using area under the receiver operating curve (AUROC) for categorical and mean absolute error (MAE) for continuous measures overall and in key subgroups. We assessed association of RVEF prediction with transplant-free survival with Cox proportional hazards models. Results: Prevalence of RV dysfunction for UKBB/MSHoriginal/MSHvalidation cohorts was 1.0%/18.0%/15.7%, respectively. RV dysfunction model AUROC for UKBB/MSHoriginal/MSHvalidation cohorts was 0.86/0.81/0.77, respectively. Prevalence of RV dilation for UKBB/MSHoriginal/MSHvalidation cohorts was 1.6%/10.6%/4.3%. RV dilation model AUROC for UKBB/MSHoriginal/MSHvalidation cohorts 0.91/0.81/0.92, respectively. MSHoriginal MAE was RVEF=7.8% and RVEDV=17.6 ml/m2. Performance was similar in key subgroups including with and without left ventricular dysfunction. Over median follow-up of 2.3 years, predicted RVEF was independently associated with composite outcome (HR 1.37 for each 10% decrease, p=0.046). Conclusions: DL-ECG analysis can accurately identify significant RV dysfunction and dilation both overall and in key subgroups. Predicted RVEF is independently associated with clinical outcome.

Semiautomated segmentation of hepatocellular carcinoma tumors with MRI using convolutional neural networks.

OBJECTIVE: To assess the performance of convolutional neural networks (CNNs) for semiautomated segmentation of hepatocellular carcinoma (HCC) tumors on MRI. METHODS: This retrospective single-center study included 292 patients (237 M/55F, mean age 61 years) with pathologically confirmed HCC between 08/2015 and 06/2019 and who underwent MRI before surgery. The dataset was randomly divided into training (n = 195), validation (n = 66), and test sets (n = 31). Volumes of interest (VOIs) were manually placed on index lesions by 3 independent radiologists on different sequences (T2-weighted imaging [WI], T1WI pre-and post-contrast on arterial [AP], portal venous [PVP], delayed [DP, 3 min post-contrast] and hepatobiliary phases [HBP, when using gadoxetate], and diffusion-weighted imaging [DWI]). Manual segmentation was used as ground truth to train and validate a CNN-based pipeline. For semiautomated segmentation of tumors, we selected a random pixel inside the VOI, and the CNN provided two outputs: single slice and volumetric outputs. Segmentation performance and inter-observer agreement were analyzed using the 3D Dice similarity coefficient (DSC). RESULTS: A total of 261 HCCs were segmented on the training/validation sets, and 31 on the test set. The median lesion size was 3.0 cm (IQR 2.0-5.2 cm). Mean DSC (test set) varied depending on the MRI sequence with a range between 0.442 (ADC) and 0.778 (high b-value DWI) for single-slice segmentation; and between 0.305 (ADC) and 0.667 (T1WI pre) for volumetric-segmentation. Comparison between the two models showed better performance in single-slice segmentation, with statistical significance on T2WI, T1WI-PVP, DWI, and ADC. Inter-observer reproducibility of segmentation analysis showed a mean DSC of 0.71 in lesions between 1 and 2 cm, 0.85 in lesions between 2 and 5 cm, and 0.82 in lesions > 5 cm. CONCLUSION: CNN models have fair to good performance for semiautomated HCC segmentation, depending on the sequence and tumor size, with better performance for the single-slice approach. Refinement of volumetric approaches is needed in future studies. KEY POINTS: • Semiautomated single-slice and volumetric segmentation using convolutional neural networks (CNNs) models provided fair to good performance for hepatocellular carcinoma segmentation on MRI. • CNN models' performance for HCC segmentation accuracy depends on the MRI sequence and tumor size, with the best results on diffusion-weighted imaging and T1-weighted imaging pre-contrast, and for larger lesions.

Estimation of subvoxel fat infiltration in neurodegenerative muscle disorders using quantitative multi-T2 analysis.

MRI's T2 relaxation time is a valuable biomarker for neuromuscular disorders and muscle dystrophies. One of the hallmarks of these pathologies is the infiltration of adipose tissue and a loss of muscle volume. This leads to a mixture of two signal components, from fat and from water, to appear in each imaged voxel, each having a specific T2 relaxation time. In this proof-of-concept work, we present a technique that can separate the signals from water and from fat within each voxel, measure their separate T2 values, and calculate their relative fractions. The echo modulation curve (EMC) algorithm is a dictionary-based technique that offers accurate and reproducible mapping of T2 relaxation times. We present an extension of the EMC algorithm for estimating subvoxel fat and water fractions, alongside the T2 and proton-density values of each component. To facilitate data processing, calf and thigh anatomy were automatically segmented using a fully convolutional neural network and FSLeyes software. The preprocessing included creating two signal dictionaries, for water and for fat, using Bloch simulations of the prospective protocol. Postprocessing included voxelwise fitting for two components, by matching the experimental decay curve to a linear combination of the two simulated dictionaries. Subvoxel fat and water fractions and relaxation times were generated and used to calculate a new quantitative biomarker, termed viable muscle index, and reflecting disease severity. This biomarker indicates the fraction of remaining muscle out of the entire muscle region. The results were compared with those using the conventional Dixon technique, showing high agreement (R = 0.98, p < 0.001). It was concluded that the new extension of the EMC algorithm can be used to quantify abnormal fat infiltration as well as identify early inflammatory processes corresponding to elevation in the T2 value of the water (muscle) component. This new ability may improve the diagnostic accuracy of neuromuscular diseases, help stratification of patients according to disease severity, and offer an efficient tool for tracking disease progression.

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer-aided diagnosis in medical imaging.

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both "traditional" machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support.

Psychophysical Evaluation of Visual vs. Computer-Aided Detection of Brain Lesions on Magnetic Resonance Images.

BACKGROUND: Magnetic resonance imaging (MRI) diagnosis is usually performed by analyzing contrast-weighted images, where pathology is detected once it reached a certain visual threshold. Computer-aided diagnosis (CAD) has been proposed as a way for achieving higher sensitivity to early pathology. PURPOSE: To compare conventional (i.e., visual) MRI assessment of artificially generated multiple sclerosis (MS) lesions in the brain's white matter to CAD based on a deep neural network. STUDY TYPE: Prospective. POPULATION: A total of 25 neuroradiologists (15 males, age 39 ± 9, 9 ± 9.8 years of experience) independently assessed all synthetic lesions. FIELD STRENGTH/SEQUENCE: A 3.0 T, T2 -weighted multi-echo spin-echo (MESE) sequence. ASSESSMENT: MS lesions of varying severity levels were artificially generated in healthy volunteer MRI scans by manipulating T2 values. Radiologists and a neural network were tasked with detecting these lesions in a series of 48 MR images. Sixteen images presented healthy anatomy and the rest contained a single lesion at eight increasing severity levels (6%, 9%, 12%, 15%, 18%, 21%, 25%, and 30% elevation in T2 ). True positive (TP) rates, false positive (FP) rates, and odds ratios (ORs) were compared between radiological diagnosis and CAD across the range lesion severity levels. STATISTICAL TESTS: Diagnostic performance of the two approaches was compared using z-tests on TP rates, FP rates, and the logarithm of ORs across severity levels. A P-value <0.05 was considered statistically significant. RESULTS: ORs of identifying pathology were significantly higher for CAD vis-à-vis visual inspection for all lesions' severity levels. For a 6% change in T2 value (lowest severity), radiologists' TP and FP rates were not significantly different (P = 0.12), while the corresponding CAD results remained statistically significant. DATA CONCLUSION: CAD is capable of detecting the presence or absence of more subtle lesions with greater precision than the representative group of 25 radiologists chosen in this study. LEVEL OF EVIDENCE: 1 TECHNICAL EFFICACY: Stage 3.

RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning.

Purpose: To demonstrate the value of pretraining with millions of radiologic images compared with ImageNet photographic images on downstream medical applications when using transfer learning. Materials and Methods: This retrospective study included patients who underwent a radiologic study between 2005 and 2020 at an outpatient imaging facility. Key images and associated labels from the studies were retrospectively extracted from the original study interpretation. These images were used for RadImageNet model training with random weight initiation. The RadImageNet models were compared with ImageNet models using the area under the receiver operating characteristic curve (AUC) for eight classification tasks and using Dice scores for two segmentation problems. Results: The RadImageNet database consists of 1.35 million annotated medical images in 131 872 patients who underwent CT, MRI, and US for musculoskeletal, neurologic, oncologic, gastrointestinal, endocrine, abdominal, and pulmonary pathologic conditions. For transfer learning tasks on small datasets-thyroid nodules (US), breast masses (US), anterior cruciate ligament injuries (MRI), and meniscal tears (MRI)-the RadImageNet models demonstrated a significant advantage (P < .001) to ImageNet models (9.4%, 4.0%, 4.8%, and 4.5% AUC improvements, respectively). For larger datasets-pneumonia (chest radiography), COVID-19 (CT), SARS-CoV-2 (CT), and intracranial hemorrhage (CT)-the RadImageNet models also illustrated improved AUC (P < .001) by 1.9%, 6.1%, 1.7%, and 0.9%, respectively. Additionally, lesion localizations of the RadImageNet models were improved by 64.6% and 16.4% on thyroid and breast US datasets, respectively. Conclusion: RadImageNet pretrained models demonstrated better interpretability compared with ImageNet models, especially for smaller radiologic datasets.Keywords: CT, MR Imaging, US, Head/Neck, Thorax, Brain/Brain Stem, Evidence-based Medicine, Computer Applications-General (Informatics) Supplemental material is available for this article. Published under a CC BY 4.0 license.See also the commentary by Cadrin-Chênevert in this issue.

Quantification of Intra-Muscular Adipose Infiltration in Calf/Thigh MRI Using Fully and Weakly Supervised Semantic Segmentation.

Purpose: Infiltration of fat into lower limb muscles is one of the key markers for the severity of muscle pathologies. The level of fat infiltration varies in its severity across and within patients, and it is traditionally estimated using visual radiologic inspection. Precise quantification of the severity and spatial distribution of this pathological process requires accurate segmentation of lower limb anatomy into muscle and fat. Methods: Quantitative magnetic resonance imaging (qMRI) of the calf and thigh muscles is one of the most effective techniques for estimating pathological accumulation of intra-muscular adipose tissue (IMAT) in muscular dystrophies. In this work, we present a new deep learning (DL) network tool for automated and robust segmentation of lower limb anatomy that is based on the quantification of MRI's transverse (T2) relaxation time. The network was used to segment calf and thigh anatomies into viable muscle areas and IMAT using a weakly supervised learning process. A new disease biomarker was calculated, reflecting the level of abnormal fat infiltration and disease state. A biomarker was then applied on two patient populations suffering from dysferlinopathy and Charcot-Marie-Tooth (CMT) diseases. Results: Comparison of manual vs. automated segmentation of muscle anatomy, viable muscle areas, and intermuscular adipose tissue (IMAT) produced high Dice similarity coefficients (DSCs) of 96.4%, 91.7%, and 93.3%, respectively. Linear regression between the biomarker value calculated based on the ground truth segmentation and based on automatic segmentation produced high correlation coefficients of 97.7% and 95.9% for the dysferlinopathy and CMT patients, respectively. Conclusions: Using a combination of qMRI and DL-based segmentation, we present a new quantitative biomarker of disease severity. This biomarker is automatically calculated and, most importantly, provides a spatially global indication for the state of the disease across the entire thigh or calf.

A Self Supervised StyleGAN for Image Annotation and Classification With Extremely Limited Labels.

The recent success of learning-based algorithms can be greatly attributed to the immense amount of annotated data used for training. Yet, many datasets lack annotations due to the high costs associated with labeling, resulting in degraded performances of deep learning methods. Self-supervised learning is frequently adopted to mitigate the reliance on massive labeled datasets since it exploits unlabeled data to learn relevant feature representations. In this work, we propose SS-StyleGAN, a self-supervised approach for image annotation and classification suitable for extremely small annotated datasets. This novel framework adds self-supervision to the StyleGAN architecture by integrating an encoder that learns the embedding to the StyleGAN latent space, which is well-known for its disentangled properties. The learned latent space enables the smart selection of representatives from the data to be labeled for improved classification performance. We show that the proposed method attains strong classification results using small labeled datasets of sizes 50 and even 10. We demonstrate the superiority of our approach for the tasks of COVID-19 and liver tumor pathology identification.

Weakly supervised attention model for RV strain classification from volumetric CTPA scans.

BACKGROUND AND OBJECTIVE: Evaluation of the right ventricle (RV) is a key component of the clinical assessment of many cardiovascular and pulmonary disorders. In this work, we focus on RV strain classification from patients who were diagnosed with pulmonary embolism (PE) in computed tomography pulmonary angiography (CTPA) scans. PE is a life-threatening condition, often without warning signs or symptoms. Early diagnosis and accurate risk stratification are critical for decreasing mortality rates. High-risk PE relies on the presence of RV dysfunction resulting from acute pressure overload. PE severity classification and specifically, high-risk PE diagnosis are crucial for appropriate therapy. CTPA is the golden standard in the diagnostic workup of suspected PE. Therefore, it can link between diagnosis and risk stratification strategies. METHODS: We retrieved data of consecutive patients who underwent CTPA and were diagnosed with PE and extracted a single binary label of "RV strain biomarker" from the CTPA scan report. This label was used as a weak label for classification. Our solution applies a 3D DenseNet network architecture, further improved by integrating residual attention blocks into the network's layers. RESULTS: This model achieved an area under the receiver operating characteristic curve (AUC) of 0.88 for classifying RV strain. For Youden's index, the model showed a sensitivity of 87% and specificity of 83.7%. Our solution outperforms state-of-the-art 3D CNN networks. The proposed design allows for a fully automated network that can be trained easily in an end-to-end manner without requiring computationally intensive and time-consuming preprocessing or strenuous labeling of the data. CONCLUSIONS: This current solution demonstrates that a small dataset of readily available unmarked CTPAs can be used for effective RV strain classification. To our knowledge, this is the first work that attempts to solve the problem of RV strain classification from CTPA scans and this is the first work where medical images are used in such an architecture. Our generalized self-attention blocks can be incorporated into various existing classification architectures making this a general methodology that can be applied to 3D medical datasets.

Cancer-Cell Deep-Learning Classification by Integrating Quantitative-Phase Spatial and Temporal Fluctuations.

We present a new classification approach for live cells, integrating together the spatial and temporal fluctuation maps and the quantitative optical thickness map of the cell, as acquired by common-path quantitative-phase dynamic imaging and processed with a deep-learning framework. We demonstrate this approach by classifying between two types of cancer cell lines of different metastatic potential originating from the same patient. It is based on the fact that both the cancer-cell morphology and its mechanical properties, as indicated by the cell temporal and spatial fluctuations, change over the disease progression. We tested different fusion methods for inputting both the morphological optical thickness maps and the coinciding spatio-temporal fluctuation maps of the cells to the classifying network framework. We show that the proposed integrated triple-path deep-learning architecture improves over deep-learning classification that is based only on the cell morphological evaluation via its quantitative optical thickness map, demonstrating the benefit in the acquisition of the cells over time and in extracting their spatio-temporal fluctuation maps, to be used as an input to the classifying deep neural network.

A Machine Learning Approach to the Interpretation of Cardiopulmonary Exercise Tests: Development and Validation.

OBJECTIVE: At present, there is no consensus on the best strategy for interpreting the cardiopulmonary exercise test's (CPET) results. This study is aimed at assessing the potential of using computer-aided algorithms to evaluate CPET data for identifying chronic heart failure (CHF) and chronic obstructive pulmonary disease (COPD). METHODS: Data from 234 CPET files from the Pulmonary Institute, at Sheba Medical Center, and the Givat-Washington College, both in Israel, were selected for this study. The selected CPET files included patients with confirmed primary CHF (n = 73), COPD (n = 75), and healthy subjects (n = 86). Of the 234 CPETs, 150 (50 in each group) tests were used for the support vector machine (SVM) learning stage, and the remaining 84 tests were used for the model validation. The performance of the SVM interpretive module was assessed by comparing its interpretation output with the conventional clinical diagnosis using distribution analysis. RESULTS: The disease classification results show that the overall predictive power of the proposed interpretive model ranged from 96% to 100%, indicating very high predictive power. Furthermore, the sensitivity, specificity, and overall precision of the proposed interpretive module were 99%, 99%, and 99%, respectively. CONCLUSIONS: The proposed new computer-aided CPET interpretive module was found to be highly sensitive and specific in classifying patients with CHF or COPD, or healthy. Comparable modules may well be applied to additional and larger populations (pathologies and exercise limitations), thereby making this tool powerful and clinically applicable.

COVID-19 in CXR: From Detection and Severity Scoring to Patient Disease Monitoring.

This work estimates the severity of pneumonia in COVID-19 patients and reports the findings of a longitudinal study of disease progression. It presents a deep learning model for simultaneous detection and localization of pneumonia in chest Xray (CXR) images, which is shown to generalize to COVID-19 pneumonia. The localization maps are utilized to calculate a "Pneumonia Ratio" which indicates disease severity. The assessment of disease severity serves to build a temporal disease extent profile for hospitalized patients. To validate the model's applicability to the patient monitoring task, we developed a validation strategy which involves a synthesis of Digital Reconstructed Radiographs (DRRs - synthetic Xray) from serial CT scans; we then compared the disease progression profiles that were generated from the DRRs to those that were generated from CT volumes.

A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises.

Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.