Deep Generative Adversarial Reinforcement Learning for Semi-Supervised Segmentation of Low-Contrast and Small Objects in Medical Images.

Deep reinforcement learning (DRL) has demonstrated impressive performance in medical image segmentation, particularly for low-contrast and small medical objects. However, current DRL-based segmentation methods face limitations due to the optimization of error propagation in two separate stages and the need for a significant amount of labeled data. In this paper, we propose a novel deep generative adversarial reinforcement learning (DGARL) approach that, for the first time, enables end-to-end semi-supervised medical image segmentation in the DRL domain. DGARL ingeniously establishes a pipeline that integrates DRL and generative adversarial networks (GANs) to optimize both detection and segmentation tasks holistically while mutually enhancing each other. Specifically, DGARL introduces two innovative components to facilitate this integration in semi-supervised settings. First, a task-joint GAN with two discriminators links the detection results to the GAN's segmentation performance evaluation, allowing simultaneous joint evaluation and feedback. This ensures that DRL and GAN can be directly optimized based on each other's results. Second, a bidirectional exploration DRL integrates backward exploration and forward exploration to ensure the DRL agent explores the correct direction when forward exploration is disabled due to lack of explicit rewards. This mitigates the issue of unlabeled data being unable to provide rewards and rendering DRL unexplorable. Comprehensive experiments on three generalization datasets, comprising a total of 640 patients, demonstrate that our novel DGARL achieves 85.02% Dice and improves at least 1.91% for brain tumors, achieves 73.18% Dice and improves at least 4.28% for liver tumors, and achieves 70.85% Dice and improves at least 2.73% for pancreas compared to the ten most recent advanced methods, our results attest to the superiority of DGARL. Code is available at GitHub.

Spatiotemporal knowledge teacher-student reinforcement learning to detect liver tumors without contrast agents.

Detecting Liver tumors without contrast agents (CAs) has shown great potential to advance liver cancer screening. It enables the provision of a reliable liver tumor-detecting result from non-enhanced MR images comparable to the radiologists' results from CA-enhanced MR images, thus eliminating the high risk of CAs, preventing an experience gap between radiologists and simplifying clinical workflows. In this paper, we proposed a novel spatiotemporal knowledge teacher-student reinforcement learning (SKT-RL) as a safe, speedy, and inexpensive contrast-free technology for liver tumor detection. Our SKT-RL builds a teacher-student framework to realize the exploring of explicit liver tumor knowledge from a teacher network on clear contrast-enhanced images to guide a student network to detect tumors from non-enhanced images directly. Importantly, our STK-RL enables three novelties in aspects of construction, transferring, and optimization to tumor knowledge to improve the guide effect. (1) A new spatiotemporal ternary knowledge set enables the construction of accurate knowledge that allows understanding of DRL's behavior (what to do) and reason (why to do it) behind reliable detection within each state and between their related historical states. (2) A novel pixel momentum transferring strategy enables detailed and controlled knowledge transfer ability. It transfers knowledge at a pixel level to enlarge the explorable space of transferring and control how much knowledge is transferred to prevent over-rely of the student to the teacher. (3) A phase-trend reward function designs different evaluations according to different detection phases to optimal for each phase in high-precision but also allows reward trend to constraint the evaluation to improve stability. Comprehensive experiments on a generalized liver tumor dataset with 375 patients (including hemangiomas, hepatocellular carcinoma, and normal controls) show that our novel SKT-RL attains a new state-of-the-art performance (improved precision by at least 4% when comparing the six recent advanced methods) in the task of liver tumor detection without CAs. The results proved that our SKT-DRL has greatly promoted the development and deployment of contrast-free liver tumor technology.

BMAnet: Boundary Mining With Adversarial Learning for Semi-Supervised 2D Myocardial Infarction Segmentation.

Automatic segmentation of myocardial infarction (MI) regions in late gadolinium-enhanced cardiac magnetic resonance images is an essential step in the computed diagnosis of myocardial infarction. Most of the current myocardial infarction region segmentation methods are based on fully supervised deep learning. However, cardiologists' annotation of myocardial infarction regions in cardiac magnetic resonance images during the diagnosis process is time-consuming and expensive. This paper proposes a semi-supervised myocardial infarction segmentation. It consists of two models: 1) a boundary mining model and 2) an adversarial learning model. The boundary mining model can solve the boundary ambiguity problem by enlarging the gap between the foreground and background features, thus segmenting the myocardial infarction region accurately. The adversarial learning model can make the boundary mining model learn from additional unlabeled data by evaluating the segmentation performance and providing pseudo supervision, which significantly increases the robustness of the boundary mining model. We conduct extensive experiments on an in-house myocardial magnetic resonance dataset. The experimental results on six evaluation metrics demonstrate that our method achieves excellent results in myocardial infarction segmentation and outperforms the state-of-the-art semi-supervised methods.

Synthesis of gadolinium-enhanced liver tumors on nonenhanced liver MR images using pixel-level graph reinforcement learning.

If successful, synthesis of gadolinium (Gd)-enhanced liver tumors on nonenhanced liver MR images will be critical for liver tumor diagnosis and treatment. This synthesis will offer a safe, efficient, and low-cost clinical alternative to eliminate the use of contrast agents in the current clinical workflow and significantly benefit global healthcare systems. In this study, we propose a novel pixel-level graph reinforcement learning method (Pix-GRL). This method directly takes regular nonenhanced liver images as input and outputs AI-enhanced liver tumor images, thereby making them comparable to traditional Gd-enhanced liver tumor images. In Pix-GRL, each pixel has a pixel-level agent, and the agent explores the pixels features and outputs a pixel-level action to iteratively change the pixel value, ultimately generating AI-enhanced liver tumor images. Most importantly, Pix-GRL creatively embeds a graph convolution to represent all the pixel-level agents. A graph convolution is deployed to the agent for feature exploration to improve the effectiveness through the aggregation of long-range contextual features, as well as outputting the action to enhance the efficiency through shared parameter training between agents. Moreover, in our Pix-GRL method, a novel reward is used to measure pixel-level action to significantly improve the performance by considering the improvement in each action in each pixel with its own future state, as well as those of neighboring pixels. Pix-GRL significantly upgrades the existing medical DRL methods from a single agent to multiple pixel-level agents, becoming the first DRL method for medical image synthesis. Comprehensive experiments on three types of liver tumor datasets (benign, cancerous, and healthy controls) with 325 patients (24,375 images) show that our novel Pix-GRL method outperforms existing medical image synthesis learning methods. It achieved an SSIM of 0.85 ± 0.06 and a Pearson correlation coefficient of 0.92 in terms of the tumor size. These results prove that the potential exists to develop a successful clinical alternative to Gd-enhanced liver MR imaging.

Contrast agent-free synthesis and segmentation of ischemic heart disease images using progressive sequential causal GANs.

The elimination of gadolinium contrast agent (CA) injections and manual segmentation are crucial for ischemic heart disease (IHD) diagnosis and treatment. In the clinic, CA-based late gadolinium enhancement (LGE) imaging and manual segmentation remain subject to concerns about potential toxicity, interobserver variability, and ineffectiveness. In this study, progressive sequential causal GANs (PSCGAN) are proposed. This is the first one-stop CA-free IHD technology that can simultaneously synthesize an LGE-equivalent image and segment diagnosis-related tissues (i.e., scars, healthy myocardium, blood pools, and other pixels) from cine MR images. To this end, the PSCGAN offer three unique properties: 1) a progressive framework that cascades three phases (i.e., priori generation, conditional synthesis, and enhanced segmentation) for divide-and-conquer training synthesis and segmentation of images. Importantly, this framework leverages the output of the previous phase as a priori condition to input the next phase and guides its training for enhancing performance, 2) a sequential causal learning network (SCLN) that creates a multi-scale, two-stream pathway and a multi-attention weighing unit to extract spatial and temporal dependencies from cine MR images and effectively select task-specific dependence. It also integrates the GAN architecture to leverage adversarial training to further facilitate the learning of interest dependencies of the latent space of cine MR images in all phases; and 3) two specifically designed self-learning loss terms: a synthetic regularization loss term leverages the spare regularization to avoid noise during synthesis, and a segmentation auxiliary loss term leverages the number of pixels for each tissue to compensate for discrimination during segmentation. Thus, the PSCGAN gain unprecedented performance while stably training in both synthesis and segmentation. By training and testing a total of 280 clinical subjects, our PSCGAN yield a synthetic normalization root-mean-squared-error of 0.14 and an overall segmentation accuracy of 97.17%. It also produces a 0.96 correlation coefficient for the scar ratio in a real diagnostic metric evaluation. These results proved that our method is able to offer significant assistance in the standardized assessment of cardiac disease.

DeepCQ: Deep multi-task conditional quantification network for estimation of left ventricle parameters.

BACKGROUND AND OBJECTIVE: Automatic cardiac left ventricle (LV) quantification plays an important role in assessing cardiac function. Although many advanced methods have been put forward to quantify related LV parameters, automatic cardiac LV quantification is still a challenge task due to the anatomy construction complexity of heart. METHODS: In this work, we propose a novel deep multi-task conditional quantification learning model (DeepCQ) which contains Segmentation module, Quantification encoder, and Dynamic analysis module. Besides, we also use task uncertainty loss function to update the parameters of the network in training. RESULTS: The proposed framework is validated on the dataset from Left Ventricle Full Quantification Challenge MICCAI 2018 (https://lvquan18.github.io/). The experimental results show that DeepCQ outperforms the other advanced methods. CONCLUSIONS: It illustrates that our method has a great potential in comprehensive cardiac function assessment and could play an auxiliary role in clinicians' diagnosis.

Segmentation and quantification of infarction without contrast agents via spatiotemporal generative adversarial learning.

Accurate and simultaneous segmentation and full quantification (all indices are required in a clinical assessment) of the myocardial infarction (MI) area are crucial for early diagnosis and surgical planning. Current clinical methods remain subject to potential high-risk, nonreproducibility and time-consumption issues. In this study, a deep spatiotemporal adversarial network (DSTGAN) is proposed as a contrast-free, stable and automatic clinical tool to simultaneously segment and quantify MIs directly from the cine MR image. The DSTGAN is implemented using a conditional generative model, which conditions the distributions of the objective cine MR image to directly optimize the generalized error of the mapping between the input and the output. The method consists of the following: (1) A multi-level and multi-scale spatiotemporal variation encoder learns a coarse to fine hierarchical feature to effectively encode the MI-specific morphological and kinematic abnormality structures, which vary for different spatial locations and time periods. (2) The top-down and cross-task generators learn the shared representations between segmentation and quantification to use the commonalities and differences between the two related tasks and enhance the generator preference. (3) Three inter-/intra-tasks to label the relatedness discriminators are iteratively imposed on the encoder and generator to detect and correct the inconsistencies in the label relatedness between and within tasks via adversarial learning. Our proposed method yields a pixel classification accuracy of 96.98%, and the mean absolute error of the MI centroid is 0.96 mm from 165 clinical subjects. These results indicate the potential of our proposed method in aiding standardized MI assessments.

Deep Learning for Diagnosis of Chronic Myocardial Infarction on Nonenhanced Cardiac Cine MRI.

Background Renal impairment is common in patients with coronary artery disease and, if severe, late gadolinium enhancement (LGE) imaging for myocardial infarction (MI) evaluation cannot be performed. Purpose To develop a fully automatic framework for chronic MI delineation via deep learning on non-contrast material-enhanced cardiac cine MRI. Materials and Methods In this retrospective single-center study, a deep learning model was developed to extract motion features from the left ventricle and delineate MI regions on nonenhanced cardiac cine MRI collected between October 2015 and March 2017. Patients with chronic MI, as well as healthy control patients, had both nonenhanced cardiac cine (25 phases per cardiac cycle) and LGE MRI examinations. Eighty percent of MRI examinations were used for the training data set and 20% for the independent testing data set. Chronic MI regions on LGE MRI were defined as ground truth. Diagnostic performance was assessed by analysis of the area under the receiver operating characteristic curve (AUC). MI area and MI area percentage from nonenhanced cardiac cine and LGE MRI were compared by using the Pearson correlation, paired t test, and Bland-Altman analysis. Results Study participants included 212 patients with chronic MI (men, 171; age, 57.2 years ± 12.5) and 87 healthy control patients (men, 42; age, 43.3 years ± 15.5). Using the full cardiac cine MRI, the per-segment sensitivity and specificity for detecting chronic MI in the independent test set was 89.8% and 99.1%, respectively, with an AUC of 0.94. There were no differences between nonenhanced cardiac cine and LGE MRI analyses in number of MI segments (114 vs 127, respectively; P = .38), per-patient MI area (6.2 cm2 ± 2.8 vs 5.5 cm2 ± 2.3, respectively; P = .27; correlation coefficient, r = 0.88), and MI area percentage (21.5% ± 17.3 vs 18.5% ± 15.4; P = .17; correlation coefficient, r = 0.89). Conclusion The proposed deep learning framework on nonenhanced cardiac cine MRI enables the confirmation (presence), detection (position), and delineation (transmurality and size) of chronic myocardial infarction. However, future larger-scale multicenter studies are required for a full validation. Published under a CC BY 4.0 license. Online supplemental material is available for this article. See also the editorial by Leiner in this issue.

Direct delineation of myocardial infarction without contrast agents using a joint motion feature learning architecture.

Changes in mechanical properties of myocardium caused by a infarction can lead to kinematic abnormalities. This phenomenon has inspired us to develop this work for delineation of myocardial infarction area directly from non-contrast agents cardiac MR imaging sequences. The main contribution of this work is to develop a new joint motion feature learning architecture to efficiently establish direct correspondences between motion features and tissue properties. This architecture consists of three seamless connected function layers: the heart localization layers can automatically crop the region of interest (ROI) sequences involving the left ventricle from the cardiac MR imaging sequences; the motion feature extraction layers, using long short-term memory-recurrent neural networks, a) builds patch-based motion features through local intensity changes between fixed-size patch sequences (cropped from image sequences), and b) uses optical flow techniques to build image-based features through global intensity changes between adjacent images to describe the motion of each pixel; the fully connected discriminative layers can combine two types of motion features together in each pixel and then build the correspondences between motion features and tissue identities (that is, infarct or not) in each pixel. We validated the performance of our framework in 165 cine cardiac MR imaging datasets by comparing to the ground truths manually segmented from delayed Gadolinium-enhanced MR cardiac images by two radiologists with more than 10 years of experience. Our experimental results show that our proposed method has a high and stable accuracy (pixel-level: 95.03%) and consistency (Kappa statistic: 0.91; Dice: 89.87%; RMSE: 0.72  mm; Hausdorff distance: 5.91  mm) compared to manual delineation results. Overall, the advantage of our framework is that it can determine the tissue identity in each pixel from its motion pattern captured by normal cine cardiac MR images, which makes it an attractive tool for the clinical diagnosis of infarction.

Robust Segmentation of Intima-Media Borders With Different Morphologies and Dynamics During the Cardiac Cycle.

Segmentation of carotid intima-media (IM) borders from ultrasound sequences is challenging because of unknown image noise and varying IM border morphologies and/or dynamics. In this paper, we have developed a state-space framework to sequentially segment the carotid IM borders in each image throughout the cardiac cycle. In this framework, an ${\mathrm{H}}_{\mathrm{\infty }}$ filter is used to solve the state-space equations, and a grayscale-derivative constraint snake is used to provide accurate measurements for the ${\mathrm{H}}_{\mathrm{\infty }}$ filter. We have evaluated the performance of our approach by comparing our segmentation results to the manually traced contours of ultrasound image sequences of three synthetic models and 156 real subjects from four medical centers. The results show that our method has a small segmentation error (lumen intima, LI: 53 $\pm\, 67\;{\mathrm{\mu }}$m; media-adventitia, MA: 57 $\pm\, 63\;{\mathrm{\mu }}$m) for synthetic and real sequences of different image characteristics, and also agrees well with the manual segmentation (LI: bias = 1.44 ${\mathrm{\mu }}$m; MA: bias = $-$3.38 ${\mathrm{\mu }}$m). Our approach can robustly segment the carotid ultrasound sequences with various IM border morphologies, dynamics, and unknown image noise. These results indicate the potential of our framework to segment IM borders for clinical diagnosis.

Complexity Analysis of Electroencephalogram Dynamics in Patients with Parkinson's Disease.

In this study, a new combination scheme has been proposed for detecting Parkinson's disease (PD) from electroencephalogram (EEG) signal recorded from normal subjects and PD patients. The scheme is based on discrete wavelet transform (DWT), sample entropy (SampEn), and the three-way decision model in analysis of EEG signal. The EEG signal is noisy and nonstationary, and, as a consequence, it becomes difficult to distinguish it visually. However, the scheme is a well-established methodology in analysis of EEG signal in three stages. In the first stage, the DWT was applied to acquire the split frequency information; here, we use three-level DWT to decompose EEG signal into approximation and detail coefficients; in this stage, we aim to remove the useless and noise information and acquire the effective information. In the second stage, as the SampEn has advantage in analyzing the EEG signal, we use the approximation coefficient to compute the SampEn values. Finally, we detect the PD patients using three-way decision based on optimal center constructive covering algorithm (O_CCA) with the accuracy about 92.86%. Without DWT as preprocessing step, the detection rate reduces to 88.10%. Overall, the combination scheme we proposed is suitable and efficient in analyzing the EEG signal with higher accuracy.

Beat-to-Beat Blood Pressure and Two-dimensional (axial and radial) Motion of the Carotid Artery Wall: Physiological Evaluation of Arterial Stiffness.

The physiological relationship between local arterial displacement and blood pressure (BP) plays an integral role in assess- ment of the mechanical properties of arteries. In this study, we used more advanced methods to obtain reliable continuous BP and the displacement of the common carotid artery (CCA) simultaneously. We propose a novel evaluation method for arterial stiffness that relies on determining the physiological relationship between the axial and radial displacements of the CCA wall and beat-to-beat BP. Patients (total of 138) were divided into groups according to the following three criteria: essential hyper- tension (EH) and normotension, male and female, elderly and younger. The Pearson correlation test and canonical correlation analysis showed that the CCA indices were significantly correlated with BP indices (r = 0:787; p < 0:05). The slope of the CCA displacement/pressure curve showed a progressive reduction with increasing age and EH disease occurrence (EH: 0.496 vs. normotension: 0.822; age <= 60:0.585 vs. age > 60:0.783). Our method provides an explicit reference value and relationship for the manner in which the CCA wall responds to changes in BP. Short-term and continuous BP were significantly correlated with CCA displacement and exhibited a close inverse relationship with each subject's BP and EH, age, and systolic blood pressure.