Multi-modal fusion and feature enhancement U-Net coupling with stem cell niches proximity estimation for voxel-wise GBM recurrence prediction.

Objective.We aim to develop a Multi-modal Fusion and Feature Enhancement U-Net (MFFE U-Net) coupling with stem cell niche proximity estimation to improve voxel-wise Glioblastoma (GBM) recurrence prediction.Approach.57 patients with pre- and post-surgery magnetic resonance (MR) scans were retrospectively solicited from 4 databases. Post-surgery MR scans included two months before the clinical diagnosis of recurrence and the day of the radiologicaly confirmed recurrence. The recurrences were manually annotated on the T1ce. The high-risk recurrence region was first determined. Then, a sparse multi-modal feature fusion U-Net was developed. The 50 patients from 3 databases were divided into 70% training, 10% validation, and 20% testing. 7 patients from the 4th institution were used as external testing with transfer learning. Model performance was evaluated by recall, precision, F1-score, and Hausdorff Distance at the 95% percentile (HD95). The proposed MFFE U-Net was compared to the support vector machine (SVM) model and two state-of-the-art neural networks. An ablation study was performed.Main results.The MFFE U-Net achieved a precision of 0.79 ± 0.08, a recall of 0.85 ± 0.11, and an F1-score of 0.82 ± 0.09. Statistically significant improvement was observed when comparing MFFE U-Net with proximity estimation couple SVM (SVMPE), mU-Net, and Deeplabv3. The HD95 was 2.75 ± 0.44 mm and 3.91 ± 0.83 mm for the 10 patients used in the model construction and 7 patients used for external testing, respectively. The ablation test showed that all five MR sequences contributed to the performance of the final model, with T1ce contributing the most. Convergence analysis, time efficiency analysis, and visualization of the intermediate results further discovered the characteristics of the proposed method.Significance. We present an advanced MFFE learning framework, MFFE U-Net, for effective voxel-wise GBM recurrence prediction. MFFE U-Net performs significantly better than the state-of-the-art networks and can potentially guide early RT intervention of the disease recurrence.

Contrast-Enhanced Liver Magnetic Resonance Image Synthesis Using Gradient Regularized Multi-Modal Multi-Discrimination Sparse Attention Fusion GAN.

PURPOSES: To provide abdominal contrast-enhanced MR image synthesis, we developed an gradient regularized multi-modal multi-discrimination sparse attention fusion generative adversarial network (GRMM-GAN) to avoid repeated contrast injections to patients and facilitate adaptive monitoring. METHODS: With IRB approval, 165 abdominal MR studies from 61 liver cancer patients were retrospectively solicited from our institutional database. Each study included T2, T1 pre-contrast (T1pre), and T1 contrast-enhanced (T1ce) images. The GRMM-GAN synthesis pipeline consists of a sparse attention fusion network, an image gradient regularizer (GR), and a generative adversarial network with multi-discrimination. The studies were randomly divided into 115 for training, 20 for validation, and 30 for testing. The two pre-contrast MR modalities, T2 and T1pre images, were adopted as inputs in the training phase. The T1ce image at the portal venous phase was used as an output. The synthesized T1ce images were compared with the ground truth T1ce images. The evaluation metrics include peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and mean squared error (MSE). A Turing test and experts' contours evaluated the image synthesis quality. RESULTS: The proposed GRMM-GAN model achieved a PSNR of 28.56, an SSIM of 0.869, and an MSE of 83.27. The proposed model showed statistically significant improvements in all metrics tested with p-values < 0.05 over the state-of-the-art model comparisons. The average Turing test score was 52.33%, which is close to random guessing, supporting the model's effectiveness for clinical application. In the tumor-specific region analysis, the average tumor contrast-to-noise ratio (CNR) of the synthesized MR images was not statistically significant from the real MR images. The average DICE from real vs. synthetic images was 0.90 compared to the inter-operator DICE of 0.91. CONCLUSION: We demonstrated the function of a novel multi-modal MR image synthesis neural network GRMM-GAN for T1ce MR synthesis based on pre-contrast T1 and T2 MR images. GRMM-GAN shows promise for avoiding repeated contrast injections during radiation therapy treatment.

Convolutional Autoencoding and Gaussian Mixture Clustering for Unsupervised Beat-to-Beat Heart Rate Estimation of Electrocardiograms from Wearable Sensors.

Heart rate is one of the most important diagnostic bases for cardiovascular disease. This paper introduces a deep autoencoding strategy into feature extraction of electrocardiogram (ECG) signals, and proposes a beat-to-beat heart rate estimation method based on convolution autoencoding and Gaussian mixture clustering. The high-level heartbeat features were first extracted in an unsupervised manner by training the convolutional autoencoder network, and then the adaptive Gaussian mixture clustering was applied to detect the heartbeat locations from the extracted features, and calculated the beat-to-beat heart rate. Compared with the existing heartbeat classification/detection methods, the proposed unsupervised feature learning and heartbeat clustering method does not rely on accurate labeling of each heartbeat location, which could save a lot of time and effort in human annotations. Experimental results demonstrate that the proposed method maintains better accuracy and generalization ability compared with the existing ECG heart rate estimation methods and could be a robust long-time heart rate monitoring solution for wearable ECG devices.

L1 Sparsity-Regularized Attention Multiple-Instance Network for Hyperspectral Target Detection.

Attention-based deep multiple-instance learning (MIL) has been applied to many machine-learning tasks with imprecise training labels. It is also appealing in hyperspectral target detection, which only requires the label of an area containing some targets, relaxing the effort of labeling the individual pixel in the scene. This article proposes an L1 sparsity-regularized attention multiple-instance neural network (L1-attention MINN) for hyperspectral target detection with imprecise labels that enforces the discrimination of false-positive instances from positively labeled bags. The sparsity constraint applied to the attention estimated for the positive training bags strictly complies with the definition of MIL and maintains better discriminative ability. The proposed algorithm has been evaluated on both simulated and real-field hyperspectral (subpixel) target detection tasks, where advanced performance has been achieved over the state-of-the-art comparisons, showing the effectiveness of the proposed method for target detection from imprecisely labeled hyperspectral data.

Non-Invasive Heart Rate Estimation From Ballistocardiograms Using Bidirectional LSTM Regression.

Non-invasive heart rate estimation is of great importance in daily monitoring of cardiovascular diseases. In this paper, a bidirectional long short term memory (bi-LSTM) regression network is developed for non-invasive heart rate estimation from the ballistocardiograms (BCG) signals. The proposed deep regression model provides an effective solution to the existing challenges in BCG heart rate estimation, such as the mismatch between the BCG signals and ground-truth reference, multi-sensor fusion and effective time series feature learning. Allowing label uncertainty in the estimation can reduce the manual cost of data annotation while further improving the heart rate estimation performance. Compared with the state-of-the-art BCG heart rate estimation methods, the strong fitting and generalization ability of the proposed deep regression model maintains better robustness to noise (e.g., sensor noise) and perturbations (e.g., body movements) in the BCG signals and provides a more reliable solution for long term heart rate monitoring.

Heartbeat Detection and Rate Estimation from Ballistocardiograms using the Gated Recurrent Unit Network.

Inspired by the application of recurrent neural networks (RNNs) to image recognition, in this paper, we propose a heartbeat detection framework based on the Gated Recurrent Unit (GRU) network. In this contribution, the heartbeat detection task from ballistocardiogram (BCG) signals was modeled as a classification problem where the segments of BCG signals were formulated as images fed into the GRU network for feature extraction. The proposed framework has advantages in fusion of multi-channel BCG signals and effective extraction of the temporal and waveform characteristics of the heartbeat signal, thereby enhancing heart rate estimation accuracy. In laboratory collected BCG data, the proposed method achieved the best heart rate estimation results compared to previous algorithms.

Gradient regularized convolutional neural networks for low-dose CT image enhancement.

The potential risks of x-ray to patients have transferred the public's attention from normal dose CT (NDCT) to low-dose CT (LDCT). However, simply lowering the radiation dose of the CT system will significantly degrade the quality of CT images such as noise and artifacts, which compromises the diagnostic performance. Hence, various methods have been proposed to solve this problem over the past decades. Although these methods have achieved impressive results, they also suffer from a drawback of smoothing image details after denoising, which makes it difficult for clinical diagnosis and treatment. To address this issue, this paper introduces a novel gradient regularization method for LDCT enhancement. Rather than common methods which only consider the pixel-wise gray value loss in the reconstruction procedure, we also take the image gradient loss into consideration to preserve image details. By combining the gradient regularization method and the convolutional neural network (CNN) framework, a gradient regularized convolutional neural network (GRCNN) is proposed to enhance LDCT images which has achieved promising performance in our experiments both visually and quantitatively.

Multiple Instance Dictionary Learning for Beat-to-Beat Heart Rate Monitoring From Ballistocardiograms.

A multiple instance dictionary learning approach, dictionary learning using functions of multiple instances (DL-FUMI), is used to perform beat-to-beat heart rate estimation and to characterize heartbeat signatures from ballistocardiogram (BCG) signals collected with a hydraulic bed sensor. DL-FUMI estimates a "heartbeat concept" that represents an individual's personal ballistocardiogram heartbeat pattern. DL-FUMI formulates heartbeat detection and heartbeat characterization as a multiple instance learning problem to address the uncertainty inherent in aligning BCG signals with ground truth during training. Experimental results show that the estimated heartbeat concept obtained by DL-FUMI is an effective heartbeat prototype and achieves superior performance over comparison algorithms.

Discriminative Multiple Instance Hyperspectral Target Characterization.

In this paper, two methods for discriminative multiple instance target characterization, MI-SMF and MI-ACE, are presented. MI-SMF and MI-ACE estimate a discriminative target signature from imprecisely-labeled and mixed training data. In many applications, such as sub-pixel target detection in remotely-sensed hyperspectral imagery, accurate pixel-level labels on training data is often unavailable and infeasible to obtain. Furthermore, since sub-pixel targets are smaller in size than the resolution of a single pixel, training data is comprised only of mixed data points (in which target training points are mixtures of responses from both target and non-target classes). Results show improved, consistent performance over existing multiple instance concept learning methods on several hyperspectral sub-pixel target detection problems.