AI-Based Automated Lipomatous Tumor Segmentation in MR Images: Ensemble Solution to Heterogeneous Data.

Deep learning (DL) has been proposed to automate image segmentation and provide accuracy, consistency, and efficiency. Accurate segmentation of lipomatous tumors (LTs) is critical for correct tumor radiomics analysis and localization. The major challenge of this task is data heterogeneity, including tumor morphological characteristics and multicenter scanning protocols. To mitigate the issue, we aimed to develop a DL-based Super Learner (SL) ensemble framework with different data correction and normalization methods. Pathologically proven LTs on pre-operative T1-weighted/proton-density MR images of 185 patients were manually segmented. The LTs were categorized by tumor locations as distal upper limb (DUL), distal lower limb (DLL), proximal upper limb (PUL), proximal lower limb (PLL), or Trunk (T) and grouped by 80%/9%/11% for training, validation and testing. Six configurations of correction/normalization were applied to data for fivefold-cross-validation trainings, resulting in 30 base learners (BLs). A SL was obtained from the BLs by optimizing SL weights. The performance was evaluated by dice-similarity-coefficient (DSC), sensitivity, specificity, and Hausdorff distance (HD95). For predictions of the BLs, the average DSC, sensitivity, and specificity from the testing data were 0.72 [Formula: see text] 0.16, 0.73 [Formula: see text] 0.168, and 0.99 [Formula: see text] 0.012, respectively, while for SL predictions were 0.80 [Formula: see text] 0.184, 0.78 [Formula: see text] 0.193, and 1.00 [Formula: see text] 0.010. The average HD95 of the BLs were 11.5 (DUL), 23.2 (DLL), 25.9 (PUL), 32.1 (PLL), and 47.9 (T) mm, whereas of SL were 1.7, 8.4, 15.9, 2.2, and 36.6 mm, respectively. The proposed method could improve the segmentation accuracy and mitigate the performance instability and data heterogeneity aiding the differential diagnosis of LTs in real clinical situations.

Deep learning-based in vivo dose verification from proton-induced secondary-electron-bremsstrahlung images with various count level.

PURPOSE: Proton-induced secondary-electron-bremsstrahlung (SEB) imaging is a promising method for estimating the ranges of particle beam. However, SEB images do not directly represent dose distributions of particle beams. In addition, the ranges estimated from measured images were deviated because of limited spatial resolutions of the developed x-ray camera as well as statistical noise in the images. To solve these problems, we proposed a method for predicting high-resolution dose images from SEB images with various count level using a deep learning (DL) approach for range and width verification. METHODS: In this study, we adopted the double U-Net model, which is a previously proposed deep convolutional network model. The first U-Net model in the double U-Net model was used to denoise the SEB images with various count level. The first U-Net model for denoising was trained on 8000 pairs of SEB images with various count level and noise-free images which were created by a sophisticated in-house developed model function. The second U-Net model for dose prediction was trained using 8000 pairs of denoised SEB images from the first U-Net model and high-resolution dose images generated by Monte Carlo simulation. RESULTS: For both simulation and measurement data, the trained DL model could successfully predict high-resolution dose images which showed a clear Bragg peak and no statistical noise. The difference of the range and width was less than 2.1 mm, even from the SEB images measured with a decrease in the number of irradiated protons to less than 11% of 3.2 × 1011 protons. CONCLUSIONS: High-resolution dose images from measured and simulated SEB images were successfully predicted by using the trained DL model for protons. Our proposed DL model was feasible to predict dose images accurately even with smaller number of irradiated protons.

NEMA NU 4-2008 performance evaluation and MR compatibility tests of an APD-based small animal PET-insert for simultaneous PET/MR imaging.

An avalanche photodiode (APD)-based small animal positron emission tomography (PET)-insert was fully evaluated for its PET performance, as well as potential influences on magnetic resonance imaging (MRI) performance. This PET-insert has an extended axial field of view (FOV) compared with the previous design to increase system sensitivity, as well as an updated cooling and temperature regulation to enable stable and reproducible PET acquisitions. The PET performance was evaluated according to the National Electrical Manufacturers Association NU4-2008 protocol. The energy and timing resolution's full width at half maximum were 16.1% and 4.7 ns, respectively. The reconstructed radial spatial resolution of the PET-insert was 1.8 mm full width at half maximum at the center FOV using filtered back projection for reconstruction and sensitivity was 3.68%. The peak noise equivalent count rates were 70 kcps for a rat-like and 350 kcps for a mouse-like phantom, respectively. Image quality phantom values and contrast recovery were comparable to state-of-the art PET-inserts and standalone systems. Regarding MR compatibility, changes in the mean signal-to-noise ratio for turbo spin echo and echo-planar imaging sequences were below 8.6%, for gradient echo sequences below 1%. Degradation of the mean homogeneity was below 2.3% for all tested sequences. The influence of the PET-insert on theB0maps was negligible and no influence on functional MRI sequences was detected. A mouse and rat imaging study demonstrated the feasibility ofin vivosimultaneous PET/MRI.

Performance evaluation of dual-ended readout PET detectors based on BGO arrays with different reflector arrangements.

OBJECTIVE: Dual-ended readout depth-encoding detectors based on bismuth germanate (BGO) scintillation crystal arrays are good candidates for high-sensitivity small animal positron emission tomography used for very-low-dose imaging. In this paper, the performance of three dual-ended readout detectors based on 15 × 15 BGO arrays with three different reflector arrangements and 8 × 8 silicon photomultiplier arrays were evaluated and compared. APPROACH: The three BGO arrays, denoted wo-ILG (without internal light guide), wp-ILG (with partial internal light guide), and wf-ILG (with full internal light guide), share a pitch size of 1.6 mm and thickness of 20 mm. Toray E60 with a thickness of 50µm was used as inter-crystal reflector. All reflector lengths in the wo-ILG and wf-ILG BGO arrays were 20 and 18 mm, respectively; the reflectors in the wp-ILG BGO array were 18 mm at the central region of the array and 20 mm at the edge. By using 18 mm reflectors, part of the crystals in the wp-ILG and wf-ILG BGO arrays worked as internal light guides. MAIN RESULTS: The results showed that the detector based on the wo-ILG BGO array provided the best flood histogram. The energy, timing and DOI resolutions of the three detectors were similar. The energy resolutions full width at half maximum (FWHM value) based on the wo-ILG, wp-ILG and wf-ILG BGO arrays were 27.2 ± 3.9%, 28.7 ± 4.6%, and 29.5 ± 4.7%, respectively. The timing resolutions (FWHM value) were 4.7 ± 0.5 ns, 4.9 ± 0.5 ns, and 5.0 ± 0.6 ns, respectively. The DOI resolution (FWHM value) were 3.0 ± 0.2 mm, 2.9 ± 0.2 mm, and 3.0 ± 0.2 mm, respectively. Over all, the wo-ILG detector provided the best performance.

Co-localization of fluorescent signals using deep learning with Manders overlapping coefficient.

Object-based co-localization of fluorescent signals allows the assessment of interactions between two (or more) biological entities using spatial information. It relies on object identification with high accuracy to separate fluorescent signals from the background. Object detectors using convolutional neural networks (CNN) with annotated training samples could facilitate the process by detecting and counting fluorescent-labeled cells from fluorescence photomicrographs. However, datasets containing segmented annotations of colocalized cells are generally not available, and creating a new dataset with delineated masks is label-intensive. Also, the co-localization coefficient is often not used as a component during training with the CNN model. Yet, it may aid with localizing and detecting objects during training and testing. In this work, we propose to address these issues by using a quantification coefficient for co-localization called Manders overlapping coefficient (MOC)1 as a single-layer branch in a CNN. Fully convolutional one-state (FCOS)2 with a Resnet101 backbone served as the network to evaluate the effectiveness of the novel branch to assist with bounding box prediction. Training data were sourced from lab curated fluorescence images of neurons from the rat hippocampus, piriform cortex, somatosensory cortex, and amygdala. Results suggest that using modified FCOS with MOC outperformed the original FCOS model for accuracy in detecting fluorescence signals by 1.1% in mean average precision (mAP). The model could be downloaded from https://github.com/Alphafrey946/Colocalization-MOC.

Generation of Brain Dual-Energy CT from Single-Energy CT Using Deep Learning.

Deep learning (DL) has shown great potential in conversions between various imaging modalities. Similarly, DL can be applied to synthesize a high-kV computed tomography (CT) image from its corresponding low-kV CT image. This indicates the feasibility of obtaining dual-energy CT (DECT) images without purchasing a DECT scanner. In this study, we investigated whether a low-to-high kV mapping was better than a high-to-low kV mapping. We used a U-Net model to perform conversions between different kV CT images. Moreover, we proposed a double U-Net model to improve the quality of original single-energy CT images. Ninety-eight patients who underwent brain DECT scans were used to train, validate, and test the proposed DL-based model. The results showed that the low-to-high kV conversion was better than the high-to-low kV conversion. In addition, the DL-based DECT images had better signal-to-noise ratios (SNRs) than the true (original) DECT images, but at the expense of a slight loss in spatial resolution. The mean CT number differences between the true and DL-based DECT images were within [Formula: see text] 1 HU. No statistically significant difference in CT number measurements was found between the true and DL-based DECT images (p > 0.05). The DL-based DECT images with improved SNR could produce low-noise virtual monoenergetic images. Our preliminary results indicate that DL has the potential to generate brain DECT images using single-energy brain CT images.

Dose image prediction for range and width verifications from carbon ion-induced secondary electron bremsstrahlung x-rays using deep learning workflow.

PURPOSE: Imaging of the secondary electron bremsstrahlung (SEB) x rays emitted during particle-ion irradiation is a promising method for beam range estimation. However, the SEB x-ray images are not directly correlated to the dose images. In addition, limited spatial resolution of the x-ray camera and low-count situation may impede correctly estimating the beam range and width in SEB x-ray images. To overcome these limitations of the SEB x-ray images measured by the x-ray camera, a deep learning (DL) approach was proposed in this work to predict the dose images for estimating the range and width of the carbon ion beam on the measured SEB x-ray images. METHODS: To prepare enough data for the DL training efficiently, 10,000 simulated SEB x-ray and dose image pairs were generated by our in-house developed model function for different carbon ion beam energies and doses. The proposed DL neural network consists of two U-nets for SEB x ray to dose image conversion and super resolution. After the network being trained with these simulated x-ray and dose image pairs, the dose images were predicted from simulated and measured SEB x-ray testing images for performance evaluation. RESULTS: For the 500 simulated testing images, the average mean squared error (MSE) was 2.5 × 10-5 and average structural similarity index (SSIM) was 0.997 while the error of both beam range and width was within 1 mm FWHM. For the three measured SEB x-ray images, the MSE was no worse than 5.5 × 10-3 and SSIM was no worse than 0.980 while the error of the beam range and width was 2 mm and 5 mm FWHM, respectively. CONCLUSIONS: We have demonstrated the advantages of predicting dose images from not only simulated data but also measured data using our deep learning approach.

A deep learning approach for converting prompt gamma images to proton dose distributions: A Monte Carlo simulation study.

PURPOSE: In proton therapy, imaging prompt gamma (PG) rays has the potential to verify proton dose (PD) distribution. Despite the fact that there is a strong correlation between the gamma-ray emission and PD, they are still different in terms of the distribution and the Bragg peak (BP) position. In this work, we investigated the feasibility of using a deep learning approach to convert PG images to PD distributions. METHODS: We designed the Monte Carlo simulations using 20 digital brain phantoms irradiated with a 100-MeV proton pencil beam. Each phantom was used to simulate 200 pairs of PG images and PD distributions. A convolutional neural network based on the U-net architecture was trained to predict PD distributions from PG images. RESULTS: Our simulation results show that the pseudo PD distributions derived from the corresponding PG images agree well with the simulated ground truths. The mean of the BP position errors from each phantom was less than 0.4 mm. We also found that 2000 pairs of PG images and dose distributions would be sufficient to train the U-net. Moreover, the trained network could be deployed on the unseen data (i.e. different beam sizes, proton energies and real patient CT data). CONCLUSIONS: Our simulation study has shown the feasibility of predicting PD distributions from PG images using a deep learning approach, but the reliable prediction of PD distributions requires high-quality PG images. Image-degrading factors such as low counts and limited spatial resolution need to be considered in order to obtain high-quality PG images.

Partial-ring PET image restoration using a deep learning based method.

PET scanners with partial-ring geometry have been proposed for various imaging purposes. The incomplete projection data obtained from this design cause undesirable artifacts in the reconstructed images. In this study, we investigated the performance of a deep learning (DL) based method for the recovery of partial-ring PET images. Twenty digital brain phantoms were used in the Monte Carlo simulation toolkit, SimSET, to simulate 15 min full-ring PET scans. Partial-ring PET data were generated from full-ring PET data by removing coincidence events that hit these specific detector blocks. A convolutional neural network based on the residual U-Net architecture was trained to predict full-ring data from partial-ring data in either the projection or image domain. The performance of the proposed DL-based method was evaluated by comparing with the PET images reconstructed using the full-ring projection data in terms of the mean squared error (MSE), structural similarity (SSIM) index and recovery coefficient (RC). The MSE results showed the superiority of the image-domain approach in reduction of 91.7% in contrast to 14.3% for the projection-domain approach. Therefore, the image-domain approach was used to study the influence of the number of detector block removal. The SSIM results were 0.998, 0.996 and 0.993 for 3, 5 and 7 detector block removals, respectively. The activity of gray and white matters could be fully recovered even with 7 detector block removal, while the RCs of two artificially inserted small lesions (3 pixels in diameter) in the testing data were 94%, 89% and 79% for 3, 5, and 7 detector block removals, respectively. Our simulation results suggest that DL has the potential to recover partial-ring PET images.

Design and evaluation of gapless curved scintillator arrays for simultaneous high-resolution and high-sensitivity brain PET.

Brain PET scanners that simultaneously provide high-resolution across the field-of-view and high-sensitivity can be constructed using detectors based on SiPM arrays coupled to both ends of scintillator arrays with finely segmented and long detector elements. To reduce the dead space between detector modules and hence improve the sensitivity of PET scanners, crystal arrays with curved surfaces are proposed. In this paper, the performance of a proof-of-concept detector module with nine detector submodules based on SiPMs coupled to both ends of a curved LYSO array with a pitch size of 1.0 × 1.0 mm2 at the front-end and a length of 30 mm was evaluated. A simple signal multiplexing method using the shared-photodetector readout method was evaluated to identify the crystals. The results showed that all the LYSO elements in the detector module of interest could be clearly resolved. The energy resolution, depth-of-interaction resolution, and timing resolution were 14.6% ± 3.6%, 2.77 ± 0.39 mm, and 1.15 ± 0.07 ns, respectively, obtained at a bias voltage of 28.0 V and a temperature of 16.8 °C ± 0.2 °C.

Higher SNR PET image prediction using a deep learning model and MRI image.

PET images often suffer poor signal-to-noise ratio (SNR). Our objective is to improve the SNR of PET images using a deep neural network (DNN) model and MRI images without requiring any higher SNR PET images in training. Our proposed DNN model consists of three modified U-Nets (3U-net). The PET training input data and targets were reconstructed using filtered-backprojection (FBP) and maximum likelihood expectation maximization (MLEM), respectively. FBP reconstruction was used because of its computational efficiency so that the trained network not only removes noise, but also accelerates image reconstruction. Digital brain phantoms downloaded from BrainWeb were used to evaluate the proposed method. Poisson noise was added into sinogram data to simulate a 6 min brain PET scan. Attenuation effect was included and corrected before the image reconstruction. Extra Poisson noise was introduced to the training inputs to improve the network denoising capability. Three independent experiments were conducted to examine the reproducibility. A lesion was inserted into testing data to evaluate the impact of mismatched MRI information using the contrast-to-noise ratio (CNR). The negative impact on noise reduction was also studied when miscoregistration between PET and MRI images occurs. Compared with 1U-net trained with only PET images, training with PET/MRI decreased the mean squared error (MSE) by 31.3% and 34.0% for 1U-net and 3U-net, respectively. The MSE reduction is equivalent to an increase in the count level by 2.5 folds and 2.9 folds for 1U-net and 3U-net, respectively. Compared with the MLEM images, the lesion CNR was improved 2.7 folds and 1.4 folds for 1U-net and 3U-net, respectively. The results show that the proposed method could improve the PET SNR without having higher SNR PET images.

Indirect methods for improving parameter estimation of PET kinetic models.

PURPOSE: Parametric images obtained from kinetic modeling of dynamic positron emission tomography (PET) data provide a new way of visualizing quantitative parameters of the tracer kinetics. However, due to the high noise level in pixel-wise image-driven time-activity curves, parametric images often suffer from poor quality and accuracy. In this study, we propose an indirect parameter estimation framework which aims to improve the quality and quantitative accuracy of parametric images. METHODS: Three different approaches related to noise reduction and advanced curve fitting algorithm are used in the proposed framework. First, dynamic PET images are denoised using a kernel-based denoising method and the highly constrained backprojection technique. Second, gradient-free curve fitting algorithms are exploited to improve the accuracy and precision of parameter estimates. Third, a kernel-based post-filtering method is applied to parametric images to further improve the quality of parametric images. Computer simulations were performed to evaluate the performance of the proposed framework. RESULTS AND CONCLUSIONS: The simulation results showed that when compared to the Gaussian filtering, the proposed denoising method could provide better PET image quality, and consequentially improve the quality and quantitative accuracy of parametric images. In addition, gradient-free optimization algorithms (i.e., pattern search) can result in better parametric images than the gradient-based curve fitting algorithm (i.e., trust-region-reflective). Finally, our results showed that the proposed kernel-based post-filtering method could further improve the precision of parameter estimates while maintaining the accuracy of parameter estimates.

PET Image Denoising Using a Deep Neural Network Through Fine Tuning.

Positron emission tomography (PET) is a functional imaging modality widely used in clinical diagnosis. In this work, we trained a deep convolutional neural network (CNN) to improve PET image quality. Perceptual loss based on features derived from a pre-trained VGG network, instead of the conventional mean squared error, was employed as the training loss function to preserve image details. As the number of real patient data set for training is limited, we propose to pre-train the network using simulation data and fine-tune the last few layers of the network using real data sets. Results from simulation, real brain and lung data sets show that the proposed method is more effective in removing noise than the traditional Gaussian filtering method.

Improving Edge Crystal Identification in Flood Histograms Using Triangular Shape Crystals.

This work presents a method to improve the separation of edge crystals in PET block detectors. As an alternative to square-shaped crystal arrays, we used an array of triangular-shaped crystals. This increases the distance between the crystal centres at the detector edges potentially improving the separation of edge crystals. To test this design, we have compared the flood histograms of two 4×4 scintillator arrays in both square and triangular configurations. The quality of the flood histogram was quantified using the fraction of events positioned in the correct crystal based on a 2D Gaussian fit of the segmented flood histograms. In the first study, the two crystal arrays were coupled with the SiPM directly using optical grease, and the flood histogram quality for the edge and corner crystals in the triangular-shaped array were much better than that for those crystals in the square-shaped array. The average light collection efficiency for the triangular-shaped array was 5.9% higher than that for the square-shaped array. The average energy resolution for the triangular and square shape array were 11.6% and 13.2% respectively. In the second study, two light guides with thickness 1 mm and 2 mm were used between the crystal arrays and the SiPM. The thicker lightguide degraded the light collection efficiency and energy resolution due to the light loss introduced by the light guide. However, in the 2-mm thick lightguide case, the flood histogram quality for the edge and corner crystals in the square-shaped array were improved due to better separation of those crystals in the flood histogram. Comparing the performance of the two crystal arrays with three different light guides, the triangular-shaped crystal array with no lightguide gave the best performance.

A general-threshold filtering method for improving intravoxel incoherent motion parameter estimates.

In this study, we present an image denoising method for diffusion-weighted magnetic resonance imaging (DW-MRI) data. Our aim is to improve the estimation of intravoxel incoherent motion (IVIM) parameters using denoised DW-MRI data. A general-threshold filtering (GTF) reconstruction via total variation minimization has been proposed to improve image quality in few-view computed tomography. Here, we applied the combination of GTF and total difference to image denoising. Voxel-wise IVIM analysis was performed using both real and simulated DW-MRI data. Using an institutional review board-approved protocol with written informed consent, DW-MRI imaging was performed at a 3 T hybrid PET/MR system in 10 patients with Hodgkin lymphoma lesions. A simulated phantom consisting of four organs (liver, pancreas, spleen and kidney) was used to generate noisy DW-MRI data according to the IVIM model at different noise levels. DW-MRI data were denoised before IVIM parameter estimation. The proposed image denoising method was compared with the image denoising method using joint rank and edge constraints (JREC). The results of simulated data show that at the lower signal-to-noise ratios the proposed image denoising method outperformed the JREC method in terms of the accuracy and precision of the IVIM parameter estimates. The experimental results also show that the proposed image denoising method could yield better parametric images than the JREC method in terms of noise reduction and edge preservation.

A low-count reconstruction algorithm for Compton-based prompt gamma imaging.

The Compton camera is an imaging device which has been proposed to detect prompt gammas (PGs) produced by proton-nuclear interactions within tissue during proton beam irradiation. Compton-based PG imaging has been developed to verify proton ranges because PG rays, particularly characteristic ones, have strong correlations with the distribution of the proton dose. However, accurate image reconstruction from characteristic PGs is challenging because the detector efficiency and resolution are generally low. Our previous study showed that point spread functions can be incorporated into the reconstruction process to improve image resolution. In this study, we proposed a low-count reconstruction algorithm to improve the image quality of a characteristic PG emission by pooling information from other characteristic PG emissions. PGs were simulated from a proton beam irradiated on a water phantom, and a two-stage Compton camera was used for PG detection. The results show that the image quality of the reconstructed characteristic PG emission is improved with our proposed method in contrast to the standard reconstruction method using events from only one characteristic PG emission. For the 4.44 MeV PG rays, both methods can be used to predict the positions of the peak and the distal falloff with a mean accuracy of 2 mm. Moreover, only the proposed method can improve the estimated positions of the peak and the distal falloff of 5.25 MeV PG rays, and a mean accuracy of 2 mm can be reached.

Simultaneous PET-MRI reveals brain function in activated and resting state on metabolic, hemodynamic and multiple temporal scales.

Combined positron emission tomography (PET) and magnetic resonance imaging (MRI) is a new tool to study functional processes in the brain. Here we study brain function in response to a barrel-field stimulus simultaneously using PET, which traces changes in glucose metabolism on a slow time scale, and functional MRI (fMRI), which assesses fast vascular and oxygenation changes during activation. We found spatial and quantitative discrepancies between the PET and the fMRI activation data. The functional connectivity of the rat brain was assessed by both modalities: the fMRI approach determined a total of nine known neural networks, whereas the PET method identified seven glucose metabolism-related networks. These results demonstrate the feasibility of combined PET-MRI for the simultaneous study of the brain at activation and rest, revealing comprehensive and complementary information to further decode brain function and brain networks.