Quantifying U-Net uncertainty in multi-parametric MRI-based glioma segmentation by spherical image projection.

BACKGROUND: Uncertainty quantification in deep learning is an important research topic. For medical image segmentation, the uncertainty measurements are usually reported as the likelihood that each pixel belongs to the predicted segmentation region. In potential clinical applications, the uncertainty result reflects the algorithm's robustness and supports the confidence and trust of the segmentation result when the ground-truth result is absent. For commonly studied deep learning models, novel methods for quantifying segmentation uncertainty are in demand. PURPOSE: To develop a U-Net segmentation uncertainty quantification method based on spherical image projection of multi-parametric MRI (MP-MRI) in glioma segmentation. METHODS: The projection of planar MRI data onto a spherical surface is equivalent to a nonlinear image transformation that retains global anatomical information. By incorporating this image transformation process in our proposed spherical projection-based U-Net (SPU-Net) segmentation model design, multiple independent segmentation predictions can be obtained from a single MRI. The final segmentation is the average of all available results, and the variation can be visualized as a pixel-wise uncertainty map. An uncertainty score was introduced to evaluate and compare the performance of uncertainty measurements. The proposed SPU-Net model was implemented on the basis of 369 glioma patients with MP-MRI scans (T1, T1-Ce, T2, and FLAIR). Three SPU-Net models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The SPU-Net model was compared with (1) the classic U-Net model with test-time augmentation (TTA) and (2) linear scaling-based U-Net (LSU-Net) segmentation models in terms of both segmentation accuracy (Dice coefficient, sensitivity, specificity, and accuracy) and segmentation uncertainty (uncertainty map and uncertainty score). RESULTS: The developed SPU-Net model successfully achieved low uncertainty for correct segmentation predictions (e.g., tumor interior or healthy tissue interior) and high uncertainty for incorrect results (e.g., tumor boundaries). This model could allow the identification of missed tumor targets or segmentation errors in U-Net. Quantitatively, the SPU-Net model achieved the highest uncertainty scores for three segmentation targets (ET/TC/WT): 0.826/0.848/0.936, compared to 0.784/0.643/0.872 using the U-Net with TTA and 0.743/0.702/0.876 with the LSU-Net (scaling factor = 2). The SPU-Net also achieved statistically significantly higher Dice coefficients, underscoring the improved segmentation accuracy. CONCLUSION: The SPU-Net model offers a powerful tool to quantify glioma segmentation uncertainty while improving segmentation accuracy. The proposed method can be generalized to other medical image-related deep-learning applications for uncertainty evaluation.

A neural ordinary differential equation model for visualizing deep neural network behaviors in multi-parametric MRI-based glioma segmentation.

PURPOSE: To develop a neural ordinary differential equation (ODE) model for visualizing deep neural network behavior during multi-parametric MRI-based glioma segmentation as a method to enhance deep learning explainability. METHODS: By hypothesizing that deep feature extraction can be modeled as a spatiotemporally continuous process, we implemented a novel deep learning model, Neural ODE, in which deep feature extraction was governed by an ODE parameterized by a neural network. The dynamics of (1) MR images after interactions with the deep neural network and (2) segmentation formation can thus be visualized after solving the ODE. An accumulative contribution curve (ACC) was designed to quantitatively evaluate each MR image's utilization by the deep neural network toward the final segmentation results. The proposed Neural ODE model was demonstrated using 369 glioma patients with a 4-modality multi-parametric MRI protocol: T1, contrast-enhanced T1 (T1-Ce), T2, and FLAIR. Three Neural ODE models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The key MRI modalities with significant utilization by deep neural networks were identified based on ACC analysis. Segmentation results by deep neural networks using only the key MRI modalities were compared to those using all four MRI modalities in terms of Dice coefficient, accuracy, sensitivity, and specificity. RESULTS: All Neural ODE models successfully illustrated image dynamics as expected. ACC analysis identified T1-Ce as the only key modality in ET and TC segmentations, while both FLAIR and T2 were key modalities in WT segmentation. Compared to the U-Net results using all four MRI modalities, the Dice coefficient of ET (0.784→0.775), TC (0.760→0.758), and WT (0.841→0.837) using the key modalities only had minimal differences without significance. Accuracy, sensitivity, and specificity results demonstrated the same patterns. CONCLUSION: The Neural ODE model offers a new tool for optimizing the deep learning model inputs with enhanced explainability. The presented methodology can be generalized to other medical image-related deep-learning applications.

A radiomics-boosted deep-learning model for COVID-19 and non-COVID-19 pneumonia classification using chest x-ray images.

PURPOSE: To develop a deep learning model design that integrates radiomics analysis for enhanced performance of COVID-19 and non-COVID-19 pneumonia detection using chest x-ray images. METHODS: As a novel radiomics approach, a 2D sliding kernel was implemented to map the impulse response of radiomic features throughout the entire chest x-ray image; thus, each feature is rendered as a 2D map in the same dimension as the x-ray image. Based on each of the three investigated deep neural network architectures, including VGG-16, VGG-19, and DenseNet-121, a pilot model was trained using x-ray images only. Subsequently, two radiomic feature maps (RFMs) were selected based on cross-correlation analysis in reference to the pilot model saliency map results. The radiomics-boosted model was then trained based on the same deep neural network architecture using x-ray images plus the selected RFMs as input. The proposed radiomics-boosted design was developed using 812 chest x-ray images with 262/288/262 COVID-19/non-COVID-19 pneumonia/healthy cases, and 649/163 cases were assigned as training-validation/independent test sets. For each model, 50 runs were trained with random assignments of training/validation cases following the 7:1 ratio in the training-validation set. Sensitivity, specificity, accuracy, and ROC curves together with area-under-the-curve (AUC) from all three deep neural network architectures were evaluated. RESULTS: After radiomics-boosted implementation, all three investigated deep neural network architectures demonstrated improved sensitivity, specificity, accuracy, and ROC AUC results in COVID-19 and healthy individual classifications. VGG-16 showed the largest improvement in COVID-19 classification ROC (AUC from 0.963 to 0.993), and DenseNet-121 showed the largest improvement in healthy individual classification ROC (AUC from 0.962 to 0.989). The reduced variations suggested improved robustness of the model to data partition. For the challenging non-COVID-19 pneumonia classification task, radiomics-boosted implementation of VGG-16 (AUC from 0.918 to 0.969) and VGG-19 (AUC from 0.964 to 0.970) improved ROC results, while DenseNet-121 showed a slight yet insignificant ROC performance reduction (AUC from 0.963 to 0.949). The achieved highest accuracy of COVID-19/non-COVID-19 pneumonia/healthy individual classifications were 0.973 (VGG-19)/0.936 (VGG-19)/ 0.933 (VGG-16), respectively. CONCLUSIONS: The inclusion of radiomic analysis in deep learning model design improved the performance and robustness of COVID-19/non-COVID-19 pneumonia/healthy individual classification, which holds great potential for clinical applications in the COVID-19 pandemic.

Further investigation of 3D dose verification in proton therapy utilizing acoustic signal, wavelet decomposition and machine learning.

Online dose verification in proton therapy is a critical task for quality assurance. We further studied the feasibility of using a wavelet-based machine learning framework to accomplishing that goal in three dimensions, built upon our previous work in 1D. The wavelet decomposition was utilized to extract features of acoustic signals and a bidirectional long-short-term memory (Bi-LSTM) recurrent neural network (RNN) was used. The 3D dose distributions of mono-energetic proton beams (multiple beam energies) inside a 3D CT phantom, were generated using Monte-Carlo simulation. The 3D propagation of acoustic signal was modeled using the k-Wave toolbox. Three different beamlets (i.e. acoustic pathways) were tested, one with its own model. The performance was quantitatively evaluated in terms of mean relative error (MRE) of dose distribution and positioning error of Bragg peak (ΔBP), for two signal-to-noise ratios (SNRs). Due to the lack of experimental data for the time being, two SNR conditions were modeled (SNR = 1 and 5). The model is found to yield good accuracy and noise immunity for all three beamlets. The results exhibit an MRE below 0.6% (without noise) and 1.2% (SNR = 5), andΔBPbelow 1.2 mm (without noise) and 1.3 mm (SNR = 5). For the worst-case scenario (SNR = 1), the MRE andΔBPare below 2.3% and 1.9 mm, respectively. It is encouraging to find out that our model is able to identify the correlation between acoustic waveforms and dose distributions in 3D heterogeneous tissues, as in the 1D case. The work lays a good foundation for us to advance the study and fully validate the feasibility with experimental results.

Dose calculation in proton therapy using a discovery cross-domain generative adversarial network (DiscoGAN).

PURPOSE: Accurate dose calculation is a critical step in proton therapy. A novel machine learning-based approach was proposed to achieve comparable accuracy to that of Monte Carlo simulation while reducing the computational time. METHODS: Computed tomography-based patient phantoms were used and three treatment sites were selected (thorax, head, and abdomen), comprising different beam pathways and beam energies. The training data were generated using Monte Carlo simulations. A discovery cross-domain generative adversarial network (DiscoGAN) was developed to perform the mapping between two domains: stopping power and dose, with HU values from CT images incorporated as auxiliary features. The accuracy of dose calculation was quantitatively evaluated in terms of mean relative error (MRE) and mean absolute error (MAE). The relationship between the DiscoGAN performance and other factors such as absolute dose, beam energy and location within the beam cross-section (center and off-center lines) was examined. RESULTS: The DiscoGAN model is found to be effective in dose calculation. For the abdominal case, the MRE is found to 1.47% (mean), 3.30% (maximum) and 0.67% (minimum). For the thoracic case, the MRE is found to ~2.43% (mean), 4.80% (maximum) and 0.71% (minimum). For the head case, the MRE is found to ~2.83% (mean), 4.84% (maximum) and 1.01% (minimum). Comparable accuracy is found in the independent validation dataset (different CT images), achieving a mean MRE of ~1.65% (thorax), 4.02% (head) and 1.64% (abdomen). For the energy span between 80 and 130 MeV, no strong dependency of accuracy on beam energy is found. The results imply that no systematic deviation, either over-dose or under-dose, occurs between the predicted dose and raw dose. CONCLUSION: The DiscoGAN framework demonstrates great potential as a tool for dose calculation in proton therapy, achieving comparable accuracy yet being more efficient relative to Monte Carlo simulation. Its comparison with the pencil beam algorithm (PBA) will be the next step of our research. If successful, our proposed approach is expected to find its use in more advanced applications such as inverse planning and adaptive proton therapy.

Feasibility study of patient-specific dose verification in proton therapy utilizing positron emission tomography (PET) and generative adversarial network (GAN).

PURPOSE: Online dose verification based on proton-induced positron emitters is a promising strategy for quality assurance in proton therapy. Because of the nonlinear correlation between dose and the activity distributions, a machine learning-based approach was developed to establish their relationship. METHODS: Simulations were carried out using a pencil beam scanning system and a computed tomography (CT) image-based phantom. A DiscoGAN model was developed to perform dose verification for both central and off-center lines. Besides the activity as input, HU information from CT images and stopping power (SP) prior were incorporated as auxiliary features for the model. The performance was quantitatively studied in terms of mean absolute error (MAE) and mean relative error (MRE), under different signal-to-noise ratios (SNRs). In addition to a dataset comprising monoenergetic beams, two additional datasets were generated to evaluate the model's generalization capability: five reconstructed PET images based on an in-beam PET system and a dataset comprising spread-out Bragg peaks (SOBPs). RESULTS: The feasibility of dose verification was successfully demonstrated for all three datasets. For the monoenergetic case (i.e., raw activity of positron emitters), the MRE is found to be <1% for the central lines and 5% for the off-center lines, respectively. The range uncertainty is found to be less than 1 mm. The prediction based on five PET images, which take into account the detection of 511-keV photons and image reconstruction, yields slightly inferior performance. For the SOBP case, the MRE of the center lines is found to be <3% and the range uncertainty is <1 mm. The inclusion of anatomical information (HU and SP) improves both accuracy and generalization of the DiscoGAN model. CONCLUSION: The combination of proton-induced positron emitters, in-beam PET, and machine learning may become a useful tool allowing for patient-specific online dose verification in proton therapy.

Feasibility study of range verification based on proton-induced acoustic signals and recurrent neural network.

Range verification in proton therapy is a critical quality assurance task. We studied the feasibility of online range verification based on proton-induced acoustic signals, using a bidirectional long-short-term-memory recurrent neural network and various signal processing techniques. Dose distribution of 1D pencil proton beams inside a CT image-based phantom was analytically calculated. The propagation of acoustic signal inside the phantom was modeled using the k-Wave toolbox. For signal processing, five methods were investigated: down-sampling (DS), DS + HT (Hilbert transform), Wavelet decomposition (Wavedec db1, db4 and db20). The performances were quantitatively evaluated in terms of mean absolute error, mean relative error (MRE) and the Bragg peak localization error ([Formula: see text]). In addition, the study analyzed the impact of noise levels, the number of sensors, as well as the location of sensors. For the noiseless case (32 sensors), the Wavedec db1 method demonstrates the best performance: [Formula: see text] is less than one pixel and the dose accuracy over the region adjacent to the Bragg peak (MRE50) is ∼3.04%. With the presence of noise, the Wavedec db1 method demonstrates the best noise immunity, achieving [Formula: see text] less than 1 mm and an MRE50 of ∼12%. The proposed machine learning framework may become a useful tool allowing for online range verification in proton therapy.

A machine learning framework with anatomical prior for online dose verification using positron emitters and PET in proton therapy.

We developed a machine learning framework in order to establish the correlation between dose and activity distributions in proton therapy. A recurrent neural network was used to predict dose distribution in three dimensions based on the information of proton-induced positron emitters. Hounsfield Unit (HU) information from CT images and analytically derived stopping power (SP) information were incorporated as auxiliary inputs. Four different scenarios were investigated: Activity only, Activity + HU, Activity + SP and Activity + HU + SP. The performance was quantitatively studied in terms of mean absolute error (MAE) and mean relative error (MRE), under different signal-to-noise ratios (SNRs). In addition to the first dataset of mono-energetic beams, three additional datasets were validated to help evaluate the generalization capability of our proposed model: a dataset of a lower SNR, five reconstructed PET images, and a dataset of spread-out Bragg peaks. Good verification accuracy of dose verification in three dimensions is demonstrated. The inclusion of anatomical information improves both accuracy and generalization. For an activity profile with an SNR of 4 (the mono-energetic case), the framework is able to obtain an MRE of ∼ 0.99% over the whole range and a range uncertainty of ∼ 0.27 mm. The machine learning-based framework may emerge as a useful tool to allow for online dose verification and quality assurance in proton therapy.