Uncertainty quantification via localized gradients for deep learning-based medical image assessments.

OBJECTIVE: Deep learning models that aid in medical image assessment tasks must be both accurate and reliable to be deployed within clinical settings. While deep learning models have been shown to be highly accurate across a variety of tasks, measures that indicate the reliability of these models are less established. Increasingly, uncertainty quantification (UQ) methods are being introduced to inform users on the reliability of model outputs. However, most existing methods cannot be augmented to previously validated models because they are not post hoc, and they change a model's output. In this work, we overcome these limitations by introducing a novel post hoc UQ method, termed Local Gradients UQ, and demonstrate its utility for deep learning-based metastatic disease delineation. APPROACH: This method leverages a trained model's localized gradient space to assess sensitivities to trained model parameters. We compared the Local Gradients UQ method to non-gradient measures defined using model probability outputs. The performance of each uncertainty measure was assessed in four clinically relevant experiments: (1) response to artificially degraded image quality, (2) comparison between matched high- and low-quality clinical images, (3) false positive (FP) filtering, and (4) correspondence with physician-rated disease likelihood. MAIN RESULTS: (1) Response to artificially degraded image quality was enhanced by the Local Gradients UQ method, where the median percent difference between matching lesions in non-degraded and most degraded images was consistently higher for the Local Gradients uncertainty measure than the non-gradient uncertainty measures (e.g., 62.35% vs. 2.16% for additive Gaussian noise). (2) The Local Gradients UQ measure responded better to high- and low-quality clinical images (p<0.05 vs p>0.1 for both non-gradient uncertainty measures). (3) FP filtering performance was enhanced by the Local Gradients UQ method when compared to the non-gradient methods, increasing the area under the receiver operating characteristic curve (ROC AUC) by 20.1% and decreasing the false positive rate by 26%. (4) The Local Gradients UQ method also showed more favorable correspondence with physician-rated likelihood for malignant lesions by increasing ROC AUC for correspondence with physician-rated disease likelihood by 16.2%. SIGNIFICANCE: In summary, this work introduces and validates a novel gradient-based UQ method for deep learning-based medical image assessments to enhance user trust when using deployed clinical models.

Models using comprehensive, lesion-level, longitudinal [68Ga]Ga-DOTA-TATE PET-derived features lead to superior outcome prediction in neuroendocrine tumor patients treated with [177Lu]Lu-DOTA-TATE.

PURPOSE: Somatostatin receptor (SSTR) imaging features are predictive of treatment outcome for neuroendocrine tumor (NET) patients receiving peptide receptor radionuclide therapy (PRRT). However, comprehensive (all metastatic lesions), longitudinal (temporal variation), and lesion-level measured features have never been explored. Such features allow for capturing the heterogeneity in disease response to treatment. Furthermore, models combining these features are lacking. In this work we evaluated the predictive power of comprehensive, longitudinal, lesion-level 68GA-SSTR-PET features combined with a multivariate linear regression (MLR) model. METHODS: This retrospective study enrolled NET patients treated with [177Lu]Lu-DOTA-TATE and imaged with [68Ga]Ga-DOTA-TATE at baseline and post-therapy. All lesions were segmented, anatomically labeled, and longitudinally matched. Lesion-level uptake and variation in uptake were measured. Patient-level features were engineered and selected for modeling of progression-free survival (PFS). The model was validated via concordance index, patient classification (ROC analysis), and survival analysis (Kaplan-Meier and Cox proportional hazards). The MLR was benchmarked against single feature predictions. RESULTS: Thirty-six NET patients were enrolled and stratified into poor and good responders (PFS ≥ 25 months). Four patient-level features were selected, the MLR concordance index was 0.826, and the AUC was 0.88 (0.85 specificity, 0.81 sensitivity). Survival analysis led to significant patient stratification (p<.001) and hazard ratio (3⨯10-5). Lastly, in a benchmark study, the MLR modeling approach outperformed all the single feature predictors. CONCLUSION: Comprehensive, lesion-level, longitudinal 68GA-SSTR-PET analysis, combined with MLR modeling, leads to excellent predictions of PRRT outcome in NET patients, outperforming non-comprehensive, patient-level, and single time-point feature predictions. MESSAGE: Neuroendocrine tumor, peptide receptor radionuclide therapy, Somatostatin Receptor Imaging, Outcome Prediction, Treatment Response Assessment.

An automated methodology for whole-body, multimodality tracking of individual cancer lesions.

Objective. Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities.Approach. This study introduces a lesion tracking methodology and benchmarked it using 2368Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) alignment of multiple scans via image registration, (2) body-part labeling, (3) automatic lesion-wise dilation, (4) clustering of lesions based on local lesion shape metrics, (5) assignment of lesion tracks, and (6) output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed versus automatic dilation), anatomic location, image modalities (inter- versus intra-modality), registration mode (direct versus indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance.Main results. Manual contouring yielded 956 lesions, 1570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes versus 54% for ≥5 nodes).Significance. The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.

Uncertainty estimation for deep learning-based pectoral muscle segmentation via Monte Carlo dropout.

Objective. Deep Learning models are often susceptible to failures after deployment. Knowing when your model is producing inadequate predictions is crucial. In this work, we investigate the utility of Monte Carlo (MC) dropout and the efficacy of the proposed uncertainty metric (UM) for flagging of unacceptable pectoral muscle segmentations in mammograms.Approach. Segmentation of pectoral muscle was performed with modified ResNet18 convolutional neural network. MC dropout layers were kept unlocked at inference time. For each mammogram, 50 pectoral muscle segmentations were generated. The mean was used to produce the final segmentation and the standard deviation was applied for the estimation of uncertainty. From each pectoral muscle uncertainty map, the overall UM was calculated. To validate the UM, a correlation between the dice similarity coefficient (DSC) and UM was used. The UM was first validated in a training set (200 mammograms) and finally tested in an independent dataset (300 mammograms). ROC-AUC analysis was performed to test the discriminatory power of the proposed UM for flagging unacceptable segmentations.Main results. The introduction of dropout layers in the model improved segmentation performance (DSC = 0.95 ± 0.07 versus DSC = 0.93 ± 0.10). Strong anti-correlation (r= -0.76,p< 0.001) between the proposed UM and DSC was observed. A high AUC of 0.98 (97% specificity at 100% sensitivity) was obtained for the discrimination of unacceptable segmentations. Qualitative inspection by the radiologist revealed that images with high UM are difficult to segment.Significance. The use of MC dropout at inference time in combination with the proposed UM enables flagging of unacceptable pectoral muscle segmentations from mammograms with excellent discriminatory power.

Comparison of automated full-body bone metastases delineation methods and their corresponding prognostic power.

Objective.Manual disease delineation in full-body imaging of patients with multiple metastases is often impractical due to high disease burden. However, this is a clinically relevant task as quantitative image techniques assessing individual metastases, while limited, have been shown to be predictive of treatment outcome. The goal of this work was to evaluate the efficacy of deep learning-based methods for full-body delineation of skeletal metastases and to compare their performance to existing methods in terms of disease delineation accuracy and prognostic power.Approach.1833 suspicious lesions on 3718F-NaF PET/CT scans of patients with metastatic castration-resistant prostate cancer (mCRPC) were contoured and classified as malignant, equivocal, or benign by a nuclear medicine physician. Two convolutional neural network (CNN) architectures (DeepMedic and nnUNet)were trained to delineate malignant disease regions with and without three-model ensembling. Malignant disease contours using previously established methods were obtained. The performance of each method was assessed in terms of four different tasks: (1) detection, (2) segmentation, (3) PET SUV metric correlations with physician-based data, and (4) prognostic power of progression-free survival.Main Results.The nnUnet three-model ensemble achieved superior detection performance with a mean (+/- standard deviation) sensitivity of 82.9±ccc 0.1% at the selected operating point. The nnUnet single and three-model ensemble achieved comparable segmentation performance with a mean Dice coefficient of 0.80±0.12 and 0.79±0.12, respectively, both outperforming other methods. The nnUNet ensemble achieved comparable or superior SUV metric correlation performance to gold-standard data. Despite superior disease delineation performance, the nnUNet methods did not display superior prognostic power over other methods.Significance.This work showed that CNN-based (nnUNet) methods are superior to the non-CNN methods for mCRPC disease delineation in full-body18F-NaF PET/CT. The CNN-based methods, however, do not hold greater prognostic power for predicting clinical outcome. This merits more investigation on the optimal selection of delineation methods for specific clinical tasks.

Predicting gamma passing rates for portal dosimetry-based IMRT QA using machine learning.

PURPOSE: Intensity-modulated radiation therapy (IMRT) quality assurance (QA) measurements are routinely performed prior to treatment delivery to verify dose calculation and delivery accuracy. In this work, we applied a machine learning-based approach to predict portal dosimetry based IMRT QA gamma passing rates. METHODS: 182 IMRT plans for various treatment sites were planned and delivered with portal dosimetry on two TrueBeam and two Trilogy LINACs. A total of 1497 beams were collected and analyzed using gamma criteria of 2%/2 mm with a 5% threshold. The datasets for building the machine learning models consisted of 1269 beams. Ten-fold cross-validation was utilized to tune the model and prevent "overfitting." A separate test set with the remaining 228 beams was used to evaluate model performance. Each beam was characterized by a set of 31 features including both plan complexity metrics and machine characteristics. Three tree-based machine learning algorithms (AdaBoost, Random Forest, and XGBoost) were used to train the models and predict gamma passing rates. RESULTS: Both AdaBoost and Random Forest had 98% of predictions within 3% of the measured 2%/2 mm gamma passing rates with a maximum error less than 4% and a mean absolute error < 1%. XGBoost showed a slightly worse prediction accuracy with 95% of the predictions within 3% of the measured gamma passing rates and a maximum error of 4.5%. The three models identified the same nine features in the top 10 most important ones that are related to plan complexity and maximum aperture displacement from the central axis or the maximum jaw size in a beam. CONCLUSION: We have demonstrated that portal dosimetry IMRT QA gamma passing rates can be accurately predicted using tree-based ensemble learning models. The machine learning based approach allows physicists to better identify the failures of IMRT QA measurements and to develop proactive QA approaches.