Uncertainty quantification via localized gradients for deep learning-based medical image assessments.

Objective.Deep learning models that aid in medical image assessment tasks must be both accurate and reliable to be deployed within clinical settings. While deep learning models have been shown to be highly accurate across a variety of tasks, measures that indicate the reliability of these models are less established. Increasingly, uncertainty quantification (UQ) methods are being introduced to inform users on the reliability of model outputs. However, most existing methods cannot be augmented to previously validated models because they are not post hoc, and they change a model's output. In this work, we overcome these limitations by introducing a novel post hoc UQ method, termedLocal Gradients UQ, and demonstrate its utility for deep learning-based metastatic disease delineation.Approach.This method leverages a trained model's localized gradient space to assess sensitivities to trained model parameters. We compared the Local Gradients UQ method to non-gradient measures defined using model probability outputs. The performance of each uncertainty measure was assessed in four clinically relevant experiments: (1) response to artificially degraded image quality, (2) comparison between matched high- and low-quality clinical images, (3) false positive (FP) filtering, and (4) correspondence with physician-rated disease likelihood.Main results.(1) Response to artificially degraded image quality was enhanced by the Local Gradients UQ method, where the median percent difference between matching lesions in non-degraded and most degraded images was consistently higher for the Local Gradients uncertainty measure than the non-gradient uncertainty measures (e.g. 62.35% vs. 2.16% for additive Gaussian noise). (2) The Local Gradients UQ measure responded better to high- and low-quality clinical images (p< 0.05 vsp> 0.1 for both non-gradient uncertainty measures). (3) FP filtering performance was enhanced by the Local Gradients UQ method when compared to the non-gradient methods, increasing the area under the receiver operating characteristic curve (ROC AUC) by 20.1% and decreasing the false positive rate by 26%. (4) The Local Gradients UQ method also showed more favorable correspondence with physician-rated likelihood for malignant lesions by increasing ROC AUC for correspondence with physician-rated disease likelihood by 16.2%.Significance. In summary, this work introduces and validates a novel gradient-based UQ method for deep learning-based medical image assessments to enhance user trust when using deployed clinical models.

Models using comprehensive, lesion-level, longitudinal [68Ga]Ga-DOTA-TATE PET-derived features lead to superior outcome prediction in neuroendocrine tumor patients treated with [177Lu]Lu-DOTA-TATE.

PURPOSE: Somatostatin receptor (SSTR) imaging features are predictive of treatment outcome for neuroendocrine tumor (NET) patients receiving peptide receptor radionuclide therapy (PRRT). However, comprehensive (all metastatic lesions), longitudinal (temporal variation), and lesion-level measured features have never been explored. Such features allow for capturing the heterogeneity in disease response to treatment. Furthermore, models combining these features are lacking. In this work we evaluated the predictive power of comprehensive, longitudinal, lesion-level 68GA-SSTR-PET features combined with a multivariate linear regression (MLR) model. METHODS: This retrospective study enrolled NET patients treated with [177Lu]Lu-DOTA-TATE and imaged with [68Ga]Ga-DOTA-TATE at baseline and post-therapy. All lesions were segmented, anatomically labeled, and longitudinally matched. Lesion-level uptake and variation in uptake were measured. Patient-level features were engineered and selected for modeling of progression-free survival (PFS). The model was validated via concordance index, patient classification (ROC analysis), and survival analysis (Kaplan-Meier and Cox proportional hazards). The MLR was benchmarked against single feature predictions. RESULTS: Thirty-six NET patients were enrolled and stratified into poor and good responders (PFS ≥ 25 months). Four patient-level features were selected, the MLR concordance index was 0.826, and the AUC was 0.88 (0.85 specificity, 0.81 sensitivity). Survival analysis led to significant patient stratification (p<.001) and hazard ratio (3⨯10-5). Lastly, in a benchmark study, the MLR modeling approach outperformed all the single feature predictors. CONCLUSION: Comprehensive, lesion-level, longitudinal 68GA-SSTR-PET analysis, combined with MLR modeling, leads to excellent predictions of PRRT outcome in NET patients, outperforming non-comprehensive, patient-level, and single time-point feature predictions. MESSAGE: Neuroendocrine tumor, peptide receptor radionuclide therapy, Somatostatin Receptor Imaging, Outcome Prediction, Treatment Response Assessment.

Quantitative evaluation of lesion response heterogeneity for superior prognostication of clinical outcome.

PURPOSE: Standardized reporting of treatment response in oncology patients has traditionally relied on methods like RECIST, PERCIST and Deauville score. These endpoints assess only a few lesions, potentially overlooking the response heterogeneity of all disease. This study hypothesizes that comprehensive spatial-temporal evaluation of all individual lesions is necessary for superior prognostication of clinical outcome. METHODS: [18F]FDG PET/CT scans from 241 patients (127 diffuse large B-cell lymphoma (DLBCL) and 114 non-small cell lung cancer (NSCLC)) were retrospectively obtained at baseline and either during chemotherapy or post-chemoradiotherapy. An automated TRAQinform IQ software (AIQ Solutions) analyzed the images, performing quantification of change in regions of interest suspicious of cancer (lesion-ROI). Multivariable Cox proportional hazards (CoxPH) models were trained to predict overall survival (OS) with varied sets of quantitative features and lesion-ROI, compared by bootstrapping with C-index and t-tests. The best-fit model was compared to automated versions of previously established methods like RECIST, PERCIST and Deauville score. RESULTS: Multivariable CoxPH models demonstrated superior prognostic power when trained with features quantifying response heterogeneity in all individual lesion-ROI in DLBCL (C-index = 0.84, p < 0.001) and NSCLC (C-index = 0.71, p < 0.001). Prognostic power significantly deteriorated (p < 0.001) when using subsets of lesion-ROI (C-index = 0.78 and 0.67 for DLBCL and NSCLC, respectively) or excluding response heterogeneity (C-index = 0.67 and 0.70). RECIST, PERCIST, and Deauville score could not significantly associate with OS (C-index < 0.65 and p > 0.1), performing significantly worse than the multivariable models (p < 0.001). CONCLUSIONS: Quantitative evaluation of response heterogeneity of all individual lesions is necessary for the superior prognostication of clinical outcome.

Uncertainty estimation and evaluation of deformation image registration based convolutional neural networks.

Objective.Fast and accurate deformable image registration (DIR), including DIR uncertainty estimation, is essential for safe and reliable clinical deployment. While recent deep learning models have shown promise in predicting DIR with its uncertainty, challenges persist in proper uncertainty evaluation and hyperparameter optimization for these methods. This work aims to develop and evaluate a model that can perform fast DIR and predict its uncertainty in seconds.Approach.This study introduces a novel probabilistic multi-resolution image registration model utilizing convolutional neural networks to estimate a multivariate normal distributed dense displacement field (DDF) in a multimodal image registration problem. To assess the quality of the DDF distribution predicted by the model, we propose a new metric based on the Kullback-Leibler divergence. The performance of our approach was evaluated against three other DIR algorithms (VoxelMorph, Monte Carlo dropout, and Monte Carlo B-spline) capable of predicting uncertainty. The evaluation of the models included not only the quality of the deformation but also the reliability of the estimated uncertainty. Our application investigated the registration of a treatment planning computed tomography (CT) to follow-up cone beam CT for daily adaptive radiotherapy.Main results.The hyperparameter tuning of the models showed a trade-off between the estimated uncertainty's reliability and the deformation's accuracy. In the optimal trade-off, our model excelled in contour propagation and uncertainty estimation (p <0.05) compared to existing uncertainty estimation models. We obtained an average dice similarity coefficient of 0.89 and a KL-divergence of 0.15.Significance.By addressing challenges in DIR uncertainty estimation and evaluation, our work showed that both the DIR and its uncertainty can be reliably predicted, paving the way for safe deployment in a clinical environment.

An automated methodology for whole-body, multimodality tracking of individual cancer lesions.

Objective. Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities.Approach. This study introduces a lesion tracking methodology and benchmarked it using 2368Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) alignment of multiple scans via image registration, (2) body-part labeling, (3) automatic lesion-wise dilation, (4) clustering of lesions based on local lesion shape metrics, (5) assignment of lesion tracks, and (6) output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed versus automatic dilation), anatomic location, image modalities (inter- versus intra-modality), registration mode (direct versus indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance.Main results. Manual contouring yielded 956 lesions, 1570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes versus 54% for ≥5 nodes).Significance. The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.