Quantitative evaluation of lesion response heterogeneity for superior prognostication of clinical outcome.

PURPOSE: Standardized reporting of treatment response in oncology patients has traditionally relied on methods like RECIST, PERCIST and Deauville score. These endpoints assess only a few lesions, potentially overlooking the response heterogeneity of all disease. This study hypothesizes that comprehensive spatial-temporal evaluation of all individual lesions is necessary for superior prognostication of clinical outcome. METHODS: [18F]FDG PET/CT scans from 241 patients (127 diffuse large B-cell lymphoma (DLBCL) and 114 non-small cell lung cancer (NSCLC)) were retrospectively obtained at baseline and either during chemotherapy or post-chemoradiotherapy. An automated TRAQinform IQ software (AIQ Solutions) analyzed the images, performing quantification of change in regions of interest suspicious of cancer (lesion-ROI). Multivariable Cox proportional hazards (CoxPH) models were trained to predict overall survival (OS) with varied sets of quantitative features and lesion-ROI, compared by bootstrapping with C-index and t-tests. The best-fit model was compared to automated versions of previously established methods like RECIST, PERCIST and Deauville score. RESULTS: Multivariable CoxPH models demonstrated superior prognostic power when trained with features quantifying response heterogeneity in all individual lesion-ROI in DLBCL (C-index = 0.84, p < 0.001) and NSCLC (C-index = 0.71, p < 0.001). Prognostic power significantly deteriorated (p < 0.001) when using subsets of lesion-ROI (C-index = 0.78 and 0.67 for DLBCL and NSCLC, respectively) or excluding response heterogeneity (C-index = 0.67 and 0.70). RECIST, PERCIST, and Deauville score could not significantly associate with OS (C-index < 0.65 and p > 0.1), performing significantly worse than the multivariable models (p < 0.001). CONCLUSIONS: Quantitative evaluation of response heterogeneity of all individual lesions is necessary for the superior prognostication of clinical outcome.

Spatial process decomposition for quantitative imaging biomarkers using multiple images of varying shapes.

Quantitative imaging biomarkers (QIB) are extracted from medical images in radiomics for a variety of purposes including noninvasive disease detection, cancer monitoring, and precision medicine. The existing methods for QIB extraction tend to be ad hoc and not reproducible. In this article, a general and flexible statistical approach is proposed for handling up to three-dimensional medical images and reasonably capturing features with respect to specific spatial patterns. In particular, a model-based spatial process decomposition is developed where the random weights are unique to individual patients for component functions common across patients. Model fitting and selection are based on maximum likelihood, while feature extractions are via optimal prediction of the underlying true image. Simulation studies are conducted to investigate the properties of the proposed methodology. For illustration, a cancer image data set is analyzed and QIBs are extracted in association with a clinical endpoint.

Automatic Quantification of Serial PET/CT Images for Pediatric Hodgkin Lymphoma Patients Using a Longitudinally-Aware Segmentation Network.

Purpose: Automatic quantification of longitudinal changes in PET scans for lymphoma patients has proven challenging, as residual disease in interim-therapy scans is often subtle and difficult to detect. Our goal was to develop a longitudinally-aware segmentation network (LAS-Net) that can quantify serial PET/CT images for pediatric Hodgkin lymphoma patients. Materials and Methods: This retrospective study included baseline (PET1) and interim (PET2) PET/CT images from 297 patients enrolled in two Children's Oncology Group clinical trials (AHOD1331 and AHOD0831). LAS-Net incorporates longitudinal cross-attention, allowing relevant features from PET1 to inform the analysis of PET2. Model performance was evaluated using Dice coefficients for PET1 and detection F1 scores for PET2. Additionally, we extracted and compared quantitative PET metrics, including metabolic tumor volume (MTV) and total lesion glycolysis (TLG) in PET1, as well as qPET and ΔSUVmax in PET2, against physician measurements. We quantified their agreement using Spearman's ρ correlations and employed bootstrap resampling for statistical analysis. Results: LAS-Net detected residual lymphoma in PET2 with an F1 score of 0.606 (precision/recall: 0.615/0.600), outperforming all comparator methods (P<0.01). For baseline segmentation, LAS-Net achieved a mean Dice score of 0.772. In PET quantification, LAS-Net's measurements of qPET, ΔSUVmax, MTV and TLG were strongly correlated with physician measurements, with Spearman's ρ of 0.78, 0.80, 0.93 and 0.96, respectively. The performance remained high, with a slight decrease, in an external testing cohort. Conclusion: LAS-Net achieved high performance in quantifying PET metrics across serial scans, highlighting the value of longitudinal awareness in evaluating multi-time-point imaging datasets.

Comparison of automated full-body bone metastases delineation methods and their corresponding prognostic power.

Objective.Manual disease delineation in full-body imaging of patients with multiple metastases is often impractical due to high disease burden. However, this is a clinically relevant task as quantitative image techniques assessing individual metastases, while limited, have been shown to be predictive of treatment outcome. The goal of this work was to evaluate the efficacy of deep learning-based methods for full-body delineation of skeletal metastases and to compare their performance to existing methods in terms of disease delineation accuracy and prognostic power.Approach.1833 suspicious lesions on 3718F-NaF PET/CT scans of patients with metastatic castration-resistant prostate cancer (mCRPC) were contoured and classified as malignant, equivocal, or benign by a nuclear medicine physician. Two convolutional neural network (CNN) architectures (DeepMedic and nnUNet)were trained to delineate malignant disease regions with and without three-model ensembling. Malignant disease contours using previously established methods were obtained. The performance of each method was assessed in terms of four different tasks: (1) detection, (2) segmentation, (3) PET SUV metric correlations with physician-based data, and (4) prognostic power of progression-free survival.Main Results.The nnUnet three-model ensemble achieved superior detection performance with a mean (+/- standard deviation) sensitivity of 82.9±ccc 0.1% at the selected operating point. The nnUnet single and three-model ensemble achieved comparable segmentation performance with a mean Dice coefficient of 0.80±0.12 and 0.79±0.12, respectively, both outperforming other methods. The nnUNet ensemble achieved comparable or superior SUV metric correlation performance to gold-standard data. Despite superior disease delineation performance, the nnUNet methods did not display superior prognostic power over other methods.Significance.This work showed that CNN-based (nnUNet) methods are superior to the non-CNN methods for mCRPC disease delineation in full-body18F-NaF PET/CT. The CNN-based methods, however, do not hold greater prognostic power for predicting clinical outcome. This merits more investigation on the optimal selection of delineation methods for specific clinical tasks.

Multi-organ segmentation of CT via convolutional neural network: impact of training setting and scanner manufacturer.

Objective. Automated organ segmentation on CT images can enable the clinical use of advanced quantitative software devices, but model performance sensitivities must be understood before widespread adoption can occur. The goal of this study was to investigate performance differences between Convolutional Neural Networks (CNNs) trained to segment one (single-class) versus multiple (multi-class) organs, and between CNNs trained on scans from a single manufacturer versus multiple manufacturers.Methods. The multi-class CNN was trained on CT images obtained from 455 whole-body PET/CT scans (413 for training, 42 for testing) taken with Siemens, GE, and Phillips PET/CT scanners where 16 organs were segmented. The multi-class CNN was compared to 16 smaller single-class CNNs trained using the same data, but with segmentations of only one organ per model. In addition, CNNs trained on Siemens-only (N = 186) and GE-only (N = 219) scans (manufacturer-specific) were compared with CNNs trained on data from both Siemens and GE scanners (manufacturer-mixed). Segmentation performance was quantified using five performance metrics, including the Dice Similarity Coefficient (DSC).Results. The multi-class CNN performed well compared to previous studies, even in organs usually considered difficult auto-segmentation targets (e.g., pancreas, bowel). Segmentations from the multi-class CNN were significantly superior to those from smaller single-class CNNs in most organs, and the 16 single-class models took, on average, six times longer to segment all 16 organs compared to the single multi-class model. The manufacturer-mixed approach achieved minimally higher performance over the manufacturer-specific approach.Significance. A CNN trained on contours of multiple organs and CT data from multiple manufacturers yielded high-quality segmentations. Such a model is an essential enabler of image processing in a software device that quantifies and analyzes such data to determine a patient's treatment response. To date, this activity of whole organ segmentation has not been adopted due to the intense manual workload and time required.

Performance of an automated registration-based method for longitudinal lesion matching and comparison to inter-reader variability.

Objective.Patients with metastatic disease are followed throughout treatment with medical imaging, and accurately assessing changes of individual lesions is critical to properly inform clinical decisions. The goal of this work was to assess the performance of an automated lesion-matching algorithm in comparison to inter-reader variability (IRV) of matching lesions between scans of metastatic cancer patients.Approach.Forty pairs of longitudinal PET/CT and CT scans were collected and organized into four cohorts: lung cancers, head and neck cancers, lymphomas, and advanced cancers. Cases were also divided by cancer burden: low-burden (<10 lesions), intermediate-burden (10-29), and high-burden (30+). Two nuclear medicine physicians conducted independent reviews of each scan-pair and manually matched lesions. Matching differences between readers were assessed to quantify the IRV of lesion matching. The two readers met to form a consensus, which was considered a gold standard and compared against the output of an automated lesion-matching algorithm. IRV and performance of the automated method were quantified using precision, recall, F1-score, and the number of differences.Main results.The performance of the automated method did not differ significantly from IRV for any metric in any cohort (p> 0.05, Wilcoxon paired test). In high-burden cases, the F1-score (median [range]) was 0.89 [0.63, 1.00] between the automated method and reader consensus and 0.93 [0.72, 1.00] between readers. In low-burden cases, F1-scores were 1.00 [0.40, 1.00] and 1.00 [0.40, 1.00], for the automated method and IRV, respectively. Automated matching was significantly more efficient than either reader (p< 0.001). In high-burden cases, median matching time for the readers was 60 and 30 min, respectively, while automated matching took a median of 3.9 minSignificance.The automated lesion-matching algorithm was successful in performing lesion matching, meeting the benchmark of IRV. Automated lesion matching can significantly expedite and improve the consistency of longitudinal lesion-matching.

Quantitative Analysis of Serial Positron Emission Tomography Imaging in Men with Metastatic Castration-resistant Prostate Cancer Treated with Enzalutamide.

BACKGROUND: The emergence of positron emission tomography (PET) in prostate cancer is impacting clinical practice, but little is known about PET imaging as a tool to determine treatment failure in metastatic castration-resistant prostate cancer (mCRPC). OBJECTIVE: To evaluate PET imaging dynamics in mCRPC patients on enzalutamide with stable computed tomography (CT) and technetium-99m (Tc99) bone scans. DESIGN, SETTING, AND PARTICIPANTS: All patients were on treatment with enzalutamide for first-line mCRPC in a clinical trial at the National Cancer Institute (Bethesda, MD, USA). A volunteer sample had serial 18F-sodium fluoride (NaF) PET in parallel with CT and Tc99. Regions of interest (ROIs) on NaF were analyzed quantitatively for response. INTERVENTION: Patients were randomized to enzalutamide with/without a cancer immunotherapy, Prostvac. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: A post hoc, descriptive analysis was performed comparing the changes seen on CT and Tc99 as per RECIST 1.1 with NaF PET scans including the use of a quantitative analysis. RESULTS AND LIMITATIONS: Eighteen mCRPC patients had 67 NaF scans. A total of 233 ROIs resolved after treatment, 52 (22%) of which eventually retuned while on therapy. In all, 394 new ROIs were seen, but 112(28%) resolved subsequently. Of 18 patients, 14 had new ROIs that ultimately resolved after appearing. Many patients experienced progression in a minority of lesions, and one patient with radiation intervention to oligoprogression had a remarkable response. This study is limited by its small number of patients and post hoc nature. CONCLUSIONS: These data highlight the dynamic nature of NaF PET in mCRPC patients treated with enzalutamide, where not all new findings were ultimately related to disease progression. This analysis also provides a potential strategy to identify and intervene in oligoprogression in prostate cancer. PATIENT SUMMARY: In this small analysis of patients with prostate cancer on enzalutamide, changes on 18F-sodium fluoride positron emission tomography (PET) imaging were not always associated with treatment failure. Caution may be indicated when using PET imaging to determine whether new therapy is needed.

Interpretation and visualization techniques for deep learning models in medical imaging.

Deep learning (DL) approaches to medical image analysis tasks have recently become popular; however, they suffer from a lack of human interpretability critical for both increasing understanding of the methods' operation and enabling clinical translation. This review summarizes currently available methods for performing image model interpretation and critically evaluates published uses of these methods for medical imaging applications. We divide model interpretation in two categories: (1) understanding model structure and function and (2) understanding model output. Understanding model structure and function summarizes ways to inspect the learned features of the model and how those features act on an image. We discuss techniques for reducing the dimensionality of high-dimensional data and cover autoencoders, both of which can also be leveraged for model interpretation. Understanding model output covers attribution-based methods, such as saliency maps and class activation maps, which produce heatmaps describing the importance of different parts of an image to the model prediction. We describe the mathematics behind these methods, give examples of their use in medical imaging, and compare them against one another. We summarize several published toolkits for model interpretation specific to medical imaging applications, cover limitations of current model interpretation methods, provide recommendations for DL practitioners looking to incorporate model interpretation into their task, and offer general discussion on the importance of model interpretation in medical imaging contexts.

Impact of scanner harmonization on PET-based treatment response assessment in metastatic melanoma.

Patients with metastatic melanoma often receive 18F-FDG PET/CT scans on different scanners throughout their monitoring period. In this study, we quantified the impact of scanner harmonization on longitudinal changes in PET standardized uptake values using various harmonization and normalization methods, including an anthropomorphic PET phantom. Twenty metastatic melanoma patients received at least two FDG PET/CT scans, each on two different scanners with an average of 4 months (range: 2-8) between. Scans from a General Electric (GE) Discovery 710 PET CT-1 were harmonized to the GE Discovery VCT using image reconstruction settings matching recovery coefficients in an anthropomorphic phantom with bone equivalent inserts and wall-less synthetic lesions. In patient images, SUVmax was measured for each melanoma lesion and time-point. Lesions were classified as progressing, stable, or responding based on pre-defined threshold of ±30% change in SUVmax. For comparison, harmonization was also performed using simpler methods, including harmonization using a NEMA phantom, post-reconstruction filtering, reference region normalization of SUVmax, and use of SUVpeak instead of SUVmax. In the 20 patients, 90 lesions across two time-points were available for treatment response assessment. Treatment response classification changed in 47% (42/90) of cases after harmonization with anthropomorphic phantom. Before harmonization, 37% (33/90) of the lesions were classified as stable (changing less than 30% between two time-points), while the fraction of stable lesions increased to 58% (52/90) after harmonization. Harmonization with the NEMA phantom agreed with harmonization with the anthropomorphic phantom in 91% (82/90) of cases. Post-reconstruction filtering agreed with anthropomorphic phantom-based harmonization in 83% (75/90) cases. The utilization of reference regions for normalization or SUVpeak was unable to correct for changes as identified by the anthropomorphic phantom-based harmonization. Overall, PET scanner harmonization has a major impact on individual lesion treatment response classification in metastatic melanoma patients. Harmonization using the NEMA phantom yielded similar results to harmonization using anthropomorphic phantom, while the only acceptable post-reconstruction technique was post-reconstruction filtering. Phantom-based harmonization is therefore strongly recommended when comparing lesion uptake across time-points when the images have been acquired on different PET scanners.

Comparison of 11 automated PET segmentation methods in lymphoma.

Segmentation of lymphoma lesions in FDG PET/CT images is critical in both assessing individual lesions and quantifying patient disease burden. Simple thresholding methods remain common despite the large heterogeneity in lymphoma lesion location, size, and contrast. Here, we assess 11 automated PET segmentation methods for their use in two scenarios: individual lesion segmentation and patient-level disease quantification in lymphoma. Lesions on 18F-FDG PET/CT scans of 90 lymphoma patients were contoured by a nuclear medicine physician. Thresholding, active contours, clustering, adaptive region-growing, and convolutional neural network (CNN) methods were implemented on all physician-identified lesions. Lesion-level segmentation was evaluated using multiple segmentation performance metrics (Dice, Hausdorff Distance). Patient-level quantification of total disease burden (SUVtotal) and metabolic tumor volume (MTV) was assessed using Spearman's correlation coefficients between the segmentation output and physician contours. Lesion segmentation and patient quantification performance was compared to inter-physician agreement in a subset of 20 patients segmented by a second nuclear medicine physician. In total, 1223 lesions with median tumor-to-background ratio of 4.0 and volume of 1.8 cm3, were evaluated. When assessed for lesion segmentation, a 3D CNN, DeepMedic, achieved the highest performance across all evaluation metrics. DeepMedic, clustering methods, and an iterative threshold method had lesion-level segmentation performance comparable to the degree of inter-physician agreement. For patient-level SUVtotal and MTV quantification, all methods except 40% and 50% SUVmax and adaptive region-growing achieved a performance that was similar the agreement of the two physicians. Multiple methods, including a 3D CNN, clustering, and an iterative threshold method, achieved both good lesion-level segmentation and patient-level quantification performance in a population of 90 lymphoma patients. These methods are thus recommended over thresholding methods such as 40% and 50% SUVmax, which were consistently found to be significantly outside the limits defined by inter-physician agreement.

Convolutional Neural Networks for Automated PET/CT Detection of Diseased Lymph Node Burden in Patients with Lymphoma.

PURPOSE: To automatically detect lymph nodes involved in lymphoma on fluorine 18 (18F) fluorodeoxyglucose (FDG) PET/CT images using convolutional neural networks (CNNs). MATERIALS AND METHODS: In this retrospective study, baseline disease of 90 patients with lymphoma was segmented on 18F-FDG PET/CT images (acquired between 2005 and 2011) by a nuclear medicine physician. An ensemble of three-dimensional patch-based, multiresolution pathway CNNs was trained using fivefold cross-validation. Performance was assessed using the true-positive rate (TPR) and number of false-positive (FP) findings. CNN performance was compared with agreement between physicians by comparing the annotations of a second nuclear medicine physician to the first reader in 20 of the patients. Patient TPR was compared using Wilcoxon signed rank tests. RESULTS: Across all 90 patients, a range of 0-61 nodes per patient was detected. At an average of four FP findings per patient, the method achieved a TPR of 85% (923 of 1087 nodes). Performance varied widely across patients (TPR range, 33%-100%; FP range, 0-21 findings). In the 20 patients labeled by both physicians, a range of 1-49 nodes per patient was detected and labeled. The second reader identified 96% (210 of 219) of nodes with an additional 3.7 per patient compared with the first reader. In the same 20 patients, the CNN achieved a 90% (197 of 219) TPR at 3.7 FP findings per patient. CONCLUSION: An ensemble of three-dimensional CNNs detected lymph nodes at a performance nearly comparable to differences between two physicians' annotations. This preliminary study is a first step toward automated PET/CT assessment for lymphoma.© RSNA, 2020.

Automated quantification of baseline imaging PET metrics on FDG PET/CT images of pediatric Hodgkin lymphoma patients.

PURPOSE: For pediatric lymphoma, quantitative FDG PET/CT imaging features such as metabolic tumor volume (MTV) are important for prognosis and risk stratification strategies. However, feature extraction is difficult and time-consuming in cases of high disease burden. The purpose of this study was to fully automate the measurement of PET imaging features in PET/CT images of pediatric lymphoma. METHODS: 18F-FDG PET/CT baseline images of 100 pediatric Hodgkin lymphoma patients were retrospectively analyzed. Two nuclear medicine physicians identified and segmented FDG avid disease using PET thresholding methods. Both PET and CT images were used as inputs to a three-dimensional patch-based, multi-resolution pathway convolutional neural network architecture, DeepMedic. The model was trained to replicate physician segmentations using an ensemble of three networks trained with 5-fold cross-validation. The maximum SUV (SUVmax), MTV, total lesion glycolysis (TLG), surface-area-to-volume ratio (SA/MTV), and a measure of disease spread (Dmaxpatient) were extracted from the model output. Pearson's correlation coefficient and relative percent differences were calculated between automated and physician-extracted features. RESULTS: Median Dice similarity coefficient of patient contours between automated and physician contours was 0.86 (IQR 0.78-0.91). Automated SUVmax values matched exactly the physician determined values in 81/100 cases, with Pearson's correlation coefficient (R) of 0.95. Automated MTV was strongly correlated with physician MTV (R = 0.88), though it was slightly underestimated with a median (IQR) relative difference of - 4.3% (- 10.0-5.7%). Agreement of TLG was excellent (R = 0.94), with median (IQR) relative difference of - 0.4% (- 5.2-7.0%). Median relative percent differences were 6.8% (R = 0.91; IQR 1.6-4.3%) for SA/MTV, and 4.5% (R = 0.51; IQR - 7.5-40.9%) for Dmaxpatient, which was the most difficult feature to quantify automatically. CONCLUSIONS: An automated method using an ensemble of multi-resolution pathway 3D CNNs was able to quantify PET imaging features of lymphoma on baseline FDG PET/CT images with excellent agreement to reference physician PET segmentation. Automated methods with faster throughput for PET quantitation, such as MTV and TLG, show promise in more accessible clinical and research applications.

Quantification of bone flare on 18F-NaF PET/CT in metastatic castration-resistant prostate cancer.

BACKGROUND: Bone flare has been observed on 99mTc-MDP bone scans of patients with metastatic castration-resistant prostate cancer (mCRPC). This exploratory study investigates bone flare in mCRPC patients receiving androgen receptor (AR) inhibitors using 18F-NaF PET/CT. METHODS: Twenty-nine mCRPC patients undergoing AR-inhibiting therapy (abiraterone, orteronel, enzalutamide) received NaF PET/CT scans at baseline, week 6, and week 12 of treatment. SUV metrics were extracted globally for each patient (SUV) and for each individual lesion (iSUV). Bone flare was defined as increasing SUV metrics or lesion number at week 6 followed by subsequent week 12 decrease. Differences in metrics across timepoints were compared using Wilcoxon tests. Cox proportional hazard regression was conducted between global metrics and progression-free survival (PFS). RESULTS: Total SUV was most sensitive for flare detection and was identified in 14/23 (61%) patients receiving CYP17A1-inhibitors (abiraterone, orteronel), and not identified in any of six patients receiving enzalutamide. The appearance of new lesions did not account for initial increases in SUV metrics. iSUV metrics followed patient-level trends: bone flare positive patients showed a median of 72% (range: 0-100%) of lesions with total iSUV flare. Increasing mean SUV at week 6 correlated with extended PFS (HR = 0.58, p = 0.02). CONCLUSION: NaF PET bone flare was present on 61% of mCRPC patients in the first 6 weeks of treatment with CYP17A1-inhibitors. Characterization provided in this study suggests favorable PFS in patients showing bone flare. This characterization of NaF flare is important for guiding treatment assessment schedules to better distinguish between patients showing bone flare and those truly progressing, and should be performed for all emerging mCRPC treatments and imaging agents.