International Benchmark for Total Metabolic Tumor Volume Measurement in Baseline 18F-FDG PET/CT of Lymphoma Patients: A Milestone Toward Clinical Implementation.

Total metabolic tumor volume (TMTV) is prognostic in lymphoma. However, cutoff values for risk stratification vary markedly, according to the tumor delineation method used. We aimed to create a standardized TMTV benchmark dataset allowing TMTV to be tested and applied as a reproducible biomarker. Methods: Sixty baseline 18F-FDG PET/CT scans were identified with a range of disease distributions (20 follicular, 20 Hodgkin, and 20 diffuse large B-cell lymphoma). TMTV was measured by 12 nuclear medicine experts, each analyzing 20 cases split across subtypes, with each case processed by 3-4 readers. LIFEx or ACCURATE software was chosen according to reader preference. Analysis was performed stepwise: TMTV1 with automated preselection of lesions using an SUV of at least 4 and a volume of at least 3 cm3 with single-click removal of physiologic uptake; TMTV2 with additional removal of reactive bone marrow and spleen with single clicks; TMTV3 with manual editing to remove other physiologic uptake, if required; and TMTV4 with optional addition of lesions using mouse clicks with an SUV of at least 4 (no volume threshold). Results: The final TMTV (TMTV4) ranged from 8 to 2,288 cm3, showing excellent agreement among all readers in 87% of cases (52/60) with a difference of less than 10% or less than 10 cm3 In 70% of the cases, TMTV4 equaled TMTV1, requiring no additional reader interaction. Differences in the TMTV4 were exclusively related to reader interpretation of lesion inclusion or physiologic high-uptake region removal, not to the choice of software. For 5 cases, large TMTV differences (>25%) were due to disagreement about inclusion of diffuse splenic uptake. Conclusion: The proposed segmentation method enabled highly reproducible TMTV measurements, with minimal reader interaction in 70% of the patients. The inclusion or exclusion of diffuse splenic uptake requires definition of specific criteria according to lymphoma subtype. The publicly available proposed benchmark allows comparison of study results and could serve as a reference to test improvements using other segmentation approaches.

RadShap: An Explanation Tool for Highlighting the Contributions of Multiple Regions of Interest to the Prediction of Radiomic Models.

Explaining the decisions made by a radiomic model is of significant interest, as it can provide valuable insights into the information learned by complex models and foster trust in well-performing ones, thereby facilitating their clinical adoption. Promising radiomic approaches that aggregate information from multiple regions within an image currently lack suitable explanation tools that could identify the regions that most significantly influence their decisions. Here we present a model- and modality-agnostic tool (RadShap, https://github.com/ncaptier/radshap), based on Shapley values, that explains the predictions of multiregion radiomic models by highlighting the contribution of each individual region. Methods: The explanation tool leverages Shapley values to distribute the aggregative radiomic model's output among all the regions of interest of an image, highlighting their individual contribution. RadShap was validated using a retrospective cohort of 130 patients with advanced non-small cell lung cancer undergoing first-line immunotherapy. Their baseline PET scans were used to build 1,000 synthetic tasks to evaluate the degree of alignment between the tool's explanations and our data generation process. RadShap's potential was then illustrated through 2 real case studies by aggregating information from all segmented tumors: the prediction of the progression-free survival of the non-small cell lung cancer patients and the classification of the histologic tumor subtype. Results: RadShap demonstrated strong alignment with the ground truth, with a median frequency of 94% for consistently explained predictions in the synthetic tasks. In both real-case studies, the aggregative models yielded superior performance to the single-lesion models (average [±SD] time-dependent area under the receiver operating characteristic curve was 0.66 ± 0.02 for the aggregative survival model vs. 0.55 ± 0.04 for the primary tumor survival model). The tool's explanations provided relevant insights into the behavior of the aggregative models, highlighting that for the classification of the histologic subtype, the aggregative model used information beyond the biopsy site to correctly classify patients who were initially misclassified by a model focusing only on the biopsied tumor. Conclusion: RadShap aligned with ground truth explanations and provided valuable insights into radiomic models' behaviors. It is implemented as a user-friendly Python package with documentation and tutorials, facilitating its smooth integration into radiomic pipelines.

Multicenter PET image harmonization using generative adversarial networks.

PURPOSE: To improve reproducibility and predictive performance of PET radiomic features in multicentric studies by cycle-consistent generative adversarial network (GAN) harmonization approaches. METHODS: GAN-harmonization was developed to harmonize whole-body PET scans to perform image style and texture translation between different centers and scanners. GAN-harmonization was evaluated by application to two retrospectively collected open datasets and different tasks. First, GAN-harmonization was performed on a dual-center lung cancer cohort (127 female, 138 male) where the reproducibility of radiomic features in healthy liver tissue was evaluated. Second, GAN-harmonization was applied to a head and neck cancer cohort (43 female, 154 male) acquired from three centers. Here, the clinical impact of GAN-harmonization was analyzed by predicting the development of distant metastases using a logistic regression model incorporating first-order statistics and texture features from baseline 18F-FDG PET before and after harmonization. RESULTS: Image quality remained high (structural similarity: left kidney ≥ 0.800, right kidney ≥ 0.806, liver ≥ 0.780, lung ≥ 0.838, spleen ≥ 0.793, whole-body ≥ 0.832) after image harmonization across all utilized datasets. Using GAN-harmonization, inter-site reproducibility of radiomic features in healthy liver tissue increased at least by ≥ 5 ± 14% (first-order), ≥ 16 ± 7% (GLCM), ≥ 19 ± 5% (GLRLM), ≥ 16 ± 8% (GLSZM), ≥ 17 ± 6% (GLDM), and ≥ 23 ± 14% (NGTDM). In the head and neck cancer cohort, the outcome prediction improved from AUC 0.68 (95% CI 0.66-0.71) to AUC 0.73 (0.71-0.75) by application of GAN-harmonization. CONCLUSIONS: GANs are capable of performing image harmonization and increase reproducibility and predictive performance of radiomic features derived from different centers and scanners.

Promising Candidate Prognostic Biomarkers in [18F]FDG PET Images: Evaluation in Independent Cohorts of Non-Small Cell Lung Cancer Patients.

The normalized distances from the hot spot of radiotracer uptake (SUVmax) to the tumor centroid (NHOC) and to the tumor perimeter (NHOP) have recently been suggested as novel PET features reflecting tumor aggressiveness. These biomarkers characterizing the shift of SUVmax toward the lesion edge during tumor progression have been shown to be prognostic factors in breast and non-small cell lung cancer (NSCLC) patients. We assessed the impact of imaging parameters on NHOC and NHOP, their complementarity to conventional PET features, and their prognostic value for advanced-NSCLC patients. Methods: This retrospective study investigated baseline [18F]FDG PET scans: cohort 1 included 99 NSCLC patients with no treatment-related inclusion criteria (robustness study); cohort 2 included 244 NSCLC patients (survival analysis) treated with targeted therapy (93), immunotherapy (63), or immunochemotherapy (88). Although 98% of patients had metastases, radiomic features including SUVs were extracted from the primary tumor only. NHOCs and NHOPs were computed using 2 approaches: the normalized distance from the localization of SUVmax or SUVpeak to the tumor centroid or perimeter. Bland-Altman analyses were performed to investigate the impact of both spatial resolution (comparing PET images with and without gaussian postfiltering) and image sampling (comparing 2 voxel sizes) on feature values. The correlation of NHOCs and NHOPs with other features was studied using Spearman correlation coefficients (r). The ability of NHOCs and NHOPs to predict overall survival (OS) was estimated using the Kaplan-Meier method. Results: In cohort 1, NHOC and NHOP features were more robust to image filtering and to resampling than were SUVs. The correlations were weak between NHOCs and NHOPs (r ≤ 0.45) and between NHOCs or NHOPs and any other radiomic features (r ≤ 0.60). In cohort 2, the patients with short OS demonstrated higher NHOCs and lower NHOPs than those with long OS. NHOCs significantly distinguished 2 survival profiles in patients treated with immunotherapy (log-rank test, P < 0.01), whereas NHOPs stratified patients regarding OS in the targeted therapy (P = 0.02) and immunotherapy (P < 0.01) subcohorts. Conclusion: Our findings suggest that even in advanced NSCLC patients, NHOC and NHOP features pertaining to the primary tumor have prognostic potential. Moreover, these features appeared to be robust with respect to imaging protocol parameters and complementary to other radiomic features and are now available in LIFEx software to be independently tested by others.

The Image Biomarker Standardization Initiative: Standardized Convolutional Filters for Reproducible Radiomics and Enhanced Clinical Insights.

Filters are commonly used to enhance specific structures and patterns in images, such as vessels or peritumoral regions, to enable clinical insights beyond the visible image using radiomics. However, their lack of standardization restricts reproducibility and clinical translation of radiomics decision support tools. In this special report, teams of researchers who developed radiomics software participated in a three-phase study (September 2020 to December 2022) to establish a standardized set of filters. The first two phases focused on finding reference filtered images and reference feature values for commonly used convolutional filters: mean, Laplacian of Gaussian, Laws and Gabor kernels, separable and nonseparable wavelets (including decomposed forms), and Riesz transformations. In the first phase, 15 teams used digital phantoms to establish 33 reference filtered images of 36 filter configurations. In phase 2, 11 teams used a chest CT image to derive reference values for 323 of 396 features computed from filtered images using 22 filter and image processing configurations. Reference filtered images and feature values for Riesz transformations were not established. Reproducibility of standardized convolutional filters was validated on a public data set of multimodal imaging (CT, fluorodeoxyglucose PET, and T1-weighted MRI) in 51 patients with soft-tissue sarcoma. At validation, reproducibility of 486 features computed from filtered images using nine configurations × three imaging modalities was assessed using the lower bounds of 95% CIs of intraclass correlation coefficients. Out of 486 features, 458 were found to be reproducible across nine teams with lower bounds of 95% CIs of intraclass correlation coefficients greater than 0.75. In conclusion, eight filter types were standardized with reference filtered images and reference feature values for verifying and calibrating radiomics software packages. A web-based tool is available for compliance checking.

METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII.

PURPOSE: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. METHODS: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. RESULT: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. CONCLUSION: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. CRITICAL RELEVANCE STATEMENT: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. KEY POINTS: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ).