Is PET Radiomics Useful to Predict Pathologic Tumor Response and Prognosis in Locally Advanced Cervical Cancer?

This study investigated whether radiomic features extracted from pretreatment [18F]FDG PET could improve the prediction of both histopathologic tumor response and survival in patients with locally advanced cervical cancer (LACC) treated with neoadjuvant chemoradiotherapy followed by surgery compared with conventional PET parameters and histopathologic features. Methods: The medical records of all consecutive patients with LACC referred between July 2010 and July 2016 were reviewed. [18F]FDG PET/CT was performed before neoadjuvant chemoradiotherapy. Radiomic features were extracted from the primary tumor volumes delineated semiautomatically on the PET images and reduced by factor analysis. A receiver-operating-characteristic analysis was performed, and conventional and radiomic features were dichotomized with Liu's method according to pathologic response (pR) and cancer-specific death. According to the study protocol, only areas under the curve of more than 0.70 were selected for further analysis, including logistic regression analysis for response prediction and Cox regression analysis for survival prediction. Results: A total of 195 patients fulfilled the inclusion criteria. At pathologic evaluation after surgery, 131 patients (67.2%) had no or microscopic (≤3 mm) residual tumor (pR0 or pR1, respectively); 64 patients (32.8%) had macroscopic residual tumor (>3 mm, pR2). With a median follow-up of 76.0 mo (95% CI, 70.7-78.7 mo), 31.3% of patients had recurrence or progression and 20.0% died of the disease. Among conventional PET parameters, SUVmean significantly differed between pathologic responders and nonresponders. Among radiomic features, 1 shape and 3 textural features significantly differed between pathologic responders and nonresponders. Three radiomic features significantly differed between presence and absence of recurrence or progression and between presence and absence of cancer-specific death. Areas under the curve were less than 0.70 for all parameters; thus, univariate and multivariate regression analyses were not performed. Conclusion: In a large series of patients with LACC treated with neoadjuvant chemoradiotherapy followed by surgery, PET radiomic features could not predict histopathologic tumor response and survival. It is crucial to further explore the biologic mechanism underlying imaging-derived parameters and plan a large, prospective, multicenter study with standardized protocols for all phases of the process of radiomic analysis to validate radiomics before its use in clinical routine.

METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII.

PURPOSE: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. METHODS: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. RESULT: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. CONCLUSION: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. CRITICAL RELEVANCE STATEMENT: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. KEY POINTS: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ).

Getting Cartilage Thickness Measurements Right: A Systematic Inter-Method Comparison Using MRI Data from the Osteoarthritis Initiative.

OBJECTIVE: Magnetic resonance imaging is the standard imaging modality to assess articular cartilage. As the imaging surrogate of degenerative joint disease, cartilage thickness is commonly quantified after tissue segmentation. In lack of a standard method, this study systematically compared five methods for automatic cartilage thickness measurements across the knee joint and as a function of region and sub-region: 3D mesh normals (3D-MN), 3D nearest neighbors (3D-NN), 3D ray tracing (3D-RT), 2D centerline normals (2D-CN), and 2D surface normals (2D-SN). DESIGN: Based on the manually segmented femoral and tibial cartilage of 507 human knee joints, mean cartilage thickness was computed for the entire femorotibial joint, 4 joint regions, and 20 subregions using these methods. Inter-method comparisons of mean cartilage thickness and computation times were performed by one-way analysis of variance (ANOVA), Bland-Altman analyses and Lin's concordance correlation coefficient (CCC). RESULTS: Mean inter-method differences in cartilage thickness were significant in nearly all subregions (P < 0.001). By trend, mean differences were smallest between 3D-MN and 2D-SN in most (sub)regions, which is also reflected by highest quantitative inter-method agreement and CCCs. 3D-RT was prone to severe overestimation of up to 2.5 mm. 3D-MN, 3D-NN, and 2D-SN required mean processing times of ≤5.3 s per joint and were thus similarly efficient, whereas the time demand of 2D-CN and 3D-RT was much larger at 133 ± 29 and 351 ± 10 s per joint (P < 0.001). CONCLUSIONS: In automatic cartilage thickness determination, quantification accuracy and computational burden are largely affected by the underlying method. Mesh and surface normals or nearest neighbor searches should be used because they accurately capture variable geometries while being time-efficient.

Convolutional neural networks for automatic image quality control and EARL compliance of PET images.

BACKGROUND: Machine learning studies require a large number of images often obtained on different PET scanners. When merging these images, the use of harmonized images following EARL-standards is essential. However, when including retrospective images, EARL accreditation might not have been in place. The aim of this study was to develop a convolutional neural network (CNN) that can identify retrospectively if an image is EARL compliant and if it is meeting older or newer EARL-standards. MATERIALS AND METHODS: 96 PET images acquired on three PET/CT systems were included in the study. All images were reconstructed with the locally clinically preferred, EARL1, and EARL2 compliant reconstruction protocols. After image pre-processing, one CNN was trained to separate clinical and EARL compliant reconstructions. A second CNN was optimized to identify EARL1 and EARL2 compliant images. The accuracy of both CNNs was assessed using fivefold cross-validation. The CNNs were validated on 24 images acquired on a PET scanner not included in the training data. To assess the impact of image noise on the CNN decision, the 24 images were reconstructed with different scan durations. RESULTS: In the cross-validation, the first CNN classified all images correctly. When identifying EARL1 and EARL2 compliant images, the second CNN identified 100% EARL1 compliant and 85% EARL2 compliant images correctly. The accuracy in the independent dataset was comparable to the cross-validation accuracy. The scan duration had almost no impact on the results. CONCLUSION: The two CNNs trained in this study can be used to retrospectively include images in a multi-center setting by, e.g., adding additional smoothing. This method is especially important for machine learning studies where the harmonization of images from different PET systems is essential.

Mitigation of noise-induced bias of PET radiomic features.

INTRODUCTION: One major challenge in PET radiomics is its sensitivity to noise. Low signal-to-noise ratio (SNR) affects not only the precision but also the accuracy of quantitative metrics extracted from the images resulting in noise-induced bias. This phantom study aims to identify the radiomic features that are robust to noise in terms of precision and accuracy and to explore some methods that might help to correct noise-induced bias. METHODS: A phantom containing three 18F-FDG filled 3D printed inserts, reflecting heterogeneous tracer uptake and realistic tumor shapes, was used in the study. The three different phantom inserts were filled and scanned with three different tumor-to-background ratios, simulating a total of nine different tumors. From the 40-minute list-mode data, ten frames each for 5 s, 10 s, 30 s, and 120 s frame duration were reconstructed to generate images with different noise levels. Under these noise conditions, the precision and accuracy of the radiomic features were analyzed using intraclass correlation coefficient (ICC) and similarity distance metric (SDM) respectively. Based on the ICC and SDM values, the radiomic features were categorized into four groups: poor, moderate, good, and excellent precision and accuracy. A "difference image" created by subtracting two statistically equivalent replicate images was used to develop a model to correct the noise-induced bias. Several regression methods (e.g., linear, exponential, sigmoid, and power-law) were tested. The best fitting model was chosen based on Akaike information criteria. RESULTS: Several radiomic features derived from low SNR images have high repeatability, with 68% of radiomic features having ICC ≥ 0.9 for images with a frame duration of 5 s. However, most features show a systematic bias that correlates with the increase in noise level. Out of 143 features with noise-induced bias, the SDM values were improved based on a regression model (53 features to excellent and 67 to good) indicating that the noise-induced bias of these features can be, at least partially, corrected. CONCLUSION: To have a predictive value, radiomic features should reflect tumor characteristics and be minimally affected by noise. The present study has shown that it is possible to correct for noise-induced bias, at least in a subset of the features, using a regression model based on the local image noise estimates.

Comparing lesion and feature selections to predict progression in newly diagnosed DLBCL patients with FDG PET/CT radiomics features.

PURPOSE: Biomarkers that can accurately predict outcome in DLBCL patients are urgently needed. Radiomics features extracted from baseline [18F]-FDG PET/CT scans have shown promising results. This study aims to investigate which lesion- and feature-selection approaches/methods resulted in the best prediction of progression after 2 years. METHODS: A total of 296 patients were included. 485 radiomics features (n = 5 conventional PET, n = 22 morphology, n = 50 intensity, n = 408 texture) were extracted for all individual lesions and at patient level, where all lesions were aggregated into one VOI. 18 features quantifying dissemination were extracted at patient level. Several lesion selection approaches were tested (largest or hottest lesion, patient level [all with/without dissemination], maximum or median of all lesions) and compared to the predictive value of our previously published model. Several data reduction methods were applied (principal component analysis, recursive feature elimination (RFE), factor analysis, and univariate selection). The predictive value of all models was tested using a fivefold cross-validation approach with 50 repeats with and without oversampling, yielding the mean cross-validated AUC (CV-AUC). Additionally, the relative importance of individual radiomics features was determined. RESULTS: Models with conventional PET and dissemination features showed the highest predictive value (CV-AUC: 0.72-0.75). Dissemination features had the highest relative importance in these models. No lesion selection approach showed significantly higher predictive value compared to our previous model. Oversampling combined with RFE resulted in highest CV-AUCs. CONCLUSION: Regardless of the applied lesion selection or feature selection approach and feature reduction methods, patient level conventional PET features and dissemination features have the highest predictive value. Trial registration number and date: EudraCT: 2006-005174-42, 01-08-2008.

Segmentation Uncertainty Estimation as a Sanity Check for Image Biomarker Studies.

Problem. Image biomarker analysis, also known as radiomics, is a tool for tissue characterization and treatment prognosis that relies on routinely acquired clinical images and delineations. Due to the uncertainty in image acquisition, processing, and segmentation (delineation) protocols, radiomics often lack reproducibility. Radiomics harmonization techniques have been proposed as a solution to reduce these sources of uncertainty and/or their influence on the prognostic model performance. A relevant question is how to estimate the protocol-induced uncertainty of a specific image biomarker, what the effect is on the model performance, and how to optimize the model given the uncertainty. Methods. Two non-small cell lung cancer (NSCLC) cohorts, composed of 421 and 240 patients, respectively, were used for training and testing. Per patient, a Monte Carlo algorithm was used to generate three hundred synthetic contours with a surface dice tolerance measure of less than 1.18 mm with respect to the original GTV. These contours were subsequently used to derive 104 radiomic features, which were ranked on their relative sensitivity to contour perturbation, expressed in the parameter η. The top four (low η) and the bottom four (high η) features were selected for two models based on the Cox proportional hazards model. To investigate the influence of segmentation uncertainty on the prognostic model, we trained and tested the setup in 5000 augmented realizations (using a Monte Carlo sampling method); the log-rank test was used to assess the stratification performance and stability of segmentation uncertainty. Results. Although both low and high η setup showed significant testing set log-rank p-values (p = 0.01) in the original GTV delineations (without segmentation uncertainty introduced), in the model with high uncertainty, to effect ratio, only around 30% of the augmented realizations resulted in model performance with p < 0.05 in the test set. In contrast, the low η setup performed with a log-rank p < 0.05 in 90% of the augmented realizations. Moreover, the high η setup classification was uncertain in its predictions for 50% of the subjects in the testing set (for 80% agreement rate), whereas the low η setup was uncertain only in 10% of the cases. Discussion. Estimating image biomarker model performance based only on the original GTV segmentation, without considering segmentation, uncertainty may be deceiving. The model might result in a significant stratification performance, but can be unstable for delineation variations, which are inherent to manual segmentation. Simulating segmentation uncertainty using the method described allows for more stable image biomarker estimation, selection, and model development. The segmentation uncertainty estimation method described here is universal and can be extended to estimate other protocol uncertainties (such as image acquisition and pre-processing).

Noise sensitivity of 89Zr-Immuno-PET radiomics based on count-reduced clinical images.

PURPOSE: Low photon count in 89Zr-Immuno-PET results in images with a low signal-to-noise ratio (SNR). Since PET radiomics are sensitive to noise, this study focuses on the impact of noise on radiomic features from 89Zr-Immuno-PET clinical images. We hypothesise that 89Zr-Immuno-PET derived radiomic features have: (1) noise-induced variability affecting their precision and (2) noise-induced bias affecting their accuracy. This study aims to identify those features that are not or only minimally affected by noise in terms of precision and accuracy. METHODS: Count-split 89Zr-Immuno-PET patient scans from previous studies with three different 89Zr-labelled monoclonal antibodies were used to extract radiomic features at 50% (S50p) and 25% (S25p) of their original counts. Tumour lesions were manually delineated on the original full-count 89Zr-Immuno-PET scans. Noise-induced variability and bias were assessed using intraclass correlation coefficient (ICC) and similarity distance metric (SDM), respectively. Based on the ICC and SDM values, the radiomic features were categorised as having poor [0, 0.5), moderate [0.5, 0.75), good [0.75, 0.9), or excellent [0.9, 1] precision and accuracy. The number of features classified into these categories was compared between the S50p and S25p images using Fisher's exact test. All p values < 0.01 were considered statistically significant. RESULTS: For S50p, a total of 92% and 90% features were classified as having good or excellent ICC and SDM respectively, while for S25p, these decreased to 81% and 31%. In total, 148 features (31%) showed robustness to noise with good or moderate ICC and SDM in both S50p and S25p. The number of features classified into the four ICC and SDM categories between S50p and S25p was significantly different statistically. CONCLUSION: Several radiomic features derived from low SNR 89Zr-Immuno-PET images exhibit noise-induced variability and/or bias. However, 196 features (43%) that show minimal noise-induced variability and bias in S50p images have been identified. These features are less affected by noise and are, therefore, suitable candidates to be further studied as prognostic and predictive quantitative biomarkers in 89Zr-Immuno-PET studies.

Quantitative Radiomics Features in Diffuse Large B-Cell Lymphoma: Does Segmentation Method Matter?

Radiomics features may predict outcome in diffuse large B-cell lymphoma (DLBCL). Currently, multiple segmentation methods are used to calculate metabolic tumor volume (MTV). We assessed the influence of segmentation method on the discriminative power of radiomics features in DLBCL at the patient level and for the largest lesion. Methods: Fifty baseline 18F-FDG PET/CT scans of DLBCL patients with progression or relapse within 2 years after diagnosis were matched on uptake time and reconstruction method with 50 baseline PET/CT scans of DLBCL patients without progression. Scans were analyzed using 6 semiautomatic segmentation methods (SUV threshold of 4.0 [SUV4.0], SUV threshold of 2.5, 41% of SUVmax, 50% of SUVpeak, a majority vote segmenting voxels detected by ≥2 methods, and a majority vote segmenting voxels detected by ≥3 methods). On the basis of these segmentations, 490 radiomics features were extracted at the patient level, and 486 features were extracted for the largest lesion. To quantify the agreement between features extracted from different segmentation methods, the intraclass correlation (ICC) agreement was calculated for each method compared with SUV4.0. The feature space was reduced by deleting features that had high Pearson correlations (≥0.7) with the previously established predictors MTV or SUVpeak Model performance was assessed using stratified repeated cross validation with 5 folds and 2,000 repeats, yielding the mean receiver-operating-characteristics curve integral for all segmentation methods using logistic regression with backward feature selection. Results: The percentage of features yielding an ICC of at least 0.75, compared with the SUV4.0 segmentation, was lowest for 50% of SUVpeak both at the patient level and for the largest lesion, with 77.3% and 66.7% of the features yielding an ICC of at least 0.75, respectively. Features did not correlate strongly with MTV, with at least 435 features at the patient level and 409 features for the largest lesion for all segmentation methods having a correlation coefficient of less than 0.7. Features correlated strongly with SUVpeak (at least 190 at patient level and 134 for the largest lesion were uncorrelated to SUVpeak, respectively). Receiver-operating-characteristics curve integrals ranged between 0.69 ± 0.11 and 0.84 ± 0.09 at the patient level and between 0.69 ± 0.11 and 0.73 ± 0.10 at the lesion level. Conclusion: Even though there are differences in the actual radiomics feature values derived and selected features among segmentation methods, there is no substantial difference in the discriminative power of radiomics features among segmentation methods.

18F-FDG PET baseline radiomics features improve the prediction of treatment outcome in diffuse large B-cell lymphoma.

PURPOSE: Accurate prognostic markers are urgently needed to identify diffuse large B-Cell lymphoma (DLBCL) patients at high risk of progression or relapse. Our purpose was to investigate the potential added value of baseline radiomics features to the international prognostic index (IPI) in predicting outcome after first-line treatment. METHODS: Three hundred seventeen newly diagnosed DLBCL patients were included. Lesions were delineated using a semi-automated segmentation method (standardized uptake value ≥ 4.0), and 490 radiomics features were extracted. We used logistic regression with backward feature selection to predict 2-year time to progression (TTP). The area under the curve (AUC) of the receiver operator characteristic curve was calculated to assess model performance. High-risk groups were defined based on prevalence of events; diagnostic performance was assessed using positive and negative predictive values. RESULTS: The IPI model yielded an AUC of 0.68. The optimal radiomics model comprised the natural logarithms of metabolic tumor volume (MTV) and of SUVpeak and the maximal distance between the largest lesion and any other lesion (Dmaxbulk, AUC 0.76). Combining radiomics and clinical features showed that a combination of tumor- (MTV, SUVpeak and Dmaxbulk) and patient-related parameters (WHO performance status and age > 60 years) performed best (AUC 0.79). Adding radiomics features to clinical predictors increased PPV with 15%, with more accurate selection of high-risk patients compared to the IPI model (progression at 2-year TTP, 44% vs 28%, respectively). CONCLUSION: Prediction models using baseline radiomics combined with currently used clinical predictors identify patients at risk of relapse at baseline and significantly improved model performance. TRIAL REGISTRATION NUMBER AND DATE: EudraCT: 2006-005,174-42, 01-08-2008.

A systematic review and quality of reporting checklist for repeatability and reproducibility of radiomic features.

PURPOSE: Although quantitative image biomarkers (radiomics) show promising value for cancer diagnosis, prognosis, and treatment assessment, these biomarkers still lack reproducibility. In this systematic review, we aimed to assess the progress in radiomics reproducibility and repeatability in the recent years. METHODS AND MATERIALS: Four hundred fifty-one abstracts were retrieved according to the original PubMed search pattern with the publication dates ranging from 2017/05/01 to 2020/12/01. Each abstract including the keywords was independently screened by four observers. Forty-two full-text articles were selected for further analysis. Patient population data, radiomic feature classes, feature extraction software, image preprocessing, and reproducibility results were extracted from each article. To support the community with a standardized reporting strategy, we propose a specific reporting checklist to evaluate the feasibility to reproduce each study. RESULTS: Many studies continue to under-report essential reproducibility information: all but one clinical and all but two phantom studies missed to report at least one important item reporting image acquisition. The studies included in this review indicate that all radiomic features are sensitive to image acquisition, reconstruction, tumor segmentation, and interpolation. However, the amount of sensitivity is feature dependent, for instance, textural features were, in general, less robust than statistical features. CONCLUSIONS: Radiomics repeatability, reproducibility, and reporting quality can substantially be improved regarding feature extraction software and settings, image preprocessing and acquisition, cutoff values for stable feature selection. Our proposed radiomics reporting checklist can serve to simplify and improve the reporting and, eventually, guarantee the possibility to fully replicate and validate radiomic studies.

Repeatability of two semi-automatic artificial intelligence approaches for tumor segmentation in PET.

BACKGROUND: Positron emission tomography (PET) is routinely used for cancer staging and treatment follow-up. Metabolic active tumor volume (MATV) as well as total MATV (TMATV-including primary tumor, lymph nodes and metastasis) and/or total lesion glycolysis derived from PET images have been identified as prognostic factor or for the evaluation of treatment efficacy in cancer patients. To this end, a segmentation approach with high precision and repeatability is important. However, the implementation of a repeatable and accurate segmentation algorithm remains an ongoing challenge. METHODS: In this study, we compare two semi-automatic artificial intelligence (AI)-based segmentation methods with conventional semi-automatic segmentation approaches in terms of repeatability. One segmentation approach is based on a textural feature (TF) segmentation approach designed for accurate and repeatable segmentation of primary tumors and metastasis. Moreover, a convolutional neural network (CNN) is trained. The algorithms are trained, validated and tested using a lung cancer PET dataset. The segmentation accuracy of both segmentation approaches is compared using the Jaccard coefficient (JC). Additionally, the approaches are externally tested on a fully independent test-retest dataset. The repeatability of the methods is compared with those of two majority vote (MV2, MV3) approaches, 41%SUVMAX, and a SUV > 4 segmentation (SUV4). Repeatability is assessed with test-retest coefficients (TRT%) and intraclass correlation coefficient (ICC). An ICC > 0.9 was regarded as representing excellent repeatability. RESULTS: The accuracy of the segmentations with the reference segmentation was good (JC median TF: 0.7, CNN: 0.73). Both segmentation approaches outperformed most other conventional segmentation methods in terms of test-retest coefficient (TRT% mean: TF: 13.0%, CNN: 13.9%, MV2: 14.1%, MV3: 28.1%, 41%SUVMAX: 28.1%, SUV4: 18.1%) and ICC (TF: 0.98, MV2: 0.97, CNN: 0.99, MV3: 0.73, SUV4: 0.81, and 41%SUVMAX: 0.68). CONCLUSION: The semi-automatic AI-based segmentation approaches used in this study provided better repeatability than conventional segmentation approaches. Moreover, both algorithms lead to accurate segmentations for both primary tumors as well as metastasis and are therefore good candidates for PET tumor segmentation.

Machine learning-based analysis of [18F]DCFPyL PET radiomics for risk stratification in primary prostate cancer.

PURPOSE: Quantitative prostate-specific membrane antigen (PSMA) PET analysis may provide for non-invasive and objective risk stratification of primary prostate cancer (PCa) patients. We determined the ability of machine learning-based analysis of quantitative [18F]DCFPyL PET metrics to predict metastatic disease or high-risk pathological tumor features. METHODS: In a prospective cohort study, 76 patients with intermediate- to high-risk PCa scheduled for robot-assisted radical prostatectomy with extended pelvic lymph node dissection underwent pre-operative [18F]DCFPyL PET-CT. Primary tumors were delineated using 50-70% peak isocontour thresholds on images with and without partial-volume correction (PVC). Four hundred and eighty standardized radiomic features were extracted per tumor. Random forest models were trained to predict lymph node involvement (LNI), presence of any metastasis, Gleason score ≥ 8, and presence of extracapsular extension (ECE). For comparison, models were also trained using standard PET features (SUVs, volume, total PSMA uptake). Model performance was validated using 50 times repeated 5-fold cross-validation yielding the mean receiver-operator characteristic curve AUC. RESULTS: The radiomics-based machine learning models predicted LNI (AUC 0.86 ± 0.15, p < 0.01), nodal or distant metastasis (AUC 0.86 ± 0.14, p < 0.01), Gleason score (0.81 ± 0.16, p < 0.01), and ECE (0.76 ± 0.12, p < 0.01). The highest AUCs reached using standard PET metrics were lower than those of radiomics-based models. For LNI and metastasis prediction, PVC and a higher delineation threshold improved model stability. Machine learning pre-processing methods had a minor impact on model performance. CONCLUSION: Machine learning-based analysis of quantitative [18F]DCFPyL PET metrics can predict LNI and high-risk pathological tumor features in primary PCa patients. These findings indicate that PSMA expression detected on PET is related to both primary tumor histopathology and metastatic tendency. Multicenter external validation is needed to determine the benefits of using radiomics versus standard PET metrics in clinical practice.

Plausibility and redundancy analysis to select FDG-PET textural features in non-small cell lung cancer.

BACKGROUND: Radiomics refers to the extraction of a large number of image biomarker describing the tumor phenotype displayed in a medical image. Extracted from positron emission tomography (PET) images, radiomics showed diagnostic and prognostic value for several cancer types. However, a large number of radiomic features are nonreproducible or highly correlated with conventional PET metrics. Moreover, radiomic features used in the clinic should yield relevant information about tumor texture. In this study, we propose a framework to identify technical and clinical meaningful features and exemplify our results using a PET non-small cell lung cancer (NSCLC) dataset. MATERIALS AND METHODS: The proposed selection procedure consists of several steps. A priori, we only include features that were found to be reproducible in a multicenter setting. Next, we apply a voxel randomization step to identify features that reflect actual textural information, that is, that yield in 90% of the patient scans a value significantly different from random texture. Finally, the remaining features were correlated with standard PET metrics to further remove redundancy with common PET metrics. The selection procedure was performed for different volume ranges, that is, excluding lesions with smaller volumes in order to assess the effect of tumor size on the results. To exemplify our procedure, the selected features were used to predict 1-yr survival in a dataset of 150 NSCLC patients. A predictive model was built using volume as predictive factor for smaller, and one of the selected features as predictive factor for bigger lesions. The prediction accuracy of the both models were compared with the prediction accuracy of volume. RESULTS: The number of selected features depended on the lesion size included in the analysis. When including the whole dataset, from 19 features reflecting actual texture only two were found to be not strongly correlated with conventional PET metrics. When excluding lesions smaller than 11.49 and 33.10 mL (25 and 50 percentile of the dataset), four out of 27 features and 13 out of 29 features remained after eliminating features highly correlated with standard PET metrics. When excluding lesions smaller than 103.9 mL (75 percentile), 33 out of 53 features remained. For larger lesions, some of these features outperformed volume in terms of classification accuracy (increase of 4-10%). The combination of using volume as predictor for smaller and one of the selected features for larger lesions also improved the accuracy when compared with volume only (increase from 72% to 76%). CONCLUSION: When performing radiomic analysis for smaller lesions, it should be first carefully investigated if a textural feature reflects actual heterogeneity information. Next, verification of the absence of correlation with all conventional PET metrics is essential in order to assess the additional value of radiomic features. Radiomic analysis with lesions larger than 11.4 mL might give additional information to conventional metrics while at the same time reflecting actual tumor texture. Using a combination of volume and one of the selected features for prediction yields promise to increase accuracy and reliability of a radiomic model.

Predictive value of quantitative 18F-FDG-PET radiomics analysis in patients with head and neck squamous cell carcinoma.

BACKGROUND: Radiomics is aimed at image-based tumor phenotyping, enabling application within clinical-decision-support-systems to improve diagnostic accuracy and allow for personalized treatment. The purpose was to identify predictive 18-fluor-fluoro-2-deoxyglucose (18F-FDG) positron-emission tomography (PET) radiomic features to predict recurrence, distant metastasis, and overall survival in patients with head and neck squamous cell carcinoma treated with chemoradiotherapy. METHODS: Between 2012 and 2018, 103 retrospectively (training cohort) and 71 consecutively included patients (validation cohort) underwent 18F-FDG-PET/CT imaging. The 434 extracted radiomic features were subjected, after redundancy filtering, to a projection resulting in outcome-independent meta-features (factors). Correlations between clinical, first-order 18F-FDG-PET parameters (e.g., SUVmean), and factors were assessed. Factors were combined with 18F-FDG-PET and clinical parameters in a multivariable survival regression and validated. A clinically applicable risk-stratification was constructed for patients' outcome. RESULTS: Based on 124 retained radiomic features from 103 patients, 8 factors were constructed. Recurrence prediction was significantly most accurate by combining HPV-status, SUVmean, SUVpeak, factor 3 (histogram gradient and long-run-low-grey-level-emphasis), factor 4 (volume-difference, coarseness, and grey-level-non-uniformity), and factor 6 (histogram variation coefficient) (CI = 0.645). Distant metastasis prediction was most accurate assessing metabolic-active tumor volume (MATV)(CI = 0.627). Overall survival prediction was most accurate using HPV-status, SUVmean, SUVmax, factor 1 (least-axis-length, non-uniformity, high-dependence-of-high grey-levels), and factor 5 (aspherity, major-axis-length, inversed-compactness and, inversed-flatness) (CI = 0.764). CONCLUSIONS: Combining HPV-status, first-order 18F-FDG-PET parameters, and complementary radiomic factors was most accurate for time-to-event prediction. Predictive phenotype-specific tumor characteristics and interactions might be captured and retained using radiomic factors, which allows for personalized risk stratification and optimizing personalized cancer care. TRIAL REGISTRATION: Trial NL3946 (NTR4111), local ethics commission reference: Prediction 2013.191 and 2016.498. Registered 7 August 2013, https://www.trialregister.nl/trial/3946.

PET segmentation of bulky tumors: Strategies and workflows to improve inter-observer variability.

BACKGROUND: PET-based tumor delineation is an error prone and labor intensive part of image analysis. Especially for patients with advanced disease showing bulky tumor FDG load, segmentations are challenging. Reducing the amount of user-interaction in the segmentation might help to facilitate segmentation tasks especially when labeling bulky and complex tumors. Therefore, this study reports on segmentation workflows/strategies that may reduce the inter-observer variability for large tumors with complex shapes with different levels of user-interaction. METHODS: Twenty PET images of bulky tumors were delineated independently by six observers using four strategies: (I) manual, (II) interactive threshold-based, (III) interactive threshold-based segmentation with the additional presentation of the PET-gradient image and (IV) the selection of the most reasonable result out of four established semi-automatic segmentation algorithms (Select-the-best approach). The segmentations were compared using Jaccard coefficients (JC) and percentage volume differences. To obtain a reference standard, a majority vote (MV) segmentation was calculated including all segmentations of experienced observers. Performed and MV segmentations were compared regarding positive predictive value (PPV), sensitivity (SE), and percentage volume differences. RESULTS: The results show that with decreasing user-interaction the inter-observer variability decreases. JC values and percentage volume differences of Select-the-best and a workflow including gradient information were significantly better than the measurements of the other segmentation strategies (p-value<0.01). Interactive threshold-based and manual segmentations also result in significant lower and more variable PPV/SE values when compared with the MV segmentation. CONCLUSIONS: FDG PET segmentations of bulky tumors using strategies with lower user-interaction showed less inter-observer variability. None of the methods led to good results in all cases, but use of either the gradient or the Select-the-best workflow did outperform the other strategies tested and may be a good candidate for fast and reliable labeling of bulky and heterogeneous tumors.

The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping.

Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpose To standardize a set of 174 radiomic features. Materials and Methods Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Results Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). Conclusion A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Kuhl and Truhn in this issue.

Experimental Multicenter and Multivendor Evaluation of the Performance of PET Radiomic Features Using 3-Dimensionally Printed Phantom Inserts.

The sensitivity of radiomic features to several confounding factors, such as reconstruction settings, makes clinical use challenging. To investigate the impact of harmonized image reconstructions on feature consistency, a multicenter phantom study was performed using 3-dimensionally printed phantom inserts reflecting realistic tumor shapes and heterogeneity uptakes. Methods: Tumors extracted from real PET/CT scans of patients with non-small cell lung cancer served as model for three 3-dimensionally printed inserts. Different heterogeneity pattern were realized by printing separate compartments that could be filled with different activity solutions. The inserts were placed in the National Electrical Manufacturers Association image-quality phantom and scanned various times. First, a list-mode scan was acquired and 5 statistically equal replicates were reconstructed. Second, the phantom was scanned 4 times on the same scanner. Third, the phantom was scanned on 6 PET/CT systems. All images were reconstructed using EANM Research Ltd. (EARL)-compliant and locally clinically preferred reconstructions. EARL-compliant reconstructions were performed without (EARL1) or with (EARL2) point-spread function. Images were analyzed with and without resampling to 2-mm cubic voxels. Images were discretized with a fixed bin width (FBW) of 0.25 and a fixed bin number (FBN) of 64. The intraclass correlation coefficient (ICC) of each scan setup was calculated and compared across reconstruction settings. An ICC above 0.75 was regarded as high. Results: The percentage of features yielding a high ICC was largest for the statistically equal replicates (70%-91% for FBN; 90%-96% for FBW discretization). For scans acquired on the same system, the percentage decreased, but most features still resulted in a high ICC (FBN, 52%-63%; FBW, 75%-85%). The percentage of features yielding a high ICC decreased more in the multicenter setting. In this case, the percentage of features yielding a high ICC was larger for images reconstructed with EARL-compliant reconstructions: for example, 40% for EARL1 and 60% for EARL2 versus 21% for the clinically preferred setting for FBW discretization. When discretized with FBW and resampled to isotropic voxels, this benefit was more pronounced. Conclusion: EARL-compliant reconstructions harmonize a wide range of radiomic features. FBW discretization and a sampling to isotropic voxels enhances the benefits of EARL-compliant reconstructions.

RaCaT: An open source and easy to use radiomics calculator tool.

PURPOSE: The widely known field 'Radiomics' aims to provide an extensive image based phenotyping of e.g. tumors using a wide variety of feature values extracted from medical images. Therefore, it is of utmost importance that feature values calculated by different institutes follow the same feature definitions. For this purpose, the imaging biomarker standardization initiative (IBSI) provides detailed mathematical feature descriptions, as well as (mathematical) test phantoms and corresponding reference feature values. We present here an easy to use radiomic feature calculator, RaCaT, which provides the calculation of a large number of radiomic features for all kind of medical images which are in compliance with the standard. METHODS: The calculator is implemented in C++ and comes as a standalone executable. Therefore, it can be easily integrated in any programming language, but can also be called from the command line. No programming skills are required to use the calculator. The software architecture is highly modularized so that it is easily extendible. The user can also download the source code, adapt it if needed and build the calculator from source. The calculated feature values are compliant with the ones provided by the IBSI standard. Source code, example files for the software configuration, and documentation can be found online on GitHub (https://github.com/ellipfaehlerUMCG/RaCat). RESULTS: The comparison with the standard values shows that all calculated features as well as image preprocessing steps, comply with the IBSI standard. The performance is also demonstrated on clinical examples. CONCLUSIONS: The authors successfully implemented an easy to use Radiomics calculator that can be called from any programming language or from the command line. Image preprocessing and feature settings and calculations can be adjusted by the user.

Multicenter CT phantoms public dataset for radiomics reproducibility tests.

PURPOSE: The aim of this paper is to describe a public, open-access, computed tomography (CT) phantom image set acquired at three centers and collected especially for radiomics reproducibility research. The dataset is useful to test radiomic features reproducibility with respect to various parameters, such as acquisition settings, scanners, and reconstruction algorithms. ACQUISITION AND VALIDATION METHODS: Three phantoms were scanned in three independent institutions. Images of the following phantoms were acquired: Catphan 700 and COPDGene Phantom II (Phantom Laboratory, Greenwich, NY, USA), and the Triple modality 3D Abdominal Phantom (CIRS, Norfolk, VA, USA). Data were collected at three Dutch medical centers: MAASTRO Clinic (Maastricht, NL), Radboud University Medical Center (Nijmegen, NL), and University Medical Center Groningen (Groningen, NL) with scanners from two different manufacturers Siemens Healthcare and Philips Healthcare. The following acquisition parameter were varied in the phantom scans: slice thickness, reconstruction kernels, and tube current. DATA FORMAT AND USAGE NOTES: We made the dataset publically available on the Dutch instance of "Extensible Neuroimaging Archive Toolkit-XNAT" (https://xnat.bmia.nl). The dataset is freely available and reusable with attribution (Creative Commons 3.0 license). POTENTIAL APPLICATIONS: Our goal was to provide a findable, open-access, annotated, and reusable CT phantom dataset for radiomics reproducibility studies. Reproducibility testing and harmonization are fundamental requirements for wide generalizability of radiomics-based clinical prediction models. It is highly desirable to include only reproducible features into models, to be more assured of external validity across hitherto unseen contexts. In this view, phantom data from different centers represent a valuable source of information to exclude CT radiomic features that may already be unstable with respect to simplified structures and tightly controlled scan settings. The intended extension of our shared dataset is to include other modalities and phantoms with more realistic lesion simulations.