The Image Biomarker Standardization Initiative: Standardized Convolutional Filters for Reproducible Radiomics and Enhanced Clinical Insights.

Filters are commonly used to enhance specific structures and patterns in images, such as vessels or peritumoral regions, to enable clinical insights beyond the visible image using radiomics. However, their lack of standardization restricts reproducibility and clinical translation of radiomics decision support tools. In this special report, teams of researchers who developed radiomics software participated in a three-phase study (September 2020 to December 2022) to establish a standardized set of filters. The first two phases focused on finding reference filtered images and reference feature values for commonly used convolutional filters: mean, Laplacian of Gaussian, Laws and Gabor kernels, separable and nonseparable wavelets (including decomposed forms), and Riesz transformations. In the first phase, 15 teams used digital phantoms to establish 33 reference filtered images of 36 filter configurations. In phase 2, 11 teams used a chest CT image to derive reference values for 323 of 396 features computed from filtered images using 22 filter and image processing configurations. Reference filtered images and feature values for Riesz transformations were not established. Reproducibility of standardized convolutional filters was validated on a public data set of multimodal imaging (CT, fluorodeoxyglucose PET, and T1-weighted MRI) in 51 patients with soft-tissue sarcoma. At validation, reproducibility of 486 features computed from filtered images using nine configurations × three imaging modalities was assessed using the lower bounds of 95% CIs of intraclass correlation coefficients. Out of 486 features, 458 were found to be reproducible across nine teams with lower bounds of 95% CIs of intraclass correlation coefficients greater than 0.75. In conclusion, eight filter types were standardized with reference filtered images and reference feature values for verifying and calibrating radiomics software packages. A web-based tool is available for compliance checking.

METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII.

PURPOSE: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies. METHODS: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated. RESULT: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community. CONCLUSION: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers. CRITICAL RELEVANCE STATEMENT: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning. KEY POINTS: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ).

Automatic Head and Neck Tumor segmentation and outcome prediction relying on FDG-PET/CT images: Findings from the second edition of the HECKTOR challenge.

By focusing on metabolic and morphological tissue properties respectively, FluoroDeoxyGlucose (FDG)-Positron Emission Tomography (PET) and Computed Tomography (CT) modalities include complementary and synergistic information for cancerous lesion delineation and characterization (e.g. for outcome prediction), in addition to usual clinical variables. This is especially true in Head and Neck Cancer (HNC). The goal of the HEad and neCK TumOR segmentation and outcome prediction (HECKTOR) challenge was to develop and compare modern image analysis methods to best extract and leverage this information automatically. We present here the post-analysis of HECKTOR 2nd edition, at the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2021. The scope of the challenge was substantially expanded compared to the first edition, by providing a larger population (adding patients from a new clinical center) and proposing an additional task to the challengers, namely the prediction of Progression-Free Survival (PFS). To this end, the participants were given access to a training set of 224 cases from 5 different centers, each with a pre-treatment FDG-PET/CT scan and clinical variables. Their methods were subsequently evaluated on a held-out test set of 101 cases from two centers. For the segmentation task (Task 1), the ranking was based on a Borda counting of their ranks according to two metrics: mean Dice Similarity Coefficient (DSC) and median Hausdorff Distance at 95th percentile (HD95). For the PFS prediction task, challengers could use the tumor contours provided by experts (Task 3) or rely on their own (Task 2). The ranking was obtained according to the Concordance index (C-index) calculated on the predicted risk scores. A total of 103 teams registered for the challenge, for a total of 448 submissions and 29 papers. The best method in the segmentation task obtained an average DSC of 0.759, and the best predictions of PFS obtained a C-index of 0.717 (without relying on the provided contours) and 0.698 (using the expert contours). An interesting finding was that best PFS predictions were reached by relying on DL approaches (with or without explicit tumor segmentation, 4 out of the 5 best ranked) compared to standard radiomics methods using handcrafted features extracted from delineated tumors, and by exploiting alternative tumor contours (automated and/or larger volumes encompassing surrounding tissues) rather than relying on the expert contours. This second edition of the challenge confirmed the promising performance of fully automated primary tumor delineation in PET/CT images of HNC patients, although there is still a margin for improvement in some difficult cases. For the first time, the prediction of outcome was also addressed and the best methods reached relatively good performance (C-index above 0.7). Both results constitute another step forward toward large-scale outcome prediction studies in HNC.

Overview of the HECKTOR Challenge at MICCAI 2022: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT.

This paper presents an overview of the third edition of the HEad and neCK TumOR segmentation and outcome prediction (HECKTOR) challenge, organized as a satellite event of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2022. The challenge comprises two tasks related to the automatic analysis of FDG-PET/CT images for patients with Head and Neck cancer (H&N), focusing on the oropharynx region. Task 1 is the fully automatic segmentation of H&N primary Gross Tumor Volume (GTVp) and metastatic lymph nodes (GTVn) from FDG-PET/CT images. Task 2 is the fully automatic prediction of Recurrence-Free Survival (RFS) from the same FDG-PET/CT and clinical data. The data were collected from nine centers for a total of 883 cases consisting of FDG-PET/CT images and clinical information, split into 524 training and 359 test cases. The best methods obtained an aggregated Dice Similarity Coefficient (DSCagg) of 0.788 in Task 1, and a Concordance index (C-index) of 0.682 in Task 2.

Predicting Adverse Radiation Effects in Brain Tumors After Stereotactic Radiotherapy With Deep Learning and Handcrafted Radiomics.

Introduction: There is a cumulative risk of 20-40% of developing brain metastases (BM) in solid cancers. Stereotactic radiotherapy (SRT) enables the application of high focal doses of radiation to a volume and is often used for BM treatment. However, SRT can cause adverse radiation effects (ARE), such as radiation necrosis, which sometimes cause irreversible damage to the brain. It is therefore of clinical interest to identify patients at a high risk of developing ARE. We hypothesized that models trained with radiomics features, deep learning (DL) features, and patient characteristics or their combination can predict ARE risk in patients with BM before SRT. Methods: Gadolinium-enhanced T1-weighted MRIs and characteristics from patients treated with SRT for BM were collected for a training and testing cohort (N = 1,404) and a validation cohort (N = 237) from a separate institute. From each lesion in the training set, radiomics features were extracted and used to train an extreme gradient boosting (XGBoost) model. A DL model was trained on the same cohort to make a separate prediction and to extract the last layer of features. Different models using XGBoost were built using only radiomics features, DL features, and patient characteristics or a combination of them. Evaluation was performed using the area under the curve (AUC) of the receiver operating characteristic curve on the external dataset. Predictions for individual lesions and per patient developing ARE were investigated. Results: The best-performing XGBoost model on a lesion level was trained on a combination of radiomics features and DL features (AUC of 0.71 and recall of 0.80). On a patient level, a combination of radiomics features, DL features, and patient characteristics obtained the best performance (AUC of 0.72 and recall of 0.84). The DL model achieved an AUC of 0.64 and recall of 0.85 per lesion and an AUC of 0.70 and recall of 0.60 per patient. Conclusion: Machine learning models built on radiomics features and DL features extracted from BM combined with patient characteristics show potential to predict ARE at the patient and lesion levels. These models could be used in clinical decision making, informing patients on their risk of ARE and allowing physicians to opt for different therapies.

Development and Validation of Multiparametric MRI-based Radiomics Models for Preoperative Risk Stratification of Endometrial Cancer.

Background Stratifying high-risk histopathologic features in endometrial carcinoma is important for treatment planning. Radiomics analysis at preoperative MRI holds potential to identify high-risk phenotypes. Purpose To evaluate the performance of multiparametric MRI three-dimensional radiomics-based machine learning models for differentiating low- from high-risk histopathologic markers-deep myometrial invasion (MI), lymphovascular space invasion (LVSI), and high-grade status-and advanced-stage endometrial carcinoma. Materials and Methods This dual-center retrospective study included women with histologically proven endometrial carcinoma who underwent 1.5-T MRI before hysterectomy between January 2011 and July 2015. Exclusion criteria were tumor diameter less than 1 cm, missing MRI sequences or histopathology reports, neoadjuvant therapy, and malignant neoplasms other than endometrial carcinoma. Three-dimensional radiomics features were extracted after tumor segmentation at MRI (T2-weighted, diffusion-weighted, and dynamic contrast-enhanced MRI). Predictive features were selected in the training set with use of random forest (RF) models for each end point, and trained RF models were applied to the external test set. Five board-certified radiologists conducted MRI-based staging and deep MI assessment in the training set. Areas under the receiver operating characteristic curve (AUCs) were reported with balanced accuracies, and radiologists' readings were compared with radiomics with use of McNemar tests. Results In total, 157 women were included: 94 at the first institution (training set; mean age, 66 years ± 11 [SD]) and 63 at the second institution (test set; 67 years ± 12). RF models dichotomizing deep MI, LVSI, high grade, and International Federation of Gynecology and Obstetrics (FIGO) stage led to AUCs of 0.81 (95% CI: 0.68, 0.88), 0.80 (95% CI: 0.67, 0.93), 0.74 (95% CI: 0.61, 0.86), and 0.84 (95% CI: 0.72, 0.92), respectively, in the test set. In the training set, radiomics provided increased performance compared with radiologists' readings for identifying deep MI (balanced accuracy, 86% vs 79%; P = .03), while no evidence of a difference was observed in performance for advanced FIGO stage (80% vs 78%; P = .27). Conclusion Three-dimensional radiomics can stratify patients by using preoperative MRI according to high-risk histopathologic end points in endometrial carcinoma and provide nonsignificantly different or higher performance than radiologists in identifying advanced stage and deep myometrial invasion, respectively. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Kido and Nishio in this issue.

Cleaning radiotherapy contours for radiomics studies, is it worth it? A head and neck cancer study.

A vast majority of studies in the radiomics field are based on contours originating from radiotherapy planning. This kind of delineation (e.g. Gross Tumor Volume, GTV) is often larger than the true tumoral volume, sometimes including parts of other organs (e.g. trachea in Head and Neck, H&N studies) and the impact of such over-segmentation was little investigated so far. In this paper, we propose to evaluate and compare the performance between models using two contour types: those from radiotherapy planning, and those specifically delineated for radiomics studies. For the latter, we modified the radiotherapy contours to fit the true tumoral volume. The two contour types were compared when predicting Progression-Free Survival (PFS) using Cox models based on radiomics features extracted from FluoroDeoxyGlucose-Positron Emission Tomography (FDG-PET) and CT images of 239 patients with oropharyngeal H&N cancer collected from five centers, the data from the 2020 HECKTOR challenge. Using Dedicated contours demonstrated better performance for predicting PFS, where Harell's concordance indices of 0.61 and 0.69 were achieved for Radiotherapy and Dedicated contours, respectively. Using automatically Resegmented contours based on a fixed intensity range was associated with a C-index of 0.63. These results illustrate the importance of using clean dedicated contours that are close to the true tumoral volume in radiomics studies, even when tumor contours are already available from radiotherapy treatment planning.

Head and neck tumor segmentation in PET/CT: The HECKTOR challenge.

This paper relates the post-analysis of the first edition of the HEad and neCK TumOR (HECKTOR) challenge. This challenge was held as a satellite event of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2020, and was the first of its kind focusing on lesion segmentation in combined FDG-PET and CT image modalities. The challenge's task is the automatic segmentation of the Gross Tumor Volume (GTV) of Head and Neck (H&N) oropharyngeal primary tumors in FDG-PET/CT images. To this end, the participants were given a training set of 201 cases from four different centers and their methods were tested on a held-out set of 53 cases from a fifth center. The methods were ranked according to the Dice Score Coefficient (DSC) averaged across all test cases. An additional inter-observer agreement study was organized to assess the difficulty of the task from a human perspective. 64 teams registered to the challenge, among which 10 provided a paper detailing their approach. The best method obtained an average DSC of 0.7591, showing a large improvement over our proposed baseline method and the inter-observer agreement, associated with DSCs of 0.6610 and 0.61, respectively. The automatic methods proved to successfully leverage the wealth of metabolic and structural properties of combined PET and CT modalities, significantly outperforming human inter-observer agreement level, semi-automatic thresholding based on PET images as well as other single modality-based methods. This promising performance is one step forward towards large-scale radiomics studies in H&N cancer, obviating the need for error-prone and time-consuming manual delineation of GTVs.

Patient-specific microdosimetry: a proof of concept.

Microscopic energy deposition distributions from ionizing radiation vary depending on biological target size and are used to predict the biological effects of an irradiation. Ionizing radiation is thought to kill cells or inhibit the cell cycle mainly by damaging DNA in the cell nucleus. The size of cells and nuclei depends on tissue type, cell cycle, and malignancy, all of which vary between patients. The aim of this study was to develop methods to perform patient-specific microdosimetry, that being, determining microdosimetric quantities in volumes that correspond to the sizes of cells and nuclei observed in a patient's tissue. A histopathological sample extracted from a stage I lung adenocarcinoma patient was analyzed. A pouring simulation was used to generate a three-dimensional tissue model from cell and nucleus size information determined from the histopathological sample. Microdosimetric distributions includingf(y)andd(y)were determined forC60o,I192r,Y169bandI125in a patient-specific model containing a distribution of cell and nucleus sizes. Fixed radius models and a summation method were compared to the full patient-specific model to evaluate their suitability for fast determination of patient-specific microdosimetric parameters. In the summation method,f(y)from many fixed radii models are summed. Fixed radius models do not provide a close approximation of the full patient-specific modely¯fory¯dfor the lower energy sources investigated,Y169bandI125.The higher energy sources investigated,C60oandI192rare less sensitive to target size variation thanY169bandI125.The summation method yields the most accurate approximation of the full modeld(y)for all radioisotopes investigated. The use of a summation method allows for the computation of patient-specific microdosimetric distributions with the computing power of a personal computer. With appropriate biological inputs the microdosimetric distributions computed using these methods can yield a patient-specific relative biological effectiveness as part of a multiscale treatment planning approach.

Investigating the impact of the CT Hounsfield unit range on radiomic feature stability using dual energy CT data.

PURPOSE: Radiomic texture calculation requires discretizing image intensities within the region-of-interest. FBN (fixed-bin-number), FBS (fixed-bin-size) and FBN and FBS with intensity equalization (FBNequal, FBSequal) are four discretization approaches. A crucial choice is the voxel intensity (Hounsfield units, or HU) binning range. We assessed the effect of this choice on radiomic features. METHODS: The dataset comprised 95 patients with head-and-neck squamous-cell-carcinoma. Dual energy CT data was reconstructed at 21 electron energies (40, 45,… 140 keV). Each of 94 texture features were calculated with 64 extraction parameters. All features were calculated five times: original choice, left shift (-10/-20 HU), right shift (+10/+20 HU). For each feature, Spearman correlation between nominal and four variants were calculated to determine feature stability. This was done for six texture feature types (GLCM, GLRLM, GLSZM, GLDZM, NGTDM, and NGLDM) separately. This analysis was repeated for the four binning algorithms. Effect of feature instability on predictive ability was studied for lymphadenopathy as endpoint. RESULTS: FBN and FBNequal algorithms showed good stability (correlation values consistently >0.9). For FBS and FBSequal algorithms, while median values exceeded 0.9, the 95% lower bound decreased as a function of energy, with poor performance over the entire spectrum. FBNequal was the most stable algorithm, and FBS the least. CONCLUSIONS: We believe this is the first multi-energy systematic study of the impact of CT HU range used during intensity discretization for radiomic feature extraction. Future analyses should account for this source of uncertainty when evaluating the robustness of their radiomic signature.

Machine Learning-Based Prediction of COVID-19 Severity and Progression to Critical Illness Using CT Imaging and Clinical Data.

OBJECTIVE: To develop a machine learning (ML) pipeline based on radiomics to predict Coronavirus Disease 2019 (COVID-19) severity and the future deterioration to critical illness using CT and clinical variables. MATERIALS AND METHODS: Clinical data were collected from 981 patients from a multi-institutional international cohort with real-time polymerase chain reaction-confirmed COVID-19. Radiomics features were extracted from chest CT of the patients. The data of the cohort were randomly divided into training, validation, and test sets using a 7:1:2 ratio. A ML pipeline consisting of a model to predict severity and time-to-event model to predict progression to critical illness were trained on radiomics features and clinical variables. The receiver operating characteristic area under the curve (ROC-AUC), concordance index (C-index), and time-dependent ROC-AUC were calculated to determine model performance, which was compared with consensus CT severity scores obtained by visual interpretation by radiologists. RESULTS: Among 981 patients with confirmed COVID-19, 274 patients developed critical illness. Radiomics features and clinical variables resulted in the best performance for the prediction of disease severity with a highest test ROC-AUC of 0.76 compared with 0.70 (0.76 vs. 0.70, p = 0.023) for visual CT severity score and clinical variables. The progression prediction model achieved a test C-index of 0.868 when it was based on the combination of CT radiomics and clinical variables compared with 0.767 when based on CT radiomics features alone (p < 0.001), 0.847 when based on clinical variables alone (p = 0.110), and 0.860 when based on the combination of visual CT severity scores and clinical variables (p = 0.549). Furthermore, the model based on the combination of CT radiomics and clinical variables achieved time-dependent ROC-AUCs of 0.897, 0.933, and 0.927 for the prediction of progression risks at 3, 5 and 7 days, respectively. CONCLUSION: CT radiomics features combined with clinical variables were predictive of COVID-19 severity and progression to critical illness with fairly high accuracy.

Development of patient-specific 3D models from histopathological samples for applications in radiation therapy.

The biological effects of ionizing radiation depend on the tissue, tumor type, radiation quality, and patient-specific factors. Inter-patient variation in cell/nucleus size may influence patient-specific dose response. However, this variability in dose response is not well investigated due to lack of available cell/nucleus size data. The aim of this study was to develop methods to derive cell/nucleus size distributions from digital images of 2D histopathological samples and use them to build digital 3D models for use in cellular dosimetry. Nineteen of sixty hematoxylin and eosin stained lung adenocarcinoma samples investigated passed exclusion criterion to be analyzed in the study. A difference of gaussians blob detection algorithm was used to identify nucleus centers and quantify cell spacing. Hematoxylin content was measured to determine nucleus radius. Pouring simulations were conducted to generate one-hundred 3D models containing volumes of equivalent cell spacing and nuclei radius to those in histopathological samples. The nuclei radius distributions of non-tumoral and cancerous regions appearing in the same slide were significantly different (p < 0.01) in all samples analyzed. The median nuclear-cytoplasmic ratio was 0.36 for non-tumoral cells and 0.50 for cancerous cells. The average cellular and nucleus packing densities in the 3D models generated were 65.9% (SD: 1.5%) and 13.3% (SD: 0.3%) respectively. Software to determine cell spacing and nuclei radius from histopathological samples was developed. 3D digital tissue models containing volumes with equivalent cell spacing, nucleus radius, and packing density to cancerous tissues were generated.

An artificial intelligence framework integrating longitudinal electronic health records with real-world data enables continuous pan-cancer prognostication.

Despite widespread adoption of electronic health records (EHRs), most hospitals are not ready to implement data science research in the clinical pipelines. Here, we develop MEDomics, a continuously learning infrastructure through which multimodal health data are systematically organized and data quality is assessed with the goal of applying artificial intelligence for individual prognosis. Using this framework, currently composed of thousands of individuals with cancer and millions of data points over a decade of data recording, we demonstrate prognostic utility of this framework in oncology. As proof of concept, we report an analysis using this infrastructure, which identified the Framingham risk score to be robustly associated with mortality among individuals with early-stage and advanced-stage cancer, a potentially actionable finding from a real-world cohort of individuals with cancer. Finally, we show how natural language processing (NLP) of medical notes could be used to continuously update estimates of prognosis as a given individual's disease course unfolds.

FDG-PET/CT Radiomics Models for The Early Prediction of Locoregional Recurrence in Head and Neck Cancer.

PURPOSE: Both CT and PET radiomics is considered as a potential prognostic biomarker in head and neck cancer. This study investigates the value of fused pre-treatment functional imaging (18F-FDG PET/CT) radiomics for modeling of local recurrence of head and neck cancers. MATERIALS AND METHODS: Firstly, 298 patients have been divided into a training set (n = 192) and verification set (n = 106). Secondly, PETs and CTs are fused based on wavelet transform. Thirdly, radiomics features are extracted from the 3D tumor area from PETCT fusion. The training set is used to select the features reduction and predict local recurrence, and the random forest prediction models combining radiomics and clinical variables are constructed. Finally, the ROC curve and KM analysis are used to evaluate the prediction efficiency of the model on the validation set. RESULTS: Two PET/CT fusion radiomics features and three clinic parameters are extracted to construct the radiomics model. AUC value in the verification set 0.70 is better than no fused sets 0.69. The accuracy of 0.66 is not the highest value (0.67). Either consistency index CI 0.70 (from 0.67 to 0.70) or the p-value 0.025 (from 0.03 to 0.025) get the best result in all four models. CONCLUSION: The radiomics model based on the fusion of PETCT is better than the model based on PET or CT alone in predicting local recurrence, the inclusion of clinical parameters may result in more accurate predictions, which has certain guiding significance for the development of personalized, precise treatment scheme.

Machine and deep learning methods for radiomics.

Radiomics is an emerging area in quantitative image analysis that aims to relate large-scale extracted imaging information to clinical and biological endpoints. The development of quantitative imaging methods along with machine learning has enabled the opportunity to move data science research towards translation for more personalized cancer treatments. Accumulating evidence has indeed demonstrated that noninvasive advanced imaging analytics, that is, radiomics, can reveal key components of tumor phenotype for multiple three-dimensional lesions at multiple time points over and beyond the course of treatment. These developments in the use of CT, PET, US, and MR imaging could augment patient stratification and prognostication buttressing emerging targeted therapeutic approaches. In recent years, deep learning architectures have demonstrated their tremendous potential for image segmentation, reconstruction, recognition, and classification. Many powerful open-source and commercial platforms are currently available to embark in new research areas of radiomics. Quantitative imaging research, however, is complex and key statistical principles should be followed to realize its full potential. The field of radiomics, in particular, requires a renewed focus on optimal study design/reporting practices and standardization of image acquisition, feature calculation, and rigorous statistical analysis for the field to move forward. In this article, the role of machine and deep learning as a major computational vehicle for advanced model building of radiomics-based signatures or classifiers, and diverse clinical applications, working principles, research opportunities, and available computational platforms for radiomics will be reviewed with examples drawn primarily from oncology. We also address issues related to common applications in medical physics, such as standardization, feature extraction, model building, and validation.

External Validation of an MRI-Derived Radiomics Model to Predict Biochemical Recurrence after Surgery for High-Risk Prostate Cancer.

Adjuvant radiotherapy after prostatectomy was recently challenged by early salvage radiotherapy, which highlighted the need for biomarkers to improve risk stratification. Therefore, we developed an MRI ADC map-derived radiomics model to predict biochemical recurrence (BCR) and BCR-free survival (bRFS) after surgery. Our goal in this work was to externally validate this radiomics-based prediction model. EXPERIMENTAL DESIGN: A total of 195 patients with a high recurrence risk of prostate cancer (pT3-4 and/or R1 and/or Gleason's score > 7) were retrospectively included in two institutions. Patients with postoperative PSA (Prostate Specific Antigen) > 0.04 ng/mL or lymph node involvement were excluded. Radiomics features were extracted from T2 and ADC delineated tumors. A total of 107 patients from Institution 1 were used to retrain the previously published model. The retrained model was then applied to 88 patients from Institution 2 for external validation. BCR predictions were evaluated using AUC (Area Under the Curve), accuracy, and bRFS using Kaplan-Meier curves. RESULTS: With a median follow-up of 46.3 months, 52/195 patients experienced BCR. In the retraining cohort, the clinical prediction model (combining the number of risk factors and postoperative PSA) demonstrated moderate predictive power (accuracy of 63%). The radiomics model (ADC-based SZEGLSZM) predicted BCR with an accuracy of 78% and allowed for significant stratification of patients for bRFS (p < 0.0001). In Institution 2, this radiomics model remained predictive of BCR (accuracy of 0.76%) contrary to the clinical model (accuracy of 0.56%). CONCLUSIONS: The recently developed MRI ADC map-based radiomics model was validated in terms of its predictive accuracy of BCR and bRFS after prostatectomy in an external cohort.

The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping.

Background Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Purpose To standardize a set of 174 radiomic features. Materials and Methods Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Results Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). Conclusion A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Kuhl and Truhn in this issue.

Deep Learning Based on MRI for Differentiation of Low- and High-Grade in Low-Stage Renal Cell Carcinoma.

Pretreatment determination of renal cell carcinoma aggressiveness may help to guide clinical decision-making. PURPOSE: To evaluate the efficacy of residual convolutional neural network using routine MRI in differentiating low-grade (grade I-II) from high-grade (grade III-IV) in stage I and II renal cell carcinoma. STUDY TYPE: Retrospective. POPULATION: In all, 376 patients with 430 renal cell carcinoma lesions from 2008-2019 in a multicenter cohort were acquired. The 353 Fuhrman-graded renal cell carcinomas were divided into a training, validation, and test set with a 7:2:1 split. The 77 WHO/ISUP graded renal cell carcinomas were used as a separate WHO/ISUP test set. FIELD STRENGTH/SEQUENCE: 1.5T and 3.0T/T2 -weighted and T1 contrast-enhanced sequences. ASSESSMENT: The accuracy, sensitivity, and specificity of the final model were assessed. The receiver operating characteristic (ROC) curve and precision-recall curve were plotted to measure the performance of the binary classifier. A confusion matrix was drawn to show the true positive, true negative, false positive, and false negative of the model. STATISTICAL TESTS: Mann-Whitney U-test for continuous data and the chi-square test or Fisher's exact test for categorical data were used to compare the difference of clinicopathologic characteristics between the low- and high-grade groups. The adjusted Wald method was used to calculate the 95% confidence interval (CI) of accuracy, sensitivity, and specificity. RESULTS: The final deep-learning model achieved a test accuracy of 0.88 (95% CI: 0.73-0.96), sensitivity of 0.89 (95% CI: 0.74-0.96), and specificity of 0.88 (95% CI: 0.73-0.96) in the Fuhrman test set and a test accuracy of 0.83 (95% CI: 0.73-0.90), sensitivity of 0.92 (95% CI: 0.84-0.97), and specificity of 0.78 (95% CI: 0.68-0.86) in the WHO/ISUP test set. DATA CONCLUSION: Deep learning can noninvasively predict the histological grade of stage I and II renal cell carcinoma using conventional MRI in a multiinstitutional dataset with high accuracy. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY STAGE: 2.

Deep Learning to Distinguish Benign from Malignant Renal Lesions Based on Routine MR Imaging.

PURPOSE: With increasing incidence of renal mass, it is important to make a pretreatment differentiation between benign renal mass and malignant tumor. We aimed to develop a deep learning model that distinguishes benign renal tumors from renal cell carcinoma (RCC) by applying a residual convolutional neural network (ResNet) on routine MR imaging. EXPERIMENTAL DESIGN: Preoperative MR images (T2-weighted and T1-postcontrast sequences) of 1,162 renal lesions definitely diagnosed on pathology or imaging in a multicenter cohort were divided into training, validation, and test sets (70:20:10 split). An ensemble model based on ResNet was built combining clinical variables and T1C and T2WI MR images using a bagging classifier to predict renal tumor pathology. Final model performance was compared with expert interpretation and the most optimized radiomics model. RESULTS: Among the 1,162 renal lesions, 655 were malignant and 507 were benign. Compared with a baseline zero rule algorithm, the ensemble deep learning model had a statistically significant higher test accuracy (0.70 vs. 0.56, P = 0.004). Compared with all experts averaged, the ensemble deep learning model had higher test accuracy (0.70 vs. 0.60, P = 0.053), sensitivity (0.92 vs. 0.80, P = 0.017), and specificity (0.41 vs. 0.35, P = 0.450). Compared with the radiomics model, the ensemble deep learning model had higher test accuracy (0.70 vs. 0.62, P = 0.081), sensitivity (0.92 vs. 0.79, P = 0.012), and specificity (0.41 vs. 0.39, P = 0.770). CONCLUSIONS: Deep learning can noninvasively distinguish benign renal tumors from RCC using conventional MR imaging in a multi-institutional dataset with good accuracy, sensitivity, and specificity comparable with experts and radiomics.

Overlooked pitfalls in multi-class machine learning classification in radiation oncology and how to avoid them.

In radiation oncology, Machine Learning classification publications are typically related to two outcome classes, e.g. the presence or absence of distant metastasis. However, multi-class classification problems also have great clinical relevance, e.g., predicting the grade of a treatment complication following lung irradiation. This work comprised two studies aimed at making work in this domain less prone to statistical blindsides. In multi-class classification, AUC is not defined, whereas correlation coefficients are. It may seem like solely quoting the correlation coefficient value (in lieu of the AUC value) is a suitable choice. In the first study, we illustrated using Monte Carlo (MC) models why this choice is misleading. We also considered the special case where the multiple classes are not ordinal, but nominal, and explained why Pearson or Spearman correlation coefficients are not only providing incomplete information but are actually meaningless. The second study concerned surrogate biomarkers for a clinical endpoint, which have purported benefits including potential for early assessment, being inexpensive, and being non-invasive. Using a MC experiment, we showed how conclusions derived from surrogate markers can be misleading. The simulated endpoint was radiation toxicity (scale of 0-5). The surrogate marker was the true toxicity grade plus a noise term. Five patient cohorts were simulated, including one control. Two of the cohorts were designed to have a statistically significant difference in toxicity. Under 1000 repeated experiments using the biomarker, these two cohorts were often found to be statistically indistinguishable, with the fraction of such occurrences rising with the level of noise.