Expect the unexpected: investigating discordant prostate MRI and biopsy results.

OBJECTIVES: To evaluate discrepant radio-pathological outcomes in biopsy-naïve patients undergoing prostate MRI and to provide insights into the underlying causes. MATERIALS AND METHODS: A retrospective analysis was conducted on 2780 biopsy-naïve patients undergoing prostate MRI at a tertiary referral centre between October 2015 and June 2022. Exclusion criteria were biopsy not performed, indeterminate MRI findings (PI-RADS 3), and clinically insignificant PCa (Gleason score 3 + 3). Patients with discrepant findings between MRI and biopsy results were categorised into two groups: MRI-negative/Biopsy-positive and MRI-positive/Biopsy-negative (biopsy-positive defined as Gleason score ≥ 3 + 4). An expert uroradiologist reviewed discrepant cases, retrospectively re-assigning PI-RADS scores, identifying any missed MRI targets, and evaluating the quality of MRI scans. Potential explanations for discrepancies included MRI overcalls (including known pitfalls), benign pathology findings, and biopsy targeting errors. RESULTS: Patients who did not undergo biopsy (n = 1258) or who had indeterminate MRI findings (n = 204), as well as those with clinically insignificant PCa (n = 216), were excluded, with a total of 1102 patients analysed. Of these, 32/1,102 (3%) were classified as MRI-negative/biopsy-positive and 117/1102 (11%) as MRI-positive/biopsy-negative. In the MRI-negative/Biopsy-positive group, 44% of studies were considered non-diagnostic quality. Upon retrospective image review, target lesions were identified in 28% of cases. In the MRI-positive/Biopsy-negative group, 42% of cases were considered to be MRI overcalls, and 32% had an explanatory benign pathological finding, with biopsy targeting errors accounting for 11% of cases. CONCLUSION: Prostate MRI demonstrated a high diagnostic accuracy, with low occurrences of discrepant findings as defined. Common reasons for MRI-positive/Biopsy-negative cases included explanatory benign findings and MRI overcalls. CLINICAL RELEVANCE STATEMENT: This study highlights the importance of optimal prostate MRI image quality and expertise in reducing diagnostic errors, improving patient outcomes, and guiding appropriate management decisions in the prostate cancer diagnostic pathway. KEY POINTS: • Discrepancies between prostate MRI and biopsy results can occur, with higher numbers of MRI-positive/biopsy-negative relative to MRI-negative/biopsy-positive cases. • MRI-positive/biopsy-negative cases were mostly overcalls or explainable by benign biopsy findings. • In about one-third of MRI-negative/biopsy-positive cases, a target lesion was retrospectively identified.

Evaluating the quality of radiomics-based studies for endometrial cancer using RQS and METRICS tools.

OBJECTIVE: To assess the methodological quality of radiomics-based models in endometrial cancer using the radiomics quality score (RQS) and METhodological radiomICs score (METRICS). METHODS: We systematically reviewed studies published by October 30th, 2023. Inclusion criteria were original radiomics studies on endometrial cancer using CT, MRI, PET, or ultrasound. Articles underwent a quality assessment by novice and expert radiologists using RQS and METRICS. The inter-rater reliability for RQS and METRICS among radiologists with varying expertise was determined. Subgroup analyses were performed to assess whether scores varied according to study topic, imaging technique, publication year, and journal quartile. RESULTS: Sixty-eight studies were analysed, with a median RQS of 11 (IQR, 9-14) and METRICS score of 67.6% (IQR, 58.8-76.0); two different articles reached maximum RQS of 19 and METRICS of 90.7%, respectively. Most studies utilised MRI (82.3%) and machine learning methods (88.2%). Characterisation and recurrence risk stratification were the most explored outcomes, featured in 35.3% and 19.1% of articles, respectively. High inter-rater reliability was observed for both RQS (ICC: 0.897; 95% CI: 0.821, 0.946) and METRICS (ICC: 0.959; 95% CI: 0.928, 0.979). Methodological limitations such as lack of external validation suggest areas for improvement. At subgroup analyses, no statistically significant difference was noted. CONCLUSIONS: Whilst using RQS, the quality of endometrial cancer radiomics research was apparently unsatisfactory, METRICS depicts a good overall quality. Our study highlights the need for strict compliance with quality metrics. Adhering to these quality measures can increase the consistency of radiomics towards clinical application in the pre-operative management of endometrial cancer. CLINICAL RELEVANCE STATEMENT: Both the RQS and METRICS can function as instrumental tools for identifying different methodological deficiencies in endometrial cancer radiomics research. However, METRICS also reflected a focus on the practical applicability and clarity of documentation. KEY POINTS: The topic of radiomics currently lacks standardisation, limiting clinical implementation. METRICS scores were generally higher than the RQS, reflecting differences in the development process and methodological content. A positive trend in METRICS score may suggest growing attention to methodological aspects in radiomics research.

Reproducibility of radiomics quality score: an intra- and inter-rater reliability study.

OBJECTIVES: To investigate the intra- and inter-rater reliability of the total radiomics quality score (RQS) and the reproducibility of individual RQS items' score in a large multireader study. METHODS: Nine raters with different backgrounds were randomly assigned to three groups based on their proficiency with RQS utilization: Groups 1 and 2 represented the inter-rater reliability groups with or without prior training in RQS, respectively; group 3 represented the intra-rater reliability group. Thirty-three original research papers on radiomics were evaluated by raters of groups 1 and 2. Of the 33 papers, 17 were evaluated twice with an interval of 1 month by raters of group 3. Intraclass coefficient (ICC) for continuous variables, and Fleiss' and Cohen's kappa (k) statistics for categorical variables were used. RESULTS: The inter-rater reliability was poor to moderate for total RQS (ICC 0.30-055, p < 0.001) and very low to good for item's reproducibility (k - 0.12 to 0.75) within groups 1 and 2 for both inexperienced and experienced raters. The intra-rater reliability for total RQS was moderate for the less experienced rater (ICC 0.522, p = 0.009), whereas experienced raters showed excellent intra-rater reliability (ICC 0.91-0.99, p < 0.001) between the first and second read. Intra-rater reliability on RQS items' score reproducibility was higher and most of the items had moderate to good intra-rater reliability (k - 0.40 to 1). CONCLUSIONS: Reproducibility of the total RQS and the score of individual RQS items is low. There is a need for a robust and reproducible assessment method to assess the quality of radiomics research. CLINICAL RELEVANCE STATEMENT: There is a need for reproducible scoring systems to improve quality of radiomics research and consecutively close the translational gap between research and clinical implementation. KEY POINTS: • Radiomics quality score has been widely used for the evaluation of radiomics studies. • Although the intra-rater reliability was moderate to excellent, intra- and inter-rater reliability of total score and point-by-point scores were low with radiomics quality score. • A robust, easy-to-use scoring system is needed for the evaluation of radiomics research.

Radiomics for the identification of extraprostatic extension with prostate MRI: a systematic review and meta-analysis.

OBJECTIVES: Extraprostatic extension (EPE) of prostate cancer (PCa) is predicted using clinical nomograms. Incorporating MRI could represent a leap forward, although poor sensitivity and standardization represent unsolved issues. MRI radiomics has been proposed for EPE prediction. The aim of the study was to systematically review the literature and perform a meta-analysis of MRI-based radiomics approaches for EPE prediction. MATERIALS AND METHODS: Multiple databases were systematically searched for radiomics studies on EPE detection up to June 2022. Methodological quality was appraised according to Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool and radiomics quality score (RQS). The area under the receiver operating characteristic curves (AUC) was pooled to estimate predictive accuracy. A random-effects model estimated overall effect size. Statistical heterogeneity was assessed with I2 value. Publication bias was evaluated with a funnel plot. Subgroup analyses were performed to explore heterogeneity. RESULTS: Thirteen studies were included, showing limitations in study design and methodological quality (median RQS 10/36), with high statistical heterogeneity. Pooled AUC for EPE identification was 0.80. In subgroup analysis, test-set and cross-validation-based studies had pooled AUC of 0.85 and 0.89 respectively. Pooled AUC was 0.72 for deep learning (DL)-based and 0.82 for handcrafted radiomics studies and 0.79 and 0.83 for studies with multiple and single scanner data, respectively. Finally, models with the best predictive performance obtained using radiomics features showed pooled AUC of 0.82, while those including clinical data of 0.76. CONCLUSION: MRI radiomics-powered models to identify EPE in PCa showed a promising predictive performance overall. However, methodologically robust, clinically driven research evaluating their diagnostic and therapeutic impact is still needed. CLINICAL RELEVANCE STATEMENT: Radiomics might improve the management of prostate cancer patients increasing the value of MRI in the assessment of extraprostatic extension. However, it is imperative that forthcoming research prioritizes confirmation studies and a stronger clinical orientation to solidify these advancements. KEY POINTS: • MRI radiomics deserves attention as a tool to overcome the limitations of MRI in prostate cancer local staging. • Pooled AUC was 0.80 for the 13 included studies, with high heterogeneity (84.7%, p < .001), methodological issues, and poor clinical orientation. • Methodologically robust radiomics research needs to focus on increasing MRI sensitivity and bringing added value to clinical nomograms at patient level.

Ovarian imaging radiomics quality score assessment: an EuSoMII radiomics auditing group initiative.

OBJECTIVE: To evaluate the methodological rigor of radiomics-based studies using noninvasive imaging in ovarian setting. METHODS: Multiple medical literature archives (PubMed, Web of Science, and Scopus) were searched to retrieve original studies focused on computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US), or positron emission tomography (PET) radiomics for ovarian disorders' assessment. Two researchers in consensus evaluated each investigation using the radiomics quality score (RQS). Subgroup analyses were performed to assess whether the total RQS varied according to first author category, study aim and topic, imaging modality, and journal quartile. RESULTS: From a total of 531 items, 63 investigations were finally included in the analysis. The studies were greatly focused (94%) on the field of oncology, with CT representing the most used imaging technique (41%). Overall, the papers achieved a median total RQS 6 (IQR, -0.5 to 11), corresponding to a percentage of 16.7% of the maximum score (IQR, 0-30.6%). The scoring was low especially due to the lack of prospective design and formal validation of the results. At subgroup analysis, the 4 studies not focused on oncological topic showed significantly lower quality scores than the others. CONCLUSIONS: The overall methodological rigor of radiomics studies in the ovarian field is still not ideal, limiting the reproducibility of results and potential translation to clinical setting. More efforts towards a standardized methodology in the workflow are needed to allow radiomics to become a viable tool for clinical decision-making. KEY POINTS: • The 63 included studies using noninvasive imaging for ovarian applications were mostly focused on oncologic topic (94%). • The included investigations achieved a median total RQS 6 (IQR, -0.5 to 11), indicating poor methodological rigor. • The RQS was low especially due to the lack of prospective design and formal validation of the results.

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative.

OBJECTIVE: The main aim of the present systematic review was a comprehensive overview of the Radiomics Quality Score (RQS)-based systematic reviews to highlight common issues and challenges of radiomics research application and evaluate the relationship between RQS and review features. METHODS: The literature search was performed on multiple medical literature archives according to PRISMA guidelines for systematic reviews that reported radiomic quality assessment through the RQS. Reported scores were converted to a 0-100% scale. The Mann-Whitney and Kruskal-Wallis tests were used to compare RQS scores and review features. RESULTS: The literature research yielded 345 articles, from which 44 systematic reviews were finally included in the analysis. Overall, the median of RQS was 21.00% (IQR = 11.50). No significant differences of RQS were observed in subgroup analyses according to targets (oncological/not oncological target, neuroradiology/body imaging focus and one imaging technique/more than one imaging technique, characterization/prognosis/detection/other). CONCLUSIONS: Our review did not reveal a significant difference of quality of radiomic articles reported in systematic reviews, divided in different subgroups. Furthermore, low overall methodological quality of radiomics research was found independent of specific application domains. While the RQS can serve as a reference tool to improve future study designs, future research should also be aimed at improving its reliability and developing new tools to meet an ever-evolving research space. KEY POINTS: • Radiomics is a promising high-throughput method that may generate novel imaging biomarkers to improve clinical decision-making process, but it is an inherently complex analysis and often lacks reproducibility and generalizability. • The Radiomics Quality Score serves a necessary role as the de facto reference tool for assessing radiomics studies. • External auditing of radiomics studies, in addition to the standard peer-review process, is valuable to highlight common limitations and provide insights to improve future study designs and practical applicability of the radiomics models.

Cardiac CT and MRI radiomics: systematic review of the literature and radiomics quality score assessment.

OBJECTIVE: To systematically review and evaluate the methodological quality of studies using magnetic resonance imaging (MRI) and computed tomography (CT) radiomics for cardiac applications. METHODS: Multiple medical literature archives (PubMed, Web of Science, and EMBASE) were systematically searched to retrieve original studies focused on cardiac MRI and CT radiomics applications. Two researchers in consensus assessed each investigation using the radiomics quality score (RQS). Subgroup analyses were performed to assess whether the total RQS varied according to study aim, journal quartile, imaging modality, and first author category. RESULTS: From a total of 1961 items, 53 articles were finally included in the analysis. Overall, the studies reached a median total RQS of 7 (IQR, 4-12), corresponding to a percentage score of 19.4% (IQR, 11.1-33.3%). Item scores were particularly low due to lack of prospective design, cost-effectiveness analysis, and open science. Median RQS percentage score was significantly higher in papers where the first author was a medical doctor and in those published on first quartile journals. CONCLUSIONS: The overall methodological quality of radiomics studies in cardiac MRI and CT is still lacking. A higher degree of standardization of the radiomics workflow and higher publication standards for studies are required to allow its translation into clinical practice. KEY POINTS: • RQS has been recently proposed for the overall assessment of the methodological quality of radiomics-based studies. • The 53 included studies on cardiac MRI and CT radiomics applications reached a median total RQS of 7 (IQR, 4-12), corresponding to a percentage of 19.4% (IQR, 11.1-33.3%). • A more standardized methodology in the radiomics workflow is needed, especially in terms of study design, validation, and open science, in order to translate the results to clinical applications.

Deep Learning Whole-Gland and Zonal Prostate Segmentation on a Public MRI Dataset.

BACKGROUND: Prostate volume, as determined by magnetic resonance imaging (MRI), is a useful biomarker both for distinguishing between benign and malignant pathology and can be used either alone or combined with other parameters such as prostate-specific antigen. PURPOSE: This study compared different deep learning methods for whole-gland and zonal prostate segmentation. STUDY TYPE: Retrospective. POPULATION: A total of 204 patients (train/test = 99/105) from the PROSTATEx public dataset. FIELD STRENGTH/SEQUENCE: A 3 T, TSE T2 -weighted. ASSESSMENT: Four operators performed manual segmentation of the whole-gland, central zone + anterior stroma + transition zone (TZ), and peripheral zone (PZ). U-net, efficient neural network (ENet), and efficient residual factorized ConvNet (ERFNet) were trained and tuned on the training data through 5-fold cross-validation to segment the whole gland and TZ separately, while PZ automated masks were obtained by the subtraction of the first two. STATISTICAL TESTS: Networks were evaluated on the test set using various accuracy metrics, including the Dice similarity coefficient (DSC). Model DSC was compared in both the training and test sets using the analysis of variance test (ANOVA) and post hoc tests. Parameter number, disk size, training, and inference times determined network computational complexity and were also used to assess the model performance differences. A P < 0.05 was selected to indicate the statistical significance. RESULTS: The best DSC (P < 0.05) in the test set was achieved by ENet: 91% ± 4% for the whole gland, 87% ± 5% for the TZ, and 71% ± 8% for the PZ. U-net and ERFNet obtained, respectively, 88% ± 6% and 87% ± 6% for the whole gland, 86% ± 7% and 84% ± 7% for the TZ, and 70% ± 8% and 65 ± 8% for the PZ. Training and inference time were lowest for ENet. DATA CONCLUSION: Deep learning networks can accurately segment the prostate using T2 -weighted images. EVIDENCE LEVEL: 4 TECHNICAL EFFICACY: Stage 2.

MRI index lesion radiomics and machine learning for detection of extraprostatic extension of disease: a multicenter study.

OBJECTIVES: To build a machine learning (ML) model to detect extraprostatic extension (EPE) of prostate cancer (PCa), based on radiomics features extracted from prostate MRI index lesions. METHODS: Consecutive MRI exams of patients undergoing radical prostatectomy for PCa were retrospectively collected from three institutions. Axial T2-weighted and apparent diffusion coefficient map images were annotated to obtain index lesion volumes of interest for radiomics feature extraction. Data from one institution was used for training, feature selection (using reproducibility, variance and pairwise correlation analyses, and a correlation-based subset evaluator), and tuning of a support vector machine (SVM) algorithm, with stratified 10-fold cross-validation. The model was tested on the two remaining institutions' data and compared with a baseline reference and expert radiologist assessment of EPE. RESULTS: In total, 193 patients were included. From an initial dataset of 2436 features, 2287 were excluded due to either poor stability, low variance, or high collinearity. Among the remaining, 14 features were used to train the ML model, which reached an overall accuracy of 83% in the training set. In the two external test sets, the SVM achieved an accuracy of 79% and 74% respectively, not statistically different from that of the radiologist (81-83%, p = 0.39-1) and outperforming the baseline reference (p = 0.001-0.02). CONCLUSIONS: A ML model solely based on radiomics features demonstrated high accuracy for EPE detection and good generalizability in a multicenter setting. Paired to qualitative EPE assessment, this approach could aid radiologists in this challenging task. KEY POINTS: • Predicting the presence of EPE in prostate cancer patients is a challenging task for radiologists. • A support vector machine algorithm achieved high diagnostic accuracy for EPE detection, with good generalizability when tested on multiple external datasets. • The performance of the algorithm was not significantly different from that of an experienced radiologist.

Clinical value of radiomics and machine learning in breast ultrasound: a multicenter study for differential diagnosis of benign and malignant lesions.

OBJECTIVES: We aimed to assess the performance of radiomics and machine learning (ML) for classification of non-cystic benign and malignant breast lesions on ultrasound images, compare ML's accuracy with that of a breast radiologist, and verify if the radiologist's performance is improved by using ML. METHODS: Our retrospective study included patients from two institutions. A total of 135 lesions from Institution 1 were used to train and test the ML model with cross-validation. Radiomic features were extracted from manually annotated images and underwent a multistep feature selection process. Not reproducible, low variance, and highly intercorrelated features were removed from the dataset. Then, 66 lesions from Institution 2 were used as an external test set for ML and to assess the performance of a radiologist without and with the aid of ML, using McNemar's test. RESULTS: After feature selection, 10 of the 520 features extracted were employed to train a random forest algorithm. Its accuracy in the training set was 82% (standard deviation, SD, ± 6%), with an AUC of 0.90 (SD ± 0.06), while the performance on the test set was 82% (95% confidence intervals (CI) = 70-90%) with an AUC of 0.82 (95% CI = 0.70-0.93). It resulted in being significantly better than the baseline reference (p = 0.0098), but not different from the radiologist (79.4%, p = 0.815). The radiologist's performance improved when using ML (80.2%), but not significantly (p = 0.508). CONCLUSIONS: A radiomic analysis combined with ML showed promising results to differentiate benign from malignant breast lesions on ultrasound images. KEY POINTS: • Machine learning showed good accuracy in discriminating benign from malignant breast lesions • The machine learning classifier's performance was comparable to that of a breast radiologist • The radiologist's accuracy improved with machine learning, but not significantly.