Towards reproducible radiomics research: introduction of a database for radiomics studies.

OBJECTIVES: To investigate the model-, code-, and data-sharing practices in the current radiomics research landscape and to introduce a radiomics research database. METHODS: A total of 1254 articles published between January 1, 2021, and December 31, 2022, in leading radiology journals (European Radiology, European Journal of Radiology, Radiology, Radiology: Artificial Intelligence, Radiology: Cardiothoracic Imaging, Radiology: Imaging Cancer) were retrospectively screened, and 257 original research articles were included in this study. The categorical variables were compared using Fisher's exact tests or chi-square test and numerical variables using Student's t test with relation to the year of publication. RESULTS: Half of the articles (128 of 257) shared the model by either including the final model formula or reporting the coefficients of selected radiomics features. A total of 73 (28%) models were validated on an external independent dataset. Only 16 (6%) articles shared the data or used publicly available open datasets. Similarly, only 20 (7%) of the articles shared the code. A total of 7 (3%) articles both shared code and data. All collected data in this study is presented in a radiomics research database (RadBase) and could be accessed at https://github.com/EuSoMII/RadBase . CONCLUSION: According to the results of this study, the majority of published radiomics models were not technically reproducible since they shared neither model nor code and data. There is still room for improvement in carrying out reproducible and open research in the field of radiomics. CLINICAL RELEVANCE STATEMENT: To date, the reproducibility of radiomics research and open science practices within the radiomics research community are still very low. Ensuring reproducible radiomics research with model-, code-, and data-sharing practices will facilitate faster clinical translation. KEY POINTS: • There is a discrepancy between the number of published radiomics papers and the clinical implementation of these published radiomics models. • The main obstacle to clinical implementation is the lack of model-, code-, and data-sharing practices. • In order to translate radiomics research into clinical practice, the radiomics research community should adopt open science practices.

Evaluating the quality of radiomics-based studies for endometrial cancer using RQS and METRICS tools.

OBJECTIVE: To assess the methodological quality of radiomics-based models in endometrial cancer using the radiomics quality score (RQS) and METhodological radiomICs score (METRICS). METHODS: We systematically reviewed studies published by October 30th, 2023. Inclusion criteria were original radiomics studies on endometrial cancer using CT, MRI, PET, or ultrasound. Articles underwent a quality assessment by novice and expert radiologists using RQS and METRICS. The inter-rater reliability for RQS and METRICS among radiologists with varying expertise was determined. Subgroup analyses were performed to assess whether scores varied according to study topic, imaging technique, publication year, and journal quartile. RESULTS: Sixty-eight studies were analysed, with a median RQS of 11 (IQR, 9-14) and METRICS score of 67.6% (IQR, 58.8-76.0); two different articles reached maximum RQS of 19 and METRICS of 90.7%, respectively. Most studies utilised MRI (82.3%) and machine learning methods (88.2%). Characterisation and recurrence risk stratification were the most explored outcomes, featured in 35.3% and 19.1% of articles, respectively. High inter-rater reliability was observed for both RQS (ICC: 0.897; 95% CI: 0.821, 0.946) and METRICS (ICC: 0.959; 95% CI: 0.928, 0.979). Methodological limitations such as lack of external validation suggest areas for improvement. At subgroup analyses, no statistically significant difference was noted. CONCLUSIONS: Whilst using RQS, the quality of endometrial cancer radiomics research was apparently unsatisfactory, METRICS depicts a good overall quality. Our study highlights the need for strict compliance with quality metrics. Adhering to these quality measures can increase the consistency of radiomics towards clinical application in the pre-operative management of endometrial cancer. CLINICAL RELEVANCE STATEMENT: Both the RQS and METRICS can function as instrumental tools for identifying different methodological deficiencies in endometrial cancer radiomics research. However, METRICS also reflected a focus on the practical applicability and clarity of documentation. KEY POINTS: The topic of radiomics currently lacks standardisation, limiting clinical implementation. METRICS scores were generally higher than the RQS, reflecting differences in the development process and methodological content. A positive trend in METRICS score may suggest growing attention to methodological aspects in radiomics research.

Quality assessment of the MRI-radiomics studies for MGMT promoter methylation prediction in glioma: a systematic review and meta-analysis.

OBJECTIVES: To evaluate the methodological quality and diagnostic accuracy of MRI-based radiomic studies predicting O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation status in gliomas. METHODS: PubMed Medline, EMBASE, and Web of Science were searched to identify MRI-based radiomic studies on MGMT methylation in gliomas published until December 31, 2022. Three raters evaluated the study methodological quality with Radiomics Quality Score (RQS, 16 components) and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD, 22 items) scales. Risk of bias and applicability concerns were assessed with QUADAS-2 tool. A meta-analysis was performed to estimate the pooled area under the curve (AUC) and to assess inter-study heterogeneity. RESULTS: We included 26 studies, published from 2016. The median RQS total score was 8 out of 36 (22%, range 8-44%). Thirteen studies performed external validation. All studies reported AUC or accuracy, but only 4 (15%) performed calibration and decision curve analysis. No studies performed phantom analysis, cost-effectiveness analysis, and prospective validation. The overall TRIPOD adherence score was between 50% and 70% in 16 studies and below 50% in 10 studies. The pooled AUC was 0.78 (95% CI, 0.73-0.83, I2 = 94.1%) with a high inter-study heterogeneity. Studies with external validation and including only WHO-grade IV gliomas had significantly lower AUC values (0.65; 95% CI, 0.57-0.73, p < 0.01). CONCLUSIONS: Study RQS and adherence to TRIPOD guidelines was generally low. Radiomic prediction of MGMT methylation status showed great heterogeneity of results and lower performances in grade IV gliomas, which hinders its current implementation in clinical practice. CLINICAL RELEVANCE STATEMENT: MGMT promoter methylation status appears to be variably correlated with MRI radiomic features; radiomic models are not sufficiently robust to be integrated into clinical practice to accurately predict MGMT promoter methylation status in patients with glioma before surgery. KEY POINTS: • Adherence to the indications of TRIPOD guidelines was generally low, as was RQS total score. • MGMT promoter methylation status prediction with MRI radiomic features provided heterogeneous diagnostic accuracy results across studies. • Studies that included grade IV glioma only and performed external validation had significantly lower diagnostic accuracy than others.

ESR Essentials: using the right scoring system in prostate MRI-practice recommendations by ESUR.

MRI has gained prominence in the diagnostic workup of prostate cancer (PCa) patients, with the Prostate Imaging Reporting and Data System (PI-RADS) being widely used for cancer detection. Beyond PI-RADS, other MRI-based scoring tools have emerged to address broader aspects within the PCa domain. However, the multitude of available MRI-based grading systems has led to inconsistencies in their application within clinical workflows. The Prostate Cancer Radiological Estimation of Change in Sequential Evaluation (PRECISE) assesses the likelihood of clinically significant radiological changes of PCa during active surveillance, and the Prostate Imaging for Local Recurrence Reporting (PI-RR) scoring system evaluates the risk of local recurrence after whole-gland therapies with curative intent. Underlying any system is the requirement to assess image quality using the Prostate Imaging Quality Scoring System (PI-QUAL). This article offers practicing radiologists a comprehensive overview of currently available scoring systems with clinical evidence supporting their use for managing PCa patients to enhance consistency in interpretation and facilitate effective communication with referring clinicians. KEY POINTS: Assessing image quality is essential for all prostate MRI interpretations and the PI-QUAL score represents  the standardized tool for this purpose. Current urological clinical guidelines for prostate cancer diagnosis and localization recommend adhering to the PI-RADS recommendations. The PRECISE and PI-RR scoring systems can be used for assessing radiological changes of prostate cancer during active surveillance and the likelihood of local recurrence after radical treatments respectively.

Reproducibility of radiomics quality score: an intra- and inter-rater reliability study.

OBJECTIVES: To investigate the intra- and inter-rater reliability of the total radiomics quality score (RQS) and the reproducibility of individual RQS items' score in a large multireader study. METHODS: Nine raters with different backgrounds were randomly assigned to three groups based on their proficiency with RQS utilization: Groups 1 and 2 represented the inter-rater reliability groups with or without prior training in RQS, respectively; group 3 represented the intra-rater reliability group. Thirty-three original research papers on radiomics were evaluated by raters of groups 1 and 2. Of the 33 papers, 17 were evaluated twice with an interval of 1 month by raters of group 3. Intraclass coefficient (ICC) for continuous variables, and Fleiss' and Cohen's kappa (k) statistics for categorical variables were used. RESULTS: The inter-rater reliability was poor to moderate for total RQS (ICC 0.30-055, p < 0.001) and very low to good for item's reproducibility (k - 0.12 to 0.75) within groups 1 and 2 for both inexperienced and experienced raters. The intra-rater reliability for total RQS was moderate for the less experienced rater (ICC 0.522, p = 0.009), whereas experienced raters showed excellent intra-rater reliability (ICC 0.91-0.99, p < 0.001) between the first and second read. Intra-rater reliability on RQS items' score reproducibility was higher and most of the items had moderate to good intra-rater reliability (k - 0.40 to 1). CONCLUSIONS: Reproducibility of the total RQS and the score of individual RQS items is low. There is a need for a robust and reproducible assessment method to assess the quality of radiomics research. CLINICAL RELEVANCE STATEMENT: There is a need for reproducible scoring systems to improve quality of radiomics research and consecutively close the translational gap between research and clinical implementation. KEY POINTS: • Radiomics quality score has been widely used for the evaluation of radiomics studies. • Although the intra-rater reliability was moderate to excellent, intra- and inter-rater reliability of total score and point-by-point scores were low with radiomics quality score. • A robust, easy-to-use scoring system is needed for the evaluation of radiomics research.

Trends and statistics of artificial intelligence and radiomics research in Radiology, Nuclear Medicine, and Medical Imaging: bibliometric analysis.

OBJECTIVE: To conduct a comprehensive bibliometric analysis of artificial intelligence (AI) and its subfields as well as radiomics in Radiology, Nuclear Medicine, and Medical Imaging (RNMMI). METHODS: Web of Science was queried for relevant publications in RNMMI and medicine along with their associated data from 2000 to 2021. Bibliometric techniques utilised were co-occurrence, co-authorship, citation burst, and thematic evolution analyses. Growth rate and doubling time were also estimated using log-linear regression analyses. RESULTS: According to the number of publications, RNMMI (11,209; 19.8%) was the most prominent category in medicine (56,734). USA (44.6%) and China (23.1%) were the two most productive and collaborative countries. USA and Germany experienced the strongest citation bursts. Thematic evolution has recently exhibited a significant shift toward deep learning. In all analyses, the annual number of publications and citations demonstrated exponential growth, with deep learning-based publications exhibiting the most prominent growth pattern. Estimated continuous growth rate, annual growth rate, and doubling time of the AI and machine learning publications in RNMMI were 26.1% (95% confidence interval [CI], 12.0-40.2%), 29.8% (95% CI, 12.7-49.5%), and 2.7 years (95% CI, 1.7-5.8), respectively. In the sensitivity analysis using data from the last 5 and 10 years, these estimates ranged from 47.6 to 51.1%, 61.0 to 66.7%, and 1.4 to 1.5 years. CONCLUSION: This study provides an overview of AI and radiomics research conducted mainly in RNMMI. These results may assist researchers, practitioners, policymakers, and organisations in gaining a better understanding of both the evolution of these fields and the importance of supporting (e.g., financial) these research activities. KEY POINTS: • In terms of the number of publications on AI and ML, Radiology, Nuclear Medicine, and Medical Imaging was the most prominent category compared to the other categories related to medicine (e.g., Health Policy & Services, Surgery). • All evaluated analyses (i.e., AI, its subfields, and radiomics), based on the annual number of publications and citations, demonstrated exponential growth, with decreasing doubling time, which indicates increasing interest from researchers, journals, and, in turn, the medical imaging community. • The most prominent growth pattern was observed in deep learning-based publications. However, the further thematic analysis demonstrated that deep learning has been underdeveloped but highly relevant to the medical imaging community.

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative.

OBJECTIVE: The main aim of the present systematic review was a comprehensive overview of the Radiomics Quality Score (RQS)-based systematic reviews to highlight common issues and challenges of radiomics research application and evaluate the relationship between RQS and review features. METHODS: The literature search was performed on multiple medical literature archives according to PRISMA guidelines for systematic reviews that reported radiomic quality assessment through the RQS. Reported scores were converted to a 0-100% scale. The Mann-Whitney and Kruskal-Wallis tests were used to compare RQS scores and review features. RESULTS: The literature research yielded 345 articles, from which 44 systematic reviews were finally included in the analysis. Overall, the median of RQS was 21.00% (IQR = 11.50). No significant differences of RQS were observed in subgroup analyses according to targets (oncological/not oncological target, neuroradiology/body imaging focus and one imaging technique/more than one imaging technique, characterization/prognosis/detection/other). CONCLUSIONS: Our review did not reveal a significant difference of quality of radiomic articles reported in systematic reviews, divided in different subgroups. Furthermore, low overall methodological quality of radiomics research was found independent of specific application domains. While the RQS can serve as a reference tool to improve future study designs, future research should also be aimed at improving its reliability and developing new tools to meet an ever-evolving research space. KEY POINTS: • Radiomics is a promising high-throughput method that may generate novel imaging biomarkers to improve clinical decision-making process, but it is an inherently complex analysis and often lacks reproducibility and generalizability. • The Radiomics Quality Score serves a necessary role as the de facto reference tool for assessing radiomics studies. • External auditing of radiomics studies, in addition to the standard peer-review process, is valuable to highlight common limitations and provide insights to improve future study designs and practical applicability of the radiomics models.

Ovarian imaging radiomics quality score assessment: an EuSoMII radiomics auditing group initiative.

OBJECTIVE: To evaluate the methodological rigor of radiomics-based studies using noninvasive imaging in ovarian setting. METHODS: Multiple medical literature archives (PubMed, Web of Science, and Scopus) were searched to retrieve original studies focused on computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US), or positron emission tomography (PET) radiomics for ovarian disorders' assessment. Two researchers in consensus evaluated each investigation using the radiomics quality score (RQS). Subgroup analyses were performed to assess whether the total RQS varied according to first author category, study aim and topic, imaging modality, and journal quartile. RESULTS: From a total of 531 items, 63 investigations were finally included in the analysis. The studies were greatly focused (94%) on the field of oncology, with CT representing the most used imaging technique (41%). Overall, the papers achieved a median total RQS 6 (IQR, -0.5 to 11), corresponding to a percentage of 16.7% of the maximum score (IQR, 0-30.6%). The scoring was low especially due to the lack of prospective design and formal validation of the results. At subgroup analysis, the 4 studies not focused on oncological topic showed significantly lower quality scores than the others. CONCLUSIONS: The overall methodological rigor of radiomics studies in the ovarian field is still not ideal, limiting the reproducibility of results and potential translation to clinical setting. More efforts towards a standardized methodology in the workflow are needed to allow radiomics to become a viable tool for clinical decision-making. KEY POINTS: • The 63 included studies using noninvasive imaging for ovarian applications were mostly focused on oncologic topic (94%). • The included investigations achieved a median total RQS 6 (IQR, -0.5 to 11), indicating poor methodological rigor. • The RQS was low especially due to the lack of prospective design and formal validation of the results.

Radiomics for the identification of extraprostatic extension with prostate MRI: a systematic review and meta-analysis.

OBJECTIVES: Extraprostatic extension (EPE) of prostate cancer (PCa) is predicted using clinical nomograms. Incorporating MRI could represent a leap forward, although poor sensitivity and standardization represent unsolved issues. MRI radiomics has been proposed for EPE prediction. The aim of the study was to systematically review the literature and perform a meta-analysis of MRI-based radiomics approaches for EPE prediction. MATERIALS AND METHODS: Multiple databases were systematically searched for radiomics studies on EPE detection up to June 2022. Methodological quality was appraised according to Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool and radiomics quality score (RQS). The area under the receiver operating characteristic curves (AUC) was pooled to estimate predictive accuracy. A random-effects model estimated overall effect size. Statistical heterogeneity was assessed with I2 value. Publication bias was evaluated with a funnel plot. Subgroup analyses were performed to explore heterogeneity. RESULTS: Thirteen studies were included, showing limitations in study design and methodological quality (median RQS 10/36), with high statistical heterogeneity. Pooled AUC for EPE identification was 0.80. In subgroup analysis, test-set and cross-validation-based studies had pooled AUC of 0.85 and 0.89 respectively. Pooled AUC was 0.72 for deep learning (DL)-based and 0.82 for handcrafted radiomics studies and 0.79 and 0.83 for studies with multiple and single scanner data, respectively. Finally, models with the best predictive performance obtained using radiomics features showed pooled AUC of 0.82, while those including clinical data of 0.76. CONCLUSION: MRI radiomics-powered models to identify EPE in PCa showed a promising predictive performance overall. However, methodologically robust, clinically driven research evaluating their diagnostic and therapeutic impact is still needed. CLINICAL RELEVANCE STATEMENT: Radiomics might improve the management of prostate cancer patients increasing the value of MRI in the assessment of extraprostatic extension. However, it is imperative that forthcoming research prioritizes confirmation studies and a stronger clinical orientation to solidify these advancements. KEY POINTS: • MRI radiomics deserves attention as a tool to overcome the limitations of MRI in prostate cancer local staging. • Pooled AUC was 0.80 for the 13 included studies, with high heterogeneity (84.7%, p < .001), methodological issues, and poor clinical orientation. • Methodologically robust radiomics research needs to focus on increasing MRI sensitivity and bringing added value to clinical nomograms at patient level.

MRI radiomics-based machine learning for classification of deep-seated lipoma and atypical lipomatous tumor of the extremities.

PURPOSE: To determine diagnostic performance of MRI radiomics-based machine learning for classification of deep-seated lipoma and atypical lipomatous tumor (ALT) of the extremities. MATERIAL AND METHODS: This retrospective study was performed at three tertiary sarcoma centers and included 150 patients with surgically treated and histology-proven lesions. The training-validation cohort consisted of 114 patients from centers 1 and 2 (n = 64 lipoma, n = 50 ALT). The external test cohort consisted of 36 patients from center 3 (n = 24 lipoma, n = 12 ALT). 3D segmentation was manually performed on T1- and T2-weighted MRI. After extraction and selection of radiomic features, three machine learning classifiers were trained and validated using nested fivefold cross-validation. The best-performing classifier according to previous analysis was evaluated and compared to an experienced musculoskeletal radiologist in the external test cohort. RESULTS: Eight features passed feature selection and were incorporated into the machine learning models. After training and validation (74% ROC-AUC), the best-performing classifier (Random Forest) showed 92% sensitivity and 33% specificity in the external test cohort with no statistical difference compared to the radiologist (p = 0.474). CONCLUSION: MRI radiomics-based machine learning may classify deep-seated lipoma and ALT of the extremities with high sensitivity and negative predictive value, thus potentially serving as a non-invasive screening tool to reduce unnecessary referral to tertiary tumor centers.

Cardiac CT and MRI radiomics: systematic review of the literature and radiomics quality score assessment.

OBJECTIVE: To systematically review and evaluate the methodological quality of studies using magnetic resonance imaging (MRI) and computed tomography (CT) radiomics for cardiac applications. METHODS: Multiple medical literature archives (PubMed, Web of Science, and EMBASE) were systematically searched to retrieve original studies focused on cardiac MRI and CT radiomics applications. Two researchers in consensus assessed each investigation using the radiomics quality score (RQS). Subgroup analyses were performed to assess whether the total RQS varied according to study aim, journal quartile, imaging modality, and first author category. RESULTS: From a total of 1961 items, 53 articles were finally included in the analysis. Overall, the studies reached a median total RQS of 7 (IQR, 4-12), corresponding to a percentage score of 19.4% (IQR, 11.1-33.3%). Item scores were particularly low due to lack of prospective design, cost-effectiveness analysis, and open science. Median RQS percentage score was significantly higher in papers where the first author was a medical doctor and in those published on first quartile journals. CONCLUSIONS: The overall methodological quality of radiomics studies in cardiac MRI and CT is still lacking. A higher degree of standardization of the radiomics workflow and higher publication standards for studies are required to allow its translation into clinical practice. KEY POINTS: • RQS has been recently proposed for the overall assessment of the methodological quality of radiomics-based studies. • The 53 included studies on cardiac MRI and CT radiomics applications reached a median total RQS of 7 (IQR, 4-12), corresponding to a percentage of 19.4% (IQR, 11.1-33.3%). • A more standardized methodology in the radiomics workflow is needed, especially in terms of study design, validation, and open science, in order to translate the results to clinical applications.

Virtual conferences: results of an international survey on radiologist preferences and perspectives.

BACKGROUND: We explored perceptions and preferences regarding the conversion of in-person to virtual conferences as necessitated by travel and in-person meeting restrictions. METHODS: A 16-question online survey to assess preferences regarding virtual conferences during the COVID-19 pandemic and future perspectives on this subject was disseminated internationally online between June and August 2020. FINDINGS: A total of 508 responses were received from 73 countries. The largest number of responses came from Italy and the USA. The majority of respondents had already attended a virtual conference (80%) and would like to attend future virtual meetings (97%). The ideal duration of such an event was 2-3 days (42%). The preferred time format was a 2-4-h session (43%). Most respondents also noted that they would like a significant fee reduction and the possibility to attend a conference partly in-person and partly online. Respondents indicated educational sessions as the most valuable sections of virtual meetings. The reported positive factor of the virtual meeting format is the ability to re-watch lectures on demand. On the other hand, the absence of networking and human contact was recognized as a significant loss. In the future, people expressed a preference to attend conferences in person for networking purposes, but only in safer conditions. CONCLUSIONS: Respondents appreciated the opportunity to attend the main radiological congresses online and found it a good opportunity to stay updated without having to travel. However, in general, they would prefer these conferences to be structured differently. The lack of networking opportunities was the main reason for preferring an in-person meeting. KEY POINTS: • Respondents appreciated the opportunity to attend the main radiological meetings online, considering it a good opportunity to stay updated without having to travel. • In the future, it is likely for congresses to offer attendance options both in person and online, making them more accessible to a larger audience. • Respondents indicated that networking represents the most valuable advantage of in-person conferences compared to online ones.

Cardiovascular magnetic resonance native T1 mapping in Anderson-Fabry disease: a systematic review and meta-analysis.

BACKGROUND: T1 mapping is an established cardiovascular magnetic resonance (CMR) technique that can characterize myocardial tissue. We aimed to determine the weighted mean native T1 values of Anderson-Fabry disease (AFD) patients and the standardized mean differences (SMD) as compared to healthy control subjects. METHODS: A comprehensive literature search of the PubMed, Scopus and Web of Science databases was conducted according to the PRISMA statement to retrieve original studies reporting myocardial native T1 values in AFD patients and healthy controls. A random effects model was used to calculate SMD, and meta-regression analysis was conducted to explore heterogeneity sources. Subgroup analysis was also performed according to scanner field strength and sequence type. RESULTS: From a total of 151 items, 14 articles were included in the final analysis accounting for a total population of 982 subjects. Overall, the weighted mean native T1 values was 984 ± 47 ms in AFD patients and 1016 ± 26 ms in controls (P < 0.0001) with a pooled SMD of - 2.38. In AFD patients there was an inverse correlation between native T1 values and male gender (P = 0.002) and left ventricular hypertrophy (LVH) (P < 0.001). Subgroup analyses confirmed lower T1 values in AFD patients compared to controls with a pooled SMD of -  2.54, -  2.28, -  2.46 for studies performed on 1.5T with modified Look-Locker inversion recovery (MOLLI), shortened MOLLI and saturation-recovery single-shot acquisition, respectively and of -  2.41 for studies conducted on 3T. CONCLUSIONS: Our findings confirm a reduction of native T1 values in AFD patients compared to healthy controls and point out that the degree of T1 shortening in AFD is influenced by gender and LVH. Although T1 mapping is useful in proving cardiac involvement in AFD patients, there is need to standardize shreshold values according to imaging equipment and protocols.

The impact of radiomics for human papillomavirus status prediction in oropharyngeal cancer: systematic review and radiomics quality score assessment.

PURPOSE: Human papillomavirus (HPV) status assessment is crucial for decision making in oropharyngeal cancer patients. In last years, several articles have been published investigating the possible role of radiomics in distinguishing HPV-positive from HPV-negative neoplasms. Aim of this review was to perform a systematic quality assessment of radiomic studies published on this topic. METHODS: Radiomics studies on HPV status prediction in oropharyngeal cancer patients were selected. The Radiomic Quality Score (RQS) was assessed by three readers to evaluate their methodological quality. In addition, possible correlations between RQS% and journal type, year of publication, impact factor, and journal rank were investigated. RESULTS: After the literature search, 19 articles were selected whose RQS median was 33% (range 0-42%). Overall, 16/19 studies included a well-documented imaging protocol, 13/19 demonstrated phenotypic differences, and all were compared with the current gold standard. No study included a public protocol, phantom study, or imaging at multiple time points. More than half (13/19) included feature selection and only 2 were comprehensive of non-radiomic features. Mean RQS was significantly higher in clinical journals. CONCLUSION: Radiomics has been proposed for oropharyngeal cancer HPV status assessment, with promising results. However, these are supported by low methodological quality investigations. Further studies with higher methodological quality, appropriate standardization, and greater attention to validation are necessary prior to clinical adoption.

Deep Learning Whole-Gland and Zonal Prostate Segmentation on a Public MRI Dataset.

BACKGROUND: Prostate volume, as determined by magnetic resonance imaging (MRI), is a useful biomarker both for distinguishing between benign and malignant pathology and can be used either alone or combined with other parameters such as prostate-specific antigen. PURPOSE: This study compared different deep learning methods for whole-gland and zonal prostate segmentation. STUDY TYPE: Retrospective. POPULATION: A total of 204 patients (train/test = 99/105) from the PROSTATEx public dataset. FIELD STRENGTH/SEQUENCE: A 3 T, TSE T2 -weighted. ASSESSMENT: Four operators performed manual segmentation of the whole-gland, central zone + anterior stroma + transition zone (TZ), and peripheral zone (PZ). U-net, efficient neural network (ENet), and efficient residual factorized ConvNet (ERFNet) were trained and tuned on the training data through 5-fold cross-validation to segment the whole gland and TZ separately, while PZ automated masks were obtained by the subtraction of the first two. STATISTICAL TESTS: Networks were evaluated on the test set using various accuracy metrics, including the Dice similarity coefficient (DSC). Model DSC was compared in both the training and test sets using the analysis of variance test (ANOVA) and post hoc tests. Parameter number, disk size, training, and inference times determined network computational complexity and were also used to assess the model performance differences. A P < 0.05 was selected to indicate the statistical significance. RESULTS: The best DSC (P < 0.05) in the test set was achieved by ENet: 91% ± 4% for the whole gland, 87% ± 5% for the TZ, and 71% ± 8% for the PZ. U-net and ERFNet obtained, respectively, 88% ± 6% and 87% ± 6% for the whole gland, 86% ± 7% and 84% ± 7% for the TZ, and 70% ± 8% and 65 ± 8% for the PZ. Training and inference time were lowest for ENet. DATA CONCLUSION: Deep learning networks can accurately segment the prostate using T2 -weighted images. EVIDENCE LEVEL: 4 TECHNICAL EFFICACY: Stage 2.

Machine learning solutions in radiology: does the emperor have no clothes?

KEY POINTS: • Interest in radiomics and machine learning is steadily increasing and is reflected both in research output and number of commercially available solutions.• Currently available commercial products using machine learning are often supported by limited evidence of clinical usefulness and studies are often of low methodological quality.• Ethical and regulatory issues remain open and hinder implementation of machine learning software packages in daily clinical practice.

MRI index lesion radiomics and machine learning for detection of extraprostatic extension of disease: a multicenter study.

OBJECTIVES: To build a machine learning (ML) model to detect extraprostatic extension (EPE) of prostate cancer (PCa), based on radiomics features extracted from prostate MRI index lesions. METHODS: Consecutive MRI exams of patients undergoing radical prostatectomy for PCa were retrospectively collected from three institutions. Axial T2-weighted and apparent diffusion coefficient map images were annotated to obtain index lesion volumes of interest for radiomics feature extraction. Data from one institution was used for training, feature selection (using reproducibility, variance and pairwise correlation analyses, and a correlation-based subset evaluator), and tuning of a support vector machine (SVM) algorithm, with stratified 10-fold cross-validation. The model was tested on the two remaining institutions' data and compared with a baseline reference and expert radiologist assessment of EPE. RESULTS: In total, 193 patients were included. From an initial dataset of 2436 features, 2287 were excluded due to either poor stability, low variance, or high collinearity. Among the remaining, 14 features were used to train the ML model, which reached an overall accuracy of 83% in the training set. In the two external test sets, the SVM achieved an accuracy of 79% and 74% respectively, not statistically different from that of the radiologist (81-83%, p = 0.39-1) and outperforming the baseline reference (p = 0.001-0.02). CONCLUSIONS: A ML model solely based on radiomics features demonstrated high accuracy for EPE detection and good generalizability in a multicenter setting. Paired to qualitative EPE assessment, this approach could aid radiologists in this challenging task. KEY POINTS: • Predicting the presence of EPE in prostate cancer patients is a challenging task for radiologists. • A support vector machine algorithm achieved high diagnostic accuracy for EPE detection, with good generalizability when tested on multiple external datasets. • The performance of the algorithm was not significantly different from that of an experienced radiologist.

Clinical value of radiomics and machine learning in breast ultrasound: a multicenter study for differential diagnosis of benign and malignant lesions.

OBJECTIVES: We aimed to assess the performance of radiomics and machine learning (ML) for classification of non-cystic benign and malignant breast lesions on ultrasound images, compare ML's accuracy with that of a breast radiologist, and verify if the radiologist's performance is improved by using ML. METHODS: Our retrospective study included patients from two institutions. A total of 135 lesions from Institution 1 were used to train and test the ML model with cross-validation. Radiomic features were extracted from manually annotated images and underwent a multistep feature selection process. Not reproducible, low variance, and highly intercorrelated features were removed from the dataset. Then, 66 lesions from Institution 2 were used as an external test set for ML and to assess the performance of a radiologist without and with the aid of ML, using McNemar's test. RESULTS: After feature selection, 10 of the 520 features extracted were employed to train a random forest algorithm. Its accuracy in the training set was 82% (standard deviation, SD, ± 6%), with an AUC of 0.90 (SD ± 0.06), while the performance on the test set was 82% (95% confidence intervals (CI) = 70-90%) with an AUC of 0.82 (95% CI = 0.70-0.93). It resulted in being significantly better than the baseline reference (p = 0.0098), but not different from the radiologist (79.4%, p = 0.815). The radiologist's performance improved when using ML (80.2%), but not significantly (p = 0.508). CONCLUSIONS: A radiomic analysis combined with ML showed promising results to differentiate benign from malignant breast lesions on ultrasound images. KEY POINTS: • Machine learning showed good accuracy in discriminating benign from malignant breast lesions • The machine learning classifier's performance was comparable to that of a breast radiologist • The radiologist's accuracy improved with machine learning, but not significantly.

Prognostic value of coronary flow reserve in patients with suspected or known coronary artery disease referred to PET myocardial perfusion imaging: A meta-analysis.

BACKGROUND: We performed a meta-meta-analysis to evaluate the prognostic value of coronary flow reserve (CFR) assessed by cardiac positron emission tomography (PET) imaging in patients with suspected or known coronary artery disease (CAD). METHODS: Studies published until April 2019 were identified by database search. We included studies if they evaluated CFR by PET providing data on adjusted hazard ratio (HR) for the occurrence of adverse events. Annualized event rates were calculated and the incidence rate ratios with 95% confidence interval (CI) were estimated to compare patients with impaired and preserved CFR. RESULTS: We identified 13 eligible articles including 11,867 patients with a follow-up ranging from 0.6 to 7.1 years. The HR for the occurrence of major adverse cardiac events (MACE) was reported in 11 studies and pooled HR was 1.93 (95% CI 1.65-2.27). The HR for the occurrence of hard events was reported in 5 studies and pooled HR was 3.11 (95% CI 1.88-5.14). Six studies reported data useful to calculate separately the incidence rate of MACE in patients with preserved and impaired CFR and pooled IRR was 2.26 (CI 95% 1.79-2.85). Three studies reported data useful to calculate separately the incidence rate of hard events in patients with preserved and impaired CFR and pooled IRR was 4.12 (CI 95% 3.08-5.51). At meta-regression analysis, we found an association between HR for MACE and gender, diabetes and hypertension, while no significant association was found between HR for hard events and demographic and clinical variables. CONCLUSION: In patients with suspected or known CAD, an impaired CFR is associated with adverse cardiovascular events. However, the large heterogeneity in study population underlines the need for further investigations to maximize the prognostic role of CFR.

Pretest models for predicting abnormal stress single-photon emission computed tomography myocardial perfusion imaging.

BACKGROUND: The frequency of abnormal stress SPECT myocardial perfusion imaging (MPS) decreased over the past decades despite an increase in the prevalence of cardiovascular risk factors. These findings strengthen the need to develop more effective strategies for appropriately referring patients with suspected coronary artery disease (CAD) to cardiac imaging. The aim of this study was to develop pretest assessment models for predicting abnormal stress MPS. METHODS: We included 5,601 consecutive patients with suspected CAD, who underwent stress MPS at our academic center. Two different models were considered: a basic model including age, gender, and anginal symptoms; and a clinical model including also diabetes, hypertension, hypercholesterolemia, smoking, and family history of CAD. RESULTS: In patients with abnormal MPS, the basic model classified more than 75% of patients as intermediate risk, whereas only 23% were incorrectly classified as low risk. In patients with normal MPS, 45% were correctly classified as low risk and none as high risk. Basic and clinical models had a limited discriminating capacity (area under the receiver operating characteristic curve 0.644 for basic model and 0.647 for clinical model) between individuals with and without abnormal stress MPS. The decision curve analysis demonstrates a high net benefit across a range of threshold probabilities from ~ 15% to ~30% for both models. CONCLUSIONS: A pretest risk stratification based on traditional cardiovascular risk factors has a limited value for predicting an abnormal stress MPS in patients with suspected CAD. However, selecting a proper threshold probability enhances the appropriateness of referral to stress MPS.