How to read and review papers on machine learning and artificial intelligence in radiology: a survival guide to key methodological concepts.

In recent years, there has been a dramatic increase in research papers about machine learning (ML) and artificial intelligence in radiology. With so many papers around, it is of paramount importance to make a proper scientific quality assessment as to their validity, reliability, effectiveness, and clinical applicability. Due to methodological complexity, the papers on ML in radiology are often hard to evaluate, requiring a good understanding of key methodological issues. In this review, we aimed to guide the radiology community about key methodological aspects of ML to improve their academic reading and peer-review experience. Key aspects of ML pipeline were presented within four broad categories: study design, data handling, modelling, and reporting. Sixteen key methodological items and related common pitfalls were reviewed with a fresh perspective: database size, robustness of reference standard, information leakage, feature scaling, reliability of features, high dimensionality, perturbations in feature selection, class balance, bias-variance trade-off, hyperparameter tuning, performance metrics, generalisability, clinical utility, comparison with traditional tools, data sharing, and transparent reporting.Key Points• Machine learning is new and rather complex for the radiology community.• Validity, reliability, effectiveness, and clinical applicability of studies on machine learning can be evaluated with a proper understanding of key methodological concepts about study design, data handling, modelling, and reporting.• Understanding key methodological concepts will provide a better academic reading and peer-review experience for the radiology community.

Artificial Intelligence in Renal Mass Characterization: A Systematic Review of Methodologic Items Related to Modeling, Performance Evaluation, Clinical Utility, and Transparency.

OBJECTIVE. The objective of our study was to systematically review the literature about the application of artificial intelligence (AI) to renal mass characterization with a focus on the methodologic quality items. MATERIALS AND METHODS. A systematic literature search was conducted using PubMed to identify original research studies about the application of AI to renal mass characterization. Besides baseline study characteristics, a total of 15 methodologic quality items were extracted and evaluated on the basis of the following four main categories: modeling, performance evaluation, clinical utility, and transparency items. The qualitative synthesis was presented using descriptive statistics with an accompanying narrative. RESULTS. Thirty studies were included in this systematic review. Overall, the methodologic quality items were mostly favorable for modeling (63%) and performance evaluation (63%). Even so, the studies (57%) more frequently constructed their work on nonrobust features. Furthermore, only a few studies (10%) had a generalizability assessment with independent or external validation. The studies were mostly unsuccessful in terms of clinical utility evaluation (89%) and transparency (97%) items. For clinical utility, the interesting findings were lack of comparisons with both radiologists' evaluation (87%) and traditional models (70%) in most of the studies. For transparency, most studies (97%) did not share their data with the public. CONCLUSION. To bring AI-based renal mass characterization from research to practice, future studies need to improve modeling and performance evaluation strategies and pay attention to clinical utility and transparency issues.

Machine Learning in Radiomic Renal Mass Characterization: Fundamentals, Applications, Challenges, and Future Directions.

OBJECTIVE. The purpose of this study is to provide an overview of the traditional machine learning (ML)-based and deep learning-based radiomic approaches, with focus placed on renal mass characterization. CONCLUSION. ML currently has a very low barrier to entry into general medical practice because of the availability of many open-source, free, and easy-to-use toolboxes. Therefore, it should not be surprising to see its related applications in renal mass characterization. A wider picture of the previous works might be beneficial to move this field forward.