Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA.

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever­growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi­society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.

Assessment of Radiology Artificial Intelligence Software: A Validation and Evaluation Framework.

Artificial intelligence (AI) software in radiology is becoming increasingly prevalent and performance is improving rapidly with new applications for given use cases being developed continuously, oftentimes with development and validation occurring in parallel. Several guidelines have provided reporting standards for publications of AI-based research in medicine and radiology. Yet, there is an unmet need for recommendations on the assessment of AI software before adoption and after commercialization. As the radiology AI ecosystem continues to grow and mature, a formalization of system assessment and evaluation is paramount to ensure patient safety, relevance and support to clinical workflows, and optimal allocation of limited AI development and validation resources before broader implementation into clinical practice. To fulfil these needs, we provide a glossary for AI software types, use cases and roles within the clinical workflow; list healthcare needs, key performance indicators and required information about software prior to assessment; and lay out examples of software performance metrics per software category. This conceptual framework is intended to streamline communication with the AI software industry and provide healthcare decision makers and radiologists with tools to assess the potential use of these software. The proposed software evaluation framework lays the foundation for a radiologist-led prospective validation network of radiology AI software. Learning Points: The rapid expansion of AI applications in radiology requires standardization of AI software specification, classification, and evaluation. The Canadian Association of Radiologists' AI Tech & Apps Working Group Proposes an AI Specification document format and supports the implementation of a clinical expert evaluation process for Radiology AI software.

Incidence of infectious complications following ultrasound-guided percutaneous musculoskeletal interventions with the use of an uncovered transducer footprint.

OBJECTIVES: To determine the incidence of infectious complications following ultrasound-guided musculoskeletal interventions performed with a disinfected uncovered ultrasound transducer footprint. METHODS: Electronic medical records of all patients who underwent an ultrasound-guided musculoskeletal procedure (including injection, calcific lavage, or ganglion cyst aspiration) performed by any of the 14 interventional musculoskeletal radiologists at our institution between January 2013 and December 2018 were retrospectively reviewed to identify procedure site infections. Biopsies and joint aspirations were excluded. The procedures were performed using a disinfected uncovered transducer footprint. First, an automated chart review identified cases with (1) positive answers to the nurse's post-procedure call, (2) an International Classification of Diseases (ICD) diagnostic code related to a musculoskeletal infection, or (3) an antibiotic prescription within 30 days post-procedure. Then, these cases were manually reviewed for evidence of procedure site infection. RESULTS: In total, 6511 procedures were included. The automated chart review identified 3 procedures (2 patients) in which post-procedural fever was reported during the nurse's post-procedure call, 33 procedures (28 patients) with an ICD code for a musculoskeletal infection, and 220 procedures (216 patients) with an antibiotic prescription within 30 post-procedural days. The manual chart review of these patients revealed no cases of confirmed infection and 1 case (0.015%) of possible site infection. CONCLUSIONS: The incidence of infectious complications after an ultrasound-guided musculoskeletal procedure performed with an uncovered transducer footprint is extremely low. This information allows radiologists to counsel their patients more precisely when obtaining informed consent. KEY POINTS: • Infectious complications after ultrasound-guided musculoskeletal procedures performed with a disinfected uncovered transducer footprint are extremely rare.

Top 10 Reviewer Critiques of Radiology Artificial Intelligence (AI) Articles: Qualitative Thematic Analysis of Reviewer Critiques of Machine Learning/Deep Learning Manuscripts Submitted to JMRI.

BACKGROUND: Classical machine learning (ML) and deep learning (DL) articles have rapidly captured the attention of the radiology research community and comprise an increasing proportion of articles submitted to JMRI, of variable reporting and methodological quality. PURPOSE: To identify the most frequent reviewer critiques of classical ML and DL articles submitted to JMRI. STUDY TYPE: Qualitative thematic analysis. POPULATION: In all, 1396 manuscript journal articles submitted to JMRI for consideration in 2018, with thematic analysis performed of reviewer critiques of 38 artificial intelligence (AI) articles, comprised of 24 ML and 14 DL articles, from January 9, 2018 to June 2, 2018. FIELD STRENGTH/SEQUENCE: N/A. ASSESSMENT: After identifying and sampling ML and DL articles, and collecting all reviews, qualitative thematic analysis was performed to identify major and minor themes of reviewer critiques. STATISTICAL TESTS: Descriptive statistics provided of article characteristics, and thematic review of major and minor themes. RESULTS: Thirty-eight articles were sampled for thematic review: 24 (63.2%) focused on classical ML and 14 (36.8%) on DL. The overall acceptance rate of classical ML/DL articles was 28.9%, similar to the overall 2017-2019 acceptance rate of 23.1-28.1%. These articles resulted in 72 reviews analyzed, yielding a total 713 critiques that underwent formal thematic analysis consensus encoding. Ten major themes of critiques were identified, with 1-Lack of Information as the most frequent, comprising 268 (37.6%) of all critiques. Frequent minor themes of critiques concerning ML/DL-specific recommendations included performing basic clinical statistics such as to ensure similarity of training and test groups (N = 26), emphasizing strong clinical Gold Standards for the basis of training labels (N = 19), and ensuring strong radiological relevance of the topic and task performed (N = 16). DATA CONCLUSION: Standardized reporting of ML and DL methods could help address nearly one-third of all reviewer critiques made. LEVEL OF EVIDENCE: 4 Technical Efficacy Stage: 1 J. Magn. Reson. Imaging 2020;52:248-254.

Radiographic features in investigated for Pneumocystis jirovecii pneumonia: a nested case-control study.

BACKGROUND: Pneumocystis jirovecii pneumonia (PJP) can be challenging to diagnose, often requiring bronchoscopy. Since most patients suspected of PJP undergo imaging, we hypothesized that the findings of these studies could help estimate the probability of disease prior to invasive testing. METHODS: We created a cohort of patients who underwent bronchoscopy specifically to diagnose PJP and conducted a nested case-control study to compare the radiographic features between patients with (n = 72) and without (n = 288) pathologically proven PJP. We used multivariable logistic regression to identify radiographic features independently associated with PJP. RESULTS: Chest x-ray findings poorly predicted the diagnosis of PJP. However, multivariable analysis of CT scan findings found that "increased interstitial markings" (OR 4.3; 95%CI 2.2-8.2), "ground glass opacities" (OR 3.3; 95%CI 1.2-9.1) and the radiologist's impression of PJP being "possible" (OR 2.0; 95%CI 1.0-4.1) or "likely" (OR 9.3; 95%CI 3.4-25.3) were independently associated with the final diagnosis (c-statistic 0.75). CONCLUSIONS: Where there is clinical suspicion of PJP, the use of CT scan can help determine the probability of PJP. Identifying patients at low risk of PJP may enable better use of non-invasive testing to avoid bronchoscopy while higher probability patients could be prioritized.

Pattern Recognition in Musculoskeletal Imaging Using Artificial Intelligence.

Artificial intelligence (AI) has the potential to affect every step of the radiology workflow, but the AI application that has received the most press in recent years is image interpretation, with numerous articles describing how AI can help detect and characterize abnormalities as well as monitor disease response. Many AI-based image interpretation tasks for musculoskeletal (MSK) pathologies have been studied, including the diagnosis of bone tumors, detection of osseous metastases, assessment of bone age, identification of fractures, and detection and grading of osteoarthritis. This article explores the applications of AI for image interpretation of MSK pathologies.

Artificial intelligence in radiology: who's afraid of the big bad wolf?

This Editorial comment refers to the article "Medical students' attitude towards artificial intelligence: a multicenter survey," Pinto Dos Santos D, et al Eur Radiol 2018. KEY POINTS: • Medical students are not well informed of the potential consequences of AI in radiology. • The fundamental principles of AI-as well as its application in medicine-must be taught in medical schools. • The radiologist specialty must actively reflect on how to validate, approve, and integrate AI algorithms into our clinical practices.

Image-based biomarkers for solid tumor quantification.

The last few decades have witnessed tremendous technological developments in image-based biomarkers for tumor quantification and characterization. Initially limited to manual one- and two-dimensional size measurements, image biomarkers have evolved to harness developments not only in image acquisition technology but also in image processing and analysis algorithms. At the same time, clinical validation remains a major challenge for the vast majority of these novel techniques, and there is still a major gap between the latest technological developments and image biomarkers used in everyday clinical practice. Currently, the imaging biomarker field is attracting increasing attention not only because of the tremendous interest in cutting-edge therapeutic developments and personalized medicine but also because of the recent progress in the application of artificial intelligence (AI) algorithms to large-scale datasets. Thus, the goal of the present article is to review the current state of the art for image biomarkers and their use for characterization and predictive quantification of solid tumors. Beginning with an overview of validated imaging biomarkers in current clinical practice, we proceed to a review of AI-based methods for tumor characterization, such as radiomics-based approaches and deep learning.Key Points• Recent years have seen tremendous technological developments in image-based biomarkers for tumor quantification and characterization.• Image-based biomarkers can be used on an ongoing basis, in a non-invasive (or mildly invasive) way, to monitor the development and progression of the disease or its response to therapy.• We review the current state of the art for image biomarkers, as well as the recent developments in artificial intelligence (AI) algorithms for image processing and analysis.

Demystification of AI-driven medical image interpretation: past, present and future.

The recent explosion of 'big data' has ushered in a new era of artificial intelligence (AI) algorithms in every sphere of technological activity, including medicine, and in particular radiology. However, the recent success of AI in certain flagship applications has, to some extent, masked decades-long advances in computational technology development for medical image analysis. In this article, we provide an overview of the history of AI methods for radiological image analysis in order to provide a context for the latest developments. We review the functioning, strengths and limitations of more classical methods as well as of the more recent deep learning techniques. We discuss the unique characteristics of medical data and medical science that set medicine apart from other technological domains in order to highlight not only the potential of AI in radiology but also the very real and often overlooked constraints that may limit the applicability of certain AI methods. Finally, we provide a comprehensive perspective on the potential impact of AI on radiology and on how to evaluate it not only from a technical point of view but also from a clinical one, so that patients can ultimately benefit from it. KEY POINTS: • Artificial intelligence (AI) research in medical imaging has a long history • The functioning, strengths and limitations of more classical AI methods is reviewed, together with that of more recent deep learning methods. • A perspective is provided on the potential impact of AI on radiology and on its evaluation from both technical and clinical points of view.

Canadian Association of Radiologists White Paper on Ethical and Legal Issues Related to Artificial Intelligence in Radiology.

Artificial intelligence (AI) software that analyzes medical images is becoming increasingly prevalent. Unlike earlier generations of AI software, which relied on expert knowledge to identify imaging features, machine learning approaches automatically learn to recognize these features. However, the promise of accurate personalized medicine can only be fulfilled with access to large quantities of medical data from patients. This data could be used for purposes such as predicting disease, diagnosis, treatment optimization, and prognostication. Radiology is positioned to lead development and implementation of AI algorithms and to manage the associated ethical and legal challenges. This white paper from the Canadian Association of Radiologists provides a framework for study of the legal and ethical issues related to AI in medical imaging, related to patient data (privacy, confidentiality, ownership, and sharing); algorithms (levels of autonomy, liability, and jurisprudence); practice (best practices and current legal framework); and finally, opportunities in AI from the perspective of a universal health care system.

Features from Computerized Texture Analysis of Breast Cancers at Pretreatment MR Imaging Are Associated with Response to Neoadjuvant Chemotherapy.

Purpose To evaluate whether features from texture analysis of breast cancers were associated with pathologic complete response (pCR) after neoadjuvant chemotherapy and to explore the association between texture features and tumor subtypes at pretreatment magnetic resonance (MR) imaging. Materials and Methods Institutional review board approval was obtained. This retrospective study included 85 patients with 85 breast cancers who underwent breast MR imaging before neoadjuvant chemotherapy between April 10, 2008, and March 12, 2015. Two-dimensional texture analysis was performed by using software at T2-weighted MR imaging and contrast material-enhanced T1-weighted MR imaging. Quantitative parameters were compared between patients with pCR and those with non-pCR and between patients with triple-negative breast cancer and those with non-triple-negative cancer. Multiple logistic regression analysis was used to determine independent parameters. Results Eighteen tumors (22%) were triple-negative breast cancers. pCR was achieved in 30 of the 85 tumors (35%). At univariate analysis, mean pixel intensity with spatial scaling factor (SSF) of 2 and 4 on T2-weighted images and kurtosis on contrast-enhanced T1-weighted images showed a significant difference between triple-negative breast cancer and non-triple-negative breast cancer (P = .009, .003, and .001, respectively). Kurtosis (SSF, 2) on T2-weighted images showed a significant difference between pCR and non-pCR (P = .015). At multiple logistic regression, kurtosis on T2-weighted images was independently associated with pCR in non-triple-negative breast cancer (P = .033). A multivariate model incorporating T2-weighted and contrast-enhanced T1-weighted kurtosis showed good performance for the identification of triple-negative breast cancer (area under the receiver operating characteristic curve, 0.834). Conclusion At pretreatment MR imaging, kurtosis appears to be associated with pCR to neoadjuvant chemotherapy in non-triple-negative breast cancer and may be a promising biomarker for the identification of triple-negative breast cancer. © RSNA, 2017.

Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology.

Artificial intelligence (AI) is rapidly moving from an experimental phase to an implementation phase in many fields, including medicine. The combination of improved availability of large datasets, increasing computing power, and advances in learning algorithms has created major performance breakthroughs in the development of AI applications. In the last 5 years, AI techniques known as deep learning have delivered rapidly improving performance in image recognition, caption generation, and speech recognition. Radiology, in particular, is a prime candidate for early adoption of these techniques. It is anticipated that the implementation of AI in radiology over the next decade will significantly improve the quality, value, and depth of radiology's contribution to patient care and population health, and will revolutionize radiologists' workflows. The Canadian Association of Radiologists (CAR) is the national voice of radiology committed to promoting the highest standards in patient-centered imaging, lifelong learning, and research. The CAR has created an AI working group with the mandate to discuss and deliberate on practice, policy, and patient care issues related to the introduction and implementation of AI in imaging. This white paper provides recommendations for the CAR derived from deliberations between members of the AI working group. This white paper on AI in radiology will inform CAR members and policymakers on key terminology, educational needs of members, research and development, partnerships, potential clinical applications, implementation, structure and governance, role of radiologists, and potential impact of AI on radiology in Canada.