Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement from the ACR, CAR, ESR, RANZCR and RSNA.

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools. This article is simultaneously published in Insights into Imaging (DOI 10.1186/s13244-023-01541-3), Journal of Medical Imaging and Radiation Oncology (DOI 10.1111/1754-9485.13612), Canadian Association of Radiologists Journal (DOI 10.1177/08465371231222229), Journal of the American College of Radiology (DOI 10.1016/j.jacr.2023.12.005), and Radiology: Artificial Intelligence (DOI 10.1148/ryai.230513). Keywords: Artificial Intelligence, Radiology, Automation, Machine Learning Published under a CC BY 4.0 license. ©The Author(s) 2024. Editor's Note: The RSNA Board of Directors has endorsed this article. It has not undergone review or editing by this journal.

Developing, purchasing, implementing and monitoring AI tools in radiology: Practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA.

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.

A paradigm shift in point-of-care imaging in low-income and middle-income countries.

The concept of primary healthcare is now regarded as crucial for enhancing access to healthcare services in low-income and middle-income countries (LMICs). Technological advancements that have made many medical imaging devices smaller, lighter, portable and more affordable, and infrastructure advancements in power supply, Internet connectivity, and artificial intelligence, are all increasing the feasibility of POCI (point-of care imaging) in LMICs. Although providing imaging services at the same time as the clinic visit represents a paradigm shift in the way imaging care is typically provided in high-income countries where patients are typically directed to dedicated imaging centres, a POCI model is often the only way to provide timely access to imaging care for many patients in LIMCs. To address the growing burden of non-communicable diseases such as cancer and heart disease, bringing advanced imaging tools to the POCI will be necessary. Strategies tailored to the countries' specific needs, including training, safety and quality, will be of the utmost importance.

Multi-Institutional Assessment and Crowdsourcing Evaluation of Deep Learning for Automated Classification of Breast Density.

OBJECTIVE: We developed deep learning algorithms to automatically assess BI-RADS breast density. METHODS: Using a large multi-institution patient cohort of 108,230 digital screening mammograms from the Digital Mammographic Imaging Screening Trial, we investigated the effect of data, model, and training parameters on overall model performance and provided crowdsourcing evaluation from the attendees of the ACR 2019 Annual Meeting. RESULTS: Our best-performing algorithm achieved good agreement with radiologists who were qualified interpreters of mammograms, with a four-class κ of 0.667. When training was performed with randomly sampled images from the data set versus sampling equal number of images from each density category, the model predictions were biased away from the low-prevalence categories such as extremely dense breasts. The net result was an increase in sensitivity and a decrease in specificity for predicting dense breasts for equal class compared with random sampling. We also found that the performance of the model degrades when we evaluate on digital mammography data formats that differ from the one that we trained on, emphasizing the importance of multi-institutional training sets. Lastly, we showed that crowdsourced annotations, including those from attendees who routinely read mammograms, had higher agreement with our algorithm than with the original interpreting radiologists. CONCLUSION: We demonstrated the possible parameters that can influence the performance of the model and how crowdsourcing can be used for evaluation. This study was performed in tandem with the development of the ACR AI-LAB, a platform for democratizing artificial intelligence.

Evolving Use of Prebiopsy Prostate Magnetic Resonance Imaging in the Medicare Population.

PURPOSE: We assessed the changing use of prebiopsy prostate magnetic resonance imaging in Medicare beneficiaries. MATERIALS AND METHODS: Men who underwent prostate biopsy were identified in 5% Medicare RIFs (Research Identifiable Files) from October 2010 through September 2015. We evaluated the rate of prebiopsy prostate magnetic resonance imaging, defined as any pelvic MRI 6 months or less before biopsy with a prostate indication diagnosis code. Temporal changes were determined as well as variation by geography and among populations. RESULTS: In male Medicare beneficiaries the prebiopsy magnetic resonance imaging use rate increased from 0.1% in 2010 to 0.7% in 2011, to 1.2% in 2012, to 2.9% in 2013, to 4.7% in 2014 and to 10.3% in 2015. In 2015 the prebiopsy prostate magnetic resonance imaging rate varied significantly by patient age, including 5.7% for greater than 80 years vs 8.4% to 9.3% for other age ranges (p = 0.040) as well as by race, including 5.8% in African American vs 10.1% in Caucasian men (p = 0.009) and geographic region, including 6.3% in the Midwest to 12.5% in the Northeast (p <0.001). The rate was highest in Wyoming at 25.0%, New York at 23.7% and Minnesota at 20.5% but it was less than 1% in 10 states. CONCLUSIONS: Historical Medicare claims provide novel insights into the dramatically increasing adoption of magnetic resonance imaging prior to prostate biopsy. Following earlier minimal use the performance increased sharply beginning in 2013, exceeding 10% in 2015. However, substantial racial and geographic variation exists in adoption. Continued educational, research and policy efforts are warranted to optimize the role of prebiopsy magnetic resonance imaging and minimize sociodemographic and geographic disparities.

Identifying Radiology's Place in the Expanding Landscape of Episode Payment Models.

The current fee-for-service system for health care reimbursement in the United Stated is argued to encourage fragmented care delivery and a lack of accountability that predisposes to insufficient focus on quality as well as unnecessary or duplicative resource utilization. Episode payment models (EPMs) seek to improve coordination by linking payments for all services related to a patient's condition or procedure, thereby improving quality and efficiency of care. The CMS Innovation Center has implemented a broadening array of EPMs. Early models with relevance to radiologists include Bundled Payment for Care Improvement (involving 48 possible clinical conditions), Comprehensive Care for Joint Replacement (involving knee and hip replacement), and the Oncology Care Model (involving chemotherapy). In July 2016, CMS expanded the range of EPMs through three new models with mandatory hospital participation addressing inpatient and 90-day postdischarge care for acute myocardial infarction, coronary artery bypass graft, and surgical hip and femur fracture treatment. Moreover, some of the EPMs include tracks that allow participating entities to qualify as an Advanced Alternative Payment Model under the Medicare Access and CHIP Reauthorization Act (MACRA), reaping the associated reporting and payment benefits. Even though none of the available EPMs are radiology specific, the models will nevertheless likely influence reimbursements for some radiologists. Thus, radiologists should partner with hospitals and other specialties in care coordination through these episode-based initiatives, thereby having opportunities to apply their imaging expertise to help lower spending while improving quality and overall levels of health.

Professional efficiencies for diagnostic imaging services rendered by different physicians: analysis of recent medicare multiple procedure payment reduction policy.

PURPOSE: The aim of this study was to quantify potential physician work efficiencies and appropriate multiple procedure payment reductions for different same-session diagnostic imaging studies interpreted by different physicians in the same group practice. METHODS: Medicare Resource-Based Relative Value Scale data were analyzed to determine the relative contributions of various preservice, intraservice, and postservice physician diagnostic imaging work activities. An expert panel quantified potential duplications in professional work activities when separate examinations were performed during the same session by different physicians within the same group practice. Maximum potential work duplications for various imaging modalities were calculated and compared with those used as the basis of CMS payment policy. RESULTS: No potential intraservice work duplication was identified when different examination interpretations were rendered by different physicians in the same group practice. When multiple interpretations within the same modality were rendered by different physicians, maximum potential duplicated preservice and postservice activities ranged from 5% (radiography, fluoroscopy, and nuclear medicine) to 13.6% (CT). Maximum mean potential duplicated work relative value units ranged from 0.0049 (radiography and fluoroscopy) to 0.0413 (CT). This equates to overall potential total work reductions ranging from 1.39% (nuclear medicine) to 2.73% (CT). Across all modalities, this corresponds to maximum Medicare professional component physician fee reductions of 1.23 ± 0.38% (range, 0.95%-1.87%) for services within the same modality, much less than an order of magnitude smaller than those implemented by CMS. For services from different modalities, potential duplications were too small to quantify. CONCLUSIONS: Although potential efficiencies exist in physician preservice and postservice work when same-session, same-modality imaging services are rendered by different physicians in the same group practice, these are relatively minuscule and have been grossly overestimated by current CMS payment policy. Greater transparency and methodologic rigor in government payment policy development are warranted.