Comparison of Commercial AI Software Performance for Radiograph Lung Nodule Detection and Bone Age Prediction.

Background Multiple commercial artificial intelligence (AI) products exist for assessing radiographs; however, comparable performance data for these algorithms are limited. Purpose To perform an independent, stand-alone validation of commercially available AI products for bone age prediction based on hand radiographs and lung nodule detection on chest radiographs. Materials and Methods This retrospective study was carried out as part of Project AIR. Nine of 17 eligible AI products were validated on data from seven Dutch hospitals. For bone age prediction, the root mean square error (RMSE) and Pearson correlation coefficient were computed. The reference standard was set by three to five expert readers. For lung nodule detection, the area under the receiver operating characteristic curve (AUC) was computed. The reference standard was set by a chest radiologist based on CT. Randomized subsets of hand (n = 95) and chest (n = 140) radiographs were read by 14 and 17 human readers, respectively, with varying experience. Results Two bone age prediction algorithms were tested on hand radiographs (from January 2017 to January 2022) in 326 patients (mean age, 10 years ± 4 [SD]; 173 female patients) and correlated strongly with the reference standard (r = 0.99; P < .001 for both). No difference in RMSE was observed between algorithms (0.63 years [95% CI: 0.58, 0.69] and 0.57 years [95% CI: 0.52, 0.61]) and readers (0.68 years [95% CI: 0.64, 0.73]). Seven lung nodule detection algorithms were validated on chest radiographs (from January 2012 to May 2022) in 386 patients (mean age, 64 years ± 11; 223 male patients). Compared with readers (mean AUC, 0.81 [95% CI: 0.77, 0.85]), four algorithms performed better (AUC range, 0.86-0.93; P value range, <.001 to .04). Conclusions Compared with human readers, four AI algorithms for detecting lung nodules on chest radiographs showed improved performance, whereas the remaining algorithms tested showed no evidence of a difference in performance. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Omoumi and Richiardi in this issue.

Direct access to magnetic resonance imaging improved orthopaedic knee referrals in the Netherlands.

BACKGROUND: The use of magnetic resonance imaging (MRI) in primary care is under debate, and the majority of GPs have no experience with MRI. OBJECTIVES: To examine for which patients with knee injury an MRI is ordered and does direct access to MRI in primary care influence the GP referral to an orthopaedic surgeon? METHODS: Consecutive patients with knee injury who underwent an MRI examination ordered by their GP were included. On the application form for MRI, the GPs indicated their referral intention in advance, as if MRI had not been available. Six months after the MRI scan, written interviews with the GPs were used to collect data on referrals and orthopaedic intervention. The number of patients finally referred to an orthopaedic surgeon in secondary care after MRI was compared with the number of intended referrals. RESULTS: Of the 588 included, GPs referred fewer patients to the orthopaedic surgeon after receiving the MRI results than they would have done prior to MRI (60% versus 82.8%, P < 0.0001). The reduction was 16.1% for patients older than 50 years and 28.1% for patients younger than 50 years. Orthopaedic intervention was performed in 62.9% of all referred patients. Of the 101 patients whom the GP did not intend to refer prior to MRI, 48 were referred to an orthopaedic surgeon based on the MRI findings. CONCLUSION: In patients with knee injury, direct access to MRI of the knee in a primary care setting significantly reduced referrals to an orthopaedic surgeon. LEVEL OF EVIDENCE: Three prospective cohort.