Comparison of Commercial AI Software Performance for Radiograph Lung Nodule Detection and Bone Age Prediction.

Background Multiple commercial artificial intelligence (AI) products exist for assessing radiographs; however, comparable performance data for these algorithms are limited. Purpose To perform an independent, stand-alone validation of commercially available AI products for bone age prediction based on hand radiographs and lung nodule detection on chest radiographs. Materials and Methods This retrospective study was carried out as part of Project AIR. Nine of 17 eligible AI products were validated on data from seven Dutch hospitals. For bone age prediction, the root mean square error (RMSE) and Pearson correlation coefficient were computed. The reference standard was set by three to five expert readers. For lung nodule detection, the area under the receiver operating characteristic curve (AUC) was computed. The reference standard was set by a chest radiologist based on CT. Randomized subsets of hand (n = 95) and chest (n = 140) radiographs were read by 14 and 17 human readers, respectively, with varying experience. Results Two bone age prediction algorithms were tested on hand radiographs (from January 2017 to January 2022) in 326 patients (mean age, 10 years ± 4 [SD]; 173 female patients) and correlated strongly with the reference standard (r = 0.99; P < .001 for both). No difference in RMSE was observed between algorithms (0.63 years [95% CI: 0.58, 0.69] and 0.57 years [95% CI: 0.52, 0.61]) and readers (0.68 years [95% CI: 0.64, 0.73]). Seven lung nodule detection algorithms were validated on chest radiographs (from January 2012 to May 2022) in 386 patients (mean age, 64 years ± 11; 223 male patients). Compared with readers (mean AUC, 0.81 [95% CI: 0.77, 0.85]), four algorithms performed better (AUC range, 0.86-0.93; P value range, <.001 to .04). Conclusions Compared with human readers, four AI algorithms for detecting lung nodules on chest radiographs showed improved performance, whereas the remaining algorithms tested showed no evidence of a difference in performance. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Omoumi and Richiardi in this issue.

AI-based lumbar central canal stenosis classification on sagittal MR images is comparable to experienced radiologists using axial images.

OBJECTIVES: The assessment of lumbar central canal stenosis (LCCS) is crucial for diagnosing and planning treatment for patients with low back pain and neurogenic pain. However, manual assessment methods are time-consuming, variable, and require axial MRIs. The aim of this study is to develop and validate an AI-based model that automatically classifies LCCS using sagittal T2-weighted MRIs. METHODS: A pre-existing 3D AI algorithm was utilized to segment the spinal canal and intervertebral discs (IVDs), enabling quantitative measurements at each IVD level. Four musculoskeletal radiologists graded 683 IVD levels from 186 LCCS patients using the 4-class Lee grading system. A second consensus reading was conducted by readers 1 and 2, which, along with automatic measurements, formed the training dataset for a multiclass (grade 0-3) and binary (grade 0-1 vs. 2-3) random forest classifier with tenfold cross-validation. RESULTS: The multiclass model achieved a Cohen's weighted kappa of 0.86 (95% CI: 0.82-0.90), comparable to readers 3 and 4 with 0.85 (95% CI: 0.80-0.89) and 0.73 (95% CI: 0.68-0.79) respectively. The binary model demonstrated an AUC of 0.98 (95% CI: 0.97-0.99), sensitivity of 93% (95% CI: 91-96%), and specificity of 91% (95% CI: 87-95%). In comparison, readers 3 and 4 achieved a specificity of 98 and 99% and sensitivity of 74 and 54%, respectively. CONCLUSION: Both the multiclass and binary models, while only using sagittal MR images, perform on par with experienced radiologists who also had access to axial sequences. This underscores the potential of this novel algorithm in enhancing diagnostic accuracy and efficiency in medical imaging. KEY POINTS: Question How can the classification of lumbar central canal stenosis (LCCS) be made more efficient? Findings Multiclass and binary AI models, using only sagittal MR images, performed on par with experienced radiologists who also had access to axial sequences. Clinical relevance Our AI algorithm accurately classifies LCCS from sagittal MRI, matching experienced radiologists. This study offers a promising tool for automated LCCS assessment from sagittal T2 MRI, potentially reducing the reliance on additional axial imaging.

Development of an exercise testing protocol for patients with a lower limb amputation: results of a pilot study.

Due to a decrease in physical activity, lower limb amputees experience a decline in physical fitness. This causes problems in walking with a prosthesis because energy expenditure in walking with a prosthesis is much higher than in walking with two sound legs. Exercise training may therefore increase the functional walking ability of these patients. To generate a safe and effective aerobic training program, exercise testing of amputees is recommended. The objectives of this study were to develop a maximal exercise testing protocol for lower limb amputees and to compare two different testing methods: combined arm-leg ergometry and arm ergometry. The protocols were tested in five amputee patients. Combined ergometry elicited a higher oxygen uptake and heart rate than arm ergometry. Electrocardiography during combined ergometry was easier to read. Combined ergometry was judged most comfortable by the amputees. The exercise testing protocol was useful in lower limb amputees to determine their maximal aerobic capacity and their main exercise limitation. Future exercise training programs may be based on this testing protocol. Combined arm-leg ergometry is appropriate for unilateral amputees without significant claudication of the remaining leg. Continuous arm ergometry is suitable for unilateral amputees with significant claudication of the remaining limb or bilateral amputees.