Your browser doesn't support javascript.
loading
Comparison of Commercial AI Software Performance for Radiograph Lung Nodule Detection and Bone Age Prediction.
van Leeuwen, Kicky G; Schalekamp, Steven; Rutten, Matthieu J C M; Huisman, Merel; Schaefer-Prokop, Cornelia M; de Rooij, Maarten; van Ginneken, Bram; Maresch, Bas; Geurts, Bram H J; van Dijke, Cornelius F; Laupman-Koedam, Emmeline; Hulleman, Enzo V; Verhoeff, Eric L; Meys, Evelyne M J; Mohamed Hoesein, Firdaus A A; Ter Brugge, Floor M; van Hoorn, Francois; van der Wel, Frank; van den Berk, Inge A H; Luyendijk, Jacqueline M; Meakin, James; Habets, Jesse; Verbeke, Jonathan I M L; Nederend, Joost; Meys, Karlijn M E; Deden, Laura N; Langezaal, Lucianne C M; Nasrollah, Mahtab; Meij, Marleen; Boomsma, Martijn F; Vermeulen, Matthijs; Vestering, Myrthe M; Vijlbrief, Onno; Algra, Paul; Algra, Selma; Bollen, Stijn M; Samson, Tijs; von Brucken Fock, Yntor H G.
Affiliation
  • van Leeuwen KG; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Schalekamp S; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Rutten MJCM; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Huisman M; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Schaefer-Prokop CM; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • de Rooij M; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • van Ginneken B; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Maresch B; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Geurts BHJ; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • van Dijke CF; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Laupman-Koedam E; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Hulleman EV; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Verhoeff EL; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Meys EMJ; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Mohamed Hoesein FAA; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Ter Brugge FM; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • van Hoorn F; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • van der Wel F; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • van den Berk IAH; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Luyendijk JM; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Meakin J; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Habets J; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Verbeke JIML; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Nederend J; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Meys KME; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Deden LN; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Langezaal LCM; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Nasrollah M; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Meij M; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Boomsma MF; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Vermeulen M; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Vestering MM; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Vijlbrief O; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Algra P; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Algra S; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Bollen SM; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • Samson T; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
  • von Brucken Fock YHG; From the Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands (K.G.v.L., S.S., M.J.C.M.R., M.H., C.M.S.P., M.d.R., B.v.G., B.H.J.G., J.M.); Department of Radiology (M.J.C.M.R.) and Department of MICT and Imaging Techniques (T.
Radiology ; 310(1): e230981, 2024 01.
Article de En | MEDLINE | ID: mdl-38193833
ABSTRACT
Background Multiple commercial artificial intelligence (AI) products exist for assessing radiographs; however, comparable performance data for these algorithms are limited. Purpose To perform an independent, stand-alone validation of commercially available AI products for bone age prediction based on hand radiographs and lung nodule detection on chest radiographs. Materials and Methods This retrospective study was carried out as part of Project AIR. Nine of 17 eligible AI products were validated on data from seven Dutch hospitals. For bone age prediction, the root mean square error (RMSE) and Pearson correlation coefficient were computed. The reference standard was set by three to five expert readers. For lung nodule detection, the area under the receiver operating characteristic curve (AUC) was computed. The reference standard was set by a chest radiologist based on CT. Randomized subsets of hand (n = 95) and chest (n = 140) radiographs were read by 14 and 17 human readers, respectively, with varying experience. Results Two bone age prediction algorithms were tested on hand radiographs (from January 2017 to January 2022) in 326 patients (mean age, 10 years ± 4 [SD]; 173 female patients) and correlated strongly with the reference standard (r = 0.99; P < .001 for both). No difference in RMSE was observed between algorithms (0.63 years [95% CI 0.58, 0.69] and 0.57 years [95% CI 0.52, 0.61]) and readers (0.68 years [95% CI 0.64, 0.73]). Seven lung nodule detection algorithms were validated on chest radiographs (from January 2012 to May 2022) in 386 patients (mean age, 64 years ± 11; 223 male patients). Compared with readers (mean AUC, 0.81 [95% CI 0.77, 0.85]), four algorithms performed better (AUC range, 0.86-0.93; P value range, <.001 to .04). Conclusions Compared with human readers, four AI algorithms for detecting lung nodules on chest radiographs showed improved performance, whereas the remaining algorithms tested showed no evidence of a difference in performance. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Omoumi and Richiardi in this issue.
Sujet(s)
Texte intégral
Ajouter à My VHL
Imprimer
XML
PubMed Links
Recherche sur Google

Texte intégral: 1 Collection: 01-internacional Base de données: MEDLINE Sujet principal: Logiciel / Intelligence artificielle Type d'étude: Diagnostic_studies / Observational_studies / Prognostic_studies / Risk_factors_studies Limites: Child / Female / Humans / Male / Middle aged Langue: En Journal: Radiology Année: 2024 Type de document: Article Pays de publication: États-Unis d'Amérique
Texte intégral
Ajouter à My VHL
Imprimer
XML
PubMed Links
Recherche sur Google

Texte intégral: 1 Collection: 01-internacional Base de données: MEDLINE Sujet principal: Logiciel / Intelligence artificielle Type d'étude: Diagnostic_studies / Observational_studies / Prognostic_studies / Risk_factors_studies Limites: Child / Female / Humans / Male / Middle aged Langue: En Journal: Radiology Année: 2024 Type de document: Article Pays de publication: États-Unis d'Amérique