RESUMO
PURPOSE: Diagnosing pulmonary embolism (PE) is still challenging due to other conditions that can mimic its appearance, leading to incomplete or delayed management and several inter-observer variabilities. This study evaluated the performance and clinical utility of an artificial intelligence (AI)-based application designed to assist clinicians in the detection of PE on CT pulmonary angiography (CTPA). PATIENTS AND METHODS: CTPAs from 230 US cities acquired on 57 scanner models from 6 different vendors were retrospectively collected. Three US board certified expert radiologists defined the ground truth by majority agreement. The same cases were analyzed by CINA-PE, an AI-driven algorithm capable of detecting and highlighting suspected PE locations. The algorithm's performance at a per-case and per-finding level was evaluated. Furthermore, cases with PE not mentioned in the clinical report but correctly detected by the algorithm were analyzed. RESULTS: A total of 1204 CTPAs (mean age 62.1 years ± 16.6[SD], 44.4 % female, 14.9 % positive) were included in the study. Per-case sensitivity and specificity were 93.9 % (95%CI: 89.3 %-96.9 %) and 94.8 % (95%CI: 93.3 %-96.1 %), respectively. Per-finding positive predictive value was 89.5 % (95%CI: 86.7 %-91.9 %). Among the 196 positive cases, 29 (15.6 %) were not mentioned in the clinical report. The algorithm detected 22/29 (76 %) of these cases, leading to a reduction in the miss rate from 15.6 % to 3.8 % (7/186). CONCLUSIONS: The AI-based application may improve diagnostic accuracy in detecting PE and enhance patient outcomes through timely intervention. Integrating AI tools in clinical workflows can reduce missed or delayed diagnoses, and positively impact healthcare delivery and patient care.
RESUMO
BACKGROUND AND PURPOSE: Recently, AI tools have been deployed with increasing speed in educational and clinical settings. However, the use of AI by trainees across different levels of experience has not been well studied. This study investigates the impact of AI assistance on diagnostic accuracy for intracranial hemorrhage (ICH) and large vessel occlusion (LVO) by medical students (MS) and resident trainees (RT). MATERIALS AND METHODS: This prospective study was conducted between March 2023 and October 2023. MS and RT were asked to identify ICH and LVO in 100 non-contrast head CTs and 100 head CTAs, respectively. One group received diagnostic aid simulating AI for ICH only (n = 26), the other for LVO only (n = 28). Primary outcomes included accuracy, sensitivity, and specificity for ICH / LVO detection without and with aid. Study interpretation time was a secondary outcome. Individual responses were pooled and analyzed with chi-square; differences in continuous variables were assessed with ANOVA. RESULTS: 48 participants completed the study, generating 10,779 ICH or LVO interpretations. With diagnostic aid, MS accuracy improved 11.0 points (P < .001) and RT accuracy showed no significant change. ICH interpretation time increased with diagnostic aid for both groups (P < .001) while LVO interpretation time decreased for MS (P < .001). Despite worse performance in detection of the smallest vs. the largest hemorrhages at baseline, MS were not more likely to accept a true positive AI result for these more difficult tasks. Both groups were considerably less accurate when disagreeing with the AI or when supplied with an incorrect AI result. CONCLUSIONS: This study demonstrated greater improvement in diagnostic accuracy with AI for MS compared to RT. However, MS were less likely than RT to overrule incorrect AI interpretations and were less accurate, even with diagnostic aid, than the AI was by itself. ABBREVIATIONS: ICH = intracranial hemorrhage; LVO = large vessel occlusion; MS = medical students; RT = resident trainees.