Nonradiology Health-Care Professionals Significantly Benefit From AI Assistance in Emergency-Related Chest Radiography Interpretation.

BACKGROUND: Chest radiographs (CXRs) are still of crucial importance in primary diagnostics, but their interpretation poses difficulties at times. RESEARCH QUESTION: Can a convolutional neural network-based artificial intelligence (AI) system that interprets CXRs add value in an emergency unit setting? STUDY DESIGN AND METHODS: A total of 563 CXRs acquired in the emergency unit of a major university hospital were retrospectively assessed twice by three board-certified radiologists, three radiology residents, and three emergency unit-experienced nonradiology residents (NRRs). They used a two-step reading process: (1) without AI support (woAI); and (2) with AI support (wAI) providing additional images with AI overlays. Suspicion of four suspected pathologies (pleural effusion, pneumothorax, consolidations suspicious for pneumonia, and nodules) was reported on a five-point confidence scale. Confidence scores of the board-certified radiologists were converted into four binary reference standards (RFS I-IV) of different sensitivities. Performance by radiology residents and NRRs woAI/wAI were statistically compared by using receiver-operating characteristics (ROCs), Youden statistics, and operating point metrics derived from fitted ROC curves. RESULTS: NRRs could significantly improve performance, sensitivity, and accuracy wAI in all four pathologies tested. In the most sensitive RFS IV, NRR consensus improved the area under the ROC curve (mean, 95% CI) in the detection of the time-critical pathology pneumothorax from 0.846 (0.785-0.907) woAI to 0.974 (0.947-1.000) wAI (P < .001), which represented a gain of 30% in sensitivity and 2% in accuracy (while maintaining an optimized specificity). The most pronounced effect was observed in nodule detection, with NRR wAI improving sensitivity by 53% and accuracy by 7% (area under the ROC curve woAI, 0.723 [0.661-0.785]; wAI, 0.890 [0.848-0.931]; P < .001). The RR consensus wAI showed smaller, mostly nonsignificant gains in performance, sensitivity, and accuracy. INTERPRETATION: In an emergency unit setting without 24/7 radiology coverage, the presented AI solution features an excellent clinical support tool to nonradiologists, similar to a second reader, and allows for a more accurate primary diagnosis and thus earlier therapy initiation.

Clinically focused multi-cohort benchmarking as a tool for external validation of artificial intelligence algorithm performance in basic chest radiography analysis.

Artificial intelligence (AI) algorithms evaluating [supine] chest radiographs ([S]CXRs) have remarkably increased in number recently. Since training and validation are often performed on subsets of the same overall dataset, external validation is mandatory to reproduce results and reveal potential training errors. We applied a multicohort benchmarking to the publicly accessible (S)CXR analyzing AI algorithm CheXNet, comprising three clinically relevant study cohorts which differ in patient positioning ([S]CXRs), the applied reference standards (CT-/[S]CXR-based) and the possibility to also compare algorithm classification with different medical experts' reading performance. The study cohorts include [1] a cohort, characterized by 563 CXRs acquired in the emergency unit that were evaluated by 9 readers (radiologists and non-radiologists) in terms of 4 common pathologies, [2] a collection of 6,248 SCXRs annotated by radiologists in terms of pneumothorax presence, its size and presence of inserted thoracic tube material which allowed for subgroup and confounding bias analysis and [3] a cohort consisting of 166 patients with SCXRs that were evaluated by radiologists for underlying causes of basal lung opacities, all of those cases having been correlated to a timely acquired computed tomography scan (SCXR and CT within < 90 min). CheXNet non-significantly exceeded the radiology resident (RR) consensus in the detection of suspicious lung nodules (cohort [1], AUC AI/RR: 0.851/0.839, p = 0.793) and the radiological readers in the detection of basal pneumonia (cohort [3], AUC AI/reader consensus: 0.825/0.782, p = 0.390) and basal pleural effusion (cohort [3], AUC AI/reader consensus: 0.762/0.710, p = 0.336) in SCXR, partly with AUC values higher than originally published ("Nodule": 0.780, "Infiltration": 0.735, "Effusion": 0.864). The classifier "Infiltration" turned out to be very dependent on patient positioning (best in CXR, worst in SCXR). The pneumothorax SCXR cohort [2] revealed poor algorithm performance in CXRs without inserted thoracic material and in the detection of small pneumothoraces, which can be explained by a known systematic confounding error in the algorithm training process. The benefit of clinically relevant external validation is demonstrated by the differences in algorithm performance as compared to the original publication. Our multi-cohort benchmarking finally enables the consideration of confounders, different reference standards and patient positioning as well as the AI performance comparison with differentially qualified medical readers.

Reduction of missed thoracic findings in emergency whole-body computed tomography using artificial intelligence assistance.

BACKGROUND: Radiology reporting of emergency whole-body computed tomography (CT) scans is time-critical and therefore involves a significant risk of pathology under-detection. We hypothesize a relevant number of initially missed secondary thoracic findings that would have been detected by an artificial intelligence (AI) software platform including several pathology-specific AI algorithms. METHODS: This retrospective proof-of-concept-study consecutively included 105 shock-room whole-body CT scans. Image data was analyzed by platform-bundled AI-algorithms, findings were reviewed by radiology experts and compared with the original radiologist's reports. We focused on secondary thoracic findings, such as cardiomegaly, coronary artery plaques, lung lesions, aortic aneurysms and vertebral fractures. RESULTS: We identified a relevant number of initially missed findings, with their quantification based on 105 analyzed CT scans as follows: up to 25 patients (23.8%) with cardiomegaly or borderline heart size, 17 patients (16.2%) with coronary plaques, 34 patients (32.4%) with aortic ectasia, 2 patients (1.9%) with lung lesions classified as "recommended to control" and 13 initially missed vertebral fractures (two with an acute traumatic origin). A high number of false positive or non-relevant AI-based findings remain problematic especially regarding lung lesions and vertebral fractures. CONCLUSIONS: We consider AI to be a promising approach to reduce the number of missed findings in clinical settings with a necessary time-critical radiological reporting. Nevertheless, algorithm improvement is necessary focusing on a reduction of "false positive" findings and on algorithm features assessing the finding relevance, e.g., fracture age or lung lesion malignancy.

COVID-19 Pandemic and Upcoming Influenza Season-Does an Expert's Computed Tomography Assessment Differentially Identify COVID-19, Influenza and Pneumonias of Other Origin?

(1) Background: Time-consuming SARS-CoV-2 RT-PCR suffers from limited sensitivity in early infection stages whereas fast available chest CT can already raise COVID-19 suspicion. Nevertheless, radiologists' performance to differentiate COVID-19, especially from influenza pneumonia, is not sufficiently characterized. (2) Methods: A total of 201 pneumonia CTs were identified and divided into subgroups based on RT-PCR: 78 COVID-19 CTs, 65 influenza CTs and 62 Non-COVID-19-Non-influenza (NCNI) CTs. Three radiology experts (blinded from RT-PCR results) raised pathogen-specific suspicion (separately for COVID-19, influenza, bacterial pneumonia and fungal pneumonia) according to the following reading scores: 0-not typical/1-possible/2-highly suspected. Diagnostic performances were calculated with RT-PCR as a reference standard. Dependencies of radiologists' pathogen suspicion scores were characterized by Pearson's Chi2 Test for Independence. (3) Results: Depending on whether the intermediate reading score 1 was considered as positive or negative, radiologists correctly classified 83-85% (vs. NCNI)/79-82% (vs. influenza) of COVID-19 cases (sensitivity up to 94%). Contrarily, radiologists correctly classified only 52-56% (vs. NCNI)/50-60% (vs. COVID-19) of influenza cases. The COVID-19 scoring was more specific than the influenza scoring compared with suspected bacterial or fungal infection. (4) Conclusions: High-accuracy COVID-19 detection by CT might expedite patient management even during the upcoming influenza season.

Artificial Intelligence Algorithm Detecting Lung Infection in Supine Chest Radiographs of Critically Ill Patients With a Diagnostic Accuracy Similar to Board-Certified Radiologists.

OBJECTIVES: Interpretation of lung opacities in ICU supine chest radiographs remains challenging. We evaluated a prototype artificial intelligence algorithm to classify basal lung opacities according to underlying pathologies. DESIGN: Retrospective study. The deep neural network was trained on two publicly available datasets including 297,541 images of 86,876 patients. PATIENTS: One hundred sixty-six patients received both supine chest radiograph and CT scans (reference standard) within 90 minutes without any intervention in between. MEASUREMENTS AND MAIN RESULTS: Algorithm accuracy was referenced to board-certified radiologists who evaluated supine chest radiographs according to side-separate reading scores for pneumonia and effusion (0 = absent, 1 = possible, and 2 = highly suspected). Radiologists were blinded to the supine chest radiograph findings during CT interpretation. Performances of radiologists and the artificial intelligence algorithm were quantified by receiver-operating characteristic curve analysis. Diagnostic metrics (sensitivity, specificity, positive predictive value, negative predictive value, and accuracy) were calculated based on different receiver-operating characteristic operating points. Regarding pneumonia detection, radiologists achieved a maximum diagnostic accuracy of up to 0.87 (95% CI, 0.78-0.93) when considering only the supine chest radiograph reading score 2 as positive for pneumonia. Radiologist's maximum sensitivity up to 0.87 (95% CI, 0.76-0.94) was achieved by additionally rating the supine chest radiograph reading score 1 as positive for pneumonia and taking previous examinations into account. Radiologic assessment essentially achieved nonsignificantly higher results compared with the artificial intelligence algorithm: artificial intelligence-area under the receiver-operating characteristic curve of 0.737 (0.659-0.815) versus radiologist's area under the receiver-operating characteristic curve of 0.779 (0.723-0.836), diagnostic metrics of receiver-operating characteristic operating points did not significantly differ. Regarding the detection of pleural effusions, there was no significant performance difference between radiologist's and artificial intelligence algorithm: artificial intelligence-area under the receiver-operating characteristic curve of 0.740 (0.662-0.817) versus radiologist's area under the receiver-operating characteristic curve of 0.698 (0.646-0.749) with similar diagnostic metrics for receiver-operating characteristic operating points. CONCLUSIONS: Considering the minor level of performance differences between the algorithm and radiologists, we regard artificial intelligence as a promising clinical decision support tool for supine chest radiograph examinations in the clinical routine with high potential to reduce the number of missed findings in an artificial intelligence-assisted reading setting.