Pneumothorax detection in chest radiographs: optimizing artificial intelligence system for accuracy and confounding bias reduction using in-image annotations in algorithm training.

OBJECTIVES: Diagnostic accuracy of artificial intelligence (AI) pneumothorax (PTX) detection in chest radiographs (CXR) is limited by the noisy annotation quality of public training data and confounding thoracic tubes (TT). We hypothesize that in-image annotations of the dehiscent visceral pleura for algorithm training boosts algorithm's performance and suppresses confounders. METHODS: Our single-center evaluation cohort of 3062 supine CXRs includes 760 PTX-positive cases with radiological annotations of PTX size and inserted TTs. Three step-by-step improved algorithms (differing in algorithm architecture, training data from public datasets/clinical sites, and in-image annotations included in algorithm training) were characterized by area under the receiver operating characteristics (AUROC) in detailed subgroup analyses and referenced to the well-established "CheXNet" algorithm. RESULTS: Performances of established algorithms exclusively trained on publicly available data without in-image annotations are limited to AUROCs of 0.778 and strongly biased towards TTs that can completely eliminate algorithm's discriminative power in individual subgroups. Contrarily, our final "algorithm 2" which was trained on a lower number of images but additionally with in-image annotations of the dehiscent pleura achieved an overall AUROC of 0.877 for unilateral PTX detection with a significantly reduced TT-related confounding bias. CONCLUSIONS: We demonstrated strong limitations of an established PTX-detecting AI algorithm that can be significantly reduced by designing an AI system capable of learning to both classify and localize PTX. Our results are aimed at drawing attention to the necessity of high-quality in-image localization in training data to reduce the risks of unintentionally biasing the training process of pathology-detecting AI algorithms. KEY POINTS: • Established pneumothorax-detecting artificial intelligence algorithms trained on public training data are strongly limited and biased by confounding thoracic tubes. • We used high-quality in-image annotated training data to effectively boost algorithm performance and suppress the impact of confounding thoracic tubes. • Based on our results, we hypothesize that even hidden confounders might be effectively addressed by in-image annotations of pathology-related image features.

Evaluation of patients with respiratory infections during the first pandemic wave in Germany: characteristics of COVID-19 versus non-COVID-19 patients.

BACKGROUND: Characteristics of COVID-19 patients have mainly been reported within confirmed COVID-19 cohorts. By analyzing patients with respiratory infections in the emergency department during the first pandemic wave, we aim to assess differences in the characteristics of COVID-19 vs. Non-COVID-19 patients. This is particularly important regarding the second COVID-19 wave and the approaching influenza season. METHODS: We prospectively included 219 patients with suspected COVID-19 who received radiological imaging and RT-PCR for SARS-CoV-2. Demographic, clinical and laboratory parameters as well as RT-PCR results were used for subgroup analysis. Imaging data were reassessed using the following scoring system: 0 - not typical, 1 - possible, 2 - highly suspicious for COVID-19. RESULTS: COVID-19 was diagnosed in 72 (32,9%) patients. In three of them (4,2%) the initial RT-PCR was negative while initial CT scan revealed pneumonic findings. 111 (50,7%) patients, 61 of them (55,0%) COVID-19 positive, had evidence of pneumonia. Patients with COVID-19 pneumonia showed higher body temperature (37,7 ± 0,1 vs. 37,1 ± 0,1 °C; p = 0.0001) and LDH values (386,3 ± 27,1 vs. 310,4 ± 17,5 U/l; p = 0.012) as well as lower leukocytes (7,6 ± 0,5 vs. 10,1 ± 0,6G/l; p = 0.0003) than patients with other pneumonia. Among abnormal CT findings in COVID-19 patients, 57 (93,4%) were evaluated as highly suspicious or possible for COVID-19. In patients with negative RT-PCR and pneumonia, another third was evaluated as highly suspicious or possible for COVID-19 (14 out of 50; 28,0%). The sensitivity in the detection of patients requiring isolation was higher with initial chest CT than with initial RT-PCR (90,4% vs. 79,5%). CONCLUSIONS: COVID-19 patients show typical clinical, laboratory and imaging parameters which enable a sensitive detection of patients who demand isolation measures due to COVID-19.

Macrophage ablation significantly reduces uptake of imaging probe into organs of the reticuloendothelial system.

BACKGROUND: Macrophages engulf particulate contrast media, which is pivotal for biomedical imaging. PURPOSE: To introduce a macrophage ablation animal model by showing its power to manipulate the kinetics of imaging probes. MATERIAL AND METHODS: The kinetics of a particulate computed tomography (CT) contrast media was compared in macrophage ablative mice and normal mice. Liposomes (size 220 µg), loaded with clodronate, were injected into the peritoneum of three C57BL/6 mice. On the third day, 200 µL of the particulate agent ExiTron nano 6000 were injected into three macrophage-ablative mice and three control mice. CT scans were acquired before and 3 min, 1 h, 6 h, and 24 h after the ExiTron application. The animals were sacrificed, and their spleens and livers removed. Relative CT values (CTV) were measured and analyzed. RESULTS: Liver and spleen enhancement of treated mice and controls were increasing over time. The median peak values were different with 225 CTV for treated mice and 582 CTV for controls in the liver (P = 0.032) and 431 CTV for treated and 974 CTV in controls in the spleen (P = 0.016). CONCLUSION: Macrophage ablation leads to a decrease of enhancement in organs containing high numbers of macrophages, but only marginal changes in macrophage-poor organs. Macrophage ablation can influence the phagocytic activity and thus opens new potentials to investigate and manipulate the uptake of imaging probes.

Artificial Intelligence Algorithm Detecting Lung Infection in Supine Chest Radiographs of Critically Ill Patients With a Diagnostic Accuracy Similar to Board-Certified Radiologists.

OBJECTIVES: Interpretation of lung opacities in ICU supine chest radiographs remains challenging. We evaluated a prototype artificial intelligence algorithm to classify basal lung opacities according to underlying pathologies. DESIGN: Retrospective study. The deep neural network was trained on two publicly available datasets including 297,541 images of 86,876 patients. PATIENTS: One hundred sixty-six patients received both supine chest radiograph and CT scans (reference standard) within 90 minutes without any intervention in between. MEASUREMENTS AND MAIN RESULTS: Algorithm accuracy was referenced to board-certified radiologists who evaluated supine chest radiographs according to side-separate reading scores for pneumonia and effusion (0 = absent, 1 = possible, and 2 = highly suspected). Radiologists were blinded to the supine chest radiograph findings during CT interpretation. Performances of radiologists and the artificial intelligence algorithm were quantified by receiver-operating characteristic curve analysis. Diagnostic metrics (sensitivity, specificity, positive predictive value, negative predictive value, and accuracy) were calculated based on different receiver-operating characteristic operating points. Regarding pneumonia detection, radiologists achieved a maximum diagnostic accuracy of up to 0.87 (95% CI, 0.78-0.93) when considering only the supine chest radiograph reading score 2 as positive for pneumonia. Radiologist's maximum sensitivity up to 0.87 (95% CI, 0.76-0.94) was achieved by additionally rating the supine chest radiograph reading score 1 as positive for pneumonia and taking previous examinations into account. Radiologic assessment essentially achieved nonsignificantly higher results compared with the artificial intelligence algorithm: artificial intelligence-area under the receiver-operating characteristic curve of 0.737 (0.659-0.815) versus radiologist's area under the receiver-operating characteristic curve of 0.779 (0.723-0.836), diagnostic metrics of receiver-operating characteristic operating points did not significantly differ. Regarding the detection of pleural effusions, there was no significant performance difference between radiologist's and artificial intelligence algorithm: artificial intelligence-area under the receiver-operating characteristic curve of 0.740 (0.662-0.817) versus radiologist's area under the receiver-operating characteristic curve of 0.698 (0.646-0.749) with similar diagnostic metrics for receiver-operating characteristic operating points. CONCLUSIONS: Considering the minor level of performance differences between the algorithm and radiologists, we regard artificial intelligence as a promising clinical decision support tool for supine chest radiograph examinations in the clinical routine with high potential to reduce the number of missed findings in an artificial intelligence-assisted reading setting.

Language model-based labeling of German thoracic radiology reports.

The aim of this study was to explore the potential of weak supervision in a deep learning-based label prediction model. The goal was to use this model to extract labels from German free-text thoracic radiology reports on chest X-ray images and for training chest X-ray classification models.The proposed label extraction model for German thoracic radiology reports uses a German BERT encoder as a backbone and classifies a report based on the CheXpert labels. For investigating the efficient use of manually annotated data, the model was trained using manual annotations, weak rule-based labels, and both. Rule-based labels were extracted from 66071 retrospectively collected radiology reports from 2017-2021 (DS 0), and 1091 reports from 2020-2021 (DS 1) were manually labeled according to the CheXpert classes. Label extraction performance was evaluated with respect to mention extraction, negation detection, and uncertainty detection by measuring F1 scores. The influence of the label extraction method on chest X-ray classification was evaluated on a pneumothorax data set (DS 2) containing 6434 chest radiographs with associated reports and expert diagnoses of pneumothorax. For this, DenseNet-121 models trained on manual annotations, rule-based and deep learning-based label predictions, and publicly available data were compared.The proposed deep learning-based labeler (DL) performed on average considerably stronger than the rule-based labeler (RB) for all three tasks on DS 1 with F1 scores of 0.938 vs. 0.844 for mention extraction, 0.891 vs. 0.821 for negation detection, and 0.624 vs. 0.518 for uncertainty detection. Pre-training on DS 0 and fine-tuning on DS 1 performed better than only training on either DS 0 or DS 1. Chest X-ray pneumothorax classification results (DS 2) were highest when trained with DL labels with an area under the receiver operating curve (AUC) of 0.939 compared to RB labels with an AUC of 0.858. Training with manual labels performed slightly worse than training with DL labels with an AUC of 0.934. In contrast, training with a public data set resulted in an AUC of 0.720.Our results show that leveraging a rule-based report labeler for weak supervision leads to improved labeling performance. The pneumothorax classification results demonstrate that our proposed deep learning-based labeler can serve as a substitute for manual labeling requiring only 1000 manually annotated reports for training. · The proposed deep learning-based label extraction model for German thoracic radiology reports performs better than the rule-based model.. · Training with limited supervision outperformed training with a small manually labeled data set.. · Using predicted labels for pneumothorax classification from chest radiographs performed equally to using manual annotations.. Wollek A, Haitzer P, Sedlmeyr T et al. Language modelbased labeling of German thoracic radiology reports. Fortschr Röntgenstr 2024; DOI 10.1055/a-2287-5054.

German CheXpert Chest X-ray Radiology Report Labeler.

PURPOSE: The aim of this study was to develop an algorithm to automatically extract annotations from German thoracic radiology reports to train deep learning-based chest X-ray classification models. MATERIALS AND METHODS: An automatic label extraction model for German thoracic radiology reports was designed based on the CheXpert architecture. The algorithm can extract labels for twelve common chest pathologies, the presence of support devices, and "no finding". For iterative improvements and to generate a ground truth, a web-based multi-reader annotation interface was created. With the proposed annotation interface, a radiologist annotated 1086 retrospectively collected radiology reports from 2020-2021 (data set 1). The effect of automatically extracted labels on chest radiograph classification performance was evaluated on an additional, in-house pneumothorax data set (data set 2), containing 6434 chest radiographs with corresponding reports, by comparing a DenseNet-121 model trained on extracted labels from the associated reports, image-based pneumothorax labels, and publicly available data, respectively. RESULTS: Comparing automated to manual labeling on data set 1: "mention extraction" class-wise F1 scores ranged from 0.8 to 0.995, the "negation detection" F1 scores from 0.624 to 0.981, and F1 scores for "uncertainty detection" from 0.353 to 0.725. Extracted pneumothorax labels on data set 2 had a sensitivity of 0.997 [95 % CI: 0.994, 0.999] and specificity of 0.991 [95 % CI: 0.988, 0.994]. The model trained on publicly available data achieved an area under the receiver operating curve (AUC) for pneumothorax classification of 0.728 [95 % CI: 0.694, 0.760], while the models trained on automatically extracted labels and on manual annotations achieved values of 0.858 [95 % CI: 0.832, 0.882] and 0.934 [95 % CI: 0.918, 0.949], respectively. CONCLUSION: Automatic label extraction from German thoracic radiology reports is a promising substitute for manual labeling. By reducing the time required for data annotation, larger training data sets can be created, resulting in improved overall modeling performance. Our results demonstrated that a pneumothorax classifier trained on automatically extracted labels strongly outperformed the model trained on publicly available data, without the need for additional annotation time and performed competitively compared to manually labeled data. KEY POINTS: · An algorithm for automatic German thoracic radiology report annotation was developed.. · Automatic label extraction is a promising substitute for manual labeling.. · The classifier trained on extracted labels outperformed the model trained on publicly available data..

Artificial Intelligence to Assess Tracheal Tubes and Central Venous Catheters in Chest Radiographs Using an Algorithmic Approach With Adjustable Positioning Definitions.

PURPOSE: To develop and validate an artificial intelligence algorithm for the positioning assessment of tracheal tubes (TTs) and central venous catheters (CVCs) in supine chest radiographs (SCXRs) by using an algorithm approach allowing for adjustable definitions of intended device positioning. MATERIALS AND METHODS: Positioning quality of CVCs and TTs is evaluated by spatially correlating the respective tip positions with anatomical structures. For CVC analysis, a configurable region of interest is defined to approximate the expected region of well-positioned CVC tips from segmentations of anatomical landmarks. The CVC/TT information is estimated by introducing a new multitask neural network architecture for jointly performing type/existence classification, course segmentation, and tip detection. Validation data consisted of 589 SCXRs that have been radiologically annotated for inserted TTs/CVCs, including an experts' categorical positioning assessment (reading 1). In-image positions of algorithm-detected TT/CVC tips could be corrected using a validation software tool (reading 2) that finally allowed for localization accuracy quantification. Algorithmic detection of images with misplaced devices (reading 1 as reference standard) was quantified by receiver operating characteristics. RESULTS: Supine chest radiographs were correctly classified according to inserted TTs/CVCs in 100%/98% of the cases, thereby with high accuracy in also spatially localizing the medical device tips: corrections less than 3 mm in >86% (TTs) and 77% (CVCs) of the cases. Chest radiographs with malpositioned devices were detected with area under the curves of >0.98 (TTs), >0.96 (CVCs with accidental vessel turnover), and >0.93 (also suboptimal CVC insertion length considered). The receiver operating characteristics limitations regarding CVC assessment were mainly caused by limitations of the applied CXR position definitions (region of interest derived from anatomical landmarks), not by algorithmic spatial detection inaccuracies. CONCLUSIONS: The TT and CVC tips were accurately localized in SCXRs by the presented algorithms, but triaging applications for CVC positioning assessment still suffer from the vague definition of optimal CXR positioning. Our algorithm, however, allows for an adjustment of these criteria, theoretically enabling them to meet user-specific or patient subgroups requirements. Besides CVC tip analysis, future work should also include specific course analysis for accidental vessel turnover detection.

Implementing Artificial Intelligence for Emergency Radiology Impacts Physicians' Knowledge and Perception: A Prospective Pre- and Post-Analysis.

PURPOSE: The aim of this study was to evaluate the impact of implementing an artificial intelligence (AI) solution for emergency radiology into clinical routine on physicians' perception and knowledge. MATERIALS AND METHODS: A prospective interventional survey was performed pre-implementation and 3 months post-implementation of an AI algorithm for fracture detection on radiographs in late 2022. Radiologists and traumatologists were asked about their knowledge and perception of AI on a 7-point Likert scale (-3, "strongly disagree"; +3, "strongly agree"). Self-generated identification codes allowed matching the same individuals pre-intervention and post-intervention, and using Wilcoxon signed rank test for paired data. RESULTS: A total of 47/71 matched participants completed both surveys (66% follow-up rate) and were eligible for analysis (34 radiologists [72%], 13 traumatologists [28%], 15 women [32%]; mean age, 34.8 ± 7.8 years). Postintervention, there was an increase that AI "reduced missed findings" (1.28 [pre] vs 1.94 [post], P = 0.003) and made readers "safer" (1.21 vs 1.64, P = 0.048), but not "faster" (0.98 vs 1.21, P = 0.261). There was a rising disagreement that AI could "replace the radiological report" (-2.04 vs -2.34, P = 0.038), as well as an increase in self-reported knowledge about "clinical AI," its "chances," and its "risks" (0.40 vs 1.00, 1.21 vs 1.70, and 0.96 vs 1.34; all P 's ≤ 0.028). Radiologists used AI results more frequently than traumatologists ( P < 0.001) and rated benefits higher (all P 's ≤ 0.038), whereas senior physicians were less likely to use AI or endorse its benefits (negative correlation with age, -0.35 to 0.30; all P 's ≤ 0.046). CONCLUSIONS: Implementing AI for emergency radiology into clinical routine has an educative aspect and underlines the concept of AI as a "second reader," to support and not replace physicians.

Nonradiology Health-Care Professionals Significantly Benefit From AI Assistance in Emergency-Related Chest Radiography Interpretation.

BACKGROUND: Chest radiographs (CXRs) are still of crucial importance in primary diagnostics, but their interpretation poses difficulties at times. RESEARCH QUESTION: Can a convolutional neural network-based artificial intelligence (AI) system that interprets CXRs add value in an emergency unit setting? STUDY DESIGN AND METHODS: A total of 563 CXRs acquired in the emergency unit of a major university hospital were retrospectively assessed twice by three board-certified radiologists, three radiology residents, and three emergency unit-experienced nonradiology residents (NRRs). They used a two-step reading process: (1) without AI support (woAI); and (2) with AI support (wAI) providing additional images with AI overlays. Suspicion of four suspected pathologies (pleural effusion, pneumothorax, consolidations suspicious for pneumonia, and nodules) was reported on a five-point confidence scale. Confidence scores of the board-certified radiologists were converted into four binary reference standards (RFS I-IV) of different sensitivities. Performance by radiology residents and NRRs woAI/wAI were statistically compared by using receiver-operating characteristics (ROCs), Youden statistics, and operating point metrics derived from fitted ROC curves. RESULTS: NRRs could significantly improve performance, sensitivity, and accuracy wAI in all four pathologies tested. In the most sensitive RFS IV, NRR consensus improved the area under the ROC curve (mean, 95% CI) in the detection of the time-critical pathology pneumothorax from 0.846 (0.785-0.907) woAI to 0.974 (0.947-1.000) wAI (P < .001), which represented a gain of 30% in sensitivity and 2% in accuracy (while maintaining an optimized specificity). The most pronounced effect was observed in nodule detection, with NRR wAI improving sensitivity by 53% and accuracy by 7% (area under the ROC curve woAI, 0.723 [0.661-0.785]; wAI, 0.890 [0.848-0.931]; P < .001). The RR consensus wAI showed smaller, mostly nonsignificant gains in performance, sensitivity, and accuracy. INTERPRETATION: In an emergency unit setting without 24/7 radiology coverage, the presented AI solution features an excellent clinical support tool to nonradiologists, similar to a second reader, and allows for a more accurate primary diagnosis and thus earlier therapy initiation.

Clinically focused multi-cohort benchmarking as a tool for external validation of artificial intelligence algorithm performance in basic chest radiography analysis.

Artificial intelligence (AI) algorithms evaluating [supine] chest radiographs ([S]CXRs) have remarkably increased in number recently. Since training and validation are often performed on subsets of the same overall dataset, external validation is mandatory to reproduce results and reveal potential training errors. We applied a multicohort benchmarking to the publicly accessible (S)CXR analyzing AI algorithm CheXNet, comprising three clinically relevant study cohorts which differ in patient positioning ([S]CXRs), the applied reference standards (CT-/[S]CXR-based) and the possibility to also compare algorithm classification with different medical experts' reading performance. The study cohorts include [1] a cohort, characterized by 563 CXRs acquired in the emergency unit that were evaluated by 9 readers (radiologists and non-radiologists) in terms of 4 common pathologies, [2] a collection of 6,248 SCXRs annotated by radiologists in terms of pneumothorax presence, its size and presence of inserted thoracic tube material which allowed for subgroup and confounding bias analysis and [3] a cohort consisting of 166 patients with SCXRs that were evaluated by radiologists for underlying causes of basal lung opacities, all of those cases having been correlated to a timely acquired computed tomography scan (SCXR and CT within < 90 min). CheXNet non-significantly exceeded the radiology resident (RR) consensus in the detection of suspicious lung nodules (cohort [1], AUC AI/RR: 0.851/0.839, p = 0.793) and the radiological readers in the detection of basal pneumonia (cohort [3], AUC AI/reader consensus: 0.825/0.782, p = 0.390) and basal pleural effusion (cohort [3], AUC AI/reader consensus: 0.762/0.710, p = 0.336) in SCXR, partly with AUC values higher than originally published ("Nodule": 0.780, "Infiltration": 0.735, "Effusion": 0.864). The classifier "Infiltration" turned out to be very dependent on patient positioning (best in CXR, worst in SCXR). The pneumothorax SCXR cohort [2] revealed poor algorithm performance in CXRs without inserted thoracic material and in the detection of small pneumothoraces, which can be explained by a known systematic confounding error in the algorithm training process. The benefit of clinically relevant external validation is demonstrated by the differences in algorithm performance as compared to the original publication. Our multi-cohort benchmarking finally enables the consideration of confounders, different reference standards and patient positioning as well as the AI performance comparison with differentially qualified medical readers.

Artificial Intelligence in Chest Radiography Reporting Accuracy: Added Clinical Value in the Emergency Unit Setting Without 24/7 Radiology Coverage.

OBJECTIVES: Chest radiographs (CXRs) are commonly performed in emergency units (EUs), but the interpretation requires radiology experience. We developed an artificial intelligence (AI) system (precommercial) that aims to mimic board-certified radiologists' (BCRs') performance and can therefore support non-radiology residents (NRRs) in clinical settings lacking 24/7 radiology coverage. We validated by quantifying the clinical value of our AI system for radiology residents (RRs) and EU-experienced NRRs in a clinically representative EU setting. MATERIALS AND METHODS: A total of 563 EU CXRs were retrospectively assessed by 3 BCRs, 3 RRs, and 3 EU-experienced NRRs. Suspected pathologies (pleural effusion, pneumothorax, consolidations suspicious for pneumonia, lung lesions) were reported on a 5-step confidence scale (sum of 20,268 reported pathology suspicions [563 images × 9 readers × 4 pathologies]) separately by every involved reader. Board-certified radiologists' confidence scores were converted into 4 binary reference standards (RFSs) of different sensitivities. The RRs' and NRRs' performances were statistically compared with our AI system (trained on nonpublic data from different clinical sites) based on receiver operating characteristics (ROCs) and operating point metrics approximated to the maximum sum of sensitivity and specificity (Youden statistics). RESULTS: The NRRs lose diagnostic accuracy to RRs with increasingly sensitive BCRs' RFSs for all considered pathologies. Based on our external validation data set, the AI system/NRRs' consensus mimicked the most sensitive BCRs' RFSs with areas under ROC of 0.940/0.837 (pneumothorax), 0.953/0.823 (pleural effusion), and 0.883/0.747 (lung lesions), which were comparable to experienced RRs and significantly overcomes EU-experienced NRRs' diagnostic performance. For consolidation detection, the AI system performed on the NRRs' consensus level (and overcomes each individual NRR) with an area under ROC of 0.847 referenced to the BCRs' most sensitive RFS. CONCLUSIONS: Our AI system matched RRs' performance, meanwhile significantly outperformed NRRs' diagnostic accuracy for most of considered CXR pathologies (pneumothorax, pleural effusion, and lung lesions) and therefore might serve as clinical decision support for NRRs.

Reduction of missed thoracic findings in emergency whole-body computed tomography using artificial intelligence assistance.

BACKGROUND: Radiology reporting of emergency whole-body computed tomography (CT) scans is time-critical and therefore involves a significant risk of pathology under-detection. We hypothesize a relevant number of initially missed secondary thoracic findings that would have been detected by an artificial intelligence (AI) software platform including several pathology-specific AI algorithms. METHODS: This retrospective proof-of-concept-study consecutively included 105 shock-room whole-body CT scans. Image data was analyzed by platform-bundled AI-algorithms, findings were reviewed by radiology experts and compared with the original radiologist's reports. We focused on secondary thoracic findings, such as cardiomegaly, coronary artery plaques, lung lesions, aortic aneurysms and vertebral fractures. RESULTS: We identified a relevant number of initially missed findings, with their quantification based on 105 analyzed CT scans as follows: up to 25 patients (23.8%) with cardiomegaly or borderline heart size, 17 patients (16.2%) with coronary plaques, 34 patients (32.4%) with aortic ectasia, 2 patients (1.9%) with lung lesions classified as "recommended to control" and 13 initially missed vertebral fractures (two with an acute traumatic origin). A high number of false positive or non-relevant AI-based findings remain problematic especially regarding lung lesions and vertebral fractures. CONCLUSIONS: We consider AI to be a promising approach to reduce the number of missed findings in clinical settings with a necessary time-critical radiological reporting. Nevertheless, algorithm improvement is necessary focusing on a reduction of "false positive" findings and on algorithm features assessing the finding relevance, e.g., fracture age or lung lesion malignancy.

Interpretation of Thoracic Radiography Shows Large Discrepancies Depending on the Qualification of the Physician-Quantitative Evaluation of Interobserver Agreement in a Representative Emergency Department Scenario.

(1) Background: Chest radiography (CXR) is still a key diagnostic component in the emergency department (ED). Correct interpretation is essential since some pathologies require urgent treatment. This study quantifies potential discrepancies in CXR analysis between radiologists and non-radiology physicians in training with ED experience. (2) Methods: Nine differently qualified physicians (three board-certified radiologists [BCR], three radiology residents [RR], and three non-radiology residents involved in ED [NRR]) evaluated a series of 563 posterior-anterior CXR images by quantifying suspicion for four relevant pathologies: pleural effusion, pneumothorax, pneumonia, and pulmonary nodules. Reading results were noted separately for each hemithorax on a Likert scale (0-4; 0: no suspicion of pathology, 4: safe existence of pathology) adding up to a total of 40,536 reported pathology suspicions. Interrater reliability/correlation and Kruskal-Wallis tests were performed for statistical analysis. (3) Results: While interrater reliability was good among radiologists, major discrepancies between radiologists' and non-radiologists' reading results could be observed in all pathologies. Highest overall interrater agreement was found for pneumothorax detection and lowest agreement in raising suspicion for malignancy suspicious nodules. Pleural effusion and pneumonia were often suspected with indifferent choices (1-3). In terms of pneumothorax detection, all readers mainly decided for a clear option (0 or 4). Interrater reliability was usually higher when evaluating the right hemithorax (all pathologies except pneumothorax). (4) Conclusions: Quantified CXR interrater reliability analysis displays a general uncertainty and strongly depends on medical training. NRR can benefit from radiology reporting in terms of time efficiency and diagnostic accuracy. CXR evaluation of long-time trained ED specialists has not been tested.

Decomposing Acute Symptom Severity in Large Vessel Occlusion Stroke: Association With Multiparametric CT Imaging and Clinical Parameters.

Background and Purpose: Acute ischemic stroke of the anterior circulation due to large vessel occlusion (LVO) is a multifactorial process, which causes neurologic symptoms of different degree. Our aim was to examine the impact of neuromorphologic and vascular correlates as well as clinical factors on acute symptom severity in LVO stroke. Methods: We selected LVO stroke patients with known onset time from a consecutive cohort which underwent multiparametric CT including non-contrast CT, CT angiography and CT perfusion (CTP) before thrombectomy. Software-based quantification was used to calculate CTP total ischemic and ischemic core volume. Symptom severity was assessed using the National Institutes of Health Stroke Scale (NIHSS) upon admission. Multivariable regression analysis was performed to determine independent associations of admission NIHSS with imaging and clinical parameters. Receiver operating characteristics (ROC) analyses were used to examine performance of imaging parameters to classify symptom severity. Results: We included 142 patients. Linear and ordinal regression analyses for NIHSS and NIHSS severity groups identified significant associations for total ischemic volume [ß = 0.31, p = 0.01; Odds ratio (OR) = 1.11, 95%-confidence-interval (CI): 1.02-1.19], clot burden score (ß = -0.28, p = 0.01; OR = 0.76, 95%-CI: 0.64-0.90) and age (ß = 0.17, p = 0.04). No association was found for ischemic core volume, stroke side, collaterals and time from onset. Stroke topography according to the Alberta Stroke Program CT Score template did not display significant influence after correction for multiple comparisons. AUC for classification of the NIHSS threshold ≥6 by total ischemic volume was 0.81 (p < 0.001). Conclusions: We determined total ischemic volume, clot burden and age as relevant drivers for baseline NIHSS in acute LVO stroke. This suggests that not only mere volume but also degree of occlusion influences symptom severity. Use of imaging parameters as surrogate for baseline NIHSS reached limited performance underlining the need for combined clinical and imaging assessment in acute stroke management.

Bi-Centric Independent Validation of Outcome Prediction after Radioembolization of Primary and Secondary Liver Cancer.

BACKGROUND: Yttrium-90 radioembolization (RE) plays an important role in the treatment of liver malignancies. Optimal patient selection is crucial for an effective and safe treatment. In this study, we aim to validate the prognostic performance of a previously established random survival forest (RSF) with an external validation cohort from a different national center. Furthermore, we compare outcome prediction models with different established metrics. METHODS: A previously established RSF model, trained on a consecutive cohort of 366 patients who had received RE due to primary or secondary liver tumor at a national center (center 1), was used to predict the outcome of an independent consecutive cohort of 202 patients from a different national center (center 2) and vice versa. Prognostic performance was evaluated using the concordance index (C-index) and the integrated Brier score (IBS). The prognostic importance of designated baseline parameters was measured with the minimal depth concept, and the influence on the predicted outcome was analyzed with accumulated local effects plots. RSF values were compared to conventional cox proportional hazards models in terms of C-index and IBS. RESULTS: The established RSF model achieved a C-index of 0.67 for center 2, comparable to the results obtained for center 1, which it was trained on (0.66). The RSF model trained on center 2 achieved a C-index of 0.68 on center 2 data and 0.66 on center 1 data. CPH models showed comparable results on both cohorts, with C-index ranging from 0.68 to 0.72. IBS validation showed more differentiated results depending on which cohort was trained on and which cohort was predicted (range: 0.08 to 0.20). Baseline cholinesterase was the most important variable for survival prediction. CONCLUSION: The previously developed predictive RSF model was successfully validated with an independent external cohort. C-index and IBS are suitable metrics to compare outcome prediction models, with IBS showing more differentiated results. The findings corroborate that survival after RE is critically determined by functional hepatic reserve and thus baseline liver function should play a key role in patient selection.

Impact of Pharmaceutical Prophylaxis on Radiation-Induced Liver Disease Following Radioembolization.

BACKGROUND: Radioembolization (RE) with yttrium-90 (90Y) resin microspheres yields heterogeneous response rates in with primary or secondary liver cancer. Radiation-induced liver disease (RILD) is a potentially life-threatening complication with higher prevalence in cirrhotics or patients exposed to previous chemotherapies. Advances in RILD prevention may help increasing tolerable radiation doses to improve patient outcomes. This study aimed to evaluate the impact of post-therapeutic RILD-prophylaxis in a cohort of intensely pretreated liver metastatic breast cancer patients; Methods: Ninety-three patients with liver metastases of breast cancer received RE between 2007 and 2016. All Patients received RILD prophylaxis for 8 weeks post-RE. From January 2014, RILD prophylaxis was changed from ursodeoxycholic acid (UDCA) and prednisolone (standard prophylaxis [SP]; n = 59) to pentoxifylline (PTX), UDCA and low-dose low molecular weight heparin (LMWH) (modified prophylaxis (MP); n = 34). The primary endpoint was toxicity including symptoms of RILD; Results: Dose exposure of normal liver parenchyma was higher in the modified vs. standard prophylaxis group (47.2 Gy (17.8-86.8) vs. 40.2 Gy (12.5-83.5), p = 0.017). All grade RILD events (mild: bilirubin ≥ 21 µmol/L (but <30 µmol/L); severe: (bilirubin ≥ 30 µmol/L and ascites)) were observed more frequently in the SP group than in the MP group, albeit without significance (7/59 vs. 1/34; p = 0.140). Severe RILD occurred in the SP group only (n = 2; p > 0.1). ALBI grade increased in 16.7% patients in the MP and in 27.1% patients in the SP group, respectively (group difference not significant); Conclusions: At established dose levels, mild or severe RILD events proved rare in our cohort. RILD prophylaxis with PTX, UDCA and LMWH appears to have an independent positive impact on OS in patients with metastatic breast cancer and may reduce the frequency and severity of RILD. Results of this study as well as pathophysiological considerations warrant further investigations of RILD prophylaxis presumably targeting combinations of anticoagulation (MP) and antiinflammation (SP) to increase dose prescriptions in radioembolization.

An Artificial Intelligence-Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter Study.

Importance: Most early lung cancers present as pulmonary nodules on imaging, but these can be easily missed on chest radiographs. Objective: To assess if a novel artificial intelligence (AI) algorithm can help detect pulmonary nodules on radiographs at different levels of detection difficulty. Design, Setting, and Participants: This diagnostic study included 100 posteroanterior chest radiograph images taken between 2000 and 2010 of adult patients from an ambulatory health care center in Germany and a lung image database in the US. Included images were selected to represent nodules with different levels of detection difficulties (from easy to difficult), and comprised both normal and nonnormal control. Exposures: All images were processed with a novel AI algorithm, the AI Rad Companion Chest X-ray. Two thoracic radiologists established the ground truth and 9 test radiologists from Germany and the US independently reviewed all images in 2 sessions (unaided and AI-aided mode) with at least a 1-month washout period. Main Outcomes and Measures: Each test radiologist recorded the presence of 5 findings (pulmonary nodules, atelectasis, consolidation, pneumothorax, and pleural effusion) and their level of confidence for detecting the individual finding on a scale of 1 to 10 (1 representing lowest confidence; 10, highest confidence). The analyzed metrics for nodules included sensitivity, specificity, accuracy, and receiver operating characteristics curve area under the curve (AUC). Results: Images from 100 patients were included, with a mean (SD) age of 55 (20) years and including 64 men and 36 women. Mean detection accuracy across the 9 radiologists improved by 6.4% (95% CI, 2.3% to 10.6%) with AI-aided interpretation compared with unaided interpretation. Partial AUCs within the effective interval range of 0 to 0.2 false positive rate improved by 5.6% (95% CI, -1.4% to 12.0%) with AI-aided interpretation. Junior radiologists saw greater improvement in sensitivity for nodule detection with AI-aided interpretation as compared with their senior counterparts (12%; 95% CI, 4% to 19% vs 9%; 95% CI, 1% to 17%) while senior radiologists experienced similar improvement in specificity (4%; 95% CI, -2% to 9%) as compared with junior radiologists (4%; 95% CI, -3% to 5%). Conclusions and Relevance: In this diagnostic study, an AI algorithm was associated with improved detection of pulmonary nodules on chest radiographs compared with unaided interpretation for different levels of detection difficulty and for readers with different experience.

Impact of Confounding Thoracic Tubes and Pleural Dehiscence Extent on Artificial Intelligence Pneumothorax Detection in Chest Radiographs.

OBJECTIVES: We hypothesized that published performances of algorithms for artificial intelligence (AI) pneumothorax (PTX) detection in chest radiographs (CXRs) do not sufficiently consider the influence of PTX size and confounding effects caused by thoracic tubes (TTs). Therefore, we established a radiologically annotated benchmarking cohort (n = 6446) allowing for a detailed subgroup analysis. MATERIALS AND METHODS: We retrospectively identified 6434 supine CXRs, among them 1652 PTX-positive cases and 4782 PTX-negative cases. Supine CXRs were radiologically annotated for PTX size, PTX location, and inserted TTs. The diagnostic performances of 2 AI algorithms ("AI_CheXNet" [Rajpurkar et al], "AI_1.5" [Guendel et al]), both trained on publicly available datasets with labels obtained from automatic report interpretation, were quantified. The algorithms' discriminative power for PTX detection was quantified by the area under the receiver operating characteristics (AUROC), and significance analysis was based on the corresponding 95% confidence interval. A detailed subgroup analysis was performed to quantify the influence of PTX size and the confounding effects caused by inserted TTs. RESULTS: Algorithm performance was quantified as follows: overall performance with AUROCs of 0.704 (AI_1.5) / 0.765 (AI_CheXNet) for unilateral PTXs, AUROCs of 0.666 (AI_1.5) / 0.722 (AI_CheXNet) for unilateral PTXs smaller than 1 cm, and AUROCs of 0.735 (AI_1.5) / 0.818 (AI_CheXNet) for unilateral PTXs larger than 2 cm. Subgroup analysis identified TTs to be strong confounders that significantly influence algorithm performance: Discriminative power is completely eliminated by analyzing PTX-positive cases without TTs referenced to control PTX-negative cases with inserted TTs. Contrarily, AUROCs increased up to 0.875 (AI_CheXNet) for large PTX-positive cases with inserted TTs referenced to control cases without TTs. CONCLUSIONS: Our detailed subgroup analysis demonstrated that the performance of established AI algorithms for PTX detection trained on public datasets strongly depends on PTX size and is significantly biased by confounding image features, such as inserted TTS. Our established, clinically relevant and radiologically annotated benchmarking cohort might be of great benefit for ongoing algorithm development.

COVID-19 Pandemic and Upcoming Influenza Season-Does an Expert's Computed Tomography Assessment Differentially Identify COVID-19, Influenza and Pneumonias of Other Origin?

(1) Background: Time-consuming SARS-CoV-2 RT-PCR suffers from limited sensitivity in early infection stages whereas fast available chest CT can already raise COVID-19 suspicion. Nevertheless, radiologists' performance to differentiate COVID-19, especially from influenza pneumonia, is not sufficiently characterized. (2) Methods: A total of 201 pneumonia CTs were identified and divided into subgroups based on RT-PCR: 78 COVID-19 CTs, 65 influenza CTs and 62 Non-COVID-19-Non-influenza (NCNI) CTs. Three radiology experts (blinded from RT-PCR results) raised pathogen-specific suspicion (separately for COVID-19, influenza, bacterial pneumonia and fungal pneumonia) according to the following reading scores: 0-not typical/1-possible/2-highly suspected. Diagnostic performances were calculated with RT-PCR as a reference standard. Dependencies of radiologists' pathogen suspicion scores were characterized by Pearson's Chi2 Test for Independence. (3) Results: Depending on whether the intermediate reading score 1 was considered as positive or negative, radiologists correctly classified 83-85% (vs. NCNI)/79-82% (vs. influenza) of COVID-19 cases (sensitivity up to 94%). Contrarily, radiologists correctly classified only 52-56% (vs. NCNI)/50-60% (vs. COVID-19) of influenza cases. The COVID-19 scoring was more specific than the influenza scoring compared with suspected bacterial or fungal infection. (4) Conclusions: High-accuracy COVID-19 detection by CT might expedite patient management even during the upcoming influenza season.

Prognostic Value of Admission Chest CT Findings for Invasive Ventilation Therapy in COVID-19 Pneumonia.

(1) Background: To assess the value of chest CT imaging features of COVID-19 disease upon hospital admission for risk stratification of invasive ventilation (IV) versus no or non-invasive ventilation (non-IV) during hospital stay. (2) Methods: A retrospective single-center study was conducted including all patients admitted during the first three months of the pandemic at our hospital with PCR-confirmed COVID-19 disease and admission chest CT scans (n = 69). Using clinical information and CT imaging features, a 10-point ordinal risk score was developed and its diagnostic potential to differentiate a severe (IV-group) from a more moderate course (non-IV-group) of the disease was tested. (3) Results: Frequent imaging findings of COVID-19 pneumonia in both groups were ground glass opacities (91.3%), consolidations (53.6%) and crazy paving patterns (31.9%). Characteristics of later stages such as subpleural bands were observed significantly more often in the IV-group (52.2% versus 26.1%, p = 0.032). Using information directly accessible during a radiologist's reporting, a simple risk score proved to reliably differentiate between IV- and non-IV-groups (AUC: 0.89 (95% CI 0.81-0.96), p < 0.001). (4) Conclusions: Information accessible from admission CT scans can effectively and reliably be used in a scoring model to support risk stratification of COVID-19 patients to improve resource and allocation management of hospitals.