Evaluating the performance of artificial intelligence software for lung nodule detection on chest radiographs in a retrospective real-world UK population.

OBJECTIVES: Early identification of lung cancer on chest radiographs improves patient outcomes. Artificial intelligence (AI) tools may increase diagnostic accuracy and streamline this pathway. This study evaluated the performance of commercially available AI-based software trained to identify cancerous lung nodules on chest radiographs. DESIGN: This retrospective study included primary care chest radiographs acquired in a UK centre. The software evaluated each radiograph independently and outputs were compared with two reference standards: (1) the radiologist report and (2) the diagnosis of cancer by multidisciplinary team decision. Failure analysis was performed by interrogating the software marker locations on radiographs. PARTICIPANTS: 5722 consecutive chest radiographs were included from 5592 patients (median age 59 years, 53.8% women, 1.6% prevalence of cancer). RESULTS: Compared with radiologist reports for nodule detection, the software demonstrated sensitivity 54.5% (95% CI 44.2% to 64.4%), specificity 83.2% (82.2% to 84.1%), positive predictive value (PPV) 5.5% (4.6% to 6.6%) and negative predictive value (NPV) 99.0% (98.8% to 99.2%). Compared with cancer diagnosis, the software demonstrated sensitivity 60.9% (50.1% to 70.9%), specificity 83.3% (82.3% to 84.2%), PPV 5.6% (4.8% to 6.6%) and NPV 99.2% (99.0% to 99.4%). Normal or variant anatomy was misidentified as an abnormality in 69.9% of the 943 false positive cases. CONCLUSIONS: The software demonstrated considerable underperformance in this real-world patient cohort. Failure analysis suggested a lack of generalisability in the training and testing datasets as a potential factor. The low PPV carries the risk of over-investigation and limits the translation of the software to clinical practice. Our findings highlight the importance of training and testing software in representative datasets, with broader implications for the implementation of AI tools in imaging.

Acute haemorrhage rate in 28,000 Out-of-Hours CT heads.

OBJECTIVES: The aim of this paper is to assess the acute haemorrhage rate in patients who had CT head investigation out-of-hours with and without trauma and compare the rates of haemorrhage between warfarin and DOACs, at a busy teritary teaching hospital. METHODS: All CT heads performed between January 2008 and December 2019 were identified from the radiology information system (RIS) at Sheffield Teaching Hospitals (STH), with the requesting information being available from January 2015. The clinical information was assessed for the mention of trauma or anticoagulation, and the reports were categorised into acute and non-acute findings. RESULTS: Between 2008 and 2019 the number of scans increased by 63%, with scans performed out of hours increasing by 278%. Between 2015 and 2019, the incidence of acute ICH was similar over the 5-year period, averaging at 6.9% and ranging from 6.1 to 7.6%. The rate of detection of acute haemorrhage following trauma was greater in those not anticoagulated (6.8%), compared with patients on anticoagulants such as warfarin (5.2%) or DOACs (2.8%). CONCLUSIONS: Over 12 years, there has been a significant increase in the number of CT heads performed at STH. The rate of ICH has remained steady over the last 5 years indicating a justified increase in imaging demand. However, the incidence of ICH in patients prescribed DOACs is lower than the general population and those on warfarin. ADVANCES IN KNOWLEDGE: This finding in a large centre should prompt discussion of the risk of bleeding with DOACs in relation to CT head imaging guidelines.