RESUMO
INTRODUCTION: Biochemical verification of smoking status prior to recruitment into smoking cessation trials is widely used to confirm smoking status, most commonly using exhaled carbon monoxide (CO). There is variation in the level of CO used as a biochemical inclusion criterion, and thus the possibility for people reporting to be current smokers to be incorrectly excluded from trials. METHODS: As part of the Cessation of Smoking Trial in the Emergency Department, people attending the Emergency Department (ED) who reported being current daily smokers underwent CO testing to confirm eligibility. Elective semi-structured interviews were undertaken with the researchers who recruited participants. As part of the interviews, researchers were asked their views and experiences with CO testing. RESULTS: Of the 1320 participants who reported being current daily smokers and underwent CO testing, 300 (22.7%) blew a CO reading of 7 ppm or less and were excluded from taking part. Possible explanations offered by researchers for participants blowing low CO readings were (1) long wait times in the ED, therefore a long period having elapsed since people had last smoked and (2) patients having reduced smoking for the period before the ED attendance due to ill health. CONCLUSIONS: Biochemical verification has the potential to improve internal validity of smoking cessation for inclusion in trials, but at the cost of reduced generalisability through exclusion of participants who would receive the intervention if it were implemented in practice. We would recommend researchers carefully consider whether it is appropriate and necessary to include biochemical verification as an inclusion criterion.
RESUMO
PURPOSE: Most studies evaluating artificial intelligence (AI) models that detect abnormalities in neuroimaging are either tested on unrepresentative patient cohorts or are insufficiently well-validated, leading to poor generalisability to real-world tasks. The aim was to determine the diagnostic test accuracy and summarise the evidence supporting the use of AI models performing first-line, high-volume neuroimaging tasks. METHODS: Medline, Embase, Cochrane library and Web of Science were searched until September 2021 for studies that temporally or externally validated AI capable of detecting abnormalities in first-line computed tomography (CT) or magnetic resonance (MR) neuroimaging. A bivariate random effects model was used for meta-analysis where appropriate. This study was registered on PROSPERO as CRD42021269563. RESULTS: Out of 42,870 records screened, and 5734 potentially eligible full texts, only 16 studies were eligible for inclusion. Included studies were not compromised by unrepresentative datasets or inadequate validation methodology. Direct comparison with radiologists was available in 4/16 studies and 15/16 had a high risk of bias. Meta-analysis was only suitable for intracranial hemorrhage detection in CT imaging (10/16 studies), where AI systems had a pooled sensitivity and specificity 0.90 (95% confidence interval [CI] 0.85-0.94) and 0.90 (95% CI 0.83-0.95), respectively. Other AI studies using CT and MRI detected target conditions other than hemorrhage (2/16), or multiple target conditions (4/16). Only 3/16 studies implemented AI in clinical pathways, either for pre-read triage or as post-read discrepancy identifiers. CONCLUSION: The paucity of eligible studies reflects that most abnormality detection AI studies were not adequately validated in representative clinical cohorts. The few studies describing how abnormality detection AI could impact patients and clinicians did not explore the full ramifications of clinical implementation.