Interrater agreement in classifying infections during extracorporeal membrane oxygenation.

Infectious complications are common during extracorporeal membrane oxygenation (ECMO) and may negatively impact outcomes. However, there is considerable variation in the reported rates of incidence, which hampers the use of infections as a quality benchmark for ECMO centers. To assess the contributing role of poor interrater agreement, three independent raters reviewed medical records from all intensive care unit (ICU) patients who received ECMO for >24 h in our tertiary center between October 2019 and October 2021 for suspected episodes of infection, which were rated based on their date of onset and presumed site/diagnosis. To establish a gold standard, any discrepancies were resolved using an expert panel consisting of two intensivists/infectious disease specialists. During 83 ECMO-runs in 77 patients, we observed a total of 62 adjudicated infectious episodes (incidence rate 62, 95% CI: 48-80, per 1000 days at risk). Among 81 episodes suspected by at least one observer, 66 (81%) were identified by two, and only 44 (54%) by all three raters, resulting in Fleiss' kappa of 0.10 (95% CI: 0.00-0.19; slight agreement). However, if raters concurred regarding infection onset, subsequent agreement on infection site was good (concordance 89%; kappa 0.85, 95% CI: 0.72-0.98; near perfect agreement). In conclusion, adjudication of infectious episodes during ECMO is associated with poor interrater agreement regarding occurrence-but not site-of infection. This finding might partially explain the significant disparities observed in reported infection rates during ECMO, emphasizing the need for caution when interpreting infection data in this particular population due to the potential for inherent measurement error.

Using machine learning to predict blood culture outcomes in the emergency department: a single-centre, retrospective, observational study.

OBJECTIVES: To develop predictive models for blood culture (BC) outcomes in an emergency department (ED) setting. DESIGN: Retrospective observational study. SETTING: ED of a large teaching hospital in the Netherlands between 1 September 2018 and 24 June 2020. PARTICIPANTS: Adult patients from whom BCs were collected in the ED. Data of demographic information, vital signs, administered medications in the ED and laboratory and radiology results were extracted from the electronic health record, if available at the end of the ED visits. MAIN OUTCOME MEASURES: The primary outcome was the performance of two models (logistic regression and gradient boosted trees) to predict bacteraemia in ED patients, defined as at least one true positive BC collected at the ED. RESULTS: In 4885 out of 51 399 ED visits (9.5%), BCs were collected. In 598/4885 (12.2%) visits, at least one of the BCs was true positive. Both a gradient boosted tree model and a logistic regression model showed good performance in predicting BC results with area under curve of the receiver operating characteristics of 0.77 (95% CI 0.73 to 0.82) and 0.78 (95% CI 0.73 to 0.82) in the test sets, respectively. In the gradient boosted tree model, the optimal threshold would predict 69% of BCs in the test set to be negative, with a negative predictive value of over 94%. CONCLUSIONS: Both models can accurately identify patients with low risk of bacteraemia at the ED in this single-centre setting and may be useful to reduce unnecessary BCs and associated healthcare costs. Further studies are necessary for validation and to investigate the potential clinical benefits and possible risks after implementation.