Appropriate use of blood cultures in the emergency department through machine learning (ABC): study protocol for a randomised controlled non-inferiority trial.

INTRODUCTION: The liberal use of blood cultures in emergency departments (EDs) leads to low yields and high numbers of false-positive results. False-positive, contaminated cultures are associated with prolonged hospital stays, increased antibiotic usage and even higher hospital mortality rates. This trial aims to investigate whether a recently developed and validated machine learning model for predicting blood culture outcomes can safely and effectively guide clinicians in withholding unnecessary blood culture analysis. METHODS AND ANALYSIS: A randomised controlled, non-inferiority trial comparing current practice with a machine learning-guided approach. The primary objective is to determine whether the machine learning based approach is non-inferior to standard practice based on 30-day mortality. Secondary outcomes include hospital length-of stay and hospital admission rates. Other outcomes include model performance and antibiotic usage. Participants will be recruited in the EDs of multiple hospitals in the Netherlands. A total of 7584 participants will be included. ETHICS AND DISSEMINATION: Possible participants will receive verbal information and a paper information brochure regarding the trial. They will be given at least 1 hour consideration time before providing informed consent. Research results will be published in peer-reviewed journals. This study has been approved by the Amsterdam University Medical Centers' local medical ethics review committee (No 22.0567). The study will be conducted in concordance with the principles of the Declaration of Helsinki and in accordance with the Medical Research Involving Human Subjects Act, General Data Privacy Regulation and Medical Device Regulation. TRIAL REGISTRATION NUMBER: NCT06163781.

Impact of Blood Culture Contamination on Antibiotic Use, Resource Utilization, and Clinical Outcomes: A Retrospective Cohort Study in Dutch and US Hospitals.

Background: Blood culture contamination (BCC) has been associated with prolonged antibiotic use (AU) and increased health care utilization; however, this has not been widely reevaluated in the era of increased attention to antibiotic stewardship. We evaluated the impact of BCC on AU, resource utilization, and length of stay in Dutch and US patients. Methods: This retrospective observational study examined adults admitted to 2 hospitals in the Netherlands and 5 hospitals in the United States undergoing ≥2 blood culture (BC) sets. Exclusion criteria included neutropenia, no hospital admission, or death within 48 hours of hospitalization. The impact of BCC on clinical outcomes-overall inpatient days of antibiotic therapy, test utilization, length of stay, and mortality-was determined via a multivariable regression model. Results: An overall 22 927 patient admissions were evaluated: 650 (4.1%) and 339 (4.8%) with BCC and 11 437 (71.8%) and 4648 (66.3%) with negative BC results from the Netherlands and the United States, respectively. Dutch and US patients with BCC had a mean ± SE 1.74 ± 0.27 (P < .001) and 1.58 ± 0.45 (P < .001) more days of antibiotic therapy than patients with negative BC results. They also had 0.6 ± 0.1 (P < .001) more BCs drawn. Dutch but not US patients with BCC had longer hospital stays (3.36 days; P < .001). There was no difference in mortality between groups in either cohort. AU remained higher in US but not Dutch patients with BCC in a subanalysis limited to BC obtained within the first 24 hours of admission. Conclusions: BCC remains associated with higher inpatient AU and health care utilization as compared with patients with negative BC results, although the impact on these outcomes differs by country.

Detecting changes in the performance of a clinical machine learning tool over time.

BACKGROUND: Excessive use of blood cultures (BCs) in Emergency Departments (EDs) results in low yields and high contamination rates, associated with increased antibiotic use and unnecessary diagnostics. Our team previously developed and validated a machine learning model to predict BC outcomes and enhance diagnostic stewardship. While the model showed promising initial results, concerns over performance drift due to evolving patient demographics, clinical practices, and outcome rates warrant continual monitoring and evaluation of such models. METHODS: A real-time evaluation of the model's performance was conducted between October 2021 and September 2022. The model was integrated into Amsterdam UMC's Electronic Health Record system, predicting BC outcomes for all adult patients with BC draws in real time. The model's performance was assessed monthly using metrics including the Area Under the Curve (AUC), Area Under the Precision-Recall Curve (AUPRC), and Brier scores. Statistical Process Control (SPC) charts were used to monitor variation over time. FINDINGS: Across 3.035 unique adult patient visits, the model achieved an average AUC of 0.78, AUPRC of 0.41, and a Brier score of 0.10 for predicting the outcome of BCs drawn in the ED. While specific population characteristics changed over time, no statistical points outside the statistical control range were detected in the AUC, AUPRC, and Brier scores, indicating stable model performance. The average BC positivity rate during the study period was 13.4%. INTERPRETATION: Despite significant changes in clinical practice, our BC stewardship tool exhibited stable performance, suggesting its robustness to changing environments. Using SPC charts for various metrics enables simple and effective monitoring of potential performance drift. The assessment of the variation of outcome rates and population changes may guide the specific interventions, such as intercept correction or recalibration, that may be needed to maintain a stable model performance over time. This study suggested no need to recalibrate or correct our BC stewardship tool. FUNDING: No funding to disclose.

Embracing cohort heterogeneity in clinical machine learning development: a step toward generalizable models.

This study is a simple illustration of the benefit of averaging over cohorts, rather than developing a prediction model from a single cohort. We show that models trained on data from multiple cohorts can perform significantly better in new settings than models based on the same amount of training data but from just a single cohort. Although this concept seems simple and obvious, no current prediction model development guidelines recommend such an approach.

Biomedical heterogeneous data categorization and schema mapping toward data integration.

Data integration is a well-motivated problem in the clinical data science domain. Availability of patient data, reference clinical cases, and datasets for research have the potential to advance the healthcare industry. However, the unstructured (text, audio, or video data) and heterogeneous nature of the data, the variety of data standards and formats, and patient privacy constraint make data interoperability and integration a challenge. The clinical text is further categorized into different semantic groups and may be stored in different files and formats. Even the same organization may store cases in different data structures, making data integration more challenging. With such inherent complexity, domain experts and domain knowledge are often necessary to perform data integration. However, expert human labor is time and cost prohibitive. To overcome the variability in the structure, format, and content of the different data sources, we map the text into common categories and compute similarity within those. In this paper, we present a method to categorize and merge clinical data by considering the underlying semantics behind the cases and use reference information about the cases to perform data integration. Evaluation shows that we were able to merge 88% of clinical data from five different sources.

Leaving the hospital on time: hospital bed utilization and reasons for discharge delay in the Netherlands.

Inappropriate bed occupancy due to delayed hospital discharge affects both physical and psychological well-being in patients and can disrupt patient flow. The Dutch healthcare system is facing ongoing pressure, especially during the current coronavirus disease pandemic, intensifying the need for optimal use of hospital beds. The aim of this study was to quantify inappropriate patient stays and describe the underlying reasons for the delays in discharge. The Day of Care Survey (DoCS) is a validated tool used to gain information about appropriate and inappropriate bed occupancy in hospitals. Between February 2019 and January 2021, the DoCS was performed five times in three different hospitals within the region of Amsterdam, the Netherlands. All inpatients were screened, using standardized criteria, for their need for in-hospital care at the time of survey and reasons for discharge delay. A total of 782 inpatients were surveyed. Of these patients, 94 (12%) were planned for definite discharge that day. Of all other patients, 145 (21%, ranging from 14% to 35%) were without the need for acute in-hospital care. In 74% (107/145) of patients, the reason for discharge delay was due to issues outside the hospital; most frequently due to a shortage of available places in care homes (26%, 37/145). The most frequent reason for discharge delay inside the hospital was patients awaiting a decision or review by the treating physician (14%, 20/145). Patients who did not meet the criteria for hospital stay were, in general, older [median 75, interquartile range (IQR) 65-84 years, and 67, IQR 55-75 years, respectively, P < .001] and had spent more days in hospital (7, IQR 5-14 days, and 3, IQR 1-8 days respectively, P < .001). Approximately one in five admitted patients occupying hospital beds did not meet the criteria for acute in-hospital stay or care at the time of the survey. Most delays were related to issues outside the immediate control of the hospital. Improvement programmes working with stakeholders focusing on the transfer from hospital to outside areas of care need to be further developed and may offer potential for the greatest gain. The DoCS can be a tool to periodically monitor changes and improvements in patient flow.

[Can artificial intelligence write a medical-scientific article of sufficient quality?] / Kan AI eenNTvG-artikel schrijven?

In this article, we describe the process - from the first draft, through peer revision to a final manuscript - of writing a scientific article only using AI. We discuss the problems and questions that arise and make recommendations for how text-generative AI may be used in the medical-scientific world.

Host Response Biomarkers for Sepsis in the Emergency Room.

This article is one of ten reviews selected from the Annual Update in Intensive Care and Emergency Medicine 2023. Other selected articles can be found online at https://www.biomedcentral.com/collections/annualupdate2023 . Further information about the Annual Update in Intensive Care and Emergency Medicine is available from https://link.springer.com/bookseries/8901 .

Factors influencing in-hospital prescribing errors: A systematic review.

AIM: In-hospital prescribing errors (PEs) may result in patient harm, prolonged hospitalization and hospital (re)admission. These events are associated with pressure on healthcare services and significant healthcare costs. To develop targeted interventions to prevent or reduce in-hospital PEs, identification and understanding of facilitating and protective factors influencing in-hospital PEs in current daily practice is necessary, adopting a Safety-II perspective. The aim of this systematic review was to create an overview of all factors reported in the literature, both protective and facilitating, as influencing in-hospital PEs. METHODS: PubMed, EMBASE.com and the Cochrane Library (via Wiley) were searched, according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement, for studies that identified factors influencing in-hospital PEs. Both qualitative and quantitative study designs were included. RESULTS: Overall, 19 articles (6 qualitative and 13 quantitative studies) were included and 40 unique factors influencing in-hospital PEs were identified. These factors were categorized into five domains according to the Eindhoven classification ('organization-related', 'prescriber-related', 'prescription-related', 'technology-related' and 'unclassified') and visualized in an Ishikawa (Fishbone) diagram. Most of the identified factors (87.5%; n = 40) facilitated in-hospital PEs. The most frequently identified facilitating factor (39.6%; n = 19) was 'insufficient (drug) knowledge, prescribing skills and/or experience of prescribers'. CONCLUSION: The findings of this review could be used to identify points of engagement for future intervention studies and help hospitals determine how to optimize prescribing. A multifaceted intervention, targeting multiple factors might help to circumvent the complex challenge of in-hospital PEs.

The impact of a sepsis performance improvement program in the emergency department: a before-after intervention study.

PURPOSE: The latest Surviving Sepsis Campaign guidelines advocate that all hospitals use sepsis performance improvement programs. However, there is a limited evidence about how to structure such programs and what their potential impact is on sepsis management and outcomes in the emergency department (ED). In this study, we evaluated the implementation of a sepsis performance improvement program in the ED including a dedicated sepsis response team and analyzed the management and outcomes of sepsis patients before and after. METHODS: We conducted a before-after interventional study in the ED of the Amsterdam University Medical Centers, the Netherlands. The sepsis performance improvement program included regular educational meetings, daily audits and weekly feedback, a screening tool, and a dedicated multidisciplinary sepsis response team. We studied all adult patients who presented to the ED with a suspected infection and a Modified Early Warning Score (MEWS) ≥ 3 during their stay. In the postintervention phase, these patients were seen by the sepsis team. Process-related and patient-related outcomes were measured between November 2019 and February 2020 (preintervention) and December 2021-May 2022 (postintervention). RESULTS: A total of 265 patients were included in the primary study, 132 patients preintervention and 133 patients postintervention. The postintervention phase was associated with improvements in nearly all process-related outcomes, such as a shorter time to antibiotics (66 vs. 143 min; p < 0.001), increased number of lactate measurements (72.9 vs. 46.2%; p < 0.001), and improved completeness of documented MEWS scores (85.0 vs. 62.9%; p < 0.001). Except for an improvement in the number of immediate versus delayed ICU admissions (100% immediate vs. 64.3% immediate; p = 0.012), there was no improvement in the other patient-related outcomes such as 28 days mortality (14.3 vs. 9.1%; p = 0.261), during the postintervention phase. CONCLUSION: Our program stimulated physicians to make timely decisions regarding diagnostics and treatment of sepsis in the ED. Implementing the sepsis performance improvement program was associated with significant improvements in most process-related outcomes but with minimal improvements in patient-related outcomes in our cohort.

Recent infection with HCoV-OC43 may be associated with protection against SARS-CoV-2 infection.

Antibodies against seasonal human coronaviruses (HCoVs) are known to cross-react with SARS-CoV-2, but data on cross-protective effects of prior HCoV infections are conflicting. In a prospective cohort of healthcare workers (HCWs), we studied the association between seasonal HCoV (OC43, HKU1, 229E and NL63) nucleocapsid protein IgG and SARS-CoV-2 infection during the first pandemic wave in the Netherlands (March 2020 - June 2020), by 4-weekly serum sampling. HCW with HCoV-OC43 antibody levels in the highest quartile, were less likely to become SARS-CoV-2 seropositive when compared with those with lower levels (6/32, 18.8%, versus 42/97, 43.3%, respectively: p = 0.019; HR 0.37, 95% CI 0.16-0.88). We found no significant association with HCoV-OC43 spike protein IgG, or with antibodies against other HCoVs. Our results indicate that the high levels of HCoV-OC43-nucleocapsid antibodies, as an indicator of a recent infection, are associated with protection against SARS-CoV-2 infection; this supports and informs efforts to develop pancoronavirus vaccines.

Diagnostic stewardship for blood cultures in the emergency department: A multicenter validation and prospective evaluation of a machine learning prediction tool.

BACKGROUND: Overuse of blood cultures (BCs) in emergency departments (EDs) leads to low yields and high numbers of contaminated cultures, accompanied by increased diagnostics, antibiotic usage, prolonged hospitalization, and mortality. We aimed to simplify and validate a recently developed machine learning model to help safely withhold BC testing in low-risk patients. METHODS: We extracted data from the electronic health records (EHR) for 44.123 unique ED visits with BC sampling in the Amsterdam UMC (locations VUMC and AMC; the Netherlands), Zaans Medical Center (ZMC; the Netherlands), and Beth Israel Deaconess Medical Center (BIDMC; United States) in periods between 2011 and 2021. We trained a machine learning model on the VUMC data to predict blood culture outcomes and validated it in the AMC, ZMC, and BIDMC with subsequent real-time prospective evaluation in the VUMC. FINDINGS: The model had an Area Under the Receiver Operating Characteristics curve (AUROC) of 0.81 (95%-CI = 0.78-0.83) in the VUMC test set. The most important predictors were temperature, creatinine, and C-reactive protein. The AUROCs in the validation cohorts were 0.80 (AMC; 0.78-0.82), 0.76 (ZMC; 0.74-0.78), and 0.75 (BIDMC; 0.74-0.76). During real-time prospective evaluation in the EHR of the VUMC, it reached an AUROC of 0.76 (0.71-0.81) among 590 patients with BC draws in the ED. The prospective evaluation showed that the model can be used to safely withhold blood culture analyses in at least 30% of patients in the ED. INTERPRETATION: We developed a machine learning model to predict blood culture outcomes in the ED, which retained its performance during external validation and real-time prospective evaluation. Our model can identify patients at low risk of having a positive blood culture. Using the model in practice can significantly reduce the number of blood culture analyses and thus avoid the hidden costs of false-positive culture results. FUNDING: This research project was funded by the Amsterdam Public Health - Quality of Care program and the Dutch "Doen of Laten" project (project number: 839205002).

The host response in different aetiologies of community-acquired pneumonia.

BACKGROUND: Community-acquired pneumonia (CAP) can be caused by a variety of pathogens, of which Streptococcus pneumoniae, Influenza and currently SARS-CoV-2 are the most common. We sought to identify shared and pathogen-specific host response features by directly comparing different aetiologies of CAP. METHODS: We measured 72 plasma biomarkers in a cohort of 265 patients hospitalized for CAP, all sampled within 48 hours of admission, and 28 age-and sex matched non-infectious controls. We stratified the biomarkers into several pathophysiological domains- antiviral response, vascular response and function, coagulation, systemic inflammation, and immune checkpoint markers. We directly compared CAP caused by SARS-CoV-2 (COVID-19, n=39), Streptococcus pneumoniae (CAP-strep, n=27), Influenza (CAP-flu, n=22) and other or unknown pathogens (CAP-other, n=177). We adjusted the comparisons for age, sex and disease severity scores. FINDINGS: Biomarkers reflective of a stronger cell-mediated antiviral response clearly separated COVID-19 from other CAPs (most notably granzyme B). Biomarkers reflecting activation and function of the vasculature showed endothelial barrier integrity was least affected in COVID-19, while glycocalyx degradation and angiogenesis were enhanced relative to other CAPs. Notably, markers of coagulation activation, including D-dimer, were not different between the CAP groups. Ferritin was most increased in COVID-19, while other systemic inflammation biomarkers such as IL-6 and procalcitonin were highest in CAP-strep. Immune checkpoint markers showed distinctive patterns in viral and non-viral CAP, with highly elevated levels of Galectin-9 in COVID-19. INTERPRETATION: Our investigation provides insight into shared and distinct pathophysiological mechanisms in different aetiologies of CAP, which may help guide new pathogen-specific therapeutic strategies. FUNDING: This study was financially supported by the Dutch Research Council, the European Commission and the Netherlands Organization for Health Research and Development.

Decreased Passive Immunity to Respiratory Viruses through Human Milk during the COVID-19 Pandemic.

Infants may develop severe viral respiratory tract infections because their immune system is still developing in the first months after birth. Human milk provides passive humoral immunity during the first months of life. During the COVID-19 pandemic, circulation of common respiratory viruses was virtually absent due to the preventative measures resulting in reduced maternal exposure. Therefore, we hypothesized that this might result in lower antibody levels in human milk during the pandemic and, subsequently, decreased protection of infants against viral respiratory tract infections. We assessed antibody levels against respiratory syncytial virus (RSV), Influenza virus, and several seasonal coronaviruses in different periods of the COVID-19 pandemic in serum and human milk using a Luminex assay. IgG levels against RSV, Influenza, HCoV-OC43, HCoV-HKU1, and HCoV-NL63 in human milk were reduced with a factor of 1.7 (P < 0.001), 2.2 (P < 0.01), 2.6 (P < 0.05), 1.4 (P < 0.01), and 2.1 (P < 0.001), respectively, since the introduction of the COVID-19 restrictions. Furthermore, we observed that human milk of mothers that experienced COVID-19 contained increased levels of IgG and IgA binding to other respiratory viruses. Passive immunity via human milk against common respiratory viruses was reduced during the COVID-19 pandemic, which may have consequences for the protection of breastfed infants against respiratory infections. IMPORTANCE Passive immunity derived from antibodies in human milk is important for protecting young infants against invading viruses. During the COVID-19 pandemic, circulation of common respiratory viruses was virtually absent due to preventative measures. In this study, we observed a decrease in human milk antibody levels against common respiratory viruses several months into the COVID-19 pandemic. This waning of antibody levels might partially explain the previously observed surge of hospitalizations of infants, mostly due to RSV, when preventative hygiene measures were lifted. Knowledge of the association between preventative measures, antibody levels in human milk and subsequent passive immunity in infants might help predict infant hospital admissions and thereby enables anticipation to prevent capacity issues. Additionally, it is important in the consideration for strategies for future lockdowns to best prevent possible consequences for vulnerable infants.

Antibody responses against SARS-CoV-2 variants induced by four different SARS-CoV-2 vaccines in health care workers in the Netherlands: A prospective cohort study.

BACKGROUND: Emerging and future SARS-CoV-2 variants may jeopardize the effectiveness of vaccination campaigns. Therefore, it is important to know how the different vaccines perform against diverse SARS-CoV-2 variants. METHODS AND FINDINGS: In a prospective cohort of 165 SARS-CoV-2 naive health care workers in the Netherlands, vaccinated with either one of four vaccines (BNT162b2, mRNA-1273, AZD1222 or Ad26.COV2.S), we performed a head-to-head comparison of the ability of sera to recognize and neutralize SARS-CoV-2 variants of concern (VOCs; Alpha, Beta, Gamma, Delta and Omicron). Repeated serum sampling was performed 5 times during a year (from January 2021 till January 2022), including before and after booster vaccination with BNT162b2. Four weeks after completing the initial vaccination series, SARS-CoV-2 wild-type neutralizing antibody titers were highest in recipients of mRNA-1273, followed by recipients of BNT162b2 (geometric mean titers (GMT) of 358 [95% CI 231-556] and 214 [95% CI 153-299], respectively; p<0.05), and substantially lower in those vaccinated with the adenovirus vector-based vaccines AZD1222 and Ad26.COV2.S (GMT of 18 [95% CI 11-30] and 14 [95% CI 8-25] IU/ml, respectively; p<0.001). VOCs neutralization was reduced in all vaccine groups, with the greatest reduction in neutralization GMT observed against the Omicron variant (fold change 0.03 [95% CI 0.02-0.04], p<0.001). The booster BNT162b2 vaccination increased neutralizing antibody titers for all groups with substantial improvement against the VOCs including the Omicron variant. We used linear regression and linear mixed model analysis. All results were adjusted for possible confounding of age and sex. Study limitations include the lack of cellular immunity data. CONCLUSIONS: Overall, this study shows that the mRNA vaccines appear superior to adenovirus vector-based vaccines in inducing neutralizing antibodies against VOCs four weeks after initial vaccination and after booster vaccination, which implies the use of mRNA vaccines for both initial and booster vaccination.