Development and Validation of a Machine Learning Model for Early Detection of Untreated Infection.

BACKGROUND: Early diagnostic uncertainty for infection causes delays in antibiotic administration in infected patients and unnecessary antibiotic administration in noninfected patients. OBJECTIVE: To develop a machine learning model for the early detection of untreated infection (eDENTIFI), with the presence of infection determined by clinician chart review. DERIVATION COHORT: Three thousand three hundred fifty-seven adult patients hospitalized between 2006 and 2018 at two health systems in Illinois, United States. VALIDATION COHORT: We validated in 1632 patients in a third Illinois health system using area under the receiver operating characteristic curve (AUC). PREDICTION MODEL: Using a longitudinal discrete-time format, we trained a gradient boosted machine model to predict untreated infection in the next 6 hours using routinely available patient demographics, vital signs, and laboratory results. RESULTS: eDENTIFI had an AUC of 0.80 (95% CI, 0.79-0.81) in the validation cohort and outperformed the systemic inflammatory response syndrome criteria with an AUC of 0.64 (95% CI, 0.64-0.65; p < 0.001). The most important features were body mass index, age, temperature, and heart rate. Using a threshold with a 47.6% sensitivity, eDENTIFI detected infection a median 2.0 hours (interquartile range, 0.9-5.2 hr) before antimicrobial administration, with a negative predictive value of 93.6%. Antibiotic administration guided by eDENTIFI could have decreased unnecessary IV antibiotic administration in noninfected patients by 10.8% absolute or 46.4% relative percentage points compared with clinicians. CONCLUSION: eDENTIFI could both decrease the time to antimicrobial administration in infected patients and unnecessary antibiotic administration in noninfected patients. Further prospective validation is needed.

Distinct immune profiles and clinical outcomes in sepsis subphenotypes based on temperature trajectories.

PURPOSE: Sepsis is a heterogeneous syndrome. Identification of sepsis subphenotypes with distinct immune profiles could lead to targeted therapies. This study investigates the immune profiles of patients with sepsis following distinct body temperature patterns (i.e., temperature trajectory subphenotypes). METHODS: Hospitalized patients from four hospitals between 2018 and 2022 with suspicion of infection were included. A previously validated temperature trajectory algorithm was used to classify study patients into temperature trajectory subphenotypes. Microbiological profiles, clinical outcomes, and levels of 31 biomarkers were compared between these subphenotypes. RESULTS: The 3576 study patients were classified into four temperature trajectory subphenotypes: hyperthermic slow resolvers (N = 563, 16%), hyperthermic fast resolvers (N = 805, 23%), normothermic (N = 1693, 47%), hypothermic (N = 515, 14%). The mortality rate was significantly different between subphenotypes, with the highest rate in hypothermics (14.2%), followed by hyperthermic slow resolvers 6%, normothermic 5.5%, and lowest in hyperthermic fast resolvers 3.6% (p < 0.001). After multiple testing correction for the 31 biomarkers tested, 20 biomarkers remained significantly different between temperature trajectories: angiopoietin-1 (Ang-1), C-reactive protein (CRP), feline McDonough sarcoma-like tyrosine kinase 3 ligand (Flt-3l), granulocyte colony stimulating factor (G-CSF), granulocyte-macrophage colony stimulating factor (GM-CSF), interleukin (IL)-15, IL-1 receptor antagonist (RA), IL-2, IL-6, IL-7, interferon gamma-induced protein 10 (IP-10), monocyte chemoattractant protein-1 (MCP-1), human macrophage inflammatory protein 3 alpha (MIP-3a), neutrophil gelatinase-associated lipocalin (NGAL), pentraxin-3, thrombomodulin, tissue factor, soluble triggering receptor expressed on myeloid cells-1 (sTREM-1), and vascular cellular adhesion molecule-1 (vCAM-1).The hyperthermic fast and slow resolvers had the highest levels of most pro- and anti-inflammatory cytokines. Hypothermics had suppressed levels of most cytokines but the highest levels of several coagulation markers (Ang-1, thrombomodulin, tissue factor). CONCLUSION: Sepsis subphenotypes identified using the universally available measurement of body temperature had distinct immune profiles. Hypothermic patients, who had the highest mortality rate, also had the lowest levels of most pro- and anti-inflammatory cytokines.

The Precision Resuscitation With Crystalloids in Sepsis (PRECISE) Trial: A Trial Protocol.

Importance: Intravenous fluids are an essential part of treatment in sepsis, but there remains clinical equipoise on which type of crystalloid fluids to use in sepsis. A previously reported sepsis subphenotype (ie, group D) has demonstrated a substantial mortality benefit from balanced crystalloids compared with normal saline. Objective: To test the hypothesis that targeting balanced crystalloids to patients with group D sepsis through an electronic health record (EHR) alert will reduce 30-day inpatient mortality. Design, Setting, and Participants: The Precision Resuscitation With Crystalloids in Sepsis (PRECISE) trial is a parallel-group, multihospital, single-blind, pragmatic randomized clinical trial to be conducted at 6 hospitals in the Emory Healthcare system. Patients with suspicion of group D infection in whom a clinician initiates an order for normal saline in the emergency department (ED) or intensive care unit (ICU) will be randomized to usual care and intervention arms. Intervention: An EHR alert that appears in the ED and ICUs to nudge clinicians to use balanced crystalloids instead of normal saline. Main Outcomes and Measures: The primary outcome is 30-day inpatient mortality. Secondary outcomes are ICU admission, in-hospital mortality, receipt of vasoactive drugs, receipt of new kidney replacement therapy, and receipt of mechanical ventilation (vasoactive drugs, kidney replacement therapy, and mechanical ventilation are counted if they occur after randomization and within the 30-day study period). Intention-to-treat analysis will be conducted. Discussion: The PRECISE trial may be one of the first precision medicine trials of crystalloid fluids in sepsis. Using routine vital signs (temperature, heart rate, respiratory rate, and blood pressure), available even in low-resource settings, a validated machine learning algorithm will prospectively identify and enroll patients with group D sepsis who may have a substantial mortality reduction from used of balanced crystalloids compared with normal saline. Results: On finalizing participant enrollment and analyzing the data, the study's findings will be shared with the public through publication in a peer-reviewed journal. Conclusions: With use of a validated machine learning algorithm, precision resuscitation in sepsis could fundamentally redefine international standards for intravenous fluid resuscitation. Trial Registration: ClinicalTrials.gov Identifier: NCT06253585.

A Common Longitudinal Intensive Care Unit data Format (CLIF) to enable multi-institutional federated critical illness research.

Background: Critical illness, or acute organ failure requiring life support, threatens over five million American lives annually. Electronic health record (EHR) data are a source of granular information that could generate crucial insights into the nature and optimal treatment of critical illness. However, data management, security, and standardization are barriers to large-scale critical illness EHR studies. Methods: A consortium of critical care physicians and data scientists from eight US healthcare systems developed the Common Longitudinal Intensive Care Unit (ICU) data Format (CLIF), an open-source database format that harmonizes a minimum set of ICU Data Elements for use in critical illness research. We created a pipeline to process adult ICU EHR data at each site. After development and iteration, we conducted two proof-of-concept studies with a federated research architecture: 1) an external validation of an in-hospital mortality prediction model for critically ill patients and 2) an assessment of 72-hour temperature trajectories and their association with mechanical ventilation and in-hospital mortality using group-based trajectory models. Results: We converted longitudinal data from 94,356 critically ill patients treated in 2020-2021 (mean age 60.6 years [standard deviation 17.2], 30% Black, 7% Hispanic, 45% female) across 8 health systems and 33 hospitals into the CLIF format, The in-hospital mortality prediction model performed well in the health system where it was derived (0.81 AUC, 0.06 Brier score). Performance across CLIF consortium sites varied (AUCs: 0.74-0.83, Brier scores: 0.06-0.01), and demonstrated some degradation in predictive capability. Temperature trajectories were similar across health systems. Hypothermic and hyperthermic-slow-resolver patients consistently had the highest mortality. Conclusions: CLIF facilitates efficient, rigorous, and reproducible critical care research. Our federated case studies showcase CLIF's potential for disease sub-phenotyping and clinical decision-support evaluation. Future applications include pragmatic EHR-based trials, target trial emulations, foundational multi-modal AI models of critical illness, and real-time critical care quality dashboards.

Racial Differences in Accuracy of Predictive Models for High-Flow Nasal Cannula Failure in COVID-19.

OBJECTIVES: To develop and validate machine learning (ML) models to predict high-flow nasal cannula (HFNC) failure in COVID-19, compare their performance to the respiratory rate-oxygenation (ROX) index, and evaluate model accuracy by self-reported race. DESIGN: Retrospective cohort study. SETTING: Four Emory University Hospitals in Atlanta, GA. PATIENTS: Adult patients hospitalized with COVID-19 between March 2020 and April 2022 who received HFNC therapy within 24 hours of ICU admission were included. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Four types of supervised ML models were developed for predicting HFNC failure (defined as intubation or death within 7 d of HFNC initiation), using routine clinical variables from the first 24 hours of ICU admission. Models were trained on the first 60% (n = 594) of admissions and validated on the latter 40% (n = 390) of admissions to simulate prospective implementation. Among 984 patients included, 317 patients (32.2%) developed HFNC failure. eXtreme Gradient Boosting (XGB) model had the highest area under the receiver-operator characteristic curve (AUROC) for predicting HFNC failure (0.707), and was the only model with significantly better performance than the ROX index (AUROC 0.616). XGB model had significantly worse performance in Black patients compared with White patients (AUROC 0.663 vs. 0.808, p = 0.02). Racial differences in the XGB model were reduced and no longer statistically significant when restricted to patients with nonmissing arterial blood gas data, and when XGB model was developed to predict mortality (rather than the composite outcome of failure, which could be influenced by biased clinical decisions for intubation). CONCLUSIONS: Our XGB model had better discrimination for predicting HFNC failure in COVID-19 than the ROX index, but had racial differences in accuracy of predictions. Further studies are needed to understand and mitigate potential sources of biases in clinical ML models and to improve their equitability.

Using Trajectories of Bedside Vital Signs to Identify COVID-19 Subphenotypes.

BACKGROUND: Trajectories of bedside vital signs have been used to identify sepsis subphenotypes with distinct outcomes and treatment responses. The objective of this study was to validate the vitals trajectory model in a multicenter cohort of patients hospitalized with COVID-19 and to evaluate the clinical characteristics and outcomes of the resulting subphenotypes. RESEARCH QUESTION: Can the trajectory of routine bedside vital signs identify COVID-19 subphenotypes with distinct clinical characteristics and outcomes? STUDY DESIGN AND METHODS: The study included adult patients admitted with COVID-19 to four academic hospitals in the Emory Healthcare system between March 1, 2020, and May 31, 2022. Using a validated group-based trajectory model, we classified patients into previously defined vital sign trajectories using oral temperature, heart rate, respiratory rate, and systolic and diastolic BP measured in the first 8 h of hospitalization. Clinical characteristics, biomarkers, and outcomes were compared between subphenotypes. Heterogeneity of treatment effect to tocilizumab was evaluated. RESULTS: The 7,065 patients with hospitalized COVID-19 were classified into four subphenotypes: group A (n = 1,429, 20%)-high temperature, heart rate, respiratory rate, and hypotensive; group B (1,454, 21%)-high temperature, heart rate, respiratory rate, and hypertensive; group C (2,996, 42%)-low temperature, heart rate, respiratory rate, and normotensive; and group D (1,186, 17%)-low temperature, heart rate, respiratory rate, and hypotensive. Groups A and D had higher ORs of mechanical ventilation, vasopressors, and 30-day inpatient mortality (P < .001). On comparing patients receiving tocilizumab (n = 55) with those who met criteria for tocilizumab but were admitted before its use (n = 461), there was significant heterogeneity of treatment effect across subphenotypes in the association of tocilizumab with 30-day mortality (P = .001). INTERPRETATION: By using bedside vital signs available in even low-resource settings, we found novel subphenotypes associated with distinct manifestations of COVID-19, which could lead to preemptive and targeted treatments.

Leveraging Data Science and Novel Technologies to Develop and Implement Precision Medicine Strategies in Critical Care.

Precision medicine aims to identify treatments that are most likely to result in favorable outcomes for subgroups of patients with similar clinical and biological characteristics. The gaps for the development and implementation of precision medicine strategies in the critical care setting are many, but the advent of data science and multi-omics approaches, combined with the rich data ecosystem in the intensive care unit, offer unprecedented opportunities to realize the promise of precision critical care. In this article, the authors review the data-driven and technology-based approaches being leveraged to discover and implement precision medicine strategies in the critical care setting.

HIRA: Heart Rate Interval based Rapid Alert score to characterize autonomic dysfunction among patients with sepsis-related acute respiratory failure (ARF).

Objective. To examine whether heart rate interval based rapid alert (HIRA) score derived from a combination model of heart rate variability (HRV) and modified early warning score (MEWS) is a surrogate for the detection of acute respiratory failure (ARF) in critically ill sepsis patients.Approach. Retrospective HRV analysis of sepsis patients admitted to Emory healthcare intensive care unit (ICU) was performed between sepsis-related ARF and sepsis controls without ARF. HRV measures such as time domain, frequency domain, and nonlinear measures were analyzed up to 24 h after patient admission, 1 h before the onset of ARF, and a random event time in the sepsis controls. Statistical significance was computed by the Wilcoxon Rank Sum test. Machine learning algorithms such as eXtreme Gradient Boosting and logistic regression were developed to validate the HIRA score model. The performance of HIRA and early warning score models were evaluated using the area under the receiver operating characteristic (AUROC).Main Results. A total of 89 (ICU) patients with sepsis were included in this retrospective cohort study, of whom 31 (34%) developed sepsis-related ARF and 58 (65%) were sepsis controls without ARF. Time-domain HRV for Electrocardiogram (ECG) Beat-to-Beat RR intervals strongly distinguished ARF patients from controls. HRV measures for nonlinear and frequency domains were significantly altered (p< 0.05) among ARF compared to controls. The HIRA score AUC: 0.93; 95% confidence interval (CI): 0.88-0.98) showed a higher predictive ability to detect ARF when compared to MEWS (AUC: 0.71; 95% CI: 0.50-0.90).Significance. HRV was significantly impaired across patients who developed ARF when compared to controls. The HIRA score uses non-invasively derived HRV and may be used to inform diagnostic and therapeutic decisions regarding the severity of sepsis and earlier identification of the need for mechanical ventilation.

Comparison of time series clustering methods for identifying novel subphenotypes of patients with infection.

OBJECTIVE: Severe infection can lead to organ dysfunction and sepsis. Identifying subphenotypes of infected patients is essential for personalized management. It is unknown how different time series clustering algorithms compare in identifying these subphenotypes. MATERIALS AND METHODS: Patients with suspected infection admitted between 2014 and 2019 to 4 hospitals in Emory healthcare were included, split into separate training and validation cohorts. Dynamic time warping (DTW) was applied to vital signs from the first 8 h of hospitalization, and hierarchical clustering (DTW-HC) and partition around medoids (DTW-PAM) were used to cluster patients into subphenotypes. DTW-HC, DTW-PAM, and a previously published group-based trajectory model (GBTM) were evaluated for agreement in subphenotype clusters, trajectory patterns, and subphenotype associations with clinical outcomes and treatment responses. RESULTS: There were 12 473 patients in training and 8256 patients in validation cohorts. DTW-HC, DTW-PAM, and GBTM models resulted in 4 consistent vitals trajectory patterns with significant agreement in clustering (71-80% agreement, P < .001): group A was hyperthermic, tachycardic, tachypneic, and hypotensive. Group B was hyperthermic, tachycardic, tachypneic, and hypertensive. Groups C and D had lower temperatures, heart rates, and respiratory rates, with group C normotensive and group D hypotensive. Group A had higher odds ratio of 30-day inpatient mortality (P < .01) and group D had significant mortality benefit from balanced crystalloids compared to saline (P < .01) in all 3 models. DISCUSSION: DTW- and GBTM-based clustering algorithms applied to vital signs in infected patients identified consistent subphenotypes with distinct clinical outcomes and treatment responses. CONCLUSION: Time series clustering with distinct computational approaches demonstrate similar performance and significant agreement in the resulting subphenotypes.

Temperature Trajectory Subphenotypes in Oncology Patients with Neutropenia and Suspected Infection.

Rationale: Despite etiologic and severity heterogeneity in neutropenic sepsis, management is often uniform. Understanding host response clinical subphenotypes might inform treatment strategies for neutropenic sepsis. Objectives: In this retrospective two-hospital study, we analyzed whether temperature trajectory modeling could identify distinct, clinically relevant subphenotypes among oncology patients with neutropenia and suspected infection. Methods: Among adult oncologic admissions with neutropenia and blood cultures within 24 hours, a previously validated model classified patients' initial 72-hour temperature trajectories into one of four subphenotypes. We analyzed subphenotypes' independent relationships with hospital mortality and bloodstream infection using multivariable models. Measurements and Main Results: Patients (primary cohort n = 1,145, validation cohort n = 6,564) fit into one of four temperature subphenotypes. "Hyperthermic slow resolvers" (pooled n = 1,140 [14.8%], mortality n = 104 [9.1%]) and "hypothermic" encounters (n = 1,612 [20.9%], mortality n = 138 [8.6%]) had higher mortality than "hyperthermic fast resolvers" (n = 1,314 [17.0%], mortality n = 47 [3.6%]) and "normothermic" (n = 3,643 [47.3%], mortality n = 196 [5.4%]) encounters (P < 0.001). Bloodstream infections were more common among hyperthermic slow resolvers (n = 248 [21.8%]) and hyperthermic fast resolvers (n = 240 [18.3%]) than among hypothermic (n = 188 [11.7%]) or normothermic (n = 418 [11.5%]) encounters (P < 0.001). Adjusted for confounders, hyperthermic slow resolvers had increased adjusted odds for mortality (primary cohort odds ratio, 1.91 [P = 0.03]; validation cohort odds ratio, 2.19 [P < 0.001]) and bloodstream infection (primary odds ratio, 1.54 [P = 0.04]; validation cohort odds ratio, 2.15 [P < 0.001]). Conclusions: Temperature trajectory subphenotypes were independently associated with important outcomes among hospitalized patients with neutropenia in two independent cohorts.

Factors Associated With Diagnostic Accuracy of Robotic Bronchoscopy With 12-Month Follow-up.

BACKGROUND: Robotic bronchoscopy (RB) aims to increase the diagnostic yield of guided bronchoscopy by providing improved navigation, farther reach, and stability during lesion sampling. METHODS: We reviewed data on consecutive cases in which RB was used to diagnose lung lesions from June 15, 2018, to December 15, 2019, at the University of Chicago Medical Center. RESULTS: The median lesion size was 20.5 mm. All patients had at least 12 months of follow-up. The overall diagnostic accuracy was 77% (95 of 124). The diagnostic accuracy was 85%, 84%, and 38% for concentric, eccentric, and absent radial endobronchial ultrasound (r-EBUS) views, respectively (P < .001). A positive r-EBUS view and lesions size of 20 to 30 mm had higher odds of achieving a diagnosis on multivariate analysis. The 12-month diagnostic accuracy, sensitivity, specificity, and positive and negative predictive value for malignancy were 77%, 69%, 100%, 100%, and 58%, respectively. Pneumothorax was noted in 1.6% (n = 2) patients with bleeding reported in 3.2% (n = 4). No postprocedure respiratory failure was noted. CONCLUSIONS: The overall diagnostic accuracy using RB for pulmonary lesion sampling in our cohort with 12-month follow-up compared favorably with established guided bronchoscopy technologies. Lesion size ≥20 mm and confirmation by r-EBUS predicted higher accuracy independent of concentric or eccentric r-EBUS patterns.

Hospital trajectories and early predictors of clinical outcomes differ between SARS-CoV-2 and influenza pneumonia.

BACKGROUND: A comparison of pneumonias due to SARS-CoV-2 and influenza, in terms of clinical course and predictors of outcomes, might inform prognosis and resource management. We aimed to compare clinical course and outcome predictors in SARS-CoV-2 and influenza pneumonia using multi-state modelling and supervised machine learning on clinical data among hospitalised patients. METHODS: This multicenter retrospective cohort study of patients hospitalised with SARS-CoV-2 (March-December 2020) or influenza (Jan 2015-March 2020) pneumonia had the composite of hospital mortality and hospice discharge as the primary outcome. Multi-state models compared differences in oxygenation/ventilatory utilisation between pneumonias longitudinally throughout hospitalisation. Differences in predictors of outcome were modelled using supervised machine learning classifiers. FINDINGS: Among 2,529 hospitalisations with SARS-CoV-2 and 2,256 with influenza pneumonia, the primary outcome occurred in 21% and 9%, respectively. Multi-state models differentiated oxygen requirement progression between viruses, with SARS-CoV-2 manifesting rapidly-escalating early hypoxemia. Highly contributory classifier variables for the primary outcome differed substantially between viruses. INTERPRETATION: SARS-CoV-2 and influenza pneumonia differ in presentation, hospital course, and outcome predictors. These pathogen-specific differential responses in viral pneumonias suggest distinct management approaches should be investigated. FUNDING: This project was supported by NIH/NCATS UL1 TR002345, NIH/NCATS KL2 TR002346 (PGL), the Doris Duke Charitable Foundation grant 2015215 (PGL), NIH/NHLBI R35 HL140026 (CSC), and a Big Ideas Award from the BJC HealthCare and Washington University School of Medicine Healthcare Innovation Lab and NIH/NIGMS R35 GM142992 (PS).

Development and validation of novel sepsis subphenotypes using trajectories of vital signs.

PURPOSE: Sepsis is a heterogeneous syndrome and identification of sub-phenotypes is essential. This study used trajectories of vital signs to develop and validate sub-phenotypes and investigated the interaction of sub-phenotypes with treatment using randomized controlled trial data. METHODS: All patients with suspected infection admitted to four academic hospitals in Emory Healthcare between 2014-2017 (training cohort) and 2018-2019 (validation cohort) were included. Group-based trajectory modeling was applied to vital signs from the first 8 h of hospitalization to develop and validate vitals trajectory sub-phenotypes. The associations between sub-phenotypes and outcomes were evaluated in patients with sepsis. The interaction between sub-phenotype and treatment with balanced crystalloids versus saline was tested in a secondary analysis of SMART (Isotonic Solutions and Major Adverse Renal Events Trial). RESULTS: There were 12,473 patients with suspected infection in training and 8256 patients in validation cohorts, and 4 vitals trajectory sub-phenotypes were found. Group A (N = 3483, 28%) were hyperthermic, tachycardic, tachypneic, and hypotensive. Group B (N = 1578, 13%) were hyperthermic, tachycardic, tachypneic (not as pronounced as Group A) and hypertensive. Groups C (N = 4044, 32%) and D (N = 3368, 27%) had lower temperatures, heart rates, and respiratory rates, with Group C normotensive and Group D hypotensive. In the 6,919 patients with sepsis, Groups A and B were younger while Groups C and D were older. Group A had the lowest prevalence of congestive heart failure, hypertension, diabetes mellitus, and chronic kidney disease, while Group B had the highest prevalence. Groups A and D had the highest vasopressor use (p < 0.001 for all analyses above). In logistic regression, 30-day mortality was significantly higher in Groups A and D (p < 0.001 and p = 0.03, respectively). In the SMART trial, sub-phenotype significantly modified treatment effect (p = 0.03). Group D had significantly lower odds of mortality with balanced crystalloids compared to saline (odds ratio (OR) 0.39, 95% confidence interval (CI) 0.23-0.67, p < 0.001). CONCLUSION: Sepsis sub-phenotypes based on vital sign trajectory were consistent across cohorts, had distinct outcomes, and different responses to treatment with balanced crystalloids versus saline.

Identifying infected patients using semi-supervised and transfer learning.

OBJECTIVES: Early identification of infection improves outcomes, but developing models for early identification requires determining infection status with manual chart review, limiting sample size. Therefore, we aimed to compare semi-supervised and transfer learning algorithms with algorithms based solely on manual chart review for identifying infection in hospitalized patients. MATERIALS AND METHODS: This multicenter retrospective study of admissions to 6 hospitals included "gold-standard" labels of infection from manual chart review and "silver-standard" labels from nonchart-reviewed patients using the Sepsis-3 infection criteria based on antibiotic and culture orders. "Gold-standard" labeled admissions were randomly allocated to training (70%) and testing (30%) datasets. Using patient characteristics, vital signs, and laboratory data from the first 24 hours of admission, we derived deep learning and non-deep learning models using transfer learning and semi-supervised methods. Performance was compared in the gold-standard test set using discrimination and calibration metrics. RESULTS: The study comprised 432 965 admissions, of which 2724 underwent chart review. In the test set, deep learning and non-deep learning approaches had similar discrimination (area under the receiver operating characteristic curve of 0.82). Semi-supervised and transfer learning approaches did not improve discrimination over models fit using only silver- or gold-standard data. Transfer learning had the best calibration (unreliability index P value: .997, Brier score: 0.173), followed by self-learning gradient boosted machine (P value: .67, Brier score: 0.170). DISCUSSION: Deep learning and non-deep learning models performed similarly for identifying infection, as did models developed using Sepsis-3 and manual chart review labels. CONCLUSION: In a multicenter study of almost 3000 chart-reviewed patients, semi-supervised and transfer learning models showed similar performance for model discrimination as baseline XGBoost, while transfer learning improved calibration.