Enhancement in Performance of Septic Shock Prediction Using National Early Warning Score, Initial Triage Information, and Machine Learning Analysis.

BACKGROUND: Several studies reported that the National Early Warning Score (NEWS) has shown superiority over other screening tools in discriminating emergency department (ED) patients who are likely to progress to septic shock. OBJECTIVES: To improve the performance of the NEWS for septic shock prediction by adding variables collected during ED triage, and to implement a machine-learning algorithm. METHODS: The study population comprised adult ED patients with suspected infection. To detect septic shock within 24 h after ED arrival, the Sepsis-3 clinical criteria and nine variables were used: NEWS, age, gender, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, and oxygen saturation. The model was developed using logistic regression (LR), extreme gradient boosting (XGB), and artificial neural network (ANN) algorithms. The evaluations were performed using an area under the receiver operating characteristic curve (AUROC), Hosmer-Lemeshow test, and net reclassification index (NRI). RESULTS: Overall, 41,687 patients were enrolled. The AUROC of the model with NEWS, age, gender, and the six vital signs (0.835-0.845) was better than that of the baseline model (0.804). The XGB model (AUROC 0.845) was the most accurate, compared with LR (0.844) and ANN (0.835). The LR and XGB models were well calibrated; however, the ANN showed poor calibration power. The LR and XGB models showed better reclassification than the baseline model with positive NRI. CONCLUSION: The discrimination power of the model for screening septic shock using NEWS, age, gender, and the six vital signs collected at ED triage outperformed the baseline NEWS model.

Prediction of Critical Care Outcome for Adult Patients Presenting to Emergency Department Using Initial Triage Information: An XGBoost Algorithm Analysis.

BACKGROUND: The emergency department (ED) triage system to classify and prioritize patients from high risk to less urgent continues to be a challenge. OBJECTIVE: This study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcomes for adult patients using information collected during ED triage and compare the performance with that of the baseline model using the Korean Triage and Acuity Scale (KTAS). METHODS: To predict the need for critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, the time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation, and level of consciousness. The baseline model with KTAS was developed using logistic regression, and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by the area under the receiver operating characteristic (AUROC) curve. The ability of calibration with Hosmer-Lemeshow test and reclassification with net reclassification index were evaluated. The calibration plot and partial dependence plot were used in the analysis. RESULTS: The AUROC of the model with the full set of variables (0.833-0.861) was better than that of the baseline model (0.796). The XGB model of AUROC 0.861 (95% CI 0.848-0.874) showed a higher discriminative performance than the DNN model of 0.833 (95% CI 0.819-0.848). The XGB and DNN models proved better reclassification than the baseline model with a positive net reclassification index. The XGB models were well-calibrated (Hosmer-Lemeshow test; P>.05); however, the DNN showed poor calibration power (Hosmer-Lemeshow test; P<.001). We further interpreted the nonlinear association between variables and critical care prediction. CONCLUSIONS: Our study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.