Computer-aided National Early Warning Score to predict the risk of sepsis following emergency medical admission to hospital: a model development and external validation study.

BACKGROUND: In hospitals in England, patients' vital signs are monitored and summarized into the National Early Warning Score (NEWS); this score is more accurate than the Quick Sepsis-related Organ Failure Assessment (qSOFA) score at identifying patients with sepsis. We investigated the extent to which the accuracy of the NEWS is enhanced by developing and comparing 3 computer-aided NEWS (cNEWS) models (M0 = NEWS alone, M1 = M0 + age + sex, M2 = M1 + subcomponents of NEWS + diastolic blood pressure) to predict the risk of sepsis. METHODS: We included all emergency medical admissions of patients 16 years of age and older discharged over 24 months from 2 acute care hospital centres (York Hospital [YH] for model development and a combined data set from 2 hospitals [Diana, Princess of Wales Hospital and Scunthorpe General Hospital] in the Northern Lincolnshire and Goole National Health Service Foundation Trust [NH] for external model validation). We used a validated Canadian method for defining sepsis from administrative hospital data. RESULTS: The prevalence of sepsis was lower in YH (4.5%, 1596/35 807) than in NH (8.5%, 2983/35 161). The C statistic increased across models (YH: M0 0.705, M1 0.763, M2 0.777; NH: M0 0.708, M1 0.777, M2 0.791). For NEWS of 5 or higher, sensitivity increased (YH: 47.24% v. 50.56% v. 52.69%; NH: 37.91% v. 43.35% v. 48.07%), the positive likelihood ratio increased (YH: 2.77 v. 2.99 v. 3.06; NH: 3.18 v. 3.32 v. 3.45) and the positive predictive value increased (YH: 11.44% v. 12.24% v. 12.49%; NH: 22.75% v. 23.55% v. 24.21%). INTERPRETATION: From the 3 cNEWS models, model M2 is the most accurate. Given that it places no additional burden of data collection on clinicians and can be automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Development and External Validation of an Automated Computer-Aided Risk Score for Predicting Sepsis in Emergency Medical Admissions Using the Patient's First Electronically Recorded Vital Signs and Blood Test Results.

OBJECTIVES: To develop a logistic regression model to predict the risk of sepsis following emergency medical admission using the patient's first, routinely collected, electronically recorded vital signs and blood test results and to validate this novel computer-aided risk of sepsis model, using data from another hospital. DESIGN: Cross-sectional model development and external validation study reporting the C-statistic based on a validated optimized algorithm to identify sepsis and severe sepsis (including septic shock) from administrative hospital databases using International Classification of Diseases, 10th Edition, codes. SETTING: Two acute hospitals (York Hospital - development data; Northern Lincolnshire and Goole Hospital - external validation data). PATIENTS: Adult emergency medical admissions discharged over a 24-month period with vital signs and blood test results recorded at admission. INTERVENTIONS: None. MAIN RESULTS: The prevalence of sepsis and severe sepsis was lower in York Hospital (18.5% = 4,861/2,6247; 5.3% = 1,387/2,6247) than Northern Lincolnshire and Goole Hospital (25.1% = 7,773/30,996; 9.2% = 2,864/30,996). The mortality for sepsis (York Hospital: 14.5% = 704/4,861; Northern Lincolnshire and Goole Hospital: 11.6% = 899/7,773) was lower than the mortality for severe sepsis (York Hospital: 29.0% = 402/1,387; Northern Lincolnshire and Goole Hospital: 21.4% = 612/2,864). The C-statistic for computer-aided risk of sepsis in York Hospital (all sepsis 0.78; sepsis: 0.73; severe sepsis: 0.80) was similar in an external hospital setting (Northern Lincolnshire and Goole Hospital: all sepsis 0.79; sepsis: 0.70; severe sepsis: 0.81). A cutoff value of 0.2 gives reasonable performance. CONCLUSIONS: We have developed a novel, externally validated computer-aided risk of sepsis, with reasonably good performance for estimating the risk of sepsis for emergency medical admissions using the patient's first, electronically recorded, vital signs and blood tests results. Since computer-aided risk of sepsis places no additional data collection burden on clinicians and is automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

A comparison of logistic regression models with alternative machine learning methods to predict the risk of in-hospital mortality in emergency medical admissions via external validation.

We compare the performance of logistic regression with several alternative machine learning methods to estimate the risk of death for patients following an emergency admission to hospital based on the patients' first blood test results and physiological measurements using an external validation approach. We trained and tested each model using data from one hospital (n = 24,696) and compared the performance of these models in data from another hospital (n = 13,477). We used two performance measures - the calibration slope and area under the receiver operating characteristic curve. The logistic model performed reasonably well - calibration slope: 0.90, area under the receiver operating characteristic curve: 0.847 compared to the other machine learning methods. Given the complexity of choosing tuning parameters of these methods, the performance of logistic regression with transformations for in-hospital mortality prediction was competitive with the best performing alternative machine learning methods with no evidence of overfitting.

Performance of externally validated enhanced computer-aided versions of the National Early Warning Score in predicting mortality following an emergency admission to hospital in England: a cross-sectional study.

OBJECTIVES: In the English National Health Service, the patient's vital signs are monitored and summarised into a National Early Warning Score (NEWS) to support clinical decision making, but it does not provide an estimate of the patient's risk of death. We examine the extent to which the accuracy of NEWS for predicting mortality could be improved by enhanced computer versions of NEWS (cNEWS). DESIGN: Logistic regression model development and external validation study. SETTING: Two acute hospitals (YH-York Hospital for model development; NH-Northern Lincolnshire and Goole Hospital for external model validation). PARTICIPANTS: Adult (≥16 years) medical admissions discharged over a 24-month period with electronic NEWS (eNEWS) recorded on admission are used to predict mortality at four time points (in-hospital, 24 hours, 48 hours and 72 hours) using the first electronically recorded NEWS (model M0) versus a cNEWS model which included age+sex (model M1) +subcomponents of NEWS (including diastolic blood pressure) (model M2). RESULTS: The risk of dying in-hospital following emergency medical admission was 5.8% (YH: 2080/35 807) and 5.4% (NH: 1900/35 161). The c-statistics for model M2 in YH for predicting mortality (in-hospital=0.82, 24 hours=0.91, 48 hours=0.88 and 72 hours=0.88) was higher than model M0 (in-hospital=0.74, 24 hours=0.89, 48 hours=0.86 and 72 hours=0.85) with higher Positive Predictive Value (PPVs) for in-hospital mortality (M2 19.3% and M0 16.6%). Similar findings were seen in NH. Model M2 performed better than M0 in almost all major disease subgroups. CONCLUSIONS: An externally validated enhanced computer-aided NEWS model (cNEWS) incrementally improves on the performance of a NEWS only model. Since cNEWS places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated to determine if it can improve care in hospitals that have eNEWS systems.

The inclusion of delirium in version 2 of the National Early Warning Score will substantially increase the alerts for escalating levels of care: findings from a retrospective database study of emergency medical admissions in two hospitals .

BACKGROUND: The National Early Warning Score (NEWS) is being replaced with NEWS2 which adds 3 points for new confusion or delirium. We estimated the impact of adding delirium on the number of medium/high level alerts that are triggers to escalate care. METHODS: Analysis of emergency medical admissions in two acute hospitals (York Hospital (YH) and Northern Lincolnshire and Goole NHS Foundation Trust hospitals (NH)) in England. Twenty per cent were randomly assigned to have delirium. RESULTS: The number of emergency admissions (YH: 35584; NH: 35795), mortality (YH: 5.7%; NH: 5.5%), index NEWS (YH: 2.5; NH: 2.1) and numbers of NEWS recorded (YH: 879193; NH: 884072) were similar in each hospital. The mean number of patients with medium level alerts per day increased from 55.3 (NEWS) to 69.5 (NEWS2), a 25.7% increase in YH and 64.1 (NEWS) to 77.4 (NEWS2), a 20.7% increase in NH. The mean number of patients with high level alerts per day increased from 27.3 (NEWS) to 34.4 (NEWS2), a 26.0% increase in YH and 29.9 (NEWS) to 37.7 (NEWS2), a 26.1% increase in NH. CONCLUSIONS: The addition of delirium in NEWS2 will have a substantial increase in medium and high level alerts in hospitalised emergency medical patients. Rigorous evaluation of NEWS2 is required before widespread implementation because the extent to which staff can cope with this increase without adverse consequences remains unknown.

Development and validation of a novel computer-aided score to predict the risk of in-hospital mortality for acutely ill medical admissions in two acute hospitals using their first electronically recorded blood test results and vital signs: a cross-sectional study.

OBJECTIVES: There are no established mortality risk equations specifically for emergency medical patients who are admitted to a general hospital ward. Such risk equations may be useful in supporting the clinical decision-making process. We aim to develop and externally validate a computer-aided risk of mortality (CARM) score by combining the first electronically recorded vital signs and blood test results for emergency medical admissions. DESIGN: Logistic regression model development and external validation study. SETTING: Two acute hospitals (Northern Lincolnshire and Goole NHS Foundation Trust Hospital (NH)-model development data; York Hospital (YH)-external validation data). PARTICIPANTS: Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic National Early Warning Score(s) and blood test results recorded on admission. RESULTS: The risk of in-hospital mortality following emergency medical admission was 5.7% (NH: 1766/30 996) and 6.5% (YH: 1703/26 247). The C-statistic for the CARM score in NH was 0.87 (95% CI 0.86 to 0.88) and was similar in an external hospital setting YH (0.86, 95% CI 0.85 to 0.87) and the calibration slope included 1 (0.97, 95% CI 0.94 to 1.00). CONCLUSIONS: We have developed a novel, externally validated CARM score with good performance characteristics for estimating the risk of in-hospital mortality following an emergency medical admission using the patient's first, electronically recorded, vital signs and blood test results. Since the CARM score places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Impact of the level of sickness on higher mortality in emergency medical admissions to hospital at weekends.

Objective Routine administrative data have been used to show that patients admitted to hospitals over the weekend appear to have a higher mortality compared to weekday admissions. Such data do not take the severity of sickness of a patient on admission into account. Our aim was to incorporate a standardized vital signs physiological-based measure of sickness known as the National Early Warning Score to investigate if weekend admissions are: sicker as measured by their index National Early Warning Score; have an increased mortality; and experience longer delays in the recording of their index National Early Warning Score. Methods We extracted details of all adult emergency medical admissions during 2014 from hospital databases and linked these with electronic National Early Warning Score data in four acute hospitals. We analysed 47,117 emergency admissions after excluding 1657 records, where National Early Warning Score was missing or the first (index) National Early Warning Score was recorded outside ±24 h of the admission time. Results Emergency medical admissions at the weekend had higher index National Early Warning Score (weekend: 2.53 vs. weekday: 2.30, p < 0.001) with a higher mortality (weekend: 706/11,332 6.23% vs. weekday: 2039/35,785 5.70%; odds ratio = 1.10, 95% CI 1.01 to 1.20, p = 0.04) which was no longer seen after adjusting for the index National Early Warning Score (odds ratio = 0.99, 95% CI 0.90 to 1.09, p = 0.87). Index National Early Warning Score was recorded sooner (-0.45 h, 95% CI -0.52 to -0.38, p < 0.001) for weekend admissions. Conclusions Emergency medical admissions at the weekend with electronic National Early Warning Score recorded within 24 h are sicker, have earlier clinical assessments, and after adjusting for the severity of their sickness, do not appear to have a higher mortality compared to weekday admissions. A larger definitive study to confirm these findings is needed.