Review and performance evaluation of aggregate weighted 'track and trigger' systems.

OBJECTIVES: There is no up-to-date literature review of physiologically based, aggregate weighted 'track and trigger' systems (AWTTS) and few data on their predictive ability for serious adverse outcomes. The aim of this study was to describe the AWTTS in clinical use and assess their ability to discriminate between survivors and non-survivors of hospital admission, based on an initial set of vital signs. MATERIALS AND METHODS: A systematic review of the literature was performed, to describe the AWTTS, their components and their differences. Their ability to discriminate between survivors and non-survivors was evaluated using the area under the receiver-operating characteristics (AUROC) curve, and a database of 9987 vital signs datasets. RESULTS: A total of 33 unique AWTTS were identified with AUROC (+/-95% CI) ranging from 0.657 (0.636-0.678) to 0.782 (0.767-0.797). 12 AWTTS (36%) discriminated reasonably well between survivors and non-survivors, the top four performing AWTTS incorporated age as a component (AUROCs ranging from 0.722 to 0.782). The top two systems also incorporated temperature. CONCLUSIONS: There is a wide range of unique, but very similar, AWTTS in clinical use. There is no consistency regarding their physiological components, but the majority differ only in minor variations in the weightings for physiological derangement and/or the cut-off points between physiological weighting bands. The performance of most systems tested was poor when used to discriminate between survivors and non-survivors, although 36% discriminated reasonably well. Our results suggest that physiology can be used to predict outcome, but that further work is required to improve the AWTTS models.

A review, and performance evaluation, of single-parameter "track and trigger" systems.

OBJECTIVES: There is no up-to-date literature review of physiologically-based, single-parameter weighted "track and trigger" systems (SPTTS) and little data on their sensitivity and specificity to predict adverse outcomes. The aim of this study was to describe the SPTTS in clinical use and measure their sensitivity and specificity when using admission vital signs data for predicting in-hospital mortality. MATERIALS AND METHODS: We performed a systematic review of the literature to describe the SPTTS, their components and their differences. We measured their sensitivity and specificity for predicting in-hospital mortality when using a database of 9987 admission vital signs datasets. RESULTS: We identified 39 unique classes of SPTTS, of which 30 were evaluated. There was considerable variation in the physiological variables used, together with significant variation in the physiological values used to trigger a medical emergency or critical care outreach team. There was marked variation in sensitivity (7.3-52.8%), specificity (69.1-98.1%), positive predictive values (13.5-26.1%), negative predictive values (92.1-94.2%) and the potential number of calls triggered (234-3271). CONCLUSIONS: There is a wide range of unique, but very similar, SPTTS in clinical use. Although specificities were high, sensitivities were too low to provide institutions with confidence that these SPTTS could identify patients at risk of in-hospital death using admission vital signs. Institutions may wish to consider these data when selecting which, if any, single-parameter track and trigger systems to introduce.

Should age be included as a component of track and trigger systems used to identify sick adult patients?

AIM OF STUDY: Few published "track and trigger systems" used to identify sick adult patients incorporate patient age as a variable. We investigated the relationship between vital signs, patient age and in-hospital mortality and investigated the impact of patient age on the function as predictors of in-hospital mortality of the two most commonly used track and trigger systems. MATERIALS AND METHODS: Using a database of 9987 vital signs datasets, we studied the relationship between admission vital signs and in-hospital mortality for a range of selected vital signs, grouped by patient age. We also used the vital signs data set to study the impact of patient age on the relationship between patient triggers using the "MET criteria" and "MEWS", and in-hospital mortality. RESULTS: At hospital discharge, there were 9152 (91.6%) survivors and 835 (8.4%) non-survivors. As admission vital signs worsened, mortality increased for each age range. Where groups of patients had triggered a certain MET criterion, mortality was higher as patient age increased. Mortality varied significantly with age (p<0.05; Fishers exact test) for breathing rate >36breathsmin(-1), systolic BP<90mmHg and decreased conscious level. For each age group, mortality also increased as total MEWS score increased. As the number of simultaneously occurring MEWS abnormalities, or simultaneously occurring MET criteria, increased, mortality increased for each age range. CONCLUSIONS: Age has a significant impact on in-hospital mortality. Our data suggest that the inclusion of age as a component of these systems could be advantageous in improving their function.

Calculating early warning scores--a classroom comparison of pen and paper and hand-held computer methods.

To assist in the early detection of critical illness, many hospitals now use a "track and trigger" system that allocates points to routine vital signs measurements on the basis of their derangement from an arbitrarily agreed "normal" range. These points are summed to provide an early warning score (EWS). Little is known about the accuracy with which EWS are calculated and charted. We compared the speed and accuracy of charting the weighted value attributed to each vital sign, and of calculating the EWS, using the traditional pen and paper method with that using a specially programmed, personal digital assistant (VitalPAC). Incorrect entries or omissions occurred in 24 (29%) of 84 EWS computed using pen/paper compared to 8 (10%) computed using the VitalPAC method. Fewer incorrect clinical actions were indicated using EWS derived via the VitalPAC method (4/84, 5%) than from those calculated using pen/paper (12/84, 14%). The mean time (+/-S.D.) taken for participants to calculate and chart a set of weighted values and EWS using the pen/paper method was 67.6+/-35.3 s (n=84). The corresponding time taken to enter a set of physiological data using the VitalPAC was 43.0+/-23.5 s (n=84). By comparison with the conventional pen/paper method, the use of VitalPAC was on average 1.6-times faster. The use of a device such as VitalPAC offers significant advantages both in speed and accuracy of recording of EWS.

Hospital-wide physiological surveillance-a new approach to the early identification and management of the sick patient.

Hospitalised patients, who suffer cardiac arrest and require unanticipated intensive care unit (ICU) admission or die, often exhibit premonitory abnormalities in vital signs. Sometimes, the deterioration is well documented, though there is little discernable evidence of intervention. In other cases, monitoring and recording of vital signs is infrequent or incomplete. Healthcare providers have introduced "track and trigger" systems to allow early identification of patients with physiological abnormalities, and rapid response teams to facilitate rapid and appropriate management. However, even when "track and trigger" systems are used, the recording of vital signs, patient chart completion and team activation remain sub-optimal. We have developed a system for collecting routine vital signs data at the bedside using standard personal digital assistants (PDA). The PDAs act as "thin clients" linked by a wireless local area network (W-LAN) to the hospital's intranet system, where raw and derived data are integrated with other patient information, e.g., name, hospital number, laboratory results. It is possible for raw physiology data, early warning scores (EWS), vital signs charts and oxygen therapy records to be made instantaneously available to any member of the hospital healthcare team via the W-LAN or hospital intranet. Early and direct contact with members of the patient's primary clinical team or rapid response team can be made through an automated alerting system, triggered by the EWS data. The ability to capture physiological data at the bedside, and to make these available to anyone with appropriate access rights at any time and in any place, should provide previously unattainable, clinical and administrative benefits. Analysis of the raw physiological data and patient outcomes will also make it possible to validate existing and future "track and trigger" systems.

Are observation selection methods important when comparing early warning score performance?

INTRODUCTION: Sicker patients generally have more vital sign assessments, particularly immediately before an adverse outcome, and especially if the vital sign monitoring schedule is driven by an early warning score (EWS) value. This lack of independence could influence the measured discriminatory performance of an EWS. METHODS: We used a population of 1564,143 consecutive vital signs observation sets collected as a routine part of patients' care. We compared 35 published EWSs for their discrimination of the risk of death within 24h of an observation set using (1) all observations in our dataset, (2) one observation per patient care episode, chosen at random and (3) one observation per patient care episode, chosen as the closest to a randomly selected point in time in each episode. We compared the area under the ROC curve (AUROC) as a measure of discrimination for each of the 35 EWSs under each observation selection method and looked for changes in their rank order. RESULTS: There were no significant changes in rank order of the EWSs based on AUROC between the different observation selection methods, except for one EWS that included age among its components. Whichever method of observation selection was used, the National Early Warning Score (NEWS) showed the highest discrimination of risk of death within 24h. AUROCs were higher when only one observation set was used per episode of care (significantly higher for many EWSs, including NEWS). CONCLUSIONS: Vital sign measurements can be treated as if they are independent - multiple observations can be used from each episode of care--when comparing the performance and ranking of EWSs, provided no EWS includes age.

Can binary early warning scores perform as well as standard early warning scores for discriminating a patient's risk of cardiac arrest, death or unanticipated intensive care unit admission?

INTRODUCTION: Although the weightings to be summed in an early warning score (EWS) calculation are small, calculation and other errors occur frequently, potentially impacting on hospital efficiency and patient care. Use of a simpler EWS has the potential to reduce errors. METHODS: We truncated 36 published 'standard' EWSs so that, for each component, only two scores were possible: 0 when the standard EWS scored 0 and 1 when the standard EWS scored greater than 0. Using 1564,153 vital signs observation sets from 68,576 patient care episodes, we compared the discrimination (measured using the area under the receiver operator characteristic curve--AUROC) of each standard EWS and its truncated 'binary' equivalent. RESULTS: The binary EWSs had lower AUROCs than the standard EWSs in most cases, although for some the difference was not significant. One system, the binary form of the National Early Warning System (NEWS), had significantly better discrimination than all standard EWSs, except for NEWS. Overall, Binary NEWS at a trigger value of 3 would detect as many adverse outcomes as are detected by NEWS using a trigger of 5, but would require a 15% higher triggering rate. CONCLUSIONS: The performance of Binary NEWS is only exceeded by that of standard NEWS. It may be that Binary NEWS, as a simplified system, can be used with fewer errors. However, its introduction could lead to significant increases in workload for ward and rapid response team staff. The balance between fewer errors and a potentially greater workload needs further investigation.

Aggregate National Early Warning Score (NEWS) values are more important than high scores for a single vital signs parameter for discriminating the risk of adverse outcomes.

INTRODUCTION: The Royal College of Physicians (RCPL) National Early Warning Score (NEWS) escalates care to a doctor at NEWS values of ≥5 and when the score for any single vital sign is 3. METHODS: We calculated the 24-h risk of serious clinical outcomes for vital signs observation sets with NEWS values of 3, 4 and 5, separately determining risks when the score did/did not include a single score of 3. We compared workloads generated by the RCPL's escalation protocol and for aggregate NEWS value alone. RESULTS: Aggregate NEWS values of 3 or 4 (n=142,282) formed 15.1% of all vital signs sets measured; those containing a single vital sign scoring 3 (n=36,207) constituted 3.8% of all sets. Aggregate NEWS values of either 3 or 4 with a component score of 3 have significantly lower risks (OR: 0.26 and 0.53) than an aggregate value of 5 (OR: 1.0). Escalating care to a doctor when any single component of NEWS scores 3 compared to when aggregate NEWS values ≥5, would have increased doctors' workload by 40% with only a small increase in detected adverse outcomes from 2.99 to 3.08 per day (a 3% improvement in detection). CONCLUSIONS: The recommended NEWS escalation protocol produces additional work for the bedside nurse and responding doctor, disproportionate to a modest benefit in increased detection of adverse outcomes. It may have significant ramifications for efficient staff resource allocation, distort patient safety focus and risk alarm fatigue. Our findings suggest that the RCPL escalation guidance warrants review.

Decision-tree early warning score (DTEWS) validates the design of the National Early Warning Score (NEWS).

AIM OF STUDY: To compare the performance of a human-generated, trial and error-optimised early warning score (EWS), i.e., National Early Warning Score (NEWS), with one generated entirely algorithmically using Decision Tree (DT) analysis. MATERIALS AND METHODS: We used DT analysis to construct a decision-tree EWS (DTEWS) from a database of 198,755 vital signs observation sets collected from 35,585 consecutive, completed acute medical admissions. We evaluated the ability of DTEWS to discriminate patients at risk of cardiac arrest, unanticipated intensive care unit admission or death, each within 24h of a given vital signs observation. We compared the performance of DTEWS and NEWS using the area under the receiver-operating characteristic (AUROC) curve. RESULTS: The structures of DTEWS and NEWS were very similar. The AUROC (95% CI) for DTEWS for cardiac arrest, unanticipated ICU admission, death, and any of the outcomes, all within 24h, were 0.708 (0.669-0.747), 0.862 (0.852-0.872), 0.899 (0.892-0.907), and 0.877 (0.870-0.883), respectively. Values for NEWS were 0.722 (0.685-0.759) [cardiac arrest], 0.857 (0.847-0.868) [unanticipated ICU admission}, 0.894 (0.887-0.902) [death], and 0.873 (0.866-0.879) [any outcome]. CONCLUSIONS: The decision-tree technique independently validates the composition and weightings of NEWS. The DT approach quickly provided an almost identical EWS to NEWS, although one that admittedly would benefit from fine-tuning using clinical knowledge. We believe that DT analysis could be used to quickly develop candidate models for disease-specific EWSs, which may be required in future.

The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.

INTRODUCTION: Early warning scores (EWS) are recommended as part of the early recognition and response to patient deterioration. The Royal College of Physicians recommends the use of a National Early Warning Score (NEWS) for the routine clinical assessment of all adult patients. METHODS: We tested the ability of NEWS to discriminate patients at risk of cardiac arrest, unanticipated intensive care unit (ICU) admission or death within 24h of a NEWS value and compared its performance to that of 33 other EWSs currently in use, using the area under the receiver-operating characteristic (AUROC) curve and a large vital signs database (n=198,755 observation sets) collected from 35,585 consecutive, completed acute medical admissions. RESULTS: The AUROCs (95% CI) for NEWS for cardiac arrest, unanticipated ICU admission, death, and any of the outcomes, all within 24h, were 0.722 (0.685-0.759), 0.857 (0.847-0.868), 0.894 (0.887-0.902), and 0.873 (0.866-0.879), respectively. Similarly, the ranges of AUROCs (95% CI) for the other 33 EWSs were 0.611 (0.568-0.654) to 0.710 (0.675-0.745) (cardiac arrest); 0.570 (0.553-0.568) to 0.827 (0.814-0.840) (unanticipated ICU admission); 0.813 (0.802-0.824) to 0.858 (0.849-0.867) (death); and 0.736 (0.727-0.745) to 0.834 (0.826-0.842) (any outcome). CONCLUSIONS: NEWS has a greater ability to discriminate patients at risk of the combined outcome of cardiac arrest, unanticipated ICU admission or death within 24h of a NEWS value than 33 other EWSs.

Patterns in the recording of vital signs and early warning scores: compliance with a clinical escalation protocol.

BACKGROUND: The recognition of patient deterioration depends largely on identifying abnormal vital signs, yet little is known about the daily pattern of vital signs measurement and charting. METHODS: We compared the pattern of vital signs and VitalPAC Early Warning Score (ViEWS) data collected from admissions to all adult inpatient areas (except high care areas, such as critical care units) of a NHS district general hospital from 1 May 2010 to 30 April 2011, to the hospital's clinical escalation protocol. Main outcome measures were hourly and daily patterns of vital signs and ViEWS value documentation; numbers of vital signs in the periods 08:00-11:59 and 20:00-23:59 with subsequent vital signs recorded in the following 6 h; and time to next observation (TTNO) for vital signs recorded in the periods 08:00-11:59 and 20:00-23:59. RESULTS: 950 043 vital sign datasets were recorded. The daily pattern of observation documentation was not uniform; there were large morning and evening peaks, and lower night-time documentation. The pattern was identical on all days. 23.84% of vital sign datasets with ViEWS ≥ 9 were measured at night compared with 10.12-19.97% for other ViEWS values. 47.42% of patients with ViEWS=7-8 and 31.22% of those with ViEWS ≥ 9 in the period 20:00-23:59 did not have vital signs recorded in the following 6 h. TTNO decreased with increasing ViEWS value, but less than expected by the monitoring protocol. CONCLUSIONS: There was only partial adherence to the vital signs monitoring protocol. Sicker patients appear more likely to have vital signs measured overnight, but even their observations were often not followed by timely repeat assessments. The observed pattern of monitoring may reflect the impact of competing clinical priorities.

Development and validation of a decision tree early warning score based on routine laboratory test results for the discrimination of hospital mortality in emergency medical admissions.

AIM OF STUDY: To build an early warning score (EWS) based exclusively on routinely undertaken laboratory tests that might provide early discrimination of in-hospital death and could be easily implemented on paper. MATERIALS AND METHODS: Using a database of combined haematology and biochemistry results for 86,472 discharged adult patients for whom the admission specialty was Medicine, we used decision tree (DT) analysis to generate a laboratory decision tree early warning score (LDT-EWS) for each gender. LDT-EWS was developed for a single set (n=3496) (Q1) and validated in 22 other discrete sets each of three months long (Q2, Q3…Q23) (total n=82,976; range of n=3428 to 4093) by testing its ability to discriminate in-hospital death using the area under the receiver-operating characteristic (AUROC) curve. RESULTS: The data generated slightly different models for male and female patients. The ranges of AUROC values (95% CI) for LDT-EWS with in-hospital death as the outcome for the validation sets Q2-Q23 were: 0.755 (0.727-0.783) (Q16) to 0.801 (0.776-0.826) [all patients combined, n=82,976]; 0.744 (0.704-0.784, Q16) to 0.824 (0.792-0.856, Q2) [39,591 males]; and 0.742 (0.707-0.777, Q10) to 0.826 (0.796-0.856, Q12) [43,385 females]. CONCLUSIONS: This study provides evidence that the results of commonly measured laboratory tests collected soon after hospital admission can be represented in a simple, paper-based EWS (LDT-EWS) to discriminate in-hospital mortality. We hypothesise that, with appropriate modification, it might be possible to extend the use of LDT-EWS throughout the patient's hospital stay.

S(p)O(2) values in acute medical admissions breathing air--implications for the British Thoracic Society guideline for emergency oxygen use in adult patients?

S(p)O(2) is routinely used to assess the well-being of patients, but it is difficult to find an evidence-based description of its normal range. The British Thoracic Society (BTS) has published guidance for oxygen administration and recommends a target S(p)O(2) of 94-98% for most adult patients. These recommendations rely on consensus opinion and small studies using arterial blood gas measurements of saturation (S(a)O(2)). Using large datasets of routinely collected vital signs from four hospitals, we analysed the S(p)O(2) range of 37,593 acute general medical inpatients (males: 47%) observed to be breathing room air. Age at admission ranged from 16 to 105 years with a mean (SD) of 64 (21) years. 19,642 admissions (52%) were aged <70 years. S(p)O(2) ranged from 70% to 100% with a median (IQR) of 97% (95-98%). S(p)O(2) values for males and females were similar. In-hospital mortality for the study patients was 5.27% (range 4.80-6.27%). Mortality (95% CI) for patients with initial S(p)O(2) values of 97%, 96% and 95% was 3.65% (3.22-4.13); 4.47% (3.99-5.00); and 5.67% (5.03-6.38), respectively. Additional analyses of S(p)O(2) values for 37,299 medical admissions aged ≥18 years provided results that were distinctly different to those upon which the current BTS guidelines based their definition of normality. Our findings suggest that the BTS should consider changing its target saturation for actively treated patients not at risk of hypercapnic respiratory failure to 96-98%.

ViEWS--Towards a national early warning score for detecting adult inpatient deterioration.

AIM OF STUDY: To develop a validated, paper-based, aggregate weighted track and trigger system (AWTTS) that could serve as a template for a national early warning score (EWS) for the detection of patient deterioration. MATERIALS AND METHODS: Using existing knowledge of the relationship between physiological data and adverse clinical outcomes, a thorough review of the literature surrounding EWS and physiology, and a previous detailed analysis of published EWSs, we developed a new paper-based EWS - VitalPAC EWS (ViEWS). We applied ViEWS to a large vital signs database (n=198,755 observation sets) collected from 35,585 consecutive, completed acute medical admissions, and also evaluated the comparative performance of 33 other AWTTSs, for a range of outcomes using the area under the receiver-operating characteristics (AUROC) curve. RESULTS: The AUROC (95% CI) for ViEWS using in-hospital mortality with 24h of the observation set was 0.888 (0.880-0.895). The AUROCs (95% CI) for the 33 other AWTTSs tested using the same outcome ranged from 0.803 (0.792-0.815) to 0.850 (0.841-0.859). ViEWS performed better than the 33 other AWTTSs for all outcomes tested. CONCLUSIONS: We have developed a simple AWTTS - ViEWS - designed for paper-based application and demonstrated that its performance for predicting mortality (within a range of timescales) is superior to all other published AWTTSs that we tested. We have also developed a tool to provide a relative measure of the number of "triggers" that would be generated at different values of EWS and permits the comparison of the workload generated by different AWTTSs.