Effect of Digital Early Warning Scores on Hospital Vital Sign Observation Protocol Adherence: Stepped-Wedge Evaluation.

BACKGROUND: Early warning scores (EWS) are routinely used in hospitals to assess a patient's risk of deterioration. EWS are traditionally recorded on paper observation charts but are increasingly recorded digitally. In either case, evidence for the clinical effectiveness of such scores is mixed, and previous studies have not considered whether EWS leads to changes in how deteriorating patients are managed. OBJECTIVE: This study aims to examine whether the introduction of a digital EWS system was associated with more frequent observation of patients with abnormal vital signs, a precursor to earlier clinical intervention. METHODS: We conducted a 2-armed stepped-wedge study from February 2015 to December 2016, over 4 hospitals in 1 UK hospital trust. In the control arm, vital signs were recorded using paper observation charts. In the intervention arm, a digital EWS system was used. The primary outcome measure was time to next observation (TTNO), defined as the time between a patient's first elevated EWS (EWS ≥3) and subsequent observations set. Secondary outcomes were time to death in the hospital, length of stay, and time to unplanned intensive care unit admission. Differences between the 2 arms were analyzed using a mixed-effects Cox model. The usability of the system was assessed using the system usability score survey. RESULTS: We included 12,802 admissions, 1084 in the paper (control) arm and 11,718 in the digital EWS (intervention) arm. The system usability score was 77.6, indicating good usability. The median TTNO in the control and intervention arms were 128 (IQR 73-218) minutes and 131 (IQR 73-223) minutes, respectively. The corresponding hazard ratio for TTNO was 0.99 (95% CI 0.91-1.07; P=.73). CONCLUSIONS: We demonstrated strong clinical engagement with the system. We found no difference in any of the predefined patient outcomes, suggesting that the introduction of a highly usable electronic system can be achieved without impacting clinical care. Our findings contrast with previous claims that digital EWS systems are associated with improvement in clinical outcomes. Future research should investigate how digital EWS systems can be integrated with new clinical pathways adjusting staff behaviors to improve patient outcomes.

Sex and age effects on risk of non-traumatic subarachnoid hemorrhage: Retrospective cohort study of 124,234 cases using electronic health records.

OBJECTIVES: The epidemiology of non-traumatic subarachnoid hemorrhage (SAH) is unclear. This study describes the antecedent characteristics of SAH patients, compares the risk of SAH between women and men, and explores if this changes with age. MATERIALS AND METHODS: Retrospective cohort study using an electronic health records network based in the USA (TriNetX). All patients aged 18-90y with at least one healthcare visit were included. Antecedent characteristics of SAH patients (ICD-10 code I60) were measured. The incidence proportion and the relative risk between women and men, were estimated overall, in the 55-90y age group, and in five-year age categories. RESULTS: Of 58.9 million eligible patients, with 190.8 million person-years of observations, 124,234 (0.21%; 63,467 female, 60,671 male) had a first SAH, with a mean age of 56.8 (S.D. 16.8) y (women: 58.2 [16.2] y, men 55.3 [17.2] y). 9,758 SAH cases (7.8%) occurred in people aged 18-30y. Prior to the SAH, an intracranial aneurysm had been diagnosed in 4.1% (women: 5.8% men: 2.5%), hypertension in 25.1% and nicotine dependence in 9.1%. Overall, women had a lower risk of SAH compared to men (RR 0.83, 95% CI 0.83-0.84), with a progressive increase in risk ratio across age groups: from RR 0.36 (0.35-0.37) in people aged 18-24y, to RR 1.07 (1.01-1.13) aged 85-90y. CONCLUSIONS: Men are at greater risk of SAH than women overall, driven by younger adult age groups. Women are at greater risk than men only in the over 75-year age groups. The excess of SAH in young men merits investigation.

Detecting Deteriorating Patients in the Hospital: Development and Validation of a Novel Scoring System.

Rationale: Late recognition of patient deterioration in hospital is associated with worse outcomes, including higher mortality. Despite the widespread introduction of early warning score (EWS) systems and electronic health records, deterioration still goes unrecognized. Objectives: To develop and externally validate a Hospital- wide Alerting via Electronic Noticeboard (HAVEN) system to identify hospitalized patients at risk of reversible deterioration. Methods: This was a retrospective cohort study of patients 16 years of age or above admitted to four UK hospitals. The primary outcome was cardiac arrest or unplanned admission to the ICU. We used patient data (vital signs, laboratory tests, comorbidities, and frailty) from one hospital to train a machine-learning model (gradient boosting trees). We internally and externally validated the model and compared its performance with existing scoring systems (including the National EWS, laboratory-based acute physiology score, and electronic cardiac arrest risk triage score). Measurements and Main Results: We developed the HAVEN model using 230,415 patient admissions to a single hospital. We validated HAVEN on 266,295 admissions to four hospitals. HAVEN showed substantially higher discrimination (c-statistic, 0.901 [95% confidence interval, 0.898-0.903]) for the primary outcome within 24 hours of each measurement than other published scoring systems (which range from 0.700 [0.696-0.704] to 0.863 [0.860-0.865]). With a precision of 10%, HAVEN was able to identify 42% of cardiac arrests or unplanned ICU admissions with a lead time of up to 48 hours in advance, compared with 22% by the next best system. Conclusions: The HAVEN machine-learning algorithm for early identification of in-hospital deterioration significantly outperforms other published scores such as the National EWS.

Descriptors of Sepsis Using the Sepsis-3 Criteria: A Cohort Study in Critical Care Units Within the U.K. National Institute for Health Research Critical Care Health Informatics Collaborative.

OBJECTIVES: To describe the epidemiology of sepsis in critical care by applying the Sepsis-3 criteria to electronic health records. DESIGN: Retrospective cohort study using electronic health records. SETTING: Ten ICUs from four U.K. National Health Service hospital trusts contributing to the National Institute for Health Research Critical Care Health Informatics Collaborative. PATIENTS: A total of 28,456 critical care admissions (14,332 emergency medical, 4,585 emergency surgical, and 9,539 elective surgical). MEASUREMENTS AND MAIN RESULTS: Twenty-nine thousand three hundred forty-three episodes of clinical deterioration were identified with a rise in Sequential Organ Failure Assessment score of at least 2 points, of which 14,869 (50.7%) were associated with antibiotic escalation and thereby met the Sepsis-3 criteria for sepsis. A total of 4,100 episodes of sepsis (27.6%) were associated with vasopressor use and lactate greater than 2.0 mmol/L, and therefore met the Sepsis-3 criteria for septic shock. ICU mortality by source of sepsis was highest for ICU-acquired sepsis (23.7%; 95% CI, 21.9-25.6%), followed by hospital-acquired sepsis (18.6%; 95% CI, 17.5-19.9%), and community-acquired sepsis (12.9%; 95% CI, 12.1-13.6%) (p for comparison less than 0.0001). CONCLUSIONS: We successfully operationalized the Sepsis-3 criteria to an electronic health record dataset to describe the characteristics of critical care patients with sepsis. This may facilitate sepsis research using electronic health record data at scale without relying on human coding.

Treatment strategies for new onset atrial fibrillation in patients treated on an intensive care unit: a systematic scoping review.

BACKGROUND: New-onset atrial fibrillation (NOAF) in patients treated on an intensive care unit (ICU) is common and associated with significant morbidity and mortality. We undertook a systematic scoping review to summarise comparative evidence to inform NOAF management for patients admitted to ICU. METHODS: We searched MEDLINE, EMBASE, CINAHL, Web of Science, OpenGrey, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects, ISRCTN, ClinicalTrials.gov, EU Clinical Trials register, additional WHO ICTRP trial databases, and NIHR Clinical Trials Gateway in March 2019. We included studies evaluating treatment or prevention strategies for NOAF or acute anticoagulation in general medical, surgical or mixed adult ICUs. We extracted study details, population characteristics, intervention and comparator(s), methods addressing confounding, results, and recommendations for future research onto study-specific forms. RESULTS: Of 3,651 citations, 42 articles were eligible: 25 primary studies, 12 review articles and 5 surveys/opinion papers. Definitions of NOAF varied between NOAF lasting 30 s to NOAF lasting > 24 h. Only one comparative study investigated effects of anticoagulation. Evidence from small RCTs suggests calcium channel blockers (CCBs) result in slower rhythm control than beta blockers (1 study), and more cardiovascular instability than amiodarone (1 study). Evidence from 4 non-randomised studies suggests beta blocker and amiodarone therapy may be equivalent in respect to rhythm control. Beta blockers may be associated with improved survival compared to amiodarone, CCBs, and digoxin, though supporting evidence is subject to confounding. Currently, the limited evidence does not support therapeutic anticoagulation during ICU admission. CONCLUSIONS: From the limited evidence available beta blockers or amiodarone may be superior to CCBs as first line therapy in undifferentiated patients in ICU. The little evidence available does not support therapeutic anticoagulation for NOAF whilst patients are critically ill. Consensus definitions for NOAF, rate and rhythm control are needed.

Anxiety, depression and post-traumatic stress disorder management after critical illness: a UK multi-centre prospective cohort study.

BACKGROUND: Survivors of critical illness have significant psychopathological comorbidity. The treatments offered by primary health care professionals to affected patients are unstudied. AIM: To report the psychological interventions after GPs received notification of patients who showed severe symptoms of anxiety, depression or Post-Traumatic Stress Disorder. METHODS: Design: Multi-centre prospective cohort sub-study of the ICON study. SETTING: NHS primary care in the United Kingdom. PARTICIPANTS: Adult patients, November 2006-October 2010 who had received at least 24 h of intensive care, where the general practitioner recorded notification that the patient had reported severe symptoms or caseness using the Hospital Anxiety and Depression Scale (HADS) or the Post-Traumatic Stress Disorder Check List-Civilian (PCL-C). INTERVENTIONS: We notified general practitioners (GPs) by post if a patient reported severe symptoms or caseness and sent a postal questionnaire to determine interventions after notification. MAIN OUTCOME MEASURE: Primary or secondary healthcare interventions instigated by general practitioners following notification of a patient's caseness. RESULTS: Of the 11,726 patients, sent questionnaire packs containing HADS and PCL-C, 4361 (37%) responded. A notification of severe symptoms was sent to their GP in 25% (1112) of cases. Of notified GPs, 65% (725) responded to our postal questionnaire. Of these 37% (266) had no record of receipt of the original notification. Of the 459 patients where GPs had record of notification (the study group for this analysis), 21% (98) had pre-existing psychopathology. Of those without a pre-existing diagnosis 45% (162) received further psychological assessment or treatment. GP screening or follow-up alone occurred in 18% (64) whilst 27% (98) were referred to mental health services or received drug therapy following notification. CONCLUSIONS: Postal questionnaire identifies a burden of psychopathology in survivors of critical illness that have otherwise gone undiagnosed following discharge from an intensive care unit (ICU). After being alerted to the presence of psychological symptoms, GPs instigate treatment in 27% and augmented surveillance in 18% of cases. TRIAL REGISTRATION: ISRCTN69112866 (assigned 02/05/2006).

Implementing a system for the real-time risk assessment of patients considered for intensive care.

BACKGROUND: Delay in identifying deterioration in hospitalised patients is associated with delayed admission to an intensive care unit (ICU) and poor outcomes. For the HAVEN project (HICF ref.: HICF-R9-524), we have developed a mathematical model that identifies deterioration in hospitalised patients in real time and facilitates the intervention of an ICU outreach team. This paper describes the system that has been designed to implement the model. We have used innovative technologies such as Portable Format for Analytics (PFA) and Open Services Gateway initiative (OSGi) to define the predictive statistical model and implement the system respectively for greater configurability, reliability, and availability. RESULTS: The HAVEN system has been deployed as part of a research project in the Oxford University Hospitals NHS Foundation Trust. The system has so far processed > 164,000 vital signs observations and > 68,000 laboratory results for > 12,500 patients and the algorithm generated score is being evaluated to review patients who are under consideration for transfer to ICU. No clinical decisions are being made based on output from the system. The HAVEN score has been computed using a PFA model for all these patients. The intent is that this score will be displayed on a graphical user interface for clinician review and response. CONCLUSIONS: The system uses a configurable PFA model to compute the HAVEN score which makes the system easily upgradable in terms of enhancing systems' predictive capability. Further system enhancements are planned to handle new data sources and additional management screens.

Using real-time location devices (RTLD) to quantify off-unit adult intensive care registrar workload: a 1-year tertiary NHS hospital prospective observational study.

UK national guidelines state deteriorating or at risk hospital ward patients should receive care from trained critical care outreach personnel. In most tertiary hospitals this involves a team led by an Intensive Care Unit (ICU) registrar. The ICU registrar must also review patients referred for possible ICU admission. These two responsibilities require work away from the ICU. To our knowledge the burden of this work has not been described, despite its importance in ICU workforce management and patient safety. A 12-month, prospective, observational study was carried out. The primary outcome measure was ICU registrar time spent on and off-unit. The study participants were senior and junior registrars on the rota of the 16 bed, Adult Intensive Care Unit at the John Radcliffe Hospital in Oxford. To measure their work patterns, this study used AeroScout 'T2' Real Time Location Device (RTLD) tags (Stanley Healthcare, Swindon). In our hospital, senior and junior ICU registrars spend roughly one-fifth of their time off-unit, half of which is spent in ED. This workload combines to leave the unit unattended at night up to 10% of the time. RTLDs provide a reliable, automated method for quantifying ICU registrar off-unit work patterns. This method may be adopted for quantifying other clinical staff work patterns in suitably equipped hospital environments.

Trends of blood pressure and heart rate in normal pregnancies: a systematic review and meta-analysis.

BACKGROUND: Current reference ranges for blood pressure and heart rate throughout pregnancy have a poor evidence base. METHODS: This is a systematic review and meta-analysis. We included studies measuring blood pressure or heart rate from healthy pregnant women within defined gestational periods of 16 weeks or less. We analysed systolic blood pressure, diastolic blood pressure and heart rate by gestational age. We assessed effects of measurement year and method. RESULTS: We included 39 studies undertaken in 1967-2017, containing 124,349 systolic measurements from 36,239 women, 124,291 diastolic measurements from 36,181 women and 10,948 heart rate measurements from 8317 women. Mean (95% CI) systolic blood pressure was lowest at 10 weeks gestation, 110.4 (108.5, 112.3) mmHg, rising to 116.0 (113.6, 118.4) mmHg at 40 weeks, mean (95% CI) change 5.6 (4.0, 7.2) mmHg. Mean (95% CI) diastolic blood pressure was lowest at 21 weeks gestation, 65.9 (64.2, 67.7) mmHg; rising to 72.8 (71.0, 74.6) mmHg at 40 weeks, mean (95% CI) change 6.9 (6.2, 7.5) mmHg. Mean (95% CI) heart rate rose from 79.3 (75.5, 83.1) beats/min at 10 weeks to 86.9 (82.2, 91.6) beats/min at 40 weeks gestation, mean (95% CI) change 7.6 (1.8, 13.4) beats/min. Studies using manual measurement reported higher diastolic blood pressures than studies using automated measurement, mean (95 CI) difference 4.9 (0.8, 8.9) mmHg. Diastolic blood pressure increased by 0.26 (95% CI 0.10-0.43) mmHg/year. Including only higher-quality studies had little effect on findings, with heterogeneity remaining high (I2 statistic > 50%). CONCLUSIONS: Significant gestational blood pressure and heart rate changes occur that should be taken into account when assessing pregnant women. Commonly taught substantial decreases in blood pressure mid-pregnancy were not seen and heart rate increases were lower than previously thought. Manual and automated blood pressure measurement cannot be used interchangeably. Increases in diastolic blood pressure over the last half-century and differences between published studies show contemporary data are required to define current normal ranges. STUDY REGISTRATION: PROSPERO CRD42014009673.

Patient centred variables with univariate associations with unplanned ICU admission: a systematic review.

BACKGROUND: Multiple predictive scores using Electronic Patient Record data have been developed for hospitalised patients at risk of clinical deterioration. Methods used to select patient centred variables for inclusion in these scores varies. We performed a systematic review to describe univariate associations with unplanned Intensive Care Unit (ICU) admission with the aim of assisting model development for future scores that predict clinical deterioration. METHODS: Data sources were MEDLINE, EMBASE, CINAHL, CENTRAL and the Cochrane Database of Systematic Reviews. Included studies were published since 2000 describing an association between patient centred variables and unplanned ICU admission determined using univariate analysis. Two authors independently screened titles, abstracts and full texts against inclusion and exclusion criteria. DistillerSR (Evidence Partners, Canada, Ottawa, Ontario) software was used to manage the data and identify duplicate search results. All screening and data extraction forms were implemented within DistillerSR. Study quality was assessed using an adapted version of the Newcastle-Ottawa Scale. Variables were analysed for strength of association with unplanned ICU admission. RESULTS: The database search yielded 1520 unique studies; 1462 were removed after title and abstract review; 57 underwent full text screening; 16 studies were included. One hundred and eighty nine variables with an evaluated univariate association with unplanned ICU admission were described. DISCUSSION: Being male, increasing age, a history of congestive cardiac failure or diabetes, a diagnosis of hepatic disease or having abnormal vital signs were all strongly associated with ICU admission. CONCLUSION: These findings will assist variable selection during the development of future models predicting unplanned ICU admission. TRIAL REGISTRATION: This study is a component of a larger body of work registered in the ISRCTN registry ( ISRCTN12518261 ).

Shoulder Impairment Following Critical Illness: A Prospective Cohort Study.

OBJECTIVES: Identify the prevalence of shoulder impairment in ICU survivors within 6 months of discharge from ICU. Evaluate the impact of shoulder impairment on upper limb functional status in patients treated on an ICU. Identify risk factors for the development of shoulder impairment. DESIGN: Prospective cohort study. SETTING: A tertiary care medical-surgical-trauma ICU at a U.K. hospital over 18 months, with a further 6-month follow-up after hospital discharge. SUBJECTS: Adult patients with an ICU length of stay of greater than 72 hours with no preexisting or new neurologic or traumatic upper limb injury. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Patients underwent targeted shoulder assessments (pain, range of movement, Constant-Murley Score, shortened version of the disabilities of the arm, shoulder, and hand [DASH] score [QuickDASH] score) at hospital discharge, 3 and 6 months after hospital discharge. Assessments were undertaken on 96 patients, with 62 patients attending follow-up at 3 months and 61 patients at 6 months. Multivariate regression analysis was used to investigate risk factors for shoulder impairment. ICU-related shoulder impairment was present in 67% of patients at 6 months following discharge from hospital. Upper limb dysfunction occurred in 46%, with 16% having severe dysfunction (equivalent to shoulder dislocation). We were unable to identify specific risk factors for shoulder impairment. CONCLUSIONS: Shoulder impairment is a highly prevalent potential source of disability in ICU survivors. This persists at 6 months after discharge with a significant impact on upper limb function. More research is needed into potential mechanisms underlying shoulder impairment and potential targeted interventions to reduce the prevalence.

Smoothing Effect in Vital Sign Recordings: Fact or Fiction? A Retrospective Cohort Analysis of Manual and Continuous Vital Sign Measurements to Assess Data Smoothing in Postoperative Care.

BACKGROUND: Data smoothing of vital signs has been reported in the anesthesia literature, suggesting that clinical staff are biased toward measurements of normal physiology. However, these findings may be partially explained by clinicians interpolating spurious values from noisy signals and by the undersampling of physiological changes by infrequent manual observations. We explored the phenomenon of data smoothing using a method robust to these effects in a large postoperative dataset including respiratory rate, heart rate, and oxygen saturation (SpO2). We also assessed whether the presence of the vital sign taker creates an arousal effect. METHODS: Study data came from a UK upper gastrointestinal postoperative ward (May 2009 to December 2013). We compared manually recorded vital sign data with contemporaneous continuous data recorded from monitoring equipment. We proposed that data smoothing increases differences between vital sign sources as vital sign abnormality increases. The primary assessment method was a mixed-effects model relating continuous-manual differences to vital sign values, adjusting for repeated measurements. We tested the regression slope significance and predicted the continuous-manual difference at clinically important vital sign values. We calculated limits of agreement (LoA) between vital sign sources using the Bland-Altman method, adjusting for repeated measures. Similarly, we assessed whether the vital sign taker affected vital signs, comparing continuous data before and during manual recording. RESULTS: From 407 study patients, 271 had contemporaneous continuous and manual recordings, allowing 3740 respiratory rate, 3844 heart rate, and 3896 SpO2 paired measurements for analysis. For the model relating continuous-manual differences to continuous-manual average vital sign values, the regression slope (95% confidence interval) was 0.04 (-0.01 to 0.10; P = .11) for respiratory rate, 0.04 (-0.01 to 0.09; P = .11) for heart rate, and 0.10 (0.07-0.14; P < .001) for SpO2. For SpO2 measurements of 91%, the model predicted a continuous-manual difference (95% confidence interval) of -0.88% (-1.17% to -0.60%). The bias (LoA) between measurement sources was -0.74 (-7.80 to 6.32) breaths/min for respiratory rate, -1.13 (-17.4 to 15.1) beats/min for heart rate, and -0.25% (-3.35% to 2.84%) for SpO2. The bias (LoA) between continuous data before and during manual observation was -0.57 (-5.63 to 4.48) breaths/min for respiratory rate, -0.71 (-10.2 to 8.73) beats/min for heart rate, and -0.07% (-2.67% to 2.54%) for SpO2. CONCLUSIONS: We found no evidence of data smoothing for heart rate and respiratory rate measurements. Although an effect exists for SpO2 measurements, it was not clinically significant. The wide LoAs between continuous and manually recorded vital signs would commonly result in different early warning scores, impacting clinical care. There was no evidence of an arousal effect caused by the vital sign taker.

Wearable monitors for patients following discharge from an intensive care unit: practical lessons learnt from an observational study.

AIMS: To identify the practical challenges encountered when using wearable monitors for patients discharged from the intensive care unit. BACKGROUND: Patients discharged from intensive care units are a high-risk group that might benefit from continuing observation using 'wearable' monitors to enable faster identification of physiological deterioration and facilitate timely clinical action. This area of technological innovation is of key interest to nurses who manage this group of patients. DESIGN: A prospective observational study. METHODS: An observational study conducted in 2013-2014 used wearable monitors to record continuous observations for patients discharged from an intensive care unit to develop a predictive model of patients likely to deteriorate. Screening data for study eligibility and case report form data to assess monitor tolerance and comfort were collected daily and analysed using Microsoft Access. RESULTS/FINDINGS: Patients (n = 2704) were discharged from an intensive care unit during the study, 208 consented to wearing the monitor. Of the 192 included in analysis, 130 (67·7%) removed the monitor before the trial finished. Reasons cited for removal included 'discomfort and irritation' 61 (31·8%) and 'feeling too unwell' 8 (4·2%). Five hundred seventeen patients were screened following adaption of the wearable monitor. Despite design changes, 56 (10·8%) patients were unable to wear monitors for reasons related to their anatomy or condition. Of 124 patients, 65 patients (52·4%) who were approached refused participation. CONCLUSION: Work is needed to understand wireless monitor comfort and design for acutely unwell patients. Product design needs to develop further, so patients are catered for in flexibility of monitor placement and improved comfort for long-term wear.

Development of a national maternity early warning score: centile based score development and Delphi informed escalation pathways.

Objective: To derive a new maternity early warning score (MEWS) from prospectively collected data on maternity vital signs and to design clinical response pathways with a Delphi consensus exercise. Design: Centile based score development and Delphi informed escalation pathways. Setting: Pregnancy Physiology Pattern Prediction (4P) prospective UK cohort study, 1 August 2012 to 28 December 2016. Participants: Pregnant people from the 4P study, recruited before 20 weeks' gestation at three UK maternity centres (Oxford, Newcastle, and London). 841, 998, and 889 women provided data in the early antenatal, antenatal, and postnatal periods. Main outcome measures: Development of a new national MEWS, assigning numerical weights to measurements in the lower and upper extremes of distributions of individual vital signs from the 4P prospective cohort study. Comparison of escalation rates of the new national MEWS with the Scottish and Irish MEWS systems from 18 to 40 weeks' gestation. Delphi consensus exercise to agree clinical responses to raised scores. Results: A new national MEWS was developed by assigning numerical weights to measurements in the lower and upper extremes (5%, 1%) of distributions of vital signs, except for oxygen saturation where lower centiles (10%, 2%) were used. For the new national MEWS, in a healthy population, 56% of observation sets resulted in a total score of 0 points, 26% a score of 1 point, 12% a score of 2 points, and 18% a score of ≥2 points (escalation of care is triggered at a total score of ≥2 points). Corresponding values for the Irish MEWS were 37%, 25%, 22%, and 38%, respectively; and for the Scottish MEWS, 50%, 18%, 21%, and 32%, respectively. All three MEWS were similar at the beginning of pregnancy, averaging 0.7-0.9 points. The new national MEWS had a lower mean score for the rest of pregnancy, with the mean score broadly constant (0.6-0.8 points). The new national MEWS had an even distribution of healthy population alerts across the antenatal period. In the postnatal period, heart rate threshold values were adjusted to align with postnatal changes. The centile based score derivation approach meant that each vital sign component in the new national MEWS had a similar alert rate. Suggested clinical responses to different MEWS values were agreed by consensus of an independent expert panel. Conclusions: The centile based MEWS alerted escalation of care evenly across the antenatal period in a healthy population, while reducing alerts in healthy women compared with other MEWS systems. How well the tool predicted adverse outcomes, however, was not assessed and therefore external validation studies in large datasets are needed. Unlike other MEWS systems, the new national MEWS was developed with prospectively collected data on vital signs and used a systematic, expert informed process to design an associated escalation protocol.

Development of an enhanced scoring system to predict ICU readmission or in-hospital death within 24 hours using routine patient data from two NHS Foundation Trusts.

RATIONALE: Intensive care units (ICUs) admit the most severely ill patients. Once these patients are discharged from the ICU to a step-down ward, they continue to have their vital signs monitored by nursing staff, with Early Warning Score (EWS) systems being used to identify those at risk of deterioration. OBJECTIVES: We report the development and validation of an enhanced continuous scoring system for predicting adverse events, which combines vital signs measured routinely on acute care wards (as used by most EWS systems) with a risk score of a future adverse event calculated on discharge from the ICU. DESIGN: A modified Delphi process identified candidate variables commonly available in electronic records as the basis for a 'static' score of the patient's condition immediately after discharge from the ICU. L1-regularised logistic regression was used to estimate the in-hospital risk of future adverse event. We then constructed a model of physiological normality using vital sign data from the day of hospital discharge. This is combined with the static score and used continuously to quantify and update the patient's risk of deterioration throughout their hospital stay. SETTING: Data from two National Health Service Foundation Trusts (UK) were used to develop and (externally) validate the model. PARTICIPANTS: A total of 12 394 vital sign measurements were acquired from 273 patients after ICU discharge for the development set, and 4831 from 136 patients in the validation cohort. RESULTS: Outcome validation of our model yielded an area under the receiver operating characteristic curve of 0.724 for predicting ICU readmission or in-hospital death within 24 hours. It showed an improved performance with respect to other competitive risk scoring systems, including the National EWS (0.653). CONCLUSIONS: We showed that a scoring system incorporating data from a patient's stay in the ICU has better performance than commonly used EWS systems based on vital signs alone. TRIAL REGISTRATION NUMBER: ISRCTN32008295.

Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach.

OBJECTIVE: To develop and validate a questionnaire to assess the usability of clinical decision support systems (CDSS) and to assist in the early identification of usability issues that may impact patient safety and quality of care. DESIGN: Mixed research methods were used to develop and validate the questionnaire. The qualitative study involved scale item development, content and face validity. Pilot testing established construct validity using factor analysis and facilitated estimates for reliability and internal consistency using the Cronbach's alpha coefficient. SETTING: Two hospitals within a single National Health Service Trust. PARTICIPANTS: We recruited a panel of 7 experts in usability and questionnaire writing for health purposes to test content validity; 10 participants to assess face validity and 78 participants for the pilot testing. To be eligible for this last phase, participants needed to be health professionals with at least 3 months experience using the local hospital electronic patient record system. RESULTS: Feedback from the face and content validity phases contributed to the development and improvement of scale items. The final Healthcare Systems Usability Scale (HSUS) proved quick to complete, easy to understand and was mostly worded by potential users. Exploratory analysis revealed four factors related to patient safety, task execution, alerts or recommendations accuracy, the effects of the system on workflow and ease of system use. These separate into four subscales: patient safety and decision effectiveness (seven items), workflow integration (six items), work effectiveness (five items) and user control (four items). These factors affect the quality of care and clinician's ability to make informed and timely decisions when using CDSS. The HSUS has a very good reliability with global Cronbach's alpha 0.914 and between 0.702 and 0.926 for the four subscales. CONCLUSION: The HSUS is a valid and reliable tool for usability testing of CDSS and early identification of usability issues that may cause medical adverse events.