Evaluation and improvement of the National Early Warning Score (NEWS2) for COVID-19: a multi-hospital study.

BACKGROUND: The National Early Warning Score (NEWS2) is currently recommended in the UK for the risk stratification of COVID-19 patients, but little is known about its ability to detect severe cases. We aimed to evaluate NEWS2 for the prediction of severe COVID-19 outcome and identify and validate a set of blood and physiological parameters routinely collected at hospital admission to improve upon the use of NEWS2 alone for medium-term risk stratification. METHODS: Training cohorts comprised 1276 patients admitted to King's College Hospital National Health Service (NHS) Foundation Trust with COVID-19 disease from 1 March to 30 April 2020. External validation cohorts included 6237 patients from five UK NHS Trusts (Guy's and St Thomas' Hospitals, University Hospitals Southampton, University Hospitals Bristol and Weston NHS Foundation Trust, University College London Hospitals, University Hospitals Birmingham), one hospital in Norway (Oslo University Hospital), and two hospitals in Wuhan, China (Wuhan Sixth Hospital and Taikang Tongji Hospital). The outcome was severe COVID-19 disease (transfer to intensive care unit (ICU) or death) at 14 days after hospital admission. Age, physiological measures, blood biomarkers, sex, ethnicity, and comorbidities (hypertension, diabetes, cardiovascular, respiratory and kidney diseases) measured at hospital admission were considered in the models. RESULTS: A baseline model of 'NEWS2 + age' had poor-to-moderate discrimination for severe COVID-19 infection at 14 days (area under receiver operating characteristic curve (AUC) in training cohort = 0.700, 95% confidence interval (CI) 0.680, 0.722; Brier score = 0.192, 95% CI 0.186, 0.197). A supplemented model adding eight routinely collected blood and physiological parameters (supplemental oxygen flow rate, urea, age, oxygen saturation, C-reactive protein, estimated glomerular filtration rate, neutrophil count, neutrophil/lymphocyte ratio) improved discrimination (AUC = 0.735; 95% CI 0.715, 0.757), and these improvements were replicated across seven UK and non-UK sites. However, there was evidence of miscalibration with the model tending to underestimate risks in most sites. CONCLUSIONS: NEWS2 score had poor-to-moderate discrimination for medium-term COVID-19 outcome which raises questions about its use as a screening tool at hospital admission. Risk stratification was improved by including readily available blood and physiological parameters measured at hospital admission, but there was evidence of miscalibration in external sites. This highlights the need for a better understanding of the use of early warning scores for COVID.

Impact of a computerized decision support tool deployed in two intensive care units on acute kidney injury progression and guideline compliance: a prospective observational study.

BACKGROUND: Acute kidney injury (AKI) affects a large proportion of the critically ill and is associated with worse patient outcomes. Early identification of AKI can lead to earlier initiation of supportive therapy and better management. In this study, we evaluate the impact of computerized AKI decision support tool integrated with the critical care clinical information system (CCIS) on patient outcomes. Specifically, we hypothesize that integration of AKI guidelines into CCIS will decrease the proportion of patients with Stage 1 AKI deteriorating into higher stages of AKI. METHODS: The study was conducted in two intensive care units (ICUs) at University Hospitals Bristol, UK, in a before (control) and after (intervention) format. The intervention consisted of the AKIN guidelines and AKI care bundle which included guidance for medication usage, AKI advisory and dashboard with AKI score. Clinical data and patient outcomes were collected from all patients admitted to the units. AKI stage was calculated using the Acute Kidney Injury Network (AKIN) guidelines. Maximum AKI stage per admission, change in AKI stage and other metrics were calculated for the cohort. Adherence to eGFR-based enoxaparin dosing guidelines was evaluated as a proxy for clinician awareness of AKI. RESULTS: Each phase of the study lasted a year, and a total of 5044 admissions were included for analysis with equal numbers of patients for the control and intervention stages. The proportion of patients worsening from Stage 1 AKI decreased from 42% (control) to 33.5% (intervention), p = 0.002. The proportion of incorrect enoxaparin doses decreased from 1.72% (control) to 0.6% (intervention), p < 0.001. The prevalence of any AKI decreased from 43.1% (control) to 37.5% (intervention), p < 0.05. CONCLUSIONS: This observational study demonstrated a significant reduction in AKI progression from Stage 1 and a reduction in overall development of AKI. In addition, a reduction in incorrect enoxaparin dosing was also observed, indicating increased clinical awareness. This study demonstrates that AKI guidelines coupled with a newly designed AKI care bundle integrated into CCIS can impact patient outcomes positively.

COVID-19 scenario modelling for the mitigation of capacity-dependent deaths in intensive care.

Managing healthcare demand and capacity is especially difficult in the context of the COVID-19 pandemic, where limited intensive care resources can be overwhelmed by a large number of cases requiring admission in a short space of time. If patients are unable to access this specialist resource, then death is a likely outcome. In appreciating these 'capacity-dependent' deaths, this paper reports on the clinically-led development of a stochastic discrete event simulation model designed to capture the key dynamics of the intensive care admissions process for COVID-19 patients. With application to a large public hospital in England during an early stage of the pandemic, the purpose of this study was to estimate the extent to which such capacity-dependent deaths can be mitigated through demand-side initiatives involving non-pharmaceutical interventions and supply-side measures to increase surge capacity. Based on information available at the time, results suggest that total capacity-dependent deaths can be reduced by 75% through a combination of increasing capacity from 45 to 100 beds, reducing length of stay by 25%, and flattening the peak demand to 26 admissions per day. Accounting for the additional 'capacity-independent' deaths, which occur even when appropriate care is available within the intensive care setting, yields an aggregate reduction in total deaths of 30%. The modelling tool, which is freely available and open source, has since been used to support COVID-19 response planning at a number of healthcare systems within the UK National Health Service.

Explainable hierarchical clustering for patient subtyping and risk prediction.

We present a pipeline in which machine learning techniques are used to automatically identify and evaluate subtypes of hospital patients admitted between 2017 and 2021 in a large UK teaching hospital. Patient clusters are determined using routinely collected hospital data, such as those used in the UK's National Early Warning Score 2 (NEWS2). An iterative, hierarchical clustering process was used to identify the minimum set of relevant features for cluster separation. With the use of state-of-the-art explainability techniques, the identified subtypes are interpreted and assigned clinical meaning, illustrating their robustness. In parallel, clinicians assessed intracluster similarities and intercluster differences of the identified patient subtypes within the context of their clinical knowledge. For each cluster, outcome prediction models were trained and their forecasting ability was illustrated against the NEWS2 of the unclustered patient cohort. These preliminary results suggest that subtype models can outperform the established NEWS2 method, providing improved prediction of patient deterioration. By considering both the computational outputs and clinician-based explanations in patient subtyping, we aim to highlight the mutual benefit of combining machine learning techniques with clinical expertise.

Validation of the MIRACLE2 Score for Prognostication After Out-of-hospital Cardiac Arrest.

Background: Out-of-hospital cardiac arrest (OHCA) is associated with very poor clinical outcomes. An optimal pathway of care is yet to be defined, but prognostication is likely to assist in the challenging decision-making required for treatment of this high-risk patient cohort. The MIRACLE2 score provides a simple method of neuro-prognostication but as yet it has not been externally validated. The aim of this study was therefore to retrospectively apply the score to a cohort of OHCA patients to assess the predictive ability and accuracy in the identification of neurological outcome. Methods: Retrospective data of patients identified by hospital coding, over a period of 18 months, were collected from a large tertiary-level cardiac centre with a mature, multidisciplinary OHCA service. MIRACLE2 score performance was assessed against three existing OHCA prognostication scores. Results: Patients with all-comer OHCA, of presumed cardiac origin, with and without evidence of ST-elevation MI (43.4% versus 56.6%, respectively) were included. Regardless of presentation, the MIRACLE2 score performed well in neuro-prognostication, with a low MIRACLE2 score (≤2) providing a negative predictive value of 94% for poor neurological outcome at discharge, while a high score (≥5) had a positive predictive value of 95%. A high MIRACLE2 score performed well regardless of presenting ECG, with 91% of patients receiving early coronary angiography having a poor outcome. Conclusion: The MIRACLE2 score has good prognostic performance and is easily applicable to cardiac-origin OHCA presentation at the hospital front door. Prognostic scoring may assist decision-making regarding early angiographic assessment.

Requirements for a Bespoke Intensive Care Unit Dashboard in Response to the COVID-19 Pandemic: Semistructured Interview Study.

BACKGROUND: Intensive care units (ICUs) around the world are in high demand due to patients with COVID-19 requiring hospitalization. As researchers at the University of Bristol, we were approached to develop a bespoke data visualization dashboard to assist two local ICUs during the pandemic that will centralize disparate data sources in the ICU to help reduce the cognitive load on busy ICU staff in the ever-evolving pandemic. OBJECTIVE: The aim of this study was to conduct interviews with ICU staff in University Hospitals Bristol and Weston National Health Service Foundation Trust to elicit requirements for a bespoke dashboard to monitor the high volume of patients, particularly during the COVID-19 pandemic. METHODS: We conducted six semistructured interviews with clinical staff to obtain an overview of their requirements for the dashboard and to ensure its ultimate suitability for end users. Interview questions aimed to understand the job roles undertaken in the ICU, potential uses of the dashboard, specific issues associated with managing COVID-19 patients, key data of interest, and any concerns about the introduction of a dashboard into the ICU. RESULTS: From our interviews, we found the following design requirements: (1) a flexible dashboard, where the functionality can be updated quickly and effectively to respond to emerging information about the management of this new disease; (2) a mobile dashboard, which allows staff to move around on wards with a dashboard, thus potentially replacing paper forms to enable detailed and consistent data entry; (3) a customizable and intuitive dashboard, where individual users would be able to customize the appearance of the dashboard to suit their role; (4) real-time data and trend analysis via informative data visualizations that help busy ICU staff to understand a patient's clinical trajectory; and (5) the ability to manage tasks and staff, tracking both staff and patient movements, handovers, and task monitoring to ensure the highest quality of care. CONCLUSIONS: The findings of this study confirm that digital solutions for ICU use would potentially reduce the cognitive load of ICU staff and reduce clinical errors at a time of notably high demand of intensive health care.

The Value of Triage during Periods of Intense COVID-19 Demand: Simulation Modeling Study.

BACKGROUND: During the COVID-19 pandemic, many intensive care units have been overwhelmed by unprecedented levels of demand. Notwithstanding ethical considerations, the prioritization of patients with better prognoses may support a more effective use of available capacity in maximizing aggregate outcomes. This has prompted various proposed triage criteria, although in none of these has an objective assessment been made in terms of impact on number of lives and life-years saved. DESIGN: An open-source computer simulation model was constructed for approximating the intensive care admission and discharge dynamics under triage. The model was calibrated from observational data for 9505 patient admissions to UK intensive care units. To explore triage efficacy under various conditions, scenario analysis was performed using a range of demand trajectories corresponding to differing nonpharmaceutical interventions. RESULTS: Triaging patients at the point of expressed demand had negligible effect on deaths but reduces life-years lost by up to 8.4% (95% confidence interval: 2.6% to 18.7%). Greater value may be possible through "reverse triage", that is, promptly discharging any patient not meeting the criteria if admission cannot otherwise be guaranteed for one who does. Under such policy, life-years lost can be reduced by 11.7% (2.8% to 25.8%), which represents 23.0% (5.4% to 50.1%) of what is operationally feasible with no limit on capacity and in the absence of improved clinical treatments. CONCLUSIONS: The effect of simple triage is limited by a tradeoff between reduced deaths within intensive care (due to improved outcomes) and increased deaths resulting from declined admission (due to lower throughput given the longer lengths of stay of survivors). Improvements can be found through reverse triage, at the expense of potentially complex ethical considerations.

Curation of an intensive care research dataset from routinely collected patient data in an NHS trust.

In this data note we provide the details of a research database of 4831 adult intensive care patients who were treated in the Bristol Royal Infirmary, UK between 2015 and 2019. The purposes of this publication are to describe the dataset for external researchers who may be interested in making use of it, and to detail the methods used to curate the dataset in order to help other intensive care units make secondary use of their routinely collected data. The curation involves linkage between two critical care datasets within our hospital and the accompanying code is available online. For reasons of data privacy the data cannot be shared without researchers obtaining appropriate ethical consents. In the future we hope to obtain a data sharing agreement in order to publicly share the de-identified data, and to link our data with other intensive care units who use a Philips clinical information system.

Towards a decision support tool for intensive care discharge: machine learning algorithm development using electronic healthcare data from MIMIC-III and Bristol, UK.

OBJECTIVE: The primary objective is to develop an automated method for detecting patients that are ready for discharge from intensive care. DESIGN: We used two datasets of routinely collected patient data to test and improve on a set of previously proposed discharge criteria. SETTING: Bristol Royal Infirmary general intensive care unit (GICU). PATIENTS: Two cohorts derived from historical datasets: 1870 intensive care patients from GICU in Bristol, and 7592 from Medical Information Mart for Intensive Care (MIMIC)-III. RESULTS: In both cohorts few successfully discharged patients met all of the discharge criteria. Both a random forest and a logistic classifier, trained using multiple-source cross-validation, demonstrated improved performance over the original criteria and generalised well between the cohorts. The classifiers showed good agreement on which features were most predictive of readiness-for-discharge, and these were generally consistent with clinical experience. By weighting the discharge criteria according to feature importance from the logistic model we showed improved performance over the original criteria, while retaining good interpretability. CONCLUSIONS: Our findings indicate the feasibility of the proposed approach to ready-for-discharge classification, which could complement other risk models of specific adverse outcomes in a future decision support system. Avenues for improvement to produce a clinically useful tool are identified.

Protective mechanical ventilation in United Kingdom critical care units: A multicentre audit.

Lung protective ventilation is becoming increasingly used for all critically ill patients being mechanically ventilated on a mandatory ventilator mode. Compliance with the universal application of this ventilation strategy in intensive care units in the United Kingdom is unknown. This 24-h audit of ventilation practice took place in 16 intensive care units in two regions of the United Kingdom. The mean tidal volume for all patients being ventilated on a mandatory ventilator mode was 7.2(±1.4) ml kg-1 predicted body weight and overall compliance with low tidal volume ventilation (≤6.5 ml kg-1 predicted body weight) was 34%. The mean tidal volume for patients ventilated with volume-controlled ventilation was 7.0(±1.2) ml kg-1 predicted body weight and 7.9(±1.8) ml kg-1 predicted body weight for pressure-controlled ventilation (P < 0.0001). Overall compliance with recommended levels of positive end-expiratory pressure was 72%. Significant variation in practice existed both at a regional and individual unit level.

Increasing compliance with low tidal volume ventilation in the ICU with two nudge-based interventions: evaluation through intervention time-series analyses.

OBJECTIVES: Low tidal volume (TVe) ventilation improves outcomes for ventilated patients, and the majority of clinicians state they implement it. Unfortunately, most patients never receive low TVes. 'Nudges' influence decision-making with subtle cognitive mechanisms and are effective in many contexts. There have been few studies examining their impact on clinical decision-making. We investigated the impact of 2 interventions designed using principles from behavioural science on the deployment of low TVe ventilation in the intensive care unit (ICU). SETTING: University Hospitals Bristol, a tertiary, mixed medical and surgical ICU with 20 beds, admitting over 1300 patients per year. PARTICIPANTS: Data were collected from 2144 consecutive patients receiving controlled mechanical ventilation for more than 1 hour between October 2010 and September 2014. Patients on controlled mechanical ventilation for more than 20 hours were included in the final analysis. INTERVENTIONS: (1) Default ventilator settings were adjusted to comply with low TVe targets from the initiation of ventilation unless actively changed by a clinician. (2) A large dashboard was deployed displaying TVes in the format mL/kg ideal body weight (IBW) with alerts when TVes were excessive. PRIMARY OUTCOME MEASURE: TVe in mL/kg IBW. FINDINGS: TVe was significantly lower in the defaults group. In the dashboard intervention, TVe fell more quickly and by a greater amount after a TVe of 8 mL/kg IBW was breached when compared with controls. This effect improved in each subsequent year for 3 years. CONCLUSIONS: This study has demonstrated that adjustment of default ventilator settings and a dashboard with alerts for excessive TVe can significantly influence clinical decision-making. This offers a promising strategy to improve compliance with low TVe ventilation, and suggests that using insights from behavioural science has potential to improve the translation of evidence into practice.

Using 'nudge' principles for order set design: a before and after evaluation of an electronic prescribing template in critical care.

OBJECTIVE: Computerised order sets have the potential to reduce clinical variation and improve patient safety but the effect is variable. We sought to evaluate the impact of changes to the design of an order set on the delivery of chlorhexidine mouthwash and hydroxyethyl starch (HES) to patients in the intensive care unit. METHODS: The study was conducted at University Hospitals Bristol NHS Foundation Trust, UK. Our intensive care unit uses a clinical information system (CIS). All drugs and fluids are prescribed with the CIS and drug and fluid charts are stored within a database. Chlorhexidine mouthwash was added as a default prescription to the prescribing template in January 2010. HES was removed from the prescribing template in April 2009. Both interventions were available to prescribe manually throughout the study period. We conducted a database review of all patients eligible for each intervention before and after changes to the configuration of choices within the prescribing system. RESULTS: 2231 ventilated patients were identified as appropriate for treatment with chlorhexidine, 591 before the intervention and 1640 after. 55.3% were prescribed chlorhexidine before the change and 90.4% after (p<0.001). 6199 patients were considered in the HES intervention, 2177 before the intervention and 4022 after. The mean volume of HES infused per patient fell from 630 mL to 20 mL after the change (p<0.001) and the percentage of patients receiving HES fell from 54.1% to 3.1% (p<0.001). These results were well sustained with time. CONCLUSIONS: The presentation of choices within an electronic prescribing system influenced the delivery of evidence-based interventions in a predictable way and the effect was well sustained. This approach has the potential to enhance the effectiveness of computerised order sets.