Meeting medical emergency response criteria for hypertension is not associated with an increased likelihood of in-hospital mortality in a tertiary referral center.

Backgrounds: Rapid response team or medical emergency team (MET) calls are typically activated by significant alterations of vital signs in inpatients. However, the clinical significance of a specific criterion, blood pressure elevations, is uncertain. Objectives: The aim of this study was to evaluate the likelihood ratios associated with MET-activating vital signs, particularly in-patient hypertension, for predicting in-hospital mortality among general medicine inpatients who met MET criteria at any point during admission in a South Australian metropolitan teaching hospital. Results: Among the 15,734 admissions over a two-year period, 4282 (27.2%) met any MET criteria, with a positive likelihood ratio of 3.05 (95% CI 2.93 to 3.18) for in-hospital mortality. Individual MET criteria were significantly associated with in-hospital mortality, with the highest positive likelihood ratio for respiratory rate ≤ 7 breaths per minute (9.83, 95% CI 6.90 to 13.62), barring systolic pressure ≥ 200 mmHg (LR + 1.26, 95% CI 0.86 to 1.69). Conclusions: Our results show that meeting the MET criteria for hypertension, unlike other criteria, was not significant associated with in-hospital mortality. This observation warrants further research in other patient cohorts to determine whether blood pressure elevations should be routinely included in MET criteria.

Defining the correlates of lymphopenia and independent predictors of poor clinical outcome in adults hospitalized with COVID-19 in Australia.

Lymphopenia is a common feature of acute COVID-19 and is associated with increased disease severity and 30-day mortality. Here we aim to define the demographic and clinical characteristics that correlate with lymphopenia in COVID-19 and determine if lymphopenia is an independent predictor of poor clinical outcome. We analysed the ENTER-COVID (Epidemiology of hospitalized in-patient admissions following planned introduction of Epidemic SARS-CoV-2 to highly vaccinated COVID-19 naïve population) dataset of adults (N = 811) admitted for COVID-19 treatment in South Australia in a retrospective registry study, categorizing them as (a) lymphopenic (lymphocyte count < 1 × 109/L) or (b) non-lymphopenic at hospital admission. Comorbidities and laboratory parameters were compared between groups. Multiple regression analysis was performed using a linear or logistic model. Intensive care unit (ICU) patients and non-survivors exhibited lower median lymphocyte counts than non-ICU patients and survivors respectively. Univariate analysis revealed that low lymphocyte counts associated with hypertension and correlated with haemoglobin, platelet count and negatively correlated with urea, creatinine, bilirubin, and aspartate aminotransferase (AST). Multivariate analysis identified age, male, haemoglobin, platelet count, diabetes, creatinine, bilirubin, alanine transaminase, c-reactive protein (CRP) and lactate dehydrogenase (LDH) as independent predictors of poor clinical outcome in COVID-19, while lymphopenia did not emerge as a significant predictor.

Systematic review and longitudinal analysis of implementing Artificial Intelligence to predict clinical deterioration in adult hospitals: what is known and what remains uncertain.

OBJECTIVE: To identify factors influencing implementation of machine learning algorithms (MLAs) that predict clinical deterioration in hospitalized adult patients and relate these to a validated implementation framework. MATERIALS AND METHODS: A systematic review of studies of implemented or trialed real-time clinical deterioration prediction MLAs was undertaken, which identified: how MLA implementation was measured; impact of MLAs on clinical processes and patient outcomes; and barriers, enablers and uncertainties within the implementation process. Review findings were then mapped to the SALIENT end-to-end implementation framework to identify the implementation stages at which these factors applied. RESULTS: Thirty-seven articles relating to 14 groups of MLAs were identified, each trialing or implementing a bespoke algorithm. One hundred and seven distinct implementation evaluation metrics were identified. Four groups reported decreased hospital mortality, 1 significantly. We identified 24 barriers, 40 enablers, and 14 uncertainties and mapped these to the 5 stages of the SALIENT implementation framework. DISCUSSION: Algorithm performance across implementation stages decreased between in silico and trial stages. Silent plus pilot trial inclusion was associated with decreased mortality, as was the use of logistic regression algorithms that used less than 39 variables. Mitigation of alert fatigue via alert suppression and threshold configuration was commonly employed across groups. CONCLUSIONS: : There is evidence that real-world implementation of clinical deterioration prediction MLAs may improve clinical outcomes. Various factors identified as influencing success or failure of implementation can be mapped to different stages of implementation, thereby providing useful and practical guidance for implementers.

Surgery's Rosetta Stone: Natural language processing to predict discharge and readmission after general surgery.

BACKGROUND: This study aimed to examine the accuracy with which multiple natural language processing artificial intelligence models could predict discharge and readmissions after general surgery. METHODS: Natural language processing models were derived and validated to predict discharge within the next 48 hours and 7 days and readmission within 30 days (based on daily ward round notes and discharge summaries, respectively) for general surgery inpatients at 2 South Australian hospitals. Natural language processing models included logistic regression, artificial neural networks, and Bidirectional Encoder Representations from Transformers. RESULTS: For discharge prediction analyses, 14,690 admissions were included. For readmission prediction analyses, 12,457 patients were included. For prediction of discharge within 48 hours, derivation and validation data set area under the receiver operator characteristic curves were, respectively: 0.86 and 0.86 for Bidirectional Encoder Representations from Transformers, 0.82 and 0.81 for logistic regression, and 0.82 and 0.81 for artificial neural networks. For prediction of discharge within 7 days, derivation and validation data set area under the receiver operator characteristic curves were, respectively: 0.82 and 0.81 for Bidirectional Encoder Representations from Transformers, 0.75 and 0.72 for logistic regression, and 0.68 and 0.67 for artificial neural networks. For readmission prediction within 30 days, derivation and validation data set area under the receiver operator characteristic curves were, respectively: 0.55 and 0.59 for Bidirectional Encoder Representations from Transformers and 0.77 and 0.62 for logistic regression. CONCLUSION: Modern natural language processing models, particularly Bidirectional Encoder Representations from Transformers, can effectively and accurately identify general surgery patients who will be discharged in the next 48 hours. However, these approaches are less capable of identifying general surgery patients who will be discharged within the next 7 days or who will experience readmission within 30 days of discharge.

The Adelaide Score: An artificial intelligence measure of readiness for discharge after general surgery.

BACKGROUND: This study aimed to examine the performance of machine learning algorithms for the prediction of discharge within 12 and 24 h to produce a measure of readiness for discharge after general surgery. METHODS: Consecutive general surgery patients at two tertiary hospitals, over a 2-year period, were included. Observation and laboratory parameter data were stratified into training, testing and validation datasets. Random forest, XGBoost and logistic regression models were evaluated. Each ward round note time was taken as a different event. Primary outcome was classification accuracy of the algorithmic model able to predict discharge within the next 12 h on the validation data set. RESULTS: 42 572 ward round note timings were included from 8826 general surgery patients. Discharge occurred within 12 h for 8800 times (20.7%), and within 24 h for 9885 (23.2%). For predicting discharge within 12 h, model classification accuracies for derivation and validation data sets were: 0.84 and 0.85 random forest, 0.84 and 0.83 XGBoost, 0.80 and 0.81 logistic regression. For predicting discharge within 24 h, model classification accuracies for derivation and validation data sets were: 0.83 and 0.84 random forest, 0.82 and 0.81 XGBoost, 0.78 and 0.79 logistic regression. Algorithms generated a continuous number between 0 and 1 (or 0 and 100), representing readiness for discharge after general surgery. CONCLUSIONS: A derived artificial intelligence measure (the Adelaide Score) successfully predicts discharge within the next 12 and 24 h in general surgery patients. This may be useful for both treating teams and allied health staff within surgical systems.

Single count breath test for the evaluation of respiratory function in Myasthenia Gravis: A systematic review.

BACKGROUND: Myasthenia gravis (MG) can have a variety of respiratory presentations, ranging from mild symptoms through to respiratory failure. The evaluation of respiratory function in MG can be limited by accessibility to testing facilities, availability of medical equipment, and facial weakness. The single count breath test (SCBT) may be a useful adjunct in the evaluation of respiratory function in MG. METHOD: A systematic review of the databases PubMed, EMBASE, and the Cochrane Library was conducted from inception to October 2022 in accordance with PRISMA guidelines and was registered on PROSPERO. RESULTS: There were 6 studies that fulfilled the inclusion criteria. The described method of evaluating SCBT involves inhaling deeply, then counting at two counts per second, in English or Spanish, sitting upright, with normal vocal register, until another breath needs to be taken. The identified studies support that the SCBT has a moderate correlation with forced vital capacity. These results also support that SCBT can assist the identification of MG exacerbation, including via assessment over the telephone. The included studies support a threshold count of ≥ 25 as consistent with normal respiratory muscle function. Although further analysis is needed, the included studies describe the SCBT as a quick bedside tool that is inexpensive and well tolerated. CONCLUSIONS: The results of this review support the clinical utility of the SCBT in assessing respiratory function in MG and describe the most current and effective methods of administration.

Protocol describing a systematic review and mixed methods consensus process to define the deteriorated ward patient.

INTRODUCTION: Most patients admitted to hospital recover with treatments that can be administered on the general ward. A small but important group deteriorate however and require augmented organ support in areas with increased nursing to patient ratios. In observational studies evaluating this cohort, proxy outcomes such as unplanned intensive care unit admission, cardiac arrest and death are used. These outcome measures introduce subjectivity and variability, which in turn hinders the development and accuracy of the increasing numbers of electronic medical record (EMR) linked digital tools designed to predict clinical deterioration. Here, we describe a protocol for developing a new outcome measure using mixed methods to address these limitations. METHODS AND ANALYSIS: We will undertake firstly, a systematic literature review to identify existing generic, syndrome-specific and organ-specific definitions for clinically deteriorated, hospitalised adult patients. Secondly, an international modified Delphi study to generate a short list of candidate definitions. Thirdly, a nominal group technique (NGT) (using a trained facilitator) will take a diverse group of stakeholders through a structured process to generate a consensus definition. The NGT process will be informed by the data generated from the first two stages. The definition(s) for the deteriorated ward patient will be readily extractable from the EMR. ETHICS AND DISSEMINATION: This study has ethics approval (reference 16399) from the Central Adelaide Local Health Network Human Research Ethics Committee. Results generated from this study will be disseminated through publication and presentation at national and international scientific meetings.

Artificial intelligence and clinical deterioration.

PURPOSE OF REVIEW: To provide an overview of the systems being used to identify and predict clinical deterioration in hospitalised patients, with focus on the current and future role of artificial intelligence (AI). RECENT FINDINGS: There are five leading AI driven systems in this field: the Advanced Alert Monitor (AAM), the electronic Cardiac Arrest Risk Triage (eCART) score, Hospital wide Alert Via Electronic Noticeboard, the Mayo Clinic Early Warning Score, and the Rothman Index (RI). Each uses Electronic Patient Record (EPR) data and machine learning to predict adverse events. Less mature but relevant evolutions are occurring in the fields of Natural Language Processing, Time and Motion Studies, AI Sepsis and COVID-19 algorithms. SUMMARY: Research-based AI-driven systems to predict clinical deterioration are increasingly being developed, but few are being implemented into clinical workflows. Escobar et al. (AAM) provide the current gold standard for robust model development and implementation methodology. Multiple technologies show promise, however, the pathway to meaningfully affect patient outcomes remains challenging.

Evaluation of a digital system to predict unplanned admissions to the intensive care unit: A mixed-methods approach.

BACKGROUND: We have developed the Hospital Alerting Via Electronic Noticeboard (HAVEN) which aims to identify hospitalised patients most at risk of reversible deterioration. HAVEN combines patients' vital-sign measurements with laboratory results, demographics and comorbidities using a machine learnt algorithm. OBJECTIVES: The aim of this study was to identify variables or concepts that could improve HAVEN predictive performance. METHODS: This was an embedded, mixed methods study. Eligible patients with the five highest HAVEN scores in the hospital (i.e., 'HAVEN Top 5') had their medical identification details recorded. We conducted a structured medical note review on these patients 48 hours post their identifiers being recorded. Methods of constant comparison were used during data collection and to analyse patient data. RESULTS: The 129 patients not admitted to ICU then underwent constant comparison review, which produced three main groups. Group 1 were patients referred to specialist services (n = 37). Group 2 responded to ward-based treatment, (n = 38). Group 3 were frail and had documented treatment limitations (n = 47). CONCLUSIONS: Digital-only validation methods code the cohort not admitted to ICU as 'falsely positive' in sensitivity analyses however this approach limits the evaluation of model performance. Our study suggested that coding for patients referred to other specialist teams, those with treatment limitations in place, along with those who are deteriorating but then respond to ward-based therapies, would give a more accurate measure of the value of the scores, especially in relation to cost-effectiveness of resource utilisation.

Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems.

BACKGROUND AND OBJECTIVES: Electronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition. METHODS: Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist. RESULTS: The search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported. The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality." CONCLUSION: There is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics, and reporting. Standardized processes may improve evaluation and comparison electronic health record systems and improve their development and implementation.

Detecting Deteriorating Patients in the Hospital: Development and Validation of a Novel Scoring System.

Rationale: Late recognition of patient deterioration in hospital is associated with worse outcomes, including higher mortality. Despite the widespread introduction of early warning score (EWS) systems and electronic health records, deterioration still goes unrecognized. Objectives: To develop and externally validate a Hospital- wide Alerting via Electronic Noticeboard (HAVEN) system to identify hospitalized patients at risk of reversible deterioration. Methods: This was a retrospective cohort study of patients 16 years of age or above admitted to four UK hospitals. The primary outcome was cardiac arrest or unplanned admission to the ICU. We used patient data (vital signs, laboratory tests, comorbidities, and frailty) from one hospital to train a machine-learning model (gradient boosting trees). We internally and externally validated the model and compared its performance with existing scoring systems (including the National EWS, laboratory-based acute physiology score, and electronic cardiac arrest risk triage score). Measurements and Main Results: We developed the HAVEN model using 230,415 patient admissions to a single hospital. We validated HAVEN on 266,295 admissions to four hospitals. HAVEN showed substantially higher discrimination (c-statistic, 0.901 [95% confidence interval, 0.898-0.903]) for the primary outcome within 24 hours of each measurement than other published scoring systems (which range from 0.700 [0.696-0.704] to 0.863 [0.860-0.865]). With a precision of 10%, HAVEN was able to identify 42% of cardiac arrests or unplanned ICU admissions with a lead time of up to 48 hours in advance, compared with 22% by the next best system. Conclusions: The HAVEN machine-learning algorithm for early identification of in-hospital deterioration significantly outperforms other published scores such as the National EWS.

Using real-time location devices (RTLD) to quantify off-unit adult intensive care registrar workload: a 1-year tertiary NHS hospital prospective observational study.

UK national guidelines state deteriorating or at risk hospital ward patients should receive care from trained critical care outreach personnel. In most tertiary hospitals this involves a team led by an Intensive Care Unit (ICU) registrar. The ICU registrar must also review patients referred for possible ICU admission. These two responsibilities require work away from the ICU. To our knowledge the burden of this work has not been described, despite its importance in ICU workforce management and patient safety. A 12-month, prospective, observational study was carried out. The primary outcome measure was ICU registrar time spent on and off-unit. The study participants were senior and junior registrars on the rota of the 16 bed, Adult Intensive Care Unit at the John Radcliffe Hospital in Oxford. To measure their work patterns, this study used AeroScout 'T2' Real Time Location Device (RTLD) tags (Stanley Healthcare, Swindon). In our hospital, senior and junior ICU registrars spend roughly one-fifth of their time off-unit, half of which is spent in ED. This workload combines to leave the unit unattended at night up to 10% of the time. RTLDs provide a reliable, automated method for quantifying ICU registrar off-unit work patterns. This method may be adopted for quantifying other clinical staff work patterns in suitably equipped hospital environments.

Testing a digital system that ranks the risk of unplanned intensive care unit admission in all ward patients: protocol for a prospective observational cohort study.

INTRODUCTION: Traditional early warning scores (EWSs) use vital sign derangements to detect clinical deterioration in patients treated on hospital wards. Combining vital signs with demographics and laboratory results improves EWS performance. We have developed the Hospital Alerting Via Electronic Noticeboard (HAVEN) system. HAVEN uses vital signs, as well as demographic, comorbidity and laboratory data from the electronic patient record, to quantify and rank the risk of unplanned admission to an intensive care unit (ICU) within 24 hours for all ward patients. The primary aim of this study is to find additional variables, potentially missed during development, which may improve HAVEN performance. These variables will be sought in the medical record of patients misclassified by the HAVEN risk score during testing. METHODS: This will be a prospective, observational, cohort study conducted at the John Radcliffe Hospital, part of the Oxford University Hospitals NHS Foundation Trust in the UK. Each day during the study periods, we will document all highly ranked patients (ie, those with the highest risk for unplanned ICU admission) identified by the HAVEN system. After 48 hours, we will review the progress of the identified patients. Patients who were subsequently admitted to the ICU will be removed from the study (as they will have been correctly classified by HAVEN). Highly ranked patients not admitted to ICU will undergo a structured medical notes review. Additionally, at the end of the study periods, all patients who had an unplanned ICU admission but whom HAVEN failed to rank highly will have a structured medical notes review. The review will identify candidate variables, likely associated with unplanned ICU admission, not included in the HAVEN risk score. ETHICS AND DISSEMINATION: Approval has been granted for gathering the data used in this study from the South Central Oxford C Research Ethics Committee (16/SC/0264, 13 June 2016) and the Confidentiality Advisory Group (16/CAG/0066). DISCUSSION: Our study will use a clinical expert conducting a structured medical notes review to identify variables, associated with unplanned ICU admission, not included in the development of the HAVEN risk score. These variables will then be added to the risk score and evaluated for potential performance gain. To the best of our knowledge, this is the first study of this type. We anticipate that documenting the HAVEN development methods will assist other research groups developing similar technology. TRIAL REGISTRATION NUMBER: ISRCTN12518261.

Patient centred variables with univariate associations with unplanned ICU admission: a systematic review.

BACKGROUND: Multiple predictive scores using Electronic Patient Record data have been developed for hospitalised patients at risk of clinical deterioration. Methods used to select patient centred variables for inclusion in these scores varies. We performed a systematic review to describe univariate associations with unplanned Intensive Care Unit (ICU) admission with the aim of assisting model development for future scores that predict clinical deterioration. METHODS: Data sources were MEDLINE, EMBASE, CINAHL, CENTRAL and the Cochrane Database of Systematic Reviews. Included studies were published since 2000 describing an association between patient centred variables and unplanned ICU admission determined using univariate analysis. Two authors independently screened titles, abstracts and full texts against inclusion and exclusion criteria. DistillerSR (Evidence Partners, Canada, Ottawa, Ontario) software was used to manage the data and identify duplicate search results. All screening and data extraction forms were implemented within DistillerSR. Study quality was assessed using an adapted version of the Newcastle-Ottawa Scale. Variables were analysed for strength of association with unplanned ICU admission. RESULTS: The database search yielded 1520 unique studies; 1462 were removed after title and abstract review; 57 underwent full text screening; 16 studies were included. One hundred and eighty nine variables with an evaluated univariate association with unplanned ICU admission were described. DISCUSSION: Being male, increasing age, a history of congestive cardiac failure or diabetes, a diagnosis of hepatic disease or having abnormal vital signs were all strongly associated with ICU admission. CONCLUSION: These findings will assist variable selection during the development of future models predicting unplanned ICU admission. TRIAL REGISTRATION: This study is a component of a larger body of work registered in the ISRCTN registry ( ISRCTN12518261 ).

Quality metrics for the evaluation of Rapid Response Systems: Proceedings from the third international consensus conference on Rapid Response Systems.

BACKGROUND: Clinically significant deterioration of patients admitted to general wards is a recognized complication of hospital care. Rapid Response Systems (RRS) aim to reduce the number of avoidable adverse events. The authors aimed to develop a core quality metric for the evaluation of RRS. METHODS: We conducted an international consensus process. Participants included patients, carers, clinicians, research scientists, and members of the International Society for Rapid Response Systems with representatives from Europe, Australia, Africa, Asia and the US. Scoping reviews of the literature identified potential metrics. We used a modified Delphi methodology to arrive at a list of candidate indicators that were reviewed for feasibility and applicability across a broad range of healthcare systems including low and middle-income countries. The writing group refined recommendations and further characterized measurement tools. RESULTS: Consensus emerged that core outcomes for reporting for quality improvement should include ten metrics related to structure, process and outcome for RRS with outcomes following the domains of the quadruple aim. The conference recommended that hospitals should collect data on cardiac arrests and their potential predictability, timeliness of escalation, critical care interventions and presence of written treatment goals for patients remaining on general wards. Unit level reporting should include the presence of patient activated rapid response and metrics of organizational culture. We suggest two exploratory cost metrics to underpin urgently needed research in this area. CONCLUSION: A consensus process was used to develop ten metrics for better understanding the course and care of deteriorating ward patients. Others are proposed for further development.

The effect of fractional inspired oxygen concentration on early warning score performance: A database analysis.

OBJECTIVES: To calculate fractional inspired oxygen concentration (FiO2) thresholds in ward patients and add these to the National Early Warning Score (NEWS). To evaluate the performance of NEWS-FiO2 against NEWS when predicting in-hospital death and unplanned intensive care unit (ICU) admission. METHODS: A multi-centre, retrospective, observational cohort study was carried out in five hospitals from two UK NHS Trusts. Adult admissions with at least one complete set of vital sign observations recorded electronically were eligible. The primary outcome measure was an 'adverse event' which comprised either in-hospital death or unplanned ICU admission. Discrimination was assessed using the Area Under the Receiver Operating Characteristic curve (AUROC). RESULTS: A cohort of 83,304 patients from a total of 271,363 adult admissions were prescribed oxygen. In this cohort, NEWS-FiO2 (AUROC 0.823, 95% CI 0.819-0.824) outperformed NEWS (AUORC 0.811, 95% CI 0.809-0.814) when predicting in-hospital death or unplanned ICU admission within 24 h of a complete set of vital sign observations. CONCLUSIONS: NEWS-FiO2 generates a performance gain over NEWS when studied in ward patients requiring oxygen. This warrants further study, particularly in patients with respiratory disorders.