Optimal timing for the Modified Early Warning Score for prediction of short-term critical illness in the acute care chain: a prospective observational study.

INTRODUCTION: The Modified Early Warning Score (MEWS) is an effective tool to identify patients in the acute care chain who are likely to deteriorate. Although it is increasingly being implemented in the ED, the optimal moment to use the MEWS is unknown. This study aimed to determine at what moment in the acute care chain MEWS has the highest accuracy in predicting critical illness. METHODS: Adult patients brought by ambulance to the ED at both locations of the Amsterdam UMC, a level 1 trauma centre, were prospectively included between 11 March and 28 October 2021. MEWS was calculated using vital parameters measured prehospital, at ED presentation, 1 hour and 3 hours thereafter, imputing for missing temperature and/or consciousness, as these values were expected not to deviate. Critical illness was defined as requiring intensive care unit admission, myocardial infarction or death within 72 hours after ED presentation. Accuracy in predicting critical illness was assessed using the area under the receiver operating characteristics curve (AUROC). RESULTS: Of the 790 included patients, critical illness occurred in 90 (11.4%). MEWS based on vital parameters at ED presentation had the highest performance in predicting critical illness with an AUROC of 0.73 (95% CI 0.67 to 0.79) but did not significantly differ compared with other moments. Patients with an increasing MEWS over time are significantly more likely to become critical ill compared with patients with an improving MEWS. CONCLUSION: The performance of MEWS is moderate in predicting critical illness using vital parameters measured surrounding ED admission. However, an increase of MEWS during ED admission is correlated with the development of critical illness. Therefore, early recognition of deteriorating patients at the ED may be achieved by frequent MEWS calculation. Further studies should investigate the effect of continuous monitoring of these patients at the ED.

The accuracy of predicting hospital admission by emergency medical service and emergency department personnel compared to the prehospital MEWS: a prospective multicenter study.

INTRODUCTION: Overcrowding in the emergency department (ED) is a global problem. Early and accurate recognition of a patient's disposition could limit time spend at the ED and thus improve throughput and quality of care provided. This study aims to compare the accuracy among healthcare providers and the prehospital Modified Early Warning Score (MEWS) in predicting the requirement for hospital admission. METHODS: A prospective, observational, multi-centre study was performed including adult patients brought to the ED by ambulance. Involved Emergency Medical Service (EMS) personnel, ED nurses and physicians were asked to predict the need for hospital admission using a structured questionnaire. Primary endpoint was the comparison between the accuracy of healthcare providers and prehospital MEWS in predicting patients' need for hospital admission. RESULTS: In total 798 patients were included of whom 393 (49.2%) were admitted to the hospital. Sensitivity of predicting hospital admission varied from 80.0 to 91.9%, with physicians predicting hospital admission significantly more accurately than EMS and ED nurses (p < 0.001). Specificity ranged from 56.4 to 67.0%. All healthcare providers outperformed MEWS ≥ 3 score on predicting hospital admission (sensitivity 80.0-91.9% versus 44.0%; all p < 0.001). Predictions for ward admissions specifically were significantly more accurate than MEWS (specificity 94.7-95.9% versus 60.6%, all p < 0.001). CONCLUSIONS: Healthcare providers can accurately predict the need for hospital admission, and all providers outperformed the MEWS score.

Unexpected versus all-cause mortality as the endpoint for investigating the effects of a Rapid Response System in hospitalized patients.

BACKGROUND: The purpose of this study was to assess the effect of replacing all-cause mortality by death without limitation of medical treatments (LOMT) as the endpoint in a study of rapid response teams (RRTs) in hospitalized patients. We also described the time course of LOMT orders in patients dying on a general ward and the influence of RRTs on such orders. METHODS: This study is a secondary analysis of the COMET trial, a pragmatic prospective Dutch multicenter before-after study. We repeated the original analysis of the influence of RRTs on death before hospital discharge by replacing all-cause mortality by death without an LOMT order. In a subgroup of all patients dying before hospital discharge, we documented patient demographics, admission characteristics and LOMT orders of each patient. Patients age 18 ears or above were included. RESULTS: In total, 166,569 patients were included in the study. The unadjusted ORs were 0.865 (95 % CI 0.77-0.98) in the original analysis using all-cause mortality and 0.557 (95 % CI 0.40-0.78) when choosing death without LOMT as the endpoint. In total, 3408 patients died before discharge. At time of death, 2910 (85 %) had an LOMT order. Median time from last change in LOMT status and death was 2 days (IQR 1-5) in the before-phase and median time after introduction of the RRT was 1 day (IQR 1-4) (p value not significant). CONCLUSIONS: The improvement in survival of hospitalized patients after introduction of a rapid response team in the COMET study was more pronounced when choosing death without limitation of medical treatment, rather than all deaths as the endpoint. Most patients who died during hospitalization had limitation of medical treatments ordered, often shortly before death. Rapid response teams did not influence the institution of limitation of medical treatments.

Outcomes Associated With the Nationwide Introduction of Rapid Response Systems in The Netherlands.

OBJECTIVE: To describe the effect of implementation of a rapid response system on the composite endpoint of cardiopulmonary arrest, unplanned ICU admission, or death. DESIGN: Pragmatic prospective Dutch multicenter before-after trial, Cost and Outcomes analysis of Medical Emergency Teams trial. SETTING: Twelve hospitals participated, each including two surgical and two nonsurgical wards between April 2009 and November 2011. The Modified Early Warning Score and Situation-Background-Assessment-Recommendation instruments were implemented over 7 months. The rapid response team was then implemented during the following 17 months. The effects of implementing the rapid response team were measured in the last 5 months of this period. PATIENTS: All patients 18 years old and older admitted to the study wards were included. MEASUREMENTS AND MAIN RESULTS: In total, 166,569 patients were included in the study representing 1,031,172 hospital admission days. No differences were observed in patient demographics between periods. The composite endpoint of cardiopulmonary arrest, unplanned ICU admission, or death per 1,000 admissions was significantly reduced in the rapid response team versus the before phase (adjusted odds ratio, 0.847; 95% CI, 0.725-0.989; p = 0.036). Cardiopulmonary arrests and in-hospital mortality were also significantly reduced (odds ratio, 0.607; 95% CI, 0.393-0.937; p = 0.018 and odds ratio, 0.802; 95% CI, 0.644-1.0; p = 0.05, respectively). Unplanned ICU admissions showed a declining trend (odds ratio, 0.878; 95% CI, 0.755-1.021; p = 0.092), whereas severity of illness at the moment of ICU admission was not different between periods. CONCLUSIONS: In this study, introduction of nationwide implementation of rapid response systems was associated with a decrease in the composite endpoint of cardiopulmonary arrests, unplanned ICU admissions, and mortality in patients in general hospital wards. These findings support the implementation of rapid response systems in hospitals to reduce severe adverse events.

Clinical relevance of routinely measured vital signs in hospitalized patients: a systematic review.

BACKGROUND: Conflicting evidence exists on the effectiveness of routinely measured vital signs on the early detection of increased probability of adverse events. PURPOSE: To assess the clinical relevance of routinely measured vital signs in medically and surgically hospitalized patients through a systematic review. DATA SOURCES: MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), Cumulative Index to Nursing and Allied Health Literature, and Meta-analysen van diagnostisch onderzoek (in Dutch; MEDION) were searched to January 2013. STUDY SELECTION: Prospective studies evaluating routine vital sign measurements of hospitalized patients, in relation to mortality, septic or circulatory shock, intensive care unit admission, bleeding, reoperation, or infection. DATA EXTRACTION: Two reviewers independently assessed potential bias and extracted data to calculate likelihood ratios (LRs) and predictive values. DATA SYNTHESIS: Fifteen studies were performed in medical (n = 7), surgical (n = 4), or combined patient populations (n = 4; totaling 42,565 participants). Only three studies were relatively free from potential bias. For temperature, the positive LR (LR+) ranged from 0 to 9.88 (median 1.78; n = 9 studies); heart rate 0.82 to 6.79 (median 1.51; n = 5 studies); blood pressure 0.72 to 4.7 (median 2.97; n = 4 studies); oxygen saturation 0.65 to 6.35 (median 1.74; n = 2 studies); and respiratory rate 1.27 to 1.89 (n = 3 studies). Overall, three studies reported area under the Receiver Operator Characteristic (ROC) curve (AUC) data, ranging from 0.59 to 0.76. Two studies reported on combined vital signs, in which one study found an LR+ of 47.0, but in the other the AUC was not influenced. CONCLUSIONS: Some discriminative LR+ were found, suggesting the clinical relevance of routine vital sign measurements. However, the subject is poorly studied, and many studies have methodological flaws. Further rigorous research is needed specifically intended to investigate the clinical relevance of routinely measured vital signs. CLINICAL RELEVANCE: The results of this research are important for clinical nurses to underpin daily routine practices and clinical decision making.

Improving the management of medical emergency team calls due to suspected infections: A before-after study.

Objective: To introduce a management guideline for sepsis-related MET calls to increase lactate and blood culture acquisition, as well as prescription of appropriate antibiotics. Design: Prospective before (Jun-Aug 2018) and after (Oct-Dec 2018) study was designed. Setting: A public university linked hospital in Melbourne, Australia. Participants: Adult patients with MET calls related to sepsis/infection were included. Main outcome measures: The primary outcome measure was the proportion of MET calls during which both a blood culture and lactate level were ordered. Secondary outcomes included the frequency with which new antimicrobials were commenced by the MET, and the presence and class of administered antimicrobials. Results: There were 985 and 955 MET calls in the baseline and after periods, respectively. Patient features, MET triggers, limitations of treatment and disposition after the MET call were similar in both groups. Compliance with the acquisition of lactates (p = 0.101), respectively. There was a slight reduction in compliance with lactate acquisition in the after period (97% vs 99%; p = 0.06). In contrast, there was a significant increase in acquisition of blood cultures in the after period (69% vs 78%; p = 0.035). Conclusions: Introducing a sepsis management guideline and enhanced linkage with an AMS program increased blood culture acquisition and decreased broad spectrum antimicrobial use but didn't change in-hospital mortality.

How nurses and physicians judge their own quality of care for deteriorating patients on medical wards: self-assessment of quality of care is suboptimal*.

OBJECTIVE: To describe how nurses and physicians judge their own quality of care for deteriorating patients on medical wards compared with the judgment of independent experts. DESIGN: Cross-sectional study using interviews of care-providers regarding their perceived quality of care for clinically deteriorating patients compared with retrospective judgment by independent experts. SETTING: Academic Medical Center of Amsterdam, the Netherlands. PATIENTS: Between April and July 2009, all patients with cardiopulmonary arrests and unplanned intensive care unit admissions from six medical nursing wards were included. The care-providers (nurses and physicians) taking care of these patients in the previous 12 hrs were included. MEASUREMENTS AND MAIN RESULTS: Forty-seven events and 198 interviews were analyzed. Skill and knowledge level regarding the recognition of a deteriorating patient were rated on a scale of 1-10 with means (SD) of 7.9 (0.8) and 7.7 (0.9), respectively. Nurses and residents attributed coordination of care largely to themselves (74% and 76%, respectively). Communication, cooperation, and coordination were graded in a positive manner (medians between 7.3 and 8), whereas the medical staff graded these factors higher compared to the grading by nurses and residents. Negative predictive values regarding the presence of a delay compared with an expert panel was 37% for nurses and 38% for residents and specialists. CONCLUSIONS: Care-providers mostly rate their care provided to patients in the hours preceding a life-threatening adverse event as good. In contrast, independent experts had a more critical appraisal of the provided care in regards to timely recognition. These findings may partly explain the reluctance of care-providers to implement patient safety initiatives.

Artificial Intelligence for the Prediction of In-Hospital Clinical Deterioration: A Systematic Review.

To analyze the available literature on the performance of artificial intelligence-generated clinical models for the prediction of serious life-threatening events in non-ICU adult patients and evaluate their potential clinical usage. DATA SOURCES: The PubMed database was searched for relevant articles in English literature from January 1, 2000, to January 23, 2022. Search terms, including artificial intelligence, machine learning, deep learning, and deterioration, were both controlled terms and free-text terms. STUDY SELECTION: We performed a systematic search reporting studies that showed performance of artificial intelligence-based models with outcome mortality and clinical deterioration. DATA EXTRACTION: Two review authors independently performed study selection and data extraction. Studies with the same outcome were grouped, namely mortality and various forms of deterioration (including ICU admission, adverse events, and cardiac arrests). Meta-analysis was planned in case sufficient data would be extracted from each study and no considerable heterogeneity between studies was present. DATA SYNTHESIS: In total, 45 articles were included for analysis, in which multiple methods of artificial intelligence were used. Twenty-four articles described models for the prediction of mortality and 21 for clinical deterioration. Due to heterogeneity of study characteristics (patient cohort, outcomes, and prediction models), meta-analysis could not be performed. The main reported measure of performance was the area under the receiver operating characteristic (AUROC) (n = 38), of which 33 (87%) had an AUROC greater than 0.8. The highest reported performance in a model predicting mortality had an AUROC of 0.935 and an area under the precision-recall curve of 0.96. CONCLUSIONS: Currently, a growing number of studies develop and analyzes artificial intelligence-based prediction models to predict critical illness and deterioration. We show that artificial intelligence-based prediction models have an overall good performance in predicting deterioration of patients. However, external validation of existing models and its performance in a clinical setting is highly recommended.

A low aldosterone/renin ratio and high soluble ACE2 associate with COVID-19 severity.

BACKGROUND: The severity of COVID-19 after SARS-CoV-2 infection is unpredictable. Angiotensin-converting enzyme-2 (ACE2) is the receptor responsible for coronavirus binding, while subsequent cell entry relies on priming by the serine protease TMPRSS2 (transmembrane protease, serine 2). Although renin-angiotensin-aldosterone-system (RAAS) blockers have been suggested to upregulate ACE2, their use in COVID-19 patients is now considered well tolerated. The aim of our study was to investigate parameters that determine COVID-19 severity, focusing on RAAS-components and variation in the genes encoding for ACE2 and TMPRSS2. METHODS: Adult patients hospitalized due to SARS-CoV-2 infection between May 2020 and October 2020 in the Haga Teaching Hospital were included, and soluble ACE2 (sACE2), renin, aldosterone (in heparin plasma) and polymorphisms in the ACE2 and TMPRSS2 genes (in DNA obtained from EDTA blood) were determined. MEASUREMENTS AND MAIN RESULTS: Out of the 188 patients who were included, 60 were defined as severe COVID-19 (ICU and/or death). These patients more often used antidiabetic drugs, were older, had higher renin and sACE2 levels, lower aldosterone levels and a lower aldosterone/renin ratio. In addition, they displayed the TMPRSS2-rs2070788 AA genotype less frequently. No ACE2 polymorphism-related differences were observed. Multivariate regression analysis revealed independent significance for age, sACE2, the aldosterone/renin ratio, and the TMPRSS2 rs2070788 non-AA genotype as predictors of COVID-19 severity, together yielding a C-index of 0.79. Findings were independent of the use of RAAS blockers. CONCLUSION: High sACE2, a low aldosterone/renin ratio and having the TMPRSS2 rs2070788 non-AA genotype are novel independent determinants that may help to predict COVID-19 disease severity. TRIAL REGISTRATION: retrospectively registered.

Development and validation of an early warning model for hospitalized COVID-19 patients: a multi-center retrospective cohort study.

BACKGROUND: Timely identification of deteriorating COVID-19 patients is needed to guide changes in clinical management and admission to intensive care units (ICUs). There is significant concern that widely used Early warning scores (EWSs) underestimate illness severity in COVID-19 patients and therefore, we developed an early warning model specifically for COVID-19 patients. METHODS: We retrospectively collected electronic medical record data to extract predictors and used these to fit a random forest model. To simulate the situation in which the model would have been developed after the first and implemented during the second COVID-19 'wave' in the Netherlands, we performed a temporal validation by splitting all included patients into groups admitted before and after August 1, 2020. Furthermore, we propose a method for dynamic model updating to retain model performance over time. We evaluated model discrimination and calibration, performed a decision curve analysis, and quantified the importance of predictors using SHapley Additive exPlanations values. RESULTS: We included 3514 COVID-19 patient admissions from six Dutch hospitals between February 2020 and May 2021, and included a total of 18 predictors for model fitting. The model showed a higher discriminative performance in terms of partial area under the receiver operating characteristic curve (0.82 [0.80-0.84]) compared to the National early warning score (0.72 [0.69-0.74]) and the Modified early warning score (0.67 [0.65-0.69]), a greater net benefit over a range of clinically relevant model thresholds, and relatively good calibration (intercept = 0.03 [- 0.09 to 0.14], slope = 0.79 [0.73-0.86]). CONCLUSIONS: This study shows the potential benefit of moving from early warning models for the general inpatient population to models for specific patient groups. Further (independent) validation of the model is needed.

Rapid response systems in The Netherlands.

Sixty-three (approximately 80%) of the 81 hospitals that responded to a survey sent to all hospitals in The Netherlands with nonpediatric intensive care units had a rapid response system (RRS) in place or were in the final process of starting one. Among many other findings regarding RRS infrastructure and implementation, only 38% of the hospitals allowed nurses to activate the rapid response team without physician consent.

A Protocolised Once a Day Modified Early Warning Score (MEWS) Measurement Is an Appropriate Screening Tool for Major Adverse Events in a General Hospital Population.

BACKGROUND: The Modified Early Warning Score (MEWS) was developed to timely recognise clinically deteriorating hospitalised patients. However, the ability of the MEWS in predicting serious adverse events (SAEs) in a general hospital population has not been examined prospectively. The aims were to (1) analyse protocol adherence to a MEWS protocol in a real-life setting and (2) to determine the predictive value of protocolised daily MEWS measurement on SAEs: death, cardiac arrests, ICU-admissions and readmissions. METHODS: All adult patients admitted to 6 hospital wards in October and November 2015 were included. MEWS were checked each morning by the research team. For each critical score (MEWS ≥ 3), the clinical staff was inquired about the actions performed. 30-day follow-up for SAEs was performed to compare between patients with and without a critical score. RESULTS: 1053 patients with 3673 vital parameter measurements were included, 200 (19.0%) had a critical score. The protocol adherence was 89.0%. 18.2% of MEWS were calculated wrongly. Patients with critical scores had significant higher rates of unplanned ICU admissions [7.0% vs 1.3%, p < 0.001], in-hospital mortality [6.0% vs 0.8%, p < 0.001], 30-day readmission rates [18.6% vs 10.8%, p < 0.05], and a longer length of stay [15.65 (SD: 15.7 days) vs 6.09 (SD: 6.9), p < 0.001]. Specificity of MEWS related to composite adverse events was 83% with a negative predicting value of 98.1%. CONCLUSIONS: Protocol adherence was high, even though one-third of the critical scores were calculated wrongly. Patients with a MEWS ≥ 3 experienced significantly more adverse events. The negative predictive value of early morning MEWS < 3 was 98.1%, indicating the reliability of this score as a screening tool.

Delayed Recognition of Deterioration of Patients in General Wards Is Mostly Caused by Human Related Monitoring Failures: A Root Cause Analysis of Unplanned ICU Admissions.

BACKGROUND: An unplanned ICU admission of an inpatient is a serious adverse event (SAE). So far, no in depth-study has been performed to systematically analyse the root causes of unplanned ICU-admissions. The primary aim of this study was to identify the healthcare worker-, organisational-, technical,- disease- and patient- related causes that contribute to acute unplanned ICU admissions from general wards using a Root-Cause Analysis Tool called PRISMA-medical. Although a Track and Trigger System (MEWS) was introduced in our hospital a few years ago, it was implemented without a clear protocol. Therefore, the secondary aim was to assess the adherence to a Track and Trigger system to identify deterioration on general hospital wards in patients eventually transferred to the ICU. METHODS: Retrospective observational study in 49 consecutive adult patients acutely admitted to the Intensive Care Unit from a general nursing ward. 1. PRISMA-analysis on root causes of unplanned ICU admissions 2. Assessment of protocol adherence to the early warning score system. RESULTS: Out of 49 cases, 156 root causes were identified. The most frequent root causes were healthcare worker related (46%), which were mainly failures in monitoring the patient. They were followed by disease-related (45%), patient-related causes (7, 5%), and organisational root causes (3%). In only 40% of the patients vital parameters were monitored as was instructed by the doctor. 477 vital parameter sets were found in the 48 hours before ICU admission, in only 1% a correct MEWS was explicitly documented in the record. CONCLUSIONS: This in-depth analysis demonstrates that almost half of the unplanned ICU admissions from the general ward had healthcare worker related root causes, mostly due to monitoring failures in clinically deteriorating patients. In order to reduce unplanned ICU admissions, improving the monitoring of patients is therefore warranted.

Standardized measurement of the Modified Early Warning Score results in enhanced implementation of a Rapid Response System: a quasi-experimental study.

PURPOSE: To study the effect of protocolized measurement (three times daily) of the Modified Early Warning Score (MEWS) versus measurement on indication on the degree of implementation of the Rapid Response System (RRS). METHODS: A quasi-experimental study was conducted in a University Hospital in Amsterdam between September and November 2011. Patients who were admitted for at least one overnight stay were included. Wards were randomized to measure the MEWS three times daily ("protocolized") versus measuring the MEWS "when clinically indicated" in the control group. At the end of each month, for an entire seven-day week, all vital signs recorded for patients were registered. The outcomes were categorized into process measures including the degree of implementation and compliance to set monitoring standards and secondly, outcomes such as the degree of delay in physician notification and Rapid Response Team (RRT) activation in patients with raised MEWS (MEWS≥3). RESULTS: MEWS calculations from vital signs occurred in 70% (2513/3585) on the protocolized wards versus 2% (65/3013) in the control group. Compliance with the protocolized regime was presents in 68% (819/1205), compliance in the control group was present in 4% (47/1232) of the measurements. There were 90 calls to primary physicians on the protocolized and 9 calls on the control wards. Additionally on protocolized wards, there were twice as much RRT calls per admission. CONCLUSIONS: Vital signs and MEWS determination three times daily, results in better detection of physiological abnormalities and more reliable activations of the RRT.

Identification of deteriorating patients on general wards; measurement of vital parameters and potential effectiveness of the Modified Early Warning Score.

BACKGROUND AND PURPOSE: Clear and detectable signs of deterioration have been shown to be present in many patients multiple hours before undergoing a serious life-threatening event. To date, few studies are available describing normal practice and the possible effectiveness of structured tools regarding recognition of deteriorating patients. The aim of this study was to describe the current practice in measurement and documentation of vital signs and the possible usefulness of the Modified Early Warning Score (MEWS) to identify deteriorating patients on hospital wards. METHODS: A retrospective observational study of medical and surgical patients from 2007 with a severe adverse event including cardiopulmonary arrest, unplanned intensive care unit admission, emergency surgery, or unexpected death was performed. We studied all vital parameters that were collected and documented in the 48 hours before these events, and the MEWS was retrospectively calculated. RESULTS: Two hundred four patients were included. In the 48 hours before the event, a total of 2688 measurements of one or more vital signs were taken. Overall, 81% of the patients had an MEWS value of 3 or more at least once during the 48 hours before their event. Recordings of vital signs were mostly incomplete. Even when the MEWS was 3 or more, respiratory rate, diuresis, and oxygen saturation were documented in only 30% to 66% of assessments.

Measuring adherence among nurses one year after training in applying the Modified Early Warning Score and Situation-Background-Assessment-Recommendation instruments.

BACKGROUND: Patients with a cardiac arrest or unplanned intensive care admission show gradual decline in clinical condition preceding the event. This can be objectified by measuring the vital parameters and subsequently determining the Modified Early Warning Score (MEWS). Contact with the physician by nurses may be structured using the Situation-Background-Assessment-Recommendation (SBAR) communication instrument. The aim of our study was to evaluate whether nurses trained in the use of MEWS and SBAR tools were more likely to recognize a deteriorating patient. DESIGN AND SETTING: This prospective quasi-experimental trial in the Academic Medical Center in Amsterdam, the Netherlands included three medical and three surgical wards. INTERVENTIONS: A group of 47 trained and 48 non-trained nurses were presented with a case of a deteriorating patient, and subsequent assessment and actions regarding the patient case were measured. RESULTS: Of the trained nurses, 77% versus 58% of the non-trained group assessed the patient immediately. On subsequent assessment of the patient, respiratory rate was measured twice as frequently (53% trained versus 25% non-trained, p=0.025). No differences were found in the measurement of other vital parameters. The MEWS was determined by 11% of trained nurses. Subsequent notification of the physician was performed by 67% of the trained versus 43% of the non-trained nurses. The SBAR communication tool was used by only one nurse. CONCLUSIONS: Trained nurses are able to identify a deteriorating patient and react more appropriately. However, despite rigorously implementing MEWS/SBAR methodology, these tools were rarely used.

Prognostic models for predicting mortality in elderly ICU patients: a systematic review.

PURPOSE: To systematically review prognostic research literature on development and/or validation of mortality predictive models in elderly patients. METHODS: We searched the Scopus database until June 2010 for articles aimed at validating prognostic models for survival or mortality in elderly intensive care unit (ICU) patients. We assessed the models' fitness for their intended purpose on the basis of barriers for use reported in the literature, using the following categories: (1) clinical credibility, (2) methodological quality (based on an existing quality assessment framework), (3) external validity, (4) model performance, and (5) clinical effectiveness. RESULTS: Seven studies were identified which met our inclusion criteria, one of which was an external validation study. In total, 17 models were found of which six were developed for the general adult ICU population and eleven specifically for elderly patients. Cohorts ranged from 148 to 12,993 patients and only smaller ones were obtained prospectively. The area under the receiver operating characteristic curve (AUC) was most commonly used to measure performance (range 0.71-0.88). The median number of criteria met for clinical credibility was 4.5 out of 7 (range 2.5-5.5) and 17 out of 20 for methodological quality (range 15-20). CONCLUSIONS: Although the models scored relatively well on methodological quality, none of them can be currently considered sufficiently credible or valid to be applicable in clinical practice for elderly patients. Future research should focus on external validation, addressing performance measures relevant for their intended use, and on clinical credibility including the incorporation of factors specific for the elderly population.

Characterizing predictive models of mortality for older adults and their validation for use in clinical practice.

OBJECTIVES: To systematically identify and characterize prognostic models of mortality for older adults, their reported potential use, and the actual level of their (external) validity. DESIGN: The Scopus database until January 2010 was searched for articles that developed and validated new models or validated existing prognostic models of mortality or survival in older adults. SETTING: All domains of health care. PARTICIPANTS: Adults aged 50 and older. MEASUREMENTS: Study and model characteristics were summarized, including the model's development method and degree of validation, data types used, and outcomes. RESULTS: One hundred three articles describing 193 models in 10 domains and mostly originating from the United States were included. These domains were mostly secondary or tertiary care settings (54%) such as intensive care (7%) or geriatric units (8%). Half of the studies (50%) were not disease specific. Heart failure-related diseases (9%) and pneumonia (9%) constituted the major disease-specific subgroups. Most studies (67%) reported support of clinical individual (treatment) decisions as use of prognostic models, but only 34% were externally validated, and only four models (2%) were validated in more than two studies. Most studies (68%) developed at least one new model, but they did not often go beyond addressing their apparent validation (49%). CONCLUSION: Although prognostic models are regularly developed to support clinical individual decisions and could be useful for this purpose, their use is premature. Because clinical credibility and evidence of external validity build trust in prognostic models, both require much more consideration to enhance model acceptance in the future.