Association of Time to Rapid Response Team Activation With Patient Outcomes Using a Range of Physiologic Deterioration Thresholds.

Clinical deterioration of hospitalized patients is common and can lead to critical illness and death. Rapid response teams (RRTs) assess and treat high-risk patients with signs of clinical deterioration to prevent further worsening and subsequent adverse outcomes. Whether activation of the RRT early in the course of clinical deterioration impacts outcomes, however, remains unclear. We sought to characterize the relationship between increasing time to RRT activation after physiologic deterioration and short-term patient outcomes. DESIGN: Retrospective multicenter cohort study. SETTING: Three academic hospitals in Pennsylvania. PATIENTS: We included the RRT activation of a hospitalization for non-ICU inpatients greater than or equal to 18 years old. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The primary exposure was time to RRT activation after physiologic deterioration. We selected four Cardiac Arrest Risk Triage (CART) score thresholds a priori from which to measure time to RRT activation (CART score ≥ 12, ≥ 16, ≥ 20, and ≥ 24). The primary outcome was 7-day mortality-death or discharge to hospice care within 7 days of RRT activation. For each CART threshold, we modeled the association of time to RRT activation duration with 7-day mortality using multivariable fractional polynomial regression. Increased time from clinical decompensation to RRT activation was associated with higher risk of 7-day mortality. This relationship was nonlinear, with odds of mortality increasing rapidly as time to RRT activation increased from 0 to 4 hours and then plateauing. This pattern was observed across several thresholds of physiologic derangement. CONCLUSIONS: Increasing time to RRT activation was associated in a nonlinear fashion with increased 7-day mortality. This relationship appeared most marked when using a CART score greater than 20 threshold from which to measure time to RRT activation. We suggest that these empirical findings could be used to inform RRT delay definitions in further studies to determine the clinical impact of interventions focused on timely RRT activation.

Defining Physiological Decompensation: An Expert Consensus and Retrospective Outcome Validation.

OBJECTIVES: Physiological decompensation of hospitalized patients is common and is associated with substantial morbidity and mortality. Research surrounding patient decompensation has been hampered by the absence of a robust definition of decompensation and lack of standardized clinical criteria with which to identify patients who have decompensated. We aimed to: 1) develop a consensus definition of physiological decompensation and 2) to develop clinical criteria to identify patients who have decompensated. DESIGN: We utilized a three-phase, modified electronic Delphi (eDelphi) process, followed by a discussion round to generate consensus on the definition of physiological decompensation and on criteria to identify decompensation. We then validated the criteria using a retrospective cohort study of adult patients admitted to the Hospital of the University of Pennsylvania. SETTING: Quaternary academic medical center. PATIENTS: Adult patients admitted to the Hospital of the University of Pennsylvania who had triggered a rapid response team (RRT) response between January 1, 2019, and December 31, 2020. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Sixty-nine experts participated in the eDelphi. Participation was high across the three survey rounds (first round: 93%, second round: 94%, and third round: 98%). The expert panel arrived at a consensus definition of physiological decompensation, "An acute worsening of a patient's clinical status that poses a substantial increase to an individual's short-term risk of death or serious harm." Consensus was also reached on criteria for physiological decompensation. Invasive mechanical ventilation, severe hypoxemia, and use of vasopressor or inotrope medication were bundled as criteria for our novel decompensation metric: the adult inpatient decompensation event (AIDE). Patients who met greater than one AIDE criteria within 24 hours of an RRT call had increased adjusted odds of 7-day mortality (adjusted odds ratio [aOR], 4.1 [95% CI, 2.5-6.7]) and intensive care unit transfer (aOR, 20.6 [95% CI, 14.2-30.0]). CONCLUSIONS: Through the eDelphi process, we have reached a consensus definition of physiological decompensation and proposed clinical criteria with which to identify patients who have decompensated using data easily available from the electronic medical record, the AIDE criteria.

A Simulated Prospective Evaluation of a Deep Learning Model for Real-Time Prediction of Clinical Deterioration Among Ward Patients.

OBJECTIVES: The National Early Warning Score, Modified Early Warning Score, and quick Sepsis-related Organ Failure Assessment can predict clinical deterioration. These scores exhibit only moderate performance and are often evaluated using aggregated measures over time. A simulated prospective validation strategy that assesses multiple predictions per patient-day would provide the best pragmatic evaluation. We developed a deep recurrent neural network deterioration model and conducted a simulated prospective evaluation. DESIGN: Retrospective cohort study. SETTING: Four hospitals in Pennsylvania. PATIENTS: Inpatient adults discharged between July 1, 2017, and June 30, 2019. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We trained a deep recurrent neural network and logistic regression model using data from electronic health records to predict hourly the 24-hour composite outcome of transfer to ICU or death. We analyzed 146,446 hospitalizations with 16.75 million patient-hours. The hourly event rate was 1.6% (12,842 transfers or deaths, corresponding to 260,295 patient-hours within the predictive horizon). On a hold-out dataset, the deep recurrent neural network achieved an area under the precision-recall curve of 0.042 (95% CI, 0.04-0.043), comparable with logistic regression model (0.043; 95% CI 0.041 to 0.045), and outperformed National Early Warning Score (0.034; 95% CI, 0.032-0.035), Modified Early Warning Score (0.028; 95% CI, 0.027- 0.03), and quick Sepsis-related Organ Failure Assessment (0.021; 95% CI, 0.021-0.022). For a fixed sensitivity of 50%, the deep recurrent neural network achieved a positive predictive value of 3.4% (95% CI, 3.4-3.5) and outperformed logistic regression model (3.1%; 95% CI 3.1-3.2), National Early Warning Score (2.0%; 95% CI, 2.0-2.0), Modified Early Warning Score (1.5%; 95% CI, 1.5-1.5), and quick Sepsis-related Organ Failure Assessment (1.5%; 95% CI, 1.5-1.5). CONCLUSIONS: Commonly used early warning scores for clinical decompensation, along with a logistic regression model and a deep recurrent neural network model, show very poor performance characteristics when assessed using a simulated prospective validation. None of these models may be suitable for real-time deployment.

Assessing the Course of Organ Dysfunction Using Joint Longitudinal and Time-to-Event Modeling in the Vasopressin and Septic Shock Trial.

Non-mortality septic shock outcomes (e.g., Sequential Organ Failure Assessment score) are important clinical endpoints in pivotal sepsis trials. However, comparisons of observed longitudinal non-mortality outcomes between study groups can be biased if death is unequal between study groups or is associated with an intervention (i.e., informative censoring). We compared the effects of vasopressin versus norepinephrine on the Sequential Organ Failure Assessment score in the Vasopressin and Septic Shock Trial to illustrate the use of joint modeling to help minimize potential bias from informative censoring. DESIGN: Secondary analysis of the Vasopressin and Septic Shock Trial data. SETTING: Twenty-seven ICUs in Canada, Australia, and United States. SUBJECTS: Seven hundred sixty-three participants with septic shock who received blinded vasopressin (n = 389) or norepinephrine infusions (n = 374). MEASUREMENTS AND MAIN RESULTS: Sequential Organ Failure Assessment scores were calculated daily until discharge, death, or day 28 after randomization. Mortality was numerically higher in the norepinephrine arm (28 d mortality of 39% vs 35%; p = 0.25), and there was a positive association between higher Sequential Organ Failure Assessment scores and patient mortality, characteristics that suggest a potential for bias from informative censoring of Sequential Organ Failure Assessment scores by death. The best-fitting joint longitudinal (i.e., linear mixed-effects model) and survival (i.e., Cox proportional hazards model for the time-to-death) model showed that norepinephrine was associated with a more rapid improvement in the total Sequential Organ Failure Assessment score through day 4, and then the daily Sequential Organ Failure Assessment scores converged and overlapped for the remainder of the study period. CONCLUSIONS: Short-term reversal of organ dysfunction occurred more rapidly with norepinephrine compared with vasopressin, although differences between study arms did not persist after day 4. Joint models are an accessible methodology that could be used in critical care trials to assess the effects of interventions on the longitudinal progression of key outcomes (e.g., organ dysfunction, biomarkers, or quality of life) that may be informatively truncated by death or other censoring events.

Inclusion of Unstructured Clinical Text Improves Early Prediction of Death or Prolonged ICU Stay.

OBJECTIVES: Early prediction of undesired outcomes among newly hospitalized patients could improve patient triage and prompt conversations about patients' goals of care. We evaluated the performance of logistic regression, gradient boosting machine, random forest, and elastic net regression models, with and without unstructured clinical text data, to predict a binary composite outcome of in-hospital death or ICU length of stay greater than or equal to 7 days using data from the first 48 hours of hospitalization. DESIGN: Retrospective cohort study with split sampling for model training and testing. SETTING: A single urban academic hospital. PATIENTS: All hospitalized patients who required ICU care at the Beth Israel Deaconess Medical Center in Boston, MA, from 2001 to 2012. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Among eligible 25,947 hospital admissions, we observed 5,504 (21.2%) in which patients died or had ICU length of stay greater than or equal to 7 days. The gradient boosting machine model had the highest discrimination without (area under the receiver operating characteristic curve, 0.83; 95% CI, 0.81-0.84) and with (area under the receiver operating characteristic curve, 0.89; 95% CI, 0.88-0.90) text-derived variables. Both gradient boosting machines and random forests outperformed logistic regression without text data (p < 0.001), whereas all models outperformed logistic regression with text data (p < 0.02). The inclusion of text data increased the discrimination of all four model types (p < 0.001). Among those models using text data, the increasing presence of terms "intubated" and "poor prognosis" were positively associated with mortality and ICU length of stay, whereas the term "extubated" was inversely associated with them. CONCLUSIONS: Variables extracted from unstructured clinical text from the first 48 hours of hospital admission using natural language processing techniques significantly improved the abilities of logistic regression and other machine learning models to predict which patients died or had long ICU stays. Learning health systems may adapt such models using open-source approaches to capture local variation in care patterns.

Validation of an Administrative Definition of ICU Admission Using Revenue Center Codes.

OBJECTIVES: Describe the operating characteristics of a proposed set of revenue center codes to correctly identify ICU stays among hospitalized patients. DESIGN: Retrospective cohort study. We report the operating characteristics of all ICU-related revenue center codes for intensive and coronary care, excluding nursery, intermediate, and incremental care, to identify ICU stays. We use a classification and regression tree model to further refine identification of ICU stays using administrative data. The gold standard for classifying ICU admission was an electronic patient location tracking system. SETTING: The University of Pennsylvania Health System in Philadelphia, PA, United States. PATIENTS: All adult inpatient hospital admissions between July 1, 2013, and June 30, 2015. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Among 127,680 hospital admissions, the proposed combination of revenue center codes had 94.6% sensitivity (95% CI, 94.3-94.9%) and 96.1% specificity (95% CI, 96.0-96.3%) for correctly identifying hospital admissions with an ICU stay. The classification and regression tree algorithm had 92.3% sensitivity (95% CI, 91.6-93.1%) and 97.4% specificity (95% CI, 97.2-97.6%), with an overall improved accuracy (χ = 398; p < 0.001). CONCLUSIONS: Use of the proposed combination of revenue center codes has excellent sensitivity and specificity for identifying true ICU admission. A classification and regression tree algorithm with additional administrative variables offers further improvements to accuracy.