Variations in Case-Mix-Adjusted Duration of Mechanical Ventilation Among ICUs.

OBJECTIVES: To develop a model that predicts the duration of mechanical ventilation and then to use this model to compare observed versus expected duration of mechanical ventilation across ICUs. DESIGN: Retrospective cohort analysis. SETTING: Eighty-six eligible ICUs at 48 U.S. hospitals. PATIENTS: ICU patients receiving mechanical ventilation on day 1 (n = 56,336) admitted from January 2013 to September 2014. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We developed and validated a multivariable logistic regression model for predicting duration of mechanical ventilation using ICU day 1 patient characteristics. Mean observed minus expected duration of mechanical ventilation was then obtained across patients and for each ICU. The accuracy of the model was assessed using R. We defined better performing units as ICUs that had an observed minus expected duration of mechanical ventilation less than -0.5 days and a p value of less than 0.01; and poorer performing units as ICUs with an observed minus expected duration of mechanical ventilation greater than +0.5 days and a p value of less than 0.01. The factors accounting for the majority of the model's explanatory power were diagnosis (71%) and physiologic abnormalities (24%). For individual patients, the difference between observed and mean predicted duration of mechanical ventilation was 3.3 hours (95% CI, 2.8-3.9) with R equal to 21.6%. The mean observed minus expected duration of mechanical ventilation across ICUs was 3.8 hours (95% CI, 2.1-5.5), with R equal to 69.9%. Among the 86 ICUs, 66 (76.7%) had an observed mean mechanical ventilation duration that was within 0.5 days of predicted. Five ICUs had significantly (p < 0.01) poorer performance (observed minus expected duration of mechanical ventilation, > 0.5 d) and 14 ICUs significantly (p < 0.01) better performance (observed minus expected duration of mechanical ventilation, < -0.5 d). CONCLUSIONS: Comparison of observed and case-mix-adjusted predicted duration of mechanical ventilation can accurately assess and compare duration of mechanical ventilation across ICUs, but cannot accurately predict an individual patient's mechanical ventilation duration. There are substantial differences in duration of mechanical ventilation across ICU and their association with unit practices and processes of care warrants examination.

Comparing observed and predicted mortality among ICUs using different prognostic systems: why do performance assessments differ?

OBJECTIVES: To compare ICU performance using standardized mortality ratios generated by the Acute Physiology and Chronic Health Evaluation IVa and a National Quality Forum-endorsed methodology and examine potential reasons for model-based standardized mortality ratio differences. DESIGN: Retrospective analysis of day 1 hospital mortality predictions at the ICU level using Acute Physiology and Chronic Health Evaluation IVa and National Quality Forum models on the same patient cohort. SETTING: Forty-seven ICUs at 36 U.S. hospitals from January 2008 to May 2013. PATIENTS: Eighty-nine thousand three hundred fifty-three consecutive unselected ICU admissions. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We assessed standardized mortality ratios for each ICU using data for patients eligible for Acute Physiology and Chronic Health Evaluation IVa and National Quality Forum predictions in order to compare unit-level model performance, differences in ICU rankings, and how case-mix adjustment might explain standardized mortality ratio differences. Hospital mortality was 11.5%. Overall standardized mortality ratio was 0.89 using Acute Physiology and Chronic Health Evaluation IVa and 1.07 using National Quality Forum, the latter having a widely dispersed and multimodal standardized mortality ratio distribution. Model exclusion criteria eliminated mortality predictions for 10.6% of patients for Acute Physiology and Chronic Health Evaluation IVa and 27.9% for National Quality Forum. The two models agreed on the significance and direction of standardized mortality ratio only 45% of the time. Four ICUs had standardized mortality ratios significantly less than 1.0 using Acute Physiology and Chronic Health Evaluation IVa, but significantly greater than 1.0 using National Quality Forum. Two ICUs had standardized mortality ratios exceeding 1.75 using National Quality Forum, but nonsignificant performance using Acute Physiology and Chronic Health Evaluation IVa. Stratification by patient and institutional characteristics indicated that units caring for more severely ill patients and those with a higher percentage of patients on mechanical ventilation had the most discordant standardized mortality ratios between the two predictive models. CONCLUSIONS: Acute Physiology and Chronic Health Evaluation IVa and National Quality Forum models yield different ICU performance assessments due to differences in case-mix adjustment. Given the growing role of outcomes in driving prospective payment patient referral and public reporting, performance should be assessed by models with fewer exclusions, superior accuracy, and better case-mix adjustment.

Comparison of the Mortality Probability Admission Model III, National Quality Forum, and Acute Physiology and Chronic Health Evaluation IV hospital mortality models: implications for national benchmarking*.

OBJECTIVE: To examine the accuracy of the original Mortality Probability Admission Model III, ICU Outcomes Model/National Quality Forum modification of Mortality Probability Admission Model III, and Acute Physiology and Chronic Health Evaluation IVa models for comparing observed and risk-adjusted hospital mortality predictions. DESIGN: Retrospective paired analyses of day 1 hospital mortality predictions using three prognostic models. SETTING: Fifty-five ICUs at 38 U.S. hospitals from January 2008 to December 2012. PATIENTS: Among 174,001 intensive care admissions, 109,926 met model inclusion criteria and 55,304 had data for mortality prediction using all three models. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We compared patient exclusions and the discrimination, calibration, and accuracy for each model. Acute Physiology and Chronic Health Evaluation IVa excluded 10.7% of all patients, ICU Outcomes Model/National Quality Forum 20.1%, and Mortality Probability Admission Model III 24.1%. Discrimination of Acute Physiology and Chronic Health Evaluation IVa was superior with area under receiver operating curve (0.88) compared with Mortality Probability Admission Model III (0.81) and ICU Outcomes Model/National Quality Forum (0.80). Acute Physiology and Chronic Health Evaluation IVa was better calibrated (lowest Hosmer-Lemeshow statistic). The accuracy of Acute Physiology and Chronic Health Evaluation IVa was superior (adjusted Brier score = 31.0%) to that for Mortality Probability Admission Model III (16.1%) and ICU Outcomes Model/National Quality Forum (17.8%). Compared with observed mortality, Acute Physiology and Chronic Health Evaluation IVa overpredicted mortality by 1.5% and Mortality Probability Admission Model III by 3.1%; ICU Outcomes Model/National Quality Forum underpredicted mortality by 1.2%. Calibration curves showed that Acute Physiology and Chronic Health Evaluation performed well over the entire risk range, unlike the Mortality Probability Admission Model and ICU Outcomes Model/National Quality Forum models. Acute Physiology and Chronic Health Evaluation IVa had better accuracy within patient subgroups and for specific admission diagnoses. CONCLUSIONS: Acute Physiology and Chronic Health Evaluation IVa offered the best discrimination and calibration on a large common dataset and excluded fewer patients than Mortality Probability Admission Model III or ICU Outcomes Model/National Quality Forum. The choice of ICU performance benchmarks should be based on a comparison of model accuracy using data for identical patients.

A history of outcome prediction in the ICU.

PURPOSE OF REVIEW: There are few first-hand accounts that describe the history of outcome prediction in critical care. This review summarizes the authors' personal perspectives about the development and evolution of Acute Physiology and Chronic Health Evaluation over the past 35 years. RECENT FINDINGS: We emphasize what we have learned in the past and more recently our perspectives about the current status of outcome prediction, and speculate about the future of outcome prediction. SUMMARY: There is increasing evidence that superior accuracy in outcome prediction requires complex modeling with detailed adjustment for diagnosis and physiologic abnormalities. Thus, an automated electronic system is recommended for gathering data and generating predictions. Support, either public or private, is required to assist users and to update and improve models. Current outcome prediction models have increasingly focused on benchmarks for resource use, a trend that seems likely to increase in the future.

The association between ICU readmission rate and patient outcomes.

OBJECTIVE: To examine the association between ICU readmission rates and case-mix-adjusted outcomes. DESIGN: Retrospective cohort study of ICU admissions from 2002 to 2010. SETTING: One hundred five ICUs at 46 United States hospitals. PATIENTS: Of 369,129 admissions, 263,082 were first admissions that were alive at ICU discharge and candidates for readmission. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The median unit readmission rate was 5.9% (intraquartile range 5.1%-7.0%). Across all admissions, hospital mortality for patients with and without readmission was 21.3% vs. 3.6%, mean ICU stay 4.9 days vs. 3.4 days, and hospital stay 13.3 days vs. 4.5 days, respectively. We stratified ICUs according to their readmission rate: high (>7%), moderate (5%-7%), and low (<5%) rates. Observed and case-mix-adjusted hospital mortality, ICU and hospital lengths of stay were examined by readmission rate strata. Observed outcomes were much worse in the high readmission rate units. But after adjusting for patient and institutional differences, there was no association between level of unit readmission rate and case-mix-adjusted mortality. The difference between observed and predicted mortality was -0.4%, 0.4%, and -1.1%, for the high, medium, and low readmission rate strata, respectively. Additionally, the difference between observed and expected ICU length of stay was approximately zero for the three strata. CONCLUSIONS: Patients readmitted to ICUs have increased hospital mortality and lengths of stay. After case-mix adjustment, there were no significant differences in standardized mortality or case-mix-adjusted lengths of stay between units with high readmission rates compared to units with moderate or low rates. The use of readmission as a quality measure should only be implemented if patient case-mix is taken into account.

Changes in hospital mortality for United States intensive care unit admissions from 1988 to 2012.

INTRODUCTION: A decrease in disease-specific mortality over the last twenty years has been reported for patients admitted to United States (US) hospitals, but data for intensive care patients are lacking. The aim of this study was to describe changes in hospital mortality and case-mix using clinical data for patients admitted to multiple US ICUs over the last 24 years. METHODS: We carried out a retrospective time series analysis of hospital mortality using clinical data collected from 1988 to 2012. We also examined the impact of ICU admission diagnosis and other clinical characteristics on mortality over time. The potential impact of hospital discharge destination on mortality was also assessed using data from 2001 to 2012. RESULTS: For 482,601 ICU admissions there was a 35% relative decrease in mortality from 1988 to 2012 despite an increase in age and severity of illness. This decrease varied greatly by diagnosis. Mortality fell by >60% for patients with chronic obstructive pulmonary disease, seizures and surgery for aortic dissection and subarachnoid hemorrhage. Mortality fell by 51% to 59% for six diagnoses, 41% to 50% for seven diagnoses, and 10% to 40% for seven diagnoses. The decrease in mortality from 2001 to 2012 was accompanied by an increase in discharge to post-acute care facilities and a decrease in discharge to home. CONCLUSIONS: Hospital mortality for patients admitted to US ICUs has decreased significantly over the past two decades despite an increase in the severity of illness. Decreases in mortality were diagnosis specific and appear attributable to improvements in the quality of care, but changes in discharge destination and other confounders may also be responsible.

Intensive care unit readmissions in U.S. hospitals: patient characteristics, risk factors, and outcomes.

OBJECTIVE: To examine which patient characteristics increase the risk for intensive care unit readmission and assess the association of readmission with case-mix adjusted mortality and resource use. DESIGN: : Retrospective cohort study. SETTING: Ninety-seven intensive and cardiac care units at 35 hospitals in the United States. PATIENTS: A total of 229,375 initial intensive care unit admissions during 2001 through 2009 who met inclusion criteria. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: For patients who were discharged alive and candidates for readmission, we compared the characteristics of those with and without a readmission. A multivariable logistic regression analysis was used to identify potential patient-level characteristics that increase the risk for subsequent readmission. We also evaluated case-mix adjusted outcomes by comparing observed and predicted values of mortality and length of stay for patients with and without intensive care unit readmission. Among 229,375 first admissions that met inclusion criteria, 13,980 (6.1%) were eventually readmitted. Risk factors associated with the highest multivariate odds ratio for unit readmission included location before intensive care unit admission, age, comorbid conditions, diagnosis, intensive care unit length of stay, physiologic abnormalities at intensive care discharge, and discharge to a step-down unit. After adjustment for risk factors, patients who were readmitted had a four-fold greater probability of hospital mortality and a 2.5-fold increase in hospital stay compared to patients without readmission. CONCLUSIONS: Intensive care readmission is associated with patient factors that reflect a greater severity and complexity of illness, resulting in a higher risk for hospital mortality and a longer hospital stay. To improve patient safety, physicians should consider these risk factors when making intensive care discharge decisions. Because intensive care unit readmission correlates with more complex and severe illness, readmission rates require case-mix adjustment before they can be properly interpreted as quality measures.

The relationship between hospital and intensive care unit length of stay.

OBJECTIVES: To assess variations in case-mix-adjusted hospital and intensive care unit length of stay and to examine the relationship between intensive care unit and hospital stay. DESIGN: Retrospective cohort study. SETTING: Sixty-nine intensive and cardiac care units in 23 U.S. hospitals during 2002 to 2008. PATIENTS: Intensive care unit admissions (202,300) who met inclusion criteria. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We obtained hospital and intensive care unit characteristics and patient demographic, clinical, diagnostic, and physiologic variables, mortality, and lengths of stay. We developed and validated a model to assess case-mix-adjusted hospital stay and modified and updated a previously validated model to assess adjusted intensive care unit stay. We used these models to compare observed and expected hospital and intensive care unit stay for each patient by calculating the observed minus expected length of stay. Mean observed intensive care unit stay was 4.33 days and mean predicted intensive care unit stay was 4.09 days (5.9-hr difference); mean observed hospital stay was 9.93 days and mean predicted hospital stay was 9.52 days (9.7-hr difference). Observed minus expected intensive care unit and hospital length of stay were significantly shorter (p < .01) at one intensive care unit and significantly longer (p < .01) at nine intensive care units. There was a correlation between hospital and intensive care unit observed minus expected length of stay across individuals (R2 = .40), which was much stronger across units (R2 = .76). CONCLUSIONS: Case-mix-adjusted benchmarks for hospital and intensive care unit stays reveal substantial differences in unit efficiency. Hospital and intensive care unit stays are strongly correlated at the patient and unit level, suggesting that there are causal factors common to both.

Institutional variations in frequency of discharge of elderly intensive care survivors to postacute care facilities.

OBJECTIVE: To examine variations in the frequency of discharge of elderly patients to postacute care facilities across multiple intensive care units and identify the influence of institutional and patient factors on the frequency of postacute care discharge. DESIGN: Observational cohort study. SETTING: Consecutive admissions from 65 intensive and coronary care units in 24 US hospitals during 2002-2008. Each hospital had a clinical information system in place. PATIENTS: A total of 13,370 admissions in patients aged≥65 yrs who were alive at hospital discharge and met inclusion criteria. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Demographic, clinical, diagnostic, and physiological variables were obtained on all patients. In addition, information for each hospital and intensive care unit was recorded. Among hospital survivors, 46.2% were discharged to postacute care facilities with a range of 8.8-77.8%. A multivariable logistic regression model was developed that predicted discharge to a postacute care facility. The major variables affecting postacute care discharge were diagnosis, day 5 physiology, and day 5 therapy; these variables accounted for 61% of the model's explanatory power. Patient age, hospital bed size, teaching status, and intensive care unit type also affected postacute care discharge. Physiology and therapy on day 1 had little impact on postacute care discharge. The model accounted for only 31% of the variation in rates across intensive care units, indicating that unmeasured factors play a part in dictating discharge location. CONCLUSION: Discharge of elderly intensive care unit patients to postacute care facilities varies widely by institution. These variations are only partially explained by differences in patient and institutional characteristics and are affected more by diagnosis and physiology on day 5, respectively. Unmeasured factors such as admission from a postacute care facility, postacute care availability, patient preferences, and socioeconomic factors may account for unexplained variations in postacute care discharge.

A predictive model for the early identification of patients at risk for a prolonged intensive care unit length of stay.

BACKGROUND: Patients with a prolonged intensive care unit (ICU) length of stay account for a disproportionate amount of resource use. Early identification of patients at risk for a prolonged length of stay can lead to quality enhancements that reduce ICU stay. This study developed and validated a model that identifies patients at risk for a prolonged ICU stay. METHODS: We performed a retrospective cohort study of 343,555 admissions to 83 ICUs in 31 U.S. hospitals from 2002-2007. We examined the distribution of ICU length of stay to identify a threshold where clinicians might be concerned about a prolonged stay; this resulted in choosing a 5-day cut-point. From patients remaining in the ICU on day 5 we developed a multivariable regression model that predicted remaining ICU stay. Predictor variables included information gathered at admission, day 1, and ICU day 5. Data from 12,640 admissions during 2002-2005 were used to develop the model, and the remaining 12,904 admissions to internally validate the model. Finally, we used data on 11,903 admissions during 2006-2007 to externally validate the model. RESULTS: The variables that had the greatest impact on remaining ICU length of stay were those measured on day 5, not at admission or during day 1. Mechanical ventilation, PaO2: FiO2 ratio, other physiologic components, and sedation on day 5 accounted for 81.6% of the variation in predicted remaining ICU stay. In the external validation set observed ICU stay was 11.99 days and predicted total ICU stay (5 days + day 5 predicted remaining stay) was 11.62 days, a difference of 8.7 hours. For the same patients, the difference between mean observed and mean predicted ICU stay using the APACHE day 1 model was 149.3 hours. The new model's r2 was 20.2% across individuals and 44.3% across units. CONCLUSIONS: A model that uses patient data from ICU days 1 and 5 accurately predicts a prolonged ICU stay. These predictions are more accurate than those based on ICU day 1 data alone. The model can be used to benchmark ICU performance and to alert physicians to explore care alternatives aimed at reducing ICU stay.

Outcome prediction in critical care: the Acute Physiology and Chronic Health Evaluation models.

PURPOSE OF REVIEW: A new generation of predictive models for critically ill patients was described between 2005 and 2008. This review will give details of the latest version of the Acute Physiology and Chronic Health Evaluation (APACHE) predictive models, and discuss it in relation to recent critical care outcome studies. We also compare APACHE IV with other systems and address the issue of model complexity. RECENT FINDINGS: APACHE IV required the remodeling of over 40 equations. These new models calibrate better to contemporary data than older versions of APACHE and there is good predictive accuracy within diagnostic subgroups. Physiology accounts for 66% and diagnosis for 17% of the APACHE IV mortality model's predictive power. Thus, physiology and diagnosis account for 83% of the accuracy of APACHE IV. SUMMARY: Predictive models have a modest window of applicability, and therefore must be revalidated frequently. This was shown to be true for APACHE III, and hence a major reestimation of models was carried out to generate APACHE IV. Although overall model accuracy is important, it is also imperative that predictive models work well within diagnostic subgroups.

Predicting outcomes for cardiac surgery patients after intensive care unit admission.

Most performance assessments of cardiac surgery programs use models based on preoperative risk factors. Models that were primarily developed to assess performance in general intensive care unit (ICU) populations have also been used to evaluate the quality of surgical, anesthetic, and ICU management after cardiac surgery. Although there are currently 5 models for evaluating general ICU populations, only the Acute Physiology and Chronic Health Evaluation (APACHE) system has been independently validated for cardiac surgery patients. This review describes the evolution, rationale, and accuracy of APACHE models that are specific for cardiac surgery patients as well as for patients who have had vascular and thoracic procedures. In addition to performance comparisons based on observed and predicted mortality, APACHE provides similar comparisons of ICU and hospital lengths of stay and duration of mechanical ventilation. However, the low mortality incidence of many cardiac outcomes means that very large numbers of patients must be obtained to get good predictive models. Thus, the equations are not designed for predicting individual patients' outcome but have proven useful in performance comparisons and for quality improvement initiatives.

A multicenter description of intermediate-care patients: comparison with ICU low-risk monitor patients.

STUDY OBJECTIVES: To describe the characteristics and outcomes of patients admitted to intermediate-care areas (ICAs) and to compare them with those of ICU patients who receive monitoring only on day 1 and are at a low risk (i.e., < 10%) for receiving subsequent active life-supporting therapy (i.e., low-risk monitor patients). DESIGN: Nonrandomized, retrospective, cohort study. SETTING: Thirteen US teaching hospitals and 19 nonteaching hospitals. PATIENTS: A consecutive sample of 8,971 patients at 37 ICAs and 5,116 low-risk (i.e., < 10%) monitor patients at 59 ICUs in 32 US hospitals. INTERVENTIONS: None. MEASUREMENTS AND RESULTS: We recorded demographic and clinical characteristics, resource use, and outcomes for the ICA and ICU low-risk monitor patients. Patient data and outcomes for this study were collected concurrently or retrospectively. ICA and ICU low-risk monitor patients were similar in regard to gender, race, and frequency of comorbitities, but ICA patients were significantly (p < 0.001) older, had fewer physiologic abnormalities (mean acute physiology score, 16.7 vs 19.8, respectively), and were more frequently admitted due to nonoperative diagnoses. The mean length of stay for ICA patients was significantly longer (3.9 days) than for ICU low-risk monitor patients (2.6 days; p < 0.001). The hospital mortality rate was significantly higher for ICA patients (3.1%) compared to ICU low-risk monitor patients (2.3%; p = 0.002). CONCLUSIONS: The clinical features of ICA patients are similar, but not identical to, those of less severely ill ICU monitor patients. Comparisons of hospital death rates and lengths of stay for these patients should be adjusted for characteristics that previously have been shown to influence these outcomes.

The use of benchmarking to identify top performing critical care units: a preliminary assessment of their policies and practices.

PURPOSE: To describe the policies and practices of intensive care units (ICUs) with good patient survival and highly efficient resource use and to identify relevant variables for future investigation. MATERIALS AND METHODS: We used clinical data for 359,715 patients from 108 ICUs to compare the ratios of actual with Acute Physiology and Chronic Health Evaluation (APACHE) III predicted hospital mortality, ICU and hospital stay, and the proportion of low-risk monitor patients. The best performing ICUs (top 10%) were defined by a mortality ratio of 1.0 or less, and either the lowest ratio for ICU stay, hospital stay, or percentage of low-risk monitor patients. The medical and nursing directors of top performing ICUs completed a questionnaire to describe their unit's structure policies and practices. RESULTS: Among the 108 ICUs, 61 (56%) had a ratio of actual to predicted hospital mortality of 1.0 or less and the best performing units had ICU stay ratios of 0.62 to 0.79, hospital stay ratios of 0.73 to 0.77, and admitted 10% to 38% low-risk monitor patients. ICU structure varied among the best performing ICUs. Units with the shortest ICU and hospital stay had alternatives to intensive care, methods to facilitate patient throughput, used multiple protocols for high-volume diagnoses and care processes, and continuously monitored resource use. Units with the fewest low-risk monitor patients screened potential admissions, had intermediate care areas, extended-stay recovery rooms, and care pathways for high-volume diagnoses. CONCLUSIONS: Benchmarking can be used to identify ICUs with good patient survival and highly efficient resource use. The combination of policies and practices used by these units might improve resource use in other ICUs.

A model for identifying patients who may not need intensive care unit admission.

PURPOSE: This study presents a new model for identifying patients who might be too well to benefit from intensive care unit (ICU) care. PATIENTS AND METHODS: Intensive care unit admissions in 2002 to 2003 were used to develop a model to predict whether patients monitored on day one would receive one or more of 33 subsequent active life-supporting treatments. Accuracy was assessed by testing the model in a subsequent cohort of admissions in 2004 to 2006. We then assessed the frequency of active treatment among monitor patients at a low (<10%) risk for active life-supporting therapy on ICU day 1. RESULTS: Among 28 847 ICU monitor admissions in 2004 to 2006, 3153 patients (11.0%) were predicted to receive active treatment; 3296 (11.5%) actually did. There were 17 720 admissions with a low (<10%) risk for receiving subsequent active life-supporting treatment; 1238 (7.0%) received subsequent active treatment. Hospital mortality (2.5%) and mean ICU stay (1.8 days) suggests that most of these patients did not require ICU care. CONCLUSIONS: The outcome for low-risk monitor patients suggest they may be too well to benefit from intensive care. The frequency of low-risk monitor admissions provides a measure of ICU resource use. Improved resource use and reduced costs might be achieved by strategies to provide care for these patients on floors or intermediate care units.