Screening inpatient quality using post-discharge events.

BACKGROUND: Decreasing hospital lengths of stay (LOS) hamper efforts to detect and to definitively treat complications of care. Patients leave before some complications are identified. OBJECTIVES: To develop a computerized method to screen for hospital complications using readily available administrative data from outpatient and nonacute care within 90 days of discharge. DESIGN: We developed the Complications Screening Program for Outpatient data (CSP-O) by using diagnosis and procedure codes from Medicare Part A and B claims to define 50 complication screens. Seventeen apply to specific procedural cases, and 33 apply to all adult, acute, medical, or surgical hospitalizations. The CSP-O algorithm examined outpatient, physician office, home health agency, and hospice claims within 90 days following discharge. SUBJECTS: Seven hundred thirty nine thousand, two hundred and forty eight discharges of Medicare beneficiaries (age range, > or = 65 years) were admitted to 515 hospitals nationwide in 1994. RESULTS: Complete 90-day, post-discharge windows were present for 62.8% of all and 68.5% of procedural cases. The 33 general screens flagged 13.6% of all cases; only 1.8% of procedural cases were flagged by the 17 procedural screens. When we allowed the CSP-O algorithm to scan information from acute hospital readmissions, flag rates rose to 32.8% for general and 8.7% for procedural complications. Controlling for patient and hospital characteristics, flag rates were considerably higher among the very old and at small and for-profit institutions. CONCLUSIONS: Whereas several CSP-O findings have construct validity, limitations of claims raise concerns. Regardless of the CSPO's ultimate utility, examining post-discharge experiences to identify inpatient complications remains important as LOSs fall.

Predicting in-hospital deaths from coronary artery bypass graft surgery. Do different severity measures give different predictions?

OBJECTIVES: Severity-adjusted death rates for coronary artery bypass graft (CABG) surgery by provider are published throughout the country. Whether five severity measures rated severity differently for identical patients was examined in this study. METHODS: Two severity measures rate patients using clinical data taken from the first two hospital days (MedisGroups, physiology scores); three use diagnoses and other information coded on standard, computerized hospital discharge abstracts (Disease Staging, Patient Management Categories, all patient refined diagnosis related groups). The database contained 7,764 coronary artery bypass graft patients from 38 hospitals with 3.2% in-hospital deaths. Logistic regression was performed to predict deaths from age, age squared, sex, and severity scores, and c statistics from these regressions were used to indicate model discrimination. Odds ratios of death predicted by different severity measures were compared. RESULTS: Code-based measures had better c statistics than clinical measures: all patient refined diagnosis related groups, c = 0.83 (95% C.I. 0.81, 0.86) versus MedisGroups, c = 0.73 (95% C.I. 0.70, 0.76). Code-based measures predicted very different odds of dying than clinical measures for more than 30% of patients. Diagnosis codes indicting postoperative, life-threatening conditions may contribute to the superior predictive power of code-based measures. CONCLUSIONS: Clinical and code-based severity measures predicted different odds of dying for many coronary artery bypass graft patients. Although code-based measures had better statistical performance, this may reflect their reliance on diagnosis codes for life-threatening conditions occurring late in the hospitalization, possibly as complications of care. This compromises their utility for drawing inferences about quality of care based on severity-adjusted coronary artery bypass graft death rates.

Differences in procedure use, in-hospital mortality, and illness severity by gender for acute myocardial infarction patients: are answers affected by data source and severity measure?

OBJECTIVES: According to some studies, women with heart disease receive fewer procedures and have higher in-hospital death rates than men. These studies vary by data source (hospital discharge abstract versus detailed clinical information) and severity measurement methods. The authors examined whether evaluations of gender differences for acute myocardial infarction patients vary by data source and severity measure. METHODS: The authors considered 10 severity measures: four using clinical medical record data and six using discharge abstracts (diagnosis and procedure codes). The authors studied all 14,083 patients admitted in 1991 for acute myocardial infarction to 100 hospitals nationwide, examining in-hospital death and use of coronary angiography, coronary artery bypass graft surgery (CABG), and percutaneous transluminal coronary angioplasty (PTCA). Logistic regression was used to calculate odds ratios for death and procedure use for women compared with men, controlling for age and each of the severity scores. RESULTS: After adjusting only for age, women were significantly more likely than men to die and less likely to receive CABG and coronary angiography. Severity measures provided different assessments of whether women were sicker than men; for all cases, clinical data-based MedisGroups rated women's severity compared with men's, whereas four code-based severity measures viewed women as sicker. After adjusting for severity and age, women were significantly more likely than men to die in-hospital and less likely to receive coronary angiography and CABG; women and men had relatively equal adjusted odds ratios of receiving PTCA. Odds ratios reflecting gender differences in procedure use and death rates were similar across severity measures. CONCLUSIONS: Comparisons of severity-adjusted in-hospital death rates and invasive procedure use between men and women yielded similar findings regardless of data source and severity measure.

Judging hospitals by severity-adjusted mortality rates: the influence of the severity-adjustment method.

OBJECTIVES: This research examined whether judgments about a hospital's risk-adjusted mortality performance are affected by the severity-adjustment method. METHODS: Data came from 100 acute care hospitals nationwide and 11880 adults admitted in 1991 for acute myocardial infarction. Ten severity measures were used in separate multivariable logistic models predicting in-hospital death. Observed-to-expected death rates and z scores were calculated with each severity measure for each hospital. RESULTS: Unadjusted mortality rates for the 100 hospitals ranged from 4.8% to 26.4%. For 32 hospitals, observed mortality rates differed significantly from expected rates for 1 or more, but not for all 10, severity measures. Agreement between pairs of severity measures on whether hospitals were flagged as statistical mortality outliers ranged from fair to good. Severity measures based on medical records frequently disagreed with measures based on discharge abstracts. CONCLUSIONS: Although the 10 severity measures agreed about relative hospital performance more often than would be expected by chance, assessments of individual hospital mortality rates varied by different severity-adjustment methods.

Do severity measures explain differences in length of hospital stay? The case of hip fracture.

OBJECTIVE: To examine whether judgments about hospital length of stay (LOS) vary depending on the measure used to adjust for severity differences. DATA SOURCES/STUDY SETTING: Data on admissions to 80 hospitals nationwide in the 1992 MedisGroups Comparative Database. STUDY DESIGN: For each of 14 severity measures, LOS was regressed on patient age/sex, DRG, and severity score. Regressions were performed on trimmed and untrimmed data. R-squared was used to evaluate model performance. For each severity measure for each hospital, we calculated the expected LOS and the z-score, a measure of the deviation of observed from expected LOS. We ranked hospitals by z-scores. DATA EXTRACTION: All patients admitted for initial surgical repair of a hip fracture, defined by DRG, diagnosis, and procedure codes. PRINCIPAL FINDINGS: The 5,664 patients had a mean (s.d.) LOS of 11.9 (8.9) days. Cross-validated R-squared values from the multivariable regressions (trimmed data) ranged from 0.041 (Comorbidity Index) to 0.165 (APR-DRGs). Using untrimmed data, observed average LOS for hospitals ranged from 7.6 to 23.9 days. The 14 severity measures showed excellent agreement in ranking hospitals based on z-scores. No severity measure explained the differences between hospitals with the shortest and longest LOS. CONCLUSIONS: Hospitals differed widely in their mean LOS for hip fracture patients, and severity adjustment did little to explain these differences.

Predicting in-hospital mortality for stroke patients: results differ across severity-measurement methods.

OBJECTIVE: To see whether severity-adjusted predictions of likelihoods of in-hospital death for stroke patients differed among severity measures. METHODS: The study sample was 9,407 stroke patients from 94 hospitals, with 916 (9.7%) in-hospital deaths. Probability of death was calculated for each patient using logistic regression with age-sex and each of five severity measures as the independent variables: admission MedisGroups probability-of-death scores; scores based on 17 physiologic variables on admission; Disease Staging's probability-of-mortality model; the Seventy Score of Patient Management Categories (PMCs); and the All Patient-Refined Diagnosis Groups (APR-DRGs). For each patient, the odds of death predicted by the severity measures were compared. The frequencies of seven clinical indicators of poor prognosis in stroke were examined for patients with very different odds of death predicted by different severity measures. Odds ratios were considered very different when the odds of death predicted by one severity measure was less than 0.5 or greater than 2.0 of that predicted by a second measure. RESULTS: MedisGroups and the physiology scores predicted similar odds of death for 82.2% of the patients. MedisGroups and PMCs disagreed the most, with very different odds predicted for 61.6% of patients. Patients viewed as more severely III by MedisGroups and the physiology score were more likely to have the clinical stroke findings than were patients seen as sicker by the other severity measures. This suggests that MedisGroups and the physiology score are more clinically credible. CONCLUSIONS: Some pairs of severity measures ranked over 60% of patients very differently by predicted probability of death. Studies of severity-adjusted stroke outcomes may produce different results depending on which severity measure is used for risk adjustment.

Does severity explain differences in hospital length of stay for pneumonia patients?

OBJECTIVES: In the USA, the role of patient severity in determining hospital resource use has been questioned since Medicare adopted prospective hospital payment based on diagnosis-related groups (DRGs). Exactly how to measure severity, however, remains unclear. We examined whether assessments of severity-adjusted hospital lengths of stay (LOS) varied when different measures were used for severity adjustment. METHODS: The complete study sample included 18,016 patients receiving medical treatment for pneumonia at 105 acute care hospitals. We studied 11 severity measures, nine based on patient demographic and diagnosis and procedure code information and two derived from clinical findings from the medical record. For each severity measure, LOS was regressed on patient age, sex, DRG, and severity score. Analyses were performed on trimmed and untrimmed data. Trimming eliminated cases with LOS more than three standard deviations from the mean on a log scale. RESULTS: The trimmed data set contained 17,976 admissions with a mean (S.D.) LOS of 8.9 (6.1) days. Average LOS ranged from 5.0-11.8 days among the 105 hospitals. Using trimmed data, the 11 severity measures produced R-squared values ranging from 0.098-0.169 for explaining LOS for individual patients. Across all severity measures, predicted average hospital LOS varied much less than the observed LOS, with predicted mean hospital LOS ranging from about 8.4-9.8 days. DISCUSSION: No severity measure explained the two-fold differences among hospitals in average LOS. Other patient characteristics, practice patterns, or institutional factors may cause the wide differences across hospitals in LOS.

Severity measurement methods and judging hospital death rates for pneumonia.

Payers and policymakers are increasingly examining hospital mortality rates as indicators of hospital quality. To be meaningful, these death rates must be adjusted for patient severity. This research examined whether judgments about an individual hospital's risk-adjusted mortality is affected by the severity adjustment method. Data came from 105 acute care hospitals nationwide that use the Medis-Groups severity measure. The study population was 18,016 adults hospitalized in 1991 for pneumonia. Multivariable logistic models to predict in-hospital death were computed separately for 14 severity methods, controlling for patient age, sex, and diagnosis-related group (DRG). For each hospital, observed-to-expected death rates and z scores were calculated for each severity method. The overall in-hospital death rate was 9.6%. Unadjusted mortality rates for the 105 hospitals ranged from 1.4% to 19.6%. After adjusting for age, sex, DRG, and severity, 73 facilities had observed mortality rates that did not differ significantly from expected rates according to all 14 severity methods; two had rates significantly higher than expected for all 14 severity methods. For 30 hospitals, observed mortality rates differed significantly from expected rates when judged by one or more but not all 14 severity methods. Kappa analysis showed fair to excellent agreement between severity methods. The 14 severity methods agreed about relative hospital performance more often than expected by chance, but perceptions of individual hospitals' mortality rates varied using different severity adjustment methods for almost one third of facilities. Judgments about individual hospital performance using different severity adjustment approaches may reach different conclusions.

Judging hospitals by severity-adjusted mortality rates: the case of CABG surgery.

In many health care marketplaces, outcomes assessment is central to monitoring quality while controlling costs. Comparing outcomes across providers generally requires adjustment for patient severity. For mortality rates and other adverse outcomes comparisons, severity adjustment ideally aims to control for patient characteristics prior to the health care intervention. A variety of severity methodologies, specifically for hospitalized patients, are commercially available. Some have been adopted by state or regional initiatives for publicly comparing hospital outcomes. We applied 14 common severity measures to the same data set to determine whether judgments about risk-adjusted hospital death rates are sensitive to the specific severity method. We examined 7,765 patients undergoing coronary artery bypass graft (CABG) surgery at 38 hospitals. Unadjusted death rates ranged from 0% to 11.2% across hospitals. Comparisons of relative hospital performance were relatively insensitive to the severity adjustment method.

Using severity measures to predict the likelihood of death for pneumonia inpatients.

OBJECTIVE: To see whether predictions of patients, likelihood of dying in-hospital differed among severity methods. DESIGN: Retrospective cohort. PATIENTS: 18,016 persons 18 years of age and older managed medically for pneumonia; 1,732 (9.6%) in-hospital deaths. METHODS: Probability of death was calculated for each patient using logistic regression with age, age squared, sex, and each of five severity measures as the independent variables: 1) admission MedisGroups probability of death scores; 2) scores based on 17 admission physiologic variables; 3) Disease Staging's probability of mortality model; the Severity Score of Patient Management Categories (PMCs); 4) and the All Patient Refined Diagnosis-Related Groups (APR-DRGs). Patients were ranked by calculated probability of death; 5) rankings were compared across severity methods. Frequencies of 14 clinical findings considered poor prognostic indicators in pneumonia were examined for patients ranked differently by different methods. RESULTS: MedisGroups and the physiology score predicted a similar likelihood of death for 89.2% of patients. In contrast, the three code-based severity methods rated over 25% of patients differently by predicted likelihood of death when compared with the rankings of the two clinical data-based methods [MedisGroups and the physiology score]. MedisGroups and the physiology score demonstrated better clinical credibility than the three severity methods based on discharge abstract data. CONCLUSIONS: Some pairs of severity measures ranked over 25% of patients very differently by predicted probability of death. Results of outcomes studies may vary depending on which severity method is used for risk adjustment.

Predicting who dies depends on how severity is measured: implications for evaluating patient outcomes.

OBJECTIVE: To determine whether assessments of illness severity, defined as risk for in-hospital death, varied across four severity measures. DESIGN: Retrospective cohort study. SETTING: 100 hospitals using the MedisGroups severity measure. PATIENTS: 11 880 adults managed medically for acute myocardial infarction; 1574 in-hospital deaths (13.2%). MEASUREMENTS: For each patient, probability of death was predicted four times, each time by using patient age and sex and one of four common severity measures: 1) admission MedisGroups scores for probability of death scores; 2) scores based on values for 17 physiologic variables at time of admission; 3) Disease Staging's probability-of-mortality model; and 4) All Patient Refined Diagnosis Related Groups (APR-DRGs). Patients were ranked according to probability of death as predicted by each severity measure, and rankings were compared across measures. The presence or absence of each of six clinical findings considered to indicate poor prognosis in patients with myocardial infarction (congestive heart failure, pulmonary edema, coma, low systolic blood pressure, low left ventricular ejection fraction, and high blood urea nitrogen level) was determined for patients ranked differently by different severity measures. RESULTS: MedisGroups and the physiology score gave 94.7% of patients similar rankings. Disease Staging, MedisGroups, and the physiology score gave only 78% of patients similar rankings. MedisGroups and APR-DRGs gave 80% of patients similar rankings. Patients whose illnesses were more severe according to MedisGroups and the physiology score were more likely to have the six clinical findings than were patients whose illnesses were more severe according to Disease Staging and APR-DRGs. CONCLUSIONS: Some pairs of severity measures assigned very different severity levels to more than 20% of patients. Evaluations of patient outcomes need to be sensitive to the severity measures used for risk adjustment.

Using severity-adjusted stroke mortality rates to judge hospitals.

Mortality rates are commonly used to judge hospital performance. In comparing death rates across hospitals, it is important to control for differences in patient severity. Various severity tools are now actively marketed in the United States. This study asked whether one would identify different hospitals as having higher- or lower-than-expected death rates using different severity measures. We applied 11 widely-used severity measures to the same database containing 9407 medically-treated stroke patients from 94 hospitals, with 916 (9.7%) in-hospital deaths. Unadjusted hospital mortality rates ranged from 0 to 24.4%. For 27 hospitals, observed mortality rates differed significantly from expected rates when judged by one or more, but not all 11, severity methods. The agreement between pairs of severity methods for identifying the worst 10% or best 50% of hospitals was fair to good. Efforts to evaluate hospital performance based on severity-adjusted, in-hospital death rates for stroke patients are likely to be sensitive to how severity is measured.

Measuring individual plate waste in school lunches. Visual estimation and children's ratings vs. actual weighing of plate waste.

Weighing individual plate waste is time consuming and often impractical. As alternatives, two indirect measures were evaluated-visual estimation and children's ratings of plate waste. Visual estimations were highly correlated with percent waste (r = 0.93) and were quite accurate. Ratings by children were less accurate but were still considered a useful indirect measure that should be further developed.