Classifying hospitals as mortality outliers: logistic versus hierarchical logistic models.

The use of hierarchical logistic regression for provider profiling has been recommended due to the clustering of patients within hospitals, but has some associated difficulties. We assess changes in hospital outlier status based on standard logistic versus hierarchical logistic modelling of mortality. The study population consisted of all patients admitted to acute, non-specialist hospitals in England between 2007 and 2011 with a primary diagnosis of acute myocardial infarction, acute cerebrovascular disease or fracture of neck of femur or a primary procedure of coronary artery bypass graft or repair of abdominal aortic aneurysm. We compared standardised mortality ratios (SMRs) from non-hierarchical models with SMRs from hierarchical models, without and with shrinkage estimates of the predicted probabilities (Model 1 and Model 2). The SMRs from standard logistic and hierarchical models were highly statistically significantly correlated (r > 0.91, p = 0.01). More outliers were recorded in the standard logistic regression than hierarchical modelling only when using shrinkage estimates (Model 2): 21 hospitals (out of a cumulative number of 565 pairs of hospitals under study) changed from a low outlier and 8 hospitals changed from a high outlier based on the logistic regression to a not-an-outlier based on shrinkage estimates. Both standard logistic and hierarchical modelling have identified nearly the same hospitals as mortality outliers. The choice of methodological approach should, however, also consider whether the modelling aim is judgment or improvement, as shrinkage may be more appropriate for the former than the latter.

Current ICD10 codes are insufficient to clearly distinguish acute myocardial infarction type: a descriptive study.

BACKGROUND: Acute myocardial infarction (AMI) type is an important distinction to be made in both clinical and health care research context, as it determines the treatment of the patient as well as affecting outcomes. The aim of the paper was to determine the feasibility of distinguishing AMI type, either ST elevation myocardial infarction (STEMI) or non-ST elevation myocardial infarction (NSTEMI), using ICD10 codes. METHODS: We carried out a retrospective descriptive analysis of hospital administrative data on AMI emergency patients in England, for financial years 2000/1 to 2009/10. We used the performance of an angioplasty procedure on the same day and on the same or next day of hospital admission as a proxy for STEMI. RESULTS: Among the ICD10 AMI subcategories, there were inconsistent trends, with some of the codes exhibiting a gradual decline (such as I21.0 Acute transmural myocardial infarction of anterior wall, I21.1 Acute transmural myocardial infarction of inferior wall, I22.0 Subsequent myocardial infarction of anterior wall and I22.1 Subsequent myocardial infarction of inferior wall) and other codes an increase (in particular I21.9 Acute myocardial infarction, unspecified and I22.9 Subsequent myocardial infarction of unspecified site). With the exception of the codes I21.4 Acute subendocardial myocardial infarction, I21.9 Acute myocardial infarction, unspecified, I22.8 Subsequent myocardial infarction of other sites and I22.9 Subsequent myocardial infarction of unspecified site, all the other AMI subcategories appear to have undergone a significant increase in the number of angioplasty procedures performed the same or the next day of hospital admission from around 2005/6. There appear to be difficulties in accurately identifying the proportion of STEMI/NSTEMI by sole reliance on ICD10 codes. CONCLUSIONS: We suggest as the best sets of codes to select STEMI cases I21.0 to I21.3, I22.0, I22.1 and I22.8; however, without any further adaptations, ICD10 codes are insufficient to clearly distinguish acute myocardial infarction type.

Measuring and explaining mortality in Dutch hospitals; the hospital standardized mortality rate between 2003 and 2005.

BACKGROUND: Indicators of hospital quality, such as hospital standardized mortality ratios (HSMR), have been used increasingly to assess and improve hospital quality. Our aim has been to describe and explain variation in new HSMRs for the Netherlands. METHODS: HSMRs were estimated using data from the complete population of discharged patients during 2003 to 2005. We used binary logistic regression to indirectly standardize for differences in case-mix. Out of a total of 101 hospitals 89 hospitals remained in our explanatory analysis. In this analysis we explored the association between HSMRs and determinants that can and cannot be influenced by hospitals. For this analysis we used a two-level hierarchical linear regression model to explain variation in yearly HSMRs. RESULTS: The average HSMR decreased yearly with more than eight percent. The highest HSMR was about twice as high as the lowest HSMR in all years. More than 2/3 of the variation stemmed from between-hospital variation. Year (-), local number of general practitioners (-) and hospital type were significantly associated with the HSMR in all tested models. CONCLUSION: HSMR scores vary substantially between hospitals, while rankings appear stable over time. We find no evidence that the HSMR cannot be used as an indicator to monitor and compare hospital quality. Because the standardization method is indirect, the comparisons are most relevant from a societal perspective but less so from an individual perspective. We find evidence of comparatively higher HSMRs in academic hospitals. This may result from (good quality) high-risk procedures, low quality of care or inadequate case-mix correction.

In defence of the hospital standardized mortality ratio.

This commentary addresses many of the points made by Penfold and colleagues in the lead article of this issue of Healthcare Papers, including the relationships between hospital standardized mortality ratios (HSMRs) and adverse event reporting, hospital policy and discharge rates. It also discusses what the HSMR is intended to measure, the various analyses and cumulative sum statistic data that my colleagues and I provide to hospitals, interpretation of the results and the inclusion or exclusion of patients receiving comfort or palliative care. It should be noted that my colleagues and I still have the attitude that if anyone can make improvements in our methodologies, we are happy to adopt these improvements as long as they are statistically sound. We feel strongly that if a hospital has a high HSMR, then further investigation is merited to exclude or identify quality-of-care issues; this approach can result in a useful insight into mortality at the institution, which can be associated with a decrease in mortality.

Learning from death: a hospital mortality reduction programme.

PROBLEM: There are wide variations in hospital mortality. Much of this variation remains unexplained and may reflect quality of care. SETTING: A large acute hospital in an urban district in the North of England. DESIGN: Before and after evaluation of a hospital mortality reduction programme. STRATEGIES FOR CHANGE: Audit of hospital deaths to inform an evidence-based approach to identify processes of care to target for the hospital strategy. Establishment of a hospital mortality reduction group with senior leadership and support to ensure the alignment of the hospital departments to achieve a common goal. Robust measurement and regular feedback of hospital deaths using statistical process control charts and summaries of death certificates and routine hospital data. Whole system working across a health community to provide appropriate end of life care. Training and awareness in processes of high quality care such as clinical observation, medication safety and infection control. EFFECTS: Hospital standardized mortality ratios fell significantly in the 3 years following the start of the programme from 94.6 (95% confidence interval 89.4, 99.9) in 2001 to 77.5 (95% CI 73.1, 82.1) in 2005. This translates as 905 fewer hospital deaths than expected during the period 2002-2005. LESSONS LEARNT: Improving the safety of hospital care and reducing hospital deaths provides a clear and well supported goal from clinicians, managers and patients. Good leadership, good information, a quality improvement strategy based on good local evidence and a community-wide approach may be effective in improving the quality of processes of care sufficiently to reduce hospital mortality.

Scientific evaluation of community-based Parkinson's disease nurse specialists on patient outcomes and health care costs.

OBJECTIVE: To determine the effects of community based nurses specialising in Parkinson's disease on health outcomes and healthcare costs. DESIGN: Two year randomised controlled trial in 438 general practices in nine randomly selected health authority areas of England. PARTICIPANTS: 1859 patients with Parkinson's disease identified by the participating general practices. MAIN OUTCOME MEASURES: Survival, stand-up test, dot in square test, bone fracture, global health question, PDQ-39, Euroqol, and healthcare costs. RESULTS: After two years 315 (17.3%) patients had died, although mortality did not differ between those who were attended by nurse specialists and those receiving standard care from their general practitioner (hazard ratio for nurse group v control group 0.91, 95% confidence interval 0.73 to 1.13). No significant differences were found between the two groups for the stand-up test (odds ratio 1.15, 0.93 to 1.42) and dot in square score (difference -0.7, -3.25 to 1.84). Scores on the global health question were significantly better in patients attended by nurse specialists than in controls (difference -0.23, -0.4 to -0.06), but no difference was observed in the results of the PDQ-39 or Euroqol questionnaires. Direct costs for patient health care increased by an average of pound2658 during the study, although not differentially between groups: the average increase was pound266 lower among patients attended by a nurse specialist (- pound981 to pound449). CONCLUSIONS: Nurse specialists in Parkinson's disease had little effect on the clinical condition of patients, but they did improve their patients' sense of wellbeing, with no increase in patients' healthcare costs.

The US hospital standardised mortality ratio: Retrospective database study of Massachusetts hospitals.

OBJECTIVES: To present a case-mix adjustment model that can be used to calculate Massachusetts hospital standardised mortality ratios and can be further adapted for other state-wide data-sets. DESIGN: We used binary logistic regression models to predict the probability of death and to calculate the hospital standardised mortality ratios. Independent variables were patient sociodemographic characteristics (such as age, gender) and healthcare details (such as admission source). Statistical performance was evaluated using c statistics, Brier score and the Hosmer-Lemeshow test. SETTING: Massachusetts hospitals providing care to patients over financial years 2005/6 to 2007/8. PATIENTS: 1,073,122 patients admitted to Massachusetts hospitals corresponding to 36 hospital standardised mortality ratio diagnosis groups that account for 80% of in-hospital deaths nationally. MAIN OUTCOME MEASURES: Adjusted in-hospital mortality rates and hospital standardised mortality ratios. RESULTS: The significant factors determining in-hospital mortality included age, admission type, primary diagnosis, the Charlson index and do-not-resuscitate status. The Massachusetts hospital standardised mortality ratios for acute (non-specialist) hospitals ranged from 60.3 (95% confidence limits 52.7-68.6) to 130.3 (116.1-145.8). The reference standard hospital standardised mortality ratio is 100 with the values below and above 100 suggesting either random or special cause variation. The model was characterised by excellent discrimination (c statistic 0.87), high accuracy (Brier statistics 0.03) and close agreement between predicted and observed mortality rates. CONCLUSIONS: We have developed a case-mix model to give insight into mortality rates for patients served by hospitals in Massachusetts. Our analysis indicates that this technique would be applicable and relevant to Massachusetts hospital care as well as to other US hospitals.

Mortality rates. Dead confusing.

There is considerable disagreement over how hospital mortality rates should be measured. There are concerns that hospitals with a high number of geriatric beds and long-stay patients are bound to appear to do badly in terms of mortality rates. Analysis of death rates in English trusts shows mortality rates do vary for length of stay. After standardisation for age group and length of stay, the adjusted mortality rates show no bias against hospitals with more patients staying more than 28 days, or against hospitals with more geriatric beds.

Staff perceptions of quality of care: an observational study of the NHS Staff Survey in hospitals in England.

BACKGROUND: There is some evidence to suggest that higher job satisfaction among healthcare staff in specific settings may be linked to improved patient outcomes. This study aimed to assess the potential of staff satisfaction to be used as an indicator of institutional performance across all acute National Health Service (NHS) hospitals in England. METHODS: Using staff responses from the NHS Staff Survey 2009, and correlating these with hospital standardised mortality ratios (HSMR), correlation analyses were conducted at institutional level with further analyses of staff subgroups. RESULTS: Over 60 000 respondents from 147 NHS trusts were included in the analysis. There was a weak negative correlation with HSMR where staff agreed that patient care was their trust's top priority (Kendall τ = -0.22, p<0.001), and where they would be happy with the care for a friend or relative (Kendall τ = -0.30, p<0.001). These correlations were identified across clinical and non-clinical groups, with nursing staff demonstrating the most robust correlation. There was no correlation between satisfaction with the quality of care delivered by oneself and institutional HSMR. CONCLUSIONS: In the context of the continued debate about the relationship of HSMR to hospital performance, these findings of a weak correlation between staff satisfaction and HSMR are intriguing and warrant further investigation. Such measures in the future have the advantage of being intuitive for lay and specialist audiences alike, and may be useful in facilitating patient choice. Whether higher staff satisfaction drives quality or merely reflects it remains unclear.

Impact of transfer for angioplasty and distance on AMI in-hospital mortality.

BACKGROUND: The aim of the study was to evaluate the impact of transfer status and distance on in-hospital mortality for acute myocardial infarction (AMI) patients undergoing angioplasty on the same or next day of hospital admission. METHODS: Retrospective analysis of English hospital administrative data using logistic regression modelling. RESULTS: After risk adjustment for the patient baseline characteristics, transferred patients had a higher in-hospital mortality rate than those admitted directly to hospital for angioplasty performed on the same or next day: OR=1.25 (95% confidence interval: 1.02-1.52), P=0.029. There was no statistically significant increased risk of in-hospital mortality with increasing distance between home and angioplasty centre (OR=0.98 (0.84-1.16), P=0.842 for 6-15 km and 1.03 (0.87-1.22), P=0.768 for >15 km when compared with <6 km) or with increasing inter-hospital transfer distance for angioplasty (OR=0.84 (0.55-1.29), P=0.435 for 16-34 km and 0.88 (0.58-1.35), for >34 km when compared with <16 km). CONCLUSIONS: Transfer status is associated with in-hospital mortality rate for AMI patients undergoing angioplasty on the same or next day of hospital admission. No relation between in-hospital mortality and the distance from home to angioplasty centre or inter-hospital transfer distance for angioplasty was found in these patients.

Hospital standardized mortality ratios: sensitivity analyses on the impact of coding.

INTRODUCTION: Hospital standardized mortality ratios (HSMRs) are derived from administrative databases and cover 80 percent of in-hospital deaths with adjustment for available case mix variables. They have been criticized for being sensitive to issues such as clinical coding but on the basis of limited quantitative evidence. METHODS: In a set of sensitivity analyses, we compared regular HSMRs with HSMRs resulting from a variety of changes, such as a patient-based measure, not adjusting for comorbidity, not adjusting for palliative care, excluding unplanned zero-day stays ending in live discharge, and using more or fewer diagnoses. RESULTS: Overall, regular and variant HSMRs were highly correlated (ρ>0.8), but differences of up to 10 points were common. Two hospitals were particularly affected when palliative care was excluded from the risk models. Excluding unplanned stays ending in same-day live discharge had the least impact despite their high frequency. The largest impacts were seen when capturing postdischarge deaths and using just five high-mortality diagnosis groups. CONCLUSIONS: HSMRs in most hospitals changed by only small amounts from the various adjustment methods tried here, though small-to-medium changes were not uncommon. However, the position relative to funnel plot control limits could move in a significant minority even with modest changes in the HSMR.

Logistic versus hierarchical modeling: an analysis of a statewide inpatient sample.

BACKGROUND: Although logistic regression is traditionally used to calculate hospital standardized mortality ratio (HSMR), it ignores the hierarchical structure of the data that can exist within a given database. Hierarchical models allow examination of the effect of data clustering on outcomes. STUDY DESIGN: Traditional logistic regression and random intercepts fixed slopes hierarchical models were fitted to a dataset of patients hospitalized between 2005 and 2007 in Massachusetts. We compared the observed to expected (O/E) in-hospital death ratios between the 2 modeling techniques, a restricted HSMR using only those diagnosis models that converged in both methods and a full hybrid HSMR using a combination of the hierarchical diagnosis models when they converge, plus the remaining diagnoses using standard logistic regression models. RESULTS: We restricted the analysis to the 36 diagnoses accounting for 80% of in-hospital deaths nationally, based on 1,043,813 admissions (59 hospitals). A failure of the hierarchical models to converge in 15 of 36 diagnosis groups hindered full HSMR comparisons. A restricted HSMR, derived from a dataset based on the 21 diagnosis groups that converged (552,933 admissions) showed very high correlation (Pearson r = 0.99). Both traditional logistic regression and hierarchical model identified 12 statistical outliers in common, 7 with high O/E values and 5 with low O/E values. In addition, the multilevel analysis identified 5 additional unique high outliers and 1 additional unique low outlier, and the conventional model identified 2 additional unique low outliers. CONCLUSIONS: Similar results were obtained from the 2 modeling techniques in terms of O/E ratios. However, because a hierarchical model is associated with convergence problems, traditional logistic regression remains our recommended procedure for computing HSMRs.

Using care bundles to reduce in-hospital mortality: quantitative survey.

PROBLEM: To reduce hospital inpatient mortality and thus increase public confidence in the quality of patient care in an urban acute hospital trust after adverse media coverage. DESIGN: Eight care bundles of treatments known to be effective in reducing in-hospital mortality were used in the intervention year; adjusted mortality (from hospital episode statistics) was compared to the preceding year for the 13 diagnoses targeted by the intervention care bundles, 43 non-targeted diagnoses, and overall mortality for the 56 hospital standardised mortality ratio (HSMR) diagnoses covering 80% of hospital deaths. SETTING: Acute hospital trust in north west London. STRATEGIES FOR CHANGE: Use of clinical guidelines in care bundles in eight targeted clinical areas. INTERVENTIONS: Use of care bundles in treatment areas for the diagnoses leading to most deaths in the trust in 2006-7. KEY MEASURES FOR IMPROVEMENT: Change in adjusted mortality in targeted and non-targeted diagnostic groups; hospital standardised mortality ratio (HSMR) during the intervention year compared with the preceding year. Effect of the change The standardised mortality ratio (SMR) of the targeted diagnoses and the HSMR both showed significant reductions, and the non-targeted diagnoses showed a slight reduction. Cumulative sum charts showed significant reductions of SMRs in 11 of the 13 diagnoses targeted in the year of the quality improvements, compared with the preceding year The HSMR of the trust fell from 89.6 in 2006-7 to 71.1 in 2007-8 to become the lowest among acute trusts in England. 255 fewer deaths occurred in the trust (174 of these in the targeted diagnoses) in 2007-8 for the HSMR diagnoses than if the 2006-7 HSMR had been applicable. From 2006-7 to 2007-8 there was a 5.7% increase in admissions, 7.9% increase in expected deaths, and 14.5% decrease in actual deaths. LESSONS LEARNT: Implementing care bundles can lead to reductions in death rates in the clinical diagnostic areas targeted and in the overall hospital mortality rate.