Hospital-wide mortality as a quality metric: conceptual and methodological challenges.

Hospital-wide mortality rates are used as a measure of overall hospital quality. However, their parsimony and apparent simplicity belie significant conceptual and methodological concerns. For many diagnoses included in hospital-wide mortality, the association between short-term mortality and quality of care is not well established. Furthermore, compared with condition-specific or procedure-specific mortality, hospital-wide mortality rates pose greater methodological challenges (ie, eligibility and exclusion criteria, risk adjustment, statistical techniques for aggregating across diagnoses, usability). Many of these result from substantial interprovider heterogeneity in diagnosis frequency, sample sizes, and patient severity. Hospital-wide mortality is problematic as a quality metric for public reporting, although hospitals may elect to use such measures for other purposes. Potential alternative approaches include multidimensional composite metrics or mortality measurement limited to selected conditions and procedures for which the link between hospital mortality and quality is clear, legitimate exclusions are uncommon, and sample sizes, end points, and risk adjustment are adequate.

Variability in the measurement of hospital-wide mortality rates.

BACKGROUND: Several countries use hospital-wide mortality rates to evaluate the quality of hospital care, although the usefulness of this metric has been questioned. Massachusetts policymakers recently requested an assessment of methods to calculate this aggregate mortality metric for use as a measure of hospital quality. METHODS: The Massachusetts Division of Health Care Finance and Policy provided four vendors with identical information on 2,528,624 discharges from Massachusetts acute care hospitals from October 1, 2004, through September 30, 2007. Vendors applied their risk-adjustment algorithms and provided predicted probabilities of in-hospital death for each discharge and for hospital-level observed and expected mortality rates. We compared the numbers and characteristics of discharges and hospitals included by each of the four methods. We also compared hospitals' standardized mortality ratios and classification of hospitals with mortality rates that were higher or lower than expected, according to each method. RESULTS: The proportions of discharges that were included by each method ranged from 28% to 95%, and the severity of patients' diagnoses varied widely. Because of their discharge-selection criteria, two methods calculated in-hospital mortality rates (4.0% and 5.9%) that were twice the state average (2.1%). Pairwise associations (Pearson correlation coefficients) of discharge-level predicted mortality probabilities ranged from 0.46 to 0.70. Hospital-performance categorizations varied substantially and were sometimes completely discordant. In 2006, a total of 12 of 28 hospitals that had higher-than-expected hospital-wide mortality when classified by one method had lower-than-expected mortality when classified by one or more of the other methods. CONCLUSIONS: Four common methods for calculating hospital-wide mortality produced substantially different results. This may have resulted from a lack of standardized national eligibility and exclusion criteria, different statistical methods, or fundamental flaws in the hypothesized association between hospital-wide mortality and quality of care. (Funded by the Massachusetts Division of Health Care Finance and Policy.).

Expert panel on weight loss surgery: executive report update.

Rapid shifts in the demographics and techniques of weight loss surgery (WLS) have led to new issues, new data, new concerns, and new challenges. In 2004, this journal published comprehensive evidence-based guidelines on WLS. In this issue, we've updated those guidelines to assure patient safety in this fast-changing field. WLS involves a uniquely vulnerable population in need of specialized resources and ongoing multidisciplinary care. Timely best-practice updates are required to identify new risks, develop strategies to address them, and optimize treatment. Findings in these reports are based on a comprehensive review of the most current literature on WLS; they directly link patient safety to methods for setting evidence-based guidelines developed from peer-reviewed scientific publications. Among other outcomes, these reports show that WLS reduces chronic disease risk factors, improves health, and confers a survival benefit on those who undergo it. The literature also shows that laparoscopy has displaced open surgery as the predominant approach; that government agencies and insurers only reimburse procedures performed at accredited WLS centers; that best practice care requires close collaboration between members of a multidisciplinary team; and that new and existing facilities require wide-ranging changes to accommodate growing numbers of severely obese patients. More than 100 specialists from across the state of Massachusetts and across the many disciplines involved in WLS came together to develop these new standards. We expect them to have far-reaching effects of the development of health care policy and the practice of WLS.

Comparing patient-reported hospital adverse events with medical record review: do patients know something that hospitals do not?

BACKGROUND: Hospitals routinely survey patients about the quality of care they receive, but little is known about whether patient interviews can detect adverse events that medical record reviews do not. OBJECTIVE: To compare adverse events reported in postdischarge patient interviews with adverse events detected by medical record review. DESIGN: Random sample survey. SETTING: Massachusetts, 2003. PATIENTS: Recently hospitalized adults. MEASUREMENTS: By using parallel methods, physicians reviewed postdischarge interviews and medical records to classify hospital adverse events. RESULTS: Among 998 study patients, 23% had at least 1 adverse event detected by an interview and 11% had at least 1 adverse event identified by record review. The kappa statistic showed relatively poor agreement between interviews and medical records for occurrence of any type of adverse event (kappa = 0.20 [95% CI, 0.03 to 0.27]) and somewhat better agreement between interviews and medical records for life-threatening or serious events (kappa = 0.33 [CI, 0.20 to 0.45]). Record review identified 11 serious, preventable events (1.1% of patients). Interviews identified an additional 21 serious and preventable events that were not documented in the medical record, including 12 predischarge events and 9 postdischarge events, in which symptoms occurred after the patient left the hospital. LIMITATIONS: Patients had to be healthy enough to be interviewed. Delay in reaching patients (6 to 12 months after discharge) may have resulted in poor recall of events during the hospital stay. CONCLUSION: Patients report many events that are not documented in the medical record; some are serious and preventable. Hospitals should consider monitoring patient safety by adding questions about adverse events to postdischarge interviews.

Reconciling medications at admission: safe practice recommendations and implementation strategies.

BACKGROUND: Fifty hospitals collaborated in a patient safety initiative developed and implemented by the Massachusetts Coalition for the Prevention of Medical Errors and the Massachusetts Hospital Association. METHODS: A consensus group identified safe practices and suggested implementation strategies. Four collaborative learning sessions were offered, and teams monitored their progress and shared successful strategies and lessons learned. Reports from participating teams and an evaluation survey were then used to identify successful techniques for reconciling medications. RESULTS: For the 50 participating hospitals, implementation strategies most strongly correlated with success included active physician and nursing engagement, having an effective improvement team, using small tests of change, having an actively engaged senior administrator, and sending a team to multiple collaborative sessions. DISCUSSION: Adoption of the reconciling safe practices proved challenging. The process of writing medication orders at patient transfer points is complex. The hospitals' experiences demonstrated that implementing the proposed safe practices requires a team effort with leadership support and vigilant measurement.

Error reporting and disclosure systems: views from hospital leaders.

CONTEXT: The Institute of Medicine has recommended establishing mandatory error reporting systems for hospitals and other health settings. OBJECTIVE: To examine the opinions and experiences of hospital leaders with state reporting systems. DESIGN AND SETTING: Survey of chief executive and chief operating officers (CEOs/COOs) from randomly selected hospitals in 2 states with mandatory reporting and public disclosure, 2 states with mandatory reporting without public disclosure, and 2 states without mandatory systems in 2002-2003. MAIN OUTCOME MEASURES: Perceptions of the effects of mandatory systems on error reporting, likelihood of lawsuits, and overall patient safety; attitudes regarding release of incident reports to the public; and likelihood of reporting incidents to the state or to the affected patient based on hypothetical clinical vignettes that varied the type and severity of patient injury. RESULTS: Responses were received from 203 of 320 hospitals (response rate = 63%). Most CEOs/COOs thought that a mandatory, nonconfidential system would discourage reporting of patient safety incidents to their hospital's own internal reporting system (69%) and encourage lawsuits (79%) while having no effect or a negative effect on patient safety (73%). More than 80% felt that the names of both the hospital and the involved professionals should be kept confidential, although respondents from states with mandatory public disclosure systems were more willing than respondents from the other states to release the hospital name (22% vs 4%-6%, P = .005). Based on the vignettes, more than 90% of hospital leaders said their hospital would report incidents involving serious injury to the state, but far fewer would report moderate or minor injuries, even when the incident was of sufficient consequence that they would tell the affected patient or family. CONCLUSIONS: Most hospital leaders expressed substantial concerns about the impact of mandatory, nonconfidential reporting systems on hospital internal reporting, lawsuits, and overall patient safety. While hospital leaders generally favor disclosure of patient safety incidents to involved patients, fewer would disclose incidents involving moderate or minor injury to state reporting systems.