The Mayo Clinic Hospital Mortality Reduction Project: Description and Results.

EXECUTIVE SUMMARY: Quality improvement, regulatory, and payer organizations use various definitions of hospital mortality as clinical outcome measures. In this prospective study, the authors evaluated a multicomponent intervention aimed at reducing inpatient mortality in a multistate healthcare delivery system. The project was initiated because of a statistically nonsignificant upward trend in mortality suggested by a six-quarter rise in the observed/expected mortality ratio generated by the Vizient Clinical Data Base and Resource Manager. The design of the mortality reduction plan was influenced by the known limitations of using hospital-wide mortality as a quality improvement measure. The primary objective was to reduce mortality through focused care redesign. The project leadership team attempted to implement standardized system-wide improvements while allowing individual hospitals to simultaneously pursue site-specific practice redesign opportunities. Between Q3, 2015, and Q4, 2017, system-wide mortality reduced from 1.78 to 1.53 (per 100 admissions; p = .01). The actual plan implemented in Mayo Clinic's hospitals is included as Appendix A to this article, published online as Supplemental Digital Content. The authors included it to allow comparison with similar efforts at other healthcare systems, as well as to stimulate criticism and discussion by readers.

Concordance Between Registry and Administrative Data in the Determination of Comorbidity: A Multi-institutional Study.

OBJECTIVE: To characterize agreement between administrative and registry data in the determination of patient-level comorbidities. BACKGROUND: Previous research finds poor agreement between these 2 types of data in the determination of outcomes. We hypothesized that concordance between administrative and registry data would also be poor. METHODS: A cohort of inpatient operations (length of stay 1 day or greater) was obtained from a consortium of 8 hospitals. Within each hospital, National Surgical Quality Improvement Program (NSQIP) data were merged with intra-institutional inpatient administrative data. Twelve different comorbidities (diabetes, hypertension, congestive heart failure, hemodialysis-dependence, cancer diagnosis, chronic obstructive pulmonary disease, ascites, sepsis, smoking, steroid, congestive heart failure, acute renal failure, and dyspnea) were analyzed in terms of agreement between administrative and NSQIP data. RESULTS: Forty-one thousand four hundred thirty-two inpatient surgical hospitalizations were analyzed in this study. Concordance (Cohen Kappa value) between the 2 data sources varied from 0.79 (diabetes) to 0.02 (dyspnea). Hospital variation in concordance (intersite variation) was quantified using a test of homogeneity. This test found significant intersite variation at a level of P < 0.001 for each of the comorbidities except for dialysis (P = 0.07) and acute renal failure (P = 0.19). These findings imply significant differences between hospitals in their generation of comorbidity data. CONCLUSION: This study finds significant differences in how administrative versus registry data assess patient-level comorbidity. These differences are of concern to patients, payers, and providers, each of which had a stake in the integrity of these data. Standardized definitions of comorbidity and periodic audits are necessary to ensure data accuracy and minimize bias.

Venous Thromboembolism after Inpatient Surgery in Administrative Data vs NSQIP: A Multi-Institutional Study.

BACKGROUND: Previous studies have documented significant differences between administrative data and registry data in the determination of postoperative venous thromboembolism (VTE). The goal of this study was to characterize the discordance between administrative and registry data in the determination of postoperative VTE. STUDY DESIGN: This study was performed using data from the American College of Surgeons NSQIP merged with administrative data from 8 different hospitals (5 different medical centers) between 2013 and 2015. Occurrences of postoperative vein thrombosis (VT) and pulmonary embolism (PE) as ascertained by administrative data and NSQIP data were compared. In each situation where the 2 sources disagreed (discordance), a 2-clinician chart review was performed to characterize the reasons for discordance. RESULTS: The cohort used for analysis included 43,336 patients, of which 53.3% were female and the mean age was 59.5 years. Concordance between administrative and NSQIP data was worse for VT (κ 0.57; 95% CI 0.51 to 0.62) than for PE (κ 0.83; 95% CI 0.78 to 0.89). A total of 136 cases of discordance were noted in the assessment of VT; of these, 50 (37%) were explained by differences in the criteria used by administrative vs NSQIP systems. In the assessment of postoperative PE, administrative data had a higher accuracy than NSQIP data (odds ratio for accuracy 2.86; 95% CI 1.11 to 7.14) when compared with the 2-clinician chart review. CONCLUSIONS: This study identifies significant problems in ability of both NSQIP and administrative data to assess postoperative VT/PE. Administrative data functioned more accurately than NSQIP data in the identification of postoperative PE. The mechanisms used to translate VTE measurement into quality improvement should be standardized and improved.

Infectious Surgical Complications are Not Dichotomous: Characterizing Discordance Between Administrative Data and Registry Data.

OBJECTIVE: To characterize reasons for discordance between administrative data and registry data in the determination of postoperative infectious complications. BACKGROUND: Data regarding the occurrence of postoperative surgical complications are identified through either administrative or registry data. Rates of complications vary significantly between these two types of data; the reasons for this are not well-understood. METHODS: The occurrence of 30-day inpatient infectious complications (pneumonia, sepsis, surgical site infection, and urinary tract infection) was compared between the NSQIP and administrative mechanisms at 4 academic hospitals between 2012 and 2014. In each situation where the NSQIP and administrative data were discordant regarding the occurrence of a specific complication, a 2-clinician chart abstraction was performed to characterize the reasons for discordance as (i) administrative coding error, (ii) NSQIP coding error, (iii) "question of criteria", where the discordance was the result of differences in criteria, or (iv) "dually incorrect", where both data sources coded the complication incorrectly. RESULTS: The cohort included 19,163 patients undergoing surgery in 4 different academic hospitals. Rates of infectious complications varied up to 5-fold between the two data sources. A total of 717 discordant complications were identified. Of these, the greatest portion (43%) was due to "question of criteria," followed by administrative coding error (37%), NSQIP error (15%), and dually incorrect (5%). CONCLUSIONS: With a goal of improving existing mechanisms for measuring surgical quality, definitions for the occurrence of a postoperative complication need to be developed and applied consistently. Progress toward this goal will enable patients and payers to better take advantage of recent advances in healthcare data transparency.

Postoperative Myocardial Infarction in Administrative Data vs Clinical Registry: A Multi-Institutional Study.

BACKGROUND: Previous studies have documented significant differences between administrative data and registry data in the determination of postoperative MI. The goal of this study was to characterize discordance between administrative and registry data in the determination of postoperative myocardial infarction (MI). STUDY DESIGN: This study was performed using data from the American College of Surgeons NSQIP merged with administrative data from 8 different hospitals, between 2013 and 2015. From each of these sources, the occurrence of a postoperative MI, as ascertained by administrative data and NSQIP data, were compared. In each situation in which the 2 sources disagreed (discordance), a 2-clinician chart review was performed to generate a "gold standard" determination as to the occurrence of postoperative MI. RESULTS: A total of 43,289 operations met our inclusion criteria for analysis. Within this cohort a total of 230 cases of MI were identified by administrative data and/or NSQIP data (administrative rate 0.41%, NSQIP rate 0.42%). A total of 89 discordant ascertainments were identified, of which 42 were admin+/NSQIP- and 47 were admin-/NSQIP+. Accuracy (99.9% for both) and concordance (kappa = 0.89 [95% CI 0.86 to 0.92] for administrative data, kappa = 0.87 [95% CI 0.84 to 0.91] for NSQIP data) of the 2 systems were similar when compared against our gold standard (chart review). The majority of errors were related to false negatives, with sensitivity rates of 81% in both data sources. CONCLUSIONS: In this multi-institutional study, administrative data and NSQIP demonstrated a similar ability to determine the occurrence of postoperative MI. These findings do not demonstrate an advantage of registry data over administrative data in the determination of postoperative MI.