Convergent Validity of 2 Widely Used Methodologies for Calculating the Hospital Standardized Mortality Ratio in Flanders, Belgium.

OBJECTIVES: To assess their construct validity, we compared results from 2 models used for estimating hospital standardized mortality ratios (HSMRs) in Belgium. The method of the Flemish Hospital Network (FHN) is based on a logistic regression for each of the 64 All Patient Refined Diagnosis-Related Groups that explain 80% of mortality and uses the Elixhauser score to correct for comorbidities. (H)SMRs published on the 3M-Benchmark-Portal are calculated by a simpler indirect standardization for All Patient Refined Diagnosis-Related Groups and risk of mortality (ROM) at discharge. METHODS: We used administrative data from all eligible hospital admissions in 22 Flemish hospitals between 2016 and 2019 (FHN, n = 682,935; 3M, n = 2,122,305). We evaluated model discrimination and accuracy and assessed agreement in estimated HSMRs between methods. RESULTS: The Spearman correlation between HSMRs generated by the FHN model and the standard 3M model was 0.79. Although 2 of 22 hospitals showed opposite classification results, that is, an HSMR significantly <1 according to the FHN method but significantly >1 according to the 3M model, classification agreement between methods was significant (agreement for 59.1% of hospitals, κ = 0.45). The 3M model ( c statistic = 0.96, adjusted Brier score = 26%) outperformed the FHN model (0.87, 17%). However, using ROM at admission instead of at discharge in the 3M model significantly reduced model performance ( c statistic = 0.94, adjusted Brier score = 21%), but yielded similar HSMR estimates and eliminated part of the discrepancy with FHN results. CONCLUSIONS: Results of both models agreed relatively well, supporting convergent validity. Whereas the FHN method only adjusts for disease severity at admission, the ROM indicator of the 3M model includes diagnoses not present on admission. Although diagnosis codes generated by complications during hospitalization have the tendency to increase the predictive performance of a model, these should not be included in risk adjustment procedures.

Unwarranted Between-hospital Variation in Mortality, Readmission, and Length of Stay of Urological Admissions: An Important Trigger for Prioritising Quality Targets.

BACKGROUND: Unwarranted between-hospital variation is a persistent health care quality issue. It is unknown whether urology patients are prone to this variation. OBJECTIVE: To examine between-hospital variation in mortality, readmission, and length of stay for all 22 urological All Patient Refined Diagnosis Related Groups (APR-DRGs). DESIGN, SETTING, AND PARTICIPANTS: This study included administrative data from 320640 urological admissions in 99 (98%) Belgian acute-care hospitals between 2016 and 2018. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: We used hierarchical mixed-effect logistic regression models to estimate hospital-specific and APR-DRG-specific risk-standardised rates for in-hospital mortality, 30-d readmission, and length of stay above the APR-DRG-specific 90th percentile. Between-hospital variation was assessed based on the estimated variance components. Associations of outcomes with patient and hospital characteristics and time trends were examined. RESULTS AND LIMITATIONS: Our analysis revealed important between-hospital variation in mortality, readmission, and length of stay for urological pathologies, particularly for medical diagnoses. Significant variation was shown in all three outcomes for kidney and urinary tract infections; other kidney and urinary tract diagnoses, signs, and symptoms; urinary stones and acquired upper urinary tract obstruction; and kidney and urinary tract procedures for nonmalignancy. Lowering of mortality rates in upper-quartile hospitals to the median could potentially save 41.5% of deaths in these hospitals, with the largest absolute gain for kidney and urinary tract infections and kidney and urinary tract malignancy. Limitations included a likely underestimation of readmission rates. CONCLUSIONS: Urological patient outcomes are characterised by unwarranted between-hospital variation. We recommend improvement initiatives to prioritise kidney and urinary tract infections because of significant variation across the three outcomes and the largest potential gain in lives saved. PATIENT SUMMARY: We found notable between-hospital variation in mortality, readmission, and length of stay for urological hospital admissions in Belgium. As much as 41.5% of deaths could potentially be avoided if underperforming hospitals improved. Targeting kidney and urinary tract infections could help reduce variation.

Standardised mortality ratios as a user-friendly performance metric and trigger for quality improvement in a Flemish hospital network: multicentre retrospective study.

OBJECTIVE: To illustrate the development and use of standardised mortality rates (SMRs) as a trigger for quality improvement in a network of 27 hospitals. DESIGN: This research was a retrospective observational study. The primary outcome was in-hospital mortality. SMRs were calculated for All Patient Refined-Diagnosis-Related Groups (APR-DRGs) that reflect 80% of the Flemish hospital network mortality. Hospital mortality was modelled using logistic regression. The metrics were communicated to the member hospitals using a custom-made R-Shiny web application showing results at the level of the hospital, patient groups and individual patients. Experiences with the metric and strategies for improvement were shared in chief medical officer meetings organised by the Flemish hospital network. SETTING: 27 Belgian hospitals. PARTICIPANTS: 1 198 717 hospital admissions for registration years 2009-2016. RESULTS: Patient gender, age, comorbidity as well as admission source and type were important predictors of mortality. Altogether the SMR models had a C-statistic of 88%, indicating good discriminatory capability. Seven out of ten APR-DRGs with the highest percentage of hospitals statistically significantly deviating from the benchmark involved malignancy. The custom-built web application and the trusted environment of the Flemish hospital network created an interoperable strategy to get to work with SMR findings. Use of the web application increased over time, with peaks before and after key discussion meetings within the Flemish hospital network. A concomitant reduction in crude mortality for the selected APR-DRGs from 6.7% in 2009 to 5.9% in 2016 was observed. CONCLUSIONS: This study reported on the phased approach for introducing SMR reporting to trigger quality improvement. Prerequisites for the successful use of quality metrics in hospital benchmarks are a collaborative approach based on trust among the participants and a reporting platform that allows stakeholders to interpret and analyse the results at multiple levels.

Revision of the Belgian Nursing Minimum Dataset: From data to information.

The Ministry of Public Health commissioned a research project to the Catholic University of Leuven and the University Hospital of Liège to revise the Belgian Nursing Minimum Dataset (B-NMDS). The study started in 2000 and will end with the implementation of the revised B-NMDS in January 2007. The study entailed four major phases. The first phase involved the development of a conceptual framework based on a literature review and secondary data analysis. The second phase focused on language development and development of a data collection tool. The third phase focused on data collection and validation of the new tool. In the fourth phase the validity and reliability of the dataset was tested. The new dataset is without avail if it is not leading to new information. Four applications of the dataset has been defined from the beginning: evaluation of the appropriateness of stay (AEP) in the hospital, nurse staffing, hospital financing and quality management. The aim of this paper is to describe how the B-NMDS can contribute to each of these applications.