Associations between hospital deaths (HSMR), readmission and length of stay (LOS): a longitudinal assessment of performance results and facility characteristics of teaching and large-sized hospitals in Canada between 2013-2014 and 2017-2018.

OBJECTIVES: To examine the association between hospital deaths (hospital standardised mortality ratio, HSMR), readmission, length of stay (LOS) and eight hospital characteristics. DESIGN: Longitudinal observational study. SETTING: A total of 119 teaching and large-sized hospitals in Canada between fiscal years 2013-2014 and 2017-2018. PARTICIPANTS: Analysis focused on indicator results and characteristics of individual Canadian hospitals. PRIMARY AND SECONDARY OUTCOMES: Hospital deaths (HSMR); all patients readmitted to hospital; average LOS and a series of eight hospital characteristic summary measures: number of acute care hospital stays; number of acute care beds; number of emergency department visits; average acute care resource intensity weight; total acute care resource intensity weight; hospital occupancy rate; patients admitted through the emergency department (%); patient days in alternate level of care (%). RESULTS: Comparing 2013-2014 to 2017-2018, hospital deaths (HSMR) largely declined, while readmissions increased; 69% of hospitals decreased their hospital deaths (HSMR), while 65% of hospitals increased their readmissions rates. A greater proportion of community-large hospitals (31%, n=14) improved on both hospital deaths (HSMR) and readmission compared to Teaching hospitals (13.9%, n=5). Hospital deaths (HSMR), readmission and LOS largely showed very weak and non-significant correlations. LOS was largely positively and statistically significantly correlated with the suite of eight hospital characteristics. Hospital deaths (HSMR) was largely negatively (not statistically significantly) correlated with the hospital characteristics. Readmission was largely not statistically significantly correlated and showed no clear pattern of correlation (direction) with hospital characteristics. CONCLUSIONS: Examining publicly reported hospital performance results can reveal meaningful insights into the association among outcome indicators and hospital characteristics. Good or bad hospital performance in one care domain does not necessarily reflect similar performance in other care domains. Thus, caution is warranted in a narrow use of outcome indicators in the design and operationalisation of hospital performance measurement and governance models (namely pay-for-performance schemes). Analysis such as this can also inform quality improvement strategies and targeted efforts to address domains of care experiencing declining performance over time; further granular subdivision of the analyses, for example, by hospital peer-groups, can reveal notable differences in performance.

Appropriateness, effectiveness and safety of care delivered in Canadian hospitals: a longitudinal assessment on the utility of publicly reported performance trend data between 2012-2013 and 2016-2017.

OBJECTIVES: To assess the utility of publicly reported performance trend results of Canadian hospitals (by hospital size/type and jurisdiction). DESIGN: Longitudinal observational study. SETTING: 489 hospitals in Canada between fiscal years 2012-2013 and 2016-2017. PARTICIPANTS: Analysis focused on indicator results of individual Canadian hospitals. PRIMARY AND SECONDARY OUTCOMES: Eight outcome indicators of hospital performance: in-hospital mortality (2), readmissions (4) and adverse events (2). Performance trend outcomes of improving, weakening or no change over time. Comparators in performance by hospital size/type of above, below or same as average. RESULTS: At the national level, between 2012-2013 and 2016-2017, Canadian hospitals largely reduced in-hospital mortality: hospital deaths (hospital standardised mortality ratio) -9%; hospital deaths following major surgery -11.1%. Conversely, readmission to hospital increased nationwide: medical 1.5%; obstetric 5%; patients aged 19 years and younger 4.6% and surgical 3%. In-hospital sepsis declined -7.1%. Approximately 10% of the 489 hospitals in this study had a trend of improving performance over time (n=49) in one or more indicators, and a similar number showed a weakening performance over time (n=52). Roughly half of the hospitals in this study (n=224) had no change in performance over time for at least four out of the eight indicators. No single hospital had an improving or weakening trend in more than two indicators. Teaching and larger-sized hospitals showed a higher ratio of improving performance compared with smaller-sized hospitals. CONCLUSIONS: Analysis of Canadian hospital performance through eight indicators shows improvement of in-hospital mortality and in-hospital sepsis, but rising rates of readmissions. Subdividing the analysis by hospital size/type shows greater instances of improvement in teaching and larger-sized hospitals. There is no clear pattern of a particular province/territory with a significant number of hospitals with improving or weakening trends. The overall assessment of trends of improving and weakening as presented in this study can be used more systematically in monitoring progress.

Health system performance assessment in the WHO European Region: which domains and indicators have been used by Member States for its measurement?

Health systems performance assessment (HSPA) varies across the WHO European Region. This review summarizes HSPA domains and indicators used by Member States in their HSPA or health system-related reports. Thirty Member States published in the English language and from their latest documents, 1485 distinct indicators were extracted. The number of indicators reported per Member State ranged from 9 to 146, with a mean of 50. Among the 14 domains of the WHO 2007 framework, service delivery and improved health were covered by virtually all Member States analysed (30 and 29, respectively), but coverage varied for the other 12 domains, with health workforce and financing having good coverage (25 and 26, respectively) but others, such as safety, efficiency, coverage or responsiveness, covered in only 20­30% of documents. Further refinement of frameworks, both in clarity on scope and function and in the conceptual robustness of domains, is warranted and further standardization of generic sets of indicators should be sought.

Оценка эффективности систем здравоохранения в Европейском регионе ВОЗ: какие сферы и показатели используют государства-члены при проведении измерений?

Особенности оценки эффективности систем здравоохранения зависят от конкретных условий в тех или иных странах Европейского региона ВОЗ. В настоящем обзоре вкратце описаны основные сферы такой оценки и показатели, которые государства-члены используют при проведении оценок и при отчетности по системам здравоохранения. Материалы на английском языке публиковались тридцатью государствами-членами, и из новейших документов авторы выделили 1485 четких показателей. Число показателей, по которым предоставляли отчетность государства-члены, варьировалось от 9 до 146 (срединное значение – 50). Из 14 сфер, вошедших в рамочную основу ВОЗ от 2007 г., практически все рассмотренные государства-члены охватили предоставление услуг и улучшение показателей здоровья (соответственно, 30 и 29 стран), однако по другим 12 сферам показатели охвата значительно варьировались. Например, такие сферы, как кадровые ресурсы здравоохранения и финансирование, были охвачены, соответственно, 25 и 26 странами, но вопросы безопасности, эффективности, охвата услугами и отзывчивости освещались лишь в 20–30% документов. Рамочные основы нуждаются в дальнейшей доработке в том, что касается сферы применения и функций, а также концептуальной верности выбранных сфер. Помимо этого, следует стремиться к дальнейшей стандартизации общих наборов показателей.

Health system performance assessment in the WHO European Region: which domains and indicators have been used by Member States for its measurement?

Health systems performance assessment (HSPA) varies across the WHO European Region. This review summarizes HSPA domains and indicators used by Member States in their HSPA or health system-related reports. Thirty Member States published in the English language and from their latest documents, 1485 distinct indicators were extracted. The number of indicators reported per Member State ranged from 9 to 146, with a mean of 50. Among the 14 domains of the WHO 2007 framework, service delivery and improved health were covered by virtually all Member States analysed (30 and 29, respectively), but coverage varied for the other 12 domains, with health workforce and financing having good coverage (25 and 26, respectively) but others, such as safety, efficiency, coverage or responsiveness, covered in only 20–30% of documents. Further refinement of frameworks, both in clarity on scope and function and in the conceptual robustness of domains, is warranted and further standardization of generic sets of indicators should be sought.

Systematic approach to evaluating and confirming the utility of a suite of national health system performance (HSP) indicators in Canada: a modified Delphi study.

OBJECTIVES: Evaluating an existing suite of health system performance (HSP) indicators for continued reporting using a systematic criteria-based assessment and national consensus conference. DESIGN: Modified Delphi approach with technical and leadership groups, an online survey of stakeholders and convening a national consensus conference. SETTING: A national health information steward, the Canadian Institute for Health Information (CIHI). PARTICIPANTS: A total of 73 participants, comprised 61 conference attendants/stakeholders from across Canada and 12 national health information steward staff. PRIMARY AND SECONDARY OUTCOME MEASURES: Indicator dispositions of retention, additional stakeholder consultation, further redevelopment or retirement. RESULTS: 4 dimensions (usability, importance, scientific soundness and feasibility) typically used to select measures for reporting were expanded to 18 criteria grouped under the 4 dimensions through a process of research and testing. Definitions for each criterion were developed and piloted. Once the definitions were established, 56 of CIHI's publicly reported HSP indicators were evaluated against the criteria using modified Delphi approaches. Of the 56 HSP indicators evaluated, 9 measures were ratified for retirement, 7 were identified for additional consultation and 3 for further research and development. A pre-Consensus Conference survey soliciting feedback from stakeholders on indicator recommendations received 48 responses (response rate of 79%). CONCLUSIONS: A systematic evaluation of HSP indicators informed the development of objective recommendations for continued reporting. The evaluation was a fruitful exercise to identify technical considerations for calculating indicators, furthering our understanding of how measures are used by stakeholders, as well as harmonising actions that could be taken to ensure relevancy, reduce indicator chaos and build consensus with stakeholders.

Palliative care coding practices in Canada since the introduction of guidelines and the HSMR indicator.

OBJECTIVES: This study examines palliative care (PC) coding practices since the introduction of a national coding standard and assesses a potential association with hospital standardised mortality ratio (HSMR) results. SETTING: Acute-care hospitals in Canada. PARTICIPANTS: ∼16 million hospital discharges recorded in Canadian Institute for Health Information (CIHI)'s Discharge Abstract Database from April 2006 to March 2013. PRIMARY AND SECONDARY OUTCOME MEASURES: In-hospital mortality, patient characteristics and service utilisation among all hospitalisations, HSMR cases and palliative patients. METHODS: We assessed all separations in the Discharge Abstract Database between fiscal years 2006-2007 and 2012-2013 for PC cases at national, provincial and facility levels. In-hospital mortality was measured among all hospitalisations (including HSMR cases) and palliative patients. We calculated a variant HSMR-PC that included PC cases. RESULTS: There was an increase in the frequency of PC coding over the study period (from 0.78% to 1.12% of all separations), and year-over-year improvement in adherence to PC coding guidelines. Characteristics and resource utilisation of PC patients remained stable within provinces. Crude mortality among HSMR cases declined from 8.7% to 7.3%. National HSMR declined by 22% during the study period, compared with a 17% decline in HSMR-PC. Provincial results for HSMR-PC are not significantly different from regular HSMR calculation. CONCLUSIONS: The introduction of a national coding standard resulted in increased identification of palliative patients and services. Aside from PC coding practices, we note numerous independent drivers of improving HSMR results, notably, a significant reduction of in-hospital mortality, and increase in admissions accompanied by a greater number of coded comorbidities. While PC impacts the HSMR indicator, its influence remains modest.