A comparison of four quasi-experimental methods: an analysis of the introduction of activity-based funding in Ireland.

BACKGROUND: Health services research often relies on quasi-experimental study designs in the estimation of treatment effects of a policy change or an intervention. The aim of this study is to compare some of the commonly used non-experimental methods in estimating intervention effects, and to highlight their relative strengths and weaknesses. We estimate the effects of Activity-Based Funding, a hospital financing reform of Irish public hospitals, introduced in 2016. METHODS: We estimate and compare four analytical methods: Interrupted time series analysis, Difference-in-Differences, Propensity Score Matching Difference-in-Differences and the Synthetic Control method. Specifically, we focus on the comparison between the control-treatment methods and the non-control-treatment approach, interrupted time series analysis. Our empirical example evaluated the length of stay impact post hip replacement surgery, following the introduction of Activity-Based Funding in Ireland. We also contribute to the very limited research reporting the impacts of Activity-Based-Funding within the Irish context. RESULTS: Interrupted time-series analysis produced statistically significant results different in interpretation, while the Difference-in-Differences, Propensity Score Matching Difference-in-Differences and Synthetic Control methods incorporating control groups, suggested no statistically significant intervention effect, on patient length of stay. CONCLUSION: Our analysis confirms that different analytical methods for estimating intervention effects provide different assessments of the intervention effects. It is crucial that researchers employ appropriate designs which incorporate a counterfactual framework. Such methods tend to be more robust and provide a stronger basis for evidence-based policy-making.

Hospital funding reforms in Canada: a narrative review of Ontario and Quebec strategies.

BACKGROUND: In the early 2000s, Ontario and Quebec, two provinces of Canada, began to introduce hospital payment reforms to improve quality and access to care. This paper (1) critically reviews patient-based funding (PBF) implementation approaches used by Quebec and Ontario over 15 years, and (2) identifies factors that support or limit PBF implementation to inform future decisions regarding the use of PBF models in both provinces. METHODS: We adopted a narrative review approach to document and critically analyse Quebec and Ontario experiences with the implementation of patient-based funding. We searched for documents in the scientific and grey literature and contacted key stakeholders to identify relevant policy documents. RESULTS: Both provinces targeted similar hospital services-aligned with nationwide policy goals-fulfilling in part patient-based funding programmes' objectives. We identified four factors that played a role in ensuring the successful-or not-implementation of these strategies: (1) adoption supports, (2) alignment with programme objectives, (3) funding incentives and (4) stakeholder engagement. CONCLUSIONS: This review provides lessons in the complexity of implementing hospital payment reforms. Implementation is enabled by adoption supports and funding incentives that align with policy objectives and by engaging stakeholders in the design of incentives.

Weighted activity unit effect: evaluating the cost of diagnosis-related group coding.

BACKGROUND: Activity-based funding (ABF) is a means of healthcare reimbursement, where hospitals are allocated funding based on the number and mix of clinical activity. The ABF model is based solely on Australian refined diagnosis-related group (AR-DRG) classifications of hospital encounters. Each AR-DRG is allocated a weighted activity unit (WAU) translating to cost value to determine ongoing funding allocations for each hospital annually. AIM: We explored cost consequences of AR-DRG coding variances within our Medical Oncology department over a 6-month period. METHODS: All inpatient encounters for medical oncology from 1 January to 30 June 2014 were identified and paired with actual AR-DRG coding sheets submitted by the hospital coders. Inpatient charts were manually reviewed by a Medical Oncology Registrar to capture any changes or additional AR-DRGs, which were subsequently evaluated for total WAU value variance. Applying 1 WAU = $4676 as per the 2014 Queensland model, cost consequences were calculated. RESULTS: A total of 116 encounters was identified for 72 patients. Of 116 patients, 95 (81%) had additional diagnoses captured, leading to an AR-DRG and WAU change in 26 encounters. The total reimbursement variance for this period was $143 404.07. Cost consequences resulted from: (i) use of abbreviations in clinical notes unable to be coded; and (ii) diagnoses not documented despite treatment delivered as per medication charts. CONCLUSION: Clinical note documentation ultimately determines the future funding of our healthcare system. Appropriate communication and education of medical staff and hospital coders are vital to ensure precise documentation and accurate AR-DRG coding for optimal and appropriate reimbursement in this funding model.

Out of sync: a Shared Mental Models perspective on policy implementation in healthcare.

The impact of policy ambiguity on implementation is a perennial concern in policy circles. The degree of ambiguity of policy goals and the means to achieve them influences the likelihood that a policy will be uniformly understood and implemented across implementation sites. We argue that the application of institutional and organisational theories to policy implementation must be supplemented by a socio-cognitive lens in which stakeholders' interpretations of policy are investigated and compared. We borrow the concept of 'Shared Mental Models' from the literature on industrial psychology to examine the microprocesses of policy implementation. Drawing from interviews with 45 key informants involved in the implementation of a hospital funding reform, known as Quality-Based Procedures in Ontario, Canada, we identify divergent mental models and explain how these divergences may have affected implementation and change management. We close with considerations for future research and practice.

[Determinants of time required by medical information technicians for quality control of hospital activity coding, in French medico-administrative system]. / Déterminants du temps nécessaire au contrôle qualité dans le cadre du programme de médicalisation des systèmes d'information (PMSI), dans le champ médecine, chirurgie et obstétrique (MCO) d'un centre hospitalier régional, par les techniciens d'information médicale.

BACKGROUND: Since 2008, in France, hospital funding is determined by the nature of activities provided (activity-based funding). Quality control of hospital activity coding is essential to optimize hospital remuneration. There is a need for reliable tools to allocate human resources wisely in order to improve these controls. METHODS: The main objective of this study was to identify the determinants of time needed by medical information technicians to control hospital activity coding in a Regional Hospital Center. From March 2016 to the beginning of January 2017, medical information technicians reported the time they spent on each quality control, and the time they needed when they had to code the entire stay. Multiple linear regressions were performed to identify the determinants of quality control or coding duration. A split sample validation was used: model was created on one half of the sample and validated on the remaining half. RESULTS: Among the controls, 5431 were included in the analysis of determinants of control duration (2715 kept aside for model validation). Seven determinants have been identified (stay duration, level of complexity, month of control, type of control, medical information technician, rank of classing information, and major diagnostic category). The correlation coefficient between predicted and real control duration was 0.71 (P<10-4); 808 stays were included in the analysis of determinants of coding duration (404 kept aside for model validation). Two determinants have been identified. The correlation coefficient, between predicted and real coding duration, was 0.47 (P<10-3). We performed the same multiple regression, on 2017 activity data, to estimate the weight of each hospital activity pole, regarding quality control of hospital activity coding. CONCLUSION: We succeeded in modeling time needed for quality control of hospital stays. These results helped to estimate human resources required for quality control of each hospital pole. Nevertheless, the second analysis did not give satisfactory results: we failed in modeling time needed to code hospital stays.

Activity-based funding for safety and quality: A policy discussion of issues and directions for nursing-focused health services outcomes research.

AIMS: A discussion of the implications and opportunities arising from the Commonwealth of Australia health care reform agenda; linking pricing with quality, with particular reference to directions for nursing-focused health services outcomes research directed to improve the safety and quality of health care practices. BACKGROUND: National activity-based funding in Australia is a policy-focused development. As the relationship between cost and quality becomes apparent, the role of clinicians and their contribution to high quality care has become a pressing issue for leadership, teaching, and research. DESIGN: Discussion paper DATA SOURCES: This paper is based on seven years' experience as a member of a Commonwealth of Australia statutory committee-the Clinical Advisory Committee of the Independent Hospital Pricing Authority-and is supported by relevant literature and theory. IMPLICATIONS FOR NURSING: To date, unravelling the linkage, especially causal relationships, between direct care nursing and patient safety outcomes has not been well established. New activity-based funding data elements developed for national implementation in Australia provide accessible and meaningful standardised data for measurement of never events, hospital-acquired complications, and preventable readmissions.

Aligning policy objectives and payment design in palliative care.

BACKGROUND: Payment models for palliative care vary across nations, with few adopting contemporary payments designs that apply to other parts of the health system. AIM: To propose optimal payment arrangements for palliative care. APPROACH: Review of relevant literature on funding mechanisms in health care generally and palliative care in particular. RESULTS: Payment models for palliative care should move toward activity-based funding using an agreed classification, be uncapped funding with performance monitoring, and make explicit use of performance metrics and reporting. CONCLUSIONS: If palliative care is to become a universally accessible service, new approaches to funding, based on the experience of funding reforms in other parts of the health system, need to be adopted.

Understanding ED performance after the implementation of activity-based funding.

OBJECTIVE: The aim of this study was to describe emergency department (ED) activities and staffing after the introduction of activity-based funding (ABF) to highlight the challenges of new funding arrangements and their implementation. METHODS: A retrospective study of public hospital EDs in Queensland, Australia, was undertaken for 2013-2014. The ED and hospital characteristics are described to evaluate the alignment between activity and resourcing levels and their impact on performance. RESULTS: Twenty EDs participated (74% response rate). Weighted activity units (WAUs) and nursing staff varied based on hospital type and size. Larger hospital EDs had on average 9076 WAUs and 13 full time equivalent (FTE) nursing staff per 1000 WAUs; smaller EDs had on average 4587 WAUs and 10.3 FTE nursing staff per 1000 WAUs. Medical staff was relatively consistent (8.1-8.7 FTE per 1000 WAUs). The proportion of patients admitted, discharged, or transferred within 4 hours ranged from 73% to 79%. The ED medical and nursing staffing numbers did not correlate with the 4-hour performance. CONCLUSION: Substantial variation exists across Queensland EDs when resourcing service delivery in an activity-based funding environment. Historical inequity persists in the staffing profiles for regional and outer metropolitan departments. The lack of association between resourcing and performance metrics provides opportunity for further investigation of efficient models of care.

Activity-based funding in mental health: a disastrous path.

OBJECTIVES: On the basis of the experience of the Netherlands, this critical commentary will argue why activity-based funding (ABF) in mental health care is a disastrous path that Australia should not take. CONCLUSIONS: ABF leads to an exponential growth in health care spending as it encourages diagnostic inflation and overproductivity. It also leads to fraud and an increased bureaucracy that goes hand in hand with demoralisation among health workers. And finally, the increasing treatment claims leads to the reintroduction of productivity limitations, waiting lists and ultimately austerity measures in order to halt the untamed growth of spending.

Activity-based funding: implications for mental health services and consultation-liaison psychiatry.

OBJECTIVE: The aim of this paper is to inform mental health professionals about Activity-based funding (ABF) and the implications for data collection and clinical practice, in particular for consultation-liaison (CL) psychiatry. CONCLUSIONS: Activity-based funding may provide an opportunity for mental health services to be more equitably resourced, but much needs to be done to demonstrate that the funding model works in mental health. It is important to ensure that data collected is meaningful and accurate and reflects the diverse roles of mental health clinicians, including in CL. Inpatient and community services should be integrated in the model, as well as safeguards against potential abuse. Clinicians, in partnership with initiatives such as the Australian Mental Health Outcomes and Classification Network, are best placed to guide the development of an ABF system for mental health which appropriately recognises the complexity and variability between patients in different settings.

Multicultural emergency medicine epidemiology: A health economic analysis of patient visits.

OBJECTIVE: There is growing evidence to suggest that culturally and linguistically diverse (CALD) patients cost the health system more than non-CALD patients because of a higher burden of disease and increased resource consumption. The present study aimed to compare the ED resource utilisation of CALD and non-CALD patients at a tertiary hospital in Sydney, Australia. METHODS: The total ED resource utilisation was calculated by separating each visit into diagnostic test cost and time spent in ED components. The time component was calculated using the product of the total length of stay and a resource cost per unit time measure. Diagnostic tests were costed using the Australian Medicare Benefit Schedule. A generalised additive model was developed to estimate the isolated effect of CALD status on the resource utilisation during an ED visit. RESULTS: CALD patients had a higher median resource utilisation than non-CALD patients ($736.93 vs $701.36, P < 0.0001); however, the generalised additive model demonstrated that CALD status was not independently associated with increased resource utilisation. CONCLUSION: CALD status is not an independent influence on ED resource utilisation but other explanatory variables such as increased age and altered case-mix appear to have a much greater influence. There may, however, be other reasons to consider CALD loading such as equity in healthcare and to address poorer overall health outcomes for CALD patients.

Hospital response to Activity-Based Funding and price incentives: Evidence from Ireland.

Activity-Based Funding (ABF) is a funding policy incentivising hospitals to deliver more efficient care. ABF can be complemented by additional price incentives to further drive hospital efficiency. In 2016, ABF was introduced for public patients admitted to Irish public hospitals. Additionally, a price incentive to perform laparoscopic cholecystectomy as day-case surgery was introduced in 2018. Private patient activity in public hospitals was subject to neither ABF nor price incentive. Using national Hospital In-Patient-Enquiry activity data 2013-2019, we evaluated the impact of ABF and the price incentive for laparoscopic cholecystectomy surgery in Ireland. We exploit variation in hospital payment for public and private patients treated in public acute Irish hospitals and employ a Propensity Score Matching Difference-in-Differences approach. We estimate the funding change impacts across outcomes measuring the proportion of day-case admissions and length of stay. We found no significant impact for either outcomes linked to ABF introduction. Similarly, no impacts linked to the price incentive were observed. It appears providers of laparoscopic cholecystectomy in Irish public hospitals did not react to the new funding mechanisms. The implementation of the funding policies did not improve hospital efficiency. Further strengthening of these new funding mechanisms are required to deliver more efficient care.

The importance of good governance in hospital payment reform - A case study from Ukraine.

In 2005, Ukraine embarked on hospital financing reforms that included the introduction of a Diagnosis Related Group (DRG) based payment system for acute inpatient care. The primary purpose of introducing activity-based funding was to provide incentives for hospitals to use their limited resources more efficiently. Following an extended period of preparation and planning during which technical assistance was provided by various development agencies, Ukraine took action to implement the DRG system at a national level in April 2018, through a World Bank project. While some progress was made, the execution of the reform faced challenges with the organisation and administration of the implementation process, and duplication of effort. The consequence of these shortcomings was that the newly introduced system was not capable of measuring inpatient DRG activity at a level of accuracy necessary for the determination of hospital performance and the subsequent calculation of payments. If the expected outcomes of DRG implementation in Ukraine are to be realised, stakeholders including both beneficiary agencies and development organisations, will need to improve program governance through better coordination of their activities towards a common goal.

Analytical methods to assess the impacts of activity-based funding (ABF): a scoping review.

BACKGROUND: Activity-Based Funding (ABF) has been implemented across many countries as a means to incentivise efficient hospital care delivery and resource use. Previous reviews have assessed the impact of ABF implementation on a range of outcomes across health systems. However, no comprehensive review of the methods used to generate this evidence has been undertaken. The aim of this review is to identify and assess the analytical methods employed in research on ABF hospital performance outcomes. METHODS: We conducted a scoping review in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews. Five academic databases and reference lists of included studies were used to identify studies assessing the impact of ABF on hospital performance outcomes. Peer-reviewed quantitative studies published between 2000 and 2019 considering ABF implementation outside the U.S. were included. Qualitative studies, policy discussions and commentaries were excluded. Abstracts and full text studies were double screened to ensure consistency. All analytical approaches and their relative strengths and weaknesses were charted and summarised. RESULTS: We identified 19 studies that assessed hospital performance outcomes from introduction of ABF in England, Korea, Norway, Portugal, Israel, the Netherlands, Canada, Italy, Japan, Belgium, China, and Austria. Quasi-experimental methods were used across most reviewed studies. The most commonly used assessment methods were different forms of interrupted time series analyses. Few studies used difference-in-differences or similar methods to compare outcome changes over time relative to comparator groups. The main hospital performance outcome measures examined were case numbers, length of stay, mortality and readmission. CONCLUSIONS: Non-experimental study designs continue to be the most widely used method in the assessment of ABF impacts. Quasi-experimental approaches examining the impact of ABF implementation on outcomes relative to comparator groups not subject to the reform should be applied where possible to facilitate identification of effects. These approaches provide a more robust evidence-base for informing future financing reform and policy.

Improving the accuracy of stroke clinical coding with open-source software and natural language processing.

Clinical coding is an important task, which is required for accurate activity-based funding. Natural language processing may be able to assist with improving the efficiency and accuracy of clinical coding. The aims of this study were to explore the feasibility of using natural language processing for stroke hospital admissions, employed with open-source software libraries, to aid in the identification of potentially misclassified (1) category of Adjacent Diagnosis Related Groups (ADRG), (2) the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Australian Modification (ICD-10-AM) diagnoses, and (3) Diagnosis Related Groups (DRG). Data was collected for consecutive individuals admitted to the Royal Adelaide Hospital Stroke Unit over a five-month period for misclassification identification analysis. 152 admissions were included in the study. Using free-text discharge summaries, a random forest classifier correctly identified two cases classified as B70 ("Stroke and Other Cerebrovascular Disorders") that should be classified as B02 (having received endovascular thrombectomy). A regular expression-based analysis correctly identified 33 cases in which ataxia was present but was not coded. Two cases were identified that should have been classified as B70D, rather than B70A/B/C, based on transfer to another centre within five days of admission. A variety of techniques may be useful to help identify misclassifications in ADRG, ICD-10-AM and DRG codes. Such techniques can be implemented with open-source software libraries, and may have significant financial implications. Future studies may seek to apply open-source software libraries to the identification of misclassifications of all ICD-10-AM diagnoses in stroke patients.

Divergent notions of "quality" in healthcare policy implementation: a framing perspective.

PURPOSE: This paper examines how "quality" was framed in the design and implementation of a policy to reform hospital funding and associated care delivery. The aims of the study were: (1) To describe how government policy-makers who designed the policy and managers and clinicians who implemented the policy framed the concept of "quality" and (2) To explore how frames of quality and the framing process may have influenced policy implementation. DESIGN/METHODOLOGY/APPROACH: The authors conducted a secondary analysis of data from a qualitative case study involving semi-structured interviews with 45 purposefully selected key informants involved in the design and implementation of the quality-based procedures policy in Ontario, Canada. The authors used framing theory to inform coding and analysis. FINDINGS: The authors found that policy designers perpetuated a broader frame of quality than implementers who held more narrow frames of quality. Frame divergence was further characterized by how informants framed the relationship between clinical and financial domains of quality. Several environmental and organizational factors influenced how quality was framed by implementers. ORIGINALITY/VALUE: As health systems around the world increasingly implement new models of governance and financing to strengthen quality of care, there is a need to consider how "quality" is framed in the context of these policies and with what effect. This is the first framing analysis of "quality" in health policy.

Level of agreement between medical record and ICD-10-AM coding of mental health, alcohol and drug conditions in trauma patients.

BACKGROUND: Despite the reliance on administrative data in epidemiological studies, there is little information on the completeness of co-morbidities in administrative data coded from medical records. OBJECTIVE: The aim of this study was to quantify the agreement between the International Classification of Diseases, Tenth Revision, Australian Modification (ICD-10-AM) administrative coding of mental health, drug and alcohol co-morbidities and medical records in a severely injured patient population. METHOD: A random sample of patients (n = 500) captured by the Victorian State Trauma Registry and definitively managed at the state's adult major trauma services was selected for the study. Retrospective medical record review was conducted to collect data about documented co-morbidities. The agreement between ICD-10-AM data generated from routine hospital coding and medical record-based co-morbidities was determined using Cohen's κ and prevalence-adjusted bias-adjusted kappa (PABAK) statistics. RESULTS: The percentage of agreement between the medical record and ICD-10-AM coding for mental health, drug and alcohol co-morbidities was 72.8%, and the PABAK showed moderate agreement (PABAK = 0.46; 95% confidence interval (CI): 0.37, 0.54). There was no difference in agreement between unintentional injury patients (PABAK = 0.52; 95% CI: 0.42, 0.62) compared with intentional injury patients (PABAK = 0.36, 95% CI: 0.23, 0.49), and no change in agreement for patients admitted before (PABAK = 0.40; 95% CI: 0.30, 0.50) and after the introduction of mandatory co-morbidity coding (PABAK = 0.46; 95% CI: 0.37, 0.54). CONCLUSION: Despite documentation in the medical record, a large proportion of mental health, drug and alcohol conditions were not coded in ICD-10-AM. Acknowledgement of these limitations is needed when using ICD-10-AM coded co-morbidities in research studies and health policy development. IMPLICATIONS: This work has implications for researchers of drug and alcohol abuse; mental health; accidents and injuries; workers' compensation; health workforce; health services; and policy decisions for healthcare, emergency services, insurance industry, national productivity and welfare costings reliant on those research outcomes.

Reconciling hospital-acquired complications and CHADx+ in Victorian coded hospital data.

BACKGROUND: The Council of Australian Governments has focused the attention of health service managers and state health departments on a list of hospital-acquired complications (HACs) proposed as the basis of funding adjustments for poor quality of hospital inpatient care. These were devised for the Australian Commission on Safety and Quality in Health Care as a subset of their earlier classification of hospital-acquired complications (CHADx) and designed to be used by health services to monitor safety performance for their admitted patients. OBJECTIVE: To improve uptake of both classification systems by clarifying their purposes and by reconciling the ICD-10-AM code sets used in HACs and the Victorian revisions to the CHADx system (CHADx+). METHOD: Frequency analysis of individual clinical codes with condition onset flag (COF 1) included in both classification systems using the Victorian Admitted Episodes Dataset for 2014/2015 ( n = 2,623,275 separations). Narrative description of the resulting differences in definition of "adverse events" embodied in the two systems. RESULTS: As expected, a high proportion of ICD-10-AM codes used in the HACs also appear in CHADx+, and given the wider scope of CHADx+, it uses a higher proportion of all COF 1 diagnoses than HACs (82% vs. 10%). This leads to differing estimates of rates of adverse events: 2.12% of cases for HACs and 11.13% for CHADx+. Most CHADx classes (70%) are not covered by the HAC system; discrepancies result from the exclusion from HACs of several major CHADx+ groups and from a narrower definition of detailed HAC classes compared with CHADx+. Case exclusion criteria in HACs (primarily mental health admissions) resulted in a very small proportion of discrepancies (0.13%) between systems. DISCUSSION: Issues of purpose and focus of these two Australian systems, HACs for clinical governance and CHADx+ for local quality improvement, explain many of the differences between them, and their approach to preventability, and risk stratification. CONCLUSION: A clearer delineation between these two systems using routinely coded hospital data will assist funders, clinicians, quality improvement professionals and health information managers to understand discrepancies in case identification between them and support their different information needs.

Ethical leadership and why health information management professionals need to be involved. Commentary on Health information is central to changes in healthcare: a clinician's view (Hoyle, 2019).

Philip Hoyle presents a compelling argument for the significant and highly valued role that the management of health information plays in the Australian healthcare system and the delivery of health services in this country. However, he also brings to our attention the ill-defined nature of the ethical oversight of this very information. Hoyle uses words such as "honesty," "commitment to beneficence," "commitment to equity" and "respect for variation" when describing the characteristics of ethical leadership. He singles out health information management professionals - Health Information Managers (HIMs) and Clinical Coders (CCs) - as the key professional group who need to step up and seize the initiative, get conversations going, form partnerships, do research and publish findings, so the knowledge and insights that the health information management profession has the potential to offer are not only more widely known and understood but also more useful to others working in the healthcare arena. Hoyle calls on health information management professionals to step out from behind the scenes and take responsibility for the ethical use of the information they help produce. Hoyle's words resonated powerfully with me, particularly with respect to the clinical coding workforce in Australia, which is made up of trained CCs and qualified HIMs. In a truly ethical environment, HIMs and CCs would not be asked to meet performance indicators for increased funding metrics or to change codes to avoid triggering certain indicators; they would simply be asked to ensure complete, accurate coding for every episode of care. This is what ethical leadership would look like. I am concerned about our clinical coding workforce. I am now asking, are our CCs and HIMs up to the task of taking back absolute and unchallenged ownership of their particular skill set, which makes them the keepers of the clinical coding standards and the experts in accurate and complete code assignment?

Paying for efficiency: Incentivising same-day discharges in the English NHS.

We study a pay-for-efficiency scheme that encourages hospitals to admit and discharge patients on the same calendar day when clinically appropriate. Since 2010, hospitals in the English NHS are incentivised by a higher price for patients treated as same-day discharge than for overnight stays, despite the former being less costly. We analyse administrative data for patients treated during 2006-2014 for 191 conditions for which same-day discharge is clinically appropriate - of which 32 are incentivised. Using difference-in-difference and synthetic control methods, we find that the policy had generally a positive impact with a statistically significant effect in 14 out of the 32 conditions. The median elasticity is 0.24 for planned and 0.01 for emergency conditions. Condition-specific design features explain some, but not all, of the differential responses.