Implementation of large, multi-site hospital interventions: a realist evaluation of strategies for developing capability.

BACKGROUND: This study presents guidelines for implementation distilled from the findings of a realist evaluation. The setting was local health districts in New South Wales, Australia that implemented three clinical improvement initiatives as part of a state-wide program. We focussed on implementation strategies designed to develop health professionals' capability to deliver value-based care initiatives for multisite programs. Capability, which increases implementers' ability to cope with unexpected scenarios is key to managing change. METHODS: We used a mixed methods realist evaluation which tested and refined program theories elucidating the complex dynamic between context (C), mechanism (M) and outcome (O) to determine what works, for whom, under what circumstances. Data was drawn from program documents, a realist synthesis, informal discussions with implementation designers, and interviews with 10 key informants (out of 37 identified) from seven sites. Data analysis employed a retroductive approach to interrogate the causal factors identified as contributors to outcomes. RESULTS: CMO statements were refined for four initial program theories: Making it Relevant- where participation in activities was increased when targeted to the needs of the staff; Investment in Quality Improvement- where engagement in capability development was enhanced when it was valued by all levels of the organisation; Turnover and Capability Loss- where the effects of staff turnover were mitigated; and Community-Wide Priority- where there was a strategy of spanning sites. From these data five guiding principles for implementers were distilled: (1) Involve all levels of the health system to effectively implement large-scale capability development, (2) Design capability development activities in a way that supports a learning culture, (3) Plan capability development activities with staff turnover in mind, (4) Increased capability should be distributed across teams to avoid bottlenecks in workflows and the risk of losing key staff, (5) Foster cross-site collaboration to focus effort, reduce variation in practice and promote greater cohesion in patient care. CONCLUSIONS: A key implementation strategy for interventions to standardise high quality practice is development of clinical capability. We illustrate how leadership support, attention to staff turnover patterns, and making activities relevant to current issues, can lead to an emergent learning culture.

Socio-environmental models of allied health disability support: an exploration of narrative experiences in the Australian National Disability and Insurance Scheme.

Objective The primary aims of this study were to explore: (1) manifestations of socio-environmental models of allied health support provision in the disability sector; and (2) narrative experiences of individual allied health professionals in the disability sector. Methods A narrative qualitative study using interviews from a purposive sample of two allied health professionals working in the disability sector explored manifestations of socio-environmental models of allied health support provision and their experiences from case examples. The key informants had more than 10 years of experience in the disability support services setting. Results Seven key themes exploring manifestations of socio-environmental models of allied health professional practice in the disability sector emerged: (1) dignity of risk; (2) models of care; (3) considerations when working in the supported person's environment; (4) goal-oriented work; (5) informed choice and informed consent; (6) reactive and flexible plans; and (7) training and education role. Conclusions Socio-environmental models of allied health support provision in the disability sector focus on empowering people with disability to achieve their goals. This may require displacement of cultural norms within the allied health professions. What is known about the topic? Socio-environmental models of allied health support provision in the disability sector focus on empowering people with disability to achieve their goals. What does this paper add? Displacement of cultural norms within the allied health professions may be needed to promote positive risk taking. Challenges for allied health professionals remain in navigating conflicting goals between clients and family members, empowering informed choice and consent, and working in uncontrolled environments. What are the implications for practitioners? Adopting training and education roles for clients, family members and carers when implementing National Disability and Insurance Scheme plans may represent one of the many pragmatic and flexible approaches to achieve people's goals.

Effectiveness of knowledge brokering and recommendation dissemination for influencing healthcare resource allocation decisions: A cluster randomised controlled implementation trial.

BACKGROUND: Implementing evidence into clinical practice is a key focus of healthcare improvements to reduce unwarranted variation. Dissemination of evidence-based recommendations and knowledge brokering have emerged as potential strategies to achieve evidence implementation by influencing resource allocation decisions. The aim of this study was to determine the effectiveness of these two research implementation strategies to facilitate evidence-informed healthcare management decisions for the provision of inpatient weekend allied health services. METHODS AND FINDINGS: This multicentre, single-blinded (data collection and analysis), three-group parallel cluster randomised controlled trial with concealed allocation was conducted in Australian and New Zealand hospitals between February 2018 and January 2020. Clustering and randomisation took place at the organisation level where weekend allied health staffing decisions were made (e.g., network of hospitals or single hospital). Hospital wards were nested within these decision-making structures. Three conditions were compared over a 12-month period: (1) usual practice waitlist control; (2) dissemination of written evidence-based practice recommendations; and (3) access to a webinar-based knowledge broker in addition to the recommendations. The primary outcome was the alignment of weekend allied health provision with practice recommendations at the cluster and ward levels, addressing the adoption, penetration, and fidelity to the recommendations. The secondary outcome was mean hospital length of stay at the ward level. Outcomes were collected at baseline and 12 months later. A total of 45 clusters (n = 833 wards) were randomised to either control (n = 15), recommendation (n = 16), or knowledge broker (n = 14) conditions. Four (9%) did not provide follow-up data, and no adverse events were recorded. No significant effect was found with either implementation strategy for the primary outcome at the cluster level (recommendation versus control ß 18.11 [95% CI -8,721.81 to 8,758.02] p = 0.997; knowledge broker versus control ß 1.24 [95% CI -6,992.60 to 6,995.07] p = 1.000; recommendation versus knowledge broker ß -9.12 [95% CI -3,878.39 to 3,860.16] p = 0.996) or ward level (recommendation versus control ß 0.01 [95% CI 0.74 to 0.75] p = 0.983; knowledge broker versus control ß -0.12 [95% CI -0.54 to 0.30] p = 0.581; recommendation versus knowledge broker ß -0.19 [-1.04 to 0.65] p = 0.651). There was no significant effect between strategies for the secondary outcome at ward level (recommendation versus control ß 2.19 [95% CI -1.36 to 5.74] p = 0.219; knowledge broker versus control ß -0.55 [95% CI -1.16 to 0.06] p = 0.075; recommendation versus knowledge broker ß -3.75 [95% CI -8.33 to 0.82] p = 0.102). None of the control or knowledge broker clusters transitioned to partial or full alignment with the recommendations. Three (20%) of the clusters who only received the written recommendations transitioned from nonalignment to partial alignment. Limitations include underpowering at the cluster level sample due to the grouping of multiple geographically distinct hospitals to avoid contamination. CONCLUSIONS: Owing to a lack of power at the cluster level, this trial was unable to identify a difference between the knowledge broker strategy and dissemination of recommendations compared with usual practice for the promotion of evidence-informed resource allocation to inpatient weekend allied health services. Future research is needed to determine the interactions between different implementation strategies and healthcare contexts when translating evidence into healthcare practice. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry ACTRN12618000029291.

Applying a framework to assess the impact of cardiovascular outcomes improvement research.

BACKGROUND: Health and medical research funding agencies are increasingly interested in measuring the impact of funded research. We present a research impact case study for the first four years of an Australian National Health and Medical Research Council funded Centre of Research Excellence in Cardiovascular Outcomes Improvement (2016-2020). The primary aim of this paper was to explore the application of a research impact matrix to assess the impact of cardiovascular outcomes improvement research. METHODS: We applied a research impact matrix developed from a systematic review of existing methodological frameworks used to measure research impact. This impact matrix was used as a bespoke tool to identify and understand various research impacts over different time frames. Data sources included a review of existing internal documentation from the research centre and publicly available information sources, informal iterative discussions with 10 centre investigators, and confirmation of information from centre grant and scholarship recipients. RESULTS: By July 2019, the impact on the short-term research domain category included over 41 direct publications, which were cited over 87 times (median journal impact factor of 2.84). There were over 61 conference presentations, seven PhD candidacies, five new academic collaborations, and six new database linkages conducted. The impact on the mid-term research domain category involved contributions towards the development of a national cardiac registry, cardiovascular guidelines, application for a Medicare Benefits Schedule reimbursement item number, introduction of patient-reported outcome measures into several databases, and the establishment of nine new industry collaborations. Evidence of long-term impacts were described as the development and use of contemporary management for aortic stenosis, a cardiovascular risk prediction model and prevention targets in several data registries, and the establishment of cost-effectiveness for stenting compared to surgery. CONCLUSIONS: We considered the research impact matrix a feasible tool to identify evidence of academic and policy impact in the short- to midterm; however, we experienced challenges in capturing long-term impacts. Cost containment and broader economic impacts represented another difficult area of impact to measure.

How do health professionals prioritize clinical areas for implementation of evidence into practice? A cross-sectional qualitative study.

AIM: The current study aimed to identify and understand the reasons why allied health professionals think certain areas of healthcare service provision are a high priority for implementation of evidence into practice. METHODS: A cross-sectional online survey using open-ended questions was conducted between April and May 2018 to identify potential areas for practice change and characterize how participants justified identified areas of priority. Eligible participants were invited by email and included allied health professionals from public or private health services, governance agencies and universities across Australia. Responses were analysed using qualitative content analysis. RESULTS: There were 149 surveys commenced with 146 respondents completing the survey. Of the 146 respondents, 128 were female, 17 male and one unknown. Most of the respondents were between 40 and 49 years old and had a master's degree. In total respondents from more than 13 different professions completed the survey with 110 respondents having more than 10 years of experience in allied health. Ten themes emerged outlining the main reasons respondents felt that their nominated areas of practice change were a high priority for action. These included closing gaps between practice and policy/recommendation/guideline; closing research evidence to practice gaps; improving access to services; perceived cost-effectiveness of service delivery; improving effectiveness of allied health services; current imbalance between service supply and demand; amount of resources involved in service delivery; extent of the health problem; areas of allied health care futility; and equality of workload across allied health professionals. CONCLUSION: The current research provides insights into the decision-making processes of allied health professionals when prioritizing areas of clinical practice for implementation of evidence into practice. Despite an appetite for evidence-based practice, behaviour change was not always implemented in a consistent and systematic manner. There was variability in the type and application of evidence used by allied health professionals to support clinical practice. Whether a more systematic approach to research translation fosters evidence uptake awaits confirmation. Also awaiting investigation are the economic and societal impacts of consistently implementing research-informed clinical decision-making.

Additional weekend allied health services reduce length of stay in subacute rehabilitation wards but their effectiveness and cost-effectiveness are unclear in acute general medical and surgical hospital wards: a systematic review.

QUESTION: Are additional weekend allied health services effective and cost-effective for acute general medical and surgical wards, and subacute rehabilitation hospital wards? DESIGN: Systematic review and meta-analysis of studies published between January 2000 and May 2017. Two reviewers independently screened studies for inclusion, extracted data, and assessed methodological quality. Meta-analyses were conducted for relative measures of effect estimates. PARTICIPANTS: Patients admitted to acute general medical and surgical wards, and subacute rehabilitation wards. INTERVENTION: All services delivered by allied health professionals during weekends (Saturday and/or Sunday). This study limited allied health professions to: occupational therapy, physiotherapy, social work, speech pathology, dietetics, art therapy, chiropractic, exercise physiology, music therapy, oral health (not dentistry), osteopathy, podiatry, psychology, and allied health assistants. OUTCOME MEASURES: Hospital length of stay, hospital re-admission, adverse events, discharge destination, functional independence, health-related quality of life, and cost of hospital care. RESULTS: Nineteen articles (20 studies) were identified, comprising 10 randomised and 10 non-randomised trials. Physiotherapy was the most commonly investigated profession. A meta-analysis of randomised, controlled trials showed that providing additional weekend allied health services in subacute rehabilitation wards reduced hospital length of stay by 2.35days (95% CI 0.45 to 4.24, I2=0%), and may be a cost-effective way to improve function (SMD 0.09, 95% CI -0.01 to 0.19, I2=0%), and health-related quality of life (SMD 0.10, 95% CI -0.01 to 0.20, I2=0%). For acute general medical and surgical hospital wards, it was unclear whether the weekend allied health service model provided in the two identified randomised trials led to significant changes in measured outcomes. CONCLUSION: The benefit of providing additional allied health services is clearer in subacute rehabilitation settings than for acute general medical and surgical wards in hospitals. REGISTRATION: PROSPERO CRD76771. [Sarkies MN, White J, Henderson K, Haas R, Bowles J, Evidence Translation in Allied Health (EviTAH) Group (2018) Additional weekend allied health services reduce length of stay in subacute rehabilitation wards but their effectiveness and cost-effectiveness are unclear in acute general medical and surgical hospital wards: a systematic review. Journal of Physiotherapy 64: 142-158].

Implementation of evidence-based weekend service recommendations for allied health managers: a cluster randomised controlled trial protocol.

BACKGROUND: It is widely acknowledged that health policy and practice do not always reflect current research evidence. Whether knowledge transfer from research to practice is more successful when specific implementation approaches are used remains unclear. A model to assist engagement of allied health managers and clinicians with research implementation could involve disseminating evidence-based policy recommendations, along with the use of knowledge brokers. We developed such a model to aid decision-making for the provision of weekend allied health services. This protocol outlines the design and methods for a multi-centre cluster randomised controlled trial to evaluate the success of research implementation strategies to promote evidence-informed weekend allied health resource allocation decisions, especially in hospital managers. METHODS: This multi-centre study will be a three-group parallel cluster randomised controlled trial. Allied health managers from Australian and New Zealand hospitals will be randomised to receive either (1) an evidence-based policy recommendation document to guide weekend allied health resource allocation decisions, (2) the same policy recommendation document with support from a knowledge broker to help implement weekend allied health policy recommendations, or (3) a usual practice control group. The primary outcome will be alignment of weekend allied health service provision with policy recommendations. This will be measured by the number of allied health service events (occasions of service) occurring on weekends as a proportion of total allied health service events for the relevant hospital wards at baseline and 12-month follow-up. DISCUSSION: Evidence-based policy recommendation documents communicate key research findings in an accessible format. This comparatively low-cost research implementation strategy could be combined with using a knowledge broker to work collaboratively with decision-makers to promote knowledge transfer. The results will assist managers to make decisions on resource allocation, based on evidence. More generally, the findings will inform the development of an allied health model for translating research into practice. TRIAL REGISTRATION: This trial is registered with the Australian New Zealand Clinical Trials Registry (ANZCTR) ( ACTRN12618000029291 ). Universal Trial Number (UTN): U1111-1205-2621.

Impact of disinvestment from weekend allied health services across acute medical and surgical wards: 2 stepped-wedge cluster randomised controlled trials.

BACKGROUND: Disinvestment (removal, reduction, or reallocation) of routinely provided health services can be difficult when there is little published evidence examining whether the services are effective or not. Evidence is required to understand if removing these services produces outcomes that are inferior to keeping such services in place. However, organisational imperatives, such as budget cuts, may force healthcare providers to disinvest from these services before the required evidence becomes available. There are presently no experimental studies examining the effectiveness of allied health services (e.g., physical therapy, occupational therapy, and social work) provided on weekends across acute medical and surgical hospital wards, despite these services being routinely provided internationally. The aim of this study was to understand the impact of removing weekend allied health services from acute medical and surgical wards using a disinvestment-specific non-inferiority research design. METHODS AND FINDINGS: We conducted 2 stepped-wedge cluster randomised controlled trials between 1 February 2014 and 30 April 2015 among patients on 12 acute medical or surgical hospital wards spread across 2 hospitals. The hospitals involved were 2 metropolitan teaching hospitals in Melbourne, Australia. Data from n = 14,834 patients were collected for inclusion in Trial 1, and n = 12,674 in Trial 2. Trial 1 was a disinvestment-specific non-inferiority stepped-wedge trial where the 'current' weekend allied health service was incrementally removed from participating wards each calendar month, in a random order, while Trial 2 used a conventional non-inferiority stepped-wedge design, where a 'newly developed' service was incrementally reinstated on the same wards as in Trial 1. Primary outcome measures were patient length of stay (proportion staying longer than expected and mean length of stay), the proportion of patients experiencing any adverse event, and the proportion with an unplanned readmission within 28 days of discharge. The 'no weekend allied health service' condition was considered to be not inferior if the 95% CIs of the differences between this condition and the condition with weekend allied health service delivery were below a 2% increase in the proportion of patients who stayed in hospital longer than expected, a 2% increase in the proportion who had an unplanned readmission within 28 days, a 2% increase in the proportion who had any adverse event, and a 1-day increase in the mean length of stay. The current weekend allied health service included physical therapy, occupational therapy, speech therapy, dietetics, social work, and allied health assistant services in line with usual care at the participating sites. The newly developed weekend allied health service allowed managers at each site to reprioritise tasks being performed and the balance of hours provided by each professional group and on which days they were provided. Analyses conducted on an intention-to-treat basis demonstrated that there was no estimated effect size difference between groups in the proportion of patients staying longer than expected (weekend versus no weekend; estimated effect size difference [95% CI], p-value) in Trial 1 (0.40 versus 0.38; estimated effect size difference 0.01 [-0.01 to 0.04], p = 0.31, CI was both above and below non-inferiority margin), but the proportion staying longer than expected was greater with the newly developed service compared to its no weekend service control condition (0.39 versus 0.40; estimated effect size difference 0.02 [0.01 to 0.04], p = 0.04, CI was completely below non-inferiority margin) in Trial 2. Trial 1 and 2 findings were discordant for the mean length of stay outcome (Trial 1: 5.5 versus 6.3 days; estimated effect size difference 1.3 days [0.9 to 1.8], p < 0.001, CI was both above and below non-inferiority margin; Trial 2: 5.9 versus 5.0 days; estimated effect size difference -1.6 days [-2.0 to -1.1], p < 0.001, CI was completely below non-inferiority margin). There was no difference between conditions for the proportion who had an unplanned readmission within 28 days in either trial (Trial 1: 0.01 [-0.01 to 0.03], p = 0.18, CI was both above and below non-inferiority margin; Trial 2: -0.01 [-0.02 to 0.01], p = 0.62, CI completely below non-inferiority margin). There was no difference between conditions in the proportion of patients who experienced any adverse event in Trial 1 (0.01 [-0.01 to 0.03], p = 0.33, CI was both above and below non-inferiority margin), but a lower proportion of patients had an adverse event in Trial 2 when exposed to the no weekend allied health condition (-0.03 [-0.05 to -0.004], p = 0.02, CI completely below non-inferiority margin). Limitations of this research were that 1 of the trial wards was closed by the healthcare provider after Trial 1 and could not be included in Trial 2, and that both withdrawing the current weekend allied health service model and installing a new one may have led to an accommodation period for staff to adapt to the new service settings. Stepped-wedge trials are potentially susceptible to bias from naturally occurring change over time at the service level; however, this was adjusted for in our analyses. CONCLUSIONS: In Trial 1, criteria to say that the no weekend allied health condition was non-inferior to current weekend allied health condition were not met, while neither the no weekend nor current weekend allied health condition demonstrated superiority. In Trial 2, the no weekend allied health condition was non-inferior to the newly developed weekend allied health condition across all primary outcomes, and superior for the outcomes proportion of patients staying longer than expected, proportion experiencing any adverse event, and mean length of stay. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry ACTRN12613001231730 and ACTRN12613001361796.