Network evaluation of an innovation platform in continuous quality improvement in Australian Indigenous primary healthcare.

BACKGROUND: From 2014 to 2019, the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI) was evaluated as an innovation platform focusing on continuous quality improvement in Indigenous Australian primary healthcare. Although social network analysis (SNA) is a recognized method for evaluating the functioning, collaboration and effectiveness of innovation platforms, applied research is limited. This study applies SNA to evaluate the CRE-IQI's functioning as an innovation platform. METHODS: Two surveys (2017, 2019) were conducted using social survey and network methods. Survey items covered respondent characteristics, their perceptions of the CRE-IQI's performance, and its impact and sociometric relationships. Members' relationship information was captured for the CRE-IQI at three time points, namely start (retrospectively), midpoint and final year, on three network types (knew, shared information, collaborated). SNA software was used to compute standard network metrics including diameter, density and centrality, and to develop visualizations. Survey and network results were addressed in a workshop held by members to develop improvement strategies. RESULTS: The response rate was 80% in 2017 and 65% in 2019 (n = 49 and 47, respectively). Between 2017 and 2019, respondents' mean ratings of the CRE-IQI's functioning and achievements in meeting its goals were sustained. They perceived the CRE-IQI as multidisciplinary, having effective management and governance, and incorporating Indigenous research leadership, representation and ways of working. Respondents recognized high levels of trust amongst members, rated "good communication and coordination with participants" highly, and "facilitating collaboration" as the CRE's most strongly recognized achievement. In collaboration and information-sharing networks, average path length remained low in 2017 and 2019, indicating good small-world network properties for relaying information. On average, respondents shared information and collaborated with more CRE members in 2017 than 2019. However, in both 2017 and 2019 there were new collaborations and information-sharing outside of direct collaborations. CRE-IQI outcomes included: evidence generation; knowledge transfer and skills development in quality improvement; research capacity-building, career development; mentoring; grant support; development of new projects; health service support; and policy impact. CONCLUSIONS: This study shows the utility of network analysis in evaluating the functioning, and collaboration, at the individual, organizational and health system levels, of an innovation platform, and adds to our understanding of factors enabling successful innovation platforms.

Collaboration and knowledge generation in an 18-year quality improvement research programme in Australian Indigenous primary healthcare: a coauthorship network analysis.

OBJECTIVES: Though multidisciplinary research networks support the practice and effectiveness of continuous quality improvement (CQI) programmes, their characteristics and development are poorly understood. In this study, we examine publication outputs from a research network in Australian Indigenous primary healthcare (PHC) to assess to what extent the research network changed over time. SETTING: Australian CQI research network in Indigenous PHC from 2002 to 2019. PARTICIPANTS: Authors from peer-reviewed journal articles and books published by the network. DESIGN: Coauthor networks across four phases of the network (2002-2004; 2005-2009; 2010-2014; 2015-2019) were constructed based on author affiliations and examined using social network analysis methods. Descriptive characteristics included organisation types, Indigenous representation, gender, student authorship and thematic research trends. RESULTS: We identified 128 publications written by 308 individual authors from 79 different organisations. Publications increased in number and diversity over each funding phase. During the final phase, publication outputs accelerated for organisations, students, project officers, Indigenous and female authors. Over time there was also a shift in research themes to encompass new clinical areas and social, environmental or behavioural determinants of health. Average degree (8.1), clustering (0.81) and diameter (3) indicated a well-connected network, with a core-periphery structure in each phase (p≤0.03) rather than a single central organisation (degree centralisation=0.55-0.65). Academic organisations dominated the core structure in all funding phases. CONCLUSION: Collaboration in publications increased with network consolidation and expansion. Increased productivity was associated with increased authorship diversity and a decentralised network, suggesting these may be important factors in enhancing research impact and advancing the knowledge and practice of CQI in PHC. Publication diversity and growth occurred mainly in the fourth phase, suggesting long-term relationship building among diverse partners is required to facilitate participatory research in CQI. Despite improvements, further work is needed to address inequities in female authorship and Indigenous authorship.

Using an adaptive, codesign approach to strengthen clinic-level immunisation services in Khayelitsha, Western Cape Province, South Africa.

INTRODUCTION: Optimal immunisation programme service delivery and childhood vaccine coverage remains an ongoing challenge in South Africa. Previous health systems approaches have made recommendations on how to address identified barriers but detailed local implementation studies are lacking. This study aimed to improve immunisation service delivery in children under 24 months in Khayelitsha, Western Cape Province using an adaptive, co-design approach to assess and improve childhood immunisation service delivery at the clinic level. METHODS: A rapid, adaptive approach to identification of barriers and assessment of current childhood immunisation service delivery was developed with three clinics in Khayelitsha, Western Cape Province. This informed a short co-design process with key stakeholders and service providers to develop local interventions targeted at high priority barriers. Interventions were implemented for 4-6 months and evaluated using theory-based evaluation tools. Clinic service delivery, satisfaction and changes to clinic processes and parent engagement and knowledge were measured. RESULTS: Interventions developed included weekly community immunisation education radio sessions, daily clinic health talks, immunisation education and promotion materials and service provider and parent quality checklists. Evaluation post-intervention showed improvement in parents'/guardians' knowledge about immunisation, parent engagement and service provider commitment to improvement in service quality. Radio sessions and immunisation education and communication materials were deemed most useful by parents and providers. CONCLUSION: Immunisation service delivery can be strengthened using an adaptive, clinic-led assessment process which can effectively identify barriers, inform co-designed interventions and be evaluated over a short period. This approach provides a framework to guide future local participatory action research to more effectively improve childhood immunisation service delivery and other child health services in under-resourced settings.

What was the impact of a participatory research project in Australian Indigenous primary healthcare services? Applying a comprehensive framework for assessing translational health research to Lessons for the Best.

OBJECTIVES: To (1) apply the Framework to Assess the Impact from Translational health research (FAIT) to Lessons from the Best to Better the Rest (LFTB), (2) report on impacts from LFTB and (3) assess the feasibility and outcomes from a retrospective application of FAIT. SETTING: Three Indigenous primary healthcare (PHC) centres in the Northern Territory, Australia; project coordinating centre distributed between Townsville, Darwin and Cairns and the broader LFTB learning community across Australia. PARTICIPANTS: LFTB research team and one representative from each PHC centre. PRIMARY AND SECONDARY OUTCOME MEASURES: Impact reported as (1) quantitative metrics within domains of benefit using a modified Payback Framework, (2) a cost-consequence analysis given a return on investment was not appropriate and (3) a narrative incorporating qualitative evidence of impact. Data were gathered through in-depth stakeholder interviews and a review of project documentation, outputs and relevant websites. RESULTS: LFTB contributed to knowledge advancement in Indigenous PHC service delivery; enhanced existing capacity of health centre staff, researchers and health service users; enhanced supportive networks for quality improvement; and used a strengths-based approach highly valued by health centres. LFTB also leveraged between $A1.4 and $A1.6 million for the subsequent Leveraging Effective Ambulatory Practice (LEAP) Project to apply LFTB learnings to resource development and creation of a learning community to empower striving PHC centres. CONCLUSION: Retrospective application of FAIT to LFTB, although not ideal, was feasible. Prospective application would have allowed Indigenous community perspectives to be included. Greater appreciation of the full benefit of LFTB including a measure of return on investment will be possible when LEAP is complete. Future assessments of impact need to account for the limitations of fully capturing impact when intermediate/final impacts have not yet been realised and captured.

Principles guiding ethical research in a collaboration to strengthen Indigenous primary healthcare in Australia: learning from experience.

INTRODUCTION: Indigenous communities worldwide are leading calls for all research involving Indigenous people to be underpinned by values and principles articulated by them. Many researchers are explicitly adopting these principles to guide what, where, how and when research is undertaken with Indigenous people. With critical reflection to support the implementation of such principles largely absent from published literature, this paper explores both the implementation of, and the outcomes from a set of guiding principles used in a large-scale Australian research collaboration to improve Indigenous health. METHODS: In this inductive qualitative study, we adopted a principles-focused evaluation approach. Based on interviews with 35 actors in the collaboration and a review of project documents, we generated themes that were then iteratively discussed, refined and categorised into (1) 'strategies'-activities by which implementation of our guiding principles were recognised; (2) 'outcomes'-results seen from implementing the principles and (3) 'conditions'-aspects of the context that facilitated and constrained implementation of the principles. RESULTS: Respondents found it difficult to articulate how the guiding principles were actually implemented, and frequently referred to them as part of the fabric of the collaboration. They viewed the set of principles as mutually reinforcing, and as providing a rudder for navigating complexity and conflict. Implementation of the principles occurred through five strategies-honouring the principles; being dynamic and adaptable; sharing and dispersing leadership; collaborating purposefully and adopting a culture of mutual learning. Outcomes included increased Indigenous leadership and participation; the ability to attract principled and values-driven researchers and stakeholders, and the development of trusting and respectful relationships. The conditions that facilitated the implementation of the principles were collaborating over time; an increasing number of Indigenous researchers and taking an 'innovation platform' approach. CONCLUSION: Our findings show that principles guiding collaborations are valuable in providing a focus, direction and a way of working together when they are collaboratively developed, hold genuine meaning for all members and are implemented within a culture of continuous critical reflection, learning and adaptation, with ongoing reinterpretation of the principles over time.

Using developmental evaluation to enhance continuous reflection, learning and adaptation of an innovation platform in Australian Indigenous primary healthcare.

Effective efforts to strengthen health systems need diverse, multi-stakeholder networks working together on complex or 'wicked' problems such as prevention and control of chronic diseases, solutions to which go beyond the role and capability of one organisation. The contextual complexities inherent in 'wicked' problems mean that solutions warrant a systems approach that encompasses innovation and new ways of thinking about, facilitating and implementing collective decision-making processes and change practices.Innovation platforms are a mechanism for facilitating communication and collaboration among diverse stakeholders, promoting joint action and stimulating innovation. Developmental evaluation is an approach that is increasingly being used to evaluate innovative and emergent programmes and projects, as it enables evaluators to provide real-time feedback so that evaluation findings can be used to guide development and adaptations. Developmental evaluation emphasises learning and adaptation, and aligns well with the implementation of innovation platforms that have continuous reflection, learning and adaptation as a specific design principle.Here, we outline our rationale for applying a developmental evaluation to enhance the formation, functioning and outcomes of an innovation platform aimed at accelerating and strengthening large-scale quality improvement efforts in Australian Aboriginal and Torres Strait Islander primary healthcare. We provide examples to explain how the developmental evaluation findings were used for adaptation of the innovation platform and assess to what extent our application of developmental evaluation was consistent with, and reflective of, its essential principles.Our evaluation aligned strongly with the principles of developmental evaluation, and the approach we took was well suited to situations with a developmental purpose, innovation niche and complexity such as innovation platforms. As a result, along with the increasing interest in multi-stakeholder platforms (e.g. innovation platforms) and the inherent challenges with evaluating these complex networks, we anticipate our use of this approach being of interest globally.

Comparing and contrasting 'innovation platforms' with other forms of professional networks for strengthening primary healthcare systems for Indigenous Australians.

Efforts to strengthen health systems require the engagement of diverse, multidisciplinary stakeholder networks. Networks provide a forum for experimentation and knowledge creation, information exchange and the spread of good ideas and practice. They might be useful in addressing complex issues or 'wicked' problems, the solutions to which go beyond the control and scope of any one agency. Innovation platforms are proposed as a novel type of network because of their diverse stakeholder composition and focus on problem solving within complex systems. Thus, they have potential applicability to health systems strengthening initiatives, even though they have been predominantly applied in the international agricultural development sector. In this paper, we compare and contrast the concept of innovation platforms with other types of networks that can be used in efforts to strengthen primary healthcare systems, such as communities of practice, practice-based research networks and quality improvement collaboratives. We reflect on our ongoing research programme that applies innovation platform concepts to drive large-scale quality improvement in primary healthcare for Aboriginal and Torres Strait Islander Australians and outline our plans for evaluation. Lessons from our experience will find resonance with others working on similar initiatives in global health.

Encouraging translation and assessing impact of the Centre for Research Excellence in Integrated Quality Improvement: rationale and protocol for a research impact assessment.

INTRODUCTION: There is growing recognition among health researchers and funders that the wider benefits of research such as economic, social and health impacts ought to be assessed and valued alongside academic outputs such as peer-reviewed papers. Research translation needs to increase and the pathways to impact ought to be more transparent. These processes are particularly pertinent to the Indigenous health sector given continued concerns that Indigenous communities are over-researched with little corresponding improvement in health outcomes. This paper describes the research protocol of a mixed methods study to apply FAIT (Framework to Assess the Impact from Translational health research) to the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI). FAIT will be applied to five selected CRE-IQI Flagship projects to encourage research translation and assess the wider impact of that research. METHODS AND ANALYSIS: Phase I will develop a modified programme logic model for each Flagship project including identifying process, output and impact metrics so progress can be monitored. A scoping review will inform potential benefits. In phase II, programme logic models will be updated to account for changes in the research pathways over time. Audit and feedback will be used to encourage research translation and collect evidence of achievement of any process, output and interim impacts. In phase III, three proven methodologies for measuring research impact-Payback, economic assessment and narratives-will be applied. Data on the application of FAIT will be collected and analysed to inform and improve FAIT's performance. ETHICS AND DISSEMINATION: This study is funded by a nationally competitive grant (ID 1078927) from the Australian National Health and Medical Research Council. Ethics approval was obtained from the University of Newcastle's Human Research Ethics Committee (ID: H-2017-0026). The results from the study will be presented in several peer-reviewed publications, through conference presentations and via social media.

Improving Health Promotion Using Quality Improvement Techniques in Australian Indigenous Primary Health Care.

Although some areas of clinical health care are becoming adept at implementing continuous quality improvement (CQI) projects, there has been limited experimentation of CQI in health promotion. In this study, we examined the impact of a CQI intervention on health promotion in four Australian Indigenous primary health care centers. Our study objectives were to (a) describe the scope and quality of health promotion activities, (b) describe the status of health center system support for health promotion activities, and (c) introduce a CQI intervention and examine the impact on health promotion activities and health centers systems over 2 years. Baseline assessments showed suboptimal health center systems support for health promotion and significant evidence-practice gaps. After two annual CQI cycles, there were improvements in staff understanding of health promotion and systems for planning and documenting health promotion activities had been introduced. Actions to improve best practice health promotion, such as community engagement and intersectoral partnerships, were inhibited by the way health center systems were organized, predominately to support clinical and curative services. These findings suggest that CQI can improve the delivery of evidence-based health promotion by engaging front line health practitioners in decision-making processes about the design/redesign of health center systems to support the delivery of best practice health promotion. However, further and sustained improvements in health promotion will require broader engagement of management, senior staff, and members of the local community to address organizational and policy level barriers.

Determinants and Gaps in Preventive Care Delivery for Indigenous Australians: A Cross-sectional Analysis.

BACKGROUND: Potentially preventable chronic diseases are the greatest contributor to the health gap between Aboriginal and Torres Strait Islander peoples and non--Indigenous Australians. Preventive care is important for earlier detection and control of chronic disease, and a number of recent policy initiatives have aimed to enhance delivery of preventive care. We examined documented delivery of recommended preventive services for Indigenous peoples across Australia and investigated the influence of health center and client level factors on adherence to best practice guidelines. METHODS: Clinical audit data from 2012 to 2014 for 3,623 well adult clients (aged 15-54) of 101 health centers from four Australian states and territories were analyzed to determine adherence to delivery of 26 recommended preventive services classified into five different modes of care on the basis of the way in which they are delivered (e.g., basic measurement; laboratory tests and imaging; assessment and brief interventions, eye, ear, and oral checks; follow-up of abnormal findings). Summary statistics were used to describe the delivery of each service item across jurisdictions. Multilevel regression models were used to quantify the variation in service delivery attributable to health center and client level factors and to identify factors associated with higher quality care. RESULTS: Delivery of recommended preventive care varied widely between service items, with good delivery of most basic measurements but poor follow-up of abnormal findings. Health center characteristics were associated with most variation. Higher quality care was associated with Northern Territory location, urban services, and smaller service population size. Client factors associated with higher quality care included age between 25 and 34 years, female sex, and more regular attendance. CONCLUSION: Wide variation in documented preventive care delivery, poor follow-up of abnormal findings, and system factors that influence quality of care should be addressed through continuous quality improvement approaches that engage stakeholders at multiple levels (including, for example, access to care in the community, appropriate decision support for practitioners, and financial incentives and context appropriate guidelines).

Responses of Aboriginal and Torres Strait Islander Primary Health-Care Services to Continuous Quality Improvement Initiatives.

BACKGROUND: Indigenous primary health-care (PHC) services participating in continuous quality improvement (CQI) cycles show varying patterns of performance over time. Understanding this variation is essential to scaling up and sustaining quality improvement initiatives. The aim of this study is to examine trends in quality of care for services participating in the ABCD National Research Partnership and describe patterns of change over time and examine health service characteristics associated with positive and negative trends in quality of care. SETTING AND PARTICIPANTS: PHC services providing care for Indigenous people in urban, rural, and remote northern Australia that had completed at least three annual audits of service delivery for at least one aspect of care (n = 73). METHODS/DESIGN: Longitudinal clinical audit data from use of four clinical audit tools (maternal health, child health, preventive health, Type 2 diabetes) between 2005 and 2013 were analyzed. Health center performance was classified into six patterns of change over time: consistent high improvement (positive), sustained high performance (positive), decline (negative), marked variability (negative), consistent low performance (negative), and no specific increase or decrease (neutral). Backwards stepwise multiple logistic regression analyses were used to examine the associations between health service characteristics and positive or negative trends in quality of care. RESULTS: Trends in quality of care varied widely between health services across the four audit tools. Regression analyses of health service characteristics revealed no consistent statistically significant associations of population size, remoteness, governance model, or accreditation status with positive or negative trends in quality of care. CONCLUSION: The variable trends in quality of care as reflected by CQI audit tools do not appear to be related to easily measurable health service characteristics. This points to the need for a deeper or more nuanced understanding of factors that moderate the effect of CQI on health service performance for the purpose of strengthening enablers and overcoming barriers to improvement.