Poor reporting limited consideration of EDI in the Australian guidelines for the clinical care of people with COVID-19.

OBJECTIVES: Actively addressing issues of equity, diversity, and inclusion (EDI) in healthcare guidelines provides an important avenue ensure that individuals and communities receive high-quality healthcare that meets their needs. In 2020, the National Clinical Evidence Taskforce was charged with developing Australian living guidelines for COVID-19 (the Guidelines). It was intended that the Guidelines would consider the biological and social determinants of health (BSDH) underpinning evidence-based recommendations for of the treatment of COVID-19. The objective of this paper is to describe the evidence available on BSDH that is reported in published trials of disease-modifying therapies for COVID-19. STUDY DESIGN AND SETTING: Published papers of randomized controlled trials that informed clinical recommendations (for and against drug therapies for COVID-19) in the Guidelines were reviewed retrospectively using a case series design. We extracted reported characteristics relating to BSDH. These included age, sex, gender, geographical location, ethnicity (including indigenous), disability, migrant status, income, education, employment, and social support. A descriptive analysis was conducted to illustrate the characteristics available for use in guideline development. RESULTS: A total of 115 peer-reviewed papers describing randomized control trials of drug interventions for the treatment of COVID-19 were included. BSDH characteristics were poorly reported. Geographical location of the study was the only category reported in all papers. While age and sex were reported in most papers (n = 109 and 108, respectively), ethnicity was reported in only one-third of papers (n = 40), social support was reported in only three papers, and employment in one paper. No paper reported on gender, disability, migrant status, income, or education. CONCLUSION: Consideration of EDI issues is a crucial component of guideline development. Although these issues were widely recognized to impact on health outcomes from COVID-19, reporting of these characteristics was poor in COVID trials. Urgent action is needed to improve reporting of EDI characteristics if they are to be meaningfully considered in guideline processes, and health inequity is overcome.

What could health technology assessment learn from living clinical practice guidelines?

A "living" approach to clinical practice guidelines is when the identification, appraisal and synthesis of evidence is maintained and repeated at an agreed frequency, with a clear process for when and how new evidence is to be incorporated. The value of a living approach to guidelines was emphasised during the COVID-19 pandemic when health professionals and policymakers needed to make decisions regarding patient care in the context of a nascent but rapidly evolving evidence base. In this perspective, we draw on our recent experience developing Australian and international living guidelines and reflect on the feasibility of applying living guideline methods and processes to a lifecycle approach to health technology assessment (HTA). We believe the opportunities and challenges of adopting a living approach in HTA fall into five key themes: identification, appraisal and synthesis of evidence; optimising the frequency of updates; embedding ongoing multi-stakeholder engagement; linking the emergence of new evidence to reimbursement; and system capacity to support a living approach. We acknowledge that the suitability of specific living approaches to HTA will be heavily influenced by the type of health technology, its intended use in the health system, local reimbursement pathways, and other policy settings. But we believe that the methods and processes applied successfully to guideline development to manage evidentiary uncertainty could be applied in the context of HTA and reimbursement decision-making to help manage similar sources of uncertainty.

Using the WHO-INTEGRATE evidence-to-decision framework to develop recommendations for induction of labour.

BACKGROUND: In 2019, WHO prioritized updating recommendations relating to three labour induction topics: labour induction at or beyond term, mechanical methods for labour induction, and outpatient labour induction. As part of this process, we aimed to review the evidence addressing factors beyond clinical effectiveness (values, human rights and sociocultural acceptability, health equity, and economic and feasibility considerations) to inform WHO Guideline Development Group decision-making using the WHO-INTEGRATE evidence-to-decision framework, and to reflect on how methods for identifying, synthesizing and integrating this evidence could be improved. METHODS: We adapted the framework to consider the key criteria and sub-criteria relevant to our intervention. We searched for qualitative and other evidence across a variety of sources and mapped the eligible evidence to country income setting and perspective. Eligibility assessment and quality appraisal of qualitative evidence syntheses was undertaken using a two-step process informed by the ENTREQ statement. We adopted an iterative approach to interpret the evidence and provided both summary and detailed findings to the decision-makers. We also undertook a review to reflect on opportunities to improve the process of applying the framework and identifying the evidence. RESULTS: Using the WHO-INTEGRATE framework allowed us to explore health rights and equity in a systematic and transparent way. We identified a lack of qualitative and other evidence from low- and middle-income settings and in populations that are most impacted by structural inequities or traditionally excluded from research. Our process review highlighted opportunities for future improvement, including adopting more systematic evidence mapping methods and working with social science researchers to strengthen theoretical understanding, methods and interpretation of the evidence. CONCLUSIONS: Using the WHO-INTEGRATE evidence-to-decision framework to inform decision-making in a global guideline for induction of labour, we identified both challenges and opportunities relating to the lack of evidence in populations and settings of need and interest; the theoretical approach informing the development and application of WHO-INTEGRATE; and interpretation of the evidence. We hope these insights will be useful for primary researchers as well as the evidence synthesis and health decision-making communities, and ultimately contribute to a reduction in health inequities.

Building a bright, evidence-informed future: a conversation starter from the incoming editors.

Health Research and Policy Systems (HARPS) has gone from strength to strength since it was established in 2003. As new Editors-in-Chief, we look forward to a bright future for HARPS, and we would like to start a conversation with you, HARPS readers, authors, editors and others, about how HARPS can best support ongoing progress and debate on evidence-informed health research policy and systems, particularly in developing countries. As a starting point for discussion, we would like to highlight three areas that we are passionate about, namely supporting an integrated community of researchers and policy-makers; building a focus on how health research and policy systems can support achievement of the Sustainable Development Goals; and strengthening our commitment to communicating and disseminating the work published in HARPS. We invite you to contribute your thoughts, ideas and suggestions on the future of HARPS, as we work together towards an evidence-informed future.

Does knowledge brokering improve the quality of rapid review proposals? A before and after study.

BACKGROUND: Rapid reviews are increasingly being used to help policy makers access research in short time frames. A clear articulation of the review's purpose, questions, scope, methods and reporting format is thought to improve the quality and generalisability of review findings. The aim of the study is to explore the effectiveness of knowledge brokering in improving the perceived clarity of rapid review proposals from the perspective of potential reviewers. To conduct the study, we drew on the Evidence Check program, where policy makers draft a review proposal (a pre knowledge brokering proposal) and have a 1-hour session with a knowledge broker, who re-drafts the proposal based on the discussion (a post knowledge brokering proposal). METHODS: We asked 30 reviewers who had previously undertaken Evidence Check reviews to examine the quality of 60 pre and 60 post knowledge brokering proposals. Reviewers were blind to whether the review proposals they received were pre or post knowledge brokering. Using a six-point Likert scale, reviewers scored six questions examining clarity of information about the review's purpose, questions, scope, method and format and reviewers' confidence that they could meet policy makers' needs. Each reviewer was allocated two pre and two post knowledge brokering proposals, randomly ordered, from the 60 reviews, ensuring no reviewer received a pre and post knowledge brokering proposal from the same review. RESULTS: The results showed that knowledge brokering significantly improved the scores for all six questions addressing the perceived clarity of the review proposal and confidence in meeting policy makers' needs; with average changes of 0.68 to 1.23 from pre to post across the six domains. CONCLUSIONS: This study found that knowledge brokering increased the perceived clarity of information provided in Evidence Check rapid review proposals and the confidence of reviewers that they could meet policy makers' needs. Further research is needed to identify how the knowledge brokering process achieves these improvements and to test the applicability of the findings in other rapid review programs.

Development and validation of SEER (Seeking, Engaging with and Evaluating Research): a measure of policymakers' capacity to engage with and use research.

BACKGROUND: Capacity building strategies are widely used to increase the use of research in policy development. However, a lack of well-validated measures for policy contexts has hampered efforts to identify priorities for capacity building and to evaluate the impact of strategies. We aimed to address this gap by developing SEER (Seeking, Engaging with and Evaluating Research), a self-report measure of individual policymakers' capacity to engage with and use research. METHODS: We used the SPIRIT Action Framework to identify pertinent domains and guide development of items for measuring each domain. Scales covered (1) individual capacity to use research (confidence in using research, value placed on research, individual perceptions of the value their organisation places on research, supporting tools and systems), (2) actions taken to engage with research and researchers, and (3) use of research to inform policy (extent and type of research use). A sample of policymakers engaged in health policy development provided data to examine scale reliability (internal consistency, test-retest) and validity (relation to measures of similar concepts, relation to a measure of intention to use research, internal structure of the individual capacity scales). RESULTS: Response rates were 55% (150/272 people, 12 agencies) for the validity and internal consistency analyses, and 54% (57/105 people, 9 agencies) for test-retest reliability. The individual capacity scales demonstrated adequate internal consistency reliability (alpha coefficients > 0.7, all four scales) and test-retest reliability (intra-class correlation coefficients > 0.7 for three scales and 0.59 for fourth scale). Scores on individual capacity scales converged as predicted with measures of similar concepts (moderate correlations of > 0.4), and confirmatory factor analysis provided evidence that the scales measured related but distinct concepts. Items in each of these four scales related as predicted to concepts in the measurement model derived from the SPIRIT Action Framework. Evidence about the reliability and validity of the research engagement actions and research use scales was equivocal. CONCLUSIONS: Initial testing of SEER suggests that the four individual capacity scales may be used in policy settings to examine current capacity and identify areas for capacity building. The relation between capacity, research engagement actions and research use requires further investigation.

The development of ORACLe: a measure of an organisation's capacity to engage in evidence-informed health policy.

BACKGROUND: Evidence-informed policymaking is more likely if organisations have cultures that promote research use and invest in resources that facilitate staff engagement with research. Measures of organisations' research use culture and capacity are needed to assess current capacity, identify opportunities for improvement, and examine the impact of capacity-building interventions. The aim of the current study was to develop a comprehensive system to measure and score organisations' capacity to engage with and use research in policymaking, which we entitled ORACLe (Organisational Research Access, Culture, and Leadership). METHOD: We used a multifaceted approach to develop ORACLe. Firstly, we reviewed the available literature to identify key domains of organisational tools and systems that may facilitate research use by staff. We interviewed senior health policymakers to verify the relevance and applicability of these domains. This information was used to generate an interview schedule that focused on seven key domains of organisational capacity. The interview was pilot-tested within four Australian policy agencies. A discrete choice experiment (DCE) was then undertaken using an expert sample to establish the relative importance of these domains. This data was used to produce a scoring system for ORACLe. RESULTS: The ORACLe interview was developed, comprised of 23 questions addressing seven domains of organisational capacity and tools that support research use, including (1) documented processes for policymaking; (2) leadership training; (3) staff training; (4) research resources (e.g. database access); and systems to (5) generate new research, (6) undertake evaluations, and (7) strengthen relationships with researchers. From the DCE data, a conditional logit model was estimated to calculate total scores that took into account the relative importance of the seven domains. The model indicated that our expert sample placed the greatest importance on domains (2), (3) and (4). CONCLUSION: We utilised qualitative and quantitative methods to develop a system to assess and score organisations' capacity to engage with and apply research to policy. Our measure assesses a broad range of capacity domains and identifies the relative importance of these capacities. ORACLe data can be used by organisations keen to increase their use of evidence to identify areas for further development.

The SPIRIT Action Framework: A structured approach to selecting and testing strategies to increase the use of research in policy.

The recent proliferation of strategies designed to increase the use of research in health policy (knowledge exchange) demands better application of contemporary conceptual understandings of how research shapes policy. Predictive models, or action frameworks, are needed to organise existing knowledge and enable a more systematic approach to the selection and testing of intervention strategies. Useful action frameworks need to meet four criteria: have a clearly articulated purpose; be informed by existing knowledge; provide an organising structure to build new knowledge; and be capable of guiding the development and testing of interventions. This paper describes the development of the SPIRIT Action Framework. A literature search and interviews with policy makers identified modifiable factors likely to influence the use of research in policy. An iterative process was used to combine these factors into a pragmatic tool which meets the four criteria. The SPIRIT Action Framework can guide conceptually-informed practical decisions in the selection and testing of interventions to increase the use of research in policy. The SPIRIT Action Framework hypothesises that a catalyst is required for the use of research, the response to which is determined by the capacity of the organisation to engage with research. Where there is sufficient capacity, a series of research engagement actions might occur that facilitate research use. These hypotheses are being tested in ongoing empirical work.

Barriers to and enablers of evidence-based practice in perinatal care in the SEA-ORCHID project.

RATIONALE, AIMS AND OBJECTIVES: The South-East Asia Optimising Reproductive and Child Health in Developing Countries (SEA-ORCHID) project aimed to improve health outcomes for mothers and babies in nine hospitals in South-East Asia by supporting evidence-based perinatal health care. In this research, we aimed to identify and explore the factors that may have acted as barriers to or enablers of evidence-based practice change at each of the hospitals. METHODS: During the final 6 months of the intervention phase of the project, semi-structured, face-to-face interviews were undertaken with 179 nurses, midwives and doctors from the maternal and neonatal departments at each of the nine participating South-East Asian hospitals. RESULTS: The interviews identified several factors that participants believed had a substantial impact on the effectiveness of the SEA-ORCHID intervention. These included knowledge, skills, hierarchy, multidisciplinarity and leadership, beliefs about consequences, resources, and the nature of the behaviours. The success of the SEA-ORCHID intervention in improving practice may reflect the extent to which tailored strategies were effective in overcoming these barriers. CONCLUSION: Effective interventions to align practice with evidence rely on identifying and addressing barriers to practice change. The barriers identified in this study may be useful for those designing similar clinical practice improvement projects, as well as for continued efforts to improve practice in the SEA-ORCHID hospitals.

Supporting evidence-based health care in crises: what information do humanitarian organizations need?

In crisis situations, there is an enormous burden of disease and very limited resources. To achieve the best possible health outcomes in these situations and ensure that scarce resources are not wasted, knowledge from health research needs to be translated into practice. We investigated what information from health research was needed by humanitarian aid workers in crisis settings and how it could be best provided. Semistructured interviews were conducted by telephone with 19 humanitarian aid workers from a range of organizations around the world and the results analyzed thematically. Participants identified a clear and currently unmet need for access to high-quality health research to support evidence-based practice in crisis situations. They emphasized that research into delivery of health care was potentially more valuable than research into the effectiveness of particular clinical interventions and highlighted the importance of including contextual information to enable the relevance of the research to be assessed. They suggested that providers of health research information and humanitarian aid organizations work together to develop these resources.

Building capacity for evidence generation, synthesis and implementation to improve the care of mothers and babies in South East Asia: methods and design of the SEA-ORCHID Project using a logical framework approach.

BACKGROUND: Rates of maternal and perinatal mortality remain high in developing countries despite the existence of effective interventions. Efforts to strengthen evidence-based approaches to improve health in these settings are partly hindered by restricted access to the best available evidence, limited training in evidence-based practice and concerns about the relevance of existing evidence. South East Asia--Optimising Reproductive and Child Health in Developing Countries (SEA-ORCHID) was a five-year project that aimed to determine whether a multifaceted intervention designed to strengthen the capacity for research synthesis, evidence-based care and knowledge implementation improved clinical practice and led to better health outcomes for mothers and babies. This paper describes the development and design of the SEA-ORCHID intervention plan using a logical framework approach. METHODS: SEA-ORCHID used a before-and-after design to evaluate the impact of a multifaceted tailored intervention at nine sites across Thailand, Malaysia, Philippines and Indonesia, supported by three centres in Australia. We used a logical framework approach to systematically prepare and summarise the project plan in a clear and logical way. The development and design of the SEA-ORCHID project was based around the three components of a logical framework (problem analysis, project plan and evaluation strategy). RESULTS: The SEA-ORCHID logical framework defined the project's goal and purpose (To improve the health of mothers and babies in South East Asia and To improve clinical practice in reproductive health in South East Asia), and outlined a series of project objectives and activities designed to achieve these. The logical framework also established outcome and process measures appropriate to each level of the project plan, and guided project work in each of the participating countries and hospitals. CONCLUSIONS: Development of a logical framework in the SEA-ORCHID project enabled a reasoned, logical approach to the project design that ensured the project activities would achieve the desired outcomes and that the evaluation plan would assess both the process and outcome of the project. The logical framework was also valuable over the course of the project to facilitate communication, assess progress and build a shared understanding of the project activities, purpose and goal.

Improving capacity for evidence-based practice in South East Asia: evaluating the role of research fellowships in the SEA-ORCHID Project.

BACKGROUND: Fellowships are a component of many professional education programs. They provide opportunities to develop skills and competencies in an environment where time is protected and resources and technical support are more readily available. The SEA-ORCHID fellowships program aimed to increase capacity for evidence-based practice and research synthesis, and to encourage fellows to become leaders in these areas. METHODS: Fellows included doctors, nurses, midwives and librarians working in the maternal and neonatal areas of nine hospitals in South East Asia. Fellowships were undertaken in Australia and involved specific outputs related to evidence-based practice or research synthesis. Training and support was tailored according to the type of output and the fellow's experience and expertise. We evaluated the fellowships program quantitatively and qualitatively through written evaluations, interviews and follow-up of fellowship activities. RESULTS: During 2006-07, 23 fellows from Thailand, Indonesia, Malaysia and the Philippines undertook short-term fellowships (median four weeks) in Australia. The main outputs were drafts of Cochrane systematic reviews, clinical practice guidelines and protocols for randomised trials, and training materials to support evidence-based practice. Protocols for Cochrane systematic reviews were more likely to be completed than other outcomes. The fellows identified several components that were critical to the program's overall success; these included protected time, tailored training, and access to technical expertise and resources. On returning home, fellows identified a lack of time and limited access to the internet and evidence-based resources as barriers to completing their outputs. The support of colleagues and senior staff was noted as an important enabler of progress, and research collaborators from other institutions and countries were also important sources of support. CONCLUSIONS: The SEA-ORCHID fellowships program provided protected time to work on an output which would facilitate evidence-based practice. While the fellows faced substantial barriers to completing their fellowship outputs once they returned home, these fellowships resulted in a greater understanding, enthusiasm and skills for evidence-based practice. The experience of the SEA-ORCHID fellowships program may be useful for other initiatives aiming to build capacity in evidence-based practice.