Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation.

BACKGROUND: Stakeholder engagement has become widely accepted as a necessary component of guideline development and implementation. While frameworks for developing guidelines express the need for those potentially affected by guideline recommendations to be involved in their development, there is a lack of consensus on how this should be done in practice. Further, there is a lack of guidance on how to equitably and meaningfully engage multiple stakeholders. We aim to develop guidance for the meaningful and equitable engagement of multiple stakeholders in guideline development and implementation. METHODS: This will be a multi-stage project. The first stage is to conduct a series of four systematic reviews. These will (1) describe existing guidance and methods for stakeholder engagement in guideline development and implementation, (2) characterize barriers and facilitators to stakeholder engagement in guideline development and implementation, (3) explore the impact of stakeholder engagement on guideline development and implementation, and (4) identify issues related to conflicts of interest when engaging multiple stakeholders in guideline development and implementation. DISCUSSION: We will collaborate with our multiple and diverse stakeholders to develop guidance for multi-stakeholder engagement in guideline development and implementation. We will use the results of the systematic reviews to develop a candidate list of draft guidance recommendations and will seek broad feedback on the draft guidance via an online survey of guideline developers and external stakeholders. An invited group of representatives from all stakeholder groups will discuss the results of the survey at a consensus meeting which will inform the development of the final guidance papers. Our overall goal is to improve the development of guidelines through meaningful and equitable multi-stakeholder engagement, and subsequently to improve health outcomes and reduce inequities in health.

Challenges in implementing The Institute of Medicine systematic review standards.

In 2011, The Institute of Medicine (IOM) identified a set of methodological standards to improve the validity, trustworthiness, and usefulness of systematic reviews. These standards, based on a mix of theoretical principles, empiric evidence, and commonly considered best practices, set a high bar for authors of systematic reviews.Based on over 15 years of experience conducting systematic reviews, the Agency for Healthcare Research and Quality Evidence-based Practice Center (EPC) program has examined the EPC's adherence and agreement with the IOM standards. Even such a large program, with infrastructure and resource support, found challenges in implementing all of the IOM standards. We summarize some of the challenges in implementing the IOM standards as a whole and suggest some considerations for individual or smaller research groups needing to prioritize which standards to adhere to, yet still achieve the highest quality and utility possible for their systematic reviews.

Identifying research needs for improving health care.

Insights from systematic reviews can help new studies better meet the priorities and needs of patients and communities. However, systematic reviews unfortunately have not yet achieved this position to direct and guide new research studies. The Agency for Healthcare Research and Quality's Evidence-based Practice Center Program uses systematic reviews to identify gaps in current evidence and has developed a systematic process of prioritizing these gaps with stakeholder input into clearly defined "future research needs." Eight Evidence-based Practice Centers began to apply this effort in 2010 to various clinical and policy topics. Gaps that prevented systematic reviewers from answering central questions of the review may include insufficient studies on subpopulations, insufficient studies with appropriate comparators, lack of appropriate outcomes measured, and methods problems. Stakeholder panels, consisting of advocacy groups, patients, researchers, clinicians, funders, and policymakers, help refine the gaps through multiple conference calls and prioritization exercises. Each report highlights a focused set of 4 to 15 high-priority needs with an accompanying description of possible considerations for study design. Identification of high-priority research needs could potentially speed the development and implementation of high-priority, stakeholder-engaged research.

Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program.

OBJECTIVE: To describe a systematic approach for identifying, reporting, and synthesizing information to allow consistent and transparent consideration of the applicability of the evidence in a systematic review according to the Population, Intervention, Comparator, Outcome, Setting domains. STUDY DESIGN AND SETTING: Comparative effectiveness reviews need to consider whether available evidence is applicable to specific clinical or policy questions to be useful to decision makers. Authors reviewed the literature and developed guidance for the Effective Health Care program. RESULTS: Because applicability depends on the specific questions and needs of the users, it is difficult to devise a valid uniform scale for rating the overall applicability of individual studies or body of evidence. We recommend consulting stakeholders to identify the factors most relevant to applicability for their decisions. Applicability should be considered separately for benefits and harms. Observational studies can help determine whether trial populations and interventions are representative of "real world" practice. Reviewers should describe differences between available evidence and the ideally applicable evidence for the question being asked and offer a qualitative judgment about the importance and potential effect of those differences. CONCLUSION: Careful consideration of applicability may improve the usefulness of systematic reviews in informing practice and policy.