A multistakeholder development process to prioritize and translate COVID-19 health recommendations for patients, caregivers and the public. A case study of the COVID-19 recommendation map.

BACKGROUND AND OBJECTIVES: To develop a digital communication tool to improve the implementation of up-to-date COVID-19 recommendations. Specifically, to improve patient, caregiver and public understanding of healthcare recommendations on prevention, diagnoses and treatment. METHODS: Multi-stakeholder engagement design. In conjunction with the COVID-19 Recommendations and Gateway to Contextualization RecMap, we co-developed a stakeholder prioritization, drafting and editing process to enhance guideline communication and understanding. RESULTS: This paper presents the multi-stakeholder development process with three distinct plain language recommendation formats: formal recommendation, good practice statement, and additional guidance. Our case study of COVID-19 plain language recommendations PLRs addresses both public health interventions (e.g., vaccination, face masks) and clinical interventions (e.g., home pulse oximetry). CONCLUSION: This paper presents a novel approach to engaging stakeholders in improving the communication and understanding of published guidelines during the COVID-19 pandemic.

Standardized wording to improve efficiency and clarity of GRADE EtD frameworks in health guidelines.

BACKGROUND: Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence-to-Decision (EtDs) frameworks are increasingly applied by health guideline developers to improve the use of evidence and transparency of health recommendations. Typically, EtDs include 12 criteria but these are flexible and EtDs have been adapted to different types of health decisions. However, developers of health recommendations struggle with the content that they should include in the EtD. The goal of this work was to provide a standardized template that facilitate the development of GRADE EtDs in health guidelines and examples for practical training. METHODS: We began by establishing the need for standardized wording templates in 10 American Society of Hematology guidelines with over 250 recommendations. We drafted template wording, and examples, and sought iterative feedback from methodologists and guideline panels in this guideline and two additional guidelines. RESULTS: We generated templates for all EtD criteria describing the type of research evidence considered, ideally based on systematic reviews, using standardized reporting of effect size, integrating the certainty of evidence and additional considerations. We also produced templates to inform the completion of the EtD conclusions section that includes recommendations, justification, implementation considerations, monitoring and evaluation and research priorities. CONCLUSION: We developed a pragmatic and useful tool to support clarity, transparency, and efficiency of the guideline EtD process.

Comparing the usability of the World Health Organization's conventional tuberculosis guidelines to the eTB recommendations map: A two-arm superiority randomised controlled trial.

Best practices for the dissemination of global health guidelines has not undergone rigorous research. We used a new approach to digitizing World Health Organization (WHO) global tuberculosis guideline recommendations (eTB RecMap) and compared its usability to the conventional method of accessing TB recommendations using the WHO website. We conducted a two-arm superiority randomised controlled trial using a survey among global stakeholders who were past or planned future users of TB guidelines, recommendations, or policy advice. We assigned participants randomly (1:1) to complete an activity using the WHO eTB RecMap or the conventional website. The primary outcome was the accessibility of information and secondary outcomes understanding, satisfaction, and preference for one of the two formats. Between February 26 and August 29, 2021, we received 478 responses from stakeholders, of whom 244 (122 per group) were eligible and provided analysable results. Participants rated the eTB RecMap as more accessible, on average, when compared to the conventional website (on a seven-point scale, the mean difference {MD} was 0.9; 95% confidence interval {CI}: 0.6, 1.2; p < 0.001) and were more likely to correctly answer understanding questions. This is the first trial comparing digitized dissemination formats of health guideline recommendations. Stakeholders rated the WHO eTB RecMap as more accessible than the conventional WHO website for the tested recommendations. They also understood presented information better. The findings support better usability of TB information through the eTB RecMap and contribute to the effort to end the TB epidemic. Trial registration: This trial was registered with ClinicalTrials.gov (NCT04745897) on February 9, 2021.

Recommendation mapping of the World Health Organization's guidelines on tuberculosis: A new approach to digitizing and presenting recommendations.

OBJECTIVE: Having up-to-date health policy recommendations accessible in one location is in high demand by guideline users. We developed an easy to navigate interactive approach to organize recommendations and applied it to tuberculosis (TB) guidelines of the World Health Organization (WHO). STUDY DESIGN: We used a mixed-methods study design to develop a framework for recommendation mapping with seven key methodological considerations. We define a recommendation map as an online repository of recommendations from several guidelines on a condition, providing links to the underlying evidence and expert judgments that inform them, allowing users to filter and cross-tabulate the search results. We engaged guideline developers, users, and health software engineers in an iterative process to elaborate the WHO eTB recommendation map. RESULTS: Applying the seven-step framework, we included 228 recommendations, linked to 103 guideline questions and organized the recommendation map according to key components of the health question, including the original recommendations and rationale (https://who.tuberculosis.recmap.org/). CONCLUSION: The recommendation mapping framework provides the entire continuum of evidence mapping by framing recommendations within a guideline questions' population, interventions, and comparators domains. Recommendation maps should allow guideline developers to organize their work meaningfully, standardize the automated publication of guidelines through links to the GRADEpro guideline development tool, and increase their accessibility and usability.

Using GRADE evidence to decision frameworks to choose from multiple interventions.

BACKGROUND AND OBJECTIVE: Guideline development groups or other health care decision makers frequently encounter situations that require a simultaneous comparison of multiple interventions. This sometimes becomes apparent either when they identify questions of interest, before they formulate recommendations, or it may surface only when recommendations have already been formulated based on pairwise comparisons. METHODS: Using examples from the World Health Organization, the European Commission, and a professional society, we developed a flexible approach to developing recommendations when a multiple-intervention comparison (MC) is needed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence to decision (EtD) frameworks. We iteratively refined this approach through user testing and then included a module in GRADE's official software GRADEpro to test the approach in two real and one theoretical guideline recommendations. RESULTS: We found the approach feasible and that all EtD criteria should be considered in an MC approach. We judged that guideline development groups and other decision makers will benefit from the availability of a network meta-analyses (NMA) of intervention effects to support decisions; however, NMA supports only one of many criteria, that is, the balance of health benefits and harms, and is therefore helpful, but not essential to the approach we propose. When similar but not identical comparators are used to address MC, challenges may arise with intransitivity and the relative rankings of interventions. CONCLUSION: We successfully applied the MC approach and software module in generating recommendations across different scenarios and identified challenges. The MC approach allows guideline groups and other decision makers to transparently and critically assess multiple options for a given health question. Application of the approach by others may lead to refinement and allow for better understanding of its impact in developing recommendations and making choices.