Decision aids linked to evidence summaries and clinical practice guidelines: results from user-testing in clinical encounters.

BACKGROUND: Tools for shared decision-making (e.g. decision aids) are intended to support health care professionals and patients engaged in clinical encounters involving shared decision-making. However, decision aids are hard to produce, and onerous to update. Consequently, they often do not reflect best current evidence, and show limited uptake in practice. In response, we initiated the Sharing Evidence to Inform Treatment decisions (SHARE-IT) project. Our goal was to develop and refine a new generation of decision aids that are generically produced along digitally structured guidelines and evidence summaries. METHODS: Applying principles of human-centred design and following the International Patient Decision Aid Standards (IPDAS) and GRADE methods for trustworthy evidence summaries we developed a decision aid prototype in collaboration with the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence project (DECIDE). We iteratively user-tested the prototype in clinical consultations between clinicians and patients. Semi-structured interviews of participating clinicians and patients were conducted. Qualitative content analysis of both user-testing sessions and interviews was performed and results categorized according to a revised Morville's framework of user-experience. We made it possible to produce, publish and use these decision aids in an electronic guideline authoring and publication platform (MAGICapp). RESULTS: Direct observations and analysis of user-testing of 28 clinical consultations between physicians and patients informed four major iterations that addressed readability, understandability, usability and ways to cope with information overload. Participants reported that the tool supported natural flow of the conversation and induced a positive shift in consultation habits towards shared decision-making. We integrated the functionality of SHARE-IT decision aids in MAGICapp, which has since generated numerous decision aids. CONCLUSION: Our study provides a proof of concept that encounter decision aids can be generically produced from GRADE evidence summaries and clinical guidelines. Online authoring and publication platforms can help scale up production including continuous updating of electronic encounter decision aids, fully integrated with evidence summaries and clinical practice guidelines.

A framework for practical issues was developed to inform shared decision-making tools and clinical guidelines.

OBJECTIVES: The objective of the study was to develop and test feasibility of a framework of patient-important practical issues. STUDY DESIGN AND SETTING: Guidelines and shared decision-making tools help facilitate discussions about patient-important outcomes of care alternatives, but typically ignore practical issues patients consider when implementing care into their daily routines. Using grounded theory, practical issues in the HealthTalk.org registry and in Option Grids were identified and categorized into a framework. We integrated the framework into the MAGIC authoring and publication platform and digitally structured authoring and publication platform and appraised its use in The BMJ Rapid Recommendations. RESULTS: The framework included the following 15 categories: medication routine, tests and visits, procedure and device, recovery and adaptation, coordination of care, adverse effects, interactions and antidote, physical well-being, emotional well-being, pregnancy and nursing, costs and access, food and drinks, exercise and activities, social life and relationships, work and education, travel and driving. Implementation in 15 BMJ Rapid Recommendations added 283 issues to 35 recommendations. The most frequently used category was procedure and device, and the least frequent was social life and relationship. CONCLUSION: Adding practical issues systematically to evidence summaries is feasible and can inform guidelines and tools for shared decision-making. How this inclusion can improve patient-centered care remains to be determined.

Are systematic review and guideline development tools useful? A Guidelines International Network survey of user preferences.

BACKGROUND: There are now over 140 tools/programs that can assist in developing systematic reviews or clinical practice guidelines (CPGs). It is currently unclear which tools are used by systematic reviewers and CPG developers, which development processes they are used for, and what facilitators or barriers to their use exist. METHODS: To determine which tools are currently being used by systematic reviewers and CPG developers, an online survey was administered during July-August 2017. Guidelines International Network individual and organizational members were invited to participate. Survey questions focused on the nature and frequency of members' use of tools to support systematic review and CPG development. RESULTS: The overall response rate was 34%. The largest number of respondents developed one to five guidelines a year (48%). GRADEpro GDT was the most popular tool (26% of respondents) followed by Dropbox (16%) and RevMan (14%). From the options provided, the reason most respondents (85%) used particular tools was 'to be more efficient'. Most users stated they would use the tool again (95%), and 95% would recommend it to other organizations. However, respondents reported that tool efficiency and facilitators such as data sharing functionality were offset by their availability and cost, issues with structured data fields (that did not allow customization), and other technical and usability factors (e.g., features, workflows). CONCLUSION: The results of this survey provide a focus for discussing improvements in tools to meet the needs of systematic reviewers and CPG developers, and a basis from which to test the efficacy and appropriateness of various tools and platforms across a number of purposes and contexts.

A systematic review of trials evaluating success factors of interventions with computerised clinical decision support.

BACKGROUND: Computerised clinical decision support (CDS) can potentially better inform decisions, and it can help with the management of information overload. It is perceived to be a key component of a learning health care system. Despite its increasing implementation worldwide, it remains uncertain why the effect of CDS varies and which factors make CDS more effective. OBJECTIVE: To examine which factors make CDS strategies more effective on a number of outcomes, including adherence to recommended practice, patient outcome measures, economic measures, provider or patient satisfaction, and medical decision quality. METHODS: We identified randomised controlled trials, non-randomised trials, and controlled before-and-after studies that directly compared CDS implementation with a given factor to CDS without that factor by searching CENTRAL, MEDLINE, EMBASE, and CINAHL and checking reference lists of relevant studies. We considered CDS with any objective for any condition in any healthcare setting. We included CDS interventions that were either displayed on screen or provided on paper and that were directed at healthcare professionals or targeted at both professionals and patients. The reviewers screened the potentially relevant studies in duplicate. They extracted data and assessed risk of bias in independent pairs or individually followed by a double check by another reviewer. We summarised results using medians and interquartile ranges and rated our certainty in the evidence using the GRADE system. RESULTS: We identified 66 head-to-head trials that we synthesised across 14 comparisons of CDS intervention factors. Providing CDS automatically versus on demand led to large improvements in adherence. Displaying CDS on-screen versus on paper led to moderate improvements and making CDS more versus less patient-specific improved adherence modestly. When CDS interventions were combined with professional-oriented strategies, combined with patient-oriented strategies, or combined with staff-oriented strategies, then adherence improved slightly. Providing CDS to patients slightly increased adherence versus CDS aimed at the healthcare provider only. Making CDS advice more explicit and requiring users to respond to the advice made little or no difference. The CDS intervention factors made little or no difference to patient outcomes. The results for economic outcomes and satisfaction outcomes were sparse. CONCLUSION: Multiple factors may affect the success of CDS interventions. CDS may be more effective when the advice is provided automatically and displayed on-screen and when the suggestions are more patient-specific. CDS interventions combined with other strategies probably also improves adherence. Providing CDS directly to patients may also positively affect adherence. The certainty of the evidence was low to moderate for all factors. TRIAL REGISTRATION: PROSPERO, CRD42016033738.

Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial.

OBJECTIVES: To investigate practicing physicians' preferences, perceived usefulness and understanding of a new multilayered guideline presentation format-compared to a standard format-as well as conceptual understanding of trustworthy guideline concepts. DESIGN: Participants attended a standardised lecture in which they were presented with a clinical scenario and randomised to view a guideline recommendation in a multilayered format or standard format after which they answered multiple-choice questions using clickers. Both groups were also presented and asked about guideline concepts. SETTING: Mandatory educational lectures in 7 non-academic and academic hospitals, and 2 settings involving primary care in Lebanon, Norway, Spain and the UK. PARTICIPANTS: 181 practicing physicians in internal medicine (156) and general practice (25). INTERVENTIONS: A new digitally structured, multilayered guideline presentation format and a standard narrative presentation format currently in widespread use. PRIMARY AND SECONDARY OUTCOME MEASURES: Our primary outcome was preference for presentation format. Understanding, perceived usefulness and perception of absolute effects were secondary outcomes. RESULTS: 72% (95% CI 65 to 79) of participants preferred the multilayered format and 16% (95% CI 10 to 22) preferred the standard format. A majority agreed that recommendations (multilayered 86% vs standard 91%, p value=0.31) and evidence summaries (79% vs 77%, p value=0.76) were useful in the context of the clinical scenario. 72% of participants randomised to the multilayered format vs 58% for standard formats reported correct understanding of the recommendations (p value=0.06). Most participants elected an appropriate clinical action after viewing the recommendations (98% vs 92%, p value=0.10). 82% of the participants considered absolute effect estimates in evidence summaries helpful or crucial. CONCLUSIONS: Clinicians clearly preferred a novel multilayered presentation format to the standard format. Whether the preferred format improves decision-making and has an impact on patient important outcomes merits further investigation.

Development of a novel, multilayered presentation format for clinical practice guidelines.

BACKGROUND: Bridging the gap between clinical research and everyday health-care practice requires effective communication strategies. To address current shortcomings in conveying practice recommendations and supporting evidence, we are creating and testing presentation formats for clinical practice guidelines (CPGs). METHODS: We carried out multiple cycles of brainstorming and sketching, developing a prototype. Physicians participating in the user testing viewed CPG formats linked to clinical scenarios and engaged in semistructured interviews applying a think-aloud method for exploring important aspects of user experience. RESULTS: We developed a multilayered presentation format that allows clinicians to successively view more in-depth information. Starting with the recommendations, clinicians can, on demand, access a rationale and a key information section containing statements on quality of the evidence, balance between desirable and undesirable consequences, values and preferences, and resource considerations. We collected feedback from 27 stakeholders and performed user testing with 47 practicing physicians from six countries. Advisory group feedback and user testing of the first version revealed problems with conceptual understanding of underlying CPG methodology, as well as difficulties with the complexity of the layout and content. Extensive revisions made before the second round of user testing resulted in most participants expressing overall satisfaction with the final presentation format. CONCLUSIONS: We have developed an electronic, multilayered, CPG format that enhances the usability of CPGs for frontline clinicians. We have implemented the format in electronic guideline tools that guideline organizations can now use when authoring and publishing their guidelines.

Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis.

We systematically reviewed randomized controlled trials (RCTs) assessing the effectiveness of computerized decision support systems (CDSSs) featuring rule- or algorithm-based software integrated with electronic health records (EHRs) and evidence-based knowledge. We searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Abstracts of Reviews of Effects. Information on system design, capabilities, acquisition, implementation context, and effects on mortality, morbidity, and economic outcomes were extracted. Twenty-eight RCTs were included. CDSS use did not affect mortality (16 trials, 37395 patients; 2282 deaths; risk ratio [RR] = 0.96; 95% confidence interval [CI] = 0.85, 1.08; I(2) = 41%). A statistically significant effect was evident in the prevention of morbidity, any disease (9 RCTs; 13868 patients; RR = 0.82; 95% CI = 0.68, 0.99; I(2) = 64%), but selective outcome reporting or publication bias cannot be excluded. We observed differences for costs and health service utilization, although these were often small in magnitude. Across clinical settings, new generation CDSSs integrated with EHRs do not affect mortality and might moderately improve morbidity outcomes.

Adaptation of trustworthy guidelines developed using the GRADE methodology: a novel five-step process.

BACKGROUND: Adaptation of guidelines for use at the national or local level can facilitate their implementation. We developed and evaluated an adaptation process in adherence with standards for trustworthy guidelines and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, aiming for efficiency and transparency. This article is the first in a series describing our adaptation of Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines for a Norwegian setting. METHODS: Informed by the ADAPTE framework, we developed a five-step adaptation process customized to guidelines developed using GRADE: (1) planning, (2) initial assessment of the recommendations, (3) modification, (4) publication, and (5) evaluation. We developed a taxonomy for describing how and why recommendations from the parent guideline were modified and applied a mixed-methods case study design for evaluation of the process. RESULTS: We published the adapted guideline in November 2013 in a novel multilayered format. The taxonomy for adaptation facilitated transparency of the modification process for both the guideline developers and the end users. We excluded 30 and modified 131 of the 333 original recommendations according to the taxonomy and developed eight new recommendations. Unforeseen obstacles related to acquiring a licensing agreement and procuring a publisher resulted in a 9-month delay. We propose modifications of the adaptation process to overcome these obstacles in the future. CONCLUSIONS: This case study demonstrates the feasibility of a novel guideline adaptation process. Replication is needed to further validate the usefulness of the process in increasing the organizational and methodologic efficiency of guideline adaptation.

Applying new strategies for the national adaptation, updating, and dissemination of trustworthy guidelines: results from the Norwegian adaptation of the Antithrombotic Therapy and the Prevention of Thrombosis, 9th Ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines.

BACKGROUND: The Antithrombotic Therapy and the Prevention of Thrombosis, 9th Edition: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (AT9) represent trustworthy international guidelines for antithrombotic treatment and thromboprophylaxis. We describe major changes to the format and content resulting from applying new strategies for guideline adaptation and dissemination. METHODS: A Norwegian guideline panel of 46 experts completed a structured and systematic adaptation process, updated the recommendations based on new evidence, and rewrote the recommendations in an electronic multilayered presentation format. We published the adapted guideline using the web-based Making GRADE the Irresistible Choice Guideline Authoring and Publication Platform. RESULTS: We applied a novel presentation format to 333 recommendations from 11 of the 15 management chapters in AT9 and condensed and restructured them into 249 recommendations in a multilayered format. We added additional relevant information, such as 29 best-practice statements about new oral anticoagulants and practical information sections for 121 recommendations. Common reasons for modifications included feasibility of the recommendations in a national context, disagreement with applied baseline risk estimates, and reevaluation of the balance between the benefits and harms of interventions in relation to assumed typical patient preferences and values. The adapted guideline was published and disseminated online in November 2013. CONCLUSIONS: New strategies for adapting, updating, and disseminating trustworthy guidelines proved feasible and will provide Norwegian health-care professionals and patients with up-to-date guidance tailored to national circumstances.

[A new generation of reliable clinical practice guidelines through MAGIC]. / Una nueva generación de guías de práctica clínica confiables a través de MAGIC.

Safe and effective disease diagnosis and treatment requires that health personnel can access the best evidence, preferably through reliable clinical practice guidelines. Most guidelines have methodological weaknesses, suboptimal reporting formats, and frequently fail to update content. New standards developed by the US Institute of Medicine and the Guidelines International Network and Systems for Trustworthy Guidelines offer better opportunities for success in the development of guidelines, but also increase the demand for methodological competence, clinical experience and time. It is important to provide clinical practice guidelines with reliable content, and achieve their dissemination and update as needed. In this article we describe how to apply new standards, methods and tools for the creation, dissemination and updating of reliable clinical practice guidelines. Key steps were set for the elaboration of guidelines by using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. It will be explained how an innovative research program, Making GRADE the Irresistible Choice (MAGIC) (www.magicproject.org), through its authorship and publication platform (MAGICapp), offers new solutions to facilitate the production, dissemination and dynamic update of reliable clinical practice guidelines. An example will be described about a recently published Norwegian guideline on the new oral anticoagulants for atrial fibrillation, showing how a guideline published in MAGICapp can be used in medical practice.

Una nueva generación de guías de práctica clínica confiables a través de MAGIC / A new generation of reliable clinical practice guidelines through MAGIC

El diagnóstico y el tratamiento seguro y eficaz de enfermedades en la consulta requieren que el personal de salud pueda acceder a la mejor evidencia, preferentemente a través de guías de práctica clínica confiables. La mayoría de guías sufren debilidades metodológicas, formatos de presentación subóptimos y frecuente falta de actualización de su contenido. Nuevos estándares desarrollados por el Institute of Medicine de los Estados Unidos y el Guideline International Network and Systems for Trustworthy Guidelines, ofrecen mejores oportunidades para el éxito en el desarrollo de guías, pero también aumentan la demanda de competencia metodológica, experiencia clínica y tiempo. Tan importante como proporcionar un contenido fiable de guías de práctica clínica es lograr la difusión y actualización de estas. En este artículo describimos cómo aplicar nuevos estándares, métodos y herramientas para la creación, difusión y actualización de las guías de práctica clínica confiables. Trazamos pasos clave para el desarrollo de guías a través de la utilización del sistema Grading of Recommendations Assessment, Development and Evaluation (GRADE). Explicamos cómo un programa de investigación innovador, MAking GRADE the Irresistible Choice (MAGIC) (www.magicproject.org) a través de su plataforma autoría y publicación (MAGICapp) ofrece nuevas soluciones para facilitar la producción, la difusión y la actualización dinámica de guías de práctica clínica confiables. Describimos un ejemplo práctico de una guía noruega publicada recientemente sobre los nuevos anticoagulantes orales para fibrilación auricular, mostrando cómo una guía publicada en el MAGICapp se puede utilizar la consulta médica.

Safe and effective disease diagnosis and treatment requires that health personnel can access the best evidence, preferably through reliable clinical practice guidelines. Most guidelines have methodological weaknesses, suboptimal reporting formats, and frequently fail to update content. New standards developed by the US Institute of Medicine and the Guidelines International Network and Systems for Trustworthy Guidelines offer better opportunities for success in the development of guidelines, but also increase the demand for methodological competence, clinical experience and time. It is important to provide clinical practice guidelines with reliable content, and achieve their dissemination and update as needed. In this article we describe how to apply new standards, methods and tools for the creation, dissemination and updating of reliable clinical practice guidelines. Key steps were set for the elaboration of guidelines by using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. It will be explained how an innovative research program, Making GRADE the Irresistible Choice (MAGIC) (www.magicproject.org), through its authorship and publication platform (MAGICapp), offers new solutions to facilitate the production, dissemination and dynamic update of reliable clinical practice guidelines. An example will be described about a recently published Norwegian guideline on the new oral anticoagulants for atrial fibrillation, showing how a guideline published in MAGICapp can be used in medical practice.

Creating clinical practice guidelines we can trust, use, and share: a new era is imminent.

Standards and guidance for developing trustworthy clinical practice guidelines are now available, and a number of leading guidelines adhere to the key standards. Even current trustworthy guidelines, however, generally suffer from a cumbersome development process, suboptimal presentation formats, inefficient dissemination to clinicians at the point of care, high risk of becoming quickly outdated, and suboptimal facilitation of shared decision-making with patients. To address these limitations, we have--in our innovative research program and nonprofit organization, MAGIC (Making GRADE the Irresistible Choice)--constructed a conceptual framework and tools to facilitate the creation, dissemination, and dynamic updating of trustworthy guidelines. We have developed an online application that constitutes an authoring and publication platform that allows guideline content to be written and structured in a database, published directly on our web platform or exported in a computer-interpretable language (eg, XML) enabling dissemination through a wide range of outputs that include electronic medical record systems, web portals, and applications for smartphones/tablets. Modifications in guidelines, such as recommendation updates, will lead to automatic alterations in these outputs with minimal additional labor for guideline authors and publishers, greatly facilitating dynamic updating of guidelines. Semiautomated creation of a new generation of decision aids linked to guideline recommendations should facilitate face-to-face shared decision-making in the clinical encounter. We invite guideline organizations to partner with us (www.magicproject.org) to apply and further improve the tools for their purposes. This work will result in clinical practice guidelines that we cannot only trust, but also easily share and use.

Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE): protocol and preliminary results.

BACKGROUND: Healthcare decision makers face challenges when using guidelines, including understanding the quality of the evidence or the values and preferences upon which recommendations are made, which are often not clear. METHODS: GRADE is a systematic approach towards assessing the quality of evidence and the strength of recommendations in healthcare. GRADE also gives advice on how to go from evidence to decisions. It has been developed to address the weaknesses of other grading systems and is now widely used internationally. The Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE) consortium (http://www.decide-collaboration.eu/), which includes members of the GRADE Working Group and other partners, will explore methods to ensure effective communication of evidence-based recommendations targeted at key stakeholders: healthcare professionals, policymakers, and managers, as well as patients and the general public. Surveys and interviews with guideline producers and other stakeholders will explore how presentation of the evidence could be improved to better meet their information needs. We will collect further stakeholder input from advisory groups, via consultations and user testing; this will be done across a wide range of healthcare systems in Europe, North America, and other countries. Targeted communication strategies will be developed, evaluated in randomized trials, refined, and assessed during the development of real guidelines. DISCUSSION: Results of the DECIDE project will improve the communication of evidence-based healthcare recommendations. Building on the work of the GRADE Working Group, DECIDE will develop and evaluate methods that address communication needs of guideline users. The project will produce strategies for communicating recommendations that have been rigorously evaluated in diverse settings, and it will support the transfer of research into practice in healthcare systems globally.

Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial.

OBJECTIVE: To determine the effects of formatting alternatives in Grading of Recommendations Assessment, Development, and Evaluation (GRADE) evidence profiles on guideline panelists' preferences, comprehension, and accessibility. STUDY DESIGN AND SETTING: We randomized 116 antithrombotic therapy guideline panelists to review one of two table formats with four formatting alternatives. After answering relevant questions, panelists reviewed the other format and reported their preferences for specific formatting alternatives. RESULTS: Panelists (88 of 116 invited [76%]) preferred presentation of study event rates over no study event rates (median 1 [interquartile range (IQR) 1] on 1-7 scale), absolute risk differences over absolute risks (median 2 [IQR 3]), and additional information in table cells over footnotes (median 1 [IQR 2]). Panelists presented with time frame information in the tables, and not only in footnotes, were more likely to correctly answer questions regarding time frame (58% vs. 11%, P<0.0001), and those presented with risk differences and not absolute risks were more likely to correctly interpret confidence intervals for absolute effects (95% vs. 54%, P<0.0001). Information was considered easy to find, easy to comprehend, and helpful in making recommendations regardless of table format (median 6, IQR 0-1). CONCLUSION: Panelists found information in GRADE evidence profiles accessible. Correct comprehension of some key information was improved by providing additional information in table and presenting risk differences.