Increasing uptake through collaboration in the development of core outcome sets: Lessons learned at OMERACT 2023.

OBJECTIVE: This manuscript highlights the importance of enhancing the uptake of Core Outcome Sets (COS) by building partnerships with Collaborators and addressing their needs in COS development. METHODS AND SETTING: This session was structured as a simulation, resembling a format akin to a classic television game show. The moderator posed a series of questions to eight different Collaborator groups who briefly described the importance of COS within their areas of interest. Previous studies examining the uptake of individual core outcomes revealed disparities in uptake rates. The Identified barriers to the uptake of COS include the lack of recommendations for validated instruments for each domain, insufficient involvement of patients and key Collaborator groups in COS development, and a lack of awareness regarding the existence of COS. CONCLUSIONS: This analysis underscores the need for COS development approaches that prioritize the inclusion of patients and diverse Collaborator groups at every stage. While current studies on COS uptake are limited, future research should explore the broader implementation of COS across diverse disease categories and delve into the factors that hinder or facilitate their uptake such as, the importance of COS developers extending their work to recommending domains with well validated instruments. Embracing patient leadership and multifaceted engagement is essential for advancing the relevance and impact of COS in clinical research.

Operationalizing the GRADE-equity criterion to inform guideline recommendations: application to a medical cannabis guideline.

OBJECTIVES: Incorporating health equity considerations into guideline development often requires information beyond that gathered through traditional evidence synthesis methodology. This article outlines an operationalization plan for the Grading of Recommendations Assessment, Development, and Evaluation (GRADE)-equity criterion to gather and assess evidence from primary studies within systematic reviews, enhancing guideline recommendations to promote equity. We demonstrate its use in a clinical guideline on medical cannabis for chronic pain. STUDY DESIGN AND SETTING: We reviewed GRADE guidance and resources recommended by team members regarding the use of evidence for equity considerations, drafted an operationalization plan, and iteratively refined it through team discussion and feedback and piloted it on a medicinal cannabis guideline. RESULTS: We propose a seven-step approach: 1) identify disadvantaged populations, 2) examine available data for specific populations, 3) evaluate population baseline risk for primary outcomes, 4) assess representation of these populations in primary studies, 5) appraise analyses, 6) note barriers to implementation of effective interventions for these populations, and 7) suggest supportive strategies to facilitate implementation of effective interventions. CONCLUSION: Our approach assists guideline developers in recognizing equity considerations, particularly in resource-constrained settings. Its application across various guideline topics can verify its feasibility and necessary adjustments.

Crowd-sourcing and automation facilitated the identification and classification of randomized controlled trials in a living review.

OBJECTIVES: To evaluate an approach using automation and crowdsourcing to identify and classify randomized controlled trials (RCTs) for rheumatoid arthritis (RA) in a living systematic review (LSR). METHODS: Records from a database search for RCTs in RA were screened first by machine learning and Cochrane Crowd to exclude non-RCTs, then by trainee reviewers using a Population, Intervention, Comparison, and Outcome (PICO) annotator platform to assess eligibility and classify the trial to the appropriate review. Disagreements were resolved by experts using a custom online tool. We evaluated the efficiency gains, sensitivity, accuracy, and interrater agreement (kappa scores) between reviewers. RESULTS: From 42,452 records, machine learning and Cochrane Crowd excluded 28,777 (68%), trainee reviewers excluded 4,529 (11%), and experts excluded 7,200 (17%). The 1,946 records eligible for our LSR represented 220 RCTs and included 148/149 (99.3%) of known eligible trials from prior reviews. Although excluded from our LSRs, 6,420 records were classified as other RCTs in RA to inform future reviews. False negative rates among trainees were highest for the RCT domain (12%), although only 1.1% of these were for the primary record. Kappa scores for two reviewers ranged from moderate to substantial agreement (0.40-0.69). CONCLUSION: A screening approach combining machine learning, crowdsourcing, and trainee participation substantially reduced the screening burden for expert reviewers and was highly sensitive.

Incidence Morbidity and Mortality From Falls in Skilled Nursing Facilities: A Systematic Review and Meta-Analysis.

OBJECTIVES: The primary objective of this study was to systematically review and meta-analyze the incidence and consequential morbidity and mortality from falls in skilled nursing facilities. Our secondary objective is to synthesize current evidence on risk factors for injurious falls. DESIGN: Systematic review and meta-analysis. SETTING AND PARTICIPANTS: Older adults residing in skilled nursing facilities or similar settings. METHODS: We completed study screening, data extraction, and quality assessment in duplicate. Random effects models were used for meta-analysis of fall incidence rates and proportions of outcomes per fall. Sensitivity analysis and meta-regression were completed to assess differences based on study design, quality, and population characteristics. The Newcastle Ottawa Scale and Cochrane Risk of Bias tools were used to assess quality of observational and intervention-based studies, respectively. The GRADE tool was used to evaluate strength of evidence for fall risk factors. RESULTS: We identified 3103 unique references, of which 38 were included in systematic review and 37 in meta-analysis. Pooled incidence of falls was 121 per 100 person-years (95% CI 86-170). Outcomes of transfer to hospital, admission to hospital, overall injury, head injury, fracture, 30-day mortality, death in hospital, and disability were reported by included studies. Sensitivity analysis indicated no significant difference in fall rates between study designs. Meta-regression indicated no significant relationship between fall rate and age or sex; however, a weak positive correlation was identified with increasing prevalence of dementia. No fall risk factors were supported by high-quality evidence. CONCLUSION/IMPLICATIONS: Our study confirms that falls in skilled nursing facilities are common and cause significant morbidity, mortality and health system use. As populations in high-income countries age, falls will become increasingly prevalent. Future research should be directed at preventing injurious falls and determining when hospital care will benefit a faller.

Canadian Rheumatology Association Recommendations for the Screening, Monitoring, and Treatment of Juvenile Idiopathic Arthritis-Associated Uveitis.

OBJECTIVE: To develop Canadian recommendations for the screening, monitoring, and treatment of uveitis associated with juvenile idiopathic arthritis (JIA). METHODS: Recommendations were developed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE)-ADOLOPMENT approach. A working group of 14 pediatric rheumatologists, 6 ophthalmologists, 2 methodologists, and 3 caregiver/patient representatives reviewed recent American College of Rheumatology (ACR)/Arthritis Foundation (AF) recommendations and worked in pairs to develop evidence-to-decision (EtD) tables. A survey to assess agreement and recommendations requiring group discussion was completed. EtD tables were presented, discussed, and voted upon at a virtual meeting, to produce the final recommendations. A health equity framework was applied to all aspects of the adolopment process including the EtD tables, survey responses, and virtual meeting discussion. RESULTS: The survey identified that 7 of the 19 recommendations required rigorous discussion. Seventy-five percent of working group members attended the virtual meeting to discuss controversial topics as they pertained to the Canadian environment, including timing to first eye exam, frequency of screening, escalation criteria for systemic and biologic therapy, and the role of nonbiologic therapies. Equity issues related to access to care and advanced therapeutics across Canadian provinces and territories were highlighted. Following the virtual meeting, 5 recommendations were adapted, 2 recommendations were removed, and 1 was developed de novo. CONCLUSION: Recommendations for JIA-associated uveitis were adapted to the Canadian context by a working group of pediatric rheumatologists, ophthalmologists with expertise in the management of uveitis, and parent/patient input, taking into consideration cost, equity, and access.

A Framework for the Development of Living Practice Guidelines in Health Care.

BACKGROUND: Living practice guidelines are increasingly being used to ensure that recommendations are responsive to rapidly emerging evidence. OBJECTIVE: To develop a framework that characterizes the processes of development of living practice guidelines in health care. DESIGN: First, 3 background reviews were conducted: a scoping review of methods papers, a review of handbooks of guideline-producing organizations, and an analytic review of selected living practice guidelines. Second, the core team drafted the first version of the framework. Finally, the core team refined the framework through an online survey and online discussions with a multidisciplinary international group of stakeholders. SETTING: International. PARTICIPANTS: Multidisciplinary group of 51 persons who have experience with guidelines. MEASUREMENTS: Not applicable. RESULTS: A major principle of the framework is that the unit of update in a living guideline is the individual recommendation. In addition to providing definitions, the framework addresses several processes. The planning process should address the organization's adoption of the living methodology as well as each specific guideline project. The production process consists of initiation, maintenance, and retirement phases. The reporting should cover the evidence surveillance time stamp, the outcome of reassessment of the body of evidence (when applicable), and the outcome of revisiting a recommendation (when applicable). The dissemination process may necessitate the use of different venues, including one for formal publication. LIMITATION: This study does not provide detailed or practical guidance for how the described concepts would be best implemented. CONCLUSION: The framework will help guideline developers in planning, producing, reporting, and disseminating living guideline projects. It will also help research methodologists study the processes of living guidelines. PRIMARY FUNDING SOURCE: None.

Canadian Rheumatology Association Living Guidelines for the Pharmacological Management of Rheumatoid Arthritis With Disease-Modifying Antirheumatic Drugs.

OBJECTIVE: To provide the initial installment of a living guideline that will provide up-to-date guidance on the pharmacological management of patients with rheumatoid arthritis (RA) in Canada. METHODS: The Canadian Rheumatology Association (CRA) formed a multidisciplinary panel composed of rheumatologists, researchers, methodologists, and patients. In this first installment of our living guideline, the panel developed a recommendation for the tapering of biologic and targeted synthetic disease-modifying antirheumatic drug (b/ts DMARD) therapy in patients in sustained remission using the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach, including a health equity framework developed for the Canadian RA population. The recommendation was adapted from a living guideline of the Australia & New Zealand Musculoskeletal Clinical Trials Network. RESULTS: In people with RA who are in sustained low disease activity or remission for at least 6 months, we suggest offering stepwise reduction in the dose of b/tsDMARD without discontinuation, in the context of a shared decision, provided patients are able to rapidly access rheumatology care and reestablish their medications if needed. In patients where rapid access to care or reestablishing access to medications is challenging, we conditionally recommend against tapering. A patient decision aid was developed to complement the recommendation. CONCLUSION: This living guideline will provide contemporary RA management recommendations for Canadian practice. New recommendations will be added over time and updated, with the latest recommendation, evidence summaries, and Evidence to Decision summaries available through the CRA website (www.rheum.ca).

Crowdsourcing trainees in a living systematic review provided valuable experiential learning opportunities: a mixed-methods study.

OBJECTIVES: To understand trainee experiences of participating in a living systematic review (LSR) for rheumatoid arthritis and the potential benefits in terms of experiential evidence-based medicine (EBM) education. STUDY DESIGN AND SETTING: We conducted a mixed-methods study with trainees who participated in the LSR and who were recruited broadly from training programs in two countries. Trainees received task-specific training and completed one or more tasks in the review: assessing article eligibility, data extraction, and quality assessment. Trainees completed a survey followed by a one-on-one interview. Data were triangulated to produce broad themes. RESULTS: Twenty one trainees, most of whom had a little prior experience with systematic reviews, reported a positive overall experience. Key benefits included learning opportunities, task segmentation (ability to focus on a single task, as opposed to an entire review), working in a supportive environment, international collaboration, and incentives such as authorship or acknowledgment. Trainees reported improvement in their competency as a Scholar, Collaborator, Leader, and Medical Expert. Challenges included communication and technical difficulties and appropriate matching of tasks to trainee skillsets. CONCLUSION: Participating in an LSR provided benefits to a wide range of trainees and may provide an opportunity for experiential EBM training, while helping LSR sustainability.

Substantial heterogeneity found in reporting mortality in Cochrane systematic reviews and Core Outcome Sets in COMET database.

OBJECTIVE: To explore mortality outcome usage in Cochrane systematic reviews and Core Outcome Sets for research. STUDY DESIGN AND SETTING: Cochrane PICO searches identified Cochrane reviews (published January 2015-March 2021) including mortality outcomes. These outcomes were categorized according to terminology used: all-cause mortality, cause-specific mortality, infant mortality, maternal mortality, survival. Mortality outcomes in Core Outcome Sets (published until 2019 on the Core Outcome Measures in Effectiveness Trials (COMET) database) were also extracted and categorized. RESULTS: In total, 2454 mortality outcomes were reported in 49% (1978/3999) of Cochrane reviews published January 2015-March 2021: all-cause (37%), infant (23%), maternal (11%), survival (10%), cause-specific (9%). Due to reviews not specifying mortality outcome type or including studies reporting no data, 11% (273/2208) remained uncategorized. Infant mortality and maternal mortality were frequently used together in reviews reporting two mortality outcomes. In total, 226 mortality outcomes were reported in 37% (165/449) of Core Outcome Sets: all-cause (48%), survival (27%), cause-specific (12%), infant (9%), maternal (4%). Mortality measurement timing varied. CONCLUSION: Mortality outcome usage varies in Cochrane reviews and Core Outcome Sets. This is problematic for evidence-based decision-making. Greater standardization is necessary for effective utilization of health research.

Canadian Rheumatology Association Recommendation for the Use of COVID-19 Vaccination for Patients With Autoimmune Rheumatic Diseases.

OBJECTIVE: To develop guidance on the use of coronavirus disease 2019 (COVID-19) vaccines in patients with autoimmune rheumatic diseases (ARD). METHODS: The Canadian Rheumatology Association (CRA) formed a multidisciplinary panel including rheumatologists, researchers, methodologists, vaccine experts, and patients. The panel used the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach. Outcomes were prioritized according to their importance for patients and clinicians. Evidence from the COVID-19 clinical trials was summarized. Indirect evidence for non-COVID-19 vaccines in ARD was also considered. The GRADE evidence-to-decision (EtD) framework was used to develop a recommendation for the use of the 4 COVID-19 vaccines approved in Canada as of March 25, 2021 (BNT162b2, mRNA-1273, ChAdOx1, and Ad26.COV2.S), over 4 virtual panel meetings. RESULTS: The CRA guideline panel suggests using COVID-19 vaccination in persons with ARD. The panel unanimously agreed that for the majority of patients, the potential health benefits of vaccination outweigh the potential harms in people with ARDs. The recommendation was graded as conditional because of low or very low certainty of the evidence on the effects in the population of interest, primarily due to indirectness and imprecise effect estimates. The panel felt strongly that persons with autoimmune rheumatic diseases who meet local eligibility should not be required to take additional steps compared to people without ARDs to obtain their vaccination. Guidance on medications, implementation, monitoring of vaccine uptake, and research priorities are also provided. CONCLUSION: This recommendation will be updated over time as new evidence emerges, with the latest recommendation, evidence summaries, and EtD available on the CRA website.

The 22nd Anniversary of the Cochrane Back and Neck Group.

STUDY DESIGN: Retrospective review and literature review. OBJECTIVE: The aim of this study was to provide an update on The Cochrane Back and Neck (CBN) activities. SUMMARY OF BACKGROUND DATA: Low back pain (LBP) affects 80% of people at some time in their lives. CBN Group has been housed in Toronto at the Institute for Work & Health since 1996 and has published 85 reviews and 32 protocols in the Cochrane Library. METHODS: Narrative review of CBN publications, impact factor, usage data, and social media impact. RESULTS: In the past 3 years, CBN conducted priority setting with organizations that develop clinical practice guidelines for LBP. CBN editors and associate editors published key methodological articles in the field of back and neck pain research. The methodological quality of CBN reviews has been assessed by external groups in a variety of areas, which found that CBN reviews had higher methodological quality than non-Cochrane reviews. CBN reviews have been included in 35 clinical practice guidelines for back and neck conditions. The 2018 journal impact factor of CBN is 11.154, which is higher than the 2018 impact factor for CDSR (7.755). CBN reviews ranked 4th among 53 Cochrane review groups in terms of Cochrane Library usage data. The most accessed CBN review was "Yoga treatment for chronic non-specific low-back pain" which had 9689 full-text downloads. CBN is active on Twitter with 3958 followers. CONCLUSION: CBN has published highly utilized systematic reviews and made important methodological contributions to the field of spine research over the past 22 years within Cochrane. LEVEL OF EVIDENCE: 4.

Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation.

BACKGROUND: Stakeholder engagement has become widely accepted as a necessary component of guideline development and implementation. While frameworks for developing guidelines express the need for those potentially affected by guideline recommendations to be involved in their development, there is a lack of consensus on how this should be done in practice. Further, there is a lack of guidance on how to equitably and meaningfully engage multiple stakeholders. We aim to develop guidance for the meaningful and equitable engagement of multiple stakeholders in guideline development and implementation. METHODS: This will be a multi-stage project. The first stage is to conduct a series of four systematic reviews. These will (1) describe existing guidance and methods for stakeholder engagement in guideline development and implementation, (2) characterize barriers and facilitators to stakeholder engagement in guideline development and implementation, (3) explore the impact of stakeholder engagement on guideline development and implementation, and (4) identify issues related to conflicts of interest when engaging multiple stakeholders in guideline development and implementation. DISCUSSION: We will collaborate with our multiple and diverse stakeholders to develop guidance for multi-stakeholder engagement in guideline development and implementation. We will use the results of the systematic reviews to develop a candidate list of draft guidance recommendations and will seek broad feedback on the draft guidance via an online survey of guideline developers and external stakeholders. An invited group of representatives from all stakeholder groups will discuss the results of the survey at a consensus meeting which will inform the development of the final guidance papers. Our overall goal is to improve the development of guidelines through meaningful and equitable multi-stakeholder engagement, and subsequently to improve health outcomes and reduce inequities in health.

PROTOCOL: When and how to replicate systematic reviews.

This is a protocol for a co-registered Cochrane and Campbell Review (Methodology). The objectives are as follows: To identify, describe and assess methods for: when to replicate a systematic review; how to replicate a systematic review.

Evidence synthesis of types and intensity of therapeutic land-based exercises to reduce pain in individuals with knee osteoarthritis.

The objective of this study is to construct an evidence synthesis to identify the types of land-based exercises most investigated in the current literature, the intervention duration, frequency of the programs and the exercises which are most frequently implemented. A search was performed on the reference list of included and excluded studies of one systematic review, on land-based exercises for knee osteoarthritis and, an updated search of The Cochrane Library, Embase, CINAHL and PEDro was completed. Two authors independently selected the studies and a third author was consulted for an additional opinion. The inclusion criteria were male or female with tibiofemoral knee osteoarthritis, land-based exercises, non-exercise control group and randomized clinical trials. The exclusion criteria were mixed diagnosis or comparison to other types of exercise. The data were extracted by two authors. Fifty-five full-text articles were included. Strengthening, proprioception and aerobic exercises resulted in significant pain reduction. The intervention durations which were significant for pain reduction were either the period of 8-11 weeks or 12-15 weeks. The frequency of three times per week was found significant in comparison to a non-exercise control group. The results, which formed an evidence synthesis, demonstrate that there is substantial evidence regarding the benefits of strengthening exercises to reduce pain in knee osteoarthritis patients. Based on the included studies analysis, exercises should be performed three times weekly for a duration of 8-11 or 12-15 weeks. Health professionals working with knee osteoarthritis patients can use this evidence synthesis as a fast and pragmatic instrument to obtain information about several effective types of exercises for pain reduction.

Some Cochrane risk-of-bias items are not important in osteoarthritis trials: a meta-epidemiological study based on Cochrane reviews.

OBJECTIVE: To evaluate the impact of bias-related study characteristics on treatment effects in osteoarthritis (OA) trials. STUDY DESIGN AND SETTING: Based on OA trials included in Cochrane reviews, the impact of study characteristics on treatment effect estimates was evaluated. Characteristics included items of the risk of bias (RoB) tool, trial size, single vs. multisite, and source of funding. Effect sizes (ESs) were calculated as standardized mean differences (SMDs). Meta-regression was performed to identify "relevant study-level covariates" that decrease the between-study variance (τˆ2). RESULTS: Twenty reviews, including 126 OA trials with a high degree of heterogeneity, were included (τˆ2 = 0.1247). Among the RoB domains, only patient blinding had an impact on the results (reducing heterogeneity according to τˆ2 < 7%). Inadequate blinding of patients yielded larger effects (SMDDifference = 0.15; 95% confidence interval [CI]: 0.01-0.29, P = 0.035). The most important study characteristic was trial size (heterogeneity reduced by 25%), with small trials reporting larger effects (SMDDifference = 0.29; 95% CI: 0.16-0.42, P < 0.001). CONCLUSION: In musculoskeletal reviews addressing pain, all the items included in the Cochrane RoB tool might not be equally important. OA trial results may be affected by bias constructs that are not yet fully elucidated.

Translating Evidence to Facilitate Shared Decision Making: Development and Usability of a Consult Decision Aid Prototype.

AIM: The purpose of this study was to translate evidence from Cochrane Reviews into a format that can be used to facilitate shared decision making during the consultation, namely patient decision aids. METHODS: A systematic development process (a) established a stakeholder committee; (b) developed a prototype according to the International Patient Decision Aid Standards; (c) applied the prototype to a Cochrane Review and used an interview-guided survey to evaluate acceptability/usability; (d) created 12 consult decision aids; and (e) used a Delphi process to reach consensus on considerations for creating a consult decision aid. RESULTS: The 1-page prototype includes (a) a title specifying the decision; (b) information on the health condition, options, benefits/harms with probabilities; (c) an explicit values clarification exercise; and (d) questions to screen for decisional conflict. Hyperlinks provide additional information on definitions, probabilities presented graphically, and references. Fourteen Cochrane Consumer Network members and Cochrane Editorial Unit staff participated. Thirteen reported that it would help patient/clinician discussions and were willing to use and/or recommend it. Seven indicated the right amount of information, six not enough, and one too much. Changes to the prototype were more links to definitions, more white space, and details on GRADE evidence ratings. Creating 12 consult decision aids took about 4 h each. We identified ten considerations when selecting Cochrane Reviews for creating consult decision aids. CONCLUSIONS: Using a systematic process, we developed a consult decision aid prototype to be populated with evidence from Cochrane Reviews. It was acceptable and easy to apply. Future studies will evaluate implementation of consult decision aids.

A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial.

OBJECTIVES: To evaluate a new format of a summary, which presents research from synthesized evidence to patients and the public. STUDY DESIGN AND SETTING: We conducted a randomized controlled trial in 143 members of the public from five countries (Canada, Norway, Spain, Argentina, and Italy). Participants received either a new summary format (a plain language summary [PLS]) or the current format used in Cochrane systematic reviews. The new PLS presents information about the condition and intervention, a narrative summary of results, and a table of results with absolute numbers for effects of the intervention and quality of the evidence using Grading of Recommendations Assessment, Development, and Evaluation. RESULTS: With the new PLS, more participants understood the benefits and harms and quality of evidence (53% vs. 18%, P < 0.001); more answered each of the five questions correctly (P ≤ 0.001 for four questions); and they answered more questions correctly, median 3 (interquartile range [IQR]: 1-4) vs. 1 (IQR: 0-1), P < 0.001). Better understanding was independent of education level. More participants found information in the new PLS reliable, easy to find, easy to understand, and presented in a way that helped make decisions. Overall, participants preferred the new PLS. CONCLUSION: This new PLS format for patients and the public is a promising tool to translate evidence from synthesized research.