Guideline Assessment Project II: statistical calibration informed the development of an AGREE II extension for surgical guidelines.

OBJECTIVE: To inform the development of an AGREE II extension specifically tailored for surgical guidelines. AGREE II was designed to inform the development, reporting, and appraisal of clinical practice guidelines. Previous research has suggested substantial room for improvement of the quality of surgical guidelines. METHODS: A previously published search in MEDLINE for clinical practice guidelines published by surgical scientific organizations with an international scope between 2008 and 2017, resulted in a total of 67 guidelines. The quality of these guidelines was assessed using AGREE II. We performed a series of statistical analyses (reliability, correlation and Factor Analysis, Item Response Theory) with the objective to calibrate AGREE II for use specifically in surgical guidelines. RESULTS: Reliability/correlation/factor analysis and Item Response Theory produced similar results and suggested that a structure of 5 domains, instead of 6 domains of the original instrument, might be more appropriate. Furthermore, exclusion and re-arrangement of items to other domains was found to increase the reliability of AGREE II when applied in surgical guidelines. CONCLUSIONS: The findings of this study suggest that statistical calibration of AGREE II might improve the development, reporting, and appraisal of surgical guidelines.

Guideline Assessment Project: Filling the GAP in Surgical Guidelines: Quality Improvement Initiative by an International Working Group.

OBJECTIVE: The aim of the study was to identify clinical practice guidelines published by surgical scientific organizations, assess their quality, and investigate the association between defined factors and quality. The ultimate objective was to develop a framework to improve the quality of surgical guidelines. SUMMARY BACKGROUND DATA: Evidence on the quality of surgical guidelines is lacking. METHODS: We searched MEDLINE for clinical practice guidelines published by surgical scientific organizations with an international scope between 2008 and 2017. We investigated the association between the following factors and guideline quality, as assessed using the AGREE II instrument: number of guidelines published within the study period by a scientific organization, the presence of a guidelines committee, applying the GRADE methodology, consensus project design, and the presence of intersociety collaboration. RESULTS: Ten surgical scientific organizations developed 67 guidelines over the study period. The median overall score using AGREE II tool was 4 out of a maximum of 7, whereas 27 (40%) guidelines were not considered suitable for use. Guidelines produced by a scientific organization with an output of ≥9 guidelines over the study period [odds ratio (OR) 3.79, 95% confidence interval (CI), 1.01-12.66, P = 0.048], the presence of a guidelines committee (OR 4.15, 95% CI, 1.47-11.77, P = 0.007), and applying the GRADE methodology (OR 8.17, 95% CI, 2.54-26.29, P < 0.0001) were associated with higher odds of being recommended for use. CONCLUSIONS: Development by a guidelines committee, routine guideline output, and adhering to the GRADE methodology were found to be associated with higher guideline quality in the field of surgery.

Uncertain times: A survey of Canadian women's perspectives toward mammography screening.

Evolving scientific evidence about mammography has raised new questions about the net benefits of organized screening, yet gaps remain about women's current screening practices, knowledge, attitudes and values toward screening to support informed decision making in this area. We addressed this gap through an online survey of 2000 screen-eligible women from Ontario, Canada in January 2016. Likert-scaled and categorical questions were used to collect information about screening practices, knowledge of benefits and risks of screening and underlying attitudes and values toward screening. Results for all responses were summarized using descriptive statistics. Comparison of results between ever screened versus never screened respondents was performed using chi-squared tests. Most women felt informed about screening yet had doubts about how informed their decisions were. They were more confident in their knowledge of the benefits than the risks which aligned with the emphasis given to benefits in discussions with health care providers. The benefits of screening were linked with lowered anxiety about breast cancer. The never screened were less likely to overstate the benefits of screening, more likely to give weight to the risks, and less likely to report anxiety or worry about breast cancer. Findings highlight the need for improved communication strategies and decision supports that emphasize the provision of current, balanced information about the benefits and risks of screening, both at the population-level (through mass media) and within patient-provider interactions. Sensitivity to the psychosocial factors that shape women's attitudes toward mammography screening should be central to any strategy.

Variable participation of knowledge users in cancer health services research: results of a multiple case study.

BACKGROUND: Integrated knowledge translation (IKT) is a research approach in which knowledge users (KUs) co-produce research. The rationale for IKT is that it leads to research that is more relevant and useful to KUs, thereby accelerating uptake of findings. The aim of the current study was to evaluate IKT activities within a cancer health services research network in Ontario, Canada. METHODS: An embedded multiple case study design was used. The cases were 5 individual studies within an overarching cancer health services research network. These studies focused on one of the following topics: case costing of cancer treatment, lung cancer surgery policy analysis, patient and provider-reported outcomes, colorectal cancer screening, and a team approach to women's survivorship. We conducted document reviews and held semi-structured interviews with researchers, KUs, and other stakeholders within a cancer system organization. The analysis examined patterns across and within cases. RESULTS: Researchers and their respective knowledge users from 4 of the 5 cases agreed to participate. Eighteen individuals from 4 cases were interviewed. In 3 of 4 cases, there were mismatched expectations between researchers and KUs regarding KU role; participants recommended that expectations be made explicit from the beginning of the collaboration. KUs perceived that frequent KU turnover may have affected both KU engagement and the uptake of study results within the organization. Researchers and KUs found that sharing research results was challenging because the organization lacked a framework for knowledge translation. Uptake of research findings appeared to be related to the researcher having an embedded role in the cancer system organization and/or close alignment of the study with organizational priorities. Document reviews found evidence of planned IKT strategies in 3 of 4 cases; however, actual KU role/engagement on research teams was variable. CONCLUSIONS: Barriers to KU co-production of cancer health services research include mismatched expectations of KU role and frequent KU turnover. When a research study directly aligns with organizational priorities, it appears more likely that results will be considered in programming. Research teams that take an IKT approach should consider specific strategies to address barriers to KU engagement.

Validity and usability testing of a health systems guidance appraisal tool, the AGREE-HS.

BACKGROUND: Health systems guidance (HSG) provides recommendations to address health systems challenges. No tools exist to inform HSG developers and users about the components of high quality HSG and to differentiate between HSG of varying quality. In response, we developed a tool to assist with the development, reporting and appraisal of HSG - the Appraisal of Guidelines for Research and Evaluation-Health Systems (AGREE-HS). This paper reports on the validity, usability and initial measurement properties of the AGREE-HS. METHODS: To establish face validity (Study 1), stakeholders completed a survey about the AGREE-HS and provided feedback on its content and structure. Revisions to the tool were made in response. To establish usability (Study 2), the revised tool was applied to 85 HSG documents and the appraisers provided feedback about their experiences via an online survey. An initial test of the revised tool's measurement properties, including internal consistency, inter-rater reliability and criterion validity, was conducted. Additional revisions to the tool were made in response. RESULTS: In Study 1, the AGREE-HS Overview, User Manual, quality item content and structure, and overall assessment questions were rated favourably. Participants indicated that the AGREE-HS would be useful, feasible to use, and that they would apply it in their context. In Study 2, participants indicated that the quality items were easy to understand and apply, and the User Manual, usefulness and usability of the tool were rated favourably. Study 2 participants also indicated intentions to use the AGREE-HS. CONCLUSIONS: The AGREE-HS comprises a User Manual, five quality items and two overall assessment questions. It is available at agreetrust.org.

Understanding optimal approaches to patient and caregiver engagement in the development of cancer practice guidelines: a mixed methods study.

BACKGROUND: Practice guidelines (PGs) can assist health care practitioners and patients to make decisions about health care options. A key component of high quality PGs is the consideration of patient values and preferences. A mixed methods study was conducted to understand optimal approaches to patient engagement in the development of cancer PGs. METHODS: Cancer patients, survivors, family members and caregivers were recruited from cancer clinics, follow-up clinics, community support programs, a provincial patient and family advisory committee, and a provincial cancer PG development program. Participants attended a workshop, completed a survey, or participated in a telephone interview, to provide information about PG awareness, attitudes, information needs, training, engagement approaches and barriers and facilitators. RESULTS: Forty-one participants (12 workshop attendees, 21 survey respondents and 8 interviewees) provided data. For those with no PG development experience, fewer than half were previously aware of PGs but perceived several benefits to the inclusion of this perspective. Common barriers to participation across the groups were time commitment, duration of the PG development process, and financial costs. Positive beliefs about the contributions that could be made and practical considerations (e.g., orientation and training, defined roles and expectations) were identified as key features in the successful integration of patients into the PG development process. There was no single model of engagement favored over another. CONCLUSIONS: Study results align with similar studies in other contexts and with international patient engagement efforts. Findings are being used to test new patient engagement models in a programmatic PG development initiative in Ontario, Canada.

Documenting coordination of cancer care between primary care providers and oncology specialists in Canada.

OBJECTIVE: To report on the findings of the CanIMPACT (Canadian Team to Improve Community-Based Cancer Care along the Continuum) Casebook project, which systematically documented Canadian initiatives (ie, programs and projects) designed to improve or support coordination and continuity of cancer care between primary care providers (PCPs) and oncology specialists. DESIGN: Pan-Canadian environmental scan. SETTING: Canada. PARTICIPANTS: Individuals representing the various initiatives provided data for the analysis. METHODS: Initiatives included in the Casebook met the following criteria: they supported coordination and collaboration between PCPs and oncology specialists; they were related to diagnosis, treatment, survivorship, or personalized medicine; and they included breast or colorectal cancer or both. Data were collected on forms that were compiled into summaries (ie, profiles) for each initiative. Casebook initiatives were organized based on the targeted stage of the cancer care continuum, jurisdiction, and strategy (ie, model of care or type of intervention) employed. Thematic analysis identified similarities and differences among employed strategies, the level of primary care engagement, implementation barriers and facilitators, and initiative evaluation. MAIN FINDINGS: The CanIMPACT Casebook profiles 24 initiatives. Eleven initiatives targeted the survivorship stage of the cancer care continuum and 15 focused specifically on breast or colorectal cancer or both. Initiative teams implemented the following strategies: nurse patient navigation, multidisciplinary care teams, electronic communication or information systems, PCP education, and multicomponent initiatives. Initiatives engaged PCPs at various levels. Implementation barriers included lack of care standardization across jurisdictions and incompatibility among electronic communication systems. Implementation facilitators included having clinical and program leaders publicly support the initiative, repurposing existing resources, receiving financial support, and establishing a motivated and skilled project or program team. The lack of evaluative data made it difficult to identify the most effective interventions or models of care. CONCLUSION: The CanIMPACT Casebook documents Canadian efforts to improve or support the coordination of cancer care by PCPs and oncology specialists as a means to improve patient outcomes and cancer system performance.

Increasing participation in colorectal cancer screening: results from a cluster randomized trial of directly mailed gFOBT kits to previous nonresponders.

Regular screening using guaiac fecal occult blood test (gFOBT) reduces mortality from colorectal cancer (CRC). The objective of this study was to determine whether the addition of a gFOBT kit to a second mailed invitation compared to a second mailed invitation alone increases CRC screening among eligible persons who did not respond to an initial mailed invitation. We conducted a cluster randomized controlled trial, with the physician as the unit of randomization. Participants were persons who had been invited but who had not responded to an invitation for CRC screening in an earlier pilot project. The intervention group received a mailed gFOBT kit and second mailed CRC screening invitation (n = 2,008) while the control group received a second mailed CRC screening invitation alone (n = 1,586). The primary outcome was the uptake of gFOBT within 6 months of the second mailing. We found that the uptake of gFOBT was more than twice as high in the intervention group (20.1%) compared to the control group (9.6%). The absolute difference between the two groups was 10.5% (95% CI: 7.5-13.4%, p ≤ 0.0001). In a subsequent adjusted analysis, participants in the intervention group were twice as likely to complete the test as those in the control group (OR = 2.1; 95% CI: 1.6-2.6). These findings suggest that directly mailed gFOBT kits increase CRC screening participation among previous nonresponders to a mailed invitation and that approximately 10 gFOBT kits would have to be sent by mail in order to screen 1 additional person. (ClinicalTrials.gov: NCT01629004).

The quality of guidelines in pediatric surgery: can we all AGREE?

OBJECTIVE: Guidelines are meant to facilitate evidence-based clinical decision-making but vary in methodological rigor and quality of reporting. We assessed the quality of guidelines published in major pediatric surgery journals. METHODS: A MEDLINE search of 4 key pediatric surgery journals was performed. Included studies had guidelines, clinical practice guidelines, and consensus statements as a subject heading or keyword. Evaluations of guidelines were excluded. Eligible guidelines were assessed by three reviewers using the Appraisal of Guidelines for Research and Evaluation (AGREE II) Instrument. RESULTS: Our search identified ten guidelines for review. Agreement for study selection was excellent [K = 0.81 (95 % CI 0.63-0.99)]. The mean AGREE II score for individual guidelines was 18 % (SD 5.7 %). The best-scored quality domains were "scope and purpose" [mean score 49 % (SD 8.7 %)] and "clarity of presentation" [mean score 40 % (SD 18.7 %)]. The poorest score was for "editorial independence" [mean score 2 % (SD 3.7 %)]. CONCLUSIONS: The overall quality of guidelines in pediatric surgery, using AGREE II, is poor and may lead to inappropriate clinical decisions. Increased awareness of proper reporting and the methodological requirements for guideline development are needed to optimize the potential of guideline recommendations to improve practice. LEVEL OF EVIDENCE: n/a (Quality Appraisal).

How does context influence collaborative decision-making for health services planning, delivery and evaluation?

BACKGROUND: Collaboration among researchers (clinician, non-clinician) and decision makers (managers, policy-makers, clinicians), referred to as integrated knowledge translation (IKT), enhances the relevance and use of research, leading to improved decision-making, policies, practice, and health care outcomes. However IKT is not widely practiced due to numerous challenges. This research explored how context influenced IKT as a means of identifying how IKT could be strengthened. METHODS: This research investigated IKT in three health services programs for colon cancer screening, prostate cancer diagnosis, and the treatment of pancreatic cancer. Qualitative methods were used to explore contextual factors that influenced how IKT occurred, and its impact. Data were collected between September 1, 2012 and May 15, 2013 from relevant documents, observation of meetings, and interviews with researchers and decision-makers, analyzed using qualitative methods, and integrated. RESULTS: Data were analyzed from 39 documents, observation of 6 meetings, and 36 interviews. IKT included interaction at meetings, joint undertaking of research, and development of guidelines. IKT was most prevalent in one program with leadership, clear goals, dedicated funding and other infrastructural resources, and an embedded researcher responsible for, and actively engaged in IKT. This program achieved a variety of social, research and health service outcomes despite mixed individual views about the value of IKT and the absence of a programmatic culture of IKT. Participants noted numerous challenges including lack of time and incentives, and recommendations to support IKT. A conceptual framework of factors that influence IKT and associated outcomes was generated, and can be used by others to plan or evaluate IKT. CONCLUSIONS: The findings can be applied by researchers, clinicians, managers or policy-makers to plan or improve collaborative decision-making for health services planning, delivery, evaluation or quality improvement. Further research is needed to explore whether these findings are widespread, and further understand how IKT can be optimized.

Awareness and knowledge of the Canadian 24-Hour Movement Guidelines for Adults among adults living in Canada.

Awareness and knowledge of national movement behaviour guidelines are needed to influence individual behaviour and public health policies. This study assessed the awareness and knowledge of the Canadian 24-Hour Movement Guidelines for Adults Aged 18-64 Years and Adults Aged 65 Years or Older (24HMG) recommendations among adults living in Canada across three timepoints. Online surveys were distributed to representative samples of adults living in Canada over a 6-month period. Findings suggest that short-term dissemination efforts were successful in increasing awareness of the 24HMG following guideline release. However, other strategies, such as education, may be needed to influence knowledge of guideline recommendations.

Development of a ParticipACTION App-Based Intervention for Improving Postsecondary Students' 24-Hour Movement Guideline Behaviors: Protocol for the Application of Intervention Mapping.

BACKGROUND: The Canadian 24-Hour Movement Guidelines for adults provide specific recommendations for levels of physical activity, sedentary behavior, and sleep (ie, the movement behaviors) required for optimal health. Performance of the movement behaviors is associated with improved mental well-being. However, most postsecondary students do not meet the movement behavior recommendations within the Canadian 24-Hour Movement Guidelines and experience increased stress and declining well-being, suggesting the need for an intervention targeting students' movement behaviors. OBJECTIVE: We aimed to develop and implement a theory-informed intervention intended to improve the movement behaviors and mental well-being of first-year postsecondary students. METHODS: The Intervention Mapping protocol was applied in the development and implementation of the intervention. Intervention Mapping entailed performing a needs assessment, determining the intervention outcomes, selecting theory- and evidence-based change methods and applications, preparing and producing intervention plans and materials, developing the implementation plan, and finally developing an evaluation plan. The Theoretical Domains Framework and the Behavior Change Wheel were also used in conjunction with the Intervention Mapping protocol to ensure a solid theoretical basis for the intervention. This protocol led to the development and implementation of a 6-week, theory-informed ParticipACTION app-based intervention aimed at helping first-year postsecondary students improve their movement behaviors and mental well-being. The developed app content provided students with information on each of the movement behaviors and behavioral strategies (ie, goal setting, action planning, monitoring, and coping planning). The use of Intervention Mapping allowed for the continuous involvement of various multidisciplinary partners and end users, ensuring that the intervention design and implementation was appropriate for the target audience. The feasibility, acceptability, and potential impact of the intervention will be examined in a subsequent proof-of-concept study at 2 Canadian university campuses. RESULTS: Participant recruitment occurred during September 2021, and the intervention was conducted from October to December 2021. The deadline for completion of the postintervention questionnaire by participants was mid-December 2021. The analysis of data examining the feasibility, acceptability, and potential impact of the intervention began in January 2022, with the publication of the proof-of-concept evaluation expected in 2023. CONCLUSIONS: Intervention Mapping with the Theoretical Domains Framework and Behavior Change Wheel was a useful approach to combine evidence and theoretical concepts to guide the design and implementation of a ParticipACTION app-based intervention targeting postsecondary students' movement behaviors and mental well-being. This process may serve as an example for other researchers developing multiple behavior change app-based interventions. Should the forthcoming evaluation demonstrate the intervention's acceptability, feasibility, and potential impact, the intervention may provide a scalable method of improving postsecondary students' movement behaviors and mental well-being. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR1-10.2196/39977.

Development of the AGREE II, part 2: assessment of validity of items and tools to support application.

BACKGROUND: We established a program of research to improve the development, reporting and evaluation of practice guidelines. We assessed the construct validity of the items and user's manual in the beta version of the AGREE II. METHODS: We designed guideline excerpts reflecting high-and low-quality guideline content for 21 of the 23 items in the tool. We designed two study packages so that one low-quality and one high-quality version of each item were randomly assigned to each package. We randomly assigned 30 participants to one of the two packages. Participants reviewed and rated the guideline content according to the instructions of the user's manual and completed a survey assessing the manual. RESULTS: In all cases, content designed to be of high quality was rated higher than low-quality content; in 18 of 21 cases, the differences were significant (p < 0.05). The manual was rated by participants as appropriate, easy to use, and helpful in differentiating guidelines of varying quality, with all scores above the mid-point of the seven-point scale. Considerable feedback was offered on how the items and manual of the beta-AGREE II could be improved. INTERPRETATION: The validity of the items was established and the user's manual was rated as highly useful by users. We used these results and those of our study presented in part 1 to modify the items and user's manual. We recommend AGREE II (available at www.agreetrust.org) as the revised standard for guideline development, reporting and evaluation.

Development of the AGREE II, part 1: performance, usefulness and areas for improvement.

BACKGROUND: We undertook research to improve the AGREE instrument, a tool used to evaluate guidelines. We tested a new seven-point scale, evaluated the usefulness of the original items in the instrument, investigated evidence to support shorter, tailored versions of the tool, and identified areas for improvement. METHOD: We report on one component of a larger study that used a mixed design with four factors (user type, clinical topic, guideline and condition). For the analysis reported in this article, we asked participants to read a guideline and use the AGREE items to evaluate it based on a seven-point scale, to complete three outcome measures related to adoption of the guideline, and to provide feedback on the instrument's usefulness and how to improve it. RESULTS: Guideline developers gave lower-quality ratings than did clinicians or policy-makers. Five of six domains were significant predictors of participants' outcome measures (p < 0.05). All domains and items were rated as useful by stakeholders (mean scores > 4.0) with no significant differences by user type (p > 0.05). Internal consistency ranged between 0.64 and 0.89. Inter-rater reliability was satisfactory. We received feedback on how to improve the instrument. INTERPRETATION: Quality ratings of the AGREE domains were significant predictors of outcome measures associated with guideline adoption: guideline endorsements, overall intentions to use guidelines, and overall quality of guidelines. All AGREE items were assessed as useful in determining whether a participant would use a guideline. No clusters of items were found more useful by some users than others. The measurement properties of the seven-point scale were promising. These data contributed to the refinements and release of the AGREE II.

Recruitment of multiple stakeholders to health services research: lessons from the front lines.

BACKGROUND: Self-administered surveys are an essential methodological tool for health services and knowledge translation research, and engaging end-users of the research is critical. However, few documented accounts of the efforts invested in recruitment of multiple different stakeholders to one health services research study exist. Here, we highlight the challenges of recruiting key stakeholders (policy-makers, clinicians, guideline developers) to a Canadian Institutes of Health Research (CIHR) funded health services research (HSR) study aimed to develop an updated and refined version of a guideline appraisal tool, the AGREE. METHODS: Using evidence-based methods of recruitment, our goal was to recruit 192 individuals: 80 international guideline developers, 80 Canadian clinicians and 32 Canadian policy/decision-makers. We calculated the participation rate and the recruitment efficiency. RESULTS: We mailed 873 invitation letters. Of 838 approached, our participation rate was 29%(240) and recruitment efficiency, 19%(156). One policy-maker manager did not allow policy staff to participate in the study. CONCLUSIONS: Based on the results from this study, we suggest that future studies aiming to engage similar stakeholders in HSR over sample by at least 5 times to achieve their target sample size and allow for participant withdrawals. We need continued efforts to communicate the value of research between researchers and end-users of research (policy-makers, clinicians, and other researchers), integration of participatory research strategies, and promotion of the value of end-user involvement in research. Future research to understand methods of improving recruitment efficiency and engaging key stakeholders in HSR is warranted.

Effectiveness of guideline dissemination and implementation strategies on health care professionals' behaviour and patient outcomes in the cancer care context: a systematic review.

BACKGROUND: Health care professionals (HCPs) use clinical practice guidelines (CPGs) to make evidence-informed decisions regarding patient care. Although a large number of cancer-related CPGs exist, it is unknown which CPG dissemination and implementation strategies are effective for improving HCP behaviour and patient outcomes in a cancer care context. This review aimed to determine the effectiveness of CPG dissemination and/or implementation strategies among HCPs in a cancer care context. METHODS: A comprehensive search of five electronic databases was conducted. Studies were limited to the dissemination and/or implementation of a CPG targeting both medical and/or allied HCPs in cancer care. Two reviewers independently coded strategies using the Mazza taxonomy, extracted study findings, and assessed study quality. RESULTS: The search strategy identified 33 studies targeting medical and/or allied HCPs. Across the 33 studies, 23 of a possible 49 strategies in the Mazza taxonomy were used, with a mean number of 3.25 (SD = 1.45) strategies per intervention. The number of strategies used per intervention was not associated with positive outcomes. Educational strategies (n = 24), feedback on guideline compliance (n = 11), and providing reminders (n = 10) were the most utilized strategies. When used independently, providing reminders and feedback on CPG compliance corresponded with positive significant changes in outcomes. Further, when used as part of multi-strategy interventions, group education and organizational strategies (e.g. creation of an implementation team) corresponded with positive significant changes in outcomes. CONCLUSIONS: Future CPG dissemination and implementation interventions for cancer care HCPs may benefit from utilizing the identified strategies. Research in this area should aim for better alignment between study objectives, intervention design, and evaluation measures, and should seek to incorporate theory in intervention design, so that behavioural antecedents are considered and measured; doing so would enhance the field's understanding of the causal mechanisms by which interventions lead, or do not lead, to changes in outcomes at all levels.

Optimal messaging of the Canadian 24-Hour Movement Guidelines for Adults aged 18-64 years and Adults aged 65 years and older.

The Canadian 24-Hour Movement Guidelines for Adults aged 18-64 years and Adults aged 65 years and older ("Guidelines") integrate recommendations for physical activity, sedentary, and sleep behaviours. Given the novelty of these integrated Guidelines, it was important to consider messaging strategies that would be most effective in reaching Canadian adults. The purpose of this study was to examine optimal messaging of the Guidelines as it pertains to communication channels and messages. Representative samples of Guideline end-users (N = 1017) and stakeholders (N = 877) each completed a cross-sectional survey. Descriptive statistics were calculated along with tests of statistical significance. Inductive content analysis was used to code stakeholders' comments (i.e., suggestions, concerns) on a draft version of the Guidelines. Most end-users had recently referred to online medical resources; family, friends, and co-workers; and physicians as communication channels for information regarding the movement behaviours. End-users and stakeholders felt that generic messages would foster self-efficacy to meet the Guidelines. Stakeholders highlighted a variety of considerations to ensure the Guidelines are inclusive towards diverse groups within the Canadian population. Findings will inform Guideline messaging. Novelty Most end-users referred to online medical resources; family, friends, and co-workers; and physicians as communication channels. End-users and stakeholders indicated that generic messages would foster self-efficacy to meet the Guidelines. Stakeholders expressed concerns about the inclusivity of the Guidelines for diverse socioeconomic groups.

Assessment of the quality of recommendations from 161 clinical practice guidelines using the Appraisal of Guidelines for Research and Evaluation-Recommendations Excellence (AGREE-REX) instrument shows there is room for improvement.

OBJECTIVE: To assess the quality of recommendations from 161 clinical practice guidelines (CPGs) using AGREE-REX-D (Appraisal of Guidelines REsearch and Evaluation-Recommendations Excellence Draft). DESIGN: Cross-sectional study SETTING: International CPG community. PARTICIPANTS: Three hundred twenty-two international CPG developers, users, and researchers. INTERVENTION: Participants were assigned to appraise one of 161 CPGs selected for the study using the AGREE-REX-D tool MAIN OUTCOME MEASURES: AGREE-REX-D scores of 161 CPGs (7-point scale, maximum 7). RESULTS: Recommendations from 161 CPGs were appraised by 322 participants using the AGREE-REX-D. CPGs were developed by 67 different organizations. The total overall average score of the CPG recommendations was 4.23 (standard deviation (SD) = 1.14). AGREE-REX-D items that scored the highest were (mean; SD): evidence (5.51; 1.14), clinical relevance (5.95; SD 0.8), and patients/population relevance (4.87; SD 1.33), while the lowest scores were observed for the policy values (3.44; SD 1.53), local applicability (3,56; SD 1.47), and resources, tools, and capacity (3.49; SD 1.44) items. CPGs developed by government-supported organizations and developed in the UK and Canada had significantly higher recommendation quality scores with the AGREE-REX-D tool (p < 0.05) than their comparators. CONCLUSIONS: We found that there is significant room for improvement of some CPGs such as the considerations of patient/population values, policy values, local applicability and resources, tools, and capacity. These findings may be considered a baseline upon which to measure future improvements in the quality of CPGs.