Evolution of appraisal tool usage preferences in PROSPERO records: a study of non-Cochrane systematic reviews.

OBJECTIVES: This research-on-research substudy uses a data-driven approach to investigate the range of appraisal tools in non-Cochrane systematic reviews and meta-analyses registered in the International Prospective Register of Systematic Reviews (PROSPERO). STUDY DESIGN AND SETTING: A comprehensive web scraping of all completed non-Cochrane registrations in PROSPERO from February 2011 to December 2017 was performed. The focus was classifying the appraisal tools based on study type, assessment aspects, and research topics. RESULTS: After analyzing 17,708 complete records, we found a predominant use of methodological quality assessment tools compared to those for reporting quality or risk of bias (RoB). This indicates a greater emphasis on methodological rigor in the studied protocols. Various tools for assessing methodological quality were observed, reflecting the complexity of such evaluations. Instruments designed for evaluating methodological or reporting quality were mainly intended for non-randomized clinical trials or observational studies, unlike RoB tools more commonly used in randomized clinical trials. No distinct trends in tool usage were observed in specific research conditions or domains, suggesting that tool choice is influenced more by study design than research topic. CONCLUSION: This study provides insights into the preferential use of various assessment tools in conducting non-Cochrane systematic reviews, as evidenced in PROSPERO records. The findings reveal various methodological assessment tools, underscoring their versatility across different study designs and research areas.

Critical Appraisal Tools to Aid Pharmacists in Evidence-Based Practice: A Narrative Review.

Background: Pharmacists and allied health researchers need to ensure that their practice is supported by current, evidence-based information. Critical appraisal tools have been developed to aid in this process. Objectives: To analyze the current landscape of critical appraisal tools and to create an aid for pharmacists and other allied health researchers to use in comparing various tools and choosing the best one for each particular study design. Data Sources: A literature search of the PubMed, University of Toronto Libraries, and Cochrane Library databases was conducted in December 2021, to generate an up-to-date list of critical appraisal tools. The tools were then summarized in a descriptive table. Study Selection and Data Extraction: Review articles, original manuscripts, and tool webpages were examined to develop a comparison chart based on the user-friendliness, efficiency, comprehensiveness, and reliability of each tool. Results: Fourteen tools were found through the literature search. These tools were compared using the findings of included review articles, and a comparison chart was created to aid pharmacists and allied health researchers in selecting the appropriate tool for their practice. Conclusions: There are many standardized critical appraisal tools that can help in assessing the quality of evidence, and the summary list of tools developed and reported here can help health care researchers to compare among them and choose the best one. No tools were found that have been specifically adapted to serve the needs of pharmacists when assessing scientific articles. Future research should examine how existing critical appraisal tools can better identify common data elements that are essential to evidence-based decision-making in pharmacy practice.

Contexte: Les pharmaciens et les chercheurs en soins de la santé doivent faire en sorte que leur pratique soit étayée par des informations actualisées et fondées sur des données probantes. Des outils d'évaluation critique ont été développés pour faciliter ce processus. Objectifs: Analyser le paysage actuel des outils d'évaluation critique et créer une aide que les pharmaciens et les autres chercheurs paramédicaux peuvent utiliser pour comparer divers outils et choisir le meilleur pour chaque conception d'étude particulière. Sources des données: Une recherche documentaire dans trois bases de données (PubMed, les University of Toronto Libraries et la Cochrane Library) a été menée en décembre 2021 afin de générer une liste actualisée d'outils d'évaluation critique qui ont ensuite été résumés dans un tableau descriptif. Sélection des études et extraction des données: Des articles de synthèse, des manuscrits originaux et des pages Internet d'outils ont été examinés pour dresser un tableau comparatif basé sur la convivialité, l'efficacité, l'exhaustivité et la fiabilité de chaque outil. Résultats: Quatorze outils ont été trouvés grâce à la recherche documentaire. Ils ont été comparés à l'aide des résultats des articles de synthèse inclus, et un tableau comparatif a été créé pour aider les pharmaciens et les chercheurs en soins de la santé à sélectionner l'outil approprié pour leur pratique. Conclusions: De nombreux outils d'évaluation critique normalisés peuvent aider à évaluer la qualité des données probantes, et la liste récapitulative des outils développés et rapportés ici peut aider les chercheurs en soins de santé à les comparer et à choisir le meilleur. Aucun outil spécifiquement adapté pour répondre aux besoins des pharmaciens lors de l'évaluation d'articles scientifiques n'a été trouvé. Les recherches futures devraient se pencher sur la manière dont les outils d'évaluation critique existants peuvent mieux identifier les éléments de données communs qui sont essentiels à la prise de décision fondée sur des données probantes dans la pratique de la pharmacie.

JBI series paper 2: tailored evidence synthesis approaches are required to answer diverse questions: a pragmatic evidence synthesis toolkit from JBI.

Evidence synthesis is critical in evidence-based healthcare and is a core program of JBI. JBI evidence synthesis is characterised by a pluralistic view of what constitutes evidence and is underpinned by a pragmatic ethos to facilitate the use of evidence to inform practice and policy. This second paper in this series provides a descriptive overview of the JBI evidence synthesis toolkit with reference to resources for 11 different types of reviews. Unique methodologies such as qualitative syntheses, mixed methods reviews, and scoping reviews are highlighted. Key features include standardised and collaborative processes for development of methodologies and a broad range of tailored resources to facilitate the conduct of a JBI evidence synthesis, including appraisal and data extraction tools, software to support the conduct of a systematic review and an intensive systematic review training program. JBI is one of the leading international protagonists for evidence synthesis, providing those who want to answer health-related questions with a toolkit of resources to synthesize the evidence.

Promoting cultural rigour through critical appraisal tools in First Nations peoples' research.

OBJECTIVE: To highlight the emerging ethos of cultural rigour in the use of critical appraisal tools in research involving First Nations peoples. METHODS: Critical reflection on recent systematic review experience. RESULTS: The concept of cultural rigour is notably undefined in peer-reviewed journal articles but is evident in the development of critical appraisal tools developed by First Nations peoples. CONCLUSIONS: Conventional critical appraisal tools for assessing study quality are built on a limited view of health that excludes the cultural knowledge of First Nations peoples. Cultural rigour is an emerging field of activity that epitomises First Nations peoples' diverse cultural knowledge through community participation in all aspects of research. Implications for public health: Critical appraisal tools developed by First Nations peoples are available to researchers and direct attention to the social, cultural, political and human rights basis of health research.

Systematic review finds that appraisal tools for medical research studies address conflicts of interest superficially.

OBJECTIVES: The objective of this study was to identify and summarize 1) appraisal tools and other guides which address conflicts of interest in medical research studies; and 2) top journals with policies on managing conflicts of interest in journal papers. STUDY DESIGN AND SETTING: We searched bibliographic databases, other sources, and websites of 30 top medical journals. Two authors selected documents and extracted data. RESULTS: We included 27 appraisal tools. None were designed specifically for addressing conflicts of interest and they included only 1-2 short items on conflicts of interest. We also included eight other types of guides. Of 27 appraisal tools, 23 addressed study funding, and 19 authors' conflicts of interest. Nine tools addressed availability of conflicts of interest information, 13 reported conflicts of interest, and five influence from conflicts of interest. Twelve of 30 top journals had conflicts of interest managing policies (beyond disclosure). One journal restricted nonresearch papers (e.g., editorials) to authors without financial conflicts of interest and ten only restricted under certain circumstances. CONCLUSION: Appraisal tools that address conflicts of interest typically do so superficially and rarely address how conflicts of interest may influence studies. Less than half of top medical journals have explicit policies on managing conflicts of interest.

Critical Appraisal Tools and Reporting Guidelines for Evidence-Based Practice.

BACKGROUND: Nurses engaged in evidence-based practice (EBP) have two important sets of tools: Critical appraisal tools and reporting guidelines. Critical appraisal tools facilitate the appraisal process and guide a consumer of evidence through an objective, analytical, evaluation process. Reporting guidelines, checklists of items that should be included in a publication or report, ensure that the project or guidelines are reported on with clarity, completeness, and transparency. PURPOSE: The primary purpose of this paper is to help nurses understand the difference between critical appraisal tools and reporting guidelines. A secondary purpose is to help nurses locate the appropriate tool for the appraisal or reporting of evidence. METHODS: A systematic search was conducted to find commonly used critical appraisal tools and reporting guidelines for EBP in nursing. RATIONALE: This article serves as a resource to help nurse navigate the often-overwhelming terrain of critical appraisal tools and reporting guidelines, and will help both novice and experienced consumers of evidence more easily select the appropriate tool(s) to use for critical appraisal and reporting of evidence. Having the skills to select the appropriate tool or guideline is an essential part of meeting EBP competencies for both practicing registered nurses and advanced practice nurses (Melnyk & Gallagher-Ford, 2015; Melnyk, Gallagher-Ford, & Fineout-Overholt, 2017). RESULTS: Nine commonly used critical appraisal tools and eight reporting guidelines were found and are described in this manuscript. Specific steps for selecting an appropriate tool as well as examples of each tool's use in a publication are provided. LINKING EVIDENCE TO ACTION: Practicing registered nurses and advance practice nurses must be able to critically appraise and disseminate evidence in order to meet EBP competencies. This article is a resource for understanding the difference between critical appraisal tools and reporting guidelines, and identifying and accessing appropriate tools or guidelines.

The impact of concussion on cardiac autonomic function: A systematic review.

PRIMARY OBJECTIVE: To evaluate the evidence regarding the effect of concussion on cardiac autonomic function (CAF). INCLUSION CRITERIA: Original research; available in English; included participants with concussion or mild traumatic brain injury (mTBI) and a comparison group; included measures of heart rate (HR) and/or heart rate variability (HRV) as outcomes. Studies of humans (greater than 6 years old) and animals were included. Critical appraisal tools: The Downs and Black (DB) criteria and Structured Effectiveness Quality Evaluation Scale (SEQES). RESULTS: Nine full-length articles and four abstracts were identified. There is conflicting evidence regarding CAF at rest following concussion. There is evidence of elevated HR and reduced HRV with low-intensity, steady-state exercise up to 10 days following concussion. There was no significant difference in HRV during isometric handgrip testing or HR while performing cognitive tasks following concussion. The validity of current literature is limited by small sample sizes, lack of female or paediatric participants, methodological heterogeneity and lack of follow-up. CONCLUSIONS: While there is some evidence to suggest CAF is altered during physical activity following concussion, methodological limitations highlight the need for further research. Understanding the effect of concussion on CAF will contribute to the development of more comprehensive concussion management strategies.

Evaluation of natriuretic peptide recommendations in heart failure clinical practice guidelines.

BACKGROUND: The B-type naturietic peptides (NPs) are associated with heart failure (HF). This investigation was designed to evaluate heart failure clinical practice guideline (CPG) recommendations for the use of NPs. METHODS: A search for English language CPGs for HF published since 2011 was conducted. A search for systematic reviews (SR) and meta-analysis for NPs in HF was conducted for the years 2004-2012. Each HF CPG was evaluated by two independent reviewers. Key recommendations for NPs and the supporting references were abstracted. The key findings from each SR were abstracted. RESULTS: Seven English language HF CPGs were found, all of which made recommendations for the use of NPs in diagnosis. Four made recommendations for prognosis and three for management. The European CPG scored highly for rigor of development with the Appraisal of Guidelines for Research and Evaluation Instrument (AGREE II) while the others did not. North American CPGs made stronger recommendations citing higher grades of evidence for the use of NPs in HF than the European or Australian CPGs. The CPGs mostly cited primary studies 47/66 to support the recommendations. From twelve available SRs, five were cited by CPGs. One CPG conducted a SR. CONCLUSIONS: The SR evidence to support NP use in CPGs has not been well cited in the CPGs and the recommendations are only partially supported by the SR evidence. Future CPGs should consider improving the methodology used to evaluate laboratory tests.

Office-Based Tools and Primary Care Visit Communication, Length, and Preventive Service Delivery.

BACKGROUND: The use of physician office-based tools such as electronic health records (EHRs), health risk appraisal (HRA) instruments, and written patient reminder lists is encouraged to support efficient, high-quality, patient-centered care. We evaluate the association of exam room use of EHRs, HRA instruments, and self-generated written patient reminder lists with patient-physician communication behaviors, recommended preventive health service delivery, and visit length. RESEARCH METHODS: Observational study of 485 office visits with 64 primary care physicians practicing in a health system serving the Detroit metropolitan area. Study data were obtained from patient surveys, direct observation, office visit audio-recordings, and automated health system records. Outcome measures included visit length in minutes, patient use of active communication behaviors, physician use of supportive talk and partnership-building communication behaviors, and percentage of delivered guideline-recommended preventive health services for which patients are eligible and due. Simultaneous linear regression models were used to evaluate associations between tool use and outcomes. Adjusted models controlled for patient characteristics, physician characteristics, characteristics of the relationship between the patient and physician, and characteristics of the environment in which the visit took place. RESULTS: Prior to adjusting for other factors, visits in which the EHR was used on average were significantly (p < .05) longer (27.6 vs. 23.8 minutes) and contained fewer preventive services for which patients were eligible and due (56.5 percent vs. 62.7 percent) compared to those without EHR use. Patient written reminder lists were also significantly associated with longer visits (30.0 vs. 26.5 minutes), and less use of physician communication behaviors facilitating patient involvement (2.1 vs. 2.6 occurrences), but more use of active patient communication behaviors (4.4 vs. 2.6). Likewise, HRA use was significantly associated with increased preventive services delivery (62.1 percent vs. 57.0 percent). All relationships remained significant (p > .05) in adjusted models with the exception of that between HRA use and preventive service delivery. DISSEMINATION AND IMPLEMENTATION IMPLICATIONS: Office-based tools intended to facilitate the implementation of desired primary care practice redesign are associated with both positive and negative cost and quality outcomes. Findings highlight the need for monitoring both intended and unintended consequences of office-based tools commonly used in primary care practice redesign.

Some notes on critical appraisal of prevalence studies: Comment on: "The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence".

Decisions in healthcare should be based on information obtained according to the principles of Evidence-Based Medicine (EBM). An increasing number of systematic reviews are published which summarize the results of prevalence studies. Interpretation of the results of these reviews should be accompanied by an appraisal of the methodological quality of the included data and studies. The critical appraisal tool for prevalence studies developed and tested by Munn et al. comprises 10 items and aims at targeting all kinds of prevalence studies. This comment discusses the pros and cons of different designs of quality appraisal tools and highlights their importance for systematic reviews of prevalence studies. Beyond piloting, which has been performed in the study by Munn et al., it is suggested here that the validity of the tool should be tested, including reproducibility and inter-rater reliability. It is concluded that studies as the one by Munn et al. will help to establish a critical understanding of methodological quality and will support the use of systematic reviews of non-intervention studies for health policy making.