[Interpretation of the reporting specifications for mixed methods systematic reviews].

The number of mixed methods systematic reviews (MMSRs) published internationally is increasing day by day, thanks to the continuous development and improvement of MMSRs methodological guidelines and reporting specification, which effectively promote the depth and breadth of evidence synthesis and integration results. However, the application of this method has yet to be popularized in China. With the continuous development of mixed methods research and evidence-based medicine in our country, the number of MMSRs will gradually increase. This paper aims to analyze the reporting specifications for MMSRs with cases to improve the quality of evidence integration and reporting standardization of domestic relevant researchers in MMSRs.

Critical reading of systematic review articles.

A systematic review provides an overview of primary studies investigating a given research question, e.g., the effect of a certain treatment. Individual study results are sometimes synthesised in a meta-analysis. A critical reader should consider whether the systematic review is relevant and reliable, e.g., does it follow a protocol, address the risk of bias, and consider potential heterogeneity. PRISMA 2020 guideline recommends a minimum set of items that should be reported in a systematic review article, and AMSTAR 2 and ROBIS are tools for critical appraisal of systematic reviews.

Some common, fatal flaws in systematic reviews of observational studies.

When evidence from randomized controlled trials about the effectiveness and safety of an intervention is unclear, researchers may choose to review the nonrandomized evidence. All systematic reviews pose considerable challenges, and the level of methodological expertise required to undertake a useful review of nonrandomized intervention studies is both high and often severely underestimated. Using the example of the endometrial receptivity array, we review some common, critical flaws in systematic reviews of this nature, including errors in critical appraisal and meta-analysis.

A scoping review on the methodological and reporting quality of scoping reviews in China.

BACKGROUND: Scoping reviews have emerged as a valuable method for synthesizing emerging evidence, providing a comprehensive contextual overview, and influencing policy and practice developments. The objective of this study is to provide an overview of scoping reviews conducted in Chinese academic institutions over the last decades. METHOD: We conducted a comprehensive search of nine databases and six grey literature databases for scoping reviews conducted in Chinese academic institutions. The reporting quality of the included reviews was assessed using the Preferred Reporting Items for PRISMA-ScR checklist. We performed both quantitative and qualitative analyses, examining the conduct of the scoping reviews and exploring the breadth of research topics covered. We used Chi-squared and Wilcoxon rank-sum tests to compare methodological issues and reporting quality in English and Chinese-language reviews. RESULTS: A total of 392 reviews published between 2013 and 2022 were included, 238 English-reported reviews and 154 Chinese-reported reviews, respectively. The primary purposes of these reviews were to map and summarize the evidence, with a particular focus on health and nursing topics. 98.7% of reviews explicitly used the term "scoping review", and the Arksey and O'Malley framework was the most frequently cited framework. Thirty-five English-reported scoping reviews provided a protocol for scoping review. PubMed was the most common source in English-reported reviews and CNKI in Chinese-reported reviews. Reviews published in English were more likely to search the grey literature (P = 0.005), consult information specialists (P < 0.001) and conduct an updated search (P = 0.012) than those in Chinese. Reviews published in English had a significantly high score compared to those published in Chinese (16 vs. 14; P < 0.001). The reporting rates in English-reported reviews were higher than those in Chinese reviews for seven items, but lower for structured summary (P < 0.001), eligibility criteria (P < 0.001), data charting process (P = 0.009) and data items (P = 0.015). CONCLUSION: There has been a significant increase in the number of scoping reviews conducted in Chinese academic institutions each year since 2020. While the research topics covered are diverse, the overall reporting quality of these reviews is need to be improved. And there is a need for greater standardization in the conduct of scoping reviews in Chinese academic institutions.

Investigate the relationship between the retraction reasons and the quality of methodology in non-Cochrane retracted systematic reviews: a systematic review.

BACKGROUND: This systematic review aimed to investigate the relationship between retraction status and the methodology quality in the retracted non-Cochrane systematic review. METHOD: PubMed, Web of Science, and Scopus databases were searched with keywords including systematic review, meta-analysis, and retraction or retracted as a type of publication until September 2023. There were no time or language restrictions. Non-Cochrane medical systematic review studies that were retracted were included in the present study. The data related to the retraction status of the articles were extracted from the retraction notice and Retraction Watch, and the quality of the methodology was evaluated with the AMSTAR-2 checklist by two independent researchers. Data were analyzed in the Excel 2019 and SPSS 21 software. RESULT: Of the 282 systematic reviews, the corresponding authors of 208 (73.75%) articles were from China. The average interval between publish and retraction of the article was about 23 months and about half of the non-Cochrane systematic reviews were retracted in the last 4 years. The most common reasons for retractions were fake peer reviews and unreliable data, respectively. Editors and publishers were the most retractors or requestors for retractions. More than 86% of the retracted non-Cochrane SRs were published in journals with an impact factor above two and had a critically low quality. Items 7, 9, and 13 among the critical items of the AMSTAR-2 checklist received the lowest scores. DISCUSSION AND CONCLUSION: There was a significant relationship between the reasons of retraction and the quality of the methodology (P-value < 0.05). Plagiarism software and using the Cope guidelines may decrease the time of retraction. In some countries, strict rules for promoting researchers increase the risk of misconduct. To avoid scientific errors and improve the quality of systematic reviews/meta-analyses (SRs/MAs), it is better to create protocol registration and retraction guidelines in each journal for SRs/MAs.

Evaluation of methodological and reporting quality of systematic reviews on conservative non-pharmacological musculoskeletal pain management in children and adolescents: A methodological analysis.

BACKGROUND: There are no studies investigating the methodological and report quality of systematic reviews of non-pharmacological interventions for musculoskeletal pain management among children and adolescents. OBJECTIVE: To evaluate the methodological and reporting quality of systematic reviews on conservative non-pharmacological pain management in children and adolescents with musculoskeletal pain. METHODS: Searches were conducted on the Cochrane Database of Systematic Reviews, Medline, Embase, and three other databases. Two pairs of reviewers independently assessed each article according to the predetermined selection criteria. We assessed the methodological quality of systematic reviews, using the AMSTAR 2 checklist and the quality of reporting, using PRISMA checklist. Descriptive analysis was used to summarise the characteristics of all included systematic reviews. The percentage of systematic reviews achieving each item from the AMSTAR 2, PRISMA checklist and the overall confidence in the results were described. RESULTS: We included 17 systematic reviews of conservative non-pharmacological pain management for musculoskeletal pain in children and adolescents. Of the 17 systematic reviews included, nine (53%) were rated as "critically low", seven (41%) were rated as "low", and one (6%) was rated as "high" methodological quality by AMSTAR-2. The reporting quality by items from PRISMA range from 17.6% (95% CI 6.2 to 41) to 100% (95% CI 81.6 to 100). CONCLUSION: This systematic review of physical interventions in children and adolescents showed overall 'very low' to 'high' methodological quality and usually poor reporting quality.

Principles for good scholarship in systematic reviews.

Many sources document problems that jeopardize the trustworthiness of systematic reviews. This is a major concern given their potential to influence patient care and impact people's lives. Responsibility for producing trustworthy conclusions on the evidence in systematic reviews is borne primarily by authors who need the necessary training and resources to correctly report on the current knowledge base. Peer reviewers and editors are also accountable; they must ensure that systematic reviews are accurate by demonstrating proper methods. To support all these stakeholders, we attempt to distill the sprawling guidance that is currently available in our recent co-publication about best tools and practices for systematic reviews. We specifically address how to meet methodological conduct standards applicable to key components of systematic reviews. In this complementary invited review, we place these standards in the context of good scholarship principles for systematic review development. Our intention is to reach a broad audience and potentially improve the trustworthiness of evidence syntheses published in the developmental medicine literature and beyond.

Guidance to best tools and practices for systematic reviews.

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Effect of PRISMA 2009 on reporting quality in systematic reviews and meta-analyses in high-impact dental medicine journals between 1993-2018.

INTRODUCTION: The PRISMA guidelines were published in 2009 to address inadequate reporting of key methodological details in systematic reviews and meta-analyses (SRs/MAs). This study sought to assess the impact of PRISMA on the quality of reporting in the full text of dental medicine journals. METHODS: This study assessed the impact of PRISMA (2009) on thirteen methodological details in SRs/MAs published in the highest-impact dental medicine journals between 1993-2009 (n = 211) and 2012-2018 (n = 618). The study further examined the rate of described use of PRISMA in the abstract or full text of included studies published post- PRISMA and the impact of described use of PRISMA on level of reporting. This study also examined potential effects of inclusion of PRISMA in Instructions for Authors, along with study team characteristics. RESULTS: The number of items reported in SRs/MAs increased following the publication of PRISMA (pre-PRISMA: M = 7.83, SD = 3.267; post-PRISMA: M = 10.55, SD = 1.4). Post-PRISMA, authors rarely mention PRISMA in abstracts (8.9%) and describe the use of PRISMA in the full text in 59.87% of SRs/MAs. The described use of PRISMA within the full text indicates that its intent (guidance for reporting) is not well understood, with over a third of SRs/MAs (35.6%) describing PRISMA as guiding the conduct of the review. However, any described use of PRISMA was associated with improved reporting. Among author team characteristics examined, only author team size had a positive relationship with improved reporting. CONCLUSION: Following the 2009 publication of PRISMA, the level of reporting of key methodological details improved for systematic reviews/meta-analyses published in the highest-impact dental medicine journals. The positive relationship between reference to PRISMA in the full text and level of reporting provides further evidence of the impact of PRISMA on improving transparent reporting in dental medicine SRs/MAs.

Guidance to Best Tools and Practices for Systematic Reviews.

¼ Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.¼ A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.¼ Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Efficacy and safety of acupuncture in the treatment of foot drop in post-stroke: A protocol for systematic review and meta-analysis.

BACKGROUND: Foot drop is a common complication in post-stroke. Patients with foot drop are at high risk for falls and fall-related injuries. Accordingly, it can reduce independence and quality of life in patients. Clinical studies have confirmed that acupuncture is effective in treating foot drop in post-stroke. However, there is a lack of systematic review exploring the efficacy and safety of acupuncture treatment. This study aims to assess the efficacy and safety of acupuncture in the treatment of foot drop in poststroke from the results of randomized controlled trials. METHODS: We will search articles in 8 electronic databases including the Cochrane Central Register of Controlled Trials, the Web of Science, PubMed, Embase, the China National Knowledge Infrastructure, the Chinese Biomedical Literature Database, Wanfang Data Database, and the Chinese Scientific Journal Database for RCTs of acupuncture treated foot drop in post-stroke from their inception to 10 August 2022. We will analyze the data meeting the inclusion criteria with the RevMan V.5.4 software. Two authors will assess the quality of the study with the Cochrane collaborative risk bias tool. We will evaluate the certainty of the estimated evidence with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) method. Data analysis will be performed using STATA 16.0. RESULTS: This study will review and evaluate the available evidence for the treatment of foot drop in post-stroke using acupuncture. CONCLUSION SUBSECTIONS: This study will determine the efficacy and safety of acupuncture applied to post-stroke individuals with foot drop.

The Campbell Collaboration's systematic review of school-based anti-bullying interventions does not meet mandatory methodological standards.

BACKGROUND: Many published reviews do not meet the widely accepted PRISMA standards for systematic reviews and meta-analysis. Campbell Collaboration and Cochrane reviews are expected to meet even more rigorous standards, but their adherence to these standards is uneven. For example, a newly updated Campbell systematic review of school-based anti-bullying interventions does not appear to meet many of the Campbell Collaboration's mandatory methodological standards. ISSUES: In this commentary, we document methodological problems in the Campbell Collaboration's new school-based anti-bullying interventions review, including (1) unexplained deviations from the protocol; (2) inadequate documentation of search strategies; (3) inconsistent reports on the number of included studies; (4) undocumented risk of bias ratings; (5) assessments of selective outcome reporting bias that are not transparent, not replicable, and appear to systematically underestimate risk of bias; (6) unreliable assessments of risk of publication bias; (7) use of a composite scale that conflates distinct risks of bias; and (8) failure to consider issues related to the strength of the evidence and risks of bias in interpreting results and drawing conclusions. Readers who are unaware of these problems may place more confidence in this review than is warranted. Campbell Collaboration editors declined to publish our comments and declined to issue a public statement of concern about this review. CONCLUSIONS: Systematic reviews are expected to use transparent methods and follow relevant methodological standards. Readers should be concerned when these expectations are not met, because transparency and rigor enhance the trustworthiness of results and conclusions. In the tradition of Donald T. Campbell, there is need for more public debate about the methods and conclusions of systematic reviews, and greater clarity regarding applications of (and adherence to) published standards for systematic reviews.

Practical guide to undertaking scoping reviews for pharmacy clinicians, researchers and policymakers.

WHAT IS KNOWN AND OBJECTIVE: Scoping reviews are a valuable evidence synthesis methodology. They can be used to map the evidence related to any topic to allow examination of practice, methods, policy and where (and how) future research could be undertaken. As such, they are a useful form of evidence synthesis for pharmacy clinicians, researchers and policymakers to review a broad range of evidence sources. COMMENT: This commentary presents the most comprehensive and up to date methodology for scoping reviews published by Joanna Briggs Institute (JBI). This approach builds upon two older approaches by Arksey and O'Malley, and Levac. To assist reviewers working in the field of pharmacy with planning and conducting scoping reviews, this paper describes how to undertake scoping reviews from inception to publication with specific examples related to pharmacy topics. WHAT IS NEW AND CONCLUSION: The JBI scoping review methodology is a valuable evidence synthesis approach to the field of pharmacy and therapeutics. This approach can assist pharmacy clinicians, researchers and policymakers to gain an understanding of the extant literature, to identify gaps, to explore concepts, characteristics and to examine current practice.

Systematic Reviews and Quality Appraisal of In Vitro Cancer Studies: Investigation of Current Practice.

BACKGROUND/AIM: To assess the quantity and quality of systematic reviews of in vitro cancer studies. MATERIALS AND METHODS: PubMed, MEDLINE, Embase, Web of Knowledge and PROSPERO databases were searched. Articles described as systematic reviews of in vitro studies, focused on or relevant to cancer and published in English were selected and appraised using an adapted version of AMSTAR 2 'critical domains'. RESULTS: From 4,021 records, 41 reviews described as systematic and cancer-related were identified. Publication dates indicate increasing frequency of systematic review conduct. Mean number of databases searched was three (range=1-8). Thirty-six reviews (88%) reported search methods, 35 (85%) specified inclusion criteria, 26 (63%) reported study selection methods, and 21 (51%) used reporting guidelines. Only 13 reviews (32%) involved formal quality assessment. CONCLUSION: Detailed investigation of reviews of cancer-relevant in vitro studies indicates need for further development and use of robust search strategies, appropriate quality assessment tools, and researchers with relevant skills.

How to Interpret and Use a Clinical Practice Guideline or Recommendation: Users' Guides to the Medical Literature.

IMPORTANCE: Clinicians may rely on recommendations from clinical practice guidelines for management of patients. OBSERVATIONS: A clinical practice guideline is a published statement that includes recommendations that are intended to optimize patient care. In the guideline development process, a panel of experts formulates recommendation questions that guide the retrieval of evidence that is used to inform the recommendations. Typically, methods of guideline development, a summary of the supporting evidence, and a justification of the panel's decisions accompany the recommendations. To use such guidelines optimally, clinicians must understand the implications of the recommendations, assess the trustworthiness of the development process, and evaluate the extent to which the recommendations are applicable to patients in their practice settings. Helpful recommendations are clear and actionable, and explicitly specify whether they are strong or weak, are appropriate for all patients, or depend on individual patients' circumstances and values. Rigorous guidelines and recommendations are informed by appropriately conducted, up-to-date systematic reviews that consider outcomes important to patients. Because judgments are involved in the interpretation of the evidence and the process of moving from evidence to recommendations, useful guidelines consider all relevant factors that have a bearing in a clinical decision and are not influenced by conflicts of interest. CONCLUSIONS AND RELEVANCE: In considering a guideline's recommendations, clinicians must decide whether there are important differences between the factors the guideline panel has considered in making recommendations and their own practice setting.

Methodological quality of systematic reviews comprising clinical practice guidelines for cardiovascular risk assessment and management for noncardiac surgery.

BACKGROUND: Cardiac assessment in noncardiac surgery clinical practice guidelines should be supported by the highest-quality evidence such as that offered by systematic reviews. Currently, the methodological and reporting quality of these studies remains unknown. METHODS: We used PubMed to search for all clinical practice guidelines related to perioperative cardiovascular patients undergoing noncardiac surgery from 2010 to 2021. The included clinical practice guidelines were analysed for all systematic reviews and meta-analyses. The primary objective of this study was to determine reporting and methodological quality using the PRISMA (Preferred Reporting Instrument for Systematic Reviews and Meta-Analyses) and AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews-2) instruments. Our secondary objective was to compare systematic reviews conducted by the Cochrane Collaboration with non-Cochrane studies. RESULTS: Three clinical practice guidelines were included in our study. Within these, 78 systematic reviews were included. PRISMA completion ranged from 34.8% to 100.0% with a mean of 76.9%. AMSTAR-2 completion ranged from 15.6% to 96.9% with a mean of 58.0%. Fifty-four systematic reviews underpinned a clinical practice guidelines recommendation, of which 25 were rated 'critically low' by AMSTAR-2 appraisal. Cochrane systematic reviews typically performed better than non-Cochrane studies, but were a minority of the included studies (10/78). CONCLUSION: We found deficiencies in several key areas regarding the methodological and reporting qualities of systematic reviews included in cardiac assessment in noncardiac surgery clinical practice guidelines. As these clinical practice guidelines are instrumental to clinical decision-making and patient care in cardiac assessment in noncardiac surgery, we advocate for improved reporting quality among systematic reviews cited as supportive evidence for these recommendations.