Predatory journals and their practices present a conundrum for systematic reviewers and evidence synthesisers of health research: A qualitative descriptive study.

Predatory journals are a blemish on scholarly publishing and academia and the studies published within them are more likely to contain data that is false. The inclusion of studies from predatory journals in evidence syntheses is potentially problematic due to this propensity for false data to be included. To date, there has been little exploration of the opinions and experiences of evidence synthesisers when dealing with predatory journals in the conduct of their evidence synthesis. In this paper, the thoughts, opinions, and attitudes of evidence synthesisers towards predatory journals and the inclusion of studies published within these journals in evidence syntheses were sought. Focus groups were held with participants who were experienced evidence synthesisers from JBI (previously the Joanna Briggs Institute) collaboration. Utilising qualitative content analysis, two generic categories were identified: predatory journals within evidence synthesis, and predatory journals within academia. Our findings suggest that evidence synthesisers believe predatory journals are hard to identify and that there is no current consensus on the management of these studies if they have been included in an evidence synthesis. There is a critical need for further research, education, guidance, and development of clear processes to assist evidence synthesisers in the management of studies from predatory journals.

Effectiveness of dual active ingredient insecticide-treated nets in preventing malaria: A systematic review and meta-analysis.

Malaria vectors have demonstrated resistance to pyrethroid-based insecticides used in insecticide-treated nets, diminishing their effectiveness. This systematic review and meta-analysis investigated two forms of dual active-ingredient (DAI) insecticide-treated nets (ITN(s)) for malaria prevention. A comprehensive search was conducted on July 6th 2022. The databases searched included PubMed, Embase, CINAHL, amongst others. Trials were eligible if they were conducted in a region with ongoing malaria transmission. The first DAI ITN investigated were those that combined a pyrethroid with a non-pyrethroid insecticides. The second DAI ITN investigated were that combined a pyrethroid with an insect growth regulator. These interventions were compared against either a pyrethroid-only ITN, or ITNs treated with pyrethroid and piperonyl-butoxide. Assessment of risk of bias was conducted in duplicate using the Cochrane risk of bias 2 tool for cluster-randomised trials. Summary data was extracted using a custom data-extraction instrument. This was conducted by authors THB, JCS and SH. Malaria case incidence was the primary outcome and has been meta-analysed, adverse events were narratively synthesised. The review protocol is registered on PROSPERO (CRD42022333044). From 9494 records, 48 reports were screened and 13 reports for three studies were included. These studies contained data from 186 clusters and all reported a low risk of bias. Compared to pyrethroid-only ITNs, clusters that received pyrethroid-non-pyrethroid DAI ITNs were associated with 305 fewer cases per 1000-person years (from 380 fewer cases to 216 fewer cases) (IRR = 0.55, 95%CI: 0.44-0.68). However, this trend was not observed in clusters that received pyrethroid-insect growth regulator ITNs compared to pyrethroid-only ITNs (from 280 fewer cases to 135 more) (IRR = 0.90, 95%CI: 0.73-1.13). Pyrethroid-non-pyrethroid DAI ITNs demonstrated consistent reductions in malaria case incidence and other outcomes across multiple comparisons. Pyrethroid-non-pyrethroid DAI ITNs may present a novel intervention for the control of malaria.

Residual insecticide surface treatment for preventing malaria: a systematic review protocol.

INTRODUCTION: Malaria presents a significant global public health burden, although substantial progress has been made, with vector control initiatives such as indoor residual surface spraying with insecticides and insecticide-treated nets. There now exists many different approaches to apply residual insecticide to indoor and outdoor surfaces in malaria-endemic settings, although no comprehensive systematic reviews exist evaluating these interventions. This manuscript outlines the protocol for a systematic review which aims to synthesise the best available evidence regarding full or partial indoor or outdoor residual insecticide surface treatment for preventing malaria. METHODS AND ANALYSIS: This review will comprehensively search the literature (both published and unpublished) for any studies investigating the effectiveness of residual insecticide surface treatment for malaria. Studies will be screened to meet the inclusion criteria by a minimum of two authors, followed by assessment of risk of bias (using appropriate risk-of-bias tools for randomised and non-randomised studies) and extraction of relevant information using structured forms by two independent authors. Meta-analysis will be carried out where possible for epidemiological outcomes such as malaria, anaemia, malaria-related mortality, all-cause mortality and adverse effects. Certainty in the evidence will be established with GRADE assessments. ETHICS AND DISSEMINATION: A full review report will be submitted to the Vector Control & Insecticide Resistance Unit, Global Malaria Program, WHO. A version of this report will be submitted for publication in an open access peer-reviewed journal. The report will inform the development of WHO recommendations regarding residual insecticide treatment for malaria. This systematic review does not require ethics approval as it is a review of primary studies. SYSTEMATIC REVIEW REGISTRATION: PROSPERO 293194.

Effectiveness of perioperative anticoagulation interruption without heparin bridging on thromboembolic events in patients with atrial fibrillation undergoing elective invasive procedures: a systematic review protocol.

OBJECTIVE: This review will determine whether withholding heparin bridging is superior to bridging in patients with atrial fibrillation requiring temporary interruption of anticoagulation therapy in the perioperative period of an elective invasive procedure. INTRODUCTION: Atrial fibrillation is the most commonly diagnosed clinical arrhythmia. It is an important cause of cardioembolic events, requiring the use of oral anticoagulation in most patients. It is unclear whether heparin bridging during temporary interruption of anticoagulants has superior outcomes compared with no bridging in the perioperative setting. INCLUSION CRITERIA: This review will consider studies that compare adults aged 18 years or older; diagnosed with atrial fibrillation; undergoing elective invasive procedures; and who have had oral anticoagulants temporarily withheld with heparin bridging with patients without heparin bridging. Participants will be excluded if they had an alternative reason for anticoagulation or were admitted for emergency surgery. Outcomes will include arterial or venous thromboembolism (including stroke, transient ischemic attack, systemic embolism), major bleeding events, non-major bleeding events, length of hospital stay, and all-cause mortality. METHODS: The review will follow the JBI methodology for systematic reviews of effectiveness. Databases including MEDLINE, Embase, CINAHL, and CENTRAL will be searched for randomized and non-randomized trials from inception until the present. Two independent reviewers will screen citations by title and abstract, and again at full text. Risk of bias will be assessed using the JBI critical appraisal instrument, and data will be extracted using a modified extraction tool. Results will be synthesized using a random effects meta-analysis and presented in a forest plot. Heterogeneity will be tested for using the standard χ 2 and I2 tests. Overall certainty of evidence will be evaluated using the GRADE approach. REVIEW REGISTRATION: PROSPERO CRD42022348538.

From critical appraisal to risk of bias assessment: clarifying the terminology for study evaluation in JBI systematic reviews.

The foundations for critical appraisal of literature have largely progressed through the development of epidemiologic research methods and the use of research to inform medical teaching and practice. This practical application of research is referred to as evidence-based medicine and has delivered a standard for the health care profession where clinicians are equally as engaged in conducting scientific research as they are in the practice of delivering treatments. Evidence-based medicine, now referred to as evidence-based health care, has generally been operationalized through empirically supported treatments, whereby the choice of treatments is substantiated by scientific support, usually by means of an evidence synthesis. As evidence synthesis methodology has advanced, guidance for the critical appraisal of primary research has emphasized a distinction from the assessment of internal validity required for synthesized research. This assessment is conceptualized and branded in various ways in the literature, such as risk of bias, critical appraisal, study validity, methodological quality, and methodological limitations. This paper provides a discussion of the definitions and characteristics of these terms, concluding with a recommendation for JBI to adopt the term "risk of bias" assessment.

The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials.

JBI recently began the process of updating and revising its suite of critical appraisal tools to ensure that these tools remain compatible with recent developments within risk of bias science. Following a rigorous development process led by the JBI Effectiveness Methodology Group, this paper presents the revised critical appraisal tool for the assessment of risk of bias for randomized controlled trials. This paper also presents practical guidance on how the questions of this tool are to be interpreted and applied by systematic reviewers, while providing topical examples. We also discuss the major changes made to this tool compared to the previous version and justification for why these changes facilitate best-practice methodologies in this field.

Reporting quality and risk of bias in JBI systematic reviews evaluating the effectiveness of interventions: a methodological review protocol.

OBJECTIVE: The objective of this methodological review is to evaluate the adherence of systematic reviews of effectiveness published in JBI Evidence Synthesis to reporting guidelines and methodological quality. INTRODUCTION: Systematic reviews of effectiveness are essential tools for health practitioners and policy-makers. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines and the Risk of Bias in Systematic Reviews (ROBIS) tool are used to ensure maintenance of high reporting standards and methodological quality, respectively. This review will utilize these tools to identify strengths and shortfalls in the reporting quality of JBI systematic reviews of effectiveness. INCLUSION CRITERIA: This review will include the 20 most recent systematic reviews of effectiveness published in JBI Evidence Synthesis . METHODS: This review will search MEDLINE (PubMed) for effectiveness reviews published in JBI Evidence Synthesis . Abstract and full-text screening will be performed by 2 independent reviewers, and the most recent 20 studies will be selected for inclusion. Data regarding adherence to PRISMA 2020 and ROBIS will be extracted by 2 independent reviewers. Data will be presented descriptively with tables and synthesized narratively.

Assessing the risk of bias of quantitative analytical studies: introducing the vision for critical appraisal within JBI systematic reviews.

A key step in the systematic review process is the assessment of the methodological quality (or risk of bias) of the included studies. At JBI, we have developed several tools to assist with this evaluation. As evidence synthesis methods continue to evolve, it has been necessary to revise and reflect on JBI's current approach to critical appraisal and to plan a strategy for the future. In this first paper of a series focusing on risk of bias assessment, we introduce our vision for risk of bias assessment for JBI. In future papers in this series, the methodological approach taken for this revision process will be discussed, along with the revised tools and guidance for using these tools.

Investigating different typologies for the synthesis of evidence: a scoping review protocol.

OBJECTIVE: The objective of this scoping review is to identify evidence synthesis types and previously proposed classification systems, typologies, or taxonomies that have guided evidence synthesis. INTRODUCTION: Evidence synthesis is a constantly evolving field. There is now a plethora of evidence synthesis approaches used across many different disciplines. Historically, there have been numerous attempts to organize the types and methods of evidence synthesis in the form of classification systems, typologies, or taxonomies. This scoping review will seek to identify all the available classification systems, typologies, or taxonomies; how they were developed; their characteristics; and the types of evidence syntheses included within them. INCLUSION CRITERIA: This scoping review will include discussion papers, commentaries, books, editorials, manuals, handbooks, and guidance from major organizations that describe multiple approaches to evidence synthesis in any discipline. METHODS: The Evidence Synthesis Taxonomy Initiative will support this scoping review. The search strategy will aim to locate both published and unpublished documents utilizing a three-step search strategy. An exploratory search of MEDLINE has identified keywords and MeSH terms. A second search of MEDLINE, Embase, CINAHL with Full Text, ERIC, Scopus, Compendex, and JSTOR will be conducted. The websites of relevant evidence synthesis organizations will be searched. Identified documents will be independently screened, selected, and extracted by two researchers, and the data will be presented in tables and summarized descriptively. DETAILS OF THIS REVIEW PROJECT ARE AVAILABLE AT: Open Science Framework https://osf.io/qwc27.

Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process.

JBI offers a suite of critical appraisal instruments that are freely available to systematic reviewers and researchers investigating the methodological limitations of primary research studies. The JBI instruments are designed to be study-specific and are presented as questions in a checklist. The JBI instruments have existed in a checklist-style format for approximately 20 years; however, as the field of research synthesis expands, many of the tools offered by JBI have become outdated. The JBI critical appraisal tools for quantitative studies (eg, randomized controlled trials, quasi-experimental studies) must be updated to reflect the current methodologies in this field. Cognizant of this and the recent developments in risk-of-bias science, the JBI Effectiveness Methodology Group was tasked with updating the current quantitative critical appraisal instruments. This paper details the methods and rationale that the JBI Effectiveness Methodology Group followed when updating the JBI critical appraisal instruments for quantitative study designs. We detail the key changes made to the tools and highlight how these changes reflect current methodological developments in this field.

Addressing Challenges When Applying GRADE to Public Health Guidelines: A Scoping Review Protocol and Pilot Analysis.

This is a protocol for a scoping review that aims to determine how guideline authors using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach have addressed previously identified challenges related to public health. The Joanna Briggs Institute (JBI) methodology for scoping reviews will be followed. We will search and screen titles of guidelines for all languages published in 2013-2021 in: the GIN library, BIGG database, Epistemonikos GRADE guidelines repository, GRADEpro Database, MAGICapp, NICE and WHO websites. Two reviewers will independently screen full texts of the documents identified. The following information will be extracted: methods used for identifying different stakeholders and incorporating their perspectives; methods for identification and prioritization of non-health outcomes; methods for determining thresholds for decision-making; methods for incorporating and grading evidence from non-randomized studies; methods for addressing concerns with conditional recommendations in public health; methods for reaching consensus; additional methodological concerns; and any modifications made to GRADE. A combination of directed content analysis and descriptive statistics will be used for data analysis, and the findings presented narratively in a tabular and graphical form. In this protocol, we present the pilot results from 13 identified eligible guidelines issued between January and August 2021. We will publish the full review results when they become available.

Meta-analysis of prevalence: I2 statistic and how to deal with heterogeneity.

Over the last decade, there has been a 10-fold increase in the number of published systematic reviews of prevalence. In meta-analyses of prevalence, the summary estimate represents an average prevalence from included studies. This estimate is truly informative only if there is no substantial heterogeneity among the different contexts being pooled. In systematic reviews, heterogeneity is usually explored with I-squared statistic (I2 ), but this statistic does not directly inform us about the distribution of effects and frequently systematic reviewers and readers misinterpret this result. In a sample of 134 meta-analyses of prevalence, the median I2 was 96.9% (IQR 90.5-98.7). We observed larger I2 in meta-analysis with higher number of studies and extreme pooled estimates (defined as <10% or >90%). Studies with high I2 values were more likely to have conducted a sensitivity analysis, including subgroup analysis but only three (2%) systematic reviews reported prediction intervals. We observed that meta-analyses of prevalence often present high I2 values. However, the number of studies included in the meta-analysis and the point estimate can be associated with the I2 value, and a high I2 value is not always synonymous with high heterogeneity. In meta-analyses of prevalence, I2 statistics may not be discriminative and should be interpreted with caution, avoiding arbitrary thresholds. To discuss heterogeneity, reviewers should focus on the description of the expected range of estimates, which can be done using prediction intervals and planned sensitivity analysis.

Conducting proportional meta-analysis in different types of systematic reviews: a guide for synthesisers of evidence.

BACKGROUND: Single group data present unique challenges for synthesises of evidence. Proportional meta-analysis is becoming an increasingly common technique employed for the synthesis of single group data. Proportional meta-analysis shares many similarities with the conduct and reporting of comparative, or pairwise, meta-analysis. While robust and comprehensive methods exist detailing how researchers can conduct a meta-analysis that compares two (or more) groups against a common intervention, there is a scarcity of methodological guidance available to assist synthesisers of evidence in the conduct, interpretation, and importance of proportional meta-analysis in systematic reviews. MAIN BODY: This paper presents an overview targeted to synthesisers of evidence and systematic review authors that details the methods, importance, and interpretation of a proportional meta-analysis. We provide worked examples of how proportional meta-analyses have been conducted in research syntheses previously and consider the methods, statistical considerations, and presentation of this technique. CONCLUSION: This overview is designed to serve as practical guidance for synthesisers of evidence in the conduct of proportional meta-analyses.

Guidelines rarely used GRADE and applied methods inconsistently: A methodological study of Australian guidelines.

OBJECTIVES: The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach is accepted methodology to assess the certainty of the evidence included in systematic reviews and clinical practice guidelines. The GRADE approach is endorsed globally, in Australia, the National Health and Medical Research Council advocated for the use of the GRADE approach in 2011. The purpose of this methodological review was to assess how GRADE has been adopted for Australian practice guidelines. STUDY DESIGN AND SETTING: This methodological review searched of the National Health and Medical Research Council Clinical Practice Guidelines Portal from 2011 to 2018, in an effort to retrieve all practice guidelines available via this medium. RESULTS: 240 guidelines were retrieved authored by 51 different organizations. 15 guidelines followed GRADE methodology. Application of GRADE methods varied between guidelines, some misreported and altered aspects of the GRADE process. Guidelines that closely adhered to the guidance from the GRADE Working Group scored higher in domain 3 (rigor of development) of the Appraisal of Guidelines for Research and Evaluation II tool, indicating a positive linear relationship between GRADE adherence and rigor of development scores. CONCLUSION: The results of our project suggest that the use of GRADE in Australian guidelines is increasing, however, strategies to increase uptake and reporting within the guideline community need to be explored.

Methodological quality of case series studies: an introduction to the JBI critical appraisal tool.

INTRODUCTION: Systematic reviews provide a rigorous synthesis of the best available evidence regarding a certain question. Where high-quality evidence is lacking, systematic reviewers may choose to rely on case series studies to provide information in relation to their question. However, to date there has been limited guidance on how to incorporate case series studies within systematic reviews assessing the effectiveness of an intervention, particularly with reference to assessing the methodological quality or risk of bias of these studies. METHODS: An international working group was formed to review the methodological literature regarding case series as a form of evidence for inclusion in systematic reviews. The group then developed a critical appraisal tool based on the epidemiological literature relating to bias within these studies. This was then piloted, reviewed, and approved by JBI's international Scientific Committee. RESULTS: The JBI critical appraisal tool for case series studies includes 10 questions addressing the internal validity and risk of bias of case series designs, particularly confounding, selection, and information bias, in addition to the importance of clear reporting. CONCLUSION: In certain situations, case series designs may represent the best available evidence to inform clinical practice. The JBI critical appraisal tool for case series offers systematic reviewers an approved method to assess the methodological quality of these studies.

How are systematic reviews of prevalence conducted? A methodological study.

BACKGROUND: There is a notable lack of methodological and reporting guidance for systematic reviews of prevalence data. This information void has the potential to result in reviews that are inconsistent and inadequate to inform healthcare policy and decision making. The aim of this meta-epidemiological study is to describe the methodology of recently published prevalence systematic reviews. METHODS: We searched MEDLINE (via PubMed) from February 2017 to February 2018 for systematic reviews of prevalence studies. We included systematic reviews assessing the prevalence of any clinical condition using patients as the unit of measurement and we summarized data related to reporting and methodology of the reviews. RESULTS: A total of 235 systematic reviews of prevalence were analyzed. The median number of authors was 5 (interquartile range [IQR] 4-7), the median number of databases searched was 4 (3-6) and the median number of studies included in each review was 24 (IQR 15-41.5). Search strategies were presented for 68% of reviews. Forty five percent of reviews received external funding, and 24% did not provide funding information. Twenty three percent of included reviews had published or registered the systematic review protocol. Reporting guidelines were used in 72% of reviews. The quality of included studies was assessed in 80% of reviews. Nine reviews assessed the overall quality of evidence (4 using GRADE). Meta-analysis was conducted in 65% of reviews; 1% used Bayesian methods. Random effect meta-analysis was used in 94% of reviews; among them, 75% did not report the variance estimator used. Among the reviews with meta-analysis, 70% did not report how data was transformed; 59% percent conducted subgroup analysis, 38% conducted meta-regression and 2% estimated prediction interval; I2 was estimated in 95% of analysis. Publication bias was examined in 48%. The most common software used was STATA (55%). CONCLUSIONS: Our results indicate that there are significant inconsistencies regarding how these reviews are conducted. Many of these differences arose in the assessment of methodological quality and the formal synthesis of comparable data. This variability indicates the need for clearer reporting standards and consensus on methodological guidance for systematic reviews of prevalence data.