Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement.

OBJECTIVE: To develop a reporting guideline for overviews of reviews of healthcare interventions. DESIGN: Development of the preferred reporting items for overviews of reviews (PRIOR) statement. PARTICIPANTS: Core team (seven individuals) led day-to-day operations, and an expert advisory group (three individuals) provided methodological advice. A panel of 100 experts (authors, editors, readers including members of the public or patients) was invited to participate in a modified Delphi exercise. 11 expert panellists (chosen on the basis of expertise, and representing relevant stakeholder groups) were invited to take part in a virtual face-to-face meeting to reach agreement (≥70%) on final checklist items. 21 authors of recently published overviews were invited to pilot test the checklist. SETTING: International consensus. INTERVENTION: Four stage process established by the EQUATOR Network for developing reporting guidelines in health research: project launch (establish a core team and expert advisory group, register intent), evidence reviews (systematic review of published overviews to describe reporting quality, scoping review of methodological guidance and author reported challenges related to undertaking overviews of reviews), modified Delphi exercise (two online Delphi surveys to reach agreement (≥70%) on relevant reporting items followed by a virtual face-to-face meeting), and development of the reporting guideline. RESULTS: From the evidence reviews, we drafted an initial list of 47 potentially relevant reporting items. An international group of 52 experts participated in the first Delphi survey (52% participation rate); agreement was reached for inclusion of 43 (91%) items. 44 experts (85% retention rate) completed the second Delphi survey, which included the four items lacking agreement from the first survey and five new items based on respondent comments. During the second round, agreement was not reached for the inclusion or exclusion of the nine remaining items. 19 individuals (6 core team and 3 expert advisory group members, and 10 expert panellists) attended the virtual face-to-face meeting. Among the nine items discussed, high agreement was reached for the inclusion of three and exclusion of six. Six authors participated in pilot testing, resulting in minor wording changes. The final checklist includes 27 main items (with 19 sub-items) across all stages of an overview of reviews. CONCLUSIONS: PRIOR fills an important gap in reporting guidance for overviews of reviews of healthcare interventions. The checklist, along with rationale and example for each item, provides guidance for authors that will facilitate complete and transparent reporting. This will allow readers to assess the methods used in overviews of reviews of healthcare interventions and understand the trustworthiness and applicability of their findings.

Screening for Gestational Diabetes: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force.

Importance: Gestational diabetes is associated with several poor health outcomes. Objective: To update the 2012 review on screening for gestational diabetes to inform the US Preventive Services Task Force. Data Sources: MEDLINE, EMBASE, and CINAHL (2010 to May 2020), ClinicalTrials.gov, reference lists; surveillance through June 2021. Study Selection: English-language intervention studies for screening and treatment; observational studies on screening; prospective studies on screening test accuracy. Data Extraction and Synthesis: Dual review of titles/abstracts, full-text articles, and study quality. Single-reviewer data abstraction with verification. Random-effects meta-analysis or bivariate analysis (accuracy). Main Outcomes and Measures: Pregnancy, fetal/neonatal, and long-term health outcomes; harms of screening; accuracy. Results: A total of 76 studies were included (18 randomized clinical trials [RCTs] [n = 31 241], 2 nonrandomized intervention studies [n = 190], 56 observational studies [n = 261 678]). Direct evidence on benefits of screening vs no screening was limited to 4 observational studies with inconsistent findings and methodological limitations. Screening was not significantly associated with serious or long-term harm. In 5 RCTs (n = 25 772), 1-step (International Association of Diabetes and Pregnancy Study Group) vs 2-step (Carpenter and Coustan) screening was significantly associated with increased likelihood of gestational diabetes (11.5% vs 4.9%) but no improved health outcomes. At or after 24 weeks of gestation, oral glucose challenge tests with 140- and 135-mg/dL cutoffs had sensitivities of 82% and 93%, respectively, and specificities of 82% and 79%, respectively, against Carpenter and Coustan criteria, and a test with a 140-mg/dL cutoff had sensitivity of 85% and specificity of 81% against the National Diabetes Group Data criteria. Fasting plasma glucose tests with cutoffs of 85 and 90 mg/dL had sensitivities of 88% and 81% and specificities of 73% and 82%, respectively, against Carpenter and Coustan criteria. Based on 8 RCTs and 1 nonrandomized study (n = 3982), treatment was significantly associated with decreased risk of primary cesarean deliveries (relative risk [RR], 0.70 [95% CI, 0.54-0.91]; absolute risk difference [ARD], 5.3%), shoulder dystocia (RR, 0.42 [95% CI, 0.23-0.77]; ARD, 1.3%), macrosomia (RR, 0.53 [95% CI, 0.41-0.68]; ARD, 8.9%), large for gestational age (RR, 0.56 [95% CI, 0.47-0.66]; ARD, 8.4%), birth injuries (odds ratio, 0.33 [95% CI, 0.11-0.99]; ARD, 0.2%), and neonatal intensive care unit admissions (RR, 0.73 [95% CI, 0.53-0.99]; ARD, 2.0%). The association with reduction in preterm deliveries was not significant (RR, 0.75 [95% CI, 0.56-1.01]). Conclusions and Relevance: Direct evidence on screening vs no screening remains limited. One- vs 2-step screening was not significantly associated with improved health outcomes. At or after 24 weeks of gestation, treatment of gestational diabetes was significantly associated with improved health outcomes.

Effectiveness and safety of interventions to manage childhood overweight and obesity: An Overview of Cochrane systematic reviews.

BACKGROUND: Childhood overweight and obesity are associated with adverse physical, social, and psychological outcomes. OBJECTIVES: We conducted an overview of Cochrane systematic reviews on the effectiveness and risks of interventions to treat overweight and obesity in children and adolescents. METHODS: In June 2019, we searched the Cochrane Database of Systematic Reviews for eligible reviews. The primary outcomes were change in adiposity (body mass and body mass index [BMI] z-score) and adverse events. Two reviewers screened studies and one reviewer extracted and another verified data. Two reviewers assessed methodological quality and reached consensus. Data were synthesized narratively. RESULTS: We included seven Cochrane reviews published between 2011 and 2017 containing evidence from 167 randomized controlled trials with 21,050 participants. Lifestyle and behavioural interventions more effectively reduced weight compared with no intervention, usual care, or another behavioural treatment (three reviews, low-to-moderate certainty). Parent-child lifestyle and behavioural interventions more effectively reduced BMI z-score compared with no intervention (one review, low certainty). Decision support tools for healthcare providers more effectively limited increases in BMI z-score compared with usual care (one review, moderate certainty). Pharmacologic treatments combined with behavioural modification more effectively reduced adiposity compared with placebo or usual care (one review, low certainty), but the risk of adverse events was greater than non-pharmacologic therapy. Surgical interventions (e.g., LAP-BAND) combined with behavioural modification more effectively reduced adiposity compared with behavioural modification alone (one review, low certainty). Those who underwent surgery reported a higher number of adverse events compared with those treated with lifestyle modification. CONCLUSIONS: There is low-certainty evidence that lifestyle and behavioural interventions, pharmacologic interventions, and surgical interventions are effective in weight management for children with overweight and obesity. Safety data remain lacking across all intervention modalities. Future research should focus on implementation strategies. Further, a focus on overall well-being may be more beneficial than weight management specifically.

Creating efficiencies in the extraction of data from randomized trials: a prospective evaluation of a machine learning and text mining tool.

BACKGROUND: Machine learning tools that semi-automate data extraction may create efficiencies in systematic review production. We evaluated a machine learning and text mining tool's ability to (a) automatically extract data elements from randomized trials, and (b) save time compared with manual extraction and verification. METHODS: For 75 randomized trials, we manually extracted and verified data for 21 data elements. We uploaded the randomized trials to an online machine learning and text mining tool, and quantified performance by evaluating its ability to identify the reporting of data elements (reported or not reported), and the relevance of the extracted sentences, fragments, and overall solutions. For each randomized trial, we measured the time to complete manual extraction and verification, and to review and amend the data extracted by the tool. We calculated the median (interquartile range [IQR]) time for manual and semi-automated data extraction, and overall time savings. RESULTS: The tool identified the reporting (reported or not reported) of data elements with median (IQR) 91% (75% to 99%) accuracy. Among the top five sentences for each data element at least one sentence was relevant in a median (IQR) 88% (83% to 99%) of cases. Among a median (IQR) 90% (86% to 97%) of relevant sentences, pertinent fragments had been highlighted by the tool; exact matches were unreliable (median (IQR) 52% [33% to 73%]). A median 48% of solutions were fully correct, but performance varied greatly across data elements (IQR 21% to 71%). Using ExaCT to assist the first reviewer resulted in a modest time savings compared with manual extraction by a single reviewer (17.9 vs. 21.6 h total extraction time across 75 randomized trials). CONCLUSIONS: Using ExaCT to assist with data extraction resulted in modest gains in efficiency compared with manual extraction. The tool was reliable for identifying the reporting of most data elements. The tool's ability to identify at least one relevant sentence and highlight pertinent fragments was generally good, but changes to sentence selection and/or highlighting were often required. PROTOCOL: https://doi.org/10.7939/DVN/RQPJKS.

Health inequities related to vaccination: An evidence map of potentially influential factors and systematic review of interventions.

INTRODUCTION: The National Advisory Committee on Immunization (NACI) makes recommendations for vaccines in Canada. To inform considerations for equity when making recommendations, the NACI Secretariat developed a matrix of factors that may influence vaccine equity. To inform the matrix we mapped the evidence for P2ROGRESS And Other factors potentially associated with unequal levels of illness or death from vaccine-preventable diseases (VPDs) and systematically reviewed the evidence for interventions aimed at reducing inequities. METHODS: In October 2019 we searched Medline, Embase, and CINAHL. Two reviewers agreed on the included studies. Our primary outcomes were VPD-related hospitalizations and deaths. Secondary outcomes were differential vaccine access, and exposure, susceptibility, severity, and consequences of VPDs. Two reviewers appraised the certainty of evidence. We mapped the evidence for P2ROGRESS And Other factors and summarized the findings descriptively. We summarized the interventions narratively. RESULTS: We identified 413 studies reporting on P2ROGRESS And Other factors. The most commonly investigated factors included age (n = 374, 89%), pre-existing conditions (n = 179, 42%), and gender identity or sex (n = 144, 34%). We identified 2 trials investigating the effects of interventions. One (n = 1249) provided very low certainty evidence that staff vaccination policies may reduce hospitalizations and deaths from influenza among private care home residents. The other (n not reported) provided very low certainty evidence that universal vaccination by nurses in clinics may reduce hospitalizations for rotavirus gastroenteritis compared with vaccination by physicians or no intervention. CONCLUSIONS: There is a large body of studies reporting on hospitalizations and deaths from VPDs stratified by P2ROGRESS And Other factors. We found only two trials examining the effects of interventions on hospitalization for or mortality from VPDs. This review has been helpful to NACI and will be helpful to similar organizations aiming to systematically identify and target health inequities through the development of vaccine program recommendations.

LOCATE: a prospective evaluation of the value of Leveraging Ongoing Citation Acquisition Techniques for living Evidence syntheses.

BACKGROUND: Living systematic reviews (LSRs) can expedite evidence synthesis by incorporating new evidence in real time. However, the methods needed to identify new studies in a timely manner are not well established. OBJECTIVES: To explore the value of complementary search approaches in terms of search performance, impact on results and conclusions, screening workload, and feasibility compared to the reference standard. METHODS: We developed three complementary search approaches for a systematic review on treatments for bronchiolitis: Automated Full Search, PubMed Similar Articles, and Scopus Citing References. These were automated to retrieve results monthly; pairs of reviewers screened the records and commented on feasibility. After 1 year, we conducted a full update search (reference standard). For each complementary approach, we compared search performance (proportion missed, number needed to read [NNR]) and reviewer workload (number of records screened, time required) to the reference standard. We investigated the impact of the new trials on the effect estimate and certainty of evidence for the primary outcomes. We summarized comments about feasibility. RESULTS: Via the reference standard, reviewers screened 505 titles/abstracts, 24 full texts, and identified four new trials (NNR 127; 12.4 h). Of the complementary approaches, only the Automated Full Search located all four trials; these were located 6 to 12 months sooner than via the reference standard but did not alter the results nor certainty in the evidence. The Automated Full Search was the most resource-intensive approach (816 records screened; NNR 204; 17.1 h). The PubMed Similar Articles and Scopus Citing References approaches located far fewer records (452 and 244, respectively), thereby requiring less screening time (9.4 and 5.2 h); however, each approach located only one of the four new trials. Reviewers found it feasible and convenient to conduct monthly screening for searches of this yield (median 15-65 records/month). CONCLUSIONS: The Automated Full Search was the most resource-intensive approach, but also the only to locate all of the newly published trials. Although the monthly screening time for the PubMed Similar Articles and Scopus Citing Articles was far less, most relevant records were missed. These approaches were feasible to integrate into reviewer work processes. SYSTEMATIC REVIEW REGISTRATION: Open Science Framework. https://doi.org/10.17605/OSF.IO/6M28H .

P value and Bayesian analysis in randomized-controlled trials in child health research published over 10 years, 2007 to 2017: a methodological review protocol.

BACKGROUND: There is an unresolved debate about the reliability of the interpretation of P value. Some investigators have suggested that an alternative Bayesian method is preferred in conducting health research. As randomized-controlled trials (RCTs) are important in generating research evidence, we decided to investigate the extent, if any, the inferential statistical framework in published RCTs in child health research have changed over 10 years. We aim to examine the change in P value and Bayesian analysis in RCTs in child health research papers published from 2007 to 2017. METHODS: We will search the Cochrane Central Register of Controlled Trials (Wiley) to identify relevant citations. We will leverage a pre-existing sample of child health RCTs published in 2007 (n=300) used in our previous study of reporting quality of pediatric RCTs. Using the same strategy and study selection methods, we will identify a comparable random sample of child health RCTs published in 2017 (n=300). Eligible studies will include RCTs in health research among individuals aged 21 years and below. One reviewer will select studies for inclusion and extract the data and another reviewer will verify these. Disagreements will be resolved by a discussion between reviewers or by involving another reviewer. We will perform a descriptive analysis of 2007 and 2017 samples and analyze the results using both the frequentist and Bayesian methods. We will present specific characteristics of the clinical trials from 2007 and 2017 in tabular and graphical forms. We will report the difference in the proportion of P value and Bayesian analysis between 2007 and 2017 to assess the 10-year change. Clustering around P values of significance, if observed, will be reported. DISCUSSION: This review will present the difference in the proportion of trials that reported on P value and Bayesian analysis between 2007 and 2017 to assess the 10-year change. The implications for future clinical research will be discussed and this research work will be published in a peer-reviewed journal. This review has the potential to help inform the need for a change in the methodological approach from the null hypothesis significance test to Bayesian methods. SYSTEMATIC REVIEW REGISTRATION: Open Science Framework https://osf.io/aj2df.

Screening for the prevention and early detection of cervical cancer: protocol for systematic reviews to inform Canadian recommendations.

PURPOSE: To inform recommendations by the Canadian Task Force on Preventive Health Care on screening in primary care for the prevention and early detection of cervical cancer by systematically reviewing evidence of (a) effectiveness; (b) test accuracy; (c) individuals' values and preferences; and (d) strategies aimed at improving screening rates. METHODS: De novo reviews will be conducted to evaluate effectiveness and to assess values and preferences. For test accuracy and strategies to improve screening rates, we will integrate studies from existing systematic reviews with search updates to the present. Two Cochrane reviews will provide evidence of adverse pregnancy outcomes from the conservative management of cervical intraepithelial neoplasia. We will search Medline, Embase, and Cochrane Central (except for individuals' values and preferences, where Medline, Scopus, and EconLit will be searched) via peer-reviewed search strategies and the reference lists of included studies and reviews. We will search ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform for ongoing trials. Two reviewers will screen potentially eligible studies and agree on those to include. Data will be extracted by one reviewer with verification by another. Two reviewers will independently assess risk of bias and reach consensus. Where possible and suitable, we will pool studies via meta-analysis. We will compare accuracy data per outcome and per comparison using the Rutter and Gatsonis hierarchical summary receiver operating characteristic model and report relative sensitivities and specificities. Findings on values and preferences will be synthesized using a narrative synthesis approach and thematic analysis, depending on study designs. Two reviewers will appraise the certainty of evidence for all outcomes using GRADE (Grading of Recommendations Assessment, Development and Evaluation) and come to consensus. DISCUSSION: The publication of guidance on screening in primary care for the prevention and early detection of cervical cancer by the Task Force in 2013 focused on cytology. Since 2013, new studies using human papillomavirus tests for cervical screening have been published that will improve our understanding of screening in primary care settings. This review will inform updated recommendations based on currently available studies and address key evidence gaps noted in our previous review.

A systematic review of factors that influence the acceptability of vaccines among Canadians.

BACKGROUND: Canada's National Advisory Committee on Immunization (NACI) provides guidance on the use of vaccines in Canada. To support the expansion of its mandate to include considerations for vaccine acceptability when making recommendations, the NACI Secretariat developed a matrix of factors that influence acceptability. To inform and validate the matrix, we systematically reviewed evidence for factors that influence vaccine acceptability, and for interventions aimed at improving acceptability. METHODS: On 10-11 October 2018 we searched four bibliographic databases, the Theses Canada Portal, and ClinicalTrials.gov. Two reviewers agreed on the included studies. From each study, we extracted information about the participants, intervention or exposure, comparator, and relevant outcomes. Due to heterogeneity in the reported factors and acceptability indicators we synthesized the findings narratively. We appraised the certainty of evidence using GRADE. For each vaccine-preventable disease we populated a matrix of factors for which there was evidence of an influence on acceptability. RESULTS: One hundred studies (>1 million participants) contributed data relevant to the public, 16 (6191 participants) to healthcare providers, and three (84 participants) to policymakers. There were 43 intervention studies (~2 million participants). Across vaccines, we identified low certainty evidence for 70 factors relevant to the general population, 56 to high-risk groups, and 30 to healthcare providers. The perceived safety and importance of the vaccine, vaccination history, and receiving a recommendation from a healthcare provider were common influential factors. We found low certainty evidence that reminders for childhood vaccines and policies or delivery models for rotavirus vaccines could improve uptake and coverage. Evidence for other interventions was of very low certainty. CONCLUSIONS: The NACI vaccine acceptability matrix is useful for categorizing acceptability factors for the general public. Reminder systems may improve the uptake of childhood vaccines. Policies that make the rotavirus vaccine universally available and easily accessible may improve coverage. FUNDING: This systematic review was completed under contract to the Public Health Agency of Canada, Contract #4600001536.

Decoding semi-automated title-abstract screening: findings from a convenience sample of reviews.

BACKGROUND: We evaluated the benefits and risks of using the Abstrackr machine learning (ML) tool to semi-automate title-abstract screening and explored whether Abstrackr's predictions varied by review or study-level characteristics. METHODS: For a convenience sample of 16 reviews for which adequate data were available to address our objectives (11 systematic reviews and 5 rapid reviews), we screened a 200-record training set in Abstrackr and downloaded the relevance (relevant or irrelevant) of the remaining records, as predicted by the tool. We retrospectively simulated the liberal-accelerated screening approach. We estimated the time savings and proportion missed compared with dual independent screening. For reviews with pairwise meta-analyses, we evaluated changes to the pooled effects after removing the missed studies. We explored whether the tool's predictions varied by review and study-level characteristics. RESULTS: Using the ML-assisted liberal-accelerated approach, we wrongly excluded 0 to 3 (0 to 14%) records that were included in the final reports, but saved a median (IQR) 26 (9, 42) h of screening time. One missed study was included in eight pairwise meta-analyses in one systematic review. The pooled effect for just one of those meta-analyses changed considerably (from MD (95% CI) - 1.53 (- 2.92, - 0.15) to - 1.17 (- 2.70, 0.36)). Of 802 records in the final reports, 87% were correctly predicted as relevant. The correctness of the predictions did not differ by review (systematic or rapid, P = 0.37) or intervention type (simple or complex, P = 0.47). The predictions were more often correct in reviews with multiple (89%) vs. single (83%) research questions (P = 0.01), or that included only trials (95%) vs. multiple designs (86%) (P = 0.003). At the study level, trials (91%), mixed methods (100%), and qualitative (93%) studies were more often correctly predicted as relevant compared with observational studies (79%) or reviews (83%) (P = 0.0006). Studies at high or unclear (88%) vs. low risk of bias (80%) (P = 0.039), and those published more recently (mean (SD) 2008 (7) vs. 2006 (10), P = 0.02) were more often correctly predicted as relevant. CONCLUSION: Our screening approach saved time and may be suitable in conditions where the limited risk of missing relevant records is acceptable. Several of our findings are paradoxical and require further study to fully understand the tasks to which ML-assisted screening is best suited. The findings should be interpreted in light of the fact that the protocol was prepared for the funder, but not published a priori. Because we used a convenience sample, the findings may be prone to selection bias. The results may not be generalizable to other samples of reviews, ML tools, or screening approaches. The small number of missed studies across reviews with pairwise meta-analyses hindered strong conclusions about the effect of missed studies on the results and conclusions of systematic reviews.

Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review.

BACKGROUND: Overviews of reviews (overviews) provide an invaluable resource for healthcare decision-making by combining large volumes of systematic review (SR) data into a single synthesis. The production of high-quality overviews hinges on the availability of practical evidence-based guidance for conduct and reporting. OBJECTIVES: Within the broad purpose of informing the development of a reporting guideline for overviews, we aimed to provide an up-to-date map of existing guidance related to the conduct of overviews, and to identify common challenges that authors face when undertaking overviews. METHODS: We updated a scoping review published in 2016 using the search methods that had produced the highest yield: ongoing reference tracking (2014 to March 2020 in PubMed, Scopus, and Google Scholar), hand-searching conference proceedings and websites, and contacting authors of published overviews. Using a qualitative meta-summary approach, one reviewer extracted, organized, and summarized the guidance and challenges presented within the included documents. A second reviewer verified the data and synthesis. RESULTS: We located 28 new guidance documents, for a total of 77 documents produced by 34 research groups. The new guidance helps to resolve some earlier identified challenges in the production of overviews. Important developments include strengthened guidance on handling primary study overlap at the study selection and analysis stages. Despite marked progress, several areas continue to be hampered by inconsistent or lacking guidance. There is ongoing debate about whether, when, and how supplemental primary studies should be included in overviews. Guidance remains scant on how to extract and use appraisals of quality of the primary studies within the included SRs and how to adapt GRADE methodology to overviews. The challenges that overview authors face are often related to the above-described steps in the process where evidence-based guidance is lacking or conflicting. CONCLUSION: The rising popularity of overviews has been accompanied by a steady accumulation of new, and sometimes conflicting, guidance. While recent guidance has helped to address some of the challenges that overview authors face, areas of uncertainty remain. Practical tools supported by empirical evidence are needed to assist authors with the many methodological decision points that are encountered in the production of overviews.

The semi-automation of title and abstract screening: a retrospective exploration of ways to leverage Abstrackr's relevance predictions in systematic and rapid reviews.

BACKGROUND: We investigated the feasibility of using a machine learning tool's relevance predictions to expedite title and abstract screening. METHODS: We subjected 11 systematic reviews and six rapid reviews to four retrospective screening simulations (automated and semi-automated approaches to single-reviewer and dual independent screening) in Abstrackr, a freely-available machine learning software. We calculated the proportion missed, workload savings, and time savings compared to single-reviewer and dual independent screening by human reviewers. We performed cited reference searches to determine if missed studies would be identified via reference list scanning. RESULTS: For systematic reviews, the semi-automated, dual independent screening approach provided the best balance of time savings (median (range) 20 (3-82) hours) and reliability (median (range) proportion missed records, 1 (0-14)%). The cited references search identified 59% (n = 10/17) of the records missed. For the rapid reviews, the fully and semi-automated approaches saved time (median (range) 9 (2-18) hours and 3 (1-10) hours, respectively), but less so than for the systematic reviews. The median (range) proportion missed records for both approaches was 6 (0-22)%. CONCLUSION: Using Abstrackr to assist one of two reviewers in systematic reviews saves time with little risk of missing relevant records. Many missed records would be identified via other means.

Quality and risk of bias appraisals of systematic reviews are inconsistent across reviewers and centers.

OBJECTIVE: The objective of the study was to evaluate the inter-rater and intercenter reliability, usability, and utility of A MeaSurement Tool to Assess systematic Reviews (AMSTAR), AMSTAR 2, and Risk Of Bias In Systematic reviews (ROBIS). STUDY DESIGN AND SETTING: This is a prospective evaluation using 30 systematic reviews of randomized trials, undertaken at three international centers. RESULTS: Reviewers completed AMSTAR, AMSTAR 2, and ROBIS in median (interquartile range) 15.7 (11.3), 19.7 (12.1), and 28.7 (17.4) minutes and reached consensus in 2.6 (3.2), 4.6 (5.3), and 10.9 (10.8) minutes, respectively. Across all centers, inter-rater reliability was substantial to almost perfect for 8/11 AMSTAR, 9/16 AMSTAR 2, and 12/24 ROBIS items. Intercenter reliability was substantial to almost perfect for 6/11 AMSTAR, 12/16 AMSTAR 2, and 7/24 ROBIS items. Intercenter reliability for confidence in the results of the review or overall risk of bias was moderate (Gwet's first-order agreement coefficient (AC1) 0.58, 95% confidence intervals [CI]: 0.30 to 0.85) to substantial (AC1 0.74, 95% CI: 0.30 to 0.85) for AMSTAR 2 and poor (AC1 -0.21, 95% CI: -0.55 to 0.13) to moderate (AC1 0.56, 95% CI: 0.30 to 0.83) for ROBIS. It is not clear whether using the appraisals of any tool as an inclusion criterion would alter an overview's findings. CONCLUSIONS: Improved guidance may be needed to facilitate the consistent interpretation and application of the newer tools (especially ROBIS).

How Perceptions Impact Recovery from Concussion in Childhood and Adolescence: a Systematic Review.

Perceptions about the causes and consequences of concussion, and individual representations and interpretations of these factors, can influence the post-concussive recovery process. The goal of this project was to synthesize evidence on perceptions related to concussions as experienced by children, adolescents, and parents, and to evaluate how these perceptions impact post-concussive recovery in physical, behavioural, cognitive, and psychological domains. We undertook a systematic review based on the Cochrane Handbook, conducting a comprehensive search of six databases and Google Scholar. Duplicate, independent screening was employed and the quality of studies was assessed using the Mixed Methods Appraisal Tool (MMAT). A total of 1552 unique records were identified, and six records (5 scientific articles and 1 thesis, published between 1990 and 2018; N = 26 to 412, age range from 2 to 18 years) were included. Perceptions about concussions were assessed differently between studies, with a range in types of measures and respondents. Some evidence suggested that perceptions could negatively impact concussion recovery, mostly post-concussive symptoms. However, results were not consistent between studies and the methodological quality was variable (and often low). There is limited evidence of the impact of perceptions of children, adolescents, and their parents on concussion recovery. Priorities for future research investigating concussion recovery should include recruiting representative samples, accounting for potential confounders, and measuring perceptions in children, adolescents and parents using validated measures. Higher quality studies are needed to better understand the role of perceptions in concussion recovery and to inform clinical care.

Preferred Reporting Items for Overviews of Reviews (PRIOR): a protocol for development of a reporting guideline for overviews of reviews of healthcare interventions.

BACKGROUND: Overviews of reviews (i.e., overviews) compile information from multiple systematic reviews to provide a single synthesis of relevant evidence for healthcare decision-making. Despite their increasing popularity, there are currently no systematically developed reporting guidelines for overviews. This is problematic because the reporting of published overviews varies considerably and is often substandard. Our objective is to use explicit, systematic, and transparent methods to develop an evidence-based and agreement-based reporting guideline for overviews of reviews of healthcare interventions (PRIOR, Preferred Reporting Items for Overviews of Reviews). METHODS: We will develop the PRIOR reporting guideline in four stages, using established methods for developing reporting guidelines in health research. First, we will establish an international and multidisciplinary expert advisory board that will oversee the conduct of the project and provide methodological support. Second, we will use the results of comprehensive literature reviews to develop a list of prospective checklist items for the reporting guideline. Third, we will use a modified Delphi exercise to achieve a high level of expert agreement on the list of items to be included in the PRIOR reporting guideline. We will identify and recruit a group of up to 100 international experts who will provide input into the guideline in three Delphi rounds: the first two rounds will occur via online survey, and the third round will occur during a smaller (8 to 10 participants) in-person meeting that will use a nominal group technique. Fourth, we will produce and publish the PRIOR reporting guideline. DISCUSSION: A systematically developed reporting guideline for overviews could help to improve the accuracy, completeness, and transparency of overviews. This, in turn, could help maximize the value and impact of overviews by allowing more efficient interpretation and use of their research findings.

Performance and usability of machine learning for screening in systematic reviews: a comparative evaluation of three tools.

BACKGROUND: We explored the performance of three machine learning tools designed to facilitate title and abstract screening in systematic reviews (SRs) when used to (a) eliminate irrelevant records (automated simulation) and (b) complement the work of a single reviewer (semi-automated simulation). We evaluated user experiences for each tool. METHODS: We subjected three SRs to two retrospective screening simulations. In each tool (Abstrackr, DistillerSR, RobotAnalyst), we screened a 200-record training set and downloaded the predicted relevance of the remaining records. We calculated the proportion missed and workload and time savings compared to dual independent screening. To test user experiences, eight research staff tried each tool and completed a survey. RESULTS: Using Abstrackr, DistillerSR, and RobotAnalyst, respectively, the median (range) proportion missed was 5 (0 to 28) percent, 97 (96 to 100) percent, and 70 (23 to 100) percent for the automated simulation and 1 (0 to 2) percent, 2 (0 to 7) percent, and 2 (0 to 4) percent for the semi-automated simulation. The median (range) workload savings was 90 (82 to 93) percent, 99 (98 to 99) percent, and 85 (85 to 88) percent for the automated simulation and 40 (32 to 43) percent, 49 (48 to 49) percent, and 35 (34 to 38) percent for the semi-automated simulation. The median (range) time savings was 154 (91 to 183), 185 (95 to 201), and 157 (86 to 172) hours for the automated simulation and 61 (42 to 82), 92 (46 to 100), and 64 (37 to 71) hours for the semi-automated simulation. Abstrackr identified 33-90% of records missed by a single reviewer. RobotAnalyst performed less well and DistillerSR provided no relative advantage. User experiences depended on user friendliness, qualities of the user interface, features and functions, trustworthiness, ease and speed of obtaining predictions, and practicality of the export file(s). CONCLUSIONS: The workload savings afforded in the automated simulation came with increased risk of missing relevant records. Supplementing a single reviewer's decisions with relevance predictions (semi-automated simulation) sometimes reduced the proportion missed, but performance varied by tool and SR. Designing tools based on reviewers' self-identified preferences may improve their compatibility with present workflows. SYSTEMATIC REVIEW REGISTRATION: Not applicable.

Evidence available for patient-identified priorities in depression research: results of 11 rapid responses.

OBJECTIVES: Patient priority setting projects (PPSPs) can reduce research agenda bias. A key element of PPSPs is a review of available literature to determine if the proposed research priorities have been addressed, identify research gaps, recognise opportunities for knowledge translation (KT) and avoid duplication of research efforts. We conducted rapid responses for 11 patient-identified priorities in depression to provide a map of the existing evidence. DESIGN: Eleven rapid responses. DATA SOURCES: Single electronic database (PubMed). ELIGIBILITY CRITERIA: Each rapid response had unique eligibility criteria. For study designs, we used a stepwise inclusion process that started with systematic reviews (SRs) if available, then randomised controlled trials and observational studies as necessary. RESULTS: For all but one of the rapid responses we identified existing SRs (median 7 SRs per rapid response, range 0-179). There were questions where extensive evidence exists (ie, hundreds of primary studies), yet uncertainties remain. For example, there is evidence supporting the effectiveness of many non-pharmacological interventions (including psychological interventions and exercise) to reduce depressive symptoms. However, targeted research is needed that addresses comparative effectiveness of promising interventions, specific populations of interest (eg, children, minority groups) and adverse effects. CONCLUSIONS: We identified an extensive body of evidence addressing patient priorities in depression and mapped the results and limitations of existing evidence, areas of uncertainty and general directions for future research. This work can serve as a solid foundation to guide future research in depression and KT activities. Integrated knowledge syntheses bring value to the PPSP process; however, the role of knowledge synthesis in PPSPs and methodological approaches are not well defined at present.

Reporting of data monitoring committees and adverse events in paediatric trials: a descriptive analysis.

OBJECTIVES: For 300 paediatric trials, we evaluated the reporting of: a data monitoring committee (DMC); interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints. METHODS: For this cross-sectional evaluation, we randomly selected 300 paediatric trials published in 2012 from the Cochrane Central Register of Controlled Trials. We collected data on the reporting of a DMC; interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints. We reported the findings descriptively and stratified by trial characteristics. RESULTS: Eighty-five (28%) of the trials investigated drugs, and 18% (n=55/300) reported a DMC. The reporting of a DMC was more common among multicentre than single centre trials (n=41/132, 31% vs n=14/139, 10%, p<0.001) and industry-sponsored trials compared with those sponsored by other sources (n=16/50, 32% vs n=39/250, 16%, p=0.009). Trials that reported a DMC enrolled more participants than those that did not (median [range]): 224 (10-60480) vs 91 (10-9528) (p<0.001). Only 25% of these trials reported interim analyses, and 42% reported stopping rules. Less than half (n=143/300, 48%) of trials reported on adverse events, and 72% (n=215/300) reported on harm-related endpoints. Trials that reported a DMC compared with those that did not were more likely to report adverse events (n=43/55, 78% vs 100/245, 41%, p<0.001) and harm-related endpoints (n=52/55, 95% vs. 163/245, 67%, p<0.001). Only 32% of drug trials reported a DMC; 18% and 19% did not report on adverse events or harm-related endpoints, respectively. CONCLUSIONS: The reporting of a DMC was infrequent, even among drug trials. Few trials reported stopping rules or interim analyses. Reporting of adverse events and harm-related endpoints was suboptimal.

Glucocorticoids for Croup in Children.

Clinical Question: For children with croup, do glucocorticoids reduce clinical symptoms, minimize return visits and/or (re)admissions to the hospital, shorten lengths of stay, or reduce the need for additional treatments? Bottom Line: Glucocorticoids reduce symptoms of croup at 2 hours and for at least 24 hours, reduce return visits to care, and shorten hospital stays by about 15 hours. Serious adverse events are infrequent.