Involvement of information specialists and statisticians in systematic reviews.

BACKGROUND: Systematic reviews (SRs) are usually conducted by a highly specialized group of researchers. The routine involvement of methodological experts is a core methodological recommendation. The present commentary describes the qualifications required for information specialists and statisticians involved in SRs, as well as their tasks, the methodological challenges they face, and potential future areas of involvement. TASKS AND QUALIFICATIONS: Information specialists select the information sources, develop search strategies, conduct the searches, and report the results. Statisticians select the methods for evidence synthesis, assess the risk of bias, and interpret the results. The minimum requirements for their involvement in SRs are a suitable university degree (e.g., in statistics or librarian/information science or an equivalent degree), methodological and content expertise, and several years of experience. KEY ARGUMENTS: The complexity of conducting SRs has greatly increased due to a massive rise in the amount of available evidence and the number and complexity of SR methods, largely statistical and information retrieval methods. Additional challenges exist in the actual conduct of an SR, such as judging how complex the research question could become and what hurdles could arise during the course of the project. CONCLUSION: SRs are becoming more and more complex to conduct and information specialists and statisticians should routinely be involved right from the start of the SR. This increases the trustworthiness of SRs as the basis for reliable, unbiased and reproducible health policy, and clinical decision making.

PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews.

BACKGROUND: Literature searches underlie the foundations of systematic reviews and related review types. Yet, the literature searching component of systematic reviews and related review types is often poorly reported. Guidance for literature search reporting has been diverse and, in many cases, does not offer enough detail to authors who need more specific information about reporting search methods and information sources in a clear, reproducible way. This document presents the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) checklist, and explanation and elaboration. METHODS: The checklist was developed using a three-stage Delphi survey process, followed by a consensus conference and public review process. RESULTS: The final checklist includes sixteen reporting items, each of which is detailed with exemplar reporting and rationale. CONCLUSIONS: The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and, therefore, reproducible.

Information specialist collaboration in Europe: collaborative methods, processes, and infrastructure through EUnetHTA.

The history of European health technology assessment (HTA) goes back more than 30 years. Almost as old as HTA agencies themselves is the desire to achieve European collaboration. This gained further impetus with the establishment of the European Network of Health Technology Assessment (EUnetHTA) in 2006. In this context, the field of information management faced specific challenges. Although these services are an integral part of HTA and information specialists play a key role here, this field is often not adequately represented in the HTA agencies within EUnetHTA. Furthermore, the organization of HTA production, including the types of HTAs produced, as well as funding, varies considerably. In order to meet these different conditions, information specialists have created various products and defined processes. With the EUnetHTA guideline, a common methodological understanding for the production of rapid Relative Effectiveness Assessments now exists. Furthermore, the Standard Operating Procedures map the complex information retrieval processes within EUnetHTA in a hands-on manner. The newly established Information Specialist Network (ISN) will in future ensure that information specialists are involved in all EUnetHTA assessments and that the methods are applied consistently in all assessments. In addition, the steering committee of the ISN manages enquiries and can be contacted to discuss methodological issues. Major barriers such as heterogeneity in the daily work of the EUnetHTA members can only be overcome through more collaboration and training.

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review.

BACKGROUND: Stringent requirements exist regarding the transparency of the study selection process and the reliability of results. A 2-step selection process is generally recommended; this is conducted by 2 reviewers independently of each other (conventional double-screening). However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget. The aim of the following methodological systematic review was to analyse the evidence available on whether single screening is equivalent to double screening in the screening process conducted in systematic reviews. METHODS: We searched Medline, PubMed and the Cochrane Methodology Register (last search 10/2018). We also used supplementary search techniques and sources ("similar articles" function in PubMed, conference abstracts and reference lists). We included all evaluations comparing single with double screening. Data were summarized in a structured, narrative way. RESULTS: The 4 evaluations included investigated a total of 23 single screenings (12 sets for screening involving 9 reviewers). The median proportion of missed studies was 5% (range 0 to 58%). The median proportion of missed studies was 3% for the 6 experienced reviewers (range: 0 to 21%) and 13% for the 3 reviewers with less experience (range: 0 to 58%). The impact of missing studies on the findings of meta-analyses had been reported in 2 evaluations for 7 single screenings including a total of 18,148 references. In 3 of these 7 single screenings - all conducted by the same reviewer (with less experience) - the findings would have changed substantially. The remaining 4 of these 7 screenings were conducted by experienced reviewers and the missing studies had no impact or a negligible on the findings of the meta-analyses. CONCLUSIONS: Single screening of the titles and abstracts of studies retrieved in bibliographic searches is not equivalent to double screening, as substantially more studies are missed. However, in our opinion such an approach could still represent an appropriate methodological shortcut in rapid reviews, as long as it is conducted by an experienced reviewer. Further research on single screening is required, for instance, regarding factors influencing the number of studies missed.

Study filters for non-randomized studies of interventions consistently lacked sensitivity upon external validation.

BACKGROUND: Little evidence is available on searches for non-randomized studies (NRS) in bibliographic databases within the framework of systematic reviews. For instance, it is currently unclear whether, when searching for NRS, effective restriction of the search strategy to certain study types is possible. The following challenges need to be considered: 1) For non-randomized controlled trials (NRCTs): whether they can be identified by established filters for randomized controlled trials (RCTs). 2) For other NRS types (such as cohort studies): whether study filters exist for each study type and, if so, which performance measures they have. The aims of the present analysis were to identify and validate existing NRS filters in MEDLINE as well as to evaluate established RCT filters using a set of MEDLINE citations. METHODS: Our analysis is a retrospective analysis of study filters based on MEDLINE citations of NRS from Cochrane reviews. In a first step we identified existing NRS filters. For the generation of the reference set, we screened Cochrane reviews evaluating NRS, which covered a broad range of study types. The citations of the studies included in the Cochrane reviews were identified via the reviews' bibliographies and the corresponding PubMed identification numbers (PMIDs) were extracted from PubMed. Random samples comprising up to 200 citations (i.e. 200 PMIDs) each were created for each study type to generate the test sets. RESULTS: A total of 271 Cochrane reviews from 41 different Cochrane groups were eligible for data extraction. We identified 14 NRS filters published since 2001. The study filters generated between 660,000 and 9.5 million hits in MEDLINE. Most filters covered several study types. The reference set included 2890 publications classified as NRS for the generation of the test sets. Twelve test sets were generated (one for each study type), of which 8 included 200 citations each. None of the study filters achieved sufficient sensitivity (≥ 92%) for all of the study types targeted. CONCLUSIONS: The performance of current NRS filters is insufficient for effective use in daily practice. It is therefore necessary to develop new strategies (e.g. new NRS filters in combination with other search techniques). The challenges related to NRS should be taken into account.

World Allergy Organization (WAO) Diagnosis and Rationale for Action against Cow's Milk Allergy (DRACMA) guideline update - XII - Recommendations on milk formula supplements with and without probiotics for infants and toddlers with CMA.

Background: Cow's milk allergy (CMA) is the most common food allergy in infants. The replacement with specialized formulas is an established clinical approach to ensure adequate growth and minimize the risk of severe allergic reactions when breastfeeding is not possible. Still, given the availability of multiple options, such as extensively hydrolyzed cow's milk protein formula (eHF-CM), amino acid formula (AAF), hydrolyzed rice formula (HRF) and soy formulas (SF), there is some uncertainty as to the most suitable choice with respect to health outcomes. Furthermore, the addition of probiotics to a formula has been proposed as a potential approach to maximize benefit. Objective: These evidence-based guidelines from the World Allergy Organization (WAO) intend to support patients, clinicians, and others in decisions about the use of milk specialized formulas, with and without probiotics, for individuals with CMA. Methods: WAO formed a multidisciplinary guideline panel balanced to include the views of all stakeholders and to minimize potential biases from competing interests. The McMaster University GRADE Centre supported the guideline-development process, including updating or performing systematic evidence reviews. The panel prioritized clinical questions and outcomes according to their importance for clinicians and patients. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach was used, including GRADE Evidence-to-Decision frameworks, which were subject to review by stakeholders. Results: After reviewing the summarized evidence and thoroughly discussing the different management options, the WAO guideline panel suggests: a) using an extensively hydrolyzed (cow's milk) formula or a hydrolyzed rice formula as the first option for managing infants with immunoglobulin E (IgE) and non-IgE-mediated CMA who are not being breastfed. An amino-acid formula or a soy formula could be regarded as second and third options respectively; b) using either a formula without a probiotic or a casein-based extensively hydrolyzed formula containing Lacticaseibacillus rhamnosus GG (LGG) for infants with either IgE or non-IgE-mediated CMA.The issued recommendations are labeled as "conditional" following the GRADE approach due to the very low certainty about the health effects based on the available evidence. Conclusions: If breastfeeding is not available, clinicians, patients, and their family members might want to discuss all the potential desirable and undesirable consequences of each formula in infants with CMA, integrating them with the patients' and caregivers' values and preferences, local availability, and cost, before deciding on a treatment option. We also suggest what research is needed to determine with greater certainty which formulas are likely to be the most beneficial, cost-effective, and equitable.

Increasing the efficiency of study selection for systematic reviews using prioritization tools and a single-screening approach.

BACKGROUND: Systematic literature screening is a key component in systematic reviews. However, this approach is resource intensive as generally two persons independently of each other (double screening) screen a vast number of search results. To develop approaches for increasing efficiency, we tested the use of text mining to prioritize search results as well as the involvement of only one person (single screening) in the study selection process. METHOD: Our study is based on health technology assessments (HTAs) of drug and non-drug interventions. Using a sample size calculation, we consecutively included 11 searches resulting in 33 study selection processes. Of the three screeners for each search, two used screening tools with prioritization (Rayyan, EPPI Reviewer) and one a tool without prioritization. For each prioritization tool, we investigated the proportion of citations classified as relevant at three cut-offs or STOP criteria (after screening 25%, 50% and 75% of the citation set). For each STOP criterion, we measured sensitivity (number of correctly identified relevant studies divided by the total number of relevant studies in the study pool). In addition, we determined the number of relevant studies identified per single screening round and investigated whether missed studies were relevant to the HTA conclusion. RESULTS: Overall, EPPI Reviewer performed better than Rayyan and identified the vast majority (88%, Rayyan 66%) of relevant citations after screening half of the citation set. As long as additional information sources were screened, it was sufficient to apply a single-screening approach to identify all studies relevant to the HTA conclusion. Although many relevant publications (n = 63) and studies (n = 29) were incorrectly excluded, ultimately only 5 studies could not be identified at all in 2 of the 11 searches (1x 1 study, 1x 4 studies). However, their omission did not change the overall conclusion in any HTA. CONCLUSIONS: EPPI Reviewer helped to identify relevant citations earlier in the screening process than Rayyan. Single screening would have been sufficient to identify all studies relevant to the HTA conclusion. However, this requires screening of further information sources. It also needs to be considered that the credibility of an HTA may be questioned if studies are missing, even if they are not relevant to the HTA conclusion.

Rapid reviews methods series: Guidance on literature search.

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and planning, information sources and search methods, search strategy development, quality assurance, reporting, and record management. Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches and (2) reducing the size of the search result. Because screening search results is usually more resource-intensive than conducting the search, we suggest investing time upfront in planning and optimising the search to save time by reducing the literature screening workload. To achieve this goal, RR teams should work with an information specialist. They should select a small number of relevant information sources (eg, databases) and use search methods that are highly likely to identify relevant literature for their topic. Database search strategies should aim to optimise both precision and sensitivity, and quality assurance measures (peer review and validation of search strategies) should be applied to minimise errors.

World Allergy Organization (WAO) Diagnosis and Rationale for Action against Cow's Milk Allergy (DRACMA) guideline update - XIII - Oral immunotherapy for CMA - Systematic review.

Background: Allergy to cow's milk is the most common food allergy in infants and it is usually outgrown by 5 years of age. In some individuals it persists beyond early childhood. Oral immunotherapy (OIT, oral desensitization, specific oral tolerance induction) has been proposed as a promising therapeutic strategy for persistent IgE-mediated cow's milk allergy. We previously published the systematic review of OIT for cow's milk allergy (CMA) in 2010 as part of the World Allergy Organization (WAO) Diagnosis and Rationale for Action against Cow's Milk Allergy (DRACMA) Guidelines. Objective: To systematically synthesize the currently available evidence about OIT for IgE-mediated CMA and to inform the updated 2022 WAO guidelines. Methods: We searched the electronic databases including PubMed, Medline, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), and the websites of selected allergy organizations. We included all studies irrespective of the language of the original publication. The last search was conducted in February 2021. We registered the protocol on Open Science Framework (10.17605/OSF.IO/AH2DT). Results: We identified 2147 unique records published between 2010 and 2021, including 13 randomized trials and 109 observational studies addressing cow's milk OIT. We found low-certainty evidence that OIT with unheated cow's milk, compared to elimination diet alone, increased the likelihood of being able to consume ≥150 ml of cow's milk in controlled settings (risk ratio (RR): 12.3, 95% CI: 5.9 to 26.0; risk difference (RD): 25 more per 100, 95% CI 11 to 56) as well as accidently ingest a small amount (≥5 ml) of cow's milk (RR: 8.7, 95% CI: 4.7 to 16.1; RD: 25 more per 100, 95% CI 12 to 50). However, 2-8 weeks after discontinuation of a successful OIT, tolerance of cow's milk persisted in only 36% (range: 20%-91%) of patients. OIT increased the frequency of anaphylaxis (rate ratio: 60.0, 95% CI 15 to 244; rate difference 5 more anaphylactic reactions per 1 person per year, 95% CI: 4 to 6; moderate evidence) and the frequency of epinephrine use (rate ratio: 35.2, 95% CI: 9 to 136.5; rate difference 268 more events per 100 person-years, 95% CI: 203 to 333; high certainty). OIT also increased the risk of gastrointestinal symptoms (RR 6.9, 95% CI 1.6-30.9; RD 28 more per 100, CI 3 to 100) and respiratory symptoms (RR 49.0, 95% CI 3.12-770.6; RD 77 more per 100, CI 62 to 92), compared with avoidance diet alone. Single-arm observational studies showed that on average 6.9% of OIT patients (95% CI: 3.8%-10%) developed eosinophilic esophagitis (very low certainty evidence). We found 1 trial and 2 small case series of OIT with baked milk. Conclusions: Moderate certainty evidence shows that OIT with unheated cow's milk in patients with IgE-mediated CMA is associated with an increased probability of being able to drink milk and, at the same time, an increased risk of serious adverse effects.

Analytical frameworks in colorectal cancer guidelines: development of methods for systematic reviews, their application and practical guidance for their use.

OBJECTIVES: Analytical frameworks are graphical representation of the key questions answered by a systematic review and can support the development of guideline recommendations. Our objectives were to a) conduct a systematic review to identify, describe and compare all analytical frameworks published as part of a systematic and guideline development process related to colorectal cancer (CRC), and b) to use this case study to develop guidance on how to conduct systematic reviews of analytical frameworks. METHODS: We developed a search strategy to identify eligible studies in Medline and Embase from 1996 until December 2020. We also manually searched guideline databases and websites to identify all guidelines and systematic reviews in CRC that used an analytical framework. We assessed the quality of the guidelines using the Appraisal of Guidelines for Research and Evaluation II tool. The systematic review was registered in International Prospective Register of Systematic Reviews, registration CRD42020172117. RESULTS: We screened 34,505 records and identified 1,166 guidelines and 3,127 systematic reviews on CRC of which five met our inclusion criteria. These five publications included four analytical frameworks in colorectal cancer (one update). We also describe our methodological approach to systematic reviews for analytical frameworks and underlying concepts for developing analytical framework using a bottom-up or top-down approach. CONCLUSION: Few guidelines and systematic reviews are utilizing analytical frameworks in the development of recommendations. Development of analytical frameworks should begin with a systematic search for existing analytical frameworks and follow a structured conceptual approach for their development to support guideline recommendations. Our methods may be helpful in achieving these objectives.

World Allergy Organization (WAO) Diagnosis and Rationale for Action against Cow's Milk Allergy (DRACMA) Guideline update - XIV - Recommendations on CMA immunotherapy.

Background: The prevalence of cow's milk allergy (CMA) is approximately 2-4.5% in infants and less than 0.5% in adults. Most children outgrow cow's milk allergy in early childhood, particularly that to the baked milk products. Immunotherapy with unheated cow's milk has been used as a treatment option for those who have not yet outgrown CMA, but the benefits must be balanced with the adverse effects. Objective: These evidence-based guidelines from the World Allergy Organization (WAO) intend to support patients, clinicians, and others in decisions about the use of oral and epicutaneous immunotherapy for the treatment of IgE-mediated CMA. Methods: WAO formed a multidisciplinary guideline panel balanced to include the views of all stakeholders and to minimize potential biases from competing interests. The McMaster University GRADE Centre supported the guideline-development process, including updating or performing systematic evidence reviews. The panel prioritized clinical questions and outcomes according to their importance for clinicians and patients. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach was used, including GRADE Evidence-to-Decision frameworks, which were subject to public comment. Results: After a careful review of the summarized evidence and thorough discussions the WAO guideline panel suggests: a) using oral immunotherapy with unheated cow's milk in those individuals with confirmed IgE-mediated CMA who value the ability to consume controlled quantities of milk more than avoiding the large adverse effects of therapy, b) not using oral immunotherapy with unheated cow's milk in those who value avoiding large adverse effects of therapy more than the ability to consume controlled quantities of milk, c) using omalizumab in those starting oral immunotherapy with unheated cow's milk, d) not using oral immunotherapy with baked cow's milk in those who do not tolerate both unheated and baked milk, and e) not using epicutaneous immunotherapy outside of a research setting. The recommendations are labeled "conditional" due to the low certainty about the health effects based on the available evidence. Conclusions: Clinicians, patients, and their family members might want to discuss all the potential desirable and undesirable effects of oral immunotherapy for IgE-mediated CMA and integrate them with the patients' values and preferences before deciding on a treatment option. More robust research is needed to determine with greater certainty which interventions are likely to be the most beneficial with the least harms, and to develop safer, low-cost, and equitable treatments.

PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews.

BACKGROUND: Literature searches underlie the foundations of systematic reviews and related review types. Yet, the literature searching component of systematic reviews and related review types is often poorly reported. Guidance for literature search reporting has been diverse, and, in many cases, does not offer enough detail to authors who need more specific information about reporting search methods and information sources in a clear, reproducible way. This document presents the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) checklist, and explanation and elaboration. METHODS: The checklist was developed using a 3-stage Delphi survey process, followed by a consensus conference and public review process. RESULTS: The final checklist includes 16 reporting items, each of which is detailed with exemplar reporting and rationale. CONCLUSIONS: The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and therefore reproducible.

An evaluation of searching the German CCMed database for the production of systematic reviews.

OBJECTIVES: National databases may be useful sources in the production of a systematic review (SR). The aim of this study was to assess the potential benefit of a systematic search in the German database 'Current Contents Medizin' (CCMed). METHODS: The study was conducted on the basis of published SRs that included CCMed as a literature source. Eligible SRs were identified through a systematic search in medline, embase and The Cochrane Library. The websites of German Health Technology Assessment agencies were also screened. Citations of primary studies included as relevant in the SRs were extracted and then categorised. RESULTS: The search yielded 52 eligible SRs. A total of 1505 relevant citations were extracted. Seventy-seven of these articles were published in journals indexed in CCMed. Thirty-two of the 77 citations were indexed in CCMed, but only eight of the 32 were unique. Of these eight citations, seven were not identified by a systematic search, but by handsearching. Only one unique citation, an observational study, was identified in CCMed by a systematic search. CONCLUSIONS: In the production of SRs, a systematic search in CCMed identifies relevant studies only in exceptional cases. Therefore, the routine inclusion of this database in systematic searches does not appear meaningful.

Development and validation of study filters for identifying controlled non-randomized studies in PubMed and Ovid MEDLINE.

A retrospective analysis published by the German Institute for Quality and Efficiency in Health Care (IQWiG) in 2018 concluded that no filter for non-randomized studies (NRS) achieved sufficient sensitivity (≥92%), a precondition for comprehensive information retrieval. New NRS filters are therefore required, taking into account the challenges related to this study type. Our evaluation focused on the development of study filters for NRS with a control group ("controlled NRS"), as this study type allows the calculation of an effect size. In addition, we assumed that due to the more explicit search syntax, controlled NRS are easier to identify than non-controlled ones, potentially resulting in better performance measures of study filters for controlled NRS. Our aim was to develop study filters for identifying controlled NRS in PubMed and Ovid MEDLINE. We developed two new search filters that can assist clinicians and researchers in identifying controlled NRS in PubMed and Ovid MEDLINE. The reference set was based on 2110 publications in Medline extracted from 271 Cochrane reviews and on 4333 irrelevant references. The first filter maximizes sensitivity (92.42%; specificity 79.67%, precision 68.49%) and should be used when a comprehensive search is needed. The second filter maximizes specificity (92.06%; precision 82.98%, sensitivity 80.94%) and should be used when a more focused search is sufficient.

Combining abbreviated literature searches with single-reviewer screening: three case studies of rapid reviews.

BACKGROUND: Decision-makers increasingly request rapid answers to clinical or public health questions. To save time, personnel, and financial resources, rapid reviews streamline the methodological steps of the systematic review process. We aimed to explore the validity of a rapid review approach that combines a substantially abbreviated literature search with a single-reviewer screening of abstracts and full texts using three case studies. METHODS: We used a convenience sample of three ongoing Cochrane reviews as reference standards. Two reviews addressed oncological topics and one addressed a public health topic. For each of the three topics, three reviewers screened the literature independently. Our primary outcome was the change in conclusions between the rapid reviews and the respective Cochrane reviews. In case the rapid approach missed studies, we recalculated the meta-analyses for the main outcomes and asked Cochrane review authors if the new body of evidence would change their original conclusion compared with the reference standards. Additionally, we assessed the sensitivity of the rapid review approach compared with the results of the original Cochrane reviews. RESULTS: For the two oncological topics (case studies 1 and 2), the three rapid reviews each yielded the same conclusions as the Cochrane reviews. However, the authors would have had less certainty about their conclusion in case study 2. For case study 3, the public health topic, only one of the three rapid reviews led to the same conclusion as the Cochrane review. The other two rapid reviews provided insufficient information for the authors to draw conclusions. Using the rapid review approach, the sensitivity was 100% (3 of 3) for case study 1. For case study 2, the three rapid reviews identified 40% (4 of 10), 50% (5 of 10), and 60% (6 of 10) of the included studies, respectively; for case study 3, the respective numbers were 38% (8 of 21), 43% (9 of 21), and 48% (10 of 21). CONCLUSIONS: Within the limitations of these case studies, a rapid review approach that combines abbreviated literature searches with single-reviewer screening may be feasible for focused clinical questions. For complex public health topics, sensitivity seems to be insufficient.

Effective study selection using text mining or a single-screening approach: a study protocol.

BACKGROUND: Systematic information retrieval generally requires a two-step selection process for studies, which is conducted by two persons independently of one another (double-screening approach). To increase efficiency, two methods seem promising, which will be tested in the planned study: the use of text mining to prioritize search results as well as the involvement of only one person in the study selection process (single-screening approach). The aim of the present study is to examine the following questions related to the process of study selection: Can the use of the Rayyan or EPPI Reviewer tools to prioritize the results of study selection increase efficiency? How accurately does a single-screening approach identify relevant studies? Which advantages or disadvantages (e.g., shortened screening time or increase in the number of full texts ordered) does a single-screening versus a double-screening approach have? METHODS: Our study is a prospective analysis of study selection processes based on benefit assessments of drug and non-drug interventions. It consists of two parts: firstly, the evaluation of a single-screening approach based on a sample size calculation (11 study selection processes, including 33 single screenings) and involving different screening tools and, secondly, the evaluation of the conventional double-screening approach based on five conventional study selection processes. In addition, the advantages and disadvantages of the single-screening versus the double-screening approach with regard to the outcomes "number of full texts ordered" and "time required for study selection" are analyzed. The previous work experience of the screeners is considered as a potential effect modifier. DISCUSSION: No study comparing the features of prioritization tools is currently available. Our study can thus contribute to filling this evidence gap. This study is also the first to investigate a range of questions surrounding the screening process and to include an a priori sample size calculation, thus enabling statistical conclusions. In addition, the impact of missing studies on the conclusion of a benefit assessment is calculated. SYSTEMATIC REVIEW REGISTRATION: Not applicable.

Trial registry searches for randomized controlled trials of new drugs required registry-specific adaptation to achieve adequate sensitivity.

OBJECTIVES: To analyze the availability of randomized controlled trials (RCTs) of new drugs in trial registries and to develop and test different search strategies in ClinicalTrials.gov (CT.gov), the EU Clinical Trials Register (EU-CTR), and the International Clinical Trials Registry Platform (ICTRP). STUDY DESIGN AND SETTING: Information from dossiers submitted by pharmaceutical companies was analyzed regarding the registration of the included RCTs in CT.gov, EU-CTR and ICTRP; different search strategies were developed and tested to determine performance. RESULTS: A total of 192 (95%) of 203 RCTs on newly approved drugs were registered in CT.gov; the 11 nonregistered trials were completed before 2005 or represented non-RCTs. Simple searches for RCTs on 18 new drugs using the generic drug name yielded a sensitivity of 94% in CT.gov (EU-CTR: 71%; ICTRP: 60%). The main reason for study nondetection was the sole use of the drug code in the registry entries. Simple searches for RCTs on 13 conditions using reasonably inferred search terms yielded a sensitivity of 100% in CT.gov. CONCLUSION: Almost all relevant RCTs on newly approved drugs will probably be identified in CT.gov alone. A sensitive search in CT.gov can be conducted using single search terms. The searches in ICTRP and EU-CTR should include several search terms (e.g., derived via text analysis).

No suitable precise or optimized epidemiologic search filters were available for bibliographic databases.

OBJECTIVES: To determine a suitable approach to a systematic search for epidemiologic publications in bibliographic databases. For this purpose, suitable sensitive, precise, and optimized filters were to be selected for MEDLINE searches. In addition, the relevance of bibliographic databases was determined. STUDY DESIGN AND SETTING: Epidemiologic systematic reviews (SRs) retrieved in a systematic search and company dossiers were screened to identify epidemiologic publications (primary studies and SRs) published since 2007. These publications were used to generate a test and validation set. Furthermore, each SR's search strategy was reviewed, and epidemiologic filters were extracted. The search syntaxes were validated using the relative recall method. RESULTS: The test set comprises 729 relevant epidemiologic publications, of which 566 were MEDLINE-indexed. About 27 epidemiologic filters were extracted. One suitable sensitive filter was identified (Larney et al. 2013: 95.94% sensitivity). Precision was presumably underestimated so that no precise or optimized filters can be recommended. About 77.64% of the publications were found in MEDLINE. CONCLUSION: There is currently no suitable approach to conducting efficient systematic searches for epidemiologic publications in bibliographic databases. The filter by Larney et al. (2013) can be used for sensitive MEDLINE searches. No robust conclusions can be drawn on precise or optimized filters. Additional search approaches should be considered.