A method for translating search strategies efficiently into HMIC and SPP.

BACKGROUND: Small databases, such as Health Management Information Consortium (HMIC) and Social Policy and Practice (SPP), can add value to systematic searches. Search strategies designed for large databases may not be appropriate in small sources. A different approach to translating strategies could ensure that small databases are searched efficiently. OBJECTIVES: To establish the contribution HMIC and SPP made to public health guidelines (PHGs); and to recommend an efficient method of translating search strategies. METHODS: Eight PHGs were analysed to establish how many included publications were retrieved from HMIC and SPP. Six options for translating strategies from MEDLINE, using variations of free text and subject terms, were compared. RESULTS: Health Management Information Consortium contributed 15 and SPP eight of the 483 publications cited in the PHGs. The free-text only search was the one option to miss an included publication. The heading word (with truncation) option was more precise than applying subject headings. DISCUSSION: There is a risk of missing relevant publications in free-text only searches and it is preferable to include subject terms efficiently. CONCLUSION: The heading word (with truncation) option did not miss the evidence included in the PHGs and was the most efficient method for translating MEDLINE to HMIC and SPP.

Literature searching methods or guidance and their application to public health topics: A narrative review.

BACKGROUND: Information specialists conducting searches for systematic reviews need to consider key questions around which and how many sources to search. This is particularly important for public health topics where evidence may be found in diverse sources. OBJECTIVES: The objective of this review is to give an overview of recent studies on information retrieval guidance and methods that could be applied to public health evidence and used to guide future searches. METHODS: A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed. RESULTS: Seventy-two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature. DISCUSSION: Many researchers follow the recommendations to search multiple databases. The review topic influences decisions about sources. Additional sources covering grey literature eliminate bias but are time-consuming and difficult to search systematically. Public health searching is complex, often requiring searches in multidisciplinary sources and using additional methods. CONCLUSIONS: Search planning is advisable to enable decisions about which and how many sources to search. This could improve with more work on modelling search scenarios, particularly in public health topics, to examine where publications were found and guide future research.

The National Institute for Health and Care Excellence information specialist development pathway: Developing the skills, knowledge and confidence to quality assure search strategies.

Quality assurance (QA) is an important process in ensuring that systematic reviews and other evidence syntheses are supported by a high-quality search. This paper describes how the National Institute for Health and Care Excellence (NICE) in the UK established a development pathway to ensure its information specialists had the skills, knowledge and confidence to undertake search QA. The key component of the pathway is that it blends technical knowledge with interpersonal skills. The pathway develops technical skills in the early steps before using peer support activities to build confidence while undertaking a range of searches. QA is effective when the search lead communicates the contextual information that has influenced search development. QA is treated as a collaboration to get the right search for the review. The key requirements for search QA, alongside technical knowledge, are communication, collaboration and negotiation skills.

The NICE OECD countries' geographic search filters: Part 1-methodology for developing the draft MEDLINE and Embase (Ovid) filters.

OBJECTIVE: There are no existing validated search filters for the group of 37 Organisation for Economic Co-operation and Development (OECD) countries. This study describes how information specialists from the United Kingdom's National Institute for Health and Care Excellence (NICE) developed and evaluated novel OECD countries' geographic search filters for MEDLINE and Embase (Ovid) to improve literature search effectiveness for evidence about OECD countries. METHODS: We created the draft filters using an alternative approach to standard filter construction. They are composed entirely of geographic subject headings and are designed to retain OECD country evidence by excluding non-OECD country evidence using the NOT Boolean operator. To evaluate the draft filters' effectiveness, we used MEDLINE and Embase literature searches for three NICE guidelines that retrieved >5,000 search results. A 10% sample of the excluded references was screened to check that OECD country evidence was not inadvertently excluded. RESULTS: The draft MEDLINE filter reduced results for each NICE guideline by 9.5% to 12.9%. In Embase, search results were reduced by 10.7% to 14%. Of the sample references, 7 of 910 (0.8%) were excluded inadvertently. These references were from a guideline about looked-after minors that concerns both OECD and non-OECD countries. CONCLUSION: The draft filters look promising-they reduced search result volumes while retaining most OECD country evidence from MEDLINE and Embase. However, we advise caution when using them in topics about both non-OECD and OECD countries. We have created final versions of the search filters and will validate them in a future study.

The NICE OECD countries' geographic search filters: Part 2-validation of the MEDLINE and Embase (Ovid) filters.

OBJECTIVE: We previously developed draft MEDLINE and Embase (Ovid) geographic search filters for Organisation for Economic Co-operation and Development (OECD) countries to assess their feasibility for finding evidence about the countries. Here, we describe the validation of these search filters. METHODS: We identified OECD country references from thirty National Institute for Health and Care Excellence (NICE) guidelines to generate gold standard sets for MEDLINE (n=2,065) and Embase (n=2,023). We validated the filters by calculating their recall against these sets. We then applied the filters to existing search strategies for three OECD-focused NICE guideline reviews (NG103 on flu vaccination, NG140 on abortion care, and NG146 on workplace health) to calculate the filters' impact on the number needed to read (NNR) of the searches. RESULTS: The filters both achieved 99.95% recall against the gold standard sets. Both filters achieved 100% recall for the three NICE guideline reviews. The MEDLINE filter reduced NNR from 256 to 232 for the NG103 review, from 38 to 27 for the NG140 review, and from 631 to 591 for the NG146 review. The Embase filter reduced NNR from 373 to 341 for the NG103 review, from 101 to 76 for the NG140 review, and from 989 to 925 for the NG146 review. CONCLUSION: The NICE OECD countries' search filters are the first validated filters for the countries. They can save time for research topics about OECD countries by finding the majority of evidence about OECD countries while reducing search result volumes in comparison to no filter use.

The Embase UK filter: validation of a geographic search filter to retrieve research about the UK from OVID Embase.

BACKGROUND: The authors developed a validated geographic search filter to retrieve research about the United Kingdom (UK) from OVID Embase. It was created to be used alongside their previously published OVID MEDLINE UK filter in systematic literature searches for context-sensitive topics. OBJECTIVES: To develop a validated geographic search filter to retrieve research about the UK from OVID Embase. METHODS: The Embase UK filter was translated from the MEDLINE UK filter. A gold standard set of references was generated using the relative recall method. The set contained references to publications about the UK that had informed National Institute for Health and Care Excellence (NICE) guidance and it was used to validate the filter. Recall, precision and number-needed-to-read (NNR) were calculated using a case study. RESULTS: The validated Embase UK filter demonstrated 99.8% recall against the references with UK identifiers in the gold standard set. In the case study, the Embase UK filter demonstrated 98.5% recall, 7.6% precision and a NNR of 13. CONCLUSION: The Embase UK filter can be used alongside the MEDLINE UK filter. The filters have the potential to save time and associated resource costs when they are used for context-sensitive topics that require research about UK settings.

The medline UK filter: development and validation of a geographic search filter to retrieve research about the UK from OVID medline.

BACKGROUND: A validated geographic search filter for the retrieval of research about the United Kingdom (UK) from bibliographic databases had not previously been published. OBJECTIVES: To develop and validate a geographic search filter to retrieve research about the UK from OVID medline with high recall and precision. METHODS: Three gold standard sets of references were generated using the relative recall method. The sets contained references to studies about the UK which had informed National Institute for Health and Care Excellence (NICE) guidance. The first and second sets were used to develop and refine the medline UK filter. The third set was used to validate the filter. Recall, precision and number-needed-to-read (NNR) were calculated using a case study. RESULTS: The validated medline UK filter demonstrated 87.6% relative recall against the third gold standard set. In the case study, the medline UK filter demonstrated 100% recall, 11.4% precision and a NNR of nine. CONCLUSION: A validated geographic search filter to retrieve research about the UK with high recall and precision has been developed. The medline UK filter can be applied to systematic literature searches in OVID medline for topics with a UK focus.

Efficient searching for NICE public health guidelines: Would using fewer sources still find the evidence?

Systematic searches are integral to identifying the evidence that is used in National Institute for Health and Care Excellence (NICE) public health guidelines (PHGs). This study analyses the sources, including bibliographic databases and other techniques, required for PHGs. The aims were to analyse the sources used to identify the publications included in NICE PHGs; and to assess whether fewer sources could have been searched to retrieve these publications. Data showing how the included publications had been identified was collated using search summary tables. Three scenarios were created to test various combinations of sources to determine whether fewer sources could have been used. The sample included 29 evidence reviews, compiled using 13 searches, to support 10 PHG topics. Across the PHGs, 23 databases and six other techniques retrieved included publications. A mean reduction in total results of 6.5% could have been made if the minimum set of sources plus Cochrane Library, Embase, and MEDLINE were searched. On average, Cochrane Library, Embase, and MEDLINE contributed 76.8% of the included publications, with other databases adding 11% and other techniques 12.2%. None of the searches had a minimum set that was comprised entirely of databases. There was not a core set of sources for PHGs. A range of databases and techniques, covering a multi-disciplinary evidence base, was required to identify all included publications. It would be possible to reduce the number of sources searched and make some gains in productivity. It is important to create a tailored set of sources to do an efficient search.

The NICE UK geographic search filters for MEDLINE and Embase (Ovid): Post-development study to further evaluate precision and number-needed-to-read when retrieving UK evidence.

BACKGROUND: The National Institute for Health and Care Excellence's (NICE) United Kingdom (UK) geographic search filters for MEDLINE and Embase (OVID) retrieve evidence in literature searches for UK-focused research topics with high recall. Their precision and number-needed-to-read (NNR) was examined previously in case studies using a single review. This paper details a larger post-development study that was conducted to test the NICE UK filters' precision and NNR more extensively. METHODS: The filters' recall of included UK references from 100 reviews was calculated. As reproducible search strategies were not available for every review, the MEDLINE filter's precision and NNR were calculated using strategies from 25 reviews. Strategies from nine reviews were used for the Embase filter. RESULTS: The MEDLINE filter achieved an average of 96.4% recall for the included UK references from the 100 reviews and the Embase filter achieved an average of 97.4% recall. Compared to not using a filter, the MEDLINE filter achieved an average of 98.9% recall for the 25 reviews. Precision was increased by an average of 7.8 times, reducing the NNR from 357 to 46. The Embase filter achieved an average of 97.1% recall for the nine reviews. Precision was increased by an average of 5.1 times, reducing the NNR from 746 to 146. CONCLUSION: There is more evidence to demonstrate that the NICE UK filters retrieve the majority of UK evidence from MEDLINE and Embase while increasing precision and reducing NNR. The filters can save time spent on selecting evidence for UK-focused research topics.

Identifying evidence for public health guidance: a comparison of citation searching with Web of Science and Google Scholar.

AIM: To examine how effectively forwards citation searching with Web of Science (WOS) or Google Scholar (GS) identified evidence to support public health guidance published by the National Institute for Health and Care Excellence. METHOD: Forwards citation searching was performed using GS on a base set of 46 publications and replicated using WOS. OUTCOMES: WOS and GS were compared in terms of recall; precision; number needed to read (NNR); administrative time and costs; and screening time and costs. Outcomes for all publications were compared with those for a subset of highly important publications. RESULTS: The searches identified 43 relevant publications. The WOS process had 86.05% recall and 1.58% precision. The GS process had 90.7% recall and 1.62% precision. The NNR to identify one relevant publication was 63.3 with WOS and 61.72 with GS. There were nine highly important publications. WOS had 100% recall, 0.38% precision and NNR of 260.22. GS had 88.89% recall, 0.33% precision and NNR of 300.88. Administering the WOS results took 4 h and cost £88-£136, compared with 75 h and £1650-£2550 with GS. CONCLUSION: WOS is recommended over GS, as citation searching was more effective, while the administrative and screening times and costs were lower.

A population search filter for hard-to-reach populations increased search efficiency for a systematic review.

OBJECTIVES: This article discusses how hard-to-reach population groups were conceptualized into a search filter. The objectives of this article were to (1) discuss how the authors designed a multistranded population search filter and (2) retrospectively test the effectiveness of the search filter in capturing all relevant populations (eg, homeless people, immigrants, substance misusers) in a public health systematic review. STUDY DESIGN AND SETTING: Systematic and retrospective analysis via a case study. Retrospective analysis of the search filter was conducted by comparing the MEDLINE search results retrieved without using the search filter against those retrieved with the search filter. A total of 5,465 additional results from the unfiltered search were screened to the same criteria as the filtered search. RESULTS: No additional populations were identified in the unfiltered sample. The search filter reduced the volume of MEDLINE hits to screen by 64%, with no impact on inclusion of populations. CONCLUSIONS: The results demonstrate the effectiveness of the filter in capturing all relevant UK populations for the review. This suggests that well-planned search filters can be written for reviews that analyze imprecisely defined population groups. This filter could be used in topic areas of associated comorbidities, for rapid clinical searches, or for investigating hard-to-reach populations.