Limits of search filter development.

OBJECTIVE: The research attempted to develop search filters for biomedical literature databases that improve retrieval of studies of clinical relevance for the nursing and rehabilitation professions. METHODS: Diagnostic testing framework compared machine-culled and practitioner-nominated search terms with a hand-tagged clinical literature database. RESULTS: We were unable to: (1) develop filters for nursing, likely because of the overlapping and expanding scope of practice for nurses in comparison with medical professionals, or (2) develop filters for rehabilitation, because of its broad scope and the profession's multifaceted understanding of "health and ability." CONCLUSIONS: We found limitations on search filter development for these health professions: nursing and rehabilitation.

High-performance information search filters for CKD content in PubMed, Ovid MEDLINE, and EMBASE.

BACKGROUND: Finding relevant articles in large bibliographic databases such as PubMed, Ovid MEDLINE, and EMBASE to inform care and future research is challenging. Articles relevant to chronic kidney disease (CKD) are particularly difficult to find because they are often published under different terminology and are found across a wide range of journal types. STUDY DESIGN: We used computer automation within a diagnostic test assessment framework to develop and validate information search filters to identify CKD articles in large bibliographic databases. SETTING & PARTICIPANTS: 22,992 full-text articles in PubMed, Ovid MEDLINE, or EMBASE. INDEX TEST: 1,374,148 unique search filters. REFERENCE TEST: We established the reference standard of article relevance to CKD by manual review of all full-text articles using prespecified criteria to determine whether each article contained CKD content or not. We then assessed filter performance by calculating sensitivity, specificity, and positive predictive value for the retrieval of CKD articles. Filters with high sensitivity and specificity for the identification of CKD articles in the development phase (two-thirds of the sample) were then retested in the validation phase (remaining one-third of the sample). RESULTS: We developed and validated high-performance CKD search filters for each bibliographic database. Filters optimized for sensitivity reached at least 99% sensitivity, and filters optimized for specificity reached at least 97% specificity. The filters were complex; for example, one PubMed filter included more than 89 terms used in combination, including "chronic kidney disease," "renal insufficiency," and "renal fibrosis." In proof-of-concept searches, physicians found more articles relevant to the topic of CKD with the use of these filters. LIMITATIONS: As knowledge of the pathogenesis of CKD grows and definitions change, these filters will need to be updated to incorporate new terminology used to index relevant articles. CONCLUSIONS: PubMed, Ovid MEDLINE, and EMBASE can be filtered reliably for articles relevant to CKD. These high-performance information filters are now available online and can be used to better identify CKD content in large bibliographic databases.

High-performance information search filters for acute kidney injury content in PubMed, Ovid Medline and Embase.

BACKGROUND: We frequently fail to identify articles relevant to the subject of acute kidney injury (AKI) when searching the large bibliographic databases such as PubMed, Ovid Medline or Embase. To address this issue, we used computer automation to create information search filters to better identify articles relevant to AKI in these databases. METHODS: We first manually reviewed a sample of 22 992 full-text articles and used prespecified criteria to determine whether each article contained AKI content or not. In the development phase (two-thirds of the sample), we developed and tested the performance of >1.3-million unique filters. Filters with high sensitivity and high specificity for the identification of AKI articles were then retested in the validation phase (remaining third of the sample). RESULTS: We succeeded in developing and validating high-performance AKI search filters for each bibliographic database with sensitivities and specificities in excess of 90%. Filters optimized for sensitivity reached at least 97.2% sensitivity, and filters optimized for specificity reached at least 99.5% specificity. The filters were complex; for example one PubMed filter included >140 terms used in combination, including 'acute kidney injury', 'tubular necrosis', 'azotemia' and 'ischemic injury'. In proof-of-concept searches, physicians found more articles relevant to topics in AKI with the use of the filters. CONCLUSIONS: PubMed, Ovid Medline and Embase can be filtered for articles relevant to AKI in a reliable manner. These high-performance information filters are now available online and can be used to better identify AKI content in large bibliographic databases.

Interventions for enhancing medication adherence.

BACKGROUND: People who are prescribed self administered medications typically take only about half their prescribed doses. Efforts to assist patients with adherence to medications might improve the benefits of prescribed medications. OBJECTIVES: The primary objective of this review is to assess the effects of interventions intended to enhance patient adherence to prescribed medications for medical conditions, on both medication adherence and clinical outcomes. SEARCH METHODS: We updated searches of The Cochrane Library, including CENTRAL (via http://onlinelibrary.wiley.com/cochranelibrary/search/), MEDLINE, EMBASE, PsycINFO (all via Ovid), CINAHL (via EBSCO), and Sociological Abstracts (via ProQuest) on 11 January 2013 with no language restriction. We also reviewed bibliographies in articles on patient adherence, and contacted authors of relevant original and review articles. SELECTION CRITERIA: We included unconfounded RCTs of interventions to improve adherence with prescribed medications, measuring both medication adherence and clinical outcome, with at least 80% follow-up of each group studied and, for long-term treatments, at least six months follow-up for studies with positive findings at earlier time points. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted all data and a third author resolved disagreements. The studies differed widely according to medical condition, patient population, intervention, measures of adherence, and clinical outcomes. Pooling results according to one of these characteristics still leaves highly heterogeneous groups, and we could not justify meta-analysis. Instead, we conducted a qualitative analysis with a focus on the RCTs with the lowest risk of bias for study design and the primary clinical outcome. MAIN RESULTS: The present update included 109 new RCTs published since the previous update in January 2007, bringing the total number of RCTs to 182; we found five RCTs from the previous update to be ineligible and excluded them. Studies were heterogeneous for patients, medical problems, treatment regimens, adherence interventions, and adherence and clinical outcome measurements, and most had high risk of bias. The main changes in comparison with the previous update include that we now: 1) report a lack of convincing evidence also specifically among the studies with the lowest risk of bias; 2) do not try to classify studies according to intervention type any more, due to the large heterogeneity; 3) make our database available for collaboration on sub-analyses, in acknowledgement of the need to make collective advancement in this difficult field of research. Of all 182 RCTs, 17 had the lowest risk of bias for study design features and their primary clinical outcome, 11 from the present update and six from the previous update. The RCTs at lowest risk of bias generally involved complex interventions with multiple components, trying to overcome barriers to adherence by means of tailored ongoing support from allied health professionals such as pharmacists, who often delivered intense education, counseling (including motivational interviewing or cognitive behavioral therapy by professionals) or daily treatment support (or both), and sometimes additional support from family or peers. Only five of these RCTs reported improvements in both adherence and clinical outcomes, and no common intervention characteristics were apparent. Even the most effective interventions did not lead to large improvements in adherence or clinical outcomes. AUTHORS' CONCLUSIONS: Across the body of evidence, effects were inconsistent from study to study, and only a minority of lowest risk of bias RCTs improved both adherence and clinical outcomes. Current methods of improving medication adherence for chronic health problems are mostly complex and not very effective, so that the full benefits of treatment cannot be realized. The research in this field needs advances, including improved design of feasible long-term interventions, objective adherence measures, and sufficient study power to detect improvements in patient-important clinical outcomes. By making our comprehensive database available for sharing we hope to contribute to achieving these advances.

Net improvement of correct answers to therapy questions after pubmed searches: pre/post comparison.

BACKGROUND: Clinicians search PubMed for answers to clinical questions although it is time consuming and not always successful. OBJECTIVE: To determine if PubMed used with its Clinical Queries feature to filter results based on study quality would improve search success (more correct answers to clinical questions related to therapy). METHODS: We invited 528 primary care physicians to participate, 143 (27.1%) consented, and 111 (21.0% of the total and 77.6% of those who consented) completed the study. Participants answered 14 yes/no therapy questions and were given 4 of these (2 originally answered correctly and 2 originally answered incorrectly) to search using either the PubMed main screen or PubMed Clinical Queries narrow therapy filter via a purpose-built system with identical search screens. Participants also picked 3 of the first 20 retrieved citations that best addressed each question. They were then asked to re-answer the original 14 questions. RESULTS: We found no statistically significant differences in the rates of correct or incorrect answers using the PubMed main screen or PubMed Clinical Queries. The rate of correct answers increased from 50.0% to 61.4% (95% CI 55.0%-67.8%) for the PubMed main screen searches and from 50.0% to 59.1% (95% CI 52.6%-65.6%) for Clinical Queries searches. These net absolute increases of 11.4% and 9.1%, respectively, included previously correct answers changing to incorrect at a rate of 9.5% (95% CI 5.6%-13.4%) for PubMed main screen searches and 9.1% (95% CI 5.3%-12.9%) for Clinical Queries searches, combined with increases in the rate of being correct of 20.5% (95% CI 15.2%-25.8%) for PubMed main screen searches and 17.7% (95% CI 12.7%-22.7%) for Clinical Queries searches. CONCLUSIONS: PubMed can assist clinicians answering clinical questions with an approximately 10% absolute rate of improvement in correct answers. This small increase includes more correct answers partially offset by a decrease in previously correct answers.

Impact of PubMed search filters on the retrieval of evidence by physicians.

BACKGROUND: Physicians face challenges when searching PubMed for research evidence, and they may miss relevant articles while retrieving too many nonrelevant articles. We investigated whether the use of search filters in PubMed improves searching by physicians. METHODS: We asked a random sample of Canadian nephrologists to answer unique clinical questions derived from 100 systematic reviews of renal therapy. Physicians provided the search terms that they would type into PubMed to locate articles to answer these questions. We entered the physician-provided search terms into PubMed and applied two types of search filters alone or in combination: a methods-based filter designed to identify high-quality studies about treatment (clinical queries "therapy") and a topic-based filter designed to identify studies with renal content. We evaluated the comprehensiveness (proportion of relevant articles found) and efficiency (ratio of relevant to nonrelevant articles) of the filtered and nonfiltered searches. Primary studies included in the systematic reviews served as the reference standard for relevant articles. RESULTS: The average physician-provided search terms retrieved 46% of the relevant articles, while 6% of the retrieved articles were relevant (corrected) (the ratio of relevant to nonrelevant articles was 1:16). The use of both filters together produced a marked improvement in efficiency, resulting in a ratio of relevant to nonrelevant articles of 1:5 (16 percentage point improvement; 99% confidence interval 9% to 22%; p < 0.003) with no substantive change in comprehensiveness (44% of relevant articles found; p = 0.55). INTERPRETATION: The use of PubMed search filters improves the efficiency of physician searches. Improved search performance may enhance the transfer of research into practice and improve patient care.

How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks.

BACKGROUND: The consistency of treatment recommendations of evidence-based medical textbooks with more recently published evidence has not been investigated to date. Inconsistencies could affect the quality of medical care. OBJECTIVE: To determine the frequency with which topics in leading online evidence-based medical textbooks report treatment recommendations consistent with more recently published research evidence. METHODS: Summarized treatment recommendations in 200 clinical topics (ie, disease states) covered in four evidence-based textbooks--UpToDate, Physicians' Information Education Resource (PIER), DynaMed, and Best Practice--were compared with articles identified in an evidence rating service (McMaster Premium Literature Service, PLUS) since the date of the most recent topic updates in each textbook. Textbook treatment recommendations were compared with article results to determine if the articles provided different, new conclusions. From these findings, the proportion of topics which potentially require updating in each textbook was calculated. RESULTS: 478 clinical topics were assessed for inclusion to find 200 topics that were addressed by all four textbooks. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks' treatment recommendations was 23% (95% CI 17-29%) for DynaMed, 52% (95% CI 45-59%) for UpToDate, 55% (95% CI 48-61%) for PIER, and 60% (95% CI 53-66%) for Best Practice (χ(2) (3)=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131-209) for DynaMed, to 488 days (range 423-554) for PIER (P<.001 across all textbooks). CONCLUSIONS: In online evidence-based textbooks, the proportion of topics with potentially outdated treatment recommendations varies substantially.

Glomerular disease search filters for Pubmed, Ovid Medline, and Embase: a development and validation study.

BACKGROUND: Tools to enhance physician searches of Medline and other bibliographic databases have potential to improve the application of new knowledge in patient care. This is particularly true for articles about glomerular disease, which are published across multiple disciplines and are often difficult to track down. Our objective was to develop and test search filters for PubMed, Ovid Medline, and Embase that allow physicians to search within a subset of the database to retrieve articles relevant to glomerular disease. METHODS: We used a diagnostic test assessment framework with development and validation phases. We read a total of 22,992 full text articles for relevance and assigned them to the development or validation set to define the reference standard. We then used combinations of search terms to develop 997,298 unique glomerular disease filters. Outcome measures for each filter included sensitivity, specificity, precision, and accuracy. We selected optimal sensitive and specific search filters for each database and applied them to the validation set to test performance. RESULTS: High performance filters achieved at least 93.8% sensitivity and specificity in the development set. Filters optimized for sensitivity reached at least 96.7% sensitivity and filters optimized for specificity reached at least 98.4% specificity. Performance of these filters was consistent in the validation set and similar among all three databases. CONCLUSIONS: PubMed, Ovid Medline, and Embase can be filtered for articles relevant to glomerular disease in a reliable manner. These filters can now be used to facilitate physician searching.

How well are journal and clinical article characteristics associated with the journal impact factor? a retrospective cohort study.

OBJECTIVE: Journal impact factor (JIF) is often used as a measure of journal quality. A retrospective cohort study determined the ability of clinical article and journal characteristics, including appraisal measures collected at the time of publication, to predict subsequent JIFs. METHODS: Clinical research articles that passed methods quality criteria were included. Each article was rated for relevance and newsworthiness by 3 to 24 physicians from a panel of more than 4,000 practicing clinicians. The 1,267 articles (from 103 journals) were divided 60∶40 into derivation (760 articles) and validation sets (507 articles), representing 99 and 88 journals, respectively. A multiple regression model was produced determining the association of 10 journal and article measures with the 2007 JIF. RESULTS: Four of the 10 measures were significant in the regression model: number of authors, number of databases indexing the journal, proportion of articles passing methods criteria, and mean clinical newsworthiness scores. With the number of disciplines rating the article, the 5 variables accounted for 61% of the variation in JIF (R(2) = 0.607, 95% CI 0.444 to 0.706, P<0.001). CONCLUSION: For the clinical literature, measures of scientific quality and clinical newsworthiness available at the time of publication can predict JIFs with 60% accuracy.

Development and validation of filters for the retrieval of studies of clinical examination from Medline.

BACKGROUND: Efficiently finding clinical examination studies--studies that quantify the value of symptoms and signs in the diagnosis of disease-is becoming increasingly difficult. Filters developed to retrieve studies of diagnosis from Medline lack specificity because they also retrieve large numbers of studies on the diagnostic value of imaging and laboratory tests. OBJECTIVE: The objective was to develop filters for retrieving clinical examination studies from Medline. METHODS: We developed filters in a training dataset and validated them in a testing database. We created the training database by hand searching 161 journals (n = 52,636 studies). We evaluated the recall and precision of 65 candidate single-term filters in identifying studies that reported the sensitivity and specificity of symptoms or signs in the training database. To identify best combinations of these search terms, we used recursive partitioning. The best-performing filters in the training database as well as 13 previously developed filters were evaluated in a testing database (n = 431,120 studies). We also examined the impact of examining reference lists of included articles on recall. RESULTS: In the training database, the single-term filters with the highest recall (95%) and the highest precision (8.4%) were diagnosis[subheading] and "medical history taking"[MeSH], respectively. The multiple-term filter developed using recursive partitioning (the RP filter) had a recall of 100% and a precision of 89% in the training database. In the testing database, the Haynes-2004-Sensitive filter (recall 98%, precision 0.13%) and the RP filter (recall 89%, precision 0.52%) showed the best performance. The recall of these two filters increased to 99% and 94% respectively with review of the reference lists of the included articles. CONCLUSIONS: Recursive partitioning appears to be a useful method of developing search filters. The empirical search filters proposed here can assist in the retrieval of clinical examination studies from Medline; however, because of the low precision of the search strategies, retrieving relevant studies remains challenging. Improving precision may require systematic changes in the tagging of articles by the National Library of Medicine.

Optimal search filters for renal information in EMBASE.

BACKGROUND: EMBASE is a popular database used to retrieve biomedical information. Our objective was to develop and test search filters to help clinicians and researchers efficiently retrieve articles with renal information in EMBASE. STUDY DESIGN: We used a diagnostic test assessment framework because filters operate similarly to screening tests. SETTINGS & PARTICIPANTS: We divided a sample of 5,302 articles from 39 journals into development and validation sets of articles. INDEX TEST: Information retrieval properties were assessed by treating each search filter as a "diagnostic test" or screening procedure for the detection of relevant articles. We tested the performance of 1,936,799 search filters made of unique renal terms and their combinations. REFERENCE STANDARD & OUTCOME: The reference standard was manual review of each article. We calculated the sensitivity and specificity of each filter to identify articles with renal information. RESULTS: The best renal filters consisted of multiple search terms, such as "renal replacement therapy," "renal," "kidney disease," and "proteinuria," and the truncated terms "kidney," "dialy," "neph," "glomerul," and "hemodial." These filters achieved peak sensitivities of 98.7% (95% CI, 97.9-99.6) and specificities of 98.5% (95% CI, 98.0-99.0). The retrieval performance of these filters remained excellent in the validation set of independent articles. LIMITATIONS: The retrieval performance of any search will vary depending on the quality of all search concepts used, not just renal terms. CONCLUSIONS: We empirically developed and validated high-performance renal search filters for EMBASE. These filters can be programmed into the search engine or used on their own to improve the efficiency of searching.

Finding knowledge translation articles in CINAHL.

BACKGROUND: The process of moving research into practice has a number of names including knowledge translation (KT). Researchers and decision makers need to be able to readily access the literature on KT for the field to grow and to evaluate the existing evidence. METHODS: To develop and validate search filters for finding KT articles in the database Cumulative Index to Nursing and Allied Health (CINAHL). A gold standard database was constructed by hand searching and classifying articles from 12 journals as KT Content, KT Applications and KT Theory. MAIN OUTCOME MEASURES: Sensitivity, specificity, precision, and accuracy of the search filters. RESULTS: Optimized search filters had fairly low sensitivity and specificity for KT Content (58.4% and 64.9% respectively), while sensitivity and specificity increased for retrieving KT Application (67.5% and 70.2%) and KT Theory articles (70.4% and 77.8%). CONCLUSION: Search filter performance was suboptimal marking the broad base of disciplines and vocabularies used by KT researchers. Such diversity makes retrieval of KT studies in CINAHL difficult.

Towards automatic recognition of scientifically rigorous clinical research evidence.

The growing numbers of topically relevant biomedical publications readily available due to advances in document retrieval methods pose a challenge to clinicians practicing evidence-based medicine. It is increasingly time consuming to acquire and critically appraise the available evidence. This problem could be addressed in part if methods were available to automatically recognize rigorous studies immediately applicable in a specific clinical situation. We approach the problem of recognizing studies containing useable clinical advice from retrieved topically relevant articles as a binary classification problem. The gold standard used in the development of PubMed clinical query filters forms the basis of our approach. We identify scientifically rigorous studies using supervised machine learning techniques (Naïve Bayes, support vector machine (SVM), and boosting) trained on high-level semantic features. We combine these methods using an ensemble learning method (stacking). The performance of learning methods is evaluated using precision, recall and F(1) score, in addition to area under the receiver operating characteristic (ROC) curve (AUC). Using a training set of 10,000 manually annotated MEDLINE citations, and a test set of an additional 2,000 citations, we achieve 73.7% precision and 61.5% recall in identifying rigorous, clinically relevant studies, with stacking over five feature-classifier combinations and 82.5% precision and 84.3% recall in recognizing rigorous studies with treatment focus using stacking over word + metadata feature vector. Our results demonstrate that a high quality gold standard and advanced classification methods can help clinicians acquire best evidence from the medical literature.

Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication--before-and-after study.

PURPOSE: To determine the quality of reporting of diagnostic accuracy studies before and after the Standards for Reporting of Diagnostic Accuracy (STARD) statement publication and to determine whether there is a difference in the quality of reporting by comparing STARD (endorsing) and non-STARD (nonendorsing) journals. MATERIALS AND METHODS: Diagnostic accuracy studies were identified by hand searching six STARD and six non-STARD journals for 2001, 2002, 2004, and 2005. Diagnostic accuracy studies (n = 240) were assessed by using a checklist of 13 of 25 STARD items. The change in the mean total score on the modified STARD checklist was evaluated with analysis of covariance. The change in proportion of times that each individual STARD item was reported before and after STARD statement publication was evaluated (chi(2) tests for linear trend). RESULTS: With mean total score as dependent factor, analysis of covariance showed that the interaction between the two independent factors (STARD or non-STARD journal and year of publication) was not significant (F = 0.664, df = 3, partial eta(2) = 0.009, P = .58). Additionally, the frequency with which individual items on the STARD checklist were reported before and after STARD statement publication has remained relatively constant, with little difference between STARD and non-STARD journals. CONCLUSION: After publication of the STARD statement in 2003, the quality of reporting of diagnostic accuracy studies remained similar to pre-STARD statement publication levels, and there was no meaningful difference (ie, one additional item on the checklist of 13 of 25 STARD items being reported) in the quality of reporting between those journals that published the STARD statement and those that did not.

PsycINFO search strategies identified methodologically sound therapy studies and review articles for use by clinicians and researchers.

OBJECTIVE: This study evaluated search strategies for finding high-quality studies on treatment and systematic reviews in PsycINFO. STUDY DESIGN AND SETTING: Sixty-four journals were hand searched at McMaster University. Methodologic criteria were applied to clinically relevant articles to identify "pass" and "fail" articles; 4,985 candidate terms were compiled, 7,463 combinations for therapy articles, and 5,246 combinations for reviews. Candidate search strategy results were compared with hand searches. The proposed strategies served as "diagnostic tests" for sound studies; the hand searches were the "gold standard." Sensitivity, specificity, precision, and accuracy were calculated. RESULTS: Of 716 treatment articles, 233 (32.5%) met criteria for scientific merit, and 58 (11.5%) of 506 review articles met criteria for systematic reviews. For treatment studies, combined terms had a peak sensitivity of 97.9% (specificity 52.2%). Maximum specificity was 97.7% (sensitivity 51.5%). Sensitivity and specificity were each 79% when optimizing both while minimizing their difference. For review articles, combined terms had a peak sensitivity of 81.0% (specificity 54.4%). Maximum specificity was 98.1% (sensitivity 51.7%). Sensitivity and specificity were each 65% when optimizing both while minimizing their difference. CONCLUSIONS: Empirically derived search strategies can achieve high sensitivity and specificity for retrieving sound treatment studies and review articles from PsycINFO.

Systematic reviews and original articles differ in relevance, novelty, and use in an evidence-based service for physicians: PLUS project.

OBJECTIVES: To describe the ratings from physicians, and use by physicians, of high quality, clinically pertinent original articles and systematic reviews from over 110 clinical journals and the Cochrane Database of Systematic Reviews (CDSRs). STUDY DESIGN AND SETTING: Prospective observational study. Data were collected via an online clinical rating system of relevance and newsworthiness for quality-filtered clinical articles and via an online delivery service for practicing physicians, during the course of the McMaster Premium LiteratUre Service Trial. Clinical ratings of articles in the MORE system by over 1,900 physicians were compared and the usage rates over 13 months of these articles by physicians, who were not raters, were examined. RESULTS: Systematic reviews were rated significantly higher than original articles for relevance (P<0.001), but significantly lower for newsworthiness (P<0.001). Reviews published in the CDSR had significantly lower ratings for both relevance (P<0.001) and newsworthiness (P<0.001) than reviews published in other journals. Participants accessed reviews more often than original articles (P<0.001), and accessed reviews from journals more often than from CDSR (P<0.001). CONCLUSION: Physician ratings and the use of high-quality original articles and systematic reviews differed, generally favoring systematic reviews over original articles. Reviews published in journals were rated higher and accessed more often than Cochrane reviews.

Sample size determination for bibliographic retrieval studies.

BACKGROUND: Research for developing search strategies to retrieve high-quality clinical journal articles from MEDLINE is expensive and time-consuming. The objective of this study was to determine the minimal number of high-quality articles in a journal subset that would need to be hand-searched to update or create new MEDLINE search strategies for treatment, diagnosis, and prognosis studies. METHODS: The desired width of the 95% confidence intervals (W) for the lowest sensitivity among existing search strategies was used to calculate the number of high-quality articles needed to reliably update search strategies. New search strategies were derived in journal subsets formed by 2 approaches: random sampling of journals and top journals (having the most high-quality articles). The new strategies were tested in both the original large journal database and in a low-yielding journal (having few high-quality articles) subset. RESULTS: For treatment studies, if W was 10% or less for the lowest sensitivity among our existing search strategies, a subset of 15 randomly selected journals or 2 top journals were adequate for updating search strategies, based on each approach having at least 99 high-quality articles. The new strategies derived in 15 randomly selected journals or 2 top journals performed well in the original large journal database. Nevertheless, the new search strategies developed using the random sampling approach performed better than those developed using the top journal approach in a low-yielding journal subset. For studies of diagnosis and prognosis, no journal subset had enough high-quality articles to achieve the expected W (10%). CONCLUSION: The approach of randomly sampling a small subset of journals that includes sufficient high-quality articles is an efficient way to update or create search strategies for high-quality articles on therapy in MEDLINE. The concentrations of diagnosis and prognosis articles are too low for this approach.

Cumulative Index to Nursing and Allied Health Literature search strategies for identifying methodologically sound causation and prognosis studies.

We developed search strategies for detecting sound articles on causation and prognosis in Cumulative Index to Nursing and Allied Health Literature (CINAHL) in the year 2000. An analytic survey was conducted, comparing hand searches of 75 journals with retrievals from CINAHL for 5,020 search terms and 11,784 combinations for causation and 9,946 combinations for prognosis. For detecting sound causation studies, a three-term strategy maximized sensitivity at 97.0% with a specificity of 52.3%. For detecting sound prognosis studies, a three-term strategy maximized sensitivity at 92.2% with a specificity of 50.0%. These search filters will enhance the searching efforts of clinicians and researchers.

EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews.

OBJECTIVES: Systematic reviews of the literature are instrumental for bridging research to health care practice and are widely available through databases such as MEDLINE and EMBASE. Search strategies have been developed to aid users in MEDLINE, but no empirical work has been done for EMBASE. The objective of this study was to develop search strategies that optimize the retrieval of methodologically sound systematic reviews from EMBASE. STUDY DESIGN AND SETTING: An analytic survey was conducted, comparing hand searches of 55 journals with retrievals from EMBASE for 4,843 candidate search terms and 17,004 combinations. Candidate search strategies were run in EMBASE, the retrievals being compared with the hand search data. The sensitivity, specificity, precision, and accuracy of the search strategies were calculated. RESULTS: Two hundred twenty (16.2%) of the 1,354 articles classified as a review met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 94.6% with specificity at 63.7%, whereas combinations of search terms to optimize specificity reached peak specificities of 99.3% with sensitivity at 61.4%. CONCLUSION: Empirically derived search strategies can achieve high sensitivity and specificity for retrieving methodologically sound systematic reviews from EMBASE.