The NICE search filters for treating and managing COVID-19: validation in MEDLINE and Embase (Ovid).

Objective: In this paper we report how the United Kingdom's National Institute for Health and Care Excellence (NICE) search filters for treating and managing COVID-19 were validated for use in MEDLINE (Ovid) and Embase (Ovid). The objective was to achieve at least 98.9% for recall and 64% for precision. Methods: We did two tests of recall to finalize the draft search filters. We updated the data from an earlier peer-reviewed publication for the first recall test. For the second test, we collated a set of systematic reviews from Epistemonikos COVID-19 L.OVE and extracted their primary studies. We calculated precision by screening all the results retrieved by the draft search filters from a targeted sample covering 2020-23. We developed a gold-standard set to validate the search filter by using all articles available from the "Treatment and Management" subject filter in the Cochrane COVID-19 Study Register. Results: In the first recall test, both filters had 99.5% recall. In the second test, recall was 99.7% and 99.8% in MEDLINE and Embase respectively. Precision was 91.1% in a deduplicated sample of records. In validation, we found the MEDLINE filter had recall of 99.86% of the 14,625 records in the gold-standard set. The Embase filter had 99.88% recall of 19,371 records. Conclusion: We have validated search filters to identify records on treating and managing COVID-19. The filters may require subsequent updates, if new SARS-CoV-2 variants of concern or interest are discussed in future literature.

MEDLINE citation tool accuracy: an analysis in two platforms.

Background: Libraries provide access to databases with auto-cite features embedded into the services; however, the accuracy of these auto-cite buttons is not very high in humanities and social sciences databases. Case Presentation: This case compares two biomedical databases, Ovid MEDLINE and PubMed, to see if either is reliable enough to confidently recommend to students for use when writing papers. A total of 60 citations were assessed, 30 citations from each citation generator, based on the top 30 articles in PubMed from 2010 to 2020. Conclusions: Error rates were higher in Ovid MEDLINE than PubMed but neither database platform provided error-free references. The auto-cite tools were not reliable. Zero of the 60 citations examined were 100% correct. Librarians should continue to advise students not to rely solely upon citation generators in these biomedical databases.

Multinational Association of Supportive Care in Cancer (MASCC) guidelines: cannabis for psychological symptoms including insomnia, anxiety, and depression.

PURPOSE: During the treatment of cancer, 18% of patients use cannabis for symptom management. Anxiety, depression, and sleep disturbances are common symptoms in cancer. A systematic review of the evidence for cannabis use for psychological symptoms in cancer patients was undertaken to develop a guideline. METHODS: A literature search of randomized trials and systematic reviews was undertaken up to November 12, 2021. Studies were independently assessed for evidence by two authors and then evaluated by all authors for approval. The literature search involved MEDLINE, CCTR, EMBASE, and PsychINFO databases. Inclusion criteria included randomized control trials and systematic reviews on cannabis versus placebo or active comparator in patients with cancer and psychological symptom management (anxiety, depression, and insomnia). RESULTS: The search yielded 829 articles; 145 from Medline, 419 from Embase, 62 from PsychINFO, and 203 from CCTR. Two systematic reviews and 15 randomized trials (4 on sleep, 5 on mood, 6 on both) met eligibility criteria. However, no studies specifically assessed the efficacy of cannabis on psychological symptoms as primary outcomes in cancer patients. The studies varied widely in terms of interventions, control, duration, and outcome measures. Six of 15 RCTs suggested benefits (five for sleep, one for mood). CONCLUSION: There is no high-quality evidence to recommend the use of cannabis as an intervention for psychological symptoms in patients with cancer until more high-quality research demonstrates benefit.

Precision and Recall of Search Strategies for Identifying Studies on Work-Related Psychosocial Risk Factors in PubMed.

PURPOSE: This study aims to report on the effectiveness of various search strategies and keywords to find studies on work-related psychosocial risk factors (PRF) in the PubMed bibliographic database. METHODS: We first selected by hand-searching 191articles published on PRF and indexed in PubMed. We extracted 30 relevant MeSH terms and 38 additional textwords. We then searched PubMed combining these 68 keywords and 27 general keywords on work-related factors. Among the 2953 articles published in January 2020, we identified 446 articles concerning exposure to PRF, which were gathered in a Gold Standard database. We then computed the Recall, Precision, and Number Needed to Read of each keyword or combination of keywords. RESULTS: Overall, 189 search-words alone or in combination were tested. The highest Recall with a single MeSH term or textword was 43% and 35%, respectively. Subsequently, we developed two different search strings, one optimizing Recall while keeping Precision acceptable (Recall 98.2%, Precision 5.9%, NNR 16.9) and one optimizing Precision while keeping Recall acceptable (Recall 73.1%, Precision 25.5%, NNR 9.7). CONCLUSIONS: No single MeSH term is available to identify relevant studies on PRF in PubMed. Locating these types of studies requires the use of various MeSH and non-MeSH terms in combination to obtain a satisfactory Recall. Nevertheless, enhancing the Recall of search strategies may lead to lower Precision, and higher NNR, although with a non-linear trend. This factor must be taken into consideration when searching PubMed.

Optimizing the literature search: coverage of included references in systematic reviews in Medline and Embase.

Objective: The aim of this study was to investigate if the included references in a set of completed systematic reviews are indexed in Ovid MEDLINE and Ovid Embase, and how many references would be missed if we were to constrict our literature searches to one of these sources, or the two databases in combination. Methods: We conducted a cross-sectional study where we searched for each included reference (n = 4,709) in 274 reviews produced by the Norwegian Institute of Public Health to find out if the references were indexed in the respective databases. The data was recorded in an Excel spreadsheet where we calculated the indexing rate. The reviews were sorted into eight categories to see if the indexing rate differs from subject to subject. Results: The indexing rate in MEDLINE (86.6%) was slightly lower than in Embase (88.2%). Without the MEDLINE records in Embase, the indexing rate in Embase was 71.8%. The highest indexing rate was achieved by combining both databases (90.2%). The indexing rate was highest in the category "Physical health - treatment" (97.4%). The category "Welfare" had the lowest indexing rate (58.9%). Conclusion: Our data reveals that 9.8% of the references are not indexed in either database. Furthermore, in 5% of the reviews, the indexing rate was 50% or lower.

Automated indexing using NLM's Medical Text Indexer (MTI) compared to human indexing in Medline: a pilot study.

Objective: In 2002, the National Library of Medicine (NLM) introduced semi-automated indexing of Medline using the Medical Text Indexer (MTI). In 2021, NLM announced that it would fully automate its indexing in Medline with an improved MTI by mid-2022. This pilot study examines indexing using a sample of records in Medline from 2000, and how an early, public version of MTI's outputs compares to records created by human indexers. Methods: This pilot study examines twenty Medline records from 2000, a year before the MTI was introduced as a MeSH term recommender. We identified twenty higher- and lower-impact biomedical journals based on Journal Impact Factor (JIF) and examined the indexing of papers by feeding their PubMed records into the Interactive MTI tool. Results: In the sample, we found key differences between automated and human-indexed Medline records: MTI assigned more terms and used them more accurately for citations in the higher JIF group, and MTI tended to rank the Male check tag more highly than the Female check tag and to omit Aged check tags. Sometimes MTI chose more specific terms than human indexers but was inconsistent in applying specificity principles. Conclusion: NLM's transition to fully automated indexing of the biomedical literature could introduce or perpetuate inconsistencies and biases in Medline. Librarians and searchers should assess changes to index terms, and their impact on PubMed's mapping features for a range of topics. Future research should evaluate automated indexing as it pertains to finding clinical information effectively, and in performing systematic searches.

PubMed's core clinical journals filter: redesigned for contemporary clinical impact and utility.

Objective: The Core Clinical Journals (CCJ) list, produced by the U.S. National Library of Medicine (NLM), has been used by clinicians and librarians for half a century for two main purposes: narrowing a literature search to clinically useful journals and identifying high priority titles for library collections. After documentation of low usage of the existing CCJ, a review was undertaken to assess current validity, followed by an update to current clinical needs. Methods: As the subject coverage of the 50-year-old list had never been evaluated, the CCJ committee began its innovative step-wise approach by analyzing the existing subject scope. To determine whether clinical subjects had changed over the last half-century, the committee collected data on journal usage in hospitals and medical facilities, adding journal usage from Morning Report blogs recording the journal article citations used by physicians and residents in response to clinical questions. Patient-driven high-frequency diagnoses and subjects added contextual data by depicting the clinical environment. Results: The analysis identified a total of 80 subjects and selected 241 journals for the updated Clinical Journals filter, based on actual clinical utility of each journal. Discussion: These data-driven methods created a different framework for evaluating the structure and content of this filter. It is the real-world evidence needed to highlight CCJ clinical impact and push clinically useful journals to first page results. Since the new process resulted in a new product, the name warrants a change from Core Clinical Journals (CCJ) to Clinically Useful Journals (CUJ). Therefore, the redesigned NLM Core Clinical Journals/AIM set from this point forward will be referred to as Clinically Useful Journals (CUJ). The evidence-based process used to reframe evaluation of the clinical impact and utility of biomedical journals is documented in this article.

Sub-Saharan Africa's biomedical journal coverage in scholarly databases: a comparison of Web of Science, Scopus, EMBASE, MEDLINE, African Index Medicus, and African Journals Online.

Objective: This study aims to find out the coverage of biomedical journals published in Sub-Saharan Africa in four authoritative international databases-Web of Science, Scopus, MEDLINE and EMBASE and two Africa-focused scholarly databases-Africa Journals Online (AJOL) and African Index Medicus (AIM). Methods: Lists of active journals that are published in the 46 Sub-Saharan African countries were retrieved from the Ulrich periodical directory to create master journal lists. Unique journals from other databases that were not found in Ulrich were added to the master journal list. The six databases included in this study were searched for journals on the master lists. Results: Only 23 of the 46 Sub-Saharan African countries had at least one biomedical journal. Only about one-quarter (152) of the 560 biomedical journals from Sub-Saharan Africa were found in at least one of the biomedical databases. South African journals accounted for more than 50% of all the Sub-Saharan journals in the international scholarly databases. AJOL contains the highest number of biomedical journals from Sub-Saharan Africa, followed by Scopus and EMBASE. AJOL asserts its importance by covering the highest number of unique journals and having a representative number of journals in all biomedical sub-disciplines. Conclusion: The majority of studies from Sub-Saharan Africa are left out when biomedical evidence-based researchers only retrieve studies from authoritative international databases. Searching Google Scholar and the African research databases of AJOL and AIM would increase the number of studies from the region.

Application of text mining to the development and validation of a geographic search filter to facilitate evidence retrieval in Ovid MEDLINE: An example from the United States.

BACKGROUND: Given the increasing volume of published research in bibliographic databases, efficient retrieval of evidence is crucial and represents an opportunity to integrate novel techniques such as text mining. OBJECTIVES: To develop and validate a geographic search filter for identifying research from the United States (US) in Ovid MEDLINE. METHODS: US and non-US citations were collected from bibliographies of evidence-based reviews. Citations were partitioned by US/non-US status and randomly divided to a training and testing set. Using text mining, common one- and two-word terms in title/abstract fields were identified, and frequencies compared between US/non-US citations. RESULTS: Common US-related terms included (as ratio of frequency in US/non-US citations) US populations and geographic terms [e.g., 'Americans' (15.5), 'Baltimore' (20.0)]. Common non-US terms were non-US geographic terms [e.g., 'Japan' (0.04), 'French' (0.05)]. A search filter was developed with 98.3% sensitivity and 82.7% specificity. DISCUSSION: This search filter will streamline the identification of evidence from the US. Periodic updates may be necessary to reflect changes in MEDLINE's controlled vocabulary. CONCLUSION: Text mining was instrumental to the development of this search filter. A novel technique generated a gold standard set comprising >20,000 citations. This method may be adapted to develop subsequent geographic search filters.

Updated generic search filters for finding studies of adverse drug effects in Ovid medline and Embase may retrieve up to 90% of relevant studies.

BACKGROUND: The most current objectively derived search filters for adverse drug effects are 15 years old and other strategies have not been developed and tested empirically. OBJECTIVE: To develop and validate search filters to retrieve evidence on adverse drug effects from Ovid medline and Ovid Embase. METHODS: We identified systematic reviews of adverse drug effects in Epistemonikos. From these reviews, we collated their included studies which we then randomly divided into three tests and one validation set of records. We constructed a search strategy to maximise relative recall using word frequency analysis with test set one. This search strategy was then refined using test sets two and three and validated on the final set of records. RESULTS: Of 107 systematic reviews which met our inclusion criteria, 1948 unique included studies were available from medline and 1980 from Embase. Generic adverse drug effects searches in medline and Embase achieved 90% and 89% relative recall, respectively. When specific adverse effects terms were added recall was improved. CONCLUSION: We have derived and validated search filters that retrieve around 90% of records with adverse drug effects data in medline and Embase. The addition of specific adverse effects terms is required to achieve higher recall.

Development of a validated search filter for Ovid Embase for degenerative cervical myelopathy.

BACKGROUND: Degenerative cervical myelopathy (DCM) is a recently proposed umbrella term for symptomatic cervical spinal cord compression secondary to degeneration of the spine. Currently literature searching for DCM is challenged by the inconsistent uptake of the term 'DCM' with many overlapping keywords and numerous synonyms. OBJECTIVES: Here, we adapt our previous Ovid medline search filter for the Ovid embase database, to support comprehensive literature searching. Both embase and medline are recommended as a minimum for systematic reviews. METHODS: References contained within embase identified in our prior study formed a 'development gold standard' reference database (N = 220). The search filter was adapted for embase and checked against the reference database. The filter was then validated against the 'validation gold standard'. RESULTS: A direct translation was not possible, as medline indexing for DCM and the keywords search field were not available in embase. We also used the 'focus' function to improve precision. The resulting search filter has 100% sensitivity in testing. DISCUSSION AND CONCLUSION: We have developed a validated search filter capable of retrieving DCM references in embase with high sensitivity. In the absence of consistent terminology and indexing, this will support more efficient and robust evidence synthesis in the field.

Development and validation of search filters to retrieve medication discontinuation articles in Medline and Embase.

BACKGROUND: Medication discontinuation studies explore the outcomes of stopping a medication compared to continuing it. Comprehensively identifying medication discontinuation articles in bibliographic databases remains challenging due to variability in terminology. OBJECTIVES: To develop and validate search filters to retrieve medication discontinuation articles in Medline and Embase. METHODS: We identified medication discontinuation articles in a convenience sample of systematic reviews. We used primary articles to create two reference sets for Medline and Embase, respectively. The reference sets were equally divided by randomization in development sets and validation sets. Terms relevant for discontinuation were identified by term frequency analysis in development sets and combined to develop two search filters that maximized relative recalls. The filters were validated against validation sets. Relative recalls were calculated with their 95% confidences intervals (95% CI). RESULTS: We included 316 articles for Medline and 407 articles for Embase, from 15 systematic reviews. The Medline optimized search filter combined 7 terms. The Embase optimized search filter combined 8 terms. The relative recalls were respectively 92% (95% CI: 87-96) and 91% (95% CI: 86-94). CONCLUSIONS: We developed two search filters for retrieving medication discontinuation articles in Medline and Embase. Further research is needed to estimate precision and specificity of the filters.

Clinical studies investigating the role of mesenchymal stem cells in healing of fracture non-unions: a systematic review.

Objectives: To identify and evaluate the effectiveness of mesenchymal stem cells (MSCs) in augmenting healing in fracture non-unions. METHODS: A focused literature search was performed on the PubMed/MEDLINE index using the keywords: "non-union", "mesenchymal stem cells", "bone healing", "MSC", "stem cells", and their MeSH terms. The search was reiterated until the 10th of August 2022. Clinical studies were included that assessed the effect of MSCs on fracture non-unions. RESULTS: Thirteen human clinical trials, studying a total of 318 participants were identified and studied. MSCs with and without biological or synthetic scaffolds were found to be effective in healing of non-unions. CONCLUSIONS: MSCs has been demonstrated to have promising outcomes in the treatment of bone non-union and tissue engineering methods utilizing MSCs may well prove to be valuable in accelerating the process of bone union. However, clinical application of MSCs as a standard method in achieving union in fracture non-unions requires larger clinical trials with a standardised approach to analyzing outcomes.

Trends in Modernization at the Nation's Medical Library.

This manuscript describes the ongoing efforts to meet ever-changing patron needs by modernizing operations, infrastructure, workforce, and customer service at the National Library of Medicine (NLM). Charged with collecting, organizing, preserving, and disseminating biomedical and life sciences information to the public, the NLM constantly adapts to evolving technologies and scientific advancements in order to better fulfill its mission. Historical context and an overview of operational decision-making offer a window into the workings of our nation's medical library.

Free MEDLINE access worldwide.

When Donald A.B. Lindberg M.D. was sworn in as Director of the National Library of Medicine (NLM) in 1984, MEDLINE, NLM's online database of citations and abstracts to biomedical journal articles, was searched primarily by librarians trained to use its command language interface. There were fees for searching, primarily to recover the cost of using commercial value-added telecommunications networks. Thirteen years later, in 1997, MEDLINE became free to anyone with an Internet connection and a Web browser. This chapter provides an insider's view of how Dr. Lindberg's vision and leadership - combined with new technology, astute handling of policy issues, and key help from political supporters and influential advocates - enabled a tremendous expansion in access to biomedical and health information for scientists, health professionals, patients, and the public.

The multiple dimensions of expanded access to health information: Don Lindberg and the U.S. National Library of Medicine.

When Donald A. B. Lindberg M.D. became Director of the U.S. National Library of Medicine in 1984, trained searchers, primarily librarians, conducted less than three million searches of NLM databases. They paid for their fair share of the commercial telecommunications costs to reach NLM's computer system. In 2015 when Lindberg retired, millions of scientists, health professionals, patients, members of the public, and librarians conducted billions of free searches of NLM's greatly expanded electronic resources via the Internet. Lindberg came to NLM intending to expand access to biomedical and health information along multiple dimensions: reaching more users, providing more types and volumes of information and data; and improving the conceptual, technical, and organizational connections needed to provide information to users when and where it is needed. By any measure he and NLM were spectacularly successful. This chapter discusses some key decisions and developments that contributed to that success.

Dr. Lindberg and scholarly publishing.

Donald A.B. Lindberg M.D., Director of the U.S. National Library of Medicine (NLM) from August 1984 to March 2015, had a remarkable vision for NLM's scope, goals, and function. This vision resulted in many external partnerships and initiatives with the publishing industry, commercial and non-profit, journal editors, and professional organizations. These partnerships ranged from ongoing collaboration and dialogue, such as the NLM Publisher's Committee and the International Committee of Medical Journal Editors (ICMJE). to the more practical, such as the creation of HINARI and the Emergency Access Initiative (EAI). Dr. Lindberg fostered partnerships outside the NLM to expand the use and reach of Library resources, including MEDLINE and ClinicalTrials.gov to support innovations in the processes that build them, and to improve the quality of biomedical journals. Dr. Lindberg also encouraged the use of technology to enhance medical information and supported the early development of fully interactive publications. Attitudes that contained a measure of skepticism and distrust faded as collaborators came to have a better understanding of both NLM and their partners. This chapter discusses these relationships and accomplishments that NLM achieved working with publishers and other creators and disseminators of medical information under Dr. Lindberg's leadership.

The NICE OECD countries' geographic search filters: Part 1-methodology for developing the draft MEDLINE and Embase (Ovid) filters.

OBJECTIVE: There are no existing validated search filters for the group of 37 Organisation for Economic Co-operation and Development (OECD) countries. This study describes how information specialists from the United Kingdom's National Institute for Health and Care Excellence (NICE) developed and evaluated novel OECD countries' geographic search filters for MEDLINE and Embase (Ovid) to improve literature search effectiveness for evidence about OECD countries. METHODS: We created the draft filters using an alternative approach to standard filter construction. They are composed entirely of geographic subject headings and are designed to retain OECD country evidence by excluding non-OECD country evidence using the NOT Boolean operator. To evaluate the draft filters' effectiveness, we used MEDLINE and Embase literature searches for three NICE guidelines that retrieved >5,000 search results. A 10% sample of the excluded references was screened to check that OECD country evidence was not inadvertently excluded. RESULTS: The draft MEDLINE filter reduced results for each NICE guideline by 9.5% to 12.9%. In Embase, search results were reduced by 10.7% to 14%. Of the sample references, 7 of 910 (0.8%) were excluded inadvertently. These references were from a guideline about looked-after minors that concerns both OECD and non-OECD countries. CONCLUSION: The draft filters look promising-they reduced search result volumes while retaining most OECD country evidence from MEDLINE and Embase. However, we advise caution when using them in topics about both non-OECD and OECD countries. We have created final versions of the search filters and will validate them in a future study.

Performance evaluation of three semantic expansions to query PubMed.

BACKGROUND: PubMed is one of the most important basic tools to access medical literature. Semantic query expansion using synonyms can improve retrieval efficacy. OBJECTIVE: The objective was to evaluate the performance of three semantic query expansion strategies. METHODS: Queries were built for forty MeSH descriptors using three semantic expansion strategies (MeSH synonyms, UMLS mappings, and mappings created by the CISMeF team), then sent to PubMed. To evaluate expansion performances for each query, the first twenty citations were selected, and their relevance were judged by three independent evaluators based on the title and abstract. RESULTS: Queries built with the UMLS expansion provided new citations with a slightly higher mean precision (74.19%) than with the CISMeF expansion (70.28%), although the difference was not significant. Inter-rater agreement was 0.28. Results varied greatly depending on the descriptor selected. DISCUSSION: The number of citations retrieved by the three strategies and their precision varied greatly according to the descriptor. This heterogeneity could be explained by the quality of the synonyms. Optimal use of these different expansions would be through various combinations of UMLS and CISMeF intersections or unions. CONCLUSION: Information retrieval tools should propose different semantic expansions depending on the descriptor and the search objectives.

BioRel: towards large-scale biomedical relation extraction.

BACKGROUND: Although biomedical publications and literature are growing rapidly, there still lacks structured knowledge that can be easily processed by computer programs. In order to extract such knowledge from plain text and transform them into structural form, the relation extraction problem becomes an important issue. Datasets play a critical role in the development of relation extraction methods. However, existing relation extraction datasets in biomedical domain are mainly human-annotated, whose scales are usually limited due to their labor-intensive and time-consuming nature. RESULTS: We construct BioRel, a large-scale dataset for biomedical relation extraction problem, by using Unified Medical Language System as knowledge base and Medline as corpus. We first identify mentions of entities in sentences of Medline and link them to Unified Medical Language System with Metamap. Then, we assign each sentence a relation label by using distant supervision. Finally, we adapt the state-of-the-art deep learning and statistical machine learning methods as baseline models and conduct comprehensive experiments on the BioRel dataset. CONCLUSIONS: Based on the extensive experimental results, we have shown that BioRel is a suitable large-scale datasets for biomedical relation extraction, which provides both reasonable baseline performance and many remaining challenges for both deep learning and statistical methods.