Performance evaluation of Unified Medical Language System®'s synonyms expansion to query PubMed.

BACKGROUND: PubMed is the main access to medical literature on the Internet. In order to enhance the performance of its information retrieval tools, primarily non-indexed citations, the authors propose a method: expanding users' queries using Unified Medical Language System' (UMLS) synonyms i.e. all the terms gathered under one unique Concept Unique Identifier. METHODS: This method was evaluated using queries constructed to emphasize the differences between this new method and the current PubMed automatic term mapping. Four experts assessed citation relevance. RESULTS: Using UMLS, we were able to retrieve new citations in 45.5% of queries, which implies a small increase in recall. The new strategy led to a heterogeneous 23.7% mean increase in non-indexed citation retrieved. Of these, 82% have been published less than 4 months earlier. The overall mean precision was 48.4% but differed according to the evaluators, ranging from 36.7% to 88.1% (Inter rater agreement was poor: kappa = 0.34). CONCLUSIONS: This study highlights the need for specific search tools for each type of user and use-cases. The proposed strategy may be useful to retrieve recent scientific advancement.

Improving information retrieval using Medical Subject Headings Concepts: a test case on rare and chronic diseases.

BACKGROUND: As more scientific work is published, it is important to improve access to the biomedical literature. Since 2000, when Medical Subject Headings (MeSH) Concepts were introduced, the MeSH Thesaurus has been concept based. Nevertheless, information retrieval is still performed at the MeSH Descriptor or Supplementary Concept level. OBJECTIVE: The study assesses the benefit of using MeSH Concepts for indexing and information retrieval. METHODS: Three sets of queries were built for thirty-two rare diseases and twenty-two chronic diseases: (1) using PubMed Automatic Term Mapping (ATM), (2) using Catalog and Index of French-language Health Internet (CISMeF) ATM, and (3) extrapolating the MEDLINE citations that should be indexed with a MeSH Concept. RESULTS: Type 3 queries retrieve significantly fewer results than type 1 or type 2 queries (about 18,000 citations versus 200,000 for rare diseases; about 300,000 citations versus 2,000,000 for chronic diseases). CISMeF ATM also provides better precision than PubMed ATM for both disease categories. DISCUSSION: Using MeSH Concept indexing instead of ATM is theoretically possible to improve retrieval performance with the current indexing policy. However, using MeSH Concept information retrieval and indexing rules would be a fundamentally better approach. These modifications have already been implemented in the CISMeF search engine.

Health multi-terminology portal: a semantic added-value for patient safety.

Since the mid-90s, several quality-controlled health gateways were developed. In France, CISMeF is the leading health gateway. It indexes Internet resources from the main institutions, using the MeSH thesaurus and the Dublin Core metadata element set. Since 2005, the CISMeF Information System (IS) includes 24 health terminologies, classifications and thesauri for indexing and information retrieval. This work aims at creating a Health Multi-Terminology Portal (HMTP) and connect it to the CISMeF Terminology Database mainly for searching concepts and terms among all the health controlled vocabularies available in French (or in English and translated in French) and browsing it dynamically. To integrate the terminologies in the CISMeF IS, three steps are necessary: (1) designing a meta-model into which each terminology can be integrated, (2) developing a process to include terminologies into the HMTP, (3) building and integrating existing and new inter-terminology mappings into the HMTP. A total of 24 terminologies are included in the HMTP, with 575,300 concepts, 852,000 synonyms, 222,800 definitions and 1,180,000 relations. Heightteen of these terminologies are not included yet in the UMLS among them, some from the World Health Organization. Since January 2010, HMTP is daily used by CISMeF librarians to index in multi-terminology mode. A health multiterminology portal is a valuable tool helping the indexing and the retrieval of resources from a quality-controlled patient safety gateway. It can also be very useful for teaching or performing audits in terminology management.

Evaluation of multi-terminology super-concepts for information retrieval.

BACKGROUND: Following a recent change in the indexing policy for French quality controlled health gateway CISMeF, multiple terminologies are now being used for indexing in addition to MeSH®. OBJECTIVE: To evaluate precision and recall of super-concepts for information retrieval in a multi-terminology paradigm compared to MeSH-only. METHODS: We evaluate the relevance of resources retrieved by multi-terminology super-concepts and MeSH-only super-concepts queries. RESULTS: Recall was 8-14% higher for multi-terminology super-concepts compared to MeSH only super-concepts. Precision decreased from 0.66 for MeSH only super-concepts to 0.61 for multi-terminology super-concepts. Retrieval performance was found to vary significantly depending on the super-concepts (p<10-4) and indexing methods (manual vs automatic; p<0.004). CONCLUSION: A multi-terminology paradigm contributes to increase recall but lowers precision. Automated tools for indexing are not accurate enough to allow a very precise information retrieval.

Optimization of the PubMed Automatic Term Mapping.

PubMed, freely available on the internet, is the best known database for medical information. We propose a method of optimization of the PubMed Automatic Term Mapping (ATM) that includes MeSH terms. This method is evaluated using two queries constructed to emphasize the differences between the PubMed queries as they are at present and also between these queries and the optimized one. The proposed query is significantly more precise than the current PubMed query (54.5% vs. 27%). The optimized query proposed would be easy to implement into PubMed.

[Quality and use of websites presenting public health education and training opportunities]. / Utilisation et qualité des sites présentant l'offre de formations en santé publique.

The objective of this work was to study the use of the Internet and the quality of the websites for postgraduate public health courses in France, and to compare them with equivalent courses in the United States of America. Between June 2004 and January 2005, the authorized public health diplomas proposed in France and in the United States were inventoried and listed, and then all websites of these public health diplomas were systematically visited and reviewed using a standardized questionnaire. In France, 36 public health courses (7 post graduate diplomas [DEA], 13 Masters degrees [DESS] and 16 masters of public health [MPH]) were identified and selected from 53 websites. Information on student profiles, prerequisite skills, the courses' curricula and program descriptions and the potential career opportunities were more frequently available for the MPH compared to the DEA and DESS. In United States, 66 MPH and 127 Master of Science in Public Health (MSPH) programs were accredited. The target public and validation methods were more often indicated on the American sites, while the prerequisite skills were more frequently found on the French sites. The recent implementation of the LMD (Bachelor's-Masters-Doctoral degrees) education system in France has encouraged the utilisation of Internet as an information and communications tool for the presentation and marketing of these new diplomas.

A MEDLINE categorization algorithm.

BACKGROUND: Categorization is designed to enhance resource description by organizing content description so as to enable the reader to grasp quickly and easily what are the main topics discussed in it. The objective of this work is to propose a categorization algorithm to classify a set of scientific articles indexed with the MeSH thesaurus, and in particular those of the MEDLINE bibliographic database. In a large bibliographic database such as MEDLINE, finding materials of particular interest to a specialty group, or relevant to a particular audience, can be difficult. The categorization refines the retrieval of indexed material. In the CISMeF terminology, metaterms can be considered as super-concepts. They were primarily conceived to improve recall in the CISMeF quality-controlled health gateway. METHODS: The MEDLINE categorization algorithm (MCA) is based on semantic links existing between MeSH terms and metaterms on the one hand and between MeSH subheadings and metaterms on the other hand. These links are used to automatically infer a list of metaterms from any MeSH term/subheading indexing. Medical librarians manually select the semantic links. RESULTS: The MEDLINE categorization algorithm lists the medical specialties relevant to a MEDLINE file by decreasing order of their importance. The MEDLINE categorization algorithm is available on a Web site. It can run on any MEDLINE file in a batch mode. As an example, the top 3 medical specialties for the set of 60 articles published in BioMed Central Medical Informatics & Decision Making, which are currently indexed in MEDLINE are: information science, organization and administration and medical informatics. CONCLUSION: We have presented a MEDLINE categorization algorithm in order to classify the medical specialties addressed in any MEDLINE file in the form of a ranked list of relevant specialties. The categorization method introduced in this paper is based on the manual indexing of resources with MeSH (terms/subheadings) pairs by NLM indexers. This algorithm may be used as a new bibliometric tool.

A web-based teaching resource to prepare for final undergraduate examination: a French pilot study.

Access to accurate and quality-controlled health information on the Internet for medical students is not an easy task. CISMeF is the search tool of a MeSH-indexed directory of medical Internet resources in French. Since 2004, a new French Pre-Residency Examination (PRE) is compulsory for all medical students in the 6th year of the curriculum. The goal of this study is to evaluate CISMeF as a tool to provide teaching resources available on the Internet covering PRE material. The CISMeF terminology and the PRE CISMeF module are described. To assess the CISMeF performance in covering PRE program, its precision (number of relevant resources/number of overall resources extracted by CISMeF) and coverage (number of PRE questions covered by at least one resource in the CISMeF gateway) were computed. The CISMeF module for the new French Pre-Residency Examination is efficient as it already covers 95.7% of the program with a precision of 82.2%. Our data demonstrates that CISMeF is acceptable to guide students' learning and should be a useful teaching resource for the preparation of the French Pre-Residency Examination.

An introduction to the Semantic Web for health sciences librarians.

OBJECTIVES: The paper (1) introduces health sciences librarians to the main concepts and principles of the Semantic Web (SW) and (2) briefly reviews a number of projects on the handling of biomedical information that uses SW technology. METHODOLOGY: The paper is structured into two main parts. "Semantic Web Technology" provides a high-level description, with examples, of the main standards and concepts: extensible markup language (XML), Resource Description Framework (RDF), RDF Schema (RDFS), ontologies, and their utility in information retrieval, concluding with mention of more advanced SW languages and their characteristics. "Semantic Web Applications and Research Projects in the Biomedical Field" is a brief review of the Unified Medical Language System (UMLS), Generalised Architecture for Languages, Encyclopedias and Nomenclatures in Medicine (GALEN), HealthCyberMap, LinkBase, and the thesaurus of the National Cancer Institute (NCI). The paper also mentions other benefits and by-products of the SW, citing projects related to them. DISCUSSION AND CONCLUSIONS: Some of the problems facing the SW vision are presented, especially the ways in which the librarians' expertise in organizing knowledge and in structuring information may contribute to SW projects.

Strategies for health information retrieval.

BACKGROUND: The amount of health data accessible on the Web is increasing and Internet has become a major source of health information. Many tools and search engines are available but medical information retrieval remains difficult for both the health professional and the patients. OBJECTIVE: In this paper we describe heuristics that aim at matching as much as possible queries with the content of the documents in the context of the CISMeF catalogue (Catalogue and Index of Health Resources in French) and its Doc'CISMeF search tool. The queries are represented by terms and the content of the documents is indexed by a terminology based on the MeSH thesaurus. RESULTS: Several operations are performed to match the terms of the terminology: natural language processing techniques on multi-words queries, phonemisation, spelling correction, plain text search with adjacency etc.. Each one is tested to evaluate its contribution in matching the terminology and the indexed documents. CONCLUSION: The implemented heuristics contribute significantly with good results in maximising as much as possible the recall of the Doc'CISMeF search tool.