Physician satisfaction with transition from CPOE to paper-based prescription.

INTRODUCTION: In January 2015, Rouen University Hospital's information system experienced serious issues. It was necessary to rapidly switch from the computerized provider order entry (CPOE) system towards a paper-based order entry (PBOE) system. This was an opportunity to evaluate prescriber opinion on the two provider order entry (POE) systems. METHODS: All residents were asked to fill an augmented version of the POE satisfaction and usage survey for both POE systems. The results were compared to identify the strengths and weaknesses of each system. RESULTS: Fifty-one respondents had used the CPOE system and the PBOE system. Overall, satisfaction was higher with PBOE than CPOE (odds ratio (OR)=3.74; p<0.001). Usability (OR=4.00; p<0.001), reliability (OR=8.54; p<0.001), time consumption (OR=0.50; p<0.05 - survey statement was formulated negatively), and communication with nurses (OR=14.27; p<0.0001) reached statistically better agreement. The more experience with CPOE the more residents were disillusioned with the reliability (OR=6.55; p<0.01), the usability (OR=5.68; p<0.01) and the patient safety (OR=0.27; p<0.05 - survey statement was formulated negatively) of CPOE. Although safety issues were reported for both systems, the causes were different; PBOE imposed frequent rewriting of the order while CPOE lack of usability might be unsafe. Another important issue with both POE systems was time consumption. CONCLUSION: Residents did not report any increase in safety issues with the rapid switch from CPOE to PBOE. They even seemed more satisfied with the rollback to paper, which remains a possible degraded mode in case of health information technology collapse.

[Benefits and usability of a pharmaceutical record in medical practice. A survey of hospital doctors and pharmacists (MATRIX study)]. / Intérêt et utilisabilité du dossier pharmaceutique en pratique médicale. Enquête auprès de médecins et pharmaciens hospitaliers (étude MATRIX).

BACKGROUND: To evaluate the impact of the pharmaceutical patient record use in emergency, geriatric and anaesthesia and intensive care departments, an experimentation was launched in 2013 in 55 hospitals. The purpose of the study was to assess the opinions of physicians and pharmacists about the benefits and usability of the patient pharmaceutical record. METHODS: An e-mailed self-administered questionnaire was sent to all the pharmacists, anaesthesiologists, geriatricians and emergency physicians of the 55 hospitals involved in the patient pharmaceutical record experimentation. The questionnaire assessed the usability of the patient pharmaceutical record using the "System Usability Scale", as well as its use, its benefits and limitations perceived in clinical practice, and overall user satisfaction. Questionnaires were collected from November 2014 to January 2015. RESULTS: Ninety-six questionnaires were collected, from 47 hospitals, representing 86% of the hospitals involved in the experimentation. The patient pharmaceutical record was effectively operational in 36 hospitals. Data from 73 questionnaires filled by physicians and pharmacists with potential experience with the patient pharmaceutical record were used for evaluation. Forty-two respondents were pharmacists (57%) and 31 were physicians (43%), including 13 geriatricians, 11 emergency physicians and 7 anaesthesiologists. Patient pharmaceutical record overall usability score was 62.5 out of 100. It did not vary with the profession or seniority of the respondent. It was positively correlated with the frequency of use. More than half of respondents reported that they never or uncommonly used the patient pharmaceutical record. The length of access to data period was considered as insufficient. Main obstacles to more utilization of the patient pharmaceutical record were the lack of information about the dosage of dispensed drugs, the low number of patients in possession of their health card and the low number of patients with an activated patient pharmaceutical record. CONCLUSION: Two years after the beginning of the experiment aiming to broaden the access to the patient pharmaceutical record to physicians, these first evaluation results are encouraging. The evaluation of the consequences of the access to the patient pharmaceutical record for physicians remains necessary.

Searching for rare diseases in PubMed: a blind comparison of Orphanet expert query and query based on terminological knowledge.

BACKGROUND: Despite international initiatives like Orphanet, it remains difficult to find up-to-date information about rare diseases. The aim of this study is to propose an exhaustive set of queries for PubMed based on terminological knowledge and to evaluate it versus the queries based on expertise provided by the most frequently used resource in Europe: Orphanet. METHODS: Four rare disease terminologies (MeSH, OMIM, HPO and HRDO) were manually mapped to each other permitting the automatic creation of expended terminological queries for rare diseases. For 30 rare diseases, 30 citations retrieved by Orphanet expert query and/or query based on terminological knowledge were assessed for relevance by two independent reviewers unaware of the query's origin. An adjudication procedure was used to resolve any discrepancy. Precision, relative recall and F-measure were all computed. RESULTS: For each Orphanet rare disease (n = 8982), there was a corresponding terminological query, in contrast with only 2284 queries provided by Orphanet. Only 553 citations were evaluated due to queries with 0 or only a few hits. There were no significant differences between the Orpha query and terminological query in terms of precision, respectively 0.61 vs 0.52 (p = 0.13). Nevertheless, terminological queries retrieved more citations more often than Orpha queries (0.57 vs. 0.33; p = 0.01). Interestingly, Orpha queries seemed to retrieve older citations than terminological queries (p < 0.0001). CONCLUSION: The terminological queries proposed in this study are now currently available for all rare diseases. They may be a useful tool for both precision or recall oriented literature search.

Knowledge Representation and Management. From Ontology to Annotation. Findings from the Yearbook 2015 Section on Knowledge Representation and Management.

OBJECTIVE: To summarize the best papers in the field of Knowledge Representation and Management (KRM). METHODS: A comprehensive review of medical informatics literature was performed to select some of the most interesting papers of KRM published in 2014. RESULTS: Four articles were selected, two focused on annotation and information retrieval using an ontology. The two others focused mainly on ontologies, one dealing with the usage of a temporal ontology in order to analyze the content of narrative document, one describing a methodology for building multilingual ontologies. CONCLUSION: Semantic models began to show their efficiency, coupled with annotation tools.

[Limiting a Medline/PubMed query to the "best" articles using the JCR relative impact factor]. / Limiter une requête Medline/PubMed aux articles publiés dans les revues les plus citées de chaque discipline.

BACKGROUND: Medline/PubMed is the most frequently used medical bibliographic research database. The aim of this study was to propose a new generic method to limit any Medline/PubMed query based on the relative impact factor and the A & B categories of the SIGAPS score. MATERIAL AND METHODS: The entire PubMed corpus was used for the feasibility study, then ten frequent diseases in terms of PubMed indexing and the citations of four Nobel prize winners. The relative impact factor (RIF) was calculated by medical specialty defined in Journal Citation Reports. The two queries, which included all the journals in category A (or A OR B), were added to any Medline/PubMed query as a central point of the feasibility study. RESULTS: Limitation using the SIGAPS category A was larger than the when using the Core Clinical Journals (CCJ): 15.65% of PubMed corpus vs 8.64% for CCJ. The response time of this limit applied to the entire PubMed corpus was less than two seconds. For five diseases out of ten, limiting the citations with the RIF was more effective than with the CCJ. For the four Nobel prize winners, limiting the citations with the RIF was more effective than the CCJ. CONCLUSION: The feasibility study to apply a new filter based on the relative impact factor on any Medline/PubMed query was positive.

Managing free text for secondary use of health data.

OBJECTIVE: To summarize the best papers in the field of Knowledge Representation and Management (KRM). METHODS: A comprehensive review of medical informatics literature was performed to select some of the most interesting papers of KRM and natural language processing (NLP) published in 2013. RESULTS: Four articles were selected, one focuses on Electronic Health Record (EHR) interoperability for clinical pathway personalization based on structured data. The other three focus on NLP (corpus creation, de-identification, and co-reference resolution) and highlight the increase in NLP tools performances. CONCLUSION: NLP tools are close to being seriously concurrent to humans in some annotation tasks. Their use could increase drastically the amount of data usable for meaningful use of EHR.

Attitude of physicians towards automatic alerting in computerized physician order entry systems. A comparative international survey.

OBJECTIVES: To analyze the attitude of physicians towards alerting in CPOE systems in different hospitals in different countries, addressing various organizational and technical settings and the view of physicians not currently using a CPOE. METHODS: A cross-sectional quantitative and qualitative questionnaire survey. We invited 2,600 physicians in eleven hospitals from nine countries to participate. Eight of the hospitals had different CPOE systems in use, and three of the participating hospitals were not using a CPOE system. RESULTS: 1,018 physicians participated. The general attitude of the physicians towards CPOE alerting is positive and is found to be mostly independent of the country, the specific organizational settings in the hospitals and their personal experience with CPOE systems. Both quantitative and qualitative results show that the majority of the physicians, both CPOE-users and non-users, appreciate the benefits of alerting in CPOE systems on medication safety. However, alerting should be better adapted to the clinical context and make use of more sophisticated ways to present alert information. The vast majority of physicians agree that additional information regarding interactions is useful on demand. Around half of the respondents see possible alert overload as a major problem; in this regard, physicians in hospitals with sophisticated alerting strategies show partly better attitude scores. CONCLUSIONS: Our results indicate that the way alerting information is presented to the physicians may play a role in their general attitude towards alerting, and that hospitals with a sophisticated alerting strategy with less interruptive alerts tend towards more positive attitudes. This aspect needs to be further investigated in future studies.

French Infobutton: an academic and business perspective.

Infobuttons are context-specific links between clinical information systems and other online information resources. The objective of this study is to describe a French Infobutton, which will be sold in the French-speaking Health Information market.

Cataloguing and displaying Web feeds from French language health sites: a Web 2.0 add-on to a health gateway.

Among the numerous new functionalities of the Internet, commonly called Web 2.0, Web syndication illustrates the trend for better and faster information sharing. Web feeds (a.k.a RSS feeds), which were used mostly on weblogs at first, are now also widely used in academic, scientific and institutional websites such as PubMed. As very few French language feeds were listed or catalogued in the Health field by the year of 2007, it was decided to implement them in the quality-controlled health gateway CISMeF ([French] acronym for Catalogue and Index of French Language Health Resources on the Internet). Furthermore, making full use of the nature of Web syndication, a Web feed aggregator was put online in to provide a dynamic news gateway called "CISMeF actualités" (http://www.chu-rouen.fr/actualites/). This article describes the process to retrieve and implement the Web feeds in the catalogue and how its terminology was adjusted to describe this new content. It also describes how the aggregator was put online and the features of this news gateway. CISMeF actualités was built accordingly to the editorial policy of CISMeF. Only a part of the Web feeds of the catalogue were included to display the most authoritative sources. Web feeds were also grouped by medical specialties and by countries using the prior indexing of websites with MeSH terms and the so-called metaterms. CISMeF actualités now displays 131 Web feeds across 40 different medical specialities, coming from 5 different countries. It is one example, among many, that static hypertext links can now easily and beneficially be completed, or replaced, by dynamic display of Web content using syndication feeds.

Use of super-concepts to customize electronic medical records data display.

Patient medical record systems (MRS) merely offer static applications, in which mostly unstructured text is linked to coded data. In these applications the more common presentation is a time oriented one, which does not allow easily for data and information retrieval. Concept oriented views based on supper-concepts (metaterms) initially defined in CISMeF to optimize Web medical search, was implemented in our MRS as specialties views. This work shows that these terminological tools are able to facilitate information retrieval.

Doc'UMVF: two search tools to provide quality-controlled teaching resources in French to students and teachers.

The UMVF project is a federation of medical teaching resources covering 32 medical schools in France. Today, the indexing of these resources is carried out manually by the CISMeF team at the University Hospital of Rouen. This indexing is based on MeSH thesaurus. We use a subset of SCORM metadata standard. This choice was defined in collaboration with the French Medical Virtual University consortium (French acronym: UMVF). Currently, with the UMVF searching tool (called Doc'UMVF), medical students can reach more than 3300 resources useful in their curriculum. Doc'UMVF is developed in close collaboration between the medical informatics laboratories of Rennes and Rouen. In this paper we present two complementary searching tools based on different methods and which are integrated and used to improve both the relevance and the coverage rate of the answers. A specific searching module has been built to retrieve specific resources concerning the National Medical Exam ENC ("Examen National Classant") is also available. Nevertheless, due to lack of time, numerous resources are not yet indexed. Therefore we have decided to use also automatic indexing method (Nomindex). This approach will be improved by further research works, resulting from Rouen and Geneva teams. After having built a searching meta-motor, our objective is to develop a meta-tool intended to index the whole set of digital pedagogical resources produced by the UMVF framework. This manual re-indexing will be carried out only for the most important resources (national references), with a more or less fine granularity.

Seal of transparency heritage in the CISMeF quality-controlled health gateway.

BACKGROUND: It is an absolute necessity to continually assess the quality of health information on the Internet. Quality-controlled subject gateways are Internet services which apply a selected set of targeted measures to support systematic resource discovery. METHODS: The CISMeF health gateway became a contributor to the MedCIRCLE project to evaluate 270 health information providers. The transparency heritage consists of using the evaluation performed on providers that are referenced in the CISMeF catalogue for evaluating the documents they publish, thus passing on the transparency label from the publishers to their documents. RESULTS: Each site rated in CISMeF has a record in the CISMeF database that generates an RDF into HTML file. The search tool Doc'CISMeF displays information originating from every publisher evaluated with a specific MedCIRCLE button, which is linked to the MedCIRCLE central repository. Starting with 270 websites, this trust heritage has led to 6,480 evaluated resources in CISMeF (49.8% of the 13,012 resources included in CISMeF). CONCLUSION: With the MedCIRCLE project and transparency heritage, CISMeF became an explicit third party.

MedCIRCLE: collaboration for Internet rating, certification, labelling and evaluation of health information on the World-Wide-Web.

We describe MedCIRCLE, an EU-funded semantic web project to implement the first steps towards a global, collaborative rating and guidance system for health information proposed in the MedCERTAIN project. In MedCIRCLE, three European gateway sites for consumer health information will implement the metadata vocabulary HIDDEL (Health Information Disclosure, Description and Evaluation Language). HIDDEL allows portals and gateways to make the results of their evaluations accessible as XML/RDF. The three participating national portals are: AQUMED (Agency for Quality in Medicine) patienten-information, de, COMB (Official Medical College of Barcelona) and CISMeF, a quality-controlled health gateway developed at Rouen University Hospital. Other health subject gateways, accreditation, or rating services are invited to join the collaboration simply by implementing HIDDEL on their gateways. Widespread implementation HIDDEL will allow intelligent agents or client-side software to harvest statements and opinions about the trustworthiness of other websites, assisting users in selecting trustworthy websites. The MedCIRCLE project builds on, expands and continues work on rating health information on the Internet piloted within the MedCERTAIN project. While MedCERTAIN provided the core technologies and software for rating and "trustmarking" health information, MedCIRCLE is built around these technologies and involves a wider medical community to assess health information, demonstrating the power of collaborative and interoperable evaluations in a semantic web environment. MedCIRCLE is a project with the overall objective to develop and promote technologies able to guide consumers to trustworthy health information on the Internet, to establish a global web of trust for networked health information, and to empower consumers to positively select high quality health information on the web. Other aims include refinement and expansion of HIDDEL, to become a standard vocabulary and interchange format for self- and third-party ratings of health information.

Level of evidence as a future gold standard for the content quality of health resources on the internet.

OBJECTIVE: An assessment of the quality of health information on the Internet is an absolute necessity. In this study 'sensitive' information was defined as information found in documents published on the Internet, which could be used in a medical decision. For sensitive information, the main criterion chosen for the quality of the information was an indication of the level of evidence. A survey was conducted using the CISMeF health catalogue to assess how often a score of the level of evidence is mentioned in the information accessible on the Internet in French-language health resources. METHODS: Since 1999, members of the CISMeF team have systematically been searching for all documents containing 'sensitive' information and verifying whether the level of evidence was explicitly indicated as a score at least once in the document. RESULTS: As of June 2001, 10,190 resources were included in CISMeF; including 2964 textual 'sensitive' resources (29.1%). Out of all these resources, only 4.7% (95% confidence interval: 4.0 - 5.5%) indicated the level of evidence. A statistically significant difference in the prevalence of indicating the level of evidence according to resource types (e.g., 18.1% for guidelines compared to 0.0% for teaching material), year of publication (almost three times greater in 1997-2001 compared with 1990-1996) and publishers was observed. CONCLUSION: As the number of people accessing the growing amount of information on the Internet is increasing daily, publishers have an ethical obligation to inform their readers about the validity of 'sensitive' information their sites contain. However, the vast majority of the French language Internet resources that were surveyed do not mention a score of the level of evidence for their sensitive information.

VUMeF: extending the French involvement in the UMLS Metathesaurus.

A considerable number of robust vocabularies and thesauri have been developed for the healthcare and biomedical domain. No single vocabulary, however; provides complete coverage of the information needs from a public health perspective. The results of an investigation of vocabulary sources for the development of a comprehensive controlled vocabulary for the public health domain at the Centers for Disease Control and Prevention (CDC) is presented.

An assessment of the visibility of MeSH-indexed medical web catalogs through search engines.

Manually indexed Internet health catalogs such as CliniWeb or CISMeF provide resources for retrieving high-quality health information. Users of these quality-controlled subject gateways are most often referred to them by general search engines such as Google, AltaVista, etc. This raises several questions, among which the following: what is the relative visibility of medical Internet catalogs through search engines? This study addresses this issue by measuring and comparing the visibility of six major, MeSH-indexed health catalogs through four different search engines (AltaVista, Google, Lycos, Northern Light) in two languages (English and French). Over half a million queries were sent to the search engines; for most of these search engines, according to our measures at the time the queries were sent, the most visible catalog for English MeSH terms was CliniWeb and the most visible one for French MeSH terms was CISMeF.

A resource server for medical training.

OBJECTIVES: We have developed a RESOURCE SERVER to collect and store various elements used by a professor during his lecture. METHODS: The server manages four types of objects: ELEMENTS, RESOURCES (set of elements referring to a given topic), INDEXES (to organize the resources for further search and use), and USERS (to identify providers, users, and access rights). If an ELEMENT s modified, the RESOURCE is automatically updated. RESULTS: An example (preparation of an anatomy lecture) explains how the RESOURCE SERVER works in three steps: organization of the training material, indexing, and retrieval. CONCLUSIONS: The RESOURCE SERVER will help instructors develop, update and share pedagogic resources for supporting their training courses, lessons and conferences. Moreover, these techniques, based on Internet technologies for easy handling of and access to these resources, allow local and distant access. Within the general framework of the French-speaking Virtual Medical University, the RESOURCE SERVER will represent an important link between data collection and its use in intelligent pedagogic training.

A preliminary study to assess a French code of ethics for health teaching resources on the Internet.

BACKGROUND: Constant assessment of the quality of health information on the Internet is an absolute necessity as peer review is often lacking in this media. OBJECTIVE: To develop a simple and easy French Code of Ethics, which will enable medical students to judge quality of health information in teaching material available on the Internet. DESIGN: Three medical informaticians selected ten criteria from previously established codes of ethics from Europe and the USA. This instrument was tested on a sample of 30 health Internet teaching resources. RESULTS: For the panel of experts, chance corrected inter-observer agreement (kappa) for quality rating ranged from k = -0.19 and k = 0.33, demonstrating poor agreement among the raters. CONCLUSION: If negative results of this preliminary study are confirmed by further research, this finding may detrimentally affect projects in Europe to accredit or certify Internet health resources.

A search tool based on 'encapsulated' MeSH thesaurus to retrieve quality health resources on the internet.

In the year 2001, the Internet has become a major source of health information for the health professional and the Netizen. The objective of Doc' CISMeF (D'C) was to create a powerful generic search tool based on a structured information model which 'encapsulates' the MeSH thesaurus to index and retrieve quality health resources on the Internet. To index resources, D'C uses four sections in its information model: 'meta-term', keyword, subheading, and resource type. Two search options are available: simple and advanced. The simple search requires the end-user to input a single term or expression. If this term belongs to the D'C information structure model, it will be exploded. If not, a full-text search is performed. In the advanced search, complex searches are possible combining Boolean operators with meta-terms, keywords, subheadings and resource types. D'C uses two standard tools for organising information: the MeSH thesaurus and the Dublin Core metadata format. Resources included in D'C are described according to the following elements: title, author or creator, subject and keywords, description, publishers, date, resource type, format, identifier, and language.

Doc'CISMEF: a search tool based on "encapsulated" MeSH thesaurus.

In the year 2000, the Internet became a major source of health information for the health professional and the Netizen. The objective of Doc'CISMeF (D'C) was to create a powerful generic search tool based on an structured information model which â encapsulates' the MeSH thesaurus to index and retrieve quality health resources on the Internet. To index resources, D'C uses four sections in its information model: 'meta-term', keyword, subheading, and resource type. Two search options are available: simple and advanced. The simple search requires the end-user to input a single term or expression. If this term belongs to the D'C information structure model, it will be exploded. If not, a full-text search is performed. In the advanced search, complex searches are possible combining Boolean operators with meta-terms, keywords, subheadings and resource types. D'C uses two standard tools for organising information: the MeSH thesaurus and the Dublin Core metadata format. Resources included in D'C are described according to the following elements: title, author or creator, subject and keywords, description, publisher, date, resource type, format, identifier, and language.