RAVEL: retrieval and visualization in ELectronic health records.

Because of the ever-increasing amount of information in patients' EHRs, healthcare professionals may face difficulties for making diagnoses and/or therapeutic decisions. Moreover, patients may misunderstand their health status. These medical practitioners need effective tools to locate in real time relevant elements within the patients' EHR and visualize them according to synthetic and intuitive presentation models. The RAVEL project aims at achieving this goal by performing a high profile industrial research and development program on the EHR considering the following areas: (i) semantic indexing, (ii) information retrieval, and (iii) data visualization. The RAVEL project is expected to implement a generic, loosely coupled to data sources prototype so that it can be transposed into different university hospitals information systems.

Validation of completeness, correctness, relevance and understandability of the PSIP CDSS for medication safety.

Medication errors and resulting Adverse Drug Events (ADEs) are an important issue of global healthcare. Within the European PSIP project that aims at developing solutions to improve medication safety, contextualized decision support modules aiming to prevent ADEs are being developed. The objective of this work was to thoroughly validate part of the CDSS (Clinical Decision Support System) and the underlying Knowledge Base, in order to detect incorrect or unclear alerts. We systematically developed a repository of test cases and used them for validation. The development of the test cases showed that there are differences among experts in interpreting the correctness of an alert, and that the clinical context is important when judging whether it is adequate. Overall, validation did not find major errors in the Knoweldge Base, but developed several recommendations for further improvement.

Evaluation of multi-terminology super-concepts for information retrieval.

BACKGROUND: Following a recent change in the indexing policy for French quality controlled health gateway CISMeF, multiple terminologies are now being used for indexing in addition to MeSH®. OBJECTIVE: To evaluate precision and recall of super-concepts for information retrieval in a multi-terminology paradigm compared to MeSH-only. METHODS: We evaluate the relevance of resources retrieved by multi-terminology super-concepts and MeSH-only super-concepts queries. RESULTS: Recall was 8-14% higher for multi-terminology super-concepts compared to MeSH only super-concepts. Precision decreased from 0.66 for MeSH only super-concepts to 0.61 for multi-terminology super-concepts. Retrieval performance was found to vary significantly depending on the super-concepts (p<10-4) and indexing methods (manual vs automatic; p<0.004). CONCLUSION: A multi-terminology paradigm contributes to increase recall but lowers precision. Automated tools for indexing are not accurate enough to allow a very precise information retrieval.

Relevance of Google-customized search engine vs. CISMeF quality-controlled health gateway.

CISMeF (acronym for Catalog and Index of French Language Health Resources on the Internet) is a quality-controlled health gateway conceived to catalog and index the most important and quality-controlled sources of institutional health information in French. The goal of this study is to compare the relevance of results provided by this gateway from a small set of documents selected and described by human experts to those provided by a search engine from a large set of automatically indexed and ranked resources. The Google-Customized search engine (CSE) was used. The evaluation was made using the 10th first results of 15 queries and two blinded physician evaluators. There was no significant difference between the relevance of information retrieval in CISMeF and Google CSE. In conclusion, automatic indexing does not lead to lower relevance than a manual MeSH indexing and may help to cope with the increasing number of references to be indexed in a controlled health quality gateway.

Construction of a semi-automated ICD-10 coding help system to optimize medical and economic coding.

INTRODUCTION: In order to measure the medical activity in hospitals, physicians are required to code manually information concerning a patient's stay using ICD-10. This requires trained staff and a lot of time. We propose to help speed up and facilitate the tedious task of coding patient information. METHODS: we show two methods. First, we propose an automated ICD-10-based coding help system using an automated MeSH-based indexing system and a mapping between MeSH and ICD-10 extracted from the UMLS metathesaurus. Secondly, we propose the use of drug prescriptions to complete the previous coding with the use of a mapping between a given prescription drug and the relevant ICD-10 codes (in compliance with the drug approval). RESULTS: the results of a preliminary experiment indicate that the precision of the indexing system is 40% and the recall is 30% when we compare to an economic rules-based coding and to a descriptive coding. DISCUSSION: moreover, we show that the use of prescription coding is relevant as the recall reaches 68% when the Vidal tool is used. CONCLUSION: Then, it is very interesting to complete the coding obtained automatically by the indexing/mapping system by the coding obtained from the prescriptions.

Data quality evaluation in medical database watermarking.

The use of watermarking in the protection of medical relational databases requires that the introduced distortion does not hinder records interpretation. In this paper, we present the preliminary results of a watermarked data quality evaluation protocol developed so as to analyze the perception the practitioner has of the watermark. These results show that some attributes are more appropriate for watermarking than others and also that incoherent or unlikely records resulting from careless watermarking are easily identified by an expert.

Quality indicators from laboratory and radiology information systems.

Consequences of the computerization of laboratory and radiology information system (LIS and RIS) are not well documented. The aim of this study was to evaluate the impact of computerization of LIS and RIS of four hospitals on performance and quality of care. The study was divided into three phases. First, the subprocesses and information flows of LIS and RIS were described. Then, a literature review was performed in order to identify the indicators used to assess the impact of computerization. Finally, comparisons were made between 2 hospitals. Using the initial framework, each partner described its process mapping concerning LIS and RIS. The review identified a wide panel of indicators. Only 41 were useful to assess the impact of information systems. For each two by two comparison, lists of relevant indicators have been selected from the identified indicators and according to the process mapping comparison. Two by two comparisons have to be completed. Eventually, these indicators may be integrated in the quality process of hospital information systems.

Preservation of information in terminology transcoding.

Linking interface terminologies (IT) to reference terminologies (RT) in flow of terminologies may allow health information system to be both usable and interoperable. Two French university hospitals worked independently on such a flow from the prescription of lab-test to the display of the corresponding results. The aim of this study was to evaluate the transmission of information in these two hospitals communication. An expert, supported by natural language processing tool, created the gold standard link between the 2 prescription ITs i.e. the terms that share the same meaning. A semantic pathway was defined to allow the mapping of one prescription IT to the other, through LOINC® and SNOMED®, the RT chosen by each hospital, respectively. The capacity of the semantic pathway to identify the correct links was computed. The expert found 218 links between the 2 prescription ITs (containing 580 and 374 terms that are linked to RT). The semantic pathway correctly identifies 96 of these links (44.0% [37.4-50.6]). The recall was not as good as expected, even though the semantic pathway was created in order to maximize it. Using different RT in different hospitals is not an efficient solution. The use of LOINC® seems to be preferable as a RT for prescription.

Improving information retrieval with multiple health terminologies in a quality-controlled gateway.

BACKGROUND: The Catalog and Index of French-language Health Internet resources (CISMeF) is a quality-controlled health gateway, primarily for Web resources in French (n=89,751). Recently, we achieved a major improvement in the structure of the catalogue by setting-up multiple terminologies, based on twelve health terminologies available in French, to overcome the potential weakness of the MeSH thesaurus, which is the main and pivotal terminology we use for indexing and retrieval since 1995. The main aim of this study was to estimate the added-value of exploiting several terminologies and their semantic relationships to improve Web resource indexing and retrieval in CISMeF, in order to provide additional health resources which meet the users' expectations. METHODS: Twelve terminologies were integrated into the CISMeF information system to set up multiple-terminologies indexing and retrieval. The same sets of thirty queries were run: (i) by exploiting the hierarchical structure of the MeSH, and (ii) by exploiting the additional twelve terminologies and their semantic links. The two search modes were evaluated and compared. RESULTS: The overall coverage of the multiple-terminologies search mode was improved by comparison to the coverage of using the MeSH (16,283 vs. 14,159) (+15%). These additional findings were estimated at 56.6% relevant results, 24.7% intermediate results and 18.7% irrelevant. CONCLUSION: The multiple-terminologies approach improved information retrieval. These results suggest that integrating additional health terminologies was able to improve recall. Since performing the study, 21 other terminologies have been added which should enable us to make broader studies in multiple-terminologies information retrieval.

An interface terminology for medical imaging ordering purposes.

The need for structured data in electronic health records has not been fully addressed by reference terminologies (RT) due to difficulties of use for end-users. Interface terminologies (IT), built for specific usage and users, and linked to RT, may solve this issue. We propose an IT for medical imaging prescription, based on the French nomenclature for procedure (CCAM), and its qualitative evaluation. The creation and evaluation processes were adapted from published guidelines. Prescription IT is available on the web (http://pts.chu-rouen.fr). It contains 290 orderable terms linked to 249 CCAM codes. The synonymy of prescription IT is significantly richer than the CCAM one and labels are significantly shorter. The main problem came from the CCAM, which is dedicated to billing purposes. We are planning to map prescription IT to other international RT such as RadLex or SNOMED. Prescription IT might quicken the adoption of computerized ordering processes in France.