Quantitative analysis of manual annotation of clinical text samples.

BACKGROUND: Semantic interoperability of eHealth services within and across countries has been the main topic in several research projects. It is a key consideration for the European Commission to overcome the complexity of making different health information systems work together. This paper describes a study within the EU-funded project ASSESS CT, which focuses on assessing the potential of SNOMED CT as core reference terminology for semantic interoperability at European level. OBJECTIVE: This paper presents a quantitative analysis of the results obtained in ASSESS CT to determine the fitness of SNOMED CT for semantic interoperability. METHODS: The quantitative analysis consists of concept coverage, term coverage and inter-annotator agreement analysis of the annotation experiments related to six European languages (English, Swedish, French, Dutch, German and Finnish) and three scenarios: (i) ADOPT, where only SNOMED CT was used by the annotators; (ii) ALTERNATIVE, where a fixed set of terminologies from UMLS, excluding SNOMED CT, was used; and (iii) ABSTAIN, where any terminologies available in the current national infrastructure of the annotators' country were used. For each language and each scenario, we configured the different terminology settings of the annotation experiments. RESULTS: There was a positive correlation between the number of concepts in each terminology setting and their concept and term coverage values. Inter-annotator agreement is low, irrespective of the terminology setting. CONCLUSIONS: No significant differences were found between the analyses for the three scenarios, but availability of SNOMED CT for the assessed language is associated with increased concept coverage. Terminology setting size and concept and term coverage correlate positively up to a limit where more concepts do not significantly impact the coverage values. The results did not confirm the hypothesis of an inverse correlation between concept coverage and IAA due to a lower amount of choices available. The overall low IAA results pose a challenge for interoperability and indicate the need for further research to assess whether consistent terminology implementation is possible across Europe, e.g., improving term coverage by adding localized versions of the selected terminologies, analysing causes of low inter-annotator agreement, and improving tooling and guidance for annotators. The much lower term coverage for the Swedish version of SNOMED CT compared to English together with the similarly high concept coverage obtained with English and Swedish SNOMED CT reflects its relevance as a hub to connect user interface terminologies and serving a variety of user needs.

On Contributing to the Progress of Medical Informatics as Publisher.

May 1st, 2017, will mark Dieter Bergemann's 80th birthday. As Chief Executive Officer and Owner of Schattauer Publishers from 1983 to 2016, the biomedical and health informatics community owes him a great debt of gratitude. The past and present editors of Methods of Information in Medicine, the IMIA Yearbook of Medical Informatics, and Applied Clinical Informatics want to honour and thank Dieter Bergemann by providing a brief biography that emphasizes his contributions, by reviewing his critical role as an exceptionally supportive publisher for Schattauer's three biomedical and health informatics periodicals, and by sharing some personal anecdotes. Over the past 40 years, Dieter Bergemann has been an influential, if behind-the-scenes, driving force in biomedical and health informatics publications, helping to ensure success in the dissemination of our field's research and practice.

Troubled Waters: Navigating Unintended Consequences of Health Information Technology.

OBJECTIVES: To provide an introduction to the 2016 IMIA Yearbook of Medical Informatics by the editors. METHODS: We present a brief overview of the 2016 special topic "Unintended consequences of Health IT: new problems, new solutions", we review our choice of special topic section editors, and discuss the transitions in the editorial team for next year. RESULTS: This edition of the Yearbook acknowledges the fact that implementation and use of Health Information Technology (HIT) may result in unintended consequences, which may lead to both adverse and sometimes beneficial outcomes. However to date, in the literature, undesired outcomes are emphasized with a focus on the complex causes and the many sources that may generate them. The growing awareness of the importance of HIT's unintended consequences and their increasing documentation reflect a wider acceptance of HIT by users (more use generating more consequences) and and a new type of users (a shift from early adopters to late adopters and laggards), whith great expectations regarding the improvement of care quality through HIT solutions. Different points of view on new problems and new solutions of unintended consequences of Health IT are presented through the keynote paper, survey papers, and the working group contributions. CONCLUSIONS: The regular 2016 issue of the IMIA yearbook focuses on new unintended consequences of Health IT - brought on by wider adoption and different types of users as well as solutions to addressing them.

Silver Anniversary: 25 Editions of the IMIA Yearbook.

OBJECTIVES: To provide an editorial introduction into the special 25th anniversary edition of the IMIA Yearbook of Medical Informatics with discussion of the significance of the Yearbook, past and current editorial teams, and a look into the future. METHODS: A brief overview of the 2016 anniversary edition of the Yearbook allows for a discussion of the significance and value of the Yearbook to the Biomedical Informatics community as well as a review of changes in Yearbook team and format over time. RESULTS: The IMIA Yearbook celebrates its 25th edition bearing witness to the quality of the IMIA brand, the Yearbook content, as well as to the dedication of and the inordinate amount of labor from the authors and editors of the Yearbook. Editorial teams are to be applauded for their hard work and for their foresight in steering the Yearbook from a paperback to an open access online publication. The special edition provides reviews of past editorials with the knowledge of today. CONCLUSIONS: The IMIA Yearbook celebrates a remarkable milestone providing a testament to the maturity of the Biomedical Informatics field. Informaticians across the world are encouraged to thank past editorial teams and celebrate with IMIA.

Health Information Technology Challenges to Support Patient-Centered Care Coordination.

OBJECTIVES: To provide an editorial introduction to the 2015 IMIA Yearbook of Medical Informatics. METHODS: We provide a brief overview of the 2015 special topic "Patient-Centered Care Coordination", discuss the addition of two new sections to the Yearbook, Natural Language Processing and Public Health & Epidemiology Informatics, and present our editorial plans for the upcoming celebration of the 25th anniversary of the Yearbook. RESULTS: Care delivery currently occurs through the processing of complex clinical pathways designed for increasingly multi-morbid patients by various practitioners in different settings. To avoid the consequences of the fragmentation of services, care should be organized to coordinate all providers, giving them the opportunity to share the same holistic view of the patient's condition, and to be informed of the planned clinical pathway that establishes the roles and interventions of each one. The adoption and use of electronic health records (EHRs) is a solution to address health information sharing and care coordination challenges. However, while EHRs are necessary, they are not sufficient to achieve care coordination, creating information availability does not mean the information will be accessed. This edition of the Yearbook acknowledges the fact that health information technology (HIT), and EHRs in particular, are not yet fully addressing the challenges in care coordination. Emerging trends, tools, and applications of HIT to support care coordination are presented through the keynote paper, survey papers, and working group contributions. CONCLUSIONS: In 2015, the IMIA Yearbook has been extended to emphasize two fields of biomedical informatics through new sections. Next year, the 25th anniversary of the Yearbook will be celebrated in grand style! A special issue with a touch of reflection, a bit of rediscovery, and some "science-fiction" will be published in addition to the usual edition.

Toward a formalization of the process to select IMIA Yearbook best papers.

BACKGROUND: Each year, the International Medical Informatics Association Yearbook recognizes significant scientific papers, labelled as "best papers", published the previous year in the subfields of biomedical informatics that correspond to the different section topics of the journal. For each section, about fifteen pre-selected "candidate" best papers are externally peer-reviewed to select the actual best papers. Although based on the available literature, little is known about the pre-selection process. OBJECTIVE: To move toward an explicit formalization of the candidate best papers selection process to reduce variability in the literature search across sections and over years. METHODS: A methodological framework is proposed to build for each section topic specific queries tailored to PubMed and Web of Science citation databases. The two sets of returned papers are merged and reviewed by two independent section editors and citations are tagged as "discarded", "pending", and "kept". A protocolized consolidation step is then jointly conducted to resolve conflicts. A bibliographic software tool, BibReview, was developed to support the whole process. RESULTS: The proposed search strategy was fully applied to the Decision Support section of the 2013 edition of the Yearbook. For this section, 1124 references were returned (689 PubMed-specific, 254 WoS-specific, 181 common to both databases) among which the 15 candidate best papers were selected. CONCLUSIONS: The search strategy for determining candidate best papers for an IMIA Yearbook's section is now explicitly specified and allows for reproducibility. However, some aspects of the whole process remain reviewer-dependent, mostly because there is no characterization of a "best paper".

Bridging data models and terminologies to support adverse drug event reporting using EHR data.

INTRODUCTION: This article is part of the Focus Theme of METHODs of Information in Medicine on "Managing Interoperability and Complexity in Health Systems". BACKGROUND: SALUS project aims at building an interoperability platform and a dedicated toolkit to enable secondary use of electronic health records (EHR) data for post marketing drug surveillance. An important component of this toolkit is a drug-related adverse events (AE) reporting system designed to facilitate and accelerate the reporting process using automatic prepopulation mechanisms. OBJECTIVE: To demonstrate SALUS approach for establishing syntactic and semantic interoperability for AE reporting. METHOD: Standard (e.g. HL7 CDA-CCD) and proprietary EHR data models are mapped to the E2B(R2) data model via SALUS Common Information Model. Terminology mapping and terminology reasoning services are designed to ensure the automatic conversion of source EHR terminologies (e.g. ICD-9-CM, ICD-10, LOINC or SNOMED-CT) to the target terminology MedDRA which is expected in AE reporting forms. A validated set of terminology mappings is used to ensure the reliability of the reasoning mechanisms. RESULTS: The percentage of data elements of a standard E2B report that can be completed automatically has been estimated for two pilot sites. In the best scenario (i.e. the available fields in the EHR have actually been filled), only 36% (pilot site 1) and 38% (pilot site 2) of E2B data elements remain to be filled manually. In addition, most of these data elements shall not be filled in each report. CONCLUSION: SALUS platform's interoperability solutions enable partial automation of the AE reporting process, which could contribute to improve current spontaneous reporting practices and reduce under-reporting, which is currently one major obstacle in the process of acquisition of pharmacovigilance data.

Big³. Editorial.

OBJECTIVES: To provide an editorial introduction into the 2014 IMIA Yearbook of Medical Informatics with an overview of the content, the new publishing scheme, and upcoming 25th anniversary. METHODS: A brief overview of the 2014 special topic, Big Data - Smart Health Strategies, and an outline of the novel publishing model is provided in conjunction with a call for proposals to celebrate the 25th anniversary of the Yearbook. RESULTS: 'Big Data' has become the latest buzzword in informatics and promise new approaches and interventions that can improve health, well-being, and quality of life. This edition of the Yearbook acknowledges the fact that we just started to explore the opportunities that 'Big Data' will bring. However, it will become apparent to the reader that its pervasive nature has invaded all aspects of biomedical informatics - some to a higher degree than others. It was our goal to provide a comprehensive view at the state of 'Big Data' today, explore its strengths and weaknesses, as well as its risks, discuss emerging trends, tools, and applications, and stimulate the development of the field through the aggregation of excellent survey papers and working group contributions to the topic. CONCLUSIONS: For the first time in history will the IMIA Yearbook be published in an open access online format allowing a broader readership especially in resource poor countries. For the first time, thanks to the online format, will the IMIA Yearbook be published twice in the year, with two different tracks of papers. We anticipate that the important role of the IMIA yearbook will further increase with these changes just in time for its 25th anniversary in 2016.

Looking for the evidence: value of health informatics. Editorial.

OBJECTIVES: To provide an editorial introduction to the 2013 IMIA Yearbook of Medical Informatics with an overview of its contents and contributors. METHODS: A brief overview of the main theme, and an outline of the purposes, contents, format, and acknowledgment of contributions. RESULTS: Health information technology (HIT) is currently widely implemented to improve healthcare quality and patient safety while reducing costs. Although these benefits are expected and largely advertised, the evidence for these benefits is still missing. Unintended consequences are often reported and some applications have been shown to be wasteful, harmful, and even fatal. Evidence-based health informatics has been defined as "the conscientious, explicit and judicious use of current best evidence when making decisions about the introduction and operation of information technology in a given health care setting". The 2013 issue of the IMIA Yearbook highlights important contributions about the significant challenges that arise from the assessment of HIT solutions. Progress towards evidence-based health informatics is identified to elicit what works, what doesn't work, and why. In an environment where resources are limited, budgets lower than in past years, and the need to improve care is becoming ever more pressing, focusing on this topic should guide institutions and providers in the implementation of the best health information technology. CONCLUSION: This overview of progress and current challenges across the spectrum of the discipline shows many great examples of evidence that have been gathered on the effectiveness of HIT. However, evidence remains limited and a significant work should be conducted to improve the development, testing, and implementation of HIT applications.

Trends on integrating technologies for mobile and personalized care. Findings from the section on health and clinical management.

OBJECTIVES: Summarize current excellent research and trends in the field of Health and Clinical Management. METHOD: Synopsis of the articles selected for the IMIA Yearbook 2013. RESULTS: Five papers from international peer-reviewed journals have been selected for the section of Health and Clinical Management. CONCLUSIONS: The selected articles illustrate current research regarding the Health and Clinical Management and showed that it is impacted by the importance of personalizing the care and integrating the use of mobile and connected technologies.

Use of electronic health records to evaluate practice individualization.

Medical decisions are usually evaluated by checking their compliance with guidelines. We propose an approach to determine to which extent and how decisions are individualized to patients' particular needs, beyond or against guidelines. For this purpose, we have to link the content of electronic health records with a specific ontology to allow formal and detailed representations of cases.

Left ventricular ejection fraction calculation from automatically selected and processed diastolic and systolic frames in short-axis cine-MRI.

The calculation of the left ventricular ejection fraction (LVEF) is dependent upon the accurate measurement of diastolic and systolic left ventricular volumes. Although breath-hold cine magnetic resonance imaging (MRI) allows coverage of the whole cardiac cycle with an excellent time resolution, many authors rely on the visual selection of diastolic and the systolic short-axis slices in order to reduce the postprocessing time. An automatic method was developed to detect the endocardial contour on each image, allowing an automatic selection of the systolic frame. The calculated ejection fraction was compared with radionuclide ventriculography (RNV). Sixty-five patients were examined using an electrocardiogram (ECG)-gated gradient echo sequence. Among these examinations, manual and automatic processing with MRI were compared when the time of the systolic frame concorded. Good correlations have been found between the automatic MRI approach and RNV, and between manual and automatic processing on MRI alone. The results show that the automatic determination of the ejection fraction is feasible, and should constitute an important step toward a larger acceptance of MRI as a routine tool in heart disease imaging. One major benefit of using automatic postprocessing is that it may eliminate the visual choice of the systolic frame, inaccurate in more than 50% of the studied patients.

Component-based mediation services for the integration of medical applications.

Allowing exchange of information and cooperation among network-wide distributed and heterogeneous applications is a major need of current health-care information systems. The European project SynEx aims at developing an integration platform for both new and legacy applications on each partner's site. We developed, in this project, mediation services based on the generic and reusable software components that facilitate the construction of an integration platform and ease the communication and the meaningful transformation among distributed and heterogeneous applications. The main component of the mediation services is named Pilot, which serves as an intelligent broker. It uses a multi-agents service model allowing the integration platform to be multi-servers. It transforms a client request into a valid high level service on the platform. Each service is broken up into several elementary steps by the Pilot. For each step, the Pilot uses an agent to realize the operation configured by the step. At runtime, the Pilot synchronizes the execution of different steps. To ease the communication and the interaction with the heterogeneous systems, an agent can integrate a Mediator. The Mediators are the communication and interpretation tools within the mediation services. We have developed a generic model that can be specialized for creating specific mediators for the different use cases. The mediator model uses two interfaces to connect the mediator with two systems that need to communicate. Each interface deals with the three aspects through three managers (the Communication Manager, the Syntax Manager and the Semantic Manager). Some ready-to-use specializations are developed for some well defined cases which can reduce the development effort. Once a manager is specialized, it can be used in different combinations with other managers to resolve different problems. The meaningful transformation is ensured on a semantic level in each mediator through the Semantic Model component. This last component allows the mapping among different vocabularies used by different systems through a shared ontology which allows the mapping process to focus on the meaning of the transformed information. We have used XML in different components of the mediation services as the interchange format and the description format. This has enhanced the flexibility of the components. The component based approach allows the generic components to be reused in different contexts and also allows the mediations services to be open to integrate other available technologies thus largely reduce the development efforts.

Integrating medical applications in an open architecture through generic and reusable components.

Allowing exchange of information and cooperation among network-wide distributed and heterogeneous applications is a major need of current health care information systems. It forces the development of open and modular integration architectures. Major issues in the development include defining a flexible and robust federation model, developing interaction and communication facilities as well as the mechanism insuring semantic interoperability. We developed generic and reusable software components to ease the construction of any integration platform. The Pilot and the Mediator Service components facilitate the execution of services and the meaningful transformation of information. They have been tested in the context of the SynEx European project to construct a multi-agents based integration architecture. The possibility of such architectures to take into account the issue of semantic interoperability is further discussed.

Expression and meaning of medical language: building an epistemological framework for the study of semantic distance.

The use of language for communication purposes introduces a semantic gap in data processing. We have developed a new tool called semantic distance to overcome this gap. A previous presentation of our tool was based on an empirical approach. Here, we examine the properties and theoretical foundation of semantic distance from a rationalistic perspective. We present an epistemological framework to explain the meaning of semantic distance. As a tool, the purpose of semantic distance is the exchange of data or knowledge based on linguistic expressions between software components. We propose a review of the relationship between language and its representation for data processing based on the underlying philosophical assumptions. Description logic is a new paradigm in the medical informatics community to express relations between terms in "is-a" hierarchies. The utility of semantic distance is compared to description logic for communication purposes in different use cases.

From paper-based to electronic guidelines: application to French guidelines.

In order to develop an internet-based decision support system, making available for French general practitioners several prevention guidelines is was necessary to implement paper based guideline. We propose a framework allowing to transform paper based practice guideline into their electronic form. Three different problems were identified: computability (e.g. determinism of the eCPG), logic (e.g. ambiguities when combining Booleans operators) and external validity (i.e. stability of decision for variations around thresholds and proportion of subjects classified in the various terminal nodes). The last problem concerned documentation of evidence: the level of evidence was associated only with the terminal decision node and not with the pathway through the decision tree. We concluded that computerisation of guidelines is not possible without expertise or authors advice. To improve computability it is necessary to provide authors with a framework that checks ambiguities, and logical errors.

Logistic regression model: an assessment of variability of predictions.

Risk prediction models available for cardiovascular prevention are statistical or based on machine learning methods. This paper investigates whether the logistic regression method can be considered as reference for validation of other methods. In order to test the stability of the predictions using this method, we performed two types of analyses on 50 random training and test samples drawn from the same database. In first analyses three models were obtained by forced entry of different sets of four variables. In second analyses, models were built with increasing number of predictive variables. The predictive performance was assessed by the area under the ROC curve. Although across-samples variability is low for a given model, it is large enough to lead to wrong conclusions when comparing different prediction methods. We also suggest that a low events-per-variable ratio alters the stability of a model's coefficients but does not affect the variability of prediction performance.

Modeling uncertainty in computerized guidelines using fuzzy logic.

Computerized Clinical Practice Guidelines (CPGs) improve quality of care by assisting physicians in their decision making. A number of problems emerges since patients with close characteristics are given contradictory recommendations. In this article, we propose to use fuzzy logic to model uncertainty due to the use of thresholds in CPGs. A fuzzy classification procedure has been developed that provides for each message of the CPG, a strength of recommendation that rates the appropriateness of the recommendation for the patient under consideration. This work is done in the context of a CPG for the diagnosis and the management of hypertension, published in 1997 by the French agency ANAES. A population of 82 patients with mild to moderate hypertension was selected and the results of the classification system were compared to whose given by a classical decision tree. Observed agreement is 86.6% and the variability of recommendations for patients with close characteristics is reduced.

A consensus approach to maintain a knowledge based system in pathology.

The IDEM (Images and Diagnosis from Example in Medicine) software is a computerized environment able to store unambiguous descriptions of histopathologic images from pathologists. Medical imaging could benefit from such environments if they can easily and continuously be maintained. Within the IDEM environment, we developed a knowledge management module coupled with a consensus module to support knowledge acquisition and maintenance by the experts of the domain. Two pathologists, a senior and junior pathologist, reviewed fifty-three cases of breast pathology. Our findings indicate 1) that the IDEM knowledge management module allows experts to describe images by selecting terms and defining new ones if necessary, allowing the construction of a glossary for the domain and 2) that the consensus module, developed to store valid multi-experts cases, contributes also to validate new terms of the glossary and to refine semantic distance between terms. Such methodology could be applied to others highly evolving medical domains.

Using semantic distance for the efficient coding of medical concepts.

OBJECTIVE: To use the notion of semantic distance to find the nearest neighbors of a medical concept in a controlled vocabulary. MATERIAL AND METHOD: 392 concepts from the cardiovascular chapter of the ICD-10 were projected on the axes of SNOMED III. Distances were measured on each axis and the resulting distance was found using a Lp norm. RESULTS: The distance between a set of ischemic diseases and a set of non-ischemic diseases was significant (p < 0.0001). Our method was validated by finding the k nearest neighbors of ten different diagnoses from the ICD-10 cardiovascular chapter. DISCUSSION: The availability of SNOMED-RT should improve our method. Several more steps are necessary to provide an ideal coding tool.