A community-of-practice-based evaluation methodology for knowledge intensive computational methods and its application to multimorbidity decision support.

OBJECTIVE: The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans. MATERIALS AND METHODS: Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods. RESULTS: Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support. Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods. DISCUSSION: The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS. CONCLUSION: We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

A model-driven transformation approach for the modelling of processes in clinical practice guidelines.

Clinical Practice Guidelines (CPGs) include recommendations aimed at optimising patient care, informed by a review of the available clinical evidence. To achieve their potential benefits, CPG should be readily available at the point of care. This can be done by translating CPG recommendations into one of the languages for Computer-Interpretable Guidelines (CIGs). This is a difficult task for which the collaboration of clinical and technical staff is crucial. However, in general CIG languages are not accessible to non-technical staff. We propose to support the modelling of CPG processes (and hence the authoring of CIGs) based on a transformation, from a preliminary specification in a more accessible language into an implementation in a CIG language. In this paper, we approach this transformation following the Model-Driven Development (MDD) paradigm, in which models and transformations are key elements for software development. To demonstrate the approach, we implemented and tested an algorithm for the transformation from the BPMN language for business processes to the PROforma CIG language. This implementation uses transformations defined in the ATLAS Transformation Language. Additionally, we conducted a small experiment to assess the hypothesis that a language such as BPMN can facilitate the modelling of CPG processes by clinical and technical staff.

CLIN-IK-LINKS: A platform for the design and execution of clinical data transformation and reasoning workflows.

BACKGROUND AND OBJECTIVE: Effective sharing and reuse of Electronic Health Records (EHR) requires technological solutions which deal with different representations and different models of data. This includes information models, domain models and, ideally, inference models, which enable clinical decision support based on a knowledge base and facts. Our goal is to develop a framework to support EHR interoperability based on transformation and reasoning services intended for clinical data and knowledge. METHODS: Our framework is based on workflows whose primary components are reusable mappings. Key features are an integrated representation, storage, and exploitation of different types of mappings for clinical data transformation purposes, as well as the support for the discovery of new workflows. The current framework supports mappings which take advantage of the best features of EHR standards and ontologies. Our proposal is based on our previous results and experience working with both technological infrastructures. RESULTS: We have implemented CLIN-IK-LINKS, a web-based platform that enables users to create, modify and delete mappings as well as to define and execute workflows. The platform has been applied in two use cases: semantic publishing of clinical laboratory test results; and implementation of two colorectal cancer screening protocols. Real data have been used in both use cases. CONCLUSIONS: The CLIN-IK-LINKS platform allows the composition and execution of clinical data transformation workflows to convert EHR data into EHR and/or semantic web standards. Having proved its usefulness to implement clinical data transformation applications of interest, CLIN-IK-LINKS can be regarded as a valuable contribution to improve the semantic interoperability of EHR systems.

Towards the semantic enrichment of Computer Interpretable Guidelines: a method for the identification of relevant ontological terms.

Clinical Practice Guidelines (CPGs) contain recommendations intended to optimize patient care, produced based on a systematic review of evidence. In turn, Computer-Interpretable Guidelines (CIGs) are formalized versions of CPGs for use as decision-support systems. We consider the enrichment of the CIG by means of an OWL ontology that describes the clinical domain of the CIG, which could be exploited e.g. for the interoperability with the Electronic Health Record (EHR). As a first step, in this paper we describe a method to support the development of such an ontology starting from a CIG. The method uses an alignment algorithm for the automated identification of ontological terms relevant to the clinical domain of the CIG, as well as a web platform to manually review the alignments and select the appropriate ones. Finally, we present the results of the application of the method to a small corpus of CIGs.

Using SNOMED CT Expression Constraints to Bridge the Gap Between Clinical Decision-Support Systems and Electronic Health Records.

Clinical decision-support systems (CDSSs) should be able to interact with the electronic health record (EHR) to obtain the patient data they require. A recent solution for the interoperability of CDSSs and EHR systems consists in the use of a mediated schema which provides a unified view of their two schemas. The use of such a mediated schema requires the definition of a mapping between this schema and the EHR one. In this paper we investigate the use of the SNOMED CT Expression Constraint Language to characterize these mappings.

A platform for exploration into chaining of web services for clinical data transformation and reasoning.

The heterogeneity of clinical data is a key problem in the sharing and reuse of Electronic Health Record (EHR) data. We approach this problem through the combined use of EHR standards and semantic web technologies, concretely by means of clinical data transformation applications that convert EHR data in proprietary format, first into clinical information models based on archetypes, and then into RDF/OWL extracts which can be used for automated reasoning. In this paper we describe a proof-of-concept platform to facilitate the (re)configuration of such clinical data transformation applications. The platform is built upon a number of web services dealing with transformations at different levels (such as normalization or abstraction), and relies on a collection of reusable mappings designed to solve specific transformation steps in a particular clinical domain. The platform has been used in the development of two different data transformation applications in the area of colorectal cancer.

Leveraging electronic healthcare record standards and semantic web technologies for the identification of patient cohorts.

BACKGROUND: The secondary use of electronic healthcare records (EHRs) often requires the identification of patient cohorts. In this context, an important problem is the heterogeneity of clinical data sources, which can be overcome with the combined use of standardized information models, virtual health records, and semantic technologies, since each of them contributes to solving aspects related to the semantic interoperability of EHR data. OBJECTIVE: To develop methods allowing for a direct use of EHR data for the identification of patient cohorts leveraging current EHR standards and semantic web technologies. MATERIALS AND METHODS: We propose to take advantage of the best features of working with EHR standards and ontologies. Our proposal is based on our previous results and experience working with both technological infrastructures. Our main principle is to perform each activity at the abstraction level with the most appropriate technology available. This means that part of the processing will be performed using archetypes (ie, data level) and the rest using ontologies (ie, knowledge level). Our approach will start working with EHR data in proprietary format, which will be first normalized and elaborated using EHR standards and then transformed into a semantic representation, which will be exploited by automated reasoning. RESULTS: We have applied our approach to protocols for colorectal cancer screening. The results comprise the archetypes, ontologies, and datasets developed for the standardization and semantic analysis of EHR data. Anonymized real data have been used and the patients have been successfully classified by the risk of developing colorectal cancer. CONCLUSIONS: This work provides new insights in how archetypes and ontologies can be effectively combined for EHR-driven phenotyping. The methodological approach can be applied to other problems provided that suitable archetypes, ontologies, and classification rules can be designed.

Interoperability of clinical decision-support systems and electronic health records using archetypes: a case study in clinical trial eligibility.

Clinical decision-support systems (CDSSs) comprise systems as diverse as sophisticated platforms to store and manage clinical data, tools to alert clinicians of problematic situations, or decision-making tools to assist clinicians. Irrespective of the kind of decision-support task CDSSs should be smoothly integrated within the clinical information system, interacting with other components, in particular with the electronic health record (EHR). However, despite decades of developments, most CDSSs lack interoperability features. We deal with the interoperability problem of CDSSs and EHRs by exploiting the dual-model methodology. This methodology distinguishes a reference model and archetypes. A reference model is represented by a stable and small object-oriented model that describes the generic properties of health record information. For their part, archetypes are reusable and domain-specific definitions of clinical concepts in the form of structured and constrained combinations of the entities of the reference model. We rely on archetypes to make the CDSS compatible with EHRs from different institutions. Concretely, we use archetypes for modelling the clinical concepts that the CDSS requires, in conjunction with a series of knowledge-intensive mappings relating the archetypes to the data sources (EHR and/or other archetypes) they depend on. We introduce a comprehensive approach, including a set of tools as well as methodological guidelines, to deal with the interoperability of CDSSs and EHRs based on archetypes. Archetypes are used to build a conceptual layer of the kind of a virtual health record (VHR) over the EHR whose contents need to be integrated and used in the CDSS, associating them with structural and terminology-based semantics. Subsequently, the archetypes are mapped to the EHR by means of an expressive mapping language and specific-purpose tools. We also describe a case study where the tools and methodology have been employed in a CDSS to support patient recruitment in the framework of a clinical trial for colorectal cancer screening. The utilisation of archetypes not only has proved satisfactory to achieve interoperability between CDSSs and EHRs but also offers various advantages, in particular from a data model perspective. First, the VHR/data models we work with are of a high level of abstraction and can incorporate semantic descriptions. Second, archetypes can potentially deal with different EHR architectures, due to their deliberate independence of the reference model. Third, the archetype instances we obtain are valid instances of the underlying reference model, which would enable e.g. feeding back the EHR with data derived by abstraction mechanisms. Lastly, the medical and technical validity of archetype models would be assured, since in principle clinicians should be the main actors in their development.

A constraint-based approach to medical guidelines and protocols.

Medical guidelines and protocols are documents aimed at improving the quality of medical care by offering support in medical decision making in the form of management recommendations based on scientific evidence. Whereas medical guidelines are intended for nation-wide use, and thus omit medical management details that may differ among hospitals, medical protocols are aimed at local use, e.g., within hospitals, and, therefore, include more detailed information. Although a medical guideline and an associated protocol concerning the management of a particular disorder are related to each other, one question is to what extent they are different. Formal methods are applied to shed light on this issue. A Dutch medical guideline regarding the treatment of breast cancer, and a Dutch protocol based on it, are taken as an example.