Leveraging the EHR4CR platform to support patient inclusion in academic studies: challenges and lessons learned.

BACKGROUND: The development of Electronic Health Records (EHRs) in hospitals offers the ability to reuse data from patient care activities for clinical research. EHR4CR is a European public-private partnership aiming to develop a computerized platform that enables the re-use of data collected from EHRs over its network. However, the reproducibility of queries may depend on attributes of the local data. Our objective was 1/ to describe the different steps that were achieved in order to use the EHR4CR platform and 2/ to identify the specific issues that could impact the final performance of the platform. METHODS: We selected three institutional studies covering various medical domains. The studies included a total of 67 inclusion and exclusion criteria and ran in two University Hospitals. We described the steps required to use the EHR4CR platform for a feasibility study. We also defined metrics to assess each of the steps (including criteria complexity, normalization quality, and data completeness of EHRs). RESULTS: We identified 114 distinct medical concepts from a total of 67 eligibility criteria Among the 114 concepts: 23 (20.2%) corresponded to non-structured data (i.e. for which transformation is needed before analysis), 92 (81%) could be mapped to terminologies used in EHR4CR, and 86 (75%) could be mapped to local terminologies. We identified 51 computable criteria following the normalization process. The normalization was considered by experts to be satisfactory or higher for 64.2% (43/67) of the computable criteria. All of the computable criteria could be expressed using the EHR4CR platform. CONCLUSIONS: We identified a set of issues that could affect the future results of the platform: (a) the normalization of free-text criteria, (b) the translation into computer-friendly criteria and (c) issues related to the execution of the query to clinical data warehouses. We developed and evaluated metrics to better describe the platforms and their result. These metrics could be used for future reports of Clinical Trial Recruitment Support Systems assessment studies, and provide experts and readers with tools to insure the quality of constructed dataset.

A Fast Healthcare Interoperability Resources (FHIR) layer implemented over i2b2.

BACKGROUND: Standards and technical specifications have been developed to define how the information contained in Electronic Health Records (EHRs) should be structured, semantically described, and communicated. Current trends rely on differentiating the representation of data instances from the definition of clinical information models. The dual model approach, which combines a reference model (RM) and a clinical information model (CIM), sets in practice this software design pattern. The most recent initiative, proposed by HL7, is called Fast Health Interoperability Resources (FHIR). The aim of our study was to investigate the feasibility of applying the FHIR standard to modeling and exposing EHR data of the Georges Pompidou European Hospital (HEGP) integrating biology and the bedside (i2b2) clinical data warehouse (CDW). RESULTS: We implemented a FHIR server over i2b2 to expose EHR data in relation with five FHIR resources: DiagnosisReport, MedicationOrder, Patient, Encounter, and Medication. The architecture of the server combines a Data Access Object design pattern and FHIR resource providers, implemented using the Java HAPI FHIR API. Two types of queries were tested: query type #1 requests the server to display DiagnosticReport resources, for which the diagnosis code is equal to a given ICD-10 code. A total of 80 DiagnosticReport resources, corresponding to 36 patients, were displayed. Query type #2, requests the server to display MedicationOrder, for which the FHIR Medication identification code is equal to a given code expressed in a French coding system. A total of 503 MedicationOrder resources, corresponding to 290 patients, were displayed. Results were validated by manually comparing the results of each request to the results displayed by an ad-hoc SQL query. CONCLUSION: We showed the feasibility of implementing a Java layer over the i2b2 database model to expose data of the CDW as a set of FHIR resources. An important part of this work was the structural and semantic mapping between the i2b2 model and the FHIR RM. To accomplish this, developers must manually browse the specifications of the FHIR standard. Our source code is freely available and can be adapted for use in other i2b2 sites.

Common data elements for secondary use of electronic health record data for clinical trial execution and serious adverse event reporting.

BACKGROUND: Data capture is one of the most expensive phases during the conduct of a clinical trial and the increasing use of electronic health records (EHR) offers significant savings to clinical research. To facilitate these secondary uses of routinely collected patient data, it is beneficial to know what data elements are captured in clinical trials. Therefore our aim here is to determine the most commonly used data elements in clinical trials and their availability in hospital EHR systems. METHODS: Case report forms for 23 clinical trials in differing disease areas were analyzed. Through an iterative and consensus-based process of medical informatics professionals from academia and trial experts from the European pharmaceutical industry, data elements were compiled for all disease areas and with special focus on the reporting of adverse events. Afterwards, data elements were identified and statistics acquired from hospital sites providing data to the EHR4CR project. RESULTS: The analysis identified 133 unique data elements. Fifty elements were congruent with a published data inventory for patient recruitment and 83 new elements were identified for clinical trial execution, including adverse event reporting. Demographic and laboratory elements lead the list of available elements in hospitals EHR systems. For the reporting of serious adverse events only very few elements could be identified in the patient records. CONCLUSIONS: Common data elements in clinical trials have been identified and their availability in hospital systems elucidated. Several elements, often those related to reimbursement, are frequently available whereas more specialized elements are ranked at the bottom of the data inventory list. Hospitals that want to obtain the benefits of reusing data for research from their EHR are now able to prioritize their efforts based on this common data element list.

Efficiency and effectiveness evaluation of an automated multi-country patient count cohort system.

BACKGROUND: With the increase of clinical trial costs during the last decades, the design of feasibility studies has become an essential process to reduce avoidable and costly protocol amendments. This design includes timelines, targeted sites and budget, together with a list of eligibility criteria that potential participants need to match. The present work was designed to assess the value of obtaining potential study participant counts using an automated patient count cohort system for large multi-country and multi-site trials: the Electronic Health Records for Clinical Research (EHR4CR) system. METHODS: The evaluation focuses on the accuracy of the patient counts and the time invested to obtain these using the EHR4CR platform compared to the current questionnaire based process. This evaluation will assess the patient counts from ten clinical trials at two different sites. In order to assess the accuracy of the results, the numbers obtained following the two processes need to be compared to a baseline number, the "alloyed" gold standard, which was produced by a manual check of patient records. RESULTS: The patient counts obtained using the EHR4CR system were in three evaluated trials more accurate than the ones obtained following the current process whereas in six other trials the current process counts were more accurate. In two of the trials both of the processes had counts within the gold standard's confidence interval. In terms of efficiency the EHR4CR protocol feasibility system proved to save approximately seven calendar days in the process of obtaining patient counts compared to the current manual process. CONCLUSIONS: At the current stage, electronic health record data sources need to be enhanced with better structured data so that these can be re-used for research purposes. With this kind of data, systems such as the EHR4CR are able to provide accurate objective patient counts in a more efficient way than the current methods. Additional research using both structured and unstructured data search technology is needed to assess the value of unstructured data and to compare the amount of efforts needed for data preparation.

Effects of automated alerts on unnecessarily repeated serology tests in a cardiovascular surgery department: a time series analysis.

BACKGROUND: Laboratory testing is frequently unnecessary, particularly repetitive testing. Among the interventions proposed to reduce unnecessary testing, Computerized Decision Support Systems (CDSS) have been shown to be effective, but their impact depends on their technical characteristics. The objective of the study was to evaluate the impact of a Serology-CDSS providing point of care reminders of previous existing serology results, embedded in a Computerized Physician Order Entry at a university teaching hospital in Paris, France. METHODS: A CDSS was implemented in the Cardiovascular Surgery department of the hospital in order to decrease inappropriate repetitions of viral serology tests (HBV).A time series analysis was performed to assess the impact of the alert on physicians' practices. The study took place between January 2004 and December 2007. The primary outcome was the proportion of unnecessarily repeated HBs antigen tests over the periods of the study. A test was considered unnecessary when it was ordered within 90 days after a previous test for the same patient. A secondary outcome was the proportion of potentially unnecessary HBs antigen test orders cancelled after an alert display. RESULTS: In the pre-intervention period, 3,480 viral serology tests were ordered, of which 538 (15.5%) were unnecessarily repeated. During the intervention period, of the 2,095 HBs antigen tests performed, 330 unnecessary repetitions (15.8%) were observed. Before the intervention, the mean proportion of unnecessarily repeated HBs antigen tests increased by 0.4% per month (absolute increase, 95% CI 0.2% to 0.6%, p < 0.001). After the intervention, a significant trend change occurred, with a monthly difference estimated at -0.4% (95% CI -0.7% to -0.1%, p = 0.02) resulting in a stable proportion of unnecessarily repeated HBs antigen tests. A total of 380 unnecessary tests were ordered among 500 alerts displayed (compliance rate 24%). CONCLUSIONS: The proportion of unnecessarily repeated tests immediately dropped after CDSS implementation and remained stable, contrasting with the significant continuous increase observed before. The compliance rate confirmed the effect of the alerts. It is necessary to continue experimentation with dedicated systems in order to improve understanding of the diversity of CDSS and their impact on clinical practice.

Methodology of integration of a clinical data warehouse with a clinical information system: the HEGP case.

Clinical Data Warehouses (CDW) can complement current Clinical Information Systems (CIS) with functions that are not easily implemented by traditional operational database systems. Here, we describe the design and deployment strategy used at the Pompidou University Hospital in southwest Paris. Four realms are described: technological realm, data realm, restitution realm, and administration realm. The corresponding UML use cases and the mapping rules from the shared integrated electronic health records to the five axes of the i2b2 CDW star model are presented. Priority is given to the anonymization and security principles used for the 1.2 million patient records currently stored in the CDW. Exploitation of a CDW by clinicians and investigators can facilitate clinical research, quality evaluations and outcome studies. These indirect benefits are among the reasons for the continuous use of an integrated CIS.

Effect of alerts for drug dosage adjustment in inpatients with renal insufficiency.

OBJECTIVES: Medication errors constitute a major problem in all hospitals. Between 20% and 46% of prescriptions requiring dosage adjustments based on renal function are inappropriate. This study aimed to determine whether implementing alerts at the time of ordering medication integrated into the computerized physician order entry decreases the proportion of inappropriate prescriptions based on the renal function of inpatients. DESIGN: Six alternating 2-month control and intervention periods were conducted between August 2006 and August 2007 in two medical departments of a teaching hospital in France. A total of 603 patients and 38 physicians were included. During the intervention periods, alerts were triggered if a patient with renal impairment was prescribed one of the 24 targeted drugs that required adjustment according to estimated glomerular filtration rate (eGFR). MEASUREMENTS: The main outcome measure was the proportion of inappropriate first prescriptions, according to recommendation. RESULTS: A total of 1,122 alerts were triggered. The rate of inappropriate first prescriptions did not differ significantly between intervention and control periods (19.9% vs. 21.3%; p=0.63). The effect of intervention differed significantly between residents and senior physicians (p=0.03). Residents tended to make fewer errors in intervention versus control periods (Odds ratio 0.69; 95% confidence interval 0.41 to 1.15), whereas senior physicians tended to make more inappropriate prescriptions in intervention periods (odds ratio 1.88; 95% confidence interval 0.91 to 3.89). CONCLUSION: Alert activation was not followed by a significant decrease in inappropriate prescriptions in our study. Thus, it is still necessary to evaluate the impact of these systems if newly implemented in other settings thanks to studies also designed to watch for possible unanticipated effects of decision supports and their underlying causes.

Integrating Multimodal Radiation Therapy Data into i2b2.

BACKGROUND: Clinical data warehouses are now widely used to foster clinical and translational research and the Informatics for Integrating Biology and the Bedside (i2b2) platform has become a de facto standard for storing clinical data in many projects. However, to design predictive models and assist in personalized treatment planning in cancer or radiation oncology, all available patient data need to be integrated into i2b2, including radiation therapy data that are currently not addressed in many existing i2b2 sites. OBJECTIVE: To use radiation therapy data in projects related to rectal cancer patients, we assessed the feasibility of integrating radiation oncology data into the i2b2 platform. METHODS: The Georges Pompidou European Hospital, a hospital from the Assistance Publique - Hôpitaux de Paris group, has developed an i2b2-based clinical data warehouse of various structured and unstructured clinical data for research since 2008. To store and reuse various radiation therapy data-dose details, activities scheduling, and dose-volume histogram (DVH) curves-in this repository, we first extracted raw data by using some reverse engineering techniques and a vendor's application programming interface. Then, we implemented a hybrid storage approach by combining the standard i2b2 "Entity-Attribute-Value" storage mechanism with a "JavaScript Object Notation (JSON) document-based" storage mechanism without modifying the i2b2 core tables. Validation was performed using (1) the Business Objects framework for replicating vendor's application screens showing dose details and activities scheduling data and (2) the R software for displaying the DVH curves. RESULTS: We developed a pipeline to integrate the radiation therapy data into the Georges Pompidou European Hospital i2b2 instance and evaluated it on a cohort of 262 patients. We were able to use the radiation therapy data on a preliminary use case by fetching the DVH curve data from the clinical data warehouse and displaying them in a R chart. CONCLUSION: By adding radiation therapy data into the clinical data warehouse, we were able to analyze radiation therapy response in cancer patients and we have leveraged the i2b2 platform to store radiation therapy data, including detailed information such as the DVH to create new ontology-based modules that provides research investigators with a wider spectrum of clinical data.

Labeling for Big Data in radiation oncology: The Radiation Oncology Structures ontology.

PURPOSE: Leveraging Electronic Health Records (EHR) and Oncology Information Systems (OIS) has great potential to generate hypotheses for cancer treatment, since they directly provide medical data on a large scale. In order to gather a significant amount of patients with a high level of clinical details, multicenter studies are necessary. A challenge in creating high quality Big Data studies involving several treatment centers is the lack of semantic interoperability between data sources. We present the ontology we developed to address this issue. METHODS: Radiation Oncology anatomical and target volumes were categorized in anatomical and treatment planning classes. International delineation guidelines specific to radiation oncology were used for lymph nodes areas and target volumes. Hierarchical classes were created to generate The Radiation Oncology Structures (ROS) Ontology. The ROS was then applied to the data from our institution. RESULTS: Four hundred and seventeen classes were created with a maximum of 14 children classes (average = 5). The ontology was then converted into a Web Ontology Language (.owl) format and made available online on Bioportal and GitHub under an Apache 2.0 License. We extracted all structures delineated in our department since the opening in 2001. 20,758 structures were exported from our "record-and-verify" system, demonstrating a significant heterogeneity within a single center. All structures were matched to the ROS ontology before integration into our clinical data warehouse (CDW). CONCLUSION: In this study we describe a new ontology, specific to radiation oncology, that reports all anatomical and treatment planning structures that can be delineated. This ontology will be used to integrate dosimetric data in the Assistance Publique-Hôpitaux de Paris CDW that stores data from 6.5 million patients (as of February 2017).

The Georges Pompidou University Hospital Clinical Data Warehouse: A 8-years follow-up experience.

BACKGROUND: When developed jointly with clinical information systems, clinical data warehouses (CDWs) facilitate the reuse of healthcare data and leverage clinical research. OBJECTIVE: To describe both data access and use for clinical research, epidemiology and health service research of the "Hôpital Européen Georges Pompidou" (HEGP) CDW. METHODS: The CDW has been developed since 2008 using an i2b2 platform. It was made available to health professionals and researchers in October 2010. Procedures to access data have been implemented and different access levels have been distinguished according to the nature of queries. RESULTS: As of July 2016, the CDW contained the consolidated data of over 860,000 patients followed since the opening of the HEGP hospital in July 2000. These data correspond to more than 122 million clinical item values, 124 million biological item values, and 3.7 million free text reports. The ethics committee of the hospital evaluates all CDW projects that generate secondary data marts. Characteristics of the 74 research projects validated between January 2011 and December 2015 are described. CONCLUSION: The use of HEGP CDWs is a key facilitator for clinical research studies. It required however important methodological and organizational support efforts from a biomedical informatics department.

Computation of semantic similarity within an ontology of breast pathology to assist inter-observer consensus.

Computer-assisted consensus in medical imaging involves automatic comparison of morphological abnormalities observed by physicians in images. We built an ontology of morphological abnormalities in breast pathology to assist inter-observer consensus. Concepts of morphological abnormalities extracted from existing terminologies, published grading systems and medical reports were organized in an taxonomic hierarchy and furthermore linked by the relation "is a diagnostic criterion of" according to diagnostic meaning. We implemented position-based, content-based and mixed semantic similarity measures between concepts in this ontology and compared the results with experts' judgment. The position-based similarity measure using both taxonomic and non-taxonomic relations performed as well as the other measures and was used for automatic comparison of morphological abnormalities within the IDEM computer-assisted consensus platform.

Cross border semantic interoperability for clinical research: the EHR4CR semantic resources and services.

With the development of platforms enabling the use of routinely collected clinical data in the context of international clinical research, scalable solutions for cross border semantic interoperability need to be developed. Within the context of the IMI EHR4CR project, we first defined the requirements and evaluation criteria of the EHR4CR semantic interoperability platform and then developed the semantic resources and supportive services and tooling to assist hospital sites in standardizing their data for allowing the execution of the project use cases. The experience gained from the evaluation of the EHR4CR platform accessing to semantically equivalent data elements across 11 European participating EHR systems from 5 countries demonstrated how far the mediation model and mapping efforts met the expected requirements of the project. Developers of semantic interoperability platforms are beginning to address a core set of requirements in order to reach the goal of developing cross border semantic integration of data.

Reviewing 741 patients records in two hours with FASTVISU.

The secondary use of electronic health records opens up new perspectives. They provide researchers with structured data and unstructured data, including free text reports. Many applications been developed to leverage knowledge from free-text reports, but manual review of documents is still a complex process. We developed FASTVISU a web-based application to assist clinicians in reviewing documents. We used FASTVISU to review a set of 6340 documents from 741 patients suffering from the celiac disease. A first automated selection pruned the original set to 847 documents from 276 patients' records. The records were reviewed by two trained physicians to identify the presence of 15 auto-immune diseases. It took respectively two hours and two hours and a half to evaluate the entire corpus. Inter-annotator agreement was high (Cohen's kappa at 0.89). FASTVISU is a user-friendly modular solution to validate entities extracted by NLP methods from free-text documents stored in clinical data warehouses.

A collaborative platform for consensus sessions in pathology over Internet.

The design of valid databases in pathology faces the problem of diagnostic disagreement between pathologists. Organizing consensus sessions between experts to reduce the variability is a difficult task. The TRIDEM platform addresses the issue to organize consensus sessions in pathology over the Internet. In this paper, we present the basis to achieve such collaborative platform. On the one hand, the platform integrates the functionalities of the IDEM consensus module that alleviates the consensus task by presenting to pathologists preliminary computed consensus through ergonomic interfaces (automatic step). On the other hand, a set of lightweight interaction tools such as vocal annotations are implemented to ease the communication between experts as they discuss a case (interactive step). The architecture of the TRIDEM platform is based on a Java-Server-Page web server that communicate with the ObjectStore PSE/PRO database used for the object storage. The HTML pages generated by the web server run Java applets to perform the different steps (automatic and interactive) of the consensus. The current limitations of the platform is to only handle a synchronous process. Moreover, improvements like re-writing the consensus workflow with a protocol such as BPML are already forecast.

Specifications and implementation of a new exchange format to support computerized consensus in pathology.

In the pathology domain, consensus sessions around multi-headed microscopes enhance reproducibility and can reduce inter- and intra-observer variability. Computerized tools and Web technology could facilitate the organization of consensus sessions and assist pathologists to agree on features that are relevant to diagnosis. In the context of the IDEM project, whose aim is to achieve a computerized platform to allow pathologists to derive consensual diagnostic during Internet-based collaborative sessions, we propose a new extension of the existing TELESLIDE format. This new extended format enables the storage and the exchange of multi-experts descriptions that will be processed by the IDEM consensus engine to produce consensual descriptions. We describe this new format and its implementation in the IDEM teleconsensus platform.

A clinical data warehouse-based process for refining medication orders alerts.

The objective of this case report is to evaluate the use of a clinical data warehouse coupled with a clinical information system to test and refine alerts for medication orders control before they were fully implemented. A clinical decision rule refinement process was used to assess alerts. The criteria assessed were the frequencies of alerts for initial prescriptions of 10 medications whose dosage levels depend on renal function thresholds. In the first iteration of the process, the frequency of the 'exceeds maximum daily dose' alerts was 7.10% (617/8692), while that of the 'under dose' alerts was 3.14% (273/8692). Indicators were presented to the experts. During the different iterations of the process, 45 (16.07%) decision rules were removed, 105 (37.5%) were changed and 136 new rules were introduced. Extensive retrospective analysis of physicians' medication orders stored in a clinical data warehouse facilitates alert optimization toward the goal of maximizing the safety of the patient and minimizing overridden alerts.

Integration of multiple ontologies in breast cancer pathology.

The diagnostic variability in pathology, widely reported in the literature, is partly due to the use of different classification systems by pathologists. The descriptions of morphological characteristics on the same image within different classification systems can be considered as different points of view of pathologists. Our aim is to represent the points of view of the experts in pathology during image interpretation and to propose a method ological and technical solution in order to implement interoperability between these points of view. According to the hybrid ontology approach, we developed a system in three stages consisting in 1) the representation of the various points of view in local ontologies 2) the realization of a shared vocabulary and the development of a mapping tool used to allow the matching of local ontologies and shared vocabulary 3) the development of a transcoding algorithm for the translation of a case description from one point of view to another. A first evaluation of the transcoding algorithm was conducted for 33 cases of breast pathology. Our results show that the pathologists generally produce descriptions of the cases which do not follow rigorously the interpretation rules corresponding to the point of view they assert to adopt. While most of the concepts of local ontologies can be transcoded from a local ontology to another one (varying from 62.5 % to 100% according to the local ontology), the transcoding of a description which is valid according to a certain point of view, often results in a description which is not rigorously in accordance with the new point of view. These results underline the differences of interpretation rules existing in the different points of view.

A customizable similarity measure between histological cases.

IDEM, a computerized environment dedicated to pathologists, includes a Case Based Reasoning (CBR) procedure to retrieve similar histological cases in the database. The relevancy of a retrieved case strongly depends on the similarity measure comparing case descriptions. The present work deals with the definition of a similarity measure in the context of IDEM. In a first step, a theoretical measure (relational, numerical and informed), based on the domain constraints, was selected. In a second step, the theoretical measure is optimized according to the current case base. Results are presented for a database of 53 cases of breast tumors. The contribution of this work is to give to pathologists an interactive environment that optimizes the similarity measure between histological cases. This work is also a contribution to the CBR cycle life since the similarity measure can be adapted while new cases are added to the base.