External validation of the unified model of information systems continuance (UMISC): An international comparison.

CONTEXT: The Unified Model of Information Systems Continuance (UMISC) is a metamodel for the evaluation of clinical information systems (CISs) that integrates constructs from five models that have previously been published in the literature. UMISC was developed at the Georges Pompidou University Hospital (HEGP) in Paris and was partially validated at the Saint Joseph Hospital Group (HPSJ), another acute care institution using the same CIS as HEGP. OBJECTIVE: The aim of this replication study was twofold: (1) to perform an external validation of UMISC in two different hospitals and country contexts: the Italian Hospital of Buenos Aires (HIBA) in Argentina and the Hospital Sirio Libanes in Sao Paulo, Brazil (HSL); (2) to compare, using the same evaluation model, the determinants of satisfaction, use, and continuance intention observed at HIBA and HSL with those previously observed at HEGP and HPSJ. METHODS: The UMISC evaluation questionnaires were translated from their original languages (English and French) to Brazilian Portuguese and Spanish following the translation/back-translation method. These questionnaires were then applied at each target site. The 21 UMISC-associated hypotheses were tested using structural equation modeling (SEM). RESULTS: A total of 3020 users, 1079 at HIBA and 1941 at the HSL, were included in the analysis. The respondents included 1406 medical staff and 1001 nursing staff. The average profession-adjusted use, overall satisfaction and continuance intention were significantly higher at HIBA than at HSL in the medical and nursing groups. In SEM analysis, UMISC explained 23% and 11% of the CIS use dimension, 72% and 85% of health professionals' satisfaction, and 41% and 60% of continuance intention at HIBA and HSL, respectively. Twenty of the 21 UMISC-related hypotheses were validated in at least one of the four evaluation sites, and 16 were validated in two or more sites. CONCLUSION: The UMISC evaluation metamodel appears to be a robust comparison and explanatory model of satisfaction, use and continuance intention for CISs in late post adoption situations.

Designing a Coding Data Warehouse to Support and Enhance Medical Act Coding.

Standards Data Warehouse has been implemented in many hospitals. It has enormous potential to improve performance measurement and health care quality. Accessing, organizing, and using these data to optimize clinical coding are evolving challenges for hospital systems. This paper describes development of a coding data warehouse based Entities-Attribute-Value (EAV) that we created by importing data from the clinical data warehouse (CDW) of public hospital. In particular, it focuses on design, implementation, and evaluation of the warehouse. Moreover, it defines the rules to convert a conceptual model of coding into an EAV logical model and his implementation using integrating biology and the bedside (i2b2). We evaluate it using data research mono and multi-criteria and then calculate the precision of our model. The result shows that, the coding data warehouse provides with good accuracy, an association of diagnostic code and medical act closer the patient's clinical landscape. Doctors without knowledge of coding rules could use this information to optimize and improve the diagnostic coding.

Evaluation of the unified model of information systems continuance (UMISC) in two hospital environments.

CONTEXT: The deployment and long-term acceptance of clinical information systems (CISs) are faced with multiple difficulties. They include insufficient quality of the systems in place and resistance to the multiple changes they induce in care processes. Permanent evaluation of deployed solutions is a prerequisite to their continuous improvement. OBJECTIVE: The purpose of this study was twofold: (1) To validate the post-adoption unified model of information systems continuance (UMISC) progressively developed at the Georges Pompidou University Hospital (HEGP) in Paris (internal validation); and (2) To compare, using the same evaluation model, the results observed at HEGP with those of the Saint-Joseph Hospital Group (HPSJ), another Paris acute care institution (external validation). METHODS: The UMISC post-adoption model is built around nine dimensions: end-user characteristics, social norm (SN), IS quality (ISQ), facilitating conditions (FC), perceived usefulness (PU), confirmation of expectations (CE), profession-adjusted use (PAU), satisfaction (SAT), and continuance intention (CI). Two semi-quantitative evaluation surveys were performed at HEGP in 2014 and 2015, and one at HPSJ in 2015. Statistical analysis included multiple regression analysis and structural equation modeling (SEM). RESULTS: The analysis concerned 459 responders, 264 at HEGP and 195 at HPSJ. UMISC indicators, with the exception of SN, are superior at HEGP than at HPSJ, which had a shorter CIS anteriority than HEGP. In SEM analysis, the UMISC model explained 25% and 40% of the CIS use, 92% and 93% of health professionals' satisfaction, and 72% and 71% of continuance intention at HEGP and HPSJ, respectively. Seventeen of the 21 tested UMISC hypotheses were supported in at least one of the two sites. CONCLUSION: The UMISC evaluation model can be used as a comparison and explanatory model of CIS use, satisfaction and continuance intention in post-CIS adoption situations that become prevalent in current electronic hospitals.

Design and Implementation of Big Data-Based Documents to Optimize Medical Coding.

Clinical information systems (CISs) in some hospitals streamline the data management from data warehouses. These warehouses contain heterogeneous information from all medical specialties that offer patient care services. It is increasingly difficult to manage large volumes of data in a specific clinical context such as quality coding of medical services. The document-based Not Only SQL (NO-SQL) model can provide an accessible, extensive and robust coding data management framework while maintaining certain flexibility. This paper focus on the design and implementation of a big data-coding warehouse, it also defines the rules to convert a conceptual model of coding into a document-oriented logical model. Using that model, we implemented, analyzed a big data-coding warehouse via the Mongodb database, and evaluated it using data research mono- and multi-criteria and then calculated the precision of our model.

The Georges Pompidou University Hospital Clinical Data Warehouse: A 8-years follow-up experience.

BACKGROUND: When developed jointly with clinical information systems, clinical data warehouses (CDWs) facilitate the reuse of healthcare data and leverage clinical research. OBJECTIVE: To describe both data access and use for clinical research, epidemiology and health service research of the "Hôpital Européen Georges Pompidou" (HEGP) CDW. METHODS: The CDW has been developed since 2008 using an i2b2 platform. It was made available to health professionals and researchers in October 2010. Procedures to access data have been implemented and different access levels have been distinguished according to the nature of queries. RESULTS: As of July 2016, the CDW contained the consolidated data of over 860,000 patients followed since the opening of the HEGP hospital in July 2000. These data correspond to more than 122 million clinical item values, 124 million biological item values, and 3.7 million free text reports. The ethics committee of the hospital evaluates all CDW projects that generate secondary data marts. Characteristics of the 74 research projects validated between January 2011 and December 2015 are described. CONCLUSION: The use of HEGP CDWs is a key facilitator for clinical research studies. It required however important methodological and organizational support efforts from a biomedical informatics department.

Constructing a Pre-Emptive System Based on a Multidimentional Matrix and Autocompletion to Improve Diagnostic Coding in Acute Care Hospitals.

Short-stay MSO (Medicine, Surgery, Obstetrics) hospitalization activities in public and private hospitals providing public services are funded through charges for the services provided (T2A in French). Coding must be well matched to the severity of the patient's condition, to ensure that appropriate funding is provided to the hospital. We propose the use of an autocompletion process and multidimensional matrix, to help physicians to improve the expression of information and to optimize clinical coding. With this approach, physicians without knowledge of the encoding rules begin from a rough concept, which is gradually refined through semantic proximity and uses information on the associated codes stemming of optimized knowledge bases of diagnosis code.

Information system end-user satisfaction and continuance intention: A unified modeling approach.

CONTEXT: Permanent evaluation of end-user satisfaction and continuance intention is a critical issue at each phase of a clinical information system (CIS) project, but most validation studies are concerned with the pre- or early post-adoption phases. OBJECTIVE: The purpose of this study was twofold: to validate at the Pompidou University Hospital (HEGP) an information technology late post-adoption model built from four validated models and to propose a unified metamodel of evaluation that could be adapted to each context or deployment phase of a CIS project. METHODS: Five dimensions, i.e., CIS quality (CISQ), perceived usefulness (PU), confirmation of expectations (CE), user satisfaction (SAT), and continuance intention (CI) were selected to constitute the CI evaluation model. The validity of the model was tested using the combined answers to four surveys performed between 2011 and 2015, i.e., more than ten years after the opening of HEGP in July 2000. Structural equation modeling was used to test the eight model-associated hypotheses. RESULTS: The multi-professional study group of 571 responders consisted of 158 doctors, 282 nurses, and 131 secretaries. The evaluation model accounted for 84% of variance of satisfaction and 53% of CI variance for the period 2011-2015 and for 92% and 69% for the period 2014-2015. In very late post adoption, CISQ appears to be the major determinant of satisfaction and CI. Combining the results obtained at various phases of CIS deployment, a Unified Model of Information System Continuance (UMISC) is proposed. CONCLUSION: In a meaningful CIS use situation at HEGP, this study confirms the importance of CISQ in explaining satisfaction and CI. The proposed UMISC model that can be adapted to each phase of CIS deployment could facilitate the necessary efforts of permanent CIS acceptance and continuance evaluation.

14 Years longitudinal evaluation of clinical information systems acceptance: The HEGP case.

CONTEXT: Meaningful use and end-user satisfaction are two major components of the success of a clinical information system (CIS). The purpose of this study was to longitudinally measure and analyze the CIS use and satisfaction determinants in a multi-professional group at the Georges Pompidou university hospital (HEGP) in Paris. METHODS: From the different evaluation surveys performed at HEGP, three periods were considered corresponding to 4, 8 and over 10 years after the first CIS deployment in 2000, respectively. Six acceptance dimensions were considered: CIS quality (CISQ), facilitating conditions (FC), perceived usefulness (PU), confirmation of expectations (CE), use, and global satisfaction (GS). Relationships between these constructs were tested through multiple regressions analysis and structural equation modeling (SEM). RESULTS: Responses were obtained from 298, 332, and 448 users for the three periods considered. CIS acceptance dimensions progressively and significantly increased over time. Significant differences between professions were observed with an initial low PU among medical staff. In the early deployment phase, GS appeared to be determined by CIS use, CISQ and PU (R(2)=.53 in SEM). In the very late post-adoption phase, GS was strongly determined by CISQ, CE, and PU (R(2)=.86 in SEM) and was no longer associated with CIS use. CONCLUSION: Acceptance models should be adapted to the phase of deployment of a CIS and integrate end-users' individual characteristics. Progressive reduction over time of the positive relationships between CIS use and satisfaction could possibly be considered as a maturity indicator of CIS deployment. These observations validate the introduction in post-adoption models of a continuance intention dimension.

Detection of Drug-Drug Interactions Inducing Acute Kidney Injury by Electronic Health Records Mining.

BACKGROUND AND OBJECTIVE: While risk of acute kidney injury (AKI) is a well documented adverse effect of some drugs, few studies have assessed the relationship between drug-drug interactions (DDIs) and AKI. Our objective was to develop an algorithm capable of detecting potential signals on this relationship by retrospectively mining data from electronic health records. MATERIAL AND METHODS: Data were extracted from the clinical data warehouse (CDW) of the Hôpital Européen Georges Pompidou (HEGP). AKI was defined as the first level of the RIFLE criteria, that is, an increase ≥50 % of creatinine basis. Algorithm accuracy was tested on 20 single drugs, 10 nephrotoxic and 10 non-nephrotoxic. We then tested 45 pairs of non-nephrotoxic drugs, among the most prescribed at our hospital and representing distinct pharmacological classes for DDIs. RESULTS: Sensitivity and specificity were 50 % [95 % confidence interval (CI) 23.66-76.34] and 90 % (95 % CI 59.58-98.21), respectively, for single drugs. Our algorithm confirmed a previously identified signal concerning clarithromycin and calcium-channel blockers (unadjusted odds ratio (ORu) 2.92; 95 % CI 1.11-7.69, p = 0.04). Among the 45 drug pairs investigated, we identified a signal concerning 55 patients in association with bromazepam and hydroxyzine (ORu 1.66; 95 % CI 1.23-2.23). This signal was not confirmed after a chart review. Even so, AKI and co-prescription were confirmed for 96 % (95 % CI 88-99) and 88 % (95 % CI 76-94) of these patients, respectively. CONCLUSION: Data mining techniques on CDW can foster the detection of adverse drug reactions when drugs are used alone or in combination.

Using a snowflake data model and autocompletion to support diagnostic coding in acute care hospitals.

PURPOSE: Efficient and adequate coding is essential for all hospitals to optimize funding, follow activity, and perform epidemiological studies. OBJECTIVE: We propose an autocompletion method for optimizing diagnostic coding in acute care hospitals. METHODS: Using a terminology snowflake model integrating SNOMED 3.5 and ICD-10 codes, autocompletion algorithms generate a list of diagnostic expressions from partial input concepts. RESULTS: A general autocompletion component has been developed and tested on a set of inpatient summary reports. Concepts expressed as strings of three or four characters return a noisy list of diagnostic labels or codes. Concepts expressed as groups of strings return lists that are semantically close to the labels present in hospital reports. The most pertinent information lies in the length of the expressions entered. CONCLUSION: Autocompletion can be a complementary tool to existing coding support systems.

Translational research platforms integrating clinical and omics data: a review of publicly available solutions.

The rise of personalized medicine and the availability of high-throughput molecular analyses in the context of clinical care have increased the need for adequate tools for translational researchers to manage and explore these data. We reviewed the biomedical literature for translational platforms allowing the management and exploration of clinical and omics data, and identified seven platforms: BRISK, caTRIP, cBio Cancer Portal, G-DOC, iCOD, iDASH and tranSMART. We analyzed these platforms along seven major axes. (1) The community axis regrouped information regarding initiators and funders of the project, as well as availability status and references. (2) We regrouped under the information content axis the nature of the clinical and omics data handled by each system. (3) The privacy management environment axis encompassed functionalities allowing control over data privacy. (4) In the analysis support axis, we detailed the analytical and statistical tools provided by the platforms. We also explored (5) interoperability support and (6) system requirements. The final axis (7) platform support listed the availability of documentation and installation procedures. A large heterogeneity was observed in regard to the capability to manage phenotype information in addition to omics data, their security and interoperability features. The analytical and visualization features strongly depend on the considered platform. Similarly, the availability of the systems is variable. This review aims at providing the reader with the background to choose the platform best suited to their needs. To conclude, we discuss the desiderata for optimal translational research platforms, in terms of privacy, interoperability and technical features.

Observations and lessons learnt from non health professionals evaluating a health search engine.

This article presents the results of one of the stages of the user-centered evaluation conducted in a framework of the EU project Khresmoi. In a controlled environment, users were asked to perform health-related tasks using a search engine specifically developed for trustworthy online health information. Twenty seven participants from largely the Czech Republic and France took part in the evaluation. All reported overall a positive experience, while some features caused some criticism. Learning points are summed up regarding running such types of evaluations with the general public and specifically with patients.

Determinants of continuance intention in a post-adoption satisfaction evaluation of a clinical information system.

PURPOSE: The evaluation of end-user satisfaction is an essential part of any clinical information system (CIS) project. The purpose of this study is to evaluate the determinants of CIS continuance intention in a late post-adoption phase at the Georges Pompidou University Hospital (HEGP) in Paris. METHODS: We designed an electronic survey instrument based on an IT post-adoption model (ITPAM) developed from three previous models, i.e., the Delone and McLean Information Success Model, the Davis TAM model and the Bhattacherjee information system continuance intention model. RESULTS: 419 questionnaires were collected from CIS users directly involved in patient care. The perceived CIS quality, usefulness and user satisfaction are significantly lower for medical professions than other professional groups. Continuance intention is very high within all professional subgroups. In a multiple regression analysis, the global satisfaction (R(2) = .780) was positively and significantly correlated with CIS quality, confirmation of expectations and perceived CIS usefulness. The continuance intention (R(2) = .392) was positively and significantly correlated with perceived CIS usefulness, confirmation of expectations and global satisfaction. CONCLUSION: In a late post-adoption CIS deployment phase, continuance intention does not significantly depend on individual end user characteristics but is significantly associated with the perceived CIS usefulness, confirmation of expectations and global satisfaction.

Assessing the relationships between hospital resources and activities: a systematic review.

Face the challenge of minimizing their resource utilization without reducing the quality of healthcare. Achieving this aim requires precise analysis and optimization of various inputs and outputs. This paper presents a systematic review of the relationships between hospital resources (considered productivity inputs) and financial and activity outcomes (considered productivity outputs). Several electronic bibliographic databases and the Internet were searched for articles published between January 1990 and December 2013 that examined the relationships between hospital resources and financial and activity outcomes. We assessed the quality of the study design, the nature of the sample, input and output indicators, and the statistical methods used for each selected study. Thirty-eight original papers were selected. Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA) were the most statistical methods used. Based on our analysis, we retained 18 input and 19 output indicators that could constitute the basis for hospital productivity benchmarking. Selecting a small set of shared economic and activity indicators is relevant for assessing the productivity of a hospital, measuring trends and performing national or international benchmarking. Such indicators should be combined with quality measures for a comprehensive evaluation approach.

Phenome-wide association studies on a quantitative trait: application to TPMT enzyme activity and thiopurine therapy in pharmacogenomics.

Phenome-Wide Association Studies (PheWAS) investigate whether genetic polymorphisms associated with a phenotype are also associated with other diagnoses. In this study, we have developed new methods to perform a PheWAS based on ICD-10 codes and biological test results, and to use a quantitative trait as the selection criterion. We tested our approach on thiopurine S-methyltransferase (TPMT) activity in patients treated by thiopurine drugs. We developed 2 aggregation methods for the ICD-10 codes: an ICD-10 hierarchy and a mapping to existing ICD-9-CM based PheWAS codes. Eleven biological test results were also analyzed using discretization algorithms. We applied these methods in patients having a TPMT activity assessment from the clinical data warehouse of a French academic hospital between January 2000 and July 2013. Data after initiation of thiopurine treatment were analyzed and patient groups were compared according to their TPMT activity level. A total of 442 patient records were analyzed representing 10,252 ICD-10 codes and 72,711 biological test results. The results from the ICD-9-CM based PheWAS codes and ICD-10 hierarchy codes were concordant. Cross-validation with the biological test results allowed us to validate the ICD phenotypes. Iron-deficiency anemia and diabetes mellitus were associated with a very high TPMT activity (p = 0.0004 and p = 0.0015, respectively). We describe here an original method to perform PheWAS on a quantitative trait using both ICD-10 diagnosis codes and biological test results to identify associated phenotypes. In the field of pharmacogenomics, PheWAS allow for the identification of new subgroups of patients who require personalized clinical and therapeutic management.

Pharmaceutical validation of medication orders using an OWL Ontology and Business Rules.

In this paper we present an application of pharmaceutical validation of medication based on an OWL ontology and business rules or more specifically clinical decision rules. This application has been developed based on a prototype that enables business users to author, execute and manage their Business Rules over OWL Ontology. This prototype is based on the Business Rule Management System (BRMS) IBM WebSphere ILOG JRules.

A clinical data warehouse-based process for refining medication orders alerts.

The objective of this case report is to evaluate the use of a clinical data warehouse coupled with a clinical information system to test and refine alerts for medication orders control before they were fully implemented. A clinical decision rule refinement process was used to assess alerts. The criteria assessed were the frequencies of alerts for initial prescriptions of 10 medications whose dosage levels depend on renal function thresholds. In the first iteration of the process, the frequency of the 'exceeds maximum daily dose' alerts was 7.10% (617/8692), while that of the 'under dose' alerts was 3.14% (273/8692). Indicators were presented to the experts. During the different iterations of the process, 45 (16.07%) decision rules were removed, 105 (37.5%) were changed and 136 new rules were introduced. Extensive retrospective analysis of physicians' medication orders stored in a clinical data warehouse facilitates alert optimization toward the goal of maximizing the safety of the patient and minimizing overridden alerts.

Methodology of integration of a clinical data warehouse with a clinical information system: the HEGP case.

Clinical Data Warehouses (CDW) can complement current Clinical Information Systems (CIS) with functions that are not easily implemented by traditional operational database systems. Here, we describe the design and deployment strategy used at the Pompidou University Hospital in southwest Paris. Four realms are described: technological realm, data realm, restitution realm, and administration realm. The corresponding UML use cases and the mapping rules from the shared integrated electronic health records to the five axes of the i2b2 CDW star model are presented. Priority is given to the anonymization and security principles used for the 1.2 million patient records currently stored in the CDW. Exploitation of a CDW by clinicians and investigators can facilitate clinical research, quality evaluations and outcome studies. These indirect benefits are among the reasons for the continuous use of an integrated CIS.

Determinants of clinical information system post-adoption success.

The diffusion of information technology (IT) in healthcare systems to support clinical processes makes the evaluation of physician and nurse post-adoption an important challenge for clinical information systems (CIS). This paper examines the relationships between the determinants of success of a CIS based on an expectation-confirmation paradigm in a cross-sectional survey performed at the Sherbrooke University Hospital (CHUS). 32.2% (161) of physicians and 27.1% (352) of nurses responded to the survey questionnaires. Results suggested that physician and nurse satisfaction is determined differently according to post-adoption expectations: compatibility, confirmation of expectations, usefulness, ease of use, and support. The best predictor of physician satisfaction was perceived usefulness (r=.25, p=.0003) whereas for nurses it was ease of use (r=.18, p=.0003). Confirmation of expectations was strongly associated with each post-adoption expectation and positions its importance in CIS design and redesign. This study draws attention to the differences between physician and nurse perceptions of information technology and emphasizes post-adoption evaluation to measure CIS success. Physicians and nurses post-adoption expectations were key factors to warn again potential discontinuance.

Choosing the right amount of healthcare information technologies investments.

OBJECTIVES: Choosing and justifying the right amount of investment in healthcare information technologies (HITECH or HIT) in hospitals is an ever increasing challenge. Our objectives are to assess the financial impact of HIT on hospital outcome, and propose decision-helping tools that could be used to rationalize the distribution of hospital finances. DESIGN: We used a production function and microeconomic tools on data of 21 Paris university hospitals recorded from 1998 to 2006 to compute the elasticity coefficients of HIT versus non-HIT capital and labor as regards to hospital financial outcome and optimize the distribution of investments according to the productivity associated with each input. RESULTS: HIT inputs and non-HIT inputs both have a positive and significant impact on hospital production (elasticity coefficients respectively of 0.106 and 0.893; R(2) of 0.92). We forecast 2006 results from the 1998 to 2005 dataset with an accuracy of +0.61%. With the model used, the best proportion of HIT investments was estimated to be 10.6% of total input and this was predicted to lead to a total saving of 388 million Euros for the 2006 dataset. CONCLUSION: Considering HIT investment from the point of view of a global portfolio and applying econometric and microeconomic tools allow the required confidence level to be attained for choosing the right amount of HIT investments. It could also allow hospitals using these tools to make substantial savings, and help them forecast their choices for the following year for better HITECH governance in the current stimulation context.