Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes.

Poor usability of health technology is thought to diminish work system performance, increase error rates and, potentially, harm patients. The present study (i) used a combination of usability evaluation methods to highlight the chain that leads from usability flaws to usage problems experienced by users and, ultimately, to negative patient outcomes, and (ii) validated this approach by studying two different discharge summary production systems. To comply with quality guidelines, the process of drafting and sending discharge summaries is increasingly being automated. However, the usability of these systems may modify their impact (or the absence thereof) in terms of production times and quality, and must therefore be evaluated. Here, we applied three successive techniques for usability evaluation (heuristic evaluation, user testing and field observation) to two discharge summary production systems (underpinned by different technologies). The systems' main usability flaws led respectively to an increase in the time need to produce a discharge summary and the risk of patient misidentification. Our results are discussed with regard to the possibility of linking the usability flaws, usage problems and the negative outcomes by successively applying three methods for evaluating usability (heuristic evaluation, user testing and in situ observations) throughout the system development life cycle.

Development of a Smart Mobile Data Module for Fetal Monitoring in E-Healthcare.

The fetal heart rate (FHR) is a marker of fetal well-being in utero (when monitoring maternal and/or fetal pathologies) and during labor. Here, we developed a smart mobile data module for the remote acquisition and transmission (via a Wi-Fi or 4G connection) of FHR recordings, together with a web-based viewer for displaying the FHR datasets on a computer, smartphone or tablet. In order to define the features required by users, we modelled the fetal monitoring procedure (in home and hospital settings) via semi-structured interviews with midwives and obstetricians. Using this information, we developed a mobile data transfer module based on a Raspberry Pi. When connected to a standalone fetal monitor, the module acquires the FHR signal and sends it (via a Wi-Fi or a 3G/4G mobile internet connection) to a secure server within our hospital information system. The archived, digitized signal data are linked to the patient's electronic medical records. An HTML5/JavaScript web viewer converts the digitized FHR data into easily readable and interpretable graphs for viewing on a computer (running Windows, Linux or MacOS) or a mobile device (running Android, iOS or Windows Phone OS). The data can be viewed in real time or offline. The application includes tools required for correct interpretation of the data (signal loss calculation, scale adjustment, and precise measurements of the signal's characteristics). We performed a proof-of-concept case study of the transmission, reception and visualization of FHR data for a pregnant woman at 30 weeks of amenorrhea. She was hospitalized in the pregnancy assessment unit and FHR data were acquired three times a day with a Philips Avalon® FM30 fetal monitor. The prototype (Raspberry Pi) was connected to the fetal monitor's RS232 port. The emission and reception of prerecorded signals were tested and the web server correctly received the signals, and the FHR recording was visualized in real time on a computer, a tablet and smartphones (running Android and iOS) via the web viewer. This process did not perturb the hospital's computer network. There was no data delay or loss during a 60-min test. The web viewer was tested successfully in the various usage situations. The system was as user-friendly as expected, and enabled rapid, secure archiving. We have developed a system for the acquisition, transmission, recording and visualization of RCF data. Healthcare professionals can view the FHR data remotely on their computer, tablet or smartphone. Integration of FHR data into a hospital information system enables optimal, secure, long-term data archiving.

Case-crossover study to examine the change in postpartum risk of pulmonary embolism over time.

BACKGROUND: Although the current guidelines recommend anticoagulation up until 6 weeks after delivery in women at high risk of venous thromboembolism (VTE), the risk of VTE may extend beyond 6 weeks. Our objective was to estimate the risk of a pulmonary embolism in successive 2-week intervals during the postpartum period. METHODS: In a population-based, case-crossover study, we analyzed the French national inpatient database from 2007 to 2013 (n = 5,517,680 singleton deliveries). Using ICD-10 codes, we identified women who were diagnosed with a postpartum pulmonary embolism between July 1st, 2008, and December 31st, 2013. Deliveries were identified during a case "period" immediately before the pulmonary embolism, and five different control periods one year before the pulmonary embolism. Using conditional logistic regression, Odds ratios (ORs) and 95% confidential intervals (CIs) were estimated for ten successive 2-week intervals that preceded the diagnosis of pulmonary embolism. RESULTS: We identified 167,103 cases with a pulmonary embolism during the inclusion period. After delivery, the risk of pulmonary embolism declined progressively over time, with an OR [95%CI] of 17.2 [14.0-21.3] in postpartum weeks 1 to 2 and 1.9 [1.4-2.7] in postpartum weeks 11 to 12. The OR [95%CI] in postpartum weeks 13 to 14 was 1.4 [0.9-2.0], and the OR did not fall significantly after postpartum week 14. CONCLUSIONS: Our findings indicate that women are at risk of a pulmonary embolism up to 12 weeks after delivery. The shape of the risk curve suggests that the risk decreases exponentially over time. Future research is needed to establish whether the duration of postpartum anticoagulation should be extended beyond 6 weeks.

Hospital-based health technology assessment in France: A focus on medical devices.

Hospital-based health technology assessment (HTA) guides decisions as to whether new healthcare products should be made available within hospital structures. Its extension to medical devices (MDs) makes it possible to analyse several relevant aspects of these healthcare products in addition to their clinical value, and such evaluations are of interest to national health authorities, other healthcare establishments and industry. The aim of this work was to formulate several recommendations for a blueprint for hospital-based HTA for MDs in France. Five themes based on the work of the European Adopting hospital-based HTA in the EU (AdHopHTA) project were defined. Each member of the roundtable was then allocated a documentation task based on their experience of the theme concerned, and a literature review was carried out. An inventory of hospital-based HTA was performed and six recommendations aiming to strengthen and improve this approach were put forward: (1) encouragement of the spread of the hospital-based HTA culture and participation in communications and the promotion of this approach to hospital decision-makers; (2) adaptation of hospital-based HTA to the needs of decision-makers, taking into account the financial timetable and strategic objectives of the healthcare establishment; (3) harmonisation of the dossiers requested from industry between healthcare establishments, based on a common core; (4) promotion of the sharing of hospital-based HTA data under certain conditions, with data dissociable from the HTA report and the use of a validated methodology for the literature review; (5) creation of a composite indicator reflecting data production effort and the sharing of HTA activities, to be taken into account in the distribution of funds allocated for teaching, research and innovation missions considered of general interest; (6) the transmission of information directly from local to national level by pioneering centres. This work highlights the major issues at stake in hospital-based HTA and the need to valorise such activities in France.

The risks of pulmonary embolism and upper gastrointestinal bleeding beyond 35days after total hip replacement for coxarthrosis among middle-aged patients: A cross-over cohort.

Prophylactic anticoagulation is recommended up to 35days after total hip replacement (THR). Although several observational studies have assessed the incidence of thrombotic events or bleeding events after THR, the corresponding measures of association have never been studied concomitantly. Here, we evaluated the duration of the elevated risks (relative to the baseline risk) of both venous thromboembolic events and bleeding events after THR for coxarthrosis among middle-aged patients. This was a population-based, cross-over cohort study of data extracted from the French national inpatient database between 2007 and 2013. We included middle-aged patients (aged 45 to 69) having undergone THR for coxarthrosis. We compared the numbers of pulmonary embolisms (PEs) (respectively upper gastrointestinal bleedings (UGIBs)) following the THR with the numbers occurring during three unexposed periods one year later. This enabled us to estimate the odds ratio (OR) [95% confidence interval (CI)] for each of six successive 35-day intervals. The study included 108,099 patients. The ORs for PE were respectively 12.4 (95% CI, 8.6-17.8) (absolute risk difference rate per 100,000 (ARD/100,000)=130) and 5.0 (95% CI, 3.4-7.4) (ARD/100,000=52) for the first two 35-day intervals, and the risk was close to 1 thereafter. The risk of UGIB fell quickly, with an OR of 6.5 (95% CI, 4.6-9.1) (ARD/100,000=83) and 0.8 (95% CI, 0.4-1.6) for the first two 35-day intervals, respectively. The majority of UGIBs occurred during the inpatient stay for THR. Among middle-aged patients, the risk of a PE remains elevated beyond 35days after THR for coxarthrosis, whereas the risk of a UGIB remains elevated for the first 35days only.

Mapping patient path in the Pediatric Emergency Department: A workflow model driven approach.

The workflow models of the patient journey in a Pediatric Emergency Department (PED) seems to be an effective approach to develop an accurate and complete representation of the PED processes. This model can drive the collection of comprehensive quantitative and qualitative service delivery and patient treatment data as an evidence base for the PED service planning. Our objective in this study is to identify crowded situation indicators and bottlenecks that contribute to over-crowding. The greatest source of delay in patient flow is the waiting time from the health care request, and especially the bed request to exit from the PED for hospital admission. It represented 70% of the time that these patients occupied in the PED waiting rooms. The use of real data to construct the workflow model of the patient path is effective in identifying sources of delay in patient flow, and aspects of the PED activity that could be improved. The development of this model was based on accurate visits made in the PED of the Regional University Hospital Center (CHRU) of Lille (France). This modeling, which has to represent most faithfully possible the reality of the PED of CHRU of Lille, is necessary. It must be detailed enough to produce an analysis allowing to identify the dysfunctions of the PED and also to propose and to estimate prevention indicators of crowded situations. Our survey is integrated into the French National Research Agency (ANR) project, titled: "Hospital: Optimization, Simulation and avoidance of strain" (HOST).

Adverse drug events with hyperkalaemia during inpatient stays: evaluation of an automated method for retrospective detection in hospital databases.

BACKGROUND: Adverse drug reactions and adverse drug events (ADEs) are major public health issues. Many different prospective tools for the automated detection of ADEs in hospital databases have been developed and evaluated. The objective of the present study was to evaluate an automated method for the retrospective detection of ADEs with hyperkalaemia during inpatient stays. METHODS: We used a set of complex detection rules to take account of the patient's clinical and biological context and the chronological relationship between the causes and the expected outcome. The dataset consisted of 3,444 inpatient stays in a French general hospital. An automated review was performed for all data and the results were compared with those of an expert chart review. The complex detection rules' analytical quality was evaluated for ADEs. RESULTS: In terms of recall, 89.5% of ADEs with hyperkalaemia "with or without an abnormal symptom" were automatically identified (including all three serious ADEs). In terms of precision, 63.7% of the automatically identified ADEs with hyperkalaemia were true ADEs. CONCLUSIONS: The use of context-sensitive rules appears to improve the automated detection of ADEs with hyperkalaemia. This type of tool may have an important role in pharmacoepidemiology via the routine analysis of large inter-hospital databases.

Clinical evaluation of the ADE scorecards as a decision support tool for adverse drug event analysis and medication safety management.

AIMS: The prevention of adverse drug events (ADEs) demands co-ordination of different health care professionals. ADE scorecards are a novel approach to raise the team awareness regarding ADE risks and causes. It makes information on numbers and on possible causes of possible ADE cases available to the clinical team. The aim of the study was to investigate the usage and acceptance of ADE scorecards by healthcare professionals and their impact on rates of possible ADEs. METHODS: ADE scorecards were introduced in three departments of a French hospital. A controlled time series analysis of ADE data was conducted to assess the impact of the ADE scorecards. In addition, qualitative interviews and a standardized survey with all participating staff members were performed. RESULTS: Physicians, nurses and pharmacists found ADE scorecards effective to increase medication safety and recommended future usage. The time-series analysis did not show changes in rates of possible ADEs. CONCLUSION: ADE scorecards appear to be useful to raise awareness of ADE-related issues among professionals. Although the evaluation did not show significant reductions of ADE rates, the participating physicians, nurses and pharmacists believed that the ADE scorecards could contribute to increased patient safety and to a reduction in ADE rates. Strategies need to be designed to integrate ADE scorecards better into the clinical routine and to increase the precision of ADE detection.

Integrating Human Factors in an international research project: lessons learned from the PSIP project.

This paper presents how a Human Factors (HF) engineering approach has been applied to a European Project aiming at preventing Adverse Drug Events. Methods used by HF experts and their main contributions are depicted. Main lessons learned from HF involvement in this project are proposed in terms of methods for ADE prevention interventions and project management.

Knowledge engineering for adverse drug event prevention: on the design and development of a uniform, contextualized and sustainable knowledge-based framework.

The primary aim of this work was the development of a uniform, contextualized and sustainable knowledge-based framework to support adverse drug event (ADE) prevention via Clinical Decision Support Systems (CDSSs). In this regard, the employed methodology involved first the systematic analysis and formalization of the knowledge sources elaborated in the scope of this work, through which an application-specific knowledge model has been defined. The entire framework architecture has been then specified and implemented by adopting Computer Interpretable Guidelines (CIGs) as the knowledge engineering formalism for its construction. The framework integrates diverse and dynamic knowledge sources in the form of rule-based ADE signals, all under a uniform Knowledge Base (KB) structure, according to the defined knowledge model. Equally important, it employs the means to contextualize the encapsulated knowledge, in order to provide appropriate support considering the specific local environment (hospital, medical department, language, etc.), as well as the mechanisms for knowledge querying, inference, sharing, and management. In this paper, we present thoroughly the establishment of the proposed knowledge framework by presenting the employed methodology and the results obtained as regards implementation, performance and validation aspects that highlight its applicability and virtue in medication safety.

A statistics-based approach of contextualization for adverse drug events detection and prevention.

Several papers propose to take contexts into account for adverse drug events (ADE) detection and prevention, notably to decrease over-alerting of clinical decision support systems (CDSS). However, no statistical argument has been published till now. This works demonstrates, based on statistical analysis, that contextualization is necessary for ADE detection and prevention by 3 steps. A database of 115,447 inpatients stays from 6 hospitals, and 236 ADE detection rules are used. Step 1: the patients differ significantly between and within hospitals, regarding their medical background, their medication and several outcomes. Step 2: The estimated ADE rates vary between and within hospitals. Step 3: even when comparable conditions are present, the probability of ADE occurrence may differ between and within hospitals. Those 3 steps demonstrate that contextualization is necessary, and pave the way for a statistics-based method to contextualize ADE prevention (CDSS) and ADE detection tools.

PSIP: an overview of the results and clinical implications.

Adverse Drug Events (ADEs) are injuries due to medication management rather than the underlying condition of the patient. They endanger the patients and most of them could be avoided and prevented. The detection of ADEs usually relies on spontaneous reporting or medical chart reviews. The first objective of the PSIP Project is to automatically detect cases of ADEs by means of Data Mining, and to provide these cases to healthcare professionals. The second objective is to prevent ADEs by means of contextualised Clinical Decision Support Systems (Cx-CDSS) connected with Computerised Physician Order Entry (CPOE) or Electronic Health Record (EHR) systems. The detection of ADEs has been made possible through a set of rules able to identify relevant cases is a set of 92,000 medical cases. The results of this detection are provided through "ADE Scorecards". Contextualized Decision Support Systems have been developed by using the same set of rules and implemented in different software environments. The initial objectives of the PSIP project have been reached. The evaluation of the clinical impact has to be completed.

Comparison of the validity, perceived usefulness and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability: a study protocol.

INTRODUCTION: Research has shown that improvements to the usability of medication alert systems are needed. For designers and decisions-makers to assess usability of their alert systems, two paper-based tools are currently available: the instrument for evaluating human-factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aims to compare the validity, usability and usefulness of both tools to identify their strengths and limitations and assist designers and decision-makers in making an informed decision about which tool is most suitable for assessing their current or prospective system. METHODS AND ANALYSIS: First, TEMAS and I-MeDeSA will be translated into French. This translation will be validated by three experts in human factors. Then, in 12 French hospitals with a medication alert system in place, staff with expertise in the system will evaluate their alert system using the two tools successively. After the use of each tool, participants will be asked to fill in the System Usability Scale (SUS) and complete a survey on the understandability and perceived usefulness of each tool. Following the completion of both assessments, participants will be asked to nominate their preferred tool and relay their opinions on the tools. The design philosophy of TEMAS and I-MeDeSA differs on the calculation of a score, impacting the way the comparison between the tools can be performed. Convergent validity will be evaluated by matching the items of the two tools with respect to the usability dimensions they assess. SUS scores and answers to the survey will be statistically compared for I-MeDeSA and TEMAS to identify differences. Free-text responses in surveys will be analysed using an inductive approach. ETHICS AND DISSEMINATION: Ethical approval is not required in France for a study of this nature. The results will be published in a peer-reviewed journal.

A Deep Learning Framework for Automated ICD-10 Coding.

The International Statistical Classification of Diseases and Related Health Problems (ICD) is one of the widely used classification system for diagnoses and procedures to assign diagnosis codes to Electronic Health Record (EHR) associated with a patient's stay. The aim of this paper is to propose an automated coding system to assist physicians in the assignment of ICD codes to EHR. For this purpose, we created a pipeline of Natural Language Processing (NLP) and Deep Learning (DL) models able to extract the useful information from French medical texts and to perform classification. After the evaluation phase, our approach was able to predict 346 diagnosis codes from heterogeneous medical units with an accuracy average of 83%. Our results were finally validated by physicians of the Medical Information Department (MID) in charge of coding hospital stays.

Can F-MTI semantic-mined drug codes be used for adverse drug events detection when no CPOE is available?

BACKGROUND: Adverse Drug Events (ADEs) endanger the patients. Their detection and prevention is essential to improve the patients' safety. In the absence of computerized physician order entry (CPOE), discharge summaries are the only source of information about the drugs prescribed during a hospitalization. The French Multierminology Indexer (F-MTI) can help to extract drug-related information from those records. METHODS: In first and second validation steps, the performance of the F-MTI tool is evaluated to extract ICD10 and ATC codes from free-text documents. In third step, potential ADE detection rules are used and the confidences of those rules are compared in several hospitals: using a CPOE vs. using semantic mining of free-text documents, diagnoses and lab results being available in both cases. RESULTS: The F-MTI tool is able to extract ATC codes from documents. Moreover, the evaluation shows coherent and comparable results between the hospitals with CPOEs and the hospital with drugs information extracted from the reports for potential ADE detection. CONCLUSION: semantic mining using F-MTI can help to identify previous cases of potential ADEs in absence of CPOE.

Patient safety through intelligent procedures in medication: the PSIP project.

The European project Patient Safety through Intelligent Procedures in medication (PSIP) aims at preventing medical errors. The objective are: (1) to facilitate the systematic production of epidemiological knowledge on Adverse Drug Events (ADE) and (2) to improve the entire medication cycle in a hospital environment. The first sub-objective is to produce knowledge on ADE: to know, as exactly as possible, per hospital, per medical department, their number, type, consequences and causes, including human factors. Data Mining of structured hospital data bases, and semantic mining of free-texts will provide a list of observed ADE, with frequencies and probabilities, thus giving a better understanding of potential risks. The second sub-objective is to develop innovative knowledge based on the mining results and to deliver professionals and patients contextualized alerts and recommendations fitting the local risk parameters. This knowledge will be implemented in a PSIP-Platform independent of existing ICT applications.

Adverse drug events prevention rules: multi-site evaluation of rules from various sources.

Adverse drug events are a public health issue (98,000 deaths in the USA every year). Some computerized physician order entry (CPOEs) coupled with clinical decision support systems (CDSS) allow to prevent ADEs thanks to decision rules. Those rules can come from many sources: academic knowledge, record reviews, and data mining. Whatever their origin, the rules may induce too numerous alerts of poor accuracy when identically applied in different places. In this work we formalized rules from various sources in XML and enforced their execution on several medical departments to evaluate their local confidence. The article details the process and shows examples of evaluated rules from various sources. Several needs are enlightened to improve confidences: segmentation, contextualization, and evaluation of the rules over time.

Data-mining-based detection of adverse drug events.

Every year adverse drug events (ADEs) are known to be responsible for 98,000 deaths in the USA. Classical methods rely on report statements, expert knowledge, and staff operated record review. One of our objectives, in the PSIP project framework, is to use data mining (e.g., decision trees) to electronically identify situations leading to risk of ADEs. 10,500 hospitalization records from Denmark and France were used. 500 rules were automatically obtained, which are currently being validated by experts. A decision support system to prevent ADEs is then to be developed. The article examines a decision tree and the rules in the field of vitamin K antagonists.

Toward automatic detection and prevention of adverse drug events.

Adverse Drug Events (ADE) due to medication errors and human factors are a major public health issue. They endanger patient safety and cause considerable extra healthcare costs. The European project PSIP (Patient Safety through Intelligent Procedures in medication) aims to identify and prevent ADE. Data mining of the structured hospital data bases will give a list of observed ADE with frequencies and probabilities, thereby giving a better understanding of potential risks. The main objective of the project is to develop innovative knowledge based on the mining results and to deliver to professionals and patients, in the form of alerts and decision support functions, a contextualized knowledge fitting the local risk parameters.