Evidence-based usability design principles for medication alerting systems.

BACKGROUND: Usability flaws in medication alerting systems may have a negative impact on clinical use and patient safety. In order to prevent the release of alerting systems that contain such flaws, it is necessary to provide designers and evaluators with evidence-based usability design principles. The objective of the present study was to develop a comprehensive, structured list of evidence-based usability design principles for medication alerting systems. METHODS: Nine sets of design principles for medication alerting systems were analyzed, summarized, and structured. We then matched the summarized principles with a list of usability flaws in order to determine the level of underlying evidence. RESULTS: Fifty-eight principles were summarized from the literature and two additional principles were defined, so that each flaw was matched with a principle. We organized the 60 summarized usability design principles into 6 meta-principles, 38 principles, and 16 sub-principles. Only 15 principles were not matched with a usability flaw. The 6 meta-principles respectively covered the improvement of the signal-to-noise ratio, the support for collaborative working, the fit with a clinician's workflow, the data display, the transparency of the alerting system, and the actionable tools to be provided within an alert. CONCLUSIONS: It is possible to develop an evidence-based, structured, comprehensive list of usability design principles that are specific to medication alerting systems and are illustrated by the corresponding usability flaws. This list represents an improvement over the current literature. Each principle is now associated with the best available evidence of its violation. This knowledge may help to improve the usability of medication alerting systems and, ultimately, decrease the harmful consequences of the systems' usability flaws.

Knowledge engineering for adverse drug event prevention: on the design and development of a uniform, contextualized and sustainable knowledge-based framework.

The primary aim of this work was the development of a uniform, contextualized and sustainable knowledge-based framework to support adverse drug event (ADE) prevention via Clinical Decision Support Systems (CDSSs). In this regard, the employed methodology involved first the systematic analysis and formalization of the knowledge sources elaborated in the scope of this work, through which an application-specific knowledge model has been defined. The entire framework architecture has been then specified and implemented by adopting Computer Interpretable Guidelines (CIGs) as the knowledge engineering formalism for its construction. The framework integrates diverse and dynamic knowledge sources in the form of rule-based ADE signals, all under a uniform Knowledge Base (KB) structure, according to the defined knowledge model. Equally important, it employs the means to contextualize the encapsulated knowledge, in order to provide appropriate support considering the specific local environment (hospital, medical department, language, etc.), as well as the mechanisms for knowledge querying, inference, sharing, and management. In this paper, we present thoroughly the establishment of the proposed knowledge framework by presenting the employed methodology and the results obtained as regards implementation, performance and validation aspects that highlight its applicability and virtue in medication safety.

Improvement of inter-services communication through a CDSS dedicated to myocardial perfusion scintigraphy.

This study addresses the question of communication between medical wards and the nuclear medicine department for the realization of myocardial perfusion scintigraphy. It analyses the effects of a reminder for completing the content of an order form. It shows that the CDSS impacted ordering practices. It could be seen as a system enabling to structure the information and improve the quality of orders.

Information contextualization in decision support modules for adverse drug event prevention.

This paper presents an analysis of hospitals' organization and Hospital Information Systems' features which can contribute in contextualization of Clinical Decision Support Systems (CDSS) for Adverse Drug Event (ADE) prevention. We identified four categories of contextualization: ENVIRONMENT, TASKS, USERS and TEMPORAL ASPECTS. Based on this analysis, we studied the technical possibilities at the architectural level to determine which component(s) of a standalone knowledge platform could technically handle contextualization. The results impact three types of components of this platform: (1) a CDSS providing decision support based on ADE signals mined in large data repositories; (2) a Connectivity Platform providing transformation and routing services (enabling any application to connect to the CDSS); (3) three prototype applications for accessing the decision support services realized within an industrial Computerized Physician Order Entry, an industrial Electronic Health Record and in an independent Web prototype, respectively. In each of the above components we present the dimension(s) of contextualization that has/have been determined to cope with and the design followed in the implementation phase.

Three different cases of exploiting decision support services for adverse drug event prevention.

Clinical Decision Support Systems (CDSSs) are implemented in clinical settings in order to improve patient outcomes and/or clinical practices. However, they are still not widely accepted by healthcare professionals due to over-alerting. The aim of the "Patient Safety through Intelligent Procedures in medication" (PSIP) project is to develop and demonstrate innovative tools so as to generate and provide relevant knowledge to healthcare professionals and patients for Adverse Drug Event (ADE) prevention by means of Information and Communication Technologies (ICT). PSIP employs a Knowledge Base (KB) as the core of its CDSS. This KB encapsulates signals capable of automatically detecting potential ADEs and contextualizing the CDSS output to the patient and healthcare professionals. To exploit the KB, a Global Knowledge Platform (GKP) has been created comprising of a KB system, a Connectivity Platform and appropriate user interface modules. The GKP has been tested to demonstrate integration of the KB in different work situations and it has been deployed in three different medical applications. The first is a Web application; the second involves a commercial French EHR (Electronic Health Record) and the third is a Danish CPOE (Computerised Physician Order Entry) system. This paper presents recent progress as regards the exploitation of the PSIP KB and the results obtained in the three different medical applications.

Effects of automated alerts on unnecessarily repeated serology tests in a cardiovascular surgery department: a time series analysis.

BACKGROUND: Laboratory testing is frequently unnecessary, particularly repetitive testing. Among the interventions proposed to reduce unnecessary testing, Computerized Decision Support Systems (CDSS) have been shown to be effective, but their impact depends on their technical characteristics. The objective of the study was to evaluate the impact of a Serology-CDSS providing point of care reminders of previous existing serology results, embedded in a Computerized Physician Order Entry at a university teaching hospital in Paris, France. METHODS: A CDSS was implemented in the Cardiovascular Surgery department of the hospital in order to decrease inappropriate repetitions of viral serology tests (HBV).A time series analysis was performed to assess the impact of the alert on physicians' practices. The study took place between January 2004 and December 2007. The primary outcome was the proportion of unnecessarily repeated HBs antigen tests over the periods of the study. A test was considered unnecessary when it was ordered within 90 days after a previous test for the same patient. A secondary outcome was the proportion of potentially unnecessary HBs antigen test orders cancelled after an alert display. RESULTS: In the pre-intervention period, 3,480 viral serology tests were ordered, of which 538 (15.5%) were unnecessarily repeated. During the intervention period, of the 2,095 HBs antigen tests performed, 330 unnecessary repetitions (15.8%) were observed. Before the intervention, the mean proportion of unnecessarily repeated HBs antigen tests increased by 0.4% per month (absolute increase, 95% CI 0.2% to 0.6%, p < 0.001). After the intervention, a significant trend change occurred, with a monthly difference estimated at -0.4% (95% CI -0.7% to -0.1%, p = 0.02) resulting in a stable proportion of unnecessarily repeated HBs antigen tests. A total of 380 unnecessary tests were ordered among 500 alerts displayed (compliance rate 24%). CONCLUSIONS: The proportion of unnecessarily repeated tests immediately dropped after CDSS implementation and remained stable, contrasting with the significant continuous increase observed before. The compliance rate confirmed the effect of the alerts. It is necessary to continue experimentation with dedicated systems in order to improve understanding of the diversity of CDSS and their impact on clinical practice.

Effect of alerts for drug dosage adjustment in inpatients with renal insufficiency.

OBJECTIVES: Medication errors constitute a major problem in all hospitals. Between 20% and 46% of prescriptions requiring dosage adjustments based on renal function are inappropriate. This study aimed to determine whether implementing alerts at the time of ordering medication integrated into the computerized physician order entry decreases the proportion of inappropriate prescriptions based on the renal function of inpatients. DESIGN: Six alternating 2-month control and intervention periods were conducted between August 2006 and August 2007 in two medical departments of a teaching hospital in France. A total of 603 patients and 38 physicians were included. During the intervention periods, alerts were triggered if a patient with renal impairment was prescribed one of the 24 targeted drugs that required adjustment according to estimated glomerular filtration rate (eGFR). MEASUREMENTS: The main outcome measure was the proportion of inappropriate first prescriptions, according to recommendation. RESULTS: A total of 1,122 alerts were triggered. The rate of inappropriate first prescriptions did not differ significantly between intervention and control periods (19.9% vs. 21.3%; p=0.63). The effect of intervention differed significantly between residents and senior physicians (p=0.03). Residents tended to make fewer errors in intervention versus control periods (Odds ratio 0.69; 95% confidence interval 0.41 to 1.15), whereas senior physicians tended to make more inappropriate prescriptions in intervention periods (odds ratio 1.88; 95% confidence interval 0.91 to 3.89). CONCLUSION: Alert activation was not followed by a significant decrease in inappropriate prescriptions in our study. Thus, it is still necessary to evaluate the impact of these systems if newly implemented in other settings thanks to studies also designed to watch for possible unanticipated effects of decision supports and their underlying causes.

Computerized advice on drug dosage to improve prescribing practice.

BACKGROUND: Maintaining therapeutic concentrations of drugs with a narrow therapeutic window is a complex task. Several computer systems have been designed to help doctors determine optimum drug dosage. Significant improvements in health care could be achieved if computer advice improved health outcomes and could be implemented in routine practice in a cost effective fashion. This is an updated version of an earlier Cochrane systematic review, by Walton et al, published in 2001. OBJECTIVES: To assess whether computerised advice on drug dosage has beneficial effects on the process or outcome of health care. SEARCH STRATEGY: We searched the Cochrane Effective Practice and Organisation of Care Group specialized register (June 1996 to December 2006), MEDLINE (1966 to December 2006), EMBASE (1980 to December 2006), hand searched the journal Therapeutic Drug Monitoring (1979 to March 2007) and the Journal of the American Medical Informatics Association (1996 to March 2007) as well as reference lists from primary articles. SELECTION CRITERIA: Randomized controlled trials, controlled trials, controlled before and after studies and interrupted time series analyses of computerized advice on drug dosage were included. The participants were health professionals responsible for patient care. The outcomes were: any objectively measured change in the behaviour of the health care provider (such as changes in the dose of drug used); any change in the health of patients resulting from computerized advice (such as adverse reactions to drugs). DATA COLLECTION AND ANALYSIS: Two reviewers independently extracted data and assessed study quality. MAIN RESULTS: Twenty-six comparisons (23 articles) were included (as compared to fifteen comparisons in the original review) including a wide range of drugs in inpatient and outpatient settings. Interventions usually targeted doctors although some studies attempted to influence prescriptions by pharmacists and nurses. Although all studies used reliable outcome measures, their quality was generally low. Computerized advice for drug dosage gave significant benefits by:1.increasing the initial dose (standardised mean difference 1.12, 95% CI 0.33 to 1.92)2.increasing serum concentrations (standradised mean difference 1.12, 95% CI 0.43 to 1.82)3.reducing the time to therapeutic stabilisation (standardised mean difference -0.55, 95%CI -1.03 to -0.08)4.reducing the risk of toxic drug level (rate ratio 0.45, 95% CI 0.30 to 0.70)5.reducing the length of hospital stay (standardised mean difference -0.35, 95% CI -0.52 to -0.17). AUTHORS' CONCLUSIONS: This review suggests that computerized advice for drug dosage has some benefits: it increased the initial dose of drug, increased serum drug concentrations and led to a more rapid therapeutic control. It also reduced the risk of toxic drug levels and the length of time spent in the hospital. However, it had no effect on adverse reactions. In addition, there was no evidence to suggest that some decision support technical features (such as its integration into a computer physician order entry system) or aspects of organization of care (such as the setting) could optimise the effect of computerised advice.

From users involvement to users' needs understanding: a case study.

UNLABELLED: Companies developing and commercializing Healthcare IT applications may decide to involve the users in the software development lifecycle in order to better understand the users' needs and to optimize their products. Unfortunately direct developers-users dialogues are not sufficient to ensure a proper understanding of the users' needs. It is also necessary to involve human factors specialists to analyze the users' expression of their needs and to properly formalize the requirements for design purposes. The objective of this paper is to present a case study reporting the collaborative work between HF experts and a company developing and commercializing a CPOE. This study shows how this collaboration helps resolve the limits of direct users involvement and usual problems pertaining to users' needs description and understanding. METHOD: The company participating in the study has implemented a procedure to convene regular meetings allowing direct exchanges between the development team and users' representatives. Those meetings aim at getting users' feedbacks on the existing products and at validating further developments. In parallel with usual HF methods supporting the analysis of the work system (onsite observations followed by debriefing interviews) and the usability evaluation of the application (usability inspection and usability tests), HF experts took the opportunity of the meetings organized by the company to collect, re-interpret and re-formulate the needs expressed by the users. RESULTS: The developers perceive the physicians' requirements concerning the display of the patient's list of medication as contradictory. In a previous meeting round the users had required a detailed view of the medication list against the synthesized existing one. Once this requirement satisfied, the users participating in the current meeting round require a synthesized view against the existing detailed one. The development team is unable to understand what they perceive as a reverse claim. Relying on a cognitive analysis of the physicians' decision making concerning the patient's treatment, the HF experts help re-formulate the physicians' cognitive needs in terms of synthesized/detailed display of the medication list depending on the stage of the decision making process. This led to an astute re-engineering of the application allowing the physicians to easily navigate back and forth between the synthesized and detailed views depending on the progress of their decision making. CONCLUSION: This study demonstrates that the integration of users' representatives in the software lifecycle is a good point for the end users. But it remains insufficient to resolve the complex usability problems of the system. Such solutions require the integration of HF expertise. Moreover, such an involvement of HF experts may generate benefits in terms of reduction of (i) the number of iterative developments and (ii) the users' training costs.

Archetypes as interface between patient data and a decision support system.

We propose an experiment to validate the hypothesis that archetypes enable better access and reliable use of patient data by a decision support system, mainly because they are designed to consistently link patient data with terminological systems and metadata.

Determinants of success for computerized clinical decision support systems integrated in CPOE systems: a systematic review.

We carried out a systematic review of published trials to identify the methodological characteristics of studies and technical characteristics of computerized clinical decision support systems (CCDSSs) associated with efficacy for the main outcome of the study. Four characteristics of the content of decision support and the way in which the user is provided with assistance seem to be associated with the success of CCDSSs: a) System-initiated interventions, b) Assistance without user control over output, c) Systems in which data are automatically retrieved from the electronic medical record and d) Systems providing corollary actions in CPOE. Major differences in outcome reporting between studies could be reduced by the use of dedicated tools to standardize methodological reporting.