Applying a Human-Centered Design to Develop a Patient Prioritization Tool for a Pediatric Emergency Department: Detailed Case Study of First Iterations.

BACKGROUND: Overcrowding in the emergency departments has become an increasingly significant problem. Patient triage strategies are acknowledged to help clinicians manage patient flow and reduce patients' waiting time. However, electronic patient triage systems are not developed so that they comply with clinicians' workflow. OBJECTIVE: This case study presents the development of a patient prioritization tool (PPT) and of the related patient prioritization algorithm (PPA) for a pediatric emergency department (PED), relying on a human-centered design process. METHODS: We followed a human-centered design process, wherein we (1) performed a work system analysis through observations and interviews in an academic hospital's PED; (2) deduced design specifications; (3) designed a mock PPT and the related PPA; and (4) performed user testing to assess the intuitiveness of the icons, the effectiveness in communicating patient priority, the fit between the prioritization model implemented and the participants' prioritization rules, and the participants' satisfaction. RESULTS: The workflow analysis identified that the PPT interface should meet the needs of physicians and nurses, represent the stages of patient care, and contain patient information such as waiting time, test status (eg, prescribed, in progress), age, and a suggestion for prioritization. The mock-up developed gives the status of patients progressing through the PED; a strip represents the patient and the patient's characteristics, including a delay indicator that compares the patient's waiting time to the average waiting time of patients with a comparable reason for emergency. User tests revealed issues with icon intuitiveness, information gaps, and possible refinements in the prioritization algorithm. CONCLUSIONS: The results of the user tests have led to modifications to improve the usability and usefulness of the PPT and its PPA. We discuss the value of integrating human factors into the design process for a PPT for PED. The PPT/PPA has been developed and installed in Lille University Hospital's PED. Studies are carried out to evaluate the use and impact of this tool on clinicians' situation awareness and prioritization-related cognitive load, prioritization of patients, waiting time, and patients' experience.

Building Usability Knowledge for Health Information Technology: A Usability-Oriented Analysis of Incident Reports.

BACKGROUND: The contribution of usability flaws to patient safety issues is acknowledged but not well-investigated. Free-text descriptions of incident reports may provide useful data to identify the connection between health information technology (HIT) usability flaws and patient safety. OBJECTIVES: This article examines the feasibility of using incident reports about HIT to learn about the usability flaws that affect patient safety. We posed three questions: (1) To what extent can we gain knowledge about usability issues from incident reports? (2) What types of usability flaws, related usage problems, and negative outcomes are reported in incidents reports? (3) What are the reported usability issues that give rise to patient safety issues? METHODS: A sample of 359 reports from the U.S. Food and Drug Administration Manufacturer and User Facility Device Experience database was examined. Descriptions of usability flaws, usage problems, and negative outcomes were extracted and categorized. A supplementary analysis was performed on the incidents which contained the full chain going from a usability flaw up to a patient safety issue to identify the usability issues that gave rise to patient safety incidents. RESULTS: A total of 249 reports were included. We found that incident reports can provide knowledge about usability flaws, usage problems, and negative outcomes. Thirty-six incidents report how usability flaws affected patient safety (ranging from incidents without consequence, to death) involving electronic patient scales, imaging systems, and HIT for medication management. The most significant class of involved usability flaws is related to the reliability, the understandability, and the availability of the clinical information. CONCLUSION: Incidents reports involving HIT are an exploitable source of information to learn about usability flaws and their effects on patient safety. Results can be used to convince all stakeholders involved in the HIT system lifecycle that usability should be considered seriously to prevent patient safety incidents.

Evidence-based usability design principles for medication alerting systems.

BACKGROUND: Usability flaws in medication alerting systems may have a negative impact on clinical use and patient safety. In order to prevent the release of alerting systems that contain such flaws, it is necessary to provide designers and evaluators with evidence-based usability design principles. The objective of the present study was to develop a comprehensive, structured list of evidence-based usability design principles for medication alerting systems. METHODS: Nine sets of design principles for medication alerting systems were analyzed, summarized, and structured. We then matched the summarized principles with a list of usability flaws in order to determine the level of underlying evidence. RESULTS: Fifty-eight principles were summarized from the literature and two additional principles were defined, so that each flaw was matched with a principle. We organized the 60 summarized usability design principles into 6 meta-principles, 38 principles, and 16 sub-principles. Only 15 principles were not matched with a usability flaw. The 6 meta-principles respectively covered the improvement of the signal-to-noise ratio, the support for collaborative working, the fit with a clinician's workflow, the data display, the transparency of the alerting system, and the actionable tools to be provided within an alert. CONCLUSIONS: It is possible to develop an evidence-based, structured, comprehensive list of usability design principles that are specific to medication alerting systems and are illustrated by the corresponding usability flaws. This list represents an improvement over the current literature. Each principle is now associated with the best available evidence of its violation. This knowledge may help to improve the usability of medication alerting systems and, ultimately, decrease the harmful consequences of the systems' usability flaws.

Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes.

Poor usability of health technology is thought to diminish work system performance, increase error rates and, potentially, harm patients. The present study (i) used a combination of usability evaluation methods to highlight the chain that leads from usability flaws to usage problems experienced by users and, ultimately, to negative patient outcomes, and (ii) validated this approach by studying two different discharge summary production systems. To comply with quality guidelines, the process of drafting and sending discharge summaries is increasingly being automated. However, the usability of these systems may modify their impact (or the absence thereof) in terms of production times and quality, and must therefore be evaluated. Here, we applied three successive techniques for usability evaluation (heuristic evaluation, user testing and field observation) to two discharge summary production systems (underpinned by different technologies). The systems' main usability flaws led respectively to an increase in the time need to produce a discharge summary and the risk of patient misidentification. Our results are discussed with regard to the possibility of linking the usability flaws, usage problems and the negative outcomes by successively applying three methods for evaluating usability (heuristic evaluation, user testing and in situ observations) throughout the system development life cycle.

How to Present Evidence-Based Usability Design Principles Dedicated to Medication-Related Alerting Systems to Designers and Evaluators? Results from a Workshop.

Medication alerting system use errors and lack of adoption are often attributed to usability issues. Previous work has used evidence from the literature to reveal usability principles specific to medication alerting systems and identify potential consequences of violating these principles. The current study sought to explore how best to convey these principles to designers and evaluators of these systems to facilitate their work. To this aim, a workshop with 19 participants was used to generate ideas and opinions on how to deliver these topic-specific design principles in a way that would be most helpful for them. Participants generated ideas for how (e.g., a collaborative, continuously updated forum) and what (e.g., illustrations, checklists, evidence sources and strength, consequences of violations) information is most useful to disseminate usability principles for medication alerting systems. Participants, especially designers, expressed desire to use these principles in practice and avoid previously documented mistakes and therefore make design and evaluation of these systems more effective and efficient. Those insights are discussed in terms of feasibility and logistical challenges to developing the proposed documentation). To move this work forward, a more collaborative approach of Human Factors specialists in medical informatics is necessary.

From Usability Engineering to Evidence-based Usability in Health IT.

Usability is a critical factor in the acceptance, safe use, and success of health IT. The User-Centred Design process is widely promoted to improve usability. However, this traditional case by case approach that is rooted in the sound understanding of users' needs is not sufficient to improve technologies' usability and prevent usability-induced use-errors that may harm patients. It should be enriched with empirical evidence. This evidence is on design elements (what are the most valuable design principles, and the worst usability mistakes), and on the usability evaluation methods (which combination of methods is most suitable in which context). To achieve this evidence, several steps must be fulfilled and challenges must be overcome. Some attempts to search evidence for designing elements of health IT and for usability evaluation methods exist and are summarized. A concrete instance of evidence-based usability design principles for medication-related alerting systems is briefly described.

Application of Human Factors Methods to Design Healthcare Work Systems: Instance of the prevention of Adverse Drug Events.

Human Factors (HF) methods are increasingly needed to support the design of new technologies in order to avoid that introducing those technologies into healthcare work systems induces use errors with potentially catastrophic consequences for the patients. This chapter illustrates the application of HF methods in developing two health technologies aiming at securing the hospital medication management process. Lessons learned from this project highlight the importance of (i) analyzing the work system in which the technology is intended to be implemented, (ii) involving end users in the design process and (iii) the intermediation role of HF between end users and scientific/technical experts.

Towards Evidence Based Usability in Health Informatics?

In a Health Information Technology (HIT) regulatory context in which the usability of this technology is more and more a critical issue, there is an increasing need for evidence based usability practice. However, a clear definition of evidence based usability practice and how to achieve it is still lacking. This paper underlines the need for evidence based HIT design and provides a definition of evidence based usability practice as the conscientious, explicit and judicious use of current best evidence in making decisions in design of interactive systems in health by applying usability engineering and usability design principles that have proven their value in practice. Current issues that hamper evidence based usability practice are highlighted and steps needed to achieve evidence are presented.

Medication Review: Human Factors Study Aiming at Helping an Acute Geriatric Unit to Sustain and Systematize the Process.

BACKGROUND: Medication Review (MRev) has been implemented in many hospitals to improve patient safety and well-being. However, it seems sometimes difficult to implement, maintain and systematize this process, especially when key-elements are absent. This study focuses on the analysis of a MRev process implemented in an Acute Geriatric Unit (AGU) which, at the time of the study, had no Computerized Physician Order Entry (CPOE) and no sufficient staff to - normally - support the process. OBJECTIVE: This study describes the MRev process as existing in the AGU with a particular focus on the preparatory MRev meeting phase and presents our recommendations to maintain and optimize it. METHODS: Human Factor experts have collected and analyzed data during MRev process by interviews, shadowing observations and video recording from April to October 2014 at Lille University Hospital. RESULTS: MRev process consists of three phases (meeting preparation, MRev meeting and patient discharge) and includes seven main tasks for which actors, documented supports, outcomes and difficulties are identified. Although allocating a fulltime pharmacist for the AGU would solve several problems, the main realistic recommendations concern training for junior and senior actors according to their roles and the improvement of some tasks processes. CONCLUSION: Despite less than optimal conditions as compared to those recommended by the literature, the observed AGU performs an efficient review based on well designed tools and processes.

Work System Characteristics Impacting the Performance and Quality of the Discharge Letter Process.

Studies on the impact of a Health Information Technology seldom consider socio-technical characteristics of the work system in which the technology is implemented. Yet those dimensions may act as hidden variables that could explain the inconsistency of impact studies' results in terms of performance, quality and satisfaction. This paper reports on the identification of those variables in the discharge letter (DL) process. Human Factors experts performed an analysis of the work system of the DL process in 17 medical units. The DL process is composed of three sub-processes running with work system differing according to the distribution of tasks, the technology implemented and the work organization. Hidden variables identified are: verification by the physician, technology's integration, number of editing cycles, physicians' preferences etc. Those variables can be collected automatically or by questionnaire. Statistical analyses will have to be performed to know which variable explain impact indicators.

Insights and limits of usability evaluation methods along the health information technology lifecycle.

A great variety of usability evaluation methods exist but they do not provide the same kind of results and do not address the same stage of the Health Information Technology (HIT) lifecycle. This paper takes stock of the application of expert evaluation, usability testing, clinical simulation, clinical trials and post-implementation surveillance to provide an overview of their main similarities and differences. Results from this comparison will help in choosing methods that are best able to evaluate a HIT and improve its usability and ultimately its safety of use.

Usability flaws of medication-related alerting functions: A systematic qualitative review.

INTRODUCTION: Medication-related alerting functions may include usability flaws that limit their optimal use. A first step on the way to preventing usability flaws is to understand the characteristics of these usability flaws. This systematic qualitative review aims to analyze the type of usability flaws found in medication-related alerting functions. METHOD: Papers were searched via PubMed, Scopus and Ergonomics Abstracts databases, along with references lists. Paper selection, data extraction and data analysis was performed by two to three Human Factors experts. Meaningful semantic units representing instances of usability flaws were the main data extracted. They were analyzed through qualitative methods: categorization following general usability heuristics and through an inductive process for the flaws specific to medication-related alerting functions. MAIN RESULTS: From the 6380 papers initially identified, 26 met all eligibility criteria. The analysis of the papers identified a total of 168 instances of usability flaws that could be classified into 13 categories of usability flaws representing either violations of general usability principles (i.e. they could be found in any system, e.g. guidance and workload issues) or infractions specific to medication-related alerting functions. The latter refer to issues of low signal-to-noise ratio, incomplete content of alerts, transparency, presentation mode and timing, missing alert features, tasks and control distribution. MAIN CONCLUSION: The list of 168 instances of usability flaws of medication-related alerting functions provides a source of knowledge for checking the usability of medication-related alerting functions during their design and evaluation process and ultimately constructs evidence-based usability design principles for these functions.

Improving access to clinical practice guidelines with an interactive graphical interface using an iconic language.

BACKGROUND: Clinical practice guidelines are useful for physicians, and guidelines are available on the Internet from various websites such as Vidal Recos. However, these guidelines are long and difficult to read, especially during consultation. Similar difficulties have been encountered with drug summaries of product characteristics. In a previous work, we have proposed an iconic language (called VCM, for Visualization of Concepts in Medicine) for representing patient conditions, treatments and laboratory tests, and we have used these icons to design a user interface that graphically indexes summaries of product characteristics. In the current study, our objective was to design and evaluate an iconic user interface for the consultation of clinical practice guidelines by physicians. METHODS: Focus groups of physicians were set up to identify the difficulties encountered when reading guidelines. Icons were integrated into Vidal Recos, taking human factors into account. The resulting interface includes a graphical summary and an iconic indexation of the guideline. The new interface was evaluated. We compared the response times and the number of errors recorded when physicians answered questions about two clinical scenarios using the interactive iconic interface or a textual interface. Users' perceived usability was evaluated with the System Usability Scale. RESULTS: The main difficulties encountered by physicians when reading guidelines were obtaining an overview and finding recommendations for patients corresponding to "particular cases". We designed a graphical interface for guideline consultation, using icons to identify particular cases and providing a graphical summary of the icons organized by anatomy and etiology. The evaluation showed that physicians gave clinical responses more rapidly with the iconic interface than the textual interface (25.2 seconds versus 45.6, p < 0.05). The physicians appreciated the new interface, and the System Usability Scale score value was 75 (between good and excellent). CONCLUSION: An interactive iconic interface can provide physicians with an overview of clinical practice guidelines, and can decrease the time required to access the content of such guidelines.

Using an iconic language to improve access to electronic medical records in general medicine.

Physicians have difficulties to access and analyse information in a medical record. In a previous work on drug databanks, we have shown that with an iconic language as VCM, an icon-based presentation can help physicians to access medical information. Our objective, herein, is to study whether VCM can be used in an electronic medical record for facilitating physician access in general practice. We identify the data and the functionalities of an electronic medical record that could benefit from VCM icons representing clinical findings, patient history, etc. We also present a preliminary evaluation of this new icon-focused interface. We conclude by discussing the results like the assessment of the user's satisfaction and pointing out the importance of coding data.

Methods uncovering usability issues in medication-related alerting functions: results from a systematic review.

This paper aims at listing the methods used to evaluate the usability of medication-related alerting functions and at knowing what type of usability issues those methods allow to detect. A sub-analysis of data from this systematic review has been performed. Methods applied in the included papers were collected. Then, included papers were sorted in four types of evaluation: "expert evaluation", "user- testing/simulation", "on site observation" and "impact studies". The types of usability issues (usability flaws, usage problems and negative outcomes) uncovered by those evaluations were analyzed. Results show that a large set of methods are used. The largest proportion of papers uses "on site observation" evaluation. This is the only evaluation type for which every kind of usability flaws, usage problems and outcomes are detected. It is somehow surprising that, in a usability systematic review, most of the papers included use a method that is not often presented as a usability method. Results are discussed about the opportunity to provide usability information collected after the implementation of the technology during their design process, i.e. before their implementation.

Context sensitive health informatics: concepts, methods and tools.

Context is a key consideration when designing and evaluating health information technology (HIT) and cannot be overstated. Unintended consequences are common post HIT implementation and even well designed technology may not achieve desired outcomes because of contextual issues. While context should be considered in the design and evaluation of health information systems (HISs) there is a shortcoming of empirical research on contextual aspects of HIT. This conference integrates the sociotechnical and Human-Centered-Design (HCD) approaches and showcases current research on context sensitive health informatics. The papers and presentations outlines theories and models for studying contextual issues and insights on how we can better design HIT to accommodate different healthcare contexts.

A framework for reporting on human factor/usability studies of health information technologies.

Increasingly, studies are being published on the potential negative effect of introducing poor designed Health Information Technology (HIT) into clinical settings, relating to technology-induced errors and adverse events. Academic research on HIT design and evaluation is an extremely important source of information in providing new insights into factors contributing to successful system (re)design efforts, system user-friendliness and usability issues and safety critical aspects of HIT design. However, these studies have been inconsistent and incomprehensive in their reporting, complicating the appraisal of outcomes, generalizability of study findings, meta-analysis and harmonization of the available evidence. To improve identification of type of use errors and safety related issues regarding design and implementation of HIT, consensus on issues to be reported on in scientific publications is a necessary step forward. This study presents the first approach to a framework providing a set of principles to follow for comprehensive and unambiguous reporting of HIT design and usability evaluation studies with the objective to reduce variation, improve on the publication reporting quality and proper indexation of these studies. This framework may be helpful in expanding the knowledge base not only concerning the application of Human Factors (HF)/Usability studies of HIT but also improve the knowledge base of how to (re)design and implement effective, efficient and safe HIT.

The usability-error ontology.

Clinical Systems have become standard partners with clinicians in the care of patients. As these systems become integral parts of the clinical workflow, they have the potential to help improve patient outcomes, however they have also in some cases have led to adverse events and has resulted in patients coming to harm. Often the root cause analysis of these adverse events can be traced back to Usability Errors in the Health Information Technology (HIT) or its interaction with users. Interoperability of the documentation of HIT related Usability Errors in a consistent fashion can improve our ability to do systematic reviews and meta-analyses. In an effort to support improved and more interoperable data capture regarding Usability Errors, we have created the Usability Error Ontology (UEO) as a classification method for representing knowledge regarding Usability Errors. We expect the UEO will grow over time to support an increasing number of HIT system types. In this manuscript, we present this Ontology of Usability Error Types and specifically address Computerized Physician Order Entry (CPOE), Electronic Health Records (EHR) and Revenue Cycle HIT systems.

Safety-oriented usability test of a semi-automated unit dose system: role of task allocation between human and machine.

The distribution of tasks between humans and machines in the design of healthcare systems is an important issue for patient safety. This paper presents a usability test performed to compare a semi-automated unit dose system (UDS) with the usual/manual preparation procedure for preparing and administering drugs. The results show that the UDS prevents the frequent administration errors encountered with the usual cabinets and produces a better performance in terms of time for filling the pill dispensers (6.52 sec ± 1.1 vs. 8.5 sec ± 1.5 (t(9, 16) = 3.12, p <.007)). But the results also stress that the UDS takes entire control of the preparation and administration tasks, thus leading to a loss of control of the process by nurses (difficulties experienced by them in resuming their actions, difficulties in memorizing drugs, lack of confidence in the UDS while they "blindly" rely on it). The distribution of tasks between the nurses and the UDS should be modified to give back control to the nurses. Design suggestions were provided in this way. For instance, the UDS may guide the nurses for the drugs localization in the cabinet but leaves the validation of the drugs to the nurses.

Impact of the context of use analysis for the extension of an existing medical device: an analgesia monitor case study.

The EU revised Medical Device Directive introduces a major change in the CE marking of medical devices (MD) aiming at improving their safety. Manufacturers must now comply with an "ergonomics" essential requirement to prevent risks of use errors. This requirement is characterized by the integration of a usability engineering process in the MD design cycle to be documented in a usability engineering file. This study focuses on the first step of the usability engineering process, i.e. the analysis of the intended context of use of the MD, and shows the benefits of this analysis when performed early in the MD design cycle. Usability experts have conducted an analysis of the intended contexts of use for the extension of an existing device (analgesia monitor) in order to support manufacturer's design choices. Observations and interviews were carried out in two neonatology units (Maternity and Neonatology Intensive Care units) with a particular focus on pain management activities performed by physicians and nursery nurses. The results highlight irreducible differences between the two environments which led to identify different risks of use errors and specific ergonomics requirements. The results provided the manufacturer enough information to make informed decision about the extension of the device.