One Health: Insights from Organizational & Social, Technology Assessment and Human Factors Perspectives.

OBJECTIVES: To offer diverse but complementary perspectives on how biomedical and health informatics can be informed by and help to achieve the vision of One Health. METHODS: Overview of key considerations and critical discussion of common themes, barriers and opportunities, based on collaborative review by International Medical Informatics Association (IMIA) working group members active in related fields. RESULTS: Health and care systems are complex sociotechnical systems that need explicit design and implementation strategies to align with the goals of One Health. The evidence-based health informatics paradigm and associated frameworks for evaluation of digital health technologies need to broaden their scope to take full account of the One Health approach. Informatics has specific contributions to make to One Health, for example by improved user experience reducing energy consumption and effective app design enhancing medication adherence. CONCLUSIONS: One Health is inherently intertwined with ergonomic, sociotechnical and evaluation perspectives in biomedical and health informatics. Health is a planetary issue that requires interdisciplinary collaborative action. The theories and principles of biomedical and health informatics offer many opportunities to transform digital health technology to better serve the One Health agenda.

Medication-Related Outcomes and Health Equity: Evidence for Pharmaceutical Care.

Marginalised people experience diminished access to pharmaceutical care and worse medication-related outcomes than the general population. Health equity is a global priority. This article explores the key evidence of health inequity and medication use, structures the causes and contributory factors and suggests opportunities that can be taken to advance the pharmaceutical care agenda so as to achieve health equity. The causes of, and contributors to, this inequity are multi-fold, with patient- and person-related factors being the most commonly reported. Limited evidence is available to identify risk factors related to other aspects of a personal medication use system, such as technology, tasks, tools and the internal and the external environments. Multiple opportunities exist to enhance equity in medication-related outcomes through pharmaceutical care research and practice. To optimise the effects and the sustainable implementation of these opportunities, it is important to (1) ensure the meaningful inclusion and engagement of members of marginalised groups, (2) use a person-centred approach and (3) apply a systems-based approach to address all of the necessary components of a system that interact and form a network as work processes that produce system outcomes.

Master's Degree in Health Data Science: Implementation and Assessment After Five Years.

Health data science is an emerging discipline that bridges computer science, statistics and health domain knowledge. This consists of taking advantage of the large volume of data, often complex, to extract information to improve decision-making. We have created a Master's degree in Health Data Science to meet the growing need for data scientists in companies and institutions. The training offers, over two years, courses covering computer science, mathematics and statistics, health and biology. With more than 60 professors and lecturers, a total of 835 hours of classes (not including the mandatory 5 months of internship per year), this curriculum has enrolled a total of 53 students today. The feedback from the students and alumni allowed us identifying new needs in terms of training, which may help us to adapt the program for the coming academic years. In particular, we will offer an additional module covering data management, from the edition of the clinical report form to the implementation of a data warehouse with an ETL process. Git and application lifecycle management will be included in programming courses or multidisciplinary projects.

The Optimal Sample Size for Usability Testing, From the Manufacturer's Perspective: A Value-of-Information Approach.

OBJECTIVES: For medical devices, a usability assessment is mandatory for market access; the objective is to detect potentially harmful use errors that stem from the device's design. The manufacturer assesses the final version of the device and determines the risk-benefit ratio for remaining errors. Nevertheless, the decision rule currently used to determine the sample size for this testing has statistical limitations and the lack of a clear decision-making perspective. METHODS: As an alternative, we developed a value-of-information analysis from the medical device manufacturer's perspective. The consequences of use errors not detected during usability testing and the errors' probability of occurrence were embedded in a loss function. The value of further testing was assessed as a reduction in the expected loss for the manufacturer. The optimal sample size was determined using the expected net benefit of sampling (ENBS) (the difference between the value provided by new participants and the cost of their inclusion). RESULTS: The value-of-information approach was applied to a real usability test of a needle-free adrenaline autoinjector. The initial estimate (performed on the first n = 20 participants) gave an optimal sample size of 100 participants and an ENBS of €255 453. This estimation was updated iteratively as new participants were included. After the inclusion of 90 participants, the ENBS was null for any sample size; hence, the cost of adding more participants outweighed the expected value of information, and therefore, the study could be stopped. CONCLUSIONS: On the basis of these results, our method seems to be highly suitable for sample size estimation in the usability testing of medical devices before market access.

A Tool for Evaluating Medication Alerting Systems: Development and Initial Assessment.

BACKGROUND: It is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems. OBJECTIVE: This study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital's computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems. METHODS: The Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability. RESULTS: The participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals' knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity. CONCLUSIONS: This initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.

Human factors engineering for medical devices: European regulation and current issues.

A large proportion of the patient injuries or deaths attributable to medical device (MD) misuse can be eliminated and/or mitigated by adopting an effective human factors and ergonomics (HFE) approach. The implementation of a usability engineering process is now mandatory for MD manufacturers seeking to obtain the European Union's CE Mark. Here, we describe the European Union's HFE regulation and highlight the challenges faced by (i) manufacturers implementing this regulation and (ii) regulatory bodies charged with assessing the compliance of usability files. In Europe, 95% of MD manufacturers are small- and medium-sized enterprises; compliance with the CE Mark regulations is a real challenge to their competitiveness. Levels of knowledge about HFE vary greatly from one regulatory organization to another, which can sometimes lead to very different expectations. We also present the specific use-related risk management approach required by the HFE regulation. Lastly, we focus on the limitations of the HFE regulation for MDs and on future HFE challenges in further reducing and/or eliminating MD use errors. The main challenge is the need to go beyond technology design and the premarket assessment and to look at the postproduction stage; the coupling between an MD and a sociotechnical system can lead to consequences that were not predicted during the design process. This implies the need to consider the emerging properties of technologies in use by involving all the stakeholders.

Building Usability Knowledge for Health Information Technology: A Usability-Oriented Analysis of Incident Reports.

BACKGROUND: The contribution of usability flaws to patient safety issues is acknowledged but not well-investigated. Free-text descriptions of incident reports may provide useful data to identify the connection between health information technology (HIT) usability flaws and patient safety. OBJECTIVES: This article examines the feasibility of using incident reports about HIT to learn about the usability flaws that affect patient safety. We posed three questions: (1) To what extent can we gain knowledge about usability issues from incident reports? (2) What types of usability flaws, related usage problems, and negative outcomes are reported in incidents reports? (3) What are the reported usability issues that give rise to patient safety issues? METHODS: A sample of 359 reports from the U.S. Food and Drug Administration Manufacturer and User Facility Device Experience database was examined. Descriptions of usability flaws, usage problems, and negative outcomes were extracted and categorized. A supplementary analysis was performed on the incidents which contained the full chain going from a usability flaw up to a patient safety issue to identify the usability issues that gave rise to patient safety incidents. RESULTS: A total of 249 reports were included. We found that incident reports can provide knowledge about usability flaws, usage problems, and negative outcomes. Thirty-six incidents report how usability flaws affected patient safety (ranging from incidents without consequence, to death) involving electronic patient scales, imaging systems, and HIT for medication management. The most significant class of involved usability flaws is related to the reliability, the understandability, and the availability of the clinical information. CONCLUSION: Incidents reports involving HIT are an exploitable source of information to learn about usability flaws and their effects on patient safety. Results can be used to convince all stakeholders involved in the HIT system lifecycle that usability should be considered seriously to prevent patient safety incidents.

How to Reach Evidence-Based Usability Evaluation Methods.

This paper discusses how and why to build evidence-based knowledge on usability evaluation methods. At each step of building evidence, requisites and difficulties to achieve it are highlighted. Specifically, the paper presents how usability evaluation studies should be designed to allow capitalizing evidence. Reciprocally, it presents how evidence-based usability knowledge will help improve usability practice. Finally, it underlines that evaluation and evidence participate in a virtuous circle that will help improve scientific knowledge and evaluation practice.

Usability Validation of Medical Devices: Issues in Identifying Potential Use Errors.

In order to prevent use errors with their medical devices, manufacturers have to integrate a safety-oriented usability engineering process in their product development lifecycle. A critical step of this process is the identification of potential use-errors. Standards and guidelines recommend to triangulate several sources of information e.g. scientific literature, incident reports, manufacturer's files and user's feedbacks. This paper presents lessons learned from applying these recommendations during an international project. We identify issues with (i) searching literature and databases, and (ii) interpreting collected data. Nevertheless triangulation of information sources allows to identify different types of use errors therefore providing valuable lists of potential use errors. Issuing recommendations aim at making easier this critical task.

Insights and limits of usability evaluation methods along the health information technology lifecycle.

A great variety of usability evaluation methods exist but they do not provide the same kind of results and do not address the same stage of the Health Information Technology (HIT) lifecycle. This paper takes stock of the application of expert evaluation, usability testing, clinical simulation, clinical trials and post-implementation surveillance to provide an overview of their main similarities and differences. Results from this comparison will help in choosing methods that are best able to evaluate a HIT and improve its usability and ultimately its safety of use.

Perceived usefulness of a usability issues reporting form to help understand "usability-induced use-errors": a preliminary study.

The Medical Device regulation requires manufacturers to anticipate and prevent risks of use errors of their medical device. However, manufacturers experience difficulties to understand the concept of "usability-induced use-errors". Based on a "usability framework" aiming at describing the relationship between usability design principles, usability flaws, usage problems, and outcomes, a usability evaluation reporting form had been designed to support understanding the use-error concept. This paper reports the preliminary evaluation of the perceived usefulness of this form. Results show that manufacturers found helpful the presentation of the results of a usability evaluation through this form for it supports the understanding of the usability origins and the consequences of use-errors. Even if the use of this reporting form should be made easier as usability experts experience difficulties to fill it, it seems a promising way to clearly present "usability-induced use-errors" to manufacturers.

A framework for reporting on human factor/usability studies of health information technologies.

Increasingly, studies are being published on the potential negative effect of introducing poor designed Health Information Technology (HIT) into clinical settings, relating to technology-induced errors and adverse events. Academic research on HIT design and evaluation is an extremely important source of information in providing new insights into factors contributing to successful system (re)design efforts, system user-friendliness and usability issues and safety critical aspects of HIT design. However, these studies have been inconsistent and incomprehensive in their reporting, complicating the appraisal of outcomes, generalizability of study findings, meta-analysis and harmonization of the available evidence. To improve identification of type of use errors and safety related issues regarding design and implementation of HIT, consensus on issues to be reported on in scientific publications is a necessary step forward. This study presents the first approach to a framework providing a set of principles to follow for comprehensive and unambiguous reporting of HIT design and usability evaluation studies with the objective to reduce variation, improve on the publication reporting quality and proper indexation of these studies. This framework may be helpful in expanding the knowledge base not only concerning the application of Human Factors (HF)/Usability studies of HIT but also improve the knowledge base of how to (re)design and implement effective, efficient and safe HIT.