Human factors engineering for medical devices: European regulation and current issues.

A large proportion of the patient injuries or deaths attributable to medical device (MD) misuse can be eliminated and/or mitigated by adopting an effective human factors and ergonomics (HFE) approach. The implementation of a usability engineering process is now mandatory for MD manufacturers seeking to obtain the European Union's CE Mark. Here, we describe the European Union's HFE regulation and highlight the challenges faced by (i) manufacturers implementing this regulation and (ii) regulatory bodies charged with assessing the compliance of usability files. In Europe, 95% of MD manufacturers are small- and medium-sized enterprises; compliance with the CE Mark regulations is a real challenge to their competitiveness. Levels of knowledge about HFE vary greatly from one regulatory organization to another, which can sometimes lead to very different expectations. We also present the specific use-related risk management approach required by the HFE regulation. Lastly, we focus on the limitations of the HFE regulation for MDs and on future HFE challenges in further reducing and/or eliminating MD use errors. The main challenge is the need to go beyond technology design and the premarket assessment and to look at the postproduction stage; the coupling between an MD and a sociotechnical system can lead to consequences that were not predicted during the design process. This implies the need to consider the emerging properties of technologies in use by involving all the stakeholders.

Estimating the number of usability problems affecting medical devices: modelling the discovery matrix.

BACKGROUND: Usability testing of medical devices are mandatory for market access. The testings' goal is to identify usability problems that could cause harm to the user or limit the device's effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability testing is estimating the total number of problems, in order to assess the completeness of the discovery process. Today's margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems. METHODS: The matrix-based model was based on the full discovery matrix (including unobserved columns) and not solely on a summary of the data (e.g. the margins). This model also circumvents a drawback of margin-based methods by simultaneously estimating the model's parameters and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which reflects a real-life setting. As suggested in the usability literature, we assumed that the probability of detection had a logit-normal distribution. RESULTS: We assessed the matrix-based method's performance in a range of settings reflecting real-life usability testing and with heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability) in a wide range of settings. We also applied our method to five real datasets from usability testing. CONCLUSIONS: Estimation models (and particularly matrix-based models) are of value in estimating and monitoring the detection process during usability testing. Matrix-based models have a solid mathematical grounding and, with a view to facilitating the decision-making process for both regulators and device manufacturers, should be incorporated into current standards.

Teaching Module to Train Students in the Participatory Design of Digital Technologies for Disability Assistance.

Teaching the principles of participatory design to students mainly interested in digital skills is important because user-centered approaches have become essential in the field of new technologies when we want to guarantee that a product will meet the needs of its end-users. Working with technologies dedicated to the disability assistance was considered to be the right application domain. In 2021, ESIEE Paris, a school training students to become engineers with digital skills, created and opened a new teaching module to learn how to follow the principles of participatory design to improve the quality of a project. This paper describes the organization and the conclusions of this experience after two editions of the module.

Detectability of use errors in summative usability tests of medical devices: Impact of the test environment.

The regulations on summative usability evaluations of medical devices (MDs) emphasize that the test environment must have sufficient ecological validity for generalization to real-life use. Here, we examined the influence of environmental fidelity (a component of ecological validity) on the detectability of MD use errors. A total of 140 participants participated in a summative usability evaluation of an anaphylactic shock auto-injector device under either a high-fidelity condition or a condition acceptable from the manufacturer's perspective, lower-fidelity condition. The numbers of errors detected in each condition were compared by applying descriptive statistics and logistic and Poisson multivariate regressions. We found that the level of fidelity did not influence the overall number of use errors detected but did influence the detection of certain use errors. To optimize the test environment and increase the detection of use errors, each environmental feature's role in the test task should first be examined.

Software as a medical device: regulatory critical issues.

The revised Medical Device Directive has been adopted by the EU in 2010. A major change is that software for certain purposes is now considered a medical device. This entails that a new view needs to be developed on the design, development, evaluation and post-market surveillance of medical software that meets the definition of a medical device. This paper identifies some issues at stake and discusses them.

Integration of a new technology into a work system: a case study of a smart drinking glass in French nursing homes.

PURPOSE: To reduce the risk of dehydration in older adults, the French company Auxivia has developed a smart drinking glass (SDG) that can measure the amount of water drunk. The present study looked at the various work systems (WSs) designed for use of the SDG in a nursing home. The study's objectives were to (i) determine the WSs' impact on the staff's ability to comply with the device's prerequisites and ensure the device's effective use and (ii) draw up guidelines on designing work systems. MATERIALS AND METHODS: At three nursing homes in France, two independent observers performed 9 h of observations at each site and a total of 29 interviews. RESULTS: Decisions concerning implementation and the resulting WSs have an impact on the tasks to be performed, the tasks' inherent constraints and the use of the SDG. It is essential to take account of the sociotechnical system as a whole before integrating a technology. Ideally, the introduction of an SDG will go unnoticed by staff and residents; however, our results emphasize the value of highlighting work constraints via a human factors analysis. CONCLUSIONS: It is essential to take account of sociotechnical WSs as a whole when integrating a technology.Implications for rehabilitationFor caregivers, better measurement of the amount of water drunk by older adults might help to prevent dehydration.For service providers, better knowledge of how work systems influence the nursing home staff's ability to comply with the smart drinking glass's prerequisites might improve the device implementation process.For managers, a nursing home's work systems should take account of the smart drinking glass's prerequisites on one hand and the home's environment and constraints on the other.

Improving the usability and usefulness of computerized decision support systems for medication review by clinical pharmacists: A convergent, parallel evaluation.

BACKGROUND: Computerized decision support systems (CDSSs) help hospital-based clinical pharmacists to perform medication reviews and so are promising tools for improving medication safety. However, their poor usability can reduce effectiveness and acceptability. OBJECTIVES: To evaluate the usability and perceived usefulness of a CDSS for medication review by hospital-based pharmacists and to draw up guidelines on improving its usability. METHODS: We performed a convergent, parallel evaluation. Firstly, three researchers conducted a heuristic evaluation of the CDSS. Secondly, clinical pharmacists who use the CDSS filled out the Usefulness, Satisfaction and Ease of Use (USE) questionnaire. Lastly, semi-structured interviews with the pharmacists enabled us to understand their opinions and experiences. The results of the heuristic evaluation were used to identify potential improvements in the CDSS. We performed a statistical analysis of the USE questionnaire data. Interviews were analyzed based on the unified theory of acceptance and use of technology (UTAUT), together with a task-technology fit model. The results generated by these three approaches were compared in order to determine convergences and divergences, identify challenges related to the usability and usefulness of the CDSS, and draw up guidelines for its improvement. RESULTS: Forty-seven usability problems were discovered; they variously concerned the graphical user interface, the pharmacists' needs, and the medication review model implemented in the CDSS. Only the "usefulness" dimension of the USE was not scored positively. All the UTAUT dimensions and the task-technology fit dimension emerged in the interviews. Cross-comparisons of the results from the three approaches led to the identification of four challenges and the corresponding formulation of 23 guidelines. CONCLUSIONS: The guidelines developed here should help to improve the design and acceptability of CDSSs. Hence, CDSSs will be able to assist clinical pharmacists more fully with their medication reviews and help to further improve patient safety.

Comparison of the validity, perceived usefulness, and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability.

OBJECTIVE: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aimed to compare their convergent validity, perceived usability, usefulness, strengths, and weaknesses, as well as users' preferences. METHOD: To evaluate convergent validity, two experts mapped TEMAS' items against I-MeDeSA's items with respect to the usability dimensions they target. To assess perceived usability, usefulness, strengths, and weaknesses of both tools, staff with expertise in their medication alerting system were asked to use French versions of the TEMAS and I-MeDeSA. After the use of each tool, participants were asked to complete the System Usability Scale (SUS) and answer questions about the understandability and usefulness of each tool. Finally, participants were asked to name their preferred tool. Numeric scores were statistically compared. Free-text responses were analyzed using an inductive approach. RESULTS: Forty-five participants from 10 hospitals took part in the study. In terms of convergent validity, I-MeDeSA focuses more on the usability of the graphical user interface while TEMAS considers a wider range of usability principles. Both tools have a fair level of perceived usability (I-MeDeSA' SUS score = 61.85 and TEMAS' SUS score = 62.87), but results highlight that revisions are necessary to both tools to improve their usability. Participants found TEMAS more useful than I-MeDeSA (t = -3.63, p =.005) and had a clear preference for TEMAS to identify problems in formative evaluation (39 of 45; 0.867, p <.001) and to compare the usability of alert systems during the procurement process (36 of 45; 0.8, p <.001). CONCLUSIONS: The TEMAS is perceived as more useful and is preferred by participants. The I-MeDeSA seems more relevant for quick evaluations that focus on the graphical user interface. The TEMAS seems to be more suitable for in-depth usability evaluations of alert systems. Even if both tools are perceived to be equally usable, they suffer from wording, instructional, and organizational problems that hinder their use. The results of this study will be used to improve the design of I-MeDeSA and TEMAS.

Why and how can we improve patient and caregiver information for the proper use of their medical devices throughout the care pathway?

Information is generally a major pillar in the relationship of trust between the patient and their healthcare team. In the case of medical devices, which are operator-dependent and for which the proper "handling" of the device will directly affect its performance, information on the proper use becomes a major therapeutic issue. To do this, patients must have access to quality information about health products. Twenty years after the Kouchner Act introduced a right to information for patients, our panel questioned the quality of information provided to patients about the medical devices they use during their care pathway. We mapped off the information process throughout the life of the medical device (manufacturer, prescription, dispensing, use and monitoring) to identify deficiencies and potential corrective solutions. Thus, the device could respond to the 5B as for the drug (right device, right patient, right indication, right time, and right use). Seven recommendations have emerged and are proposed in this article.

Jupyter Notebooks for Introducing Data Science to Novice Users.

Data science is a bridge discipline involving computer science, statistics, and knowledge of the health field. We developed a Jupyter Notebook to enable novice users to easily and autonomously analyze data from social networks. We conducted an experimentation with non-programmer students. They had to adapt a R Notebook and complete 14 questions and to perform descriptive analyses. The average rate of correct answers was 90.7. Jupyter Notebook enabled novice users to easily and autonomously analyze data from Twitter.

The rehabilitation robot: factors influencing its use, advantages and limitations in clinical rehabilitation.

PURPOSE: Despite the proven effectiveness of rehabilitation robots (RR) in the literature, they are still little used in clinical rehabilitation. The aim of this study was to analyse the factors influencing the use of RR and the perception of therapists who used RR. METHOD: In order to characterize the factors influencing the use of RR by therapists, a semi-structured interview was conducted with 18 therapists. These interviews are based on an interview guide inspired by the Unified Theory of Acceptance and Use of Technology model. The interviews were recorded and then transcribed, summarized and finally synthesized cross-sectionally. In addition and in parallel, the System Usability Scale (SUS) was also proposed to clinicians in order to collect quantitative data. RESULTS: The interviews highlight the facilitators perceived by the therapists, such as the intensity of the movement, the complementarity with conventional rehabilitation. The results also showed the possible barriers perceived, these can be sometimes inconclusive (e.g., bugs). The SUS results show no effect, either on the gender of the users, their therapists, or the duration of use of the tool. CONCLUSION: Better communication on the functionality of the robot and the construction of achievable goals would lead to more results that are conclusive but also better patient care. To date, and despite the evidence for the effectiveness of RRs, therapists believe that there are still many barriers to their use. They agree, however, that if changes are made, RRs will become an integral part of therapy.IMPLICATIONS FOR REHABILITATIONThe study idenfied and highlighted the factors influencing the use of the rehabilitation robot in the clinics through metric and ergonomic evaluations.The study allowed to quantify the level of acceptance of the Lokomat among therapists.This study allowed to identify negative factors that could be resolved through the implementation of a structured and generalized protocol for patients and thus improve their care.

Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study.

BACKGROUND: A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula is an important part of search engine evaluation. However, the methods currently used to evaluate ranking formulae mainly collect quantitative data and do not gather qualitative data, which help to understand what needs to be improved to tailor the formulae to their end users. OBJECTIVE: This study aims to evaluate 2 different parameter settings of the ranking formula of LiSSa (the French acronym for scientific literature in health care; Department of Medical Informatics and Information), a tool that provides access to health scientific literature in French, to adapt the formula to the needs of the end users. METHODS: To collect quantitative and qualitative data, user tests were carried out with representative end users of LiSSa: 10 general practitioners and 10 registrars. Participants first assessed the relevance of the search results and then rated the ranking criteria used in the 2 formulae. Verbalizations were analyzed to characterize each criterion. RESULTS: A formula that prioritized articles representing a consensus in the field was preferred. When users assess an article's relevance, they judge its topic, methods, and value in clinical practice. CONCLUSIONS: Following the evaluation, several improvements were implemented to give more weight to articles that match the search topic and to downgrade articles that have less informative or scientific value for the reader. Applying a qualitative methodology generates valuable user inputs to improve the ranking formula and move toward a highly usable search engine.

Medication related computerized decision support system (CDSS): make it a clinicians' partner!

Medication related Computerized Decision Support System (CDSS) are known to have a positive impact on Adverse Drug Events (ADE) prevention but they face acceptance problems due to over alerting and usability issues. We present here a Human factors approach to the design of these Clinical Decision Support (CDS) functions and to their integration into different Electronic Health Record (EHR) / Computerized Physicians Order Entry (CPOE) systems, so that the resulting CDSS corresponds to the users needs and fits clinical workflows and cognitive processes. We used ethnographic observations completed with semi-structured interviews to analyse existing work situations and work processes. These were then described in detail using the SHEL (Software, Hardware, Environment & Liveware) formalism, which enables a structured description of the work system and provides an appropriate classification of human errors potentially leading to ADEs. We then propose a Unified Modelling Language (UML) model supporting the characterization by the CDSS of the drug monitoring and clinical context of patients at risk of ADE. This model combines the status of the lab test orders on the one hand with the validity and normality of the lab results on the other hand. This makes the system able to catch the context of the monitoring of the drugs through their corresponding lab tests and lab results (e.g. kalemia for potassium) and also part of the context of the clinical status of the patient (actual lab values, but also diseases and other pathologies that are identified as potential causes of the ADE e.g. renal insufficiency and potassium). We show that making the system able to catch the monitoring and clinical contexts opens interesting opportunities for the design of the CDS information content and display mode. Implementing this model would allow the CDSS to take into account the actions already engaged by the healthcare team and to adapt the information delivered to the monitoring and clinical context, thus making the CDSS a partner to the clinicians, nurses and pharmacists.

Human factors based recommendations for the design of medication related clinical decision support systems (CDSS).

This study is part of a research project aiming at developing advanced functions of medication related CDSS to support the monitoring of patients' therapeutic treatments based mainly on corresponding lab values. We adopted a user-centred approach to the design of these advanced CDSS functions. We collected existing recommendations in the literature and completed previous Human Factors (HF) field studies and analyses by focused observations and modeling. We present resulting HF based recommendations for the design of such advanced medication CDSS and focus more specifically on two innovative high level recommendations completing those already existing in the literature. For illustration purposes, an example of the operationalization of one of the recommendation is presented.

Competitive Usability Evaluation of Electronic Health Records: Preliminary Results of a Case Study.

It is necessary for hospitals to be able to compare the usability of electronic health records (EHR) before acquisition. Adding usability as a critical element of the procurement process is therefore crucial. During the competitive usability evaluation of several EHRs, the usability walkthrough method has the potential of making end-users more active in the procurement process than demonstrations. This case study presents first results of a comparison of three EHRs performed by nine representative end-users. All users uncovered usability problems while performing their scenarios. The results show that none of the EHRs evaluated is without major usability problems. These problems have been well-known to human factors researchers for a long time.

Aero-Sim: A User-Centred Testbed for the Evaluation of Devices as Protection from Droplets and/or Aerosol Contaminations.

Concerns over high transmission risk of SARS-CoV-2 have led to innovation and usage of an aerosol box to protect healthcare workers during airway intubation in patients with COVID-19. However, only a few studies have examined the impact of these aerosol boxes on the dispersion of droplets and aerosols, which are both thought to be significant contributors to the spread of COVID-19. In addition, to our knowledge, only a few studies have included in the concerned testbeds elements of the work context, which nevertheless have an impact on the use of the device.

Context-Sensitive Implementation of Clinical Pharmacy Missions Within an Academic Hospital.

Clinical pharmacy activities contribute to improve patient safety. Yet, the work system's characteristics influence how clinical pharmacy activities are performed and conversely clinical pharmacy causes that work system to evolve. This exploratory study aims to identify the different ways in which clinical pharmacy activities are performed in different units of a large academic hospital. Interviews and observations have been performed to identify in each ward the clinical pharmacy activities implemented and how they are carried out.

Medication review and reconciliation in older adults.

Older people are frequently exposed to polypharmacy, inappropriate prescribing, and adverse drug events. Two clinical processes can help geriatricians to optimize and increase the safety of drug prescriptions for older adults: medication reconciliation and medication review. Medication reconciliation provides the best possible medication history and identifies and resolves discrepancies in drug prescriptions. During the medication review, the best possible medication history is crosschecked against other data, including morbidities, patient's preferences, or geriatric syndromes, to produce a personalized medication strategy. Alignment of treatment recommendations with patient preferences and goals through shared decision-making is particularly important in medication review. Medication reconciliation and medication review have proven to be effective, but their broad implementation remains difficult. Indeed, these procedures are time-consuming and require specific skills, coordination between different healthcare professionals, organizations and dedicated means. The involvement of geriatricians therefore remains essential for the successful implementation of medication reconciliation and medication review in geriatric settings and among frail older people.

Organizational considerations for the implementation of a computerized physician order entry.

Several studies stressed that the introduction of CPOE applications deteriorates the doctor-nurse communication. But there are many factors that might influence communication behaviors, as for example the way these communications are organized. The present study aims at showing that the impact of a CPOE system on the cooperative activities can be controlled given that a good understanding of the cooperative workflows support the implementation. By analyzing the doctors-nurses communications during the medication use process, the study demonstrates that the technical system has no impact on the cooperative activities within a given organization. CPOE does not induce differences in the dialogs' durations and contents.