Detectability of use errors in summative usability tests of medical devices: Impact of the test environment.

The regulations on summative usability evaluations of medical devices (MDs) emphasize that the test environment must have sufficient ecological validity for generalization to real-life use. Here, we examined the influence of environmental fidelity (a component of ecological validity) on the detectability of MD use errors. A total of 140 participants participated in a summative usability evaluation of an anaphylactic shock auto-injector device under either a high-fidelity condition or a condition acceptable from the manufacturer's perspective, lower-fidelity condition. The numbers of errors detected in each condition were compared by applying descriptive statistics and logistic and Poisson multivariate regressions. We found that the level of fidelity did not influence the overall number of use errors detected but did influence the detection of certain use errors. To optimize the test environment and increase the detection of use errors, each environmental feature's role in the test task should first be examined.

One Health: Insights from Organizational & Social, Technology Assessment and Human Factors Perspectives.

OBJECTIVES: To offer diverse but complementary perspectives on how biomedical and health informatics can be informed by and help to achieve the vision of One Health. METHODS: Overview of key considerations and critical discussion of common themes, barriers and opportunities, based on collaborative review by International Medical Informatics Association (IMIA) working group members active in related fields. RESULTS: Health and care systems are complex sociotechnical systems that need explicit design and implementation strategies to align with the goals of One Health. The evidence-based health informatics paradigm and associated frameworks for evaluation of digital health technologies need to broaden their scope to take full account of the One Health approach. Informatics has specific contributions to make to One Health, for example by improved user experience reducing energy consumption and effective app design enhancing medication adherence. CONCLUSIONS: One Health is inherently intertwined with ergonomic, sociotechnical and evaluation perspectives in biomedical and health informatics. Health is a planetary issue that requires interdisciplinary collaborative action. The theories and principles of biomedical and health informatics offer many opportunities to transform digital health technology to better serve the One Health agenda.

The medication self-management work system of patients and informal carers from a human factors & ergonomics perspective: A scoping review protocol.

Background: Healthcare is increasingly delivered closer to the patients' homes, which increases the level of responsibility that patients and informal carers take for managing their medication-taking, although this is associated with hazards. Medication self-management has been conceptualised as work taking place in non-formal settings ( e.g., households), which are complex systems. Human factors and ergonomics (HFE) models provide a framework for studying such systems. The Systems Engineering Initiative for Patient Safety (SEIPS) is one framework that considers work system elements and how they interact with each other to shape processes that lead to outcomes ( e.g., safety). Given the increasing amount of diverse research on patient and carer work and on system-shaping factors, the objectives of this review are to: (i) identify available evidence in a structured and systems-oriented way, (ii) explore approaches that have been applied and (iii) highlight research gaps. Methods: An evidence-informed patient, public and carer involvement (PPCI) approach will be implemented at all post-protocol stages to ensure the relevance, uptake and translation of the scoping review. The review will systematically search MEDLINE, Embase, PsycInfo, CINAHL and Web of Science to identify relevant qualitative studies. The methodological approach will be guided by Johanna Briggs Institute methodology and will be reported according to the PRISMA-ScR standards. Data charting and qualitative content analysis directed by SEIPS will explore how the work system and its constituting elements have been described in the literature and identify specific gaps and opportunities for future research. Borrowing from realist approaches, included studies will be assessed in terms of richness and relevance to our review question. Discussion: Strengths of this scoping review include PPCI and a converging focus on medication safety, medication self-management and HFE. Ultimately, this approach will advance our understanding of this complex system and guide opportunities to broaden and strengthen the evidence base.

Organizational and Human Factors in Green Medical Informatics - A Case Study in Dutch Hospitals.

Medical Informatics brings methods and solutions that could support reducing healthcare's ecological footprint. Initial frameworks for Green Medical Informatics solutions are available, however these do not address organizational and human factors. Including these factors in evaluation or analysis of (technical) interventions aimed at making healthcare more sustainable, is essential for improving usability as well as effectiveness of these interventions. Interviews with healthcare professionals from Dutch hospitals led to preliminary insights into which organizational and human factors impact the implementation and adoption of sustainable solutions. Results indicate that forming multi-disciplinary teams is considered an important factor for realizing intended outcomes in terms of reducing carbon emissions and waste. Some other key factors mentioned are formalizing tasks, allocating budget and time, creating awareness and changing protocols to promote sustainable diagnosis and treatment procedures.

Considering Work Systems and Processes in Assessing the Impact of a CDSS Intervention: Preliminary Results.

The DetecIP project aims to implement multifactorial dynamic rules within a computerized decision support system (CDSS) for pharmaceutical analysis of orders to reduce the rate and severity of iatrogenic hyperkalemia and acute kidney injury. However, understanding the impact of this intervention (if any) requires that the way in which it influences the work systems and processes also be studied. This study presents the preliminary results of the analysis of the work contexts in which these rules will be implemented. A series of semi-structured interviews exploring the dimensions of the systems engineering initiative for patient safety (SEIPS) were conducted with healthcare professionals involved in the prevention and management of iatrogenic risks in five hospital units. Data were analyzed to identify current barriers and facilitators to the prevention and management of iatrogenic risks. Preliminary results from a geriatric unit and a cardiology unit reveal that, despite overall similarities in work processes, differences in the availability and location of physicians and clinical pharmacists influence how iatrogenic risks are managed. These contextual differences could influence the impact of the new CDSS rules once implemented.

Inter-Professional Communications During Follow-Up of Type 2 Diabetes Patients: An Exploratory Study.

Follow-up of patients with type 2 diabetes mellitus (T2DM) involves several healthcare professionals. The quality of their communication is crucial for optimizing care. This exploratory work aims to characterize those communications and their problems. Interviews were performed with general practitioners (GP), patients and other professionals. Data were analyzed deductively, and results were structured through a people map. We performed 25 interviews. GP, patients, nurses, community pharmacists, medical specialists and diabetologists are the main actors of the T2DM patients' follow-up. Three communication issues were identified: difficulties in reaching the hospital diabetologist, delays in receiving reports, and difficulties for patient to transmit information. Results were discussed in terms of tools, care pathways and new roles to support communications during T2DM patients' follow-up.

Comparison of the validity, perceived usefulness, and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability.

OBJECTIVE: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aimed to compare their convergent validity, perceived usability, usefulness, strengths, and weaknesses, as well as users' preferences. METHOD: To evaluate convergent validity, two experts mapped TEMAS' items against I-MeDeSA's items with respect to the usability dimensions they target. To assess perceived usability, usefulness, strengths, and weaknesses of both tools, staff with expertise in their medication alerting system were asked to use French versions of the TEMAS and I-MeDeSA. After the use of each tool, participants were asked to complete the System Usability Scale (SUS) and answer questions about the understandability and usefulness of each tool. Finally, participants were asked to name their preferred tool. Numeric scores were statistically compared. Free-text responses were analyzed using an inductive approach. RESULTS: Forty-five participants from 10 hospitals took part in the study. In terms of convergent validity, I-MeDeSA focuses more on the usability of the graphical user interface while TEMAS considers a wider range of usability principles. Both tools have a fair level of perceived usability (I-MeDeSA' SUS score = 61.85 and TEMAS' SUS score = 62.87), but results highlight that revisions are necessary to both tools to improve their usability. Participants found TEMAS more useful than I-MeDeSA (t = -3.63, p =.005) and had a clear preference for TEMAS to identify problems in formative evaluation (39 of 45; 0.867, p <.001) and to compare the usability of alert systems during the procurement process (36 of 45; 0.8, p <.001). CONCLUSIONS: The TEMAS is perceived as more useful and is preferred by participants. The I-MeDeSA seems more relevant for quick evaluations that focus on the graphical user interface. The TEMAS seems to be more suitable for in-depth usability evaluations of alert systems. Even if both tools are perceived to be equally usable, they suffer from wording, instructional, and organizational problems that hinder their use. The results of this study will be used to improve the design of I-MeDeSA and TEMAS.

Medication-Related Outcomes and Health Equity: Evidence for Pharmaceutical Care.

Marginalised people experience diminished access to pharmaceutical care and worse medication-related outcomes than the general population. Health equity is a global priority. This article explores the key evidence of health inequity and medication use, structures the causes and contributory factors and suggests opportunities that can be taken to advance the pharmaceutical care agenda so as to achieve health equity. The causes of, and contributors to, this inequity are multi-fold, with patient- and person-related factors being the most commonly reported. Limited evidence is available to identify risk factors related to other aspects of a personal medication use system, such as technology, tasks, tools and the internal and the external environments. Multiple opportunities exist to enhance equity in medication-related outcomes through pharmaceutical care research and practice. To optimise the effects and the sustainable implementation of these opportunities, it is important to (1) ensure the meaningful inclusion and engagement of members of marginalised groups, (2) use a person-centred approach and (3) apply a systems-based approach to address all of the necessary components of a system that interact and form a network as work processes that produce system outcomes.

Integration of a new technology into a work system: a case study of a smart drinking glass in French nursing homes.

PURPOSE: To reduce the risk of dehydration in older adults, the French company Auxivia has developed a smart drinking glass (SDG) that can measure the amount of water drunk. The present study looked at the various work systems (WSs) designed for use of the SDG in a nursing home. The study's objectives were to (i) determine the WSs' impact on the staff's ability to comply with the device's prerequisites and ensure the device's effective use and (ii) draw up guidelines on designing work systems. MATERIALS AND METHODS: At three nursing homes in France, two independent observers performed 9 h of observations at each site and a total of 29 interviews. RESULTS: Decisions concerning implementation and the resulting WSs have an impact on the tasks to be performed, the tasks' inherent constraints and the use of the SDG. It is essential to take account of the sociotechnical system as a whole before integrating a technology. Ideally, the introduction of an SDG will go unnoticed by staff and residents; however, our results emphasize the value of highlighting work constraints via a human factors analysis. CONCLUSIONS: It is essential to take account of sociotechnical WSs as a whole when integrating a technology.Implications for rehabilitationFor caregivers, better measurement of the amount of water drunk by older adults might help to prevent dehydration.For service providers, better knowledge of how work systems influence the nursing home staff's ability to comply with the smart drinking glass's prerequisites might improve the device implementation process.For managers, a nursing home's work systems should take account of the smart drinking glass's prerequisites on one hand and the home's environment and constraints on the other.

Definition of a Practical Taxonomy for Referencing Data Quality Problems in Health Care Databases.

INTRODUCTION: Health care information systems can generate and/or record huge volumes of data, some of which may be reused for research, clinical trials, or teaching. However, these databases can be affected by data quality problems; hence, an important step in the data reuse process consists in detecting and rectifying these issues. With a view to facilitating the assessment of data quality, we developed a taxonomy of data quality problems in operational databases. MATERIAL: We searched the literature for publications that mentioned "data quality problems," "data quality taxonomy," "data quality assessment," or "dirty data." The publications were then reviewed, compared, summarized, and structured using a bottom-up approach, to provide an operational taxonomy of data quality problems. The latter were illustrated with fictional examples (though based on reality) from clinical databases. RESULTS: Twelve publications were selected, and 286 instances of data quality problems were identified and were classified according to six distinct levels of granularity. We used the classification defined by Oliveira et al to structure our taxonomy. The extracted items were grouped into 53 data quality problems. DISCUSSION: This taxonomy facilitated the systematic assessment of data quality in databases by presenting the data's quality according to their granularity. The definition of this taxonomy is the first step in the data cleaning process. The subsequent steps include the definition of associated quality assessment methods and data cleaning methods. CONCLUSION: Our new taxonomy enabled the classification and illustration of 53 data quality problems found in hospital databases.

Improving the usability and usefulness of computerized decision support systems for medication review by clinical pharmacists: A convergent, parallel evaluation.

BACKGROUND: Computerized decision support systems (CDSSs) help hospital-based clinical pharmacists to perform medication reviews and so are promising tools for improving medication safety. However, their poor usability can reduce effectiveness and acceptability. OBJECTIVES: To evaluate the usability and perceived usefulness of a CDSS for medication review by hospital-based pharmacists and to draw up guidelines on improving its usability. METHODS: We performed a convergent, parallel evaluation. Firstly, three researchers conducted a heuristic evaluation of the CDSS. Secondly, clinical pharmacists who use the CDSS filled out the Usefulness, Satisfaction and Ease of Use (USE) questionnaire. Lastly, semi-structured interviews with the pharmacists enabled us to understand their opinions and experiences. The results of the heuristic evaluation were used to identify potential improvements in the CDSS. We performed a statistical analysis of the USE questionnaire data. Interviews were analyzed based on the unified theory of acceptance and use of technology (UTAUT), together with a task-technology fit model. The results generated by these three approaches were compared in order to determine convergences and divergences, identify challenges related to the usability and usefulness of the CDSS, and draw up guidelines for its improvement. RESULTS: Forty-seven usability problems were discovered; they variously concerned the graphical user interface, the pharmacists' needs, and the medication review model implemented in the CDSS. Only the "usefulness" dimension of the USE was not scored positively. All the UTAUT dimensions and the task-technology fit dimension emerged in the interviews. Cross-comparisons of the results from the three approaches led to the identification of four challenges and the corresponding formulation of 23 guidelines. CONCLUSIONS: The guidelines developed here should help to improve the design and acceptability of CDSSs. Hence, CDSSs will be able to assist clinical pharmacists more fully with their medication reviews and help to further improve patient safety.

Development and usage of an anesthesia data warehouse: lessons learnt from a 10-year project.

This paper describes the development and implementation of an anesthesia data warehouse in the Lille University Hospital. We share the lessons learned from a ten-year project and provide guidance for the implementation of such a project. Our clinical data warehouse is mainly fed with data collected by the anesthesia information management system and hospital discharge reports. The data warehouse stores historical and accurate data with an accuracy level of the day for administrative data, and of the second for monitoring data. Datamarts complete the architecture and provide secondary computed data and indicators, in order to execute queries faster and easily. Between 2010 and 2021, 636 784 anesthesia records were integrated for 353 152 patients. We reported the main concerns and barriers during the development of this project and we provided 8 tips to handle them. We have implemented our data warehouse into the OMOP common data model as a complementary downstream data model. The next step of the project will be to disseminate the use of the OMOP data model for anesthesia and critical care, and drive the trend towards federated learning to enhance collaborations and multicenter studies.

Master's Degree in Health Data Science: Implementation and Assessment After Five Years.

Health data science is an emerging discipline that bridges computer science, statistics and health domain knowledge. This consists of taking advantage of the large volume of data, often complex, to extract information to improve decision-making. We have created a Master's degree in Health Data Science to meet the growing need for data scientists in companies and institutions. The training offers, over two years, courses covering computer science, mathematics and statistics, health and biology. With more than 60 professors and lecturers, a total of 835 hours of classes (not including the mandatory 5 months of internship per year), this curriculum has enrolled a total of 53 students today. The feedback from the students and alumni allowed us identifying new needs in terms of training, which may help us to adapt the program for the coming academic years. In particular, we will offer an additional module covering data management, from the edition of the clinical report form to the implementation of a data warehouse with an ETL process. Git and application lifecycle management will be included in programming courses or multidisciplinary projects.

Overcoming Challenges to Inclusive User-based Testing of Health Information Technology with Vulnerable Older Adults: Recommendations from a Human Factors Engineering Expert Inquiry.

OBJECTIVES: Involving representative users in usability testing of health information technology (HIT) is central to user-centered design. However, (vulnerable) older adults as representative users have unique requirements. Aging processes may affect physical capabilities and cognitive skills, which can hamper testing with this demographic and may require special attention and revised protocols. This study was performed to provide expert-based recommendations for HIT user-testing with (vulnerable) older adults to support inclusive HIT design and evaluation. METHODS: First, we conducted a structured workshop with ten experts in HIT implementation and research, recruited through purposeful sampling, to generate insights into how characteristics of older adults may influence user-testing. Next, five Human Factor researchers experienced in HIT user-testing with (vulnerable) older adults validated the results and provided additional textual insights to gain consensus on the most important recommendations. A thematic analysis was performed on the resulting inquiries. Applied codes were based on the User-Centered Design framework. RESULTS: The analysis resulted in nine recommendations for user-testing of HIT with older adults, divided into three main themes: (1) empathetic approach and trust-building, (2) new requirements for testing and study design, and (3) adjustments to usability evaluation methods. For each theme a checklist of relevant items to follow-up on the recommendation is provided. CONCLUSIONS: The recommendations generated through expert inquiry contribute to more effective usability testing of HIT with older adults. This provides an important step towards improved accessibility of HIT amongst older adults through inclusive user-centered design.

Improving Parents' Experience in the Pediatric Emergency Waiting Room: Researching the Most Optimal Design for an Information Tool.

To improve parents' experience in the pediatric emergency waiting room by means of an information tool, the information needs of parents and pediatric patients need to be determined. It is necessary to find an optimal design for the information needed to be understandable by and useful to the parents. This paper presents the first step of designing an information tool aiming to improve patient/parent experience, encourage the understanding of the care process, and optimize the patient-caregiver relationship. In order to acquire insight into parents' information needs, exploratory interviews were conducted and benchmark research had been performed. Furthermore, one focus group session with parents was conducted to identify their information needs, determine the format of the information tool, and visualize the best ideas by using paper mock-ups. The focus group concluded that parents prefer an information screen to a smartphone application. The designed mock-up visualizes and explains the care process, and represents one's position in the waiting queue. The mock-up was evaluated through questionnaires.

Awareness and Perception of Google® Reviews Among French GPs.

The objective of this study was to assess the perception of French general practitioners (GPs) and the impact of the Google® online physician rating system. We questioned the French GPs with a self-administered questionnaire. A total of 412 GPs had answered the complete questionnaire. 83.25% of respondents did not validate the relevance of the physician rating websites. The most decried limitations were the lack of validity of these opinions and the negative impact on physicians.

Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study.

BACKGROUND: A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula is an important part of search engine evaluation. However, the methods currently used to evaluate ranking formulae mainly collect quantitative data and do not gather qualitative data, which help to understand what needs to be improved to tailor the formulae to their end users. OBJECTIVE: This study aims to evaluate 2 different parameter settings of the ranking formula of LiSSa (the French acronym for scientific literature in health care; Department of Medical Informatics and Information), a tool that provides access to health scientific literature in French, to adapt the formula to the needs of the end users. METHODS: To collect quantitative and qualitative data, user tests were carried out with representative end users of LiSSa: 10 general practitioners and 10 registrars. Participants first assessed the relevance of the search results and then rated the ranking criteria used in the 2 formulae. Verbalizations were analyzed to characterize each criterion. RESULTS: A formula that prioritized articles representing a consensus in the field was preferred. When users assess an article's relevance, they judge its topic, methods, and value in clinical practice. CONCLUSIONS: Following the evaluation, several improvements were implemented to give more weight to articles that match the search topic and to downgrade articles that have less informative or scientific value for the reader. Applying a qualitative methodology generates valuable user inputs to improve the ranking formula and move toward a highly usable search engine.

The Optimal Sample Size for Usability Testing, From the Manufacturer's Perspective: A Value-of-Information Approach.

OBJECTIVES: For medical devices, a usability assessment is mandatory for market access; the objective is to detect potentially harmful use errors that stem from the device's design. The manufacturer assesses the final version of the device and determines the risk-benefit ratio for remaining errors. Nevertheless, the decision rule currently used to determine the sample size for this testing has statistical limitations and the lack of a clear decision-making perspective. METHODS: As an alternative, we developed a value-of-information analysis from the medical device manufacturer's perspective. The consequences of use errors not detected during usability testing and the errors' probability of occurrence were embedded in a loss function. The value of further testing was assessed as a reduction in the expected loss for the manufacturer. The optimal sample size was determined using the expected net benefit of sampling (ENBS) (the difference between the value provided by new participants and the cost of their inclusion). RESULTS: The value-of-information approach was applied to a real usability test of a needle-free adrenaline autoinjector. The initial estimate (performed on the first n = 20 participants) gave an optimal sample size of 100 participants and an ENBS of €255 453. This estimation was updated iteratively as new participants were included. After the inclusion of 90 participants, the ENBS was null for any sample size; hence, the cost of adding more participants outweighed the expected value of information, and therefore, the study could be stopped. CONCLUSIONS: On the basis of these results, our method seems to be highly suitable for sample size estimation in the usability testing of medical devices before market access.