Barriers encountered with clinical data warehouses: Recommendations from a focus group.

BACKGROUND AND OBJECTIVE: The increasing implementation and use of electronic health records over the last few decades has made a significant volume of clinical data being available. Over the past 20 years, hospitals have also adopted and implemented data warehouse technology to facilitate the reuse of administrative and clinical data for research. However, the implementation of clinical data warehouses encounters a set of barriers: ethical, legislative, technical, human and organizational. This paper proposes an overview of difficulties and barriers encountered during a clinical data warehouse (CDW) development and implementation project. METHODS: We conducted a focus group at the 2023 Medical Informatics Europe Conference and invited professionals involved in the implementation of CDW. These experts described their CDW and the difficulties and barriers they encountered at each phase: (i) launching of the data warehouse project, (ii) implementing the data warehouse and (iii) using a data warehouse in routine operations. They were also asked to propose solutions they were able to implement to address the barriers previously reported. RESULTS: After synthesis and consensus, a total of 26 barriers were identified, 10 pertained to tasks, 5 to tools and technologies, 4 to persons, 4 to organization, and 3 to the external environment. To address these challenges, a set of 15 practical recommendations was offered, covering essential aspects such as governance, stakeholder engagement, interdisciplinary collaboration, and external expertise utilization. CONCLUSIONS: These recommendations serve as a valuable resource for healthcare institutions seeking to establish and optimize CDWs, offering a roadmap for leveraging clinical data for research, quality enhancement, and improved patient care.

Usability Checklists for Health Technology: Case Study and Experts' Opinions.

Application of usability evaluations throughout the health technology lifecycle is necessary to improve the efficiency, safety, and effectiveness of health service delivery. Unfortunately, technology vendors and healthcare organizations may not have funding, time or expertise to conduct usability studies. In this paper, we describe how usability checklists can potentially fill this gap. First, we introduce a case study using a checklist to identify usability issues with a primary care dashboard. Then we provide an expert summary of the strengths and limitations of usability checklists. Findings suggest that checklists are efficient to identify important usability issues. They can be used effectively by project team members - including clinicians - without formal usability training. However, checklists should complement rather than replace usability evaluations with representative users.

Usability Evaluation Ecological Validity: Is More Always Better?

BACKGROUND: The ecological validity associated with usability testing of health information technologies (HITs) can affect test results and the predictability of real-world performance. It is, therefore, necessary to identify conditions with the greatest effect on validity. METHOD: We conducted a comparative analysis of two usability testing conditions. We tested a HIT designed for anesthesiologists to detect pain signals and compared two fidelity levels of ecological validity. We measured the difference in the number and type of use errors identified between high and low-fidelity experimental conditions. RESULTS: We identified the same error types in both test conditions, although the number of errors varied as a function of the condition. The difference in total error counts was relatively modest and not consistent across levels of severity. CONCLUSIONS: Increasing ecological validity does not invariably increase the ability to detect use errors. Our findings suggest that low-fidelity tests are an efficient way to identify and mitigate usability issues affecting ease of use, effectiveness, and safety. We believe early low-fidelity testing is an efficient but underused way to maximize the value of usability testing.

Detectability of use errors in summative usability tests of medical devices: Impact of the test environment.

The regulations on summative usability evaluations of medical devices (MDs) emphasize that the test environment must have sufficient ecological validity for generalization to real-life use. Here, we examined the influence of environmental fidelity (a component of ecological validity) on the detectability of MD use errors. A total of 140 participants participated in a summative usability evaluation of an anaphylactic shock auto-injector device under either a high-fidelity condition or a condition acceptable from the manufacturer's perspective, lower-fidelity condition. The numbers of errors detected in each condition were compared by applying descriptive statistics and logistic and Poisson multivariate regressions. We found that the level of fidelity did not influence the overall number of use errors detected but did influence the detection of certain use errors. To optimize the test environment and increase the detection of use errors, each environmental feature's role in the test task should first be examined.

One Health: Insights from Organizational & Social, Technology Assessment and Human Factors Perspectives.

OBJECTIVES: To offer diverse but complementary perspectives on how biomedical and health informatics can be informed by and help to achieve the vision of One Health. METHODS: Overview of key considerations and critical discussion of common themes, barriers and opportunities, based on collaborative review by International Medical Informatics Association (IMIA) working group members active in related fields. RESULTS: Health and care systems are complex sociotechnical systems that need explicit design and implementation strategies to align with the goals of One Health. The evidence-based health informatics paradigm and associated frameworks for evaluation of digital health technologies need to broaden their scope to take full account of the One Health approach. Informatics has specific contributions to make to One Health, for example by improved user experience reducing energy consumption and effective app design enhancing medication adherence. CONCLUSIONS: One Health is inherently intertwined with ergonomic, sociotechnical and evaluation perspectives in biomedical and health informatics. Health is a planetary issue that requires interdisciplinary collaborative action. The theories and principles of biomedical and health informatics offer many opportunities to transform digital health technology to better serve the One Health agenda.

The medication self-management work system of patients and informal carers from a human factors & ergonomics perspective: A scoping review protocol.

Background: Healthcare is increasingly delivered closer to the patients' homes, which increases the level of responsibility that patients and informal carers take for managing their medication-taking, although this is associated with hazards. Medication self-management has been conceptualised as work taking place in non-formal settings ( e.g., households), which are complex systems. Human factors and ergonomics (HFE) models provide a framework for studying such systems. The Systems Engineering Initiative for Patient Safety (SEIPS) is one framework that considers work system elements and how they interact with each other to shape processes that lead to outcomes ( e.g., safety). Given the increasing amount of diverse research on patient and carer work and on system-shaping factors, the objectives of this review are to: (i) identify available evidence in a structured and systems-oriented way, (ii) explore approaches that have been applied and (iii) highlight research gaps. Methods: An evidence-informed patient, public and carer involvement (PPCI) approach will be implemented at all post-protocol stages to ensure the relevance, uptake and translation of the scoping review. The review will systematically search MEDLINE, Embase, PsycInfo, CINAHL and Web of Science to identify relevant qualitative studies. The methodological approach will be guided by Johanna Briggs Institute methodology and will be reported according to the PRISMA-ScR standards. Data charting and qualitative content analysis directed by SEIPS will explore how the work system and its constituting elements have been described in the literature and identify specific gaps and opportunities for future research. Borrowing from realist approaches, included studies will be assessed in terms of richness and relevance to our review question. Discussion: Strengths of this scoping review include PPCI and a converging focus on medication safety, medication self-management and HFE. Ultimately, this approach will advance our understanding of this complex system and guide opportunities to broaden and strengthen the evidence base.

Organizational and Human Factors in Green Medical Informatics - A Case Study in Dutch Hospitals.

Medical Informatics brings methods and solutions that could support reducing healthcare's ecological footprint. Initial frameworks for Green Medical Informatics solutions are available, however these do not address organizational and human factors. Including these factors in evaluation or analysis of (technical) interventions aimed at making healthcare more sustainable, is essential for improving usability as well as effectiveness of these interventions. Interviews with healthcare professionals from Dutch hospitals led to preliminary insights into which organizational and human factors impact the implementation and adoption of sustainable solutions. Results indicate that forming multi-disciplinary teams is considered an important factor for realizing intended outcomes in terms of reducing carbon emissions and waste. Some other key factors mentioned are formalizing tasks, allocating budget and time, creating awareness and changing protocols to promote sustainable diagnosis and treatment procedures.

Considering Work Systems and Processes in Assessing the Impact of a CDSS Intervention: Preliminary Results.

The DetecIP project aims to implement multifactorial dynamic rules within a computerized decision support system (CDSS) for pharmaceutical analysis of orders to reduce the rate and severity of iatrogenic hyperkalemia and acute kidney injury. However, understanding the impact of this intervention (if any) requires that the way in which it influences the work systems and processes also be studied. This study presents the preliminary results of the analysis of the work contexts in which these rules will be implemented. A series of semi-structured interviews exploring the dimensions of the systems engineering initiative for patient safety (SEIPS) were conducted with healthcare professionals involved in the prevention and management of iatrogenic risks in five hospital units. Data were analyzed to identify current barriers and facilitators to the prevention and management of iatrogenic risks. Preliminary results from a geriatric unit and a cardiology unit reveal that, despite overall similarities in work processes, differences in the availability and location of physicians and clinical pharmacists influence how iatrogenic risks are managed. These contextual differences could influence the impact of the new CDSS rules once implemented.

Comparison of the validity, perceived usefulness, and usability of I-MeDeSA and TEMAS, two tools to evaluate alert system usability.

OBJECTIVE: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aimed to compare their convergent validity, perceived usability, usefulness, strengths, and weaknesses, as well as users' preferences. METHOD: To evaluate convergent validity, two experts mapped TEMAS' items against I-MeDeSA's items with respect to the usability dimensions they target. To assess perceived usability, usefulness, strengths, and weaknesses of both tools, staff with expertise in their medication alerting system were asked to use French versions of the TEMAS and I-MeDeSA. After the use of each tool, participants were asked to complete the System Usability Scale (SUS) and answer questions about the understandability and usefulness of each tool. Finally, participants were asked to name their preferred tool. Numeric scores were statistically compared. Free-text responses were analyzed using an inductive approach. RESULTS: Forty-five participants from 10 hospitals took part in the study. In terms of convergent validity, I-MeDeSA focuses more on the usability of the graphical user interface while TEMAS considers a wider range of usability principles. Both tools have a fair level of perceived usability (I-MeDeSA' SUS score = 61.85 and TEMAS' SUS score = 62.87), but results highlight that revisions are necessary to both tools to improve their usability. Participants found TEMAS more useful than I-MeDeSA (t = -3.63, p =.005) and had a clear preference for TEMAS to identify problems in formative evaluation (39 of 45; 0.867, p <.001) and to compare the usability of alert systems during the procurement process (36 of 45; 0.8, p <.001). CONCLUSIONS: The TEMAS is perceived as more useful and is preferred by participants. The I-MeDeSA seems more relevant for quick evaluations that focus on the graphical user interface. The TEMAS seems to be more suitable for in-depth usability evaluations of alert systems. Even if both tools are perceived to be equally usable, they suffer from wording, instructional, and organizational problems that hinder their use. The results of this study will be used to improve the design of I-MeDeSA and TEMAS.

Inter-Professional Communications During Follow-Up of Type 2 Diabetes Patients: An Exploratory Study.

Follow-up of patients with type 2 diabetes mellitus (T2DM) involves several healthcare professionals. The quality of their communication is crucial for optimizing care. This exploratory work aims to characterize those communications and their problems. Interviews were performed with general practitioners (GP), patients and other professionals. Data were analyzed deductively, and results were structured through a people map. We performed 25 interviews. GP, patients, nurses, community pharmacists, medical specialists and diabetologists are the main actors of the T2DM patients' follow-up. Three communication issues were identified: difficulties in reaching the hospital diabetologist, delays in receiving reports, and difficulties for patient to transmit information. Results were discussed in terms of tools, care pathways and new roles to support communications during T2DM patients' follow-up.

Medication-Related Outcomes and Health Equity: Evidence for Pharmaceutical Care.

Marginalised people experience diminished access to pharmaceutical care and worse medication-related outcomes than the general population. Health equity is a global priority. This article explores the key evidence of health inequity and medication use, structures the causes and contributory factors and suggests opportunities that can be taken to advance the pharmaceutical care agenda so as to achieve health equity. The causes of, and contributors to, this inequity are multi-fold, with patient- and person-related factors being the most commonly reported. Limited evidence is available to identify risk factors related to other aspects of a personal medication use system, such as technology, tasks, tools and the internal and the external environments. Multiple opportunities exist to enhance equity in medication-related outcomes through pharmaceutical care research and practice. To optimise the effects and the sustainable implementation of these opportunities, it is important to (1) ensure the meaningful inclusion and engagement of members of marginalised groups, (2) use a person-centred approach and (3) apply a systems-based approach to address all of the necessary components of a system that interact and form a network as work processes that produce system outcomes.

Integration of a new technology into a work system: a case study of a smart drinking glass in French nursing homes.

PURPOSE: To reduce the risk of dehydration in older adults, the French company Auxivia has developed a smart drinking glass (SDG) that can measure the amount of water drunk. The present study looked at the various work systems (WSs) designed for use of the SDG in a nursing home. The study's objectives were to (i) determine the WSs' impact on the staff's ability to comply with the device's prerequisites and ensure the device's effective use and (ii) draw up guidelines on designing work systems. MATERIALS AND METHODS: At three nursing homes in France, two independent observers performed 9 h of observations at each site and a total of 29 interviews. RESULTS: Decisions concerning implementation and the resulting WSs have an impact on the tasks to be performed, the tasks' inherent constraints and the use of the SDG. It is essential to take account of the sociotechnical system as a whole before integrating a technology. Ideally, the introduction of an SDG will go unnoticed by staff and residents; however, our results emphasize the value of highlighting work constraints via a human factors analysis. CONCLUSIONS: It is essential to take account of sociotechnical WSs as a whole when integrating a technology.Implications for rehabilitationFor caregivers, better measurement of the amount of water drunk by older adults might help to prevent dehydration.For service providers, better knowledge of how work systems influence the nursing home staff's ability to comply with the smart drinking glass's prerequisites might improve the device implementation process.For managers, a nursing home's work systems should take account of the smart drinking glass's prerequisites on one hand and the home's environment and constraints on the other.

Definition of a Practical Taxonomy for Referencing Data Quality Problems in Health Care Databases.

INTRODUCTION: Health care information systems can generate and/or record huge volumes of data, some of which may be reused for research, clinical trials, or teaching. However, these databases can be affected by data quality problems; hence, an important step in the data reuse process consists in detecting and rectifying these issues. With a view to facilitating the assessment of data quality, we developed a taxonomy of data quality problems in operational databases. MATERIAL: We searched the literature for publications that mentioned "data quality problems," "data quality taxonomy," "data quality assessment," or "dirty data." The publications were then reviewed, compared, summarized, and structured using a bottom-up approach, to provide an operational taxonomy of data quality problems. The latter were illustrated with fictional examples (though based on reality) from clinical databases. RESULTS: Twelve publications were selected, and 286 instances of data quality problems were identified and were classified according to six distinct levels of granularity. We used the classification defined by Oliveira et al to structure our taxonomy. The extracted items were grouped into 53 data quality problems. DISCUSSION: This taxonomy facilitated the systematic assessment of data quality in databases by presenting the data's quality according to their granularity. The definition of this taxonomy is the first step in the data cleaning process. The subsequent steps include the definition of associated quality assessment methods and data cleaning methods. CONCLUSION: Our new taxonomy enabled the classification and illustration of 53 data quality problems found in hospital databases.

Improving the usability and usefulness of computerized decision support systems for medication review by clinical pharmacists: A convergent, parallel evaluation.

BACKGROUND: Computerized decision support systems (CDSSs) help hospital-based clinical pharmacists to perform medication reviews and so are promising tools for improving medication safety. However, their poor usability can reduce effectiveness and acceptability. OBJECTIVES: To evaluate the usability and perceived usefulness of a CDSS for medication review by hospital-based pharmacists and to draw up guidelines on improving its usability. METHODS: We performed a convergent, parallel evaluation. Firstly, three researchers conducted a heuristic evaluation of the CDSS. Secondly, clinical pharmacists who use the CDSS filled out the Usefulness, Satisfaction and Ease of Use (USE) questionnaire. Lastly, semi-structured interviews with the pharmacists enabled us to understand their opinions and experiences. The results of the heuristic evaluation were used to identify potential improvements in the CDSS. We performed a statistical analysis of the USE questionnaire data. Interviews were analyzed based on the unified theory of acceptance and use of technology (UTAUT), together with a task-technology fit model. The results generated by these three approaches were compared in order to determine convergences and divergences, identify challenges related to the usability and usefulness of the CDSS, and draw up guidelines for its improvement. RESULTS: Forty-seven usability problems were discovered; they variously concerned the graphical user interface, the pharmacists' needs, and the medication review model implemented in the CDSS. Only the "usefulness" dimension of the USE was not scored positively. All the UTAUT dimensions and the task-technology fit dimension emerged in the interviews. Cross-comparisons of the results from the three approaches led to the identification of four challenges and the corresponding formulation of 23 guidelines. CONCLUSIONS: The guidelines developed here should help to improve the design and acceptability of CDSSs. Hence, CDSSs will be able to assist clinical pharmacists more fully with their medication reviews and help to further improve patient safety.

Development and usage of an anesthesia data warehouse: lessons learnt from a 10-year project.

This paper describes the development and implementation of an anesthesia data warehouse in the Lille University Hospital. We share the lessons learned from a ten-year project and provide guidance for the implementation of such a project. Our clinical data warehouse is mainly fed with data collected by the anesthesia information management system and hospital discharge reports. The data warehouse stores historical and accurate data with an accuracy level of the day for administrative data, and of the second for monitoring data. Datamarts complete the architecture and provide secondary computed data and indicators, in order to execute queries faster and easily. Between 2010 and 2021, 636 784 anesthesia records were integrated for 353 152 patients. We reported the main concerns and barriers during the development of this project and we provided 8 tips to handle them. We have implemented our data warehouse into the OMOP common data model as a complementary downstream data model. The next step of the project will be to disseminate the use of the OMOP data model for anesthesia and critical care, and drive the trend towards federated learning to enhance collaborations and multicenter studies.

Master's Degree in Health Data Science: Implementation and Assessment After Five Years.

Health data science is an emerging discipline that bridges computer science, statistics and health domain knowledge. This consists of taking advantage of the large volume of data, often complex, to extract information to improve decision-making. We have created a Master's degree in Health Data Science to meet the growing need for data scientists in companies and institutions. The training offers, over two years, courses covering computer science, mathematics and statistics, health and biology. With more than 60 professors and lecturers, a total of 835 hours of classes (not including the mandatory 5 months of internship per year), this curriculum has enrolled a total of 53 students today. The feedback from the students and alumni allowed us identifying new needs in terms of training, which may help us to adapt the program for the coming academic years. In particular, we will offer an additional module covering data management, from the edition of the clinical report form to the implementation of a data warehouse with an ETL process. Git and application lifecycle management will be included in programming courses or multidisciplinary projects.

Overcoming Challenges to Inclusive User-based Testing of Health Information Technology with Vulnerable Older Adults: Recommendations from a Human Factors Engineering Expert Inquiry.

OBJECTIVES: Involving representative users in usability testing of health information technology (HIT) is central to user-centered design. However, (vulnerable) older adults as representative users have unique requirements. Aging processes may affect physical capabilities and cognitive skills, which can hamper testing with this demographic and may require special attention and revised protocols. This study was performed to provide expert-based recommendations for HIT user-testing with (vulnerable) older adults to support inclusive HIT design and evaluation. METHODS: First, we conducted a structured workshop with ten experts in HIT implementation and research, recruited through purposeful sampling, to generate insights into how characteristics of older adults may influence user-testing. Next, five Human Factor researchers experienced in HIT user-testing with (vulnerable) older adults validated the results and provided additional textual insights to gain consensus on the most important recommendations. A thematic analysis was performed on the resulting inquiries. Applied codes were based on the User-Centered Design framework. RESULTS: The analysis resulted in nine recommendations for user-testing of HIT with older adults, divided into three main themes: (1) empathetic approach and trust-building, (2) new requirements for testing and study design, and (3) adjustments to usability evaluation methods. For each theme a checklist of relevant items to follow-up on the recommendation is provided. CONCLUSIONS: The recommendations generated through expert inquiry contribute to more effective usability testing of HIT with older adults. This provides an important step towards improved accessibility of HIT amongst older adults through inclusive user-centered design.

Improving Parents' Experience in the Pediatric Emergency Waiting Room: Researching the Most Optimal Design for an Information Tool.

To improve parents' experience in the pediatric emergency waiting room by means of an information tool, the information needs of parents and pediatric patients need to be determined. It is necessary to find an optimal design for the information needed to be understandable by and useful to the parents. This paper presents the first step of designing an information tool aiming to improve patient/parent experience, encourage the understanding of the care process, and optimize the patient-caregiver relationship. In order to acquire insight into parents' information needs, exploratory interviews were conducted and benchmark research had been performed. Furthermore, one focus group session with parents was conducted to identify their information needs, determine the format of the information tool, and visualize the best ideas by using paper mock-ups. The focus group concluded that parents prefer an information screen to a smartphone application. The designed mock-up visualizes and explains the care process, and represents one's position in the waiting queue. The mock-up was evaluated through questionnaires.

Awareness and Perception of Google® Reviews Among French GPs.

The objective of this study was to assess the perception of French general practitioners (GPs) and the impact of the Google® online physician rating system. We questioned the French GPs with a self-administered questionnaire. A total of 412 GPs had answered the complete questionnaire. 83.25% of respondents did not validate the relevance of the physician rating websites. The most decried limitations were the lack of validity of these opinions and the negative impact on physicians.