Development and usage of an anesthesia data warehouse: lessons learnt from a 10-year project.

This paper describes the development and implementation of an anesthesia data warehouse in the Lille University Hospital. We share the lessons learned from a ten-year project and provide guidance for the implementation of such a project. Our clinical data warehouse is mainly fed with data collected by the anesthesia information management system and hospital discharge reports. The data warehouse stores historical and accurate data with an accuracy level of the day for administrative data, and of the second for monitoring data. Datamarts complete the architecture and provide secondary computed data and indicators, in order to execute queries faster and easily. Between 2010 and 2021, 636 784 anesthesia records were integrated for 353 152 patients. We reported the main concerns and barriers during the development of this project and we provided 8 tips to handle them. We have implemented our data warehouse into the OMOP common data model as a complementary downstream data model. The next step of the project will be to disseminate the use of the OMOP data model for anesthesia and critical care, and drive the trend towards federated learning to enhance collaborations and multicenter studies.

The Optimal Sample Size for Usability Testing, From the Manufacturer's Perspective: A Value-of-Information Approach.

OBJECTIVES: For medical devices, a usability assessment is mandatory for market access; the objective is to detect potentially harmful use errors that stem from the device's design. The manufacturer assesses the final version of the device and determines the risk-benefit ratio for remaining errors. Nevertheless, the decision rule currently used to determine the sample size for this testing has statistical limitations and the lack of a clear decision-making perspective. METHODS: As an alternative, we developed a value-of-information analysis from the medical device manufacturer's perspective. The consequences of use errors not detected during usability testing and the errors' probability of occurrence were embedded in a loss function. The value of further testing was assessed as a reduction in the expected loss for the manufacturer. The optimal sample size was determined using the expected net benefit of sampling (ENBS) (the difference between the value provided by new participants and the cost of their inclusion). RESULTS: The value-of-information approach was applied to a real usability test of a needle-free adrenaline autoinjector. The initial estimate (performed on the first n = 20 participants) gave an optimal sample size of 100 participants and an ENBS of €255 453. This estimation was updated iteratively as new participants were included. After the inclusion of 90 participants, the ENBS was null for any sample size; hence, the cost of adding more participants outweighed the expected value of information, and therefore, the study could be stopped. CONCLUSIONS: On the basis of these results, our method seems to be highly suitable for sample size estimation in the usability testing of medical devices before market access.

Assessing the hospital volume-outcome relationship in surgery: a scoping review.

INTRODUCTION: Many recent studies have investigated the hospital volume-outcome relationship in surgery. In some cases, the results have prompted the centralization of surgical activity. However, the methodologies and interpretations differ markedly from one study to another. The objective of the present scoping review was to describe the various features used to assess the volume-outcome relationship: the analyzed datasets, study population, outcome, covariates, confounders, volume modalities, and statistical methods. METHODS AND ANALYSIS: The review was conducted according to a study protocol published in BMJ Open in 2020. Two authors (both of whom had helped to design the study protocol) screened publications independently according to the title, the abstract and then the full text. To ensure exhaustivity, all the papers included by each reviewer went through to the next step. INTERPRETATION: The 403 included studies covered 90 types of surgery, 61 types of outcome, and 72 covariates or potential confounders. 191 (47.5%) studies focussed on oncological surgery and 37.8% focussed visceral or digestive tract surgery. Overall, 86.6% of the studies found a statistically significant volume-outcome relationship, although the findings differed from one type of surgery to another. Furthermore, the types of outcome and the covariates were highly diverse. The majority of studies were performed in Western countries, and oncological and visceral surgical procedures were over-represented; this might limit the generalizability and comparability of the studies' results.

Human factors engineering for medical devices: European regulation and current issues.

A large proportion of the patient injuries or deaths attributable to medical device (MD) misuse can be eliminated and/or mitigated by adopting an effective human factors and ergonomics (HFE) approach. The implementation of a usability engineering process is now mandatory for MD manufacturers seeking to obtain the European Union's CE Mark. Here, we describe the European Union's HFE regulation and highlight the challenges faced by (i) manufacturers implementing this regulation and (ii) regulatory bodies charged with assessing the compliance of usability files. In Europe, 95% of MD manufacturers are small- and medium-sized enterprises; compliance with the CE Mark regulations is a real challenge to their competitiveness. Levels of knowledge about HFE vary greatly from one regulatory organization to another, which can sometimes lead to very different expectations. We also present the specific use-related risk management approach required by the HFE regulation. Lastly, we focus on the limitations of the HFE regulation for MDs and on future HFE challenges in further reducing and/or eliminating MD use errors. The main challenge is the need to go beyond technology design and the premarket assessment and to look at the postproduction stage; the coupling between an MD and a sociotechnical system can lead to consequences that were not predicted during the design process. This implies the need to consider the emerging properties of technologies in use by involving all the stakeholders.

Development, implementation and preliminary evaluation of clinical dashboards in a department of anesthesia.

Clinical dashboards summarize indicators of high-volume patient data in a concise, user-friendly visual format. There are few studies of the use of dashboards to improve professional practice in anesthesiology. The objective of the present study was to describe the user-centered development, implementation and preliminary evaluation of clinical dashboards dealing with anesthesia unit management and quality assessment in a French university medical center. User needs and technical requirements were identified in end user interviews and then synthesized. Several representations were then developed (according to good visualization practice) and submitted to end users for appraisal. Lastly, dashboards were implemented and made accessible for everyday use via the medical center's network. After a period of use, end user feedback on the dashboard platform was collected as a system usability score (range 0 to 100). Seventeen themes (corresponding to 29 questions and 42 indicators) were identified. After prioritization and feasibility assessment, 10 dashboards were ultimately implemented and deployed. The dashboards variously addressed the unit's overall activity, compliance with guidelines on intraoperative hemodynamics, ventilation and monitoring, and documentation of the anesthesia procedure. The mean (standard deviation) system usability score was 82.6 (11.5), which corresponded to excellent usability. We developed clinical dashboards for a university medical center's anesthesia units. The dashboards' deployment was well received by the center's anesthesiologists. The dashboards' impact on activity and practice after several months of use will now have to be assessed.

How to interact with medical terminologies? Formative usability evaluations comparing three approaches for supporting the use of MedDRA by pharmacovigilance specialists.

BACKGROUND: Medical terminologies are commonly used in medicine. For instance, to answer a pharmacovigilance question, pharmacovigilance specialists (PVS) search in a pharmacovigilance database for reports in relation to a given drug. To do that, they first need to identify all MedDRA terms that might have been used to code an adverse reaction in the database, but terms may be numerous and difficult to select as they may belong to different parts of the hierarchy. In previous studies, three tools have been developed to help PVS identify and group all relevant MedDRA terms using three different approaches: forms, structured query-builder, and icons. Yet, a poor usability of the tools may increase PVS' workload and reduce their performance. This study aims to evaluate, compare and improve the three tools during two rounds of formative usability evaluation. METHODS: First, a cognitive walkthrough was performed. Based on the design recommendations obtained from this evaluation, designers made modifications to their tools to improve usability. Once this re-engineering phase completed, six PVS took part in a usability test: difficulties, errors and verbalizations during their interaction with the three tools were collected. Their satisfaction was measured through the System Usability Scale. The design recommendations issued from the tests were used to adapt the tools. RESULTS: All tools had usability problems related to the lack of guidance in the graphical user interface (e.g., unintuitive labels). In two tools, the use of the SNOMED CT to find MedDRA terms hampered their use because French PVS were not used to it. For the most obvious and common terms, the icons-based interface would appear to be more useful. For the less frequently used MedDRA terms or those distributed in different parts of the hierarchy, the structured query-builder would be preferable thanks to its great power and flexibility. The form-based tool seems to be a compromise. CONCLUSION: These evaluations made it possible to identify the strengths of each tool but also their weaknesses to address them before further evaluation. Next step is to assess the acceptability of tools and the expressiveness of their results to help identify and group MedDRA terms.

Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes.

Poor usability of health technology is thought to diminish work system performance, increase error rates and, potentially, harm patients. The present study (i) used a combination of usability evaluation methods to highlight the chain that leads from usability flaws to usage problems experienced by users and, ultimately, to negative patient outcomes, and (ii) validated this approach by studying two different discharge summary production systems. To comply with quality guidelines, the process of drafting and sending discharge summaries is increasingly being automated. However, the usability of these systems may modify their impact (or the absence thereof) in terms of production times and quality, and must therefore be evaluated. Here, we applied three successive techniques for usability evaluation (heuristic evaluation, user testing and field observation) to two discharge summary production systems (underpinned by different technologies). The systems' main usability flaws led respectively to an increase in the time need to produce a discharge summary and the risk of patient misidentification. Our results are discussed with regard to the possibility of linking the usability flaws, usage problems and the negative outcomes by successively applying three methods for evaluating usability (heuristic evaluation, user testing and in situ observations) throughout the system development life cycle.

Evidence-based usability design principles for medication alerting systems.

BACKGROUND: Usability flaws in medication alerting systems may have a negative impact on clinical use and patient safety. In order to prevent the release of alerting systems that contain such flaws, it is necessary to provide designers and evaluators with evidence-based usability design principles. The objective of the present study was to develop a comprehensive, structured list of evidence-based usability design principles for medication alerting systems. METHODS: Nine sets of design principles for medication alerting systems were analyzed, summarized, and structured. We then matched the summarized principles with a list of usability flaws in order to determine the level of underlying evidence. RESULTS: Fifty-eight principles were summarized from the literature and two additional principles were defined, so that each flaw was matched with a principle. We organized the 60 summarized usability design principles into 6 meta-principles, 38 principles, and 16 sub-principles. Only 15 principles were not matched with a usability flaw. The 6 meta-principles respectively covered the improvement of the signal-to-noise ratio, the support for collaborative working, the fit with a clinician's workflow, the data display, the transparency of the alerting system, and the actionable tools to be provided within an alert. CONCLUSIONS: It is possible to develop an evidence-based, structured, comprehensive list of usability design principles that are specific to medication alerting systems and are illustrated by the corresponding usability flaws. This list represents an improvement over the current literature. Each principle is now associated with the best available evidence of its violation. This knowledge may help to improve the usability of medication alerting systems and, ultimately, decrease the harmful consequences of the systems' usability flaws.

Usability flaws of medication-related alerting functions: A systematic qualitative review.

INTRODUCTION: Medication-related alerting functions may include usability flaws that limit their optimal use. A first step on the way to preventing usability flaws is to understand the characteristics of these usability flaws. This systematic qualitative review aims to analyze the type of usability flaws found in medication-related alerting functions. METHOD: Papers were searched via PubMed, Scopus and Ergonomics Abstracts databases, along with references lists. Paper selection, data extraction and data analysis was performed by two to three Human Factors experts. Meaningful semantic units representing instances of usability flaws were the main data extracted. They were analyzed through qualitative methods: categorization following general usability heuristics and through an inductive process for the flaws specific to medication-related alerting functions. MAIN RESULTS: From the 6380 papers initially identified, 26 met all eligibility criteria. The analysis of the papers identified a total of 168 instances of usability flaws that could be classified into 13 categories of usability flaws representing either violations of general usability principles (i.e. they could be found in any system, e.g. guidance and workload issues) or infractions specific to medication-related alerting functions. The latter refer to issues of low signal-to-noise ratio, incomplete content of alerts, transparency, presentation mode and timing, missing alert features, tasks and control distribution. MAIN CONCLUSION: The list of 168 instances of usability flaws of medication-related alerting functions provides a source of knowledge for checking the usability of medication-related alerting functions during their design and evaluation process and ultimately constructs evidence-based usability design principles for these functions.

A substitution method to improve completeness of events documentation in anesthesia records.

AIMS are optimized to find and display data and curves about one specific intervention but is not retrospective analysis on a huge volume of interventions. Such a system present two main limitation; (1) the transactional database architecture, (2) the completeness of documentation. In order to solve the architectural problem, data warehouses were developed to propose architecture suitable for analysis. However, completeness of documentation stays unsolved. In this paper, we describe a method which allows determining of substitution rules in order to detect missing anesthesia events in an anesthesia record. Our method is based on the principle that missing event could be detected using a substitution one defined as the nearest documented event. As an example, we focused on the automatic detection of the start and the end of anesthesia procedure when these events were not documented by the clinicians. We applied our method on a set of records in order to evaluate; (1) the event detection accuracy, (2) the improvement of valid records. For the year 2010-2012, we obtained event detection with a precision of 0.00 (-2.22; 2.00) min for the start of anesthesia and 0.10 (0.00; 0.35) min for the end of anesthesia. On the other hand, we increased by 21.1% the data completeness (from 80.3 to 97.2% of the total database) for the start and the end of anesthesia events. This method seems to be efficient to replace missing "start and end of anesthesia" events. This method could also be used to replace other missing time events in this particular data warehouse as well as in other kind of data warehouses.

Detectability of use errors in summative usability tests of medical devices: Impact of the test environment.

The regulations on summative usability evaluations of medical devices (MDs) emphasize that the test environment must have sufficient ecological validity for generalization to real-life use. Here, we examined the influence of environmental fidelity (a component of ecological validity) on the detectability of MD use errors. A total of 140 participants participated in a summative usability evaluation of an anaphylactic shock auto-injector device under either a high-fidelity condition or a condition acceptable from the manufacturer's perspective, lower-fidelity condition. The numbers of errors detected in each condition were compared by applying descriptive statistics and logistic and Poisson multivariate regressions. We found that the level of fidelity did not influence the overall number of use errors detected but did influence the detection of certain use errors. To optimize the test environment and increase the detection of use errors, each environmental feature's role in the test task should first be examined.

Clinical evaluation of the ADE scorecards as a decision support tool for adverse drug event analysis and medication safety management.

AIMS: The prevention of adverse drug events (ADEs) demands co-ordination of different health care professionals. ADE scorecards are a novel approach to raise the team awareness regarding ADE risks and causes. It makes information on numbers and on possible causes of possible ADE cases available to the clinical team. The aim of the study was to investigate the usage and acceptance of ADE scorecards by healthcare professionals and their impact on rates of possible ADEs. METHODS: ADE scorecards were introduced in three departments of a French hospital. A controlled time series analysis of ADE data was conducted to assess the impact of the ADE scorecards. In addition, qualitative interviews and a standardized survey with all participating staff members were performed. RESULTS: Physicians, nurses and pharmacists found ADE scorecards effective to increase medication safety and recommended future usage. The time-series analysis did not show changes in rates of possible ADEs. CONCLUSION: ADE scorecards appear to be useful to raise awareness of ADE-related issues among professionals. Although the evaluation did not show significant reductions of ADE rates, the participating physicians, nurses and pharmacists believed that the ADE scorecards could contribute to increased patient safety and to a reduction in ADE rates. Strategies need to be designed to integrate ADE scorecards better into the clinical routine and to increase the precision of ADE detection.

Integrating Human Factors in an international research project: lessons learned from the PSIP project.

This paper presents how a Human Factors (HF) engineering approach has been applied to a European Project aiming at preventing Adverse Drug Events. Methods used by HF experts and their main contributions are depicted. Main lessons learned from HF involvement in this project are proposed in terms of methods for ADE prevention interventions and project management.

Organizational and Human Factors in Green Medical Informatics - A Case Study in Dutch Hospitals.

Medical Informatics brings methods and solutions that could support reducing healthcare's ecological footprint. Initial frameworks for Green Medical Informatics solutions are available, however these do not address organizational and human factors. Including these factors in evaluation or analysis of (technical) interventions aimed at making healthcare more sustainable, is essential for improving usability as well as effectiveness of these interventions. Interviews with healthcare professionals from Dutch hospitals led to preliminary insights into which organizational and human factors impact the implementation and adoption of sustainable solutions. Results indicate that forming multi-disciplinary teams is considered an important factor for realizing intended outcomes in terms of reducing carbon emissions and waste. Some other key factors mentioned are formalizing tasks, allocating budget and time, creating awareness and changing protocols to promote sustainable diagnosis and treatment procedures.

Considering Work Systems and Processes in Assessing the Impact of a CDSS Intervention: Preliminary Results.

The DetecIP project aims to implement multifactorial dynamic rules within a computerized decision support system (CDSS) for pharmaceutical analysis of orders to reduce the rate and severity of iatrogenic hyperkalemia and acute kidney injury. However, understanding the impact of this intervention (if any) requires that the way in which it influences the work systems and processes also be studied. This study presents the preliminary results of the analysis of the work contexts in which these rules will be implemented. A series of semi-structured interviews exploring the dimensions of the systems engineering initiative for patient safety (SEIPS) were conducted with healthcare professionals involved in the prevention and management of iatrogenic risks in five hospital units. Data were analyzed to identify current barriers and facilitators to the prevention and management of iatrogenic risks. Preliminary results from a geriatric unit and a cardiology unit reveal that, despite overall similarities in work processes, differences in the availability and location of physicians and clinical pharmacists influence how iatrogenic risks are managed. These contextual differences could influence the impact of the new CDSS rules once implemented.

Medication-Related Outcomes and Health Equity: Evidence for Pharmaceutical Care.

Marginalised people experience diminished access to pharmaceutical care and worse medication-related outcomes than the general population. Health equity is a global priority. This article explores the key evidence of health inequity and medication use, structures the causes and contributory factors and suggests opportunities that can be taken to advance the pharmaceutical care agenda so as to achieve health equity. The causes of, and contributors to, this inequity are multi-fold, with patient- and person-related factors being the most commonly reported. Limited evidence is available to identify risk factors related to other aspects of a personal medication use system, such as technology, tasks, tools and the internal and the external environments. Multiple opportunities exist to enhance equity in medication-related outcomes through pharmaceutical care research and practice. To optimise the effects and the sustainable implementation of these opportunities, it is important to (1) ensure the meaningful inclusion and engagement of members of marginalised groups, (2) use a person-centred approach and (3) apply a systems-based approach to address all of the necessary components of a system that interact and form a network as work processes that produce system outcomes.

Integration of a new technology into a work system: a case study of a smart drinking glass in French nursing homes.

PURPOSE: To reduce the risk of dehydration in older adults, the French company Auxivia has developed a smart drinking glass (SDG) that can measure the amount of water drunk. The present study looked at the various work systems (WSs) designed for use of the SDG in a nursing home. The study's objectives were to (i) determine the WSs' impact on the staff's ability to comply with the device's prerequisites and ensure the device's effective use and (ii) draw up guidelines on designing work systems. MATERIALS AND METHODS: At three nursing homes in France, two independent observers performed 9 h of observations at each site and a total of 29 interviews. RESULTS: Decisions concerning implementation and the resulting WSs have an impact on the tasks to be performed, the tasks' inherent constraints and the use of the SDG. It is essential to take account of the sociotechnical system as a whole before integrating a technology. Ideally, the introduction of an SDG will go unnoticed by staff and residents; however, our results emphasize the value of highlighting work constraints via a human factors analysis. CONCLUSIONS: It is essential to take account of sociotechnical WSs as a whole when integrating a technology.Implications for rehabilitationFor caregivers, better measurement of the amount of water drunk by older adults might help to prevent dehydration.For service providers, better knowledge of how work systems influence the nursing home staff's ability to comply with the smart drinking glass's prerequisites might improve the device implementation process.For managers, a nursing home's work systems should take account of the smart drinking glass's prerequisites on one hand and the home's environment and constraints on the other.

The medication self-management work system of patients and informal carers from a human factors & ergonomics perspective: A scoping review protocol.

Background: Healthcare is increasingly delivered closer to the patients' homes, which increases the level of responsibility that patients and informal carers take for managing their medication-taking, although this is associated with hazards. Medication self-management has been conceptualised as work taking place in non-formal settings ( e.g., households), which are complex systems. Human factors and ergonomics (HFE) models provide a framework for studying such systems. The Systems Engineering Initiative for Patient Safety (SEIPS) is one framework that considers work system elements and how they interact with each other to shape processes that lead to outcomes ( e.g., safety). Given the increasing amount of diverse research on patient and carer work and on system-shaping factors, the objectives of this review are to: (i) identify available evidence in a structured and systems-oriented way, (ii) explore approaches that have been applied and (iii) highlight research gaps. Methods: An evidence-informed patient, public and carer involvement (PPCI) approach will be implemented at all post-protocol stages to ensure the relevance, uptake and translation of the scoping review. The review will systematically search MEDLINE, Embase, PsycInfo, CINAHL and Web of Science to identify relevant qualitative studies. The methodological approach will be guided by Johanna Briggs Institute methodology and will be reported according to the PRISMA-ScR standards. Data charting and qualitative content analysis directed by SEIPS will explore how the work system and its constituting elements have been described in the literature and identify specific gaps and opportunities for future research. Borrowing from realist approaches, included studies will be assessed in terms of richness and relevance to our review question. Discussion: Strengths of this scoping review include PPCI and a converging focus on medication safety, medication self-management and HFE. Ultimately, this approach will advance our understanding of this complex system and guide opportunities to broaden and strengthen the evidence base.

Improving the usability and usefulness of computerized decision support systems for medication review by clinical pharmacists: A convergent, parallel evaluation.

BACKGROUND: Computerized decision support systems (CDSSs) help hospital-based clinical pharmacists to perform medication reviews and so are promising tools for improving medication safety. However, their poor usability can reduce effectiveness and acceptability. OBJECTIVES: To evaluate the usability and perceived usefulness of a CDSS for medication review by hospital-based pharmacists and to draw up guidelines on improving its usability. METHODS: We performed a convergent, parallel evaluation. Firstly, three researchers conducted a heuristic evaluation of the CDSS. Secondly, clinical pharmacists who use the CDSS filled out the Usefulness, Satisfaction and Ease of Use (USE) questionnaire. Lastly, semi-structured interviews with the pharmacists enabled us to understand their opinions and experiences. The results of the heuristic evaluation were used to identify potential improvements in the CDSS. We performed a statistical analysis of the USE questionnaire data. Interviews were analyzed based on the unified theory of acceptance and use of technology (UTAUT), together with a task-technology fit model. The results generated by these three approaches were compared in order to determine convergences and divergences, identify challenges related to the usability and usefulness of the CDSS, and draw up guidelines for its improvement. RESULTS: Forty-seven usability problems were discovered; they variously concerned the graphical user interface, the pharmacists' needs, and the medication review model implemented in the CDSS. Only the "usefulness" dimension of the USE was not scored positively. All the UTAUT dimensions and the task-technology fit dimension emerged in the interviews. Cross-comparisons of the results from the three approaches led to the identification of four challenges and the corresponding formulation of 23 guidelines. CONCLUSIONS: The guidelines developed here should help to improve the design and acceptability of CDSSs. Hence, CDSSs will be able to assist clinical pharmacists more fully with their medication reviews and help to further improve patient safety.