The ethical requirement of explainability for AI-DSS in healthcare: a systematic review of reasons.

BACKGROUND: Despite continuous performance improvements, especially in clinical contexts, a major challenge of Artificial Intelligence based Decision Support Systems (AI-DSS) remains their degree of epistemic opacity. The conditions of and the solutions for the justified use of the occasionally unexplainable technology in healthcare are an active field of research. In March 2024, the European Union agreed upon the Artificial Intelligence Act (AIA), requiring medical AI-DSS to be ad-hoc explainable or to use post-hoc explainability methods. The ethical debate does not seem to settle on this requirement yet. This systematic review aims to outline and categorize the positions and arguments in the ethical debate. METHODS: We conducted a literature search on PubMed, BASE, and Scopus for English-speaking scientific peer-reviewed publications from 2016 to 2024. The inclusion criterion was to give explicit requirements of explainability for AI-DSS in healthcare and reason for it. Non-domain-specific documents, as well as surveys, reviews, and meta-analyses were excluded. The ethical requirements for explainability outlined in the documents were qualitatively analyzed with respect to arguments for the requirement of explainability and the required level of explainability. RESULTS: The literature search resulted in 1662 documents; 44 documents were included in the review after eligibility screening of the remaining full texts. Our analysis showed that 17 records argue in favor of the requirement of explainable AI methods (xAI) or ad-hoc explainable models, providing 9 categories of arguments. The other 27 records argued against a general requirement, providing 11 categories of arguments. Also, we found that 14 works advocate the need for context-dependent levels of explainability, as opposed to 30 documents, arguing for context-independent, absolute standards. CONCLUSIONS: The systematic review of reasons shows no clear agreement on the requirement of post-hoc explainability methods or ad-hoc explainable models for AI-DSS in healthcare. The arguments found in the debate were referenced and responded to from different perspectives, demonstrating an interactive discourse. Policymakers and researchers should watch the development of the debate closely. Conversely, ethicists should be well informed by empirical and technical research, given the frequency of advancements in the field.

[Supporting medical and nursing activities with AI: recommendations for responsible design and use]. / Unterstützung ärztlicher und pflegerischer Tätigkeit durch KI: Handlungsempfehlungen für eine verantwortbare Gestaltung und Nutzung.

Clinical decision support systems (CDSS) based on artificial intelligence (AI) are complex socio-technical innovations and are increasingly being used in medicine and nursing to improve the overall quality and efficiency of care, while also addressing limited financial and human resources. However, in addition to such intended clinical and organisational effects, far-reaching ethical, social and legal implications of AI-based CDSS on patient care and nursing are to be expected. To date, these normative-social implications have not been sufficiently investigated. The BMBF-funded project DESIREE (DEcision Support In Routine and Emergency HEalth Care: Ethical and Social Implications) has developed recommendations for the responsible design and use of clinical decision support systems. This article focuses primarily on ethical and social aspects of AI-based CDSS that could have a negative impact on patient health. Our recommendations are intended as additions to existing recommendations and are divided into the following action fields with relevance across all stakeholder groups: development, clinical use, information and consent, education and training, and (accompanying) research.

Examining Mental Workload Relating to Digital Health Technologies in Health Care: Systematic Review.

BACKGROUND: The workload in health care is increasing and hence, mental health issues are on the rise among health care professionals (HCPs). The digitization of patient care could be related to the increase in stress levels. It remains unclear whether the health information system or systems and digital health technologies (DHTs) being used in health care relieve the professionals or whether they represent a further burden. The mental construct that best describes this burden of technologies is mental workload (MWL). The measurement methods of MWL are particularly relevant in this sensitive setting. OBJECTIVE: This review aimed to address 2 different but related objectives: identifying the factors that contribute to the MWL of HCPs when using DHT and examining and exploring the applied assessments for the measurement of MWL with a special focus on eye tracking. METHODS: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 statement, we conducted a systematic review and processed a literature search in the following databases: MEDLINE (PubMed), Web of Science, Academic Search Premier and CINAHL (EBSCO), and PsycINFO. Studies were eligible if they assessed the MWL of HCPs related to DHT. The review was conducted as per the following steps: literature search, article selection, data extraction, quality assessment (using the Standard Quality Assessment Criteria for Evaluation Primary Research Papers From a Variety of Fields [QualSyst]), data analysis, and data synthesis (narrative and tabular). The process was performed by 2 reviewers (in cases of disagreement, a third reviewer was involved). RESULTS: The literature search process resulted in 25 studies that fit the inclusion criteria and examined the MWL of health care workers resulting from the use of DHT in health care settings. Most studies had sample sizes of 10-50 participants, were conducted in the laboratory, and had quasi-experimental or cross-sectional designs. The main results can be grouped into two categories: assessment methods and factors related to DHT that contribute to MWL. Most studies applied subjective methods for the assessment of MWL. Eye tracking did not play a major role in the selected studies. The factors contributing to a higher MWL were clustered into organizational and systemic factors. CONCLUSIONS: Our review of 25 papers shows a diverse assessment approach toward the MWL of HCPs related to DHT as well as 2 groups of relevant contributing factors to MWL. Our results are limited in terms of interpretability and causality due to methodological weaknesses of the included studies and may be limited by some shortcomings in the search process. Future research should concentrate on adequate assessments of the MWL of HCPs dependent on the setting, the evaluation of quality criteria, and further assessment of the contributing factors to MWL. TRIAL REGISTRATION: PROSPERO (International Prospective Register of Systematic Reviews) CRD42021233271; https://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42021233271.

Causes of adverse events in home mechanical ventilation: a nursing perspective.

BACKGROUND: Adverse events (AE) are ubiquitous in home mechanical ventilation (HMV) and can jeopardise patient safety. One particular source of error is human interaction with life-sustaining medical devices, such as the ventilator. The objective is to understand these errors and to be able to take appropriate action. With a systematic analysis of the hazards associated with HMV and their causes, measures can be taken to prevent damage to patient health. METHODS: A systematic adverse events analysis process was conducted to identify the causes of AE in intensive home care. The analysis process consisted of three steps. 1) An input phase consisting of an expert interview and a questionnaire. 2) Analysis and categorisation of the data into a root-cause diagram to help identify the causes of AE. 3) Derivation of risk mitigation measures to help avoid AE. RESULTS: The nursing staff reported that patient transportation, suction and tracheostomy decannulation were the main factors that cause AE. They would welcome support measures such as checklists for care activities and a reminder function, for e.g. tube changes. Risk mitigation measures are given for many of the causes listed in the root-cause diagram. These include measures such as device and care competence, as well as improvements to be made by the equipment providers and manufacturers. The first step in addressing AE is transparency and an open approach to errors and near misses. A systematic error analysis can prevent patient harm through a preventive approach. CONCLUSION: Risks in HMV were identified based on a qualitative approach. The collected data was systematically mapped onto a root-cause diagram. Using the root-cause diagram, some of the causes were analysed for risk mitigation. For manufacturers, caregivers and care services requirements for intervention offers the possibility to create a checklist for particularly risky care activities.

Visualising Data Models of Patient Registries and Clinical Studies - A Method for Quality Check of EDC Systems.

INTRODUCTION: The configuration of electronic data capture (EDC) systems has a relevant impact on data quality in studies and patient registries. The objective was to develop a method to visualise the configuration of an EDC system to check the completeness and correctness of the data definition and rules. METHODS: Step 1: transformation of the EDC data model into a graphical model, step 2: Checking the completeness and consistency of the data model, step 3: correction of identified findings. This process model was evaluated on the patient registry EpiReg. RESULTS: Using the graphical visualisation as a basis, 21 problems in the EDC configuration were identified, discussed with an interdisciplinary team, and corrected. CONCLUSION: The tested methodological approach enables an improvement in data quality by optimising the underlying EDC configuration.

Improving detection of oral lesions: Eye tracking insights from a randomized controlled trial comparing standardized to conventional approach.

BACKGROUND: Early detection of oral cancer (OC) or its precursors is the most effective measure to improve outcome. The reasons for missing them on conventional oral examination (COE) or possible countermeasures are still unclear. METHODS: In this randomized controlled trial, we investigated the effects of standardized oral examination (SOE) compared to COE. 49 dentists, specialists, and dental students wearing an eye tracker had to detect 10 simulated oral lesions drawn into a volunteer's oral cavity. RESULTS: SOE had a higher detection rate at 85.4% sensitivity compared to 78.8% in the control (p = 0.017) due to higher completeness (p < 0.001). Detection rate correlated with examination duration (p = 0.002). CONCLUSIONS: A standardized approach can improve systematics and thereby detection rates in oral examinations. It should take at least 5 min. Perceptual and cognitive errors and improper technique cause oral lesions to be missed. Its wide implementation could be an additional strategy to enhance early detection of OC.

Multi-Source Data ETL (Extract, Transform, Load) for a Genetic Epilepsy Diagnosis and Treatment Dashboard.

The growing number of genes identified in relation to epilepsy represents a major breakthrough in diagnosis and treatment, but experts face the challenge of efficiently accessing and consolidating the vast amount of genetic data available. Therefore, we present the process of transforming data from different sources and formats into an Entity-Attribute-Value (EAV) model database. Combined with the use of standard coding systems, this approach will provide a scalable and adaptable database to present the data in a comprehensive way to experts via a dashboard.

Methodology for the Description of Socio-Technical Systems: A Case Study Approach.

The ethical implications and regulatory requirements of AI applications and decision support systems are generally the subjects of interdisciplinary research. Case studies are a suitable means to prepare AI applications and clinical decision support systems for research. This paper proposes an approach that describes a procedure model and a categorization of the contents of cases for socio-technical systems. The developed methodology was applied to three cases and serve the researchers in the DESIREE research project as a basis for qualitative research and for ethical, social, and regulatory analyses.

Usability-Tests of Mechanical Ventilators: A Systematic Review.

The usability of ventilators is critical for patient safety. This systematic review shows the methods used in usability studies on ventilators, if those are similar in methodology. Furthermore, the usability tasks are compared to the requirements for manufactures during approval. Results show that the methodology and procedure of the studies are similar, but only cover part of the primary operating functions from their corresponding ISO Norm. Therefore optimisation of aspects of the study design, e.g., scope of tested scenarios, is possible.

Usability Engineering of Dynamic Biosignal Displays Using Ventilation Data.

The aim of this work is to develop and evaluate a multi-stage procedure model for the identification of use problems and optimization of usability using biosignal data. The concept is divided into 5 steps: 1. static analysis of data to identify use problems; 2. conducting interviews within the context of use and requirements analysis to investigate problems in more detail; 3. developing new interface concepts to implement the requirements and a prototype of an interface including dynamic visualization of data; 4. formative evaluation using an unmoderated remote usability test; 5. usability test with realistic scenarios and influencing factors in the simulation room. The concept was evaluated in the ventilation setting as an example. The procedure allowed the identification of use problems in the ventilation of patients as well as the development of suitable concepts and their evaluation to counteract use problems. To relieve users, ongoing analyses of biosignals with respect to the use problem are to be carried out. To overcome technical barriers, further development is needed in this area.

Modeling Clinical Guidelines for an Epilepsy-CDSS: The EDiTh Project.

The knowledge transformation process involves the guideline for the diagnosis and therapy of epilepsy to an executable and computable knowledge base that serves as the basis for a decision-support system. We present a transparent knowledge representation model which facilitates technical implementation and verification. Knowledge is represented in a plain table, used in the frontend code of the software where simple reasoning is performed. The simple structure is sufficient and comprehensible also for non-technical persons (i.e., clinicians).

Effect of head-mounted displays on students' acquisition of surgical suturing techniques compared to an e-learning and tutor-led course: a randomized controlled trial.

BACKGROUND: Although surgical suturing is one of the most important basic skills, many medical school graduates do not acquire sufficient knowledge of it due to its lack of integration into the curriculum or a shortage of tutors. E-learning approaches attempt to address this issue but still rely on the involvement of tutors. Furthermore, the learning experience and visual-spatial ability appear to play a critical role in surgical skill acquisition. Virtual reality head-mounted displays (HMDs) could address this, but the benefits of immersive and stereoscopic learning of surgical suturing techniques are still unclear. MATERIAL AND METHODS: In this multi-arm randomized controlled trial, 150 novices participated. Three teaching modalities were compared: an e-learning course (monoscopic), an HMD-based course (stereoscopic, immersive), both self-directed and a tutor-led course with feedback. Suturing performance was recorded by video camera both before and after course participation (>26 h of video material) and assessed in a blinded fashion using the Objective Structured Assessment of Technical Skills (OSATS) Global Rating Score (GRS). Furthermore, the optical flow of the videos was determined using an algorithm. The number of sutures performed was counted, the visual-spatial ability was measured with the Mental Rotation Test (MRT), and courses were assessed with questionnaires. RESULTS: Students' self-assessment in the HMD-based course was comparable to that of the tutor-led course and significantly better than in the e-learning course ( P =0.003). Course suitability was rated best for the tutor-led course ( x̄ =4.8), followed by the HMD-based ( x̄ =3.6) and e-learning ( x̄ =2.5) courses. The median ΔGRS between courses was comparable ( P =0.15) at 12.4 (95% CI 10.0-12.7) for the e-learning course, 14.1 (95% CI 13.0-15.0) for the HMD-based course, and 12.7 (95% CI 10.3-14.2) for the tutor-led course. However, the ΔGRS was significantly correlated with the number of sutures performed during the training session ( P =0.002), but not with visual-spatial ability ( P =0.615). Optical flow ( R2 =0.15, P <0.001) and the number of sutures performed ( R2 =0.73, P <0.001) can be used as additional measures to GRS. CONCLUSION: The use of HMDs with stereoscopic and immersive video provides advantages in the learning experience and should be preferred over a traditional web application for e-learning. Contrary to expectations, feedback is not necessary for novices to achieve a sufficient level in suturing; only the number of surgical sutures performed during training is a good determinant of competence improvement. Nevertheless, feedback still enhances the learning experience. Therefore, automated assessment as an alternative feedback approach could further improve self-directed learning modalities. As a next step, the data from this study could be used to develop such automated AI-based assessments.

Online Usability Tool - First Experiences from Testing a Ventilator Interface.

Usability tests of medical devices are mainly conducted on-site, but remote tests can also be suitable for quick feedback. Using the online survey tool SoSci Survey and videos of a ventilator interface prototype, a usability test environment was developed which allows participation independent of time and place.

Causes of use errors in ventilation devices - Systematic review.

A systematic review according to the PRISMA reporting standard was performed to identify causes of use errors in mechanical ventilators described in the literature. The PubMed search resulted in the inclusion of 16 papers. The errors described were systematically analyzed with regard to their causes and categorized in an adapted cause-and-effect diagram. The causes of use errors were related to specific usability issues and to the general condition that medical staff often work with different ventilators. When many devices are used, the different user interfaces are a source of use errors, since, for example, the same ventilation modes have different names. In order to avoid the identified causes for use errors in the future, this work offers manufacturers of ventilation devices design recommendations and the possibility to include the results in their risk management. In addition, standardizing user interface content across all ventilators, as in ISO 19223, can help reduce use errors.

Augmented Reality-Based Surgery on the Human Cadaver Using a New Generation of Optical Head-Mounted Displays: Development and Feasibility Study.

BACKGROUND: Although nearly one-third of the world's disease burden requires surgical care, only a small proportion of digital health applications are directly used in the surgical field. In the coming decades, the application of augmented reality (AR) with a new generation of optical-see-through head-mounted displays (OST-HMDs) like the HoloLens (Microsoft Corp) has the potential to bring digital health into the surgical field. However, for the application to be performed on a living person, proof of performance must first be provided due to regulatory requirements. In this regard, cadaver studies could provide initial evidence. OBJECTIVE: The goal of the research was to develop an open-source system for AR-based surgery on human cadavers using freely available technologies. METHODS: We tested our system using an easy-to-understand scenario in which fractured zygomatic arches of the face had to be repositioned with visual and auditory feedback to the investigators using a HoloLens. Results were verified with postoperative imaging and assessed in a blinded fashion by 2 investigators. The developed system and scenario were qualitatively evaluated by consensus interview and individual questionnaires. RESULTS: The development and implementation of our system was feasible and could be realized in the course of a cadaver study. The AR system was found helpful by the investigators for spatial perception in addition to the combination of visual as well as auditory feedback. The surgical end point could be determined metrically as well as by assessment. CONCLUSIONS: The development and application of an AR-based surgical system using freely available technologies to perform OST-HMD-guided surgical procedures in cadavers is feasible. Cadaver studies are suitable for OST-HMD-guided interventions to measure a surgical end point and provide an initial data foundation for future clinical trials. The availability of free systems for researchers could be helpful for a possible translation process from digital health to AR-based surgery using OST-HMDs in the operating theater via cadaver studies.

Examining the Mental Workload Associated With Digital Health Technologies in Health Care: Protocol for a Systematic Review Focusing on Assessment Methods.

BACKGROUND: The workload in health care is high; physicians and nurses report high stress levels due to a demanding environment where they often have to perform multiple tasks simultaneously. As a result, mental health issues among health care professionals (HCPs) are on the rise and the prevalence of errors in their daily tasks could increase. Processes of demographic change are partly responsible for even higher stress levels among HCPs. The digitization of patient care is intended to counteract these processes. However, it remains unclear whether these health information systems (HIS) and digital health technologies (DHT) support the HCPs and relieve stress, or if they represent a further burden. The mental construct that describes this burden of technologies is mental workload (MWL). Work in the clinic can be viewed as working in safety-critical environments. Particularly in this sensitive setting, the measurement methods of MWL are relevant, mainly due to their strongly differing levels of intrusiveness and sensitivity. The method of eye tracking could be a useful way to measure MWL directly in the field. OBJECTIVE: The systematic review aims to address the following questions: (1) In which manner do DHT contribute to the overall MWL of HCPs? (2) Can we observe a direct or indirect effect of DHT on MWL? (3) Which aspects or factors of DHT contribute to an increase in MWL? (4) Which methods/assessments are applied to measure MWL related to HIS/DHT? (5) What role does eye tracking/pupillometry play in the context of measuring MWL? (6) Which outcomes are being assessed via eye tracking? METHODS: Following the PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analysis) statement, we will conduct a systematic review. Based on the research questions, we define keywords that we then combine in search terms. The review follows the following steps: literature search, article selection, data extraction, risk of bias assessment, data analysis, and data synthesis. RESULTS: We expect results as well as a finalization of the review in the summer of 2021. CONCLUSIONS: This review will evaluate the impact of DHT on the MWL of HCPs. In addition, assessment methods of MWL in the context of digital technologies will be systematically analyzed. TRIAL REGISTRATION: PROSPERO (International Prospective Register of Systematic Reviews) CRD42021233271; https://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42021233271. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/29126.

Usability Issues with Mechanical Ventilation Devices.

The user interface of a mechanical ventilator is safety critical, as use errors can lead to patient harm. A systematic review was conducted to identify published usability issues and contributing factors that can lead to use errors. The findings were grouped in an Ishikawa diagram. Many of the problems mentioned based on inconsistent labeling and manufacturer-specific naming of ventilation modes. In the studies, usability was often measured quantitatively and did not allow any conclusions to be drawn about concrete problems.

Feasibility of Three Head Mounted Eye-Tracker in Anesthesia: A Feasibility Study.

Many studies use eye-tracker to analyse the socio-technical system, also in medical research. Only a few articles describe the use of eye-tracker to examine human-computer interaction in a critical care environment, especially in the field of anaesthesia or surgery. Therefore, we have tested in a feasibility study head-mounted eye-tracker of three different manufactures in a simulated anesthesia surrounding with mankind patient simulators. The research question was to analyse whether the field scene camera of the eye-tracker can be used in the light conditions and changes in brightness of the operating room. In addition, it was tested whether the eye-tracker was still calibrated and held on the subject's head during the resuscitation movement. All eye-trackers tested had a good adaptation on changing light or changing distances.

[Experiences with health technologies to support physical activity in people over the age of 65: A qualitative survey of the requirements for the development of preventive technologies for a heterogeneous target group]. / Erfahrungen mit Gesundheitstechnologien zur Unterstützung der körperlichen Aktivität bei Menschen ab 65 Jahren ­ Eine qualitative Erhebung der Anforderungen für die Entwicklung präventiver Technologien für eine heterogene Zielgruppe.

BACKGROUND: For elderly people, physical activity (PA) is an important prerequisite for a healthy and self-determined life and provides preventive protection against many chronic diseases. Since a high proportion of this group does not exercise sufficiently in everyday life, new preventive strategies have been developed. In recent years, health technologies have also become increasingly important and offer a potential for primary prevention to promote PA. The aim of this study was to analyze experience with and acceptance of the use of technologies to support PA and to monitor health parameters in people over the age of 65. In addition to identifying concerns, uncertainties and subjective ideas about technologies, requirements for preventive technology should be derived. METHOD: Guided (semi-structured) interviews were conducted. Interviewing topics included "PA" and "technology". Recruitment took place through a public call and the distribution of flyers. The evaluation was carried out by content analysis with inductive and deductive category formation. In addition, an interdisciplinary requirements analysis was performed on the basis of the transcripts in order to derive needs and requirements for preventive technologies. RESULTS: The interviews were conducted with 33 persons (19 female). The average age was 75 years. Nine participants with a migration background originated from four countries. All participants were active in everyday life. The barriers identified included a lack of motivation and weather conditions. Many participants used health technologies in their everyday lives and were quite open to these developments. However, concern was expressed regarding topics such as data protection and security. CONCLUSION: Experience and acceptance in dealing with future preventive technologies as well as concerns, ideas and requirements could be inferred from discussions with the heterogeneous target group. General strategies (e.g. notices, press) as well as specific strategies (e.g. multipliers) and sufficient time are important to ensure a target group-specific access to different groups of people.

Evaluation of Electronic and Paper-Pen Data Capturing Tools for Data Quality in a Public Health Survey in a Health and Demographic Surveillance Site, Ethiopia: Randomized Controlled Crossover Health Care Information Technology Evaluation.

BACKGROUND: Periodic demographic health surveillance and surveys are the main sources of health information in developing countries. Conducting a survey requires extensive use of paper-pen and manual work and lengthy processes to generate the required information. Despite the rise of popularity in using electronic data collection systems to alleviate the problems, sufficient evidence is not available to support the use of electronic data capture (EDC) tools in interviewer-administered data collection processes. OBJECTIVE: This study aimed to compare data quality parameters in the data collected using mobile electronic and standard paper-based data capture tools in one of the health and demographic surveillance sites in northwest Ethiopia. METHODS: A randomized controlled crossover health care information technology evaluation was conducted from May 10, 2016, to June 3, 2016, in a demographic and surveillance site. A total of 12 interviewers, as 2 individuals (one of them with a tablet computer and the other with a paper-based questionnaire) in 6 groups were assigned in the 6 towns of the surveillance premises. Data collectors switched the data collection method based on computer-generated random order. Data were cleaned using a MySQL program and transferred to SPSS (IBM SPSS Statistics for Windows, Version 24.0) and R statistical software (R version 3.4.3, the R Foundation for Statistical Computing Platform) for analysis. Descriptive and mixed ordinal logistic analyses were employed. The qualitative interview audio record from the system users was transcribed, coded, categorized, and linked to the International Organization for Standardization 9241-part 10 dialogue principles for system usability. The usability of this open data kit-based system was assessed using quantitative System Usability Scale (SUS) and matching of qualitative data with the isometric dialogue principles. RESULTS: From the submitted 1246 complete records of questionnaires in each tool, 41.89% (522/1246) of the paper and pen data capture (PPDC) and 30.89% (385/1246) of the EDC tool questionnaires had one or more types of data quality errors. The overall error rates were 1.67% and 0.60% for PPDC and EDC, respectively. The chances of more errors on the PPDC tool were multiplied by 1.015 for each additional question in the interview compared with EDC. The SUS score of the data collectors was 85.6. In the qualitative data response mapping, EDC had more positive suitability of task responses with few error tolerance characteristics. CONCLUSIONS: EDC possessed significantly better data quality and efficiency compared with PPDC, explained with fewer errors, instant data submission, and easy handling. The EDC proved to be a usable data collection tool in the rural study setting. Implementation organization needs to consider consistent power source, decent internet connection, standby technical support, and security assurance for the mobile device users for planning full-fledged implementation and integration of the system in the surveillance site.