AI Quality Standards in Health Care: Rapid Umbrella Review.

BACKGROUND: In recent years, there has been an upwelling of artificial intelligence (AI) studies in the health care literature. During this period, there has been an increasing number of proposed standards to evaluate the quality of health care AI studies. OBJECTIVE: This rapid umbrella review examines the use of AI quality standards in a sample of health care AI systematic review articles published over a 36-month period. METHODS: We used a modified version of the Joanna Briggs Institute umbrella review method. Our rapid approach was informed by the practical guide by Tricco and colleagues for conducting rapid reviews. Our search was focused on the MEDLINE database supplemented with Google Scholar. The inclusion criteria were English-language systematic reviews regardless of review type, with mention of AI and health in the abstract, published during a 36-month period. For the synthesis, we summarized the AI quality standards used and issues noted in these reviews drawing on a set of published health care AI standards, harmonized the terms used, and offered guidance to improve the quality of future health care AI studies. RESULTS: We selected 33 review articles published between 2020 and 2022 in our synthesis. The reviews covered a wide range of objectives, topics, settings, designs, and results. Over 60 AI approaches across different domains were identified with varying levels of detail spanning different AI life cycle stages, making comparisons difficult. Health care AI quality standards were applied in only 39% (13/33) of the reviews and in 14% (25/178) of the original studies from the reviews examined, mostly to appraise their methodological or reporting quality. Only a handful mentioned the transparency, explainability, trustworthiness, ethics, and privacy aspects. A total of 23 AI quality standard-related issues were identified in the reviews. There was a recognized need to standardize the planning, conduct, and reporting of health care AI studies and address their broader societal, ethical, and regulatory implications. CONCLUSIONS: Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With increasing desire to adopt AI in different health topics, domains, and settings, practitioners and researchers must stay abreast of and adapt to the evolving landscape of health care AI quality standards and apply these standards to improve the quality of their AI studies.

The Role of Human and Organizational Factors in the Pursuit of One Digital Health.

OBJECTIVE: This paper surveys a subset of the 2022 human and organizational factor (HOF) literature to provide guidance on building a One Digital Health ecosystem. METHODS: We searched a subset of journals in PubMed/Medline for studies with "human factors" or "organization" in the title or abstract. Papers published in 2022 were eligible for inclusion in the survey. Selected papers were categorized into structural and behavioural aspects to understand digital health enabled interactions across micro, meso, and macro systems. RESULTS: Our survey of the 2022 HOF literature showed that while we continue to make meaningful progress at digital health enabled interactions across systems levels, there are still challenges that must be overcome. For example, we must continue to grow the breadth of HOF research beyond individual users and systems to assist with the scale up of digital health systems across and beyond organizations. We summarize the findings by providing five HOF considerations to help build a One Digital Health ecosystem. CONCLUSION: One Digital Health challenges us to improve coordination, communication, and collaboration between the health, environmental and veterinary sectors. Doing so requires us to develop both the structural and behavioural capacity of digital health systems at the organizational level and beyond so that we can develop more robust and integrated systems across health, environmental and veterinary sectors. The HOF community has much to offer and must play a leading role in designing a One Digital Health ecosystem.

Automation vs. Innovation: Unexplored Strategies to Improve Virtual Care.

While there is a global desire to increase digital health capacity, digital health should transform health services delivery rather than simply automate - or worse - replicate existing practices. Failing to capitalize on this transformative potential misses an opportunity to engage patients and other users to provide a more person-centered experience. However, digital transformation done recklessly can disrupt workflow, alienate users, and jeopardize patient safety, as we have observed with implementation of many digital health tools. This paper uses a telemedicine example to provide insight into how digital health innovation can be a meaningful enabler of health system transformation. Examining different ways to leverage digital health technologies is crucial to best capitalize on their potential.

Ethics in Telehealth: Comparison between Guidelines and Practice-based Experience -the Case for Learning Health Systems.

OBJECTIVES: To understand ethical issues within the tele-health domain, specifically how well established macro level telehealth guidelines map with micro level practitioner perspectives. METHODS: We developed four overarching issues to use as a starting point for developing an ethical framework for telehealth. We then reviewed telemedicine ethics guidelines elaborated by the American Medical Association (AMA), the World Medical Association (WMA), and the telehealth component of the Health Professions council of South Africa (HPCSA). We then compared these guidelines with practitioner perspectives to identify the similarities and differences between them. Finally, we generated suggestions to bridge the gap between ethics guidelines and the micro level use of telehealth. RESULTS: Clear differences emerged between the ethics guidelines and the practitioner perspectives. The main reason for the differences were the different contexts where telehealth was used, for example, variability in international practice and variations in the complexity of patient-provider interactions. Overall, published guidelines largely focus on macro level issues related to technology and maintaining data security in patient-provider interactions while practitioner concern is focused on applying the guidelines to specific micro level contexts. CONCLUSIONS: Ethics guidelines on telehealth have a macro level focus in contrast to the micro level needs of practitioners. Work is needed to close this gap. We recommend that both telehealth practitioners and ethics guideline developers better understand healthcare systems and adopt a learning health system approach that draws upon different contexts of clinical practice, innovative models of care delivery, emergent data and evidence-based outcomes. This would help develop a clearer set of priorities and guidelines for the ethical conduct of telehealth.

Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies.

OBJECTIVE: Human and Organizational Factors (HOF) studies in health technology involve human beings and thus require Institutional Review Board (IRB) approval. Yet HOF studies have specific constraints and methods that may not fit standard regulations and IRB practices. Gaining IRB approval may pose difficulties for HOF researchers. This paper aims to provide a first overview of HOF study challenges to get IRB review by exploring differences and best practices across different countries. METHODS: HOF researchers were contacted by email to provide a testimony about their experience with IRB review and approval. Testimonies were thematically analyzed and synthesized to identify and discuss shared themes. RESULTS: Researchers from seven European countries, Argentina, Canada, Australia, and the United States answered the call. Four themes emerged that indicate shared challenges in legislation, IRB inefficiencies and inconsistencies, general regulation and costs, and lack of HOF study knowledge by IRB members. We propose a model for IRB review of HOF studies based on best practices. CONCLUSION: International criteria are needed that define low and high-risk HOF studies, to allow identification of studies that can undergo an expedited (or exempted) process from those that need full IRB review. Enhancing IRB processes in such a way would be beneficial to the conduct of HOF studies. Greater knowledge and promotion of HOF methods and evidence-based HOF study designs may support the evolving discipline. Based on these insights, training and guidance to IRB members may be developed to support them in ensuring that appropriate ethical issues for HOF studies are considered.

Evidence-Based Health Informatics as the Foundation for the COVID-19 Response: A Joint Call for Action.

BACKGROUND: As a major public health crisis, the novel coronavirus disease 2019 (COVID-19) pandemic demonstrates the urgent need for safe, effective, and evidence-based implementations of digital health. The urgency stems from the frequent tendency to focus attention on seemingly high promising digital health interventions despite being poorly validated in times of crisis. AIM: In this paper, we describe a joint call for action to use and leverage evidence-based health informatics as the foundation for the COVID-19 response and public health interventions. Tangible examples are provided for how the working groups and special interest groups of the International Medical Informatics Association (IMIA) are helping to build an evidence-based response to this crisis. METHODS: Leaders of working and special interest groups of the IMIA, a total of 26 groups, were contacted via e-mail to provide a summary of the scientific-based efforts taken to combat COVID-19 pandemic and participate in the discussion toward the creation of this manuscript. A total of 13 groups participated in this manuscript. RESULTS: Various efforts were exerted by members of IMIA including (1) developing evidence-based guidelines for the design and deployment of digital health solutions during COVID-19; (2) surveying clinical informaticians internationally about key digital solutions deployed to combat COVID-19 and the challenges faced when implementing and using them; and (3) offering necessary resources for clinicians about the use of digital tools in clinical practice, education, and research during COVID-19. DISCUSSION: Rigor and evidence need to be taken into consideration when designing, implementing, and using digital tools to combat COVID-19 to avoid delays and unforeseen negative consequences. It is paramount to employ a multidisciplinary approach for the development and implementation of digital health tools that have been rapidly deployed in response to the pandemic bearing in mind human factors, ethics, data privacy, and the diversity of context at the local, national, and international levels. The training and capacity building of front-line workers is crucial and must be linked to a clear strategy for evaluation of ongoing experiences.

Sustainable Health Informatics: Health Informaticians as Alchemists.

The digital transformation of health care delivery remains an elusive work in progress. Contextual variation continues to be a significant barrier to the development of sustainable health information systems. In this paper we characterize health informaticians as modern alchemists and use this characterization to describe informatics progress in addressing four key healthcare challenges. We highlight the need for informaticians to be diligent and loyal to basic methodological principles while also appreciating the role that contextual variation plays in informatics research. We also emphasize that meaningful health systems transformation takes time. The insight presented in this paper helps informaticians in our quest to develop sustainable health information systems.

Context and Meaning in EHR Displays.

Many Electronic Health Record (EHRs) data displays are insensitive to their settings, contexts, and to clinicians' needs. Yet, the contexts in which the data are displayed critically affect EHR usability and patient safety. Medication prescribing is a complex task; especially sensitive to contextual variation in EHR displays as vast variations in formats and logic are often unnecessarily confusing, leading to unwanted cognitive burdens and medical errors. With examples of EHR screenshots, we illustrate contextual variations in medication and allergy displays across different EHR systems and implementations-noting often seemingly haphazard differences that can lead to misunderstandings and misinterpretations.

Digital Process Innovation for Patient Centred Cancer Symptom Management.

Digital process innovation in healthcare enables us to enhance functionality beyond what is enabled in physical process execution. However, the movement from paper artifacts and manual processes to the use of information and communication technologies (ICTs) to support healthcare delivery is challenging because system design requirements for digital processes can be much less structured and more difficult to define than for physical processes. This paper addresses the above issue and presents on our research in converting paper-based cancer symptom management practice guides to digital format for use by patients. We provide an overall architecture and three system design considerations for digital process innovation to support patient centered cancer symptom management.

Balancing Health Information Exchange and Privacy Governance from a Patient-Centred Connected Health and Telehealth Perspective.

OBJECTIVES: Connected healthcare is an essential part of patient-centred care delivery. Technology such as telehealth is a critical part of connected healthcare. However, exchanging health information brings the risk of privacy issues. To better manage privacy risks we first need to understand the different patterns of patient-centred care in order to tailor solutions to address privacy risks. METHODS: Drawing upon published literature, we develop a business model to enable patient-centred care via telehealth. The model identifies three patient-centred connected health patterns. We then use the patterns to analyse potential privacy risks and possible solutions from different types of telehealth delivery. RESULTS: Connected healthcare raises the risk of unwarranted access to health data and related invasion of privacy. However, the risk and extent of privacy issues differ according to the pattern of patient-centred care delivery and the type of particular challenge as they enable the highest degree of connectivity and thus the greatest potential for privacy breaches. CONCLUSION: Privacy issues are a major concern in telehealth systems and patients, providers, and administrators need to be aware of these privacy issues and have guidance on how to manage them. This paper integrates patient-centred connected health care, telehealth, and privacy risks to provide an understanding of how risks vary across different patterns of patient-centred connected health and different types of telehealth delivery.

Context Sensitive Health Informatics: Delivering 21st Century Healthcare - Building a Quality-and-Efficiency Driven System.

Health informatics applications will be a cornerstone in the next generation quality-and-efficiency health care system. Health care is delivered from many different specialties, to many different patients with complex diseases and comorbidity. A one size fits all approach is not adequate to reach the Triple Aim of improving the patient experience of care, improving the health of populations, and reducing the per capita cost of health care. Health informatics applications must be built to be adaptable and sensitive to the complex contexts they will be used in. To enhance patient-centeredness in the 21st Century healthcare, research attention should be focused on investigating and designing models contributing to effective health information retrieval process.

ICT Design for Collaborative and Community Driven Disaster Management.

Information and communication technologies (ICT) have the potential to greatly enhance our ability to develop community reliance and sustainability to support disaster management. However, developing community resilience requires the sharing of numerous resources and the development of collaborative capacity, both of which make ICT design a challenge. This paper presents a framework that integrates community based participatory research (CBPR) and participatory design (PD). We discuss how the framework provides bounding to support community driven ICT design and evaluation.

Studying the HIT-Complexity Interchange.

The design and implementation of health information technology (HIT) is challenging, particularly when it is being introduced into complex settings. While complex adaptive system (CASs) can be a valuable means of understanding relationships between users, HIT and tasks, much of the existing work using CASs is descriptive in nature. This paper addresses that issue by integrating a model for analyzing task complexity with approaches for HIT evaluation and systems analysis. The resulting framework classifies HIT-user tasks and issues as simple, complicated or complex, and provides insight on how to study them.

A Connectivity Framework for Social Information Systems Design in Healthcare.

Social information systems (SISs) will play a key role in healthcare systems' transformation into collaborative patient-centered systems that support care delivery across the entire continuum of care. SISs enable the development of collaborative networks andfacilitate relationships to integrate people and processes across time and space. However, we believe that a "connectivity" issue, which refers to the scope and extent of system requirements for a SIS, is a significant challenge of SIS design. This paper's contribution is the development of the Social Information System Connectivity Framework for supporting SIS design in healthcare. The framework has three parts. First, it defines the structure of a SIS as a set of social triads. Second, it identifies six dimensions that represent the behaviour of a SIS. Third, it proposes the Social Information System Connectivity Factor as our approximation of the extent of connectivity and degree of complexity in a SIS.

Examining the role of collaboration in studies of health information technologies in biomedical informatics: A systematic review of 25 years of research.

PURPOSE: Our objective was to identify and examine studies of collaboration in relation to the use of health information technologies (HIT) in the biomedical informatics field. METHODS: We conducted a systematic literature review of articles through PubMed searches as well as reviewing a variety of individual journals and proceedings. Our search period was from 1990-2015. We identified 98 articles that met our inclusion criteria. We excluded articles that were not published in English, did not deal with technology, and did not focus primarily on individuals collaborating. RESULTS: We categorized the studies by technology type, user groups, study location, methodology, processes related to collaboration, and desired outcomes. We identified three major processes: workflow, communication, and information exchange and two outcomes: maintaining awareness and establishing common ground. Researchers most frequently studied collaboration within hospitals using qualitative methods. DISCUSSION: Based on our findings, we present the "collaboration space model", which is a model to help researchers study collaboration and technology in healthcare. We also discuss issues related to collaboration and future research directions. CONCLUSION: While collaboration is being increasingly recognized in the biomedical informatics community as essential to healthcare delivery, collaboration is often implicitly discussed or intertwined with other similar concepts. In order to evaluate how HIT affects collaboration and how we can build HIT to effectively support collaboration, we need more studies that explicitly focus on collaborative issues.

Review of Social and Organizational Issues in Health Information Technology.

OBJECTIVES: This paper reviews organizational and social issues (OSIs) in health information technology (HIT). METHODS: A review and synthesis of the literature on OSIs in HIT was conducted. RESULTS: Five overarching themes with respect to OSIs in HIT were identified and discussed: scope and frameworks for defining OSIs in HIT, context matters, process immaturity and complexity, trade-offs will happen and need to be discussed openly, and means of studying OSIs in HIT. CONCLUSIONS: There is a wide body of literature that provides insight into OSIs in HIT, even if many of the studies are not explicitly labelled as such. The two biggest research needs are more explicit and theoretical studies of OSI in HITs and more research on integrating micro and macro perspectives of HIT use in organizations.

A Model of Tradeoffs for Understanding Health Information Technology Implementation.

Implementing health information technology (HIT) is a challenge that frequently results in unintended consequences post implementation. To better manage these consequences we need approaches that can proactively identify issues so we deal with them pre-implementation. It can be suggested that a reason unintended consequences occur is because of trade-offs between people's work practices and pre and post HIT implementation. If we can identify what these trade-offs are we can use them for proactive management of unintended consequences. This paper uses a case study of a perioperative information system and principles of social BPM and qualitative content analysis to develop a model of seven trade-off patterns that can be used to study HIT mediated change. It also discusses the implications of the model on the design and evaluation of HIT.

Theories and Methods for Context Sensitive Health Informatics.

Context is a key issue when designing, implementing, and evaluating health information technology. Advanced and well-designed systems may not achieve desired outcomes because of complex contextual issues, and unintended consequences are often reported in the literature. The conference introduced in this article integrates sociotechnical and human factors based theories and methods for analysis and evaluation of complex health information technologies in diverse environments demanding high context sensitivity.

Understanding the Context of Patient Safety Through The Lenses of Three IMIA Working Groups.

Delivering safe patient centered care remains an important yet elusive goal across healthcare systems worldwide. The complexity of healthcare delivery and the unique contexts where it is delivered necessitates patient safety solutions that go beyond individual perspectives. This paper articulates the current state of patient safety research and HIT from the perspective of three International Medical Informatics Association (IMIA) working groups. Each WG will describe patient safety issues within their domain. We then integrate the three WG perspectives into an integrated model to support research, education and policy development for patient safety where HIT is concerned.

A Framework of 'p'-Benefits in Health Information Technology Implementation.

Implementing health information technology (HIT) into clinical settings is still problematic despite extensive efforts and research. HIT issues are caused by a gap in fit between users and technology. The transition from paper (p) to electronic (e) systems is a key factor in the chasm between users and technology. While a significant body of research exists on behavioral or usability evaluation of HIT, there is far less research that looks how to support the transition from p to e systems. This paper presents a framework of 'p' benefits to help us understand why issues occur in the transition from 'p' to 'e' systems. The framework can help us design and evaluate HIT in a manner that bridges the transition from p to e and helps close the user-technology chasm.