AI Quality Standards in Health Care: Rapid Umbrella Review.

BACKGROUND: In recent years, there has been an upwelling of artificial intelligence (AI) studies in the health care literature. During this period, there has been an increasing number of proposed standards to evaluate the quality of health care AI studies. OBJECTIVE: This rapid umbrella review examines the use of AI quality standards in a sample of health care AI systematic review articles published over a 36-month period. METHODS: We used a modified version of the Joanna Briggs Institute umbrella review method. Our rapid approach was informed by the practical guide by Tricco and colleagues for conducting rapid reviews. Our search was focused on the MEDLINE database supplemented with Google Scholar. The inclusion criteria were English-language systematic reviews regardless of review type, with mention of AI and health in the abstract, published during a 36-month period. For the synthesis, we summarized the AI quality standards used and issues noted in these reviews drawing on a set of published health care AI standards, harmonized the terms used, and offered guidance to improve the quality of future health care AI studies. RESULTS: We selected 33 review articles published between 2020 and 2022 in our synthesis. The reviews covered a wide range of objectives, topics, settings, designs, and results. Over 60 AI approaches across different domains were identified with varying levels of detail spanning different AI life cycle stages, making comparisons difficult. Health care AI quality standards were applied in only 39% (13/33) of the reviews and in 14% (25/178) of the original studies from the reviews examined, mostly to appraise their methodological or reporting quality. Only a handful mentioned the transparency, explainability, trustworthiness, ethics, and privacy aspects. A total of 23 AI quality standard-related issues were identified in the reviews. There was a recognized need to standardize the planning, conduct, and reporting of health care AI studies and address their broader societal, ethical, and regulatory implications. CONCLUSIONS: Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With increasing desire to adopt AI in different health topics, domains, and settings, practitioners and researchers must stay abreast of and adapt to the evolving landscape of health care AI quality standards and apply these standards to improve the quality of their AI studies.

Examining the role of collaboration in studies of health information technologies in biomedical informatics: A systematic review of 25 years of research.

PURPOSE: Our objective was to identify and examine studies of collaboration in relation to the use of health information technologies (HIT) in the biomedical informatics field. METHODS: We conducted a systematic literature review of articles through PubMed searches as well as reviewing a variety of individual journals and proceedings. Our search period was from 1990-2015. We identified 98 articles that met our inclusion criteria. We excluded articles that were not published in English, did not deal with technology, and did not focus primarily on individuals collaborating. RESULTS: We categorized the studies by technology type, user groups, study location, methodology, processes related to collaboration, and desired outcomes. We identified three major processes: workflow, communication, and information exchange and two outcomes: maintaining awareness and establishing common ground. Researchers most frequently studied collaboration within hospitals using qualitative methods. DISCUSSION: Based on our findings, we present the "collaboration space model", which is a model to help researchers study collaboration and technology in healthcare. We also discuss issues related to collaboration and future research directions. CONCLUSION: While collaboration is being increasingly recognized in the biomedical informatics community as essential to healthcare delivery, collaboration is often implicitly discussed or intertwined with other similar concepts. In order to evaluate how HIT affects collaboration and how we can build HIT to effectively support collaboration, we need more studies that explicitly focus on collaborative issues.

Wikis and collaborative writing applications in health care: a scoping review.

BACKGROUND: Collaborative writing applications (eg, wikis and Google Documents) hold the potential to improve the use of evidence in both public health and health care. The rapid rise in their use has created the need for a systematic synthesis of the evidence of their impact as knowledge translation (KT) tools in the health care sector and for an inventory of the factors that affect their use. OBJECTIVE: Through the Levac six-stage methodology, a scoping review was undertaken to explore the depth and breadth of evidence about the effective, safe, and ethical use of wikis and collaborative writing applications (CWAs) in health care. METHODS: Multiple strategies were used to locate studies. Seven scientific databases and 6 grey literature sources were queried for articles on wikis and CWAs published between 2001 and September 16, 2011. In total, 4436 citations and 1921 grey literature items were screened. Two reviewers independently reviewed citations, selected eligible studies, and extracted data using a standardized form. We included any paper presenting qualitative or quantitative empirical evidence concerning health care and CWAs. We defined a CWA as any technology that enables the joint and simultaneous editing of a webpage or an online document by many end users. We performed qualitative content analysis to identify the factors that affect the use of CWAs using the Gagnon framework and their effects on health care using the Donabedian framework. RESULTS: Of the 111 studies included, 4 were experimental, 5 quasi-experimental, 5 observational, 52 case studies, 23 surveys about wiki use, and 22 descriptive studies about the quality of information in wikis. We classified them by theme: patterns of use of CWAs (n=26), quality of information in existing CWAs (n=25), and CWAs as KT tools (n=73). A high prevalence of CWA use (ie, more than 50%) is reported in 58% (7/12) of surveys conducted with health care professionals and students. However, we found only one longitudinal study showing that CWA use is increasing in health care. Moreover, contribution rates remain low and the quality of information contained in different CWAs needs improvement. We identified 48 barriers and 91 facilitators in 4 major themes (factors related to the CWA, users' knowledge and attitude towards CWAs, human environment, and organizational environment). We also found 57 positive and 23 negative effects that we classified into processes and outcomes. CONCLUSIONS: Although we found some experimental and quasi-experimental studies of the effectiveness and safety of CWAs as educational and KT interventions, the vast majority of included studies were observational case studies about CWAs being used by health professionals and patients. More primary research is needed to find ways to address the different barriers to their use and to make these applications more useful for different stakeholders.

A model of collaborative agency and common ground.

As more healthcare delivery is provided via collaborative means there is a need to understand how to design information and communication technologies (ICTs) to support collaboration. Existing research has largely focused on individual aspects of ICT usage and not how they can support the coordination of collaborative activities. In order to understand how we can design ICTs to support collaboration we need to understand how agents, technologies, information and processes integrate while providing collaborative care delivery. Co-agency and common ground have both provided insight about the integration of different entities as part of collaboration practices. However there is still a lack of understanding about how to coordinate the integration of agents, processes and technologies to support collaboration. This paper combines co-agency and common ground to develop a model of collaborative agency and specific categories of common ground to facilitate its coordination.

The Role of Human and Organizational Factors in the Pursuit of One Digital Health.

OBJECTIVE: This paper surveys a subset of the 2022 human and organizational factor (HOF) literature to provide guidance on building a One Digital Health ecosystem. METHODS: We searched a subset of journals in PubMed/Medline for studies with "human factors" or "organization" in the title or abstract. Papers published in 2022 were eligible for inclusion in the survey. Selected papers were categorized into structural and behavioural aspects to understand digital health enabled interactions across micro, meso, and macro systems. RESULTS: Our survey of the 2022 HOF literature showed that while we continue to make meaningful progress at digital health enabled interactions across systems levels, there are still challenges that must be overcome. For example, we must continue to grow the breadth of HOF research beyond individual users and systems to assist with the scale up of digital health systems across and beyond organizations. We summarize the findings by providing five HOF considerations to help build a One Digital Health ecosystem. CONCLUSION: One Digital Health challenges us to improve coordination, communication, and collaboration between the health, environmental and veterinary sectors. Doing so requires us to develop both the structural and behavioural capacity of digital health systems at the organizational level and beyond so that we can develop more robust and integrated systems across health, environmental and veterinary sectors. The HOF community has much to offer and must play a leading role in designing a One Digital Health ecosystem.

Automation vs. Innovation: Unexplored Strategies to Improve Virtual Care.

While there is a global desire to increase digital health capacity, digital health should transform health services delivery rather than simply automate - or worse - replicate existing practices. Failing to capitalize on this transformative potential misses an opportunity to engage patients and other users to provide a more person-centered experience. However, digital transformation done recklessly can disrupt workflow, alienate users, and jeopardize patient safety, as we have observed with implementation of many digital health tools. This paper uses a telemedicine example to provide insight into how digital health innovation can be a meaningful enabler of health system transformation. Examining different ways to leverage digital health technologies is crucial to best capitalize on their potential.

The nature of unintended benefits in health information systems.

Health information systems (HISs) have been shown to introduce unintended consequences post implementation. Much of the current research on these consequences has focused on the negative aspects of them. However unintended consequences of HIS usage can also be beneficial to various aspects of healthcare delivery. This paper uses several case studies of HIS implementation to develop a model of unintended benefits of HIS usage with three categories of benefits: patient, service delivery and administrative. We also discuss the implications of these benefits on the design and evaluation of HISs.

A model for representing formal, informal and hybrid communication in the clinical communication space.

The clinical communication space refers to the activities that take place in the context of clinical communication. Communication is an important part of healthcare delivery yet the communication space has been described a source of medical errors and other communication issues such as interruptions. Information and communication technologies (ICTs) can enhance the clinical communication space but before we can design ICTs we need to describe the space. In particular we need to understand the role that informal communication plays. This paper reports on an exploratory study of the clinical communication space. It uses an ontology to illustrate the range of processes, modalities and contexts within the communication space and the role of formal, informal and hybrid communication.

A methodology and supply chain management inspired reference ontology for modeling healthcare teams.

Numerous studies and strategic plans are advocating more team based healthcare delivery that is facilitated by information and communication technologies (ICTs). However before we can design ICTs to support teams we need a solid conceptual model of team processes and a methodology for using such a model in healthcare settings. This paper draws upon success in the supply chain management domain to develop a reference ontology of healthcare teams and a methodology for modeling teams to instantiate the ontology in specific settings. This research can help us understand how teams function and how we can design ICTs to support teams.

Ethics in Telehealth: Comparison between Guidelines and Practice-based Experience -the Case for Learning Health Systems.

OBJECTIVES: To understand ethical issues within the tele-health domain, specifically how well established macro level telehealth guidelines map with micro level practitioner perspectives. METHODS: We developed four overarching issues to use as a starting point for developing an ethical framework for telehealth. We then reviewed telemedicine ethics guidelines elaborated by the American Medical Association (AMA), the World Medical Association (WMA), and the telehealth component of the Health Professions council of South Africa (HPCSA). We then compared these guidelines with practitioner perspectives to identify the similarities and differences between them. Finally, we generated suggestions to bridge the gap between ethics guidelines and the micro level use of telehealth. RESULTS: Clear differences emerged between the ethics guidelines and the practitioner perspectives. The main reason for the differences were the different contexts where telehealth was used, for example, variability in international practice and variations in the complexity of patient-provider interactions. Overall, published guidelines largely focus on macro level issues related to technology and maintaining data security in patient-provider interactions while practitioner concern is focused on applying the guidelines to specific micro level contexts. CONCLUSIONS: Ethics guidelines on telehealth have a macro level focus in contrast to the micro level needs of practitioners. Work is needed to close this gap. We recommend that both telehealth practitioners and ethics guideline developers better understand healthcare systems and adopt a learning health system approach that draws upon different contexts of clinical practice, innovative models of care delivery, emergent data and evidence-based outcomes. This would help develop a clearer set of priorities and guidelines for the ethical conduct of telehealth.

Challenges and Best Practices in Ethical Review of Human and Organizational Factors Studies in Health Technology: a Synthesis of Testimonies.

OBJECTIVE: Human and Organizational Factors (HOF) studies in health technology involve human beings and thus require Institutional Review Board (IRB) approval. Yet HOF studies have specific constraints and methods that may not fit standard regulations and IRB practices. Gaining IRB approval may pose difficulties for HOF researchers. This paper aims to provide a first overview of HOF study challenges to get IRB review by exploring differences and best practices across different countries. METHODS: HOF researchers were contacted by email to provide a testimony about their experience with IRB review and approval. Testimonies were thematically analyzed and synthesized to identify and discuss shared themes. RESULTS: Researchers from seven European countries, Argentina, Canada, Australia, and the United States answered the call. Four themes emerged that indicate shared challenges in legislation, IRB inefficiencies and inconsistencies, general regulation and costs, and lack of HOF study knowledge by IRB members. We propose a model for IRB review of HOF studies based on best practices. CONCLUSION: International criteria are needed that define low and high-risk HOF studies, to allow identification of studies that can undergo an expedited (or exempted) process from those that need full IRB review. Enhancing IRB processes in such a way would be beneficial to the conduct of HOF studies. Greater knowledge and promotion of HOF methods and evidence-based HOF study designs may support the evolving discipline. Based on these insights, training and guidance to IRB members may be developed to support them in ensuring that appropriate ethical issues for HOF studies are considered.

Evidence-Based Health Informatics as the Foundation for the COVID-19 Response: A Joint Call for Action.

BACKGROUND: As a major public health crisis, the novel coronavirus disease 2019 (COVID-19) pandemic demonstrates the urgent need for safe, effective, and evidence-based implementations of digital health. The urgency stems from the frequent tendency to focus attention on seemingly high promising digital health interventions despite being poorly validated in times of crisis. AIM: In this paper, we describe a joint call for action to use and leverage evidence-based health informatics as the foundation for the COVID-19 response and public health interventions. Tangible examples are provided for how the working groups and special interest groups of the International Medical Informatics Association (IMIA) are helping to build an evidence-based response to this crisis. METHODS: Leaders of working and special interest groups of the IMIA, a total of 26 groups, were contacted via e-mail to provide a summary of the scientific-based efforts taken to combat COVID-19 pandemic and participate in the discussion toward the creation of this manuscript. A total of 13 groups participated in this manuscript. RESULTS: Various efforts were exerted by members of IMIA including (1) developing evidence-based guidelines for the design and deployment of digital health solutions during COVID-19; (2) surveying clinical informaticians internationally about key digital solutions deployed to combat COVID-19 and the challenges faced when implementing and using them; and (3) offering necessary resources for clinicians about the use of digital tools in clinical practice, education, and research during COVID-19. DISCUSSION: Rigor and evidence need to be taken into consideration when designing, implementing, and using digital tools to combat COVID-19 to avoid delays and unforeseen negative consequences. It is paramount to employ a multidisciplinary approach for the development and implementation of digital health tools that have been rapidly deployed in response to the pandemic bearing in mind human factors, ethics, data privacy, and the diversity of context at the local, national, and international levels. The training and capacity building of front-line workers is crucial and must be linked to a clear strategy for evaluation of ongoing experiences.

An interdisciplinary team communication framework and its application to healthcare 'e-teams' systems design.

BACKGROUND: There are few studies that examine the processes that interdisciplinary teams engage in and how we can design health information systems (HIS) to support those team processes. This was an exploratory study with two purposes: (1) To develop a framework for interdisciplinary team communication based on structures, processes and outcomes that were identified as having occurred during weekly team meetings. (2) To use the framework to guide 'e-teams' HIS design to support interdisciplinary team meeting communication. METHODS: An ethnographic approach was used to collect data on two interdisciplinary teams. Qualitative content analysis was used to analyze the data according to structures, processes and outcomes. RESULTS: We present details for team meta-concepts of structures, processes and outcomes and the concepts and sub concepts within each meta-concept. We also provide an exploratory framework for interdisciplinary team communication and describe how the framework can guide HIS design to support 'e-teams'. CONCLUSION: The structures, processes and outcomes that describe interdisciplinary teams are complex and often occur in a non-linear fashion. Electronic data support, process facilitation and team video conferencing are three HIS tools that can enhance team function.

Health information systems design to support a nursing model of care: opportunities and challenges.

The design and implementation of health information systems (HISs) in team-based settings is complex owing to the multiple users with different perspectives who interact with the system. We argue that such perspectives must be understood prior to designing and implementing HISs. One specific type of team-based model is a nursing care model. In such a model, care is provided through an interdisciplinary team that is lead by the nursing staff. We analyze a nursing-based model of care according to the context of the organization, clinical unit, and individual as defined by the Contextual Implementation Model [1]. We then discuss how the nursing model will be affected by automation using different HISs.

Digital Process Innovation for Patient Centred Cancer Symptom Management.

Digital process innovation in healthcare enables us to enhance functionality beyond what is enabled in physical process execution. However, the movement from paper artifacts and manual processes to the use of information and communication technologies (ICTs) to support healthcare delivery is challenging because system design requirements for digital processes can be much less structured and more difficult to define than for physical processes. This paper addresses the above issue and presents on our research in converting paper-based cancer symptom management practice guides to digital format for use by patients. We provide an overall architecture and three system design considerations for digital process innovation to support patient centered cancer symptom management.

Sustainable Health Informatics: Health Informaticians as Alchemists.

The digital transformation of health care delivery remains an elusive work in progress. Contextual variation continues to be a significant barrier to the development of sustainable health information systems. In this paper we characterize health informaticians as modern alchemists and use this characterization to describe informatics progress in addressing four key healthcare challenges. We highlight the need for informaticians to be diligent and loyal to basic methodological principles while also appreciating the role that contextual variation plays in informatics research. We also emphasize that meaningful health systems transformation takes time. The insight presented in this paper helps informaticians in our quest to develop sustainable health information systems.

Context and Meaning in EHR Displays.

Many Electronic Health Record (EHRs) data displays are insensitive to their settings, contexts, and to clinicians' needs. Yet, the contexts in which the data are displayed critically affect EHR usability and patient safety. Medication prescribing is a complex task; especially sensitive to contextual variation in EHR displays as vast variations in formats and logic are often unnecessarily confusing, leading to unwanted cognitive burdens and medical errors. With examples of EHR screenshots, we illustrate contextual variations in medication and allergy displays across different EHR systems and implementations-noting often seemingly haphazard differences that can lead to misunderstandings and misinterpretations.

An interdisciplinary computer-based information tool for palliative severe pain management.

OBJECTIVES: As patient care becomes more collaborative in nature, there is a need for information technology that supports interdisciplinary practices of care. This study developed and performed usability testing of a standalone computer-based information tool to support the interdisciplinary practice of palliative severe pain management (SPM). DESIGN: A grounded theory-participatory design (GT-PD) approach was used with three distinct palliative data sources to obtain and understand user requirements for SPM practice and how a computer-based information tool could be designed to support those requirements. RESULTS: The GT-PD concepts and categories provided a rich perspective of palliative SPM and the process and information support required for different SPM tasks. A conceptual framework consisting of an ontology and a set of three problem-solving methods was developed to reconcile the requirements of different interdisciplinary team members. The conceptual framework was then implemented as a prototype computer-based information tool that has different modes of use to support both day-to-day case management and education of palliative SPM. Usability testing of the computer tool was performed, and the tool tested favorably in a laboratory setting. CONCLUSION: An interdisciplinary computer-based information tool can be developed to support the different work practices and information needs of interdisciplinary team members, but explicit requirements must be sought from all prospective users of such a tool. Qualitative methods such as the hybrid GT-PD approach used in this research are particularly helpful for articulating computer tool design requirements.

A comparative analysis of computer based hospice palliative care datasets in Canada.

BACKGROUND: To analyze how seven Canadian hospice palliative care (HPC) centres and one national surveillance dataset compare with respect to the collection of forty data elements. Research and service delivery implications of the findings are discussed. METHODS: The data sources consisted of data element names and their definitions collected in a computer based format from seven HPC centres and one surveillance dataset. The data elements were structured into five themes: demographic, patient death, support, contact or informal caregiver; program/consultations/service request, and clinical. Each theme contains a number of data elements with a total of 40 elements included in the analysis. Comparative analysis was done on the data elements to compare their names and definitions. RESULTS: Much variation exists in data collection around HPC delivery. Such variation prevents any timely and meaningful comparison of service and care delivery across HPC centres. Patient death data, service/program data and clinical data is particularly varied. CONCLUSION: Developing a common minimum data set is a logical starting point to help overcome data variations between care centres. Greater coordination is needed between care centres and the development of national standards and policies. Moving towards electronic data collection would help facilitate common policy and practice norms.

Balancing Health Information Exchange and Privacy Governance from a Patient-Centred Connected Health and Telehealth Perspective.

OBJECTIVES: Connected healthcare is an essential part of patient-centred care delivery. Technology such as telehealth is a critical part of connected healthcare. However, exchanging health information brings the risk of privacy issues. To better manage privacy risks we first need to understand the different patterns of patient-centred care in order to tailor solutions to address privacy risks. METHODS: Drawing upon published literature, we develop a business model to enable patient-centred care via telehealth. The model identifies three patient-centred connected health patterns. We then use the patterns to analyse potential privacy risks and possible solutions from different types of telehealth delivery. RESULTS: Connected healthcare raises the risk of unwarranted access to health data and related invasion of privacy. However, the risk and extent of privacy issues differ according to the pattern of patient-centred care delivery and the type of particular challenge as they enable the highest degree of connectivity and thus the greatest potential for privacy breaches. CONCLUSION: Privacy issues are a major concern in telehealth systems and patients, providers, and administrators need to be aware of these privacy issues and have guidance on how to manage them. This paper integrates patient-centred connected health care, telehealth, and privacy risks to provide an understanding of how risks vary across different patterns of patient-centred connected health and different types of telehealth delivery.