Human-Centered Design to Address Biases in Artificial Intelligence.

The potential of artificial intelligence (AI) to reduce health care disparities and inequities is recognized, but it can also exacerbate these issues if not implemented in an equitable manner. This perspective identifies potential biases in each stage of the AI life cycle, including data collection, annotation, machine learning model development, evaluation, deployment, operationalization, monitoring, and feedback integration. To mitigate these biases, we suggest involving a diverse group of stakeholders, using human-centered AI principles. Human-centered AI can help ensure that AI systems are designed and used in a way that benefits patients and society, which can reduce health disparities and inequities. By recognizing and addressing biases at each stage of the AI life cycle, AI can achieve its potential in health care.

Understanding the Information Needs and Context of Trauma Handoffs to Design Automated Sensing Clinical Documentation Technologies: Qualitative Mixed-Method Study of Military and Civilian Cases.

BACKGROUND: Current methods of communication between the point of injury and receiving medical facilities rely on verbal communication, supported by brief notes and the memory of the field medic. This communication can be made more complete and reliable with technologies that automatically document the actions of field medics. However, designing state-of-the-art technology for military field personnel and civilian first responders is challenging due to the barriers researchers face in accessing the environment and understanding situated actions and cognitive models employed in the field. OBJECTIVE: To identify design insights for an automated sensing clinical documentation (ASCD) system, we sought to understand what information is transferred in trauma cases between prehospital and hospital personnel, and what contextual factors influence the collection, management, and handover of information in trauma cases, in both military and civilian cases. METHODS: Using a multi-method approach including video review and focus groups, we developed an understanding of the information needs of trauma handoffs and the context of field documentation to inform the design of an automated sensing documentation system that uses wearables, cameras, and environmental sensors to passively infer clinical activity and automatically produce documentation. RESULTS: Comparing military and civilian trauma documentation and handoff, we found similarities in the types of data collected and the prioritization of information. We found that military environments involved many more contextual factors that have implications for design, such as the physical environment (eg, heat, lack of lighting, lack of power) and the potential for active combat and triage, creating additional complexity. CONCLUSIONS: An ineffectiveness of communication is evident in both the civilian and military worlds. We used multiple methods of inquiry to study the information needs of trauma care and handoff, and the context of medical work in the field. Our findings informed the design and evaluation of an automated documentation tool. The data illustrated the need for more accurate recordkeeping, specifically temporal aspects, during transportation, and characterized the environment in which field testing of the developed tool will take place. The employment of a systems perspective in this project produced design insights that our team would not have identified otherwise. These insights created exciting and interesting challenges for the technical team to resolve.

Clinical use of artificial intelligence requires AI-capable organizations.

Artificial intelligence-based algorithms are being widely implemented in health care, even as evidence is emerging of bias in their design, problems with implementation, and potential harm to patients. To achieve the promise of using of AI-based tools to improve health, healthcare organizations will need to be AI-capable, with internal and external systems functioning in tandem to ensure the safe, ethical, and effective use of AI-based tools. Ideas are starting to emerge about the organizational routines, competencies, resources, and infrastructures that will be required for safe and effective deployment of AI in health care, but there has been little empirical research. Infrastructures that provide legal and regulatory guidance for managers, clinician competencies for the safe and effective use of AI-based tools, and learner-centric resources such as clear AI documentation and local health ecosystem impact reviews can help drive continuous improvement.

Enabling adoption and use of new health information technology during implementation: Roles and strategies for internal and external support personnel.

OBJECTIVE: Successful technological implementations frequently involve individuals who serve as mediators between end users, management, and technology developers. The goal for this project was to evaluate the structure and activities of such mediators in a large-scale electronic health record implementation. MATERIALS AND METHODS: Field notes from observations taken during implementation beginning in November 2017 were analyzed qualitatively using a thematic analysis framework to examine the relationship between specific types of mediators and the type and level of support to end users. RESULTS: We found that support personnel possessing both contextual knowledge of the institution's workflow and training in the new technology were the most successful in mediation of adoption and use. Those that lacked context of either technology or institutional workflow often displayed barriers in communication, trust, and active problem solving. CONCLUSIONS: These findings suggest that institutional investment in technology training and explicit programs to foster skills in mediation, including roles for professionals with career development opportunities, prior to implementation can be beneficial in easing the pain of system transition.

Disappearing expertise in clinical automation: Barcode medication administration and nurse autonomy.

OBJECTIVE: Using the case of barcode medication administration (BCMA), our objective is to describe the challenges nurses face when informatics tools are not designed to accommodate the full complexity of their work. MATERIALS AND METHODS: Autonomy is associated with nurse satisfaction and quality of care. BCMA organizes patient information and verifies medication administration. However, it presents challenges to nurse autonomy. Qualitative fieldwork, including observations of everyday work and interviews, was conducted during the implementation of BCMA in a large academic medical center. Fieldnotes and interview transcripts were coded and analyzed to describe nurses' perspectives on medication safety. RESULTS: Nurses adopt orienting frames to structure work routines and require autonomy to ensure safe task completion. Nurses exerted agency by trusting their own judgment over system information when the system did not consider workload complexity. Our results indicate that the system's rigidity clashed with adaptive needs embodied by nurses' orienting frames. DISCUSSION: Despite the fact that the concept of nurse as knowledge worker is foundational to informatics, nurses may be perceived as doers, rather than knowledge workers. In practice, nurses not only make decisions, but also engage in highly complex task-related work that is not well supported by process-oriented information technology tools. CONCLUSIONS: Information technology developers and healthcare organization managers should engage and better understand nursing work in order to develop technological and social systems to support it.

A Perioperative Care Display for Understanding High Acuity Patients.

BACKGROUND: The data visualization literature asserts that the details of the optimal data display must be tailored to the specific task, the background of the user, and the characteristics of the data. The general organizing principle of a concept-oriented display is known to be useful for many tasks and data types. OBJECTIVES: In this project, we used general principles of data visualization and a co-design process to produce a clinical display tailored to a specific cognitive task, chosen from the anesthesia domain, but with clear generalizability to other clinical tasks. To support the work of the anesthesia-in-charge (AIC) our task was, for a given day, to depict the acuity level and complexity of each patient in the collection of those that will be operated on the following day. The AIC uses this information to optimally allocate anesthesia staff and providers across operating rooms. METHODS: We used a co-design process to collaborate with participants who work in the AIC role. We conducted two in-depth interviews with AICs and engaged them in subsequent input on iterative design solutions. RESULTS: Through a co-design process, we found (1) the need to carefully match the level of detail in the display to the level required by the clinical task, (2) the impedance caused by irrelevant information on the screen such as icons relevant only to other tasks, and (3) the desire for a specific but optional trajectory of increasingly detailed textual summaries. CONCLUSION: This study reports a real-world clinical informatics development project that engaged users as co-designers. Our process led to the user-preferred design of a single binary flag to identify the subset of patients needing further investigation, and then a trajectory of increasingly detailed, text-based abstractions for each patient that can be displayed when more information is needed.

Risky business: a scoping review for communicating results of predictive models between providers and patients.

OBJECTIVE: Given widespread excitement around predictive analytics and the proliferation of machine learning algorithms that predict outcomes, a key next step is understanding how this information is-or should be-communicated with patients. MATERIALS AND METHODS: We conducted a scoping review informed by PRISMA-ScR guidelines to identify current knowledge and gaps in this domain. RESULTS: Ten studies met inclusion criteria for full text review. The following topics were represented in the studies, some of which involved more than 1 topic: disease prevention (N = 5/10, 50%), treatment decisions (N = 5/10, 50%), medication harms reduction (N = 1/10, 10%), and presentation of cardiovascular risk information (N = 5/10, 50%). A single study included 6- and 12-month clinical outcome metrics. DISCUSSION: As predictive models are increasingly published, marketed by industry, and implemented, this paucity of relevant research poses important gaps. Published studies identified the importance of (1) identifying the most effective source of information for patient communications; (2) contextualizing risk information and associated design elements based on users' needs and problem areas; and (3) understanding potential impacts on risk factor modification and behavior change dependent on risk presentation. CONCLUSION: An opportunity remains for researchers and practitioners to share strategies for effective selection of predictive algorithms for clinical practice, approaches for educating clinicians and patients in effectively using predictive data, and new approaches for framing patient-provider communication in the era of artificial intelligence.

Improving health IT through understanding the cultural production of safety in clinical settings.

Health IT is said to have the potential to improve the safety and effectiveness of care. However, it is known that the implementation of health IT can introduce new risks into the environment of care as a result of design failures, implementation failures, and unintended consequences. The design and implementation of health IT systems reflect explicit or implicit assumptions about what constitutes safe practice. These assumptions may clash with those held by patients and clinicians who are the intended users and subjects of the technology. Current perspectives on risk are discussed and an anthropological approach to understanding the construction of safe practices in the clinical setting is explored using research in barcode medication administration.

Everyday objects and spaces: How they afford resilience in diabetes routines.

Thirty million Americans currently have diabetes, and a substantial portion do not reach the goals of clinical treatment. This is in part due to the complex barriers to effective self-care faced by people with diabetes. This study uses a patient work perspective, focusing on the everyday, lived experience of managing diabetes. Our primary research goal was to explore how the work of self-care is embedded in the other routines of everyday living. We found that everyday objects and spaces were instrumental in the incorporation of diabetes work into daily routines. Objects anchored diabetes tasks by linking illness-specific artifacts to space and time (e.g. a morning routine), and by enabling the performance on diabetes tasks while on the move in either planned or unplanned ways.

Organizational diagnostics: a systematic approach to identifying technology and workflow issues in clinical settings.

OBJECTIVES: Healthcare organizations need to rapidly adapt to new technology, policy changes, evolving payment strategies, and other environmental changes. We report on the development and application of a structured methodology to support technology and process improvement in healthcare organizations, Systematic Iterative Organizational Diagnostics (SIOD). SIOD was designed to evaluate clinical work practices, diagnose technology and workflow issues, and recommend potential solutions. MATERIALS AND METHODS: SIOD consists of five stages: (1) Background Scan, (2) Engagement Building, (3) Data Acquisition, (4) Data Analysis, and (5) Reporting and Debriefing. Our team applied the SIOD approach in two ambulatory clinics and an integrated ambulatory care center and used SIOD components during an evaluation of a large-scale health information technology transition. RESULTS: During the initial SIOD application in two ambulatory clinics, five major analysis themes were identified, grounded in the data: putting patients first, reducing the chaos, matching space to function, technology making work harder, and staffing is more than numbers. Additional themes were identified based on SIOD application to a multidisciplinary clinical center. The team also developed contextually grounded recommendations to address issues identified through applying SIOD. DISCUSSION: The SIOD methodology fills a problem identification gap in existing process improvement systems through an emphasis on issue discovery, holistic clinic functionality, and inclusion of diverse perspectives. SIOD can diagnose issues where approaches as Lean, Six Sigma, and other organizational interventions can be applied. CONCLUSION: The complex structure of work and technology in healthcare requires specialized diagnostic strategies to identify and resolve issues, and SIOD fills this need.

Patient Stories Can Make a Difference in Patient-Centered Research Design.

Amid increasing interest in improving the patient-centeredness of research, new forms of engagement are emerging that enable researchers to get input from community members on research goals, methods, and implementation. This input often includes stories, which are useful for understanding lived experiences of illness and encounters with health care organizations, and for locating these experiences within larger meta-narratives of specific communities. We analyzed the stories in transcripts of 13 Community Engagement Studios and identified 4 major functions that the stories served in the sessions. Major functions included: (1) establishing mutual understanding, (2) adding expansion and depth, (3) characterizing abstract concepts, and (4) providing context for experience, with the latter being the most frequent. We assert that stories can serve to better communicate the complex contexts of patient experiences, helping to align research priorities and research design with community interests, leading to more patient-centered innovations in clinical practice.

Improving the Effectiveness of Health Information Technology: The Case for Situational Analytics.

Health information technology has contributed to improvements in quality and safety in clinical settings. However, the implementation of new technologies in health care has also been associated with the introduction of new sociotechnical hazards, produced through a range of complex interactions that vary with social, physical, temporal, and technological context. Other industries have been confronted with this problem and have developed advanced analytics to examine context-specific activities of workers and related outcomes. The skills and data exist in health care to develop similar insights through situational analytics, defined as the application of analytic methods to characterize human activity in situations and identify patterns in activity and outcomes that are influenced by contextual factors. This article describes the approach of situational analytics and potentially useful data sources, including trace data from electronic health record activity, reports from users, qualitative field data, and locational data. Key implementation requirements are discussed, including the need for collaboration among qualitative researchers and data scientists, organizational and federal level infrastructure requirements, and the need to implement a parallel research program in ethics to understand how the data are being used by organizations and policy makers.

Using a sociotechnical framework to understand adaptations in health IT implementation.

PURPOSE: When barcode medication administration (BCMA) is implemented nurses are required to integrate not only a new set of procedures or artifacts into everyday work, but also an orientation to medication safety itself that is sometimes at odds with their own. This paper describes how the nurses' orientation (the Practice Frame) can collide with the orientation that is represented by the technology and its implementation (the System Frame), resulting in adaptations at the individual and organization levels. METHODS: The paper draws on two qualitative research studies that examined the implementation of BCMA in inpatient settings using observation and ethnographic fieldwork, content analysis of email communications, and interviews with healthcare professionals. RESULTS: Two frames of reference are described: the System Frame and the Practice Frame. We found collisions of these frames that prompted adaptations at the individual and organization levels. The System Frame was less integrated and flexible than the Practice Frame, less able to account for all of the dimensions of everyday patient care to which medication administration is tied. CONCLUSION: Collisions in frames during implementation of new technology result in adaptations at the individual and organization level that can have a variety of effects. We found adaptations to be a means of evolving both the work routines and the technology. Understanding the frames of clinical workers when new technology is being designed and implemented can inform changes to technology or organizational structure and policy that can preclude unproductive or unsafe adaptations.

Barcode medication administration: supporting transitions in articulation work.

Articulation work is that which enables coordinated activity among colleagues distributed in time and space. Despite its important role in clinical settings, this work remains largely invisible in process flowcharts. When process-oriented information systems are implemented, the informal, flexible, contingent activities of participants that enable coordinated work are suddenly placed in a new context. Articulation work must adapt to new contexts of automation, and there are opportunities for clinical systems to better support coordination activities. This research explores the articulation work involved in medication administration, how it is affected by the implementation of barcoding, and strategies for support and problem resolution in this arena.

Implementing barcode medication administration: nurses' attitudes.

To reduce administration-related medication errors hospitals are implementing Barcode Medication Administration (BCMA). This 3-phase study examines nurses' expectations, experiences and attitudes related to BCMA before, during and after implementation. Physical artifacts (carts, scanners, batteries) associated with BCMA can be sources of negative attitudes toward the system.