Bridging Communication Gaps Between Radiologists, Referring Physicians, and Patients Through Standardized Structured Cancer Imaging Reporting: The Experience with Female Pelvic MRI Assessment Using O-RADS and a Simulated Cohort Patient Group.

RATIONALE AND OBJECTIVES: This study aimed to evaluate whether implementing structured reporting based on Ovarian-Adnexal Reporting and Data System (O-RADS) magnetic resonance imaging (MRI) in women with sonographically indeterminate adnexal masses improves communication between radiologists, referrers, and patients/caregivers and enhances diagnostic performance for determining adnexal malignancy. MATERIALS AND METHODS: We retrospectively analyzed prospectively issued MRI reports in 2019-2022 performed for characterizing adnexal masses before and after implementing O-RADS MRI; 56 patients/caregivers and nine gynecologic oncologists ("referrers") were surveyed about report interpretability/clarity/satisfaction; responses for pre- and post-implementation reports were compared using Fisher's exact and Chi-squared tests. Diagnostic performance was assessed using receiver operating characteristic curves. RESULTS: A total of 123 reports from before and 119 reports from after O-RADS MRI implementation were included. Survey response rates were 35.7% (20/56) for patients/caregivers and 66.7% (6/9) for referrers. For patients/caregivers, O-RADS MRI reports were clearer (p < 0.001) and more satisfactory (p < 0.001) than unstructured reports, but interpretability did not differ significantly (p = 0.14), as 28.0% (28/100) of postimplementation and 38.0% (38/100) of preimplementation reports were considered difficult to interpret. For referrers, O-RADS MRI reports were clearer, more satisfactory, and easier to interpret (p < 0.001); only 1.3% (1/77) were considered difficult to interpret. For differentiating benign from malignant adnexal lesions, O-RADS MRI showed area under the curve of 0.92 (95% confidence interval [CI], 0.85-0.99), sensitivity of 0.81 (95% CI, 0.58-0.95), and specificity of 0.91 (95% CI, 0.83-0.96). Diagnostic performance of reports before implementation could not be calculated due to many different phrases used to describe the likelihood of malignancy. CONCLUSION: Implementing standardized structured reporting using O-RADS MRI for characterizing adnexal masses improved clarity and satisfaction for patients/caregivers and referrers. Interpretability improved for referrers but remained limited for patients/caregivers.

Work system barriers and facilitators of a team health information technology.

Designing health IT aimed at supporting team-based care and improving patient safety is difficult. This requires a work system (i.e., SEIPS) evaluation of the technology by care team members. This study aimed to identify work system barriers and facilitators to the use of a team health IT that supports care transitions for pediatric trauma patients. We conducted an analysis on 36 interviews - representing 12 roles - collected from a scenario-based evaluation of T3. We identified eight dimensions with both barriers and facilitators in all five work system elements: person (experience), task (task performance, workload/efficiency), technology (usability, specific features of T3), environment (space, location), and organization (communication/coordination). Designing technology that meets every role's needs is challenging; in particular, when trade-offs need to be managed, e.g., additional workload for one role or divergent perspectives regarding specific features. Our results confirm the usefulness of a continuous work system approach to technology design and implementation.

Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation.

BACKGROUND: Heuristic evaluations, while commonly used, may inadequately capture the severity of identified usability issues. In the domain of health care, usability issues can pose different levels of risk to patients. Incorporating diverse expertise (eg, clinical and patient) in the heuristic evaluation process can help assess and address potential negative impacts on patient safety that may otherwise go unnoticed. One document that should be highly usable for patients-with the potential to prevent adverse outcomes-is the after visit summary (AVS). The AVS is the document given to a patient upon discharge from the emergency department (ED), which contains instructions on how to manage symptoms, medications, and follow-up care. OBJECTIVE: This study aims to assess a multistage method for integrating diverse expertise (ie, clinical, an older adult care partner, and health IT) with human factors engineering (HFE) expertise in the usability evaluation of the patient-facing ED AVS. METHODS: We conducted a three-staged heuristic evaluation of an ED AVS using heuristics developed for use in evaluating patient-facing documentation. In stage 1, HFE experts reviewed the AVS to identify usability issues. In stage 2, 6 experts of varying expertise (ie, emergency medicine physicians, ED nurses, geriatricians, transitional care nurses, and an older adult care partner) rated each previously identified usability issue on its potential impact on patient comprehension and patient safety. Finally, in stage 3, an IT expert reviewed each usability issue to identify the likelihood of successfully addressing the issue. RESULTS: In stage 1, we identified 60 usability issues that violated a total of 108 heuristics. In stage 2, 18 additional usability issues that violated 27 heuristics were identified by the study experts. Impact ratings ranged from all experts rating the issue as "no impact" to 5 out of 6 experts rating the issue as having a "large negative impact." On average, the older adult care partner representative rated usability issues as being more significant more of the time. In stage 3, 31 usability issues were rated by an IT professional as "impossible to address," 21 as "maybe," and 24 as "can be addressed." CONCLUSIONS: Integrating diverse expertise when evaluating usability is important when patient safety is at stake. The non-HFE experts, included in stage 2 of our evaluation, identified 23% (18/78) of all the usability issues and, depending on their expertise, rated those issues as having differing impacts on patient comprehension and safety. Our findings suggest that, to conduct a comprehensive heuristic evaluation, expertise from all the contexts in which the AVS is used must be considered. Combining those findings with ratings from an IT expert, usability issues can be strategically addressed through redesign. Thus, a 3-staged heuristic evaluation method offers a framework for integrating context-specific expertise efficiently, while providing practical insights to guide human-centered design.

Satisfaction of Older Patients With Emergency Department Care: Psychometric Properties and Construct Validity of the Consumer Emergency Care Satisfaction Scale.

BACKGROUND: Patient satisfaction is an important indicator of quality of care, but its measurement remains challenging. The Consumer Emergency Care Satisfaction Scale (CECSS) was developed to measure patient satisfaction in the emergency department (ED). Although this is a valid and reliable tool, several aspects of the CECSS need to be improved, including the definition, dimension, and scoring of scales. PURPOSE: The purpose of this study was to examine the construct validity of the CECSS and make suggestions on how to improve the tool to measure overall satisfaction with ED care. METHODS: We administered 2 surveys to older adults who presented with a fall to the ED and used electronic health record data to examine construct validity of the CECSS and ceiling effects. RESULTS: Using several criteria, we improved construct validity of the CECSS, reduced ceiling effects, and standardized scoring. CONCLUSION: We addressed several methodological issues with the CECSS and provided recommendations for improvement.

Managing multiple perspectives in the collaborative design process of a team health information technology.

We need to design technologies that support the work of health care teams; designing such solutions should integrate different clinical roles. However, we know little about the actual collaboration that occurs in the design process for a team-based care solution. This study examines how multiple perspectives were managed in the design of a team health IT solution aimed at supporting clinician information needs during pediatric trauma care transitions. We focused our analysis on four co-design sessions that involved multiple clinicians caring for pediatric trauma patients. We analyzed design session transcripts using content analysis and process coding guided by Détienne's (2006) co-design framework. We expanded upon Détienne (2006) three collaborative activities to identify specific themes and processes of collaboration between care team members engaged in the design process. The themes and processes describe how team members collaborated in a team health IT design process that resulted in a highly usable technology.

A cross-sector systematic review and synthesis of knowledge on telemedicine interventions in chronic wound management-Implications from a system perspective.

Based on initially identified needs for further telemedicine (TM) and chronic wound management research, the objective of this article is twofold: to conduct a systematic review of existing knowledge on TM interventions in chronic wound management-including barriers and opportunities-across the specialist and primary care sectors, and to incorporate the review findings into a system framework that can be further developed and validated through empirical data. We conclude that there is a pressing need for broader and more comprehensive empirical explorations into quality improvement and integration of TM in chronic wound management, including using system frameworks that can capture cross-sector system perspectives and associated implications. Of practical consideration, we suggest that the design and execution of TM improvement interventions and associated research projects should be conducted in close cooperation with managers and practitioners knowledgeable about barriers and opportunities that can influence the implementation of important interventions within chronic wound management.

Retrospective analysis of the human-centered design process used to develop a clinical decision support in the emergency department: PE Dx Study Part 2.

With the growing implementation and use of health IT such as Clinical Decision Support (CDS), there is increasing attention on the potential negative impact of these technologies on patients (e.g., medication errors) and clinicians (e.g., increased workload, decreased job satisfaction, burnout). Human-Centered Design (HCD) and Human Factors (HF) principles are recommended to improve the usability of health IT and reduce its negative impact on patients and clinicians; however, challenges persist. The objective of this study is to understand how an HCD process influences the usability of health IT. We conducted a systematic retrospective analysis of the HCD process used in the design of a CDS for pulmonary embolism diagnosis in the emergency department (ED). Guided by the usability outcomes (e.g., barriers and facilitators) of the CDS use "in the wild" (see Part 1 of this research in the accompanying manuscript), we performed deductive content analysis of 17 documents (e.g., design session transcripts) produced during the HCD process. We describe if and how the design team considered the barriers and facilitators during the HCD process. We identified 7 design outcomes of the HCD process, for instance designing a workaround and making a design change to the CDS. We identify gaps in the current HCD process and demonstrate the need for a continuous health IT design process.

Post-implementation usability evaluation of a human factors-based clinical decision support for pulmonary embolism (PE) diagnosis (Dx): PE Dx Study Part 1.

While there is promise for health IT, such as Clinical Decision Support (CDS), to improve patient safety and clinician efficiency, poor usability has hindered widespread use of these tools. Human Factors (HF) principles and methods remain the gold standard for health IT design; however, there is limited information on how HF methods and principles influence CDS usability "in the wild". In this study, we explore the usability of an HF-based CDS used in the clinical environment; the CDS was designed according to a human-centered design process, which is described in Carayon et al. (2020). In this study, we interviewed 12 emergency medicine physicians, identifying 294 excerpts of barriers and facilitators of the CDS. Sixty-eight percent of excerpts related to the HF principles applied in the human-centered design of the CDS. The remaining 32% of excerpts related to 18 inductively-created categories, which highlight gaps in the CDS design process. Several barriers were related to the physical environment and organization work system elements as well as physicians' broader workflow in the emergency department (e.g., teamwork). This study expands our understanding of the usability outcomes of HF-based CDS "in the wild". We demonstrate the value of HF principles in the usability of CDS and identify areas for improvement to future human-centered design of CDS. The relationship between these usability outcomes and the HCD process is explored in an accompanying Part 2 manuscript.

A Systematic Review of the Consumer Emergency Care Satisfaction Scale (CECSS).

BACKGROUND: Patient experience is receiving increasing attention in the context of patient-centered care. However, there are relatively few instruments that measure patient experience that are valid and reliable. OBJECTIVE: In this study, we systematically review the literature on the Consumer Emergency Care Satisfaction Scale (CECSS) and examine its psychometric properties. METHODS: We conducted a systematic literature search in the Cumulative Index to Nursing and Allied Health Literature, PubMed, PsycINFO, and Web of Science databases on articles that contain information on the CECSS. The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. RESULTS: Our systematic literature search resulted in 28 articles in which the CECSS was used. CONCLUSIONS: Results of our literature review show that from a psychometric perspective, the CECSS is a valid and reliable instrument. However, the results of our study also show that the CECSS has several weaknesses. We have made recommendations to improve the CECSS.

Team Cognition in Handoffs: Relating System Factors, Team Cognition Functions and Outcomes in Two Handoff Processes.

OBJECTIVE: This study investigates how team cognition occurs in care transitions from operating room (OR) to intensive care unit (ICU). We then seek to understand how the sociotechnical system and team cognition are related. BACKGROUND: Effective handoffs are critical to ensuring patient safety and have been the subject of many improvement efforts. However, the types of team-level cognitive processing during handoffs have not been explored, nor is it clear how the sociotechnical system shapes team cognition. METHOD: We conducted this study in an academic, Level 1 trauma center in the Midwestern United States. Twenty-eight physicians (surgery, anesthesia, pediatric critical care) and nurses (OR, ICU) participated in semi-structured interviews. We performed qualitative content analysis and epistemic network analysis to understand the relationships between system factors, team cognition in handoffs and outcomes. RESULTS: Participants described three team cognition functions in handoffs-(1) information exchange, (2) assessment, and (3) planning and decision making; information exchange was mentioned most. Work system factors influenced team cognition. Inter-professional handoffs facilitated information exchange but included large teams with diverse backgrounds communicating, which can be inefficient. Intra-professional handoffs decreased team size and role diversity, which may simplify communication but increase information loss. Participants in inter-professional handoffs reflected on outcomes significantly more in relation to system factors and team cognition (p < 0.001), while participants in intra-professional handoffs discussed handoffs as a task. CONCLUSION: Handoffs include team cognition, which was influenced by work system design. Opportunities for handoff improvement include a flexibly standardized process and supportive tools/technologies. We recommend incorporating perspectives of the patient and family in future work.

Older Adult Patients and Care Partners as Knowledge Brokers in Fragmented Health Care.

OBJECTIVE: To describe older adult patients' and care partners' knowledge broker roles during emergency department (ED) visits. BACKGROUND: Older adult patients are vulnerable to communication and coordination challenges during an ED visit, which can be exacerbated by the time and resource constrained ED environment. Yet, as a constant throughout the patient journey, patients and care partners can act as an information conduit, or knowledge broker, between fragmented care systems to attain high-quality, safe care. METHODS: Participants included 14 older adult patients (≥ 65 years old) and their care partners (e.g., spouse, adult child) who presented to the ED after having experienced a fall. Human factors researchers collected observation data from patients, care partners and clinician interactions during the patient's ED visit. We used an inductive content analysis to determine the role of patients and care partners as knowledge brokers. RESULTS: We found that patients and care partners act as knowledge brokers by providing information about diagnostic testing, medications, the patient's health history, and care accommodations at the disposition location. Patients and care partners filled the role of knowledge broker proactively (i.e. offer information) and reactively (i.e. are asked to provide information by clinicians or staff), within-ED work system and across work systems (e.g., between the ED and hospital), and in anticipation of future knowledge brokering. CONCLUSION: Patients and care partners, acting as knowledge brokers, often fill gaps in communication and participate in care coordination that assists in mitigating health care fragmentation.

Usability of a Human Factors-based Clinical Decision Support in the Emergency Department: Lessons Learned for Design and Implementation.

OBJECTIVE: To evaluate the usability and use of human factors (HF)-based clinical decision support (CDS) implemented in the emergency department (ED). BACKGROUND: Clinical decision support can improve patient safety; however, the acceptance and use of CDS has faced challenges. Following a human-centered design process, we designed a CDS to support pulmonary embolism (PE) diagnosis in the ED. We demonstrated high usability of the CDS during scenario-based usability testing. We implemented the HF-based CDS in one ED in December 2018. METHOD: We conducted a survey of ED physicians to evaluate the usability and use of the HF-based CDS. We distributed the survey via Qualtrics, a web-based survey platform. We compared the computer system usability questionnaire scores of the CDS between those collected in the usability testing to use of the CDS in the real environment. We asked physicians about their acceptance and use of the CDS, barriers to using the CDS, and areas for improvement. RESULTS: Forty-seven physicians (56%) completed the survey. Physicians agreed that diagnosing PE is a major problem and risk scores can support the PE diagnostic process. Usability of the CDS was reported as high, both in the experimental setting and the real clinical setting. However, use of the CDS was low. We identified several barriers to the CDS use in the clinical environment, in particular a lack of workflow integration. CONCLUSION: Design of CDS should be a continuous process and focus on the technology's usability in the context of the broad work system and clinician workflow.

Human-centered design of team health IT for pediatric trauma care transitions.

BACKGROUND: As problems of acceptance, usability and workflow integration continue to emerge with health information technologies (IT), it is critical to incorporate human factors and ergonomics (HFE) methods and design principles. Human-centered design (HCD) provides an approach to integrate HFE and produce usable technologies. However, HCD has been rarely used for designing team health IT, even though team-based care is expanding. OBJECTIVE: To describe the HCD process used to develop a usable team health IT (T3 or Teamwork Transition Technology) that provides cognitive support to pediatric trauma care teams during transitions from the emergency department to the operating room and the pediatric intensive care unit. METHODS: The HCD process included seven steps in three phases of analysis, design activities and feedback. RESULTS: The HCD process involved multiple perspectives and clinical roles that were engaged in inter-related activities, leading to design requirements, i.e., goals for the technology, a set of 47 information elements, and a list of HFE design principles applied to T3. Results of the evaluation showed a high usability score for T3. CONCLUSIONS: HFE can be integrated in the HCD process through a range of methods and design principles. That design process can produce a usable technology that provides cognitive support to a large diverse team involved in pediatric trauma care transitions. Future research should continue to focus on HFE-based design of team health IT.

Scenario-Based Evaluation of Team Health Information Technology to Support Pediatric Trauma Care Transitions.

BACKGROUND: Clinicians need health information technology (IT) that better supports their work. Currently, most health IT is designed to support individuals; however, more and more often, clinicians work in cross-functional teams. Trauma is one of the leading preventable causes of children's death. Trauma care by its very nature is team based but due to the emergent nature of trauma, critical clinical information is often missed in the transition of these patients from one service or unit to another. Teamwork transition technology can help support these transitions and minimize information loss while enhancing information gathering and storage. In this study, we created a large screen technology to support shared situational awareness across multiple clinical roles and departments. OBJECTIVES: This study aimed to examine if the Teamwork Transition Technology (T3) supports teams and team cognition. METHODS: We used a scenario-based mock-up methodology with 36 clinicians and staff from the different units and departments who are involved in pediatric trauma to examine T3. RESULTS: Results of the evaluation show that most participants agreed that the technology helps achieve the goals set out in the design phase. Respondents thought that T3 organizes and presents information in a different way that was helpful to them. CONCLUSION: In this study, we examined a health IT (T3) that was designed to support teams and team cognition. The results of our evaluation show that participants agreed that T3 does support them in their work and increases their situation awareness.

Care transition of trauma patients: Processes with articulation work before and after handoff.

While care transitions influence quality of care, less work studies transitions between hospital units. We studied care transitions from the operating room (OR) to pediatric and adult intensive critical care units (ICU) using Systems Engineering Initiative for Patient Safety (SEIPS)-based process modeling. We interviewed twenty-nine physicians (surgery, anesthesia, pediatric critical care) and nurses (OR, ICU) and administered the AHRQ Hospital Survey on Patient Safety Culture items about handoffs, care transitions and teamwork. Care transitions are complex, spatio-temporal processes and involve work during the transition (i.e., handoff and transport) and preparation and follow up activities (i.e., articulation work). Physicians defined the transition as starting earlier and ending later than nurses. Clinicians in the OR to adult ICU transition without a team handoff reported significantly less information loss and better cooperation, despite positive interview data. A team handoff and supporting articulation work should increase awareness, improving quality and safety of care transitions.

Collaborative design and implementation of a clinical decision support system for automated fall-risk identification and referrals in emergency departments.

Of the 3 million older adults seeking fall-related emergency care each year, nearly one-third visited the Emergency Department (ED) in the previous 6 months. ED providers have a great opportunity to refer patients for fall prevention services at these initial visits, but lack feasible tools for identifying those at highest-risk. Existing fall screening tools have been poorly adopted due to ED staff/provider burden and lack of workflow integration. To address this, we developed an automated clinical decision support (CDS) system for identifying and referring older adult ED patients at risk of future falls. We engaged an interdisciplinary design team (ED providers, health services researchers, information technology/predictive analytics professionals, and outpatient Falls Clinic staff) to collaboratively develop a system that successfully met user requirements and integrated seamlessly into existing ED workflows. Our rapid-cycle development and evaluation process employed a novel combination of human-centered design, implementation science, and patient experience strategies, facilitating simultaneous design of the CDS tool and intervention implementation strategies. This included defining system requirements, systematically identifying and resolving usability problems, assessing barriers and facilitators to implementation (e.g., data accessibility, lack of time, high patient volumes, appointment availability) from multiple vantage points, and refining protocols for communicating with referred patients at discharge. ED physician, nurse, and patient stakeholders were also engaged through online surveys and user testing. Successful CDS design and implementation required integration of multiple new technologies and processes into existing workflows, necessitating interdisciplinary collaboration from the onset. By using this iterative approach, we were able to design and implement an intervention meeting all project goals. Processes used in this Clinical-IT-Research partnership can be applied to other use cases involving automated risk-stratification, CDS development, and EHR-facilitated care coordination.

A Sociotechnical Systems Framework for the Application of Artificial Intelligence in Health Care Delivery.

In the coming years, artificial intelligence (AI) will pervade almost every aspect of the health care delivery system. AI has the potential to improve patient safety (e.g. diagnostic accuracy) as well as reduce the burden on clinicians (e.g. documentation-related workload); however, these benefits are yet to be realized. AI is only one element of a larger sociotechnical system that needs to be considered for effective AI application. In this paper, we describe the current challenges of integrating AI into clinical care and propose a sociotechnical systems (STS) approach for AI design and implementation. We demonstrate the importance of an STS approach through a case study on the design and implementation of a clinical decision support (CDS). In order for AI to reach its potential, the entire work system as well as clinical workflow must be systematically considered throughout the design of AI technology.

Usability barriers and facilitators of a human factors engineering-based clinical decision support technology for diagnosing pulmonary embolism.

BACKGROUND: Health IT, such as clinical decision support (CDS), has the potential to improve patient safety. However, poor usability of health IT continues to be a major concern. Human factors engineering (HFE) approaches are recommended to improve the usability of health IT. Limited evidence exists on the actual impact of HFE methods and principles on the usability of health IT. OBJECTIVE: To identify and describe the usability barriers and facilitators of an HFE-based CDS prior to implementation in the emergency department (ED). METHODS: We conducted debrief interviews with 32 emergency medicine physicians as a part of a scenario-based simulation study evaluating the usability of the HFE-based CDS. We performed a deductive content analysis of the interviews using the usability criteria of Scapin and Bastien as a framework. RESULTS: We identified 271 occurrences of usability barriers (94) and facilitators (177) of the HFE-based CDS. For instance, we found a facilitator relating to the usability criteria prompting as the PE Dx helps the physician order diagnostic tests following the risk assessment. We found the most facilitators relating to the criteria, minimal actions, e.g. as the PE Dx automatically populates vitals signs (e.g., heart rate) from the chart into the CDS. The majority of the usability barriers related to the usability criteria, compatibility (i.e., workflow integration), which was not explicitly considered in the HFE design of the CDS. For example, the CDS did not support resident and attending physician teamwork in the PE diagnostic process. CONCLUSION: The systematic use of HFE principles in the design of CDS improves the usability of these technologies. In order to further reduce usability barriers, workflow integration should be explicitly considered in the design of health IT.

Health Care 4.0: A Vision for Smart and Connected Health Care.

Industry 4.0 has transformed manufacturing industry into a new paradigm. In a manner similar to manufacturing, health care delivery is at the dawn of a foundational change into the new era of smart and connected health care, referred to as Health Care 4.0. In this paper, we discuss the historical evolution of Health Care 1.0 to 4.0, describe the characteristics of smart and connected care in Health Care 4.0, identify multiple research challenges and opportunities of Health Care 4.0 in terms of data, model, dynamics, and integration, and outline the implications of people, process, system and health outcomes. Finally, conclusions and recommendations are presented in the areas of (1) involvement of multiple disciplines and perspectives, (2) development of technologies and methodologies with combination of quantitative and qualitative approaches, (3) closed-loop integration of sociotechnical system, and (4) design of person-centered system with specific attention to human needs and health equity.

Workflow integration analysis of a human factors-based clinical decision support in the emergency department.

Numerous challenges with the implementation, acceptance, and use of health IT are related to poor usability and a lack of integration of the technologies into clinical workflow, and have, therefore, limited the potential of these technologies to improve patient safety. We propose a definition and conceptual model of health IT workflow integration. Using interviews of 12 emergency department (ED) physicians, we identify 134 excerpts of barriers and facilitators to workflow integration of a human factors (HF)-based clinical decision support (CDS) implemented in the ED. Using data on these 134 barriers and facilitators, we distinguish 25 components of workflow integration of the CDS, which are described according to four dimensions of workflow integration: time, flow, scope of patient journey, and level. The proposed definition and conceptual model of workflow integration can be used to inform health IT design; this is the purpose of the proposed checklist that can help to ensure consideration of workflow integration during the development of health IT.