Satisfaction of Older Patients With Emergency Department Care: Psychometric Properties and Construct Validity of the Consumer Emergency Care Satisfaction Scale.

BACKGROUND: Patient satisfaction is an important indicator of quality of care, but its measurement remains challenging. The Consumer Emergency Care Satisfaction Scale (CECSS) was developed to measure patient satisfaction in the emergency department (ED). Although this is a valid and reliable tool, several aspects of the CECSS need to be improved, including the definition, dimension, and scoring of scales. PURPOSE: The purpose of this study was to examine the construct validity of the CECSS and make suggestions on how to improve the tool to measure overall satisfaction with ED care. METHODS: We administered 2 surveys to older adults who presented with a fall to the ED and used electronic health record data to examine construct validity of the CECSS and ceiling effects. RESULTS: Using several criteria, we improved construct validity of the CECSS, reduced ceiling effects, and standardized scoring. CONCLUSION: We addressed several methodological issues with the CECSS and provided recommendations for improvement.

Caregiver perceptions of in-home COVID-19 testing for children with medical complexity: a qualitative study.

BACKGROUND: In-home direct antigen rapid testing (DART) plays a major role in COVID-19 mitigation and policy. However, perceptions of DART within high-risk, intellectually impaired child populations are unknown. This lack of research could negatively influence DART uptake and utility among those who stand to benefit most from DART. The purpose of this study was to describe caregivers' perceptions of an in-home COVID-19 DART regimen in children with medical complexity, including the benefits and limitations of DART use. METHODS: This qualitative study was a subproject of the NIH Rapid Acceleration of Diagnostics Underserved Populations research program at the University of Wisconsin. We combined survey data and the thematic analysis of semi-structured interview data to understand caregivers' perceptions of in-home COVID-19 testing and motivators to perform testing. Caregivers of children with medical complexity were recruited from the Pediatric Complex Care Program at the University of Wisconsin (PCCP). Data were collected between May and August 2021. RESULTS: Among n = 20 caregivers, 16/20 (80%) of their children had neurologic conditions and 12/20 (60%) used home oxygen. Survey data revealed that the largest caregiver motivators to test their child were to get early treatment if positive (18/20 [90%] of respondents agreed) and to let the child's school know if the child was safe to attend (17/20 [85%] agreed). Demotivators to testing included that the child could still get COVID-19 later (7/20 [35%] agreed), and the need for officials to reach out to close contacts (6/20 [30%] agreed). From interview data, four overarching themes described perceptions of in-home COVID-19 testing: Caregivers perceived DART on a spectrum of 1) benign to traumatic and 2) simple to complex. Caregivers varied in the 3) extent to which DART contributed to their peace of mind and 4) implications of test results for their child. CONCLUSIONS: Although participants often described DART as easy to administer and contributing to peace of mind, they also faced critical challenges and limitations using DART. Future research should investigate how to minimize the complexity of DART within high-risk populations, while leveraging DART to facilitate safe school attendance for children with medical complexity and reduce caregiver burden.

Older Adult Patients and Care Partners as Knowledge Brokers in Fragmented Health Care.

OBJECTIVE: To describe older adult patients' and care partners' knowledge broker roles during emergency department (ED) visits. BACKGROUND: Older adult patients are vulnerable to communication and coordination challenges during an ED visit, which can be exacerbated by the time and resource constrained ED environment. Yet, as a constant throughout the patient journey, patients and care partners can act as an information conduit, or knowledge broker, between fragmented care systems to attain high-quality, safe care. METHODS: Participants included 14 older adult patients (≥ 65 years old) and their care partners (e.g., spouse, adult child) who presented to the ED after having experienced a fall. Human factors researchers collected observation data from patients, care partners and clinician interactions during the patient's ED visit. We used an inductive content analysis to determine the role of patients and care partners as knowledge brokers. RESULTS: We found that patients and care partners act as knowledge brokers by providing information about diagnostic testing, medications, the patient's health history, and care accommodations at the disposition location. Patients and care partners filled the role of knowledge broker proactively (i.e. offer information) and reactively (i.e. are asked to provide information by clinicians or staff), within-ED work system and across work systems (e.g., between the ED and hospital), and in anticipation of future knowledge brokering. CONCLUSION: Patients and care partners, acting as knowledge brokers, often fill gaps in communication and participate in care coordination that assists in mitigating health care fragmentation.

Dashboarding to Monitor Machine-Learning-Based Clinical Decision Support Interventions.

BACKGROUND: Existing monitoring of machine-learning-based clinical decision support (ML-CDS) is focused predominantly on the ML outputs and accuracy thereof. Improving patient care requires not only accurate algorithms but also systems of care that enable the output of these algorithms to drive specific actions by care teams, necessitating expanding their monitoring. OBJECTIVES: In this case report, we describe the creation of a dashboard that allows the intervention development team and operational stakeholders to govern and identify potential issues that may require corrective action by bridging the monitoring gap between model outputs and patient outcomes. METHODS: We used an iterative development process to build a dashboard to monitor the performance of our intervention in the broader context of the care system. RESULTS: Our investigation of best practices elsewhere, iterative design, and expert consultation led us to anchor our dashboard on alluvial charts and control charts. Both the development process and the dashboard itself illuminated areas to improve the broader intervention. CONCLUSION: We propose that monitoring ML-CDS algorithms with regular dashboards that allow both a context-level view of the system and a drilled down view of specific components is a critical part of implementing these algorithms to ensure that these tools function appropriately within the broader care system.

Academic Detailing as a Health Information Technology Implementation Method: Supporting the Design and Implementation of an Emergency Department-Based Clinical Decision Support Tool to Prevent Future Falls.

BACKGROUND: Clinical decision support (CDS) tools that incorporate machine learning-derived content have the potential to transform clinical care by augmenting clinicians' expertise. To realize this potential, such tools must be designed to fit the dynamic work systems of the clinicians who use them. We propose the use of academic detailing-personal visits to clinicians by an expert in a specific health IT tool-as a method for both ensuring the correct understanding of that tool and its evidence base and identifying factors influencing the tool's implementation. OBJECTIVE: This study aimed to assess academic detailing as a method for simultaneously ensuring the correct understanding of an emergency department-based CDS tool to prevent future falls and identifying factors impacting clinicians' use of the tool through an analysis of the resultant qualitative data. METHODS: Previously, our team designed a CDS tool to identify patients aged 65 years and older who are at the highest risk of future falls and prompt an interruptive alert to clinicians, suggesting the patient be referred to a mobility and falls clinic for an evidence-based preventative intervention. We conducted 10-minute academic detailing interviews (n=16) with resident emergency medicine physicians and advanced practice providers who had encountered our CDS tool in practice. We conducted an inductive, team-based content analysis to identify factors that influenced clinicians' use of the CDS tool. RESULTS: The following categories of factors that impacted clinicians' use of the CDS were identified: (1) aspects of the CDS tool's design (2) clinicians' understanding (or misunderstanding) of the CDS or referral process, (3) the busy nature of the emergency department environment, (4) clinicians' perceptions of the patient and their associated fall risk, and (5) the opacity of the referral process. Additionally, clinician education was done to address any misconceptions about the CDS tool or referral process, for example, demonstrating how simple it is to place a referral via the CDS and clarifying which clinic the referral goes to. CONCLUSIONS: Our study demonstrates the use of academic detailing for supporting the implementation of health information technologies, allowing us to identify factors that impacted clinicians' use of the CDS while concurrently educating clinicians to ensure the correct understanding of the CDS tool and intervention. Thus, academic detailing can inform both real-time adjustments of a tool's implementation, for example, refinement of the language used to introduce the tool, and larger scale redesign of the CDS tool to better fit the dynamic work environment of clinicians.

Human Factors Engineering for the Pediatric Hospitalist.

As pediatrics hospitalists, we care for a diverse population of hospitalized children with increasing acuity and complexity in large, multidisciplinary medical teams. In this Method/ology paper, we summarize how human factors engineering (HFE) can provide a framework and tools to help us understand and improve our complex care processes and resulting outcomes. First, we define and discuss the 3 domains of HFE (ie, physical, cognitive, and organizational) and offer examples of HFE's application to pediatric hospital medicine. Next, we highlight an HFE-based framework, the Systems Engineering for Patient Safety model, which conceptualizes how our work system shapes health care processes and outcomes. We provide tools for leveraging this model to better understand the context in which our work is done, which, consequently, informs how we design our systems and processes to improve the quality and safety of care. Finally, we outline the basics of human-centered design and highlight a case study of a project completed in a pediatric hospital setting focused on making rounds more family-centered. In addition, we provide resources for those interested in learning more about HFE.

Effectiveness of an Emergency Department-Based Machine Learning Clinical Decision Support Tool to Prevent Outpatient Falls Among Older Adults: Protocol for a Quasi-Experimental Study.

BACKGROUND: Emergency department (ED) providers are important collaborators in preventing falls for older adults because they are often the first health care providers to see a patient after a fall and because at-home falls are often preceded by previous ED visits. Previous work has shown that ED referrals to falls interventions can reduce the risk of an at-home fall by 38%. Screening patients at risk for a fall can be time-consuming and difficult to implement in the ED setting. Machine learning (ML) and clinical decision support (CDS) offer the potential of automating the screening process. However, it remains unclear whether automation of screening and referrals can reduce the risk of future falls among older patients. OBJECTIVE: The goal of this paper is to describe a research protocol for evaluating the effectiveness of an automated screening and referral intervention. These findings will inform ongoing discussions about the use of ML and artificial intelligence to augment medical decision-making. METHODS: To assess the effectiveness of our program for patients receiving the falls risk intervention, our primary analysis will be to obtain referral completion rates at 3 different EDs. We will use a quasi-experimental design known as a sharp regression discontinuity with regard to intent-to-treat, since the intervention is administered to patients whose risk score falls above a threshold. A conditional logistic regression model will be built to describe 6-month fall risk at each site as a function of the intervention, patient demographics, and risk score. The odds ratio of a return visit for a fall and the 95% CI will be estimated by comparing those identified as high risk by the ML-based CDS (ML-CDS) and those who were not but had a similar risk profile. RESULTS: The ML-CDS tool under study has been implemented at 2 of the 3 EDs in our study. As of April 2023, a total of 1326 patient encounters have been flagged for providers, and 339 unique patients have been referred to the mobility and falls clinic. To date, 15% (45/339) of patients have scheduled an appointment with the clinic. CONCLUSIONS: This study seeks to quantify the impact of an ML-CDS intervention on patient behavior and outcomes. Our end-to-end data set allows for a more meaningful analysis of patient outcomes than other studies focused on interim outcomes, and our multisite implementation plan will demonstrate applicability to a broad population and the possibility to adapt the intervention to other EDs and achieve similar results. Our statistical methodology, regression discontinuity design, allows for causal inference from observational data and a staggered implementation strategy allows for the identification of secular trends that could affect causal associations and allow mitigation as necessary. TRIAL REGISTRATION: ClinicalTrials.gov NCT05810064; https://www.clinicaltrials.gov/study/NCT05810064. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/48128.

What makes a home? Designing home personas to represent the homes of families caring for children with medical complexity.

Personas are widely recognized as valuable design tools for communicating dimensions of individuals, yet they often lack critical contextual factors. For those people managing chronic health conditions, the home is a critical context of their patient work system (PWS). We propose the development of 'home personas' to convey essential aspects of the home context to those tasked with designing technologies and interventions to fit it. We used an iterative, multi-stakeholder design process to design 'home personas' for a model population, families caring for children with medical complexity. Each of the four resultant home personas-Multi-level, Customized, Ranch, and Rental-has a unique home layout, pain points, and are described on three dimensions that emerged from the data. This study builds on a foundation of work in the emerging field of Patient Ergonomics, describing a mechanism for distilling rich descriptions of the PWS into brief yet informative design tools.

Evaluating the Usability of an Emergency Department After Visit Summary: Staged Heuristic Evaluation.

BACKGROUND: Heuristic evaluations, while commonly used, may inadequately capture the severity of identified usability issues. In the domain of health care, usability issues can pose different levels of risk to patients. Incorporating diverse expertise (eg, clinical and patient) in the heuristic evaluation process can help assess and address potential negative impacts on patient safety that may otherwise go unnoticed. One document that should be highly usable for patients-with the potential to prevent adverse outcomes-is the after visit summary (AVS). The AVS is the document given to a patient upon discharge from the emergency department (ED), which contains instructions on how to manage symptoms, medications, and follow-up care. OBJECTIVE: This study aims to assess a multistage method for integrating diverse expertise (ie, clinical, an older adult care partner, and health IT) with human factors engineering (HFE) expertise in the usability evaluation of the patient-facing ED AVS. METHODS: We conducted a three-staged heuristic evaluation of an ED AVS using heuristics developed for use in evaluating patient-facing documentation. In stage 1, HFE experts reviewed the AVS to identify usability issues. In stage 2, 6 experts of varying expertise (ie, emergency medicine physicians, ED nurses, geriatricians, transitional care nurses, and an older adult care partner) rated each previously identified usability issue on its potential impact on patient comprehension and patient safety. Finally, in stage 3, an IT expert reviewed each usability issue to identify the likelihood of successfully addressing the issue. RESULTS: In stage 1, we identified 60 usability issues that violated a total of 108 heuristics. In stage 2, 18 additional usability issues that violated 27 heuristics were identified by the study experts. Impact ratings ranged from all experts rating the issue as "no impact" to 5 out of 6 experts rating the issue as having a "large negative impact." On average, the older adult care partner representative rated usability issues as being more significant more of the time. In stage 3, 31 usability issues were rated by an IT professional as "impossible to address," 21 as "maybe," and 24 as "can be addressed." CONCLUSIONS: Integrating diverse expertise when evaluating usability is important when patient safety is at stake. The non-HFE experts, included in stage 2 of our evaluation, identified 23% (18/78) of all the usability issues and, depending on their expertise, rated those issues as having differing impacts on patient comprehension and safety. Our findings suggest that, to conduct a comprehensive heuristic evaluation, expertise from all the contexts in which the AVS is used must be considered. Combining those findings with ratings from an IT expert, usability issues can be strategically addressed through redesign. Thus, a 3-staged heuristic evaluation method offers a framework for integrating context-specific expertise efficiently, while providing practical insights to guide human-centered design.

Feasibility Testing of Tubes@HOME: A Mobile Application to Support Family-Delivered Enteral Care.

BACKGROUND: Assistance from medical devices is common for children with medical complexity (CMC) but introduces caregiving challenges. We tested the feasibility of "Tubes@HOME," a mobile application supporting CMC family-delivered care using enteral care as a model. METHODS: Caregivers of CMC with enteral tubes participated in a 30-day feasibility study of Tubes@HOME November 2020 through January 2021. Tubes@HOME was available on mobile devices and designed to support collaborative care and tracking over time. Key features include child profile, caregiving network management, care routines, feedback loop, and action plans. Care routines delineated nutrition, medication, and procedural tasks needed for the child: frequencies, completions, and reminders. Metadata summarized feature use among users. Feasibility was evaluated with postuse questionnaires and interviews. Measures of Tubes@HOME's usability and usefulness included the NASA Task Load Index (TLX), System Usability Scale (SUS), and Acceptability and Use of Technology Questionnaire (AUTQ). RESULTS: Among n = 30 children, there were 30 primary (eg, parent) and n = 22 nonprimary caregivers using Tubes@HOME. Children had a median (IQR) 10 (5.5-13) care routines created. For care routines created, 93% were marked complete at least once during the study period, with participants engaging with routines throughout study weeks 2 to 4. Results (mean [SD]) indicated low mental workload (TLX) 30.9 (12.2), good usability (SUS) 75.4 (14.7), and above-average usefulness (AUTQ) 4.0 (0.7) associated with Tubes@HOME, respectively. Interviews contextualized usefulness and suggested improvements. CONCLUSIONS: Longitudinal use of Tubes@HOME among caregiving networks appeared feasible. Efficacy testing is needed, and outcomes could include reliability of care delivered in home and community.

Priorities for Safer In-Person School for Children With Medical Complexity During COVID-19.

OBJECTIVES: To establish statewide consensus priorities for safer in-person school for children with medical complexity (CMC) during the coronavirus disease 2019 (COVID-19) pandemic using a rapid, replicable, and transparent priority-setting method. METHODS: We adapted the Child Health and Nutrition Research Initiative Method, which allows for crowdsourcing ideas from diverse stakeholders and engages technical experts in prioritizing these ideas using predefined scoring criteria. Crowdsourcing surveys solicited ideas from CMC families, school staff, clinicians and administrators through statewide distribution groups/listservs using the prompt: "It is safe for children with complex health issues and those around them (families, teachers, classmates, etc.) to go to school in-person during the COVID-19 pandemic if/when…" Ideas were aggregated and synthesized into a unique list of candidate priorities. Thirty-four experts then scored each candidate priority against 5 criteria (equity, impact on COVID-19, practicality, sustainability, and cost) using a 5-point Likert scale. Scores were weighted and predefined thresholds applied to identify consensus priorities. RESULTS: From May to June 2021, 460 stakeholders contributed 1166 ideas resulting in 87 candidate priorities. After applying weighted expert scores, 10 consensus CMC-specific priorities exceeded predetermined thresholds. These priorities centered on integrating COVID-19 safety and respiratory action planning into individualized education plans, educating school communities about CMC's unique COVID-19 risks, using medical equipment safely, maintaining curricular flexibility, ensuring masking and vaccination, assigning seats during transportation, and availability of testing and medical staff at school. CONCLUSIONS: Priorities for CMC, identified by statewide stakeholders, complement and extend existing recommendations. These priorities can guide implementation efforts to support safer in-person education for CMC.

Disparate perspectives: Exploring healthcare professionals' misaligned mental models of older adults' transitions of care between the emergency department and skilled nursing facility.

Care transitions that occur across healthcare system boundaries represent a unique challenge for maintaining high quality care and patient safety, as these systems are typically not aligned to perform the care transition process. We explored healthcare professionals' mental models of older adults' transitions between the emergency department (ED) and skilled nursing facility (SNF). We conducted a thematic analysis of interviews with ED and SNF healthcare professionals and identified three themes: 1) ED and SNF healthcare professionals had misaligned mental models regarding communication processes and tools used during care transitions, 2) ED and SNF healthcare professionals had misaligned mental models regarding healthcare system capability, and 3) Misalignments led to individual and organizational consequences. Overall, we found that SNF and ED healthcare professionals are part of the same process but have different perceptions of the process. Future work must take steps to redesign and realign these distinct work systems such that those involved conceptualize themselves as part of a joint process.

Medical Device Workarounds in Providing Care for Children With Medical Complexity in the Home.

OBJECTIVES: Children with medical complexity (CMC) are commonly assisted by medical devices, which family caregivers are responsible for managing and troubleshooting in the home. Optimizing device use by maximizing the benefits and minimizing the complications is a critical goal for CMC but is relatively unexplored. In this study, we sought to identify and describe workarounds families have developed to optimize medical device use for their needs. METHODS: We conducted 30 contextual inquiry interviews with families of CMC in homes. Interviews were recorded, transcribed, and analyzed for barriers and workarounds specific to medical device usage through a directed content analysis. We used observation notes and photographs to confirm and elaborate on interview findings. RESULTS: We identified 4 barriers to using medical devices in the home: (1) the quantity and type of devices allotted do not meet family needs, (2) the device is not designed to be used in locations families require, (3) device use is physically or organizationally disruptive to the home, and (4) the device is not designed to fit the user. We also identified 11 categories of workarounds to the barriers. CONCLUSIONS: Families face many barriers in using medical devices to care for CMC. Our findings offer rich narrative and photographic data revealing the ways in which caregivers work around these barriers. Future researchers should explore the downstream effects of these ubiquitous, necessary workarounds on CMC outcomes toward developing interventions that optimize device use for families.

Codesign and Usability Testing of a Mobile Application to Support Family-Delivered Enteral Tube Care.

BACKGROUND: Enteral tubes are prevalent among children with medical complexity (CMC), and complications can lead to costly health care use. Our objective was to design and test the usability of a mobile application (app) to support family-delivered enteral tube care. METHODS: Human-centered design methods (affinity diagramming, persona development, and software development) were applied with family caregivers of CMC to develop a prototype. During 3 waves of usability testing with design refinement between waves, screen capture software collected user-app interactions and inductive content analysis of narrative feedback identified areas for design improvement. The National Aeronautics and Space Administration Task Load Index and the System Usability Scale quantified mental workload and ease of use. RESULTS: Design participants identified core app functions, including displaying care routines, reminders, tracking inventory and health data, caregiver communication, and troubleshooting. Usability testing participants were 80% non-Hispanic white, 28% lived in rural settings, and 20% had not completed high school. Median years providing enteral care was 2 (range 1-14). Design iterations improved app function, simplification, and user experience. The mean System Usability Scale score was 76, indicating above-average usability. National Aeronautics and Space Administration Task Load Index revealed low mental demand, frustration, and effort. All 14 participants reported that they would recommend the app, and that the app would help with organization, communication, and caregiver transitions. CONCLUSIONS: Using a human-centered codesign process, we created a highly usable mobile application to support enteral tube caregiving at home. Future work involves evaluating the feasibility of longitudinal use and effectiveness in improving self-efficacy and reduce device complications.

Physician Perceptions of Disposition Decision-making for Older Adults in the Emergency Department: A Preliminary Analysis.

Disposition decision-making in the emergency department (ED) is critical to patient safety and quality of care. Disposition decision-making has particularly important implications for older adults who comprise a significant portion of ED visits annually and are vulnerable to suboptimal outcomes throughout ED care transitions. We conducted a secondary inductive content analysis of interviews with ED physicians (N= 11) to explore their perceptions of who they involve in disposition decision-making and what information they use to make disposition decisions for older adults. ED physicians cited 7 roles (5 types of clinicians, patients and families) and 11 information types, both clinical (e.g. test/lab results) and non-clinical (e.g. family's preference). Our preliminary findings represent a key first step toward the development of interventions that promote patient safety and quality of care for older adults in the ED by supporting the cognitive and communicative aspects of disposition decision-making.