A Qualitative Analysis of Barriers to Evidence-Based Care in the Prehospital Management of Patients with Suspected Acute Coronary Syndrome.

OBJECTIVES: Prehospital electrocardiogram (ECG) and administration of aspirin are evidence-based strategies for patients with acute coronary syndrome (ACS). However, emergency medical services (EMS) compliance in patients with suspected ACS varies widely. We sought to understand the barriers to prehospital ECG acquisition and aspirin administration for patients with suspected ACS. METHODS: In this qualitative study, we interviewed EMS clinicians at three geographically diverse United States (U.S.)-based EMS agencies. We interviewed practicing clinicians and quality and operations leaders at these agencies. Based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework, interviews were recorded, transcribed, and analyzed using a grounded qualitative approach with open coding. The Systems Engineering Initiative for Patient Safety (SEIPS) framework and a constant comparison technique were used to identify and refine themes. RESULTS: Twenty-five paramedics and 20 additional agency personnel participated. Median age was 41 (IQR: 34-51) years and 13 (29%) were female. Themes were organized using SEIPS and longitudinally through four phases of an EMS call. During the pre-arrival phase, staffing challenges, training quality, and dispatch may anchor EMS clinicians on a diagnosis. During the diagnosis and treatment phase, safety and communication barriers may take priority over care delivery. Additionally, EMS clinicians must allocate assets (e.g. whether to send an advanced life support unit) and financial resources; veteran EMS clinicians identified their experience whereas newer clinicians cited their recent education when making these decisions. Also, diagnostic uncertainty due to increasing patient complexity and atypical presentations contributed to diagnostic errors. During the response and transport phase, the scope of practice limits the use and interpretation of the ECG, with clinicians reporting that liberal use of ECG led to more rapid decision-making. Finally, in the after phase, personnel reported the "psychologically taxing" nature of the job contributing to biases, bad habits, and burnout. Performance feedback was desired for personal development, though currently perceived as infrequent and punitive. CONCLUSIONS: Multiple, interrelated themes underscored the complexities of delivering evidence-based care to prehospital patients with ACS. Education in ECG interpretation, resource allocation, bias, and enhancing feedback may serve as strategies to address the identified barriers.

Determinants of appropriate antibiotic and NSAID prescribing in unscheduled outpatient settings in the veterans health administration.

BACKGROUND: Despite efforts to enhance the quality of medication prescribing in outpatient settings, potentially inappropriate prescribing remains common, particularly in unscheduled settings where patients can present with infectious and pain-related complaints. Two of the most commonly prescribed medication classes in outpatient settings with frequent rates of potentially inappropriate prescribing include antibiotics and nonsteroidal anti-inflammatory drugs (NSAIDs). In the setting of persistent inappropriate prescribing, we sought to understand a diverse set of perspectives on the determinants of inappropriate prescribing of antibiotics and NSAIDs in the Veterans Health Administration. METHODS: We conducted a qualitative study guided by the Consolidated Framework for Implementation Research and Theory of Planned Behavior. Semi-structured interviews were conducted with clinicians, stakeholders, and Veterans from March 1, 2021 through December 31, 2021 within the Veteran Affairs Health System in unscheduled outpatient settings at the Tennessee Valley Healthcare System. Stakeholders included clinical operations leadership and methodological experts. Audio-recorded interviews were transcribed and de-identified. Data coding and analysis were conducted by experienced qualitative methodologists adhering to the Consolidated Criteria for Reporting Qualitative Studies guidelines. Analysis was conducted using an iterative inductive/deductive process. RESULTS: We conducted semi-structured interviews with 66 participants: clinicians (N = 25), stakeholders (N = 24), and Veterans (N = 17). We identified six themes contributing to potentially inappropriate prescribing of antibiotics and NSAIDs: 1) Perceived versus actual Veterans expectations about prescribing; 2) the influence of a time-pressured clinical environment on prescribing stewardship; 3) Limited clinician knowledge, awareness, and willingness to use evidence-based care; 4) Prescriber uncertainties about the Veteran condition at the time of the clinical encounter; 5) Limited communication; and 6) Technology barriers of the electronic health record and patient portal. CONCLUSIONS: The diverse perspectives on prescribing underscore the need for interventions that recognize the detrimental impact of high workload on prescribing stewardship and the need to design interventions with the end-user in mind. This study revealed actionable themes that could be addressed to improve guideline concordant prescribing to enhance the quality of prescribing and to reduce patient harm.

Medication Safety Amid Technological Change: Usability Evaluation to Inform Inpatient Nurses' Electronic Health Record System Transition.

BACKGROUND: Electronic health record (EHR) system transitions are challenging for healthcare organizations. High-volume, safety-critical tasks like barcode medication administration (BCMA) should be evaluated, yet standards for ensuring safety during transition have not been established. OBJECTIVE: Identify risks in common and problem-prone medication tasks to inform safe transition between BCMA systems and establish benchmarks for future system changes. DESIGN: Staff nurses completed simulation-based usability testing in the legacy system (R1) and new system pre- (R2) and post-go-live (R3). Tasks included (1) Hold/Administer, (2) IV Fluids, (3) PRN Pain, (4) Insulin, (5) Downtime/PRN, and (6) Messaging. Audiovisual recordings of task performance were systematically analyzed for time, navigation, and errors. The System Usability Scale measured perceived usability and satisfaction. Post-simulation interviews captured nurses' qualitative comments and perceptions of the systems. PARTICIPANTS: Fifteen staff nurses completed 2-3-h simulation sessions. Eleven completed both R1 and R2, and seven completed all three rounds. Clinical experience ranged from novice (< 1 year) to experienced (> 10 years). Practice settings included adult and pediatric patient populations in ICU, stepdown, and acute care departments. MAIN MEASURES: Task completion rates/times, safety and non-safety-related use errors (interaction difficulties), and user satisfaction. KEY RESULTS: Overall success rates remained relatively stable in all tasks except two: IV Fluids task success increased substantially (R1: 17%, R2: 54%, R3: 100%) and Downtime/PRN task success decreased (R1: 92%, R2: 64%, R3: 22%). Among the seven nurses who completed all rounds, overall safety-related errors decreased 53% from R1 to R3 and 50% from R2 to R3, and average task times for successfully completed tasks decreased 22% from R1 to R3 and 38% from R2 to R3. CONCLUSIONS: Usability testing is a reasonable approach to compare different BCMA tasks to anticipate transition problems and establish benchmarks with which to monitor and evaluate system changes going forward.

Human-Centered Design to Address Biases in Artificial Intelligence.

The potential of artificial intelligence (AI) to reduce health care disparities and inequities is recognized, but it can also exacerbate these issues if not implemented in an equitable manner. This perspective identifies potential biases in each stage of the AI life cycle, including data collection, annotation, machine learning model development, evaluation, deployment, operationalization, monitoring, and feedback integration. To mitigate these biases, we suggest involving a diverse group of stakeholders, using human-centered AI principles. Human-centered AI can help ensure that AI systems are designed and used in a way that benefits patients and society, which can reduce health disparities and inequities. By recognizing and addressing biases at each stage of the AI life cycle, AI can achieve its potential in health care.

A Time and Motion Analysis of Nursing Workload and Electronic Health Record Use in the Emergency Department.

INTRODUCTION: The use of an electronic health record may create unanticipated consequences for emergency care delivery. We sought to describe emergency department nursing task distribution and the use of the electronic health record. METHODS: This was a prospective observational study of nurses in the emergency department using a time-and-motion methodology. Three trained research assistants conducted 1:1 observations between March and September 2019. Nurse tasks were classified into 6 established categories: electronic health record, direct/indirect patient care, communication, personal time, and other. Nurses' perceived workload was assessed using the National Aeronautics and Space Administration (NASA) Task Load Index. RESULTS: Twenty-three observations were conducted over 46 hours. Overall, nurses spent 27% of their time on electronic health record tasks, 25% on direct patient care, 17% on personal time, 15% on indirect patient care, and 6% on communication. During morning (7 am-12 pm) and afternoon shifts (12 pm-3 pm), the use of the health record was the most commonly performed task, whereas indirect patient care was the task most performed during evening shifts (3 pm-12 pm). Using the National Aeronautics and Space Administration (NASA) Task Load Index, nurses reported an increase in mental demand and effort during afternoon shifts compared with morning shifts. DISCUSSION: We observed that emergency nurses spent more time using the electronic health record as compared to other tasks. Increased usability of the electronic health record, particularly during high occupancy periods, may be a target for improvement.

Primary Care Clinicians' Beliefs and Strategies for Managing Chronic Pain in an Era of a National Opioid Epidemic.

BACKGROUND: Little is known about how primary care clinicians (PCCs) approach chronic pain management in the current climate of rapidly changing guidelines and the growing body of research about risks and benefits of opioid therapy. OBJECTIVE: To better understand PCCs' approaches to managing patients with chronic pain and explore implications for technological and administrative interventions. DESIGN: We conducted adapted critical decision method interviews with 20 PCCs. Each PCC participated in 1-5 interviews. PARTICIPANTS: PCCs interviewed had a mean of 14 years of experience. They were sampled from 13 different clinics in rural, suburban, and urban health settings across the state of Indiana. APPROACH: Interviews included discussion of participants' general approach to managing chronic pain, as well as in-depth discussion of specific patients with chronic pain. Interviews were audio recorded. Transcripts were analyzed thematically. KEY RESULTS: PCCs reflected on strategies they use to encourage and motivate patients. We identified four associated strategic themes: (1) developing trust, (2) eliciting information from the patient, (3) diverting attention from pain to function, and (4) articulating realistic goals for the patient. In discussion of chronic pain management, PCCs often explained their beliefs about opioid therapy. Three themes emerged: (1) Opioid use tends to reduce function, (2) Opioids are often not effective for long-term pain treatment, and (3) Response to pain and opioids is highly variable. CONCLUSIONS: PCC beliefs about opioid therapy generally align with the clinical evidence, but may have some important gaps. These findings suggest the potential value of interventions that include improved access to research findings; organizational changes to support PCCs in spending time with patients to develop rapport and trust, elicit information about pain, and manage patient expectations; and the need for innovative clinical cognitive support.

Health Information Technologies in the Support Systems of Pregnant Women and Their Caregivers: Mixed-Methods Study.

BACKGROUND: The quality and quantity of families' support systems during pregnancy can affect maternal and fetal outcomes. The support systems of expecting families can include many elements, such as family members, friends, and work or community groups. Emerging health information technologies (eg, social media, internet websites, and mobile apps) provide new resources for pregnant families to augment their support systems and to fill information gaps. OBJECTIVE: This study sought to determine the number and nature of the components of the support systems of pregnant women and their caregivers (eg, family members) and the role of health information technologies in these support systems. We examined the differences between pregnant women's support systems and those of their caregivers and the associations between support system composition and stress levels. METHODS: We enrolled pregnant women and caregivers from advanced maternal-fetal and group prenatal care clinics. Participants completed surveys assessing sociodemographic characteristics, health literacy, numeracy, and stress levels and were asked to draw a picture of their support system. Support system elements were extracted from drawings, categorized by type (ie, individual persons, groups, technologies, and other) and summarized for pregnant women and caregivers. Participant characteristics and support system elements were compared using the Pearson chi-square test for categorical variables and Wilcoxon ranked sum test for continuous variables. Associations between support system characteristics and stress levels were measured with Spearman correlation coefficient. RESULTS: The study enrolled 100 participants: 71 pregnant women and 29 caregivers. The support systems of pregnant women were significantly larger than those of caregivers-an average of 7.4 components for pregnant women and 5.4 components for caregivers (P=.003). For all participants, the most commonly reported support system elements were individual persons (408/680, 60.0%), followed by people groups (132/680, 19.4%), technologies (112/680, 16.5%), and other resources (28/680, 4.1%). Pregnant women's and caregivers' technology preferences within their support systems differed-pregnant women more often identified informational websites, apps, and social media as parts of their support systems, whereas caregivers more frequently reported general internet search engines. The size and components of these support systems were not associated with levels of stress. CONCLUSIONS: This study is one of the first demonstrating that technologies comprise a substantial portion of the support systems of pregnant women and their caregivers. Pregnant women more frequently reported specific medical information websites as part of their support system, whereas caregivers more often reported general internet search engines. Although social support is important for maternal and fetal health outcomes, no associations among stress, support system size, and support system components were found in this study. As health information technologies continue to evolve and their adoption increases, their role in patient and caregiver support systems and their effects should be further explored.

Facilitated Nurse Medication-Related Event Reporting to Improve Medication Management Quality and Safety in Intensive Care Units.

BACKGROUND: Medication safety presents an ongoing challenge for nurses working in complex, fast-paced, intensive care unit (ICU) environments. Studying ICU nurse's medication management-especially medication-related events (MREs)-provides an approach to analyze and improve medication safety and quality. OBJECTIVES: The goal of this study was to explore the utility of facilitated MRE reporting in identifying system deficiencies and the relationship between MREs and nurses' work in the ICUs. METHODS: We conducted 124 structured 4-hour observations of nurses in three different ICUs. Each observation included measurement of nurse's moment-to-moment activity and self-reports of workload and negative mood. The observer then obtained MRE reports from the nurse using a structured tool. The MREs were analyzed by three experts. RESULTS: MREs were reported in 35% of observations. The 60 total MREs included four medication errors and seven adverse drug events. Of the 49 remaining MREs, 65% were associated with negative patient impact. Task/process deficiencies were the most common contributory factor for MREs. MRE occurrence was correlated with increased total task volume. MREs also correlated with increased workload, especially during night shifts. DISCUSSION: Most of these MREs would not be captured by traditional event reporting systems. Facilitated MRE reporting provides a robust information source about potential breakdowns in medication management safety and opportunities for system improvement.

Paper or plastic? Simulation based evaluation of two versions of a cognitive aid for managing pediatric peri-operative critical events by anesthesia trainees: evaluation of the society for pediatric anesthesia emergency checklist.

Cognitive aids (CA), including emergency manuals and checklists, are tools designed to assist users in prioritizing and performing complex tasks during time sensitive, high stress situations (Marshall in Anesth Analgesia 117(5):1162-1171, 2013; Marshall and Mehra in Anaesthesia 69(7):669-677, 2014). The society for pediatric anesthesia (SPA) has developed a series of emergency checklists tailored for use by pediatric perioperative teams that cover a wide range of intraoperative critical events (Shaffner et al. in Anesth Analgesia 117(4):960-979, 2013). In this study, we evaluated user preferences for a CA (SPA checklist) using two different presentation formats, paper and electronic, during management of simulated critical events. Anesthesia trainees managed the simulated critical events under one of three randomized conditions: (1) memory alone, (2) with a paper version of the CA, (3) with an electronic version of the CA. Following participation in the simulated critical events, participants were asked to complete a survey regarding their experience using the different versions of the CA. The percentage of favorable responses for each format of the CA was compared using a mixed effects proportional odds model. There were 143 simulated events managed by 89 anesthesia trainees. Approximately one out of three trainees (electronic 29 %, paper 30 %) assigned to use the CA chose not to use it and completed the scenario from memory alone. The survey was completed by 68 % of participants, 58 % of trainees preferred the paper version and 35 % preferred the electronic version. All survey responses that reached statistical significance favored the paper version. In this study, anesthesia trainees had a favorable opinion of the content and perceived clinical relevance of both versions of the CA. In both quantitative and qualitative analysis, the paper version of the CA was preferred over the electronic version by participants. Despite overall favorable responses to the CA, a sizeable number of participants chose not to use either version the CA during the crisis.

Randomized Controlled Comparative Effectiveness Trial of Risk Model-Guided Clinical Decision Support for Suicide Screening.

Suicide prevention requires risk identification, appropriate intervention, and follow-up. Traditional risk identification relies on patient self-reporting, support network reporting, or face-to-face screening with validated instruments or history and physical exam. In the last decade, statistical risk models have been studied and more recently deployed to augment clinical judgment. Models have generally been found to be low precision or problematic at scale due to low incidence. Few have been tested in clinical practice, and none have been tested in clinical trials to our knowledge. Methods: We report the results of a pragmatic randomized controlled trial (RCT) in three outpatient adult Neurology clinic settings. This two-arm trial compared the effectiveness of Interruptive and Non-Interruptive Clinical Decision Support (CDS) to prompt further screening of suicidal ideation for those predicted to be high risk using a real-time, validated statistical risk model of suicide attempt risk, with the decision to screen as the primary end point. Secondary outcomes included rates of suicidal ideation and attempts in both arms. Manual chart review of every trial encounter was used to determine if suicide risk assessment was subsequently documented. Results: From August 16, 2022, through February 16, 2023, our study randomized 596 patient encounters across 561 patients for providers to receive either Interruptive or Non-Interruptive CDS in a 1:1 ratio. Adjusting for provider cluster effects, Interruptive CDS led to significantly higher numbers of decisions to screen (42%=121/289 encounters) compared to Non-Interruptive CDS (4%=12/307) (odds ratio=17.7, p-value <0.001). Secondarily, no documented episodes of suicidal ideation or attempts occurred in either arm. While the proportion of documented assessments among those noting the decision to screen was higher for providers in the Non-Interruptive arm (92%=11/12) than in the Interruptive arm (52%=63/121), the interruptive CDS was associated with more frequent documentation of suicide risk assessment (63/289 encounters compared to 11/307, p-value<0.001). Conclusions: In this pragmatic RCT of real-time predictive CDS to guide suicide risk assessment, Interruptive CDS led to higher numbers of decisions to screen and documented suicide risk assessments. Well-powered large-scale trials randomizing this type of CDS compared to standard of care are indicated to measure effectiveness in reducing suicidal self-harm. ClinicalTrials.gov Identifier: NCT05312437.

Methodology and bias in assessing compliance with a surgical safety checklist.

BACKGROUND: Surgical safety checklists, such as the perioperative time-out, have been shown to improve performance on a variety of patient safety measures. A variety of methods have been used to assess compliance with the perioperative time-out, but no standardized methodology with a reliable observer group currently exists. An observation-based methodology was used to assess time-out compliance at an academic medical center. METHODS: A single observer group made up of medical students and nurses recorded compliance with each of the 11 standardized items of the time-out. A total of 193 time-out procedures were observed, 48 by medical students and 145 by nurses. RESULTS: One item (procedure to be performed) achieved > 95% compliance. Three items (surgical site; availability of necessary blood products, implants, devices; and start of antibiotics) achieved 80%-95% compliance. Seven items achieved < 80% compliance (presence of required members of procedure team, presence of person who marked patient, patient identity, side marking, relevant images, allergies, and discussion of relevant special considerations). Compliance with the four core time-out items was 78.2%. Of the 11 items on the time-out being evaluated, there was a statistically significant difference between medical student and nursing observations for 10 items (p < .05). CONCLUSIONS: In our cohort of observed time-outs, the compliance rate was low, calling into question time-out quality, and, more importantly, patient safety. Measures must be taken by large hospitals to regularly audit time-out compliance and create effective programming to improve performance. Although observational assessment is an effective method to assess compliance with surgical safety checklists, observer group bias has the potential to skew results.

Inpatient nurses' preferences and decisions with risk information visualization.

OBJECTIVE: We examined the influence of 4 different risk information formats on inpatient nurses' preferences and decisions with an acute clinical deterioration decision-support system. MATERIALS AND METHODS: We conducted a comparative usability evaluation in which participants provided responses to multiple user interface options in a simulated setting. We collected qualitative data using think aloud methods. We collected quantitative data by asking participants which action they would perform after each time point in 3 different patient scenarios. RESULTS: More participants (n = 6) preferred the probability format over relative risk ratios (n = 2), absolute differences (n = 2), and number of persons out of 100 (n = 0). Participants liked average lines, having a trend graph to supplement the risk estimate, and consistent colors between trend graphs and possible actions. Participants did not like too much text information or the presence of confidence intervals. From a decision-making perspective, use of the probability format was associated with greater concordance in actions taken by participants compared to the other 3 risk information formats. DISCUSSION: By focusing on nurses' preferences and decisions with several risk information display formats and collecting both qualitative and quantitative data, we have provided meaningful insights for the design of clinical decision-support systems containing complex quantitative information. CONCLUSION: This study adds to our knowledge of presenting risk information to nurses within clinical decision-support systems. We encourage those developing risk-based systems for inpatient nurses to consider expressing risk in a probability format and include a graph (with average line) to display the patient's recent trends.

Understanding Patient and Clinician Reported Nonroutine Events in Ambulatory Surgery.

OBJECTIVE: Nonroutine events (NREs, i.e., deviations from optimal care) can identify care process deficiencies and safety risks. Nonroutine events reported by clinicians have been shown to identify systems failures, but this methodology fails to capture the patient perspective. The objective of this prospective observational study is to understand the incidence and nature of patient- and clinician-reported NREs in ambulatory surgery. METHODS: We interviewed patients about NREs that occurred during their perioperative care using a structured interview tool before discharge and in a 7-day follow-up call. Concurrently, we interviewed the clinicians caring for these patients immediately postoperatively to collect NREs. We trained 2 experienced clinicians and 2 patients to assess and code each reported NRE for type, theme, severity, and likelihood of reoccurrence (i.e., likelihood that the same event would occur for another patient). RESULTS: One hundred one of 145 ambulatory surgery cases (70%) contained at least one NRE. Overall, 214 NREs were reported-88 by patients and 126 by clinicians. Cases containing clinician-reported NREs were associated with increased patient body mass index ( P = 0.023) and lower postcase patient ratings of being treated with respect ( P = 0.032). Cases containing patient-reported NREs were associated with longer case duration ( P = 0.040), higher postcase clinician frustration ratings ( P < 0.001), higher ratings of patient stress ( P = 0.019), and lower patient ratings of their quality of life ( P = 0.010), of the quality of clinician teamwork ( P = 0.010), being treated with respect ( P = 0.003), and being listened to carefully ( P = 0.012). Trained patient raters evaluated NRE severity significantly higher than did clinician raters ( P < 0.001), while clinicians rated recurrence likelihood significantly higher than patients for both clinician ( P = 0.032) and patient-reported NREs ( P = 0.001). CONCLUSIONS: Both patients and clinicians readily report events during clinical care that they believe deviate from optimal care expectations. These 2 primary stakeholders in safe, high-quality surgical care have different experiences and perspectives regarding NREs. The combination of patient- and clinician-reported NREs seems to be a promising patient-centered method of identifying healthcare system deficiencies and opportunities for improvement.

Evaluation of inpatient medication guidance from an artificial intelligence chatbot.

PURPOSE: To analyze the clinical completeness, correctness, usefulness, and safety of chatbot and medication database responses to everyday inpatient medication-use questions. METHODS: We evaluated the responses from an artificial intelligence chatbot, a medication database, and clinical pharmacists to 200 real-world medication-use questions. Answer quality was rated by a blinded group of pharmacists, providers, and nurses. Chatbot and medication database responses were deemed "acceptable" if the mean reviewer rating was within 3 points of the mean rating for pharmacists' answers. We used descriptive statistics for reviewer ratings and Kendall's coefficient to evaluate interrater agreement. RESULTS: The medication database generated responses to 194 (97%) questions, with 88% considered acceptable for clinical correctness, 76% considered acceptable for completeness, 83% considered acceptable for safety, and 81% considered acceptable for usefulness compared to pharmacists' answers. The chatbot responded to only 160 (80%) questions, with 85% considered acceptable for clinical correctness, 65% considered acceptable for completeness, 71% considered acceptable for safety, and 68% considered acceptable for usefulness. CONCLUSION: Traditional search methods using a drug database provide more clinically correct, complete, safe, and useful answers than a chatbot. When the chatbot generated a response, the clinical correctness was similar to that of a drug database; however, it was not rated as favorably for clinical completeness, safety, or usefulness. Our results highlight the need for ongoing training and continued improvements to artificial intelligence chatbots for them to be incorporated reliably into the clinical workflow. With continued improvement in chatbot functionality, chatbots could be a useful pharmacist adjunct, providing healthcare providers with quick and reliable answers to medication-use questions.

Empowering patients to address diabetes care gaps: formative usability testing of a novel patient portal intervention.

Objective: The aim of this study was to design and assess the formative usability of a novel patient portal intervention designed to empower patients with diabetes to initiate orders for diabetes-related monitoring and preventive services. Materials and Methods: We used a user-centered Design Sprint methodology to create our intervention prototype and assess its usability with 3 rounds of iterative testing. Participants (5/round) were presented with the prototype and asked to perform common, standardized tasks using think-aloud procedures. A facilitator rated task performance using a scale: (1) completed with ease, (2) completed with difficulty, and (3) failed. Participants completed the System Usability Scale (SUS) scored 0-worst to 100-best. All testing occurred remotely via Zoom. Results: We identified 3 main categories of usability issues: distrust about the automated system, content concerns, and layout difficulties. Changes included improving clarity about the ordering process and simplifying language; however, design constraints inherent to the electronic health record system limited our ability to respond to all usability issues (eg, could not modify fixed elements in layout). Percent of tasks completed with ease across each round were 67%, 60%, and 80%, respectively. Average SUS scores were 87, 74, and 93, respectively. Across rounds, participants found the intervention valuable and appreciated the concept of patient-initiated ordering. Conclusions: Through iterative user-centered design and testing, we improved the usability of the patient portal intervention. A tool that empowers patients to initiate orders for disease-specific services as part of their existing patient portal account has potential to enhance the completion of recommended health services and improve clinical outcomes.

Adapting Cognitive Task Analysis Methods for Use in a Large Sample Simulation Study of High-Risk Healthcare Events.

Cognitive task analysis (CTA) methods are traditionally used to conduct small-sample, in-depth studies. In this case study, CTA methods were adapted for a large multi-site study in which 102 anesthesiologists worked through four different high-fidelity simulated high-consequence incidents. Cognitive interviews were used to elicit decision processes following each simulated incident. In this paper, we highlight three practical challenges that arose: (1) standardizing the interview techniques for use across a large, distributed team of diverse backgrounds; (2) developing effective training; and (3) developing a strategy to analyze the resulting large amount of qualitative data. We reflect on how we addressed these challenges by increasing standardization, developing focused training, overcoming social norms that hindered interview effectiveness, and conducting a staged analysis. We share findings from a preliminary analysis that provides early validation of the strategy employed. Analysis of a subset of 64 interview transcripts using a decompositional analysis approach suggests that interviewers successfully elicited descriptions of decision processes that varied due to the different challenges presented by the four simulated incidents. A holistic analysis of the same 64 transcripts revealed individual differences in how anesthesiologists interpreted and managed the same case.

Decision-Making During High-Risk Events: A Systematic Literature Review.

Effective decision-making in crisis events is challenging due to time pressure, uncertainty, and dynamic decisional environments. We conducted a systematic literature review in PubMed and PsycINFO, identifying 32 empiric research papers that examine how trained professionals make naturalistic decisions under pressure. We used structured qualitative analysis methods to extract key themes. The studies explored different aspects of decision-making across multiple domains. The majority (19) focused on healthcare; military, fire and rescue, oil installation, and aviation domains were also represented. We found appreciable variability in research focus, methodology, and decision-making descriptions. We identified five main themes: (1) decision-making strategy, (2) time pressure, (3) stress, (4) uncertainty, and (5) errors. Recognition-primed decision-making (RPD) strategies were reported in all studies that analyzed this aspect. Analytical strategies were also prominent, appearing more frequently in contexts with less time pressure and explicit training to generate multiple explanations. Practitioner experience, time pressure, stress, and uncertainty were major influencing factors. Professionals must adapt to the time available, types of uncertainty, and individual skills when making decisions in high-risk situations. Improved understanding of these decisional factors can inform evidence-based enhancements to training, technology, and process design.

Using intervention mapping to design and implement a multicomponent intervention to improve antibiotic and NSAID prescribing.

Successfully changing prescribing behavior to reduce inappropriate antibiotic and nonsteroidal anti-inflammatory drug (NSAID) prescriptions often requires combining components into a multicomponent intervention. However, multicomponent interventions often fail because of development and implementation complexity. To increase the likelihood of successfully changing prescribing behavior, we applied a systematic process to design and implement a multicomponent intervention. We used Intervention Mapping to create a roadmap for a multicomponent intervention in unscheduled outpatient care settings in the Veterans Health Administration. Intervention Mapping is a systematic process consisting of six steps that we grouped into three phases: (i) understand behavioral determinants and barriers to implementation, (ii) develop the intervention, and (iii) define evaluation plan and implementation strategies. A targeted literature review, combined with 25 prescriber and 25 stakeholder interviews, helped identify key behavioral determinants to inappropriate prescribing (e.g. perceived social pressure from patients to prescribe). We targeted three desired prescriber behaviors: (i) review guideline-concordant prescribing and patient outcomes, (ii) manage diagnostic and treatment uncertainty, and (iii) educate patients and caregivers. The intervention consisted of components for academic detailing, prescribing feedback, and alternative prescription order sets. Implementation strategies consisted of preparing clinical champions, conducting readiness assessments, and incentivizing use of the intervention. We chose a mixed-method study design with a commonly used evaluation framework to assess effectiveness and implementation outcomes in a subsequent trial. This study furthers knowledge about causes of inappropriate antibiotic and NSAID prescribing and demonstrates how theoretical, empirical, and practical information can be systematically applied to develop a multicomponent intervention to help address these causes.

Reducing adverse drug events from antibiotics and nonsteroidal anti-inflammatory drugs (NSAIDs) is a patient safety priority. Successfully changing prescribing behavior to reduce inappropriate prescriptions can require combining intervention components, each with different mechanisms for behavior change, into a multicomponent intervention. However, multicomponent interventions often fail because of development and implementation complexity. To increase the chance of successfully changing antibiotic and NSAID prescribing, the objective this study was to apply a systematic process to design and implement a multicomponent intervention. Three desired prescriber behaviors were targeted: (i) review guideline-concordant prescribing and patient outcomes, (ii) manage diagnostic and treatment uncertainty, and (iii) educate patients and caregivers. The designed intervention consisted of components for prescribing feedback, academic detailing, and alternative prescription order sets. Strategies to improve use of the intervention consisted of preparing clinical champions, conducting readiness assessments prior to study onset, and incentivizing use of the intervention. We chose a mixed-method study design with a commonly used evaluation framework to assess effectiveness and implementation outcomes of the multicomponent intervention in a subsequent trial.

Developing Ethics and Equity Principles, Terms, and Engagement Tools to Advance Health Equity and Researcher Diversity in AI and Machine Learning: Modified Delphi Approach.

BACKGROUND: Artificial intelligence (AI) and machine learning (ML) technology design and development continues to be rapid, despite major limitations in its current form as a practice and discipline to address all sociohumanitarian issues and complexities. From these limitations emerges an imperative to strengthen AI and ML literacy in underserved communities and build a more diverse AI and ML design and development workforce engaged in health research. OBJECTIVE: AI and ML has the potential to account for and assess a variety of factors that contribute to health and disease and to improve prevention, diagnosis, and therapy. Here, we describe recent activities within the Artificial Intelligence/Machine Learning Consortium to Advance Health Equity and Researcher Diversity (AIM-AHEAD) Ethics and Equity Workgroup (EEWG) that led to the development of deliverables that will help put ethics and fairness at the forefront of AI and ML applications to build equity in biomedical research, education, and health care. METHODS: The AIM-AHEAD EEWG was created in 2021 with 3 cochairs and 51 members in year 1 and 2 cochairs and ~40 members in year 2. Members in both years included AIM-AHEAD principal investigators, coinvestigators, leadership fellows, and research fellows. The EEWG used a modified Delphi approach using polling, ranking, and other exercises to facilitate discussions around tangible steps, key terms, and definitions needed to ensure that ethics and fairness are at the forefront of AI and ML applications to build equity in biomedical research, education, and health care. RESULTS: The EEWG developed a set of ethics and equity principles, a glossary, and an interview guide. The ethics and equity principles comprise 5 core principles, each with subparts, which articulate best practices for working with stakeholders from historically and presently underrepresented communities. The glossary contains 12 terms and definitions, with particular emphasis on optimal development, refinement, and implementation of AI and ML in health equity research. To accompany the glossary, the EEWG developed a concept relationship diagram that describes the logical flow of and relationship between the definitional concepts. Lastly, the interview guide provides questions that can be used or adapted to garner stakeholder and community perspectives on the principles and glossary. CONCLUSIONS: Ongoing engagement is needed around our principles and glossary to identify and predict potential limitations in their uses in AI and ML research settings, especially for institutions with limited resources. This requires time, careful consideration, and honest discussions around what classifies an engagement incentive as meaningful to support and sustain their full engagement. By slowing down to meet historically and presently underresourced institutions and communities where they are and where they are capable of engaging and competing, there is higher potential to achieve needed diversity, ethics, and equity in AI and ML implementation in health research.