Design of patient-facing immunization visualizations affects task performance: an experimental comparison of 4 electronic visualizations.

OBJECTIVE: This study experimentally evaluated how well lay individuals could interpret and use 4 types of electronic health record (EHR) patient-facing immunization visualizations. MATERIALS AND METHODS: Participants (n = 69) completed the study using a secure online survey platform. Participants viewed the same immunization information in 1 of 4 EHR-based immunization visualizations: 2 different patient portals (Epic MyChart and eClinicWorks), a downloadable EHR record, and a clinic-generated electronic letter (eLetter). Participants completed a common task, created a standard vaccine schedule form, and answered questions about their perceived workload, subjective numeracy and health literacy, demographic variables, and familiarity with the task. RESULTS: The design of the immunization visualization significantly affected both task performance measures (time taken to complete the task and number of correct dates). In particular, those using Epic MyChart took significantly longer to complete the task than those using eLetter or eClinicWorks. Those using Epic MyChart entered fewer correct dates than those using the eLetter or eClinicWorks. There were no systematic statistically significant differences in task performance measures based on the numeracy, health literacy, demographic, and experience-related questions we asked. DISCUSSION: The 4 immunization visualizations had unique design elements that likely contributed to these performance differences. CONCLUSION: Based on our findings, we provide practical guidance for the design of immunization visualizations, and future studies. Future research should focus on understanding the contexts of use and design elements that make tables an effective type of health data visualization.

Longitudinal clinical decision support for assessing decisions over time: State-of-the-art and future directions.

Objective: Patients and clinicians rarely experience healthcare decisions as snapshots in time, but clinical decision support (CDS) systems often represent decisions as snapshots. This scoping review systematically maps challenges and facilitators to longitudinal CDS that are applied at two or more timepoints for the same decision made by the same patient or clinician. Methods: We searched Embase, PubMed, and Medline databases for articles describing development, validation, or implementation of patient- or clinician-facing longitudinal CDS. Validated quality assessment tools were used for article selection. Challenges and facilitators to longitudinal CDS are reported according to PRISMA-ScR guidelines. Results: Eight articles met inclusion criteria; each article described a unique CDS. None used entirely automated data entry, none used living guidelines for updating the evidence base or knowledge engine as new evidence emerged during the longitudinal study, and one included formal readiness for change assessments. Seven of eight CDS were implemented and evaluated prospectively. Challenges were primarily related to suboptimal study design (with unique challenges for each study) or user interface. Facilitators included use of randomized trial designs for prospective enrollment, increased CDS uptake during longitudinal exposure, and machine-learning applications that are tailored to the CDS use case. Conclusions: Despite the intuitive advantages of representing healthcare decisions longitudinally, peer-reviewed literature on longitudinal CDS is sparse. Existing reports suggest opportunities to incorporate longitudinal CDS frameworks, automated data entry, living guidelines, and user readiness assessments. Generating best practice guidelines for longitudinal CDS would require a greater depth and breadth of published work and expert opinion.

Development and Usability Evaluation of an Opioid Management App.

We describe the development and usability evaluation of a novel patient engagement tool (OPY) in its early stage from perspectives of both experts and end-users. The tool is aimed at engaging patients in positive behaviors surrounding the use, weaning, and disposal of opioid medications in the post-surgical setting. The messaging and design of the application were created through a behavioral economics lens. Expert-based heuristic analysis and user testing were conducted and demonstrated that while patients found the tool to be easy to use and subjectively somewhat useful, additional work to enhance the user interface and features is needed in close partnership with developers and stakeholders.

A Systematic Review of Nurses' Perceptions of Electronic Health Record Usability Based on the Human Factor Goals of Satisfaction, Performance, and Safety.

The poor usability of electronic health records contributes to increased nurses' workload, workarounds, and potential threats to patient safety. Understanding nurses' perceptions of electronic health record usability and incorporating human factors engineering principles are essential for improving electronic health records and aligning them with nursing workflows. This review aimed to synthesize studies focused on nurses' perceived electronic health record usability and categorize the findings in alignment with three human factor goals: satisfaction, performance, and safety. This systematic review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis. Five hundred forty-nine studies were identified from January 2009 to June 2023. Twenty-one studies were included in this review. The majority of the studies utilized reliable and validated questionnaires (n = 15) to capture the viewpoints of hospital-based nurses (n = 20). When categorizing usability-related findings according to the goals of good human factor design, namely, improving satisfaction, performance, and safety, studies used performance-related measures most. Only four studies measured safety-related aspects of electronic health record usability. Electronic health record redesign is necessary to improve nurses' perceptions of electronic health record usability, but future efforts should systematically address all three goals of good human factor design.

Buy or Build: Challenges Developing Consumer Digital Health Interventions.

BACKGROUND: Digital health interventions offer opportunities to improve collaborative care between clinicians and patients. Designing and implementing digital health interventions requires decisions about buying or building each technology-related component, all of which can lead to unanticipated issues. OBJECTIVES: This study aimed to describe issues encountered from our "buy or build" decisions developing two digital health interventions over different timeframes, designed to use patient-generated health data to: (1) improve hypertension control and (2) measure and improve adherence to HIV-related medications. METHODS: CONDUIT-HID (CONtrolling Disease Using Information Technology-Hypertension In Diabetes) was developed during 2010 to 2015 to allow patients receiving care from a multispecialty group practice to easily upload home blood pressure readings into their electronic health record and trigger clinician action if mean blood pressure values indicated inadequate control. USE-MI (Unobtrusive SEnsing of Medication Intake) was developed from 2016 to 2022 to allow entry of patients' HIV-related medication regimens, send reminders if patients had not taken their medications by the scheduled time(s), attempt to detect medication ingestion through machine learning analysis of smartwatch motion data, and present graphical adherence summaries to patients and clinicians. RESULTS: Both projects required multiple "buy or build" decisions across all system components, including data collection, transfer, analysis, and display. We used commercial, off-the-shelf technology where possible, but virtually all of these components still required substantial custom development. We found that, even though our projects spanned years, issues related to our "buy or build" decisions stemmed from several common themes, including mismatches between existing and new technologies, our use case being new or unanticipated, technology stability, technology longevity, and resource limitations. CONCLUSION: Those designing and implementing digital health interventions need to make numerous "buy or build" decisions as they create the technologies that underpin their intervention. These "buy or build" decisions, and the ensuing issues that will arise because of them, require careful planning, particularly if they represent an "edge case" use of existing commercial systems.

Clinical Judgment and Informatics: Encoding Simulations Using the Omaha System.

BACKGROUND: Clinical judgment is a critical nursing competency. Unfolding case study is a pedagogy used to develop clinical judgment. The Omaha System is an accepted taxonomy for standardizing nursing documentation. METHOD: An unfolding case study was developed from a simulation scenario by encoding 33 nursing interventions with the Omaha System, then developed multiple true-false response items which were sent electronically in survey format to prelicensure baccalaureate nursing students. Differences between identified essential and distractor interventions were evaluated. RESULTS: Participants (n = 101) identified correct interventions (M = 74.6%, standard deviation [SD] = 12%). A paired t-test indicated the percentage of correctly identified essential interventions (M = 78%, SD = 18.7%) was significantly higher than distractor interventions (M = 67%, SD = 18%). DISCUSSION: Nursing students can identify appropriate interventions using the Omaha System, demonstrating potential to extend highly effective and low-cost learning experiences using unfolding case study and multiple true-false response items. [J Nurs Educ. 2023;62(4):237-239.].

Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design.

BACKGROUND: Trauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems. METHODS: A user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues. RESULTS: Seventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems-transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods. CONCLUSION: We found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.

Free-Text Computerized Provider Order Entry Orders Used as Workaround for Communicating Medication Information.

OBJECTIVES: Medication information is frequently communicated via free-text computerized provider order entry (CPOE) orders in electronic health records. When such information is transmitted separately from a structured CPOE medication order, there is a significant risk of medication error. Although prior studies have described the frequency of using free-text CPOE orders for communicating medication information, there is a gap in understanding the nature of the medication information contained in the free-text CPOE orders. The aims of this study are to (1) identify the most common medication names communicated in free-text CPOE orders and their risk levels and (2) identify what actions physicians expect that nurses will complete when they place free-text CPOE orders, and (3) describe differences in these patterns across hospitals. METHODS: This study was a retrospective analysis of a sample of 26,524 free-text CPOE orders from 6 hospitals in the mid-Atlantic U.S. region. RESULTS: Free-text CPOE orders contained in the sample mentioned 193 medication names. Free-text CPOE orders were used frequently to communicate information about naloxone, heparin, flumazenil, and dextrose. Twenty-two percent of the free-text CPOE orders related to discontinuing medication(s), whereas 7% of the free-text CPOE orders relate to giving medication(s). There was high variation across hospitals both in the percentage of free-text CPOE orders mentioning medication information and in the proportion of those that referred to high-risk medications. CONCLUSIONS: The prevalence of medication information in free-text CPOE orders may suggest specific communication challenges in respect to urgency, uncertainty, planning, and other aspects of communication and clinical needs. Understanding and addressing communication challenges around commonly mentioned medication names and actions, especially those that are high risk, can help reduce the risk of medication errors.

Implementation and Analysis of an Institution-Wide EHR-Integrated Handoff Note.

EHR-Integrated Handoff Notes are becoming increasingly prevalent, especially among inpatient clinical service teams composed of physicians in training (resident physicians). We describe the implementation of such a tool at our institution, and then describe the usage of the tool, as well as changes made to its content, structure, and format, years after original implementation. We also describe frequency and temporal distribution of updates made to free text elements of the Handoff Note. At the initial implementation there were three versions available; medical/surgical, pediatric, and ICU. Years after implementation, 57% of clinical service teams continue to utilize the note, including all medical, pediatric, behavioral health, obstetrics/gynecology, and neurology services, as well as most surgical services, that serve as "primary" teams. Several interesting changes were noted to the content and structure of the Handoff Note, namely that more complicated versions were abandoned in favor of simpler versions. The Patient Summary and To Do free text boxes are updated an average of 1.0 and 1.6 times per day. Around 60% of updates to both free text boxes occur between 12 pm - 5:59 pm, likely in preparation for the daytime-nightime team handoff.

Usability Experience of a Personal Sleep Monitoring Device to Self-manage Sleep Among Persons 65 Years or Older With Self-reported Sleep Disturbances.

Increasingly, persons with self-reported health symptoms are using mobile health technologies to better understand, validate, and manage their symptoms. These off-the-shelf devices primarily utilize actigraphy to estimate sleep and activity. The purpose of this study was to describe qualitatively the experience of using a personal sleep monitoring device for sleep self-management in adults 65 years or older with self-reported sleep disturbances. This study followed a hybrid qualitative design using deductive and emergent coding derived from open-ended interviews (n = 25) after a period of 4 weeks using a wearable personal sleep monitoring device. Results expanded existing theoretical models on usability with the theme of personal meaning in the interaction between health and self-monitoring technology that were associated with age and technology use, privacy, and capability. Future studies for sleep health self-management and personally tailored interventions using personal sleep monitoring devices should continue to collect qualitative information in extending the understanding of user experience across different symptom clusters, such as sleep disturbances, that manifest more commonly in older age populations. This research is important for application in the use of mobile health technologies for nursing led health self-management interventions.

Human Factors and Organizational Issues in Health Informatics: Innovations and Opportunities.

OBJECTIVE: Human factors and ergonomics (HF/E) frameworks and methods are becoming embedded in the health informatics community. There is now broad recognition that health informatics tools must account for the diverse needs, characteristics, and abilities of end users, as well as their context of use. The objective of this review is to synthesize the current nature and scope of HF/E integration into the health informatics community. METHODS: Because the focus of this synthesis is on understanding the current integration of the HF/E and health informatics research communities, we manually reviewed all manuscripts published in primary HF/E and health informatics journals during 2020. RESULTS: HF/E-focused health informatics studies included in this synthesis focused heavily on EHR customizations, specifically clinical decision support customizations and customized data displays, and on mobile health innovations. While HF/E methods aimed to jointly improve end user safety, performance, and satisfaction, most HF/E-focused health informatics studies measured only end user satisfaction. CONCLUSION: HF/E-focused health informatics researchers need to identify and communicate methodological standards specific to health informatics, to better synthesize findings across resource intensive HF/E-focused health informatics studies. Important gaps in the HF/E design and evaluation process should be addressed in future work, including support for technology development platforms and training programs so that health informatics designers are as diverse as end users.

Clinician Perceptions on the Use of Free-Text Communication Orders.

OBJECTIVE: The aim of this study was to investigate (1) why ordering clinicians use free-text orders to communicate medication information; (2) what risks physicians and nurses perceive when free-text orders are used for communicating medication information; and (3) how electronic health records (EHRs) could be improved to encourage the safe communication of medication information. METHODS: We performed semi-structured, scenario-based interviews with eight physicians and eight nurses. Interview responses were analyzed and grouped into common themes. RESULTS: Participants described eight reasons why clinicians use free-text medication orders, five risks relating to the use of free-text medication orders, and five recommendations for improving EHR medication-related communication. Poor usability, including reduced efficiency and limited functionality associated with structured order entry, was the primary reason clinicians used free-text orders to communicate medication information. Common risks to using free-text orders for medication communication included the increased likelihood of missing orders and the increased workload on nurses responsible for executing orders. DISCUSSION: Clinicians' use of free-text orders is primarily due to limitations in the current structured order entry design. To encourage the safe communication of medication information between clinicians, the EHR's structured order entry must be redesigned to support clinicians' cognitive and workflow needs that are currently being addressed via the use of free-text orders. CONCLUSION: Clinicians' use of free-text orders as a workaround to insufficient structured order entry can create unintended patient safety risks. Thoughtful solutions designed to address these workarounds can improve the medication ordering process and the subsequent medication administration process.

Communication through the electronic health record: frequency and implications of free text orders.

Communication for non-medication order (CNMO) is a type of free text communication order providers use for asynchronous communication about patient care. The objective of this study was to understand the extent to which non-medication orders are being used for medication-related communication. We analyzed a sample of 26 524 CNMOs placed in 6 hospitals. A total of 42% of non-medication orders contained medication information. There was large variation in the usage of CNMOs across hospitals, provider settings, and provider types. The use of CNMOs for communicating medication-related information may result in delayed or missed medications, receiving medications that should have been discontinued, or important clinical decision being made based on inaccurate information. Future studies should quantify the implications of these data entry patterns on actual medication error rates and resultant safety issues.

The Use of Technology to Support Precision Health in Nursing Science.

PURPOSE: This article outlines how current nursing research can utilize technology to advance symptom and self-management science for precision health and provides a roadmap for the development and use of technologies designed for this purpose. APPROACH: At the 2018 annual conference of the National Institute of Nursing Research (NINR) Research Centers, nursing and interdisciplinary scientists discussed the use of technology to support precision health in nursing research projects and programs of study. Key themes derived from the presentations and discussion were summarized to create a proposed roadmap for advancement of technologies to support health and well-being. CONCLUSIONS: Technology to support precision health must be centered on the user and designed to be desirable, feasible, and viable. The proposed roadmap is composed of five iterative steps for the development, testing, and implementation of technology-based/enhanced self-management interventions. These steps are (a) contextual inquiry, focused on the relationships among humans, and the tools and equipment used in day-to-day life; (b) value specification, translating end-user values into end-user requirements; (c) design, verifying that the technology/device can be created and developing the prototype(s); (d) operationalization, testing the intervention in a real-world setting; and (e) summative evaluation, collecting and analyzing viability metrics, including process data, to evaluate whether the technology and the intervention have the desired effect. CLINICAL RELEVANCE: Interventions using technology are increasingly popular in precision health. Use of a standard multistep process for the development and testing of technology is essential.

Resident-Physician Preferences for Electronic Handoff Note Content: Implications for Implementation of a System-Wide Electronic Health Record-Integrated Handoff Tool.

Increasing attention is paid to the Handoff Process and EHR- integrated tools to enhance the Handoff Process and aid in creating Handoff Notes are becoming more prevalent. In this study, we attempted to determine the ideal content of the Handoff Note based on the preferences of the resident physicians for whom the tool was being constructed. This commenced with an initial semi-structured interview and culminated in a large-scale survey. Overall, 315 resident physicians completed the survey. Plan of Care, Illness Severity, and Patient Summary were the most important content elements to resident physicians. The importance and trustworthiness of other content elements, as well as their preferred display and level of granularity within the Handoff Note, varied considerably. Subjective assessment by a colleague of a patient's hospital course and plan of care, rather than any single piece of objective data, are preferred as ideal content for Handoff Note composition by resident physicians.

Electronic Progress Note Reading Patterns: An Eye Tracking Analysis.

This study used eye-tracking to understand how the order of note sections influences the way physicians read electronic progress notes. Participants (n = 7) wore an eye-tracking device while reviewing progress notes for four patient cases and then provided a verbal summary. We reviewed and analyzed verbal summaries and eye tracking recordings. Wide variation in reading behaviors existed. There was no relationship between time spent reading a section and section origin of verbal summaries.

Challenges and Opportunities to Improve the Clinician Experience Reviewing Electronic Progress Notes.

BACKGROUND: High-quality clinical notes are essential to effective clinical communication. However, electronic clinical notes are often long, difficult to review, and contain information that is potentially extraneous or out of date. Additionally, many clinicians write electronic clinical notes using customized templates, resulting in notes with significant variability in structure. There is a need to understand better how clinicians review electronic notes and how note structure variability may impact clinicians' note-reviewing experiences. OBJECTIVE: This article aims to understand how physicians review electronic clinical notes and what impact section order has on note-reviewing patterns. MATERIALS AND METHODS: We conducted an experiment utilizing an electronic health record (EHR) system prototype containing four anonymized patient cases, each composed of nine progress notes that were presented with note sections organized in different orders to different subjects (i.e., Subjective, Objective, Assessment, and Plan, Assessment, Plan, Subjective, and Objective, Subjective, Assessment, Objective, and Plan, and Mixed). Participants, who were mid-level residents and fellows, reviewed the cases and provided a brief summary after reviewing each case. Time-related data were collected and analyzed using descriptive statistics. Surveys were administered and interviews regarding experiences reviewing notes were collected and analyzed qualitatively. RESULTS: Qualitatively, participants reported challenges related to reviewing electronic clinical notes. Experimentally, time spent reviewing notes varied based on the note section organization. Consistency in note section organization improved performance (e.g., less scrolling and searching) compared with Mixed section organization when reviewing progress notes. DISCUSSION: Clinicians face significant challenges reviewing electronic clinical notes. Our findings support minimizing extraneous information in notes, removing information that can be found in other parts of the EHR, and standardizing the display and order of note sections to improve clinicians' note review experience. CONCLUSION: Our findings support the need to improve EHR note design and presentation to support optimal note review patterns for clinicians.

Designing a wrist-worn sensor to improve medication adherence: accommodating diverse user behaviors and technology preferences.

OBJECTIVES: High medication adherence is important for HIV suppression (antiretroviral therapy) and pre-exposure prophylaxis efficacy. We are developing sensor-based technologies to detect pill-taking gestures, trigger reminders, and generate adherence reports. MATERIALS AND METHODS: We collected interview, observation, and questionnaire data from individuals with and at-risk for HIV (N = 17). We assessed their medication-taking practices and physical actions, and feedback on our initial design. RESULTS: While participants displayed diverse medication taking practices and physical actions, most (67%) wanted to use the system to receive real-time and summative feedback, and most (69%) wanted to share data with their physicians. Participants preferred reminders via the wrist-worn device or mobile app, and summative feedback via mobile app or email. DISCUSSION: Adoption of these systems is promising if designs accommodate diverse behaviors and preferences. CONCLUSION: Our findings may help improve the accuracy and adoption of the system by accounting for user behaviors, physical actions, and preferences.

Predictors of Art and PrEP Adherence and Medication-Taking Practices and Preferences to Inform Development of a Wrist-Worn Adherence System.

We examined adherence, medication-taking practices, and preferences to inform development of a wrist-worn adherence system. Two convenience samples of persons taking antiretroviral therapy and HIV pre-exposure prophylaxis completed a survey. Additional online questions asked about willingness to use a wrist-worn device and reminder and feedback preferences. Among 225 participants, 13% reported adherence < 90%; this was associated with younger age and clinic sample. Compared to pill bottle-using participants, mediset users less commonly reported adherence < 90% (aOR = 0.16, p = .02), and blister pack users (aOR = 6.3, p = .02) and pill roll users (aOR = 3.3, p = .04) more commonly reported adherence < 90%. Sixty-two percent of the online participants reporting adherence (< 100%) had some interest in receiving adherence reminders, including 42% with interest in receiving reminders by smartwatch notifications. Although confounders are likely, formative work identified potential users and interest in using a wrist-worn adherence system. Future work will determine its acceptability and efficacy.