Clinical Knowledge Model for the Prevention of Healthcare-Associated Venous Thromboembolism.

Knowledge models inform organizational behavior through the logical association of documentation processes, definitions, data elements, and value sets. The development of a well-designed knowledge model allows for the reuse of electronic health record data to promote efficiency in practice, data interoperability, and the extensibility of data to new capabilities or functionality such as clinical decision support, quality improvement, and research. The purpose of this article is to describe the development and validation of a knowledge model for healthcare-associated venous thromboembolism prevention. The team used FloMap, an Internet-based survey resource, to compare metadata from six healthcare organizations to an initial draft model. The team used consensus decision-making over time to compare survey results. The resulting model included seven panels, 41 questions, and 231 values. A second validation step included completion of an Internet-based survey with 26 staff nurse respondents representing 15 healthcare organizations, two electronic health record vendors, and one academic institution. The final knowledge model contained nine Logical Observation Identifiers Names and Codes panels, 32 concepts, and 195 values representing an additional six panels (groupings), 15 concepts (questions), and the specification of 195 values (answers). The final model is useful for consistent documentation to demonstrate the contribution of nursing practice to the prevention of venous thromboembolism.

Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare.

OBJECTIVE: The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution's approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. MATERIALS AND METHODS: Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. RESULTS: An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. DISCUSSION: By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. CONCLUSIONS: We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.

Implementing Best Practices to Redesign Workflow and Optimize Nursing Documentation in the Electronic Health Record.

BACKGROUND: Documentation burden associated with electronic health records (EHR) is well documented in the literature. Usability and functionality of the EHR are considered fragmented and disorganized making it difficult to synthesize clinical information. Few best practices are reported in the literature to support streamlining the configuration of documentation fields to align clinical workflow with EHR data entry elements. OBJECTIVE: The primary objective was to improve performance, reduce duplication, and remove nonvalue-added tasks by redesigning the patient assessment template in the EHR using best practice approaches. METHODS: A quality improvement approach and pre-/postdesign was used to implement and evaluate best approaches to redesign standardized flowsheet documentation workflow. We implemented standards for usability modifications targeting efficiency, reducing redundancy, and improving workflow navigation. The assessment type row was removed; a reassessment section was added to the first three flowsheet rows and documentation practices were revised to document changes from the initial assessment by selecting the corresponding body system from the dropdown menu. Vendor-supplied timestamp data were used to evaluate documentation times. Video motion-time recording was used to capture click and scroll burden, defined as steps in documentation, and was analyzed using the Keystrok Level Model. RESULTS: This study's results included an 18.5% decreased time in the EHR; decrease of 7 to 12% of total time in flowsheets; time savings of 1.5 to 6.5 minutes per reassessment per patient; and a decrease of 88 to 97% in number of steps to perform reassessment documentation. CONCLUSION: Workflow redesign to improve the usability and functionality decreased documentation time, redundancy, and click burden resulting in improved productivity. The time savings correlate to several hours per 12-hour shift which could be reallocated to value-added patient care activities. Revising documentation practices in alignment with redesign benefits staff by decreasing workload, improving quality, and satisfaction.

A framework for the oversight and local deployment of safe and high-quality prediction models.

Artificial intelligence/machine learning models are being rapidly developed and used in clinical practice. However, many models are deployed without a clear understanding of clinical or operational impact and frequently lack monitoring plans that can detect potential safety signals. There is a lack of consensus in establishing governance to deploy, pilot, and monitor algorithms within operational healthcare delivery workflows. Here, we describe a governance framework that combines current regulatory best practices and lifecycle management of predictive models being used for clinical care. Since January 2021, we have successfully added models to our governance portfolio and are currently managing 52 models.

Information Models Offer Value to Standardize Electronic Health Record Flowsheet Data: A Fall Prevention Exemplar.

PURPOSE: The rapid implementation of electronic health records (EHRs) resulted in a lack of data standardization and created considerable difficulty for secondary use of EHR documentation data within and between organizations. While EHRs contain documentation data (input), nurses and healthcare organizations rarely have useable documentation data (output). The purpose of this article is to describe a method of standardizing EHR flowsheet documentation data using information models (IMs) to support exchange, quality improvement, and big data research. As an exemplar, EHR flowsheet metadata (input) from multiple organizations was used to validate a fall prevention IM. DESIGN: A consensus-based, qualitative, descriptive approach was used to identify a minimum set of essential fall prevention data concepts documented by staff nurses in acute care. The goal was to increase generalizable and comparable nurse-sensitive data on the prevention of falls across organizations for big data research. METHODS: The research team conducted a retrospective, observational study using an iterative, consensus-based approach to map, analyze, and evaluate nursing flowsheet metadata contributed by eight health systems. The team used FloMap software to aggregate flowsheet data across organizations for mapping and comparison of data to a reference IM. The FloMap analysis was refined with input from staff nurse subject matter experts, review of published evidence, current documentation standards, Magnet Recognition nursing standards, and informal fall prevention nursing use cases. FINDINGS: Flowsheet metadata analyzed from the EHR systems represented 6.6 million patients, 27 million encounters, and 683 million observations. Compared to the original reference IM, five new IM classes were added, concepts were reduced by 14 (from 57 to 43), and 157 value set items were added. The final fall prevention IM incorporated 11 condition or age-specific fall risk screening tools and a fall event details class with 14 concepts. CONCLUSION: The iterative, consensus-based refinement and validation of the fall prevention IM from actual EHR fall prevention flowsheet documentation contributes to the ability to semantically exchange and compare fall prevention data across multiple health systems and organizations. This method and approach provides a process for standardizing flowsheet data as coded data for information exchange and use in big data research. CLINICAL RELEVANCE: Opportunities exist to work with EHR vendors and the Office of the National Coordinator for Health Information Technology to implement standardized IMs within EHRs to expand interoperability of nurse-sensitive data.

Evaluation of ML-Based Clinical Decision Support Tool to Replace an Existing Tool in an Academic Health System: Lessons Learned.

There is increasing application of machine learning tools to problems in healthcare, with an ultimate goal to improve patient safety and health outcomes. When applied appropriately, machine learning tools can augment clinical care provided to patients. However, even if a model has impressive performance characteristics, prospectively evaluating and effectively implementing models into clinical care remains difficult. The primary objective of this paper is to recount our experiences and challenges in comparing a novel machine learning-based clinical decision support tool to legacy, non-machine learning tools addressing potential safety events in the hospitals and to summarize the obstacles which prevented evaluation of clinical efficacy of tools prior to widespread institutional use. We collected and compared safety events data, specifically patient falls and pressure injuries, between the standard of care approach and machine learning (ML)-based clinical decision support (CDS). Our assessment was limited to performance of the model rather than the workflow due to challenges in directly comparing both approaches. We did note a modest improvement in falls with ML-based CDS; however, it was not possible to determine that overall improvement was due to model characteristics.

A refined methodology for validation of information models derived from flowsheet data and applied to a genitourinary case.

Use of electronic health record data is expanding to support quality improvement and research; however, this requires standardization of the data and validation within and across organizations. Information models (IMs) are created to standardize data elements into a logical organization that includes data elements, definitions, data types, values, and relationships. To be generalizable, these models need to be validated across organizations. The purpose of this case report is to describe a refined methodology for validation of flowsheet IMs and apply the revised process to a genitourinary IM created in one organization. The refined IM process, adding evidence and input from experts, produced a clinically relevant and evidence-based model of genitourinary care. The refined IM process provides a foundation for optimizing electronic health records with comparable nurse sensitive data that can add to common data models for continuity of care and ongoing use for quality improvement and research.

A Closer Look at the "Right" Format for Clinical Decision Support: Methods for Evaluating a Storyboard BestPractice Advisory.

(1) Background: The five rights of clinical decision support (CDS) are a well-known framework for planning the nuances of CDS, but recent advancements have given us more options to modify the format of the alert. One-size-fits-all assessments fail to capture the nuance of different BestPractice Advisory (BPA) formats. To demonstrate a tailored evaluation methodology, we assessed a BPA after implementation of Storyboard for changes in alert fatigue, behavior influence, and task completion; (2) Methods: Data from 19 weeks before and after implementation were used to evaluate differences in each domain. Individual clinics were evaluated for task completion and compared for changes pre- and post-redesign; (3) Results: The change in format was correlated with an increase in alert fatigue, a decrease in erroneous free text answers, and worsened task completion at a system level. At a local level, however, 14% of clinics had improved task completion; (4) Conclusions: While the change in BPA format was correlated with decreased performance, the changes may have been driven primarily by the COVID-19 pandemic. The framework and metrics proposed can be used in future studies to assess the impact of new CDS formats. Although the changes in this study seemed undesirable in aggregate, some positive changes were observed at the level of individual clinics. Personalized implementations of CDS tools based on local need should be considered.

Academic-Practice Partnership for Caregiver Training and Support: The Duke Elder Family/Caregiver Training (DEFT) Center.

BACKGROUND After a hospital stay, many older adults rely on their caregivers for assistance at home. Empirical evidence demonstrates that caregiver support programs in hospital-to-home transitions are associated with favorable caregiver and patient outcomes. We tested the feasibility of implementing the Duke Elder Family/Caregiver Training (DEFT) program in an academic medical center.METHODS: We recruited adult caregivers of homebound patients who were aged 55 years or older from Duke University Hospital in Durham, North Carolina. Caregivers attended a face-to-face caregiver training and received two telephone checks after hospital discharge with DEFT services ending at 14 days of hospital discharge. We used a one-item survey to measure overall DEFT satisfaction. We also monitored 30-day readmissions of patients whose caregivers completed the DEFT program.RESULTS: The DEFT Center received 104 consult orders in six months. Of these, 61 agreed to participate but nine caregivers were unable to schedule the DEFT training and three decided to eventually withdraw from participation. Forty-nine caregivers received the DEFT training, 12 of whom were ineligible to continue because of change in patients' disposition plan. Of the remaining 37 caregivers, 15 completed the full program and reported high satisfaction; one patient was readmitted within 30 days of discharge.LIMITATIONS: The DEFT implementation was based on academic-medical partnership and relied on electronic medical records for consult and documentation. Replicability and generalizability of findings are limited to settings with similar capabilities and resources.CONCLUSION: The implementation of a caregiver training and support program in an academic medical center was feasible and was associated with favorable preliminary outcomes.

Validation and Refinement of a Pain Information Model from EHR Flowsheet Data.

BACKGROUND: Secondary use of electronic health record (EHR) data can reduce costs of research and quality reporting. However, EHR data must be consistent within and across organizations. Flowsheet data provide a rich source of interprofessional data and represents a high volume of documentation; however, content is not standardized. Health care organizations design and implement customized content for different care areas creating duplicative data that is noncomparable. In a prior study, 10 information models (IMs) were derived from an EHR that included 2.4 million patients. There was a need to evaluate the generalizability of the models across organizations. The pain IM was selected for evaluation and refinement because pain is a commonly occurring problem associated with high costs for pain management. OBJECTIVE: The purpose of our study was to validate and further refine a pain IM from EHR flowsheet data that standardizes pain concepts, definitions, and associated value sets for assessments, goals, interventions, and outcomes. METHODS: A retrospective observational study was conducted using an iterative consensus-based approach to map, analyze, and evaluate data from 10 organizations. RESULTS: The aggregated metadata from the EHRs of 8 large health care organizations and the design build in 2 additional organizations represented flowsheet data from 6.6 million patients, 27 million encounters, and 683 million observations. The final pain IM has 30 concepts, 4 panels (classes), and 396 value set items. Results are built on Logical Observation Identifiers Names and Codes (LOINC) pain assessment terms and extend the need for additional terms to support interoperability. CONCLUSION: The resulting pain IM is a consensus model based on actual EHR documentation in the participating health systems. The IM captures the most important concepts related to pain.