Clinical Knowledge Model for the Prevention of Healthcare-Associated Venous Thromboembolism.

Knowledge models inform organizational behavior through the logical association of documentation processes, definitions, data elements, and value sets. The development of a well-designed knowledge model allows for the reuse of electronic health record data to promote efficiency in practice, data interoperability, and the extensibility of data to new capabilities or functionality such as clinical decision support, quality improvement, and research. The purpose of this article is to describe the development and validation of a knowledge model for healthcare-associated venous thromboembolism prevention. The team used FloMap, an Internet-based survey resource, to compare metadata from six healthcare organizations to an initial draft model. The team used consensus decision-making over time to compare survey results. The resulting model included seven panels, 41 questions, and 231 values. A second validation step included completion of an Internet-based survey with 26 staff nurse respondents representing 15 healthcare organizations, two electronic health record vendors, and one academic institution. The final knowledge model contained nine Logical Observation Identifiers Names and Codes panels, 32 concepts, and 195 values representing an additional six panels (groupings), 15 concepts (questions), and the specification of 195 values (answers). The final model is useful for consistent documentation to demonstrate the contribution of nursing practice to the prevention of venous thromboembolism.

Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare.

OBJECTIVE: The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution's approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. MATERIALS AND METHODS: Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. RESULTS: An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. DISCUSSION: By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. CONCLUSIONS: We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.

Information Models Offer Value to Standardize Electronic Health Record Flowsheet Data: A Fall Prevention Exemplar.

PURPOSE: The rapid implementation of electronic health records (EHRs) resulted in a lack of data standardization and created considerable difficulty for secondary use of EHR documentation data within and between organizations. While EHRs contain documentation data (input), nurses and healthcare organizations rarely have useable documentation data (output). The purpose of this article is to describe a method of standardizing EHR flowsheet documentation data using information models (IMs) to support exchange, quality improvement, and big data research. As an exemplar, EHR flowsheet metadata (input) from multiple organizations was used to validate a fall prevention IM. DESIGN: A consensus-based, qualitative, descriptive approach was used to identify a minimum set of essential fall prevention data concepts documented by staff nurses in acute care. The goal was to increase generalizable and comparable nurse-sensitive data on the prevention of falls across organizations for big data research. METHODS: The research team conducted a retrospective, observational study using an iterative, consensus-based approach to map, analyze, and evaluate nursing flowsheet metadata contributed by eight health systems. The team used FloMap software to aggregate flowsheet data across organizations for mapping and comparison of data to a reference IM. The FloMap analysis was refined with input from staff nurse subject matter experts, review of published evidence, current documentation standards, Magnet Recognition nursing standards, and informal fall prevention nursing use cases. FINDINGS: Flowsheet metadata analyzed from the EHR systems represented 6.6 million patients, 27 million encounters, and 683 million observations. Compared to the original reference IM, five new IM classes were added, concepts were reduced by 14 (from 57 to 43), and 157 value set items were added. The final fall prevention IM incorporated 11 condition or age-specific fall risk screening tools and a fall event details class with 14 concepts. CONCLUSION: The iterative, consensus-based refinement and validation of the fall prevention IM from actual EHR fall prevention flowsheet documentation contributes to the ability to semantically exchange and compare fall prevention data across multiple health systems and organizations. This method and approach provides a process for standardizing flowsheet data as coded data for information exchange and use in big data research. CLINICAL RELEVANCE: Opportunities exist to work with EHR vendors and the Office of the National Coordinator for Health Information Technology to implement standardized IMs within EHRs to expand interoperability of nurse-sensitive data.

A Closer Look at the "Right" Format for Clinical Decision Support: Methods for Evaluating a Storyboard BestPractice Advisory.

(1) Background: The five rights of clinical decision support (CDS) are a well-known framework for planning the nuances of CDS, but recent advancements have given us more options to modify the format of the alert. One-size-fits-all assessments fail to capture the nuance of different BestPractice Advisory (BPA) formats. To demonstrate a tailored evaluation methodology, we assessed a BPA after implementation of Storyboard for changes in alert fatigue, behavior influence, and task completion; (2) Methods: Data from 19 weeks before and after implementation were used to evaluate differences in each domain. Individual clinics were evaluated for task completion and compared for changes pre- and post-redesign; (3) Results: The change in format was correlated with an increase in alert fatigue, a decrease in erroneous free text answers, and worsened task completion at a system level. At a local level, however, 14% of clinics had improved task completion; (4) Conclusions: While the change in BPA format was correlated with decreased performance, the changes may have been driven primarily by the COVID-19 pandemic. The framework and metrics proposed can be used in future studies to assess the impact of new CDS formats. Although the changes in this study seemed undesirable in aggregate, some positive changes were observed at the level of individual clinics. Personalized implementations of CDS tools based on local need should be considered.

A refined methodology for validation of information models derived from flowsheet data and applied to a genitourinary case.

Use of electronic health record data is expanding to support quality improvement and research; however, this requires standardization of the data and validation within and across organizations. Information models (IMs) are created to standardize data elements into a logical organization that includes data elements, definitions, data types, values, and relationships. To be generalizable, these models need to be validated across organizations. The purpose of this case report is to describe a refined methodology for validation of flowsheet IMs and apply the revised process to a genitourinary IM created in one organization. The refined IM process, adding evidence and input from experts, produced a clinically relevant and evidence-based model of genitourinary care. The refined IM process provides a foundation for optimizing electronic health records with comparable nurse sensitive data that can add to common data models for continuity of care and ongoing use for quality improvement and research.

Evaluation of ML-Based Clinical Decision Support Tool to Replace an Existing Tool in an Academic Health System: Lessons Learned.

There is increasing application of machine learning tools to problems in healthcare, with an ultimate goal to improve patient safety and health outcomes. When applied appropriately, machine learning tools can augment clinical care provided to patients. However, even if a model has impressive performance characteristics, prospectively evaluating and effectively implementing models into clinical care remains difficult. The primary objective of this paper is to recount our experiences and challenges in comparing a novel machine learning-based clinical decision support tool to legacy, non-machine learning tools addressing potential safety events in the hospitals and to summarize the obstacles which prevented evaluation of clinical efficacy of tools prior to widespread institutional use. We collected and compared safety events data, specifically patient falls and pressure injuries, between the standard of care approach and machine learning (ML)-based clinical decision support (CDS). Our assessment was limited to performance of the model rather than the workflow due to challenges in directly comparing both approaches. We did note a modest improvement in falls with ML-based CDS; however, it was not possible to determine that overall improvement was due to model characteristics.