Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults.

BACKGROUND: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. OBJECTIVE: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. METHODS: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. RESULTS: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. CONCLUSIONS: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence-driven CDS. TRIAL REGISTRATION: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480.

Operationalizing a real-time scoring model to predict fall risk among older adults in the emergency department.

Predictive models are increasingly being developed and implemented to improve patient care across a variety of clinical scenarios. While a body of literature exists on the development of models using existing data, less focus has been placed on practical operationalization of these models for deployment in real-time production environments. This case-study describes challenges and barriers identified and overcome in such an operationalization for a model aimed at predicting risk of outpatient falls after Emergency Department (ED) visits among older adults. Based on our experience, we provide general principles for translating an EHR-based predictive model from research and reporting environments into real-time operation.

Governance of Clinical AI applications to facilitate safe and equitable deployment in a large health system: Key elements and early successes.

One of the key challenges in successful deployment and meaningful adoption of AI in healthcare is health system-level governance of AI applications. Such governance is critical not only for patient safety and accountability by a health system, but to foster clinician trust to improve adoption and facilitate meaningful health outcomes. In this case study, we describe the development of such a governance structure at University of Wisconsin Health (UWH) that provides oversight of AI applications from assessment of validity and user acceptability through safe deployment with continuous monitoring for effectiveness. Our structure leverages a multi-disciplinary steering committee along with project specific sub-committees. Members of the committee formulate a multi-stakeholder perspective spanning informatics, data science, clinical operations, ethics, and equity. Our structure includes guiding principles that provide tangible parameters for endorsement of both initial deployment and ongoing usage of AI applications. The committee is tasked with ensuring principles of interpretability, accuracy, and fairness across all applications. To operationalize these principles, we provide a value stream to apply the principles of AI governance at different stages of clinical implementation. This structure has enabled effective clinical adoption of AI applications. Effective governance has provided several outcomes: (1) a clear and institutional structure for oversight and endorsement; (2) a path towards successful deployment that encompasses technologic, clinical, and operational, considerations; (3) a process for ongoing monitoring to ensure the solution remains acceptable as clinical practice and disease prevalence evolve; (4) incorporation of guidelines for the ethical and equitable use of AI applications.

An Assessment of Climate Change Impacts on Los Angeles (California USA) Hospitals, Wildfires Highest Priority.

Introduction Although many studies have delineated the variety and magnitude of impacts that climate change is likely to have on health, very little is known about how well hospitals are poised to respond to these impacts. Hypothesis/Problem The hypothesis is that most modern hospitals in urban areas in the United States need to augment their current disaster planning to include climate-related impacts. METHODS: Using Los Angeles County (California USA) as a case study, historical data for emergency department (ED) visits and projections for extreme-heat events were used to determine how much climate change is likely to increase ED visits by mid-century for each hospital. In addition, historical data about the location of wildfires in Los Angeles County and projections for increased frequency of both wildfires and flooding related to sea-level rise were used to identify which area hospitals will have an increased risk of climate-related wildfires or flooding at mid-century. RESULTS: Only a small fraction of the total number of predicted ED visits at mid-century would likely to be due to climate change. By contrast, a significant portion of hospitals in Los Angeles County are in close proximity to very high fire hazard severity zones (VHFHSZs) and would be at greater risk to wildfire impacts as a result of climate change by mid-century. One hospital in Los Angeles County was anticipated to be at greater risk due to flooding by mid-century as a result of climate-related sea-level rise. CONCLUSION: This analysis suggests that several Los Angeles County hospitals should focus their climate-change-related planning on building resiliency to wildfires. Adelaine SA , Sato M , Jin Y , Godwin H . An assessment of climate change impacts on Los Angeles (California USA) hospitals, wildfires highest priority. Prehosp Disaster Med. 2017;32(5):556-562.

An Assessment of Collaboration and Disasters: A Hospital Perspective.

INTRODUCTION: There is no standard guidance for strategies for hospitals to use to coordinate with other agencies during a disaster. HYPOTHESIS/PROBLEM: This study analyzes successful strategies and barriers encountered by hospitals across the nation in coordinating and collaborating with other response agencies. METHODS: Quantitative and qualitative data were collected from a web-based study from 577 acute care hospitals sampled from the 2013 American Hospital Association (AHA) database. The results were analyzed using descriptive statistics. RESULTS: The most common barriers to collaboration are related to finances, ability to communicate, and personnel.