Advanced Care Planning Content Encoding with Natural Language Processing.

While advanced care planning (ACP) is an essential practice for ensuring patient-centered care, its adoption remains poor and the completeness of its documentation variable. Natural language processing (NLP) approaches hold promise for supporting ACP, including its use for decision support to improve ACP gaps at the point of care. ACP themes were annotated on palliative care notes across four annotators (Fleiss kappa = 0.753) and supervised models trained (Huggingface models bert-base-uncased and Bio_ClinicalBERT) using 5-fold cross validation (F1=0.8, precision=0.75, recall=0.86, any theme). When applied across the full note corpus of 12,711 notes, we observed variability in documentation of ACP information. Our findings demonstrate the promise of NLP approaches for informatics-based approaches for ACP and patient-centered care.

Building to learn: Information technology innovations to enable rapid pragmatic evaluation in a learning health system.

Background: Learning health systems (LHSs) iteratively generate evidence that can be implemented into practice to improve care and produce generalizable knowledge. Pragmatic clinical trials fit well within LHSs as they combine real-world data and experiences with a degree of methodological rigor which supports generalizability. Objectives: We established a pragmatic clinical trial unit ("RapidEval") to support the development of an LHS. To further advance the field of LHS, we sought to further characterize the role of health information technology (HIT), including innovative solutions and challenges that occur, to improve LHS project delivery. Methods: During the period from December 2021 to February 2023, eight projects were selected out of 51 applications to the RapidEval program, of which five were implemented, one is currently in pilot testing, and two are in planning. We evaluated pre-study planning, implementation, analysis, and study closure approaches across all RapidEval initiatives to summarize approaches across studies and identify key innovations and learnings by gathering data from study investigators, quality staff, and IT staff, as well as RapidEval staff and leadership. Implementation Results: Implementation approaches spanned a range of HIT capabilities including interruptive alerts, clinical decision support integrated into order systems, patient navigators, embedded micro-education, targeted outpatient hand-off documentation, and patient communication. Study approaches include pre-post with time-concordant controls (1), randomized stepped-wedge (1), cluster randomized across providers (1) and location (3), and simple patient level randomization (2). Conclusions: Study selection, design, deployment, data collection, and analysis required close collaboration between data analysts, informaticists, and the RapidEval team.

Health Information Technology Use among Chronic Disease Patients: An Analysis of the United States Health Information National Trends Survey.

BACKGROUND: Chronic disease is the leading cause of mortality in the United States. Health information technology (HIT) tools show promise for improving disease management. OBJECTIVES: This study aims to understand the following: (1) how self-perceptions of health compare between those with and without disease; (2) how HIT usage varies between chronic disease profiles (diabetes, hypertension, cardiovascular disease, pulmonary disease, depression, cancer, and comorbidities); (3) how HIT trends have changed in the past 6 years; and (4) the likelihood that a given chronic disease patient uses specific HIT tools. METHODS: The Health Information National Trends Survey (HINTS) inclusive of 2014 to 2020 served as the primary data source with statistical analysis completed using Stata. Bivariate analyses and two-tailed t-tests were conducted to compare self-perceived health and HIT usage to chronic disease. Logistic regression models were created to examine the odds of a specific patient using various forms of HIT, controlling for demographics and comorbidities. RESULTS: Logistic regression models controlling for sociodemographic factors and comorbidities showed that pulmonary disease, depression, and cancer patients had an increased likelihood of using HIT tools, for example, depression patients had an 81.1% increased likelihood of looking up health information (p < 0.0001). In contrast, diabetic, high blood pressure, and cardiovascular disease patients appeared to use HIT tools at similar rates to patients without chronic disease. Overall HIT usage has increased during the timeframe examined. CONCLUSION: This study demonstrates that certain chronic disease cohorts appear to have greater HIT usage than others. Further analysis should be done to understand what factors influence patients to utilize HIT which may provide additional insights into improving design and user experience for other populations with the goal of improving management of disease. Such analyses could also establish a new baseline to account for differences in HIT usage as a direct consequence of the novel coronavirus disease 2019 (COVID-19) pandemic.

Rapid implementation of a COVID-19 remote patient monitoring program.

OBJECTIVE: The study sought to evaluate early lessons from a remote patient monitoring engagement and education technology solution for patients with coronavirus disease 2019 (COVID-19) symptoms. MATERIALS AND METHODS: A COVID-19-specific remote patient monitoring solution (GetWell Loop) was offered to patients with COVID-19 symptoms. The program engaged patients and provided educational materials and the opportunity to share concerns. Alerts were resolved through a virtual care workforce of providers and medical students. RESULTS: Between March 18 and April 20, 2020, 2255 of 3701 (60.93%) patients with COVID-19 symptoms enrolled, resulting in over 2303 alerts, 4613 messages, 13 hospital admissions, and 91 emergency room visits. A satisfaction survey was given to 300 patient respondents, 74% of whom would be extremely likely to recommend their doctor. DISCUSSION: This program provided a safe and satisfying experience for patients while minimizing COVID-19 exposure and in-person healthcare utilization. CONCLUSIONS: Remote patient monitoring appears to be an effective approach for managing COVID-19 symptoms at home.

Natural language processing of prehospital emergency medical services trauma records allows for automated characterization of treatment appropriateness.

BACKGROUND: Incomplete prehospital trauma care is a significant contributor to preventable deaths. Current databases lack timelines easily constructible of clinical events. Temporal associations and procedural indications are critical to characterize treatment appropriateness. Natural language processing (NLP) methods present a novel approach to bridge this gap. We sought to evaluate the efficacy of a novel and automated NLP pipeline to determine treatment appropriateness from a sample of prehospital EMS motor vehicle crash records. METHODS: A total of 142 records were used to extract airway procedures, intraosseous/intravenous access, packed red blood cell transfusion, crystalloid bolus, chest compression system, tranexamic acid bolus, and needle decompression. Reports were processed using four clinical NLP systems and augmented via a word2phrase method leveraging a large integrated health system clinical note repository to identify terms semantically similar with treatment indications. Indications were matched with treatments and categorized as indicated, missed (indicated but not performed), or nonindicated. Automated results were then compared with manual review, and precision and recall were calculated for each treatment determination. RESULTS: Natural language processing identified 184 treatments. Automated timeline summarization was completed for all patients. Treatments were characterized as indicated in a subset of cases including the following: 69% (18 of 26 patients) for airway, 54.5% (6 of 11 patients) for intraosseous access, 11.1% (1 of 9 patients) for needle decompression, 55.6% (10 of 18 patients) for tranexamic acid, 60% (9 of 15 patients) for packed red blood cell, 12.9% (4 of 31 patients) for crystalloid bolus, and 60% (3 of 5 patients) for chest compression system. The most commonly nonindicated treatment was crystalloid bolus (22 of 142 patients). Overall, the automated NLP system performed with high precision and recall with over 70% of comparisons achieving precision and recall of greater than 80%. CONCLUSION: Natural language processing methodologies show promise for enabling automated extraction of procedural indication data and timeline summarization. Future directions should focus on optimizing and expanding these techniques to scale and facilitate broader trauma care performance monitoring. LEVEL OF EVIDENCE: Diagnostic tests or criteria, level III.

Predictors of elderly mortality after trauma: A novel outcome score.

INTRODUCTION: Elderly trauma patients are at high risk for mortality, even when presenting with minor injuries. Previous prognostic models are poorly used because of their reliance on elements unavailable during the index hospitalization. The purpose of this study was to develop a predictive algorithm to accurately estimate in-hospital mortality using easily available metrics. METHODS: The National Trauma Databank was used to identify patients 65 years and older. Data were split into derivation (2007-2013) and validation (2014-2015) data sets. There was no overlap between data sets. Factors included age, comorbidities, physiologic parameters, and injury types. A two-tiered scoring system to predict in-hospital mortality was developed: a quick elderly mortality after trauma (qEMAT) score for use at initial patient presentation and a full EMAT (fEMAT) score for use after radiologic evaluation. The final model (stepwise forward selection, p < 0.05) was chosen based on calibration and discrimination analysis. Calibration (Brier score) and discrimination (area under the receiving operating characteristic curve [AuROC]) were evaluated. Because National Trauma Databank did not include blood product transfusion, an element of the Geriatric Trauma Outcome Score (GTOS), a regional trauma registry was used to compare qEMAT versus GTOS. A mobile-based application is currently available for cost-free utilization. RESULTS: A total of 840,294 patients were included in the derivation data set and 427,358 patients in the validation data set. The fEMAT score (median, 91; S.D., 82-102) included 26 factors, and the qEMAT score included eight factors. The AuROC was 0.86 for fEMAT (Brier, 0.04) and 0.84 for qEMAT. The fEMAT outperformed other trauma mortality prediction models (e.g., Trauma and Injury Severity Score-Penetrating and Trauma and Injury Severity Score-Blunt, age + Injury Severity Score). The qEMAT outperformed the GTOS (AuROC, 0.87 vs. 0.83). CONCLUSION: The qEMAT and fEMAT accurately estimate the probability of in-hospital mortality and can be easily calculated on admission. This information could aid in deciding transfer to tertiary referral center, patient/family counseling, and palliative care utilization. LEVEL OF EVIDENCE: Epidemiological Study, level IV.

Using word embeddings to expand terminology of dietary supplements on clinical notes.

OBJECTIVE: The objective of this study is to demonstrate the feasibility of applying word embeddings to expand the terminology of dietary supplements (DS) using over 26 million clinical notes. METHODS: Word embedding models (ie, word2vec and GloVe) trained on clinical notes were used to predefine a list of top 40 semantically related terms for each of 14 commonly used DS. Each list was further evaluated by experts to generate semantically similar terms. We investigated the effect of corpus size and other settings (ie, vector size and window size) as well as the 2 word embedding models on performance for DS term expansion. We compared the number of clinical notes (and patients they represent) that were retrieved using the word embedding expanded terms to both the baseline terms and external DS sources exandped terms. RESULTS: Using the word embedding models trained on clinical notes, we could identify 1-12 semantically similar terms for each DS. Using the word embedding exandped terms, we were able to retrieve averagely 8.39% more clinical notes and 11.68% more patients for each DS compared with 2 sets of terms. The increasing corpus size results in more misspellings, but not more semantic variants brand names. Word2vec model is also found more capable of detecting semantically similar terms than GloVe. CONCLUSION: Our study demonstrates the utility of word embeddings on clinical notes for terminology expansion on 14 DS. We propose that this method can be potentially applied to create a DS vocabulary for downstream applications, such as information extraction.

Electronic Progress Note Reading Patterns: An Eye Tracking Analysis.

This study used eye-tracking to understand how the order of note sections influences the way physicians read electronic progress notes. Participants (n = 7) wore an eye-tracking device while reviewing progress notes for four patient cases and then provided a verbal summary. We reviewed and analyzed verbal summaries and eye tracking recordings. Wide variation in reading behaviors existed. There was no relationship between time spent reading a section and section origin of verbal summaries.

Recurrent Deep Network Models for Clinical NLP Tasks: Use Case with Sentence Boundary Disambiguation.

Although a number of foundational natural language processing (NLP) tasks like text segmentation are considered a simple problem in the general English domain dominated by well-formed text, complexities of clinical documentation lead to poor performance of existing solutions designed for the general English domain. We present an alternative solution that relies on a convolutional neural network layer followed by a bidirectional long short-term memory layer (CNN-Bi-LSTM) for the task of sentence boundary disambiguation and describe an ensemble approach for domain adaptation using two training corpora. Implementations using the Keras neural-networks API are available at https://github.com/NLPIE/clinical-sentences.

Challenges and Opportunities to Improve the Clinician Experience Reviewing Electronic Progress Notes.

BACKGROUND: High-quality clinical notes are essential to effective clinical communication. However, electronic clinical notes are often long, difficult to review, and contain information that is potentially extraneous or out of date. Additionally, many clinicians write electronic clinical notes using customized templates, resulting in notes with significant variability in structure. There is a need to understand better how clinicians review electronic notes and how note structure variability may impact clinicians' note-reviewing experiences. OBJECTIVE: This article aims to understand how physicians review electronic clinical notes and what impact section order has on note-reviewing patterns. MATERIALS AND METHODS: We conducted an experiment utilizing an electronic health record (EHR) system prototype containing four anonymized patient cases, each composed of nine progress notes that were presented with note sections organized in different orders to different subjects (i.e., Subjective, Objective, Assessment, and Plan, Assessment, Plan, Subjective, and Objective, Subjective, Assessment, Objective, and Plan, and Mixed). Participants, who were mid-level residents and fellows, reviewed the cases and provided a brief summary after reviewing each case. Time-related data were collected and analyzed using descriptive statistics. Surveys were administered and interviews regarding experiences reviewing notes were collected and analyzed qualitatively. RESULTS: Qualitatively, participants reported challenges related to reviewing electronic clinical notes. Experimentally, time spent reviewing notes varied based on the note section organization. Consistency in note section organization improved performance (e.g., less scrolling and searching) compared with Mixed section organization when reviewing progress notes. DISCUSSION: Clinicians face significant challenges reviewing electronic clinical notes. Our findings support minimizing extraneous information in notes, removing information that can be found in other parts of the EHR, and standardizing the display and order of note sections to improve clinicians' note review experience. CONCLUSION: Our findings support the need to improve EHR note design and presentation to support optimal note review patterns for clinicians.

Named Entity Recognition in Prehospital Trauma Care.

Natural language processing (NLP) methods would improve outcomes in the area of prehospital Emergency Medical Services (EMS) data collection and abstraction. This study evaluated off-the-shelf solutions for automating labelling of clinically relevant data from EMS reports. A qualitative approach for choosing the best possible ensemble of pretrained NLP systems was developed and validated along with a feature using word embeddings to test phrase synonymy. The ensemble showed increased performance over individual systems.

Usability Evaluation of an Unstructured Clinical Document Query Tool for Researchers.

Natural Language Processing - Patient Information Extraction for Researchers (NLP-PIER) was developed for clinical researchers for self-service Natural Language Processing (NLP) queries with clinical notes. This study was to conduct a user-centered analysis with clinical researchers to gain insight into NLP-PIER's usability and to gain an understanding of the needs of clinical researchers when using an application for searching clinical notes. Clinical researcher participants (n=11) completed tasks using the system's two existing search interfaces and completed a set of surveys and an exit interview. Quantitative data including time on task, task completion rate, and survey responses were collected. Interviews were analyzed qualitatively. Survey scores, time on task and task completion proportions varied widely. Qualitative analysis indicated that participants found the system to be useful and usable in specific projects. This study identified several usability challenges and our findings will guide the improvement of NLP-PIER 's interfaces.

Representation of occupational information across resources and validation of the occupational data for health model.

Reports by the National Academy of Medicine and leading public health organizations advocate including occupational information as part of an individual's social context. Given recent National Academy of Medicine recommendations on occupation-related data in the electronic health record, there is a critical need for improved representation. The National Institute for Occupational Safety and Health has developed an Occupational Data for Health (ODH) model, currently in draft format. This study aimed to validate the ODH model by mapping occupation-related elements from resources representing recommendations, standards, public health reports and surveys, and research measures, along with preliminary evaluation of associated value sets. All 247 occupation-related items across 20 resources mapped to the ODH model. Recommended value sets had high variability across the evaluated resources. This study demonstrates the ODH model's value, the multifaceted nature of occupation information, and the critical need for occupation value sets to support clinical care, population health, and research.

Characterizing Functional Health Status of Surgical Patients in Clinical Notes.

Functional health status is an important factor not only for determining overall health, but also for measuring risks of adverse events. Our hypothesis is that important functional status data is contained in clinical notes. We found that several categories of phrases related to functional status including diagnoses, activity and care assessments, physical exam, functional scores, assistive equipment, symptoms, and surgical history were important factors. Use of functional health status level terms from our chart review compared to National Surgical Quality Improvement Program determination had varying sensitivities for correct functional status category identification, with 96% for independent patients, 60% for partially dependent patients, and 44% for totally dependent patients. Inter-rater agreement assessing term relevance to functional health status was high at 91% (Kappa=0.74). Functional status-related terms in clinical notes show potential for use in future methodologies for automated detection of functional health status for quality improvement registries and other clinical assessments.

Comparing Existing Resources to Represent Dietary Supplements.

Dietary supplements, often considered as food, are widely consumed despite of limited knowledge around their safety/efficacy and any well-established regulatory policies, unlike their drug counterparts. Informatics methods may be useful in filling this knowledge gap, however, the lack of standardized representation of DS hinders this progress. In this pilot study, five electronic DS resources, i.e., NM, DSID & NHPID (ingredient level) and DSLD & LNHPD (product level), were evaluated and compared both quantitatively and qualitatively employing four phases. Essential data elements needed for comprehensive DS representation were compiled based on LanguaL code (food) & AHFSA (drugs) guidelines and employed as a check-list. We further investigated the completeness of DS representation by incorporating Ginseng and Fish oil as examples. We found fragmented and inconsistent distribution of DS representation in terms of essential data elements across five resources. This study provides a preliminary platform for development of standardized DS terminology/ontology model.

Representation of Social History Factors Across Age Groups: A Topic Analysis of Free-Text Social Documentation.

As individuals age, there is potential for dramatic changes in the social and behavioral determinants that affect health status and outcomes. The importance of these determinants has been increasingly recognized in clinical decision-making. We sought to characterize how social and behavioral health determinants vary in different demographic groups using a previously established schema of 28 social history types through both manual analysis and automated topic analysis of social documentation in the electronic health record across the population of an entire integrated healthcare system. Our manual analysis generated 8,335 annotations over 1,400 documents, representing 24 (86%) social history types. In contrast, automated topic analysis generated 22 (79%) social history types. A comparative evaluation demonstrated both similarities and differences in coverage between the manual and topic analyses. Our findings validate the widespread nature of social and behavioral determinants that affect health status over populations of individuals over their lifespan.

Residence, Living Situation, and Living Conditions Information Documentation in Clinical Practice.

Social determinants of health (SDOH) have an important role in diagnosis, prevention, health outcomes, and quality of life. Currently, SDOH information in electronic health record (EHR) systems is often contained in unstructured text. The objective of this study is to examine an important subset of SDOH documentation for Residence, Living Situation and Living Conditions in an enterprise EHR informed by previous model representations. In addition to two publically available clinical note sources, notes created by Social Work, Physical Therapy, and Occupational Therapy, along with free text Social Documentation entries were reviewed. Sentences were classified, annotated, and evaluated once mapped to element entities and attributes. Overall, 2,491 total notes yielded 616, 813, and 30 sentences related to Residence, Living Situation, and Living Conditions. This study demonstrated the need for additional elements in the model representation, more representative values and content culminating in a more comprehensive model representation for these key SDOH.

Assessing the Representation of Occupation Information in Free-Text Clinical Documents Across Multiple Sources.

There has been increasing recognition of the key role of social determinants like occupation on health. Given the relatively poor understanding of occupation information in electronic health records (EHRs), we sought to characterize occupation information within free-text clinical document sources. From six distinct clinical sources, 868 total occupation-related sentences were identified for the study corpus. Building off approaches from previous studies, refined annotation guidelines were created using the National Institute for Occupational Safety and Health Occupational Data for Health data model with elements added to increase granularity. Our corpus generated 2,005 total annotations representing 39 of 41 entity types from the enhanced data model. Highest frequency entities were: Occupation Description (17.7%); Employment Status - Not Specified (12.5%); Employer Name (11.0%); Subject (9.8%); Industry Description (6.2%). Our findings support the value of standardizing entry of EHR occupation information to improve data quality for improved patient care and secondary uses of this information.

Usability Evaluation of NLP-PIER: A Clinical Document Search Engine for Researchers.

NLP-PIER (Natural Language Processing - Patient Information Extraction for Research) is a self-service platform with a search engine for clinical researchers to perform natural language processing (NLP) queries using clinical notes. We conducted user-centered testing of NLP-PIER's usability to inform future design decisions. Quantitative and qualitative data were analyzed. Our findings will be used to improve the usability of NLP-PIER.

Investigating Longitudinal Tobacco Use Information from Social History and Clinical Notes in the Electronic Health Record.

The electronic health record (EHR) provides an opportunity for improved use of clinical documentation including leveraging tobacco use information by clinicians and researchers. In this study, we investigated the content, consistency, and completeness of tobacco use data from structured and unstructured sources in the EHR. A natural language process (NLP) pipeline was utilized to extract details about tobacco use from clinical notes and free-text tobacco use comments within the social history module of an EHR system. We analyzed the consistency of tobacco use information within clinical notes, comments, and available structured fields for tobacco use. Our results indicate that structured fields for tobacco use alone may not be able to provide complete tobacco use information. While there was better consistency for some elements (e.g., status and type), inconsistencies were found particularly for temporal information. Further work is needed to improve tobacco use information integration from different parts of the EHR.