Applying generative AI with retrieval augmented generation to summarize and extract key clinical information from electronic health records.

BACKGROUND: Malnutrition is a prevalent issue in aged care facilities (RACFs), leading to adverse health outcomes. The ability to efficiently extract key clinical information from a large volume of data in electronic health records (EHR) can improve understanding about the extent of the problem and developing effective interventions. This research aimed to test the efficacy of zero-shot prompt engineering applied to generative artificial intelligence (AI) models on their own and in combination with retrieval augmented generation (RAG), for the automating tasks of summarizing both structured and unstructured data in EHR and extracting important malnutrition information. METHODOLOGY: We utilized Llama 2 13B model with zero-shot prompting. The dataset comprises unstructured and structured EHRs related to malnutrition management in 40 Australian RACFs. We employed zero-shot learning to the model alone first, then combined it with RAG to accomplish two tasks: generate structured summaries about the nutritional status of a client and extract key information about malnutrition risk factors. We utilized 25 notes in the first task and 1,399 in the second task. We evaluated the model's output of each task manually against a gold standard dataset. RESULT: The evaluation outcomes indicated that zero-shot learning applied to generative AI model is highly effective in summarizing and extracting information about nutritional status of RACFs' clients. The generated summaries provided concise and accurate representation of the original data with an overall accuracy of 93.25%. The addition of RAG improved the summarization process, leading to a 6% increase and achieving an accuracy of 99.25%. The model also proved its capability in extracting risk factors with an accuracy of 90%. However, adding RAG did not further improve accuracy in this task. Overall, the model has shown a robust performance when information was explicitly stated in the notes; however, it could encounter hallucination limitations, particularly when details were not explicitly provided. CONCLUSION: This study demonstrates the high performance and limitations of applying zero-shot learning to generative AI models to automatic generation of structured summarization of EHRs data and extracting key clinical information. The inclusion of the RAG approach improved the model performance and mitigated the hallucination problem.

Machine Learning Model to Extract Malnutrition Data from Nursing Notes.

Malnutrition is a severe health problem that is prevalent in older people residing in residential aged care facilities. Recent advancements in machine learning have made it possible to extract key insight from electronic health records. To date, few researchers applied these techniques to classify nursing notes automatically. Therefore, we propose a model based on ClinicalBioBert to identify malnutrition notes. We evaluated our approach with two mainstream approaches. Our approach had the highest F1-score of 0.90.

A Five-Step Workflow to Manually Annotate Unstructured Data into Training Dataset for Natural Language Processing.

Natural Language Processing (NLP) is a powerful technique for extracting valuable information from unstructured electronic health records (EHRs). However, a prerequisite for NLP is the availability of high-quality annotated datasets. To date, there is a lack of effective methods to guide the research effort of manually annotating unstructured datasets, which can hinder NLP performance. Therefore, this study develops a five-step workflow for manually annotating unstructured datasets, including (1) annotator training and familiarising with the text corpus, (2) vocabulary identification, (3) annotation schema development, (4) annotation execution, and (5) result validation. This framework was then applied to annotate agitation symptoms from the unstructured EHRs of 40 Australian residential aged care facilities. The annotated corpus achieved an accuracy rate of 96%. This suggests that our proposed annotation workflow can be used in manual data processing to develop annotated training corpus for developing NLP algorithms.

Extracting Symptoms of Agitation in Dementia from Free-Text Nursing Notes Using Advanced Natural Language Processing.

Nursing staff record observations about older people under their care in free-text nursing notes. These notes contain older people's care needs, disease symptoms, frequency of symptom occurrence, nursing actions, etc. Therefore, it is vital to develop a technique to uncover important data from these notes. This study developed and evaluated a deep learning and transfer learning-based named entity recognition (NER) model for extracting symptoms of agitation in dementia from the nursing notes. We employed a Clinical BioBERT model for word embedding. Then we applied bidirectional long-short-term memory (BiLSTM) and conditional random field (CRF) models for NER on nursing notes from Australian residential aged care facilities. The proposed NER model achieves satisfactory performance in extracting symptoms of agitation in dementia with a 75% F1 score and 78% accuracy. We will further develop machine learning models to recommend the optimal nursing actions to manage agitation.

Developing an ontology of non-pharmacological treatment for emotional and mood disturbances in dementia.

Emotional and mood disturbances are common in people with dementia. Non-pharmacological interventions are beneficial for managing these disturbances. However, effectively applying these interventions, particularly in the person-centred approach, is a complex and knowledge-intensive task. Healthcare professionals need the assistance of tools to obtain all relevant information that is often buried in a vast amount of clinical data to form a holistic understanding of the person for successfully applying non-pharmacological interventions. A machine-readable knowledge model, e.g., ontology, can codify the research evidence to underpin these tools. For the first time, this study aims to develop an ontology entitled Dementia-Related Emotional And Mood Disturbance Non-Pharmacological Treatment Ontology (DREAMDNPTO). DREAMDNPTO consists of 1258 unique classes (concepts) and 70 object properties that represent relationships between these classes. It meets the requirements and quality standards for biomedical ontology. As DREAMDNPTO provides a computerisable semantic representation of knowledge specific to non-pharmacological treatment for emotional and mood disturbances in dementia, it will facilitate the application of machine learning to this particular and important health domain of emotional and mood disturbance management for people with dementia.

Malnutrition and its contributing factors for older people living in residential aged care facilities: Insights from natural language processing of aged care records.

BACKGROUND: Malnutrition is a serious health risk facing older people living in residential aged care facilities. Aged care staff record observations and concerns about older people in electronic health records (EHR), including free-text progress notes. These insights are yet to be unleashed. OBJECTIVE: This study explored the risk factors for malnutrition in structured and unstructured electronic health data. METHODS: Data of weight loss and malnutrition were extracted from the de-identified EHR records of a large aged care organization in Australia. A literature review was conducted to identify causative factors for malnutrition. Natural language processing (NLP) techniques were applied to progress notes to extract these causative factors. The NLP performance was evaluated by the parameters of sensitivity, specificity and F1-Score. RESULTS: The NLP methods were highly accurate in extracting the key data, values for 46 causative variables, from the free-text client progress notes. Thirty three percent (1,469 out of 4,405) of the clients were malnourished. The structured, tabulated data only recorded 48% of these malnourished clients, far less than that (82%) identified from the progress notes, suggesting the importance of using NLP technology to uncover the information from nursing notes to fully understand the health status of the vulnerable older people in residential aged care. CONCLUSION: This study identified 33% of older people suffered from malnutrition, lower than those reported in the similar setting in previous studies. Our study demonstrates that NLP technology is important for uncovering the key information about health risks for older people in residential aged care. Future research can apply NLP to predict other health risks for older people in this setting.

Agitation Prevalence in People With Dementia in Australian Residential Aged Care Facilities: Findings From Machine Learning of Electronic Health Records.

Using a suite of artificial intelligence technologies, the current study sought to determine the prevalence of agitated behaviors in people with dementia in residential aged care facilities (RACFs) in Australia. Computerized natural language processing allowed extraction of agitation instances from the free-text nursing progress notes, a component of electronic health records in RACFs. In total, 59 observable agitated behaviors were found. No difference was found in dementia prevalence between female and male clients (44.1%), across metropolitan and regional facilities (42.1% [SD = 17.9%]), or for agitation prevalence in dementia (76.5% [SD = 18.4%]). The top 10 behaviors were resisting, wandering, speaking in excessively loud voice, pacing, restlessness, pushing, shouting, complaining, frustration, and using profane language. Four to 17 agitated behaviors coexisted in 53% of people with dementia agitation, indicating high caregiver burden in these RACFs. Improving workforce training and redesigning care models are urgent for sustainability of dementia care in RACFs. [Journal of Gerontological Nursing, 48(4), 57-64.].

Developing an Intuitive Graph Representation of Knowledge for Nonpharmacological Treatment of Psychotic Symptoms in Dementia.

Applying person-centered, nonpharmacological interventions to manage psychotic symptoms of dementia is promoted for health care professionals, particularly gerontological nurses, who are responsible for care of older adults in nursing homes. A knowledge graph is a graph consisting of a set of concepts that are linked together by their interrelationship and has been widely used as a formal representation of domain knowledge in health. However, there is lack of a knowledge graph for nonpharmacological treatment of psychotic symptoms in dementia. Therefore, we developed a comprehensive, human- and machine-understandable knowledge graph for this domain, named Dementia-Related Psychotic Symptom Nonpharmacological Treatment Ontology (DRPSNPTO). This graph was built by adopting the established NeOn methodology, a knowledge graph engineering method, to meet the quality standards for biomedical knowledge graphs. This intuitive graph representation of the domain knowledge sets a new direction for visualizing and computerizing gerontological knowledge to facilitate human comprehension and build intelligent aged care information systems. [Journal of Gerontological Nursing, 48(4), 49-55.].

Developing an ontology for representing the domain knowledge specific to non-pharmacological treatment for agitation in dementia.

INTRODUCTION: A large volume of clinical care data has been generated for managing agitation in dementia. However, the valuable information in these data has not been used effectively to generate insights for improving the quality of care. Application of artificial intelligence technologies offers us enormous opportunities to reuse these data. For health data science to achieve this, this study focuses on using ontology to coding clinical knowledge for non-pharmacological treatment of agitation in a machine-readable format. METHODS: The resultant ontology-Dementia-Related Agitation Non-Pharmacological Treatment Ontology (DRANPTO)-was developed using a method adopted from the NeOn methodology. RESULTS: DRANPTO consisted of 569 concepts and 48 object properties. It meets the standards for biomedical ontology. DISCUSSION: DRANPTO is the first comprehensive semantic representation of non-pharmacological management for agitation in dementia in the long-term care setting. As a knowledge base, it will play a vital role to facilitate the development of intelligent systems for managing agitation in dementia.