Topic evolution before fall incidents in new fallers through natural language processing of general practitioners' clinical notes.

BACKGROUND: Falls involve dynamic risk factors that change over time, but most studies on fall-risk factors are cross-sectional and do not capture this temporal aspect. The longitudinal clinical notes within electronic health records (EHR) provide an opportunity to analyse fall risk factor trajectories through Natural Language Processing techniques, specifically dynamic topic modelling (DTM). This study aims to uncover fall-related topics for new fallers and track their evolving trends leading up to falls. METHODS: This case-cohort study utilised primary care EHR data covering information on older adults between 2016 and 2019. Cases were individuals who fell in 2019 but had no falls in the preceding three years (2016-18). The control group was randomly sampled individuals, with similar size to the cases group, who did not endure falls during the whole study follow-up period. We applied DTM on the clinical notes collected between 2016 and 2018. We compared the trend lines of the case and control groups using the slopes, which indicate direction and steepness of the change over time. RESULTS: A total of 2,384 fallers (cases) and an equal number of controls were included. We identified 25 topics that showed significant differences in trends between the case and control groups. Topics such as medications, renal care, family caregivers, hospital admission/discharge and referral/streamlining diagnostic pathways exhibited a consistent increase in steepness over time within the cases group before the occurrence of falls. CONCLUSIONS: Early recognition of health conditions demanding care is crucial for applying proactive and comprehensive multifactorial assessments that address underlying causes, ultimately reducing falls and fall-related injuries.

Predicting future falls in older people using natural language processing of general practitioners' clinical notes.

BACKGROUND: Falls in older people are common and morbid. Prediction models can help identifying individuals at higher fall risk. Electronic health records (EHR) offer an opportunity to develop automated prediction tools that may help to identify fall-prone individuals and lower clinical workload. However, existing models primarily utilise structured EHR data and neglect information in unstructured data. Using machine learning and natural language processing (NLP), we aimed to examine the predictive performance provided by unstructured clinical notes, and their incremental performance over structured data to predict falls. METHODS: We used primary care EHR data of people aged 65 or over. We developed three logistic regression models using the least absolute shrinkage and selection operator: one using structured clinical variables (Baseline), one with topics extracted from unstructured clinical notes (Topic-based) and one by adding clinical variables to the extracted topics (Combi). Model performance was assessed in terms of discrimination using the area under the receiver operating characteristic curve (AUC), and calibration by calibration plots. We used 10-fold cross-validation to validate the approach. RESULTS: Data of 35,357 individuals were analysed, of which 4,734 experienced falls. Our NLP topic modelling technique discovered 151 topics from the unstructured clinical notes. AUCs and 95% confidence intervals of the Baseline, Topic-based and Combi models were 0.709 (0.700-0.719), 0.685 (0.676-0.694) and 0.718 (0.708-0.727), respectively. All the models showed good calibration. CONCLUSIONS: Unstructured clinical notes are an additional viable data source to develop and improve prediction models for falls compared to traditional prediction models, but the clinical relevance remains limited.

Development and Internal Validation of a Prediction Model for Falls Using Electronic Health Records in a Hospital Setting.

OBJECTIVE: Fall prevention is important in many hospitals. Current fall-risk-screening tools have limited predictive accuracy specifically for older inpatients. Their administration can be time-consuming. A reliable and easy-to-administer tool is desirable to identify older inpatients at higher fall risk. We aimed to develop and internally validate a prognostic prediction model for inpatient falls for older patients. DESIGN: Retrospective analysis of a large cohort drawn from hospital electronic health record data. SETTING AND PARTICIPANTS: Older patients (≥70 years) admitted to a university medical center (2016 until 2021). METHODS: The outcome was an inpatient fall (≥24 hours of admission). Two prediction models were developed using regularized logistic regression in 5 imputed data sets: one model without predictors indicating missing values (Model-without) and one model with these additional predictors indicating missing values (Model-with). We internally validated our whole model development strategy using 10-fold stratified cross-validation. The models were evaluated using discrimination (area under the receiver operating characteristic curve) and calibration (plot assessment). We determined whether the areas under the receiver operating characteristic curves (AUCs) of the models were significantly different using DeLong test. RESULTS: Our data set included 21,286 admissions. In total, 470 (2.2%) had a fall after 24 hours of admission. The Model-without had 12 predictors and Model-with 13, of which 4 were indicators of missing values. The AUCs of the Model-without and Model-with were 0.676 (95% CI 0.646-0.707) and 0.695 (95% CI 0.667-0.724). The AUCs between both models were significantly different (P = .013). Calibration was good for both models. CONCLUSIONS AND IMPLICATIONS: Both the Model-with and Model-without indicators of missing values showed good calibration and fair discrimination, where the Model-with performed better. Our models showed competitive performance to well-established fall-risk-screening tools, and they have the advantage of being based on routinely collected data. This may substantially reduce the burden on nurses, compared with nonautomatic fall-risk-screening tools.

External Validation of a Prediction Model for Falls in Older People Based on Electronic Health Records in Primary Care.

OBJECTIVE: Early identification of older people at risk of falling is the cornerstone of fall prevention. Many fall prediction tools exist but their external validity is lacking. External validation is a prerequisite before application in clinical practice. Models developed with electronic health record (EHR) data are especially challenging because of the uncontrolled nature of routinely collected data. We aimed to externally validate our previously developed and published prediction model for falls, using a large cohort of community-dwelling older people derived from primary care EHR data. DESIGN: Retrospective analysis of a prospective cohort drawn from EHR data. SETTING AND PARTICIPANTS: Pseudonymized EHR data were collected from individuals aged ≥65 years, who were enlisted in any of the participating 59 general practices between 2015 and 2020 in the Netherlands. METHODS: Ten predictors were defined and obtained using the same methods as in the development study. The outcome was 1-year fall and was obtained from free text. Both reproducibility and transportability were evaluated. Model performance was assessed in terms of discrimination using the area under the receiver operating characteristic curve (ROC-AUC), and in terms of calibration, using calibration-in-the-large, calibration slope and calibration plots. RESULTS: Among 39,342 older people, 5124 (13.4%) fell in the 1-year follow-up. The characteristics of the validation and the development cohorts were similar. ROC-AUCs of the validation and development cohort were 0.690 and 0.705, respectively. Calibration-in-the-large and calibration slope were 0.012 and 0.878, respectively. Calibration plots revealed overprediction for high-risk groups in a small number of individuals. CONCLUSIONS AND IMPLICATIONS: Our previously developed prediction model for falls demonstrated good external validity by reproducing its predictive performance in the validation cohort. The implementation of this model in the primary care setting could be considered after impact assessment.

Development and Internal Validation of a Risk Prediction Model for Falls Among Older People Using Primary Care Electronic Health Records.

BACKGROUND: Currently used prediction tools have limited ability to identify community-dwelling older people at high risk for falls. Prediction models utilizing electronic health records (EHRs) provide opportunities but up to now showed limited clinical value as risk stratification tool, because of among others the underestimation of falls prevalence. The aim of this study was to develop a fall prediction model for community-dwelling older people using a combination of structured data and free text of primary care EHRs and to internally validate its predictive performance. METHODS: We used EHR data of individuals aged 65 or older. Age, sex, history of falls, medications, and medical conditions were included as potential predictors. Falls were ascertained from the free text. We employed the Bootstrap-enhanced penalized logistic regression with the least absolute shrinkage and selection operator to develop the prediction model. We used 10-fold cross-validation to internally validate the prediction strategy. Model performance was assessed in terms of discrimination and calibration. RESULTS: Data of 36 470 eligible participants were extracted from the data set. The number of participants who fell at least once was 4 778 (13.1%). The final prediction model included age, sex, history of falls, 2 medications, and 5 medical conditions. The model had a median area under the receiver operating curve of 0.705 (interquartile range 0.700-0.714). CONCLUSIONS: Our prediction model to identify older people at high risk for falls achieved fair discrimination and had reasonable calibration. It can be applied in clinical practice as it relies on routinely collected variables and does not require mobility assessment tests.

Impact of Altering Data Granularity Levels on Predictive Modelling: A Case Study of Fall Risk Prediction in Older Persons.

Classification systems are widely used in medicine for knowledge representation. The hierarchical relationships between concepts in a classification system can be exploited in prediction models by looking for the optimal predictive granularity level. In this study, we used the Anatomical Therapeutic Chemical (ATC) classification system to cluster medications in the context of predicting medication-related falls in older persons. We compared the performance of fall risk prediction by describing medications at varying granularity levels of the ATC classification system. We found that the level of abstraction significantly affects the predictive performance in terms of both discrimination (measured by the receiver operating characteristic curve AUC-ROC) and calibration. An implication of these findings to the researchers is that data representation at different granularity levels can influence the predictive performance. The optimal granularity level can be determined by experimentation.