Artificial intelligence approaches for phenotyping heart failure in U.S. Veterans Health Administration electronic health record.

AIMS: Heart failure (HF) is a clinical syndrome with no definitive diagnostic tests. HF registries are often based on manual reviews of medical records of hospitalized HF patients identified using International Classification of Diseases (ICD) codes. However, most HF patients are not hospitalized, and manual review of big electronic health record (EHR) data is not practical. The US Department of Veterans Affairs (VA) has the largest integrated healthcare system in the nation, and an estimated 1.5 million patients have ICD codes for HF (HF ICD-code universe) in their VA EHR. The objective of our study was to develop artificial intelligence (AI) models to phenotype HF in these patients. METHODS AND RESULTS: The model development cohort (n = 20 000: training, 16 000; validation 2000; testing, 2000) included 10 000 patients with HF and 10 000 without HF who were matched by age, sex, race, inpatient/outpatient status, hospital, and encounter date (within 60 days). HF status was ascertained by manual chart reviews in VA's External Peer Review Program for HF (EPRP-HF) and non-HF status was ascertained by the absence of ICD codes for HF in VA EHR. Two clinicians annotated 1000 random snippets with HF-related keywords and labelled 436 as HF, which was then used to train and test a natural language processing (NLP) model to classify HF (positive predictive value or PPV, 0.81; sensitivity, 0.77). A machine learning (ML) model using linear support vector machine architecture was trained and tested to classify HF using EPRP-HF as cases (PPV, 0.86; sensitivity, 0.86). From the 'HF ICD-code universe', we randomly selected 200 patients (gold standard cohort) and two clinicians manually adjudicated HF (gold standard HF) in 145 of those patients by chart reviews. We calculated NLP, ML, and NLP + ML scores and used weighted F scores to derive their optimal threshold values for HF classification, which resulted in PPVs of 0.83, 0.77, and 0.85 and sensitivities of 0.86, 0.88, and 0.83, respectively. HF patients classified by the NLP + ML model were characteristically and prognostically similar to those with gold standard HF. All three models performed better than ICD code approaches: one principal hospital discharge diagnosis code for HF (PPV, 0.97; sensitivity, 0.21) or two primary outpatient encounter diagnosis codes for HF (PPV, 0.88; sensitivity, 0.54). CONCLUSIONS: These findings suggest that NLP and ML models are efficient AI tools to phenotype HF in big EHR data to create contemporary HF registries for clinical studies of effectiveness, quality improvement, and hypothesis generation.

A Novel Explainable AI Method to Assess Associations between Temporal Patterns in Patient Trajectories and Adverse Outcome Risks: Analyzing Fitness as a Risk Factor of ADRD.

We present a novel explainable artificial intelligence (XAI) method to assess the associations between the temporal patterns in the patient trajectories recorded in longitudinal clinical data and the adverse outcome risks, through explanations for a type of deep neural network model called Hybrid Value-Aware Transformer (HVAT) model. The HVAT models can learn jointly from longitudinal and non-longitudinal clinical data, and in particular can leverage the time-varying numerical values associated with the clinical codes or concepts within the longitudinal data for outcome prediction. The key component of the XAI method is the definitions of two derived variables, the temporal mean and the temporal slope, which are defined for the clinical concepts with associated time-varying numerical values. The two variables represent the overall level and the rate of change over time, respectively, in the trajectory formed by the values associated with the clinical concept. Two operations on the original values are designed for changing the values of the two derived variables separately. The effects of the two variables on the outcome risks learned by the HVAT model are calculated in terms of impact scores and impacts. Interpretations of the impact scores and impacts as being similar to those of odds ratios are also provided. We applied the XAI method to the study of cardiorespiratory fitness (CRF) as a risk factor of Alzheimer's disease and related dementias (ADRD). Using a retrospective case-control study design, we found that each one-unit increase in the overall CRF level is associated with a 5% reduction in ADRD risk, while each one-unit increase in the changing rate of CRF over time is associated with a 1% reduction. A closer investigation revealed that the association between the changing rate of CRF level and the ADRD risk is nonlinear, or more specifically, approximately piecewise linear along the axis of the changing rate on two pieces: the piece of negative changing rates and the piece of positive changing rates.

Sexual and Gender Minority Status and Suicide Mortality: An Explainable Artificial Intelligence Analysis.

Objectives: Suicide risk is elevated in lesbian, gay, bisexual, and transgender (LGBT) individuals. Limited data on LGBT status in healthcare systems hinder our understanding of this risk. This study used natural language processing to extract LGBT status and a deep neural network (DNN) to examine suicidal death risk factors among US Veterans. Methods: Data on 8.8 million veterans with visits between 2010 and 2017 was used. A case-control study was performed, and suicide death risk was analyzed by a DNN. Feature impacts and interactions on the outcome were evaluated. Results: The crude suicide mortality rate was higher in LGBT patients. However, after adjusting for over 200 risk and protective factors, known LGBT status was associated with reduced risk compared to LGBT-Unknown status. Among LGBT patients, black, female, married, and older Veterans have a higher risk, while Veterans of various religions have a lower risk. Conclusion: Our results suggest that disclosed LGBT status is not directly associated with an increase suicide death risk, however, other factors (e.g., depression and anxiety caused by stigma) are associated with suicide death risks.

Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data.

Transformer is the latest deep neural network (DNN) architecture for sequence data learning, which has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clinical data and natural language data, clinical data presents unique complexities that make adapting Transformer to this domain challenging. To address this issue, we have designed a new Transformer-based DNN architecture, referred to as Hybrid Value-Aware Transformer (HVAT), which can jointly learn from longitudinal and non-longitudinal clinical data. HVAT is unique in the ability to learn from the numerical values associated with clinical codes/concepts such as labs, and in the use of a flexible longitudinal data representation called clinical tokens. We have also trained a prototype HVAT model on a case-control dataset, achieving high performance in predicting Alzheimer's disease and related dementias as the patient outcome. The results demonstrate the potential of HVAT for broader clinical data-learning tasks.

Patient Dietary Supplements Use: Do Results from Natural Language Processing of Clinical Notes Agree with Survey Data?

There is widespread use of dietary supplements, some prescribed but many taken without a physician's guidance. There are many potential interactions between supplements and both over-the-counter and prescription medications in ways that are unknown to patients. Structured medical records do not adequately document supplement use; however, unstructured clinical notes often contain extra information on supplements. We studied a group of 377 patients from three healthcare facilities and developed a natural language processing (NLP) tool to detect supplement use. Using surveys of these patients, we investigated the correlation between self-reported supplement use and NLP extractions from the clinical notes. Our model achieved an F1 score of 0.914 for detecting all supplements. Individual supplement detection had a variable correlation with survey responses, ranging from an F1 of 0.83 for calcium to an F1 of 0.39 for folic acid. Our study demonstrated good NLP performance while also finding that self-reported supplement use is not always consistent with the documented use in clinical records.

Phenotype fingerprinting of bipolar disorder prodrome.

BACKGROUND: Detecting prodromal symptoms of bipolar disorder (BD) has garnered significant attention in recent research, as early intervention could potentially improve therapeutic efficacy and improve patient outcomes. The heterogeneous nature of the prodromal phase in BD, however, poses considerable challenges for investigators. Our study aimed to identify distinct prodromal phenotypes or "fingerprints" in patients diagnosed with BD and subsequently examine correlations between these fingerprints and relevant clinical outcomes. METHODS: 20,000 veterans diagnosed with BD were randomly selected for this study. K-means clustering analysis was performed on temporal graphs of the clinical features of each patient. We applied what we call "temporal blurring" to each patient image in order to allow clustering to focus on the clinical features, and not cluster patients based upon their varying temporal patterns in diagnosis, which lead to the desired types of clusters. We evaluated several outcomes including mortality rate, hospitalization rate, mean number of hospitalizations, mean length of stay, and the occurrence of a psychosis diagnosis within one year following the initial BD diagnosis. To determine the statistical significance of the observed differences for each outcome, we conducted appropriate tests, such as ANOVA or Chi-square. RESULTS: Our analysis yielded 8 clusters which appear to represent distinct phenotypes with differing clinical attributes. Each of these clusters also has statistically significant differences across all outcomes (p < 0.0001). The clinical features in many of the clusters were consistent with findings in the literature concerning prodromal symptoms in patients with BD. One cluster, notably characterized by patients lacking discernible prodromal symptoms, exhibited the most favorable results across all measured outcomes. CONCLUSION: Our study successfully identified distinct prodromal phenotypes in patients diagnosed with BD. We also found that these distinct prodromal phenotypes are associated with different clinical outcomes.

Cardiorespiratory fitness and risk of Alzheimer's disease and related dementias among American veterans.

INTRODUCTION: Cardiorespiratory fitness (CRF) is associated with improved health and survival. Less is known about its association with Alzheimer's disease and related dementias (ADRD). METHODS: We identified 649,605 US veterans 30 to 95 years of age and free of ADRD who completed a standardized exercise tolerance test between 2000 and 2017 with no evidence of ischemia. We examined the association between five age- and sex-specific CRF categories and ADRD incidence using multivariate Cox regression models. RESULTS: During up to 20 (median 8.3) years of follow-up, incident ADRD occurred in 44,105 (6.8%) participants, with an incidence rate of 7.7/1000 person-years. Compared to the least-fit, multivariable-adjusted hazard ratios (95% confidence intervals) for incident ADRD were: 0.87 (0.85-0.90), 0.80 (0.78-0.83), 0.74 (0.72-0.76), and 0.67 (0.65-0.70), for low-fit, moderate-fit, fit, and high-fit individuals, respectively. DISSCUSSION: These findings demonstrate an independent, inverse, and graded association between CRF and incident ADRD. Future studies may determine the amount and duration of physical activity needed to optimize ADRD risk reduction.

Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data.

Transformer is the latest deep neural network (DNN) architecture for sequence data learning that has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clinical data and natural language data, clinical data presents unique complexities that make adapting Transformer to this domain challenging. To address this issue, we have designed a new Transformer-based DNN architecture, referred to as Hybrid Value-Aware Transformer (HVAT), which can jointly learn from longitudinal and non-longitudinal clinical data. HVAT is unique in the ability to learn from the numerical values associated with clinical codes/concepts such as labs, and also the use of a flexible longitudinal data representation called clinical tokens. We trained a prototype HVAT model on a case-control dataset, achieving high performance in predicting Alzheimer’s disease and related dementias as the patient outcome. The result demonstrates the potential of HVAT for broader clinical data learning tasks.

Identifying probable dementia in undiagnosed Black and White Americans using machine learning in Veterans Health Administration electronic health records.

The application of machine learning (ML) tools in electronic health records (EHRs) can help reduce the underdiagnosis of dementia, but models that are not designed to reflect minority population may perpetuate that underdiagnosis. To address the underdiagnosis of dementia in both Black Americans (BAs) and white Americans (WAs), we sought to develop and validate ML models that assign race-specific risk scores. These scores were used to identify undiagnosed dementia in BA and WA Veterans in EHRs. More specifically, risk scores were generated separately for BAs (n=10K) and WAs (n=10K) in training samples of cases and controls by performing ML, equivalence mapping, topic modeling, and a support vector-machine (SVM) in structured and unstructured EHR data. Scores were validated via blinded manual chart reviews (n=1.2K) of controls from a separate sample (n=20K). AUCs and negative and positive predictive values (NPVs and PPVs) were calculated to evaluate the models. There was a strong positive relationship between SVM-generated risk scores and undiagnosed dementia. BAs were more likely than WAs to have undiagnosed dementia per chart review, both overall (15.3% vs 9.5%) and among Veterans with >90th percentile cutoff scores (25.6% vs 15.3%). With chart reviews as the reference standard and varied cutoff scores, the BA model performed slightly better than the WA model (AUC=0.86 with NPV=0.98 and PPV=0.26 at >90th percentile cutoff vs AUC=0.77 with NPV=0.98 and PPV=0.15 at >90th). The AUCs, NPVs, and PPVs suggest that race-specific ML models can assist in the identification of undiagnosed dementia, particularly in BAs. Future studies should investigate implementing EHR-based risk scores in clinics that serve both BA and WA Veterans.

Enhancing Clinical Data Analysis by Explaining Interaction Effects between Covariates in Deep Neural Network Models.

Deep neural network (DNN) is a powerful technology that is being utilized by a growing number and range of research projects, including disease risk prediction models. One of the key strengths of DNN is its ability to model non-linear relationships, which include covariate interactions. We developed a novel method called interaction scores for measuring the covariate interactions captured by DNN models. As the method is model-agnostic, it can also be applied to other types of machine learning models. It is designed to be a generalization of the coefficient of the interaction term in a logistic regression; hence, its values are easily interpretable. The interaction score can be calculated at both an individual level and population level. The individual-level score provides an individualized explanation for covariate interactions. We applied this method to two simulated datasets and a real-world clinical dataset on Alzheimer's disease and related dementia (ADRD). We also applied two existing interaction measurement methods to those datasets for comparison. The results on the simulated datasets showed that the interaction score method can explain the underlying interaction effects, there are strong correlations between the population-level interaction scores and the ground truth values, and the individual-level interaction scores vary when the interaction was designed to be non-uniform. Another validation of our new method is that the interactions discovered from the ADRD data included both known and novel relationships.

Identification of colorectal cancer using structured and free text clinical data.

Colorectal cancer incidence has continually fallen among those 50 years old and over. However, the incidence has increased in those under 50. Even with the recent screening guidelines recommending that screening begins at age 45, nearly half of all early-onset colorectal cancer will be missed. Methods are needed to identify high-risk individuals in this age group for targeted screening. Colorectal cancer studies, as with other clinical studies, have required labor intensive chart review for the identification of those affected and risk factors. Natural language processing and machine learning can be used to automate the process and enable the screening of large numbers of patients. This study developed and compared four machine learning and statistical models: logistic regression, support vector machine, random forest, and deep neural network, in their performance in classifying colorectal cancer patients. Excellent classification performance is achieved with AUCs over 97%.

Medication-Wide Association Study Plus (MWAS+): A Proof of Concept Study on Drug Repurposing.

The high cost and time for developing a new drug or repositioning a partially-developed drug has fueled interest in "repurposing" drugs. Drug repurposing is particularly of interest for Alzheimer's disease (AD) or AD-related dementias (ADRD) because there are no unrestricted disease-modifying treatments for ADRD. We have designed and pilot tested a 3-Step Medication-Wide Association Study Plus (MWAS+) approach to rigorously accelerate the identification of drugs with a high potential to be repurposed for delaying and preventing AD/ADRD: Step 1 is a hypothesis-free exploration; Step 2 is mechanistic filtering; And Step 3 is hypothesis testing using observational data and prospective cohort design. Our results demonstrated the feasibility of the MWAS+ approach. The Step 1 analysis identified potential candidate drugs including atorvastatin and GLP1. The literature search in Step 2 found evidence supporting the mechanistic plausibility of the statin-ADRD association. Finally, Step 3 confirmed our hypothesis that statin may lower the risk of incident ADRD, which was statistically significant using a target trial design that emulated randomized controlled trials.

Accuracies of Training Labels and Machine Learning Models: Experiments on Delirium and Simulated Data.

Supervised predictive models require labeled data for training purposes. Complete and accurate labeled data is not always available, and imperfectly labeled data may need to serve as an alternative. An important question is if the accuracy of the labeled data creates a performance ceiling for the trained model. In this study, we trained several models to recognize the presence of delirium in clinical documents using data with annotations that are not completely accurate. In the external evaluation, the support vector machine model with a linear kernel performed best, achieving an area under the curve of 89.3% and accuracy of 88%, surpassing the 80% accuracy of the training sample. We then generated a set of simulated data and carried out a series of experiments which demonstrated that models trained on imperfect data can (but do not always) outperform the accuracy of the training data.

Genomic landscape of single-stranded DNA gapped intermediates in Escherichia coli.

Single-stranded (ss) gapped regions in bacterial genomes (gDNA) are formed on W- and C-strands during replication, repair, and recombination. Using non-denaturing bisulfite treatment to convert C to U on ssDNA, combined with deep sequencing, we have mapped gDNA gap locations, sizes, and distributions in Escherichia coli for cells grown in mid-log phase in the presence and absence of UV irradiation, and in stationary phase cells. The fraction of ssDNA on gDNA is similar for W- and C-strands, ∼1.3% for log phase cells, ∼4.8% for irradiated log phase cells, and ∼8.5% for stationary phase cells. After UV irradiation, gaps increased in numbers and average lengths. A monotonic reduction in ssDNA occurred symmetrically between the DNA replication origin of (OriC) and terminus (Ter) for log phase cells with and without UV, a hallmark feature of DNA replication. Stationary phase cells showed no OriC → Ter ssDNA gradient. We have identified a spatially diverse gapped DNA landscape containing thousands of highly enriched 'hot' ssDNA regions along with smaller numbers of 'cold' regions. This analysis can be used for a wide variety of conditions to map ssDNA gaps generated when DNA metabolic pathways have been altered, and to identify proteins bound in the gaps.

Cardiovascular Safety of Hydroxychloroquine in US Veterans With Rheumatoid Arthritis.

OBJECTIVE: Hydroxychloroquine (HCQ) may prolong the QT interval, a risk factor for torsade de pointes, a potentially fatal ventricular arrhythmia. This study was undertaken to examine the cardiovascular safety of HCQ in patients with rheumatoid arthritis (RA). METHODS: We conducted an active comparator safety study of HCQ in a propensity score-matched cohort of 8,852 US veterans newly diagnosed as having RA between October 1, 2001 and December 31, 2017. Patients were started on HCQ (n = 4,426) or another nonbiologic disease-modifying antirheumatic drug (DMARD; n = 4,426) after RA diagnosis, up to December 31, 2018, and followed up for 12 months after therapy initiation, up to December 31, 2019. RESULTS: Patients had a mean ± SD age of 64 ± 12 years, 14% were women, and 28% were African American. The treatment groups were balanced with regard to 87 baseline characteristics. There were 3 long QT syndrome events (0.03%), 2 of which occurred in patients receiving HCQ. Of the 56 arrhythmia-related hospitalizations (0.63%), 30 occurred in patients in the HCQ group (hazard ratio [HR] associated with HCQ 1.16 [95% confidence interval (95% CI) 0.68-1.95]). All-cause mortality occurred in 144 (3.25%) and 136 (3.07%) of the patients in the HCQ and non-HCQ groups, respectively (HR associated with HCQ 1.06 [95% CI, 0.84-1.34]). During the first 30 days of follow-up, there were no long QT syndrome events, 2 arrhythmia-related hospitalizations (none in the HCQ group), and 13 deaths (6 in the HCQ group). CONCLUSION: Our findings indicate that the incidence of long QT syndrome and arrhythmia-related hospitalization is low in patients with RA during the first year after the initiation of HCQ or another nonbiologic DMARD. We found no evidence that HCQ therapy is associated with a higher risk of adverse cardiovascular events or death.

Understanding Demographic Risk Factors for Adverse Outcomes in COVID-19 Patients: Explanation of a Deep Learning Model.

This study was to understand the impacts of three key demographic variables, age, gender, and race, on the adverse outcome of all-cause hospitalization or all-cause mortality in patients with COVID-19, using a deep neural network (DNN) analysis. We created a cohort of Veterans who were tested positive for COVID-19, extracted data on age, gender, and race, and clinical characteristics from their electronic health records, and trained a DNN model for predicting the adverse outcome. Then, we analyzed the association of the demographic variables with the risks of the adverse outcome using the impact scores and interaction scores for explaining DNN models. The results showed that, on average, older age and African American race were associated with higher risks while female gender was associated with lower risks. However, individual-level impact scores of age showed that age was a more impactful risk factor in younger patients and in older patients with fewer comorbidities. The individual-level impact scores of gender and race variables had a wide span covering both positive and negative values. The interaction scores between the demographic variables showed that the interaction effects were minimal compared to the impact scores associated with them. In conclusion, the DNN model is able to capture the non-linear relationship between the risk factors and the adverse outcome, and the impact scores and interaction scores can help explain the complicated non-linear effects between the demographic variables and the risk of the outcome.

Shedding Light on the Black Box: Explaining Deep Neural Network Prediction of Clinical Outcomes.

Deep neural network models are emerging as an important method in healthcare delivery, following the recent success in other domains such as image recognition. Due to the multiple non-linear inner transformations, deep neural networks are viewed by many as black boxes. For practical use, deep learning models require explanations that are intuitive to clinicians. In this study, we developed a deep neural network model to predict outcomes following major cardiovascular procedures, using temporal image representation of past medical history as input. We created a novel explanation for the prediction of the model by defining impact scores that associate clinical observations with the outcome. For comparison, a logistic regression model was fitted to the same dataset. We compared the impact scores and log odds ratios by calculating three types of correlations, which provided a partial validation of the impact scores. The deep neural network model achieved an area under the receiver operating characteristics curve (AUC) of 0.787, compared to 0.746 for the logistic regression model. Moderate correlations were found between the impact scores and the log odds ratios. Impact scores generated by the explanation algorithm has the potential to shed light on the "black box" deep neural network model and could facilitate its adoption by clinicians.

Personal black carbon and ultrafine particles exposures among high school students in urban China.

Air pollution is a major public health challenge in the highly urbanized megacities of China. However, knowledge on exposure to ambient unregulated air pollutants such as black carbon (BC) and ultrafine particles (UFP) among the Chinese population, especially among urban high school students who may have highly variable time-activity patterns, is scarce. To address this, the personal exposures to BC and UFP of high school students (aged 17 to 18) in Chengdu, China were measured at 1-min intervals via portable samplers. Monitoring lasted for 2 consecutive 24-h periods with days classified as "school days" or "non-school days". Time-activity diaries and measurements were combined to explore spatial, temporal, and behavioral factors that contribute to different exposure profiles. The overall geometric means of BC and UFP were 3.60 µg/m3 and 1.83 × 104p/cm3, respectively with notable spatiotemporal variation in exposures observed. In general, the household and transport microenvironments were the predominant contributors to total BC (74.5%) and UFP (36.5%) exposure. However, the outdoor public microenvironment was found to have significantly higher overall average levels of BC than the household and transport microenvironments (p < 0.001) while also presenting the greatest exposure dose intensity (EDI - a measure of exposure in a microenvironment in proportion to time spent in that environment) of 4.79. The largest overall average level of UFP occurred in the indoor public microenvironment followed by transport. The outdoor public microenvironment also presented the greatest EDI of UFP (4.17). This study shows notable spatiotemporal variety in exposure patterns and will inform future exposure and population health studies. The high EDI outdoors may mean that health positive activities, such as exercise, may be being undermined by ambient pollution.

Urinary bisphenol analogues concentrations and biomarkers of oxidative DNA and RNA damage in Chinese school children in East China: A repeated measures study.

The associations between bisphenol analogues (BPs) exposure and oxidative damage was explored in this 3-year longitudinal study of 275 school children in East China. Nine BPs in first morning urine samples were measured to assess BPs exposure, and 8-hydroxydeoxyguanosine (8-OHdG) and 8-oxo-7,8-dihydroguanosine (8-OHG) were measured as biomarkers of oxidative DNA and RNA damage. Linear mixed model (LMM) was used for repeated measures analysis. School children were mainly exposed to BPA, BPS, BPF, and BPAF (detection frequencies: 97.9%, 42.2%, 13.3%, and 12.8%) with median concentrations of 1.55, 0.355, 0.236 and 0.238 µg g-1Cre, respectively. An interquartile range (IQR) increase in urinary BPA was significantly associated with 12.9% (95% CI: 6.1%, 19.6%) increase in 8-OHdG and 19.4% (95% CI: 11.7%, 27.1%) increase in 8-OHG, and for total of BPs (the sum of BPA, BPS, BPF, and BPAF), they were 17.4% (95% CI: 8.9%, 26.0%) for 8-OHdG and 25.9% (95% CI: 16.1%, 35.7%) for 8-OHG, respectively. BPS was positively associated with 8-OHG, but not with 8-OHdG. The study found positive associations of urinary levels of BPA and total BPs with 8-OHdG and 8-OHG and indicated that BPs exposure might cause oxidative RNA damage.

A Proficient Spelling Analysis Method Applied to a Pharmacovigilance Task.

Misspellings in clinical free text present potential challenges to pharmacovigilance tasks, such as monitoring for potential ineffective treatment of drug-resistant infections. We developed a novel method using Word2Vec, Levenshtein edit distance constraints, and a customized lexicon to identify correct and misspelled pharmaceutical word forms. We processed a large corpus of clinical notes in a real-world pharmacovigilance task, achieving positive predictive values of 0.929 and 0.909 in identifying valid misspellings and correct spellings, respectively, and negative predictive values of 0.994 and 0.333 as assessments where the program did not produce output. In a specific Methicillin-Resistant Staphylococcus Aureus use case, the method identified 9,815 additional instances in the corpus for potential inaffective drug administration inspection. The findings suggest that this method could potentially achieve satisfactory results for other pharmacovigilance tasks.