Algorithmic Identification of Treatment-Emergent Adverse Events From Clinical Notes Using Large Language Models: A Pilot Study in Inflammatory Bowel Disease.

Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLMs) like Bidirectional Encoder Representations from Transformers (BERT) have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event (AE) detection. We adapted a new clinical LLM, University of California - San Francisco (UCSF)-BERT, to identify serious AEs (SAEs) occurring after treatment with a non-steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. We annotated 928 outpatient IBD notes corresponding to 928 individual patients with IBD for all SAE-associated hospitalizations occurring after treatment with a non-steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of eight candidate models, UCSF-BERT achieved the highest numerical performance on identifying drug-SAE pairs from this corpus (accuracy 88-92%, macro F1 61-68%), with 5-10% greater accuracy than previously published models. UCSF-BERT was significantly superior at identifying hospitalization events emergent to medication use (P < 0.01). LLMs like UCSF-BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared with prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multicenter data and newer architectures like Generative pre-trained transformer (GPT). Our findings support the potential value of using large language models to enhance pharmacovigilance.

Accurate, Robust, and Scalable Machine Abstraction of Mayo Endoscopic Subscores From Colonoscopy Reports.

BACKGROUND: The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES. METHODS: We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness. RESULTS: Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P = .65) or area deprivation index (P = .80). CONCLUSIONS: We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.

Our accurate pair of models automatically classify colonoscopy reports by Mayo endoscopic subscore and abstain from prediction appropriately. Our methods can enable large-scale electronic health record studies of treatment effectiveness, prospective identification of patients for clinical trials, and quality improvement efforts in ulcerative colitis.

Algorithmic identification of treatment-emergent adverse events from clinical notes using large language models: a pilot study in inflammatory bowel disease.

Background and Aims: Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLM) like BERT have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event detection. Methods: We adapted a new clinical LLM, UCSF BERT, to identify serious adverse events (SAEs) occurring after treatment with a non-steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. Results: We annotated 928 outpatient IBD notes corresponding to 928 individual IBD patients for all SAE-associated hospitalizations occurring after treatment with a non-steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of 8 candidate models, UCSF BERT achieved the highest numerical performance on identifying drug-SAE pairs from this corpus (accuracy 88-92%, macro F1 61-68%), with 5-10% greater accuracy than previously published models. UCSF BERT was significantly superior at identifying hospitalization events emergent to medication use (p < 0.01). Conclusions: LLMs like UCSF BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared to prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multi-center data and newer architectures like GPT. Our findings support the potential value of using large language models to enhance pharmacovigilance.

Case-control study of the association of chronic acid suppression and social determinants of health with COVID-19 infection.

Acid suppressants are widely-used classes of medications linked to increased risks of aerodigestive infections. Prior studies of these medications as potentially reversible risk factors for COVID-19 have been conflicting. We aimed to determine the impact of chronic acid suppression use on COVID-19 infection risk while simultaneously evaluating the influence of social determinants of health to validate known and discover novel risk factors. We assessed the association of chronic acid suppression with incident COVID-19 in a 1:1 case-control study of 900 patients tested across three academic medical centers in California, USA. Medical comorbidities and history of chronic acid suppression use were manually extracted from health records by physicians following a pre-specified protocol. Socio-behavioral factors by geomapping publicly-available data to patient zip codes were incorporated. We identified no evidence to support an association between chronic acid suppression and COVID-19 (adjusted odds ratio 1.04, 95% CI 0.92-1.17, P = 0.515). However, several medical and social features were positive (Latinx ethnicity, BMI ≥ 30, dementia, public transportation use, month of the pandemic) and negative (female sex, concurrent solid tumor, alcohol use disorder) predictors of new infection. These findings demonstrate the value of integrating publicly-available databases with medical data to identify critical features of communicable diseases.

Diagnostic Accuracy of the Spectralis and Cirrus Reference Databases in Differentiating between Healthy and Early Glaucoma Eyes.

PURPOSE: To evaluate and compare the diagnostic accuracy of global and sector analyses for detection of early visual field (VF) damage using the retinal nerve fiber layer (RNFL) reference databases of the Spectralis (Heidelberg Engineering, Heidelberg, Germany) and Cirrus (Carl Zeiss Meditec, Dublin, CA) spectral-domain optical coherence tomography (SD OCT) devices. METHODS: Healthy subjects and glaucoma suspects from the Diagnostic Innovations in Glaucoma Study (DIGS) and African Descent and Glaucoma Evaluation Study (ADAGES) with at least 2 years of follow-up were included. Global and sectoral RNFL measures were classified as within normal limits, borderline (BL), and outside normal limits (ONL) on the basis of the device reference databases. The sensitivity of ONL classification was estimated in glaucoma suspect eyes that developed repeatable VF damage. RESULTS: A total of 353 glaucoma suspect eyes and 279 healthy eyes were included. A total of 34 (9.6%) of the glaucoma suspect eyes developed VF damage. In glaucoma suspect eyes, Spectralis and Cirrus ONL classification was present in 47 eyes (13.3%) and 24 eyes (6.8%), respectively. The sensitivity of the global RNFL ONL classification among eyes that developed VF damage was 23.5% for Cirrus and 32.4% for Spectralis. The specificity of within-normal-limits global classification in healthy eyes was 100% for Cirrus and 99.6% for Spectralis. There was moderate to substantial agreement between Cirrus and Spectralis classification as ONL. CONCLUSIONS: The Spectralis and Cirrus reference databases have a high specificity for identifying healthy eyes and good agreement for detection of eyes with early glaucoma damage.

Assessment of optic nerve head drusen using enhanced depth imaging and swept source optical coherence tomography.

BACKGROUND: Optic nerve head drusen (ONHD) are calcific deposits buried or at the surface of the optic disc. Although ONHD may be associated with progressive visual field defects, the mechanism of drusen-related field loss is poorly understood. Methods for detecting and imaging disc drusen include B-scan ultrasonography, fundus autofluorescence, and optical coherence tomography (OCT). These modalities are useful for drusen detection but are limited by low resolution or poor penetration of deep structures. This review was designed to assess the potential role of new OCT technologies in imaging ONHD. EVIDENCE ACQUISITION: Critical appraisal of published literature and comparison of new imaging devices to established technology. RESULTS: The new imaging modalities of enhanced depth imaging optical coherence tomography (EDI-OCT) and swept source optical coherence tomography (SS-OCT) are able to provide unprecedented in vivo detail of ONHD. Using these devices it is now possible to quantify optic disc drusen dimensions and assess integrity of neighboring retinal structures, including the retinal nerve fiber layer. CONCLUSIONS: EDI-OCT and SS-OCT have the potential to allow better detection of longitudinal changes in drusen and neural retina and improve our understanding of drusen-related visual field loss.

Contradicting an established mantra: microsurgeon experience determines free flap outcome.

BACKGROUND: Since the inception and popularization of microsurgery in the 1960s and 1970s, it has been commonly accepted that the outcome of free tissue transfer directly correlates with surgeon experience. METHODS: The clinical outcomes of three young microsurgeons at a single institution were retrospectively reviewed. Free flaps performed by these individuals were categorized according to the surgeon's years of practice and analyzed using statistical methods. RESULTS: A total of 410 free flaps were identified. No correlation was found between the surgeon's years of experience and the outcomes measured. CONCLUSIONS: There has been increased exposure to microsurgery during plastic surgery training at many programs, and consequently, residents have often already surpassed the learning curve. The imperfect correlation between experience and superior outcomes in medicine serves to suggest that further research in the specific underlying principles of surgical learning is needed to understand the relationship between experience and superior surgical outcomes.

Reconstruction of the burned hand.

This article summarizes the initial management of acute burn injuries to the hand, in addition to treatment and reconstructive options. The goal of treatment for a burn injury to the hand is primarily a functional hand. This is best achieved by appropriate early treatment, the right selection from a wide range of possible reconstructive procedures, and focused occupational hand therapy.