Algorithmic Identification of Treatment-Emergent Adverse Events From Clinical Notes Using Large Language Models: A Pilot Study in Inflammatory Bowel Disease.

Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLMs) like Bidirectional Encoder Representations from Transformers (BERT) have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event (AE) detection. We adapted a new clinical LLM, University of California - San Francisco (UCSF)-BERT, to identify serious AEs (SAEs) occurring after treatment with a non-steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. We annotated 928 outpatient IBD notes corresponding to 928 individual patients with IBD for all SAE-associated hospitalizations occurring after treatment with a non-steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of eight candidate models, UCSF-BERT achieved the highest numerical performance on identifying drug-SAE pairs from this corpus (accuracy 88-92%, macro F1 61-68%), with 5-10% greater accuracy than previously published models. UCSF-BERT was significantly superior at identifying hospitalization events emergent to medication use (P < 0.01). LLMs like UCSF-BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared with prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multicenter data and newer architectures like Generative pre-trained transformer (GPT). Our findings support the potential value of using large language models to enhance pharmacovigilance.

Algorithmic identification of treatment-emergent adverse events from clinical notes using large language models: a pilot study in inflammatory bowel disease.

Background and Aims: Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLM) like BERT have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event detection. Methods: We adapted a new clinical LLM, UCSF BERT, to identify serious adverse events (SAEs) occurring after treatment with a non-steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. Results: We annotated 928 outpatient IBD notes corresponding to 928 individual IBD patients for all SAE-associated hospitalizations occurring after treatment with a non-steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of 8 candidate models, UCSF BERT achieved the highest numerical performance on identifying drug-SAE pairs from this corpus (accuracy 88-92%, macro F1 61-68%), with 5-10% greater accuracy than previously published models. UCSF BERT was significantly superior at identifying hospitalization events emergent to medication use (p < 0.01). Conclusions: LLMs like UCSF BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared to prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multi-center data and newer architectures like GPT. Our findings support the potential value of using large language models to enhance pharmacovigilance.

Centrins, cell cycle regulation proteins in human malaria parasite Plasmodium falciparum.

Molecules and cellular mechanisms that regulate the process of cell division in malaria parasites remain poorly understood. In this study we isolate and characterize the four Plasmodium falciparum centrins (PfCENs) and, by growth complementation studies, provide evidence for their involvement in cell division. Centrins are cytoskeleton proteins with key roles in cell division, including centrosome duplication, and possess four Ca(2+)-binding EF hand domains. By means of phylogenetic analysis, we were able to decipher the evolutionary history of centrins in eukaryotes with particular emphasis on the situation in apicomplexans and other alveolates. Plasmodium possesses orthologs of four distinct centrin paralogs traceable to the ancestral alveolate, including two that are unique to alveolates. By real time PCR and/or immunofluorescence, we determined the expression of PfCEN mRNA or protein in sporozoites, asexual blood forms, gametocytes, and in the oocysts developing inside mosquito mid-gut. Immunoelectron microscopy studies showed that centrin is expressed in close proximity with the nucleus of sporozoites and asexual schizonts. Furthermore, confocal and widefield microscopy using the double staining with alpha-tubulin and centrin antibodies strongly suggested that centrin is associated with the parasite centrosome. Following the episomal expression of the four PfCENs in a centrin knock-out Leishmania donovani parasite line that exhibited a severe growth defect, one of the PfCENs was able to partially restore Leishmania growth rate and overcome the defect in cytokinesis in such mutant cell line. To our knowledge, this study is the first characterization of a Plasmodium molecule that is involved in the process of cell division. These results provide the opportunity to further explore the role of centrins in cell division in malaria parasites and suggest novel targets to construct genetically modified, live attenuated malaria vaccines.

Quality prediction of cell substrate using gene expression profiling.

Changes in cell culture conditions influence the metabolism of cells, which consequently affects the quality of the products that they produce, such as viral vectors, recombinant proteins, or vaccines. Currently there is no effective technique available to monitor global quality of cells in cell culture. Here we describe a new method using gene expression profiling by microarray to predict the quality of cell substrates. Human embryonic kidney 293 cells are a commonly used cell substrate in the production of biological products. We demonstrate that the yield of adenoviral vectors was lower in over-confluent 293 cells, compared to 40 or 90% confluent cells. Total RNA derived from these cells of different confluence states was reverse transcribed, labeled, and used to hybridize 10K cDNA arrays to determine biomarkers for confluence states. Phenotype scatter-plot analysis and cluster analysis were used for class discovery. Based on this approach, we identified genes that were either up-regulated or down-modulated in response to different cell confluence states. By multivariate predictive models we identified a set of 37 genes that were either down-regulated or up-regulated compared to 90% confluent cells as a predictor of cell confluence and quality of 293 cell cultures. The predictive accuracy of these models was assessed by the leave-one-out cross-validation method. The expression of selected gene predictors was validated by quantitative PCR analysis. Our results demonstrate that gene expression profiling can assess the quality of cell substrates prior to large-scale production of a biological product.