Impact of Translation on Biomedical Information Extraction: Experiment on Real-Life Clinical Notes.

Background: Biomedical natural language processing tasks are best performed with English models, and translation tools have undergone major improvements. On the other hand, building annotated biomedical data sets remains a challenge. Objective: The aim of our study is to determine whether the use of English tools to extract and normalize French medical concepts based on translations provides comparable performance to that of French models trained on a set of annotated French clinical notes. Methods: We compared 2 methods: 1 involving French-language models and 1 involving English-language models. For the native French method, the named entity recognition and normalization steps were performed separately. For the translated English method, after the first translation step, we compared a 2-step method and a terminology-oriented method that performs extraction and normalization at the same time. We used French, English, and bilingual annotated data sets to evaluate all stages (named entity recognition, normalization, and translation) of our algorithms. Results: The native French method outperformed the translated English method, with an overall F1-score of 0.51 (95% CI 0.47-0.55), compared with 0.39 (95% CI 0.34-0.44) and 0.38 (95% CI 0.36-0.40) for the 2 English methods tested. Conclusions: Despite recent improvements in translation models, there is a significant difference in performance between the 2 approaches in favor of the native French method, which is more effective on French medical texts, even with few annotated documents.

Collaborative and privacy-enhancing workflows on a clinical data warehouse: an example developing natural language processing pipelines to detect medical conditions.

OBJECTIVE: To develop and validate a natural language processing (NLP) pipeline that detects 18 conditions in French clinical notes, including 16 comorbidities of the Charlson index, while exploring a collaborative and privacy-enhancing workflow. MATERIALS AND METHODS: The detection pipeline relied both on rule-based and machine learning algorithms, respectively, for named entity recognition and entity qualification, respectively. We used a large language model pre-trained on millions of clinical notes along with annotated clinical notes in the context of 3 cohort studies related to oncology, cardiology, and rheumatology. The overall workflow was conceived to foster collaboration between studies while respecting the privacy constraints of the data warehouse. We estimated the added values of the advanced technologies and of the collaborative setting. RESULTS: The pipeline reached macro-averaged F1-score positive predictive value, sensitivity, and specificity of 95.7 (95%CI 94.5-96.3), 95.4 (95%CI 94.0-96.3), 96.0 (95%CI 94.0-96.7), and 99.2 (95%CI 99.0-99.4), respectively. F1-scores were superior to those observed using alternative technologies or non-collaborative settings. The models were shared through a secured registry. CONCLUSIONS: We demonstrated that a community of investigators working on a common clinical data warehouse could efficiently and securely collaborate to develop, validate and use sensitive artificial intelligence models. In particular, we provided an efficient and robust NLP pipeline that detects conditions mentioned in clinical notes.

Impact of translation on biomedical information extraction: an experiment on real-life clinical notes.

BACKGROUND: Biomedical natural language processing tasks are best performed with English models, and translation tools have undergone major improvements. On the other hand, building annotated biomedical datasets remains a challenge. OBJECTIVE: The aim of our study is to determine whether the use of English tools to extract and normalize French medical concepts on translations provides comparable performance to that of French models trained on a set of annotated French clinical notes. METHODS: We compare two methods: one involving French-language models and one involving English-language models. For the native French method, the Named Entity Recognition (NER) and normalization steps are performed separately. For the translated English method, after the first translation step, we compare a two-step method and a terminology-oriented method that performs extraction and normalization at the same time. We used French, English and bilingual annotated datasets to evaluate all stages (NER, normalization and translation) of our algorithms. RESULTS: The native French method outperformed the translated English method, with an overall f1 score of 0.51 [0.47;0.55], compared with 0.39 [0.34;0.44] and 0.38 [0.36;0.40] for the two English methods tested. CONCLUSIONS: Despite recent improvements in translation models, there is a significant difference in performance between the two approaches in favor of the native French method, which is more effective on French medical texts, even with few annotated documents.

Users' Reactions to Announced Vaccines Against COVID-19 Before Marketing in France: Analysis of Twitter Posts.

BACKGROUND: Within a few months, the COVID-19 pandemic had spread to many countries and had been a real challenge for health systems all around the world. This unprecedented crisis has led to a surge of online discussions about potential cures for the disease. Among them, vaccines have been at the heart of the debates and have faced lack of confidence before marketing in France. OBJECTIVE: This study aims to identify and investigate the opinions of French Twitter users on the announced vaccines against COVID-19 through sentiment analysis. METHODS: This study was conducted in 2 phases. First, we filtered a collection of tweets related to COVID-19 available on Twitter from February 2020 to August 2020 with a set of keywords associated with vaccine mistrust using word embeddings. Second, we performed sentiment analysis using deep learning to identify the characteristics of vaccine mistrust. The model was trained on a hand-labeled subset of 4548 tweets. RESULTS: A set of 69 relevant keywords were identified as the semantic concept of the word "vaccin" (vaccine in French) and focused mainly on conspiracies, pharmaceutical companies, and alternative treatments. Those keywords enabled us to extract nearly 350,000 tweets in French. The sentiment analysis model achieved 0.75 accuracy. The model then predicted 16% of positive tweets, 41% of negative tweets, and 43% of neutral tweets. This allowed us to explore the semantic concepts of positive and negative tweets and to plot the trends of each sentiment. The main negative rhetoric identified from users' tweets was that vaccines are perceived as having a political purpose and that COVID-19 is a commercial argument for the pharmaceutical companies. CONCLUSIONS: Twitter might be a useful tool to investigate the arguments for vaccine mistrust because it unveils political criticism contrasting with the usual concerns on adverse drug reactions. As the opposition rhetoric is more consistent and more widely spread than the positive rhetoric, we believe that this research provides effective tools to help health authorities better characterize the risk of vaccine mistrust.

Construction of Cohorts of Similar Patients From Automatic Extraction of Medical Concepts: Phenotype Extraction Study.

BACKGROUND: Reliable and interpretable automatic extraction of clinical phenotypes from large electronic medical record databases remains a challenge, especially in a language other than English. OBJECTIVE: We aimed to provide an automated end-to-end extraction of cohorts of similar patients from electronic health records for systemic diseases. METHODS: Our multistep algorithm includes a named-entity recognition step, a multilabel classification using medical subject headings ontology, and the computation of patient similarity. A selection of cohorts of similar patients on a priori annotated phenotypes was performed. Six phenotypes were selected for their clinical significance: P1, osteoporosis; P2, nephritis in systemic erythematosus lupus; P3, interstitial lung disease in systemic sclerosis; P4, lung infection; P5, obstetric antiphospholipid syndrome; and P6, Takayasu arteritis. We used a training set of 151 clinical notes and an independent validation set of 256 clinical notes, with annotated phenotypes, both extracted from the Assistance Publique-Hôpitaux de Paris data warehouse. We evaluated the precision of the 3 patients closest to the index patient for each phenotype with precision-at-3 and recall and average precision. RESULTS: For P1-P4, the precision-at-3 ranged from 0.85 (95% CI 0.75-0.95) to 0.99 (95% CI 0.98-1), the recall ranged from 0.53 (95% CI 0.50-0.55) to 0.83 (95% CI 0.81-0.84), and the average precision ranged from 0.58 (95% CI 0.54-0.62) to 0.88 (95% CI 0.85-0.90). P5-P6 phenotypes could not be analyzed due to the limited number of phenotypes. CONCLUSIONS: Using a method close to clinical reasoning, we built a scalable and interpretable end-to-end algorithm for extracting cohorts of similar patients.

Multilabel classification of medical concepts for patient clinical profile identification.

BACKGROUND: The development of electronic health records has provided a large volume of unstructured biomedical information. Extracting patient characteristics from these data has become a major challenge, especially in languages other than English. METHODS: Inspired by the French Text Mining Challenge (DEFT 2021) [1] in which we participated, our study proposes a multilabel classification of clinical narratives, allowing us to automatically extract the main features of a patient report. Our system is an end-to-end pipeline from raw text to labels with two main steps: named entity recognition and multilabel classification. Both steps are based on a neural network architecture based on transformers. To train our final classifier, we extended the dataset with all English and French Unified Medical Language System (UMLS) vocabularies related to human diseases. We focus our study on the multilingualism of training resources and models, with experiments combining French and English in different ways (multilingual embeddings or translation). RESULTS: We obtained an overall average micro-F1 score of 0.811 for the multilingual version, 0.807 for the French-only version and 0.797 for the translated version. CONCLUSION: Our study proposes an original multilabel classification of French clinical notes for patient phenotyping. We show that a multilingual algorithm trained on annotated real clinical notes and UMLS vocabularies leads to the best results.

Drug-induced antiphospholipid syndrome: Analysis of the WHO international database.

OBJECTIVE: As with drug-induced lupus, some drugs may induce an antiphospholipid syndrome (APS). With the always growing numbers of new molecules, the list of the liable treatments evolves rapidly. We herein analyzed VigiBase, the international pharmacovigilance database, to identify drugs suspected of inducing APS. METHODS: All the reported cases associated with "anti-phospholipid syndrome" using the preferred term level of medDRA (dictionary of regulated drug activity) when associated with anti-phospholipid antibodies in VigiBase were analyzed. For each treatment, a Bayesian disproportionality indicator (i.e. information component, IC) was calculated. A drug was significantly associated with APS if the 95% lower-end of the IC credibility interval was positive (IC025 > 0). Drugs with potential protopathic bias were excluded. RESULTS: From 01/11/2000 to 25/07/2021, 790 reports of suspected drug-induced APS were found in VigiBase. After excluding drugs reported by a single country and drugs with protopathic bias, fourteen drugs (n = 359 reports) were associated with APS with an IC0 25 > 0. These drugs were hormons: ethinylestradiol-etonogestrel and drospirenone-ethynilestradiol; platelet growth factors: eltrombopag, romiplostim; vaccines: Human Papillomavirus vaccine, hepatitis A and B vaccines and typhoid vaccine; antibiotics: minocycline; nonstreroidal anti-inflammatory: rofecoxib; biotherapy: interferon beta-1-a, etanercept; anti-hypertensive drug: hydralazine; bisphosphonates: alendronic acid and antipsychotic: olanzapine. The mean age at diagnosis of drug-induced APS was 39.2 years [29.3;47.9] and there were 63.5% of female patients. The mean delay from first exposition to drug-induced APS was 19.7 months [4.5; 38.8]. Drug-induced APS was reported as a severe side effect in 66.3% of cases: 8.4% with a life-threatening event and 2.5% of death (n = 9). A third (n = 118, 32.9%) pulmonary embolism events were reported and 4.2% (15) cerebral infarctions. 14.8% (53) cases were associated with a systemic lupus, a sub-analysis without lupus cases showed the same severity of cases. CONCLUSION: This study identified 14 drugs potentially associated with drug-induced APS that may prove useful in the investigational work-up in any new diagnosis of APS. TRIAL REGISTRATION NUMBER: NCT03994302.

Survival and specific outcome of sickle cell disease patients after renal transplantation.

The prognosis of sickle cell disease (SCD) patients who need dialysis is poor, but experience with kidney transplantation is limited. This study assessed the characteristics of 36 SCD patients undergoing renal transplantation. Immediate post-surgical complications occurred in 25% of cases. Cytomegalovirus and bacterial infections were frequently observed. Twelve patients died after a median follow-up period of 17·4 months. Overall patient survival was significantly lower in SCD than in the control group without significant difference for overall death-censored graft survival. Our data suggest that renal transplantation should be systematically considered in SCD patients with end-stage renal disease.

Increased FDG-PET/CT pericardial uptake identifies acute pericarditis patients at high risk for relapse.

OBJECTIVES: We aimed to evaluate the prognostic value of FDG pericardial uptake using FDG-PET/CT in patients admitted for acute pericarditis with pericardial effusion. METHODS: In this monocentric retrospective cohort study, all patients admitted for idiopathic acute pericarditis with pericardial effusion from January 2009 to December 2016 who underwent a FDG-PET/CT at diagnosis were considered. Pericardial FDG uptake was measured by generating a volume of interest to calculate the maximal standardized uptake value. The primary outcome was the pericarditis relapse rate during follow-up. RESULTS: FDG-PET/CT was performed 23 [7-99] days after diagnosis in 39 patients (52 [18-83] years, 43.6% of women) admitted for acute pericarditis with pericardial effusion. During a median follow-up period of 7.6 [2.4-77.2] months, 7 (17.9%) patients suffered pericarditis relapse that occurred 3.8 [1.6-14.6] months after FDG-PET CT. In the multivariable analysis, pericardial FDG uptake at diagnosis (OR: 16.6; 95% confidence interval [CI]: 1.25 to 220.8; p = 0.033) was independently associated with pericarditis relapse. Eventually, patients with pericardial FDG uptake at diagnosis had a higher recurrence rate during follow up (p = 0.047). CONCLUSIONS: In acute pericarditis with pericardial effusion, increased FDG-PET/CT pericardial uptake is associated with a higher risk for relapse.