RESUMO
The ability to fine-tune pre-trained deep learning models to learn how to process a downstream task using a large training set allow to significantly improve performances of named entity recognition. Large language models are recent models based on the Transformers architecture that may be conditioned on a new task with in-context learning, by providing a series of instructions or prompt. These models only require few examples and such approach is defined as few shot learning. Our objective was to compare performances of named entity recognition of adverse drug events between state of the art deep learning models fine-tuned on Pubmed abstracts and a large language model using few-shot learning. Hussain et al's state of the art model (PMID: 34422092) significantly outperformed the ChatGPT-3.5 model (F1-Score: 97.6% vs 86.0%). Few-shot learning is a convenient way to perform named entity recognition when training examples are rare, but performances are still inferior to those of a deep learning model fine-tuned with several training examples. Perspectives are to evaluate few-shot prompting with GPT-4 and perform fine-tuning on GPT-3.5.
Assuntos
Aprendizado Profundo , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Humanos , Processamento de Linguagem Natural , Sistemas de Notificação de Reações Adversas a MedicamentosRESUMO
BACKGROUND: To mitigate safety concerns, regulatory agencies must make informed decisions regarding drug usage and adverse drug events (ADEs). The primary pharmacovigilance data stem from spontaneous reports by health care professionals. However, underreporting poses a notable challenge within the current system. Explorations into alternative sources, including electronic patient records and social media, have been undertaken. Nevertheless, social media's potential remains largely untapped in real-world scenarios. OBJECTIVE: The challenge faced by regulatory agencies in using social media is primarily attributed to the absence of suitable tools to support decision makers. An effective tool should enable access to information via a graphical user interface, presenting data in a user-friendly manner rather than in their raw form. This interface should offer various visualization options, empowering users to choose representations that best convey the data and facilitate informed decision-making. Thus, this study aims to assess the potential of integrating social media into pharmacovigilance and enhancing decision-making with this novel data source. To achieve this, our objective was to develop and assess a pipeline that processes data from the extraction of web forum posts to the generation of indicators and alerts within a visual and interactive environment. The goal was to create a user-friendly tool that enables regulatory authorities to make better-informed decisions effectively. METHODS: To enhance pharmacovigilance efforts, we have devised a pipeline comprising 4 distinct modules, each independently editable, aimed at efficiently analyzing health-related French web forums. These modules were (1) web forums' posts extraction, (2) web forums' posts annotation, (3) statistics and signal detection algorithm, and (4) a graphical user interface (GUI). We showcase the efficacy of the GUI through an illustrative case study involving the introduction of the new formula of Levothyrox in France. This event led to a surge in reports to the French regulatory authority. RESULTS: Between January 1, 2017, and February 28, 2021, a total of 2,081,296 posts were extracted from 23 French web forums. These posts contained 437,192 normalized drug-ADE couples, annotated with the Anatomical Therapeutic Chemical (ATC) Classification and Medical Dictionary for Regulatory Activities (MedDRA). The analysis of the Levothyrox new formula revealed a notable pattern. In August 2017, there was a sharp increase in posts related to this medication on social media platforms, which coincided with a substantial uptick in reports submitted by patients to the national regulatory authority during the same period. CONCLUSIONS: We demonstrated that conducting quantitative analysis using the GUI is straightforward and requires no coding. The results aligned with prior research and also offered potential insights into drug-related matters. Our hypothesis received partial confirmation because the final users were not involved in the evaluation process. Further studies, concentrating on ergonomics and the impact on professionals within regulatory agencies, are imperative for future research endeavors. We emphasized the versatility of our approach and the seamless interoperability between different modules over the performance of individual modules. Specifically, the annotation module was integrated early in the development process and could undergo substantial enhancement by leveraging contemporary techniques rooted in the Transformers architecture. Our pipeline holds potential applications in health surveillance by regulatory agencies or pharmaceutical companies, aiding in the identification of safety concerns. Moreover, it could be used by research teams for retrospective analysis of events.
Assuntos
Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Farmacovigilância , Mídias Sociais , Humanos , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos/prevenção & controle , Sistemas de Notificação de Reações Adversas a Medicamentos/estatística & dados numéricos , InternetRESUMO
The proliferation of health misinformation in recent years has prompted the development of various methods for detecting and combatting this issue. This review aims to provide an overview of the implementation strategies and characteristics of publicly available datasets that can be used for health misinformation detection. Since 2020, a large number of such datasets have emerged, half of which are focused on COVID-19. Most of the datasets are based on fact-checkable websites, while only a few are annotated by experts. Furthermore, some datasets provide additional information such as social engagement and explanations, which can be utilized to study the spread of misinformation. Overall, these datasets offer a valuable resource for researchers working to combat the spread and consequences of health misinformation.
Assuntos
COVID-19 , Humanos , Pesquisadores , Participação SocialRESUMO
BACKGROUND: Within a few months, the COVID-19 pandemic had spread to many countries and had been a real challenge for health systems all around the world. This unprecedented crisis has led to a surge of online discussions about potential cures for the disease. Among them, vaccines have been at the heart of the debates and have faced lack of confidence before marketing in France. OBJECTIVE: This study aims to identify and investigate the opinions of French Twitter users on the announced vaccines against COVID-19 through sentiment analysis. METHODS: This study was conducted in 2 phases. First, we filtered a collection of tweets related to COVID-19 available on Twitter from February 2020 to August 2020 with a set of keywords associated with vaccine mistrust using word embeddings. Second, we performed sentiment analysis using deep learning to identify the characteristics of vaccine mistrust. The model was trained on a hand-labeled subset of 4548 tweets. RESULTS: A set of 69 relevant keywords were identified as the semantic concept of the word "vaccin" (vaccine in French) and focused mainly on conspiracies, pharmaceutical companies, and alternative treatments. Those keywords enabled us to extract nearly 350,000 tweets in French. The sentiment analysis model achieved 0.75 accuracy. The model then predicted 16% of positive tweets, 41% of negative tweets, and 43% of neutral tweets. This allowed us to explore the semantic concepts of positive and negative tweets and to plot the trends of each sentiment. The main negative rhetoric identified from users' tweets was that vaccines are perceived as having a political purpose and that COVID-19 is a commercial argument for the pharmaceutical companies. CONCLUSIONS: Twitter might be a useful tool to investigate the arguments for vaccine mistrust because it unveils political criticism contrasting with the usual concerns on adverse drug reactions. As the opposition rhetoric is more consistent and more widely spread than the positive rhetoric, we believe that this research provides effective tools to help health authorities better characterize the risk of vaccine mistrust.
Assuntos
COVID-19 , Mídias Sociais , Humanos , Vacinas contra COVID-19 , Pandemias , Marketing , Preparações FarmacêuticasRESUMO
Artificial Intelligence (AI) has made major progress in recent years in many fields. With regard of medicine however, the utilization of AI raises numerous ethical questions, especially since newer and much more accurate algorithms function as black boxes. A trade-off must then be made between having algorithms being very accurate and effective, and algorithms that are explainable but less proficient. In this paper we address the ethical implications of utilizing black box algorithms in medicine.
Assuntos
Inteligência Artificial , Medicina , AlgoritmosRESUMO
In previous work, we implemented a deep learning model with CamemBERT and PyTorch, and built a microservices architecture using the TorchServe serving library. Without TorchServe, inference time was three times faster when the model was loaded once in memory compared when the model was loaded each time. The preloaded model without TorchServe presented comparable inference time with the TorchServe instance. However, using a PyTorch preloaded model in a web application without TorchServe would necessitate to implement functionalities already present in TorchServe.
Assuntos
COVID-19 , Aprendizado Profundo , Mídias Sociais , Vacinas , COVID-19/prevenção & controle , Vacinas contra COVID-19 , Humanos , Análise de SentimentosRESUMO
In 2022, the Medical Informatics Europe conference created a special topic called "Challenges of trustable AI and added-value on health" which was centered around the theme of eXplainable Artificial Intelligence. Unfortunately, two opposite views remain for biomedical applications of machine learning: accepting to use reliable but opaque models, vs. enforce models to be explainable. In this contribution we discuss these two opposite approaches and illustrate with examples the differences between them.
Assuntos
Informática Médica , Medicina , Inteligência Artificial , Europa (Continente) , Aprendizado de MáquinaRESUMO
A strong trend in the software industry is to merge the activities of deployment and operationalization through the DevOps approach, which in the case of artificial intelligence is called Machine Learning Operations (MLOps). We present here a microservices architecture containing the whole pipeline (frontend, backend, data predictions) hosted in Docker containers which exposes a model implemented for opinion prediction in Twitter on the COVID vaccines. This is the first description in the literature of implementing a microservice architecture using TorchServe, a library for serving Pytorch models.
Assuntos
COVID-19 , Mídias Sociais , Inteligência Artificial , COVID-19/prevenção & controle , Vacinas contra COVID-19 , Humanos , SARS-CoV-2RESUMO
Our objective was to improve the accuracy of bacteria and resistance coding in a hospital case mix database. Data sources consisted of 50,074 files on bacteriological susceptibility tests transmitted with the HPRIM protocol from laboratory management system to electronic health record of the University hospital of Saint Etienne in July 2017. An algorithm was implemented to detect susceptibility tests containing information corresponding to codes whose addition in the case mix database was susceptible to increase the severity level of a diagnosis related group. Among 132 hospital stays fulfilling the conditions, 27 were lacking bacteria and/or resistance codes, and the tariff was increased for 9 stays, with earnings of 54,612. Analyzing Antimicrobial susceptibility tests helps to improve clinical coding and optimize the financial gain.
Assuntos
Anti-Infecciosos , Infecções Bacterianas , Infecções Bacterianas/tratamento farmacológico , Codificação Clínica , Bases de Dados Factuais , Grupos Diagnósticos Relacionados , HumanosRESUMO
Methods of natural language processing associated with machine learning or deep learning can support detection of adverse drug reactions in abstracts of case reports available on Pubmed. In 2012, Gurulingappa et al. proposed a training set for the recognition of named entities corresponding to drugs and adverse reactions on 3000 Pubmed abstracts. We implemented a classifier using deep learning with a Bi-LSTM and a CRF layer that achieves an F-measure of 87.8%. Perspectives consist in using BERT for improving the classifier, and applying it to a large number of Pubmed abstract to build a database of case reports available in the literature.
Assuntos
Aprendizado Profundo , Farmacovigilância , Aprendizado de Máquina , Processamento de Linguagem Natural , PubMedRESUMO
Polypharmacy in elderly is a public health problem with both clinical (increase of adverse drug events) and economic issues. One solution is medication review, a structured assessment of patients' drug orders by the pharmacist for optimizing the therapy. However, this task is tedious, cognitively complex and error-prone, and only a few clinical decision support systems have been proposed for supporting it. Existing systems are either rule-based systems implementing guidelines, or documentary systems presenting drug knowledge. In this paper, we present the ABiMed research project, and, through literature reviews and brainstorming, we identified five candidate innovations for a decision support system for medication review: patient data transfer from GP to pharmacists, use of semantic technologies, association of rule-based and documentary approaches, use of machine learning, and a two-way discussion between pharmacist and GP after the medication review.
Assuntos
Sistemas de Apoio a Decisões Clínicas , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Idoso , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos/prevenção & controle , Humanos , Revisão de Medicamentos , Farmacêuticos , PolimedicaçãoRESUMO
Since December 2019 and the first reported cases of COVID-19 in Wuhan, China, there have been 199,466,211 confirmed cases of COVID-19 in the World. The WHO defined vaccination hesitancy as one of the top ten threats to global health in 2019. Our objective was thus to identify topics and trends about COVID-19 vaccines from French web forums to understand the perception of the French population on these vaccines before the vaccination campaign started. We performed a topic model analysis on 485 web forums' posts. 10 topics were found. We reviewed 120 posts from 6 of these 10 topics. One topic was about vaccine hesitancy, refusal, and mistrust, and two topics were related to what the users think about the government, the political and economic choices made towards this epidemic.
Assuntos
COVID-19 , Mídias Sociais , Vacinas , Vacinas contra COVID-19 , Humanos , SARS-CoV-2 , Vacinação , Hesitação VacinalRESUMO
BACKGROUND: Adverse drug reactions (ADRs) are statistically characterized within randomized clinical trials and postmarketing pharmacovigilance, but their molecular mechanism remains unknown in most cases. This is true even for hepatic or skin toxicities, which are classically monitored during drug design. Aside from clinical trials, many elements of knowledge about drug ingredients are available in open-access knowledge graphs, such as their properties, interactions, or involvements in pathways. In addition, drug classifications that label drugs as either causative or not for several ADRs, have been established. METHODS: We propose in this paper to mine knowledge graphs for identifying biomolecular features that may enable automatically reproducing expert classifications that distinguish drugs causative or not for a given type of ADR. In an Explainable AI perspective, we explore simple classification techniques such as Decision Trees and Classification Rules because they provide human-readable models, which explain the classification itself, but may also provide elements of explanation for molecular mechanisms behind ADRs. In summary, (1) we mine a knowledge graph for features; (2) we train classifiers at distinguishing, on the basis of extracted features, drugs associated or not with two commonly monitored ADRs: drug-induced liver injuries (DILI) and severe cutaneous adverse reactions (SCAR); (3) we isolate features that are both efficient in reproducing expert classifications and interpretable by experts (i.e., Gene Ontology terms, drug targets, or pathway names); and (4) we manually evaluate in a mini-study how they may be explanatory. RESULTS: Extracted features reproduce with a good fidelity classifications of drugs causative or not for DILI and SCAR (Accuracy = 0.74 and 0.81, respectively). Experts fully agreed that 73% and 38% of the most discriminative features are possibly explanatory for DILI and SCAR, respectively; and partially agreed (2/3) for 90% and 77% of them. CONCLUSION: Knowledge graphs provide sufficiently diverse features to enable simple and explainable models to distinguish between drugs that are causative or not for ADRs. In addition to explaining classifications, most discriminative features appear to be good candidates for investigating ADR mechanisms further.
Assuntos
Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Reconhecimento Automatizado de Padrão , Sistemas de Notificação de Reações Adversas a Medicamentos , Inteligência Artificial , Estudos de Viabilidade , Humanos , FarmacovigilânciaRESUMO
As social media are an interesting source of information for pharmacovigilance, we implemented a novel visualisation method for pharmacovigilance specialists applied to French discussion forums. A word embedding model was trained on posts to facilitate the identification of patterns associated with adverse drug reactions.
Assuntos
Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Mídias Sociais , Sistemas de Notificação de Reações Adversas a Medicamentos , Humanos , FarmacovigilânciaRESUMO
BACKGROUND: Medical terminologies are commonly used in medicine. For instance, to answer a pharmacovigilance question, pharmacovigilance specialists (PVS) search in a pharmacovigilance database for reports in relation to a given drug. To do that, they first need to identify all MedDRA terms that might have been used to code an adverse reaction in the database, but terms may be numerous and difficult to select as they may belong to different parts of the hierarchy. In previous studies, three tools have been developed to help PVS identify and group all relevant MedDRA terms using three different approaches: forms, structured query-builder, and icons. Yet, a poor usability of the tools may increase PVS' workload and reduce their performance. This study aims to evaluate, compare and improve the three tools during two rounds of formative usability evaluation. METHODS: First, a cognitive walkthrough was performed. Based on the design recommendations obtained from this evaluation, designers made modifications to their tools to improve usability. Once this re-engineering phase completed, six PVS took part in a usability test: difficulties, errors and verbalizations during their interaction with the three tools were collected. Their satisfaction was measured through the System Usability Scale. The design recommendations issued from the tests were used to adapt the tools. RESULTS: All tools had usability problems related to the lack of guidance in the graphical user interface (e.g., unintuitive labels). In two tools, the use of the SNOMED CT to find MedDRA terms hampered their use because French PVS were not used to it. For the most obvious and common terms, the icons-based interface would appear to be more useful. For the less frequently used MedDRA terms or those distributed in different parts of the hierarchy, the structured query-builder would be preferable thanks to its great power and flexibility. The form-based tool seems to be a compromise. CONCLUSION: These evaluations made it possible to identify the strengths of each tool but also their weaknesses to address them before further evaluation. Next step is to assess the acceptability of tools and the expressiveness of their results to help identify and group MedDRA terms.
Assuntos
Sistemas de Notificação de Reações Adversas a Medicamentos , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Farmacovigilância , Systematized Nomenclature of Medicine , Humanos , EspecializaçãoRESUMO
While vaccines are intended to protect people from infectious diseases, public confidence in vaccination has evolved as patients have reservation about vaccination, with a major concern about its safety. Social media may help regulatory authorities to better understand opposition to vaccination and make informed decisions for better promotion of vaccines' benefits towards the public. Our objective was to explore French web forums for potential pharmacovigilance signals associated with human papillomavirus infections (HPV) vaccines. Among 138 posts associated with a signal randomly chosen for manual review, 29% were actually adverse drug reactions to the vaccine described in clinical studies, and only 2 were personal experiences. Only 14% of the reviewed posts described positive opinion about the vaccine whereas 46% were neutral and 40% were negative. While few personal experiences of adverse reactions were actually reported by users, our case study showed a large proportion of negative opinions.
Assuntos
Infecções por Papillomavirus , Vacinas contra Papillomavirus , Mídias Sociais , Humanos , Infecções por Papillomavirus/prevenção & controle , Farmacovigilância , VacinaçãoRESUMO
The large-scale use of social media by the population has gained the attention of stakeholders and researchers in various fields. In the domain of pharmacovigilance, this new resource was initially considered as an opportunity to overcome underreporting and monitor the safety of drugs in real time in close connection with patients. Research is still required to overcome technical challenges related to data extraction, annotation, and filtering, and there is not yet a clear consensus concerning the systematic exploration and use of social media in pharmacovigilance. Although the literature has mainly considered signal detection, the potential value of social media to support other pharmacovigilance activities should also be explored. The objective of this paper is to present the main findings and subsequent recommendations from the French research project Vigi4Med, which evaluated the use of social media, mainly web forums, for pharmacovigilance activities. This project included an analysis of the existing literature, which contributed to the recommendations presented herein. The recommendations are categorized into three categories: ethical (related to privacy, confidentiality, and follow-up), qualitative (related to the quality of the information), and quantitative (related to statistical analysis). We argue that the progress in information technology and the societal need to consider patients' experiences should motivate future research on social media surveillance for the reinforcement of classical pharmacovigilance.
Assuntos
Sistemas de Notificação de Reações Adversas a Medicamentos , Farmacovigilância , Mídias Sociais , França , Humanos , Projetos de PesquisaRESUMO
Information relevant to pharmacogenomics studies is available in several open databases, which makes it difficult to synthetize the available data. Within the PractikPharma project, several databases were integrated to PGxLOD, a resource dedicated to the generation and verification of pharmacogenomic influence on drug responses. The Comparative Toxicogenomic Database (CTD) describes the toxic effects of many chemicals on living species based on the literature. Since drugs are peculiar chemicals and side effects are peculiar toxic effects, we aimed at extracting information from CTD that matches drug side effects in the human specie.
Assuntos
Doença/etiologia , Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos , Substâncias Perigosas/toxicidade , Farmacogenética , Toxicogenética , Bases de Dados Factuais , Doença/genética , Humanos , Pesquisa , Integração de SistemasRESUMO
This poster presents a non-exhaustive study of machine learning classification algorithms on pharmacovigilance data. In this study, we have taken into account the patient's clinical data such as medical history, medications taken and their indications for prescriptions, and the observed side effects. From these elements we determine whether the patient case is considered serious or not. We show the performances of the different algorithms by their precision, recall and accuracy as well as their learning curves.