Automated Drug Coding Using Artificial Intelligence: An Evaluation of WHODrug Koda on Adverse Event Reports.

INTRODUCTION: Coding medicinal products described on adverse event (AE) reports to specific entries in standardised drug dictionaries, such as WHODrug Global, is a time-consuming step in case processing activities despite its potential for automation. Many organisations are already partially automating drug coding using text-processing methods and synonym lists, however addressing challenges such as misspellings, abbreviations or ambiguous trade names requires more advanced methods. WHODrug Koda is a drug coding engine using text-processing algorithms, built-in coding rules and machine learning to code drug verbatims to WHODrug Global. OBJECTIVE: Our aim was to evaluate the drug coding performance of WHODrug Koda on AE reports from VigiBase, the World Health Organization's global database of individual case safety reports, in terms of level of automation and coding quality. METHODS: Koda was evaluated on 4.8 million drug entries from VigiBase. Automation level was computed as the proportion of drug entries automatically coded by Koda and was compared to a simple case-insensitive text-matching algorithm. Coding quality was evaluated in terms of coding accuracy, by comparing Koda's prediction to the WHODrug entries found on the AE reports in VigiBase. To better understand the cases in which Koda's coding results did not match with the WHODrug entries in VigiBase, a manual assessment of 600 samples of disagreeing encodings was performed by two teams of expert drug coders. RESULTS: Compared with a simple direct-match baseline, Koda can increase the automation level from 61% to 89%, while providing high coding quality with an accuracy of 97%. CONCLUSIONS: Even though Koda was designed for use in clinical trials, Koda achieves automation level and coding quality for drug coding of AE reports comparable with the performance observed in a previous evaluation of Koda on clinical trial data. Koda can thus help organisations to automate their drug coding of AE reports to a large degree.

Post-Marketing Safety Profile of Vortioxetine Using a Cluster Analysis and a Disproportionality Analysis of Global Adverse Event Reports.

INTRODUCTION: Vortioxetine, a multimodal serotonergic drug, is widely used as treatment for major depressive disorder. Although on the market since late 2013, the data of the relative safety of vortioxetine, especially compared to selective serotonin reuptake inhibitors, are still scarce. OBJECTIVE: The aim of this study was to explore the adverse event reporting pattern of vortioxetine through a cluster analysis. Furthermore, to compare the adverse event reporting pattern for vortioxetine with that of the selective serotonin reuptake inhibitors. METHODS: Individual case safety reports for vortioxetine in VigiBase up to 1 November, 2019 were subjected to consensus clustering, to identify and describe natural groupings of reports based on their reported adverse events. A vigiPoint exploratory analysis compared vortioxetine to the selective serotonin reuptake inhibitors in terms of relative frequencies for a wide range of covariates, including patient sex and age, reported drugs and adverse events, and reporting country. Important differences were identified using odds ratios with adaptive statistical shrinkage. RESULTS: Thirty-six clusters containing at least five reports were identified and analysed. The two largest clusters included 48% of the vortioxetine reports and appeared to represent gastrointestinal adverse events and hypersensitivity adverse events. Other distinct clusters were related to, respectively, fatigue, aggression/suicidality, convulsion, medication errors, arthralgia/myalgia, increased weight, paraesthesia and anticholinergic effects. Some of these clusters are not labelled for vortioxetine, such as arthralgia/myalgia and paraesthesia, but are known adverse events for selective serotonin reuptake inhibitors. A vigiPoint analysis revealed a higher proportion of reports from consumers and non-health professionals for vortioxetine as well as higher relative reporting rates of gastrointestinal symptoms, pruritus and mood-related symptoms, consistent with the cluster analysis. CONCLUSIONS: A pattern of co-reported adverse events that is consistent with labelled adverse events for vortioxetine and the safety profile for selective serotonin reuptake inhibitors in general was revealed. Clusters of unlabelled adverse events were identified that reflect clinical entities that might represent signals of previously unknown adverse events. More extensive analyses of spontaneous reports may help to further understand the reporting pattern of adverse events.

Consensus clustering for case series identification and adverse event profiles in pharmacovigilance.

OBJECTIVE: To describe and evaluate vigiGroup - a consensus clustering algorithm which can identify groups of individual case reports referring to similar suspected adverse drug reactions and describe associated adverse event profiles, accounting for co-reported adverse event terms. MATERIALS AND METHODS: Consensus clustering is achieved by grouping pairs of reports that are repeatedly placed together in the same clusters across a set of mixture model-based cluster analyses. The latter use empirical Bayes statistical shrinkage for improved performance. As baseline comparison, we considered a regular mixture model-based cluster analysis. Three randomly selected drugs in VigiBase, the World Health Organization's global database of Individual Case Safety Reports were analyzed: sumatriptan, ambroxol and tacrolimus. Clustering stability was assessed using the adjusted Rand index, ranging between -1 and +1, and clinical coherence was assessed through an intruder detection analysis. RESULTS: For the three drugs considered, vigiGroup achieved stable and coherent results with adjusted Rand indices between +0.80 and +0.92, and intruder detection rates between 86% and 94%. Consensus clustering improved both stability and clinical coherence compared to mixture model-based clustering alone. Statistical shrinkage improved the stability of clusters compared to the baseline mixture model, as well as the cross-validated log-likelihood. CONCLUSIONS: The proposed algorithm can achieve adequate stability and clinical coherence in clustering individual case reports, thereby enabling better identification of case series and associated adverse event profiles in pharmacovigilance. The use of empirical Bayes shrinkage and consensus clustering each led to meaningful improvements in performance.