The Reporting of a Disproportionality Analysis for Drug Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Development and Statement.

BACKGROUND AND AIM: Disproportionality analyses using reports of suspected adverse drug reactions are the most commonly used quantitative methods for detecting safety signals in pharmacovigilance. However, their methods and results are generally poorly reported in published articles and existing guidelines do not capture the specific features of disproportionality analyses. We here describe the development of a guideline (REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance [READUS-PV]) for reporting the results of disproportionality analyses in articles and abstracts. METHODS: We established a group of 34 international experts from universities, the pharmaceutical industry, and regulatory agencies, with expertise in pharmacovigilance, disproportionality analyses, and assessment of safety signals. We followed a three-step process to develop the checklist: (1) an open-text survey to generate a first list of items; (2) an online Delphi method to select and rephrase the most important items; (3) a final online consensus meeting. RESULTS: Among the panel members, 33 experts responded to round 1 and 30 to round 2 of the Delphi and 25 participated to the consensus meeting. Overall, 60 recommendations for the main body of the manuscript and 13 recommendations for the abstracts were retained by participants after the Delphi method. After merging of some items together and the online consensus meeting, the READUS-PV guidelines comprise a checklist of 32 recommendations, in 14 items, for the reporting of disproportionality analyses in the main body text and four items, comprising 12 recommendations, for abstracts. CONCLUSIONS: The READUS-PV guidelines will support authors, editors, peer-reviewers, and users of disproportionality analyses using individual case safety report databases. Adopting these guidelines will lead to more transparent, comprehensive, and accurate reporting and interpretation of disproportionality analyses, facilitating the integration with other sources of evidence.

The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration.

In pharmacovigilance, disproportionality analyses based on individual case safety reports are widely used to detect safety signals. Unfortunately, publishing disproportionality analyses lacks specific guidelines, often leading to incomplete and ambiguous reporting, and carries the risk of incorrect conclusions when data are not placed in the correct context. The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) statement was developed to address this issue by promoting transparent and comprehensive reporting of disproportionality studies. While the statement paper explains in greater detail the procedure followed to develop these guidelines, with this explanation paper we present the 14 items retained for READUS-PV guidelines, together with an in-depth explanation of their rationale and bullet points to illustrate their practical implementation. Our primary objective is to foster the adoption of the READUS-PV guidelines among authors, editors, peer reviewers, and readers of disproportionality analyses. Enhancing transparency, completeness, and accuracy of reporting, as well as proper interpretation of their results, READUS-PV guidelines will ultimately facilitate evidence-based decision making in pharmacovigilance.

A Comparison of Signals of Designated Medical Events and Non-designated Medical Events: Results from a Scoping Review.

INTRODUCTION AND OBJECTIVE: The European Medicines Agency (EMA) maintains a list of designated medical events (DMEs), events that are inherently serious and are prioritized for signal detection, irrespective of statistical criteria. We have analysed the results of our previously published scoping review to determine whether DME signals differ from those of other adverse events in terms of time to communication and characteristics of supporting reports of suspected adverse drug reactions. METHODS: For all signals, we obtained the launch year of medicinal products from textbooks or regulatory agencies, extracted the year of the first report in VigiBase and calculated the interval between the first report and communication (time to communication, TTC). We further retrieved the average completeness (via vigiGrade) of the reports in each case series in the years before the communication. We categorised as DME signals those concerning an event in the EMA's list. We described the two groups of signals using medians and interquartile ranges (IQR) and compared them using the Brunner-Munzel test, calculating 95% confidence intervals (95% CI) and P values. RESULTS: Of 4520 signals, 919 concerned DMEs and 3601 concerned non-DMEs. Signals of DMEs were supported by a median of 15 reports (IQR 6-38 reports) with a completeness score of 0.52 (IQR 0.43-0.62) and signals of non-DMEs by 20 reports (IQR 6-84 reports) with a completeness score of 0.46 (IQR 0.38-0.56). The probability that a random DME signal was supported by fewer reports than non-DME signals was 0.56 (95% CI 0.54-0.58, P < 0.001) and that of one having lower average completeness was 0.39 (95% CI 0.36-0.41, P < 0.001). The median TTCs of DME and non-DME signals did not differ (10 years), but the TTC was as low as 2 years when signals (irrespective of classification) were supported by reports whose average completeness was > 0.80. CONCLUSIONS: Signals of designated medical events were supported by fewer reports and higher completeness scores than signals of other adverse events. Although statistically significant, the differences in effect sizes between the two groups were small. This suggests that listing certain adverse events as DMEs is not having the expected effect of encouraging a focus on reports of the types of suspected adverse reactions that deserve special attention. Further enhancing the completeness of the reports of suspected adverse drug reactions supporting signals of designated medical events might shorten their time to communication and reduce the number of reports required to support them.

Supporting Pharmacovigilance Signal Validation and Prioritization with Analyses of Routinely Collected Health Data: Lessons Learned from an EHDEN Network Study.

INTRODUCTION: Individual case reports are the main asset in pharmacovigilance signal management. Signal validation is the first stage after signal detection and aims to determine if there is sufficient evidence to justify further assessment. Throughout signal management, a prioritization of signals is continually made. Routinely collected health data can provide relevant contextual information but are primarily used at a later stage in pharmacoepidemiological studies to assess communicated signals. OBJECTIVE: The aim of this study was to examine the feasibility and utility of analysing routine health data from a multinational distributed network to support signal validation and prioritization and to reflect on key user requirements for these analyses to become an integral part of this process. METHODS: Statistical signal detection was performed in VigiBase, the WHO global database of individual case safety reports, targeting generic manufacturer drugs and 16 prespecified adverse events. During a 5-day study-a-thon, signal validation and prioritization were performed using information from VigiBase, regulatory documents and the scientific literature alongside descriptive analyses of routine health data from 10 partners of the European Health Data and Evidence Network (EHDEN). Databases included in the study were from the UK, Spain, Norway, the Netherlands and Serbia, capturing records from primary care and/or hospitals. RESULTS: Ninety-five statistical signals were subjected to signal validation, of which eight were considered for descriptive analyses in the routine health data. Design, execution and interpretation of results from these analyses took up to a few hours for each signal (of which 15-60 minutes were for execution) and informed decisions for five out of eight signals. The impact of insights from the routine health data varied and included possible alternative explanations, potential public health and clinical impact and feasibility of follow-up pharmacoepidemiological studies. Three signals were selected for signal assessment, two of these decisions were supported by insights from the routine health data. Standardization of analytical code, availability of adverse event phenotypes including bridges between different source vocabularies, and governance around the access and use of routine health data were identified as important aspects for future development. CONCLUSIONS: Analyses of routine health data from a distributed network to support signal validation and prioritization are feasible in the given time limits and can inform decision making. The cost-benefit of integrating these analyses at this stage of signal management requires further research.

Signals of Adverse Drug Reactions Communicated by Pharmacovigilance Stakeholders: A Scoping Review of the Global Literature.

INTRODUCTION AND OBJECTIVE: Signals of adverse drug reactions (ADRs) can be supported by reports of ADRs and by interventional and non-interventional studies. The evidence base and features of ADR reports that are used to support signals remain to be comprehensively described. To this end, we have undertaken a scoping review. METHODS: We searched the following databases: PubMed, EMBASE, PsycINFO, Web of Science, and Google Scholar, without language or time restrictions. We also hand searched the bibliographies of relevant studies. We included studies of any design if the results were described as signals. We assessed the levels of evidence using the Oxford Centre for Evidence-Based Medicine (OCEBM) criteria and coded features of reports of ADRs using the Bradford Hill guidelines. RESULTS: Overall, 1974 publications reported 2421 studies of signals; 1683/2421 were clinical assessments of anecdotal reports of ADRs, but only 225 (13%) of these included explicit judgments on which features of the ADR reports were supportive of a signal. These 225 studies yielded 228 signals; these were supported by features, which were: 'experimental evidence' (i.e., positive dechallenge or rechallenge, 154 instances [68%]), 'temporality' (i.e., time to onset, 130 [57%]), 'exclusion of competing causes' (49 [21%]), and others (40 [17%]). Positive dechallenge/rechallenge often co-occurred with temporality (77/228). OCEBM 4 (i.e., case series and case-control studies) was the most frequent level of evidence (2078 studies). Between 2013 and 2019, there was a three-fold increase in clinical assessments of reports of ADRs compared with a less than two-fold increase in studies supported by higher levels of evidence (i.e., OCEBM 1-3). We identified an increased rate between 2013 and 2019 in disproportionality analyses (about 15 studies per year), mostly from academia. CONCLUSIONS: Most signals were supported by temporality and dechallenge/rechallenge, but clear reporting of judgments on causality remains infrequent. The number of studies supported only by anecdotal reports of ADRs increased from year to year. The impact of a growing number of signals of disproportionate reporting communicated without an accompanying clinical assessment should be evaluated.

Post-Marketing Safety Profile of Vortioxetine Using a Cluster Analysis and a Disproportionality Analysis of Global Adverse Event Reports.

INTRODUCTION: Vortioxetine, a multimodal serotonergic drug, is widely used as treatment for major depressive disorder. Although on the market since late 2013, the data of the relative safety of vortioxetine, especially compared to selective serotonin reuptake inhibitors, are still scarce. OBJECTIVE: The aim of this study was to explore the adverse event reporting pattern of vortioxetine through a cluster analysis. Furthermore, to compare the adverse event reporting pattern for vortioxetine with that of the selective serotonin reuptake inhibitors. METHODS: Individual case safety reports for vortioxetine in VigiBase up to 1 November, 2019 were subjected to consensus clustering, to identify and describe natural groupings of reports based on their reported adverse events. A vigiPoint exploratory analysis compared vortioxetine to the selective serotonin reuptake inhibitors in terms of relative frequencies for a wide range of covariates, including patient sex and age, reported drugs and adverse events, and reporting country. Important differences were identified using odds ratios with adaptive statistical shrinkage. RESULTS: Thirty-six clusters containing at least five reports were identified and analysed. The two largest clusters included 48% of the vortioxetine reports and appeared to represent gastrointestinal adverse events and hypersensitivity adverse events. Other distinct clusters were related to, respectively, fatigue, aggression/suicidality, convulsion, medication errors, arthralgia/myalgia, increased weight, paraesthesia and anticholinergic effects. Some of these clusters are not labelled for vortioxetine, such as arthralgia/myalgia and paraesthesia, but are known adverse events for selective serotonin reuptake inhibitors. A vigiPoint analysis revealed a higher proportion of reports from consumers and non-health professionals for vortioxetine as well as higher relative reporting rates of gastrointestinal symptoms, pruritus and mood-related symptoms, consistent with the cluster analysis. CONCLUSIONS: A pattern of co-reported adverse events that is consistent with labelled adverse events for vortioxetine and the safety profile for selective serotonin reuptake inhibitors in general was revealed. Clusters of unlabelled adverse events were identified that reflect clinical entities that might represent signals of previously unknown adverse events. More extensive analyses of spontaneous reports may help to further understand the reporting pattern of adverse events.

Consensus clustering for case series identification and adverse event profiles in pharmacovigilance.

OBJECTIVE: To describe and evaluate vigiGroup - a consensus clustering algorithm which can identify groups of individual case reports referring to similar suspected adverse drug reactions and describe associated adverse event profiles, accounting for co-reported adverse event terms. MATERIALS AND METHODS: Consensus clustering is achieved by grouping pairs of reports that are repeatedly placed together in the same clusters across a set of mixture model-based cluster analyses. The latter use empirical Bayes statistical shrinkage for improved performance. As baseline comparison, we considered a regular mixture model-based cluster analysis. Three randomly selected drugs in VigiBase, the World Health Organization's global database of Individual Case Safety Reports were analyzed: sumatriptan, ambroxol and tacrolimus. Clustering stability was assessed using the adjusted Rand index, ranging between -1 and +1, and clinical coherence was assessed through an intruder detection analysis. RESULTS: For the three drugs considered, vigiGroup achieved stable and coherent results with adjusted Rand indices between +0.80 and +0.92, and intruder detection rates between 86% and 94%. Consensus clustering improved both stability and clinical coherence compared to mixture model-based clustering alone. Statistical shrinkage improved the stability of clusters compared to the baseline mixture model, as well as the cross-validated log-likelihood. CONCLUSIONS: The proposed algorithm can achieve adequate stability and clinical coherence in clustering individual case reports, thereby enabling better identification of case series and associated adverse event profiles in pharmacovigilance. The use of empirical Bayes shrinkage and consensus clustering each led to meaningful improvements in performance.

Interstitial Lung Disease as an Adverse Drug Reaction in Japan: Exploration of Regulatory Actions as a Basis for High Reporting.

INTRODUCTION: Increased post-marketing reports of interstitial lung disease in Japan have been recognized. An understanding of its regional groundings can be important for the global pharmacovigilance community. OBJECTIVE: The objective of this study was to explore the correlation between high rates of interstitial lung disease reporting and regulatory actions in Japan. METHODS: Post-marketing interstitial lung disease-related label changes and interstitial lung disease reports were classified by the anatomical therapeutic chemical classification groups of the suspected drugs. Regulatory actions for the top interstitial lung disease-reporting drugs were compared. The interstitial lung disease reporting patterns of protein kinase inhibitors were compared to those of methotrexate. RESULTS: Interstitial lung disease-related label changes predominantly occurred for drugs in the anatomical therapeutic chemical classification groups L, J, C, and herbal medicines. Interstitial lung disease was reported most frequently for L group, especially for the protein kinase inhibitors. The regulatory actions for those drugs with the highest number of interstitial lung disease reports (methotrexate, protease kinase inhibitors, gemcitabine, docetaxel) plus monoclonal antibodies were analyzed. The ratio of interstitial lung disease reports to all reports over time was initially high in the re-examination period, while it was constantly low after the period expired. The increase in interstitial lung disease reporting was observed for the drugs for which interstitial lung disease was designated as a priority item in the use-results survey. Methotrexate had more interstitial lung disease reports with multiple suspected drugs and fewer reports with high completeness than the protease kinase inhibitors. CONCLUSIONS: The high rates of interstitial lung disease reporting derived from mainly the anatomical therapeutic chemical classification group L drugs. Interstitial lung disease is the targeted adverse drug reaction in the use-results survey mandated in the re-examination of those drugs. This system provides at least one explanation for the high reporting of interstitial lung disease in Japan.

Correction to: Communicating Adverse Drug Reaction Insights Through Patient Organizations: Experiences from a Pilot Study in the Netherlands.

The article "Communicating Adverse Drug Reaction Insights Through Patient Organizations: Experiences from a Pilot Study in the Netherlands", written by Linda Härmark, Gerda Weits, Rietje Meijer, Federica Santoro, G. Niklas Norén, Florence van Hunsel, was originally published electronically on 16 May 2020 without open access.

A Feasibility Study of Drug-Drug Interaction Signal Detection in Regular Pharmacovigilance.

INTRODUCTION: Adverse drug reactions related to drug-drug interactions cause harm to patients. There is a body of research on signal detection for drug interactions in collections of individual case reports, but limited use in regular pharmacovigilance. OBJECTIVE: The aim of this study was to evaluate the feasibility of signal detection of drug-drug interactions in collections of individual case reports of suspected adverse drug reactions. METHODS: This study was conducted in VigiBase, the WHO global database of individual case safety reports. The data lock point was 31 August 2016, which provided 13.6 million reports for analysis after deduplication. Statistical signal detection was performed using a previously developed predictive model for possible drug interactions. The model accounts for an interaction disproportionality measure, expressed suspicion of an interaction by the reporter, potential for interaction through cytochrome P450 activity of drugs, and reported information indicative of unexpected therapeutic response or altered therapeutic effect. Triage filters focused the preliminary signal assessment on combinations relating to serious adverse events with case series of no more than 30 reports from at least two countries, with at least one report during the previous 2 years. Additional filters sought to eliminate already known drug interactions through text mining of standard literature sources. Preliminary signal assessment was performed by a multidisciplinary group of pharmacovigilance professionals from Uppsala Monitoring Centre and collaborating organizations, whereas in-depth signal assessment was performed by experienced pharmacovigilance assessors. RESULTS: We performed preliminary signal assessment for 407 unique drug pairs. Of these, 157 drug pairs were considered already known to interact, whereas 232 were closed after preliminary assessment for other reasons. Ten drug pairs were subjected to in-depth signal assessment and an additional eight were decided to be kept under review awaiting additional reports. The triage filters had a major impact in focusing our preliminary signal assessment on just 14% of the statistical signals generated by the predictive model for drug interactions. In-depth assessment led to three signals communicated with the broader pharmacovigilance community, six closed signals and one to be kept under review. CONCLUSION: This study shows that signals of adverse drug interactions can be detected through broad statistical screening of individual case reports. It further shows that signal assessment related to possible drug interactions requires more detailed information on the temporal relationship between different drugs and the adverse event. Future research may consider whether interaction signal detection should be performed not for individual adverse event terms but for pairs of drugs across a spectrum of adverse events.

Risk Factor Considerations in Statistical Signal Detection: Using Subgroup Disproportionality to Uncover Risk Groups for Adverse Drug Reactions in VigiBase.

INTRODUCTION: In the treatment of the individual patient, a vision is to achieve the best possible balance between benefit and harm. Such tailored therapy relies upon the identification and characterisation of risk factors for adverse drug reactions. Information relevant to risk factor considerations can be captured in adverse event reports and could be utilised in statistical signal detection. OBJECTIVE: The aim of this study was to explore whether statistical screening of a broad range of risk factors within a global database of adverse event reports could uncover signals of risk groups for adverse drug reactions. METHODS: Subgroup disproportionality analysis was applied to 15.4 million reports entered in VigiBase, the World Health Organization (WHO) global database of individual case safety reports, up to August 2017. Disproportionality analyses for drug-adverse event pairs were performed (1) in the full database and (2) across a range of subgroups defined by the following covariates: patient age, sex, body mass index, pregnancy, underlying condition, reporting country, and geographical region. Drug-adverse event pairs disproportionately over-reported in such subgroups, but not in the full database, and with a substantial difference between the two observed-to-expected ratios, were highlighted as statistical signals. These were further prioritised, through filtering and sorting, for clinical assessment, whereafter clinically relevant signals were communicated to the pharmacovigilance community and the public. RESULTS: Assessments were performed for 354 prioritised statistical signals, resulting in seven communicated signals describing previously unrecognised potential risk groups related to age (elderly), sex (male and female), body mass index (underweight and obese), and geographical region (Asia), all except one for already established adverse drug reactions. Important aspects considered in the assessments included an evaluation of the disproportionate over-reporting in the subgroup by reviewing alternative explanations and reporting patterns for similar drugs/adverse events/subgroups, and a search for plausible mechanisms to support the risk hypothesis. CONCLUSIONS: This study reveals that it is possible to uncover signals of risk groups for adverse drug reactions through incorporation of broad risk factor screening into statistical signal detection in a global database of adverse event reports. Our findings suggest the potential to use such statistical methodologies for risk characterisation in subpopulations of concern.

Communicating Adverse Drug Reaction Insights Through Patient Organizations: Experiences from a Pilot Study in the Netherlands.

INTRODUCTION: To improve therapeutic decision making, it is crucial that information regarding adverse drug reactions reaches patients. It is not enough to disseminate such findings through regulatory and scientific channels; targeted efforts to reach patients are necessary. One possible avenue is to collaborate with patient organizations. OBJECTIVES: The aim of this pilot study was to explore how adverse drug reactions can be communicated through patient organizations. METHODS: A text describing a signal of levothyroxine and panic attacks was tailored to patients' needs, in terms of language, style and content, with emphasis placed on what to do when experiencing the symptoms described. The signal was communicated via the Dutch thyroid organization's digital newsletter, social media channels, website and print magazine. RESULTS: The digital newsletter was distributed to around 5000 subscribers. On Facebook, 13,820 people viewed the message, with 2346 clicks in the message, indicating an intention to read the whole post. The interactions on social media were positive, and the tone was respectful. CONCLUSION: Patient organizations can help enable effective communication of adverse drug reactions to a relevant audience. The social media post generated more engagement than other communications from the patient organization, indicating a strong interest in this information. The additional patient experiences that were shared in the comments on social media further strengthened the original signal and its relevance to patients, creating an interesting feedback loop. The favourable experiences in this study support further consideration and exploration of this approach to communicate adverse drug reactions to patients.

Data-Driven Identification of Adverse Event Reporting Patterns for Japan in VigiBase, the WHO Global Database of Individual Case Safety Reports.

INTRODUCTION: Adverse event reporting patterns vary between countries, reflecting differences in reporting culture, clinical practice and underlying patient populations. Japan collects about 60,000 domestic adverse event reports yearly and shares serious reports with the World Health Organization (WHO) Programme for International Drug Monitoring in VigiBase, the WHO global database of individual case safety reports. Understanding these reports in the global context can be helpful for regulators worldwide and can aid hypothesis-generation for Japanese-specific vulnerabilities to adverse drug reactions. OBJECTIVE: The objective of this study was to explore differences in the reporting of adverse events between Japan and other countries. METHODS: vigiPoint is a method for data-driven exploration in pharmacovigilance. It outlines data subsets, pinpoints key features and facilitates expert review, using odds ratios subjected to statistical shrinkage to distinguish one data subset from another. Here, we compared 260,000 Japanese reports in E2B format classified as serious and received in VigiBase between 2013 and 2018 with 2.5 million reports from the rest of the world (of which 51% are from the USA). Reporting patterns for which the 99% credibility interval of the shrunk log-odds ratios were above 0.5 or below - 0.5 were flagged as key features. The shrinkage was set to the vigiPoint default corresponding to 1% of the size of the Japanese data subset. As a sensitivity analysis, additional vigiPoint comparisons were performed between Japan and, in turn, Africa, the Americas, the Americas except the USA and Canada, Asia and Europe. RESULTS: There were higher reporting rates in Japan from physicians (83% vs. 39%) and pharmacists (17% vs. 10%). It was also more common to see reports with more than five drugs per report (22% vs. 14%) and with a single adverse event (72% vs. 45%). More than half of the Japanese reports had a vigiGrade completeness score above 0.8 compared with about one in five from the rest of the world. There were more reports than expected for patients aged 70-89 years and fewer reports for adults aged 20-59 years. Adverse events reported more often in Japan included interstitial lung disease, abnormal hepatic function, decreased platelet count, decreased neutrophil count and drug eruption. Adverse events reported less often included death, fatigue, dyspnoea, pain and headache. Drugs reported more often in Japan included prednisolone, methotrexate and peginterferon alfa-2b. Drugs reported less often included rosiglitazone and adalimumab as well as blood substitutes and perfusion solutions. The findings were generally robust to the sensitivity analysis except for the less often reported drugs, many of which were rarely reported in most countries, except in the USA. CONCLUSION: Analysis of Japanese adverse event reporting patterns in a global context has revealed key features that may reflect possible pharmaco-ethnic vulnerabilities in the Japanese, as well as differences in adverse event reporting and clinical practice. This knowledge is essential in the global collaboration of signal detection afforded by the WHO Programme for International Drug Monitoring.

Recommendations for the Use of Social Media in Pharmacovigilance: Lessons from IMI WEB-RADR.

Over a period of 3 years, the European Union's Innovative Medicines Initiative WEB-RADR project has explored the value of social media (i.e., information exchanged through the internet, typically via online social networks) for identifying adverse events as well as for safety signal detection. Many patients and clinicians have taken to social media to discuss their positive and negative experiences of medications, creating a source of publicly available information that has the potential to provide insights into medicinal product safety concerns. The WEB-RADR project has developed a collaborative English language workspace for visualising and analysing social media data for a number of medicinal products. Further, novel text and data mining methods for social media analysis have been developed and evaluated. From this original research, several recommendations are presented with supporting rationale and consideration of the limitations. Recommendations for further research that extend beyond the scope of the current project are also presented.

Correction to: vigiGrade: A Tool to Identify Well-Documented Individual Case Reports and Highlight Systematic Data Quality Issues.

The article vigiGrade: A Tool to Identify Well-Documented Individual Case Reports and Highlight Systematic Data Quality Issues, written by Tomas Bergvall. G. Niklas Norén. Marie Lindquist, was originally published Online First without open access.

Assessment of the Utility of Social Media for Broad-Ranging Statistical Signal Detection in Pharmacovigilance: Results from the WEB-RADR Project.

INTRODUCTION AND OBJECTIVE: Social media has been proposed as a possibly useful data source for pharmacovigilance signal detection. This study primarily aimed to evaluate the performance of established statistical signal detection algorithms in Twitter/Facebook for a broad range of drugs and adverse events. METHODS: Performance was assessed using a reference set by Harpaz et al., consisting of 62 US Food and Drug Administration labelling changes, and an internal WEB-RADR reference set consisting of 200 validated safety signals. In total, 75 drugs were studied. Twitter/Facebook posts were retrieved for the period March 2012 to March 2015, and drugs/events were extracted from the posts. We retrieved 4.3 million and 2.0 million posts for the WEB-RADR and Harpaz drugs, respectively. Individual case reports were extracted from VigiBase for the same period. Disproportionality algorithms based on the Information Component or the Proportional Reporting Ratio and crude post/report counting were applied in Twitter/Facebook and VigiBase. Receiver operating characteristic curves were generated, and the relative timing of alerting was analysed. RESULTS: Across all algorithms, the area under the receiver operating characteristic curve for Twitter/Facebook varied between 0.47 and 0.53 for the WEB-RADR reference set and between 0.48 and 0.53 for the Harpaz reference set. For VigiBase, the ranges were 0.64-0.69 and 0.55-0.67, respectively. In Twitter/Facebook, at best, 31 (16%) and four (6%) positive controls were detected prior to their index dates in the WEB-RADR and Harpaz references, respectively. In VigiBase, the corresponding numbers were 66 (33%) and 17 (27%). CONCLUSIONS: Our results clearly suggest that broad-ranging statistical signal detection in Twitter and Facebook, using currently available methods for adverse event recognition, performs poorly and cannot be recommended at the expense of other pharmacovigilance activities.

A method for data-driven exploration to pinpoint key features in medical data and facilitate expert review.

PURPOSE: To develop a method for data-driven exploration in pharmacovigilance and illustrate its use by identifying the key features of individual case safety reports related to medication errors. METHODS: We propose vigiPoint, a method that contrasts the relative frequency of covariate values in a data subset of interest to those within one or more comparators, utilizing odds ratios with adaptive statistical shrinkage. Nested analyses identify higher order patterns, and permutation analysis is employed to protect against chance findings. For illustration, a total of 164 000 adverse event reports related to medication errors were characterized and contrasted to the other 7 833 000 reports in VigiBase, the WHO global database of individual case safety reports, as of May 2013. The initial scope included 2000 features, such as patient age groups, reporter qualifications, and countries of origin. RESULTS: vigiPoint highlighted 109 key features of medication error reports. The most prominent were that the vast majority of medication error reports were from the United States (89% compared with 49% for other reports in VigiBase); that the majority of reports were sent by consumers (53% vs 17% for other reports); that pharmacists (12% vs 5.3%) and lawyers (2.9% vs 1.5%) were overrepresented; and that there were more medication error reports than expected for patients aged 2-11 years (10% vs 5.7%), particularly in Germany (16%). CONCLUSIONS: vigiPoint effectively identified key features of medication error reports in VigiBase. More generally, it reduces lead times for analysis and ensures reproducibility and transparency. An important next step is to evaluate its use in other data.

vigiRank for statistical signal detection in pharmacovigilance: First results from prospective real-world use.

PURPOSE: vigiRank is a data-driven predictive model for emerging safety signals. In addition to disproportionate reporting patterns, it also accounts for the completeness, recency, and geographic spread of individual case reporting, as well as the availability of case narratives. Previous retrospective analysis suggested that vigiRank performed better than disproportionality analysis alone. The purpose of the present analysis was to evaluate its prospective performance. METHODS: The evaluation of vigiRank was based on real-world signal detection in VigiBase. In May 2014, vigiRank scores were computed for pairs of new drugs and WHO Adverse Reaction Terminology critical terms with at most 30 reports from at least 2 countries. Initial manual assessments were performed in order of descending score, selecting a subset of drug-adverse drug reaction pairs for in-depth expert assessment. The primary performance metric was the proportion of initial assessments that were decided signals during in-depth assessment. As comparator, the historical performance for disproportionality- guided signal detection in VigiBase was computed from a corresponding cohort of drug-adverse drug reaction pairs assessed between 2009 and 2013. During this period, the requirement for initial manual assessment was a positive lower endpoint of the 95% credibility interval of the Information Component measure of disproportionality, observed for the first time. RESULTS: 194 initial assessments suggested by vigiRank's ordering eventually resulted in 6 (3.1%) signals. Disproportionality analysis yielded 19 signals from 1592 initial assessments (1.2%; P < .05). CONCLUSIONS: Combining multiple strength-of-evidence aspects as in vigiRank significantly outperformed disproportionality analysis alone in real-world pharmacovigilance signal detection, for VigiBase.