Individual Case Safety Report Replication: An Analysis of Case Reporting Transmission Networks.

INTRODUCTION: The basis of pharmacovigilance is provided by the exchange of Individual Case Safety Reports (ICSRs) between the recipient of the original report and other interested parties, which include Marketing Authorization Holders (MAHs) and Health Authorities (HAs). Different regulators have different reporting requirements for report transmission. This results in replication of each ICSR that will exist in multiple locations. Adding in the fact that each case will go through multiple versions, different recipients may receive different case versions at different times, potentially influencing patient safety decisions and potentially amplifying or obscuring safety signals inappropriately. OBJECTIVE: The present study aimed to investigate the magnitude of replication, the variability among recipients, and the subsequent divergence across recipients of ICSRs. METHODS: Seven participating TransCelerate Member Companies (MCs) queried their respective safety databases covering a 3-year period and provided aggregate ICSR submission statistics for expedited safety reports to an independent project manager. As measured in the US Food and Drug Administration (FDA)'s Adverse Event Reporting System (FAERS), ICSR volume for these seven MCs makes up approximately 20% of the total case volume. Aggregate metrics were calculated from the company data, specifically: (i) number of ICSR transmissions, (ii) average number of recipients (ANR) per case version transmitted, (iii) a submission selectivity metric, which measures the percentage of recipients not having received all sequential case version numbers, and (iv) percent of common ISCRs residing in two or more MAH databases. RESULTS: The analysis reflects 2,539,802 case versions, distributed through 7,602,678 submissions. The overall mean replication rate is 3.0 submissions per case version. The distribution of the ANR replication measure was observed to be very long-tailed, with a significant fraction of case versions (~ 12.4% of all transmissions) being sent to ten or more HA recipients. Replication is higher than average for serious, unlisted, and literature cases, ranging from 3.5 to 6.1 submissions per version. Within the subset of ICSR versions sent to three recipients, a significant degree of variability in the actual recipients (i.e., HAs) was observed, indicating that there is not one single combination of the same three HAs predominantly receiving an ICSR. Submission selectivity increases with the case version. For case version 6, the range of the submission selectivity for the MAHs ranges from ~ 10% to over 50%, with a median of 30.2%. Within the participating MAHs, the percentage of cases that reside within at least two safety databases is approximately 2% across five databases. Further analysis of the data from three MAHs showed percentages of 13.4%, 15.6%, and 27.9% of ICSRs originating from HAs and any other partners such as other MAHs and other institutions. CONCLUSION: Replication of ICSRs and the variation of available safety information in recipient databases were quantified and shown to be substantial. Our work shows that multiple processors and medical reviewers will likely handle the same original ICSR as a result of replication. Aside from the obvious duplicate work, this phenomenon could conceivably lead to differing clinical assessments and decisions. If replication could be reduced or even eliminated, this would enable more focus on activities with a benefit for patient safety.

Leveraging Machine Learning to Facilitate Individual Case Causality Assessment of Adverse Drug Reactions.

INTRODUCTION: Causality assessment of individual case safety reports (ICSRs) is an important step in pharmacovigilance case-level review and aims to establish a position on whether a patient's exposure to a drug is causally related to the patient experiencing an untoward adverse event. There are many different approaches for case causality adjudication, including the use of expert opinions and algorithmic frameworks; however, a great deal of variability exists between assessment methods, products, therapeutic classes, individual physicians, change of process and conventions over time, and other factors. OBJECTIVE: The objective of this study was to develop a machine learning-based model that can predict the likelihood of a causal association of an observed drug-reaction combination in an ICSR. METHODS: In this study, we used a set of annotated solicited ICSRs (50K cases) from a company post-marketing database. These data were enriched with novel supplementary features from external and internal data sources that aim to capture facets such as temporal plausibility, scientific validity, and confoundedness that have been shown to contribute to causality adjudication. Using these features, we constructed a Bayesian network (BN) model to predict drug-event pair causality assessment. BN topology was driven by an internally developed ICSR causality decision support tool. Performance of the model was evaluated through examination of sensitivity, positive predictive value (PPV), and the area under the receiver operating characteristic curve (AUC) on an independent set of data from a temporally adjacent interval (20K cases). No external validation was performed because of a lack of publicly available ICSRs with causality assessments for drug-event pairs. RESULTS: The model demonstrated high performance in predicting the causality assessment of drug-event pairs compared with clinical judgment using global introspection (AUC 0.924; 95% confidence interval [CI] 0.922-0.927). The sensitivity of the model was 0.900 (95% CI 0.896-0.904), and the PPV of the model was 0.778 (95% CI 0.773-0.783). CONCLUSION: These results show that robust probabilistic modeling of ICSR causality is feasible, and the approach used in the development of the model can serve as a framework for such causality assessments, leading to improvements in safety decision making.

Investigating Overlap in Signals from EVDAS, FAERS, and VigiBase®.

INTRODUCTION: The Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) and VigiBase® are two established databases for safety monitoring of medicinal products, recently complemented with the EudraVigilance Data Analysis System (EVDAS). OBJECTIVE: Signals of disproportionate reporting (SDRs) can characterize the reporting profile of a drug, accounting for the distribution of all drugs and all events in the database. This study aims to quantify the redundancy among the three databases when characterized by two disproportionality-based analyses (DPA). METHODS: SDRs for 100 selected products were identified with two sets of thresholds (standard EudraVigilance SDR criteria for all vs Bayesian approach for FAERS and VigiBase®). Per product and database, the presence or absence of SDRs was determined and compared. Adverse events were considered at three levels: MedDRA® Preferred Term (PT), High Level Term (HLT), and HLT combined with Standardized MedDRA® Query (SMQ). Redundancy was measured in terms of recall (SDRs in EVDAS divided by SDRs from any database) and overlap (SDRs in EVDAS and at least one other database, divided by SDRs in EVDAS). Covariates with potential impact on results were explored with linear regression models. RESULTS: The median overlap between EVDAS and FAERS or VigiBase® was 85.0% at the PT level, 94.5% at the HLT level, and 97.7% at the HLT or SMQ level. The corresponding median recall of signals in EVDAS as a percentage of all signals generated in all three databases was 59.4%, 74.1%, and 87.9% at the PT, HLT, and HLT or SMQ levels, respectively. The overlap difference is partially explained by the relative number of EU cases in EudraVigilance and the ratio of EVDAS cases and FAERS cases, presumably due to differences in marketing authorizations, or market penetration in different regions. Products with few cases in EVDAS (< 1500) also display limited recall of signals relative to FAERs/VigiBase®. Time-on-market does not predict signal redundancy between the three databases. The choice of the DPA has an expected but somewhat small effect on redundancy. CONCLUSIONS: Organizations typically consider regulatory expectations, operating performance (e.g., positive predictive value), and procedural complexity when selecting databases for signal management. As SDRs can be seen as a proxy of general reporting characteristics identifiable in a systematic screening process, our results indicate that, for most products, these characteristics are largely similar in each of the databases.

Recommendations for the Use of Social Media in Pharmacovigilance: Lessons from IMI WEB-RADR.

Over a period of 3 years, the European Union's Innovative Medicines Initiative WEB-RADR project has explored the value of social media (i.e., information exchanged through the internet, typically via online social networks) for identifying adverse events as well as for safety signal detection. Many patients and clinicians have taken to social media to discuss their positive and negative experiences of medications, creating a source of publicly available information that has the potential to provide insights into medicinal product safety concerns. The WEB-RADR project has developed a collaborative English language workspace for visualising and analysing social media data for a number of medicinal products. Further, novel text and data mining methods for social media analysis have been developed and evaluated. From this original research, several recommendations are presented with supporting rationale and consideration of the limitations. Recommendations for further research that extend beyond the scope of the current project are also presented.

Assessment of the Utility of Social Media for Broad-Ranging Statistical Signal Detection in Pharmacovigilance: Results from the WEB-RADR Project.

INTRODUCTION AND OBJECTIVE: Social media has been proposed as a possibly useful data source for pharmacovigilance signal detection. This study primarily aimed to evaluate the performance of established statistical signal detection algorithms in Twitter/Facebook for a broad range of drugs and adverse events. METHODS: Performance was assessed using a reference set by Harpaz et al., consisting of 62 US Food and Drug Administration labelling changes, and an internal WEB-RADR reference set consisting of 200 validated safety signals. In total, 75 drugs were studied. Twitter/Facebook posts were retrieved for the period March 2012 to March 2015, and drugs/events were extracted from the posts. We retrieved 4.3 million and 2.0 million posts for the WEB-RADR and Harpaz drugs, respectively. Individual case reports were extracted from VigiBase for the same period. Disproportionality algorithms based on the Information Component or the Proportional Reporting Ratio and crude post/report counting were applied in Twitter/Facebook and VigiBase. Receiver operating characteristic curves were generated, and the relative timing of alerting was analysed. RESULTS: Across all algorithms, the area under the receiver operating characteristic curve for Twitter/Facebook varied between 0.47 and 0.53 for the WEB-RADR reference set and between 0.48 and 0.53 for the Harpaz reference set. For VigiBase, the ranges were 0.64-0.69 and 0.55-0.67, respectively. In Twitter/Facebook, at best, 31 (16%) and four (6%) positive controls were detected prior to their index dates in the WEB-RADR and Harpaz references, respectively. In VigiBase, the corresponding numbers were 66 (33%) and 17 (27%). CONCLUSIONS: Our results clearly suggest that broad-ranging statistical signal detection in Twitter and Facebook, using currently available methods for adverse event recognition, performs poorly and cannot be recommended at the expense of other pharmacovigilance activities.

Regulatory Definitions and Good Pharmacovigilance Practices in Social Media: Challenges and Recommendations.

Social media presents new challenges to the biopharmaceutical industry for conducting pharmacovigilance activities. The authors reviewed worldwide regulatory guidance documents related to monitoring of adverse events posted on social media sites and identified gaps in current regulatory definitions for pharmacovigilance. Points to consider for addressing these gaps are made to offer standards for industry consideration and a potential framework for guidance from global health authorities.