The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration.

In pharmacovigilance, disproportionality analyses based on individual case safety reports are widely used to detect safety signals. Unfortunately, publishing disproportionality analyses lacks specific guidelines, often leading to incomplete and ambiguous reporting, and carries the risk of incorrect conclusions when data are not placed in the correct context. The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) statement was developed to address this issue by promoting transparent and comprehensive reporting of disproportionality studies. While the statement paper explains in greater detail the procedure followed to develop these guidelines, with this explanation paper we present the 14 items retained for READUS-PV guidelines, together with an in-depth explanation of their rationale and bullet points to illustrate their practical implementation. Our primary objective is to foster the adoption of the READUS-PV guidelines among authors, editors, peer reviewers, and readers of disproportionality analyses. Enhancing transparency, completeness, and accuracy of reporting, as well as proper interpretation of their results, READUS-PV guidelines will ultimately facilitate evidence-based decision making in pharmacovigilance.

The Reporting of a Disproportionality Analysis for Drug Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Development and Statement.

BACKGROUND AND AIM: Disproportionality analyses using reports of suspected adverse drug reactions are the most commonly used quantitative methods for detecting safety signals in pharmacovigilance. However, their methods and results are generally poorly reported in published articles and existing guidelines do not capture the specific features of disproportionality analyses. We here describe the development of a guideline (REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance [READUS-PV]) for reporting the results of disproportionality analyses in articles and abstracts. METHODS: We established a group of 34 international experts from universities, the pharmaceutical industry, and regulatory agencies, with expertise in pharmacovigilance, disproportionality analyses, and assessment of safety signals. We followed a three-step process to develop the checklist: (1) an open-text survey to generate a first list of items; (2) an online Delphi method to select and rephrase the most important items; (3) a final online consensus meeting. RESULTS: Among the panel members, 33 experts responded to round 1 and 30 to round 2 of the Delphi and 25 participated to the consensus meeting. Overall, 60 recommendations for the main body of the manuscript and 13 recommendations for the abstracts were retained by participants after the Delphi method. After merging of some items together and the online consensus meeting, the READUS-PV guidelines comprise a checklist of 32 recommendations, in 14 items, for the reporting of disproportionality analyses in the main body text and four items, comprising 12 recommendations, for abstracts. CONCLUSIONS: The READUS-PV guidelines will support authors, editors, peer-reviewers, and users of disproportionality analyses using individual case safety report databases. Adopting these guidelines will lead to more transparent, comprehensive, and accurate reporting and interpretation of disproportionality analyses, facilitating the integration with other sources of evidence.

Updated core competencies in pharmacoepidemiology to inform contemporary curricula and training for academia, government, and industry.

PURPOSE: The first paper to specify the core content of pharmacoepidemiology as a profession was published by an ISPE (International Society for Pharmacoepidemiology) workgroup in 2012 (Jones JK et al. PDS 2012; 21[7]:677-689). Due to the broader and evolving scope of pharmacoepidemiology, ISPE considers it important to proactively identify, update and expand the list of core competencies to inform curricula of education programs; thus, better positioning pharmacoepidemiologists across academic, government (including regulatory), and industry positions. The aim of this project was to update the list of core competencies in pharmacoepidemiology. METHODS: To ensure applicability of findings to multiple areas, a working group was established consisting of ISPE members with positions in academia, industry, government, and other settings. All competencies outlined by Jones et al. were extracted from the initial manuscript and presented to the working group for review. Expert-based judgments were collated and used to identify consensus. It was noted that some competencies could contribute to multiple groups and could be directly or indirectly related to a group. RESULTS: Five core domains were proposed: (1) Epidemiology, (2) Clinical Pharmacology, (3) Regulatory Science, (4) Statistics and data science, and (5) Communication and other professional skills. In total, 55 individual competencies were proposed, of which 25 were new competencies. No competencies from the original work were dropped but aggregation or amendments were made where considered necessary. CONCLUSIONS: While many core competencies in pharmacoepidemiology have remained the same over the past 10 years, there have also been several updates to reflect new and emerging concepts in the field.

Tree-temporal scan statistics for safety signal detection in vaccine clinical trials.

The evaluation of safety is critical in all clinical trials. However, the quantitative analysis of safety data in clinical trials poses statistical difficulties because of multiple potentially overlapping endpoints. Tree-temporal scan statistic approaches address this issue and have been widely employed in other data sources, but not to date in clinical trials. We evaluated the performance of three complementary scan statistical methods for routine quantitative safety signal detection: the self-controlled tree-temporal scan (SCTTS), a tree-temporal scan based on group comparison (BGTTS), and a log-rank based tree-temporal scan (LgRTTS). Each method was evaluated using data from two phase III clinical trials, and simulated data (simulation study). In the case study, the reference set was adverse events (AEs) in the Reference Safety Information of the evaluated vaccine. The SCTTS method had higher sensitivity than other methods, and after dose 1 detected 80 true positives (TP) with a positive predictive value (PPV) of 60%. The LgRTTS detected 49 TPs with 69% PPV. The BGTTS had 90% of PPV with 38 TPs. In the simulation study, with simulated reference sets of AEs, the SCTTS method had good sensitivity to detect transient effects. The LgRTTS method showed the best performance for the detection of persistent effects, with high sensitivity and expected probability of type I error. These three methods provide complementary approaches to safety signal detection in clinical trials or across clinical development programmes. All three methods formally adjust for multiple testing of large numbers of overlapping endpoints without being excessively conservative.

Timing Matters: A Machine Learning Method for the Prioritization of Drug-Drug Interactions Through Signal Detection in the FDA Adverse Event Reporting System and Their Relationship with Time of Co-exposure.

INTRODUCTION: Current drug-drug interaction (DDI) detection methods often miss the aspect of temporal plausibility, leading to false-positive disproportionality signals in spontaneous reporting system (SRS) databases. OBJECTIVE: This study aims to develop a method for detecting and prioritizing temporally plausible disproportionality signals of DDIs in SRS databases by incorporating co-exposure time in disproportionality analysis. METHODS: The method was tested in the Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS). The CRESCENDDI dataset of positive controls served as the primary source of true-positive DDIs. Disproportionality analysis was performed considering the time of co-exposure. Temporal plausibility was assessed using the flex point of cumulative reporting of disproportionality signals. Potential confounders were identified using a machine learning method (i.e. Lasso regression). RESULTS: Disproportionality analysis was conducted on 122 triplets with more than three cases, resulting in the prioritization of 61 disproportionality signals (50.0%) involving 13 adverse events, with 61.5% of these included in the European Medicine Agency's (EMA's) Important Medical Event (IME) list. A total of 27 signals (44.3%) had at least ten cases reporting the triplet of interest, and most of them (n = 19; 70.4%) were temporally plausible. The retrieved confounders were mainly other concomitant drugs. CONCLUSIONS: Our method was able to prioritize disproportionality signals with temporal plausibility. This finding suggests a potential for our method in pinpointing signals that are more likely to be furtherly validated.

Optimizing Signal Management in a Vaccine Adverse Event Reporting System: A Proof-of-Concept with COVID-19 Vaccines Using Signs, Symptoms, and Natural Language Processing.

INTRODUCTION: The Vaccine Adverse Event Reporting System (VAERS) has already been challenged by an extreme increase in the number of individual case safety reports (ICSRs) after the market introduction of coronavirus disease 2019 (COVID-19) vaccines. Evidence from scientific literature suggests that when there is an extreme increase in the number of ICSRs recorded in spontaneous reporting databases (such as the VAERS), an accompanying increase in the number of disproportionality signals (sometimes referred to as 'statistical alerts') generated is expected. OBJECTIVES: The objective of this study was to develop a natural language processing (NLP)-based approach to optimize signal management by excluding disproportionality signals related to listed adverse events following immunization (AEFIs). COVID-19 vaccines were used as a proof-of-concept. METHODS: The VAERS was used as a data source, and the Finding Associated Concepts with Text Analysis (FACTA+) was used to extract signs and symptoms of listed AEFIs from MEDLINE for COVID-19 vaccines. Disproportionality analyses were conducted according to guidelines and recommendations provided by the US Centers for Disease Control and Prevention. By using signs and symptoms of listed AEFIs, we computed the proportion of disproportionality signals dismissed for COVID-19 vaccines using this approach. Nine NLP techniques, including Generative Pre-Trained Transformer 3.5 (GPT-3.5), were used to automatically retrieve Medical Dictionary for Regulatory Activities Preferred Terms (MedDRA PTs) from signs and symptoms extracted from FACTA+. RESULTS: Overall, 17% of disproportionality signals for COVID-19 vaccines were dismissed as they reported signs and symptoms of listed AEFIs. Eight of nine NLP techniques used to automatically retrieve MedDRA PTs from signs and symptoms extracted from FACTA+ showed suboptimal performance. GPT-3.5 achieved an accuracy of 78% in correctly assigning MedDRA PTs. CONCLUSION: Our approach reduced the need for manual exclusion of disproportionality signals related to listed AEFIs and may lead to better optimization of time and resources in signal management.

Implementation of a Taxonomy-Based Framework for the Selection of Appropriate Drugs and Outcomes for Real-World Data Signal Detection Studies.

INTRODUCTION: For signal detection studies investigating either drug safety or method evaluation, the choice of drug-outcome pairs needs to be tailored to the planned study design and vice versa. While this is well understood in hypothesis-testing epidemiology, it should be as important in signal detection, but this has not widely been considered. There is a need for a taxonomy framework to provide guidance and a systematic reproducible approach to the selection of appropriate drugs and outcomes for signal detection studies either investigating drug safety or assessing method performance using real-world data. OBJECTIVE: The aim was to design a general framework for the selection of appropriate drugs and outcomes for signal detection studies given a study design of interest. As a motivating example, we illustrate how the framework is applied to build a reference set for a study aiming to assess the performance of the self-controlled case series with active comparators. METHODS: We reviewed criteria presented in two published studies which aimed to provide practical advice for choosing the appropriate signal evaluation methodology, and assessed their relevance for signal detection. Further characteristics specific to signal detection were added. The final framework is based on: the application of study design requirements, the database(s) of interest, and the clinical importance of the drug(s) and outcome(s) under consideration. This structure was applied by selecting drug-outcome pairs as a reference set (i.e. list of drug-outcome pairs classified as positive or negative controls) for which the method is expected to work well for a signal detection study aiming to assess the performance of self-controlled case series. Eight criteria were used, related to the application of self-controlled case series assumptions, choice of active comparators, coverage in the database of interest and clinical importance of the outcomes. RESULTS: After application of the framework, two classes of antibiotics (seven drugs) were selected for the study, and 28 outcomes from all organ classes were chosen from the drug labels, out of the 273 investigated. In total, this corresponds to 104 positive controls (drug-outcome pairs) and 58 negative controls. CONCLUSIONS: We proposed and applied a framework for the selection of drugs and outcomes for both drug safety signal detection and method assessment used in signal detection to optimise their performance given a study design. This framework will eliminate part of the bias relating to drugs and outcomes not being suited to the method or database. The main difficulty lies in the choice of the criteria and their application to ensure systematic selection, especially as some information remains unknown in signal detection, and clinical judgement was needed on occasions. The same framework could be adapted for other methods.

Finding Needles in the Haystack: Clinical Utility Score for Prioritisation (CUSP), an Automated Approach for Identifying Spontaneous Reports with the Highest Clinical Utility.

INTRODUCTION: Spontaneous reporting of adverse events has increased steadily over the past decades, and although this trend has contributed to improving post-marketing surveillance pharmacovigilance activities, the consequent amount of data generated is challenging to manually review during assessment, with each individual report requiring review by pharmacovigilance experts. This highlights a clear need for alternative or complementary methodologies to help prioritise review. OBJECTIVE: Here, we aimed to develop and test an automated methodology, the Clinical Utility Score for Prioritisation (CUSP), to assist pharmacovigilance experts in prioritising clinical assessment of safety data to improve the rapidity of case series review when case volumes are large. METHODS: The CUSP method was tested on a reference dataset of individual case safety reports (ICSRs) associated to five drug-event pairs that led to labelling changes. The selected drug-event pairs were of varying characteristics across the portfolio of GSK's products. RESULTS: The mean CUSP score for 'key cases' and 'cases of low utility' was 19.7 (median: 21; range: 7-27) and 17.3 (median: 19; range: 4-27), respectively. CUSP distribution for 'key cases' were skewed toward the higher range of scores compared with 'all cases'. The overall performance across each individual drug-event pair varied considerably, showing higher predictive power for 'key cases' for three of the drug-event pairs (average CUSP between these three: 22.8; range: 22.5-23.0) and lesser power for the remaining two (average CUSP between these two: 17.6; range: 14.5-20.7). CONCLUSION: Although several tools have been developed to assess ICSR completeness and regulatory utility, this is the first attempt to successfully develop an automated clinical utility scoring system that can support the prioritisation of ICSRs for clinical review.

Developing an Artificial Intelligence-Guided Signal Detection in the Food and Drug Administration Adverse Event Reporting System (FAERS): A Proof-of-Concept Study Using Galcanezumab and Simulated Data.

INTRODUCTION: Time- and resource-demanding activities related to processing individual case safety reports (ICSRs) include manual procedures to evaluate individual causality with the final goal of dismissing false-positive safety signals. Eminent experts and a representative from pharmaceutical industries and regulatory agencies have highlighted the need to automatize time- and resource-demanding procedures in signal detection and validation. However, to date there is a sparse availability of automatized tools for such purposes. OBJECTIVES: ICSRs recorded in spontaneous reporting databases have been and continue to be the cornerstone and the most important data source in signal detection. Despite the richness of this data source, the incessantly increased amount of ICSRs recorded in spontaneous reporting databases has generated problems in signal detection and validation due to the increase in resources and time needed to process cases. This study aimed to develop a new artificial intelligence (AI)-based framework to automate resource- and time-consuming steps of signal detection and signal validation, such as (1) the selection of control groups in disproportionality analyses and (2) the identification of co-reported drugs serving as alternative causes, to look to dismiss false-positive disproportionality signals and therefore reduce the burden of case-by-case validation. METHODS: The Summary of Product Characteristics (SmPC) and the Anatomical Therapeutic Chemical (ATC) classification system were used to automatically identify control groups within and outside the chemical subgroup of the proof-of-concept drug under investigation, galcanezumab. Machine learning, specifically conditional inference trees, has been used to identify alternative causes in disproportionality signals. RESULTS: By using conditional inference trees, the framework was able to dismiss 20.00% of erenumab, 14.29% of topiramate, and 13.33% of amitriptyline disproportionality signals on the basis of purely alternative causes identified in cases. Furthermore, of the disproportionality signals that could not be dismissed purely on the basis of the alternative causes identified, we estimated a 15.32%, 25.39%, and 26.41% reduction in the number of galcanezumab cases to undergo manual validation in comparison with erenumab, topiramate, and amitriptyline, respectively. CONCLUSION: AI could significantly ease some of the most time-consuming and labor-intensive steps of signal detection and validation. The AI-based approach showed promising results, however, future work is needed to validate the framework.

Identifying Safety Subgroups at Risk: Assessing the Agreement Between Statistical Alerting and Patient Subgroup Risk.

INTRODUCTION: Identifying individual characteristics or underlying conditions linked to adverse drug reactions (ADRs) can help optimise the benefit-risk ratio for individuals. A systematic evaluation of statistical methods to identify subgroups potentially at risk using spontaneous ADR report datasets is lacking. OBJECTIVES: In this study, we aimed to assess concordance between subgroup disproportionality scores and European Medicines Agency Pharmacovigilance Risk Assessment Committee (PRAC) discussions of potential subgroup risk. METHODS: The subgroup disproportionality method described by Sandberg et al., and variants, were applied to statistically screen for subgroups at potential increased risk of ADRs, using data from the US FDA Adverse Event Reporting System (FAERS) cumulative from 2004 to quarter 2 2021. The reference set used to assess concordance was manually extracted from PRAC minutes from 2015 to 2019. Mentions of subgroups presenting potential differentiated risk and overlapping with the Sandberg method were included. RESULTS: Twenty-seven PRAC subgroup examples representing 1719 subgroup drug-event combinations (DECs) in FAERS were included. Using the Sandberg methodology, 2 of the 27 could be detected (one for age and one for sex). No subgroup examples for pregnancy and underlying condition were detected. With a methodological variant, 14 of 27 examples could be detected. CONCLUSIONS: We observed low concordance between subgroup disproportionality scores and PRAC discussions of potential subgroup risk. Subgroup analyses performed better for age and sex, while for covariates not well-captured in FAERS, such as underlying condition and pregnancy, additional data sources should be considered.

The role of Real-World Data and evidence in oncology medicines approved in EU in 2018-2019.

Use of Real-World Data (RWD) has gained the interest of different stakeholders in cancer care. The aim of this study was to identify and describe the use of RWD/RWE during the pre-authorization phase of products authorized by the EMA in 2018 and 2019 (n = 111), with the focus on oncology medicines (n = 24). Information was extracted from the European Public Assessment Report (EPAR) summaries and recorded for 5 stages (11 categories) of the drug development lifecycle (discovery, early development, clinical development, registration/market launch, lifecycle management). Specific chapters of full EPAR were reviewed to substantiate the findings on RWD/RWE use in clinical trial design, efficacy, safety, and effectiveness evaluation. RWD/RWE is present in all stages of the oncology drug development; 100.0 % in discovery, 37.5 % early development, 58.3 % in clinical development, 62.5 % in registration decision and 100.0 % in post-authorization lifecycle management. Examples showed that trial design supported by RWD/RWE included use of open label/single arm studies; efficacy was about using either comparison of results to historical controls, supplying survey data obtained outside the clinical trial or utilizing expert panel advice; safety about including literature findings in evidence; and effectiveness on comparison of trial results of the given product to historical data or existing standard of care. The findings of this study provide specific insights into how RWD/RWE is used in development of cancer therapeutics, how it contributes to regulatory decision making and can guide further policy developments in this field.

Individual Case Safety Report Replication: An Analysis of Case Reporting Transmission Networks.

INTRODUCTION: The basis of pharmacovigilance is provided by the exchange of Individual Case Safety Reports (ICSRs) between the recipient of the original report and other interested parties, which include Marketing Authorization Holders (MAHs) and Health Authorities (HAs). Different regulators have different reporting requirements for report transmission. This results in replication of each ICSR that will exist in multiple locations. Adding in the fact that each case will go through multiple versions, different recipients may receive different case versions at different times, potentially influencing patient safety decisions and potentially amplifying or obscuring safety signals inappropriately. OBJECTIVE: The present study aimed to investigate the magnitude of replication, the variability among recipients, and the subsequent divergence across recipients of ICSRs. METHODS: Seven participating TransCelerate Member Companies (MCs) queried their respective safety databases covering a 3-year period and provided aggregate ICSR submission statistics for expedited safety reports to an independent project manager. As measured in the US Food and Drug Administration (FDA)'s Adverse Event Reporting System (FAERS), ICSR volume for these seven MCs makes up approximately 20% of the total case volume. Aggregate metrics were calculated from the company data, specifically: (i) number of ICSR transmissions, (ii) average number of recipients (ANR) per case version transmitted, (iii) a submission selectivity metric, which measures the percentage of recipients not having received all sequential case version numbers, and (iv) percent of common ISCRs residing in two or more MAH databases. RESULTS: The analysis reflects 2,539,802 case versions, distributed through 7,602,678 submissions. The overall mean replication rate is 3.0 submissions per case version. The distribution of the ANR replication measure was observed to be very long-tailed, with a significant fraction of case versions (~ 12.4% of all transmissions) being sent to ten or more HA recipients. Replication is higher than average for serious, unlisted, and literature cases, ranging from 3.5 to 6.1 submissions per version. Within the subset of ICSR versions sent to three recipients, a significant degree of variability in the actual recipients (i.e., HAs) was observed, indicating that there is not one single combination of the same three HAs predominantly receiving an ICSR. Submission selectivity increases with the case version. For case version 6, the range of the submission selectivity for the MAHs ranges from ~ 10% to over 50%, with a median of 30.2%. Within the participating MAHs, the percentage of cases that reside within at least two safety databases is approximately 2% across five databases. Further analysis of the data from three MAHs showed percentages of 13.4%, 15.6%, and 27.9% of ICSRs originating from HAs and any other partners such as other MAHs and other institutions. CONCLUSION: Replication of ICSRs and the variation of available safety information in recipient databases were quantified and shown to be substantial. Our work shows that multiple processors and medical reviewers will likely handle the same original ICSR as a result of replication. Aside from the obvious duplicate work, this phenomenon could conceivably lead to differing clinical assessments and decisions. If replication could be reduced or even eliminated, this would enable more focus on activities with a benefit for patient safety.

Methods for drug safety signal detection using routinely collected observational electronic health care data: A systematic review.

PURPOSE: Signal detection is a crucial step in the discovery of post-marketing adverse drug reactions. There is a growing interest in using routinely collected data to complement established spontaneous report analyses. This work aims to systematically review the methods for drug safety signal detection using routinely collected healthcare data and their performance, both in general and for specific types of drugs and outcomes. METHODS: We conducted a systematic review following the PRISMA guidelines, and registered a protocol in PROSPERO. MEDLINE, EMBASE, PubMed, Web of Science, Scopus, and the Cochrane Library were searched until July 13, 2021. RESULTS: The review included 101 articles, among which there were 39 methodological works, 25 performance assessment papers, and 24 observational studies. Methods included adaptations from those used with spontaneous reports, traditional epidemiological designs, methods specific to signal detection with real-world data. More recently, implementations of machine learning have been studied in the literature. Twenty-five studies evaluated method performances, 16 of them using the area under the curve (AUC) for a range of positive and negative controls as their main measure. Despite the likelihood that performance measurement could vary by drug-event pair, only 10 studies reported performance stratified by drugs and outcomes, in a heterogeneous manner. The replicability of the performance assessment results was limited due to lack of transparency in reporting and the lack of a gold standard reference set. CONCLUSIONS: A variety of methods have been described in the literature for signal detection with routinely collected data. No method showed superior performance in all papers and across all drugs and outcomes, performance assessment and reporting were heterogeneous. However, there is limited evidence that self-controlled designs, high dimensional propensity scores, and machine learning can achieve higher performances than other methods.

Digital biomarkers for post-licensure safety monitoring.

Post-licensure safety data form the cornerstone of safety surveillance. However, such data have some limitations related to the subjectiveness of reporting and recording, primary purpose of the collected data, or heterogeneity. Routine capture of richer data would in part help mitigate these limitations, enabling earlier, more reliable safety insights. Digital health tools that remotely acquire health-related information are increasingly available and used by patients and the wider population. However, they are rarely used for pharmacovigilance purposes. Here, we review different cases that reveal the opportunities and challenges of using these technologies for enhanced safety assessment in routine healthcare delivery. We believe such approaches will advance our understanding of the safety of drugs and vaccines in the future.

Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting.

Transparency is increasingly promoted to instill trust in nonrandomized studies using real-world data. Graphics and data visualizations support transparency by aiding communication and understanding, and can inform study design and analysis decisions. However, other than graphical representation of a study design and flow diagrams (e.g., a Consolidated Standards of Reporting Trials [CONSORT] like diagram), specific standards on how to maximize validity and transparency with visualization are needed. This paper provides guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study-from initial study design to final report-to facilitate rationalized and transparent decision-making about study design and implementation, and clear communication of study findings. Our intent is to help researchers align their practices with current consensus statements on transparency.