The Reporting of a Disproportionality Analysis for Drug Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Development and Statement.

BACKGROUND AND AIM: Disproportionality analyses using reports of suspected adverse drug reactions are the most commonly used quantitative methods for detecting safety signals in pharmacovigilance. However, their methods and results are generally poorly reported in published articles and existing guidelines do not capture the specific features of disproportionality analyses. We here describe the development of a guideline (REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance [READUS-PV]) for reporting the results of disproportionality analyses in articles and abstracts. METHODS: We established a group of 34 international experts from universities, the pharmaceutical industry, and regulatory agencies, with expertise in pharmacovigilance, disproportionality analyses, and assessment of safety signals. We followed a three-step process to develop the checklist: (1) an open-text survey to generate a first list of items; (2) an online Delphi method to select and rephrase the most important items; (3) a final online consensus meeting. RESULTS: Among the panel members, 33 experts responded to round 1 and 30 to round 2 of the Delphi and 25 participated to the consensus meeting. Overall, 60 recommendations for the main body of the manuscript and 13 recommendations for the abstracts were retained by participants after the Delphi method. After merging of some items together and the online consensus meeting, the READUS-PV guidelines comprise a checklist of 32 recommendations, in 14 items, for the reporting of disproportionality analyses in the main body text and four items, comprising 12 recommendations, for abstracts. CONCLUSIONS: The READUS-PV guidelines will support authors, editors, peer-reviewers, and users of disproportionality analyses using individual case safety report databases. Adopting these guidelines will lead to more transparent, comprehensive, and accurate reporting and interpretation of disproportionality analyses, facilitating the integration with other sources of evidence.

The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration.

In pharmacovigilance, disproportionality analyses based on individual case safety reports are widely used to detect safety signals. Unfortunately, publishing disproportionality analyses lacks specific guidelines, often leading to incomplete and ambiguous reporting, and carries the risk of incorrect conclusions when data are not placed in the correct context. The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) statement was developed to address this issue by promoting transparent and comprehensive reporting of disproportionality studies. While the statement paper explains in greater detail the procedure followed to develop these guidelines, with this explanation paper we present the 14 items retained for READUS-PV guidelines, together with an in-depth explanation of their rationale and bullet points to illustrate their practical implementation. Our primary objective is to foster the adoption of the READUS-PV guidelines among authors, editors, peer reviewers, and readers of disproportionality analyses. Enhancing transparency, completeness, and accuracy of reporting, as well as proper interpretation of their results, READUS-PV guidelines will ultimately facilitate evidence-based decision making in pharmacovigilance.

Lessons Learned on Observed-to-Expected Analysis Using Spontaneous Reports During Mass Vaccination.

During the COVID-19 vaccination campaign, observed-to-expected analysis was used by the European Medicines Agency to contextualise data from spontaneous reports to generate real-time evidence on emerging safety concerns that may impact the benefit-risk profile of COVID-19 vaccines. Observed-to-expected analysis compares the number of cases spontaneously reported for an event of interest after vaccination ('observed') to the 'expected' number of cases anticipated to occur in the same number of individuals had they not been vaccinated. Observed-to-expected analysis is a robust methodology that relies on several assumptions that have been described in regulatory guidelines and scientific literature. The use of observed-to-expected analysis to support the safety monitoring of COVID-19 vaccines has provided valuable insights and lessons on its design and interpretability, which could prove to be beneficial in future analyses. When undertaking an observed-to-expected analysis within the context of safety monitoring, several aspects need attention. In particular, we emphasise the importance of stratified and harmonised data collection both for vaccine exposure and spontaneous reporting data, the need for alignment between coding dictionaries and the crucial role of accurate background incidence rates for adverse events of special interest. While these considerations and recommendations were determined in the context of the COVID-19 mass vaccination setting, they are generalisable in principle.

Predicting the submission frequency of periodic safety update reports: development and application of the EURD tool.

Introduction: Periodic Safety Update Reports (PSURs) are a key pharmacovigilance tool for the continuous evaluation of the benefit-risk balance of a medicinal product in the post-authorisation phase. The PSUR submission frequency for authorised active substances and combinations of active substances across the EU is individually determined. The objective of this research was the development and application of the EURD tool, a statistical method based on readily available safety data to predict PSUR frequencies and to ensure a consistent risk-based approach. Methods: First, variables considered relevant in determining the PSUR frequency were identified from data sources available at the European Medicines Agency. A subsequent first survey with National Competent Authorities in Europe lead to a prioritisation of identified variables, while a second survey was carried out to propose the PSUR frequencies for a set of substances. Finally, a regression model was built on the information collected, applied to a larger list of substances and its results tested via a third survey with the same experts. Results: The developed EURD tool was applied to the 1,032 EURD list entries with a PSUR assessment deferred to 2025 at the time of the creation of the list in 2012. As the number of procedures would have had a significant impact on the workload for the European Medicines Regulatory Network (EMRN), in a second step the workload impact was estimated after allocating the entries according to their proposed frequency. The analysis suggests that all entries could be reviewed by 2038 by increasing the median workload by 15% (from 868 to 1,000 substances/year). Conclusion: The EURD tool is the first data-driven application for supporting decision making of PSUR frequencies based on relevant active substance safety data. While we illustrated its potential for improving the assignment of PSUR submission frequencies for active substances authorised in the EU, other institutions requiring periodic assessment of safety data and balancing of the resulting workload could benefit from it.

Ondansetron use in nausea and vomiting during pregnancy: A descriptive analysis of prescription patterns and patient characteristics in UK general practice.

AIMS: The objective of this study was to describe ondansetron drug utilization patterns during pregnancy to treat nausea and vomiting in pregnancy (NVP). Moreover, we aimed to describe the maternal factors associated with NVP and antiemetic use. METHODS: The data consist of pregnancies with a live birth(s) within an IMRD-UK registered GP practice. Descriptive statistics were used to investigate patterns of ondansetron use in pregnancy and to describe maternal characteristics associated with NVP and antiemetic drug utilization. We differentiate first- from second-line use during pregnancy using antiemetic prescription pathways. RESULTS: The dataset included 733 633 recorded complete pregnancies from 2005 to 2019. NVP diagnosis and ondansetron prescription prevalence increased from 2.7% and 0.1% in 2005 to 4.8% and 2.5% in 2019 respectively. Over the period 2015-2019, the most common oral daily dosages were 4 mg/d (8.5%), 8 mg/d (37.1%), 12 mg/d (37.5%) and between 16 and 24 mg/d (16.9%). Prescription of ondansetron was initiated during the first trimester of pregnancy in 40% of the cases and was moderately used as a first-line therapy (2.8%), but preferred choice of second-line therapy. Women with mental health disorders, asthma and/or prescribed folic acid were more likely to experience NVP and use antiemetics in pregnancy than their counterparts. CONCLUSION: This study confirms that ondansetron is increasingly used off-label to treat NVP during pregnancy, also in the first trimester and before other prescription antiemetics have been prescribed. Several maternal comorbidities and folic acid use were more common among women experiencing NVP and using antiemetics, including ondansetron.

The Impact of Mandatory Reporting of Non-Serious Safety Reports to EudraVigilance on the Detection of Adverse Reactions.

INTRODUCTION AND OBJECTIVE: European Union legislation has mandated the submission of European Economic Area non-serious reports to the EudraVigilance database since November 2017. As spontaneous reports of suspected adverse reactions to medicines represent a key source of safety signals, the European Medicines Agency has undertaken this work to assess the effects of this requirement on the characteristics of the reports submitted to EudraVigilance and on the detection of adverse drug reactions through routine analyses of the database. METHODS: Changes in the numbers of serious and non-serious reports transmitted to EudraVigilance were examined over the period during which the legislation was implemented. The numbers and nature of potential safety signals emerging from established statistical algorithms used at the European Medicines Agency applied either to only the serious reports or to all reports in EudraVigilance were compared. RESULTS: Up to November 2017, less than 25% of European Economic Area reports in EudraVigilance were classified as non-serious, since than this figure was slightly above 60%. This change accompanied an increase in the total number of reports received. Addition of non-serious reports to the signal detection process resulted in a small overall increase in signals of disproportionate reporting with some new signals of disproportionate reporting appearing and some existing signals of disproportionate reporting disappearing; the sensitivity of the signal detection system was slightly increased and the proportion of signals of disproportionate reporting that corresponded to known adverse drug reactions (a measure of efficiency) was unchanged. CONCLUSIONS: The change in legislation has led to a small increase in sensitivity, without affecting the efficiency of the routine statistical measures used. The number of non-serious reports as a proportion of reports in EudraVigilance is likely to increase over time and further monitoring of the impact on signal detection is required. Further work is also required on the qualitative impact of non-serious reports on the nature of signals detected and on their evaluation.

Is There an Increased Risk of Hepatotoxicity with Metamizole? A Comparative Cohort Study in Incident Users.

INTRODUCTION: The analgesic metamizole, which has been withdrawn from the market in several countries due to the risk of agranulocytosis but is still available on the market in Germany and some other countries, has been associated with liver injury in published case reports; however, epidemiological studies on the risk of liver injury are limited. OBJECTIVE: The aim of this study was to compare the risk of liver injury up to 270 days after the first start of treatment with metamizole with the corresponding risk in patients starting treatment with paracetamol, using a retrospective cohort incident user design. METHODS: The first prescription for either metamizole or paracetamol in the Intercontinental Medical Statistics (IMS)® Disease Analyzer Germany database during the study period (2009-2018) was identified in patients with at least 365 days of observation and no prior diagnosis of liver events, cancer or HIV, or treatment within the last 6 months with hepatotoxic drugs typically administered for chronic conditions. Each patient was followed for specific liver events for 90 days after the prescription. In case of a new prescription within 90 days, a new 90-day observation period started, up to a maximum of 270 days. Cox regression was used to compare the risk of liver injury in the two groups. RESULTS: Metamizole was associated with a higher risk of liver injury compared with paracetamol (adjusted hazard ratio 1.69, 95% confidence interval 1.46-1.97). Sensitivity analyses were performed to evaluate the robustness of these findings. In all the sensitivity analyses, metamizole was still associated with a higher risk of liver injury, including an analysis where naproxen was used as a comparator instead of paracetamol. CONCLUSIONS: Results from this study support previous studies suggesting that metamizole is associated with a significant risk of liver injury. Nevertheless, a possible impact of residual confounding cannot be excluded.

Different Strategies to Execute Multi-Database Studies for Medicines Surveillance in Real-World Setting: A Reflection on the European Model.

Although postmarketing studies conducted in population-based databases often contain information on patients in the order of millions, they can still be underpowered if outcomes or exposure of interest is rare, or the interest is in subgroup effects. Combining several databases might provide the statistical power needed. A multi-database study (MDS) uses at least two healthcare databases, which are not linked with each other at an individual person level, with analyses carried out in parallel across each database applying a common study protocol. Although many MDSs have been performed in Europe in the past 10 years, there is a lack of clarity on the peculiarities and implications of the existing strategies to conduct them. In this review, we identify four strategies to execute MDSs, classified according to specific choices in the execution: (A) local analyses, where data are extracted and analyzed locally, with programs developed by each site; (B) sharing of raw data, where raw data are locally extracted and transferred without analysis to a central partner, where all the data are pooled and analyzed; (C) use of a common data model with study-specific data, where study-specific data are locally extracted, loaded into a common data model, and processed locally with centrally developed programs; and (D) use of general common data model, where all local data are extracted and loaded into a common data model, prior to and independent of any study protocol, and protocols are incorporated in centrally developed programs that run locally. We illustrate differences between strategies and analyze potential implications.

Can We Rely on Results From IQVIA Medical Research Data UK Converted to the Observational Medical Outcome Partnership Common Data Model?: A Validation Study Based on Prescribing Codeine in Children.

Exploring and combining results from more than one real-world data (RWD) source might be necessary in order to explore variability and demonstrate generalizability of the results or for regulatory requirements. However, the heterogeneous nature of RWD poses challenges when working with more than one source, some of which can be solved by analyzing databases converted into a common data model (CDM). The main objective of the study was to evaluate the implementation of the Observational Medical Outcome Partnership (OMOP) CDM on IQVIA Medical Research Data (IMRD)-UK data. A drug utilization study describing the prescribing of codeine for pain in children was used as a case study to be replicated in IMRD-UK and its corresponding OMOP CDM transformation. Differences between IMRD-UK source and OMOP CDM were identified and investigated. In IMRD-UK updated to May 2017, results were similar between source and transformed data with few discrepancies. These were the result of different conventions applied during the transformation regarding the date of birth for children younger than 15 years and the start of the observation period, and of a misclassification of two drug treatments. After the initial analysis and feedback provided, a rerun of the analysis in IMRD-UK updated to September 2018 showed almost identical results for all the measures analyzed. For this study, the conversion to OMOP CDM was adequate. Although some decisions and mapping could be improved, these impacted on the absolute results but not on the study inferences. This validation study supports six recommendations for good practice in transforming to CDMs.

A European multicentre drug utilisation study of the impact of regulatory measures on prescribing of codeine for pain in children.

PURPOSE: In June 2013, following recommendations from the World Health Organization (WHO) and Food and Drug Administration (FDA), the European Medicines Agency agreed updates to the codeine product information regarding use for pain in children younger than 12 years and children undergoing tonsillectomy or adenoidectomy (TA) for obstructive sleep apnoea. This study was conducted to (a) assess effectiveness of these measures on codeine prescribing in the "real-world" setting and (b) test feasibility of a study using a common protocol by regulators with access to databases. METHODS: The study was performed using BIFAP (Spain), CPRD (UK), and IMS® Disease Analyzer (France and Germany) databases. Prescribers included general practitioners (GPs) (France and UK), GPs and paediatricians together (Spain), and GPs, paediatricians, and ear, nose, and throat (ENT) specialists separately (Germany). Between January 2010 and June 2015, prevalence of codeine prescribing was obtained every 6 months, and a time series analysis (joinpoint) was performed. Codeine prescribing within ±30 days of TA was also identified. Furthermore, doses, durations, and prior prescribing of other analgesics were investigated. RESULTS: Over the 5-year period, codeine prescribing decreased in children younger than 12 years (by 84% in France and Spain, 44% in GP practices in Germany, and 33% in the United Kingdom). The temporal pattern was compatible with the regulatory intervention in France and the United Kingdom, whereas a decrease throughout the study period was seen in Germany and Spain. Decreased prescribing associated with TA was suggested in ENT practices in Germany. CONCLUSIONS: Codeine prescribing for children decreased in line with introduced regulatory measures. Multidatabase studies assessing impact of measures by EU regulators are feasible.

EudraVigilance Medicines Safety Database: Publicly Accessible Data for Research and Public Health Protection.

The analysis of safety data from spontaneous reporting systems has a proven value for the detection and analysis of the risks of medicines following their placement on the market and use in medical practice. EudraVigilance is the pharmacovigilance database to manage the collection and analysis of suspected adverse reactions to medicines authorised in the European Economic Area. EudraVigilance first operated in December 2001, with access to the database being governed by the EudraVigilance access policy. We performed a literature search including data up to December 2016 to demonstrate how the data from EudraVigilance has been used in scientific publications. We describe the results, including by type of publication, research topics and drugs involved. In 50% of the publications, the data are used to describe safety issues, in 44% to analyse methodologies used in pharmacovigilance activities and in 6% to support clinical perspectives. We also outline a description of the use of the database by the European Union regulatory network. Driven by the full implementation of the 2010 pharmacovigilance legislation, EudraVigilance has undergone further enhancements together with a major revision of its access policy, taking into account the use of the new individual case safety report standard developed by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use and the International Organization for Standardization. The aim of the broadened access is to facilitate more effective safety monitoring of authorised medicines, to make more data available for research and to provide better access to information on suspected adverse reactions for healthcare professionals and patients. In November 2017, the new full functionalities of EudraVigilance were launched, including the extensive web access to data on suspected adverse drug reactions and the possibilities for academic research institutions to request a more extensive dataset for the purposes of health research. The main objective of this article is to describe the new access to the database together with the opportunities that this new access can bring for research. It is intended to promote an appropriate use of the data to support the safe and effective use of medicines.

An algorithm to detect unexpected increases in frequency of reports of adverse events in EudraVigilance.

PURPOSE: The European Medicines Agency developed an algorithm to detect unexpected increases in frequency of reports, to enhance the ability to detect adverse events that manifest as increases in frequency, in particular quality defects, medication errors, and cases of abuse or misuse. METHODS: An algorithm based on a negative binomial time-series regression model run on 6 sequential observations prior to the monitored period was developed to forecast monthly counts of reports. A heuristic model to capture increases in counts when the previous 4 observations were null supplemented the regression. Count data were determined at drug-event combination. Sensitivity analyses were run to determine the effect of different methods of pooling or stratifying count data. Positive retrospective detections and positive predictive values (PPVs) were determined. RESULTS: The algorithm detected 8 of the 13 historical concerns, including all concerns of quality defects. The highest PPV (1.29%) resulted from increasing the lower count threshold from 3 to 5 and including literature reports in the counts. Both the regression model and the heuristic model components to the algorithm contributed to the detection of concerns. Sensitivity analysis indicates that stratification by commercial product reduces the PPV but suggests that pooling counts of related events may improve it. CONCLUSION: The results are encouraging and suggest that the algorithm could be useful for the detection of concerns that manifest as changes in frequency of reporting; however, further testing, including in prospective use, is warranted.

Patient Reporting in the EU: Analysis of EudraVigilance Data.

INTRODUCTION: New pharmacovigilance legislation was adopted in the EU in 2010 and became operational in July 2012. The legislation placed an obligation on all national competent authorities (NCAs) and marketing authorisation holders (MAHs) to record and report cases of suspected adverse drug reactions (ADRs) received from patients. OBJECTIVES: This descriptive study aims to provide insight into patient reporting for the totality of the EU by querying the EudraVigilance (EV) database for the period of 3 years before the new pharmacovigilance legislation became operational and the 3 years after as well as comparing patient reports with those from healthcare professionals (HCPs) where feasible. METHODS: We queried the EV database for the following characteristics of patient and HCP reports: demographics (patient sex and age), seriousness, reported ADR terms, reported indications, number of ADRs per report, time to report an ADR, and most reported substances. Wherever feasible, direct comparisons between patient reports and HCP reports were performed using relative risks. RESULTS: The EV database contained a total of 53,130 patient reports in the 3 years preceding the legislation operation period and 113,371 in the 3 years after. Member states contributing the most patient reports to the EV database were the Netherlands, the UK, Germany, France and Italy. The results for indications and substances show that patients were more likely than HCPs to report for genitourinary, hormonal and reproductive indications. Patients reported more in general disorders and administration site conditions Medical Dictionary for Regulatory Activities (MedDRA) System Organ Class (SOC), whereas HCPs reported more Preferred Terms (PTs) belonging in the Investigations SOC. However, 13 of the 20 reactions most frequently reported by patients were also among the top 20 reactions reported by HCPs. CONCLUSION: Patient reporting complemented reporting by HCPs. Patients were motivated to report ADRs, especially those that affected their quality of life. Sharing these results with NCAs and patient associations can inform training and awareness on patient reporting.

Understanding inconsistency in the results from observational pharmacoepidemiological studies: the case of antidepressant use and risk of hip/femur fractures.

PURPOSE: Results from observational studies on the same exposure-outcome association may be inconsistent because of variations in methodological factors, clinical factors or health care systems. We evaluated the consistency of results assessing the association between antidepressant use and the risk of hip/femur fractures in three European primary care databases using two different study designs. METHODS: Cohort and nested case control studies were conducted in three European primary care databases (Spanish BIFAP, Dutch Mondriaan and UK THIN) to assess the association between use of antidepressants and hip/femur fracture. A common protocol and statistical analysis plan was applied to harmonize study design and conduct between data sources. RESULTS: Current use of antidepressants was consistently associated with a 1.5 to 2.5-fold increased risk of hip/femur fractures in all data sources with both designs, with estimates for SSRIs generally higher than those for TCAs. In general, risk estimates in Mondriaan, the smallest data source, were higher compared to the other data sources. This difference may be partially explained by an interaction between SSRI and age in Mondriaan. Adjustment for GP-recorded lifestyle factors and matching on general practice had negligible impact on adjusted relative risk estimates. CONCLUSION: We found a consistent increased risk of hip/femur fracture with current use of antidepressants across different databases and different designs. Applying similar pharmacoepidemiological study methods resulted in similar risks for TCA use and some variation for SSRI use. Some of these differences may express real (or natural) variance in the exposure-outcome co-occurrences.

Case-only designs for studying the association of antidepressants and hip or femur fracture.

PURPOSE: The purpose of this study is to evaluate the performance and validity of the case-crossover (CCO) and self-controlled case-series (SCCS) designs when studying the association between hip/femur fracture (HF) and antidepressant (AD) use in general practitioner databases. In addition, comparability with cohort and case-control designs is discussed. METHODS: Adult patients with HF and who received an AD prescription during 2001-2009 were identified from UK's The Health Improvement Network (THIN) and the Dutch Mondriaan databases. AD exposure was classified into current, recent and past/non-use (reference). In the CCO, for each patient, a case moment (date of HF) and four prior control moments at -91, -182, -273 and -365 days were defined. In SCCS, incidence of HF was compared between exposure states. Conditional logistic regression was used in the CCO and Poisson regression in the SCCS to compute odds ratios and incidence rate ratios, respectively. In CCO, we adjusted for time-varying co-medication and in SCCS for age. RESULTS: Adjusted estimates for the effect of current AD exposure on HF were higher in the CCO (co-medication-adjusted odds ratio, THIN: 2.24, 95% confidence interval [CI]: 2.04-2.47; Mondriaan: 2.57, 95%CI [1.50, 4.43]) than in the SCCS (age-adjusted incidence rate ratio, THIN: 1.41, 95%CI [1.32, 1.49]; Mondriaan: 2.14, 95%CI [1.51, 3.03]). The latter were comparable with the traditional designs. CONCLUSION: Case-only designs confirmed the association between AD and HF. The CCO design violated assumptions in this study with regard to exchangeability and length of exposure, and transient effects on outcome. The SCCS seems to be an appropriate design for assessing AD-HF association.

Instrumental variables analysis using multiple databases: an example of antidepressant use and risk of hip fracture.

PURPOSE: Instrumental variable (IV) analysis can control for unmeasured confounding, yet it has not been widely used in pharmacoepidemiology. We aimed to assess the performance of IV analysis using different IVs in multiple databases in a study of antidepressant use and hip fracture. METHODS: Information on adults with at least one prescription of a selective serotonin reuptake inhibitor (SSRI) or tricyclic antidepressant (TCA) during 2001-2009 was extracted from the THIN (UK), BIFAP (Spain), and Mondriaan (Netherlands) databases. IVs were created using the proportion of SSRI prescriptions per practice or using the one, five, or ten previous prescriptions by a physician. Data were analysed using conventional Cox regression and two-stage IV models. RESULTS: In the conventional analysis, SSRI (vs. TCA) was associated with an increased risk of hip fracture, which was consistently found across databases: the adjusted hazard ratio (HR) was approximately 1.35 for time-fixed and 1.50 to 2.49 for time-varying SSRI use, while the IV analysis based on the IVs that appeared to satisfy the IV assumptions showed conflicting results, e.g. the adjusted HRs ranged from 0.55 to 2.75 for time-fixed exposure. IVs for time-varying exposure violated at least one IV assumption and were therefore invalid. CONCLUSIONS: This multiple database study shows that the performance of IV analysis varied across the databases for time-fixed and time-varying exposures and strongly depends on the definition of IVs. It remains challenging to obtain valid IVs in pharmacoepidemiological studies, particularly for time-varying exposure, and IV analysis should therefore be interpreted cautiously.

Good Signal Detection Practices: Evidence from IMI PROTECT.

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.

Performance of Stratified and Subgrouped Disproportionality Analyses in Spontaneous Databases.

INTRODUCTION: Disproportionality analyses are used in many organisations to identify adverse drug reactions (ADRs) from spontaneous report data. Reporting patterns vary over time, with patient demographics, and between different geographical regions, and therefore subgroup analyses or adjustment by stratification may be beneficial. OBJECTIVE: The objective of this study was to evaluate the performance of subgroup and stratified disproportionality analyses for a number of key covariates within spontaneous report databases of differing sizes and characteristics. METHODS: Using a reference set of established ADRs, signal detection performance (sensitivity and precision) was compared for stratified, subgroup and crude (unadjusted) analyses within five spontaneous report databases (two company, one national and two international databases). Analyses were repeated for a range of covariates: age, sex, country/region of origin, calendar time period, event seriousness, vaccine/non-vaccine, reporter qualification and report source. RESULTS: Subgroup analyses consistently performed better than stratified analyses in all databases. Subgroup analyses also showed benefits in both sensitivity and precision over crude analyses for the larger international databases, whilst for the smaller databases a gain in precision tended to result in some loss of sensitivity. Additionally, stratified analyses did not increase sensitivity or precision beyond that associated with analytical artefacts of the analysis. The most promising subgroup covariates were age and region/country of origin, although this varied between databases. CONCLUSIONS: Subgroup analyses perform better than stratified analyses and should be considered over the latter in routine first-pass signal detection. Subgroup analyses are also clearly beneficial over crude analyses for larger databases, but further validation is required for smaller databases.

Comparison of statistical signal detection methods within and across spontaneous reporting databases.

BACKGROUND: Most pharmacovigilance departments maintain a system to identify adverse drug reactions (ADRs) through analysis of spontaneous reports. The signal detection algorithms (SDAs) and the nature of the reporting databases vary between operators and it is unclear whether any algorithm can be expected to provide good performance in a wide range of environments. OBJECTIVE: The objective of this study was to compare the performance of commonly used algorithms across spontaneous reporting databases operated by pharmaceutical companies and national and international pharmacovigilance organisations. METHODS: 220 products were chosen and a reference set of ADRs was compiled. Within four company, one national and two international databases, 15 SDAs based on five disproportionality methods were tested. Signals of disproportionate reporting (SDRs) were calculated at monthly intervals and classified by comparison with the reference set. These results were summarised as sensitivity and precision for each algorithm in each database. RESULTS: Different algorithms performed differently between databases but no method dominated all others. Performance was strongly dependent on the thresholds used to define a statistical signal. However, the different disproportionality statistics did not influence the achievable performance. The relative performance of two algorithms was similar in different databases. Over the lifetime of a product there is a reduction in precision for any method. CONCLUSIONS: In designing signal detection systems, careful consideration should be given to the criteria that are used to define an SDR. The choice of disproportionality statistic does not appreciably affect the achievable range of signal detection performance and so this can primarily be based on ease of implementation, interpretation and minimisation of computing resources. The changes in sensitivity and precision obtainable by replacing one algorithm with another are predictable. However, the absolute performance of a method is specific to the database and is best assessed directly on that database. New methods may be required to gain appreciable improvements.