Good Signal Detection Practices: Evidence from IMI PROTECT.

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.

An Empirical Approach to Explore the Relationship Between Measures of Disproportionate Reporting and Relative Risks from Analytical Studies.

INTRODUCTION: Although it seems reasonable to suppose that a drug that increases the risk of an adverse event might tend to show increased disproportionality statistics in spontaneous reporting databases, that relationship is not clear. Therefore, an empirical approach was taken to investigate the relationship between proportional reporting ratios (PRRs) and relative risk (RR) estimates from formal studies in a set of known adverse drug reactions (ADRs). METHODS: Drug-event pairs that were the subject of pharmacovigilance-driven European regulatory actions from 2007 to 2010 were selected. Only pairs having RR derived from formal studies and where it was considered that there was well-established evidence supporting the actions were included. A best estimate of the RR for each ADR was chosen based on pre-specified rules. PRRs were then calculated in Eudravigilance using only those cases reported before the date of first recognition of the ADR in the medical community. An additional analysis was carried out in FEDRA, the Spanish spontaneous reports database. A descriptive analysis and an orthogonal regression model were performed. RESULTS: From an initial dataset of 78 drug-event pairs, 15 were selected. The regression model (ln RR = 0.203 + 0.463 × ln PRR) showed a significant (p < 0.001) correlation between RR and PRR in Eudravigilance. None of the ADR-related variables analysed modified the relationship. Exploratory results in FEDRA went in the same direction. CONCLUSIONS: Disproportionality measures should not replace formal studies but could provide an initial indication of the likely clinical importance of an ADR, should the signal be confirmed subsequently. Whether the same conclusions can be applied to other datasets should be further studied.

Postmarketing Safety Study Tool: A Web Based, Dynamic, and Interoperable System for Postmarketing Drug Surveillance Studies.

Postmarketing drug surveillance is a crucial aspect of the clinical research activities in pharmacovigilance and pharmacoepidemiology. Successful utilization of available Electronic Health Record (EHR) data can complement and strengthen postmarketing safety studies. In terms of the secondary use of EHRs, access and analysis of patient data across different domains are a critical factor; we address this data interoperability problem between EHR systems and clinical research systems in this paper. We demonstrate that this problem can be solved in an upper level with the use of common data elements in a standardized fashion so that clinical researchers can work with different EHR systems independently of the underlying information model. Postmarketing Safety Study Tool lets the clinical researchers extract data from different EHR systems by designing data collection set schemas through common data elements. The tool interacts with a semantic metadata registry through IHE data element exchange profile. Postmarketing Safety Study Tool and its supporting components have been implemented and deployed on the central data warehouse of the Lombardy region, Italy, which contains anonymized records of about 16 million patients with over 10-year longitudinal data on average. Clinical researchers in Roche validate the tool with real life use cases.

Comparison of statistical signal detection methods within and across spontaneous reporting databases.

BACKGROUND: Most pharmacovigilance departments maintain a system to identify adverse drug reactions (ADRs) through analysis of spontaneous reports. The signal detection algorithms (SDAs) and the nature of the reporting databases vary between operators and it is unclear whether any algorithm can be expected to provide good performance in a wide range of environments. OBJECTIVE: The objective of this study was to compare the performance of commonly used algorithms across spontaneous reporting databases operated by pharmaceutical companies and national and international pharmacovigilance organisations. METHODS: 220 products were chosen and a reference set of ADRs was compiled. Within four company, one national and two international databases, 15 SDAs based on five disproportionality methods were tested. Signals of disproportionate reporting (SDRs) were calculated at monthly intervals and classified by comparison with the reference set. These results were summarised as sensitivity and precision for each algorithm in each database. RESULTS: Different algorithms performed differently between databases but no method dominated all others. Performance was strongly dependent on the thresholds used to define a statistical signal. However, the different disproportionality statistics did not influence the achievable performance. The relative performance of two algorithms was similar in different databases. Over the lifetime of a product there is a reduction in precision for any method. CONCLUSIONS: In designing signal detection systems, careful consideration should be given to the criteria that are used to define an SDR. The choice of disproportionality statistic does not appreciably affect the achievable range of signal detection performance and so this can primarily be based on ease of implementation, interpretation and minimisation of computing resources. The changes in sensitivity and precision obtainable by replacing one algorithm with another are predictable. However, the absolute performance of a method is specific to the database and is best assessed directly on that database. New methods may be required to gain appreciable improvements.

Post-marketing assessment of neuropsychiatric adverse events in influenza patients treated with oseltamivir: an updated review.

A 2008 review by our group concluded that the risk of neuropsychiatric adverse events (NPAEs) in influenza patients was not increased by oseltamivir exposure, and did not identify any mechanism by which oseltamivir or its metabolites could cause or worsen such events. The current article reviews new information on this topic. Between September 16, 2007 and May 15, 2010, 1,805 spontaneously-reported NPAEs were identified in 1,330 patients receiving oseltamivir: 767 (42.5%) from Japan, 296 (16.4%) from the USA, and 742 (41.1%) from other countries. NPAEs were more common in children: 1,072 (59.4%) events were in those aged ≤16 years. NPAEs often occurred within 48 h of treatment initiation (953 events; 52.8%). Nearly half of the events were serious in nature (838; 46.4%). The three largest categories of events were abnormal behavior (457 events, 25.3%), miscellaneous psychiatric events (370; 20.5%), and delusions/perceptual disturbances (316 events, 17.5%). A total of 1,545 events (85.6%) in eight different categories were considered to be delirium or delirium-like. Twenty-eight suicide-related events were reported. A US healthcare claims database analysis showed that the risk of NPAEs in 7,798 oseltamivir-treated patients was no higher than that in 10,411 patients not on antivirals, but a study on oseltamivir and abnormal behavior in Japan was less conclusive. NPAE frequency in oseltamivir-exposed Japanese and Taiwanese children with influenza was the same as in unexposed children. New analysis of the UK General Practice Research Database showed that the relative adjusted risk of NPAEs in influenza patients was 2.18-times higher than in the general population. Other epidemiology studies report frequent occurrence of encephalitis and similar disorders in influenza patients independently of oseltamivir exposure. The new data support the findings of the original assessment. Evidence suggests that influenza-related encephalopathies are caused by influenza-induced inflammatory responses, but more work is needed to confirm the underlying mechanisms.

Assessment of neuropsychiatric adverse events in influenza patients treated with oseltamivir: a comprehensive review.

After reports from Japan of neuropsychiatric adverse events (NPAEs) in children taking oseltamivir phosphate (hereafter referred to as oseltamivir [Tamiflu; F. Hoffmann-La Roche Ltd, Basel, Switzerland]) during and after the 2004--5 influenza season, Roche explored possible reasons for the increase in reporting rate and presented regular updates to the US FDA and other regulatory authorities. This review summarizes the results of a comprehensive assessment of the company's own preclinical and clinical studies, post-marketing spontaneous adverse event reporting, epidemiological investigations utilizing health claims and medical records databases and an extensive review of the literature, with the aim of answering the following questions: (i) what the types and rates of neuropsychiatric abnormalities reported in patients with influenza are, and whether these differ in patients who have received oseltamivir compared with those who have not; (ii) what levels of oseltamivir and its active metabolite, oseltamivir carboxylate are achieved in the CNS; (iii) whether oseltamivir and oseltamivir carboxylate have pharmacological activity in the CNS; and (iv) whether there are genetic differences between Japanese and Caucasian patients that result in different levels of oseltamivir and/or oseltamivir carboxylate in the CNS, differences in their metabolism or differences in their pharmacological activity in the CNS. In total, 3051 spontaneous reports of NPAEs were received by Roche, involving 2466 patients who received oseltamivir between 1999 and 15 September 2007; 2772 (90.9%) events originated from Japan, 190 (6.2%) from the US and 89 (2.9%) from other countries. During this period, oseltamivir was prescribed to around 48 million people worldwide. Crude NPAE reporting rates (per 1,000,000 prescriptions) in children (aged < or =16 years) and adults, respectively, were 99 and 28 events in Japan and 19 and 8 in the US. NPAEs were more commonly reported in children (2218 events in 1808 children aged < or =16 years vs 833 in 658 adults) and generally occurred within 48 hours of the onset of influenza illness and initiation of treatment. After categorizing the reported events according to International Classification of Diseases (9th edition) codes, abnormal behaviour (1160 events, 38.0%) and delusions/perceptual disturbances (661 events, 21.7%) were the largest categories of events, and delirium or delirium-like events (as defined by the American Psychiatric Association) were very common in most categories. No difference in NPAE reporting rates between oseltamivir and placebo was found in phase III treatment studies (0.5% vs 0.6%). Analyses of US healthcare claims databases showed the risk of NPAEs in oseltamivir-treated patients (n = 159,386) was no higher than those not receiving antivirals (n = 159,386). Analysis of medical records in the UK General Practice Research Database showed that the adjusted relative risk of NPAEs in influenza patients was significantly higher (1.75-fold) than in the general population. Based on literature reports, NPAEs in Japanese and Taiwanese children with influenza have occurred before the initiation of oseltamivir treatment; events were also similar to those occurring after the initiation of oseltamivir therapy. No clinically relevant differences in plasma pharmacokinetics of oseltamivir and its active metabolite oseltamivir carboxylate were noted between Japanese and Caucasian adults or children. Penetration into the CNS of both oseltamivir and oseltamivir carboxylate was low in Japanese and Caucasian adults (cerebrospinal fluid/plasma maximum concentration and area under the plasma concentration-time curve ratios of approximately 0.03), and the capacity for converting oseltamivir to oseltamivir carboxylate in rat and human brains was low. In animal autoradiography and pharmacokinetic studies, brain : plasma radioactivity ratios were generally 20% or lower. Animal studies showed no specific CNS/behavioural effects after administration of doses corresponding to > or =100 times the clinical dose. Oseltamivir or oseltamivir carboxylate did not interact with human neuraminidases or with 155 known molecular targets in radioligand binding and functional assays. A review of the information published to date on functional variations of genes relevant to oseltamivir pharmacokinetics and pharmacodynamics and simulated gene knock-out scenarios did not identify any plausible genetic explanations for the observed NPAEs. The available data do not suggest that the incidence of NPAEs in influenza patients receiving oseltamivir is higher than in those who do not, and no mechanism by which oseltamivir or oseltamivir carboxylate could cause or worsen such events could be identified.

Detecting signals of drug-drug interactions in a spontaneous reports database.

AIMS: The spontaneous reports database is widely used for detecting signals of ADRs. We have extended the methodology to include the detection of signals of ADRs that are associated with drug-drug interactions (DDI). In particular, we have investigated two different statistical assumptions for detecting signals of DDI. METHODS: Using the FDA's spontaneous reports database, we investigated two models, a multiplicative and an additive model, to detect signals of DDI. We applied the models to four known DDIs (methotrexate-diclofenac and bone marrow depression, simvastatin-ciclosporin and myopathy, ketoconazole-terfenadine and torsades de pointes, and cisapride-erythromycin and torsades de pointes) and to four drug-event combinations where there is currently no evidence of a DDI (fexofenadine-ketoconazole and torsades de pointes, methotrexade-rofecoxib and bone marrow depression, fluvastatin-ciclosporin and myopathy, and cisapride-azithromycine and torsade de pointes) and estimated the measure of interaction on the two scales. RESULTS: The additive model correctly identified all four known DDIs by giving a statistically significant (P < 0.05) positive measure of interaction. The multiplicative model identified the first two of the known DDIs as having a statistically significant or borderline significant (P < 0.1) positive measure of interaction term, gave a nonsignificant positive trend for the third interaction (P = 0.27), and a negative trend for the last interaction. Both models correctly identified the four known non interactions by estimating a negative measure of interaction. CONCLUSIONS: The spontaneous reports database is a valuable resource for detecting signals of DDIs. In particular, the additive model is more sensitive in detecting such signals. The multiplicative model may further help qualify the strength of the signal detected by the additive model.

Health outcomes among patients receiving oseltamivir.

PURPOSE: Oseltamivir was approved by the FDA for the treatment of influenza in 1999. The primary objective was to compare health outcomes among influenza patients who were treated with oseltamivir and those who were not. METHODS: The patient population included UnitedHealthcare members who received an influenza diagnosis during the 1999-2000 influenza season, divided into those who were dispensed oseltamivir on the same day (N = 3211) and those who were not dispensed oseltamivir (N = 19,985). Cardiovascular, neuropsychiatric and respiratory outcomes were assessed from medical claims for a 30-day period. RESULTS: The adjusted incidence rate ratio for major cardiac outcome was 0.56 (95%CI: 0.34-0.93) in those without a positive history of major cardiac disease, indicating that those in the oseltamivir group tended to be at lower risk of cardiac outcomes than those without oseltamivir. The adjusted incidence rate ratio for major neuropsychiatric outcome was 0.72 (95%CI: 0.53-0.97) in the negative history of neuropsychiatric disease stratum. The incidence rate ratios for respiratory events were more variable. CONCLUSIONS: There appears to be no increased risk of cardiac or neuropsychiatric outcomes among subjects with influenza who were treated with oseltamivir in comparison with those who were not.

Safety and pharmacology of oseltamivir in clinical use.

Oseltamivir is a novel agent approved for the treatment and prevention of influenza infection and illnesses in adults and children. Assessment of data from the clinical trial programme, a US health insurance database study and postmarketing surveillance allowed a comprehensive review of the safety of oseltamivir in clinical use in subjects >1 year of age. Oseltamivir has been studied over the course of a 5-year development programme in >11000 subjects from North America, Europe and the Southern Hemisphere, including otherwise healthy adults, approximately 500 elderly/high-risk subjects, and children (>1000) aged 1-12 years. Safety evaluations included treatment-emergent adverse events, hospitalisations and deaths, as well as haematological and biochemical laboratory safety tests. The data reveals that oseltamivir has simple, uncomplicated pharmacology and lacks potential for drug-drug interactions. Electrocardiogram parameters, including corrected QT interval, were unaffected by oseltamivir even at high doses. Postmarketing studies confirmed that transient gastrointestinal disturbance is the major adverse effect of oseltamivir and that this can be reduced by taking oseltamivir after a light snack. On treatment serious adverse events were reported in 1.3% of oseltamivir 75mg twice daily, 0.7% of oseltamivir 150 mg twice daily and 1.2% of placebo recipients, respectively, in the clinical trial programme. Postmarketing, it is estimated that, to date, over 4 million oseltamivir prescriptions have been dispensed worldwide. Approximately 2300 spontaneous reports were received by the manufacturer over the three winter seasons of use. As these events are reported infrequently and from an unknown number of users, it is not possible to definitively assess causality or frequency of reported events. Most reports were of gastrointestinal and skin reactions. However, a clear association between the skin reactions and oseltamivir has not been established. A large study of insurance records, which permitted the assessment of the relative risk of medical events treated in the month following prescription of oseltamivir in general use, showed no evidence of increased risk of cardiac, neuropsychiatric or respiratory events for those receiving oseltamivir compared with those who did not. To conclude, no important safety concerns have evolved which might limit the suitability of oseltamivir for the treatment and prevention of influenza in all patient populations.