A Model for Monitoring Spontaneously Reported Medication Errors Using the Adjuvanted Recombinant Zoster Vaccine as an Example.

A European legislation was put in place for the reporting of medication errors, and guidelines were drafted to help stakeholders in the reporting, evaluation, and, ultimately, minimization of these errors. As part of pharmacovigilance reporting, a proper classification of medication errors is needed. However, this process can be tedious, time-consuming, and resource-intensive. To fulfill this obligation regarding medication errors, we developed an algorithm that classifies the reported errors in an automated way into four categories: potential medication errors, intercepted medication errors, medication errors without harm (i.e., not associated with adverse reaction(s)), and medication errors with harm (i.e., associated with adverse reaction(s)). A fifth category ("conflicting category") was created for reported cases that could not be unambiguously classified as either potential or intercepted medication errors. Our algorithm defines medication error categories based on internationally accepted terminology using the Medical Dictionary for Regulatory Activities (MedDRA®) preferred terms. We present the algorithm and the strengths of this automated way of reporting medication errors. We also give examples of visualizations using spontaneously reported vaccination error data associated with the adjuvanted recombinant zoster vaccine. For this purpose, we used a customized web-based platform that uses visualizations to support safety signal detection. The use of the algorithm facilitates and ensures a consistent way of categorizing medication errors with MedDRA® terms, thereby saving time and resources and avoiding the risk of potential mistakes versus manual classification. This allows further assessment and potential prevention of medication errors. In addition, the algorithm is easy to implement and can be used to categorize medication errors from different databases.

Post-Marketing Safety Surveillance for the Adjuvanted Recombinant Zoster Vaccine: Methodology.

A diligent, systematic, regular review of aggregate safety data is essential, particularly early after vaccine introduction, as this is when safety signals not identified during clinical development may emerge. In October 2017, the US Centers for Disease Control and Prevention Advisory Committee on Immunization Practices recommended the adjuvanted recombinant zoster vaccine (RZV; Shingrix, GSK) as the preferred vaccine for preventing herpes zoster (HZ) and related complications in immunocompetent adults aged ≥ 50 years. Subsequently, GSK experienced an unprecedented high demand for RZV. In this methodology paper, we summarize the enhanced measures undertaken to assess RZV safety during its early post-marketing experience in the USA, Canada and Germany. In addition to the routine signal-detection methods already in place for all vaccines, GSK established tailored and enhanced safety monitoring for RZV based on aggregate data of spontaneous reports and manufacturing data. Proactive, near real-time detection and evaluation of signals was a key objective. A dedicated in-house signal-detection tool customized for RZV was employed on a weekly (rather than the routine monthly) basis, allowing for a centralized, more frequent review of data on a single web-based platform. We also identified the background incidence rates of preselected medical events of interest in the first countries to introduce RZV (USA, Canada and Germany) to perform observed-to-expected analyses. This approach may offer a solution to the challenges associated with the assessment and monitoring of vaccine safety in an efficient and timely manner in the context of high vaccine uptake.

Review of the initial post-marketing safety surveillance for the recombinant zoster vaccine.

BACKGROUND: The adjuvanted recombinant zoster vaccine (RZV) received its first marketing authorization in October 2017, for prevention of herpes zoster in individuals aged ≥50 years. METHODS: We summarized safety information, following RZV administration, received by GSK via spontaneous adverse event (AE) reports submitted by healthcare providers, vaccine recipients and other reporters. Observed-to-expected (O/E) analyses were performed for selected outcomes: reports of death, Guillain-Barré syndrome and Bell's palsy. Standard case definitions were used to assess individual case reports. Data mining, using proportional reporting ratio and time-to-onset signal detection methods, was employed to identify RZV-AE pairs with disproportionate reporting or unexpected time-to-onset distribution. RESULTS: Between October 13, 2017 and February 10, 2019, an estimated 9.3 million doses were distributed and GSK received 15,638 spontaneous AE reports involving RZV. Most reports were classified as non-serious (95.3%) and originated from the United States (81.7%), where the majority of doses were distributed. Among reports with age or sex reported, individuals were mainly 50-69-year-olds (62.1%) and female (66.7%). Of all reports, 3,579 (22.9%) described vaccination errors, of which 82.7% were without associated symptoms. Of all vaccination error reports, most described errors of vaccine preparation and reconstitution (29.7%), inappropriate schedule or incomplete course of administration (26.7%), incorrect route of administration (16.4%), and storage errors (12.9%). The most commonly reported symptoms were consistent with the known RZV reactogenicity profile observed in clinical trials, including injection site reactions, pyrexia, chills, fatigue, headache. O/E analyses for selected outcomes and data mining analyses for all reported AEs did not identify any unexpected patterns. CONCLUSIONS: Review of the initial data from the post-marketing safety surveillance showed that the safety profile of RZV is consistent with that previously observed in pre-licensure clinical trials. Other studies are ongoing and planned, to continue generating real-world safety data and further characterize RZV.

Tree-based scan statistic - Application in manufacturing-related safety signal detection.

BACKGROUND AND OBJECTIVES: Over the last decades, medicinal regulations have been put into place and have considerably improved manufacturing practices. Nevertheless, safety issues may still arise. Using the simulation described in this manuscript, our aim is to develop adequate detection methods for manufacturing-related safety signals, especially in the context of biological products. METHODS: Pharmaceutical companies record the entire batch genealogies, from seed batches over intermediates to final product (FP) batches. We constructed a hierarchical tree based on this genealogy information and linked it to the spontaneous safety data available for the FP batch numbers. The tree-based scan statistic (TBSS) was used on simulated data as a proof of concept to locate the source that may have subsequently generated an excess of specific adverse events (AEs) within the manufacturing steps, and to evaluate the method's adjustment for multiple testing. All calculations were performed with a customized program in SAS v9.2. RESULTS: The TBSS generated a close to expected number of false positive signals, demonstrating that it adjusted for multiple testing. Overall, the method detected 71% of the simulated signals at the correct production step when a 6-fold increase in reports with AEs of interest (AEOI) was applied, and 31% when a 2-fold increase was applied. The relatively low detection performance may be attributed to the higher granularity associated with the lower levels of the hierarchy, leading to a lack of power and the stringent definition criteria that were applied for a true positive result. CONCLUSION: As a data-mining method for manufacturing-related safety signal detection, the TBSS may provide advantages over other disproportionality analyses (using batch information) but may benefit from complementary methods (not relaying on batch information). While the method warrants further refinement, it may improve safety signal detection and contribute to improvements in the quality of manufacturing processes.

Good Signal Detection Practices: Evidence from IMI PROTECT.

Over a period of 5 years, the Innovative Medicines Initiative PROTECT (Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium) project has addressed key research questions relevant to the science of safety signal detection. The results of studies conducted into quantitative signal detection in spontaneous reporting, clinical trial and electronic health records databases are summarised and 39 recommendations have been formulated, many based on comparative analyses across a range of databases (e.g. regulatory, pharmaceutical company). The recommendations point to pragmatic steps that those working in the pharmacovigilance community can take to improve signal detection practices, whether in a national or international agency or in a pharmaceutical company setting. PROTECT has also pointed to areas of potentially fruitful future research and some areas where further effort is likely to yield less.

Investigating Reports of Complex Regional Pain Syndrome: An Analysis of HPV-16/18-Adjuvanted Vaccine Post-Licensure Data.

Complex regional pain syndrome (CRPS) is a chronic pain disorder that typically follows trauma or surgery. Suspected CRPS reported after vaccination with human papillomavirus (HPV) vaccines led to temporary suspension of proactive recommendation of HPV vaccination in Japan. We investigated the potential CRPS signal in relation to HPV-16/18-adjuvanted vaccine (Cervarix®) by database review of CRPS cases with independent expert confirmation; a disproportionality analysis and analyses of temporality; an observed versus expected analysis using published background incidence rates; systematic reviews of aggregate safety data, and a literature review. The analysis included 17 case reports of CRPS: 10 from Japan (0.14/100,000 doses distributed) and seven from the United Kingdom (0.08/100,000). Five cases were considered by independent experts to be confirmed CRPS. Quantitative analyses did not suggest an association between CRPS and HPV-16/18-adjuvanted vaccine. Observed CRPS incidence after HPV-16/18 vaccination was statistically significantly below expected rates. Systematic database reviews using search terms varying in specificity and sensitivity did not identify new cases. No CRPS was reported during clinical development and no unexpected results found in the literature. There is not sufficient evidence to suggest an increased risk of developing CRPS following vaccination with HPV-16/18-adjuvanted vaccine. Post-licensure safety surveillance confirms the acceptable benefit-risk of HPV-16/18 vaccination.

Modelling Hospitalisation Ratios for Febrile Convulsions and Severe Varicella Under Combined Measles, Mumps, Rubella, and Varicella (MMRV-Priorix-Tetra™) Compared to Separate MMR + V Vaccination.

INTRODUCTION: Measles, mumps, rubella, and varicella combination vaccines (MMRV) facilitate varicella vaccination uptake compared with separate administration of measles, mumps, and rubella vaccine (MMR) with varicella vaccine (V). However, the risk of developing febrile convulsions (FC) is higher in children vaccinated with MMRV. OBJECTIVES: The aim was to demonstrate how to put the increased FC risk associated with MMRV into perspective by comparing it with the lower V-coverage risk associated with MMR + V. METHODS: FC and varicella burdens were measured by total numbers or duration of hospitalisations. A model, based on several assumptions and integrating parameters from heterogeneous data sources relevant to Germany, was developed to evaluate hospitalisation ratios (HRs; ratios between yearly numbers of varicella-related hospitalisation days prevented by MMRV and yearly numbers of FC-related hospitalisation days attributed to MMRV, both compared with MMR + V). A sensitivity analysis estimated HR under different scenarios beyond the German experience. RESULTS: For parameter values compatible with the German experience, where MMRV (Priorix-Tetra™, GSK, Belgium) was introduced in 2006, the model predicted that transitioning from MMR + V to MMRV would induce 225 vaccine-related FC hospitalisation days whilst preventing 1976 varicella-related hospitalisation days per year. The HR estimated by Monte Carlo simulations was 8.5 (95 % confidence interval: 1.99-25.22). A sensitivity analysis on two key parameters suggested that transitioning from MMR + V to MMRV would be favourable in situations where MMRV use would significantly impact varicella vaccination uptake. CONCLUSIONS: MMRV use instead of MMR + V can substantially reduce the number of hospitalisation days, despite increased FC risk when MMRV is used as a first dose of measles-containing vaccine.

Post-marketing monitoring of intussusception after rotavirus vaccination in Japan.

PURPOSE: Rotarix(TM) was launched in November 2011 in Japan to prevent rotavirus gastroenteritis. Some studies suggest that Rotarix(TM) may have a temporal association with a risk of intussusception (IS). We assessed a possible association between IS and Rotarix(TM) vaccination in Japan. METHODS: All IS cases spontaneously reported post-vaccination (Brighton collaboration levels 1, 2, and 3) were extracted from the GlaxoSmithKline spontaneous report database on the 11th of January 2013. Expected numbers of IS cases were estimated using the number of vaccine doses distributed and the Japanese incidence rate of IS stratified by month of age. The observed versus expected analysis considered the IS cases for each risk period (7 and 30 days post-vaccination) and for each vaccine dose (two doses). RESULTS: Before January 2013, approximately 601 000 Rotarix(TM) doses were distributed in Japan. For a risk period of 7 days post-dose 1 and post-dose 2, 10 and five IS cases were observed, whereas 3.4 and 7.6 were expected, providing an observed-to-expected ratio of 2.96 (95% confidence interval [CI]: 1.42; 5.45) and 0.66 (95% CI: 0.21; 1.53), respectively. For a risk period of 30 days post-dose 1 and post-dose 2, 14 and eight cases were observed, whereas 14.5 and 32.7 were expected, providing an observed-to-expected ratio of 0.97 (95% CI: 0.53; 1.62) and 0.24 (95% CI: 0.11; 0.48), respectively. CONCLUSION: A statistically significant excess of IS cases was observed within 7 days post-dose 1, but not post-dose 2. These results are consistent with previous observations in large post-marketing safety studies in other world regions.

Use of logistic regression to combine two causality criteria for signal detection in vaccine spontaneous report data.

PURPOSE: We evaluated the use of logistic regression to model the probabilities of spontaneously reported vaccine-event pairs being adverse reactions following immunization (ARFI), using disproportionality and unexpectedness of time-to-onset (TTO) distributions as predictive variables and the presence of events in the global product information as a dependent variable. METHODS: We used spontaneous reports of adverse events from eight vaccines and their labels as proxies for ARFIs. Three logistic regressions were built to predict ARFIs based on different combinations of the proportional reporting ratio (PRR; disproportionality measure) and two Kolmogorov-Smirnov (KS) tests ('between vaccines' and the 'between events') of TTO distribution: model 1, using the PRR estimate and its 95% lower confidence interval (CI) limit; model 2, using the p values of the two KS tests; and model 3, using the PRR (point estimate and lower CI limit) and both KS tests. The performance of the regressions (model fit statistics, calibration, and discrimination) was measured on 100 bootstrap samples. RESULTS: Model 3, using two quantified causality criteria, provided the best performance for all measures. The p value of the 'between vaccines' KS test was the most significant predictive factor. Model 1 had the worst performance. CONCLUSIONS: Logistic regression allows estimation of the probability of a vaccine-event pair being an ARFI using two causality criteria at the population level assessed in spontaneous report data: the strength of association (disproportionality measure) and temporality (TTO distribution tests). Logistic regression combines and weights these causality criteria based on their respective ability to predict known safety issues.

Signal detection based on time-to-onset: extending a new method from spontaneous reports to observational studies.

PURPOSE: A proof-of-concept study has previously highlighted the added value of a method using time-to-onset (TTO) for quantitative and non-parametric signal detection on spontaneous report data. The aim of this study was to assess the added value of this new TTO signal detection method adapted to observational studies. METHODS: For each adverse event collected during the conduct of an observational study of H1N1 pandemic influenza vaccine, the TTO distribution was tested against the 'follow-up distribution' from vaccination to 'lost to follow-up' by a Kolmogorov-Smirnov test. Events rejecting the null hypothesis of similar distribution were flagged as signals, and a safety physician evaluated their relevance for further medical assessment. We simulated ongoing surveillance by performing retrospective weekly signal detection based on TTO. RESULTS: The TTO method detected 21, 15 and 4 signals within a 30-day period post-dose 1 with confidence levels set at 90%, 95% and 99%, respectively. Of these signals, 14 (67%), 10 (67%) and 2 (50%) were considered as relevant. Among the 14, six had not been identified by previous signal detection activities. When performed weekly, the Kolmogorov-Smirnov test detected 26 events as signals (alpha = 0.05). Three weeks after first participant first dose, one of the six new signals could theoretically have been detected. CONCLUSIONS: This study provided evidence that the Kolmogorov-Smirnov method can screen all TTO distributions and objectively flag the unexpected, leading to earlier detection of signals, and thus potential safety issues.

The upper bound to the Relative Reporting Ratio-a measure of the impact of the violation of hidden assumptions underlying some disproportionality methods used in signal detection.

PURPOSE: For disproportionality measures based on the Relative Reporting Ratio (RRR) such as the Information Component (IC) and the Empirical Bayesian Geometrical Mean (EBGM), each product and event is assumed to represent a negligible fraction of the spontaneous report database (SRD). Here, we provide the tools for allowing signal detection experts to assess the consequence of the violation of this assumption on their specific SRD. METHODS: For each product-event pair (P-E), a worst-case scenario associated all the reported events-of-interest with the product of interest. The values of the RRR under this scenario were measured for different sets of stratification factors using the GlaxoSmithKline vaccines SRD. These values represent the RRR upper bound that RRR cannot exceed whatever the true strength of association. RESULTS: Depending on the choice of stratification factors, the RRR could not exceed an upper bound of 2 for up to 2.4% of the P-Es. For Engerix™, 23.4% of all reports in the SDR, the RRR could not exceed an upper bound of 2 for up to 13.8% of pairs. For the P-E Rotarix™-Intussusception, the choice of stratification factors impacted the upper bound to RRR: from 52.5 for an unstratified RRR to 2.0 for a fully stratified RRR. CONCLUSIONS: The quantification of the upper bound can indicate whether measures such as EBGM, IC, or RRR can be used for SRD for which products or events represent a non-negligible fraction of the entire SRD. In addition, at the level of the product or P-E, it can also highlight detrimental impact of overstratification.

Assessing the extent and impact of the masking effect of disproportionality analyses on two spontaneous reporting systems databases.

BACKGROUND: Masking is a statistical issue by which signals are hidden by the presence of other medicines in the database. In the absence algorithm, the impact of the masking effect has not been fully investigated. OBJECTIVE: Our study is aimed at assessing the extent and the impact of the masking effect on two large spontaneous reporting databases. STUDY DESIGN: Cross sectional study using a set of terms of importance for public health in two spontaneous reporting databases. SETTING: The analyses were performed on EudraVigilance (EV) and the Pfizer spontaneous reporting database (PfDB). MAIN OUTCOME MEASURE: Using the masking ratio, we have identified and removed the products inducing the highest masking effect. RESULTS: Studying a total of almost 50 000 drug-event combinations masking had an impact on approximately 60% of drug-event combinations were masked by another product with a masking ratio >1 in EV and 84% in PfDB. The prevalence of important masking was quite rare (0.003% of the DECs) and mainly affected events rarely reported in EV. The products involved in the highest masking effects are products known to induce the reaction. The removal of the masking effect of the highest masking product has revealed 974 signals of disproportionate reporting in EV including true signals. The study shows that the original ranking provided by the quantitative methods included in our study is marginally affected by the removal of the masking product. CONCLUSION: Our study suggests that significant masking is rare in large spontaneous databases and mostly affects events rarely reported in EV.

Signal detection on spontaneous reports of adverse events following immunisation: a comparison of the performance of a disproportionality-based algorithm and a time-to-onset-based algorithm.

PURPOSE: Disproportionality methods measure how unexpected the observed number of adverse events is. Time-to-onset (TTO) methods measure how unexpected the TTO distribution of a vaccine-event pair is compared with what is expected from other vaccines and events. Our purpose is to compare the performance associated with each method. METHODS: For the disproportionality algorithms, we defined 336 combinations of stratification factors (sex, age, region and year) and threshold values of the multi-item gamma Poisson shrinker (MGPS). For the TTO algorithms, we defined 18 combinations of significance level and time windows. We used spontaneous reports of adverse events recorded for eight vaccines. The vaccine product labels were used as proxies for true safety signals. Algorithms were ranked according to their positive predictive value (PPV) for each vaccine separately; amedian rank was attributed to each algorithm across vaccines. RESULTS: The algorithm with the highest median rank was based on TTO with a significance level of 0.01 and a time window of 60 days after immunisation. It had an overall PPV 2.5 times higher than for the highest-ranked MGPS algorithm, 16(th) rank overall, which was fully stratified and had a threshold value of 0.8. A TTO algorithm with roughly the same sensitivity as the highest-ranked MGPS had better specificity but longer time-to-detection. CONCLUSIONS: Within the scope of this study, the majority of the TTO algorithms presented a higher PPV than for any MGPS algorithm. Considering the complementarity of TTO and disproportionality methods, a signal detection strategy combining them merits further investigation. © 2013 GlaxoSmithKline. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.

Optimization of a quantitative signal detection algorithm for spontaneous reports of adverse events post immunization.

PURPOSE: To optimize the efficiency of signal detection by maximizing the proportion of true positive (TP) signals among signals detected by a disproportionality algorithm. METHODS: We compared 176 different combinations of stratification factors, sex (S), age (A), region (R) and year of report (Y), and cut-off values of a Multi-Item Gamma Poisson Schrinker (MGPS) algorithm. Spontaneous adverse event reports of eight vaccines from the GlaxoSmithKline Biologicals safety database were used. Defining events listed in the Product Information as proxy of true safety signals, we compared each algorithm performance in terms of positive predictive value (PPV). For each vaccine, each algorithm was ranked according to PPV. Median rank and overall PPV were computed across vaccines. RESULTS: For a standard cut-off of 2, the optimal stratification factors differed by vaccine and led to a set of algorithms with a median rank of 34.5 (PPV = 0.22; 34 TP). Keeping the original SARY stratification led to optimal cut-offs that differed by vaccine and a set of algorithms with a median rank of 1.75 (PPV = 0.20; 142 TP). The optimal combination of cut-off and stratification led to different algorithms by vaccine with a median rank of 1 (PPV = 0.19; 139 TP). The best unique algorithm parameterization across vaccines was 0.8-SARY (cut-off-stratification), with a median rank of 3 (PPV = 0.20; 195 TP). The original 2-SARY was one of the worst algorithms, with a median rank of 150.75 (PPV = 0.13; 8 TP). CONCLUSION: Within the scope of this study, a unique MGPS algorithm across vaccines with the original full stratification but a lowered cut-off provided major performance improvement.

Using time-to-onset for detecting safety signals in spontaneous reports of adverse events following immunization: a proof of concept study.

PURPOSE: Disproportionality analyses (DPA) are widely used in pharmacovigilance for detecting safety signals from spontaneous reports of adverse events. In these analyses, time-to-onset (TTO; the time between vaccination and the onset of the adverse event) is rarely considered. Our objective is to assess the potential use of TTO to improve signal detection (SD). METHODS: Adverse events were defined as signals for a vaccine if the TTO distribution was significantly different from the distribution of other events for the same vaccine and from the distribution obtained with the same event for other vaccines. Distributions were compared within a time window of 30 days by using the two-sample Kolmogorov-Smirnov statistic. With the use of the product label as a proxy of true positive safety signals, TTO SD was compared with a standard DPA method (based on stratified empirical Bayesian geometric mean) for an oral live pediatric vaccine (Rotarix™) and an inactivated adult vaccine (Fluarix™). RESULTS: With the use of the GlaxoSmithKline spontaneous reports database for Rotarix™, 10 Medical Dictionary for Regulatory Activities preferred terms were identified as signals, and among them, five were listed in the product label. The DPA method identified only three preferred terms from the label, that is, TTO SD showed higher sensitivity and specificity. For Fluarix™, TTO SD also showed higher sensitivity but lower specificity. CONCLUSION: This TTO SD method is complementary, conceptually and practically, to more traditional DPA and does not share the major drawback of DPA known as the masking effect. Higher sensitivity and/or specificity can be achieved using TTO SD.