ADVANCE system testing: Benefit-risk analysis of a marketed vaccine using multi-criteria decision analysis and individual-level state transition modelling.

BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using electronic health record (eHR) databases in Europe. Proof-of-concept studies were designed to assess the proposed processes and system for generating the required evidence to perform B/R assessment and near-real time monitoring of vaccines. We aimed to test B/R methodologies for vaccines, using the comparison of the B/R profiles of whole-cell (wP) and acellular pertussis (aP) vaccine formulations in children as an example. METHODS: We used multi-criteria decision analysis (MCDA) to structure the B/R assessment combined with individual-level state transition modelling to build the B/R effects table. In the state transition model, we simulated the number of events in two hypothetical cohorts of 1 million children followed from first pertussis dose till pre-school-entry booster (or six years of age, whichever occurred first), with one cohort receiving wP, and the other aP. The benefits were reductions in pertussis incidence and complications. The risks were increased incidences of febrile convulsions, fever, hypotonic-hyporesponsive episodes, injection-site reactions and persistent crying. Most model parameters were informed by estimates (coverage, background incidences, relative risks) from eHR databases from Denmark (SSI), Spain (BIFAP and SIDIAP), Italy (Pedianet) and the UK (RCGP-RSC and THIN). Preferences were elicited from clinical and epidemiological experts. RESULTS: Using state transition modelling to build the B/R effects table facilitated the comparison of different vaccine effects (e.g. immediate vaccine risks vs long-term vaccine benefits). Estimates from eHR databases could be used to inform the simulation model. The model results could be easily combined with preference weights to obtain B/R scores. CONCLUSION: Existing B/R methodology, modelling and estimates from eHR databases can be successfully used for B/R assessment of vaccines.

ADVANCE system testing: Estimating the incidence of adverse events following pertussis vaccination in healthcare databases with incomplete exposure data.

The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid vaccine benefit-risk monitoring using existing European healthcare databases. Incidence rate (IR) estimates of vaccination-associated adverse events that are needed to model vaccination risks can be calculated from existing healthcare databases when vaccination (exposure) data are available. We assessed different methods to derive IRs in risk periods following vaccination when exposure data are missing in one database, using estimated IRs and IRRs from other databases for febrile seizures, fever and persistent crying. IRs were estimated for children aged 0-5 years in outcome-specific risk and non-risk periods following the first dose of acellular pertussis (aP) vaccination in four primary care databases and one hospital database. We compared derived and observed IRs in each database using three methods: 1) multiplication of non-risk period IR for database i by IR ratio (IRR) obtained from meta-analysis of IRRs estimated using the self-controlled case-series method, from databases other than i; 2) same method as 1, but multiplying with background IR; and 3) meta-analyses of observed IRs from databases other than i. IRs for febrile seizures were lower in primary care databases than the hospital database. The derived IR for febrile seizures using data from primary care databases was lower than that observed in the hospital database, and using data from the hospital database gave a higher derived IR than that observed in the primary care database. For fever and persistent crying the opposite was observed. We demonstrated that missing IRs for a post-vaccination period can be derived but that the type of database and the method of event data capture can have an impact on potential bias. We recommend IRs are derived using data from similar database types (hospital or primary care) with caution as even this can give heterogeneous results.

Age-specific vaccination coverage estimates for influenza, human papillomavirus and measles containing vaccines from seven population-based healthcare databases from four EU countries - The ADVANCE project.

BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk monitoring of vaccines using existing healthcare databases in Europe. We estimated vaccine coverage from electronic healthcare databases as part of a fit-for-purpose assessment for vaccine benefit-risk studies. METHODS: A retrospective dynamic cohort study was conducted through a distributed network approach. Coverage with measles-vaccine for birth year 2006, human papillomavirus (HPV)-vaccine for birth years 1990-2000 and influenza-vaccine for birth years 1920-1950 was estimated using period-prevalence and inverse probability weighting methods. Seven databases from four countries participated: Italy (Pedianet, Val Padana), Spain (BIFAP, SIDIAP), UK (RCGP-RSC, THIN), Denmark (SSI/AUH). Database access providers extracted the data, transformed it into a common structure and ran an R-script locally. The created output tables were shared and pooled at a central server. RESULTS: The total study population comprised 274,616 persons for measles-vaccine, 2,011,666 persons for HPV-vaccine and 14,904,033 persons for influenza-vaccine. Measles-vaccine coverage varied from 84.3% (Denmark) to 96.5% (Italy, Val Padana) for the first dose and from 82.8% (Italy, Val Padana) to 90.9% (UK) for the second dose at the age of 7 years. The HPV-vaccine coverage, aggregated over birth years 1997-2000, ranged from 60% (UK) to 88.3% (Denmark) at the age of 15 years. The influenza-vaccine coverage for the influenza seasons from 2009 to 2015 for persons aged 65 years and more was roughly stable around 43% in Denmark and around 68% in the UK while a decrease from 58 to 50% was observed in Catalonia (Spain). CONCLUSIONS: We obtained detailed, age-specific coverage estimates though a common procedure. We discussed between database comparability and comparability to published national estimates.

ADVANCE system testing: Can safety studies be conducted using electronic healthcare data? An example using pertussis vaccination.

INTRODUCTION: The Accelerated Development of Vaccine benefit-risk Collaboration in Europe (ADVANCE) public-private collaboration, aimed to develop and test a system for rapid benefit-risk monitoring of vaccines using healthcare databases in Europe. The objective of this proof-of-concept (POC) study was to test the feasibility of the ADVANCE system to generate incidence rates (IRs) per 1000 person-years and incidence rate ratios (IRRs) for risks associated with whole cell- (wP) and acellular- (aP) pertussis vaccines, occurring in event-specific risk windows in children prior to their pre-school-entry booster. METHODS: The study population comprised almost 5.1 million children aged 1 month to <6 years vaccinated with wP or aP vaccines during the study period from 1 January 1990 to 31 December 2015. Data from two Danish hospital (H) databases (AUH and SSI) and five primary care (PC) databases from, UK (THIN and RCGP RSC), Spain (SIDIAP and BIFAP) and Italy (Pedianet) were analysed. Database-specific IRRs between risk vs. non-risk periods were estimated in a self-controlled case series study and pooled using random-effects meta-analyses. RESULTS: The overall IRs were: fever, 58.2 (95% CI: 58.1; 58.3), 96.9 (96.7; 97.1) for PC DBs and 8.56 (8.5; 8.6) for H DBs; convulsions, 7.6 (95% CI: 7.6; 7.7), 3.55 (3.5; 3.6) for PC and 12.87 (12.8; 13) for H; persistent crying, 3.9 (95% CI: 3.8; 3.9) for PC, injection-site reactions, 2.2 (95% CI 2.1; 2.2) for PC, hypotonic hypo-responsive episode (HHE), 0.4 (95% CI: 0.4; 0.4), 0.6 (0.6; 0.6) for PC and 0.2 (0.2; 0.3) for H; and somnolence: 0.3 (95% CI: 0.3; 0.3) for PC. The pooled IRRs for persistent crying, fever, and ISR, adjusted for age and healthy vaccinee period were higher after wP vs. aP vaccination, and lower for convulsions, for all doses. The IRR for HHE was slightly lower for wP than aP, while wP was associated with somnolence only for dose 1 and dose 3 compared with aP. CONCLUSIONS: The estimated IRs and IRRs were comparable with published data, therefore demonstrating that the ADVANCE system was able to combine several European healthcare databases to assess vaccine safety data for wP and aP vaccination.

ADVANCE system testing: Can coverage of pertussis vaccination be estimated in European countries using electronic healthcare databases: An example.

INTRODUCTION: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines, using existing healthcare databases in Europe. The objective of this paper was to assess the feasibility of using electronic healthcare databases to estimate dose-specific acellular pertussis (aP) and whole cell pertussis (wP) vaccine coverage. METHODS: Seven electronic healthcare databases in four European countries (Denmark (n = 2), UK (n = 2), Spain (n = 2) and Italy (n = 1)) participated in this study. Children were included from birth and followed up to age six years. Vaccination exposure was obtained from the databases and classified by type (aP or wP), and dose 1, 2 or 3. Coverage was estimated using period prevalence. For the 2006 birth cohort, two estimation methods for pertussis vaccine coverage, period prevalence and cumulative incidence were compared for each database. RESULTS: The majority of the 2,575,576 children included had been vaccinated at the country-specific recommended ages. Overall, the estimated dose 3 coverage was 88-97% in Denmark (birth cohorts from 2003 to 2014), 96-100% in the UK (2003-2014), 95-98% in Spain (2004-2014) and 94% in Italy (2006-2007). The estimated dose 3 coverage per birth cohort in Denmark and the UK differed by 1-6% compared with national estimates, with our estimates mostly higher. The estimated dose 3 coverage in Spain differed by 0-2% with no consistent over- or underestimation. In Italy, the estimates were 3% lower compared with the national estimates. Except for Italy, for which the two coverage estimation methods generated the same results, the estimated cumulative incidence coverages were consistently 1-10% lower than period prevalence estimates. CONCLUSION: This study showed that it was possible to provide consistent estimates of pertussis immunisation coverage from the electronic healthcare databases included, and that the estimates were comparable with the national estimates.

ADVANCE database characterisation and fit for purpose assessment for multi-country studies on the coverage, benefits and risks of pertussis vaccinations.

INTRODUCTION: The public-private ADVANCE consortium (Accelerated development of vaccine benefit-risk collaboration in Europe) aimed to assess if electronic healthcare databases can provide fit-for purpose data for collaborative, distributed studies and monitoring of vaccine coverage, benefits and risks of vaccines. OBJECTIVE: To evaluate if European healthcare databases can be used to estimate vaccine coverage, benefit and/or risk using pertussis-containing vaccines as an example. METHODS: Characterisation was conducted using open-source Java-based (Jerboa) software and R scripts. We obtained: (i) The general characteristics of the database and data source (meta-data) and (ii) a detailed description of the database population (size, representatively of age/sex of national population, rounding of birth dates, delay between birth and database entry), vaccinations (number of vaccine doses, recording of doses, pattern of doses by age and coverage) and events of interest (diagnosis codes, incidence rates). A total of nine databases (primary care, regional/national record linkage) provided data on events (pertussis, pneumonia, death, fever, convulsions, injection site reactions, hypotonic hypo-responsive episode, persistent crying) and vaccines (acellular pertussis and whole cell pertussis) related to the pertussis proof of concept studies. RESULTS: The databases contained data for a total population of 44 million individuals. Seven databases had recorded doses of vaccines. The pertussis coverage estimates were similar to those reported by the World Health Organisation (WHO). Incidence rates of events were comparable in magnitude and age-distribution between databases with the same characteristics. Several conditions (persistent crying and somnolence) were not captured by the databases for which outcomes were restricted to hospital discharge diagnoses. CONCLUSION: The database characterisation programs and workflows allowed for an efficient, transparent and standardised description and verification of electronic healthcare databases which may participate in pertussis vaccine coverage, benefit and risk studies. This approach is ready to be used for other vaccines/events to create readiness for participation in other vaccine related studies.

Advance system testing: Vaccine benefit studies using multi-country electronic health data - The example of pertussis vaccination.

The Accelerated Development of VAccine benefit-risk Collaboration in Europe (ADVANCE), a public-private consortium, implemented and tested a distributed network system for the generation of evidence on the benefits-risks of marketed vaccines in Europe. We tested the system by estimating the incidence rate (IR) of pertussis and pertussis-related complications in children vaccinated with acellular (aP) and whole-cell (wP) pertussis vaccine. Data from seven electronic databases from four countries (Denmark: AUH and SSI, Spain: SIDIAP and BIFAP, UK: THIN and RCGP RSC and Italy: Pedianet) were included in a retrospective cohort analysis. Exposure was defined as any pertussis vaccination (aP or wP). The follow-up time started 14 days after the first dose. Children who had received any pertussis vaccine from January 1990 to December 2015 were included (those who switched type, or had unknown type were excluded). The outcomes of interest were confirmed or suspected pertussis and pertussis-related pneumonia and generalised convulsions within one month of pertussis diagnosis and death within three months of pertussis diagnosis. The cohort comprised 2,886,367 children ≤5 years of age. Data on wP and aP vaccination were available in three and seven databases, respectively. The IRs (per 100,000 person-years) for pertussis varied largely and ranged between 0.15 (95% CI: 0.12; 0.19) and 1.15 (95% CI: 1.07; 1.23), and the trends over time was consistent with those observed from national surveillance databases for confirmed pertussis. The pertussis IRs decreased as the number of wP and aP vaccine doses increased. Pertussis-related complications were rare (89 pneumonia, 7 generalised convulsions and no deaths) and their relative risk (vs. non-pertussis) could not be reliably estimated. The study demonstrated the feasibility of the ADVANCE system to estimate the change in pertussis IRs following pertussis vaccination. Larger sample sizes would provide additional power to compare the risk for complications between children with and without pertussis. The feasibility of vaccine-type specific effectiveness studies may be considered in the future.

Quantifying outcome misclassification in multi-database studies: The case study of pertussis in the ADVANCE project.

BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using European healthcare databases. Event misclassification can result in biased estimates. Using different algorithms for identifying cases of Bordetella pertussis (BorPer) infection as a test case, we aimed to describe a strategy to quantify event misclassification, when manual chart review is not feasible. METHODS: Four participating databases retrieved data from primary care (PC) setting: BIFAP: (Spain), THIN and RCGP RSC (UK) and PEDIANET (Italy); SIDIAP (Spain) retrieved data from both PC and hospital settings. BorPer algorithms were defined by healthcare setting, data domain (diagnoses, drugs, or laboratory tests) and concept sets (specific or unspecified pertussis). Algorithm- and database-specific BorPer incidence rates (IRs) were estimated in children aged 0-14 years enrolled in 2012 and 2014 and followed up until the end of each calendar year and compared with IRs of confirmed pertussis from the ECDC surveillance system (TESSy). Novel formulas were used to approximate validity indices, based on a small set of assumptions. They were applied to approximately estimate positive predictive value (PPV) and sensitivity in SIDIAP. RESULTS: The number of cases and the estimated BorPer IRs per 100,000 person-years in PC, using data representing 3,173,268 person-years, were 0 (IR = 0.0), 21 (IR = 4.3), 21 (IR = 5.1), 79 (IR = 5.7), and 2 (IR = 2.3) in BIFAP, SIDIAP, THIN, RCGP RSC and PEDIANET respectively. The IRs for combined specific/unspecified pertussis were higher than TESSy, suggesting that some false positives had been included. In SIDIAP the estimated IR was 45.0 when discharge diagnoses were included. The sensitivity and PPV of combined PC specific and unspecific diagnoses for BorPer cases in SIDIAP were approximately 85% and 72%, respectively. CONCLUSION: Retrieving BorPer cases using only specific concepts has low sensitivity in PC databases, while including cases retrieved by unspecified concepts introduces false positives, which were approximately estimated to be 28% in one database. The share of cases that cannot be retrieved from a PC database because they are only seen in hospital was approximately estimated to be 15% in one database. This study demonstrated that quantifying the impact of different event-finding algorithms across databases and benchmarking with disease surveillance data can provide approximate estimates of algorithm validity.