Six Years of the US Food and Drug Administration's Postmarket Active Risk Identification and Analysis System in the Sentinel Initiative: Implications for Real World Evidence Generation.

Congress mandated the creation of a postmarket Active Risk Identification and Analysis (ARIA) system containing data on 100 million individuals for monitoring risks associated with drug and biologic products using data from disparate sources to complement the US Food and Drug Administration's (FDA's) existing postmarket capabilities. We report on the first 6 years of ARIA utilization in the Sentinel System (2016-2021). The FDA has used the ARIA system to evaluate 133 safety concerns; 54 of these evaluations have closed with regulatory determinations, whereas the rest remain in progress. If the ARIA system and the FDA's Adverse Event Reporting System are deemed insufficient to address a safety concern, then the FDA may issue a postmarket requirement to a product's manufacturer. One hundred ninety-seven ARIA insufficiency determinations have been made. The most common situation for which ARIA was found to be insufficient is the evaluation of adverse pregnancy and fetal outcomes following in utero drug exposure, followed by neoplasms and death. ARIA was most likely to be sufficient for thromboembolic events, which have high positive predictive value in claims data alone and do not require supplemental clinical data. The lessons learned from this experience illustrate the continued challenges using administrative claims data, especially to define novel clinical outcomes. This analysis can help to identify where more granular clinical data are needed to fill gaps to improve the use of real-world data for drug safety analyses and provide insights into what is needed to efficiently generate high-quality real-world evidence for efficacy.

Validation of Acute Pancreatitis Among Adults in an Integrated Healthcare System.

BACKGROUND: Acute pancreatitis is a serious gastrointestinal disease that is an important target for drug safety surveillance. Little is known about the accuracy of ICD-10 codes for acute pancreatitis in the United States, or their performance in specific clinical settings. We conducted a validation study to assess the accuracy of acute pancreatitis ICD-10 diagnosis codes in inpatient, emergency department (ED), and outpatient settings. METHODS: We reviewed electronic medical records for encounters with acute pancreatitis diagnosis codes in an integrated healthcare system from October 2015 to December 2019. Trained abstractors and physician adjudicators determined whether events met criteria for acute pancreatitis. RESULTS: Out of 1,844 eligible events, we randomly sampled 300 for review. Across all clinical settings, 182 events met validation criteria for an overall positive predictive value (PPV) of 61% (95% confidence intervals [CI] = 55, 66). The PPV was 87% (95% CI = 79, 92%) for inpatient codes, but only 45% for ED (95% CI = 35, 54%) and outpatient (95% CI = 34, 55%) codes. ED and outpatient encounters accounted for 43% of validated events. Acute pancreatitis codes from any encounter type with lipase >3 times the upper limit of normal had a PPV of 92% (95% CI = 86, 95%) and identified 85% of validated events (95% CI = 79, 89%), while codes with lipase <3 times the upper limit of normal had a PPV of only 22% (95% CI = 16, 30%). CONCLUSIONS: These results suggest that ICD-10 codes accurately identified acute pancreatitis in the inpatient setting, but not in the ED and outpatient settings. Laboratory data substantially improved algorithm performance.

New-Onset Cancer Cases in FDA's Sentinel System: A Large Distributed System of US Electronic Healthcare Data.

BACKGROUND: Evaluations of cancer etiology and safety and effectiveness of cancer treatments are predicated on large numbers of patients with sufficient baseline and follow-up data. To assess feasibility of FDA's Sentinel System's electronic healthcare data for surveillance of malignancy onset and examination of product safety, this study examined patterns of enrollment surrounding new-onset cancers. METHODS: Using a retrospective cohort of patients based on administrative claims, we identified incident events of 19 cancers among 292.5 million health plan members from January 2000 to February 2020 using International Classification of Diseases (ICD) diagnosis codes. Annual incident cases were stratified by sex, age, medical and drug coverage, and insurer type. Descriptive statistics were calculated for observable time prior to and following diagnosis. RESULTS: We identified 10,697,573 incident cancer events among members with medical coverage. When drug coverage was additionally required, number of incident cancers was reduced by 41%. Medicare data contributed 61% of cases, with similar duration trends as other insurers. Mean duration of follow-up prior to diagnosis ranged from 4.0 to 4.6 years, whereas follow-up post diagnosis ranged from 1.1 to 3.3 years. Approximately a third (36.1%) had at least 2 years both prior to and following diagnosis. CONCLUSIONS: The FDA Sentinel System's electronic healthcare data may be useful for characterizing relatively short latency cancer risk, examining cancer drug utilization and safety after diagnosis, and conducting surveillance for acute adverse events among patients with cancers. IMPACT: A national distributed system with electronic health data, the Sentinel system provides opportunity for rapid pharmacoepidemiologic assessments relevant in oncology.

Validity of ICD-10-CM diagnoses to identify hospitalizations for serious infections among patients treated with biologic therapies.

PURPOSE: Identifying hospitalizations for serious infections among patients dispensed biologic therapies within healthcare databases is important for post-marketing surveillance of these drugs. We determined the positive predictive value (PPV) of an ICD-10-CM-based diagnostic coding algorithm to identify hospitalization for serious infection among patients dispensed biologic therapy within the FDA's Sentinel Distributed Database. METHODS: We identified health plan members who met the following algorithm criteria: (1) hospital ICD-10-CM discharge diagnosis of serious infection between July 1, 2016 and August 31, 2018; (2) either outpatient/emergency department infection diagnosis or outpatient antimicrobial treatment within 7 days prior to hospitalization; (3) inflammatory bowel disease, psoriasis, or rheumatological diagnosis within 1 year prior to hospitalization, and (4) were dispensed outpatient biologic therapy within 90 days prior to admission. Medical records were reviewed by infectious disease clinicians to adjudicate hospitalizations for serious infection. The PPV (95% confidence interval [CI]) for confirmed events was determined after further weighting by the prevalence of the type of serious infection in the database. RESULTS: Among 223 selected health plan members who met the algorithm, 209 (93.7% [95% CI, 90.1%-96.9%]) were confirmed to have a hospitalization for serious infection. After weighting by the prevalence of the type of serious infection, the PPV of the ICD-10-CM algorithm identifying a hospitalization for serious infection was 80.2% (95% CI, 75.3%-84.7%). CONCLUSIONS: The ICD-10-CM-based algorithm for hospitalization for serious infection among patients dispensed biologic therapies within the Sentinel Distributed Database had 80% PPV for confirmed events and could be considered for use within pharmacoepidemiologic studies.

Electronic phenotyping of health outcomes of interest using a linked claims-electronic health record database: Findings from a machine learning pilot project.

OBJECTIVE: Claims-based algorithms are used in the Food and Drug Administration Sentinel Active Risk Identification and Analysis System to identify occurrences of health outcomes of interest (HOIs) for medical product safety assessment. This project aimed to apply machine learning classification techniques to demonstrate the feasibility of developing a claims-based algorithm to predict an HOI in structured electronic health record (EHR) data. MATERIALS AND METHODS: We used the 2015-2019 IBM MarketScan Explorys Claims-EMR Data Set, linking administrative claims and EHR data at the patient level. We focused on a single HOI, rhabdomyolysis, defined by EHR laboratory test results. Using claims-based predictors, we applied machine learning techniques to predict the HOI: logistic regression, LASSO (least absolute shrinkage and selection operator), random forests, support vector machines, artificial neural nets, and an ensemble method (Super Learner). RESULTS: The study cohort included 32 956 patients and 39 499 encounters. Model performance (positive predictive value [PPV], sensitivity, specificity, area under the receiver-operating characteristic curve) varied considerably across techniques. The area under the receiver-operating characteristic curve exceeded 0.80 in most model variations. DISCUSSION: For the main Food and Drug Administration use case of assessing risk of rhabdomyolysis after drug use, a model with a high PPV is typically preferred. The Super Learner ensemble model without adjustment for class imbalance achieved a PPV of 75.6%, substantially better than a previously used human expert-developed model (PPV = 44.0%). CONCLUSIONS: It is feasible to use machine learning methods to predict an EHR-derived HOI with claims-based predictors. Modeling strategies can be adapted for intended uses, including surveillance, identification of cases for chart review, and outcomes research.

Understanding utilization patterns of biologics and biosimilars in the United States to support postmarketing studies of safety and effectiveness.

PURPOSE: To describe utilization of filgrastim and infliximab, the first two products with biosimilars approved in the United States. METHODS: We identified use of filgrastim (reference, tbo-filgrastim, and filgrastim-sndz) and infliximab (reference, infliximab-dyyb, and infliximab-abda) in the Sentinel Distributed Database using Healthcare Common Procedure Coding System (HCPCS) codes and National Drug Codes (NDCs) from January 2015 to August 2018. We calculated the proportion of use by code type and assessed uptake over time. We compared baseline patient characteristics and treatment indications. Among patients with >1 exposure episode, we characterized gaps between episodes. RESULTS: Use was identified primarily via HCPCS codes (filgrastim: 86.4%-97.7%; infliximab: 87.8%-100%) although some was identified via NDCs (filgrastim: 2.2%-13.5%; infliximab: <0.1%-6.5%). Filgrastim reference product use declined from 89.4% in January 2015 to 30.3% in June 2018, with corresponding increases in filgrastim-sndz (0% to 49.3%) and tbo-filgrastim (10.6% to 20.4%). Infliximab biosimilar uptake was low (9.7% in June 2018). We identified 94 846 filgrastim reference product, 27 143 tbo-filgrastim, and 38 264 filgrastim-sndz users. For infliximab, we identified 125 412 reference product, 1034 infliximab-dyyb, 49 infliximab-abda, and 4855 undetermined biosimilar users. Patients receiving filgrastim products were largely similar, but differences in age, sex, and indication were observed across infliximab product users. The median exposure episode gap ranged from 1 to 3 days for filgrastim and 48 to 50 days for infliximab. CONCLUSION: Use of biosimilar filgrastim has increased in the United States, but infliximab biosimilar use remains low. Data on identification of biosimilars in claims data and observed gaps between exposure episodes can be used to support drug safety studies of biosimilars.

Evaluation of Use of Technologies to Facilitate Medical Chart Review.

INTRODUCTION: While medical chart review remains the gold standard to validate health conditions or events identified in administrative claims and electronic health record databases, it is time consuming, expensive and can involve subjective decisions. AIM: The aim of this study was to describe the landscape of technology-enhanced approaches that could be used to facilitate medical chart review within and across distributed data networks. METHOD: We conducted a semi-structured survey regarding processes for medical chart review with organizations that either routinely do medical chart review or use technologies that could facilitate chart review. RESULTS: Fifteen out of 17 interviewed organizations used optical character recognition (OCR) or natural language processing (NLP) in their chart review process. None used handwriting recognition software. While these organizations found OCR and NLP to be useful for expediting extraction of useful information from medical charts, they also mentioned several challenges. Quality of medical scans can be variable, interfering with the accuracy of OCR. Additionally, linguistic complexity in medical notes and heterogeneity in reporting templates used by different healthcare systems can reduce the transportability of NLP-based algorithms to diverse healthcare settings. CONCLUSION: New technologies including OCR and NLP are currently in use by various organizations involved in medical chart review. While technology-enhanced approaches could scale up capacity to validate key variables and make information about important clinical variables from medical records more generally available for research purposes, they often require considerable customization when employed in a distributed data environment with multiple, diverse healthcare settings.

Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0.

PURPOSE: Defining a study population and creating an analytic dataset from longitudinal healthcare databases involves many decisions. Our objective was to catalogue scientific decisions underpinning study execution that should be reported to facilitate replication and enable assessment of validity of studies conducted in large healthcare databases. METHODS: We reviewed key investigator decisions required to operate a sample of macros and software tools designed to create and analyze analytic cohorts from longitudinal streams of healthcare data. A panel of academic, regulatory, and industry experts in healthcare database analytics discussed and added to this list. CONCLUSION: Evidence generated from large healthcare encounter and reimbursement databases is increasingly being sought by decision-makers. Varied terminology is used around the world for the same concepts. Agreeing on terminology and which parameters from a large catalogue are the most essential to report for replicable research would improve transparency and facilitate assessment of validity. At a minimum, reporting for a database study should provide clarity regarding operational definitions for key temporal anchors and their relation to each other when creating the analytic dataset, accompanied by an attrition table and a design diagram. A substantial improvement in reproducibility, rigor and confidence in real world evidence generated from healthcare databases could be achieved with greater transparency about operational study parameters used to create analytic datasets from longitudinal healthcare databases.

Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0.

PURPOSE: Defining a study population and creating an analytic dataset from longitudinal healthcare databases involves many decisions. Our objective was to catalogue scientific decisions underpinning study execution that should be reported to facilitate replication and enable assessment of validity of studies conducted in large healthcare databases. METHODS: We reviewed key investigator decisions required to operate a sample of macros and software tools designed to create and analyze analytic cohorts from longitudinal streams of healthcare data. A panel of academic, regulatory, and industry experts in healthcare database analytics discussed and added to this list. CONCLUSION: Evidence generated from large healthcare encounter and reimbursement databases is increasingly being sought by decision-makers. Varied terminology is used around the world for the same concepts. Agreeing on terminology and which parameters from a large catalogue are the most essential to report for replicable research would improve transparency and facilitate assessment of validity. At a minimum, reporting for a database study should provide clarity regarding operational definitions for key temporal anchors and their relation to each other when creating the analytic dataset, accompanied by an attrition table and a design diagram. A substantial improvement in reproducibility, rigor and confidence in real world evidence generated from healthcare databases could be achieved with greater transparency about operational study parameters used to create analytic datasets from longitudinal healthcare databases.

Automated Algorithm for Detection of Transient Adenosine Release.

Spontaneous adenosine release events have been discovered in the brain that last only a few seconds. The identification of these adenosine events from fast-scan cyclic voltammetry (FSCV) data is difficult due to the random nature of adenosine release. In this study, we develop an algorithm that automatically identifies and characterizes adenosine transient features, including event time, concentration, and duration. Automating the data analysis reduces analysis time from 10 to 18 h to about 40 min per experiment. The algorithm identifies adenosine based on its two oxidation peaks, the time delay between them, and their current vs time peak ratios. In order to validate the program, four data sets from three independent researchers were analyzed by the algorithm and then compared to manual identification by an analyst. The algorithm resulted in 10 ± 4% false negatives and 9 ± 3% false positives. The specificity of the algorithm was verified by comparing calibration data for adenosine triphosphate (ATP), histamine, hydrogen peroxide, and pH changes and these analytes were not identified as adenosine. Stimulated histamine release in vivo was also not identified as adenosine. The code is modular in design and could be easily adjusted to detect features of spontaneous dopamine or other neurochemical transients in FSCV data.

Transient Adenosine Release Is Modulated by NMDA and GABAB Receptors.

Adenosine is a neuroprotective agent that modulates neurotransmission and is modulated by other neurotransmitters. Spontaneous, transient adenosine is a recently discovered mode of signaling where adenosine is released and cleared from the extracellular space quickly, in less than three seconds. Spontaneous adenosine release is regulated by adenosine A1 and A2a receptors, but regulation by other neurotransmitter receptors has not been studied. Here, we examined the effect of glutamate and GABA receptors on the concentration and frequency of spontaneous, transient adenosine release by measuring adenosine with fast-scan cyclic voltammetry in the rat caudate-putamen. The glutamate NMDA antagonist, 3-(R-2-carboxypiperazin-4-yl)-propyl-1-phosphonic acid (CPP, 6.25 mg/kg i.p.), increased the frequency of adenosine transients and the concentration of individual transients, but NMDA (agonist, 50 mg/kg, i.p.) did not change the frequency. In contrast, antagonists of other glutamate receptors had no effect on the frequency or concentration of transient adenosine release, including the AMPA antagonist NBQX (15 mg/kg i.p.) and the mGlu2/3 glutamate receptor antagonist LY 341495 (5 mg/kg i.p.). The GABAB antagonist CGP 52432 (30 mg/kg i.p.) significantly decreased the number of adenosine release events while the GABAB agonist baclofen (4 mg/kg i.p.) increased the frequency of adenosine release. The GABAA antagonist bicuculline (5 mg/kg i.p.) had no significant effects on adenosine. NMDA and GABAB likely act presynaptically, affecting the overall cell excitability for vesicular release. The ability to regulate adenosine with NMDA and GABAB receptors will help control the modulatory effects of transient adenosine release.

Statistical Power for Postlicensure Medical Product Safety Data Mining.

OBJECTIVE: To perform sample size calculations when using tree-based scan statistics in longitudinal observational databases. METHODS: Tree-based scan statistics enable data mining on epidemiologic datasets where thousands of disease outcomes are organized into hierarchical tree structures with automatic adjustment for multiple testing. We show how to evaluate the statistical power of the unconditional and conditional Poisson versions. The null hypothesis is that there is no increase in the risk for any of the outcomes. The alternative is that one or more outcomes have an excess risk. We varied the excess risk, total sample size, frequency of the underlying event rate, and the level of across-the-board health care utilization. We also quantified the reduction in statistical power resulting from specifying a risk window that was too long or too short. RESULTS: For 500,000 exposed people, we had at least 98 percent power to detect an excess risk of 1 event per 10,000 exposed for all outcomes. In the presence of potential temporal confounding due to across-the-board elevations of health care utilization in the risk window, the conditional tree-based scan statistic controlled type I error well, while the unconditional version did not. DISCUSSION: Data mining analyses using tree-based scan statistics expand the pharmacovigilance toolbox, ensuring adequate monitoring of thousands of outcomes of interest while controlling for multiple hypothesis testing. These power evaluations enable investigators to design and optimize implementation of retrospective data mining analyses.

Laser Treated Carbon Nanotube Yarn Microelectrodes for Rapid and Sensitive Detection of Dopamine in Vivo.

Carbon nanotube yarn microelectrodes (CNTYMEs) exhibit rapid and selective detection of dopamine with fast-scan cyclic voltammetry (FSCV); however, the sensitivity limits their application in vivo. In this study, we introduce laser treatment as a simple, reliable, and efficient approach to improve the sensitivity of CNTYMEs by three fold while maintaining high temporal resolution. The effect of laser treatment on the microelectrode surface was characterized by scanning electron microscopy, Raman spectroscopy, energy dispersion spectroscopy, and laser confocal microscopy. Laser treatment increases the surface area and oxygen containing functional groups on the surface, which provides more adsorption sites for dopamine than at unmodified CNTYMEs. Moreover, similar to unmodified CNTYMEs, the dopamine signal at laser treated CNTYMEs is not dependent on scan repetition frequency, unlike the current at carbon fiber microelectrodes (CFMEs) which decreases with increasing scan repetition frequency. This frequency independence is caused by the significantly larger surface roughness which would trap dopamine-o-quinone and amplify the dopamine signal. CNTYMEs were applied as an in vivo sensor with FSCV for the first time and laser treated CNTYMEs maintained high dopamine sensitivity compared to CFMEs with an increased scan repetition frequency of 50 Hz, which is five-fold faster than the conventional frequency. CNTYMEs with laser treatment are advantageous because of their easy fabrication, high reproducibility, fast electron transfer kinetics, high sensitivity, and rapid in vivo measurement of dopamine and could be a potential alternative to CFMEs in the future.

Prospective influenza vaccine safety surveillance using fresh data in the Sentinel System.

PURPOSE: To develop the infrastructure to conduct timely active surveillance for safety of influenza vaccines and other medical countermeasures in the Sentinel System (formerly the Mini-Sentinel Pilot), a Food and Drug Administration-sponsored national surveillance system that typically relies on data that are mature, settled, and updated quarterly. METHODS: Three Data Partners provided their earliest available ("fresh") cumulative claims data on influenza vaccination and health outcomes 3-4 times on a staggered basis during the 2013-2014 influenza season, collectively producing 10 data updates. We monitored anaphylaxis in the entire population using a cohort design and seizures in children ≤4 years of age using both a self-controlled risk interval design (primary) and a cohort design (secondary). After each data update, we conducted sequential analysis for inactivated (IIV) and live (LAIV) influenza vaccines using the Maximized Sequential Probability Ratio Test, adjusting for data-lag. RESULTS: Most of the 10 sequential analyses were conducted within 6 weeks of the last care-date in the cumulative dataset. A total of 6 682 336 doses of IIV and 782 125 doses of LAIV were captured. The primary analyses did not identify any statistical signals following IIV or LAIV. In secondary analysis, the risk of seizures was higher following concomitant IIV and PCV13 than historically after IIV in 6- to 23-month-olds (relative risk = 2.7), which requires further investigation. CONCLUSIONS: The Sentinel System can implement a sequential analysis system that uses fresh data for medical product safety surveillance. Active surveillance using sequential analysis of fresh data holds promise for detecting clinically significant health risks early. Limitations of employing fresh data for surveillance include cost and the need for careful scrutiny of signals. © 2015 The Authors. Pharmacoepidemiology and Drug Safety Published by John Wiley & Sons Ltd.

Carbon Nanotubes Grown on Metal Microelectrodes for the Detection of Dopamine.

Microelectrodes modified with carbon nanotubes (CNTs) are useful for the detection of neurotransmitters because the CNTs enhance sensitivity and have electrocatalytic effects. CNTs can be grown on carbon fiber microelectrodes (CFMEs) but the intrinsic electrochemical activity of carbon fibers makes evaluating the effect of CNT enhancement difficult. Metal wires are highly conductive and many metals have no intrinsic electrochemical activity for dopamine, so we investigated CNTs grown on metal wires as microelectrodes for neurotransmitter detection. In this work, we successfully grew CNTs on niobium substrates for the first time. Instead of planar metal surfaces, metal wires with a diameter of only 25 µm were used as CNT substrates; these have potential in tissue applications due to their minimal tissue damage and high spatial resolution. Scanning electron microscopy shows that aligned CNTs are grown on metal wires after chemical vapor deposition. By use of fast-scan cyclic voltammetry, CNT-coated niobium (CNT-Nb) microelectrodes exhibit higher sensitivity and lower ΔEp value compared to CNTs grown on carbon fibers or other metal wires. The limit of detection for dopamine at CNT-Nb microelectrodes is 11 ± 1 nM, which is approximately 2-fold lower than that of bare CFMEs. Adsorption processes were modeled with a Langmuir isotherm, and detection of other neurochemicals was also characterized, including ascorbic acid, 3,4-dihydroxyphenylacetic acid, serotonin, adenosine, and histamine. CNT-Nb microelectrodes were used to monitor stimulated dopamine release in anesthetized rats with high sensitivity. This study demonstrates that CNT-grown metal microelectrodes, especially CNTs grown on Nb microelectrodes, are useful for monitoring neurotransmitters.

Evaluation of the risk of venous thromboembolism after quadrivalent human papillomavirus vaccination among US females.

After the Food and Drug Administration (FDA) licensed quadrivalent human papillomavirus vaccine (HPV4) in 2006, reports suggesting a possible association with venous thromboembolism (VTE) emerged from the Vaccine Adverse Event Reporting System and the Vaccine Safety Datalink. Our objective was to determine whether HPV4 increased VTE risk. The subjects were 9-26-year-old female members of five data partners in the FDA's Mini-Sentinel pilot project receiving HPV4 during 2006-2013. The outcome was radiologically confirmed first-ever VTE among potential cases identified by diagnosis codes in administrative data during Days 1-77 after HPV4 vaccination. With a self-controlled risk interval design, we compared counts of first-ever VTE in risk intervals (Days 1-28 and Days 1-7 post-vaccination) and control intervals (Days 36-56 for Dose 1 and Days 36-63 for Doses 2 and 3). Combined hormonal contraceptive use was treated as a potential confounder. The main analyses were: (1) unadjusted for time-varying VTE risk from contraceptive use, (2) unadjusted but restricted to cases without such time-varying risk, and (3) adjusted by incorporating the modeled risk of VTE by week of contraceptive use in the analysis. Of 279 potential VTE cases identified following 1,423,399 HPV4 doses administered, 225 had obtainable charts, and 53 were confirmed first-ever VTE. All 30 with onsets in risk or control intervals had known risk factors for VTE. VTE risk was not elevated in the first 7 or 28 days following any dose of HPV in any analysis (e.g. relative risk estimate (95% CI) from both unrestricted analyses, for all-doses, 28-day risk interval: 0.7 (0.3-1.4)). Temporal scan statistics found no clustering of VTE onsets after any dose. Thus, we found no evidence of an increased risk of VTE associated with HPV4 among 9-26-year-old females. A particular strength of this evaluation was its control for both time-invariant and contraceptive-related time-varying potential confounding.

A vaccine study design selection framework for the postlicensure rapid immunization safety monitoring program.

The Postlicensure Rapid Immunization Safety Monitoring Program, the vaccination safety monitoring component of the US Food and Drug Administration's Mini-Sentinel project, is currently the largest cohort in the US general population for vaccine safety surveillance. We developed a study design selection framework to provide a roadmap and description of methods that may be utilized to evaluate potential associations between vaccines and health outcomes of interest in the Postlicensure Rapid Immunization Safety Monitoring Program and other systems using administrative data. The strengths and weaknesses of designs for vaccine safety monitoring, including the cohort design, the case-centered design, the risk interval design, the case-control design, the self-controlled risk interval design, the self-controlled case series method, and the case-crossover design, are described and summarized in tabular form. A structured decision table is provided to aid in planning of future vaccine safety monitoring activities, and the data components comprising the structured decision table are delineated. The study design selection framework provides a starting point for planning vaccine safety evaluations using claims-based data sources.