Validation of claims-based algorithms to identify non-live birth outcomes.

PURPOSE: Perinatal epidemiology studies using healthcare utilization databases are often restricted to live births, largely due to the lack of established algorithms to identify non-live births. The study objective was to develop and validate claims-based algorithms for the ascertainment of non-live births. METHODS: Using the Mass General Brigham Research Patient Data Registry 2000-2014, we assembled a cohort of women enrolled in Medicaid with a non-live birth. Based on ≥1 inpatient or ≥2 outpatient diagnosis/procedure codes, we identified and randomly sampled 100 potential stillbirth, spontaneous abortion, and termination cases each. For the secondary definitions, we excluded cases with codes for other pregnancy outcomes within ±5 days of the outcome of interest and relaxed the definitions for spontaneous abortion and termination by allowing cases with one outpatient diagnosis only. Cases were adjudicated based on medical chart review. We estimated the positive predictive value (PPV) for each outcome. RESULTS: The PPV was 71.0% (95% CI, 61.1-79.6) for stillbirth; 79.0% (69.7-86.5) for spontaneous abortion, and 93.0% (86.1-97.1) for termination. When excluding cases with adjacent codes for other pregnancy outcomes and further relaxing the definition, the PPV increased to 80.6% (69.5-88.9) for stillbirth, 86.6% (80.5-91.3) for spontaneous abortion and 94.9% (91.1-97.4) for termination. The PPV for the composite outcome using the relaxed definition was 94.4% (92.3-96.1). CONCLUSIONS: Our findings suggest non-live birth outcomes can be identified in a valid manner in epidemiological studies based on healthcare utilization databases.

Development and Validation of Algorithms to Estimate Live Birth Gestational Age in Medicaid Analytic eXtract Data.

BACKGROUND: While healthcare utilization data are useful for postmarketing surveillance of drug safety in pregnancy, the start of pregnancy and gestational age at birth are often incompletely recorded or missing. Our objective was to develop and validate a claims-based live birth gestational age algorithm. METHODS: Using the Medicaid Analytic eXtract (MAX) linked to birth certificates in three states, we developed four candidate algorithms based on: preterm codes; preterm or postterm codes; timing of prenatal care; and prediction models - using conventional regression and machine-learning approaches with a broad range of prespecified and empirically selected predictors. We assessed algorithm performance based on mean squared error (MSE) and proportion of pregnancies with estimated gestational age within 1 and 2 weeks of the gold standard, defined as the clinical or obstetric estimate of gestation on the birth certificate. We validated the best-performing algorithms against medical records in a nationwide sample. We quantified misclassification of select drug exposure scenarios due to estimated gestational age as positive predictive value (PPV), sensitivity, and specificity. RESULTS: Among 114,117 eligible pregnancies, the random forest model with all predictors emerged as the best performing algorithm: MSE 1.5; 84.8% within 1 week and 96.3% within 2 weeks, with similar performance in the nationwide validation cohort. For all exposure scenarios, PPVs were >93.8%, sensitivities >94.3%, and specificities >99.4%. CONCLUSIONS: We developed a highly accurate algorithm for estimating gestational age among live births in the nationwide MAX data, further supporting the value of these data for drug safety surveillance in pregnancy. See video abstract at, http://links.lww.com/EDE/B989 .

Electronic phenotyping of health outcomes of interest using a linked claims-electronic health record database: Findings from a machine learning pilot project.

OBJECTIVE: Claims-based algorithms are used in the Food and Drug Administration Sentinel Active Risk Identification and Analysis System to identify occurrences of health outcomes of interest (HOIs) for medical product safety assessment. This project aimed to apply machine learning classification techniques to demonstrate the feasibility of developing a claims-based algorithm to predict an HOI in structured electronic health record (EHR) data. MATERIALS AND METHODS: We used the 2015-2019 IBM MarketScan Explorys Claims-EMR Data Set, linking administrative claims and EHR data at the patient level. We focused on a single HOI, rhabdomyolysis, defined by EHR laboratory test results. Using claims-based predictors, we applied machine learning techniques to predict the HOI: logistic regression, LASSO (least absolute shrinkage and selection operator), random forests, support vector machines, artificial neural nets, and an ensemble method (Super Learner). RESULTS: The study cohort included 32 956 patients and 39 499 encounters. Model performance (positive predictive value [PPV], sensitivity, specificity, area under the receiver-operating characteristic curve) varied considerably across techniques. The area under the receiver-operating characteristic curve exceeded 0.80 in most model variations. DISCUSSION: For the main Food and Drug Administration use case of assessing risk of rhabdomyolysis after drug use, a model with a high PPV is typically preferred. The Super Learner ensemble model without adjustment for class imbalance achieved a PPV of 75.6%, substantially better than a previously used human expert-developed model (PPV = 44.0%). CONCLUSIONS: It is feasible to use machine learning methods to predict an EHR-derived HOI with claims-based predictors. Modeling strategies can be adapted for intended uses, including surveillance, identification of cases for chart review, and outcomes research.

A General Propensity Score for Signal Identification Using Tree-Based Scan Statistics.

The tree-based scan statistic (TreeScan; Martin Kulldorff, Harvard Medical School, Boston, Massachusetts) is a data-mining method that adjusts for multiple testing of correlated hypotheses when screening thousands of potential adverse events for signal identification. Simulation has demonstrated the promise of TreeScan with a propensity score (PS)-matched cohort design. However, it is unclear which variables to include in a PS for applied signal identification studies to simultaneously adjust for confounding across potential outcomes. We selected 4 pairs of medications with well-understood safety profiles. For each pair, we evaluated 5 candidate PSs with different combinations of 1) predefined general covariates (comorbidity, frailty, utilization), 2) empirically selected (data-driven) covariates, and 3) covariates tailored to the drug pair. For each pair, statistical alerting patterns were similar with alternative PSs (≤11 alerts in 7,996 outcomes scanned). Inclusion of covariates tailored to exposure did not appreciably affect screening results. Inclusion of empirically selected covariates can provide better proxy coverage for confounders but can also decrease statistical power. Unlike tailored covariates, empirical and predefined general covariates can be applied "out of the box" for signal identification. The choice of PS depends on the level of concern about residual confounding versus loss of power. Potential signals should be followed by pharmacoepidemiologic assessment where confounding control is tailored to the specific outcome(s) under investigation.

Extension of Disease Risk Score-Based Confounding Adjustments for Multiple Outcomes of Interest: An Empirical Evaluation.

Use of disease risk score (DRS)-based confounding adjustment when estimating treatment effects on multiple outcomes is not well studied. We designed an empirical cohort study to compare dabigatran initiators and warfarin initiators with respect to risks of ischemic stroke and major bleeding in 12 sequential monitoring periods (90 days each), using data from the Truven Marketscan database (Truven Health Analytics, Ann Arbor, Michigan). We implemented 2 approaches to combine DRS for multiple outcomes: 1) 1:1 matching on prognostic propensity scores (PPS), created using DRS for bleeding and stroke as independent variables in a propensity score (PS) model; and 2) simultaneous 1:1 matching on DRS for bleeding and stroke using Mahalanobis distance (M-distance), and compared their performance with that of traditional PS matching. M-distance matching appeared to produce more stable results in the early marketing period than both PPS and traditional PS matching; hazard ratios from unadjusted analysis, traditional PS matching, PPS matching, and M-distance matching after 4 periods were 0.72 (95% confidence interval (CI): 0.51, 1.03), 0.61 (95% CI: 0.31, 1.09), 0.55 (95% CI: 0.33, 0.91), and 0.78 (95% CI: 0.45, 1.34), respectively, for stroke and 0.65 (95% CI: 0.53, 0.80), 0.78 (95% CI: 0.60, 1.01), 0.75 (95% CI: 0.59, 0.96), and 0.78 (95% CI: 0.64, 0.95), respectively, for bleeding. In later periods, estimates were similar for traditional PS matching and M-distance matching but suggested potential residual confounding with PPS matching. These results suggest that M-distance matching may be a valid approach for extension of DRS-based confounding adjustments for multiple outcomes of interest.

A Synthesis of Current Surveillance Planning Methods for the Sequential Monitoring of Drug and Vaccine Adverse Effects Using Electronic Health Care Data.

INTRODUCTION: The large-scale assembly of electronic health care data combined with the use of sequential monitoring has made proactive postmarket drug- and vaccine-safety surveillance possible. Although sequential designs have been used extensively in randomized trials, less attention has been given to methods for applying them in observational electronic health care database settings. EXISTING METHODS: We review current sequential-surveillance planning methods from randomized trials, and the Vaccine Safety Datalink (VSD) and Mini-Sentinel Pilot projects-two national observational electronic health care database safety monitoring programs. FUTURE SURVEILLANCE PLANNING: Based on this examination, we suggest three steps for future surveillance planning in health care databases: (1) prespecify the sequential design and analysis plan, using available feasibility data to reduce assumptions and minimize later changes to initial plans; (2) assess existing drug or vaccine uptake, to determine if there is adequate information to proceed with surveillance, before conducting more resource-intensive planning; and (3) statistically evaluate and clearly communicate the sequential design with all those designing and interpreting the safety-surveillance results prior to implementation. Plans should also be flexible enough to accommodate dynamic and often unpredictable changes to the database information made by the health plans for administrative purposes. CONCLUSIONS: This paper is intended to encourage dialogue about establishing a more systematic, scalable, and transparent sequential design-planning process for medical-product safety-surveillance systems utilizing observational electronic health care databases. Creating such a framework could yield improvements over existing practices, such as designs with increased power to assess serious adverse events.

Challenges in evaluating cancer as a clinical outcome in postapproval studies of drug safety.

Pharmaceuticals approved in the United States are largely not known human carcinogens. However, cancer signals associated with pharmaceuticals may be hypothesized or arise after product approval. There are many study designs that can be used to evaluate cancer as an outcome in the postapproval setting. Because prospective systematic collection of cancer outcomes from a large number of individuals may be lengthy, expensive, and challenging, leveraging data from large existing databases are an integral approach. Such studies have the capability to evaluate the clinical experience of a large number of individuals, yet there are unique methodological challenges involved in their use to evaluate cancer outcomes. To discuss methodological challenges and potential solutions, the Food and Drug Administration and the National Cancer Institute convened a two-day public meeting in 2014. This commentary summarizes the most salient issues discussed at the meeting.

Initiation of tumor necrosis factor α antagonists and risk of fractures in patients with selected rheumatic and autoimmune diseases.

OBJECTIVE: We tested the hypothesis that initiation of tumor necrosis factor α (TNFα) antagonists reduced the risk of fractures compared to nonbiologic comparators in patients with autoimmune diseases. METHODS: Using 4 large administrative databases, we assembled retrospective cohorts of patients with autoimmune diseases who initiated either a TNFα antagonist or a nonbiologic medication. We identified 3 mutually exclusive disease groups: rheumatoid arthritis (RA), inflammatory bowel disease (IBD), and a combined group: psoriasis (PsO), psoriatic arthritis (PsA), or ankylosing spondylitis (AS). We used baseline covariate data to calculate propensity scores (PS) for each disease group and used Cox regression to calculate hazard ratios (HRs) and 95% confidence intervals (95% CIs). We compared the risk of combined hip, radius/ulna, humerus, or pelvic fractures between PS-matched cohorts of new users of TNFα antagonists and nonbiologic comparators. RESULTS: We identified 9,020, 2,014, and 2,663 new PS-matched episodes of TNFα antagonist and nonbiologic comparator use in RA, IBD, and PsO-PsA-AS cohorts, respectively. The risk of combined fractures was similar between new users of TNFα antagonists and nonbiologic comparators for each disease (HR 1.17, 95% CI 0.91-1.51; HR 1.49, 95% CI 0.72-3.11; and HR 0.92, 95% CI 0.47-1.82 for RA, IBD, and PsO-PsA-AS, respectively). In RA, the risk of combined fractures was associated with an average daily dosage of prednisone equivalents >10 mg/day at baseline compared with no glucocorticoid (HR 1.54, 95% CI 1.03-2.30). CONCLUSION: The risk of fractures did not differ between initiators of a biologic agent and a nonbiologic comparator for any disease studied. Among RA patients, use of >10 mg/day of prednisone equivalents at baseline increased the fracture risk.

Neuropsychiatric events in varenicline and nicotine replacement patch users in the Military Health System.

AIM: To determine the rate ratio of neuropsychiatric hospitalizations in new users of varenicline compared to new users of nicotine replacement therapy (NRT) patch in the Military Health System (MHS). DESIGN, SETTING AND PARTICIPANTS: Varenicline (n = 19,933) and NRT patch (n = 15,867) users who initiated therapy from 1 August 2006 to 31 August 2007 within the MHS were included in this retrospective cohort study. After matching according to propensity scores, 10,814 users remained in each cohort. The study population included those with and without a history of neuropsychiatric disease. MEASUREMENTS: Patients were followed for neuropsychiatric hospitalizations defined by primary neuropsychiatric discharge diagnosis using ICD-9 codes from in-patient administrative claims. Hazard ratios (HR) and 95% confidence intervals (CI) were estimated after propensity score matching on exposure for socio-demographic factors, health-care utilization, comorbidities, medication history and neuropsychiatric history. FINDINGS: There was no increase in the rate of neuropsychiatric hospitalizations in patients treated with varenicline compared to NRT patch when followed for 30 days (propensity-score matched HR = 1.14, 95% CI: 0.56-2.34). Results were similar after 60 days of follow-up. CONCLUSIONS: There does not appear to be an increase in neuropsychiatric hospitalizations with varenicline compared with nicotine replacement therapy patch over 30 or 60 days after drug initiation.

Recent combined hormonal contraceptives (CHCs) and the risk of thromboembolism and other cardiovascular events in new users.

BACKGROUND: Combined hormonal contraceptives (CHCs) place women at increased risk of venous thromboembolic events (VTEs) and arterial thrombotic events (ATEs), including acute myocardial infarction and ischemic stroke. There is concern that three recent CHC preparations [drospirenone-containing pills (DRSPs), the norelgestromin-containing transdermal patch (NGMN) and the etonogestrel vaginal ring (ETON)] may place women at even higher risk of thrombosis than other older low-dose CHCs with a known safety profile. STUDY DESIGN: All VTEs and all hospitalized ATEs were identified in women, ages 10-55 years, from two integrated health care programs and two state Medicaid programs during the time period covering their new use of DRSP, NGMN, ETON or one of four low-dose estrogen comparator CHCs. The relative risk of thrombotic and thromboembolic outcomes associated with the newer CHCs in relation to the comparators was assessed with Cox proportional hazards regression models adjusting for age, site and year of entry into the study. RESULTS: The hazards ratio for DRSP in relation to low-dose estrogen comparators among new users was 1.77 (95% confidence interval 1.33-2.35) for VTE and 2.01 (1.06-3.81) for ATE. The increased risk of DRSP was limited to the 10-34-year age group for VTE and the 35-55-year group for ATE. Use of the NGMN patch and ETON vaginal ring was not associated with increased risk of either thromboembolic or thrombotic outcomes. CONCLUSIONS: In new users, DRSP was associated with higher risk of thrombotic events (VTE and ATE) relative to low-dose estrogen comparator CHCs, while the use of the NGMN patch and ETON vaginal ring was not.

Rates of hospitalized bacterial infection associated with juvenile idiopathic arthritis and its treatment.

OBJECTIVE: To compare the incidence of hospitalized bacterial infections among children with and children without juvenile idiopathic arthritis (JIA) and to examine the effects of selected medications. METHODS: Using national Medicaid data from 2000 through 2005, we identified a cohort of children with JIA and a comparator cohort of children with attention deficit hyperactivity disorder (ADHD). Exposures to methotrexate (MTX), TNF inhibitors, and oral glucocorticoids (GCs) were determined using pharmacy claims. Patients hospitalized with bacterial infections were identified using coded discharge diagnoses. We calculated adjusted hazard ratios (HR(adj) ) to compare infection incidence rates while adjusting for relevant covariates. RESULTS: We identified 8,479 JIA patients with 13,003 person-years of followup; 36% took MTX and 16% took TNF inhibitors. Compared with ADHD patients, JIA patients who were not currently taking MTX or TNF inhibitors had an increased rate of infection (HR(adj) 2.0 [95% confidence interval (95% CI) 1.5, 2.5]). Among JIA patients not receiving TNF inhibitor therapy, MTX users had a similar rate of infection as those not currently taking MTX (HR(adj) 1.2 [95% CI 0.9, 1.7]). TNF inhibitor use (irrespective of MTX) resulted in a similar rate of infection as use of MTX without a TNF inhibitor (HR(adj) 1.2 [95% CI 0.8, 1.8]). Use of high-dose GCs (≥10 mg/day of prednisone or equivalent) increased the rate of infection as compared with no GC use, after adjustment for MTX and TNF inhibitor use (HR(adj) 3.1 [95% CI 2.0, 4.7]). CONCLUSION: Children with JIA had an increased rate of infection compared to children with ADHD. Among children with JIA, the rate of infection was not increased with MTX or TNF inhibitor use, but was significantly increased with high-dose GC use.

Initiation of tumor necrosis factor-α antagonists and the risk of hospitalization for infection in patients with autoimmune diseases.

CONTEXT: Although tumor necrosis factor (TNF)-α antagonists are increasingly used in place of nonbiologic comparator medications, their safety profile remains incomplete. OBJECTIVES: To determine whether initiation of TNF-α antagonists compared with nonbiologic comparators is associated with an increased risk of serious infections requiring hospitalization. DESIGN, SETTING, AND PATIENTS: Within a US multi-institutional collaboration, we assembled retrospective cohorts (1998-2007) of patients with rheumatoid arthritis (RA), inflammatory bowel disease (IBD), and psoriasis, psoriatic arthritis, or ankylosing spondylitis (psoriasis and spondyloarthropathies) combining data from Kaiser Permanente Northern California, New Jersey and Pennsylvania Pharmaceutical Assistance programs, Tennessee Medicaid, and national Medicaid/Medicare. TNF-α antagonists and nonbiologic regimens were compared in disease-specific propensity score (PS)-matched cohorts using Cox regression models with nonbiologics as the reference. Baseline glucocorticoid use was evaluated as a separate covariate. MAIN OUTCOME MEASURE: Infections requiring hospitalization (serious infections) during the first 12 months after initiation of TNF-α antagonists or nonbiologic regimens. RESULTS: Study cohorts included 10,484 RA, 2323 IBD, and 3215 psoriasis and spondyloarthropathies matched pairs using TNF-α antagonists and comparator medications. Overall, we identified 1172 serious infections, most of which (53%) were pneumonia and skin and soft tissue infections. Among patients with RA, serious infection hospitalization rates were 8.16 (TNF-α antagonists) and 7.78 (comparator regimens) per 100 person-years (adjusted hazard ratio [aHR], 1.05 [95% CI, 0.91-1.21]). Among patients with IBD, rates were 10.91 (TNF-α antagonists) and 9.60 (comparator) per 100 person-years (aHR, 1.10 [95% CI, 0.83-1.46]). Among patients with psoriasis and spondyloarthropathies, rates were 5.41 (TNF-α antagonists) and 5.37 (comparator) per 100 person-years (aHR, 1.05 [95% CI, 0.76-1.45]). Among patients with RA, infliximab was associated with a significant increase in serious infections compared with etanercept (aHR, 1.26 [95% CI, 1.07-1.47]) and adalimumab (aHR, 1.23 [95% CI, 1.02-1.48]). Baseline glucocorticoid use was associated with a dose-dependent increase in infections. CONCLUSION: Among patients with autoimmune diseases, compared with treatment with nonbiologic regimens, initiation of TNF-α antagonists was not associated with an increased risk of hospitalizations for serious infections.

Changes in cotherapies after initiation of disease-modifying antirheumatic drug therapy in patients with rheumatoid arthritis.

Objective. We hypothesized that initiation of a new disease-modifying antirheumatic drug (DMARD) for treatment of rheumatoid arthritis (RA) would decrease the use of corticosteroids, nonsteroidal antiinflammatory drugs (NSAIDs), and narcotics.Methods. Using administrative databases, we assembled 4 retrospective cohorts of RA patients (1998-2005) and identified 5 groups initiating DMARD regimens: methotrexate (MTX) with (new MTX) or without (first MTX) use of other nonbiologic DMARDs in the previous year; new hydroxychloroquine (HCQ) and/or sulfasalazine (SSZ; new HCQ/SSZ)and new leflunomide (new LEF), both with previous use of MTX; and new tumor necrosis factor α (TNFα) antagonists(new anti-TNF). We compared within-person differences in any use of cotherapies (≥ prescription) between the 6 months before and the 6-12 months after DMARD initiation.Results. Among 32,476 DMARD initiators, the prevalence of corticosteroid, NSAID, and narcotic use increased by 15%, 5%,and 6%, respectively, in the 6 months before initiation compared to the previous 6 months, suggesting worsening of the disease. In the 6-12 months after initiation for most initiator groups, more patients stopped using corticosteroids and NSAIDs than started, with overall decreases of 8.9% (95% confidence interval [95% CI] 8.4-9.4%) for corticosteroids and 12.9% (95%CI 12.3-13.4%) for NSAIDs. The proportion of narcotic users changed little (overall decrease of 2.5%; 95% CI 1.9-3.0%).Conclusion. Use of all 3 cotherapies increased in the 6 months before initiation of new DMARD regimens for RA. Use of corticosteroids and NSAIDs decreased modestly 6-12 months after initiation, but there was only a very small decrease in narcotic use. These differential changes require further study.

Risk of acute myocardial infarction, stroke, heart failure, and death in elderly Medicare patients treated with rosiglitazone or pioglitazone.

CONTEXT: Studies have suggested that the use of rosiglitazone may be associated with an increased risk of serious cardiovascular events compared with other treatments for type 2 diabetes. OBJECTIVE: To determine if the risk of serious cardiovascular harm is increased by rosiglitazone compared with pioglitazone, the other thiazolidinedione marketed in the United States. DESIGN, SETTING, AND PATIENTS: Nationwide, observational, retrospective, inception cohort of 227,571 Medicare beneficiaries aged 65 years or older (mean age, 74.4 years) who initiated treatment with rosiglitazone or pioglitazone through a Medicare Part D prescription drug plan from July 2006-June 2009 and who underwent follow-up for up to 3 years after thiazolidinedione initiation. MAIN OUTCOME MEASURES: Individual end points of acute myocardial infarction (AMI), stroke, heart failure, and all-cause mortality (death), and composite end point of AMI, stroke, heart failure, or death, assessed using incidence rates by thiazolidinedione, attributable risk, number needed to harm, Kaplan-Meier plots of time to event, and Cox proportional hazard ratios for time to event, adjusted for potential confounding factors, with pioglitazone as reference. RESULTS: A total of 8667 end points were observed during the study period. The adjusted hazard ratio for rosiglitazone compared with pioglitazone was 1.06 (95% confidence interval [CI], 0.96-1.18) for AMI; 1.27 (95% CI, 1.12-1.45) for stroke; 1.25 (95% CI, 1.16-1.34) for heart failure; 1.14 (95% CI, 1.05-1.24) for death; and 1.18 (95% CI, 1.12-1.23) for the composite of AMI, stroke, heart failure, or death. The attributable risk for this composite end point was 1.68 (95% CI, 1.27-2.08) excess events per 100 person-years of treatment with rosiglitazone compared with pioglitazone. The corresponding number needed to harm was 60 (95% CI, 48-79) treated for 1 year. CONCLUSION: Compared with prescription of pioglitazone, prescription of rosiglitazone was associated with an increased risk of stroke, heart failure, and all-cause mortality and an increased risk of the composite of AMI, stroke, heart failure, or all-cause mortality in patients 65 years or older.

Perspectives on the use of data mining in pharmaco-vigilance.

In the last 5 years, regulatory agencies and drug monitoring centres have been developing computerised data-mining methods to better identify reporting relationships in spontaneous reporting databases that could signal possible adverse drug reactions. At present, there are no guidelines or standards for the use of these methods in routine pharmaco-vigilance. In 2003, a group of statisticians, pharmaco-epidemiologists and pharmaco-vigilance professionals from the pharmaceutical industry and the US FDA formed the Pharmaceutical Research and Manufacturers of America-FDA Collaborative Working Group on Safety Evaluation Tools to review best practices for the use of these methods.In this paper, we provide an overview of: (i) the statistical and operational attributes of several currently used methods and their strengths and limitations; (ii) information about the characteristics of various postmarketing safety databases with which these tools can be deployed; (iii) analytical considerations for using safety data-mining methods and interpreting the results; and (iv) points to consider in integration of safety data mining with traditional pharmaco-vigilance methods. Perspectives from both the FDA and the industry are provided. Data mining is a potentially useful adjunct to traditional pharmaco-vigilance methods. The results of data mining should be viewed as hypothesis generating and should be evaluated in the context of other relevant data. The availability of a publicly accessible global safety database, which is updated on a frequent basis, would further enhance detection and communication about safety issues.