Bayesian safety surveillance with adaptive bias correction.

Postmarket safety surveillance is an integral part of mass vaccination programs. Typically relying on sequential analysis of real-world health data as they accrue, safety surveillance is challenged by sequential multiple testing and by biases induced by residual confounding in observational data. The current standard approach based on the maximized sequential probability ratio test (MaxSPRT) fails to satisfactorily address these practical challenges and it remains a rigid framework that requires prespecification of the surveillance schedule. We develop an alternative Bayesian surveillance procedure that addresses both aforementioned challenges using a more flexible framework. To mitigate bias, we jointly analyze a large set of negative control outcomes that are adverse events with no known association with the vaccines in order to inform an empirical bias distribution, which we then incorporate into estimating the effect of vaccine exposure on the adverse event of interest through a Bayesian hierarchical model. To address multiple testing and improve on flexibility, at each analysis timepoint, we update a posterior probability in favor of the alternative hypothesis that vaccination induces higher risks of adverse events, and then use it for sequential detection of safety signals. Through an empirical evaluation using six US observational healthcare databases covering more than 360 million patients, we benchmark the proposed procedure against MaxSPRT on testing errors and estimation accuracy, under two epidemiological designs, the historical comparator and the self-controlled case series. We demonstrate that our procedure substantially reduces Type 1 error rates, maintains high statistical power and fast signal detection, and provides considerably more accurate estimation than MaxSPRT. Given the extensiveness of the empirical study which yields more than 7 million sets of results, we present all results in a public R ShinyApp. As an effort to promote open science, we provide full implementation of our method in the open-source R package EvidenceSynthesis.

Adjusting for both sequential testing and systematic error in safety surveillance using observational data: Empirical calibration and MaxSPRT.

Post-approval safety surveillance of medical products using observational healthcare data can help identify safety issues beyond those found in pre-approval trials. When testing sequentially as data accrue, maximum sequential probability ratio testing (MaxSPRT) is a common approach to maintaining nominal type 1 error. However, the true type 1 error may still deviate from the specified one because of systematic error due to the observational nature of the analysis. This systematic error may persist even after controlling for known confounders. Here we propose to address this issue by combing MaxSPRT with empirical calibration. In empirical calibration, we assume uncertainty about the systematic error in our analysis, the source of uncertainty commonly overlooked in practice. We infer a probability distribution of systematic error by relying on a large set of negative controls: exposure-outcome pairs where no causal effect is believed to exist. Integrating this distribution into our test statistics has previously been shown to restore type 1 error to nominal. Here we show how we can calibrate the critical value central to MaxSPRT. We evaluate this novel approach using simulations and real electronic health records, using H1N1 vaccinations during the 2009-2010 season as an example. Results show that combining empirical calibration with MaxSPRT restores nominal type 1 error. In our real-world example, adjusting for systematic error using empirical calibration has a larger impact than, and hence is just as essential as, adjusting for sequential testing using MaxSPRT. We recommend performing both, using the method described here.

Padé approximant meets federated learning: A nearly lossless, one-shot algorithm for evidence synthesis in distributed research networks with rare outcomes.

OBJECTIVE: We developed and evaluated a novel one-shot distributed algorithm for evidence synthesis in distributed research networks with rare outcomes. MATERIALS AND METHODS: Fed-Padé, motivated by a classic mathematical tool, Padé approximants, reconstructs the multi-site data likelihood via Padé approximant whose key parameters can be computed distributively. Thanks to the simplicity of [2,2] Padé approximant, Fed-Padé requests an extremely simple task and low communication cost for data partners. Specifically, each data partner only needs to compute and share the log-likelihood and its first 4 gradients evaluated at an initial estimator. We evaluated the performance of our algorithm with extensive simulation studies and four observational healthcare databases. RESULTS: Our simulation studies revealed that a [2,2]-Padé approximant can well reconstruct the multi-site likelihood so that Fed-Padé produces nearly identical estimates to the pooled analysis. Across all simulation scenarios considered, the median of relative bias and rate of instability of our Fed-Padé are both <0.1%, whereas meta-analysis estimates have bias up to 50% and instability up to 75%. Furthermore, the confidence intervals derived from the Fed-Padé algorithm showed better coverage of the truth than confidence intervals based on the meta-analysis. In real data analysis, the Fed-Padé has a relative bias of <1% for all three comparisons for risks of acute liver injury and decreased libido, whereas the meta-analysis estimates have a substantially higher bias (around 10%). CONCLUSION: The Fed-Padé algorithm is nearly lossless, stable, communication-efficient, and easy to implement for models with rare outcomes. It provides an extremely suitable and convenient approach for synthesizing evidence in distributed research networks with rare outcomes.

Adjusting for indirectly measured confounding using large-scale propensity score.

Confounding remains one of the major challenges to causal inference with observational data. This problem is paramount in medicine, where we would like to answer causal questions from large observational datasets like electronic health records (EHRs) and administrative claims. Modern medical data typically contain tens of thousands of covariates. Such a large set carries hope that many of the confounders are directly measured, and further hope that others are indirectly measured through their correlation with measured covariates. How can we exploit these large sets of covariates for causal inference? To help answer this question, this paper examines the performance of the large-scale propensity score (LSPS) approach on causal analysis of medical data. We demonstrate that LSPS may adjust for indirectly measured confounders by including tens of thousands of covariates that may be correlated with them. We present conditions under which LSPS removes bias due to indirectly measured confounders, and we show that LSPS may avoid bias when inadvertently adjusting for variables (like colliders) that otherwise can induce bias. We demonstrate the performance of LSPS with both simulated medical data and real medical data.

PheValuator 2.0: Methodological improvements for the PheValuator approach to semi-automated phenotype algorithm evaluation.

PURPOSE: Phenotype algorithms are central to performing analyses using observational data. These algorithms translate the clinical idea of a health condition into an executable set of rules allowing for queries of data elements from a database. PheValuator, a software package in the Observational Health Data Sciences and Informatics (OHDSI) tool stack, provides a method to assess the performance characteristics of these algorithms, namely, sensitivity, specificity, and positive and negative predictive value. It uses machine learning to develop predictive models for determining a probabilistic gold standard of subjects for assessment of cases and non-cases of health conditions. PheValuator was developed to complement or even replace the traditional approach of algorithm validation, i.e., by expert assessment of subject records through chart review. Results in our first PheValuator paper suggest a systematic underestimation of the PPV compared to previous results using chart review. In this paper we evaluate modifications made to the method designed to improve its performance. METHODS: The major changes to PheValuator included allowing all diagnostic conditions, clinical observations, drug prescriptions, and laboratory measurements to be included as predictors within the modeling process whereas in the prior version there were significant restrictions on the included predictors. We also have allowed for the inclusion of the temporal relationships of the predictors in the model. To evaluate the performance of the new method, we compared the results from the new and original methods against results found from the literature using traditional validation of algorithms for 19 phenotypes. We performed these tests using data from five commercial databases. RESULTS: In the assessment aggregating all phenotype algorithms, the median difference between the PheValuator estimate and the gold standard estimate for PPV was reduced from -21 (IQR -34, -3) in Version 1.0 to 4 (IQR -3, 15) using Version 2.0. We found a median difference in specificity of 3 (IQR 1, 4.25) for Version 1.0 and 3 (IQR 1, 4) for Version 2.0. The median difference between the two versions of PheValuator and the gold standard for estimates of sensitivity was reduced from -39 (-51, -20) to -16 (-34, -6). CONCLUSION: PheValuator 2.0 produces estimates for the performance characteristics for phenotype algorithms that are significantly closer to estimates from traditional validation through chart review compared to version 1.0. With this tool in researcher's toolkits, methods, such as quantitative bias analysis, may now be used to improve the reliability and reproducibility of research studies using observational data.

Indirect covariate balance and residual confounding: An applied comparison of propensity score matching and cardinality matching.

PURPOSE: Propensity score matching (PSM) is subject to limitations associated with limited degrees of freedom and covariate overlap. Cardinality matching (CM), an optimization algorithm, overcomes these limitations by matching directly on the marginal distribution of covariates. This study compared the performance of PSM and CM. METHODS: Comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and ß-blocker monotherapy identified from a large U.S. administrative claims database. One-to-one matching was conducted through PSM using nearest-neighbor matching (caliper = 0.15) and CM permitting a maximum standardized mean difference (SMD) of 0, 0.01, 0.05, and 0.10 between comparison groups. Matching covariates included 37 patient demographic and clinical characteristics. Observed covariates included patient demographics, and all observed prior conditions, drug exposures, and procedures. Residual confounding was assessed based on the expected absolute systematic error of negative control outcome experiments. PSM and CM were compared in terms of post-match patient retention, matching and observed covariate balance, and residual confounding within a 10%, 1%, 0.25% and 0.125% sample group. RESULTS: The eligible study population included 182 235 (ACEI: 129363; ß-blocker: 56872) patients. CM achieved superior patient retention and matching covariate balance in all analyses. After PSM, 1.6% and 28.2% of matching covariates were imbalanced in the 10% and 0.125% sample groups, respectively. No significant difference in observed covariate balance was observed between matching techniques. CM permitting a maximum SMD <0.05 was associated with improved residual bias as compared to PSM. CONCLUSION: We recommend CM with more stringent balance criteria as an alternative to PSM when matching on a set of clinically relevant covariates.

Learning patient-level prediction models across multiple healthcare databases: evaluation of ensembles for increasing model transportability.

BACKGROUND: Prognostic models that are accurate could help aid medical decision making. Large observational databases often contain temporal medical data for large and diverse populations of patients. It may be possible to learn prognostic models using the large observational data. Often the performance of a prognostic model undesirably worsens when transported to a different database (or into a clinical setting). In this study we investigate different ensemble approaches that combine prognostic models independently developed using different databases (a simple federated learning approach) to determine whether ensembles that combine models developed across databases can improve model transportability (perform better in new data than single database models)? METHODS: For a given prediction question we independently trained five single database models each using a different observational healthcare database. We then developed and investigated numerous ensemble models (fusion, stacking and mixture of experts) that combined the different database models. Performance of each model was investigated via discrimination and calibration using a leave one dataset out technique, i.e., hold out one database to use for validation and use the remaining four datasets for model development. The internal validation of a model developed using the hold out database was calculated and presented as the 'internal benchmark' for comparison. RESULTS: In this study the fusion ensembles generally outperformed the single database models when transported to a previously unseen database and the performances were more consistent across unseen databases. Stacking ensembles performed poorly in terms of discrimination when the labels in the unseen database were limited. Calibration was consistently poor when both ensembles and single database models were applied to previously unseen databases. CONCLUSION: A simple federated learning approach that implements ensemble techniques to combine models independently developed across different databases for the same prediction question may improve the discriminative performance in new data (new database or clinical setting) but will need to be recalibrated using the new data. This could help medical decision making by improving prognostic model performance.

Applied comparison of large-scale propensity score matching and cardinality matching for causal inference in observational research.

BACKGROUND: Cardinality matching (CM), a novel matching technique, finds the largest matched sample meeting prespecified balance criteria thereby overcoming limitations of propensity score matching (PSM) associated with limited covariate overlap, which are especially pronounced in studies with small sample sizes. The current study proposes a framework for large-scale CM (LS-CM); and compares large-scale PSM (LS-PSM) and LS-CM in terms of post-match sample size, covariate balance and residual confounding at progressively smaller sample sizes. METHODS: Evaluation of LS-PSM and LS-CM within a comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and thiazide or thiazide-like diuretic monotherapy identified from a U.S. insurance claims database. Candidate covariates included patient demographics, and all observed prior conditions, drug exposures and procedures. Propensity scores were calculated using LASSO regression, and candidate covariates with non-zero beta coefficients in the propensity model were defined as matching covariates for use in LS-CM. One-to-one matching was performed using progressively tighter parameter settings. Covariate balance was assessed using standardized mean differences. Hazard ratios for negative control outcomes perceived as unassociated with treatment (i.e., true hazard ratio of 1) were estimated using unconditional Cox models. Residual confounding was assessed using the expected systematic error of the empirical null distribution of negative control effect estimates compared to the ground truth. To simulate diverse research conditions, analyses were repeated within 10 %, 1 and 0.5 % subsample groups with increasingly limited covariate overlap. RESULTS: A total of 172,117 patients (ACEI: 129,078; thiazide: 43,039) met the study criteria. As compared to LS-PSM, LS-CM was associated with increased sample retention. Although LS-PSM achieved balance across all matching covariates within the full study population, substantial matching covariate imbalance was observed within the 1 and 0.5 % subsample groups. Meanwhile, LS-CM achieved matching covariate balance across all analyses. LS-PSM was associated with better candidate covariate balance within the full study population. Otherwise, both matching techniques achieved comparable candidate covariate balance and expected systematic error. CONCLUSIONS: LS-CM found the largest matched sample meeting prespecified balance criteria while achieving comparable candidate covariate balance and residual confounding. We recommend LS-CM as an alternative to LS-PSM in studies with small sample sizes or limited covariate overlap.

Empirical assessment of case-based methods for identification of drugs associated with acute liver injury in the French National Healthcare System database (SNDS).

PURPOSES: Drug induced acute liver injury (ALI) is a frequent cause of liver failure. Case-based designs were empirically assessed and calibrated in the French National claims database (SNDS), aiming to identify the optimum design for drug safety alert generation associated with ALI. METHODS: All cases of ALI were extracted from SNDS (2009-2014) using specific and sensitive definitions. Positive and negative drug controls were used to compare 196 self-controlled case series (SCCS), case-control (CC), and case-population (CP) design variants, using area under the receiver operating curve (AUC), mean square error (MSE) and coverage probability. Parameters that had major impacts on results were identified through logistic regression. RESULTS: Using a specific ALI definition, AUCs ranged from 0.78 to 0.94, 0.64 to 0.92 and 0.48 to 0.85, for SCCS, CC and CP, respectively. MSE ranged from 0.12 to 0.40, 0.22 to 0.39 and 1.03 to 5.29, respectively. Variants adjusting for multiple drug use had higher coverage probabilities. Univariate regressions showed that high AUCs were achieved with SCCS using exposed time as the risk window. The top SCCS variant yielded an AUC = 0.93 and MSE = 0.22 and coverage = 86%, with 1/7 negative and 13/18 positive controls presenting significant estimates. CONCLUSIONS: SCCS adjusting for multiple drugs and using exposed time as the risk window performed best in generating ALI-related drug safety alert and providing estimates of the magnitude of the risk. This approach may be useful for ad-hoc pharmacoepidemiology studies to support regulatory actions.

Quantifying bias in epidemiologic studies evaluating the association between acetaminophen use and cancer.

Many observational studies explore the association between acetaminophen and cancer, but known limitations such as vulnerability to channeling, protopathic bias, and uncontrolled confounding hamper the interpretability of results. To help understand the potential magnitude of bias, we identify key design choices in these observational studies and specify 10 study design variants that represent different combinations of these design choices. We evaluate these variants by applying them to 37 negative controls - outcome presumed not to be caused by acetaminophen - as well as 4 cancer outcomes in the Clinical Practice Research Datalink (CPRD) database. The estimated odds and hazards ratios for the negative controls show substantial bias in the evaluated design variants, with far fewer of the 95% confidence intervals containing 1 than the nominal 95% expected for negative controls. The effect-size estimates for the cancer outcomes are comparable to those observed for the negative controls. A comparison of exposed and unexposed reveals many differences at baseline for which most studies do not correct. We observe that the design choices made in many of the published observational studies can lead to substantial bias. Thus, caution in the interpretation of published studies of acetaminophen and cancer is recommended.

A systematic assessment of the epidemiologic literature regarding an association between acetaminophen exposure and cancer.

Introduced in the 1950s, acetaminophen is one of the most widely used antipyretics and analgesics worldwide. In 1999, the International Agency for Research on Cancer (IARC) reviewed the epidemiologic studies of acetaminophen and the data were judged to be "inadequate" to conclude that it is carcinogenic. In 2019 the California Office of Environmental Health Hazard Assessment initiated a review process on the carcinogenic hazard potential of acetaminophen. To inform this review process, the authors performed a comprehensive literature search and identified 136 epidemiologic studies, which for most cancer types suggest no alteration in risk associated with acetaminophen use. For 3 cancer types, renal cell, liver, and some forms of lymphohematopoietic, some studies suggest an increased risk; however, multiple factors unique to acetaminophen need to be considered to determine if these results are real and clinically meaningful. The objective of this publication is to analyze the results of these epidemiologic studies using a framework that accounts for the inherent challenge of evaluating acetaminophen, including, broad population-wide use in multiple disease states, challenges with exposure measurement, protopathic bias, channeling bias, and recall bias. When evaluated using this framework, the data do not support a causal association between acetaminophen use and cancer.

Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data.

Observational healthcare data, such as electronic health records and administrative claims, offer potential to estimate effects of medical products at scale. Observational studies have often been found to be nonreproducible, however, generating conflicting results even when using the same database to answer the same question. One source of discrepancies is error, both random caused by sampling variability and systematic (for example, because of confounding, selection bias, and measurement error). Only random error is typically quantified but converges to zero as databases become larger, whereas systematic error persists independent from sample size and therefore, increases in relative importance. Negative controls are exposure-outcome pairs, where one believes no causal effect exists; they can be used to detect multiple sources of systematic error, but interpreting their results is not always straightforward. Previously, we have shown that an empirical null distribution can be derived from a sample of negative controls and used to calibrate P values, accounting for both random and systematic error. Here, we extend this work to calibration of confidence intervals (CIs). CIs require positive controls, which we synthesize by modifying negative controls. We show that our CI calibration restores nominal characteristics, such as 95% coverage of the true effect size by the 95% CI. We furthermore show that CI calibration reduces disagreement in replications of two pairs of conflicting observational studies: one related to dabigatran, warfarin, and gastrointestinal bleeding and one related to selective serotonin reuptake inhibitors and upper gastrointestinal bleeding. We recommend CI calibration to improve reproducibility of observational studies.

An empirical analysis of dealing with patients who are lost to follow-up when developing prognostic models using a cohort design.

BACKGROUND: Researchers developing prediction models are faced with numerous design choices that may impact model performance. One key decision is how to include patients who are lost to follow-up. In this paper we perform a large-scale empirical evaluation investigating the impact of this decision. In addition, we aim to provide guidelines for how to deal with loss to follow-up. METHODS: We generate a partially synthetic dataset with complete follow-up and simulate loss to follow-up based either on random selection or on selection based on comorbidity. In addition to our synthetic data study we investigate 21 real-world data prediction problems. We compare four simple strategies for developing models when using a cohort design that encounters loss to follow-up. Three strategies employ a binary classifier with data that: (1) include all patients (including those lost to follow-up), (2) exclude all patients lost to follow-up or (3) only exclude patients lost to follow-up who do not have the outcome before being lost to follow-up. The fourth strategy uses a survival model with data that include all patients. We empirically evaluate the discrimination and calibration performance. RESULTS: The partially synthetic data study results show that excluding patients who are lost to follow-up can introduce bias when loss to follow-up is common and does not occur at random. However, when loss to follow-up was completely at random, the choice of addressing it had negligible impact on model discrimination performance. Our empirical real-world data results showed that the four design choices investigated to deal with loss to follow-up resulted in comparable performance when the time-at-risk was 1-year but demonstrated differential bias when we looked into 3-year time-at-risk. Removing patients who are lost to follow-up before experiencing the outcome but keeping patients who are lost to follow-up after the outcome can bias a model and should be avoided. CONCLUSION: Based on this study we therefore recommend (1) developing models using data that includes patients that are lost to follow-up and (2) evaluate the discrimination and calibration of models twice: on a test set including patients lost to follow-up and a test set excluding patients lost to follow-up.

Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis.

BACKGROUND: Uncertainty remains about the optimal monotherapy for hypertension, with current guidelines recommending any primary agent among the first-line drug classes thiazide or thiazide-like diuretics, angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, dihydropyridine calcium channel blockers, and non-dihydropyridine calcium channel blockers, in the absence of comorbid indications. Randomised trials have not further refined this choice. METHODS: We developed a comprehensive framework for real-world evidence that enables comparative effectiveness and safety evaluation across many drugs and outcomes from observational data encompassing millions of patients, while minimising inherent bias. Using this framework, we did a systematic, large-scale study under a new-user cohort design to estimate the relative risks of three primary (acute myocardial infarction, hospitalisation for heart failure, and stroke) and six secondary effectiveness and 46 safety outcomes comparing all first-line classes across a global network of six administrative claims and three electronic health record databases. The framework addressed residual confounding, publication bias, and p-hacking using large-scale propensity adjustment, a large set of control outcomes, and full disclosure of hypotheses tested. FINDINGS: Using 4·9 million patients, we generated 22 000 calibrated, propensity-score-adjusted hazard ratios (HRs) comparing all classes and outcomes across databases. Most estimates revealed no effectiveness differences between classes; however, thiazide or thiazide-like diuretics showed better primary effectiveness than angiotensin-converting enzyme inhibitors: acute myocardial infarction (HR 0·84, 95% CI 0·75-0·95), hospitalisation for heart failure (0·83, 0·74-0·95), and stroke (0·83, 0·74-0·95) risk while on initial treatment. Safety profiles also favoured thiazide or thiazide-like diuretics over angiotensin-converting enzyme inhibitors. The non-dihydropyridine calcium channel blockers were significantly inferior to the other four classes. INTERPRETATION: This comprehensive framework introduces a new way of doing observational health-care science at scale. The approach supports equivalence between drug classes for initiating monotherapy for hypertension-in keeping with current guidelines, with the exception of thiazide or thiazide-like diuretics superiority to angiotensin-converting enzyme inhibitors and the inferiority of non-dihydropyridine calcium channel blockers. FUNDING: US National Science Foundation, US National Institutes of Health, Janssen Research & Development, IQVIA, South Korean Ministry of Health & Welfare, Australian National Health and Medical Research Council.

Empirical assessment of case-based methods for drug safety alert identification in the French National Healthcare System database (SNDS): Methodology of the ALCAPONE project.

OBJECTIVES: To introduce the methodology of the ALCAPONE project. BACKGROUND: The French National Healthcare System Database (SNDS), covering 99% of the French population, provides a potentially valuable opportunity for drug safety alert generation. ALCAPONE aimed to assess empirically in the SNDS case-based designs for alert generation related to four health outcomes of interest. METHODS: ALCAPONE used a reference set adapted from observational medical outcomes partnership (OMOP) and Exploring and Understanding Adverse Drug Reactions (EU-ADR) project, with four outcomes-acute liver injury (ALI), myocardial infarction (MI), acute kidney injury (AKI), and upper gastrointestinal bleeding (UGIB)-and positive and negative drug controls. ALCAPONE consisted of four main phases: (1) data preparation to fit the OMOP Common Data Model and select the drug controls; (2) detection of the selected controls via three case-based designs: case-population, case-control, and self-controlled case series, including design variants (varying risk window, adjustment strategy, etc.); (3) comparison of design variant performance (area under the ROC curve, mean square error, etc.); and (4) selection of the optimal design variants and their calibration for each outcome. RESULTS: Over 2009-2014, 5225 cases of ALI, 354 109 MI, 12 633 AKI, and 156 057 UGIB were identified using specific definitions. The number of detectable drugs ranged from 61 for MI to 25 for ALI. Design variants generated more than 50 000 points estimates. Results by outcome will be published in forthcoming papers. CONCLUSIONS: ALCAPONE has shown the interest of the empirical assessment of pharmacoepidemiological approaches for drug safety alert generation and may encourage other researchers to do the same in other databases.

Empirical assessment of case-based methods for identification of drugs associated with upper gastrointestinal bleeding in the French National Healthcare System database (SNDS).

PURPOSE: Upper gastrointestinal bleeding (UGIB) is a severe and frequent drug-related event. In order to enable efficient drug safety alert generation in the French National Healthcare System database (SNDS), we assessed and calibrated empirically case-based designs to identify drug associated with UGIB risk. METHODS: All cases of UGIB were extracted from SNDS (2009-2014) using two definitions. Positive and negative drug controls were used to compare 196 self-controlled case series (SCCS), case-control (CC) and case-population (CP) design variants. Each variant was evaluated in a 1/10th population sample using area under the receiver operating curve (AUC) and mean square error (MSE). Parameters that had major impacts on results were identified through logistic regression. Optimal designs were replicated in the unsampled population. RESULTS: Using a specific UGIB definition, AUCs ranged from 0.64 to 0.80, 0.44 to 0.61 and 0.50 to 0.67, for SCCS, CC and CP, respectively. MSE ranged from 0.07 to 0.39, 0.83 to 1.33 and 1.96 to 4.6, respectively. Univariate regressions showed that high AUCs were achieved with SCCS with multiple drug adjustment and a 30-day risk window starting at exposure. The top-performing SCCS variant in the unsampled population yielded an AUC = 0.84 and MSE = 0.14, with 10/36 negative controls presenting significant estimates. CONCLUSIONS: SCCS adjusting for multiple drugs and using a 30-day risk window has the potential to generate UGIB-related alerts in the SNDS and hypotheses on its potential population impact. Negative control implementation highlighted that low systematic error was generated but that protopathic bias and confounding by indication remained unaddressed issues.

Association of Ticagrelor vs Clopidogrel With Net Adverse Clinical Events in Patients With Acute Coronary Syndrome Undergoing Percutaneous Coronary Intervention.

Importance: Current guidelines recommend ticagrelor as the preferred P2Y12 platelet inhibitor for patients with acute coronary syndrome (ACS), primarily based on a single large randomized clinical trial. The benefits and risks associated with ticagrelor vs clopidogrel in routine practice merits attention. Objective: To determine the association of ticagrelor vs clopidogrel with ischemic and hemorrhagic events in patients undergoing percutaneous coronary intervention (PCI) for ACS in clinical practice. Design, Setting, and Participants: A retrospective cohort study of patients with ACS who underwent PCI and received ticagrelor or clopidogrel was conducted using 2 United States electronic health record-based databases and 1 nationwide South Korean database from November 2011 to March 2019. Patients were matched using a large-scale propensity score algorithm, and the date of final follow-up was March 2019. Exposures: Ticagrelor vs clopidogrel. Main Outcomes and Measures: The primary end point was net adverse clinical events (NACE) at 12 months, composed of ischemic events (recurrent myocardial infarction, revascularization, or ischemic stroke) and hemorrhagic events (hemorrhagic stroke or gastrointestinal bleeding). Secondary outcomes included NACE or mortality, all-cause mortality, ischemic events, hemorrhagic events, individual components of the primary outcome, and dyspnea at 12 months. The database-level hazard ratios (HRs) were pooled to calculate summary HRs by random-effects meta-analysis. Results: After propensity score matching among 31 290 propensity-matched pairs (median age group, 60-64 years; 29.3% women), 95.5% of patients took aspirin together with ticagrelor or clopidogrel. The 1-year risk of NACE was not significantly different between ticagrelor and clopidogrel (15.1% [3484/23 116 person-years] vs 14.6% [3290/22 587 person-years]; summary HR, 1.05 [95% CI, 1.00-1.10]; P = .06). There was also no significant difference in the risk of all-cause mortality (2.0% for ticagrelor vs 2.1% for clopidogrel; summary HR, 0.97 [95% CI, 0.81-1.16]; P = .74) or ischemic events (13.5% for ticagrelor vs 13.4% for clopidogrel; summary HR, 1.03 [95% CI, 0.98-1.08]; P = .32). The risks of hemorrhagic events (2.1% for ticagrelor vs 1.6% for clopidogrel; summary HR, 1.35 [95% CI, 1.13-1.61]; P = .001) and dyspnea (27.3% for ticagrelor vs 22.6% for clopidogrel; summary HR, 1.21 [95% CI, 1.17-1.26]; P < .001) were significantly higher in the ticagrelor group. Conclusions and Relevance: Among patients with ACS who underwent PCI in routine clinical practice, ticagrelor, compared with clopidogrel, was not associated with significant difference in the risk of NACE at 12 months. Because the possibility of unmeasured confounders cannot be excluded, further research is needed to determine whether ticagrelor is more effective than clopidogrel in this setting.

A plea to stop using the case-control design in retrospective database studies.

The case-control design is widely used in retrospective database studies, often leading to spectacular findings. However, results of these studies often cannot be replicated, and the advantage of this design over others is questionable. To demonstrate the shortcomings of applications of this design, we replicate two published case-control studies. The first investigates isotretinoin and ulcerative colitis using a simple case-control design. The second focuses on dipeptidyl peptidase-4 inhibitors and acute pancreatitis, using a nested case-control design. We include large sets of negative control exposures (where the true odds ratio is believed to be 1) in both studies. Both replication studies produce effect size estimates consistent with the original studies, but also generate estimates for the negative control exposures showing substantial residual bias. In contrast, applying a self-controlled design to answer the same questions using the same data reveals far less bias. Although the case-control design in general is not at fault, its application in retrospective database studies, where all exposure and covariate data for the entire cohort are available, is unnecessary, as other alternatives such as cohort and self-controlled designs are available. Moreover, by focusing on cases and controls it opens the door to inappropriate comparisons between exposure groups, leading to confounding for which the design has few options to adjust for. We argue that this design should no longer be used in these types of data. At the very least, negative control exposures should be used to prove that the concerns raised here do not apply.

Diabetic ketoacidosis in patients with type 2 diabetes treated with sodium glucose co-transporter 2 inhibitors versus other antihyperglycemic agents: An observational study of four US administrative claims databases.

PURPOSE: To compare the incidence of diabetic ketoacidosis (DKA) among patients with type 2 diabetes mellitus (T2DM) who were new users of sodium glucose co-transporter 2 inhibitors (SGLT2i) versus other classes of antihyperglycemic agents (AHAs). METHODS: Patients were identified from four large US claims databases using broad (all T2DM patients) and narrow (intended to exclude patients with type 1 diabetes or secondary diabetes misclassified as T2DM) definitions of T2DM. New users of SGLT2i and seven groups of comparator AHAs were matched (1:1) on exposure propensity scores to adjust for imbalances in baseline covariates. Cox proportional hazards regression models, conditioned on propensity score-matched pairs, were used to estimate hazard ratios (HRs) of DKA for new users of SGLT2i versus other AHAs. When I2 <40%, a combined HR across the four databases was estimated. RESULTS: Using the broad definition of T2DM, new users of SGLT2i had an increased risk of DKA versus sulfonylureas (HR [95% CI]: 1.53 [1.31-1.79]), DPP-4i (1.28 [1.11-1.47]), GLP-1 receptor agonists (1.34 [1.12-1.60]), metformin (1.31 [1.11-1.54]), and insulinotropic AHAs (1.38 [1.15-1.66]). Using the narrow definition of T2DM, new users of SGLT2i had an increased risk of DKA versus sulfonylureas (1.43 [1.01-2.01]). New users of SGLT2i had a lower risk of DKA versus insulin and a similar risk as thiazolidinediones, regardless of T2DM definition. CONCLUSIONS: Increased risk of DKA was observed for new users of SGLT2i versus several non-SGLT2i AHAs when T2DM was defined broadly. When T2DM was defined narrowly to exclude possible misclassified patients, an increased risk of DKA with SGLT2i was observed compared with sulfonylureas.

Characterizing treatment pathways at scale using the OHDSI network.

Observational research promises to complement experimental research by providing large, diverse populations that would be infeasible for an experiment. Observational research can test its own clinical hypotheses, and observational studies also can contribute to the design of experiments and inform the generalizability of experimental research. Understanding the diversity of populations and the variance in care is one component. In this study, the Observational Health Data Sciences and Informatics (OHDSI) collaboration created an international data network with 11 data sources from four countries, including electronic health records and administrative claims data on 250 million patients. All data were mapped to common data standards, patient privacy was maintained by using a distributed model, and results were aggregated centrally. Treatment pathways were elucidated for type 2 diabetes mellitus, hypertension, and depression. The pathways revealed that the world is moving toward more consistent therapy over time across diseases and across locations, but significant heterogeneity remains among sources, pointing to challenges in generalizing clinical trial results. Diabetes favored a single first-line medication, metformin, to a much greater extent than hypertension or depression. About 10% of diabetes and depression patients and almost 25% of hypertension patients followed a treatment pathway that was unique within the cohort. Aside from factors such as sample size and underlying population (academic medical center versus general population), electronic health records data and administrative claims data revealed similar results. Large-scale international observational research is feasible.