Transparency and Reproducibility of Observational Cohort Studies Using Large Healthcare Databases.

The scientific community and decision-makers are increasingly concerned about transparency and reproducibility of epidemiologic studies using longitudinal healthcare databases. We explored the extent to which published pharmacoepidemiologic studies using commercially available databases could be reproduced by other investigators. We identified a nonsystematic sample of 38 descriptive or comparative safety/effectiveness cohort studies. Seven studies were excluded from reproduction, five because of violation of fundamental design principles, and two because of grossly inadequate reporting. In the remaining studies, >1,000 patient characteristics and measures of association were reproduced with a high degree of accuracy (median differences between original and reproduction <2% and <0.1). An essential component of transparent and reproducible research with healthcare databases is more complete reporting of study implementation. Once reproducibility is achieved, the conversation can be elevated to assess whether suboptimal design choices led to avoidable bias and whether findings are replicable in other data sources.

Active safety monitoring of newly marketed medications in a distributed data network: application of a semi-automated monitoring system.

We developed a semi-automated active monitoring system that uses sequential matched-cohort analyses to assess drug safety across a distributed network of longitudinal electronic health-care data. In a retrospective analysis, we show that the system would have identified cerivastatin-induced rhabdomyolysis. In this study, we evaluated whether the system would generate alerts for three drug-outcome pairs: rosuvastatin and rhabdomyolysis (known null association), rosuvastatin and diabetes mellitus, and telithromycin and hepatotoxicity (two examples for which alerting would be questionable). Over >5 years of monitoring, rate differences (RDs) in comparisons of rosuvastatin with atorvastatin were -0.1 cases of rhabdomyolysis per 1,000 person-years (95% confidence interval (CI): -0.4, 0.1) and -2.2 diabetes cases per 1,000 person-years (95% CI: -6.0, 1.6). The RD for hepatotoxicity comparing telithromycin with azithromycin was 0.3 cases per 1,000 person-years (95% CI: -0.5, 1.0). In a setting in which false positivity is a major concern, the system did not generate alerts for the three drug-outcome pairs.

Assessing the comparative effectiveness of newly marketed medications: methodological challenges and implications for drug development.

Comparative-effectiveness research (CER) aims to produce actionable evidence regarding the effectiveness and safety of medical products and interventions as they are used outside of controlled research settings. Although CER evidence regarding medications is particularly needed shortly after market approval, key methodological challenges include (i) potential bias due to channeling of patients to the newly marketed medication because of various patient-, physician-, and system-related factors; (ii) rapid changes in the characteristics of the user population during the early phase of marketing; and (iii) lack of timely data and the often small number of users in the first few months of marketing. We propose a mix of approaches to generate comparative-effectiveness data in the early marketing period, including sequential cohort monitoring with secondary health-care data and propensity score (PS) balancing, as well as extended follow-up of phase III and phase IV trials, indirect comparisons of placebo-controlled trials, and modeling and simulation of virtual trials.