Prior event rate ratio adjustment: numerical studies of a statistical method to address unrecognized confounding in observational studies.

PURPOSE: The purpose of this study was to evaluate a statistical method, prior event rate ratio (PERR) adjustment, and an alternative, PERR-ALT, both of which have the potential to overcome "unmeasured confounding," both analytically and via simulation. METHODS: Formulae were derived for the target estimates of both PERR methods, which were compared with results from simulations to ensure their validity. In addition to the theoretical insights gained, relative biases of both PERR methods for estimating exposure effects were also investigated via simulation studies and compared empirically with electronic medical record database study results. RESULTS: Theoretical derivations closely matched simulated results. In simulation studies, both PERR methods significantly reduce bias from unmeasured confounding compared with the standard Cox model. When there is no interaction between unmeasured confounders and time intervals, the estimate from PERR-ALT is unbiased, whereas the estimate from PERR has well-controlled relative bias. When interactions exist, relative biases tend to increase but not greatly, especially when the exposure effect is relatively large in comparison with the interaction effects. When the event rate is low and the sample size is limited, PERR is more computationally stable than PERR-ALT. In empiric study comparisons with randomized controlled trials, both PERR methods show potential to reduce bias from the standard Cox model similarly when unmeasured confounding is present. CONCLUSIONS: Extensive simulation studies and theoretical derivation show that PERR-based methods may reduce bias from unmeasured confounders when the exposure effect is relatively large in comparison with confounder-exposure interaction. The rare study event situation warrants further investigation.

Advantages of large medical record database for outcomes research: Insights into post-menopausal hormone therapy.

Approximately 25 years ago, our team initiated studies to determine whether outcome results from a large medical record database would yield valid results. We utilized the data in the United Kingdom (UK) General Practice Research Database (GPRD) to replicate the randomized controlled trial (RCT) study result and compared them to confirm the database results. The initial studies compared favorably, but some subsequent studies did not. This prompted development of a new strategy to determine and correct for unrecognized confounding in the database. This strategy divided outcome rates prior to initiation of therapy in the database study (which should include both identified and unidentified confounders) into the outcome rates during the treatment interval. When they differed from Cox-adjusted results, it reflected unrecognized confounding. We called this strategy Prior Event Rate Ratio (PERR)-adjusted outcome. One of our previously published observational studies replicated the Women's Health Initiative (WHI) RCT study of hormone therapy in post-menopausal women. Our study results replicated the WHI RCT results except it did not exhibit an increase in heart attack in contrast to the RCT. Furthermore, we could not evaluate death reliably since our analytic approach to overcome unrecognized confounding does not work for this outcome. In Volume 1, Issue 1 of the Learning Health Systems open access journal, we published a new study (titled "A new method to address unmeasured confounding of mortality in observational studies") that reported a novel death method, based on our prior methodology, that could analyze unrecognized confounding of the death outcome. This new methodology, termed Post Treatment Event Rate Ratio (PTERR), permitted a reliable examination of mortality in post-menopausal women undergoing hormone therapy. These results are reported in this manuscript. The study used the data from our previous observational study. It demonstrates that estrogen therapy markedly reduced death in post-menopausal women. This work also illuminates principles of database construction and correspondingly demonstrates the use of novel methodologies for obtaining valid results, which can be applied to enable learning from such databases. Work to advance such methodologies is essential to advancing the scientific integrity Core Value underpinning learning health systems (LHSs). Indeed, in the absence of such efforts to develop and refine methodologies for learning trustworthy lessons from real-world data, we risk inadvertently creating mis-learning systems.

Hormone therapy and coronary heart disease in young women.

OBJECTIVE: Because the Women's Health Initiative randomized controlled trial (WHI RCT) primarily studied older women, it is unresolved whether hormone therapy might prevent coronary heart disease in younger women. Given the similarity between our UK General Practice Research Database (GPRD) study of older women and the WHI RCT, the GPRD methodology was used to study a cohort of younger women. DESIGN: Women ages 50 to 55 years were investigated using data from the GPRD to simulate the WHI RCT in a manner similar to our initial study of older women. This study compared 30,102 unexposed and 20,654 exposed women treated with conjugated estrogens and norgestrel. RESULTS: Myocardial infarction was not altered significantly by hormone therapy (adjusted hazard ratio 0.91; 95% CI, 0.69-1.20). Stroke, venous thromboembolic events, breast cancer, and hip fracture were similar to both the GPRD study of older women and the WHI RCT. Although death was decreased in the total cohort, similar to older women, it was not altered significantly in a subset without missing covariate data. CONCLUSION: The similar results of the GPRD studies in younger and older women and between the GPRD and the WHI RCT suggest that hormone therapy does not prevent coronary heart disease in younger postmenopausal women. This study also demonstrates that investigation using a primary care electronic medical record database can expand the generalizability of findings from an RCT.

Replicated studies of two randomized trials of angiotensin-converting enzyme inhibitors: further empiric validation of the 'prior event rate ratio' to adjust for unmeasured confounding by indication.

PURPOSE: Using data from the United Kingdom General Practice Research Database (GPRD) randomized controlled trials (RCTs) were replicated to determine whether identifiable study characteristics and/or analytic methods influence the validity of observational studies. METHODS: This study reports GPRD replications of 2 RCTs that investigated whether angiotensin-converting enzyme inhibitors (ACEIs) improve cardiovascular outcomes in patients without congestive heart failure at high risk for cardiovascular disease (heart outcomes prevention evaluation (HOPE) and EUROPA). The GPRD studies replicated to the extent feasible all aspects of these RCTs except for randomization. RESULTS: With adjustment for confounders using conventional biostatistical techniques, both GPRD studies exhibited results divergent from the RCTs. Myocardial infarction, stroke, congestive heart failure, and coronary revascularization were increased in the Exposed group of both GPRD studies; whereas these outcomes either were decreased or unchanged by ACEI therapy in both RCTs. The results also were analyzed with a new method that appears to adjust for both identified and unmeasured confounders. With this methodology that employs the ratio of event rates between the Exposed and Unexposed cohorts prior to study start time to adjust the study hazard ratio (HR), the GPRD results for myocardial infarction, stroke, and coronary revascularization were largely similar to those found in the two RCTs. CONCLUSIONS: This study provides additional empiric evidence suggesting that this new analytic methodology, 'prior events rate ratio (PERR)' adjustment, is a promising solution for the vexing problem of 'unmeasured confounding' in observational studies. Additional statistical simulations are needed to fully appreciate the applicability and limitations of this method.

Replication of the Scandinavian Simvastatin Survival Study using a primary care medical record database prompted exploration of a new method to address unmeasured confounding.

PURPOSE: To examine whether identifiable study characteristics and/or analytic methods used determine observational study validity, as assessed by replicating randomized controlled trials using observational data. METHODS: A cohort from the United Kingdom General Practice Research Database (GPRD) was used to replicate the Scandinavian Simvastatin Survival Study RCT, which investigated statin treatment of hypercholesterolemic subjects with coronary heart disease. All aspects of the RCT except randomization were replicated to the extent possible in the GPRD study, which included 2,871 Unexposed and 1,280 statin-treated Exposed subjects. RESULTS: Overall mortality [adjusted hazard ratio 0.71 (0.53-0.96)] and myocardial infarction [adjusted HR 0.79 (0.61-1.02)] decreased in the GPRD study similar to the RCT. Coronary revascularization increased two-fold in the GPRD study, whereas it decreased significantly in the RCT [0.63 (0.54-0.74)]. This latter disparity prompted use of a new methodology to adjust for unmeasured confounding, which yielded an adjusted HR [1.0 (0.75-1.33)] more comparable to the RCT. CONCLUSIONS: This study provides additional evidence that a replicated GPRD observational study can yield results reasonably similar to a RCT. More important, it provides preliminary evidence suggesting that a new analytic methodology may adjust for unmeasured confounding, the major limitation to research using observational data.

Transforming the future of health together: The Learning Health Systems Consensus Action Plan.

The Learning Health Community is an emergent global multistakeholder grassroots incipient movement bonded together by a set of consensus Core Values Underlying a National-Scale Person-Centered Continuous Learning Health System developed at the 2012 Learning Health System (LHS) Summit. The Learning Health Community's Second LHS Summit was convened on December 8 to 9, 2016 building upon LHS efforts taking shape in order to achieve consensus on actions that, if taken, will advance LHSs and the LHS vision from what remain appealing concepts to a working reality for improving the health of individuals and populations globally. An iterative half-year collaborative revision process following the Second LHS Summit led to the development of the Learning Health Systems Consensus Action Plan.

A simulation using data from a primary care practice database closely replicated the women's health initiative trial.

OBJECTIVE: In contrast to prior observational studies, hormone replacement therapy (HRT) did not prevent coronary heart disease in the Women's Health Initiative Randomized Controlled Trial (WHI RCT). To assess the validity of a novel observational study design, we compared the WHI RCT with a simulation using data from the United Kingdom General Practice Research Database (GPRD). STUDY DESIGN AND SETTING: A cohort from GPRD was used to simulate the WHI RCT by replicating, to the extent possible, all aspects of the RCT except randomization. The study included 37,730 Unexposed and 13,658 Exposed women treated with estrogen and norgestrel. RESULTS: Myocardial infarction (adjusted hazard ratio 0.95 [0.78-1.16]) was not decreased significantly in the GPRD Exposed group. Similar to the WHI RCT, stroke, venous thromboembolic events, and breast cancer were increased; and colorectal cancer was decreased. Although death appeared to decrease in the total cohort, it was unaltered in a subset of subjects without missing data on baseline covariates. CONCLUSION: A structured comparison using data from GPRD was largely concordant with the WHI RCT and did not show a cardioprotective effect of HRT. These findings further generalize the results of WHI and reinforce the potential utility of this analytic approach.

Simulation of the Syst-Eur randomized control trial using a primary care electronic medical record was feasible.

OBJECTIVE: To determine the validity of outcome research using a primary care practice database by comparing a randomized controlled trial (RCT) with a simulated one. STUDY DESIGN AND SETTING: A cohort from the United Kingdom General Practice Research Database (GPRD) was used to simulate the Systolic Hypertension in Europe (Syst-Eur) trial, a study of antihypertensive therapy of isolated systolic hypertension, by replicating all aspects of that RCT (selection criteria, study time frame, treatment, and outcomes) except randomization. RESULTS: The exposed and unexposed groups in the GPRD study exhibited similar baseline characteristics. Stroke, the primary RCT outcome, decreased significantly in both the RCT (incidence rate ratio IRR = 0.58) and the GPRD study (IRR = 0.68). Myocardial infarction decreased nonsignificantly but similarly in both studies (RCT IRR = 0.70; GPRD IRR = 0.74). With the GPRD study extended for 3 years more, myocardial infarction decreased significantly, comparable to findings from the Systolic Hypertension in the Elderly Program (SHEP), another RCT similar to Syst-Eur. CONCLUSIONS: The findings support the potential value of clinical databases to investigate treatment effectiveness. RCT simulation using the GPRD may be the most feasible way to assess observational study validity in comparison to RCTs. The extended GPRD study shows the feasibility of using a simulated study to supplement the results of an RCT of limited duration.

Use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomised controlled trial findings.

OBJECTIVES: To determine whether observational studies that use an electronic medical record database can provide valid results of therapeutic effectiveness and to develop new methods to enhance validity. DESIGN: Data from the UK general practice research database (GPRD) were used to replicate previously performed randomised controlled trials, to the extent that was feasible aside from randomisation. Studies Six published randomised controlled trials. MAIN OUTCOME MEASURE: Cardiovascular outcomes analysed by hazard ratios calculated with standard biostatistical methods and a new analytical technique, prior event rate ratio (PERR) adjustment. RESULTS: In nine of 17 outcome comparisons, there were no significant differences between results of randomised controlled trials and database studies analysed using standard biostatistical methods or PERR analysis. In eight comparisons, Cox adjusted hazard ratios in the database differed significantly from the results of the randomised controlled trials, suggesting unmeasured confounding. In seven of these eight, PERR adjusted hazard ratios differed significantly from Cox adjusted hazard ratios, whereas in five they didn't differ significantly, and in three were more similar to the hazard ratio from the randomised controlled trial, yielding PERR results more similar to the randomised controlled trial than Cox (P<0.05). CONCLUSIONS: Although observational studies using databases are subject to unmeasured confounding, our new analytical technique (PERR), applied here to cardiovascular outcomes, worked well to identify and reduce the effects of such confounding. These results suggest that electronic medical record databases can be useful to investigate therapeutic effectiveness.

The afterlife for retiring deans and other senior medical administrators.

Career options for individuals leaving the administrative role as dean of a school of medicine or other senior administrative positions are considered. Options discussed include retirement and a variety of other positions both within schools of medicines and in other venues. Many opportunities exist for a challenging and fulfilling career path after leaving the role as a senior administrator in an academic medical center.

Clinical trials in silico: rigorous assessment of treatment effect using electronic health records.

With their ability to balance measured and unmeasured confounders, randomized controlled trials (RCTs) provide the highest level of evidence regarding the effect of treatment. Analyses of administrative data have had inconsistent success when compared with the results of RCTs. Much of the error can be attributed to the quality and completeness of the data sources as well as suboptimal study design and analytical approaches which differ significantly from the RCT. The comprehensive, longitudinal data within Electronic Health Records (EHRs) can overcome the data quality issues. This report describes a new analytical method for EHR data that successfully replicates RCT results.

Perspectives on hormone replacement therapy: the Women's Health Initiative and new observational studies sampling the overall population.

This commentary integrates the most recent information on coronary heart disease from the Women's Health Initiative (WHI) trial and our new large observational studies, which complement and expand the WHI results to the overall population. Breast and colon cancer findings are also considered.

Estrogen affects post-menopausal women differently than estrogen plus progestin replacement therapy.

BACKGROUND: In the Women's Health Initiative Randomized Controlled Trial (WHI RCT), estrogen-only treatment compared with combined estrogen-progestin treatment resulted in less coronary artery disease, no increase in breast cancer and no reduction in colorectal cancer. Since we previously reasonably replicated the combined estrogen-progestin WHI RCT using the UK General Practice Research Database (GPRD), estrogen-only treatment was investigated using a similar methodology. METHODS: This GPRD study simulated the estrogen-only WHI RCT of women who had undergone a hysterectomy except for randomization. The primary analysis examined 11 572 unexposed and 6890 Exposed women (aged 55-79) treated with conjugated equine estrogen and was compared with the combined estrogen-progestin GPRD study. RESULTS: At baseline, women with a hysterectomy exhibited more cardiovascular disease than those with an intact uterus. In the estrogen-only GPRD study, adjusted hazard ratios (HRs) were 0.50 (0.38-0.67) for myocardial infarction (MI), 1.13 (0.91-1.41) for breast cancer, and 1.18 (0.72-1.92) for colorectal cancer. Compared to the HRs in the estrogen-progestin GPRD study, the estrogen-only results are significantly lower for MI and breast cancer and higher for colon cancer, a pattern similar to the WHI RCT study comparisons. CONCLUSIONS: This study confirms that post-menopausal women in the overall population respond differently to estrogen-only treatment compared with estrogen-progestin treatment, due to different hormone regimens and/or increased cardiovascular disease in hysterectomized women.