A Bayesian hierarchical surrogate outcome model for multiple sclerosis.

The development of novel therapies in multiple sclerosis (MS) is one area where a range of surrogate outcomes are used in various stages of clinical research. While the aim of treatments in MS is to prevent disability, a clinical trial for evaluating a drugs effect on disability progression would require a large sample of patients with many years of follow-up. The early stage of MS is characterized by relapses. To reduce study size and duration, clinical relapses are accepted as primary endpoints in phase III trials. For phase II studies, the primary outcomes are typically lesion counts based on magnetic resonance imaging (MRI), as these are considerably more sensitive than clinical measures for detecting MS activity. Recently, Sormani and colleagues in 'Surrogate endpoints for EDSS worsening in multiple sclerosis' provided a systematic review and used weighted regression analyses to examine the role of either MRI lesions or relapses as trial level surrogate outcomes for disability. We build on this work by developing a Bayesian three-level model, accommodating the two surrogates and the disability endpoint, and properly taking into account that treatment effects are estimated with errors. Specifically, a combination of treatment effects based on MRI lesion count outcomes and clinical relapse was used to develop a study-level surrogate outcome model for the corresponding treatment effects based on disability progression. While the primary aim for developing this model was to support decision-making in drug development, the proposed model may also be considered for future validation. Copyright © 2016 John Wiley & Sons, Ltd.

Bayesian models for subgroup analysis in clinical trials.

BACKGROUND: In a pharmaceutical drug development setting, possible interactions between the treatment and particular baseline clinical or demographic factors are often of interest. However, the subgroup analysis required to investigate such associations remains controversial. Concerns with classical hypothesis testing approaches to the problem include low power, multiple testing, and the possibility of data dredging. PURPOSE: As an alternative to hypothesis testing, the use of shrinkage estimation techniques is investigated in the context of an exploratory post hoc subgroup analysis. A range of models that have been suggested in the literature are reviewed. Building on this, we explore a general modeling strategy, considering various options for shrinkage of effect estimates. This is applied to a case-study, in which evidence was available from seven-phase II-III clinical trials examining a novel therapy, and also to two artificial datasets with the same structure. METHODS: Emphasis is placed on hierarchical modeling techniques, adopted within a Bayesian framework using freely available software. A range of possible subgroup model structures are applied, each incorporating shrinkage estimation techniques. RESULTS: The investigation of the case-study showed little evidence of subgroup effects. Because inferences appeared to be consistent across a range of well-supported models, and model diagnostic checks showed no obvious problems, it seemed this conclusion was robust. It is reassuring that the structured shrinkage techniques appeared to work well in a situation where deeper inspection of the data suggested little evidence of subgroup effects. LIMITATIONS: The post hoc examination of subgroups should be seen as an exploratory analysis, used to help make better informed decisions regarding potential future studies examining specific subgroups. To a certain extent, the degree of understanding provided by such assessments will be limited by the quality and quantity of available data. CONCLUSIONS: In light of recent interest by health authorities into the use of subgroup analysis in the context of drug development, it appears that Bayesian approaches involving shrinkage techniques could play an important role in this area. Hopefully, the developments outlined here provide useful methodology for tackling such a problem, in-turn leading to better informed decisions regarding subgroups.

Use of the false discovery rate when comparing multiple health care providers.

OBJECTIVE: Comparisons of the performance of multiple health care providers are often based on hypothesis tests, those with resulting P-values below some critical threshold being identified as potentially extreme. Because of the multiple testing involved, the classical P-value threshold of, say, 0.05 may not be considered strict enough, as it will tend to lead to too many "false positives." However, we argue that the commonly used Bonferroni-corrected threshold is in general too strict for the problem in hand. The purpose of this article is to demonstrate a suitable alternative thresholding procedure that is already well established in other fields. STUDY DESIGN AND SETTING: The suggested procedure involves control of an error measure called the "false discovery rate" (FDR). We present a worked example involving a comparison of risk-adjusted mortality rates following heart surgery in New York State hospitals during 2000-2002. It is shown that the FDR critical threshold lines can be drawn on a "funnel plot," providing a simple graphical presentation of the results. RESULTS: The FDR procedure identified more providers as potentially extreme than the Bonferroni correction, while maintaining control of an intuitively sensible error measure. CONCLUSION: Control of the FDR offers a simple guideline to determining where to draw critical thresholds when comparing multiple health care providers.