Accelerating drug development at Bristol Myers Squibb through innovation.

This paper focuses on the use of novel technologies and innovative trial designs to accelerate evidence generation and increase pharmaceutical Research and Development (R&D) productivity, at Bristol Myers Squibb. We summarize learnings with case examples, on how we prepared and continuously evolved to address the increasing cost, complexities, and external pressures in drug development, to bring innovative medicines to patients much faster. These learnings were based on review of internal efforts toward accelerating R&D focusing on four key areas: adopting innovative trial designs, optimizing trial designs, leveraging external control data, and implementing novel methods using artificial intelligence and machine learning.

Bayesian approaches to benefit-risk assessment for diagnostic tests.

Benefit-risk assessment plays an important role in the evaluation of medical devices. Unlike the therapeutic devices, the diagnostic tests usually affect patient life indirectly since subsequent therapeutic treatment interventions (such as proper treatment in time, further examination or test, no action, etc.) will depend on correct diagnosis and monitoring of the disease status. A benefit-risk score using statistical models by integrating the information from benefit (true positive and true negative) and risk (false positive and false negative) for diagnostic tests with binary outcomes (i.e., positive and negative) will help evaluation of the utility and the uncertainty of a particular diagnostic device. In this paper, we develop two types of Bayesian models with conjugate priors for constructing the benefit-risk (BR) measures with corresponding credible intervals, one based on a Multinomial model with Dirichlet prior, and the other based on independent Binomial models with independent Beta priors. We then propose a Bayesian power prior model to incorporate the historical data or the real-world data (RWD). Both the fixed and random power prior parameters are considered for Bayesian borrowing. We evaluate the performance of the methods by simulations and illustrate their implementation using a real example.

Multi-Criteria Decision Analysis Model for Assessing the Risk from Multi-Ingredient Dietary Supplements (MIDS).

Military personnel use dietary supplements (DS) for performance enhancement, bodybuilding, weight loss, and to maintain health. Adverse events, including cardiovascular (CV) effects, have been reported in military personnel taking supplements. Previous research determined that ingestion of multi-ingredient dietary supplements (MIDS), can lead to signals of safety concerns. Therefore, to assess the safety of MIDS, the Department of Defense via a contractor explored the development of a model-based risk assessment tool. We present a strategy and preliminary novel multi-criteria decision analysis (MCDA)-based tool for assessing the risk of adverse CV effects from MIDS. The tool integrates toxicology and other relevant data available on MIDS; likelihood of exposure, and biologic plausibility that could contribute to specific aspects of risk.Inputs for the model are values of four measures assigned based on the available evidence supplemented with the opinion of experts in toxicology, modeling, risk assessment etc. Measures were weighted based on the experts' assessment of measures' relative importance. Finally, all data for the four measures were integrated to provide a risk potential of 0 (low risk) to 100 (high risk) that defines the relative risk of a MIDS to cause adverse reactions.We conclude that the best available evidence must be supplemented with the opinion of experts in medicine, toxicology and pharmacology. Model-based approaches are useful to inform risk assessment in the absence of data. This MCDA model provides a foundation for refinement and validation of accuracy of the model predictions as new evidence becomes available.

A Bayesian approach to benefit-risk assessment in clinical studies with longitudinal data.

Chuang-Stein et al. proposed a method for benefit-risk assessment by formulating a five-category multinomial random variable with the first four categories as a combination of benefit and risk, and the fifth category to include subjects who withdraw from study. In this article, we subdivide the single withdrawal category into four sub-categories to consider withdrawal for different reasons. To analyze eight-category data, we propose a two-level multivariate-Dirichlet Model to identify benefit-risk measures at the population level. For individual benefit-risk, we use a log-odds ratio model with Dirichlet process prior. Two methods are applied to a hypothetical clinical trial data for illustration.

Homogeneity assessment for pivotal medical device clinical studies.

A question that routinely arises in medical device clinical studies is the homogeneity across demographic subgroups, geographical regions, or investigational sites of the enrolled patients in terms of treatment effects or outcome variables. The main objective of this paper is to discuss statistical concepts and methods for the assessment of such homogeneity and to provide the practitioner a statistical framework and points to consider in conducting homogeneity assessment. Demographic subgroups, geographical regions, and investigational sites are discussed separately as each has its unique issues. Specific considerations are also given to randomized controlled trials, non-randomized comparative studies, and single-arm studies. We point out that judicious use of statistical methods, in conjunction with sound clinical judgment, is essential in handling the issue of homogeneity of treatment effect in medical device clinical studies.

Benefit-risk assessment for binary diagnostic tests.

In diagnostic device evaluation, it is important to have an integrated benefit-risk (BR) assessment for safety and effectiveness, which is not same as the assessment for drugs and therapeutic devices. Correct diagnosis does not lead to direct clinical outcome such as longer survival, release of symptoms, tumor shrinkage, etc.; but leads to the proper treatment in time while incorrect diagnosis may result in serious consequences of unnecessary tests and wrong treatments. Some common measures used in evaluating the accuracy of a diagnostic device include sensitivity, specificity, positive predictive value and negative predictive value. Here, we propose a BR measure by incorporating information about true-positive and true-negative cases (correct diagnosis) and false-positive and false-negative cases (incorrect diagnosis) for facilitating the necessary decision-making. Three decision rules are discussed depending on the purpose of the clinical study. Different statistical models are developed for estimating the BR measure for data obtained from different sampling schemes (cross-sectional and case-control sampling). The construction of confidence intervals (CIs) for the proposed BR measure is based on (i) the asymptotic normality of the maximum likelihood estimators (MLEs), and (ii) parametric bootstrap re-sampling technique. The performance of these CIs is evaluated by intensive Monte-Carlo simulations which reveal that both CIs perform reasonably well. Finally, the proposed methodology is applied to two clinical trial datasets.

Approaches for testing noninferiority in two-arm trials for risk ratio and odds ratio.

For an existing established drug regimen, active control trials are defacto standard due to ethical reason as well as for clinical equipoise. However, when superiority claim of a new drug against the active control is unlikely to be successful, researchers often address the issue in terms of noninferiority (NI), provided the experimental drug demonstrates the evidence of other benefits beyond efficacy. Such trials aim to demonstrate that an experimental treatment is non-inferior to an existing comparator by not more than a pre-specified margin. The issue of choosing such a margin is complex. In this article, two-arm NI trials with binary outcomes are considered when margin is defined in terms of relative risk or odds ratio. A Frequentist test based on proposed NI margin is developed first. Since two-arm NI trials without placebo arm are dependent upon historical information, in order to make accurate and meaningful interpretation of their results, a Bayesian approach is developed next. Bayesian approach is flexible to incorporate the available information from the historical trial. The operating characteristics of the proposed methods are studied in terms of power and sample size for varying design factors. A clinical trial data is reanalyzed to study the properties of the proposed approach.

Bayesian hierarchical methods for meta-analysis combining randomized-controlled and single-arm studies.

Meta-analysis of interventions usually relies on randomized controlled trials. However, when the dominant source of information comes from single-arm studies, or when the results from randomized controlled trials lack generalization due to strict inclusion and exclusion criteria, it is vital to synthesize both sources of evidence. One challenge of synthesizing both sources is that single-arm studies are usually less reliable than randomized controlled trials due to selection bias and confounding factors. In this paper, we propose a Bayesian hierarchical framework for the purpose of bias reduction and efficiency gain. Under this framework, three methods are proposed: bivariate generalized linear mixed effects models, hierarchical power prior model and hierarchical commensurate prior model. Design difference and potential biases are considered in all models, within which the hierarchical power prior and hierarchical commensurate prior models further offer to downweight single-arm studies flexibly. The hierarchical commensurate prior model is recommended as the primary method for evidence synthesis because of its accuracy and robustness. We illustrate our methods by applying all models to two motivating datasets and evaluate their performance through simulation studies. We finish with a discussion of the advantages and limitations of our methods, as well as directions for future research in this area.

Bayesian approach for assessing non-inferiority in a three-arm trial with pre-specified margin.

Non-inferiority trials are becoming increasingly popular for comparative effectiveness research. However, inclusion of the placebo arm, whenever possible, gives rise to a three-arm trial which has lesser burdensome assumptions than a standard two-arm non-inferiority trial. Most of the past developments in a three-arm trial consider defining a pre-specified fraction of unknown effect size of reference drug, that is, without directly specifying a fixed non-inferiority margin. However, in some recent developments, a more direct approach is being considered with pre-specified fixed margin albeit in the frequentist setup. Bayesian paradigm provides a natural path to integrate historical and current trials' information via sequential learning. In this paper, we propose a Bayesian approach for simultaneous testing of non-inferiority and assay sensitivity in a three-arm trial with normal responses. For the experimental arm, in absence of historical information, non-informative priors are assumed under two situations, namely when (i) variance is known and (ii) variance is unknown. A Bayesian decision criteria is derived and compared with the frequentist method using simulation studies. Finally, several published clinical trial examples are reanalyzed to demonstrate the benefit of the proposed procedure.

Meta-analysis using Dirichlet process.

This article develops a Bayesian approach for meta-analysis using the Dirichlet process. The key aspect of the Dirichlet process in meta-analysis is the ability to assess evidence of statistical heterogeneity or variation in the underlying effects across study while relaxing the distributional assumptions. We assume that the study effects are generated from a Dirichlet process. Under a Dirichlet process model, the study effects parameters have support on a discrete space and enable borrowing of information across studies while facilitating clustering among studies. We illustrate the proposed method by applying it to a dataset on the Program for International Student Assessment on 30 countries. Results from the data analysis, simulation studies, and the log pseudo-marginal likelihood model selection procedure indicate that the Dirichlet process model performs better than conventional alternative methods.

Bayesian approach for clinical trial safety data using an Ising prior.

In drug safety, development of statistical methods for multiplicity adjustments has exploited potential relationships among adverse events (AEs) according to underlying medical features. Due to the coarseness of the biological features used to group AEs together, which serves as the basis for the adjustment, it is possible that a single adverse event can be simultaneously described by multiple biological features. However, existing methods are limited in that they are not structurally flexible enough to accurately exploit this multi-dimensional characteristic of an adverse event. In order to preserve the complex dependencies present in clinical safety data, a Bayesian approach for modeling the risk differentials of the AEs between the treatment and comparator arms is proposed which provides a more appropriate clinical description of the drug's safety profile. The proposed procedure uses an Ising prior to unite medically related AEs. The proposed method and an existing Bayesian method are applied to a clinical dataset, and the signals from the two methods are presented. Results from a small simulation study are also presented.

Assessment of groundwater quality: a fusion of geochemical and geophysical information via Bayesian neural networks.

Deplorable quality of groundwater arising from saltwater intrusion, natural leaching and anthropogenic activities is one of the major concerns for the society. Assessment of groundwater quality is, therefore, a primary objective of scientific research. Here, we propose an artificial neural network-based method set in a Bayesian neural network (BNN) framework and employ it to assess groundwater quality. The approach is based on analyzing 36 water samples and inverting up to 85 Schlumberger vertical electrical sounding data. We constructed a priori model by suitably parameterizing geochemical and geophysical data collected from the western part of India. The posterior model (post-inversion) was estimated using the BNN learning procedure and global hybrid Monte Carlo/Markov Chain Monte Carlo optimization scheme. By suitable parameterization of geochemical and geophysical parameters, we simulated 1,500 training samples, out of which 50 % samples were used for training and remaining 50 % were used for validation and testing. We show that the trained model is able to classify validation and test samples with 85 % and 80 % accuracy respectively. Based on cross-correlation analysis and Gibb's diagram of geochemical attributes, the groundwater qualities of the study area were classified into following three categories: "Very good", "Good", and "Unsuitable". The BNN model-based results suggest that groundwater quality falls mostly in the range of "Good" to "Very good" except for some places near the Arabian Sea. The new modeling results powered by uncertainty and statistical analyses would provide useful constrain, which could be utilized in monitoring and assessment of the groundwater quality.

A generalized p-value approach for assessing noninferiority in a three-arm trial.

The existing generalized p-value approach, from statistical literature, is applied to assess noninferiority of an experimental treatment in a three-arm clinical trial including a placebo. Two generalized test functions (GTFs) are constructed and Monte Carlo simulations are used to compute the p-value. The GTFs perform well in terms of maintaining the Type-I error probabilities, and the power of the tests are shown to increase to 1 as both the sample size and the parameter denoting the fraction of the effect of the reference drug with respect to placebo increase. The generalized confidence intervals are shown to retain the coverage probabilities. A published dataset is re-analysed using the proposed test and the results are in agreement with earlier findings.

Bayesian approach to noninferiority trials for proportions.

Noninferiority trials are unique because they are dependent upon historical information in order to make meaningful interpretation of their results. Hence, a direct application of the Bayesian paradigm in sequential learning becomes apparently useful in the analysis. This paper describes a Bayesian procedure for testing noninferiority in two-arm studies with a binary primary endpoint that allows the incorporation of historical data on an active control via the use of informative priors. In particular, the posteriors of the response in historical trials are assumed as priors for its corresponding parameters in the current trial, where that treatment serves as the active control. The Bayesian procedure includes a fully Bayesian method and two normal approximation methods on the prior and/or on the posterior distributions. Then a common Bayesian decision criterion is used but with two prespecified cutoff levels, one for the approximation methods and the other for the fully Bayesian method, to determine whether the experimental treatment is noninferior to the active control. This criterion is evaluated and compared with the frequentist method using simulation studies in keeping with regulatory framework that new methods must protect type I error and arrive at a similar conclusion with existing standard strategies. Results show that both methods arrive at comparable conclusions of noninferiority when applied to a modified real data set. The advantage of the proposed Bayesian approach lies in its ability to provide posterior probabilities for effect sizes of the experimental treatment over the active control.

Joint modeling of longitudinal data and informative dropout time in the presence of multiple changepoints.

In longitudinal studies of patients with the human immunodeficiency virus (HIV), objectives of interest often include modeling of individual-level trajectories of HIV ribonucleic acid (RNA) as a function of time. Such models can be used to predict the effects of different treatment regimens or to classify subjects into subgroups with similar trajectories. Empirical evidence, however, suggests that individual trajectories often possess multiple points of rapid change, which may vary from subject to subject. Additionally, some individuals may end up dropping out of the study and the tendency to drop out may be related to the level of the biomarker. Modeling of individual viral RNA profiles is challenging in the presence of these changes, and currently available methods do not address all the issues such as multiple changes, informative dropout, clustering, etc. in a single model. In this article, we propose a new joint model, where a multiple-changepoint model is proposed for the longitudinal viral RNA response and a proportional hazards model for the time of dropout process. Dirichlet process (DP) priors are used to model the distribution of the individual random effects and error distribution. In addition to robustifying the model against possible misspecifications, the DP leads to a natural clustering of subjects with similar trajectories which can be of importance in itself. Sharing of information among subjects with similar trajectories also results in improved parameter estimation. A fully Bayesian approach for model fitting and prediction is implemented using MCMC procedures on the ACTG 398 clinical trial data. The proposed model is seen to give rise to improved estimates of individual trajectories when compared with a parametric approach.

A modified version of Moran's I.

BACKGROUND: Investigation of global clustering patterns across regions is very important in spatial data analysis. Moran's I is a widely used spatial statistic for detecting global spatial patterns such as an east-west trend or an unusually large cluster. Here, we intend to improve Moran's I for evaluating global clustering patterns by including the weight function in the variance, introducing a population density (PD) weight function in the statistics, and conducting Monte Carlo simulation for testing. We compare our modified Moran's I with Oden's I*pop for simulated data with homogeneous populations. The proposed method is applied to a census tract data set. METHODS: We present a modified version of Moran's I which includes information about the strength of the neighboring association when estimating the variance for the statistic. We provide a power analysis on Moran's I, a modified version of Moran's I, and I*pop in a simulation study. Data were simulated under two common spatial correlation scenarios of local and global clustering. RESULTS: For simulated data with a large cluster pattern, the modified Moran's I has the highest power (43.4%) compared to Moran's I (39.9%) and I*pop (12.4%) when the adjacent weight function is used with 5%, 10%, 15%, 20%, or 30% of the total population as the geographic range for the cluster.For two global clustering patterns, the modified Moran's I (power > 25.3%) performed better than both Moran's I (> 24.6%) and I*pop (> 7.9%) with the adjacent weight function. With the population density weight function, all methods performed equally well.In the real data example, all statistics indicate the existence of a global clustering pattern in a leukemia data set. The modified Moran's I has the lowest p-value (.0014) followed by Moran's I (.0156) and I*pop (.011). CONCLUSIONS: Our power analysis and simulation study show that the modified Moran's I achieved higher power than Moran's I and I*pop for evaluating global and local clustering patterns on geographic data with homogeneous populations. The inclusion of the PD weight function which in turn redefines the neighbors seems to have a large impact on the power of detecting global clustering patterns. Our methods to improve the original version of Moran's I for homogeneous populations can also be extended to some alternative versions of Moran's I methods developed for heterogeneous populations.

Efficient interval estimation for age-adjusted cancer rates.

The age-adjusted cancer rates are defined as the weighted average of the age-specific cancer rates, where the weights are positive, known, and normalized so that their sum is 1. Fay and Feuer developed a confidence interval for a single age-adjusted rate based on the gamma approximation. Fay used the gamma approximations to construct an F interval for the ratio of two age-adjusted rates. Modifications of the gamma and F intervals are proposed and a simulation study is carried out to show that these modified gamma and modified F intervals are more efficient than the gamma and F intervals, respectively, in the sense that the proposed intervals have empirical coverage probabilities less than or equal to their counterparts, and that they also retain the nominal level. The normal and beta confidence intervals for a single age-adjusted rate are also provided, but they are shown to be slightly liberal. Finally, for comparing two correlated age-adjusted rates, the confidence intervals for the difference and for the ratio of the two age-adjusted rates are derived incorporating the correlation between the two rates. The proposed gamma and F intervals and the normal intervals for the correlated age-adjusted rates are recommended to be implemented in the Surveillance, Epidemiology and End Results Program of the National Cancer Institute.