A propensity score-integrated approach for leveraging external data in a randomized controlled trial with time-to-event endpoints.

In a randomized controlled trial with time-to-event endpoint, some commonly used statistical tests to test for various aspects of survival differences, such as survival probability at a fixed time point, survival function up to a specific time point, and restricted mean survival time, may not be directly applicable when external data are leveraged to augment an arm (or both arms) of an RCT. In this paper, we propose a propensity score-integrated approach to extend such tests when external data are leveraged. Simulation studies are conducted to evaluate the operating characteristics of three propensity score-integrated statistical tests, and an illustrative example is given to demonstrate how these proposed procedures can be implemented.

Accelerating drug development at Bristol Myers Squibb through innovation.

This paper focuses on the use of novel technologies and innovative trial designs to accelerate evidence generation and increase pharmaceutical Research and Development (R&D) productivity, at Bristol Myers Squibb. We summarize learnings with case examples, on how we prepared and continuously evolved to address the increasing cost, complexities, and external pressures in drug development, to bring innovative medicines to patients much faster. These learnings were based on review of internal efforts toward accelerating R&D focusing on four key areas: adopting innovative trial designs, optimizing trial designs, leveraging external control data, and implementing novel methods using artificial intelligence and machine learning.

Analysis of the b, p values, and the fractal dimension of aftershocks sequences following two major earthquakes in central Himalaya.

On April 25 of 2015, earthquake of 7.6 ML struck the central Himalayan region having epicenter at Barpak village in the Gorkha district of Nepal. The event was followed by 7.0 ML earthquake known as Dolakha earthquake, epicenter on the border of Dolakha and Sindhupalchowk. In this study, the b-value, aftershock decay rate p-value, and correlation fractal dimension were estimated for the aftershock sequences. The data used were obtained from the National Earthquake Monitoring and Research Centre (NEMRC), Nepal. The b-values 0.89 ± 0.05 and 0.90 ± 0.07 were computed for aftershocks after Gorkha and Dolakha event, respectively. A significant increase in b-values were reported following the Gorkha earthquake and Dolakha earthquake. The slip procured in the primary fault is respectively 60 % and 56 % after the events. The aftershocks sequences after both events were modelled by the Omori-Utsu law with p = 1.2 ± 0.11 and p = 0.76 ± 0.04. The observed b and p values after the earthquake sequences may correlate well with the large slip experienced by the seismogenic fault. This study sheds light on the mechanism behind the preparation of the significant earthquakes in the tectonic regions of the Himalaya.

Incorporating Prior Data in Quantitative Benefit-Risk Assessments: Case Study of a Bayesian Method.

BACKGROUND: Multiple criteria decision analysis (MCDA) and stochastic multi-criteria acceptability analysis (SMAA) in their current implementation cannot incorporate prior or external information on benefits and risks. We demonstrate how to incorporate prior data using a Bayesian mixture model approach while conducting quantitative benefit-risk assessments (qBRA) for medical products. METHODS: We implemented MCDA and SMAA in a Bayesian framework. To incorporate information from a prior study, we use mixture priors on each benefit and risk attribute that mixes information from a previous study with a vague prior distribution. The degree of borrowing is varied using a mixing proportion parameter. RESULTS: A demonstration case study for qBRA using the supplementary New Drug Application (sNDA) filing for Rivaroxaban for the indication of reduction in the risk of major thrombotic vascular events in patients with peripheral artery disease (PAD) was used to illustrate the method. Net utility scores, obtained from the randomized controlled trial data to support the sNDA, from the MCDA for Rivaraxoban and comparator were 0.48 and 0.56, respectively, with Rivaroxaban being the preferred alternative only 33% of the time. We show that with only 30% borrowing from a previous RCT, the MCDA and SMAA results are favorable for Rivaroxaban, accounting for the seemingly aberrant results on all-cause death in the trial data used to support the sNDA. CONCLUSION: Our method to formally incorporate prior data in MCDA and SMAA is easy to use and interpret. Software in the form of an RShiny App is available here: https://sai-dharmarajan.shinyapps.io/BayesianMCDA_SMAA/ .

Propensity score-incorporated adaptive design approaches when incorporating real-world data.

The propensity score-integrated composite likelihood (PSCL) method is one method that can be utilized to design and analyze an application when real-world data (RWD) are leveraged to augment a prospectively designed clinical study. In the PSCL, strata are formed based on propensity scores (PS) such that similar subjects in terms of the baseline covariates from both the current study and RWD sources are placed in the same stratum, and then composite likelihood method is applied to down-weight the information from the RWD. While PSCL was originally proposed for a fixed design, it can be extended to be applied under an adaptive design framework with the purpose to either potentially claim an early success or to re-estimate the sample size. In this paper, a general strategy is proposed due to the feature of PSCL. For the possibility of claiming early success, Fisher's combination test is utilized. When the purpose is to re-estimate the sample size, the proposed procedure is based on the test proposed by Cui, Hung, and Wang. The implementation of these two procedures is demonstrated via an example.

Adaptive designs for best treatment identification with top-two Thompson sampling and acceleration.

We consider outcome adaptive phase II or phase II/III trials to identify the best treatment for further development. Different from many other multi-arm multi-stage designs, we borrow approaches for the best arm identification in multi-armed bandit (MAB) approaches developed for machine learning and adapt them for clinical trial purposes. The best arm identification in MAB focuses on the error rate of identification at the end of the trial, but we are also interested in the cumulative benefit of trial patients, for example, the frequency of patients treated with the best treatment. In particular, we consider Top-Two Thompson Sampling (TTTS) and propose an acceleration approach for better performance in drug development scenarios in which the sample size is much smaller than that considered in machine learning applications. We also propose a variant of TTTS (TTTS2) which is simpler, easier for implementation, and has comparable performance in small sample settings. An extensive simulation study was conducted to evaluate the performance of the proposed approach in multiple typical scenarios in drug development.

Spatial mapping of b-value and fractal dimension prior to November 8, 2022 Doti Earthquake, Nepal.

An earthquake of magnitude 5.6 mb (6.6 ML) hit western Nepal (Doti region) in the wee hours of wednesday morning local time (2:12 AM, 2022.11.08) killing at least six people. Gutenberg-Richter b-value of earthquake distribution and correlation fractal dimension (D2) are estimated for 493 earthquakes with magnitude of completeness 3.6 prior to this earthquake. We consider earthquakes in western Nepal Himalaya and adjoining region (80.0-83.5°E and 27.3-30.5°N) for the period of 1964 to 2022 for the analysis. The b-value 0.68±0.03 implies a high stress zone and the spatial correlation dimension 1.81±0.02 implies a highly heterogeneous region where the epicenters are spatially distributed. Low b-values and high D2 values identify the study region as a high hazard zone. Focal mechanism styles and low b-values correlate with thrust nature of earthquakes and show that the earthquake's occurrence is associated with the dynamics of the faults responsible for generating the past earthquakes.

Principled leveraging of external data in the evaluation of diagnostic devices via the propensity score-integrated composite likelihood approach.

In the area of diagnostics, it is common practice to leverage external data to augment a traditional study of diagnostic accuracy consisting of prospectively enrolled subjects to potentially reduce the time and/or cost needed for the performance evaluation of an investigational diagnostic device. However, the statistical methods currently being used for such leveraging may not clearly separate study design and outcome data analysis, and they may not adequately address possible bias due to differences in clinically relevant characteristics between the subjects constituting the traditional study and those constituting the external data. This paper is intended to draw attention in the field of diagnostics to the recently developed propensity score-integrated composite likelihood approach, which originally focused on therapeutic medical products. This approach applies the outcome-free principle to separate study design and outcome data analysis and can mitigate bias due to imbalance in covariates, thereby increasing the interpretability of study results. While this approach was conceived as a statistical tool for the design and analysis of clinical studies for therapeutic medical products, here, we will show how it can also be applied to the evaluation of sensitivity and specificity of an investigational diagnostic device leveraging external data. We consider two common scenarios for the design of a traditional diagnostic device study consisting of prospectively enrolled subjects, which is to be augmented by external data. The reader will be taken through the process of implementing this approach step-by-step following the outcome-free principle that preserves study integrity.

A more powerful test for three-arm non-inferiority via risk difference: Frequentist and Bayesian approaches.

Necessity for finding improved intervention in many legacy therapeutic areas are of high priority. This has the potential to decrease the expense of medical care and poor outcomes for many patients. Typically, clinical efficacy is the primary evaluating criteria to measure any beneficial effect of a treatment. Albeit, there could be situations when several other factors (e.g. side-effects, cost-burden, less debilitating, less intensive, etc.) which can permit some slightly less efficacious treatment options favorable to a subgroup of patients. This often leads to non-inferiority (NI) testing. NI trials may or may not include a placebo arm due to ethical reasons. However, when included, the resulting three-arm trial is more prudent since it requires less stringent assumptions compared to a two-arm placebo-free trial. In this article, we consider both Frequentist and Bayesian procedures for testing NI in the three-arm trial with binary outcomes when the functional of interest is risk difference. An improved Frequentist approach is proposed first, which is then followed by a Bayesian counterpart. Bayesian methods have a natural advantage in many active-control trials, including NI trial, as it can seamlessly integrate substantial prior information. In addition, we discuss sample size calculation and draw an interesting connection between the two paradigms.

Emulating Randomized Controlled Trials with Hybrid Control Arms in Oncology: A Case Study.

This proof-of-concept study retrospectively assessed the feasibility of applying a hybrid control arm design to a completed phase III randomized controlled trial (RCT; CheckMate-057) in advanced non-small cell lung cancer using a real-world data (RWD) source. The emulated trial consists of an experimental arm (patients from the RCT experimental cohort) and a hybrid control arm (patients from the RCT and RWD control cohorts). For the RWD control cohort, this study used a nationwide electronic health record-derived de-identified database. Three frequentist statistical borrowing methods were evaluated: a two-step Cox model, a fixed Cox model, and propensity score-integrated composite likelihood ("Methods 1-3"). The experimental treatment effect for hybrid control designs were evaluated using hazard ratios (HRs) with 95% confidence interval (CI) estimated from the Cox models accounting for covariate differences. The reduction in study duration compared to the RCT was also evaluated. All three statistical borrowing methods achieved comparable experimental treatment effects to that observed in the CheckMate-057 clinical trial, with HRs of 0.73 (95% CI: 0.59, 0.92), 0.74 (95% CI: 0.61, 0.91), 0.72 (95% CI: 0.59, 0.88) for Methods 1-3, respectively. Reduction in study duration time was 99-115 days when borrowing 30-38 events for Methods 1-3, respectively. This study demonstrated that it is feasible to emulate an RCT using a hybrid control arm design using three frequentist propensity-score based statistical borrowing methods. Selection of an appropriate, fit-for-use RWD cohort is critical to minimizing bias in experimental treatment effect.

Propensity score-integrated approach to survival analysis: leveraging external evidence in single-arm studies.

External data, referred to as data external to the traditional clinical study being planned, include but are not limited to real-world data (RWD) and data collected from clinical studies being conducted in the past or in other countries. The external data are sometimes leveraged to augment a single-arm, prospectively designed study when appropriate. In such an application, recently developed propensity score-integrated approaches including PSPP and PSCL can be used for study design and data analysis when the clinical outcomes are binary or continuous. In this paper, the propensity score-integrated Kaplan-Meier (PSKM) method is proposed for a similar situation but the outcome of interest is time-to-event. The propensity score methodology is used to select external subjects that are similar to those in the current study in terms of baseline covariates and to stratify the selected subjects from both data sources into more homogeneous strata. The stratum-specific PSKM estimators are obtained based on all subjects in the stratum with the external data being down-weighted, and then these estimators are combined to obtain an overall PSKM estimator. A simulation study is conducted to assess the performance of the PSKM method, and an illustrative example is presented to demonstrate how to implement the proposed method.

Estimands in observational studies: Some considerations beyond ICH E9 (R1).

The document ICH E9 (R1) has brought much attention to the concept of estimand in the clinical trials community. ICH stands for International Conference for Harmonization. In this article, we draw attention to one facet of estimand that is not discussed in that document but is crucial in the context of observational studies, namely weighting for covariate balance. How weighting schemes are connected to estimand, or more specifically to one of its five attributes identified in ICH E9 (R1), the attribute of population, is illustrated using the Rubin Causal Model. Three estimands are examined from both theoretical and practical perspectives. Factors that may be considered in choosing among these estimands are discussed.

Conditional borrowing external data to establish a hybrid control arm in randomized clinical trials.

Utilizing external data from the real world, including data from historical clinical trials, has received increasing interest in drug development. The use of external data to support drug evaluation in clinical trials has mainly been through using various matching methods for baseline characteristics to form external control arms in single-arm trials or to augment control arms of randomized controlled trials in hybrid approaches. However, matching the baseline characteristics between the trial and the external subjects can only guarantee comparability on the level of baseline characteristics. Differences in outcomes between the two data sources may still exist due to contemporaneous and operational characteristics. Similarity between the outcomes in the trial control and the external subjects with similar baseline characteristics can be critical in leveraging the external subjects in the clinical trials. In this paper, a resampling method for augmenting control arms in randomized controlled trials is proposed under the conditional borrowing framework. The new method establishes empirical distributions for the hazard ratio in outcomes between the external and trial control subjects. The borrowing decision is then derived from this empirical distribution using a measure of similarity. Once the borrowing decision is established, the borrowing weights for the external subjects, based on the similarity measure, are incorporated in the weighted partial likelihood to evaluate the treatment effect. The operating characteristics of the hybrid control arm, under both the conditional borrowing and unconditional borrowing frameworks, are evaluated. Simulation is conducted to evaluate Type I error, bias, and power. An illustrative example using simulated data is also presented.

Zero-Inflated Binomial Model for Meta-Analysis and Safety-Signal Detection.

BACKGROUND: Meta-analysis of related trials can provide an overall measure of safety-signal accounting for variability across studies. In addition to an overall measure, researchers may often be interested in study-specific measures to assess safety of the product. Likelihood ratio tests (LRT) methods serve this purpose by identifying studies that appear to show a safety concern. In this paper, we present a Bayesian approach. Despite having good statistical properties, the LRT methods may not be suitable for the meta-analysis of randomized controlled trials (RCTs) when there are several studies with zero events in at least one arm. METHODS: In this article, we describe a Bayesian framework using a Zero-inflated binomial model with spike-and-slab parameterization for the treatment effects. In addition to providing an overall meta-analytic estimate, this method provides posterior probability of a safety-signal for each study. RESULTS: We illustrate the approach using two published data sets comprising several randomized controlled trials (RCTs) each and compare the model performance for different choices of priors for treatment effect. DISCUSSION: The proposed Bayesian methodological framework is useful to identify potential signal for single adverse event and to determine overall meta-analytic estimate of the magnitude of the signal. Practitioners may consider this approach as an alternative to the frequentist's LRT approach discussed in Jung et al. (J Biopharm Stat 31:47-54, 2020) when there are zero events in either the treatment arm or the control arm. In the future, this approach can be further extended to accommodate multiple adverse events.

A novel approach to augment single-arm clinical studies with real-world data.

In this paper, we develop a methodology for leveraging real-world data into single-arm clinical trial studies. In recent years, the idea of augmenting randomized clinical trials data with real-world data has emerged as a particularly attractive technique for health organizations and drug developers to accelerate the drug development process. Major regulatory authorities such as the Food and Drug Administration and European Medicines Agency have recognized the potential of utilizing real-world data and are advancing toward making regulatory decisions based on real-world evidence. Several statistical methods have been developed in recent years for borrowing data from real-world sources such as electronic health records, product and disease registries, as well as claims and billing data. We propose a novel approach to augment single-arm clinical trials with the real-world data derived from single or multiple data sources. Furthermore, we illustrate the proposed method in the presence of missing data and conduct simulation studies to evaluate its performance in diverse settings.

New approaches for testing non-inferiority for three-arm trials with Poisson distributed outcomes.

With the availability of limited resources, innovation for improved statistical method for the design and analysis of randomized controlled trials (RCTs) is of paramount importance for newer and better treatment discovery for any therapeutic area. Although clinical efficacy is almost always the primary evaluating criteria to measure any beneficial effect of a treatment, there are several important other factors (e.g., side effects, cost burden, less debilitating, less intensive, etc.), which can permit some less efficacious treatment options favorable to a subgroup of patients. This leads to non-inferiority (NI) testing. The objective of NI trial is to show that an experimental treatment is not worse than an active reference treatment by more than a pre-specified margin. Traditional NI trials do not include a placebo arm for ethical reason; however, this necessitates stringent and often unverifiable assumptions. On the other hand, three-arm NI trials consisting of placebo, reference, and experimental treatment, can simultaneously test the superiority of the reference over placebo and NI of experimental treatment over the reference. In this article, we proposed both novel Frequentist and Bayesian procedures for testing NI in the three-arm trial with Poisson distributed count outcome. RCTs with count data as the primary outcome are quite common in various disease areas such as lesion count in cancer trials, relapses in multiple sclerosis, dermatology, neurology, cardiovascular research, adverse event count, etc. We first propose an improved Frequentist approach, which is then followed by it's Bayesian version. Bayesian methods have natural advantage in any active-control trials, including NI trial when substantial historical information is available for placebo and established reference treatment. In addition, we discuss sample size calculation and draw an interesting connection between the two paradigms.

Leverage multiple real-world data sources in single-arm medical device clinical studies.

The interest in utilizing real-world data (RWD) has been considerably increasing in medical product development and evaluation. With proper usage and analysis of high-quality real-world data, real-world evidence (RWE) can be generated to inform regulatory and healthcare decision-making. This paper proposes a study design and data analysis approach for a prospective, single-arm clinical study that is supplemented with patients from multiple real-world data sources containing patient-level covariate and outcome data. After the amount of information to be borrowed from each real-world data source is determined, the propensity score-integrated composite likelihood method is applied to obtain an estimate of the parameter of interest based on data from the prospective clinical study and this real-world data source. This method is applied to each real-world data source. The final estimate of the parameter of interest is then obtained by taking a weighted average of all these estimates. The performance of the proposed approach is evaluated via a simulation study. A hypothetical example is presented to illustrate how to implement the proposed approach.

Ensuring exchangeability in data-based priors for a Bayesian analysis of clinical trials.

In many orphan diseases and pediatric indications, the randomized controlled trials may be infeasible because of their size, duration, and cost. Leveraging information on the control through a prior can potentially reduce sample size. However, unless an objective prior is used to impose complete ignorance for the parameter being estimated, it results in biased estimates and inflated type-I error. Hence, it is essential to assess both the confirmatory and supplementary knowledge available during the construction of the prior to avoid "cherry-picking" advantageous information. For this purpose, propensity score methods are employed to minimize selection bias by weighting supplemental control subjects according to their similarity in terms of pretreatment characteristics to the subjects in the current trial. The latter can be operationalized through a proposed measure of overlap in propensity-score distributions. In this paper, we consider single experimental arm in the current trial and the control arm is completely borrowed from the supplemental data. The simulation experiments show that the proposed method reduces prior and data conflict and improves the precision of the of the average treatment effect.

Augmenting Both Arms of a Randomized Controlled Trial Using External Data: An Application of the Propensity Score-Integrated Approaches.

Leveraging external data is a topic that have recently received much attention. The propensity score-integrated approaches are a methodological innovation for this purpose. In this paper we adapt these approaches, originally introduced to augment single-arm studies with external data, for the augmentation of both arms of a randomized controlled trial (RCT) with external data. After recapitulating the basic ideas, we provide a step-by-step tutorial of how to implement the propensity score-integrated approaches, from study design to outcome analysis, in the RCT setting in such a way that the study integrity and objectively are maintained. Both the Bayesian (power prior) approach and the frequentist (composite likelihood) approach are included. Some extensions and variations of these approaches are also outlined at the end of this paper.