Case weighted power priors for hybrid control analyses with time-to-event data.

We develop a method for hybrid analyses that uses external controls to augment internal control arms in randomized controlled trials (RCTs) where the degree of borrowing is determined based on similarity between RCT and external control patients to account for systematic differences (e.g., unmeasured confounders). The method represents a novel extension of the power prior where discounting weights are computed separately for each external control based on compatibility with the randomized control data. The discounting weights are determined using the predictive distribution for the external controls derived via the posterior distribution for time-to-event parameters estimated from the RCT. This method is applied using a proportional hazards regression model with piecewise constant baseline hazard. A simulation study and a real-data example are presented based on a completed trial in non-small cell lung cancer. It is shown that the case weighted power prior provides robust inference under various forms of incompatibility between the external controls and RCT population.

Adaptively leveraging external data with robust meta-analytical-predictive prior using empirical Bayes.

The robust meta-analytical-predictive (rMAP) prior is a popular method to robustly leverage external data. However, a mixture coefficient would need to be pre-specified based on the anticipated level of prior-data conflict. This can be very challenging at the study design stage. We propose a novel empirical Bayes robust MAP (EB-rMAP) prior to address this practical need and adaptively leverage external/historical data. Built on Box's prior predictive p-value, the EB-rMAP prior framework balances between model parsimony and flexibility through a tuning parameter. The proposed framework can be applied to binomial, normal, and time-to-event endpoints. Implementation of the EB-rMAP prior is also computationally efficient. Simulation results demonstrate that the EB-rMAP prior is robust in the presence of prior-data conflict while preserving statistical power. The proposed EB-rMAP prior is then applied to a clinical dataset that comprises 10 oncology clinical trials, including the prospective study.

Reverse-Bayes methods for evidence assessment and research synthesis.

It is now widely accepted that the standard inferential toolkit used by the scientific research community-null-hypothesis significance testing (NHST)-is not fit for purpose. Yet despite the threat posed to the scientific enterprise, there is no agreement concerning alternative approaches for evidence assessment. This lack of consensus reflects long-standing issues concerning Bayesian methods, the principal alternative to NHST. We report on recent work that builds on an approach to inference put forward over 70 years ago to address the well-known "Problem of Priors" in Bayesian analysis, by reversing the conventional prior-likelihood-posterior ("forward") use of Bayes' theorem. Such Reverse-Bayes analysis allows priors to be deduced from the likelihood by requiring that the posterior achieve a specified level of credibility. We summarise the technical underpinning of this approach, and show how it opens up new approaches to common inferential challenges, such as assessing the credibility of scientific findings, setting them in appropriate context, estimating the probability of successful replications, and extracting more insight from NHST while reducing the risk of misinterpretation. We argue that Reverse-Bayes methods have a key role to play in making Bayesian methods more accessible and attractive for evidence assessment and research synthesis. As a running example we consider a recently published meta-analysis from several randomised controlled trials (RCTs) investigating the association between corticosteroids and mortality in hospitalised patients with COVID-19.

A decision-theoretic approach to Bayesian clinical trial design and evaluation of robustness to prior-data conflict.

Bayesian clinical trials allow taking advantage of relevant external information through the elicitation of prior distributions, which influence Bayesian posterior parameter estimates and test decisions. However, incorporation of historical information can have harmful consequences on the trial's frequentist (conditional) operating characteristics in case of inconsistency between prior information and the newly collected data. A compromise between meaningful incorporation of historical information and strict control of frequentist error rates is therefore often sought. Our aim is thus to review and investigate the rationale and consequences of different approaches to relaxing strict frequentist control of error rates from a Bayesian decision-theoretic viewpoint. In particular, we define an integrated risk which incorporates losses arising from testing, estimation, and sampling. A weighted combination of the integrated risk addends arising from testing and estimation allows moving smoothly between these two targets. Furthermore, we explore different possible elicitations of the test error costs, leading to test decisions based either on posterior probabilities, or solely on Bayes factors. Sensitivity analyses are performed following the convention which makes a distinction between the prior of the data-generating process, and the analysis prior adopted to fit the data. Simulation in the case of normal and binomial outcomes and an application to a one-arm proof-of-concept trial, exemplify how such analysis can be conducted to explore sensitivity of the integrated risk, the operating characteristics, and the optimal sample size, to prior-data conflict. Robust analysis prior specifications, which gradually discount potentially conflicting prior information, are also included for comparison. Guidance with respect to cost elicitation, particularly in the context of a Phase II proof-of-concept trial, is provided.

Checking for model failure and for prior-data conflict with the constrained multinomial model.

Multinomial models can be difficult to use when constraints are placed on the probabilities. An exact model checking procedure for such models is developed based on a uniform prior on the full multinomial model. For inference, a nonuniform prior can be used and a consistency theorem is proved concerning a check for prior-data conflict with the chosen prior. Applications are presented and a new elicitation methodology is developed for multinomial models with ordered probabilities.

A Bayesian decision-theoretic approach to incorporate preclinical information into phase I oncology trials.

Leveraging preclinical animal data for a phase I oncology trial is appealing yet challenging. In this paper, we use animal data to improve decision-making in a model-based dose-escalation procedure. We make a proposal for how to measure and address a prior-data conflict in a sequential study with a small sample size. Animal data are incorporated via a robust two-component mixture prior for the parameters of the human dose-toxicity relationship. The weights placed on each component of the prior are chosen empirically and updated dynamically as the trial progresses and more data accrue. After completion of each cohort, we use a Bayesian decision-theoretic approach to evaluate the predictive utility of the animal data for the observed human toxicity outcomes, reflecting the degree of agreement between dose-toxicity relationships in animals and humans. The proposed methodology is illustrated through several data examples and an extensive simulation study.

Quantification of prior impact in terms of effective current sample size.

Bayesian methods allow borrowing of historical information through prior distributions. The concept of prior effective sample size (prior ESS) facilitates quantification and communication of such prior information by equating it to a sample size. Prior information can arise from historical observations; thus, the traditional approach identifies the ESS with such a historical sample size. However, this measure is independent of newly observed data, and thus would not capture an actual "loss of information" induced by the prior in case of prior-data conflict. We build on a recent work to relate prior impact to the number of (virtual) samples from the current data model and introduce the effective current sample size (ECSS) of a prior, tailored to the application in Bayesian clinical trial designs. Special emphasis is put on robust mixture, power, and commensurate priors. We apply the approach to an adaptive design in which the number of recruited patients is adjusted depending on the effective sample size at an interim analysis. We argue that the ECSS is the appropriate measure in this case, as the aim is to save current (as opposed to historical) patients from recruitment. Furthermore, the ECSS can help overcome lack of consensus in the ESS assessment of mixture priors and can, more broadly, provide further insights into the impact of priors. An R package accompanies the paper.

The assessment of intrinsic credibility and a new argument for p < 0.005.

The concept of intrinsic credibility has been recently introduced to check the credibility of 'out of the blue' findings without any prior support. A significant result is deemed intrinsically credible if it is in conflict with a sceptical prior derived from the very same data that would make the effect just non-significant. In this paper, I propose to use Bayesian prior-predictive tail probabilities to assess intrinsic credibility. For the standard 5% significance level, this leads to a new p-value threshold that is remarkably close to the recently proposed p < 0.005 standard. I also introduce the credibility ratio, the ratio of the upper to the lower limit (or vice versa) of a confidence interval for a significant effect size. I show that the credibility ratio has to be smaller than 5.8 such that a significant finding is also intrinsically credible. Finally, a p-value for intrinsic credibility is proposed that is a simple function of the ordinary p-value and has a direct frequentist interpretation in terms of the probability of replicating an effect. An application to data from the Open Science Collaboration study on the reproducibility of psychological science suggests that intrinsic credibility of the original experiment is better suited to predict the success of a replication experiment than standard significance.

How the Choice of Distance Measure Influences the Detection of Prior-Data Conflict.

The present paper contrasts two related criteria for the evaluation of prior-data conflict: the Data Agreement Criterion (DAC; Bousquet, 2008) and the criterion of Nott et al. (2016). One aspect that these criteria have in common is that they depend on a distance measure, of which dozens are available, but so far, only the Kullback-Leibler has been used. We describe and compare both criteria to determine whether a different choice of distance measure might impact the results. By means of a simulation study, we investigate how the choice of a specific distance measure influences the detection of prior-data conflict. The DAC seems more susceptible to the choice of distance measure, while the criterion of Nott et al. seems to lead to reasonably comparable conclusions of prior-data conflict, regardless of the distance measure choice. We conclude with some practical suggestions for the user of the DAC and the criterion of Nott et al.

Adaptive prior weighting in generalized regression.

The prior distribution is a key ingredient in Bayesian inference. Prior information on regression coefficients may come from different sources and may or may not be in conflict with the observed data. Various methods have been proposed to quantify a potential prior-data conflict, such as Box's p-value. However, there are no clear recommendations how to react to possible prior-data conflict in generalized regression models. To address this deficiency, we propose to adaptively weight a prespecified multivariate normal prior distribution on the regression coefficients. To this end, we relate empirical Bayes estimates of prior weight to Box's p-value and propose alternative fully Bayesian approaches. Prior weighting can be done for the joint prior distribution of the regression coefficients or-under prior independence-separately for prespecified blocks of regression coefficients. We outline how the proposed methodology can be implemented using integrated nested Laplace approximations (INLA) and illustrate the applicability with a Bayesian logistic regression model for data from a cross-sectional study. We also provide a simulation study that shows excellent performance of our approach in the case of prior misspecification in terms of root mean squared error and coverage. Supplementary Materials give details on software implementation and code and another application to binary longitudinal data from a randomized clinical trial using a Bayesian generalized linear mixed model.

Addressing prior-data conflict with empirical meta-analytic-predictive priors in clinical studies with historical information.

A common question in clinical studies is how to use historical data from earlier studies, leveraging relevant information into the design and analysis of a new study. Bayesian approaches are particularly well-suited to this task, with their natural ability to borrow strength across data sources. In this paper, we propose an eMAP approach for incorporating historical data into the analysis of clinical studies, and we discuss an application of this method to the analysis of observational safety studies for a class of products for patients with hemophilia A. The eMAP prior approach is flexible and robust to prior-data conflict. We conducted simulations to compare the frequentist operating characteristics of three approaches under different prior-data conflict assumptions and sample size scenarios.

Measuring statistical evidence using relative belief.

A fundamental concern of a theory of statistical inference is how one should measure statistical evidence. Certainly the words "statistical evidence," or perhaps just "evidence," are much used in statistical contexts. It is fair to say, however, that the precise characterization of this concept is somewhat elusive. Our goal here is to provide a definition of how to measure statistical evidence for any particular statistical problem. Since evidence is what causes beliefs to change, it is proposed to measure evidence by the amount beliefs change from a priori to a posteriori. As such, our definition involves prior beliefs and this raises issues of subjectivity versus objectivity in statistical analyses. This is dealt with through a principle requiring the falsifiability of any ingredients to a statistical analysis. These concerns lead to checking for prior-data conflict and measuring the a priori bias in a prior.

Addressing potential prior-data conflict when using informative priors in proof-of-concept studies.

Bayesian methods are increasingly used in proof-of-concept studies. An important benefit of these methods is the potential to use informative priors, thereby reducing sample size. This is particularly relevant for treatment arms where there is a substantial amount of historical information such as placebo and active comparators. One issue with using an informative prior is the possibility of a mismatch between the informative prior and the observed data, referred to as prior-data conflict. We focus on two methods for dealing with this: a testing approach and a mixture prior approach. The testing approach assesses prior-data conflict by comparing the observed data to the prior predictive distribution and resorting to a non-informative prior if prior-data conflict is declared. The mixture prior approach uses a prior with a precise and diffuse component. We assess these approaches for the normal case via simulation and show they have some attractive features as compared with the standard one-component informative prior. For example, when the discrepancy between the prior and the data is sufficiently marked, and intuitively, one feels less certain about the results, both the testing and mixture approaches typically yield wider posterior-credible intervals than when there is no discrepancy. In contrast, when there is no discrepancy, the results of these approaches are typically similar to the standard approach. Whilst for any specific study, the operating characteristics of any selected approach should be assessed and agreed at the design stage; we believe these two approaches are each worthy of consideration.

Bayesian inference in two-arm trials using relative belief ratios.

Various methodologies proposed for some inference problems associated with two-arm trails are known to suffer from difficulties, as documented in Senn (2001). We propose an alternative Bayesian approach to these problems that deals with these difficulties through providing an explicit measure of statistical evidence and the strength of this evidence. Bayesian methods are often criticized for their intrinsic subjectivity. We show how these concerns can be dealt with through assessing the bias induced by a prior model checking and checking for prior-data conflict.