Going beyond probability of success: Opportunities for statisticians to influence quantitative decision-making at the portfolio level.

The pharmaceutical industry is plagued with long, costly development and high risk. Therefore, a company's effective management and optimisation of a portfolio of projects is critical for success. Project metrics such as the probability of success enable modelling of a company's pipeline accounting for the high uncertainty inherent within the industry. Making portfolio decisions inherently involves managing risk, and statisticians are ideally positioned to champion not only the derivation of metrics for individual projects, but also advocate decision-making at a broader portfolio level. This article aims to examine the existing different portfolio decision-making approaches and to suggest opportunities for statisticians to add value in terms of introducing probabilistic thinking, quantitative decision-making, and increasingly advanced methodologies.

Futility Interim Analysis Based on Probability of Success Using a Surrogate Endpoint.

In clinical trials with time-to-event data, the evaluation of treatment efficacy can be a long and complex process, especially when considering long-term primary endpoints. Using surrogate endpoints to correlate the primary endpoint has become a common practice to accelerate decision-making. Moreover, the ethical need to minimize sample size and the practical need to optimize available resources have encouraged the scientific community to develop methodologies that leverage historical data. Relying on the general theory of group sequential design and using a Bayesian framework, the methodology described in this paper exploits a documented historical relationship between a clinical "final" endpoint and a surrogate endpoint to build an informative prior for the primary endpoint, using surrogate data from an early interim analysis of the clinical trial. The predictive probability of success of the trial is then used to define a futility-stopping rule. The methodology demonstrates substantial enhancements in trial operating characteristics when there is a good agreement between current and historical data. Furthermore, incorporating a robust approach that combines the surrogate prior with a vague component mitigates the impact of the minor prior-data conflicts while maintaining acceptable performance even in the presence of significant prior-data conflicts. The proposed methodology was applied to design a Phase III clinical trial in metastatic colorectal cancer, with overall survival as the primary endpoint and progression-free survival as the surrogate endpoint.

Incorporation of healthy volunteers data on receptor occupancy into a phase II proof-of-concept trial using a Bayesian dynamic borrowing design.

Receptor occupancy in targeted tissues measures the proportion of receptors occupied by a drug at equilibrium and is sometimes used as a surrogate of drug efficacy to inform dose selection in clinical trials. We propose to incorporate data on receptor occupancy from a phase I study in healthy volunteers into a phase II proof-of-concept study in patients, with the objective of using all the available evidence to make informed decisions. A minimal physiologically based pharmacokinetic modeling is used to model receptor occupancy in healthy volunteers and to predict it in the patients of a phase II proof-of-concept study, taking into account the variability of the population parameters and the specific differences arising from the pathological condition compared to healthy volunteers. Then, given an estimated relationship between receptor occupancy and the clinical endpoint, an informative prior distribution is derived for the clinical endpoint in both the treatment and control arms of the phase II study. These distributions are incorporated into a Bayesian dynamic borrowing design to supplement concurrent phase II trial data. A simulation study in immuno-inflammation demonstrates that the proposed design increases the power of the study while maintaining a type I error at acceptable levels for realistic values of the clinical endpoint.

A comparison of estimation methods adjusting for selection bias in adaptive enrichment designs with time-to-event endpoints.

Adaptive enrichment designs in clinical trials have been developed to enhance drug developments. They permit, at interim analyses during the trial, to select the sub-populations that benefits the most from the treatment. Because of this selection, the naive maximum likelihood estimation of the treatment effect, commonly used in classical randomized controlled trials, is biased. In the literature, several methods have been proposed to obtain a better estimation of the treatments' effects in such contexts. To date, most of the works have focused on normally distributed endpoints, and some estimators have been proposed for time-to-event endpoints but they have not all been compared side-by-side. In this work, we conduct an extensive simulation study, inspired by a real case-study in heart failure, to compare the maximum-likelihood estimator (MLE) with an unbiased estimator, shrinkage estimators, and bias-adjusted estimators for the estimation of the treatment effect with time-to-event data. The performances of the estimators are evaluated in terms of bias, variance, and mean squared error. Based on the results, along with the MLE, we recommend to provide the unbiased estimator and the single-iteration bias-adjusted estimator: the former completely eradicates the selection bias, but is highly variable with respect to a naive estimator; the latter is less biased than the MLE estimator and only slightly more variable.

A Bayesian approach for event predictions in clinical trials with time-to-event outcomes.

In clinical trials with time-to-event outcome as the primary endpoint, the end of study date is often based on the number of observed events, which drives the statistical power and the sample size calculation. It is of great value for study sponsors to have a good understanding of the recruitment process and the event milestones to manage the logistical tasks, which require a considerable amount of resources. The objective of the proposed statistical approach is to predict, as accurately as possible, the timing of an analysis planned once a target number of events is collected. The method takes into account the enrollment, the time to event, and the time to censor processes, using Weibull models in a Bayesian framework. We also consider a possible delay in the event reporting by the investigators, and covariates may also be included. Several metrics can be obtained, such as the probability of study completion at specific timepoints or the credible interval of the date of study completion. The approach was applied to oncology trials, with progression-free survival as primary outcome. A retrospective analysis shows the accuracy of the approach on these examples, as well as the benefit of updating the predictive probability of study completion as data are accumulating or new information becomes available. We also evaluated the performances of the proposed method in a comprehensive simulation study.

Comparison between functional and intravascular imaging approaches guiding percutaneous coronary intervention: A network meta-analysis of randomized and propensity matching studies.

BACKGROUND: The optimal approach to guide percutaneous coronary intervention (PCI) has yet to be defined. The aim of this study was to compare functional driven (fractional flow reserve) versus intravascular imaging (intravascular ultrasound, IVUS, and/or optical coherence tomography, OCT) versus standard (coronary angiography only, CA)-guided PCI. METHODS: Randomized controlled trials (RCTs) and propensity score weight-matched studies (PSWMs) comparing FFR versus IVUS versus OCT versus CA-guided PCI were included. Major adverse cardiovascular event (MACE; a composite end point of death or myocardial infarction [MI] or revascularization) was the primary endpoint, whereas definite stent thrombosis (ST) and single components of MACE were the secondary ones. Primary analyses were performed including only RCTs, secondary also with PSWMs. RESULTS: Thirty-three studies were included in the analysis, 16 RCTs and 17 PSWMs. After 2 (1-3) years, IVUS performed better for MACE than CA (odds ratio [OR] 0.75 0.52-0.88), whereas there was just a trend for FFR (OR 0.81, 0.64-1.02). These results were mainly driven by reduced risk of all cause death, MI (FFR OR 0.74:0.57-0.99 and IVUS OR 0.82:0.54-0.94) and revascularization. IVUS reduced ST while FFR did not, and at meta-regression analysis, there was a trend for superiority of IVUS versus FFR to reduce subsequent MI in acute coronary syndrome (ACS) patients. The present results were consistent also after adding studies with PSWMs. CONCLUSIONS: Functional and intravascular imaging approaches seem to perform similarly in term of clinical outcomes, while both performed better compared with the standard approach. Imaging showed a potential benefit for ACS patients. The present results stress the need for a wider use of functional or imaging driven PCI.

Predictive probability of success using surrogate endpoints.

The predictive probability of success of a future clinical trial is a key quantitative tool for decision-making in drug development. It is derived from prior knowledge and available evidence, and the latter typically comes from the accumulated data on the clinical endpoint of interest in previous clinical trials. However, a surrogate endpoint could be used as primary endpoint in early development and, usually, no or limited data are collected on the clinical endpoint of interest. We propose a general, reliable, and broadly applicable methodology to predict the success of a future trial from surrogate endpoints, in a way that makes the best use of all the available evidence. The predictions are based on an informative prior, called surrogate prior, derived from the results of past trials on one or several surrogate endpoints. If available, in a Bayesian framework, this prior could be combined with data from past trials on the clinical endpoint of interest. Two methods are proposed to address a potential discordance between the surrogate prior and the data on the clinical endpoint. We investigate the patterns of behavior of the predictions in a comprehensive simulation study, and we present an application to the development of a drug in Multiple Sclerosis. The proposed methodology is expected to support decision-making in many different situations, since the use of predictive markers is important to accelerate drug developments and to select promising drug candidates, better and earlier.

Impact of design of coronary stents and length of dual antiplatelet therapies on ischaemic and bleeding events: a network meta-analysis of 64 randomized controlled trials and 102 735 patients.

AIMS: The differential impact on ischaemic and bleeding events of the type of drug-eluting stent [durable polymer stents [DES] vs. biodegradable polymer stents vs. bioresorbable scaffolds (BRS)] and length of dual antiplatelet therapy (DAPT) remains to be defined. METHODS AND RESULTS: Randomized controlled trials comparing different types of DES and/or DAPT durations were selected. The primary endpoint was Major Adverse Cardiovascular Events (MACE) [a composite of death, myocardial infarction (MI), and target vessel revascularization]. Definite stent thrombosis (ST) and single components of MACE were secondary endpoints. The arms of interest were: BRS with 12 months of DAPT (12mDAPT), biodegradable polymer stent with 12mDAPT, durable polymer stent [everolimus-eluting (EES), zotarolimus-eluting (ZES)] with 12mDAPT, EES/ZES with <12 months of DAPT, and EES/ZES with >12 months of DAPT (DAPT > 12 m). Sixty-four studies with 150 arms and 102 735 patients were included. After a median follow-up of 20 months, MACE rates were similar in the different arms of interest. EES/ZES with DAPT > 12 m reported a lower incidence of MI than the other groups, while BRS showed a higher rate of ST when compared to EES/ZES, irrespective of DAPT length. A higher risk of major bleedings was observed for DAPT > 12 m as compared to shorter DAPT. CONCLUSION: Durable and biodegradable polymer stents along with BRS report a similar rate of MACE irrespective of DAPT length. Fewer MI are observed with EES/ZES with DAPT > 12 m, while a higher rate of ST is reported for BRS when compared to EES/ZES, independently from DAPT length. Stent type may partially affect the outcome together with DAPT length.

Decision-making in drug development using a composite definition of success.

Evidence-based quantitative methodologies have been proposed to inform decision-making in drug development, such as metrics to make go/no-go decisions or predictions of success, identified with statistical significance of future clinical trials. While these methodologies appropriately address some critical questions on the potential of a drug, they either consider the past evidence without predicting the outcome of the future trials or focus only on efficacy, failing to account for the multifaceted aspects of a successful drug development. As quantitative benefit-risk assessments could enhance decision-making, we propose a more comprehensive approach using a composite definition of success based not only on the statistical significance of the treatment effect on the primary endpoint but also on its clinical relevance and on a favorable benefit-risk balance in the next pivotal studies. For one drug, we can thus study several development strategies before starting the pivotal trials by comparing their predictive probability of success. The predictions are based on the available evidence from the previous trials, to which new hypotheses on the future development could be added. The resulting predictive probability of composite success provides a useful summary to support the discussions of the decision-makers. We present a fictive, but realistic, example in major depressive disorder inspired by a real decision-making case.

A simple way to unify multicriteria decision analysis (MCDA) and stochastic multicriteria acceptability analysis (SMAA) using a Dirichlet distribution in benefit-risk assessment.

Quantitative methodologies have been proposed to support decision making in drug development and monitoring. In particular, multicriteria decision analysis (MCDA) and stochastic multicriteria acceptability analysis (SMAA) are useful tools to assess the benefit-risk ratio of medicines according to the performances of the treatments on several criteria, accounting for the preferences of the decision makers regarding the relative importance of these criteria. However, even in its probabilistic form, MCDA requires the exact elicitations of the weights of the criteria by the decision makers, which may be difficult to achieve in practice. SMAA allows for more flexibility and can be used with unknown or partially known preferences, but it is less popular due to its increased complexity and the high degree of uncertainty in its results. In this paper, we propose a simple model as a generalization of MCDA and SMAA, by applying a Dirichlet distribution to the weights of the criteria and by making its parameters vary. This unique model permits to fit both MCDA and SMAA, and allows for a more extended exploration of the benefit-risk assessment of treatments. The precision of its results depends on the precision parameter of the Dirichlet distribution, which could be naturally interpreted as the strength of confidence of the decision makers in their elicitation of preferences.

A comparison of model-free phase I dose escalation designs for dual-agent combination therapies.

It is increasingly common for therapies in oncology to be given in combination. In some cases, patients can benefit from the interaction between two drugs, although often at the risk of higher toxicity. A large number of designs to conduct phase I trials in this setting are available, where the objective is to select the maximum tolerated dose combination. Recently, a number of model-free (also called model-assisted) designs have provoked interest, providing several practical advantages over the more conventional approaches of rule-based or model-based designs. In this paper, we demonstrate a novel calibration procedure for model-free designs to determine their most desirable parameters. Under the calibration procedure, we compare the behaviour of model-free designs to model-based designs in a comprehensive simulation study, covering a number of clinically plausible scenarios. It is found that model-free designs are competitive with the model-based designs in terms of the proportion of correct selections of the maximum tolerated dose combination. However, there are a number of scenarios in which model-free designs offer a safer alternative. This is also illustrated in the application of the designs to a case study using data from a phase I oncology trial.

A comparison of various aggregation functions in multi-criteria decision analysis for drug benefit-risk assessment.

Multi-criteria decision analysis is a quantitative approach to the drug benefit-risk assessment which allows for consistent comparisons by summarising all benefits and risks in a single score. The multi-criteria decision analysis consists of several components, one of which is the utility (or loss) score function that defines how benefits and risks are aggregated into a single quantity. While a linear utility score is one of the most widely used approach in benefit-risk assessment, it is recognised that it can result in counter-intuitive decisions, for example, recommending a treatment with extremely low benefits or high risks. To overcome this problem, alternative approaches to the scores construction, namely, product, multi-linear and Scale Loss Score models, were suggested. However, to date, the majority of arguments concerning the differences implied by these models are heuristic. In this work, we consider four models to calculate the aggregated utility/loss scores and compared their performance in an extensive simulation study over many different scenarios, and in a case study. It is found that the product and Scale Loss Score models provide more intuitive treatment recommendation decisions in the majority of scenarios compared to the linear and multi-linear models, and are more robust to the correlation in the criteria.

A novel measure of drug benefit-risk assessment based on Scale Loss Score.

Quantitative methods have been proposed to assess and compare the benefit-risk balance of treatments. Among them, multicriteria decision analysis (MCDA) is a popular decision tool as it permits to summarise the benefits and the risks of a drug in a single utility score, accounting for the preferences of the decision-makers. However, the utility score is often derived using a linear model which might lead to counter-intuitive conclusions; for example, drugs with no benefit or extreme risk could be recommended. Moreover, it assumes that the relative importance of benefits against risks is constant for all levels of benefit or risk, which might not hold for all drugs. We propose Scale Loss Score (SLoS) as a new tool for the benefit-risk assessment, which offers the same advantages as the linear multicriteria decision analysis utility score but has, in addition, desirable properties permitting to avoid recommendations of non-effective or extremely unsafe treatments, and to tolerate larger increases in risk for a given increase in benefit when the amount of benefit is small than when it is high. We present an application to a real case study on telithromycin in Community Acquired Pneumonia and Acute Bacterial Sinusitis, and we investigated the patterns of behaviour of Scale Loss Score, as compared to the linear multicriteria decision analysis, in a comprehensive simulation study.

Radial Artery Versus Right Internal Thoracic Artery Versus Saphenous Vein as the Second Conduit for Coronary Artery Bypass Surgery: A Network Meta-Analysis of Clinical Outcomes.

Background There remains uncertainty regarding the second-best conduit after the internal thoracic artery in coronary artery bypass grafting. Few studies directly compared the clinical results of the radial artery ( RA ), right internal thoracic artery ( RITA ), and saphenous vein ( SV ). No network meta-analysis has compared these 3 strategies. Methods and Results MEDLINE and EMBASE were searched for adjusted observational studies and randomized controlled trials comparing the RA , SV , and/or RITA as the second conduit for coronary artery bypass grafting. The primary end point was all-cause long-term mortality. Secondary end points were operative mortality, perioperative stroke, perioperative myocardial infarction, and deep sternal wound infection ( DSWI ). Pairwise and network meta-analyses were performed. A total of 149 902 patients (4 randomized, 31 observational studies) were included ( RA , 16 201, SV , 112 018, RITA, 21 683). At NMA , the use of SV was associated with higher long-term mortality compared with the RA (incidence rate ratio, 1.23; 95% CI , 1.12-1.34) and RITA (incidence rate ratio, 1.26; 95% CI , 1.17-1.35). The risk of DSWI for SV was similar to RA but lower than RITA (odds ratio, 0.71; 95% CI , 0.55-0.91). There were no differences for any outcome between RITA and RA , although DSWI trended higher with RITA (odds ratio, 1.39; 95% CI , 0.92-2.1). The risk of DSWI in bilateral internal thoracic artery studies was higher when the skeletonization technique was not used. Conclusions The use of the RA or the RITA is associated with a similar and statistically significant long-term clinical benefit compared with the SV . There are no differences in operative risk or complications between the 2 arterial conduits, but DSWI remains a concern with bilateral ITA when skeletonization is not used.

Pharmacological Treatment of Arterial Hypertension in Children and Adolescents: A Network Meta-Analysis.

Pharmacological treatment is indicated in children and adolescents with hypertension unresponsive to lifestyle modifications, but there is not enough evidence to recommend 1 class of antihypertensive drugs over others. We performed a network meta-analysis to compare the results of available randomized clinical trials on pharmacological treatment of pediatric hypertension. From a total of 554 potentially relevant studies, 13 randomized placebo-controlled clinical trials enrolling ≥50 patients and a follow-up ≥4 weeks were included. The reduction of systolic blood pressure (SBP) and diastolic BP (DBP) after treatment were the coprimary end points. A total of 2378 pediatric patients, with a median age of 12 years, were included in the analysis. After a median follow-up of 35 days, lisinopril and enalapril were found to be superior to placebo in reducing SBP and DBP, whereas only for DBP, losartan was found to be superior to placebo and lisinopril and enalapril were found to be superior to eplerenone. Angiotensin-converting enzyme inhibitors and angiotensin receptor blockers were associated with a greater SBP and DBP reduction compared with placebo, likewise the mineralocorticoid receptor antagonist was inferior to angiotensin-converting enzyme inhibitors in DBP reduction. The analysis was adjusted for study-level mean age, percentage of women, mean baseline blood pressure, and mean weight, only the latter significantly affected DBP reduction. According to the present analysis, angiotensin-converting enzyme inhibitors and angiotensin receptor blockers could represent the best choice as antihypertensive treatment for pediatric hypertension. However, because of the paucity of available data for the other classes of antihypertensive drugs, definitive conclusions are not allowed and further randomized controlled trials are warranted.

Network meta-analysis of studies comparing closure devices for femoral access after percutaneous coronary intervention.

INTRODUCTION: Different devices have been released for closure of femoral vascular access after coronary angiography or percutaneous coronary intervention, whereas evidence about their efficacy and safety when compared with manual compression or head to head is lacking, especially across different diameters of sheaths, age and sex. RESULTS: A total of 30 studies were included in the analysis. Manual compression was evaluated as the control group in all of the included studies (5620 patients), Angioseal in 15 studies (17-29) (1812 patients), Exoseal in two studies (30-31) (1773 patients), Perclose in six (29, 32-37) (849 patients), Vasoseal in eight (36, 38-43) (699 patients), DUETT in one study (44) (392 patients), StarClose in two studies (23, 45) (334 patients), Techstar in two studies (37, 46) (252 patients) and extravascular staple in one study (47) (242 patients). At network meta-analysis, all the devices resulted as not superior to manual compression to reduce all vascular complications, and these results did not vary at metaregression for age, sex and diameter of sheaths. Manual compression significantly increased time to hemostasis when compared with Femoseal (5.72; 1.91-19.10), Vasoseal (5.11; 2.32-11.33), Perclose (3.46; 1.70-7.06), Angioseal (14.95; 7.84-28.57) and Techstar (9.78; 1.81-53.65), while was similar to StarClose, DUETT and Exoseal. CONCLUSION: Different vascular devices for closure of femoral access did not results superior to manual compression to reduce complications, whereas offered a shorted time to hemostasis. StarClose was the device with the highest probability to perform best in terms of complication, whereas Angioseal was superior in terms of reduction of time to hemostasis.

Meta-Analysis Comparing Complete or Culprit Only Revascularization in Patients With Multivessel Disease Presenting With Cardiogenic Shock.

The optimal strategy for patients with an acute myocardial infarction (MI) and multivessel (MV) coronary artery disease complicated by cardiogenic shock (CS) remains unknown. We conducted a meta-analysis of all randomized controlled trials and observational studies that reported adjusted effect measures to evaluate the association of MV-PCI (percutaneous coronary intervention), compared with culprit only (C)-PCI, with cardiovascular events in patients admitted for CS and MV disease. We identified 12 studies (n = 1 randomized controlled trials, n = 11 observational) that included 7,417 patients (n = 1,809 treated with MV-PCI and n = 5,608 with C-PCI). When compared with C-PCI, MV-PCI was not associated with an increased risk of short-term death (odds ratio [OR] 1.14, 95% confidence interval [CI] 0.87 to 1.48, p = 0.35 and adjusted OR [ORadj] 1.00, 95% CI 0.70 to 1.43, p = 1.00). In-hospital and/or short-term mortality tended to be higher with MV-PCI, when compared with C-PCI, for CS patients needing dialysis (ß 0.12, 95% CI from 0.049 to 0.198; p= 0.001), whereas MV-PCI was associated with lower in-hospital and/or short-term mortality in patients with an anterior MI (ß -0.022, 95% CI -0.03 to -0.01; p <0.001). MV-PCI strategy was associated with a more frequent need for dialysis or contrast-induced nephropathy after revascularization (OR 1.36, 95% CI 1.06 to 1.75, p = 0.02). In conclusion, MV-PCI seems not to increase risk of death during short- or long-term follow-up when compared with C-PCI in patients admitted for MV coronary artery disease and MI complicated by CS. Furthermore, it appears a more favorable strategy in patients with anterior MI, whereas the increased risk for AKI and its negative prognostic impact should be considered in decision-making process. Further studies are needed to confirm our hypothesis on in these subpopulations of CS patients.

Computed Tomography and Adrenal Venous Sampling in the Diagnosis of Unilateral Primary Aldosteronism.

Unilateral primary aldosteronism is the most common surgically correctable form of endocrine hypertension and is usually differentiated from bilateral forms by adrenal venous sampling (AVS) or computed tomography (CT). Our objective was to compare clinical and biochemical postsurgical outcomes of patients with unilateral primary aldosteronism diagnosed by CT or AVS and identify predictors of surgical outcomes. Patient data were obtained from 18 internationally distributed centers and retrospectively analyzed for clinical and biochemical outcomes of adrenalectomy of patients with surgical management based on CT (n=235 patients, diagnosed from 1994-2016) or AVS (526 patients, diagnosed from 1994-2015) using the standardized PASO (Primary Aldosteronism Surgical Outcome) criteria. Biochemical outcomes were highly different according to surgical management approach with a smaller proportion in the CT group achieving complete biochemical success (188 of 235 [80%] patients versus 491 of 526 [93%], P<0.001) and a greater proportion with absent biochemical success (29 of 235 [12%] versus 10 of 526 [2%], P<0.001). A diagnosis by CT was associated with a decreased likelihood of complete biochemical success compared with AVS (odds ratio, 0.28; 0.16-0.50; P<0.001). Clinical outcomes were not significantly different, but the absence of a postsurgical elevated aldosterone-to-renin ratio was a strong marker of complete clinical success (odds ratio, 14.81; 1.76-124.53; P=0.013) in the CT but not in the AVS group. In conclusion, patients diagnosed by CT have a decreased likelihood of achieving complete biochemical success compared with a diagnosis by AVS.