The inclusion of real world evidence in clinical development planning.

BACKGROUND: When designing studies it is common to search the literature to investigate variability estimates to use in sample size calculations. Proprietary data of previously designed trials in a particular indication are also used to obtain estimates of variability. Estimates of treatment effects are typically obtained from randomised controlled clinical trials (RCTs). Based on the observed estimates of treatment effect, variability and the minimum clinical relevant difference to detect, the sample size for a subsequent trial is estimated. However, data from real world evidence (RWE) studies, such as observational studies and other interventional studies in patients in routine clinical practice, are not widely used in a systematic manner when designing studies. In this paper, we propose a framework for inclusion of RWE in planning of a clinical development programme. METHODS: In our proposed approach, all evidence, from both RCTs and RWE (i.e. from studies in routine clinical practice), available at the time of designing of a new clinical trial is combined in a Bayesian network meta-analysis (NMA). The results can be used to inform the design of the next clinical trial in the programme. The NMA was performed at key milestones, such as at the end of the phase II trial and prior to the design of key phase III studies. To illustrate the methods, we designed an alternative clinical development programme in multiple sclerosis using RWE through clinical trial simulations. RESULTS: Inclusion of RWE in the NMA and the resulting trial simulations demonstrated that 284 patients per arm were needed to achieve 90% power to detect effects of predetermined size in the TRANSFORMS study. For the FREEDOMS and FREEDOMS II clinical trials, 189 patients per arm were required. Overall there was a reduction in sample size of at least 40% across the three phase III studies, which translated to a time savings of at least 6 months for the undertaking of the fingolimod phase III programme. CONCLUSION: The use of RWE resulted in a reduced sample size of the pivotal phase III studies, which led to substantial time savings compared to the approach of sample size calculations without RWE.

Searching for indirect evidence and extending the network of studies for network meta-analysis: case study in venous thromboembolic events prevention following elective total knee replacement surgery.

OBJECTIVE: To evaluate the effect of study identification methods and network size on the relative effectiveness and cost-effectiveness of recommended pharmacological venous thromboembolic events (VTEs) prophylaxis for adult patients undergoing elective total knee replacement surgery in the United Kingdom. METHODS: A stepwise literature search specifically designed to identify indirect evidence was conducted to extend the original clinical review from the latest National Institute for Health and Care Excellence (NICE) VTE technology appraisal. Different network sizes or network orders, based on the successive searches, informed three network meta-analyses (NMAs), which were compared with a replicated base case. The resulting comparative estimates were inputted in an economic model to investigate the effect of network size on cost-effectiveness probabilities. RESULTS: Searches increased the number of indirect comparisons between VTE interventions, progressively widening the relevant network of studies for NMA. Precision around mean relative treatment effects was increased as the network was extended from the base case to first-order NMA, but further extensions had limited effect. Cost-effectiveness analysis results were largely insensitive to variation in clinical inputs from the different NMA orders. CONCLUSIONS: No standard methodology is currently recommended by NICE to identify the most relevant network of studies for NMA. Our study showed that optimizing the identification of studies for NMA can extend the evidence base for analysis and reduce the uncertainty in relative effectiveness estimates. Although in our example network extensions did not affect the acceptability of available treatments in VTE prevention based on cost-effectiveness results, it may in other applications.

Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons.

OBJECTIVES: To establish current guidance and practice in UK on presentation of indirect comparison and mixed treatment comparison analyses; to provide recommendations to improve indirect comparison/mixed treatment comparison reporting and to identify research priorities for improved presentation. METHODS: Existing institutional guidance for conducting indirect comparison/mixed treatment comparison alongside current practice in health technology assessment was reviewed. Reports published in UK by the Health Technology Assessment programme since 1997, which utilized indirect comparison/mixed treatment comparison methods, were reviewed with respect to the presentation of study data, statistical models and results. Recommendations for presentation were developed. RESULTS: Guidance exists that provide the details necessary to conduct a successful indirect comparison/mixed treatment comparison analysis but recommendations on presentation are limited. Of 205 health technology assessment reports that contained evidence synthesis for effectiveness, 19 used indirect comparison/mixed treatment comparison methods. These reports utilized numerous presentational formats from which the following key components were identified: network table/diagram for presenting data; model description to allow reproducibility; and tables, forest plots, matrix tables and summary forest plots for presenting a range of results. Recommendations were developed to ensure that reporting is explicit, transparent and reproducible. Approaches most understandable by non-technical decision makers, and areas where future research is required, are outlined. CONCLUSIONS: There is no standard presentational style used in UK for reporting indirect comparison/mixed treatment comparison, and the use of graphical tools is limited. Standardization of reporting and innovation in graphical representation of indirect comparison/mixed treatment comparison results is required.

Use of implicit and explicit bayesian methods in health technology assessment.

OBJECTIVES: The aim of this study was to examine the use of implicit and explicit Bayesian methods in health technology assessments and to identify whether this has changed over time. METHODS: A review of all health technology assessment (HTA) reports of secondary research published by the UK National Institute of Health Research (NIHR) between 1997 and 2011. Data were extracted on the use and implementation of Bayesian methods, whether defined as such by the original authors (i.e., explicit) or not (i.e., implicit). RESULTS: A total of 155 of 375 (41 percent) NIHR HTA reports, identified as relevant to this review, contained a Bayesian analysis. Of these, 128 (83 percent) contained an implicit Bayesian analysis, 3 (2 percent) an explicit Bayesian analysis and 24 (15 percent) both implicit and explicit Bayesian analyses. Of the twenty-seven reports that explicitly used Bayes theorem, only six included prior information in the form of (informative) prior distributions. Over time, the percentage of HTA reports that used Bayesian (implicit and/or explicit) methods increased from 0 percent in 1997 to nearly 80 percent in 2011. CONCLUSIONS: This review has shown that there has been an increase in the use of Bayesian methods in HTA, which is likely to be a result of the increase in freely available resources to implement the approach. Areas where Bayesian methods have the potential to advance healthcare evaluations in the future are considered in the discussion.

Systematic review and network meta-analysis of overall survival comparing 3 mg/kg ipilimumab with alternative therapies in the management of pretreated patients with unresectable stage III or IV melanoma.

OBJECTIVE: To compare the overall survival (OS) of patients treated with 3 mg/kg ipilimumab versus alternative systemic therapies in pretreated unresectable stage III or IV melanoma patients. METHODS: A systematic literature search was performed to identify relevant randomized clinical trials. From these trials, Kaplan-Meier survival curves for each intervention were digitized and combined by means of a Bayesian network meta-analysis (NMA) to compare different drug classes. RESULTS: Of 38 trials identified, 15 formed one interlinked network by drug class to allow for an NMA. Ipilimumab, at a dose of 3 mg/kg, was associated with a greater mean OS time (18.8 months; 95% credible interval [CrI], 15.5-23.0 months) than single-agent chemotherapy (12.3 months; 95% CrI, 6.3-28.0 months), chemotherapy combinations (12.2 months; 95% CrI, 7.1-23.3 months), biochemotherapies (11.9 months; 95% CrI, 7.0-22.0 months), single-agent immunotherapy (11.1 months; 95% CrI, 8.5-16.2 months), and immunotherapy combinations (14.1 months; 95% CrI, 9.0-23.8 months). CONCLUSION: Results of this NMA were in line with previous findings and suggest that OS with ipilimumab is expected to be greater than with alternative systemic therapies, alone or in combination, for the management of pretreated patients with unresectable stage III or IV melanoma.