Determining optimal trial size using sequential analysis.

When characterising typical human movement profiles, the optimal number of trials analysed for each participant should ensure a stable mean. Sequential analysis is one method able to establish the number of trials to stability by assessing a moving point mean against a set bandwidth. As the total trial number determining this bandwidth is selected arbitrarily, the effect of applying different total trial numbers on the results of sequential analysis was investigated. Twenty participants performed 30 trials of overarm throwing, and sequential analyses were applied to three dimensional (3-D) kinematic data over 10, 20 and 30 trial numbers. We found a total of 20 to be the preferred trial number for sequential analyses. Erroneous results were produced consistently by 10 trial number groups, while moving point means were statistically unchanged after the 10th trial. Subsequently, sequential analyses were applied to 20 trials to establish trials to stability in discrete and time series elements of the 3-D kinematic data. The results suggest that a trial size between 13 and 17 provides stable means for overarm throwing kinematics.

Clinical Trials in Chronic Arthritic Diseases with Underestimated Impact of Placebo Effects on Study Size Calculation.

Whether and to which extent placebo treatment in double-blinded randomized controlled clinical trials is effective in chronic arthritic diseases has not been studied before. Therefore, a systematic literature search was undertaken to detect eligible trials. Demographic data of the placebo groups as well as concomitant and previous disease outcomes were collected. Analyses of significant bivariate correlations and linear regression between clinical endpoints and characteristics of the placebo groups were performed. A total of 152 double-blinded randomized controlled studies, including 21,616 participants in the placebo groups, was analyzed. The results of bivariate correlations and linear regressions revealed significant positive associations between responses in the placebo groups and the following factors: (i) naïvety of previous treatment and (ii) early stage of disease. In addition to the clinical relevance, the results support the importance of the placebo effect on study size calculations, and will allow an optimized calculation of patients' numbers for early placebo-controlled trials conducted in patients with chronic arthritic diseases.

Understanding power in randomized trials: The example of vertebroplasty.

OBJECTIVES: Randomized trials (RCTs) should include a sufficient number of patients to reduce the risk that the observed outcome is a result of chance rather than from a truly different treatment effect. Trials must be even larger to claim an absence of treatment effect in a placebo-controlled trial. To estimate the size of the trial and maximize power, it is often suggested to use a comparison between the means of a continuous variable. METHODS: We examine the RCTs that have compared vertebroplasty and placebo for patients with osteoporotic fractures. Most trials compared the means of a continuous pain score to yield implausibly small trials, as small as 24 patients per group. RESULTS: The minimally significant difference between groups has no precise clinical meaning for patients when it is based on a comparison of means of pain scores. A comparison of the proportions of patients reaching a per-patient outcome measure of treatment success is much more pertinent if the trial is to inform the care of future patients. The resulting trials will admittedly need to be larger, but they will be much less likely to fall prey to the 'evidence of absence' fallacy. Furthermore, trial size should also take into consideration harder clinical outcome measures, such as death and disability. CONCLUSION: When the goal of a trial is to inform outcome-based medical care, comparing the proportions of patients reaching a clinically pertinent outcome is more appropriate than comparing the means of a continuous variable.

The small trial problem.

BACKGROUND: Many randomized trials that aim to assess new or commonly used medical or surgical interventions have been so small that the validity of conclusions becomes questionable. METHODS: We illustrate the small trial problem using the power calculation of five Cochrane-reviewed studies that compared vertebroplasty versus placebo interventions. We discuss some of the reasons why the statistical admonition not to dichotomize continuous variables may not apply to the calculation of the number of patients required for trials to be meaningful. RESULTS: Placebo-controlled vertebroplasty trials planned to recruit between 23 and 71 patients per group. Four of five studies used the standardized mean difference of a continuous pain variable (centimeters on the visual analog scale (VAS)) to plan implausibly small trials. What is needed is not a mean effect at the population level but a measure of efficacy at the patient level. Clinical practice concerns the care of individual patients that vary in many more respects than the variation around the mean of a single selected variable. The inference from trial to practice concerns the frequency of success of the experimental intervention performed one patient at a time. A comparison of the proportions of patients reaching a certain threshold is a more meaningful method that appropriately requires larger trials. CONCLUSION: Most placebo-controlled vertebroplasty trials used comparisons of means of a continuous variable and were consequently very small. Randomized trials should instead be large enough to account for the diversity of future patients and practices. They should offer an evaluation of a clinically meaningful number of interventions performed in various contexts. Implications of this principle are not specific to placebo-controlled surgical trials. Trials designed to inform practice require a per-patient comparison of outcomes and the size of the trial should be planned accordingly.

Precision Medicines Have Faster Approvals Based On Fewer And Smaller Trials Than Other Medicines.

Precision medicines can benefit patients by increasing the probability of a successful treatment response in selected patient populations. The potential for more immediate signals of efficacy during clinical trials suggests such medicines will reach the market more rapidly than traditional drugs will. Using data from the Food and Drug Administration (FDA), we examined the regulatory review and pivotal trial characteristics of precision medicines. We found that in the period January 2013-June 2017, precision medicines were developed and reviewed almost two years faster than nonprecision medicines. In addition, approximately 48 percent of the precision medicines in our study qualified for the FDA's breakthrough therapy designation. Precision medicines were also approved based on fewer pivotal trials with fewer patients, and the trials were less likely to be randomized, blinded, or controlled with either an active or placebo comparator.

Clinical evidence inputs to comparative effectiveness research could impact the development of novel treatments.

AIM: This study aims to analyze the impacts of a range of clinical evidence generation scenarios associated with comparative effectiveness research (CER) on pharmaceutical innovation. MATERIALS & METHODS: We used the Global Pharmaceutical Policy Model to project the effect of changes in pharmaceutical producer costs, revenues and timings on drug innovation and health for the age 55+ populations in the USA and Europe through year 2060 using three clinical scenarios. RESULTS: Changes in producer incentives from widespread CER evidence generation and use had varied but often large predicted impacts on simulated outcomes in 2060. Effect on the number of new drug introductions ranged from a 81.1% reduction to a 45.5% increase, and the effect on population-level life expectancy ranged from a 15.6% reduction to a 11.4% increase compared to baseline estimates. CONCLUSION: The uncertainty surrounding the consequences of increased clinical evidence generation and use on innovation calls for a carefully measured approach to CER implementation, balancing near-term benefits to spending and health with long-term implications for innovation.

Small studies are more heterogeneous than large ones: a meta-meta-analysis.

OBJECTIVES: Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. We empirically evaluated whether there is also a relationship between trial size and heterogeneity (τ). STUDY DESIGN AND SETTING: We selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n = 2,009) or continuous (n = 1,254) outcome. The association between estimated τ and trial size was evaluated across meta-analyses using regression and within meta-analyses using a Bayesian approach. Small trials were predefined as those having standard errors (SEs) over 0.2 standardized effects. RESULTS: Most meta-analyses were based on few (median 4) trials. Within the same meta-analysis, the small study τS(2) was larger than the large-study τL(2) [average ratio 2.11; 95% credible interval (1.05, 3.87) for dichotomous and 3.11 (2.00, 4.78) for continuous meta-analyses]. The imprecision of τS was larger than of τL: median SE 0.39 vs. 0.20 for dichotomous and 0.22 vs. 0.13 for continuous small-study and large-study meta-analyses. CONCLUSION: Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.