Selection bias for treatments with positive Phase 2 results.

In drug development, treatments are most often selected at Phase 2 for further development when an initial trial of a new treatment produces a result that is considered positive. This selection due to a positive result means, however, that an estimator of the treatment effect, which does not take account of the selection is likely to over-estimate the true treatment effect (ie, will be biased). This bias can be large and researchers may face a disappointingly lower estimated treatment effect in further trials. In this paper, we review a number of methods that have been proposed to correct for this bias and introduce three new methods. We present results from applying the various methods to two examples and consider extensions of the examples. We assess the methods with respect to bias of estimation of the treatment effect and compare the probabilities that a bias-corrected treatment effect estimate will exceed a decision threshold. Following previous work, we also compare average power for the situation where a Phase 3 trial is launched given that the bias-corrected observed Phase 2 treatment effect exceeds a launch threshold. Finally, we discuss our findings and potential application of the bias correction methods.

Recent Developments in the Prevention and Treatment of Missing Data.

Recent research has fostered new guidance on preventing and treating missing data, most notably the landmark expert panel report from the National Research Council (NRC) that was commissioned by FDA. One of the findings from that panel was the need for better software tools to conduct missing data sensitivity analyses and frameworks for drawing inference from them. In response to the NRC recommendations, a Scientific Working Group was formed under the Auspices of the Drug Information Association (DIASWG). The present paper is from work of the DIASWG. Specifically, the NRC panel's 18 recommendations are distilled into 3 pillars for dealing with missing data: (1) providing clearly stated objectives and causal estimands; (2) preventing as much missing data as possible; and (3) combining a sensible primary analysis with sensitivity analyses to assess robustness of inferences to missing data assumptions. Sample data sets are used to illustrate how sensitivity analyses can be used to assess robustness of inferences to missing data assumptions. The suite of software tools used to conduct the sensitivity analyses are freely available for public use at www.missingdata.org.uk.

Discounting phase 2 results when planning phase 3 clinical trials.

Sample size planning is an important design consideration for a phase 3 trial. In this paper, we consider how to improve this planning when using data from phase 2 trials. We use an approach based on the concept of assurance. We consider adjusting phase 2 results because of two possible sources of bias. The first source arises from selecting compounds with pre-specified favourable phase 2 results and using these favourable results as the basis of treatment effect for phase 3 sample size planning. The next source arises from projecting phase 2 treatment effect to the phase 3 population when this projection is optimistic because of a generally more heterogeneous patient population at the confirmatory stage. In an attempt to reduce the impact of these two sources of bias, we adjust (discount) the phase 2 estimate of treatment effect. We consider multiplicative and additive adjustment. Following a previously proposed concept, we consider the properties of several criteria, termed launch criteria, for deciding whether or not to progress development to phase 3. We use simulations to investigate launch criteria with or without bias adjustment for the sample size calculation under various scenarios. The simulation results are supplemented with empirical evidence to support the need to discount phase 2 results when the latter are used in phase 3 planning. Finally, we offer some recommendations based on both the simulations and the empirical investigations.

Laboratory data in clinical trials: a statistician's perspective.

Even though laboratory data provide the best indicators for systemic toxicities in clinical trials of investigational medications, many applied statisticians lack a basic understanding of the interpretation of such data. Understanding is essential to a statistician's ability to help evaluate a patient's overall safety experience in a trial, the latter being the primary objective for collecting laboratory data in the trial. In this paper, we discuss the purpose of conducting laboratory evaluations as well as some hidden issues concerning the current practice of laboratory data analysis. The issues include the use of reference ranges, the one-parameter-at-a-time approach, and the exploratory nature of safety data analyses.

The impact and implication of regression to the mean on the design and analysis of medical investigations.

We have examined the regression effect and its magnitude under the Gaussian distributional assumption. The impact and implication of regression to the mean on the analysis of medical investigations was discussed. For simplicity, we called the approach adjusting for the regression effect a two-stage procedure and noted its relationship to the analysis of covariance model for comparing treatment groups. We also proposed to examine the correlation structure among repeated measurements in the absence of any external interventions through a model more realistic than the one assuming equal correlations. The proposed structure led us to investigate ways to reduce or eliminate regression effect via study designs when patient selection is inevitable. Two examples were given to help illustrate the discussion in this paper.

Multiple comparisons procedures for comparing several treatments with a control based on binary data.

In this paper, we examine three approaches for comparing several treatments with a control with use of binary response data. The first approach relies on asymptotic theory applied to the Freeman-Tukey transformation of the observed proportions. The second finds an acceptance region based on the binomial distributions estimated under the joint null hypotheses. The third approach applies Dunnett's procedure to the binary data. We evaluated the actual overall type I error rates of the Freeman-Tukey test and Dunnett's procedure using both simulation and binomial calculations while we assessed those of the binomial approach using simulation. Based upon their capability to preserve the desirable overall type I error rate, we provide recommendations regarding the choices among the three approaches for various occasions. In addition, we provide comments on the power of these three approaches.

A new proposal for benefit-less-risk analysis in clinical trials.

In this paper, we propose a method to discount the observed benefit of a treatment by the observed risk in order to facilitate the benefit-less-risk comparison of treatments in a clinical trial. The discounting, applied to each individual in a trial, utilizes a method proposed by Chuang-Stein and co-authors to consolidate the safety data collected in the trial. The collating of the safety information allows one to estimate quantitatively the risk experienced by each individual, and therefore enables the construction of a risk-adjusted benefit measure for the same individual. We discuss the rationale for the adjusting method and examine its impact on the inference. When the discounting process reflects an individual's choice, the results should be interpreted at the individual level. An example is given to illustrate the approach.

Organization and analysis of safety data using a multivariate approach.

The collection of safety data is an important part of clinical trials. These safety data are often described and reported in great detail with expenditure of substantial effort and energy. Because of the wide variety of data that require scrutiny from the safety perspective, however, statistical comparisons of the safety profiles of different treatments often lack focus and structure and result in situations where the comparisons for each individual item lack power and thus are inconclusive. In this paper, we propose to organize the safety data into a more manageable form by consolidating them into a number of K classes characterized by body systems and determined in conjunction with the underlying disease as well as the treatments involved. Within each class, we propose assignment to each patient of an overall intensity grade based on all relevant information. The consolidation of the safety data as proposed provides an informative summary for the safety profile of each treatment. The analysis of such organized data concentrates on comparison of the mean intensity grades for different treatments within the K classes simultaneously with use of scores that reflect the acceptability of the various intensity levels to an individual. Furthermore, we demonstrate that the proposed multivariate comparison has much higher power than the univariate one to detect differences in certain cases. We provide examples to illustrate the proposed procedure.

Three measures for simultaneously evaluating benefits and risks using categorical data from clinical trials.

Randomized clinical trials are typically conducted to compare the efficacy (benefits) and side effects (risks) of two or more treatments. One can use results from such trials to decide on a preferable treatment that reflects one's own evaluation of the benefits and risks. To facilitate the necessary decision making, we propose in this paper three measures for simultaneously assessing benefits and risks. All three measures use weights that reflect the relative importance of the various treatment outcomes to an individual. Two of them carry the flavour of benefit/risk ratios, while the third generalizes Hilden's measure which incorporates patients' preferences. The proposed measures and procedures are illustrated using data from a phase III clinical trial of antihypertensive compounds.

A note on the analysis of titration studies.

We propose a method to incorporate all types of dropouts in the evaluation of an intervention based on binary data from a titration study. The proposed procedure does not require that the dropouts occur randomly, but instead examines each outcome individually. Our procedure has a built-in penalty factor in the estimation process that determines the amount of penalty on efficacy parameters due to efficacy-related dropouts. We also propose a method to estimate the variances associated with the estimates. The results have use in exploration of a potential dose-response relationship or to answer questions pertinent for phase III studies. We use data from a phase III antihypertension study to illustrate the procedure.

The relationship of blood transfusion, tumor staging, and cancer recurrence.

Previous research demonstrated a relationship between transfusions of whole blood, or large numbers of red cell concentrates, and later recurrence of cancers of the colon, rectum, cervix, and prostate. It is possible that the transfusion of whole blood may represent a surrogate marker for advanced or more aggressive clinical disease. The relationship of clinical or histologic tumor stage, blood transfusion status, and disease outcome was studied in detail. Patients receiving no transfusions or small numbers of red cells (less than or equal to 3 units) had uniformly better recurrence and survival experiences than patients receiving similar amounts of blood that included at least 1 unit of whole blood, regardless of the patient's clinical or histologic tumor stage. In multivariate analyses, stage was an independent predictor of outcome, and transfusion status was not a surrogate marker for stage. The effects on recurrence of stage and transfusion appear to be cumulative. These results are consistent with but do not prove the hypothesis that the transfusion of large amounts of stored plasma and cellular debris impairs the host defenses against cancer, regardless of the underlying biologic and clinical aggressiveness of the cancer.

A log-linear model for ordinal data to characterize differential change among treatments.

We propose a family of log-linear models for ordinal data that contain parameters reflecting change patterns to compare treatments relative to change from baseline. Under the most general model, rates of change can depend not only upon the direction of change, but also upon the level of the baseline classification. We describe methods for selection of a parsimonious model and for tests of hypotheses concerning treatment differences. Interpretation of treatment differences in the follow-up response profiles, within baseline strata, employs the concept of stochastic ordering. Data from two clinical trials illustrate the proposed procedure.