Design of a clinical trial using generalized pairwise comparisons to test a less intensive treatment regimen.

BACKGROUND/AIMS: Showing "similar efficacy" of a less intensive treatment typically requires a non-inferiority trial. Yet such trials may be challenging to design and conduct. In acute promyelocytic leukemia, great progress has been achieved with the introduction of targeted therapies, but toxicity remains a major clinical issue. There is a pressing need to show the favorable benefit/risk of less intensive treatment regimens. METHODS: We designed a clinical trial that uses generalized pairwise comparisons of five prioritized outcomes (alive and event-free at 2 years, grade 3/4 documented infections, differentiation syndrome, hepatotoxicity, and neuropathy) to confirm a favorable benefit/risk of a less intensive treatment regimen. We conducted simulations based on historical data and assumptions about the differences expected between the standard of care and the less intensive treatment regimen to calculate the sample size required to have high power to show a positive Net Treatment Benefit in favor of the less intensive treatment regimen. RESULTS: Across 10,000 simulations, average sample sizes of 260 to 300 patients are required for a trial using generalized pairwise comparisons to detect typical Net Treatment Benefits of 0.19 (interquartile range 0.14-0.23 for a sample size of 280). The Net Treatment Benefit is interpreted as a difference between the probability of doing better on the less intensive treatment regimen than on the standard of care, minus the probability of the opposite situation. A Net Treatment Benefit of 0.19 translates to a number needed to treat of about 5.3 patients (1/0.19 ≃ 5.3). CONCLUSION: Generalized pairwise comparisons allow for simultaneous assessment of efficacy and safety, with priority given to the former. The sample size required would be of the order of 300 patients, as compared with more than 700 patients for a non-inferiority trial using a margin of 4% against the less intensive treatment regimen for the absolute difference in event-free survival at 2 years, as considered here.

Generalized pairwise comparisons of prioritized outcomes are a powerful and patient-centric analysis of multi-domain scores.

BACKGROUND: Generalized pairwise comparisons (GPC) can be used to assess the net benefit of new treatments for rare diseases. We show the potential of GPC through simulations based on data from a natural history study in mucopolysaccharidosis type IIIA (MPS IIIA). METHODS: Using data from a historical series of untreated children with MPS IIIA aged 2 to 9 years at the time of enrolment and followed for 2 years, we performed simulations to assess the operating characteristics of GPC to detect potential (simulated) treatment effects on a multi-domain symptom assessment. Two approaches were used for GPC: one in which the various domains were prioritized, the other with all domains weighted equally. The net benefit was used as a measure of treatment effect. We used increasing thresholds of clinical relevance to reflect the magnitude of the desired treatment effects, relative to the standard deviation of the measurements in each domain. RESULTS: GPC were shown to have adequate statistical power (80% or more), even with small sample sizes, to detect treatment effects considered to be clinically worthwhile on a symptom assessment covering five domains (expressive language, daily living skills, and gross-motor, sleep and pain). The prioritized approach generally led to higher power as compared with the non-prioritized approach. CONCLUSIONS: GPC of prioritized outcomes is a statistically powerful as well as a patient-centric approach for the analysis of multi-domain scores in MPS IIIA and could be applied to other heterogeneous rare diseases.

Trial Design for Cancer Immunotherapy: A Methodological Toolkit.

Immunotherapy with checkpoint inhibitors (CPIs) and cell-based products has revolutionized the treatment of various solid tumors and hematologic malignancies. These agents have shown unprecedented response rates and long-term benefits in various settings. These clinical advances have also pointed to the need for new or adapted approaches to trial design and assessment of efficacy and safety, both in the early and late phases of drug development. Some of the conventional statistical methods and endpoints used in other areas of oncology appear to be less appropriate in immuno-oncology. Conversely, other methods and endpoints have emerged as alternatives. In this article, we discuss issues related to trial design in the early and late phases of drug development in immuno-oncology, with a focus on CPIs. For early trials, we review the most salient issues related to dose escalation, use and limitations of tumor response and progression criteria for immunotherapy, the role of duration of response as an endpoint in and of itself, and the need to conduct randomized trials as early as possible in the development of new therapies. For late phases, we discuss the choice of primary endpoints for randomized trials, review the current status of surrogate endpoints, and discuss specific statistical issues related to immunotherapy, including non-proportional hazards in the assessment of time-to-event endpoints, alternatives to the Cox model in these settings, and the method of generalized pairwise comparisons, which can provide a patient-centric assessment of clinical benefit and be used to design randomized trials.

Generalized pairwise comparisons for censored data: An overview.

The method of generalized pairwise comparisons (GPC) is an extension of the well-known nonparametric Wilcoxon-Mann-Whitney test for comparing two groups of observations. Multiple generalizations of Wilcoxon-Mann-Whitney test and other GPC methods have been proposed over the years to handle censored data. These methods apply different approaches to handling loss of information due to censoring: ignoring noninformative pairwise comparisons due to censoring (Gehan, Harrell, and Buyse); imputation using estimates of the survival distribution (Efron, Péron, and Latta); or inverse probability of censoring weighting (IPCW, Datta and Dong). Based on the GPC statistic, a measure of treatment effect, the "net benefit," can be defined. It quantifies the difference between the probabilities that a randomly selected individual from one group is doing better than an individual from the other group. This paper aims at evaluating GPC methods for censored data, both in the context of hypothesis testing and estimation, and providing recommendations related to their choice in various situations. The methods that ignore uninformative pairs have comparable power to more complex and computationally demanding methods in situations of low censoring, and are slightly superior for high proportions (>40%) of censoring. If one is interested in estimation of the net benefit, Harrell's c index is an unbiased estimator if the proportional hazards assumption holds. Otherwise, the imputation (Efron or Peron) or IPCW (Datta, Dong) methods provide unbiased estimators in case of proportions of drop-out censoring up to 60%.

Operational characteristics of generalized pairwise comparisons for hierarchically ordered endpoints.

The method of generalized pairwise comparisons (GPC) is a multivariate extension of the well-known non-parametric Wilcoxon-Mann-Whitney test. It allows comparing two groups of observations based on multiple hierarchically ordered endpoints, regardless of the number or type of the latter. The summary measure, "net benefit," quantifies the difference between the probabilities that a random observation from one group is doing better than an observation from the opposite group. The method takes into account the correlations between the endpoints. We have performed a simulation study for the case of two hierarchical endpoints to evaluate the impact of their correlation on the type-I error probability and power of the test based on GPC. The simulations show that the power of the GPC test for the primary endpoint is modified if the secondary endpoint is included in the hierarchical GPC analysis. The change in power depends on the correlation between the endpoints. Interestingly, a decrease in power can occur, regardless of whether there is any marginal treatment effect on the secondary endpoint. It appears that the overall power of the hierarchical GPC procedure depends, in a complex manner, on the entire variance-covariance structure of the set of outcomes. Any additional factors (such as thresholds of clinical relevance, drop out, or censoring scheme) will also affect the power and will have to be taken into account when designing a trial based on the hierarchical GPC procedure.

Unbiasedness and efficiency of non-parametric and UMVUE estimators of the probabilistic index and related statistics.

In reliability theory, diagnostic accuracy, and clinical trials, the quantity P(X>Y)+1/2P(X=Y), also known as the Probabilistic Index (PI), is a common treatment effect measure when comparing two groups of observations. The quantity P(X>Y)-P(Y>X), a linear transformation of PI known as the net benefit, has also been advocated as an intuitively appealing treatment effect measure. Parametric estimation of PI has received a lot of attention in the past 40 years, with the formulation of the Uniformly Minimum-Variance Unbiased Estimator (UMVUE) for many distributions. However, the non-parametric Mann-Whitney estimator of the PI is also known to be UMVUE in some situations. To understand this seeming contradiction, in this paper a systematic comparison is performed between the non-parametric estimator for the PI and parametric UMVUE estimators in various settings. We show that the Mann-Whitney estimator is always an unbiased estimator of the PI with univariate, completely observed data, while the parametric UMVUE is not when the distribution is misspecified. Additionally, the Mann-Whitney estimator is the UMVUE when observations belong to an unrestricted family. When observations come from a more restrictive family of distributions, the loss in efficiency for the non-parametric estimator is limited in realistic clinical scenarios. In conclusion, the Mann-Whitney estimator is simple to use and is a reliable estimator for the PI and net benefit in realistic clinical scenarios.