Covariate-adjusted generalized pairwise comparisons in small samples.

Semiparametric probabilistic index models allow for the comparison of two groups of observations, whilst adjusting for covariates, thereby fitting nicely within the framework of generalized pairwise comparisons (GPC). As with most regression approaches in this setting, the limited amount of data results in invalid inference as the asymptotic normality assumption is not met. In addition, separation issues might arise when considering small samples. In this article, we show that the parameters of the probabilistic index model can be estimated using generalized estimating equations, for which adjustments exist that lead to estimators of the sandwich variance-covariance matrix with improved finite sample properties and that can deal with bias due to separation. In this way, appropriate inference can be performed as is shown through extensive simulation studies. The known relationships between the probabilistic index and other GPC statistics allow to also provide valid inference for example, the net treatment benefit or the success odds.

Design of a clinical trial using generalized pairwise comparisons to test a less intensive treatment regimen.

BACKGROUND/AIMS: Showing "similar efficacy" of a less intensive treatment typically requires a non-inferiority trial. Yet such trials may be challenging to design and conduct. In acute promyelocytic leukemia, great progress has been achieved with the introduction of targeted therapies, but toxicity remains a major clinical issue. There is a pressing need to show the favorable benefit/risk of less intensive treatment regimens. METHODS: We designed a clinical trial that uses generalized pairwise comparisons of five prioritized outcomes (alive and event-free at 2 years, grade 3/4 documented infections, differentiation syndrome, hepatotoxicity, and neuropathy) to confirm a favorable benefit/risk of a less intensive treatment regimen. We conducted simulations based on historical data and assumptions about the differences expected between the standard of care and the less intensive treatment regimen to calculate the sample size required to have high power to show a positive Net Treatment Benefit in favor of the less intensive treatment regimen. RESULTS: Across 10,000 simulations, average sample sizes of 260 to 300 patients are required for a trial using generalized pairwise comparisons to detect typical Net Treatment Benefits of 0.19 (interquartile range 0.14-0.23 for a sample size of 280). The Net Treatment Benefit is interpreted as a difference between the probability of doing better on the less intensive treatment regimen than on the standard of care, minus the probability of the opposite situation. A Net Treatment Benefit of 0.19 translates to a number needed to treat of about 5.3 patients (1/0.19 ≃ 5.3). CONCLUSION: Generalized pairwise comparisons allow for simultaneous assessment of efficacy and safety, with priority given to the former. The sample size required would be of the order of 300 patients, as compared with more than 700 patients for a non-inferiority trial using a margin of 4% against the less intensive treatment regimen for the absolute difference in event-free survival at 2 years, as considered here.

On restricted mean time in favor of treatment.

The restricted mean time in favor (RMT-IF) of treatment is a nonparametric effect size for complex life history data. It is defined as the net average time the treated spend in a more favorable state than the untreated over a prespecified time window. It generalizes the familiar restricted mean survival time (RMST) from the two-state life-death model to account for intermediate stages in disease progression. The overall estimand can be additively decomposed into stage-wise effects, with the standard RMST as a component. Alternate expressions of the overall and stage-wise estimands as integrals of the marginal survival functions for a sequence of landmark transitioning events allow them to be easily estimated by plug-in Kaplan-Meier estimators. The dynamic profile of the estimated treatment effects as a function of follow-up time can be visualized using a multilayer, cone-shaped "bouquet plot." Simulation studies under realistic settings show that the RMT-IF meaningfully and accurately quantifies the treatment effect and outperforms traditional tests on time to the first event in statistical efficiency thanks to its fuller utilization of patient data. The new methods are illustrated on a colon cancer trial with relapse and death as outcomes and a cardiovascular trial with recurrent hospitalizations and death as outcomes. The R-package rmt implements the proposed methodology and is publicly available from the Comprehensive R Archive Network (CRAN).

Impact of correlations between prioritized outcomes on the net benefit and its estimate by generalized pairwise comparisons.

Benefit-risk balance is gaining interest in clinical trials. For the comprehensive assessment of benefits and risks, generalized pairwise comparisons are increasingly used to estimate the net benefit based on multiple prioritized outcomes. Although previous research has demonstrated that the correlations between the outcomes impact the net benefit and its estimate, the direction and magnitude of this impact remain unclear. In this study, we investigated the impact of correlations between two binary or Gaussian variables on the true net benefit values via theoretical and numerical analyses. We also explored the impact of correlations between survival and categorical variables on the net benefit estimates based on four existing methods (Gehan, Péron, Gehan with correction, and Péron with correction) in the presence of right censoring via simulation and application to actual oncology clinical trial data. Our theoretical and numerical analyses revealed that the true net benefit values were impacted by the correlations in various directions depending on the outcome distributions. With binary endpoints, this direction was governed by a simple rule with a threshold of 50% for a favorable outcome. Our simulation showed that the net benefit estimates based on Gehan's or Péron's scoring rule could be substantially biased in the presence of right censoring, and that the direction and magnitude of this bias were associated with the outcome correlations. The recently proposed correction method greatly reduced this bias, even in the presence of strong outcome correlations. The impact of correlations should be carefully considered when interpreting the net benefit and its estimate.

Individualized Net Benefit estimation and meta-analysis using generalized pairwise comparisons in N-of-1 trials.

BACKGROUND: The Net Benefit (Δ) is a measure of the benefit-risk balance in clinical trials, based on generalized pairwise comparisons (GPC) using several prioritized outcomes and thresholds of clinical relevance. We extended Δ to N-of-1 trials, with a focus on patient-level and population-level Δ. METHODS: We developed a Δ estimator at the individual level as an extension of the stratum-specific Δ, and at the population-level as an extension of the stratified Δ. We performed a simulation study mimicking PROFIL, a series of 38 N-of-1 trials testing sildenafil in Raynaud's phenomenon, to assess the power for such an analysis with realistic data. We then reanalyzed PROFIL using GPC. This reanalysis was finally interpreted in the context of the main analysis of PROFIL which used Bayesian individual probabilities of efficacy. RESULTS: Simulations under the null showed good size of the test for both individual and population levels. The test lacked power when being simulated from the true PROFIL data, even when increasing the number of repetitions up to 140 days per patient. PROFIL individual-level estimated Δ were well correlated with the probabilities of efficacy from the Bayesian analysis while showing similarly wide confidence intervals. Population-level estimated Δ was not significantly different from zero, consistently with the previous Bayesian analysis. CONCLUSION: GPC can be used to estimate individual Δ which can then be aggregated in a meta-analytic way in N-of-1 trials. GPC ability to easily incorporate patient preferences allow for more personalized treatment evaluation, while needing much less computing time than Bayesian modeling.

The net benefit for time-to-event outcome in oncology clinical trials with treatment switching.

BACKGROUND: The net benefit is an effect measure for any type of endpoint, including the time-to-event outcome, and can provide intuitive and clinically meaningful interpretation. It is defined as the probability of a randomly selected subject from the experimental arm surviving by at least a clinically relevant time longer than a randomly selected subject from the control arm. In oncology clinical trials, an intercurrent event such as treatment switching is common, which potentially causes informative censoring; nevertheless, conventional methods for the net benefit are not able to deal with it. In this study, we proposed a new estimator using the inverse probability of censoring weighting (IPCW) method and illustrated an oncology clinical trial with treatment switching (the SHIVA study) to apply the proposed method under the estimand framework. METHODS: The net benefit can be estimated using the survival functions of each treatment group. The proposed estimator was based on the survival functions estimated by the inverse probability of the censoring weighting method that can handle covariate-dependent censoring. The simulation study was undertaken to evaluate the operating characteristics of the proposed estimator under several scenarios; we varied the shapes of the survival curves, treatment effect, covariates effect on censoring, proportion of the censoring, threshold of the net benefit, and sample size. We also applied conventional methods (the scoring rules by Péron or Gehan) and the proposed method to the SHIVA study. RESULTS: Our simulation study showed that the proposed estimator provided less biased results under the covariate-dependent censoring than existing estimators. When applying the proposed method to the SHIVA study, we were able to estimate the net benefit by incorporating the information of the covariates with different estimand strategies to address the intercurrent event of the treatment switching. However, the estimates of the proposed method and those of the aforementioned conventional methods were similar under the hypothetical strategy. CONCLUSIONS: We proposed a new estimator of the net benefit that can include covariates to account for the possibly informative censoring. We also provided an illustrative analysis of the proposed method for the oncology clinical trial with treatment switching using the estimand framework. Our proposed new estimator is suitable for handling the intercurrent events that can potentially cause covariate-dependent censoring.

Evidence synthesis analysis with prioritized benefit outcomes in oncology clinical trials.

Overall survival, progression-free survival, objective response/complete response, and duration of (complete) response are frequently used as the primary and secondary efficacy endpoints for designs and analyses of oncology clinical trials. However, these endpoints are typically analyzed separately. In this article, we introduce an evidence synthesis approach to prioritize the benefit outcomes by applying the generalized pairwise comparisons (GPC) method, and use win statistics (win ratio, win odds and net benefit) to quantify treatment benefit. Under the framework of GPC, the main advantage of this evidence synthesis approach is the ability to combine relevant outcomes of various types into a single summary statistic without relying on any parametric assumptions. It is particularly relevant since health authorities and the pharmaceutical industry are increasingly incorporating structured quantitative methodologies in their benefit-risk assessment. We apply this evidence synthesis approach to an oncology phase 3 study in first-line renal cell carcinoma to assess the overall effect of an investigational treatment by ranking the most clinically relevant endpoints in cancer drug development. This application and a simulation study demonstrate that the proposed approach can synthesize the evidence of treatment effect from multiple prioritized benefit outcomes, and has substantial advantage over conventional methods that analyze each individual endpoint separately. We also introduce a newly developed R package WINS for statistical inference based on win statistics.

On the use of extreme value tail modeling for generalized pairwise comparisons with censored outcomes.

In randomized clinical trials, methods of pairwise comparisons such as the 'Net Benefit' or the 'win ratio' have recently gained much attention when interests lies in assessing the effect of a treatment as compared to a standard of care. Among other advantages, these methods are usually praised for delivering a treatment measure that can easily handle multiple outcomes of different nature, while keeping a meaningful interpretation for patients and clinicians. For time-to-event outcomes, a recent suggestion emerged in the literature for estimating these treatment measures by providing a natural handling of censored outcomes. However, this estimation procedure may lead to biased estimates when tails of survival functions cannot be reliably estimated using Kaplan-Meier estimators. The problem then extrapolates to the other outcomes incorporated in the pairwise comparison construction. In this work, we suggest to extend the procedure by the consideration of a hybrid survival function estimator that relies on an extreme value tail model through the Generalized Pareto distribution. We provide an estimator of treatment effect measures that notably improves on bias and remains easily apprehended for practical implementation. This is illustrated in an extensive simulation study as well as in an actual trial of a new cancer immunotherapy.

Win statistics (win ratio, win odds, and net benefit) can complement one another to show the strength of the treatment effect on time-to-event outcomes.

Conventional analyses of a composite of multiple time-to-event outcomes use the time to the first event. However, the first event may not be the most important outcome. To address this limitation, generalized pairwise comparisons and win statistics (win ratio, win odds, and net benefit) have become popular and have been applied to clinical trial practice. However, win ratio, win odds, and net benefit have typically been used separately. In this article, we examine the use of these three win statistics jointly for time-to-event outcomes. First, we explain the relation of point estimates and variances among the three win statistics, and the relation between the net benefit and the Mann-Whitney U statistic. Then we explain that the three win statistics are based on the same win proportions, and they test the same null hypothesis of equal win probabilities in two groups. We show theoretically that the Z-values of the corresponding statistical tests are approximately equal; therefore, the three win statistics provide very similar p-values and statistical powers. Finally, using simulation studies and data from a clinical trial, we demonstrate that, when there is no (or little) censoring, the three win statistics can complement one another to show the strength of the treatment effect. However, when the amount of censoring is not small, and without adjustment for censoring, the win odds and the net benefit may have an advantage for interpreting the treatment effect; with adjustment (e.g., IPCW adjustment) for censoring, the three win statistics can complement one another to show the strength of the treatment effect. For calculations we use the R package WINS, available on the CRAN (Comprehensive R Archive Network).

The stratified win statistics (win ratio, win odds, and net benefit).

The win odds and the net benefit are related directly to each other and indirectly, through ties, to the win ratio. These three win statistics test the same null hypothesis of equal win probabilities between two groups. They provide similar p-values and powers, because the Z-values of their statistical tests are approximately equal. Thus, they can complement one another to show the strength of a treatment effect. In this article, we show that the estimated variances of the win statistics are also directly related regardless of ties or indirectly related through ties. Since its introduction in 2018, the stratified win ratio has been applied in designs and analyses of clinical trials, including Phase III and Phase IV studies. This article generalizes the stratified method to the win odds and the net benefit. As a result, the relations of the three win statistics and the approximate equivalence of their statistical tests also hold for the stratified win statistics.

Generalized pairwise comparisons for censored data: An overview.

The method of generalized pairwise comparisons (GPC) is an extension of the well-known nonparametric Wilcoxon-Mann-Whitney test for comparing two groups of observations. Multiple generalizations of Wilcoxon-Mann-Whitney test and other GPC methods have been proposed over the years to handle censored data. These methods apply different approaches to handling loss of information due to censoring: ignoring noninformative pairwise comparisons due to censoring (Gehan, Harrell, and Buyse); imputation using estimates of the survival distribution (Efron, Péron, and Latta); or inverse probability of censoring weighting (IPCW, Datta and Dong). Based on the GPC statistic, a measure of treatment effect, the "net benefit," can be defined. It quantifies the difference between the probabilities that a randomly selected individual from one group is doing better than an individual from the other group. This paper aims at evaluating GPC methods for censored data, both in the context of hypothesis testing and estimation, and providing recommendations related to their choice in various situations. The methods that ignore uninformative pairs have comparable power to more complex and computationally demanding methods in situations of low censoring, and are slightly superior for high proportions (>40%) of censoring. If one is interested in estimation of the net benefit, Harrell's c index is an unbiased estimator if the proportional hazards assumption holds. Otherwise, the imputation (Efron or Peron) or IPCW (Datta, Dong) methods provide unbiased estimators in case of proportions of drop-out censoring up to 60%.

Operational characteristics of generalized pairwise comparisons for hierarchically ordered endpoints.

The method of generalized pairwise comparisons (GPC) is a multivariate extension of the well-known non-parametric Wilcoxon-Mann-Whitney test. It allows comparing two groups of observations based on multiple hierarchically ordered endpoints, regardless of the number or type of the latter. The summary measure, "net benefit," quantifies the difference between the probabilities that a random observation from one group is doing better than an observation from the opposite group. The method takes into account the correlations between the endpoints. We have performed a simulation study for the case of two hierarchical endpoints to evaluate the impact of their correlation on the type-I error probability and power of the test based on GPC. The simulations show that the power of the GPC test for the primary endpoint is modified if the secondary endpoint is included in the hierarchical GPC analysis. The change in power depends on the correlation between the endpoints. Interestingly, a decrease in power can occur, regardless of whether there is any marginal treatment effect on the secondary endpoint. It appears that the overall power of the hierarchical GPC procedure depends, in a complex manner, on the entire variance-covariance structure of the set of outcomes. Any additional factors (such as thresholds of clinical relevance, drop out, or censoring scheme) will also affect the power and will have to be taken into account when designing a trial based on the hierarchical GPC procedure.

Net benefit in the presence of correlated prioritized outcomes using generalized pairwise comparisons: A simulation study.

BACKGROUND: The prioritized net benefit (Δ) is a measure of the benefit-risk balance in clinical trials, based on generalized pairwise comparisons (GPC) using several prioritized outcomes. Its estimation requires the classification as Wins or Losses of all possible pairs of patients, one from the experimental treatment (E) group and one from the control treatment (C) group. In this simulation study, we assessed the impact of the correlation between prioritized outcomes on Δ, its estimate, bias, size, and power. METHODS: The theoretical Δ value was derived for the specific case of two correlated binary outcomes when a normal copula is used. Focusing on one efficacy and one toxicity outcome, two situations frequently met in practice were simulated: binary efficacy outcome with binary toxicity outcome, or time to event efficacy outcome with categorical toxicity outcome. Several scenarios of efficacy and toxicity were generated, with various levels of correlation. RESULTS: When E was more effective than C, positive correlations were mainly associated with a decrease in the proportion of Losses, while negative correlations were associated with a decrease in the proportion of Wins on the toxicity outcome. This resulted in an increase of Δ^ with the intensity of the positive correlation without adding any bias. Results were similar whatever the type of outcomes generated but led to power alteration. CONCLUSION: Correlations between outcomes analyzed with GPC led to substantial but predictable modifications of Δ and its estimate. Correlations should be taken into consideration when performing sample size estimations in clinical trials.

A new measure of treatment effect in clinical trials involving competing risks based on generalized pairwise comparisons.

In survival analysis with competing risks, the treatment effect is typically expressed using cause-specific or subdistribution hazard ratios, both relying on proportional hazards assumptions. This paper proposes a nonparametric approach to analyze competing risks data based on generalized pairwise comparisons (GPC). GPC estimate the net benefit, defined as the probability that a patient from the treatment group has a better outcome than a patient from the control group minus the probability of the opposite situation, by comparing all pairs of patients taking one patient from each group. GPC allow using clinically relevant thresholds and simultaneously analyzing multiple prioritized endpoints. We show that under proportional subdistribution hazards, the net benefit for competing risks settings can be expressed as a decreasing function of the subdistribution hazard ratio, taking a value 0 when the latter equals 1. We propose four net benefit estimators dealing differently with censoring. Among them, the Péron estimator uses the Aalen-Johansen estimator of the cumulative incidence functions to classify the pairs for which the patient with the best outcome could not be determined due to censoring. We use simulations to study the bias of these estimators and the size and power of the tests based on the net benefit. The Péron estimator was approximately unbiased when the sample size was large and the censoring distribution's support sufficiently wide. With one endpoint, our approach showed a comparable power to a proportional subdistribution hazards model even under proportional subdistribution hazards. An application of the methodology in oncology is provided.

Correcting the bias of the net benefit estimator due to right-censored observations.

Generalized pairwise comparisons (GPCs) are a statistical method used in randomized clinical trials to simultaneously analyze several prioritized outcomes. This procedure estimates the net benefit (Δ). Δ may be interpreted as the probability for a random patient in the treatment group to have a better overall outcome than a random patient in the control group, minus the probability of the opposite situation. However, the presence of right censoring introduces uninformative pairs that will typically bias the estimate of Δ toward 0. We propose a correction to GPCs that estimates the contribution of each uninformative pair based on the average contribution of the informative pairs. The correction can be applied to the analysis of several prioritized outcomes. We perform a simulation study to evaluate the bias associated with this correction. When only one time-to-event outcome was generated, the corrected estimates were unbiased except in the presence of very heavy censoring. The correction had no effect on the power or type-1 error of the tests based on the Δ. Finally, we illustrate the impact of the correction using data from two randomized trials. The illustrative datasets showed that the correction had limited impact when the proportion of censored observations was around 20% and was most useful when this proportion was close to 70%. Overall, we propose an estimator for the net benefit that is minimally affected by censoring under the assumption that uninformative pairs are exchangeable with informative pairs.

Restricted Net Treatment Benefit in oncology.

OBJECTIVES: The restricted Net Treatment Benefit (rNTB) is a clinically meaningful and tractable estimand of the overall treatment effect assessed in randomized trials when at least one survival endpoint with time restriction is used. Its interpretation does not rely on parametric assumptions such as proportional hazards, can be estimated without bias even in the presence of independent right-censoring, and can include a prespecified threshold of minimal clinically relevant difference. To demonstrate that the rNTB, corresponding to the NTB during a predefined time interval, is a meaningful and adaptable measure of treatment effect in clinical trials. METHODS: In this simulation study, we tested the impact on the rNTB value, estimation, and power of several factors including the presence of a delayed treatment effect, minimal clinically relevant difference threshold value, restriction time value, and the inclusion of both efficacy and toxicity in the rNTB definition. The impact of right censoring on rNTB was assessed in terms of bias. rNTB-derived statistical tests and log rank (LR) tests were compared in terms of power. RESULTS: RNTB estimates are unbiased even in case of right-censoring. rNTB may be used to estimate the benefit/risk ratio of a new treatment, for example, taking into account both survival and toxicity and include several prioritized outcomes. The estimated rNTB is much easier to interpret in this context compared to NTB in the presence of censoring since the latter is intrinsically dependent on the follow-up duration. Including toxicity increases the test power when the experimental treatment is less toxic. rNTB-derived test power increases when the experimental treatment is associated with longer survival and lower toxicity and might increase in the presence of a cure rate or a delayed treatment effect. Case applications on the PRODIGE, Checkmate-066, and Checkmate-067 trials are provided. CONCLUSIONS: RNTB is an interesting alternative to describe and test the treatment's effect in a clear and understandable way in case of restriction, particularly in scenarios with nonproportional hazards or when trying to balance benefit and safety. It can be tuned to take into consideration short- or long-term survival differences and one or more prioritized outcomes.

Generalized pairwise comparisons of prioritized outcomes are a powerful and patient-centric analysis of multi-domain scores.

BACKGROUND: Generalized pairwise comparisons (GPC) can be used to assess the net benefit of new treatments for rare diseases. We show the potential of GPC through simulations based on data from a natural history study in mucopolysaccharidosis type IIIA (MPS IIIA). METHODS: Using data from a historical series of untreated children with MPS IIIA aged 2 to 9 years at the time of enrolment and followed for 2 years, we performed simulations to assess the operating characteristics of GPC to detect potential (simulated) treatment effects on a multi-domain symptom assessment. Two approaches were used for GPC: one in which the various domains were prioritized, the other with all domains weighted equally. The net benefit was used as a measure of treatment effect. We used increasing thresholds of clinical relevance to reflect the magnitude of the desired treatment effects, relative to the standard deviation of the measurements in each domain. RESULTS: GPC were shown to have adequate statistical power (80% or more), even with small sample sizes, to detect treatment effects considered to be clinically worthwhile on a symptom assessment covering five domains (expressive language, daily living skills, and gross-motor, sleep and pain). The prioritized approach generally led to higher power as compared with the non-prioritized approach. CONCLUSIONS: GPC of prioritized outcomes is a statistically powerful as well as a patient-centric approach for the analysis of multi-domain scores in MPS IIIA and could be applied to other heterogeneous rare diseases.

Composite endpoints, including patient reported outcomes, in rare diseases.

BACKGROUND: When assessing the efficacy of a treatment in any clinical trial, it is recommended by the International Conference on Harmonisation to select a single meaningful endpoint. However, a single endpoint is often not sufficient to reflect the full clinical benefit of a treatment in multifaceted diseases, which is often the case in rare diseases. Therefore, the use of a combination of several clinically meaningful outcomes is preferred. Many methodologies that allow for combining outcomes in a so-called composite endpoint are however limited in a number of ways, not in the least in the number and type of outcomes that can be combined and in the poor small-sample properties. Moreover, patient reported outcomes, such as quality of life, often cannot be integrated in a composite analysis, in spite of their intrinsic value. RESULTS: Recently, a class of non-parametric generalized pairwise comparisons tests have been proposed, which members do allow for any number and type of outcomes, including patient reported outcomes. The class enjoys good small-sample properties. Moreover, this very flexible class of methods allows for prioritizing the outcomes by clinical severity, allows for matched designs and for adding a threshold of clinical relevance. Our aim is to introduce the generalized pairwise comparison ideas and concepts for rare disease clinical trial analysis, and demonstrate their benefit in a post-hoc analysis of a small-sample trial in epidermolysis bullosa. More precisely, we will include a patient relevant outcome (Quality of life), in a composite endpoint. This publication is part of the European Joint Programme on Rare Diseases (EJP RD) series on innovative methodologies for rare diseases clinical trials, which is based on the webinars presented within the educational activity of EJP RD. This publication covers the webinar topic on composite endpoints in rare diseases and includes participants' response to a questionnaire on this topic. CONCLUSIONS: Generalized pairwise comparisons is a promising statistical methodology for evaluating any type of composite endpoints in rare disease trials and may allow a better evaluation of therapy efficacy including patients reported outcomes in addition to outcomes related to the diseases signs and symptoms.

Generalized Pairwise Comparisons to Assess Treatment Effects: JACC Review Topic of the Week.

A time-to-first-event composite endpoint analysis has well-known shortcomings in evaluating a treatment effect in cardiovascular clinical trials. It does not fully describe the clinical benefit of therapy because the severity of the events, events repeated over time, and clinically relevant nonsurvival outcomes cannot be considered. The generalized pairwise comparisons (GPC) method adds flexibility in defining the primary endpoint by including any number and type of outcomes that best capture the clinical benefit of a therapy as compared with standard of care. Clinically important outcomes, including bleeding severity, number of interventions, and quality of life, can easily be integrated in a single analysis. The treatment effect in GPC can be expressed by the net treatment benefit, the success odds, or the win ratio. This review provides guidance on the use of GPC and the choice of treatment effect measures for the analysis and reporting of cardiovascular trials.

The Net Benefit of a treatment should take the correlation between benefits and harms into account.

OBJECTIVE: The assessment of benefits and harms from experimental treatments often ignores the association between outcomes. In a randomized trial, generalized pairwise comparisons (GPC) can be used to assess a Net Benefit that takes this association into account. STUDY DESIGN AND SETTINGS: We use GPC to analyze a fictitious trial of treatment versus control, with a binary efficacy outcome (response) and a binary toxicity outcome, as well as data from two actual randomized trials in oncology. In all cases, we compute the Net Benefit for scenarios with different orders of priority between response and toxicity, and a range of odds ratios (ORs) for the association between these outcomes. RESULTS: The GPC Net Benefit was quite different from the benefit/harm computed using marginal treatment effects on response and toxicity. In the fictitious trial using response as first priority, treatment had an unfavorable Net Benefit if OR < 1, but favorable if OR > 1. With OR = 1, the Net Benefit was 0. Results changed drastically using toxicity as first priority. CONCLUSION: Even in a simple situation, marginal treatment effects can be misleading. In contrast, GPC assesses the Net Benefit as a function of the treatment effects on each outcome, the association between outcomes, and individual patient priorities.