The win ratio in cardiology trials: lessons learnt, new developments, and wise future use.

The win ratio method for analysing a composite clinical hierarchy of outcomes is growing in popularity especially in cardiovascular trials. This article gives a perspective on its use so far and the issues derived from that experience. Specifically, it focuses on the limitations of a conventional composite outcome; how does the win ratio work, what does it mean, and how to display its findings; guidance on choosing an appropriate clinical hierarchy of outcomes including clinical events, quantitative outcomes, and other options; the additional value of the win difference as a measure of absolute benefit: extension to stratified win ratio, subgroup analysis, matched win ratio, and covariate adjustment; determining trial size for a win ratio outcome; specific insights such as adaptive designs, use of repeat events, and use of margins and time averages for quantitative outcomes; a critique of potential misuses; availability of statistical software; and a statistical appendix on the methodological details. Throughout, each principle is illustrated by examples from specific cardiology trials. The article concludes with a set of recommendations for future use of the win ratio.

Win Ratio Analyses of Piperacillin-Tazobactam Versus Meropenem for Ceftriaxone-Nonsusceptible Escherichia coli or Klebsiella pneumoniae Bloodstream Infections: Post Hoc Insights From the MERINO Trial.

BACKGROUND: Clinical trials of treatments for serious infections commonly use the primary endpoint of all-cause mortality. However, many trial participants survive their infection and this endpoint may not truly reflect important benefits and risks of therapy. The win ratio uses a hierarchical composite endpoint that can incorporate and prioritize outcome measures by relative clinical importance. METHODS: The win ratio methodology was applied post hoc to outcomes observed in the MERINO trial, which compared piperacillin-tazobactam with meropenem. We quantified the win ratio with a primary hierarchical composite endpoint, including all-cause mortality, microbiological relapse, and secondary infection. A win ratio of 1 would correspond to no difference between the 2 antibiotics, while a ratio <1 favors meropenem. Further analyses were performed to calculate the win odds and to introduce a continuous outcome variable in order to reduce ties. RESULTS: With the hierarchy of all-cause mortality, microbiological relapse, and secondary infection, the win ratio estimate was 0.40 (95% confidence interval [CI], .22-.71]; P = .002), favoring meropenem over piperacillin-tazobactam. However, 73.4% of the pairs were tied due to the small proportion of events. The win odds, a modification of the win ratio accounting for ties, was 0.79 (95% CI, .68-.92). The addition of length of stay to the primary composite greatly minimized the number of ties (4.6%) with a win ratio estimate of 0.77 (95% CI, .60-.99; P = .04). CONCLUSIONS: The application of the win ratio methodology to the MERINO trial data illustrates its utility and feasibility for use in antimicrobial trials.

Comparing Analytical Methods for Composite End Points in Clinical Trials: Insights from the Vericiguat Global Study in Subjects with Heart Failure With Reduced Ejection Fraction Trial.

BACKGROUND: In VICTORIA (Vericiguat Global Study in Subjects with Heart Failure with Reduced Ejection Fraction), participants with heart failure (HF) and reduced ejection fraction, vericiguat decreased the primary composite outcome (time to first HF hospitalization [HFH] or cardiovascular death [CVD]) (897 events) compared with placebo (972 events) (hazard ratio, 0.90; 95% confidence interval [CI], 0.82-0.98; P = .02). In this prespecified secondary analysis, we applied the weighted composite end point (WCE) and the win ratio (WR) methods to provide complementary assessments of treatment effect. METHODS AND RESULTS: The WCE method estimated the mean HFH-adjusted survival based on prespecified weights from a Delphi panel of the VICTORIA executive committee and national leaders: mild (weight per event, 0.39), moderate (0.5), or severe (0.67) HFH, and CVD (1.0). The unmatched WR was estimated for the descending hierarchy of CVD, then recurrent HFH. The WCE used all 3412 primary clinical events: 875 severe HFH (vericiguat, 416/ placebo, 459), 1614 moderate HFH (767/847), 68 mild HFH (38/30), and 855 CVD (414/441). Improved HFH-adjusted survival occurred with vericiguat (mean 78.2% vs 75.6%, difference 2.4%, 95% CI, 1.7%-3.2%, P < .0001). Based on a comparison of 6,375,624 pairs, the WR of 1.13 (95% CI 1.03-1.24, P = .01) also indicated improved clinical outcomes with vericiguat. CONCLUSIONS: The results of the WCE and WR methods were consistent with the primary analysis of the time to first HFH or CVD. Although both WCE and WR assessed recurrent events, the WCE allowed inclusion of all recurrent events, insights on the severity of HFH events, and an absolute measure of the participant-treatment experience. This approach complements conventional assessment, better informing consumers of new therapeutics and future trial designs.

Clinician's Approach to Advanced Statistical Methods: Win Ratios, Restricted Mean Survival Time, Responder Analyses, and Standardized Mean Differences.

Novel statistical methods have emerged in recent medical literature, which clinicians must understand to properly appraise and integrate evidence into their practice. Some of these key concepts include win ratios, restricted mean survival time, responder analyses, and standardized mean difference. This article offers guidance to busy clinicians on the comprehension and practical applicability of the results to patients. Win ratios provide an alternative method to analyze composite outcomes by prioritizing individual components of the composite; prioritization of the outcomes should be evidence-based, pre-specified, and patient-centered. Restricted mean survival time presents a method to analyze Kaplan-Meier curves when assumptions required for Cox proportional hazards analysis are not met. As it only considers outcomes that occur within a specific timeframe, the duration of follow-up must be appropriately defined and based on prior epidemiologic and mechanistic evidence. Researchers can analyze continuous outcomes with responder analyses, in which participants are dichotomized into "responders" or "non-responders." While clinicians and patients may more easily grasp outcomes analyzed in this way, they should be aware of the loss of information and resulting imprecision, as well as potential to manipulate data presentation. When meta-analyzing continuous outcomes, point estimates can be converted to standardized mean differences to facilitate the combination of data utilizing various outcome measures. However, clinicians may find it challenging to grasp the clinical meaningfulness of a standardized mean difference, and may benefit from converting it to well-known outcomes. By providing the background knowledge of these statistical methods, along with practical applicability, benefits, and inevitable limitations, this article aims to provide clinicians with an approach to appraise the literature and apply the results in clinical practice.

Use of win time for ordered composite endpoints in clinical trials.

Consider the choice of outcome for overall treatment benefit in a clinical trial which measures the first time to each of several clinical events. We describe several new variants of the win ratio that incorporate the time spent in each clinical state over the common follow-up, where clinical state means the worst clinical event that has occurred by that time. One version allows restriction so that death during follow-up is most important, while time spent in other clinical states is still accounted for. Three other variants are described; one is based on the average pairwise win time, one creates a continuous outcome for each participant based on expected win times against a reference distribution and another that uses the estimated distributions of clinical state to compare the treatment arms. Finally, a combination testing approach is described to give robust power for detecting treatment benefit across a broad range of alternatives. These new methods are designed to be closer to the overall treatment benefit/harm from a patient's perspective, compared to the ordinary win ratio. The new methods are compared to the composite event approach and the ordinary win ratio. Simulations show that when overall treatment benefit on death is substantial, the variants based on either the participants' expected win times (EWTs) against a reference distribution or estimated clinical state distributions have substantially higher power than either the pairwise comparison or composite event methods. The methods are illustrated by re-analysis of the trial heart failure: a controlled trial investigating outcomes of exercise training.

Components of the Atrial fibrillation Better Care pathway for holistic care of patients with atrial fibrillation: a win ratio analysis from the COOL-AF registry.

AIMS: Compliance with integrated care based on the Atrial fibrillation Better Care (ABC) pathway has been associated with improved clinical outcomes. The primary objective of this study was to compare clinical outcomes of AF patients according to the compliant status of each component of the ABC pathway in a hierarchical win ratio approach. METHODS AND RESULTS: We studied AF patients in the COOL-AF registry. Each patient was followed every 6 months until 3 years. A win ratio analysis was performed, as not all clinical outcomes are equivalent. The hierarchical outcomes were (1) all-cause death, (2) intracranial haemorrhage (ICH), (3) ischaemic stroke/systemic embolism, (4) non-ICH major bleedings, and (5) acute myocardial infarction or heart failure. We also assessed win ratio and win proportion variance over the follow-up time, and the variations over time. A total of 3405 patients (mean age 67.8 ± 11.3; 41.8% female) were studied. Win ratio of ABC-compliant (all three components) vs. ABC-not-compliant was 1.57 (1.35-1.83), P < 0.001. When adding time in therapeutic range (TTR) data for compliant criteria for those who were on warfarin, the win ratio increased to 2.28 (1.89-2.75), P < 0.001. The A-compliant group (plus TTR data), B-compliant, and C-compliant had the win ratio of 1.81 (1.51-2.12), 1.82 (1.53-2.16), and 1.39 (1.18-1.62), all P < 0.001, compared to not compliant group. CONCLUSION: Management of AF patients according to each component of the ABC pathway is associated with better clinical outcomes compared to those non-compliant to ABC pathway. This finding underscores the importance of a holistic management approach strategy for AF patients.

Analysis of composite time-to-event endpoints in cardiovascular outcome trials.

Composite time-to-event endpoints are commonly used in cardiovascular outcome trials. For example, the IMPROVE-IT trial comparing ezetimibe+simvastatin to placebo+simvastatin in 18,144 patients with acute coronary syndrome used a primary composite endpoint with five component outcomes: (1) cardiovascular death, (2) non-fatal stroke, (3) non-fatal myocardial infarction, (4) coronary revascularization ≥30 days after randomization, and (5) unstable angina requiring hospitalization. In such settings, the traditional analysis compares treatments using the observed time to the occurrence of the first (i.e. earliest) component outcome for each patient. This approach ignores information for subsequent outcome(s), possibly leading to reduced power to demonstrate the benefit of the test versus the control treatment. We use real data examples and simulations to contrast the traditional approach with several alternative approaches that use data for all the intra-patient component outcomes, not just the first.

A win ratio-based framework to combine multiple clinical endpoints in exploratory basket trials.

In contemporary exploratory phase of oncology drug development, there has been an increasing interest in evaluating investigational drug or drug combination in multiple tumor indications in a single basket trial to expedite drug development. There has been extensive research on more efficiently borrowing information across tumor indications in early phase drug development including Bayesian hierarchical modeling and the pruning-and-pooling methods. Despite the fact that the Go/No-Go decision for subsequent Phase 2 or Phase 3 trial initiation is almost always a multi-facet consideration, the statistical literature of basket trial design and analysis has largely been limited to a single binary endpoint. In this paper we explore the application of considering clinical priorities of multiple endpoints based on matched win ratio to the basket trial design and analysis. The control arm data will be simulated for each tumor indication based on the corresponding null assumptions that could be heterogeneous across tumor indications. The matched win ratio matching on the tumor indication can be performed for individual tumor indication, pooled data, or the pooled data after pruning depending on whether an individual evaluation or a simple pooling or a pruning-and-pooling method is used. We conduct the simulation studies to evaluate the performance of proposed win ratio-based framework and the results suggest the proposed framework could provide desirable operating characteristics.

Win-loss parameters for right-censored event data, with application to recurrent events.

The win ratio has become a popular method for comparing multiple event data between two groups in clinical cohort studies. The win ratio compares the event data in prioritized order, where the first prioritized event is death and a typical example for the second prioritized event is hospitalization. Literature is sparse on inference for win and loss parameters, including the win ratio, for censored event data. Inference for two prioritized censored event times has been developed for independent right-censoring. Many clinical studies include recurrent event data such as hospitalizations. In this article, we suggest inference for win-loss parameters for death and a recurrent event outcome under independent right-censoring. The small sample properties of the proposed method are studied in a simulation study showing that the variance formula is accurate even for small samples. The method is applied on a data set from a randomized clinical trial.

The maraca plot: A novel visualization of hierarchical composite endpoints.

BACKGROUND: Hierarchical composite endpoints are complex endpoints combining outcomes of different types and different clinical importance into an ordinal outcome that prioritizes the clinically most important (e.g. most severe) event of a patient. Hierarchical composite endpoint can be analysed with the win odds, an adaptation of win ratio to include ties. One of the difficulties in interpreting hierarchical composite endpoint is the lack of proper tools for visualizing the treatment effect captured by hierarchical composite endpoint, given the complex nature of the endpoint which combines events of different types. METHODS: Hierarchical composite endpoints usually combine time-to-event outcomes and continuous outcomes into a composite; hence, it is important to capture not only the shift from more severe categories to less severe categories in the active group in comparison to the control group (as in any ordinal endpoint), but also changes occurring within each category. We introduce the novel maraca plot which combines violin plots (with nested box plots) to visualize the density of the distribution of the continuous outcome and Kaplan-Meier plots for time-to-event outcomes into a comprehensive visualization. CONCLUSION: The novel maraca plot is suggested for visualization of hierarchical composite endpoints consisting of several time-to-event outcomes and a continuous outcome. It has a very simple structure and therefore easily communicates both the overall treatment effect and the effect on individual components.

A win ratio approach for comparing crossing survival curves in clinical trials.

Many clinical trials include time-to-event or survival data as an outcome. To compare two survival distributions, the log-rank test is often used to produce a P-value for a statistical test of the null hypothesis that the two survival curves are identical. However, such a P-value does not provide the magnitude of the difference between the curves regarding the treatment effect. As a result, the P-value is often accompanied by an estimate of the hazard ratio from the proportional hazards model or Cox model as a measurement of treatment difference. However, one of the most important assumptions for Cox model is that the hazard functions for the two treatment groups are proportional. When the hazard curves cross, the Cox model could lead to misleading results and the log-rank test could also perform poorly. To address the problem of crossing curves in survival analysis, we propose the use of the win ratio method put forward by Pocock et al. as an estimand for analysing such data. The subjects in the test and control treatment groups are formed into all possible pairs. For each pair, the test treatment subject is labelled a winner or a loser if it is known who had the event of interest such as death. The win ratio is the total number of winners divided by the total number of losers and its standard error can be estimated using Bebu and Lachin method. Using real trial datasets and Monte Carlo simulations, this study investigates the power and type I error and compares the win ratio method with the log-rank test and Cox model under various scenarios of crossing survival curves with different censoring rates and distribution parameters. The results show that the win ratio method has similar power as the log-rank test and Cox model to detect the treatment difference when the assumption of proportional hazards holds true, and that the win ratio method outperforms log-rank test and Cox model in terms of power to detect the treatment difference when the survival curves cross.

The win odds: statistical inference and regression.

Generalized pairwise comparisons and win statistics (i.e., win ratio, win odds and net benefit) are advantageous in analyzing and interpreting a composite of multiple outcomes in clinical trials. An important limitation of these statistics is their inability to adjust for covariates other than by stratified analysis. Because the win ratio does not account for ties, the win odds, a modification that includes ties, has attracted attention. We review and combine information on the win odds to articulate the statistical inferences for the win odds. We also show alternative variance estimators based on the exact permutation and bootstrap as well as statistical inference via the probabilistic index. Finally, we extend multiple-covariate regression probabilistic index models to the win odds with a univariate outcome. As an illustration we apply the regression models to the data in the CHARM trial.

Adjusted win ratio using the inverse probability of treatment weighting.

The win ratio method has been increasingly applied in the design and analysis of clinical trials. However, the win ratio method is a univariate approach that does not allow for adjusting for baseline imbalances in covariates, although a stratified win ratio can be calculated when the number of strata is small. This paper proposes an adjusted win ratio to control for such imbalances by inverse probability of treatment weighting (IPTW) method. We derive the adjusted win ratio with its variance and suggest three IPTW adjustments: IPTW-average treatment effect (IPTW-ATE), stabilized IPTW-ATE (SIPTW-ATE) and IPTW-average treatment effect in the treated (IPTW-ATT). The proposed adjusted methods are applied to analyse a composite outcome in the CHARM trial. The statistical properties of the methods are assessed through simulations. Results show that adjusted win ratio methods can correct the win ratio for covariate imbalances at baseline. Simulation results show that the three proposed adjusted win ratios have similar power to detect the treatment difference and have slightly lower power than the corresponding adjusted Cox models when the assumption of proportional hazards holds true but have consistently higher power than adjusted Cox models when the proportional hazard assumption is violated.

Evidence synthesis analysis with prioritized benefit outcomes in oncology clinical trials.

Overall survival, progression-free survival, objective response/complete response, and duration of (complete) response are frequently used as the primary and secondary efficacy endpoints for designs and analyses of oncology clinical trials. However, these endpoints are typically analyzed separately. In this article, we introduce an evidence synthesis approach to prioritize the benefit outcomes by applying the generalized pairwise comparisons (GPC) method, and use win statistics (win ratio, win odds and net benefit) to quantify treatment benefit. Under the framework of GPC, the main advantage of this evidence synthesis approach is the ability to combine relevant outcomes of various types into a single summary statistic without relying on any parametric assumptions. It is particularly relevant since health authorities and the pharmaceutical industry are increasingly incorporating structured quantitative methodologies in their benefit-risk assessment. We apply this evidence synthesis approach to an oncology phase 3 study in first-line renal cell carcinoma to assess the overall effect of an investigational treatment by ranking the most clinically relevant endpoints in cancer drug development. This application and a simulation study demonstrate that the proposed approach can synthesize the evidence of treatment effect from multiple prioritized benefit outcomes, and has substantial advantage over conventional methods that analyze each individual endpoint separately. We also introduce a newly developed R package WINS for statistical inference based on win statistics.

The stratified win statistics (win ratio, win odds, and net benefit).

The win odds and the net benefit are related directly to each other and indirectly, through ties, to the win ratio. These three win statistics test the same null hypothesis of equal win probabilities between two groups. They provide similar p-values and powers, because the Z-values of their statistical tests are approximately equal. Thus, they can complement one another to show the strength of a treatment effect. In this article, we show that the estimated variances of the win statistics are also directly related regardless of ties or indirectly related through ties. Since its introduction in 2018, the stratified win ratio has been applied in designs and analyses of clinical trials, including Phase III and Phase IV studies. This article generalizes the stratified method to the win odds and the net benefit. As a result, the relations of the three win statistics and the approximate equivalence of their statistical tests also hold for the stratified win statistics.

Designing a longitudinal clinical trial based on a composite endpoint: Sample size, monitoring, and adaptation.

Longitudinal clinical trials are often designed to compare treatments on the basis of multiple outcomes. For example in the case of cardiac trials, the outcomes of interest include mortality as well as cardiac events and hospitalization. For a COVID-19 trial, the outcomes of interest include mortality, time on ventilator, and time in hospital. Earlier work by these authors proposed a non-parametric test based on a composite of multiple endpoints referred to as the Finkelstein-Schoenfeld (FS) test (Finkelstein and Schoenfeld. Stat Med. 1999;18(11):1341-1354.). More recently, an estimate of the treatment comparison based on multiple endpoints (related to the FS test) was proposed (Pocock et al. Eur Heart J. 2011;33(2):176-182.). This estimate, which summarized the ratio of the number of patients who fared better vs worse on the experimental arm was coined the win ratio. The aim of this article is to provide guidance in the design of a trial that will use the FS test or the win ratio. The issues that will be considered are the sample size, sequential monitoring, and adaptive designs.

Stratified proportional win-fractions regression analysis.

The recently proposed proportional win-fractions (PW) model extends the two-sample win ratio analysis of prioritized composite endpoints to regression. Its proportionality assumption ensures that the covariate-specific win ratios are invariant to the follow-up time. However, this assumption is strong and may not be satisfied by every covariate in the model. We develop a stratified PW model that adjusts for certain prognostic factors without setting them as covariates, thus bypassing the proportionality requirement. We formulate the stratified model based on pairwise comparisons within each stratum, with a common win ratio across strata modeled as a multiplicative function of the covariates. Correspondingly, we construct an estimating function for the regression coefficients in the form of an incomplete U $$ U $$ -statistic consisting of within-stratum pairs. Two types of asymptotic variance estimators are developed depending on the number of strata relative to the sample size. This in particular allows valid inference even when the strata are extremely small, such as with matched pairs. Simulation studies in realistic settings show that the stratified model outperforms the unstratified version in robustness and efficiency. Finally, real data from a major cardiovascular trial are analyzed to illustrate the potential benefits of stratification. The proposed methods are implemented in the R package WR, publicly available on the Comprehensive R Archive Network (CRAN).

Win ratio approach for analyzing composite time-to-event endpoint with opposite treatment effects in its components.

There is an increasing interest in the use of win ratio with composite time-to-event due to its flexibility in combining component endpoints. Exploring this flexibility further, one interesting question is in assessing the impact when there is a difference in treatment effect in the component endpoints. For example, the active treatment may prolong the time to occurrence of the negative event such as death or ventilation; meanwhile, the treatment effect may also shorten the time to achieving positive events, such as recovery or improvement. Notably, this portrays a situation where the treatment effect on time to recovery is in a different direction of benefit compared to the time to ventilation or death. Under such circumstances, if a single endpoint is used, the benefit gained for other individual outcomes is not counted and is diminished. As consequence, the study may need a larger sample size to detect a significant effect of treatment. Such a scenario can be handled by win ratio in a novel way by ranking component events, which is different from the usual composite endpoint approach such as time-to-first event. To evaluate how the different directions of treatment effect on component endpoints will impact the win ratio analysis, we use a Clayton copula-based bivariate survival simulation to investigate the correlation of component time-to-event. Through simulation, we found that compared to the marginal model using single endpoints, the win ratio analysis on composite endpoint performs better, especially when the correlation between two events is weak. Then, we applied the methodology to an infectious disease progression simulated study motivated by COVID-19. The application demonstrates that the win ratio approach offers advantages in empirical power compared to the traditional Cox proportional hazard approach when there is a difference in treatment effect in the marginal events.

A class of proportional win-fractions regression models for composite outcomes.

The win ratio is gaining traction as a simple and intuitive approach to analysis of prioritized composite endpoints in clinical trials. To extend it from two-sample comparison to regression, we propose a novel class of semiparametric models that includes as special cases both the two-sample win ratio and the traditional Cox proportional hazards model on time to the first event. Under the assumption that the covariate-specific win and loss fractions are proportional over time, the regression coefficient is unrelated to the censoring distribution and can be interpreted as the log win ratio resulting from one-unit increase in the covariate. U-statistic estimating functions, in the form of an arbitrary covariate-specific weight process integrated by a pairwise residual process, are constructed to obtain consistent estimators for the regression parameter. The asymptotic properties of the estimators are derived using uniform weak convergence theory for U-processes. Visual inspection of a "score" process provides useful clues as to the plausibility of the proportionality assumption. Extensive numerical studies using both simulated and real data from a major cardiovascular trial show that the regression methods provide valid inference on covariate effects and outperform the two-sample win ratio in both efficiency and robustness. The proposed methodology is implemented in the R-package WR, publicly available from the Comprehensive R Archive Network (CRAN).

Win odds: An adaptation of the win ratio to include ties.

The win ratio, a recently proposed measure for comparing the benefit of two treatment groups, allows ties in the data but ignores ties in the inference. In this article, we highlight some difficulties that this can lead to, and we propose to focus on the win odds instead, a modification of the win ratio which takes ties into account. We construct hypothesis tests and confidence intervals for the win odds, and we investigate their properties through simulations and in a case study. We conclude that the win odds should be preferred over the win ratio.