The impact of allocation bias on test decisions in clinical trials with multiple endpoints using multiple testing strategies.

BACKGROUND: Considering multiple endpoints in clinical trials provide a more comprehensive understanding of treatment effects and may lead to increased power or reduced sample size, which may be beneficial in rare diseases. Besides the small sample sizes, allocation bias is an issue that affects the validity of these trials. We investigate the impact of allocation bias on testing decisions in clinical trials with multiple endpoints and offer a tool for selecting an appropriate randomization procedure (RP). METHODS: We derive a model for quantifying the effect of allocation bias depending on the RP in the case of two-arm parallel group trials with continuous multiple endpoints. We focus on two approaches to analyze multiple endpoints, either the Sidák procedure to show efficacy in at least one endpoint and the all-or-none procedure to show efficacy in all endpoints. RESULTS: To evaluate the impact of allocation bias on the test decision we propose a biasing policy for multiple endpoints. The impact of allocation on the test decision is measured by the family-wise error rate of the Sidák procedure and the type I error rate of the all-or-none procedure. Using the biasing policy we derive formulas to calculate these error rates. In simulations we show that, for the Sidák procedure as well as for the all-or-none procedure, allocation bias leads to inflation of the mean family-wise error and mean type I error, respectively. The strength of this inflation is affected by the choice of the RP. CONCLUSION: Allocation bias should be considered during the design phase of a trial to increase validity. The developed methodology is useful for selecting an appropriate RP for a clinical trial with multiple endpoints to minimize allocation bias effects.

Clinical trial designs and endpoints.

Trials should be designed with consideration of the individual disease context and research question. Many different approaches may be justified. In this chapter, we therefore consider some of the principal components of trial design in general and within the context of the emerging field of gene and cell therapies. Many aspects of developing a trial protocol require striking a balance between scientific rigor and practicalities for which the voice of patients and their families should form an integral part. We outline the importance of the acceptability of trial designs to participants, the determination of a target population and eligibility criteria, stratification methods that ensure a balanced control of variance across the trial, adequate controls to answer research questions including considerations of placebo allocation, blinding, and endpoints.

Increasing Power in Phase III Oncology Trials With Multivariable Regression: An Empirical Assessment of 535 Primary End Point Analyses.

PURPOSE: A previous study demonstrated that power against the (unobserved) true effect for the primary end point (PEP) of most phase III oncology trials is low, suggesting an increased risk of false-negative findings in the field of late-phase oncology. Fitting models with prognostic covariates is a potential solution to improve power; however, the extent to which trials leverage this approach, and its impact on trial interpretation at scale, is unknown. To that end, we hypothesized that phase III trials using multivariable PEP analyses are more likely to demonstrate superiority versus trials with univariable analyses. METHODS: PEP analyses were reviewed from trials registered on ClinicalTrials.gov. Adjusted odds ratios (aORs) were calculated by logistic regressions. RESULTS: Of the 535 trials enrolling 454,824 patients, 69% (n = 368) used a multivariable PEP analysis. Trials with multivariable PEP analyses were more likely to demonstrate PEP superiority (57% [209 of 368] v 42% [70 of 167]; aOR, 1.78 [95% CI, 1.18 to 2.72]; P = .007). Among trials with a multivariable PEP model, 16 conditioned on covariates and 352 stratified by covariates. However, 108 (35%) of 312 trials with stratified analyses lost power by categorizing a continuous variable, which was especially common among immunotherapy trials (aOR, 2.45 [95% CI, 1.23 to 4.92]; P = .01). CONCLUSION: Trials increasing power by fitting multivariable models were more likely to demonstrate PEP superiority than trials with unadjusted analysis. Underutilization of conditioning models and empirical power loss associated with covariate categorization required by stratification were identified as barriers to power gains. These findings underscore the opportunity to increase power in phase III trials with conventional methodology and improve patient access to effective novel therapies.

Commentary on Chen et al. (2022): The need for continued methodological research on leveraging information in secondary endpoints for more efficient RCTs.

Chen et al. (2022) recently proposed a set of estimating equations that incorporate data from secondary endpoints to improve precision in parameter estimates related to a primary endpoint. We were motivated to translate their methodology to the context of randomized controlled trials to gain precision in treatment effect estimation using data from secondary endpoints. Our results suggest that this estimator cannot gain efficiency in this context because of random treatment assignment, especially when there is a treatment effect on secondary endpoints, and that further methodological work in this area is needed.

Improving the clinical meaning of surrogate endpoints: An empirical assessment of clinical progression in phase III oncology trials.

Disease progression in clinical trials is commonly defined by radiologic measures. However, clinical progression may be more meaningful to patients, may occur even when radiologic criteria for progression are not met, and often requires a change in therapy in clinical practice. The objective of this study was to determine the utilization of clinical progression criteria within progression-based trial endpoints among phase III trials testing systemic therapies for metastatic solid tumors. The primary manuscripts and protocols of phase III trials were reviewed for whether clinical events, such as refractory pain, tumor bleeding, or neurologic compromise, could constitute a progression event. Univariable logistic regression computed odds ratios (OR) and 95% CI for associations between trial-level covariates and clinical progression. A total of 216 trials enrolling 148,190 patients were included, with publication dates from 2006 through 2020. A major change in clinical status was included in the progression criteria of 13% of trials (n = 27), most commonly as a secondary endpoint (n = 22). Only 59% of trials (n = 16) reported distinct clinical progression outcomes that constituted the composite surrogate endpoint. Compared with other disease sites, genitourinary trials were more likely to include clinical progression definitions (16/33 [48%] vs. 11/183 [6%]; OR, 14.72; 95% CI, 5.99 to 37.84; p < .0001). While major tumor-related clinical events were seldom considered as disease progression events, increased attention to clinical progression may improve the meaningfulness and clinical applicability of surrogate endpoints for patients with metastatic solid tumors.

Familywise error for multiple time-to-event endpoints in a group sequential design.

We investigate the familywise error rate (FWER) for time-to-event endpoints evaluated using a group sequential design with a hierarchical testing procedure for secondary endpoints. We show that, in this setup, the correlation between the log-rank test statistics at interim and at end of study is not congruent with the canonical correlation derived for normal-distributed endpoints. We show, both theoretically and by simulation, that the correlation also depends on the level of censoring, the hazard rates of the endpoints, and the hazard ratio. To optimize operating characteristics in this complex scenario, we propose a simulation-based method to assess the FWER which, better than the alpha-spending approach, can inform the choice of critical values for testing secondary endpoints.

Design of randomized clinical trials with a binary endpoint: Conditional versus unconditional analyses of a two-by-two table.

When designing a randomized clinical trial to compare two treatments, the sample size required to have desired power with a specified type 1 error depends on the hypothesis testing procedure. With a binary endpoint (e.g., response), the trial results can be displayed in a 2 × 2 table. If one does the analysis conditional on the number of positive responses, then using Fisher's exact test has an actual type 1 error less than or equal to the specified nominal type 1 error. Alternatively, one can use one of many unconditional "exact" tests that also preserve the type 1 error and are less conservative than Fisher's exact test. In particular, the unconditional test of Boschloo is always at least as powerful as Fisher's exact test, leading to smaller required sample sizes for clinical trials. However, many statisticians have argued over the years that the conditional analysis with Fisher's exact test is the only appropriate procedure. Since having smaller clinical trials is an extremely important consideration, we review the general arguments given for the conditional analysis of a 2 × 2 table in the context of a randomized clinical trial. We find the arguments not relevant in this context, or, if relevant, not completely convincing, suggesting the sample-size advantage of the unconditional tests should lead to their recommended use. We also briefly suggest that since designers of clinical trials practically always have target null and alternative response rates, there is the possibility of using this information to improve the power of the unconditional tests.

The role of quality of life data as an endpoint for collecting real-world evidence within geroscience clinical trials.

With geroscience research evolving at a fast pace, the need arises for human randomized controlled trials to assess the efficacy of geroprotective interventions to prevent age-related adverse outcomes, disease, and mortality in normative aging cohorts. However, to confirm efficacy requires a long-term and costly approach as time to the event of morbidity and mortality can be decades. While this could be circumvented using sensitive biomarkers of aging, current molecular, physiological, and digital endpoints require further validation. In this review, we discuss how collecting real-world evidence (RWE) by obtaining health data that is amenable for collection from large heterogeneous populations in a real-world setting can help speed up validation of geroprotective interventions. Further, we propose inclusion of quality of life (QoL) data as a biomarker of aging and candidate endpoint for geroscience clinical trials to aid in distinguishing healthy from unhealthy aging. We highlight how QoL assays can aid in accelerating data collection in studies gathering RWE on the geroprotective effects of repurposed drugs to support utilization within healthy longevity medicine. Finally, we summarize key metrics to consider when implementing QoL assays in studies, and present the short-form 36 (SF-36) as the most well-suited candidate endpoint.

Development of a Core Outcome Set for Dysphagia Interventions in Parkinson's disease (COS-DIP): study protocol.

INTRODUCTION: Current clinical trials on swallowing disorders (dysphagia) in Parkinson's disease (PD) apply a high variety of outcomes and different outcome measures making comparative effectiveness research challenging. Furthermore, views of patients and dysphagia clinicians when selecting trial outcomes have not been considered in the past, thus study results may have little importance to them. This study aims to develop an agreed standardised Core Outcome Set for Dysphagia Interventions in Parkinson's disease (COS-DIP), systematically measured and reported as a minimum for all clinical trials. It will also comprise guidance on outcome definitions, outcome measures and time points of measurement. METHODS AND ANALYSIS: The COS-DIP development will comprise five stages following established methodology: (1) a recent scoping review on all applied outcomes, their definitions, methods and time points of measurement in clinical trials in dysphagia in PD, (2) online surveys and focus groups with clinicians, patients, caregivers and family members to identify outcomes that are important to them, (3) an identified list of outcomes based on results of stage 1 and 2, (4) three round online Delphi survey with up to 200 key stakeholders to determine core outcomes and (5) two online consensus meetings with up to 40 representative key stakeholders to agree on all outcomes, definitions, methods and time points of measurement in the final COS-DIP. ETHICS AND DISSEMINATION: Full ethical approval was obtained from the Research Ethics Committee, School of Linguistic, Speech and Communication Sciences, Trinity College Dublin, on 15 May 2023 (HT27). Dissemination of the COS-DIP will be enhanced through presentations at (inter-) national conferences and through peer-reviewed, open access publications of related manuscripts. Lay and professional information sheets and infographics will be circulated through relevant patient and professional organisations and networks. TRIAL REGISTRATION NUMBER: The COS-DIP study was registered prospectively with the Core Outcome Measures in Effectiveness Trials (COMET) database on 24 September 2021 (www.comet-initiative.org/Studies/Details/1942).

Use of win time for ordered composite endpoints in clinical trials.

Consider the choice of outcome for overall treatment benefit in a clinical trial which measures the first time to each of several clinical events. We describe several new variants of the win ratio that incorporate the time spent in each clinical state over the common follow-up, where clinical state means the worst clinical event that has occurred by that time. One version allows restriction so that death during follow-up is most important, while time spent in other clinical states is still accounted for. Three other variants are described; one is based on the average pairwise win time, one creates a continuous outcome for each participant based on expected win times against a reference distribution and another that uses the estimated distributions of clinical state to compare the treatment arms. Finally, a combination testing approach is described to give robust power for detecting treatment benefit across a broad range of alternatives. These new methods are designed to be closer to the overall treatment benefit/harm from a patient's perspective, compared to the ordinary win ratio. The new methods are compared to the composite event approach and the ordinary win ratio. Simulations show that when overall treatment benefit on death is substantial, the variants based on either the participants' expected win times (EWTs) against a reference distribution or estimated clinical state distributions have substantially higher power than either the pairwise comparison or composite event methods. The methods are illustrated by re-analysis of the trial heart failure: a controlled trial investigating outcomes of exercise training.

An information-theoretic approach for the assessment of a continuous outcome as a surrogate for a binary true endpoint based on causal inference: Application to vaccine evaluation.

Within the causal association paradigm, a method is proposed to assess the validity of a continuous outcome as a surrogate for a binary true endpoint. The methodology is based on a previously introduced information-theoretic definition of surrogacy and has two main steps. In the first step, a new model is proposed to describe the joint distribution of the potential outcomes associated with the putative surrogate and the true endpoint of interest. The identifiability issues inherent to this type of models are handled via sensitivity analysis. In the second step, a metric of surrogacy new to this setting, the so-called individual causal association is presented. The methodology is studied in detail using theoretical considerations, some simulations, and data from a randomized clinical trial evaluating an inactivated quadrivalent influenza vaccine. A user-friendly R package Surrogate is provided to carry out the evaluation exercise.

Consideration of stratification in confirmatory trials with time-to-event endpoint.

Stratification in randomization and analysis are widely employed to balance treatment groups in clinical trials. However, the potential power loss due to under-stratification or over-stratification has not been thoroughly evaluated in the typical setting of confirmatory clinical trials. In cases where there are too many strata and some have small sample sizes or a small number of events, it is common practice to combine these small strata during analysis. However, there is a lack of guidance on how those small strata should be combined. This paper presents extensive simulation studies to evaluate the impact of under-stratification or over-stratification on the power of survival analysis and the estimate of hazard ratio using stratified log-rank test and Cox PH model, respectively. The difference in power between stratified and unstratified log-rank tests is also investigated under different scenarios. Our results suggest that failing to consider prognostic stratification factors with strong effects, and/or accounting for non-prognostic factors such as noise and predictive factors, may reduce the power of the stratified log-rank test. Additionally, methods of combining small strata are explored and compared.

Using short-term endpoints to improve interim decision making and trial duration in two-stage phase II trials with nested binary endpoints.

In oncology, phase II clinical trials are often planned as single-arm two-stage designs with a binary endpoint, for example, progression-free survival after 12 months, and the option to stop for futility after the first stage. Simon's two-stage design is a very popular approach but depending on the follow-up time required to measure the patients' outcomes the trial may have to be paused undesirably long. To shorten this forced interruption, it was proposed to use a short-term endpoint for the interim decision, such as progression-free survival after 3 months. We show that if the assumptions for the short-term endpoint are misspecified, the decision-making in the interim can be misleading, resulting in a great loss of statistical power. For the setting of a binary endpoint with nested measurements, such as progression-free survival, we propose two approaches that utilize all available short-term and long-term assessments of the endpoint to guide the interim decision. One approach is based on conditional power and the other is based on Bayesian posterior predictive probability of success. In extensive simulations, we show that both methods perform similarly, when appropriately calibrated, and can greatly improve power compared to the existing approach in settings with slow patient recruitment. Software code to implement the methods is made publicly available.

Assessment highlights need for improvement in standards of development of core outcome sets for rare genetic diseases.

OBJECTIVES: A rare disease is classified as such if it affects less than one person in 2,000. The Core Outcome Set STandards for Development (COS-STAD) is a set of standards that represent the minimum recommendations to be considered in the process of core outcome set (COS) development. The aim of this study was to provide a baseline assessment of COS development standards for rare genetic diseases. STUDY DESIGN AND SETTING: Core Outcome Measures in Effectiveness Trials (COMET) database contains nearly 400 published COS studies according to the latest systematic review. Studies focusing on COS development for rare genetic diseases were eligible for inclusion and were assessed by two independent evaluators. RESULTS: Nine COS studies were included in the analysis. Eight different rare genetic diseases were investigated. None of the studies met all the standards for development. The number of standards met ranged from 6 to 10, and the median was 7. CONCLUSION: This study is the first study to assess COS-STAD for rare genetic diseases, and it highlights a great need for improvement. First in terms of numbers of rare diseases considered for COS developments, second in methodology, particularly regarding the consensus process, and third in reporting of the COS development studies.

The development of a core outcome set for clinical effectiveness studies of bordered foam dressings in the treatment of complex wounds.

AIM: The aim of this project was to develop a core outcome set (COS) for clinical effectiveness studies of bordered foam dressings in the treatment of complex wounds. METHODS: The research project followed the Core Outcome Measures in Effectiveness Trials (COMET) initiative and consisted of two phases. The first phase prepared the background and process, while the second phase had three steps: outcome list generation via systematic review and qualitative study, Delphi consensus study, and consensus meeting. The study has been registered in the Core Outcome Measures in Effectiveness Trials database. RESULTS: The systematic review resulted in 82 outcomes and 20 additional outcomes were obtained during the interviews. After refinement, 111 panellists from 23 countries rated a list of 51 outcomes. In the following consensus meeting, six outcomes were prioritized to be included in the core outcome set. After the consensus meeting, a patient-reported outcome was added to the core outcome set. CONCLUSION: The COS for evaluating the effectiveness of bordered foam dressings in treating complex wounds includes 7 outcomes: "ability to stay in place", "leakage", "pain", "dressing related periwound skin changes", "change in wound size over time", and "overall satisfaction". These identified outcomes are correlated with contemporary bioengineering testing and evaluation methods for dressing performance, which underpins the need for a close multidisciplinary collaboration to advance the field of wound dressings. The outcome 'overall satisfaction' reflects the impact of complex wounds and their treatment on a patient's daily life. The use of these outcomes is recommended to improve data synthesis and promote evidence-based practice. Future developments in COS development involve creating measurement instruments and relevant endpoints for these outcomes.

Omnibus test for restricted mean survival time based on influence function.

The restricted mean survival time (RMST), which evaluates the expected survival time up to a pre-specified time point τ, has been widely used to summarize the survival distribution due to its robustness and straightforward interpretation. In comparative studies with time-to-event data, the RMST-based test has been utilized as an alternative to the classic log-rank test because the power of the log-rank test deteriorates when the proportional hazards assumption is violated. To overcome the challenge of selecting an appropriate time point τ, we develop an RMST-based omnibus Wald test to detect the survival difference between two groups throughout the study follow-up period. Treating a vector of RMSTs at multiple quantile-based time points as a statistical functional, we construct a Wald χ2 test statistic and derive its asymptotic distribution using the influence function. We further propose a new procedure based on the influence function to estimate the asymptotic covariance matrix in contrast to the usual bootstrap method. Simulations under different scenarios validate the size of our RMST-based omnibus test and demonstrate its advantage over the existing tests in power, especially when the true survival functions cross within the study follow-up period. For illustration, the proposed test is applied to two real datasets, which demonstrate its power and applicability in various situations.

Development of core outcome sets of Food for Special Medical Purposes designed for type 2 diabetes mellitus: a study protocol.

BACKGROUND: The Chinese government stipulates all food for special medical purposes (FSMP) designed for specific diseases to be tested in clinical trials before approving it for registration. The process of developing core outcome sets (COSs), the minimum sets of outcomes supposed to be measured and reported, provides an economical and practical option for stakeholders to communicate and cooperate in conducting clinical trials as well as in reporting FSMP outcomes. This study uses type 2 diabetes mellitus (T2DM) as an example to develop COS for clinical trials of FSMP. METHODS: The COS for FSMP-T2DM will be divided into 3 phases and developed following COS-STAP and COS-STAD: (1) Generate a list of relevant outcomes identified from a systematic review, in which information sources will mainly include published studies, regulatory documentation, and qualitative interviews of stakeholders. The identified outcomes will be categorized using a conceptual framework and formatted into the first round of the Delphi survey questionnaire items. (2) At least 2 rounds of Delphi surveys will be performed among stakeholders to create the COS for FSMP-T2DM. Patients, clinical dietitians, physicians, COS researchers, journal editors, FSMP manufacturers, and regulatory representatives will be invited to score each outcome from aspects of importance. (3) Hold a face-to-face or online consensus meeting to refine the content of the COS for FSMP-T2DM. Key stakeholders will be invited to attend the meeting to discuss and agree on the final COS. DISCUSSION: We have prepared an alternative solution of the Likert scale selection, Delphi survey rounds, scoring group, and consensus definitions in case of an unexpected situation. TRIAL REGISTRATION: COMET (1547). Registered on March 23, 2020.

A note on familywise error rate for a primary and secondary endpoint.

Hung et al. (2007) considered the problem of controlling the type I error rate for a primary and secondary endpoint in a clinical trial using a gatekeeping approach in which the secondary endpoint is tested only if the primary endpoint crosses its monitoring boundary. They considered a two-look trial and showed by simulation that the naive method of testing the secondary endpoint at full level α at the time the primary endpoint reaches statistical significance does not control the familywise error rate at level α. Tamhane et al. (2010) derived analytic expressions for familywise error rate and power and confirmed the inflated error rate of the naive approach. Nonetheless, many people mistakenly believe that the closure principle can be used to prove that the naive procedure controls the familywise error rate. The purpose of this note is to explain in greater detail why there is a problem with the naive approach and show that the degree of alpha inflation can be as high as that of unadjusted monitoring of a single endpoint.

Design of paediatric trials with benefit-risk endpoints using a composite score of adverse events of interest (AEI) and win-statistics.

A fundamental problem in the regulatory evaluation of a therapy is assessing whether the benefit outweighs the associated risks. This work proposes designing a trial that assesses a composite endpoint consisting of benefit and risk, hence, making the core of the design of the study, to assess benefit and risk. The proposed benefit risk measure consists of efficacy measure(s) and a risk measure that is based on a composite score obtained from pre-defined adverse events of interest (AEI). This composite score incorporates full aspects of adverse events of interest (i.e. the incidence, severity, and duration of the events). We call this newly proposed score the AEI composite score. After specifying the priorities between the components of the composite endpoint, a win-statistic (i.e. win ratio, win odds, or net benefit) is used to assess the difference between treatments in this composite endpoint. The power and sample size requirements of such a trial design are explored via simulation. Finally, using Dupixent published adult study results, we show how we can design a paediatric trial where the primary outcome is a composite of prioritized outcomes consisting of efficacy endpoints and the AEI composite score endpoint. The resulting trial design can potentially substantially reduce sample size compared to a trial designed to assess the co-primary efficacy endpoints, therefore it may address the challenge of slow enrollment and patient availability for paediatric studies.