Increasing Power in Phase III Oncology Trials With Multivariable Regression: An Empirical Assessment of 535 Primary End Point Analyses.

PURPOSE: A previous study demonstrated that power against the (unobserved) true effect for the primary end point (PEP) of most phase III oncology trials is low, suggesting an increased risk of false-negative findings in the field of late-phase oncology. Fitting models with prognostic covariates is a potential solution to improve power; however, the extent to which trials leverage this approach, and its impact on trial interpretation at scale, is unknown. To that end, we hypothesized that phase III trials using multivariable PEP analyses are more likely to demonstrate superiority versus trials with univariable analyses. METHODS: PEP analyses were reviewed from trials registered on ClinicalTrials.gov. Adjusted odds ratios (aORs) were calculated by logistic regressions. RESULTS: Of the 535 trials enrolling 454,824 patients, 69% (n = 368) used a multivariable PEP analysis. Trials with multivariable PEP analyses were more likely to demonstrate PEP superiority (57% [209 of 368] v 42% [70 of 167]; aOR, 1.78 [95% CI, 1.18 to 2.72]; P = .007). Among trials with a multivariable PEP model, 16 conditioned on covariates and 352 stratified by covariates. However, 108 (35%) of 312 trials with stratified analyses lost power by categorizing a continuous variable, which was especially common among immunotherapy trials (aOR, 2.45 [95% CI, 1.23 to 4.92]; P = .01). CONCLUSION: Trials increasing power by fitting multivariable models were more likely to demonstrate PEP superiority than trials with unadjusted analysis. Underutilization of conditioning models and empirical power loss associated with covariate categorization required by stratification were identified as barriers to power gains. These findings underscore the opportunity to increase power in phase III trials with conventional methodology and improve patient access to effective novel therapies.

Commentary on Chen et al. (2022): The need for continued methodological research on leveraging information in secondary endpoints for more efficient RCTs.

Chen et al. (2022) recently proposed a set of estimating equations that incorporate data from secondary endpoints to improve precision in parameter estimates related to a primary endpoint. We were motivated to translate their methodology to the context of randomized controlled trials to gain precision in treatment effect estimation using data from secondary endpoints. Our results suggest that this estimator cannot gain efficiency in this context because of random treatment assignment, especially when there is a treatment effect on secondary endpoints, and that further methodological work in this area is needed.

Familywise error for multiple time-to-event endpoints in a group sequential design.

We investigate the familywise error rate (FWER) for time-to-event endpoints evaluated using a group sequential design with a hierarchical testing procedure for secondary endpoints. We show that, in this setup, the correlation between the log-rank test statistics at interim and at end of study is not congruent with the canonical correlation derived for normal-distributed endpoints. We show, both theoretically and by simulation, that the correlation also depends on the level of censoring, the hazard rates of the endpoints, and the hazard ratio. To optimize operating characteristics in this complex scenario, we propose a simulation-based method to assess the FWER which, better than the alpha-spending approach, can inform the choice of critical values for testing secondary endpoints.

Design of randomized clinical trials with a binary endpoint: Conditional versus unconditional analyses of a two-by-two table.

When designing a randomized clinical trial to compare two treatments, the sample size required to have desired power with a specified type 1 error depends on the hypothesis testing procedure. With a binary endpoint (e.g., response), the trial results can be displayed in a 2 × 2 table. If one does the analysis conditional on the number of positive responses, then using Fisher's exact test has an actual type 1 error less than or equal to the specified nominal type 1 error. Alternatively, one can use one of many unconditional "exact" tests that also preserve the type 1 error and are less conservative than Fisher's exact test. In particular, the unconditional test of Boschloo is always at least as powerful as Fisher's exact test, leading to smaller required sample sizes for clinical trials. However, many statisticians have argued over the years that the conditional analysis with Fisher's exact test is the only appropriate procedure. Since having smaller clinical trials is an extremely important consideration, we review the general arguments given for the conditional analysis of a 2 × 2 table in the context of a randomized clinical trial. We find the arguments not relevant in this context, or, if relevant, not completely convincing, suggesting the sample-size advantage of the unconditional tests should lead to their recommended use. We also briefly suggest that since designers of clinical trials practically always have target null and alternative response rates, there is the possibility of using this information to improve the power of the unconditional tests.

The role of quality of life data as an endpoint for collecting real-world evidence within geroscience clinical trials.

With geroscience research evolving at a fast pace, the need arises for human randomized controlled trials to assess the efficacy of geroprotective interventions to prevent age-related adverse outcomes, disease, and mortality in normative aging cohorts. However, to confirm efficacy requires a long-term and costly approach as time to the event of morbidity and mortality can be decades. While this could be circumvented using sensitive biomarkers of aging, current molecular, physiological, and digital endpoints require further validation. In this review, we discuss how collecting real-world evidence (RWE) by obtaining health data that is amenable for collection from large heterogeneous populations in a real-world setting can help speed up validation of geroprotective interventions. Further, we propose inclusion of quality of life (QoL) data as a biomarker of aging and candidate endpoint for geroscience clinical trials to aid in distinguishing healthy from unhealthy aging. We highlight how QoL assays can aid in accelerating data collection in studies gathering RWE on the geroprotective effects of repurposed drugs to support utilization within healthy longevity medicine. Finally, we summarize key metrics to consider when implementing QoL assays in studies, and present the short-form 36 (SF-36) as the most well-suited candidate endpoint.

Development of a Core Outcome Set for Dysphagia Interventions in Parkinson's disease (COS-DIP): study protocol.

INTRODUCTION: Current clinical trials on swallowing disorders (dysphagia) in Parkinson's disease (PD) apply a high variety of outcomes and different outcome measures making comparative effectiveness research challenging. Furthermore, views of patients and dysphagia clinicians when selecting trial outcomes have not been considered in the past, thus study results may have little importance to them. This study aims to develop an agreed standardised Core Outcome Set for Dysphagia Interventions in Parkinson's disease (COS-DIP), systematically measured and reported as a minimum for all clinical trials. It will also comprise guidance on outcome definitions, outcome measures and time points of measurement. METHODS AND ANALYSIS: The COS-DIP development will comprise five stages following established methodology: (1) a recent scoping review on all applied outcomes, their definitions, methods and time points of measurement in clinical trials in dysphagia in PD, (2) online surveys and focus groups with clinicians, patients, caregivers and family members to identify outcomes that are important to them, (3) an identified list of outcomes based on results of stage 1 and 2, (4) three round online Delphi survey with up to 200 key stakeholders to determine core outcomes and (5) two online consensus meetings with up to 40 representative key stakeholders to agree on all outcomes, definitions, methods and time points of measurement in the final COS-DIP. ETHICS AND DISSEMINATION: Full ethical approval was obtained from the Research Ethics Committee, School of Linguistic, Speech and Communication Sciences, Trinity College Dublin, on 15 May 2023 (HT27). Dissemination of the COS-DIP will be enhanced through presentations at (inter-) national conferences and through peer-reviewed, open access publications of related manuscripts. Lay and professional information sheets and infographics will be circulated through relevant patient and professional organisations and networks. TRIAL REGISTRATION NUMBER: The COS-DIP study was registered prospectively with the Core Outcome Measures in Effectiveness Trials (COMET) database on 24 September 2021 (www.comet-initiative.org/Studies/Details/1942).

Use of win time for ordered composite endpoints in clinical trials.

Consider the choice of outcome for overall treatment benefit in a clinical trial which measures the first time to each of several clinical events. We describe several new variants of the win ratio that incorporate the time spent in each clinical state over the common follow-up, where clinical state means the worst clinical event that has occurred by that time. One version allows restriction so that death during follow-up is most important, while time spent in other clinical states is still accounted for. Three other variants are described; one is based on the average pairwise win time, one creates a continuous outcome for each participant based on expected win times against a reference distribution and another that uses the estimated distributions of clinical state to compare the treatment arms. Finally, a combination testing approach is described to give robust power for detecting treatment benefit across a broad range of alternatives. These new methods are designed to be closer to the overall treatment benefit/harm from a patient's perspective, compared to the ordinary win ratio. The new methods are compared to the composite event approach and the ordinary win ratio. Simulations show that when overall treatment benefit on death is substantial, the variants based on either the participants' expected win times (EWTs) against a reference distribution or estimated clinical state distributions have substantially higher power than either the pairwise comparison or composite event methods. The methods are illustrated by re-analysis of the trial heart failure: a controlled trial investigating outcomes of exercise training.

Consideration of stratification in confirmatory trials with time-to-event endpoint.

Stratification in randomization and analysis are widely employed to balance treatment groups in clinical trials. However, the potential power loss due to under-stratification or over-stratification has not been thoroughly evaluated in the typical setting of confirmatory clinical trials. In cases where there are too many strata and some have small sample sizes or a small number of events, it is common practice to combine these small strata during analysis. However, there is a lack of guidance on how those small strata should be combined. This paper presents extensive simulation studies to evaluate the impact of under-stratification or over-stratification on the power of survival analysis and the estimate of hazard ratio using stratified log-rank test and Cox PH model, respectively. The difference in power between stratified and unstratified log-rank tests is also investigated under different scenarios. Our results suggest that failing to consider prognostic stratification factors with strong effects, and/or accounting for non-prognostic factors such as noise and predictive factors, may reduce the power of the stratified log-rank test. Additionally, methods of combining small strata are explored and compared.

An information-theoretic approach for the assessment of a continuous outcome as a surrogate for a binary true endpoint based on causal inference: Application to vaccine evaluation.

Within the causal association paradigm, a method is proposed to assess the validity of a continuous outcome as a surrogate for a binary true endpoint. The methodology is based on a previously introduced information-theoretic definition of surrogacy and has two main steps. In the first step, a new model is proposed to describe the joint distribution of the potential outcomes associated with the putative surrogate and the true endpoint of interest. The identifiability issues inherent to this type of models are handled via sensitivity analysis. In the second step, a metric of surrogacy new to this setting, the so-called individual causal association is presented. The methodology is studied in detail using theoretical considerations, some simulations, and data from a randomized clinical trial evaluating an inactivated quadrivalent influenza vaccine. A user-friendly R package Surrogate is provided to carry out the evaluation exercise.

Using short-term endpoints to improve interim decision making and trial duration in two-stage phase II trials with nested binary endpoints.

In oncology, phase II clinical trials are often planned as single-arm two-stage designs with a binary endpoint, for example, progression-free survival after 12 months, and the option to stop for futility after the first stage. Simon's two-stage design is a very popular approach but depending on the follow-up time required to measure the patients' outcomes the trial may have to be paused undesirably long. To shorten this forced interruption, it was proposed to use a short-term endpoint for the interim decision, such as progression-free survival after 3 months. We show that if the assumptions for the short-term endpoint are misspecified, the decision-making in the interim can be misleading, resulting in a great loss of statistical power. For the setting of a binary endpoint with nested measurements, such as progression-free survival, we propose two approaches that utilize all available short-term and long-term assessments of the endpoint to guide the interim decision. One approach is based on conditional power and the other is based on Bayesian posterior predictive probability of success. In extensive simulations, we show that both methods perform similarly, when appropriately calibrated, and can greatly improve power compared to the existing approach in settings with slow patient recruitment. Software code to implement the methods is made publicly available.

Assessment highlights need for improvement in standards of development of core outcome sets for rare genetic diseases.

OBJECTIVES: A rare disease is classified as such if it affects less than one person in 2,000. The Core Outcome Set STandards for Development (COS-STAD) is a set of standards that represent the minimum recommendations to be considered in the process of core outcome set (COS) development. The aim of this study was to provide a baseline assessment of COS development standards for rare genetic diseases. STUDY DESIGN AND SETTING: Core Outcome Measures in Effectiveness Trials (COMET) database contains nearly 400 published COS studies according to the latest systematic review. Studies focusing on COS development for rare genetic diseases were eligible for inclusion and were assessed by two independent evaluators. RESULTS: Nine COS studies were included in the analysis. Eight different rare genetic diseases were investigated. None of the studies met all the standards for development. The number of standards met ranged from 6 to 10, and the median was 7. CONCLUSION: This study is the first study to assess COS-STAD for rare genetic diseases, and it highlights a great need for improvement. First in terms of numbers of rare diseases considered for COS developments, second in methodology, particularly regarding the consensus process, and third in reporting of the COS development studies.

The development of a core outcome set for clinical effectiveness studies of bordered foam dressings in the treatment of complex wounds.

AIM: The aim of this project was to develop a core outcome set (COS) for clinical effectiveness studies of bordered foam dressings in the treatment of complex wounds. METHODS: The research project followed the Core Outcome Measures in Effectiveness Trials (COMET) initiative and consisted of two phases. The first phase prepared the background and process, while the second phase had three steps: outcome list generation via systematic review and qualitative study, Delphi consensus study, and consensus meeting. The study has been registered in the Core Outcome Measures in Effectiveness Trials database. RESULTS: The systematic review resulted in 82 outcomes and 20 additional outcomes were obtained during the interviews. After refinement, 111 panellists from 23 countries rated a list of 51 outcomes. In the following consensus meeting, six outcomes were prioritized to be included in the core outcome set. After the consensus meeting, a patient-reported outcome was added to the core outcome set. CONCLUSION: The COS for evaluating the effectiveness of bordered foam dressings in treating complex wounds includes 7 outcomes: "ability to stay in place", "leakage", "pain", "dressing related periwound skin changes", "change in wound size over time", and "overall satisfaction". These identified outcomes are correlated with contemporary bioengineering testing and evaluation methods for dressing performance, which underpins the need for a close multidisciplinary collaboration to advance the field of wound dressings. The outcome 'overall satisfaction' reflects the impact of complex wounds and their treatment on a patient's daily life. The use of these outcomes is recommended to improve data synthesis and promote evidence-based practice. Future developments in COS development involve creating measurement instruments and relevant endpoints for these outcomes.

Omnibus test for restricted mean survival time based on influence function.

The restricted mean survival time (RMST), which evaluates the expected survival time up to a pre-specified time point τ, has been widely used to summarize the survival distribution due to its robustness and straightforward interpretation. In comparative studies with time-to-event data, the RMST-based test has been utilized as an alternative to the classic log-rank test because the power of the log-rank test deteriorates when the proportional hazards assumption is violated. To overcome the challenge of selecting an appropriate time point τ, we develop an RMST-based omnibus Wald test to detect the survival difference between two groups throughout the study follow-up period. Treating a vector of RMSTs at multiple quantile-based time points as a statistical functional, we construct a Wald χ2 test statistic and derive its asymptotic distribution using the influence function. We further propose a new procedure based on the influence function to estimate the asymptotic covariance matrix in contrast to the usual bootstrap method. Simulations under different scenarios validate the size of our RMST-based omnibus test and demonstrate its advantage over the existing tests in power, especially when the true survival functions cross within the study follow-up period. For illustration, the proposed test is applied to two real datasets, which demonstrate its power and applicability in various situations.

Development of core outcome sets of Food for Special Medical Purposes designed for type 2 diabetes mellitus: a study protocol.

BACKGROUND: The Chinese government stipulates all food for special medical purposes (FSMP) designed for specific diseases to be tested in clinical trials before approving it for registration. The process of developing core outcome sets (COSs), the minimum sets of outcomes supposed to be measured and reported, provides an economical and practical option for stakeholders to communicate and cooperate in conducting clinical trials as well as in reporting FSMP outcomes. This study uses type 2 diabetes mellitus (T2DM) as an example to develop COS for clinical trials of FSMP. METHODS: The COS for FSMP-T2DM will be divided into 3 phases and developed following COS-STAP and COS-STAD: (1) Generate a list of relevant outcomes identified from a systematic review, in which information sources will mainly include published studies, regulatory documentation, and qualitative interviews of stakeholders. The identified outcomes will be categorized using a conceptual framework and formatted into the first round of the Delphi survey questionnaire items. (2) At least 2 rounds of Delphi surveys will be performed among stakeholders to create the COS for FSMP-T2DM. Patients, clinical dietitians, physicians, COS researchers, journal editors, FSMP manufacturers, and regulatory representatives will be invited to score each outcome from aspects of importance. (3) Hold a face-to-face or online consensus meeting to refine the content of the COS for FSMP-T2DM. Key stakeholders will be invited to attend the meeting to discuss and agree on the final COS. DISCUSSION: We have prepared an alternative solution of the Likert scale selection, Delphi survey rounds, scoring group, and consensus definitions in case of an unexpected situation. TRIAL REGISTRATION: COMET (1547). Registered on March 23, 2020.

A note on familywise error rate for a primary and secondary endpoint.

Hung et al. (2007) considered the problem of controlling the type I error rate for a primary and secondary endpoint in a clinical trial using a gatekeeping approach in which the secondary endpoint is tested only if the primary endpoint crosses its monitoring boundary. They considered a two-look trial and showed by simulation that the naive method of testing the secondary endpoint at full level α at the time the primary endpoint reaches statistical significance does not control the familywise error rate at level α. Tamhane et al. (2010) derived analytic expressions for familywise error rate and power and confirmed the inflated error rate of the naive approach. Nonetheless, many people mistakenly believe that the closure principle can be used to prove that the naive procedure controls the familywise error rate. The purpose of this note is to explain in greater detail why there is a problem with the naive approach and show that the degree of alpha inflation can be as high as that of unadjusted monitoring of a single endpoint.

Design of paediatric trials with benefit-risk endpoints using a composite score of adverse events of interest (AEI) and win-statistics.

A fundamental problem in the regulatory evaluation of a therapy is assessing whether the benefit outweighs the associated risks. This work proposes designing a trial that assesses a composite endpoint consisting of benefit and risk, hence, making the core of the design of the study, to assess benefit and risk. The proposed benefit risk measure consists of efficacy measure(s) and a risk measure that is based on a composite score obtained from pre-defined adverse events of interest (AEI). This composite score incorporates full aspects of adverse events of interest (i.e. the incidence, severity, and duration of the events). We call this newly proposed score the AEI composite score. After specifying the priorities between the components of the composite endpoint, a win-statistic (i.e. win ratio, win odds, or net benefit) is used to assess the difference between treatments in this composite endpoint. The power and sample size requirements of such a trial design are explored via simulation. Finally, using Dupixent published adult study results, we show how we can design a paediatric trial where the primary outcome is a composite of prioritized outcomes consisting of efficacy endpoints and the AEI composite score endpoint. The resulting trial design can potentially substantially reduce sample size compared to a trial designed to assess the co-primary efficacy endpoints, therefore it may address the challenge of slow enrollment and patient availability for paediatric studies.

Pediatric core outcome sets had deficiencies and lacked child and family input: A methodological review.

OBJECTIVES: The Core Outcome Set-STAndards for Development (COS-STAD), published in 2017, contains 11 standards (12 criteria) describing minimum design criteria for core outcome set (COS) development. We aimed to identify and appraise all pediatric COS published prior to COS-STAD, and assess methods of child and family involvement in their development. STUDY DESIGN AND SETTING: This methodological review included documents that described the development of pediatric COS up to and including 2017. Reviewers independently assessed each COS against COS-STAD criteria, and methods of involvement were synthesized. RESULTS: A total of 56 pediatric COS were identified, meeting a median of five COS-STAD criteria. Nearly all met criteria on COS scope specification for setting, health condition, and population; 41% met criteria for intervention. Standards were more often met for the involvement of researchers/health professionals (64%) than for patients or their representatives (29%). Few met standards for achieving COS consensus (4-23%). Methods of child and family engagement varied and were limited. CONCLUSION: A large proportion of pediatric COS developed prior to COS-STAD recommendations show gaps in design methodology. Updated and newly developed pediatric COS would benefit from the inclusion of the child and family voice, implementing a priori criteria for COS consensus, and clear reporting.

Core Outcome Sets in Child Health: A Systematic Review.

Importance: Developing core outcome sets is essential to ensure that results of clinical trials are comparable and useful. A number of core outcome sets in pediatrics have been published, but a comprehensive in-depth understanding of core outcome sets in this field is lacking. Objective: To systematically identify core outcome sets in child health, collate the diseases to which core outcome sets have been applied, describe the methods used for development and stakeholder participation, and evaluate the methodological quality of existing core outcome sets. Evidence Review: MEDLINE, SCOPUS, Cochrane Library, and CINAHL were searched using relevant search terms, such as clinical trials, core outcome, and children, along with relevant websites, such as Core Outcome Measures in Effectiveness Trials (COMET). Four researchers worked in teams of 2, performed literature screening and data extraction, and evaluated the methodological quality of core outcome sets using the Core Outcome Set-Standards for Development (COS-STAD). Findings: A total of 77 pediatric core outcome sets were identified, mainly developed by organizations or researchers in Europe, North America, and Australia and mostly from the UK (22 [29%]) and the US (22 [29%]). A total of 77 conditions were addressed; the most frequent International Classification of Diseases, 11th Revision category was diseases of the digestive system (14 [18%]). Most of the outcomes in pediatric core outcome sets were unordered (34 [44%]) or presented in custom classifications (29 [38%]). Core outcome sets used 1 or more of 8 development methods; the most frequent combination of methods was systematic review/literature review/scoping review, together with the Delphi approach and consensus for decision-making (10 [14%]). Among the 6 main types of stakeholders, clinical experts were the most frequently involved (74 [100%]), while industry representatives were rarely involved (4 [5%]). Only 6 core outcome sets (8%) met the 12 criteria of COS-STAD. Conclusions and Relevance: Future quality of pediatric core outcome sets should be improved based on the standards proposed by the COMET initiative, while core outcome sets methodology and reporting standards should be extended to pediatric populations to help improve the quality of core outcome sets in child health. In addition, the COMET outcome taxonomy should also add items applicable to children.

Core outcomes for pressure ulcer prevention trials: results of an international consensus study: Classification: Outcomes and qualitative research.

BACKGROUND: There is substantial heterogeneity between trial outcomes in pressure ulcer prevention research. The development of core outcome sets is one strategy to improve comparability between trial results and thus increase the quality of evidence. OBJECTIVES: To identify core outcomes for pressure ulcer prevention trials. METHODS: A workshop was held with service users to discuss their views and understanding of the outcomes identified by a scoping review and to identify any missing outcomes. In a next step, a Delphi survey comprising three rounds was conducted to evaluate a compiled list of outcomes by their importance. Afterwards the preselection from the Delphi survey was discussed in a virtual consensus meeting with the aim of agreeing on a final set of core outcomes. Individuals who had completed all three rounds of the Delphi survey were eligible to participate in this meeting. Participants included practitioners, service users, researchers and industry representatives. The OUTPUTs project is registered in the COMET database and is part of the Cochrane Skin Core Outcome Set Initiative. RESULTS: The workshop did not reveal any missing outcomes, but highlighted the need for further efforts to make lay people understand what an outcome is in a study setting. The Delphi survey took place between December 2020 and June 2021. After the three rounds, 18 out of 37 presented outcomes were rated to be critically important. In the following consensus meeting, six outcomes were prioritized to be included in the core outcome set for pressure ulcer prevention trials: (i) pressure ulcer occurrence; (ii) pressure ulcer precursor signs and symptoms; (iii) mobility; (iv) acceptability and comfort of intervention; (v) adherence/compliance; and (vi) adverse events/safety. CONCLUSIONS: Based on a comprehensive list of outcomes in pressure ulcer prevention research, there was clear agreement on the six identified core outcomes in three international Delphi rounds and in the consensus meeting. Although outcome measurement instruments need to be identified next, the six identified core outcomes should already be considered in future trials, as service users, practitioners, researchers and industry representatives have agreed that they are critically important. What is already known about this topic? There are numerous trials on pressure ulcer prevention, but evidence on the effectiveness of preventive measures is limited due to heterogeneity between trial outcomes. The development of a core outcome set is one strategy to improve comparability between trial results. What does this study add? A service user workshop, a three-round Delphi survey and an online consensus meeting with practitioners, service users, researchers and industry representatives were conducted to identify core outcomes for pressure ulcer prevention trials. Six core outcomes were defined: (i) pressure ulcer occurrence, (ii) pressure ulcer precursor signs and symptoms, (iii) mobility, (iv) acceptability and comfort of intervention, (v) adherence/compliance and (vi) adverse events/safety. What are the clinical implications of this work? Better evidence of interventions for pressure ulcer prevention will help health professionals and service users to decide which interventions are most appropriate and effective. Better evidence may contribute to better pressure ulcer prevention.