Innovative approaches for vaccine trials as a key component of pandemic preparedness - a white paper.

BACKGROUND: WHO postulates the application of adaptive design features in the global clinical trial ecosystem. However, the adaptive platform trial (APT) methodology has not been widely adopted in clinical research on vaccines. METHODS: The VACCELERATE Consortium organized a two-day workshop to discuss the applicability of APT methodology in vaccine trials under non-pandemic as well as pandemic conditions. Core aspects of the discussions are summarized in this article. RESULTS: An "ever-warm" APT appears ideally suited to improve efficiency and speed of vaccine research. Continuous learning based on accumulating APT trial data allows for pre-planned adaptations during its course. Given the relative design complexity, alignment of all stakeholders at all stages of an APT is central. Vaccine trial modelling is crucial, both before and in a pandemic emergency. Various inferential paradigms are possible (frequentist, likelihood, or Bayesian). The focus in the interpandemic interval may be on research gaps left by industry trials. For activation in emergency, template Disease X protocols of syndromal design for pathogens yet unknown need to be stockpiled and updated regularly. Governance of a vaccine APT should be fully integrated into supranational pandemic response mechanisms. DISCUSSION: A broad range of adaptive features can be applied in platform trials on vaccines. Faster knowledge generation comes with increased complexity of trial design. Design complexity should not preclude simple execution at trial sites. Continuously generated evidence represents a return on investment that will garner societal support for sustainable funding. Adaptive design features will naturally find their way into platform trials on vaccines.

Adaptive clinical trial designs with blinded selection of binary composite endpoints and sample size reassessment.

For randomized clinical trials where a single, primary, binary endpoint would require unfeasibly large sample sizes, composite endpoints (CEs) are widely chosen as the primary endpoint. Despite being commonly used, CEs entail challenges in designing and interpreting results. Given that the components may be of different relevance and have different effect sizes, the choice of components must be made carefully. Especially, sample size calculations for composite binary endpoints depend not only on the anticipated effect sizes and event probabilities of the composite components but also on the correlation between them. However, information on the correlation between endpoints is usually not reported in the literature which can be an obstacle for designing future sound trials. We consider two-arm randomized controlled trials with a primary composite binary endpoint and an endpoint that consists only of the clinically more important component of the CE. We propose a trial design that allows an adaptive modification of the primary endpoint based on blinded information obtained at an interim analysis. Especially, we consider a decision rule to select between a CE and its most relevant component as primary endpoint. The decision rule chooses the endpoint with the lower estimated required sample size. Additionally, the sample size is reassessed using the estimated event probabilities and correlation, and the expected effect sizes of the composite components. We investigate the statistical power and significance level under the proposed design through simulations. We show that the adaptive design is equally or more powerful than designs without adaptive modification on the primary endpoint. Besides, the targeted power is achieved even if the correlation is misspecified at the planning stage while maintaining the type 1 error. All the computations are implemented in R and illustrated by means of a peritoneal dialysis trial.

Online error rate control for platform trials.

Platform trials evaluate multiple experimental treatments under a single master protocol, where new treatment arms are added to the trial over time. Given the multiple treatment comparisons, there is the potential for inflation of the overall type I error rate, which is complicated by the fact that the hypotheses are tested at different times and are not necessarily pre-specified. Online error rate control methodology provides a possible solution to the problem of multiplicity for platform trials where a relatively large number of hypotheses are expected to be tested over time. In the online multiple hypothesis testing framework, hypotheses are tested one-by-one over time, where at each time-step an analyst decides whether to reject the current null hypothesis without knowledge of future tests but based solely on past decisions. Methodology has recently been developed for online control of the false discovery rate as well as the familywise error rate (FWER). In this article, we describe how to apply online error rate control to the platform trial setting, present extensive simulation results, and give some recommendations for the use of this new methodology in practice. We show that the algorithms for online error rate control can have a substantially lower FWER than uncorrected testing, while still achieving noticeable gains in power when compared with the use of a Bonferroni correction. We also illustrate how online error rate control would have impacted a currently ongoing platform trial.

Impact of adaptive filtering on power and false discovery rate in RNA-seq experiments.

BACKGROUND: In RNA-sequencing studies a large number of hypothesis tests are performed to compare the differential expression of genes between several conditions. Filtering has been proposed to remove candidate genes with a low expression level which may not be relevant and have little or no chance of showing a difference between conditions. This step may reduce the multiple testing burden and increase power. RESULTS: We show in a simulation study that filtering can lead to some increase in power for RNA-sequencing data, too aggressive filtering, however, can lead to a decline. No uniformly optimal filter in terms of power exists. Depending on the scenario different filters may be optimal. We propose an adaptive filtering strategy which selects one of several filters to maximise the number of rejections. No additional adjustment for multiplicity has to be included, but a rule has to be considered if the number of rejections is too small. CONCLUSIONS: For a large range of simulation scenarios, the adaptive filter maximises the power while the simulated False Discovery Rate is bounded by the pre-defined significance level. Using the adaptive filter, it is not necessary to pre-specify a single individual filtering method optimised for a specific scenario.

Incidence and Outcomes of Cardiopulmonary Resuscitation in ICUs: Retrospective Cohort Analysis.

OBJECTIVES: We aim to describe incidence and outcomes of cardiopulmonary resuscitation (CPR) efforts and their outcomes in ICUs and their changes over time. DESIGN: Retrospective cohort analysis. SETTING: Patient data documented in the Austrian Center for Documentation and Quality Assurance in Intensive Care database. PATIENTS: Adult patients (age ≥ 18 yr) admitted to Austrian ICUs between 2005 and 2019. INTERVENTIONS: None. MEASUREMENTS ANDN MAIN RESULTS: Information on CPR was deduced from the Therapeutic Intervention Scoring System. End points were overall occurrence rate of CPR in the ICU and CPR for unexpected cardiac arrest after the first day of ICU stay as well as survival to discharge from the ICU and the hospital. Incidence and outcomes of ICU-CPR were compared between 2005 and 2009, 2010 and 2014, and 2015 and 2019 using chi-square test. A total of 525,518 first admissions and readmissions to ICU of 494,555 individual patients were included; of these, 72,585 patients (14.7%) died in hospital. ICU-CPR was performed in 20,668 (3.9%) admissions at least once; first events occurred on the first day of ICU admission in 15,266 cases (73.9%). ICU-CPR was first performed later during ICU stay in 5,402 admissions (1.0%). The incidence of ICU-CPR decreased slightly from 4.4% between 2005 and 2009, 3.9% between 2010 and 2014, and 3.7% between 2015 and 2019 ( p < 0.001). A total of 7,078 (34.5%) of 20,499 patients who received ICU-CPR survived until hospital discharge. Survival rates varied slightly over the observation period; 59,164 (12.0%) of all patients died during hospital stay without ever receiving CPR in the ICU. CONCLUSIONS: The incidence of ICU-CPR is approximately 40 in 1,000 admissions overall and approximately 10 in 1,000 admissions after the day of ICU admission. Short-term survival is approximately four out of 10 patients who receive ICU-CPR.

Growth prediction model for abdominal aortic aneurysms.

BACKGROUND: The most relevant determinant in scheduling monitoring intervals for abdominal aortic aneurysms (AAAs) is maximum diameter. The aim of the study was to develop a statistical model that takes into account specific characteristics of AAA growth distributions such as between-patient variability as well as within-patient variability across time, and allows probabilistic statements to be made regarding expected AAA growth. METHODS: CT angiography (CTA) data from patients monitored at 6-month intervals with maximum AAA diameters at baseline between 30 and 66 mm were used to develop the model. By extending the model of geometric Brownian motion with a log-normal random effect, a stochastic growth model was developed. An additional set of ultrasound-based growth data was used for external validation. RESULTS: The study data included 363 CTAs from 87 patients, and the external validation set comprised 390 patients. Internal and external cross-validation showed that the stochastic growth model allowed accurate description of the distribution of aneurysm growth. Median relative growth within 1 year was 4.1 (5-95 per cent quantile 0.5-13.3) per cent. Model calculations further resulted in relative 1-year growth of 7.0 (1.0-16.4) per cent for patients with previously observed rapid 1-year growth of 10 per cent, and 2.6 (0.3-8.3) per cent for those with previously observed slow growth of 1 per cent. The probability of exceeding a threshold of 55 mm was calculated to be 1.78 per cent at most when adhering to the current RESCAN guidelines for rescreening intervals. An online calculator based on the fitted model was made available. CONCLUSION: The stochastic growth model was found to provide a reliable tool for predicting AAA growth.

On model-based time trend adjustments in platform trials with non-concurrent controls.

BACKGROUND: Platform trials can evaluate the efficacy of several experimental treatments compared to a control. The number of experimental treatments is not fixed, as arms may be added or removed as the trial progresses. Platform trials are more efficient than independent parallel group trials because of using shared control groups. However, for a treatment entering the trial at a later time point, the control group is divided into concurrent controls, consisting of patients randomised to control when that treatment arm is in the platform, and non-concurrent controls, patients randomised before. Using non-concurrent controls in addition to concurrent controls can improve the trial's efficiency by increasing power and reducing the required sample size, but can introduce bias due to time trends. METHODS: We focus on a platform trial with two treatment arms and a common control arm. Assuming that the second treatment arm is added at a later time, we assess the robustness of recently proposed model-based approaches to adjust for time trends when utilizing non-concurrent controls. In particular, we consider approaches where time trends are modeled either as linear in time or as a step function, with steps at time points where treatments enter or leave the platform trial. For trials with continuous or binary outcomes, we investigate the type 1 error rate and power of testing the efficacy of the newly added arm, as well as the bias and root mean squared error of treatment effect estimates under a range of scenarios. In addition to scenarios where time trends are equal across arms, we investigate settings with different time trends or time trends that are not additive in the scale of the model. RESULTS: A step function model, fitted on data from all treatment arms, gives increased power while controlling the type 1 error, as long as the time trends are equal for the different arms and additive on the model scale. This holds even if the shape of the time trend deviates from a step function when patients are allocated to arms by block randomisation. However, if time trends differ between arms or are not additive to treatment effects in the scale of the model, the type 1 error rate may be inflated. CONCLUSIONS: The efficiency gained by using step function models to incorporate non-concurrent controls can outweigh potential risks of biases, especially in settings with small sample sizes. Such biases may arise if the model assumptions of equality and additivity of time trends are not satisfied. However, the specifics of the trial, scientific plausibility of different time trends, and robustness of results should be carefully considered.

Rationale for the update algorithm of the graphical approach to sequentially rejective multiple test procedures.

The graphical approach by Bretz et al. is a convenient tool to construct, visualize and perform multiple test procedures that are tailored to structured families of hypotheses while controlling the familywise error rate. A critical step is to update the transition weights following a pre-specified algorithm. In their original publication, however, the authors did not provide a detailed rationale for the update formula. This paper closes the gap and provides three alternative arguments for the update of the transition weights of the graphical approach. It is a legacy of the first author, based on an unpublished technical report from 2014, and after his untimely death reconstructed by the other two authors as a tribute to Willi Maurer's collaboration with Andy Grieve and contributions to biostatistics over many years.

Robust group sequential designs for trials with survival endpoints and delayed response.

Randomized clinical trials in oncology typically utilize time-to-event endpoints such as progression-free survival or overall survival as their primary efficacy endpoints, and the most commonly used statistical test to analyze these endpoints is the log-rank test. The power of the log-rank test depends on the behavior of the hazard ratio of the treatment arm to the control arm. Under the assumption of proportional hazards, the log-rank test is asymptotically fully efficient. However, this proportionality assumption does not hold true if there is a delayed treatment effect. Cancer immunology has evolved over time and several cancer vaccines are available in the market for treating existing cancers. This includes sipuleucel-T for metastatic hormone-refractory prostate cancer, nivolumab for metastatic melanoma, and pembrolizumab for advanced nonsmall-cell lung cancer. As cancer vaccines require some time to elicit an immune response, a delayed treatment effect is observed, resulting in a violation of the proportional hazards assumption. Thus, the traditional log-rank test may not be optimal for testing immuno-oncology drugs in randomized clinical trials. Moreover, the new immuno-oncology compounds have been shown to be very effective in prolonging overall survival. Therefore, it is desirable to implement a group sequential design with the possibility of early stopping for overwhelming efficacy. In this paper, we investigate the max-combo test, which utilizes the maximum of two weighted log-rank statistics, as a robust alternative to the log-rank test. The new test is implemented for two-stage designs with possible early stopping at the interim analysis time point. Two classes of weights are investigated for the max-combo test: the Fleming and Harrington (1981) Gρ,γ$G^{\rho , \gamma }$ weights and the Magirr and Burman (2019) modest (τ∗)$ (\tau ^{*})$  weights.

Association of Acute Kidney Injury Receiving Kidney Replacement Therapy With Prognosis of Critically Ill Patients With and Without Cancer: A Retrospective Study.

OBJECTIVES: To assess outcomes of cancer patients receiving kidney replacement therapy due to acute kidney injury in ICUs and compare these with other patient groups receiving kidney replacement therapy in ICUs. DESIGN: Retrospective registry analysis. SETTING: Prospectively collected database of 296,424 ICU patients. PATIENTS: Patients with and without solid cancer with acute kidney injury necessitating kidney replacement therapy were identified and compared with those without acute kidney injury necessitating kidney replacement therapy. INTERVENTIONS: Descriptive statistics were used to ascertain prevalence of acute kidney injury necessitating kidney replacement therapy and solid cancer in ICU patients. Association of acute kidney injury necessitating kidney replacement therapy and cancer with prognosis was assessed using logistic regression analysis. To compare the attributable mortality of acute kidney injury necessitating kidney replacement therapy, 20,154 noncancer patients and 2,411 cancer patients without acute kidney injury necessitating kidney replacement therapy were matched with 12,827 noncancer patients and 1,079 cancer patients with acute kidney injury necessitating kidney replacement therapy. MEASUREMENTS AND MAIN RESULTS: Thirty-five thousand three hundred fifty-six ICU patients (11.9%) had solid cancer. Acute kidney injury necessitating kidney replacement therapy was present in 1,408 (4.0%) cancer patients and 13,637 (5.2%) noncancer patients. Crude ICU and hospital mortality was higher in the cancer group (646 [45.9%] vs 4,674 [34.3%], p < 0.001, and 787 [55.9%] vs 5,935 [43.5%], p < 0.001). In multivariable logistic regression analyses, odds ratio (95% CI) for hospital mortality was 1.73 (1.62-1.85) for cancer compared with no cancer 3.57 (3.32-3.83) for acute kidney injury necessitating kidney replacement therapy and 1.07 (0.86-1.33) for their interaction. In the matched subcohort, attributable hospital mortality of acute kidney injury necessitating kidney replacement therapy was 56.7% in noncancer patients and 48.0% in cancer patients. CONCLUSIONS: Occurrence rate of acute kidney injury necessitating kidney replacement therapy and prognosis in ICU patients with solid cancer are comparable with other ICU patient groups. In cancer, acute kidney injury necessitating kidney replacement therapy is associated with higher crude hospital mortality. However, the specific attributable mortality conveyed by acute kidney injury necessitating kidney replacement therapy is actually lower in cancer patients than in noncancer patients. Diagnosis of cancer per se does not justify withholding kidney replacement therapy.

Optimizing subgroup selection in two-stage adaptive enrichment and umbrella designs.

We design two-stage confirmatory clinical trials that use adaptation to find the subgroup of patients who will benefit from a new treatment, testing for a treatment effect in each of two disjoint subgroups. Our proposal allows aspects of the trial, such as recruitment probabilities of each group, to be altered at an interim analysis. We use the conditional error rate approach to implement these adaptations with protection of overall error rates. Applying a Bayesian decision-theoretic framework, we optimize design parameters by maximizing a utility function that takes the population prevalence of the subgroups into account. We show results for traditional trials with familywise error rate control (using a closed testing procedure) as well as for umbrella trials in which only the per-comparison type 1 error rate is controlled. We present numerical examples to illustrate the optimization process and the effectiveness of the proposed designs.

Delayed treatment effects, treatment switching and heterogeneous patient populations: How to design and analyze RCTs in oncology.

In the analysis of survival times, the logrank test and the Cox model have been established as key tools, which do not require specific distributional assumptions. Under the assumption of proportional hazards, they are efficient and their results can be interpreted unambiguously. However, delayed treatment effects, disease progression, treatment switchers or the presence of subgroups with differential treatment effects may challenge the assumption of proportional hazards. In practice, weighted logrank tests emphasizing either early, intermediate or late event times via an appropriate weighting function may be used to accommodate for an expected pattern of non-proportionality. We model these sources of non-proportional hazards via a mixture of survival functions with piecewise constant hazard. The model is then applied to study the power of unweighted and weighted log-rank tests, as well as maximum tests allowing different time dependent weights. Simulation results suggest a robust performance of maximum tests across different scenarios, with little loss in power compared to the most powerful among the considered weighting schemes and huge power gain compared to unfavorable weights. The actual sources of non-proportional hazards are not obvious from resulting populationwise survival functions, highlighting the importance of detailed simulations in the planning phase of a trial when assuming non-proportional hazards.We provide the required tools in a software package, allowing to model data generating processes under complex non-proportional hazard scenarios, to simulate data from these models and to perform the weighted logrank tests.

The use of external controls: To what extent can it currently be recommended?

With more and better clinical data being captured outside of clinical studies and greater data sharing of clinical studies, external controls may become a more attractive alternative to randomized clinical trials (RCTs). Both industry and regulators recognize that in situations where a randomized study cannot be performed, external controls can provide the needed contextualization to allow a better interpretation of studies without a randomized control. It is also agreed that external controls will not fully replace RCTs as the gold standard for formal proof of efficacy in drug development and the yardstick of clinical research. However, it remains unclear in which situations conclusions about efficacy and a positive benefit/risk can reliably be based on the use of an external control. This paper will provide an overview on types of external control, their applications and the different sources of bias their use may incur, and discuss potential mitigation steps. It will also give recommendations on how the use of external controls can be justified.

A critical review of graphics for subgroup analyses in clinical trials.

Subgroup analyses are a routine part of clinical trials to investigate whether treatment effects are homogeneous across the study population. Graphical approaches play a key role in subgroup analyses to visualise effect sizes of subgroups, to aid the identification of groups that respond differentially, and to communicate the results to a wider audience. Many existing approaches do not capture the core information and are prone to lead to a misinterpretation of the subgroup effects. In this work, we critically appraise existing visualisation techniques, propose useful extensions to increase their utility and attempt to develop an effective visualisation approach. We focus on forest plots, UpSet plots, Galbraith plots, subpopulation treatment effect pattern plot, and contour plots, and comment on other approaches whose utility is more limited. We illustrate the methods using data from a prostate cancer study.

A multiple comparison procedure for dose-finding trials with subpopulations.

Identifying subgroups of patients with an enhanced response to a new treatment has become an area of increased interest in the last few years. When there is knowledge about possible subpopulations with an enhanced treatment effect before the start of a trial it might be beneficial to set up a testing strategy, which tests for a significant treatment effect not only in the full population, but also in these prespecified subpopulations. In this paper, we present a parametric multiple testing approach for tests in multiple populations for dose-finding trials. Our approach is based on the MCP-Mod methodology, which uses multiple comparison procedures (MCPs) to test for a dose-response signal, while considering multiple possible candidate dose-response shapes. Our proposed methods allow for heteroscedastic error variances between populations and control the family-wise error rate over tests in multiple populations and for multiple candidate models. We show in simulations that the proposed multipopulation testing approaches can increase the power to detect a significant dose-response signal over the standard single-population MCP-Mod, when the specified subpopulation has an enhanced treatment effect.

Methods for the analysis of multiple endpoints in small populations: A review.

While current guidelines generally recommend single endpoints for primary analyses of confirmatory clinical trials, it is recognized that certain settings require inference on multiple endpoints for comprehensive conclusions on treatment effects. Furthermore, combining treatment effect estimates from several outcome measures can increase the statistical power of tests. Such an efficient use of resources is of special relevance for trials in small populations. This paper reviews approaches based on a combination of test statistics or measurements across endpoints as well as multiple testing procedures that allow for confirmatory conclusions on individual endpoints. We especially focus on feasibility in trials with small sample sizes and do not solely rely on asymptotic considerations. A systematic literature search in the Scopus database, supplemented by a manual search, was performed to identify research papers on analysis methods for multiple endpoints with relevance to small populations. The identified methods were grouped into approaches that combine endpoints into a single measure to increase the power of statistical tests and methods to investigate differential treatment effects in several individual endpoints by multiple testing.

A Randomized Trial of Bortezomib in Late Antibody-Mediated Kidney Transplant Rejection.

Late antibody-mediated rejection (ABMR) is a leading cause of kidney allograft failure. Uncontrolled studies have suggested efficacy of the proteasome inhibitor bortezomib, but no systematic trial has been undertaken to support its use in ABMR. In this randomized, placebo-controlled trial (the Bortezomib in Late Antibody-Mediated Kidney Transplant Rejection [BORTEJECT] Trial), we investigated whether two cycles of bortezomib (each cycle: 1.3 mg/m2 intravenously on days 1, 4, 8, and 11) prevent GFR decline by halting the progression of late donor-specific antibody (DSA)-positive ABMR. Forty-four DSA-positive kidney transplant recipients with characteristic ABMR morphology (median time after transplant, 5.0 years; pretransplant DSA documented in 19 recipients), who were identified on cross-sectional screening of 741 patients, were randomly assigned to receive bortezomib (n=21) or placebo (n=23). The 0.5-ml/min per 1.73 m2 per year (95% confidence interval, -4.8 to 5.8) difference detected between bortezomib and placebo in eGFR slope (primary end point) was not significant (P=0.86). We detected no significant differences between bortezomib- and placebo-treated groups in median measured GFR at 24 months (33 versus 42 ml/min per 1.73 m2; P=0.31), 2-year graft survival (81% versus 96%; P=0.12), urinary protein concentration, DSA levels, or morphologic or molecular rejection phenotypes in 24-month follow-up biopsy specimens. Bortezomib, however, associated with gastrointestinal and hematologic toxicity. In conclusion, our trial failed to show that bortezomib prevents GFR loss, improves histologic or molecular disease features, or reduces DSA, despite significant toxicity. Our results reinforce the need for systematic trials to dissect the efficiency and safety of new treatments for late ABMR.

Nested combination tests with a time-to-event endpoint using a short-term endpoint for design adaptations.

Adaptive trial methodology for multiarmed trials and enrichment designs has been extensively discussed in the past. A general principle to construct test procedures that control the family-wise Type I error rate in the strong sense is based on combination tests within a closed test. Using survival data, a problem arises when using information of patients for adaptive decision making, which are under risk at interim. With the currently available testing procedures, either no testing of hypotheses in interim analyses is possible or there are restrictions on the interim data that can be used in the adaptation decisions as, essentially, only the interim test statistics of the primary endpoint may be used. We propose a general adaptive testing procedure, covering multiarmed and enrichment designs, which does not have these restrictions. An important application are clinical trials, where short-term surrogate endpoints are used as basis for trial adaptations, and we illustrate how such trials can be designed. We propose statistical models to assess the impact of effect sizes, the correlation structure between the short-term and the primary endpoint, the sample size, the timing of interim analyses, and the selection rule on the operating characteristics.

Design and estimation in clinical trials with subpopulation selection.

Population heterogeneity is frequently observed among patients' treatment responses in clinical trials because of various factors such as clinical background, environmental, and genetic factors. Different subpopulations defined by those baseline factors can lead to differences in the benefit or safety profile of a therapeutic intervention. Ignoring heterogeneity between subpopulations can substantially impact on medical practice. One approach to address heterogeneity necessitates designs and analysis of clinical trials with subpopulation selection. Several types of designs have been proposed for different circumstances. In this work, we discuss a class of designs that allow selection of a predefined subgroup. Using the selection based on the maximum test statistics as the worst-case scenario, we then investigate the precision and accuracy of the maximum likelihood estimator at the end of the study via simulations. We find that the required sample size is chiefly determined by the subgroup prevalence and show in simulations that the maximum likelihood estimator for these designs can be substantially biased.

Flexible alpha allocation strategies for confirmatory adaptive enrichment clinical trials with a prespecified subgroup.

Adaptive enrichment designs have recently received considerable attention as they have the potential to make drug development process for personalized medicine more efficient. Several statistical approaches have been proposed so far in the literature and the operating characteristics of these approaches are extensively investigated using simulation studies. In this paper, we improve on existing adaptive enrichment designs by assigning unequal weights to the significance levels associated with the hypotheses of the overall population and a prespecified subgroup. More specifically, we focus on the standard combination test, a modified combination test, the marginal combination test, and the partial conditional error rate approach and explore the operating characteristics of these approaches by a simulation study. We show that these approaches can lead to power gains, compared to existing approaches, if the weights are chosen carefully.