Practical guidance for planning resources required to support publicly-funded adaptive clinical trials.

Adaptive designs are a class of methods for improving efficiency and patient benefit of clinical trials. Although their use has increased in recent years, research suggests they are not used in many situations where they have potential to bring benefit. One barrier to their more widespread use is a lack of understanding about how the choice to use an adaptive design, rather than a traditional design, affects resources (staff and non-staff) required to set-up, conduct and report a trial. The Costing Adaptive Trials project investigated this issue using quantitative and qualitative research amongst UK Clinical Trials Units. Here, we present guidance that is informed by our research, on considering the appropriate resourcing of adaptive trials. We outline a five-step process to estimate the resources required and provide an accompanying costing tool. The process involves understanding the tasks required to undertake a trial, and how the adaptive design affects them. We identify barriers in the publicly funded landscape and provide recommendations to trial funders that would address them. Although our guidance and recommendations are most relevant to UK non-commercial trials, many aspects are relevant more widely.

A comparison of clinical development pathways to advance tuberculosis regimen development.

BACKGROUND: Current tuberculosis (TB) regimen development pathways are slow and in urgent need of innovation. We investigated novel phase IIc and seamless phase II/III trials utilizing multi-arm multi-stage and Bayesian response adaptive randomization trial designs to select promising combination regimens in a platform adaptive trial. METHODS: Clinical trial simulation tools were built using predictive and validated parametric survival models of time to culture conversion (intermediate endpoint) and time to TB-related unfavorable outcome (final endpoint). This integrative clinical trial simulation tool was used to explore and optimize design parameters for aforementioned trial designs. RESULTS: Both multi-arm multi-stage and Bayesian response adaptive randomization designs were able to reliably graduate desirable regimens in ≥ 95% of trial simulations and reliably stop suboptimal regimens in ≥ 90% of trial simulations. Overall, adaptive phase IIc designs reduced patient enrollment by 17% and 25% with multi-arm multi-stage and Bayesian response adaptive randomization designs respectively compared to the conventional sequential approach, while seamless designs reduced study duration by 2.6 and 3.5 years respectively (typically ≥ 8.5 years for standard sequential approach). CONCLUSIONS: In this study, we demonstrate that adaptive trial designs are suitable for TB regimen development, and we provide plausible design parameters for a platform adaptive trial. Ultimately trial design and specification of design parameters will depend on clinical trial objectives. To support decision-making for clinical trial designs in contemporary TB regimen development, we provide a flexible clinical trial simulation tool that can be used to explore and optimize design features and parameters.

Pandemic vaccine testing: Combining conventional and challenge studies.

Early into COVID, human challenge trials were considered, but usually as alternatives to conventional randomized controlled trials. Instead, assessment of authorized COVID vaccines, of further COVID vaccines, and of vaccines against future pandemics should combine both designs, in five different ways, including a wholly novel one that we elaborate, Viz., combining data from both designs to answer a single question.

Costs and staffing resource requirements for adaptive clinical trials: quantitative and qualitative results from the Costing Adaptive Trials project.

BACKGROUND: Adaptive designs offer great promise in improving the efficiency and patient-benefit of clinical trials. An important barrier to further increased use is a lack of understanding about which additional resources are required to conduct a high-quality adaptive clinical trial, compared to a traditional fixed design. The Costing Adaptive Trials (CAT) project investigated which additional resources may be required to support adaptive trials. METHODS: We conducted a mock costing exercise amongst seven Clinical Trials Units (CTUs) in the UK. Five scenarios were developed, derived from funded clinical trials, where a non-adaptive version and an adaptive version were described. Each scenario represented a different type of adaptive design. CTU staff were asked to provide the costs and staff time they estimated would be needed to support the trial, categorised into specified areas (e.g. statistics, data management, trial management). This was calculated separately for the non-adaptive and adaptive version of the trial, allowing paired comparisons. Interviews with 10 CTU staff who had completed the costing exercise were conducted by qualitative researchers to explore reasons for similarities and differences. RESULTS: Estimated resources associated with conducting an adaptive trial were always (moderately) higher than for the non-adaptive equivalent. The median increase was between 2 and 4% for all scenarios, except for sample size re-estimation which was 26.5% (as the adaptive design could lead to a lengthened study period). The highest increase was for statistical staff, with lower increases for data management and trial management staff. The percentage increase in resources varied across different CTUs. The interviews identified possible explanations for differences, including (1) experience in adaptive trials, (2) the complexity of the non-adaptive and adaptive design, and (3) the extent of non-trial specific core infrastructure funding the CTU had. CONCLUSIONS: This work sheds light on additional resources required to adequately support a high-quality adaptive trial. The percentage increase in costs for supporting an adaptive trial was generally modest and should not be a barrier to adaptive designs being cost-effective to use in practice. Informed by the results of this research, guidance for investigators and funders will be developed on appropriately resourcing adaptive trials.

Inference for multimarker adaptive enrichment trials.

Identification of treatment selection biomarkers has become very important in cancer drug development. Adaptive enrichment designs have been developed for situations where a unique treatment selection biomarker is not apparent based on the mechanism of action of the drug. With such designs, the eligibility rules may be adaptively modified at interim analysis times to exclude patients who are unlikely to benefit from the test treatment.We consider a recently proposed, particularly flexible approach that permits development of model-based multifeature predictive classifiers as well as optimized cut-points for continuous biomarkers. A single significance test, including all randomized patients, is performed at the end of the trial of the strong null hypothesis that the expected outcome on the test treatment is no better than control for any of the subset populations of patients accrued in the K stages of the clinical trial. In this paper, we address 2 issues involving inference following an adaptive enrichment design as described above. The first is specification of the intended use population and estimation of treatment effect for that population following rejection of the strong null hypothesis. The second issue is defining conditions in which rejection of the strong null hypothesis implies rejection of the null hypothesis for the intended use population.

Information-adaptive clinical trials with selective recruitment and binary outcomes.

Selective recruitment designs preferentially recruit individuals who are estimated to be statistically informative onto a clinical trial. Individuals who are expected to contribute less information have a lower probability of recruitment. Furthermore, in an information-adaptive design, recruits are allocated to treatment arms in a manner that maximises information gain. The informativeness of an individual depends on their covariate (or biomarker) values, and how information is defined is a critical element of information-adaptive designs. In this paper, we define and evaluate four different methods for quantifying statistical information. Using both experimental data and numerical simulations, we show that selective recruitment designs can offer a substantial increase in statistical power compared with randomised designs. In trials without selective recruitment, we find that allocating individuals to treatment arms according to information-adaptive protocols also leads to an increase in statistical power. Consequently, selective recruitment designs can potentially achieve successful trials using fewer recruits thereby offering economic and ethical advantages. Copyright © 2017 John Wiley & Sons, Ltd.

Seven unconfirmed ideas to improve future ICU practice.

With imprecise definitions, inexact measurement tools, and flawed study execution, our clinical science often lags behind bedside experience and simply documents what appear to be the apparent faults or validity of ongoing practices. These impressions are later confirmed, modified, or overturned by the results of the next trial. On the other hand, insights that stem from the intuitions of experienced clinicians, scientists and educators-while often neglected-help place current thinking into proper perspective and occasionally point the way toward formulating novel hypotheses that direct future research. Both streams of information and opinion contribute to progress. In this paper we present a wide-ranging set of unproven 'out of the mainstream' ideas of our FCCM faculty, each with a defensible rationale and holding clear implications for altering bedside management. Each proposition was designed deliberately to be provocative so as to raise awareness, stimulate new thinking and initiate lively dialog.

A conceptual model for the development process of confirmatory adaptive clinical trials within an emergency research network.

BACKGROUND: Adaptive clinical trials use accumulating data from enrolled subjects to alter trial conduct in pre-specified ways based on quantitative decision rules. In this research, we sought to characterize the perspectives of key stakeholders during the development process of confirmatory-phase adaptive clinical trials within an emergency clinical trials network and to build a model to guide future development of adaptive clinical trials. METHODS: We used an ethnographic, qualitative approach to evaluate key stakeholders' views about the adaptive clinical trial development process. Stakeholders participated in a series of multidisciplinary meetings during the development of five adaptive clinical trials and completed a Strengths-Weaknesses-Opportunities-Threats questionnaire. In the analysis, we elucidated overarching themes across the stakeholders' responses to develop a conceptual model. RESULTS: Four major overarching themes emerged during the analysis of stakeholders' responses to questioning: the perceived statistical complexity of adaptive clinical trials and the roles of collaboration, communication, and time during the development process. Frequent and open communication and collaboration were viewed by stakeholders as critical during the development process, as were the careful management of time and logistical issues related to the complexity of planning adaptive clinical trials. CONCLUSION: The Adaptive Design Development Model illustrates how statistical complexity, time, communication, and collaboration are moderating factors in the adaptive design development process. The intensity and iterative nature of this process underscores the need for funding mechanisms for the development of novel trial proposals in academic settings.

Maximum type I error rate inflation from sample size reassessment when investigators are blind to treatment labels.

Consider a parallel group trial for the comparison of an experimental treatment to a control, where the second-stage sample size may depend on the blinded primary endpoint data as well as on additional blinded data from a secondary endpoint. For the setting of normally distributed endpoints, we demonstrate that this may lead to an inflation of the type I error rate if the null hypothesis holds for the primary but not the secondary endpoint. We derive upper bounds for the inflation of the type I error rate, both for trials that employ random allocation and for those that use block randomization. We illustrate the worst-case sample size reassessment rule in a case study. For both randomization strategies, the maximum type I error rate increases with the effect size in the secondary endpoint and the correlation between endpoints. The maximum inflation increases with smaller block sizes if information on the block size is used in the reassessment rule. Based on our findings, we do not question the well-established use of blinded sample size reassessment methods with nuisance parameter estimates computed from the blinded interim data of the primary endpoint. However, we demonstrate that the type I error rate control of these methods relies on the application of specific, binding, pre-planned and fully algorithmic sample size reassessment rules and does not extend to general or unplanned sample size adjustments based on blinded data. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Adaptive enrichment designs for clinical trials.

Modern medicine has graduated from broad spectrum treatments to targeted therapeutics. New drugs recognize the recently discovered heterogeneity of many diseases previously considered to be fairly homogeneous. These treatments attack specific genetic pathways which are only dysregulated in some smaller subset of patients with the disease. Often this subset is only rudimentarily understood until well into large-scale clinical trials. As such, standard practice has been to enroll a broad range of patients and run post hoc subset analysis to determine those who may particularly benefit. This unnecessarily exposes many patients to hazardous side effects, and may vastly decrease the efficiency of the trial (especially if only a small subset of patients benefit). In this manuscript, we propose a class of adaptive enrichment designs that allow the eligibility criteria of a trial to be adaptively updated during the trial, restricting entry to patients likely to benefit from the new treatment. We show that our designs both preserve the type 1 error, and in a variety of cases provide a substantial increase in power.

Using short-term endpoints to improve interim decision making and trial duration in two-stage phase II trials with nested binary endpoints.

In oncology, phase II clinical trials are often planned as single-arm two-stage designs with a binary endpoint, for example, progression-free survival after 12 months, and the option to stop for futility after the first stage. Simon's two-stage design is a very popular approach but depending on the follow-up time required to measure the patients' outcomes the trial may have to be paused undesirably long. To shorten this forced interruption, it was proposed to use a short-term endpoint for the interim decision, such as progression-free survival after 3 months. We show that if the assumptions for the short-term endpoint are misspecified, the decision-making in the interim can be misleading, resulting in a great loss of statistical power. For the setting of a binary endpoint with nested measurements, such as progression-free survival, we propose two approaches that utilize all available short-term and long-term assessments of the endpoint to guide the interim decision. One approach is based on conditional power and the other is based on Bayesian posterior predictive probability of success. In extensive simulations, we show that both methods perform similarly, when appropriately calibrated, and can greatly improve power compared to the existing approach in settings with slow patient recruitment. Software code to implement the methods is made publicly available.

Towards achieving transnational research partnership equity: lessons from implementing adaptive platform trials in low- and middle-income countries.

Background: Use of adaptive clinical trials, particularly adaptive platform trials, has grown exponentially in response to the coronavirus disease (COVID-19) pandemic. Implementation of these trials in low- and middle-income countries (LMICs) has been fostered through the formation or modification of transnational research partnerships, typically between research groups from LMICs and high-income countries (HICs). While these partnerships are important to promote collaboration and overcome the structural and economic disadvantages faced by LMIC health researchers, it is critical to focus attention on the multiple dimensions of partnership equity. Methods: Based on informal literature reviews and a meeting with leaders of one of the multinational COVID-19 adaptive platform trials, we describe some important considerations about research partnership equity in this context. Results: We organize these considerations into eight thematic categories: 1) epistemic structures, 2) funding, 3) ethics oversight, 4) regulatory oversight, 5) leadership, 6) post-trial access to interventions, data, and specimens, 7) knowledge translation and dissemination, and 8) research capacity strengthening and maintenance. Within each category we review normative claims that support its relevance to research partnership equity followed by discussion of how adaptive platform trials highlight new dimensions, considerations, or challenges. Conclusion: In aggregate, these observations provide insight into procedural and substantive equity-building measures within transnational global health research partnerships more broadly.

Finite-Sample Two-Group Composite Hypothesis Testing via Machine Learning.

In the problem of composite hypothesis testing, identifying the potential uniformly most powerful (UMP) unbiased test is of great interest. Beyond typical hypothesis settings with exponential family, it is usually challenging to prove the existence and further construct such UMP unbiased tests with finite sample size. For example in the COVID-19 pandemic with limited previous assumptions on the treatment for investigation and the standard of care, adaptive clinical trials are appealing due to ethical considerations, and the ability to accommodate uncertainty while conducting the trial. Although several methods have been proposed to control Type I error rates, how to find a more powerful hypothesis testing strategy is still an open question. Motivated by this problem, we propose an automatic framework of constructing test statistics and corresponding critical values via machine learning methods to enhance power in a finite sample. In this article, we particularly illustrate the performance using Deep Neural Networks (DNN) and discuss its advantages. Simulations and two case studies of adaptive designs demonstrate that our method is automatic, general and prespecified to construct statistics with satisfactory power in finite-sample. Supplemental materials are available online including R code and an R shiny app.

Choosing Interim Sample Sizes in Group Sequential Designs.

This manuscript investigates sample sizes for interim analyses in group sequential designs. Traditional group sequential designs (GSD) rely on "information fraction" arguments to define the interim sample sizes. Then, interim maximum likelihood estimators (MLEs) are used to decide whether to stop early or continue the data collection until the next interim analysis. The possibility of early stopping changes the distribution of interim and final MLEs: possible interim decisions on trial stopping excludes some sample space elements. At each interim analysis the distribution of an interim MLE is a mixture of truncated and untruncated distributions. The distributional form of an MLE becomes more and more complicated with each additional interim analysis. Test statistics that are asymptotically normal without a possibly of early stopping, become mixtures of truncated normal distributions under local alternatives. Stage-specific information ratios are equivalent to sample size ratios for independent and identically distributed data. This equivalence is used to justify interim sample sizes in GSDs. Because stage-specific information ratios derived from normally distributed data differ from those derived from non-normally distributed data, the former equivalence is invalid when there is a possibility of early stopping. Tarima and Flournoy [3] have proposed a new GSD where interim sample sizes are determined by a pre-defined sequence of ordered alternative hypotheses, and the calculation of information fractions is not needed. This innovation allows researchers to prescribe interim analyses based on desired power properties. This work compares interim power properties of a classical one-sided three stage Pocock design with a one-sided three stage design driven by three ordered alternatives.

Single-arm oncology trials and the nature of external controls arms.

Aim: Single-arm trials with external control arms (ECAs) have gained popularity in oncology. ECAs may consist of primary data from previous trials, electronic health records (EHRs) or aggregate data from the literature. We sought to provide a description of how such studies achieve similarity of patients, comparability of data quality and outcome assessment. Materials & methods: In a stratified convenience sample of 15 studies, five used primary data from trials as ECAs, five used secondary data from EHRs and five used aggregate data from the literature. Data were collected from the published literature and public web resources, blinded to the eventual approval decision. Results: Studies using ECAs from primary data and EHR data displayed methods to achieve comparability of information, including matched baseline characteristics. Aggregate data from published studies did not attempt to match covariates. The EHR controls often showed calendar time overlap for collecting information while trial data were mostly historic. Outcome data were not consistently reported across studies. US FDA approval was only seen when primary data from trials or EHR data were used as the ECA, however no ECA in this sample directly contributed to approval. Discussion: In this nonsystematic review of ECAs for single-arm trials, the ECAs derived from primary data collected by other trials or EHRs show patterns of patient comparability, time overlap, and realistic methodological approaches to achieving balance between treatment arms. They are often submitted to regulators while literature-derived aggregate findings as ECA may serve as benchmarks for pipeline decisions.

Stakeholder perspectives on adaptive clinical trials: a scoping review.

BACKGROUND: Adaptive clinical trials (ACTs) represent an emerging approach to trial design where accumulating data are used to make decisions about future conduct. Adaptations can include comparisons of multiple dose tiers, response-adaptive randomization, sample size re-estimation, and efficacy/futility stopping rules. The objective of this scoping review is to assess stakeholder attitudes, perspectives, and understanding of adaptive trials. METHODS: We conducted a review of articles examining stakeholders encompassing the broad medical trial community's perspectives of adaptive designs (ADs). A computerized search was conducted of four electronic databases with relevant search terms. Following screening of articles, the primary findings of each included article were coded for study design, population studied, purpose, and primary implications. RESULTS: Our team retrieved 167 peer-reviewed titles in total from the database search and 5 additional titles through searching web-based search engines for gray literature. Of those 172 titles, 152 were non-duplicate citations. Of these, 119 were not given full-text reviews, as their titles and abstracts indicated that they did not meet the inclusion criteria. Thirty-three articles were carefully examined for relevance, and of those, 18 were chosen to be part of the analysis; the other 15 were excluded, as they were not relevant upon closer inspection. Perceived advantages to ADs included limiting ineffective treatments and efficiency in answering the research question; -perceived barriers included insufficient sample size for secondary outcomes, challenges of consent, potential for bias, risk of type 1 error, cost and time to adaptively design trials, unclear rationales for using Ads, and, most importantly, a lack of education regarding ADs among stakeholders within the clinical trial community. Perceptions among different types of stakeholders varied from sector to sector, with patient perspectives being noticeably absent from the literature. CONCLUSION: There are diverse perceptions regarding ADs among stakeholders. Further training, guidelines, and toolkits on the proper use of ADs are needed at all levels to overcome many of these perceived barriers. While education for principal investigators is important, it is also crucial to educate other groups in the community, such as patients, as well as clinicians and staff involved in their daily implementation.

Bringing New Meaning to the Term "Adaptive Trial": Challenges of Conducting Clinical Research During the Coronavirus Disease 2019 Pandemic and Implications for Implementation Science.

Although implementation of evidence-based practices takes an average of 17 years, in the context of the global pandemic, coronavirus disease 2019 (COVID-19) interventions were adopted in a greatly compressed time frame. This rapid uptake creates major challenges for conducting COVID-19 clinical research studies, because quickly evolving standards make it difficult to adapt in real time. The rapid dissemination and implementation of COVID-19 interventions is the realization of goals long pursued by the implementation science community. However, the downside of the rapid implementation is that low-quality evidence with little to no scientific vetting may be quickly integrated into clinical care, resulting in lost opportunities to advance our scientific understanding about how to manage infected patients. In the future, novel adaptive designs embedded into electronic health records (Embedded Quantified, Integrated-into-Practice Trial [EQuIPT] designs) that allow for easier and better access to clinical trials may simultaneously improve care and advance healthcare innovations.

Maintenance therapy and need for cessation studies in multiple myeloma: Focus on the future.

With ten years of follow-up since the advent of the modern paradigm of combination induction therapy, consolidative autologous stem-cell transplant, and lenalidomide maintenance, median survival for multiple myeloma has increased to almost 50% at 10 years. Given this outlook, the overarching goal of maintenance therapy is to spare patients from the toxicities of aggressive or otherwise intrusive therapies while ideally extending survival or, at the least, extending progression-free survival or time until next treatment. This review will focus on the current landscape of maintenance therapies for multiple myeloma. The historical context and evidence for choice of agent, duration of treatment, and current strategies and ongoing trials will be discussed - as well as a focus on unmet needs. The case for studies investigating cessation of therapy and risk and response-adapted approaches will be underscored given that the current paradigm likely results in overtreatment for some patients.

The Oxford Royal College of General Practitioners Clinical Informatics Digital Hub: Protocol to Develop Extended COVID-19 Surveillance and Trial Platforms.

BACKGROUND: Routinely recorded primary care data have been used for many years by sentinel networks for surveillance. More recently, real world data have been used for a wider range of research projects to support rapid, inexpensive clinical trials. Because the partial national lockdown in the United Kingdom due to the coronavirus disease (COVID-19) pandemic has resulted in decreasing community disease incidence, much larger numbers of general practices are needed to deliver effective COVID-19 surveillance and contribute to in-pandemic clinical trials. OBJECTIVE: The aim of this protocol is to describe the rapid design and development of the Oxford Royal College of General Practitioners Clinical Informatics Digital Hub (ORCHID) and its first two platforms. The Surveillance Platform will provide extended primary care surveillance, while the Trials Platform is a streamlined clinical trials platform that will be integrated into routine primary care practice. METHODS: We will apply the FAIR (Findable, Accessible, Interoperable, and Reusable) metadata principles to a new, integrated digital health hub that will extract routinely collected general practice electronic health data for use in clinical trials and provide enhanced communicable disease surveillance. The hub will be findable through membership in Health Data Research UK and European metadata repositories. Accessibility through an online application system will provide access to study-ready data sets or developed custom data sets. Interoperability will be facilitated by fixed linkage to other key sources such as Hospital Episodes Statistics and the Office of National Statistics using pseudonymized data. All semantic descriptors (ie, ontologies) and code used for analysis will be made available to accelerate analyses. We will also make data available using common data models, starting with the US Food and Drug Administration Sentinel and Observational Medical Outcomes Partnership approaches, to facilitate international studies. The Surveillance Platform will provide access to data for health protection and promotion work as authorized through agreements between Oxford, the Royal College of General Practitioners, and Public Health England. All studies using the Trials Platform will go through appropriate ethical and other regulatory approval processes. RESULTS: The hub will be a bottom-up, professionally led network that will provide benefits for member practices, our health service, and the population served. Data will only be used for SQUIRE (surveillance, quality improvement, research, and education) purposes. We have already received positive responses from practices, and the number of practices in the network has doubled to over 1150 since February 2020. COVID-19 surveillance has resulted in tripling of the number of virology sites to 293 (target 300), which has aided the collection of the largest ever weekly total of surveillance swabs in the United Kingdom as well as over 3000 severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) serology samples. Practices are recruiting to the PRINCIPLE (Platform Randomised trial of INterventions against COVID-19 In older PeopLE) trial, and these participants will be followed up through ORCHID. These initial outputs demonstrate the feasibility of ORCHID to provide an extended national digital health hub. CONCLUSIONS: ORCHID will provide equitable and innovative use of big data through a professionally led national primary care network and the application of FAIR principles. The secure data hub will host routinely collected general practice data linked to other key health care repositories for clinical trials and support enhanced in situ surveillance without always requiring large volume data extracts. ORCHID will support rapid data extraction, analysis, and dissemination with the aim of improving future research and development in general practice to positively impact patient care. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/19773.

21st-Century Oncology Drug Safety via New-Age Regulatory Standards and Practices.

There is no such thing as a drug that is 100% safe or effective. Determining whether or not a new oncology treatment (or an additional indication for an existing medicine) should be approved by a regulatory licensing authority is, ultimately, as much regulatory science as public health art and nuance. There are many dynamic shifts in regulatory science (expedited review pathways, biomarker validation, use of real-world evidence, expanded off-label usage, etc) interpreted and expressed within the context of 21st-century oncology drug development, and these new tools and the learnings gleaned from them are helping to advance patient care. They are also helping us to carefully reconsider the levels of uncertainty we find in benefit-risk data and clinical calculations. New-Age Pharmacovigilance can be a tool in product development, regulatory review, postmarketing surveillance and enhanced clinical outcomes.