Comparison of Bayesian and frequentist monitoring boundaries motivated by the Multiplatform Randomized Clinical Trial.

BACKGROUND: The coronavirus disease 2019 pandemic highlighted the need to conduct efficient randomized clinical trials with interim monitoring guidelines for efficacy and futility. Several randomized coronavirus disease 2019 trials, including the Multiplatform Randomized Clinical Trial (mpRCT), used Bayesian guidelines with the belief that they would lead to quicker efficacy or futility decisions than traditional "frequentist" guidelines, such as spending functions and conditional power. We explore this belief using an intuitive interpretation of Bayesian methods as translating prior opinion about the treatment effect into imaginary prior data. These imaginary observations are then combined with actual observations from the trial to make conclusions. Using this approach, we show that the Bayesian efficacy boundary used in mpRCT is actually quite similar to the frequentist Pocock boundary. METHODS: The mpRCT's efficacy monitoring guideline considered stopping if, given the observed data, there was greater than 99% probability that the treatment was effective (odds ratio greater than 1). The mpRCT's futility monitoring guideline considered stopping if, given the observed data, there was greater than 95% probability that the treatment was less than 20% effective (odds ratio less than 1.2). The mpRCT used a normal prior distribution that can be thought of as supplementing the actual patients' data with imaginary patients' data. We explore the effects of varying probability thresholds and the prior-to-actual patient ratio in the mpRCT and compare the resulting Bayesian efficacy monitoring guidelines to the well-known frequentist Pocock and O'Brien-Fleming efficacy guidelines. We also contrast Bayesian futility guidelines with a more traditional 20% conditional power futility guideline. RESULTS: A Bayesian efficacy and futility monitoring boundary using a neutral, weakly informative prior distribution and a fixed probability threshold at all interim analyses is more aggressive than the commonly used O'Brien-Fleming efficacy boundary coupled with a 20% conditional power threshold for futility. The trade-off is that more aggressive boundaries tend to stop trials earlier, but incur a loss of power. Interestingly, the Bayesian efficacy boundary with 99% probability threshold is very similar to the classic Pocock efficacy boundary. CONCLUSIONS: In a pandemic where quickly weeding out ineffective treatments and identifying effective treatments is paramount, aggressive monitoring may be preferred to conservative approaches, such as the O'Brien-Fleming boundary. This can be accomplished with either Bayesian or frequentist methods.

Changing interim monitoring in response to internal clinical trial data.

Designing clinical trials for emerging infectious diseases such as COVID-19 is challenging because information needed for proper planning may be lacking. Pre-specified adaptive designs can be attractive options, but what happens if a trial with no such design needs to be modified? For example, unexpectedly high efficacy (approximately 95%) in two COVID-19 vaccine trials might cause investigators in other COVID-19 vaccine trials to increase the number of interim analyses to allow earlier stopping for efficacy. If such a decision is based solely on external data, there are no issues, but what if internal trial data by arm are also examined? Fortunately, the conditional error principle of Müller and Schäfer (2004) can be used to ensure no inflation of the type 1 error rate, even if no interim analyses were planned. We study the properties, including limitations, of this method. We provide a shiny app to evaluate changes in timing of interim analyses in response to outcome data by arm in clinical trials.

Statistics of Rare Events.

Rare events can sometime arise in clinical development of treatments. For example, CYPIDES was a single-arm study of the CYP11A1 inhibitor ODM-208 to treat metastatic prostate cancer.1 Preclinical testing of the compound identified elevated thyroid-stimulating hormone (TSH) and bilirubin in rats and dogs. Unusual findings in preclinical testing focus attention and magnify evidence if similar results occur in humans. By analogy, imagine a murder trial in which the only evidence against the defendant arose from a database search of DNA matching the partial profile found at the crime scene. Multiple people could match, so without other evidence, the perpetrator could be any of them.

Persistence of Human Immunodeficiency Virus-Associated Cerebral Toxoplasmosis Lesions in Successfully Treated Patients Receiving Combination Antiretroviral Therapy.

Background: Toxoplasmic encephalitis (TE) is a life-threatening complication of people with human immunodeficiency virus (PWH) with severe immunodeficiency, especially those with a CD4+ T-cell count <100 cells/µL. Following a clinical response to anti-Toxoplasma therapy, and immune reconstitution after initiation of combination antiretroviral therapy (ART), anti-Toxoplasma therapy can be discontinued with a low risk of relapse. Methods: To better understand the evolution of magnetic resonance imaging (MRI)-defined TE lesions in PWH receiving ART, we undertook a retrospective study of PWH initially seen at the National Institutes of Health between 2001 and 2012, who had at least 2 serial MRI scans. Lesion size and change over time were calculated and correlated with clinical parameters. Results: Among 24 PWH with TE and serial MRI scans, only 4 had complete clearance of lesions at the last MRI (follow-up, 0.09-5.8 years). Of 10 PWH off all anti-Toxoplasma therapy (median, 3.2 years after TE diagnosis), 6 had persistent MRI enhancement. In contrast, all 5 PWH seen in a pre-ART era study who were followed for >6 months had complete clearance of lesions. TE lesion area at diagnosis was associated with the absolute change in area (P < .0001). Conclusions: Contrast enhancement can persist even when TE has been successfully treated and anti-Toxoplasma therapy has been stopped, highlighting the need to consider diagnostic alternatives in successfully treated patients with immune reconstitution presenting with new neurologic symptoms.

A note on familywise error rate for a primary and secondary endpoint.

Hung et al. (2007) considered the problem of controlling the type I error rate for a primary and secondary endpoint in a clinical trial using a gatekeeping approach in which the secondary endpoint is tested only if the primary endpoint crosses its monitoring boundary. They considered a two-look trial and showed by simulation that the naive method of testing the secondary endpoint at full level α at the time the primary endpoint reaches statistical significance does not control the familywise error rate at level α. Tamhane et al. (2010) derived analytic expressions for familywise error rate and power and confirmed the inflated error rate of the naive approach. Nonetheless, many people mistakenly believe that the closure principle can be used to prove that the naive procedure controls the familywise error rate. The purpose of this note is to explain in greater detail why there is a problem with the naive approach and show that the degree of alpha inflation can be as high as that of unadjusted monitoring of a single endpoint.

Interval between prior SARS-CoV-2 infection and booster vaccination impacts magnitude and quality of antibody and B cell responses.

SARS-CoV-2 mRNA booster vaccines provide protection from severe disease, eliciting strong immunity that is further boosted by previous infection. However, it is unclear whether these immune responses are affected by the interval between infection and vaccination. Over a 2-month period, we evaluated antibody and B cell responses to a third-dose mRNA vaccine in 66 individuals with different infection histories. Uninfected and post-boost but not previously infected individuals mounted robust ancestral and variant spike-binding and neutralizing antibodies and memory B cells. Spike-specific B cell responses from recent infection (<180 days) were elevated at pre-boost but comparatively less so at 60 days post-boost compared with uninfected individuals, and these differences were linked to baseline frequencies of CD27lo B cells. Day 60 to baseline ratio of BCR signaling measured by phosphorylation of Syk was inversely correlated to days between infection and vaccination. Thus, B cell responses to booster vaccines are impeded by recent infection.

Recent SARS-CoV-2 infection abrogates antibody and B-cell responses to booster vaccination.

SARS-CoV-2 mRNA booster vaccines provide protection from severe disease, eliciting strong immunity that is further boosted by previous infection. However, it is unclear whether these immune responses are affected by the interval between infection and vaccination. Over a two-month period, we evaluated antibody and B-cell responses to a third dose mRNA vaccine in 66 individuals with different infection histories. Uninfected and post-boost but not previously infected individuals mounted robust ancestral and variant spike-binding and neutralizing antibodies, and memory B cells. Spike-specific B-cell responses from recent infection were elevated at pre-boost but comparatively less so at 60 days post-boost compared to uninfected individuals, and these differences were linked to baseline frequencies of CD27 lo B cells. Day 60 to baseline ratio of BCR signaling measured by phosphorylation of Syk was inversely correlated to days between infection and vaccination. Thus, B-cell responses to booster vaccines are impeded by recent infection.

Combination anti-HIV antibodies provide sustained virological suppression.

Antiretroviral therapy is highly effective in suppressing human immunodeficiency virus (HIV)1. However, eradication of the virus in individuals with HIV has not been possible to date2. Given that HIV suppression requires life-long antiretroviral therapy, predominantly on a daily basis, there is a need to develop clinically effective alternatives that use long-acting antiviral agents to inhibit viral replication3. Here we report the results of a two-component clinical trial involving the passive transfer of two HIV-specific broadly neutralizing monoclonal antibodies, 3BNC117 and 10-1074. The first component was a randomized, double-blind, placebo-controlled trial that enrolled participants who initiated antiretroviral therapy during the acute/early phase of HIV infection. The second component was an open-label single-arm trial that enrolled individuals with viraemic control who were naive to antiretroviral therapy. Up to 8 infusions of 3BNC117 and 10-1074, administered over a period of 24 weeks, were well tolerated without any serious adverse events related to the infusions. Compared with the placebo, the combination broadly neutralizing monoclonal antibodies maintained complete suppression of plasma viraemia (for up to 43 weeks) after analytical treatment interruption, provided that no antibody-resistant HIV was detected at the baseline in the study participants. Similarly, potent HIV suppression was seen in the antiretroviral-therapy-naive study participants with viraemia carrying sensitive virus at the baseline. Our data demonstrate that combination therapy with broadly neutralizing monoclonal antibodies can provide long-term virological suppression without antiretroviral therapy in individuals with HIV, and our experience offers guidance for future clinical trials involving next-generation antibodies with long half-lives.

Data and Safety Monitoring Board Monitoring of Clinical Trials for Early Efficacy.

DSMB Monitoring of Clinical Trials for Early EfficacyIn this article in the NEJM Evidence DSMB mini-series, Dodd and Proschan review the clinical, ethical, and statistical issues that confront a DSMB when trial data suggest that the treatment under test is highly efficacious. Examples of such trials are reviewed.

Discussion on "Improving precision and power in randomized trials for COVID-19 treatments using covariate adjustment for binary, ordinal, and time-to-event outcomes".

Benkeser et al. present a very informative paper evaluating the efficiency gains of covariate adjustment in settings with binary, ordinal, and time-to-event outcomes. The adjustment method focuses on estimating the marginal treatment effect averaged over the covariate distribution in both arms combined. The authors show that covariate adjustment can achieve power gains that could find answers more quickly. The suggested approach is an important weapon in the armamentarium against epidemics like COVID-19. I recommend evaluating the procedure against more traditional approaches for conditional analyses (e.g., logistic regression) and against blinded methods of building prediction models followed by randomization-based inference.

Diagnosing fraudulent baseline data in clinical trials.

The first table in many articles reporting results of a randomized clinical trial compares baseline factors across arms. Results that appear inconsistent with chance trigger suspicion, and in one case, accusation and confirmation of data falsification. We confirm theoretically results of simulation analyses showing that inconsistency with chance is extremely difficult to prove in the absence of any information about correlations between baseline covariates. We offer a reasonable diagnostic to trigger further investigation.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design.

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design.

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

A primer on strong vs weak control of familywise error rate.

Multiple comparison adjustments have a long history, yet confusion remains about which procedures control type 1 error rate in a strong sense and how to show this. Part of the confusion stems from a powerful technique called the closed testing principle, whose statement is deceptively simple, but is sometimes misinterpreted. This primer presents a straightforward way to think about multiplicity adjustment.

An open-label phase 1 clinical trial of the anti-α4ß7 monoclonal antibody vedolizumab in HIV-infected individuals.

Despite the substantial clinical benefits of antiretroviral therapy (ART), complete eradication of HIV has not been possible. The gastrointestinal tract and associated lymphoid tissues may play an important role in the pathogenesis of HIV infection. The integrin α4ß7 facilitates homing of T lymphocytes to the gut by binding to the mucosal addressin cell adhesion molecule-1 (MAdCAM-1) expressed on venules in gut-associated lymphoid tissue. CD4+ T cells with increased expression of α4ß7 are susceptible to HIV infection and may be key players in subsequent virus dissemination. Data from nonhuman primate models infected with simian immunodeficiency virus (SIV) have suggested that blockade of the α4ß7/MAdCAM-1 interaction may be effective at preventing SIV infection and may have beneficial effects in animals with established viral infection. To explore whether these findings could be reproduced in HIV-infected individuals after interruption of ART, we conducted an open-label phase 1 clinical trial of vedolizumab, a monoclonal antibody against α4ß7 integrin. Vedolizumab infusions in 20 HIV-infected individuals were well tolerated with no serious adverse events related to the study drug. After interruption of ART, the median time to meeting protocol criteria to restart therapy was 13 weeks. The median duration of plasma viremia of <400 copies/ml was 5.4 weeks. Only a single subject in the trial experienced prolonged suppression of plasma viremia after interruption of ART. These results suggest that blockade of α4ß7 may not be an effective strategy for inducing virological remission in HIV-infected individuals after ART interruption.

Re-randomization tests in clinical trials.

As randomization methods use more information in more complex ways to assign patients to treatments, analysis of the resulting data becomes challenging. The treatment assignment vector and outcome vector become correlated whenever randomization probabilities depend on data correlated with outcomes. One straightforward analysis method is a re-randomization test that fixes outcome data and creates a reference distribution for the test statistic by repeatedly re-randomizing according to the same randomization method used in the trial. This article reviews re-randomization tests, especially in nonstandard settings like covariate-adaptive and response-adaptive randomization. We show that re-randomization tests provide valid inference in a wide range of settings. Nonetheless, there are simple examples demonstrating limitations.

Development process of a consensus-driven CONSORT extension for randomised trials using an adaptive design.

BACKGROUND: Adequate reporting of adaptive designs (ADs) maximises their potential benefits in the conduct of clinical trials. Transparent reporting can help address some obstacles and concerns relating to the use of ADs. Currently, there are deficiencies in the reporting of AD trials. To overcome this, we have developed a consensus-driven extension to the CONSORT statement for randomised trials using an AD. This paper describes the processes and methods used to develop this extension rather than detailed explanation of the guideline. METHODS: We developed the guideline in seven overlapping stages: 1) Building on prior research to inform the need for a guideline; 2) A scoping literature review to inform future stages; 3) Drafting the first checklist version involving an External Expert Panel; 4) A two-round Delphi process involving international, multidisciplinary, and cross-sector key stakeholders; 5) A consensus meeting to advise which reporting items to retain through voting, and to discuss the structure of what to include in the supporting explanation and elaboration (E&E) document; 6) Refining and finalising the checklist; and 7) Writing-up and dissemination of the E&E document. The CONSORT Executive Group oversaw the entire development process. RESULTS: Delphi survey response rates were 94/143 (66%), 114/156 (73%), and 79/143 (55%) in rounds 1, 2, and across both rounds, respectively. Twenty-seven delegates from Europe, the USA, and Asia attended the consensus meeting. The main checklist has seven new and nine modified items and six unchanged items with expanded E&E text to clarify further considerations for ADs. The abstract checklist has one new and one modified item together with an unchanged item with expanded E&E text. The E&E document will describe the scope of the guideline, the definition of an AD, and some types of ADs and trial adaptations and explain each reporting item in detail including case studies. CONCLUSIONS: We hope that making the development processes, methods, and all supporting information that aided decision-making transparent will enhance the acceptability and quick uptake of the guideline. This will also help other groups when developing similar CONSORT extensions. The guideline is applicable to all randomised trials with an AD and contains minimum reporting requirements.

A randomized controlled safety/efficacy trial of therapeutic vaccination in HIV-infected individuals who initiated antiretroviral therapy early in infection.

Despite substantial clinical benefits, complete eradication of HIV has not been possible using antiretroviral therapy (ART) alone. Strategies that can either eliminate persistent viral reservoirs or boost host immunity to prevent rebound of virus from these reservoirs after discontinuation of ART are needed; one possibility is therapeutic vaccination. We report the results of a randomized, placebo-controlled trial of a therapeutic vaccine regimen in patients in whom ART was initiated during the early stage of HIV infection and whose immune system was anticipated to be relatively intact. The objectives of our study were to determine whether the vaccine was safe and could induce an immune response that would maintain suppression of plasma viremia after discontinuation of ART. Vaccinations were well tolerated with no serious adverse events but produced only modest augmentation of existing HIV-specific CD4+ T cell responses, with little augmentation of CD8+ T cell responses. Compared with placebo, the vaccination regimen had no significant effect on the kinetics or magnitude of viral rebound after interruption of ART and no impact on the size of the HIV reservoir in the CD4+ T cell compartment. Notably, 26% of subjects in the placebo arm exhibited sustained suppression of viremia (<400 copies/ml) after treatment interruption, a rate of spontaneous suppression higher than previously reported. Our findings regarding the degree and kinetics of plasma viral rebound after ART interruption have potentially important implications for the design of future trials testing interventions aimed at achieving ART-free control of HIV infection.