Comparison of Bayesian and frequentist monitoring boundaries motivated by the Multiplatform Randomized Clinical Trial.

BACKGROUND: The coronavirus disease 2019 pandemic highlighted the need to conduct efficient randomized clinical trials with interim monitoring guidelines for efficacy and futility. Several randomized coronavirus disease 2019 trials, including the Multiplatform Randomized Clinical Trial (mpRCT), used Bayesian guidelines with the belief that they would lead to quicker efficacy or futility decisions than traditional "frequentist" guidelines, such as spending functions and conditional power. We explore this belief using an intuitive interpretation of Bayesian methods as translating prior opinion about the treatment effect into imaginary prior data. These imaginary observations are then combined with actual observations from the trial to make conclusions. Using this approach, we show that the Bayesian efficacy boundary used in mpRCT is actually quite similar to the frequentist Pocock boundary. METHODS: The mpRCT's efficacy monitoring guideline considered stopping if, given the observed data, there was greater than 99% probability that the treatment was effective (odds ratio greater than 1). The mpRCT's futility monitoring guideline considered stopping if, given the observed data, there was greater than 95% probability that the treatment was less than 20% effective (odds ratio less than 1.2). The mpRCT used a normal prior distribution that can be thought of as supplementing the actual patients' data with imaginary patients' data. We explore the effects of varying probability thresholds and the prior-to-actual patient ratio in the mpRCT and compare the resulting Bayesian efficacy monitoring guidelines to the well-known frequentist Pocock and O'Brien-Fleming efficacy guidelines. We also contrast Bayesian futility guidelines with a more traditional 20% conditional power futility guideline. RESULTS: A Bayesian efficacy and futility monitoring boundary using a neutral, weakly informative prior distribution and a fixed probability threshold at all interim analyses is more aggressive than the commonly used O'Brien-Fleming efficacy boundary coupled with a 20% conditional power threshold for futility. The trade-off is that more aggressive boundaries tend to stop trials earlier, but incur a loss of power. Interestingly, the Bayesian efficacy boundary with 99% probability threshold is very similar to the classic Pocock efficacy boundary. CONCLUSIONS: In a pandemic where quickly weeding out ineffective treatments and identifying effective treatments is paramount, aggressive monitoring may be preferred to conservative approaches, such as the O'Brien-Fleming boundary. This can be accomplished with either Bayesian or frequentist methods.

Changing interim monitoring in response to internal clinical trial data.

Designing clinical trials for emerging infectious diseases such as COVID-19 is challenging because information needed for proper planning may be lacking. Pre-specified adaptive designs can be attractive options, but what happens if a trial with no such design needs to be modified? For example, unexpectedly high efficacy (approximately 95%) in two COVID-19 vaccine trials might cause investigators in other COVID-19 vaccine trials to increase the number of interim analyses to allow earlier stopping for efficacy. If such a decision is based solely on external data, there are no issues, but what if internal trial data by arm are also examined? Fortunately, the conditional error principle of Müller and Schäfer (2004) can be used to ensure no inflation of the type 1 error rate, even if no interim analyses were planned. We study the properties, including limitations, of this method. We provide a shiny app to evaluate changes in timing of interim analyses in response to outcome data by arm in clinical trials.

Statistics of Rare Events.

Rare events can sometime arise in clinical development of treatments. For example, CYPIDES was a single-arm study of the CYP11A1 inhibitor ODM-208 to treat metastatic prostate cancer.1 Preclinical testing of the compound identified elevated thyroid-stimulating hormone (TSH) and bilirubin in rats and dogs. Unusual findings in preclinical testing focus attention and magnify evidence if similar results occur in humans. By analogy, imagine a murder trial in which the only evidence against the defendant arose from a database search of DNA matching the partial profile found at the crime scene. Multiple people could match, so without other evidence, the perpetrator could be any of them.

Abatacept, Cenicriviroc, or Infliximab for Treatment of Adults Hospitalized With COVID-19 Pneumonia: A Randomized Clinical Trial.

Importance: Immune dysregulation contributes to poorer outcomes in COVID-19. Objective: To investigate whether abatacept, cenicriviroc, or infliximab provides benefit when added to standard care for COVID-19 pneumonia. Design, Setting, and Participants: Randomized, double-masked, placebo-controlled clinical trial using a master protocol to investigate immunomodulators added to standard care for treatment of participants hospitalized with COVID-19 pneumonia. The results of 3 substudies are reported from 95 hospitals at 85 clinical research sites in the US and Latin America. Hospitalized patients 18 years or older with confirmed SARS-CoV-2 infection within 14 days and evidence of pulmonary involvement underwent randomization between October 2020 and December 2021. Interventions: Single infusion of abatacept (10 mg/kg; maximum dose, 1000 mg) or infliximab (5 mg/kg) or a 28-day oral course of cenicriviroc (300-mg loading dose followed by 150 mg twice per day). Main Outcomes and Measures: The primary outcome was time to recovery by day 28 evaluated using an 8-point ordinal scale (higher scores indicate better health). Recovery was defined as the first day the participant scored at least 6 on the ordinal scale. Results: Of the 1971 participants randomized across the 3 substudies, the mean (SD) age was 54.8 (14.6) years and 1218 (61.8%) were men. The primary end point of time to recovery from COVID-19 pneumonia was not significantly different for abatacept (recovery rate ratio [RRR], 1.12 [95% CI, 0.98-1.28]; P = .09), cenicriviroc (RRR, 1.01 [95% CI, 0.86-1.18]; P = .94), or infliximab (RRR, 1.12 [95% CI, 0.99-1.28]; P = .08) compared with placebo. All-cause 28-day mortality was 11.0% for abatacept vs 15.1% for placebo (odds ratio [OR], 0.62 [95% CI, 0.41-0.94]), 13.8% for cenicriviroc vs 11.9% for placebo (OR, 1.18 [95% CI 0.72-1.94]), and 10.1% for infliximab vs 14.5% for placebo (OR, 0.59 [95% CI, 0.39-0.90]). Safety outcomes were comparable between active treatment and placebo, including secondary infections, in all 3 substudies. Conclusions and Relevance: Time to recovery from COVID-19 pneumonia among hospitalized participants was not significantly different for abatacept, cenicriviroc, or infliximab vs placebo. Trial Registration: ClinicalTrials.gov Identifier: NCT04593940.

Persistence of Human Immunodeficiency Virus-Associated Cerebral Toxoplasmosis Lesions in Successfully Treated Patients Receiving Combination Antiretroviral Therapy.

Background: Toxoplasmic encephalitis (TE) is a life-threatening complication of people with human immunodeficiency virus (PWH) with severe immunodeficiency, especially those with a CD4+ T-cell count <100 cells/µL. Following a clinical response to anti-Toxoplasma therapy, and immune reconstitution after initiation of combination antiretroviral therapy (ART), anti-Toxoplasma therapy can be discontinued with a low risk of relapse. Methods: To better understand the evolution of magnetic resonance imaging (MRI)-defined TE lesions in PWH receiving ART, we undertook a retrospective study of PWH initially seen at the National Institutes of Health between 2001 and 2012, who had at least 2 serial MRI scans. Lesion size and change over time were calculated and correlated with clinical parameters. Results: Among 24 PWH with TE and serial MRI scans, only 4 had complete clearance of lesions at the last MRI (follow-up, 0.09-5.8 years). Of 10 PWH off all anti-Toxoplasma therapy (median, 3.2 years after TE diagnosis), 6 had persistent MRI enhancement. In contrast, all 5 PWH seen in a pre-ART era study who were followed for >6 months had complete clearance of lesions. TE lesion area at diagnosis was associated with the absolute change in area (P < .0001). Conclusions: Contrast enhancement can persist even when TE has been successfully treated and anti-Toxoplasma therapy has been stopped, highlighting the need to consider diagnostic alternatives in successfully treated patients with immune reconstitution presenting with new neurologic symptoms.

A note on familywise error rate for a primary and secondary endpoint.

Hung et al. (2007) considered the problem of controlling the type I error rate for a primary and secondary endpoint in a clinical trial using a gatekeeping approach in which the secondary endpoint is tested only if the primary endpoint crosses its monitoring boundary. They considered a two-look trial and showed by simulation that the naive method of testing the secondary endpoint at full level α at the time the primary endpoint reaches statistical significance does not control the familywise error rate at level α. Tamhane et al. (2010) derived analytic expressions for familywise error rate and power and confirmed the inflated error rate of the naive approach. Nonetheless, many people mistakenly believe that the closure principle can be used to prove that the naive procedure controls the familywise error rate. The purpose of this note is to explain in greater detail why there is a problem with the naive approach and show that the degree of alpha inflation can be as high as that of unadjusted monitoring of a single endpoint.

Interval between prior SARS-CoV-2 infection and booster vaccination impacts magnitude and quality of antibody and B cell responses.

SARS-CoV-2 mRNA booster vaccines provide protection from severe disease, eliciting strong immunity that is further boosted by previous infection. However, it is unclear whether these immune responses are affected by the interval between infection and vaccination. Over a 2-month period, we evaluated antibody and B cell responses to a third-dose mRNA vaccine in 66 individuals with different infection histories. Uninfected and post-boost but not previously infected individuals mounted robust ancestral and variant spike-binding and neutralizing antibodies and memory B cells. Spike-specific B cell responses from recent infection (<180 days) were elevated at pre-boost but comparatively less so at 60 days post-boost compared with uninfected individuals, and these differences were linked to baseline frequencies of CD27lo B cells. Day 60 to baseline ratio of BCR signaling measured by phosphorylation of Syk was inversely correlated to days between infection and vaccination. Thus, B cell responses to booster vaccines are impeded by recent infection.

Imaging and biopsy of HIV-infected individuals undergoing analytic treatment interruption.

Background: HIV persistence during antiretroviral therapy (ART) is the principal obstacle to cure. Lymphoid tissue is a compartment for HIV, but mechanisms of persistence during ART and viral rebound when ART is interrupted are inadequately understood. Metabolic activity in lymphoid tissue of patients on long-term ART is relatively low, and increases when ART is stopped. Increases in metabolic activity can be detected by 18F-fluorodeoxyglucose Positron Emission Tomography (FDG-PET) and may represent sites of HIV replication or immune activation in response to HIV replication. Methods: FDG-PET imaging will be used to identify areas of high and low metabolic uptake in lymphoid tissue of individuals undergoing long-term ART. Baseline tissue samples will be collected. Participants will then be randomized 1:1 to continue or interrupt ART via analytic treatment interruption (ATI). Image-guided biopsy will be repeated 10 days after ATI initiation. After ART restart criteria are met, image-guided biopsy will be repeated once viral suppression is re-achieved. Participants who continued ART will have a second FDG-PET and biopsies 12-16 weeks after the first. Genetic characteristics of HIV populations in areas of high and low FDG uptake will be assesed. Optional assessments of non-lymphoid anatomic compartments may be performed to evaluate HIV populations in distinct anatomic compartments. Anticipated results: We anticipate that PET standardized uptake values (SUV) will correlate with HIV viral RNA in biopsies of those regions and that lymph nodes with high SUV will have more viral RNA than those with low SUV within a patient. Individuals who undergo ATI are expected to have diverse viral populations upon viral rebound in lymphoid tissue. HIV populations in tissues may initially be phylogenetically diverse after ATI, with emergence of dominant viral species (clone) over time in plasma. Dominant viral species may represent the same HIV population seen before ATI. Discussion: This study will allow us to explore utility of PET for identification of HIV infected cells and determine whether high FDG uptake respresents areas of HIV replication, immune activation or both. We will also characterize HIV infected cell populations in different anatomic locations. The protocol will represent a platform to investigate persistence and agents that may target HIV populations. Study protocol registration: Identifier: NCT05419024.

Recent SARS-CoV-2 infection abrogates antibody and B-cell responses to booster vaccination.

SARS-CoV-2 mRNA booster vaccines provide protection from severe disease, eliciting strong immunity that is further boosted by previous infection. However, it is unclear whether these immune responses are affected by the interval between infection and vaccination. Over a two-month period, we evaluated antibody and B-cell responses to a third dose mRNA vaccine in 66 individuals with different infection histories. Uninfected and post-boost but not previously infected individuals mounted robust ancestral and variant spike-binding and neutralizing antibodies, and memory B cells. Spike-specific B-cell responses from recent infection were elevated at pre-boost but comparatively less so at 60 days post-boost compared to uninfected individuals, and these differences were linked to baseline frequencies of CD27 lo B cells. Day 60 to baseline ratio of BCR signaling measured by phosphorylation of Syk was inversely correlated to days between infection and vaccination. Thus, B-cell responses to booster vaccines are impeded by recent infection.

Infliximab for Treatment of Adults Hospitalized with Moderate or Severe Covid-19.

Background: Immune dysregulation contributes to poorer outcomes in severe Covid-19. Immunomodulators targeting various pathways have improved outcomes. We investigated whether infliximab provides benefit over standard of care. Methods: We conducted a master protocol investigating immunomodulators for potential benefit in treatment of participants hospitalized with Covid-19 pneumonia. We report results for infliximab (single dose infusion) versus shared placebo both with standard of care. Primary outcome was time to recovery by day 29 (28 days after randomization). Key secondary endpoints included 14-day clinical status and 28-day mortality. Results: A total of 1033 participants received study drug (517 infliximab, 516 placebo). Mean age was 54.8 years, 60.3% were male, 48.6% Hispanic or Latino, and 14% Black. No statistically significant difference in the primary endpoint was seen with infliximab compared with placebo (recovery rate ratio 1.13, 95% CI 0.99-1.29; p=0.063). Median (IQR) time to recovery was 8 days (7, 9) for infliximab and 9 days (8, 10) for placebo. Participants assigned to infliximab were more likely to have an improved clinical status at day 14 (OR 1.32, 95% CI 1.05-1.66). Twenty-eight-day mortality was 10.1% with infliximab versus 14.5% with placebo, with 41% lower odds of dying in those receiving infliximab (OR 0.59, 95% CI 0.39-0.90). No differences in risk of serious adverse events including secondary infections. Conclusions: Infliximab did not demonstrate statistically significant improvement in time to recovery. It was associated with improved 14-day clinical status and substantial reduction in 28- day mortality compared with standard of care. Trial registration: ClinicalTrials.gov ( NCT04593940 ).

Noninferiority testing with censoring when the event rate is low.

The PREDICT TB trial tests noninferiority of an abbreviated treatment regimen (arm A) vs a conventional treatment regimen (arm C). Treatment trials of drug-susceptible tuberculosis are expected to have low event rates (ie, relapse probabilities around 3-5%). We examine the question of what is the "best" way to test for noninferiority in a setting with low event rates. In a series of simulations supported by theoretical arguments, we examine operating characteristics of five tests, including normal approximation, exact, and simulation-based tests. Two of these tests are constructed from Kaplan-Meier based-estimators, which account for variable follow-up time (and those lost to follow-up). We evaluate the effect of loss to follow-up via simulations. We also examine the results of the five tests on a data set similar to PREDICT TB, the REMoxTB trial. We find that the normal approximation tests perform well, albeit with small type I error rate inflation. We also find that the Kaplan-Meier methods generally have larger power than the other tests, especially when there is between 10-30% loss to follow-up.

Combination anti-HIV antibodies provide sustained virological suppression.

Antiretroviral therapy is highly effective in suppressing human immunodeficiency virus (HIV)1. However, eradication of the virus in individuals with HIV has not been possible to date2. Given that HIV suppression requires life-long antiretroviral therapy, predominantly on a daily basis, there is a need to develop clinically effective alternatives that use long-acting antiviral agents to inhibit viral replication3. Here we report the results of a two-component clinical trial involving the passive transfer of two HIV-specific broadly neutralizing monoclonal antibodies, 3BNC117 and 10-1074. The first component was a randomized, double-blind, placebo-controlled trial that enrolled participants who initiated antiretroviral therapy during the acute/early phase of HIV infection. The second component was an open-label single-arm trial that enrolled individuals with viraemic control who were naive to antiretroviral therapy. Up to 8 infusions of 3BNC117 and 10-1074, administered over a period of 24 weeks, were well tolerated without any serious adverse events related to the infusions. Compared with the placebo, the combination broadly neutralizing monoclonal antibodies maintained complete suppression of plasma viraemia (for up to 43 weeks) after analytical treatment interruption, provided that no antibody-resistant HIV was detected at the baseline in the study participants. Similarly, potent HIV suppression was seen in the antiretroviral-therapy-naive study participants with viraemia carrying sensitive virus at the baseline. Our data demonstrate that combination therapy with broadly neutralizing monoclonal antibodies can provide long-term virological suppression without antiretroviral therapy in individuals with HIV, and our experience offers guidance for future clinical trials involving next-generation antibodies with long half-lives.

Data and Safety Monitoring Board Monitoring of Clinical Trials for Early Efficacy.

DSMB Monitoring of Clinical Trials for Early EfficacyIn this article in the NEJM Evidence DSMB mini-series, Dodd and Proschan review the clinical, ethical, and statistical issues that confront a DSMB when trial data suggest that the treatment under test is highly efficacious. Examples of such trials are reviewed.

Discussion on "Improving precision and power in randomized trials for COVID-19 treatments using covariate adjustment for binary, ordinal, and time-to-event outcomes".

Benkeser et al. present a very informative paper evaluating the efficiency gains of covariate adjustment in settings with binary, ordinal, and time-to-event outcomes. The adjustment method focuses on estimating the marginal treatment effect averaged over the covariate distribution in both arms combined. The authors show that covariate adjustment can achieve power gains that could find answers more quickly. The suggested approach is an important weapon in the armamentarium against epidemics like COVID-19. I recommend evaluating the procedure against more traditional approaches for conditional analyses (e.g., logistic regression) and against blinded methods of building prediction models followed by randomization-based inference.

Diagnosing fraudulent baseline data in clinical trials.

The first table in many articles reporting results of a randomized clinical trial compares baseline factors across arms. Results that appear inconsistent with chance trigger suspicion, and in one case, accusation and confirmation of data falsification. We confirm theoretically results of simulation analyses showing that inconsistency with chance is extremely difficult to prove in the absence of any information about correlations between baseline covariates. We offer a reasonable diagnostic to trigger further investigation.