Secondary Analysis of a Systematic Review: Are Antifungal Noninferiority Trials at Risk of Eroding Effectiveness Because of Biocreep?

Noninferiority randomized controlled trial (RCT) effectiveness may erode when results favor the active control over time and when a decreasingly effective control arm is used in serial trials. We analyzed 32 antifungal noninferiority RCTs (NI-RCTs) for these scenarios in this secondary analysis of a systematic review. Our exploratory analysis suggests that the erosion risk in the effectiveness of antifungal noninferiority trials is uncommon. Findings are limited by small sample size and overall risk of bias.

Pandemic Planning in Homeless Shelters: A Pilot Study of a Coronavirus Disease 2019 (COVID-19) Testing and Support Program to Mitigate the Risk of COVID-19 Outbreaks in Congregate Settings.

We tested 104 residents and 141 staff for coronavirus disease 2019 who failed daily symptom screening in homeless shelters in Hamilton, Canada. We detected 1 resident (1%), 7 staff (5%), and 1 case of secondary spread. Shelter restructuring to allow physical distancing, testing, and isolation can decrease outbreaks in shelters.

Methodological and Reporting Quality of Noninferiority Randomized Controlled Trials Comparing Antibiotic Therapies: A Systematic Review.

BACKGROUND: Antibiotic noninferiority randomized controlled trials (RCTs) are used for approval of new antibiotics and making changes to antibiotic prescribing in clinical practice. We conducted a systematic review to assess the methodological and reporting quality of antibiotic noninferiority RCTs. METHODS: We searched MEDLINE, Embase, the Cochrane Database of Systematic Reviews, and the Food and Drug Administration drug database from inception until November 22, 2019, for noninferiority RCTs comparing different systemic antibiotic therapies. Comparisons between antibiotic types, doses, administration routes, or durations were included. Methodological and reporting quality indicators were based on the Consolidated Standards of Reporting Trials reporting guidelines. Two independent reviewers extracted the data. RESULTS: The systematic review included 227 studies. Of these, 135 (59.5%) studies were supported by pharmaceutical industry. Only 83 (36.6%) studies provided a justification for the noninferiority margin. Reporting of both intention-to-treat (ITT) and per-protocol (PP) analyses were done in 165 (72.7%) studies. The conclusion was misleading in 34 (15.0%) studies. The studies funded by pharmaceutical industry were less likely to be stopped early because of logistical reasons (3.0% vs 19.1%; odds ratio [OR] = 0.13; 95% confidence interval [CI], .04-.37) and to show inconclusive results (11.1% vs 42.9%; OR = 0.17; 95% CI, .08-.33). The quality of studies decreased over time with respect to blinding, early stopping, reporting of ITT with PP analysis, and having misleading conclusions. CONCLUSIONS: There is room for improvement in the methodology and reporting of antibiotic noninferiority trials. Quality can be improved across the entire spectrum from investigators, funding agencies, as well as during the peer-review process.There is room for improvement in the methodology and reporting of antibiotic noninferiority trials including justification of noninferiority margin, reporting of intention-to-treat analysis with per-protocol analysis, and having conclusions that are concordant with study results.Clinical Trials Registration PROSPERO registration number CRD42020165040.

Intention-to-treat analysis may be more conservative than per protocol analysis in antibiotic non-inferiority trials: a systematic review.

BACKGROUND: In non-inferiority trials, there is a concern that intention-to-treat (ITT) analysis, by including participants who did not receive the planned interventions, may bias towards making the treatment and control arms look similar and lead to mistaken claims of non-inferiority. In contrast, per protocol (PP) analysis is viewed as less likely to make this mistake and therefore preferable in non-inferiority trials. In a systematic review of antibiotic non-inferiority trials, we compared ITT and PP analyses to determine which analysis was more conservative. METHODS: In a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, used absolute risk reduction (ARR) as the main outcome and reported both ITT and PP analyses. All estimates and confidence intervals (CIs) were oriented so that a negative ARR favored the control arm, and a positive ARR favored the treatment arm. We compared ITT to PP analyses results. The more conservative analysis between ITT and PP analyses was defined as the one having a more negative lower CI limit. RESULTS: The analysis included 164 comparisons from 154 studies. In terms of the ARR, ITT analysis yielded the more conservative point estimate and lower CI limit in 83 (50.6%) and 92 (56.1%) comparisons respectively. The lower CI limits in ITT analysis favored the control arm more than in PP analysis (median of - 7.5% vs. -6.9%, p = 0.0402). CIs were slightly wider in ITT analyses than in PP analyses (median of 13.3% vs. 12.4%, p < 0.0001). The median success rate was 89% (interquartile range IQR 82 to 93%) in the PP population and 44% (IQR 23 to 60%) in the patients who were included in the ITT population but excluded from the PP population (p < 0.0001). CONCLUSIONS: Contrary to common belief, ITT analysis was more conservative than PP analysis in the majority of antibiotic non-inferiority trials. The lower treatment success rate in the ITT analysis led to a larger variance and wider CI, resulting in a more conservative lower CI limit. ITT analysis should be mandatory and considered as either the primary or co-primary analysis for non-inferiority trials. TRIAL REGISTRATION: PROSPERO registration number CRD42020165040 .

Novel Antibiotics May Be Noninferior but Are They Becoming Less Effective?: a Systematic Review.

Novel antibiotics approved by noninferiority trials may become less effective over time in two scenarios: (i) the treatment effect in studies of novel antibiotics may be consistently worse than studies of older antibiotics; (ii) when a decreasingly effective control arm is used in a series of noninferiority trials. Our systematic review of 175 noninferiority antibiotic trials found these scenarios to be rare.

Methodological and reporting quality of non-inferiority randomized controlled trials comparing antifungal therapies: a systematic review.

BACKGROUND: Detailed reporting is essential in non-inferiority randomized controlled trials (NI-RCTs) to assess evidence quality, as these trials inform standards of care. OBJECTIVES: The primary objective was to evaluate the methodological and reporting quality of antifungal NI-RCTs. DATA SOURCES: Medline, EMBASE, the Cochrane CENTRAL and the United States Federal Drug Administration (FDA) drugs database were searched to 9 September 2020. STUDY ELIGIBILITY CRITERIA: NI-RCTs differing by antifungal formulation, type, dose, administration and/or duration were included. Articles were independently assessed in duplicate using quality indicators developed by the Consolidated Standards of Reporting Trials (CONSORT) group. PARTICIPANTS: Patients enrolled in antifungal trials for prophylactic and therapeutic use. METHODS: The Cochrane RoB 2.0 tool was used to assess risk of bias. Descriptive statistics were used; all statistical tests were two sided. RESULTS: Of 32 included studies, 22 (68.7%) did not justify the NIM. Handling of missing data was not described in 20 (62.5%). Intention-to-treat (ITT) and per-protocol (PP) analyses were both reported in 12/32 (37.5%) studies. Eleven of 32 studies (34.3%) reported potentially misleading conclusions. Industry-financed studies were more likely to report only the ITT analysis (n = 14/27, 51.9%). Methodological and reporting quality was unaffected by publication year; risk of bias from missing data changed over time. Overall risk of bias across included studies was moderate to high, with high risk in randomization process (n = 8/32, 25%), missing outcome data (n = 5/32, 15.6%), and selection of reported result (n = 9/32, 28.1%). CONCLUSIONS: Justification of the non-inferiority margin, reporting of ITT and PP analyses, missing data handling description, and ensuring conclusions are consistent with reported data is necessary to improve CONSORT adherence. Small sample size and overall risk of bias are study limitations. (Systematic Review Registration Number PROSPERO CRD42020219497).

Confidence interval of risk difference by different statistical methods and its impact on the study conclusion in antibiotic non-inferiority trials.

BACKGROUND: Numerous statistical methods can be used to calculate the confidence interval (CI) of risk differences. There is consensus in previous literature that the Wald method should be discouraged. We compared five statistical methods for estimating the CI of risk difference in terms of CI width and study conclusion in antibiotic non-inferiority trials. METHODS: In a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, reported risk differences for the primary outcome, and described the number of successes and/or failures as well as patients in each arm. For each study, we re-calculated the risk difference CI using the Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and skewness-corrected asymptotic score (SCAS) methods. The CIs by different statistical methods were compared in terms of CI width and conclusion on non-inferiority. A wider CI was considered to be more conservative. RESULTS: The analysis included 224 comparisons from 213 studies. The statistical method used to calculate CI was not reported in 134 (59.8%) cases. The median (interquartile range IQR) for CI width by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods was 13.0% (10.8%, 17.4%), 13.3% (10.9%, 18.5%), 13.6% (11.1%, 18.9%), 13.6% (11.1% and 19.0%), and 13.4% (11.1%, 18.9%), respectively. In 216 comparisons that reported a non-inferiority margin, the conclusion on non-inferiority was the same across the five statistical methods in 211 (97.7%) cases. The differences in CI width were more in trials with a sample size of 100 or less in each group and treatment success rate above 90%. Of the 18 trials in this subgroup with a specified non-inferiority margin, non-inferiority was shown in 17 (94.4%), 16 (88.9%), 14 (77.8%), 14 (77.8%), and 15 (83.3%) cases based on CI by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods, respectively. CONCLUSIONS: The statistical method used to calculate CI was not reported in the majority of antibiotic non-inferiority trials. Different statistical methods for CI resulted in different conclusions on non-inferiority in 2.3% cases. The differences in CI widths were highest in trials with a sample size of 100 or less in each group and a treatment success rate above 90%. TRIAL REGISTRATION: PROSPERO CRD42020165040 . April 28, 2020.

Comparison of four COVID-19 screening strategies to facilitate early case identification within the homeless shelter population: A structured summary of a study protocol for a randomised controlled trial.

OBJECTIVES: 1. To compare the effectiveness of four different surveillance strategies in detecting COVID-19 within the homeless shelter population. 2. To assess the participant adherence over time for each surveillance method. TRIAL DESIGN: This is a prospective cluster-randomized study to compare the effectiveness of four different surveillance regimens across eight homeless shelters in the city of Hamilton. PARTICIPANTS: Participants will include both residents of, and the staff working within, the homeless shelters. All participants aged 18 or older who consent to the study and are able to collect a swab sample (where relevant) are eligible for the study. The study will take place across eight homeless shelters (four men-only and four women-only) in the City of Hamilton in Ontario, Canada. INTERVENTION AND COMPARATOR GROUPS: The comparator group will receive active daily surveillance of symptoms and testing will only be completed in symptomatic participants (i.e. those who fail screening or who seek care for potential COVID-19 related symptoms). The three intervention arms will all receive active daily surveillance of symptoms and testing of symptomatic participants (as in the comparator group) in addition to one of the following: 1. Once weekly self-collected oral swabs (OS) regardless of symptoms using written and visual instructions. 2. Once weekly self-collected oral-nares swab (O-NS) regardless of symptoms using written and visual instructions. 3. Once weekly nurse collected nasopharyngeal swab (NPS) regardless of symptoms. Participants will follow verbal and written instructions for the collection of OS and O-NS specimens. For OS collection, participants are instructed to first moisten the swab on their tongue, insert the swab between the cheek and the lower gums and rotate the swab three times. This is repeated on the other side. For O-NS collection, after oral collection, the swab is inserted comfortably (about 2-3 cm) into one nostril, parallel to the floor and turned three times, then repeated in the other nostril. NPS specimens were collected by the nurse following standard of care procedure. All swabs were placed into a viral inactivation medium and transported to the laboratory for COVID-19 testing. Briefly, total nucleic acid was extracted from specimens and then amplified by RT-PCR for the UTR and Envelope genes of SARS-CoV-2 and the human RNase P gene, which is used as a sample adequacy marker. MAIN OUTCOMES: 1. PRIMARY OUTCOME: COVID-19 detection rate, i.e. the number of new positive cases over the study period of 8 weeks in each arm of the study. 2. SECONDARY OUTCOMES: Qualitative assessment of study enrollment over 8 weeks. Percentage of participants who performed 50% or more of the weekly swabs in the intervention arms in the 8 week study period. RANDOMIZATION: We will use a computer-generated random assignment list to randomize the shelters to one of four interventions. Shelters were stratified by gender, and the simple randomization scheme was applied within each stratum. The randomization scheme was created using WinPEPI. BLINDING: This is an open-label study in which neither participants nor assessors are blinded. NUMBERS TO BE RANDOMIZED (SAMPLE SIZE): Since we are including our total sample frame, a sample size estimation at the cluster level is not required. However, if we succeed to enroll 50 participants per shelter from 8 shelters (n=400), and the detection rate is 3 times higher in the intervention groups (0.15) than in the comparator groups (0.05), we will have 90% power to detect a statistically significant and clinically important difference at a type I error rate of alpha=0.05 (one tailed), assuming an intraclass correlation of ~0.008. These computations were done using WinPEPI, and informed by conservative estimates from other studies on respiratory illness in the homeless (see Full protocol). TRIAL STATUS: The protocol version number is 3.0. Recruitment began on April 17, 2020 and is ongoing. Due to low numbers of COVID cases in the community and shelter system during the initial study period, the trial was extended. The estimated date for the end of the extended recruitment period is Feb 1, 2021. TRIAL REGISTRATION: The trial was registered with ClinicalTrials.gov on June 18, 2020 with the identifier NCT04438070 . FULL PROTOCOL: The full protocol is attached as an additional file, accessible from the Trials website (Additional file 1). In the interest in expediting dissemination of this material, the familiar formatting has been eliminated; this Letter serves as a summary of the key elements of the full protocol.