A framework for testing non-inferiority in a three-arm, sequential, multiple assignment randomized trial.

Sequential multiple assignment randomized trial design is becoming increasingly used in the field of precision medicine. This design allows comparisons of sequences of adaptive interventions tailored to the individual patient. Superiority testing is usually the initial goal in order to determine which embedded adaptive intervention yields the best primary outcome on average. When direct superiority is not evident, yet an adaptive intervention poses other benefits, then non-inferiority testing is warranted. Non-inferiority testing in the sequential multiple assignment randomized trial setup is rather new and involves the specification of non-inferiority margin and other important assumptions that are often unverifiable internally. These challenges are not specific to sequential multiple assignment randomized trial and apply to two-arm non-inferiority trials that do not include a standard-of-care (or placebo) arm. To address some of these challenges, three-arm non-inferiority trials that include the standard-of-care arm are proposed. However, methods developed so far for three-arm non-inferiority trials are not sequential multiple assignment randomized trial-specific. This is because apart from embedded adaptive interventions, sequential multiple assignment randomized trial typically does not include a third standard-of-care arm. In this article, we consider a three-arm sequential multiple assignment randomized trial from an National Institutes of Health-funded study of symptom management strategies among people undergoing cancer treatment. Motivated by that example, we propose a novel data analytic method for non-inferiority testing in the framework of three-arm sequential multiple assignment randomized trial for the first time. Sample size and power considerations are discussed through extensive simulation studies to elucidate our method.

A more powerful test for three-arm non-inferiority via risk difference: Frequentist and Bayesian approaches.

Necessity for finding improved intervention in many legacy therapeutic areas are of high priority. This has the potential to decrease the expense of medical care and poor outcomes for many patients. Typically, clinical efficacy is the primary evaluating criteria to measure any beneficial effect of a treatment. Albeit, there could be situations when several other factors (e.g. side-effects, cost-burden, less debilitating, less intensive, etc.) which can permit some slightly less efficacious treatment options favorable to a subgroup of patients. This often leads to non-inferiority (NI) testing. NI trials may or may not include a placebo arm due to ethical reasons. However, when included, the resulting three-arm trial is more prudent since it requires less stringent assumptions compared to a two-arm placebo-free trial. In this article, we consider both Frequentist and Bayesian procedures for testing NI in the three-arm trial with binary outcomes when the functional of interest is risk difference. An improved Frequentist approach is proposed first, which is then followed by a Bayesian counterpart. Bayesian methods have a natural advantage in many active-control trials, including NI trial, as it can seamlessly integrate substantial prior information. In addition, we discuss sample size calculation and draw an interesting connection between the two paradigms.

Conversion of non-inferiority margin from hazard ratio to restricted mean survival time difference using data from multiple historical trials.

The restricted mean survival time measure has gained a lot of interests for designing and analyzing oncology trials with time-to-event endpoints due to its intuitive clinical interpretation and potentially high statistical power. In the non-inferiority trial literature, restricted mean survival time has been used as an alternative measure for reanalyzing a completed trial, which was originally designed and analyzed based on traditional proportional hazard model. However, the reanalysis procedure requires a conversion from the non-inferiority margin measured in hazard ratio to a non-inferiority margin measured by restricted mean survival time difference. An existing conversion method assumes a Weibull distribution for the population survival time of the historical active control group under the proportional hazard assumption using data from a single trial. In this article, we develop a methodology for non-inferiority margin conversion when data from multiple historical active control studies are available, and introduce a Kaplan-Meier estimator-based method for the non-inferiority margin conversion to relax the parametric assumption. We report extensive simulation studies to examine the performances of proposed methods under the Weibull data generative models and a piecewise-exponential data generative model that mimic the tumor recurrence and survival characteristics of advanced colon cancer. This work is motivated to achieve non-inferiority margin conversion, using historical patient-level data from a large colon cancer clinical database, to reanalyze an internationally collaborated non-inferiority study that evaluates 6-month versus 3-month duration of adjuvant chemotherapy in stage III colon cancer patients.

Choosing and changing the analysis scale in non-inferiority trials with a binary outcome.

BACKGROUND: The size of the margin strongly influences the required sample size in non-inferiority and equivalence trials. What is sometimes ignored, however, is that for trials with binary outcomes, the scale of the margin - risk difference, risk ratio or odds ratio - also has a large impact on power and thus on sample size requirement. When considering several scales at the design stage of a trial, these sample size consequences should be taken into account. Sometimes, changing the scale may be needed at a later stage of a trial, for example, when the event proportion in the control arm turns out different from expected. Also after completion of a trial, a switch to another scale is sometimes made, for example, when using a regression model in a secondary analysis or when combining study results in a meta-analysis that requires unifying scales. The exact consequences of such switches are currently unknown. METHODS AND RESULTS: This article first outlines sample size consequences for different choices of analysis scale at the design stage of a trial. We add a new result on sample size requirement comparing the risk difference scale with the risk ratio scale. Then, we study two different approaches to changing the analysis scale after the trial has commenced: (1) mapping the original non-inferiority margin using the event proportion in the control arm that was anticipated at the design stage or (2) mapping the original non-inferiority margin using the observed event proportion in the control arm. We use simulations to illustrate consequences on type I and type II error rates. Methods are illustrated on the INES trial, a non-inferiority trial that compared single birth rates in subfertile couples after different fertility treatments. Our results demonstrate large differences in required sample size when choosing between risk difference, risk ratio and odds ratio scales at the design stage of non-inferiority trials. In some cases, the sample size requirement is twice as large on one scale compared with another. Changing the scale after commencing the trial using anticipated proportions mainly impacts type II error rate, whereas switching using observed proportions is not advised due to not maintaining type I error rate. Differences were more pronounced with larger margins. CONCLUSIONS: Trialists should be aware that the analysis scale can have large impact on type I and type II error rates in non-inferiority trials.

New approaches for testing non-inferiority for three-arm trials with Poisson distributed outcomes.

With the availability of limited resources, innovation for improved statistical method for the design and analysis of randomized controlled trials (RCTs) is of paramount importance for newer and better treatment discovery for any therapeutic area. Although clinical efficacy is almost always the primary evaluating criteria to measure any beneficial effect of a treatment, there are several important other factors (e.g., side effects, cost burden, less debilitating, less intensive, etc.), which can permit some less efficacious treatment options favorable to a subgroup of patients. This leads to non-inferiority (NI) testing. The objective of NI trial is to show that an experimental treatment is not worse than an active reference treatment by more than a pre-specified margin. Traditional NI trials do not include a placebo arm for ethical reason; however, this necessitates stringent and often unverifiable assumptions. On the other hand, three-arm NI trials consisting of placebo, reference, and experimental treatment, can simultaneously test the superiority of the reference over placebo and NI of experimental treatment over the reference. In this article, we proposed both novel Frequentist and Bayesian procedures for testing NI in the three-arm trial with Poisson distributed count outcome. RCTs with count data as the primary outcome are quite common in various disease areas such as lesion count in cancer trials, relapses in multiple sclerosis, dermatology, neurology, cardiovascular research, adverse event count, etc. We first propose an improved Frequentist approach, which is then followed by it's Bayesian version. Bayesian methods have natural advantage in any active-control trials, including NI trial when substantial historical information is available for placebo and established reference treatment. In addition, we discuss sample size calculation and draw an interesting connection between the two paradigms.

Non-inferiority Testing for Risk Ratio, Odds Ratio and Number Needed to Treat in Three-arm Trial.

Three-arm non-inferiority (NI) trial including the experimental treatment, an active reference treatment, and a placebo where the outcome of interest is binary are considered. While the risk difference (RD) is the most common and well explored functional form for testing efficacy (or effectiveness), however, recent FDA guideline suggested measures such as relative risk (RR), odds ratio (OR), number needed to treat (NNT) among others, on the basis of which NI can be claimed for binary outcome. Albeit, developing test based on these different functions of binary outcome are challenging. This is because the construction and interpretation of NI margin for such functions are non-trivial extensions of RD based approach. A Frequentist test based on traditional fraction margin approach for RR, OR and NNT are proposed first. Furthermore a conditional testing approach is developed by incorporating assay sensitivity (AS) condition directly into NI testing. A detailed discussion of sample size/power calculation are also put forward which could be readily used while designing such trials in practice. A clinical trial data is reanalyzed to demonstrate the presented approach.

Bayesian Approach for Assessing Non-inferiority in Three-arm Trials for Risk Ratio and Odds Ratio.

In this paper we consider three-arm non-inferiority (NI) trial that includes an experimental, a reference, and a placebo arm. While for binary outcomes the risk difference (RD) is the most common and well explored functional form for testing efficacy (or effectiveness), recent FDA guideline suggested other measures such as relative risk (RR) and odds ratio (OR) on the basis of which NI of an experimental treatment can be claimed. However, developing test based on these different functions of binary outcomes are challenging since the construction and interpretation of NI margin for such functions are not trivial extensions of RD based approach. Recently, we have proposed Frequentist approaches for testing NI for these functionals. In this article we further develop Bayesian approaches for testing NI based on effect retention approach for RR and OR. Bayesian paradigm provides a natural path to integrate historical trials' information, as well as it allows the usage of patients'/clinicians' opinions as prior information via sequential learning. In addition we discuss, in detail, the sample size/power calculation which could be readily used while designing such trials in practice.

A reproducibility probability-based bias-adjustment approach on the specification of non-inferiority margin using historical data.

The effect of reference treatment over placebo, known as M1, is essential in the development of non-inferiority margin. We proposed a M1 adjustment approach to reduce the selection bias for collected data of historical trials. A quantitative illustration of selection bias of historical data is also defined. Simulation study shows that the proposed approaches would significantly reduce the bias when the proportion of positive studies in historical data is noticeably larger than the power of studies include in historical data. When historical data are constituted by only positive studies, the performance of the proposed method is also appreciable. However, when the proportion of positive studies is close to the power of studies included or the number of studies included is too small, the performance of the proposed approach may not be reliable. A real-data application is also presented. The proposed bias-adjustment approach is a reasonable method to reduce the over-estimate of effect size in the specification of non-inferiority margin. It could also be applied in most non-inferiority margin specification methods or be cooperate used with other bias-adjustment approaches.

Augmented reality-assisted pedicle screw insertion: a cadaveric proof-of-concept study.

OBJECTIVE Augmented reality (AR) is a novel technology that has the potential to increase the technical feasibility, accuracy, and safety of conventional manual and robotic computer-navigated pedicle insertion methods. Visual data are directly projected to the operator's retina and overlaid onto the surgical field, thereby removing the requirement to shift attention to a remote display. The objective of this study was to assess the comparative accuracy of AR-assisted pedicle screw insertion in comparison to conventional pedicle screw insertion methods. METHODS Five cadaveric male torsos were instrumented bilaterally from T6 to L5 for a total of 120 inserted pedicle screws. Postprocedural CT scans were obtained, and screw insertion accuracy was graded by 2 independent neuroradiologists using both the Gertzbein scale (GS) and a combination of that scale and the Heary classification, referred to in this paper as the Heary-Gertzbein scale (HGS). Non-inferiority analysis was performed, comparing the accuracy to freehand, manual computer-navigated, and robotics-assisted computer-navigated insertion accuracy rates reported in the literature. User experience analysis was conducted via a user experience questionnaire filled out by operators after the procedures. RESULTS The overall screw placement accuracy achieved with the AR system was 96.7% based on the HGS and 94.6% based on the GS. Insertion accuracy was non-inferior to accuracy reported for manual computer-navigated pedicle insertion based on both the GS and the HGS scores. When compared to accuracy reported for robotics-assisted computer-navigated insertion, accuracy achieved with the AR system was found to be non-inferior when assessed with the GS, but superior when assessed with the HGS. Last, accuracy results achieved with the AR system were found to be superior to results obtained with freehand insertion based on both the HGS and the GS scores. Accuracy results were not found to be inferior in any comparison. User experience analysis yielded "excellent" usability classification. CONCLUSIONS AR-assisted pedicle screw insertion is a technically feasible and accurate insertion method.

Evidence-based sizing of non-inferiority trials using decision models.

BACKGROUND: There are significant challenges to the successful conduct of non-inferiority trials because they require large numbers to demonstrate that an alternative intervention is "not too much worse" than the standard. In this paper, we present a novel strategy for designing non-inferiority trials using an approach for determining the appropriate non-inferiority margin (δ), which explicitly balances the benefits of interventions in the two arms of the study (e.g. lower recurrence rate or better survival) with the burden of interventions (e.g. toxicity, pain), and early and late-term morbidity. METHODS: We use a decision analytic approach to simulate a trial using a fixed value for the trial outcome of interest (e.g. cancer incidence or recurrence) under the standard intervention (pS) and systematically varying the incidence of the outcome in the alternative intervention (pA). The non-inferiority margin, pA - pS = δ, is reached when the lower event rate of the standard therapy counterbalances the higher event rate but improved morbidity burden of the alternative. We consider the appropriate non-inferiority margin as the tipping point at which the quality-adjusted life-years saved in the two arms are equal. RESULTS: Using the European Polyp Surveillance non-inferiority trial as an example, our decision analytic approach suggests an appropriate non-inferiority margin, defined here as the difference between the two study arms in the 10-year risk of being diagnosed with colorectal cancer, of 0.42% rather than the 0.50% used to design the trial. The size of the non-inferiority margin was smaller for higher assumed burden of colonoscopies. CONCLUSIONS: The example demonstrates that applying our proposed method appears feasible in real-world settings and offers the benefits of more explicit and rigorous quantification of the various considerations relevant for determining a non-inferiority margin and associated trial sample size.

Adaptive non-inferiority margins under observable non-constancy.

A central assumption in the design and conduct of non-inferiority trials is that the active-control therapy will have the same degree of effectiveness in the planned non-inferiority trial as in the prior placebo-controlled trials used to define the non-inferiority margin. This is referred to as the 'constancy' assumption. If the constancy assumption fails, decisions based on the chosen non-inferiority margin may be incorrect, and the study runs the risk of approving an inferior product or failing to approve a beneficial product. The constancy assumption cannot be validated in a trial without a placebo arm, and it is unlikely ever to be met completely. When there are strong, observable predictors of constancy, such as dosing and adherence to the active-control product, we can specify conditions where the constancy assumption will likely fail. We propose a method for using measurable predictors of active-control effectiveness to specify non-inferiority margins targeted to the planned study population characteristics. We describe a pre-specified method, using baseline characteristics or post-baseline predictors in the active-control arm, to adapt the non-inferiority margin at the end of the study if constancy is violated. Adaptive margins can help adjust for constancy violations that will inevitably occur in real clinical trials, while maintaining pre-specified levels of Type I error and power.

A more efficient three-arm non-inferiority test based on pooled estimators of the homogeneous variance.

Hida and Tango established a statistical testing framework for the three-arm non-inferiority trial including a placebo with a pre-specified non-inferiority margin to overcome the shortcomings of traditional two-arm non-inferiority trials (such as having to choose the non-inferiority margin). In this paper, we propose a new method that improves their approach with respect to two aspects. We construct our testing statistics based on the best unbiased pooled estimators of the homogeneous variance; and we use the principle of intersection-union tests to determine the rejection rule. We theoretically prove that our test is better than that of Hida and Tango for large sample sizes. Furthermore, when that sample size was small or moderate, our simulation studies showed that our approach performed better than Hida and Tango's. Although both controlled the type I error rate, their test was more conservative and the statistical power of our test was higher.

Design of Phase II Non-inferiority Trials.

With the development of inexpensive treatment regimens and less invasive surgical procedures, we are confronted with non-inferiority study objectives. A non-inferiority phase III trial requires a roughly four times larger sample size than that of a similar standard superiority trial. Because of the large required sample size, we often face feasibility issues to open a non-inferiority trial. Furthermore, due to lack of phase II non-inferiority trial design methods, we do not have an opportunity to investigate the efficacy of the experimental therapy through a phase II trial. As a result, we often fail to open a non-inferiority phase III trial and a large number of non-inferiority clinical questions still remain unanswered. In this paper, we want to develop some designs for non-inferiority randomized phase II trials with feasible sample sizes. At first, we review a design method for non-inferiority phase III trials. Subsequently, we propose three different designs for non-inferiority phase II trials that can be used under different settings. Each method is demonstrated with examples. Each of the proposed design methods is shown to require a reasonable sample size for non-inferiority phase II trials. The three different non-inferiority phase II trial designs are used under different settings, but require similar sample sizes that are typical for phase II trials.

Methods of defining the non-inferiority margin in randomized, double-blind controlled trials: a systematic review.

BACKGROUND: There is no consensus on the preferred method for defining the non-inferiority margin in non-inferiority trials, and previous studies showed that the rationale for its choice is often not reported. This study investigated how the non-inferiority margin is defined in the published literature, and whether its reporting has changed over time. METHODS: A systematic PubMed search was conducted for all published randomized, double-blind, non-inferiority trials from January 1, 1966, to February 6, 2015. The primary outcome was the number of margins that were defined by methods other than the historical evidence of the active comparator. This was evaluated for a time trend. We also assessed the under-reporting of the methods of defining the margin as a secondary outcome, and whether this changed over time. Both outcomes were analyzed using a Poisson log-linear model. Predictors for better reporting of the methods, and the use of the fixed-margin method (one of the historical evidence methods) were also analyzed using logistic regression. RESULTS: Two hundred seventy-three articles were included, which account for 273 non-inferiority margins. There was no statistically significant difference in the number of margins that were defined by other methods compared to those defined based on the historical evidence (ratio 2.17, 95% CI 0.86 to 5.82, p = 0.11), and this did not change over time. The number of margins for which methods were unreported was similar to those with reported methods (ratio 1.35, 95% CI 0.76 to 2.43, p = 0.31), with no change over time. The method of defining the margin was less often reported in journals with low-impact factors compared to journals with high-impact factors (OR 0.20; 95% CI 0.10 to 0.37, p < 0.0001). The publication of the FDA draft guidance in 2010 was associated with increased reporting of the fixed-margin method (after versus before 2010) (OR 3.54; 95% CI 1.12 to 13.35, p = 0.04). CONCLUSIONS: Non-inferiority margins are not commonly defined based on the historical evidence of the active comparator, and they are poorly reported. Authors, reviewers, and editors need to take notice of reporting this critical information to allow for better judgment of non-inferiority trials.

Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study.

BACKGROUND: Systematic reviews offer the most reliable and valid support for health policy decision-making, patient information, and guideline development. However, they are labor intensive and frequently take longer than 1 year to complete. Consequently, they often do not meet the needs of those who need to make decisions quickly. Rapid reviews have therefore become a pragmatic alternative to systematic reviews. They are knowledge syntheses that abbreviate certain methodological aspects of systematic reviews to produce information more quickly. Methodological shortcuts often take place in literature identification. A potential drawback is less reliable results. To date, the impact of abbreviated searches on estimates of treatment effects and subsequent conclusions has not been analyzed systematically across multiple bodies of evidence. We aim to answer the research question: Do bodies of evidence that are based on abbreviated literature searches lead to different conclusions about benefits and harms of interventions compared with bodies of evidence that are based on comprehensive, systematic literature searches? METHODS: We will use a non-inferiority and meta-epidemiologic design. The primary outcome is the proportion of discordant conclusions based on different search approaches. Drawing of a pool of Cochrane reports published between 2012 and 2016, we will randomly select 60 reports. Eligible reports are those that present a summary-of-findings table, draw a clear conclusion, present data for meta-analyses, and document the search strategy clearly. We will conduct several abbreviated searches to detect whether included studies in these Cochrane reviews could be detected. If searches could not detect all studies, we will revise the original summary-of-findings table and ask review authors whether the missed evidence would change conclusions of their report. We will determine the proportion of discordant conclusions for each abbreviated search approach. We will consider an abbreviated search as non-inferior if the lower limit of the 95% confidence interval of the proportion of discordant conclusions is below the non-inferiority margin, which is determined based on results of a survey for clinical and public health scenarios. DISCUSSION: This will be the first study to assess whether the reduced sensitivity of abbreviated searches has an impact on conclusions across multiple bodies of evidence, not only on effect estimates.

Statistical issues in the design, conduct and analysis of two large safety studies.

BACKGROUND/AIMS: The emergence, post approval, of serious medical events, which may be associated with the use of a particular drug or class of drugs, is an important public health and regulatory issue. The best method to address this issue is through a large, rigorously designed safety study. Therefore, it is important to elucidate the statistical issues involved in these large safety studies. METHODS: Two such studies are PRECISION and EAGLES. PRECISION is the primary focus of this article. PRECISION is a non-inferiority design with a clinically relevant non-inferiority margin. Statistical issues in the design, conduct and analysis of PRECISION are discussed. RESULTS: Quantitative and clinical aspects of the selection of the composite primary endpoint, the determination and role of the non-inferiority margin in a large safety study and the intent-to-treat and modified intent-to-treat analyses in a non-inferiority safety study are shown. Protocol changes that were necessary during the conduct of PRECISION are discussed from a statistical perspective. Issues regarding the complex analysis and interpretation of the results of PRECISION are outlined. EAGLES is presented as a large, rigorously designed safety study when a non-inferiority margin was not able to be determined by a strong clinical/scientific method. In general, when a non-inferiority margin is not able to be determined, the width of the 95% confidence interval is a way to size the study and to assess the cost-benefit of relative trial size. CONCLUSION: A non-inferiority margin, when able to be determined by a strong scientific method, should be included in a large safety study. Although these studies could not be called "pragmatic," they are examples of best real-world designs to address safety and regulatory concerns.

Non-inferiority tests for anti-infective drugs using control group quantiles.

BACKGROUND/AIMS: In testing for non-inferiority of anti-infective drugs, the primary endpoint is often the difference in the proportion of failures between the test and control group at a landmark time. The landmark time is chosen to approximately correspond to the qth historic quantile of the control group, and the non-inferiority margin is selected to be reasonable for the target level q. For designing these studies, a troubling issue is that the landmark time must be pre-specified, but there is no guarantee that the proportion of control failures at the landmark time will be close to the target level q. If the landmark time is far from the target control quantile, then the pre-specified non-inferiority margin may not longer be reasonable. Exact variable margin tests have been developed by Röhmel and Kieser to address this problem, but these tests can have poor power if the observed control failure rate at the landmark time is far from its historic value. METHODS: We develop a new variable margin non-inferiority test where we continue sampling until a pre-specified proportion of failures, q, have occurred in the control group, where q is the target quantile level. The test does not require any assumptions on the failure time distributions, and hence, no knowledge of the true [Formula: see text] control quantile for the study is needed. RESULTS: Our new test is exact and has power comparable to (or greater than) its competitors when the true control quantile from the study equals (or differs moderately from) its historic value. Our nivm R package performs the test and gives confidence intervals on the difference in failure rates at the true target control quantile. The tests can be applied to time to cure or other numeric variables as well. CONCLUSION: A substantial proportion of new anti-infective drugs being developed use non-inferiority tests in their development, and typically, a pre-specified landmark time and its associated difference margin are set at the design stage to match a specific target control quantile. If through changing standard of care or selection of a different population the target quantile for the control group changes from its historic value, then the appropriateness of the pre-specified margin at the landmark time may be questionable. Our proposed test avoids this problem by sampling until a pre-specified proportion of the controls have failed.

Bayesian approach for assessing non-inferiority in a three-arm trial with pre-specified margin.

Non-inferiority trials are becoming increasingly popular for comparative effectiveness research. However, inclusion of the placebo arm, whenever possible, gives rise to a three-arm trial which has lesser burdensome assumptions than a standard two-arm non-inferiority trial. Most of the past developments in a three-arm trial consider defining a pre-specified fraction of unknown effect size of reference drug, that is, without directly specifying a fixed non-inferiority margin. However, in some recent developments, a more direct approach is being considered with pre-specified fixed margin albeit in the frequentist setup. Bayesian paradigm provides a natural path to integrate historical and current trials' information via sequential learning. In this paper, we propose a Bayesian approach for simultaneous testing of non-inferiority and assay sensitivity in a three-arm trial with normal responses. For the experimental arm, in absence of historical information, non-informative priors are assumed under two situations, namely when (i) variance is known and (ii) variance is unknown. A Bayesian decision criteria is derived and compared with the frequentist method using simulation studies. Finally, several published clinical trial examples are reanalyzed to demonstrate the benefit of the proposed procedure.

Comparing vaccines: a systematic review of the use of the non-inferiority margin in vaccine trials.

BACKGROUND: Non-inferiority (NI) randomized controlled trials (RCTs) aim to demonstrate that a new treatment is no worse than a comparator that has already shown its efficacy over placebo within a pre-specified margin. However, clear guidelines on how the NI margin should be determined are lacking for vaccine trials. A difference (seroprevalence/risk) of 10% or a geometric mean titre/concentration (GMT) ratio of 1.5 or 2.0 in antibody levels is implicitly recommended for vaccine trials. We aimed to explore which NI margins were used in vaccine RCTs and how they were determined. METHODS: A systematic search for NI vaccine RCTs yielded 177 eligible articles. Data were extracted from these articles using a standardized form and included general characteristics and characteristics specific for NI trials. Relations between the study characteristics and the NI margin used were explored. RESULTS: Among the 143 studies using an NI margin based on difference (n=136 on immunogenicity, n=2 on efficacy and n=5 on safety), 66% used a margin of 10%, 23% used margins lower than 10% (range 1-7.5%) and 11% used margins larger than 10% (range 11.5-25%). Of the 103 studies using a NI margin based on the GMT ratio, 50% used a margin of 0.67/1.5 and 49% used 0.5/2.0. As observed, 85% of the studies did not discuss the method of margin determination; and 19% of the studies lacked a confidence interval or p-value for non-inferiority. CONCLUSION: Most NI vaccine RCTs used an NI margin of 10% for difference or a GMT ratio of 1.5 or 2.0 without a clear rationale. Most articles presented enough information for the reader to make a judgement about the NI margin used and the conclusions. The reporting on the design, margins used and results of NI vaccine trials could be improved; more explicit guidelines may help to achieve this end.