Do efficacy results obtained from randomized controlled trials translate to effectiveness data from observational studies for relapsing-remitting multiple sclerosis?

BACKGROUND: Randomized controlled trials are considered the gold standard in regulatory decision making, as observational studies are known to have important methodological limitations. However, real-world evidence may be helpful in specific situations. This review investigates how the effect estimates obtained from randomized controlled trials compare to those obtained from observational studies, using drug therapy for relapsing-remitting multiple sclerosis as an example. STUDY DESIGN AND SETTING: A systematic review of randomized controlled trials and observational studies was conducted. The primary outcome was the annualized relapse rate. Using (network) meta-analysis together with posterior predictive distributions, the drug-specific rate ratios from the network of randomized controlled trials were compared with those from the network of observational studies. RESULTS: Effect estimates from 26 observational studies showed greater magnitudes and were less precise compared to estimates obtained from 21 randomized controlled trials. Twenty of the 28 treatment comparisons between designs had similar rate ratios. Seven inconsistencies in observed rate ratios could be attributed to two specific disease-modifying therapies. CONCLUSION: In this case study, estimates from observational studies predominantly agreed with estimates from randomized controlled trials given their posterior predictive distributions. Multiple observational studies together may therefore supplement additional pivotal randomized controlled trials in relapsing-remitting multiple sclerosis, for instance facilitating the extrapolation of trial results to the broader patient population.

Inconsistency identification in network meta-analysis via stochastic search variable selection.

The reliability of the results of network meta-analysis (NMA) lies in the plausibility of the key assumption of transitivity. This assumption implies that the effect modifiers' distribution is similar across treatment comparisons. Transitivity is statistically manifested through the consistency assumption which suggests that direct and indirect evidence are in agreement. Several methods have been suggested to evaluate consistency. A popular approach suggests adding inconsistency factors to the NMA model. We follow a different direction by describing each inconsistency factor with a candidate covariate whose choice relies on variable selection techniques. Our proposed method, stochastic search inconsistency factor selection (SSIFS), evaluates the consistency assumption both locally and globally, by applying the stochastic search variable selection method to determine whether the inconsistency factors should be included in the model. The posterior inclusion probability of each inconsistency factor quantifies how likely is a specific comparison to be inconsistent. We use posterior model odds or the median probability model to decide on the importance of inconsistency factors. Differences between direct and indirect evidence can be incorporated into the inconsistency detection process. A key point of our proposed approach is the construction of a reasonable "informative" prior concerning network consistency. The prior is based on the elicitation of information derived historical data from 201 published network meta-analyses. The performance of our proposed method is evaluated in two published network meta-analyses. The proposed methodology is publicly available in an R package called ssifs, published on CRAN and developed and maintained by the authors of this work.

Combined assessment of early and late-phase outcomes in orphan drug development.

In drug development programs, proof-of-concept Phase II clinical trials typically have a biomarker as a primary outcome, or an outcome that can be observed with relatively short follow-up. Subsequently, the Phase III clinical trials aim to demonstrate the treatment effect based on a clinical outcome that often needs a longer follow-up to be assessed. Early-phase outcomes or biomarkers are typically associated with late-phase outcomes and they are often included in Phase III trials. The decision to proceed to Phase III development is based on analysis of the early-Phase II outcome data. In rare diseases, it is likely that only one Phase II trial and one Phase III trial are available. In such cases and before drug marketing authorization requests, positive results of the early-phase outcome of Phase II trials are then likely seen as supporting (or even replicating) positive Phase III results on the late-phase outcome, without a formal retrospective combined assessment and without accounting for between-study differences. We used double-regression modeling applied to the Phase II and Phase III results to numerically mimic this informal retrospective assessment. We provide an analytical solution for the bias and mean square error of the overall effect that leads to a corrected double-regression. We further propose a flexible Bayesian double-regression approach that minimizes the bias by accounting for between-study differences via discounting the Phase II early-phase outcome when they are not in line with the Phase III biomarker outcome results. We illustrate all methods with an orphan drug example for Fabry disease.

Endovascular treatment for basilar artery occlusion: A systematic review and meta-analysis.

BACKGROUND AND PURPOSE: Independent randomized controlled clinical trials (RCTs) have provided robust evidence for endovascular treatment (EVT) as the standard of care treatment for acute large vessel occlusions in the anterior circulation. We examined available studies specific to posterior cerebral circulation ischemic strokes to see if any conclusions can be drawn regarding EVT options. METHODS: We performed a systematic literature search to identify studies evaluating the safety and efficacy of EVT versus standard medical treatment for patients with acute basilar artery occlusion (BAO). We extracted data for outcomes of interest and presented associations between the two groups with the use of risk ratios (RRs) or odds ratios (ORs), with corresponding 95% confidence intervals (CIs). We used a random-effects model to pool the effect estimates. RESULTS: We identified five studies (two RCTs, three observational cohorts) including a total of 1098 patients. Patients receiving EVT had a higher risk of symptomatic intracranial hemorrhage (sICH) compared to those receiving non-interventional medical management (RR 5.42, 95% CI 2.74-10.71). Nonsignificant trends towards modified Rankin Scale (mRS) scores 0-2 (RR 1.02, 95% CI 0.74-1.41), mRS scores 0-3 (RR = 0.97, 95% CI 0.64-1.47), overall functional improvement (OR 0.93, 95% CI 0.57-1.51), and all-cause mortality (RR 1.03, 95% CI 0.78-1.35) at 3 months were seen. CONCLUSION: Although EVT increases the probability of sICH, the available data do not exclude the possibility of improved functional outcomes over standard therapy. As larger studies are challenged by the perceived lack of equipoise in this vulnerable patient population, results of ongoing RCTs are expected to provide substantial input for future meta-analyses.

Prior distributions for variance parameters in a sparse-event meta-analysis of a few small trials.

In rare diseases, typically only a small number of patients are available for a randomized clinical trial. Nevertheless, it is not uncommon that more than one study is performed to evaluate a (new) treatment. Scarcity of available evidence makes it particularly valuable to pool the data in a meta-analysis. When the primary outcome is binary, the small sample sizes increase the chance of observing zero events. The frequentist random-effects model is known to induce bias and to result in improper interval estimation of the overall treatment effect in a meta-analysis with zero events. Bayesian hierarchical modeling could be a promising alternative. Bayesian models are known for being sensitive to the choice of prior distributions for between-study variance (heterogeneity) in sparse settings. In a rare disease setting, only limited data will be available to base the prior on, therefore, robustness of estimation is desirable. We performed an extensive and diverse simulation study, aiming to provide practitioners with advice on the choice of a sufficiently robust prior distribution shape for the heterogeneity parameter. Our results show that priors that place some concentrated mass on small τ values but do not restrict the density for example, the Uniform(-10, 10) heterogeneity prior on the log(τ2 ) scale, show robust 95% coverage combined with less overestimation of the overall treatment effect, across varying degrees of heterogeneity. We illustrate the results with meta-analyzes of a few small trials.

Pharmacogenetic interactions in amyotrophic lateral sclerosis: a step closer to a cure?

Genetic mutations related to amyotrophic lateral sclerosis (ALS) act through distinct pathophysiological pathways, which may lead to varying treatment responses. Here we assess the genetic interaction between C9orf72, UNC13A, and MOBP with creatine and valproic acid treatment in two clinical trials. Genotypic data was available for 309 of the 338 participants (91.4%). The UNC13A genotype affected mortality (p = 0.012), whereas C9orf72 repeat-expansion carriers exhibited a faster rate of decline in overall (p = 0.051) and bulbar functioning (p = 0.005). A dose-response pharmacogenetic interaction was identified between creatine and the A allele of the MOBP genotype (p = 0.027), suggesting a qualitative interaction in a recessive model (HR 3.96, p = 0.015). Not taking genetic information into account may mask evidence of response to treatment or be an unrecognized source of bias. Incorporating genetic data could help investigators to identify critical treatment clues in patients with ALS.

Effectiveness of a school food aid programme in improving household food insecurity; a cluster randomized trial.

BACKGROUND: Aim of this cluster randomized trial was to examine the impact of a school feeding programme combining healthy meals provision and educational activities to reduce food insecurity. METHODS: Schools participating in the DIATROFI Program in Greece during the 2014-2015 school year were randomly allocated between a multicomponent intervention (MI: each student received a daily healthy meal along with educational actions; 28 schools) and an educational intervention (EI; 23 schools). A linear-mixed model was used to examine intervention effect on change from baseline in the food insecurity score, as measured via the Food Security Survey Module (FSSM). The analysis was based on 1442 pre-post intervention questionnaire pairs in the MI group and 986 in the EI group. RESULTS: The reduction in food insecurity score in the MI group was statistically significantly greater compared to the EI group, by 9.8% or -0.31 [95% confidence interval (CI) -0.61 to -0.01] FSSM units after adjusting for potential confounders. MI intervention was significantly more effective compared to EI, among students in food insecure households (mean -0.44, 95% CI -0.84 to -0.04), students in households facing hunger (mean -1.04, 95% CI -1.91 to -0.17) and overweight/obese students (mean -0.36, 95% CI -0.72 to -0.01). CONCLUSION: For interventions aiming to address childhood food insecurity, public health focus should be oriented towards school-based programmes combining food assistance with activities that promote healthy nutrition.

Critical design considerations for time-to-event endpoints in amyotrophic lateral sclerosis clinical trials.

BACKGROUND: Funding and resources for low prevalent neurodegenerative disorders such as amyotrophic lateral sclerosis (ALS) are limited, and optimising their use is vital for efficient drug development. In this study, we review the design assumptions for pivotal ALS clinical trials with time-to-event endpoints and provide optimised settings for future trials. METHODS: We extracted design settings from 13 completed placebo-controlled trials. Optimal assumptions were estimated using parametric survival models in individual participant data (n=4991). Designs were compared in terms of sample size, trial duration, drug use and costs. RESULTS: Previous trials overestimated the hazard rate by 18.9% (95% CI 3.4% to 34.5%, p=0.021). The median expected HR was 0.56 (range 0.33-0.66). Additionally, we found evidence for an increasing mean hazard rate over time (Weibull shape parameter of 2.03, 95% CI 1.93 to 2.15, p<0.001), which affects the design and planning of future clinical trials. Incorporating accrual time and assuming an increasing hazard rate at the design stage reduced sample size by 33.2% (95% CI 27.9 to 39.4), trial duration by 17.4% (95% CI 11.6 to 23.3), drug use by 14.3% (95% CI 9.6 to 19.0) and follow-up costs by 21.2% (95% CI 15.6 to 26.8). CONCLUSIONS: Implementing distributional knowledge and incorporating accrual at the design stage could achieve large gains in the efficiency of ALS clinical trials with time-to-event endpoints. We provide an open-source platform that helps investigators to make more accurate sample size calculations and optimise the use of their available resources.

Neuropathy associated with immunoglobulin M monoclonal gammopathy: A combined sonographic and nerve conduction study.

INTRODUCTION: We assessed the specific sonographic pattern of structural nerve abnormalities in immunoglobulin M (IgM) neuropathy and disease controls. METHODS: We enrolled 106 incident patients-32 patients with IgM neuropathy, 42 treatment-naive patients with chronic inflammatory demyelinating polyneuropathy, and 32 patients with axonal neuropathies. All patients underwent standardized ancillary testing in addition to standardized sonography of the brachial plexus and the large arm and leg nerves bilaterally. RESULTS: We found widespread nerve enlargement in IgM neuropathy and chronic inflammatory demyelinating polyneuropathy (CIDP), with specific enlargement of brachial plexus and proximal segments of median nerve but not in axonal disease controls (P < .001). Sonographic nerve hypertrophy in IgM neuropathy was not associated with nerve conduction, clinical, or laboratory characteristics. DISCUSSION: Immunoglobulin M neuropathy is characterized by widespread nerve enlargement indistinguishable from CIDP. Our data provide evidence to confirm that the disease process is not confined to the more distal parts of nerves in either classical demyelinating or axonal variants of neuropathy with associated IgM.

Optimizing the Design and Analysis of Clinical Trials for Antibacterials Against Multidrug-resistant Organisms: A White Paper From COMBACTE's STAT-Net.

Innovations are urgently required for clinical development of antibacterials against multidrug-resistant organisms. Therefore, a European, public-private working group (STAT-Net; part of Combatting Bacterial Resistance in Europe [COMBACTE]), has reviewed and tested several innovative trials designs and analytical methods for randomized clinical trials, which has resulted in 8 recommendations. The first 3 focus on pharmacokinetic and pharmacodynamic modeling, emphasizing the pertinence of population-based pharmacokinetic models, regulatory procedures for the reassessment of old antibiotics, and rigorous quality improvement. Recommendations 4 and 5 address the need for more sensitive primary end points through the use of rank-based or time-dependent composite end points. Recommendation 6 relates to the applicability of hierarchical nested-trial designs, and the last 2 recommendations propose the incorporation of historical or concomitant trial data through Bayesian methods and/or platform trials. Although not all of these recommendations are directly applicable, they provide a solid, evidence-based approach to develop new, and established, antibacterials and address this public health challenge.

Monitoring disease progression with plasma creatinine in amyotrophic lateral sclerosis clinical trials.

OBJECTIVES: Plasma creatinine is a predictor of survival in amyotrophic lateral sclerosis (ALS). It remains, however, to be established whether it can monitor disease progression and serve as surrogate endpoint in clinical trials. METHODS: We used clinical trial data from three cohorts of clinical trial participants in the LITRA, EMPOWER and PROACT studies. Longitudinal associations between functional decline, muscle strength and survival with plasma creatinine were assessed. Results were translated to trial design in terms of sample size and power. RESULTS: A total of 13 564 measurements were obtained for 1241 patients. The variability between patients in rate of decline was lower in plasma creatinine than in ALS functional rating scale-Revised (ALSFRS-R; p<0.001). The average rate of decline was faster in the ALSFRS-R, with less between-patient variability at baseline (p<0.001). Plasma creatinine had strong longitudinal correlations with the ALSFRS-R (0.43 (0.39-0.46), p<0.001), muscle strength (0.55 (0.51-0.58), p<0.001) and overall mortality (HR 0.88 (0.86-0.91, p<0.001)). Using plasma creatinine as outcome could reduce the sample size in trials by 21.5% at 18 months. For trials up to 10 months, the ALSFRS-R required a lower sample size. CONCLUSIONS: Plasma creatinine is an inexpensive and easily accessible biomarker that exhibits less variability between patients with ALS over time and is predictive for the patient's functional status, muscle strength and mortality risk. Plasma creatinine may, therefore, increase the power to detect treatment effects and could be incorporated in future ALS clinical trials as potential surrogate outcome.

Dynamic borrowing through empirical power priors that control type I error.

In order for historical data to be considered for inclusion in the design and analysis of clinical trials, prospective rules are essential. Incorporation of historical data may be of particular interest in the case of small populations where available data is scarce and heterogeneity is not as well understood, and thus conventional methods for evidence synthesis might fall short. The concept of power priors can be particularly useful for borrowing evidence from a single historical study. Power priors employ a parameter γ ∈ [ 0 , 1 ] that quantifies the heterogeneity between the historical study and the new study. However, the possibility of borrowing data from a historical trial will usually be associated with an inflation of the type I error. We suggest a new, simple method of estimating the power parameter suitable for the case when only one historical dataset is available. The method is based on predictive distributions and parameterized in such a way that the type I error can be controlled by calibrating to the degree of similarity between the new and historical data. The method is demonstrated for normal responses in a one or two group setting. Generalization to other models is straightforward.

Data-generating models of dichotomous outcomes: Heterogeneity in simulation studies for a random-effects meta-analysis.

Simulation studies to evaluate performance of statistical methods require a well-specified data-generating model. Details of these models are essential to interpret the results and arrive at proper conclusions. A case in point is random-effects meta-analysis of dichotomous outcomes. We reviewed a number of simulation studies that evaluated approximate normal models for meta-analysis of dichotomous outcomes, and we assessed the data-generating models that were used to generate events for a series of (heterogeneous) trials. We demonstrate that the performance of the statistical methods, as assessed by simulation, differs between these 3 alternative data-generating models, with larger differences apparent in the small population setting. Our findings are relevant to multilevel binomial models in general.

An analytical approach to assess the predictive value of biomarkers in Phase II decision making.

Early Phase II clinical trials typically have a biomarker as a primary outcome. The extent to which a positive result for such a Phase II trial is predictive for Phase III success is of obvious concern. In this article we extend existing approaches for assessing the probability of success in Phase III. We address the case that a dichotomous clinical endpoint of interest is to be measured in Phase III. A Beta prior distribution is suggested that quantifies information from both the data observed on the biomarker and its predictive accuracy. Based on the quantification of the impact of the predictive ability of biomarkers, it is shown that the predictive quality of a biomarker needs to be substantial to support the full development decision. The approach is illustrated with a practical example. A simulation study is presented to illustrate the findings more generally.

Graphical tools for visualizing the results of network meta-analysis of multicomponent interventions.

Network meta-analysis (NMA) is an established method for assessing the comparative efficacy and safety of competing interventions. It is often the case that we deal with interventions that consist of multiple, possibly interacting, components. Examples of interventions' components include characteristics of the intervention, mode (face-to-face, remotely etc.), location (hospital, home etc.), provider (physician, nurse etc.), time of communication (synchronous, asynchronous etc.) and other context related components. Networks of multicomponent interventions are typically sparse and classical NMA inference is not straightforward and prone to confounding. Ideally, we would like to disentangle the effect of each component to find out what works (or does not work). To this aim, we propose novel ways of visualizing the NMA results, describe their use, and illustrate their application in real-life examples. We developed an R package viscomp to produce all the suggested figures.

Development and Evaluation of a Simulation-Based Algorithm to Optimize the Planning of Interim Analyses for Clinical Trials in ALS.

BACKGROUND AND OBJECTIVES: Late-phase clinical trials for neurodegenerative diseases have a low probability of success. In this study, we introduce an algorithm that optimizes the planning of interim analyses for clinical trials in amyotrophic lateral sclerosis (ALS) to better use the time and resources available and minimize the exposure of patients to ineffective or harmful drugs. METHODS: A simulation-based algorithm was developed to determine the optimal interim analysis scheme by integrating prior knowledge about the success rate of ALS clinical trials with drug-specific information obtained in early-phase studies. Interim analysis schemes were optimized by varying the number and timing of interim analyses, together with their decision rules about when to stop a trial. The algorithm was applied retrospectively to 3 clinical trials that investigated the efficacy of diaphragm pacing or ceftriaxone on survival in patients with ALS. Outcomes were additionally compared with conventional interim designs. RESULTS: We evaluated 183-1,351 unique interim analysis schemes for each trial. Application of the optimal designs correctly established lack of efficacy, would have concluded all studies 1.2-19.4 months earlier (reduction of 4.6%-57.7% in trial duration), and could have reduced the number of randomized patients by 1.7%-58.1%. By means of simulation, we illustrate the efficiency for other treatment scenarios. The optimized interim analysis schemes outperformed conventional interim designs in most scenarios. DISCUSSION: Our algorithm uses prior knowledge to determine the uncertainty of the expected treatment effect in ALS clinical trials and optimizes the planning of interim analyses. Improving futility monitoring in ALS could minimize the exposure of patients to ineffective or harmful treatments and result in significant ethical and efficiency gains.

Exploring the Effectiveness of Self-Management Interventions in Type 2 Diabetes: A Systematic Review and Network Meta-Analysis.

BACKGROUND: Chronic diseases are a leading cause of global morbidity and mortality. In response to this challenge, self-management interventions (SMIs) have emerged as an essential tool in improving patient outcomes. However, the diverse and complex nature of SMIs pose significant challenges in measuring their effectiveness. This work aims to investigate the comparative effectiveness of SMIs on Type 2 diabetes mellitus (T2DM) outcomes. METHODS: A rigorous analytical framework was employed to assess the relative effectiveness of different SMIs, encompassing both pairwise and network meta-analysis (NMA), as well as component network meta-analysis (CNMA). Various outcomes were considered, including glycated hemoglobin (HbA1c) control, body mass index (BMI) reduction and low-density lipoprotein (LDL) cholesterol. Visualization tools were also utilized to enhance the interpretation of results. RESULTS: SMIs were found promising in improving clinical outcomes and patient-reported measures. However, considerable heterogeneity and inconsistency across studies challenged the validity of NMA results. CNMA along with various visualization tools offered insights into the contributions of individual SMI components, highlighting the complexity of these interventions. DISCUSSION/CONCLUSIONS: SMIs represent a valuable approach to managing chronic conditions, but their effectiveness is context-dependent. Further research is needed to elucidate the contextual factors influencing SMI outcomes. This work contributes to a comprehensive understanding of SMIs' role in T2DM management, aiming to aid decision-makers, clinicians, and patients in selecting tailored interventions.

Modeling Multicomponent Interventions in Network Meta-Analysis.

There is a rapid increase in trials assessing healthcare interventions consisting of a combination of drugs (polytherapies) or multiple components. In the latter type of interventions (also known as complex interventions), the aspect of complexity is of paramount importance. For example, nonpharmacological interventions, such as psychological interventions or self-management interventions, usually share common components that relate to the nature of intervention, who delivers it, or where and how. In a network of trials, there is often the need to identify the most effective (or safest) component and/or combination of components. Four key meta-analytical approaches have been presented in the literature to handle complex interventions. These include (a) the single-effect model, (b) the full interaction model, (c) the additive main effects model, and (d) the two-way interaction model. In this chapter, we present and discuss the advantages and limitations of these approaches. We illustrate these methods using a network that assesses the relative effects of self-management interventions on waist size in patients with type 2 diabetes.

An old friend who has overstayed their welcome: the ALSFRS-R total score as primary endpoint for ALS clinical trials.

Objective: The ALSFRS-R is limited by multidimensionality, which originates from the summation of various subscales. This prevents a direct comparison between patients with identical total scores. We aim to evaluate how multidimensionality affects the performance of the ALSFRS-R in clinical trials. Methods: We simulated clinical trial data with different treatment effects for the ALSFRS-R total score and its subscales (i.e. bulbar, fine motor, gross motor and respiratory). We considered scenarios where treatment reduced the rate of ALSFRS-R subscale decline either uniformly (i.e. all subscales respond identically to treatment) or non-uniformly (i.e. subscales respond differently to treatment). Two main analytical strategies were compared: (1) analyzing only the total score or (2) utilizing a subscale-based test (i.e. alternative strategy). For each analytical strategy, we calculated the empirical power and required sample size. Results: Both strategies are valid when there is no treatment benefit and provide adequate control of type 1 error. If all subscales respond identically to treatment, using the total score is the most powerful approach. As the differences in treatment responses between subscales increase, the more the total score becomes affected. For example, to detect a 40% reduction in the bulbar rate of decline with 80% power, the total score requires 1380 patients, whereas this is 336 when using the alternative strategy. Conclusions: Ignoring the multidimensional structure of the ALSFRS-R total score could have negative consequences for ALS clinical trials. We propose determining treatment benefit on a subscale level, prior to stating whether a treatment is generally effective.