Do efficacy results obtained from randomized controlled trials translate to effectiveness data from observational studies for relapsing-remitting multiple sclerosis?

BACKGROUND: Randomized controlled trials are considered the gold standard in regulatory decision making, as observational studies are known to have important methodological limitations. However, real-world evidence may be helpful in specific situations. This review investigates how the effect estimates obtained from randomized controlled trials compare to those obtained from observational studies, using drug therapy for relapsing-remitting multiple sclerosis as an example. STUDY DESIGN AND SETTING: A systematic review of randomized controlled trials and observational studies was conducted. The primary outcome was the annualized relapse rate. Using (network) meta-analysis together with posterior predictive distributions, the drug-specific rate ratios from the network of randomized controlled trials were compared with those from the network of observational studies. RESULTS: Effect estimates from 26 observational studies showed greater magnitudes and were less precise compared to estimates obtained from 21 randomized controlled trials. Twenty of the 28 treatment comparisons between designs had similar rate ratios. Seven inconsistencies in observed rate ratios could be attributed to two specific disease-modifying therapies. CONCLUSION: In this case study, estimates from observational studies predominantly agreed with estimates from randomized controlled trials given their posterior predictive distributions. Multiple observational studies together may therefore supplement additional pivotal randomized controlled trials in relapsing-remitting multiple sclerosis, for instance facilitating the extrapolation of trial results to the broader patient population.

Inconsistency identification in network meta-analysis via stochastic search variable selection.

The reliability of the results of network meta-analysis (NMA) lies in the plausibility of the key assumption of transitivity. This assumption implies that the effect modifiers' distribution is similar across treatment comparisons. Transitivity is statistically manifested through the consistency assumption which suggests that direct and indirect evidence are in agreement. Several methods have been suggested to evaluate consistency. A popular approach suggests adding inconsistency factors to the NMA model. We follow a different direction by describing each inconsistency factor with a candidate covariate whose choice relies on variable selection techniques. Our proposed method, stochastic search inconsistency factor selection (SSIFS), evaluates the consistency assumption both locally and globally, by applying the stochastic search variable selection method to determine whether the inconsistency factors should be included in the model. The posterior inclusion probability of each inconsistency factor quantifies how likely is a specific comparison to be inconsistent. We use posterior model odds or the median probability model to decide on the importance of inconsistency factors. Differences between direct and indirect evidence can be incorporated into the inconsistency detection process. A key point of our proposed approach is the construction of a reasonable "informative" prior concerning network consistency. The prior is based on the elicitation of information derived historical data from 201 published network meta-analyses. The performance of our proposed method is evaluated in two published network meta-analyses. The proposed methodology is publicly available in an R package called ssifs, published on CRAN and developed and maintained by the authors of this work.

Development and Evaluation of a Simulation-Based Algorithm to Optimize the Planning of Interim Analyses for Clinical Trials in ALS.

BACKGROUND AND OBJECTIVES: Late-phase clinical trials for neurodegenerative diseases have a low probability of success. In this study, we introduce an algorithm that optimizes the planning of interim analyses for clinical trials in amyotrophic lateral sclerosis (ALS) to better use the time and resources available and minimize the exposure of patients to ineffective or harmful drugs. METHODS: A simulation-based algorithm was developed to determine the optimal interim analysis scheme by integrating prior knowledge about the success rate of ALS clinical trials with drug-specific information obtained in early-phase studies. Interim analysis schemes were optimized by varying the number and timing of interim analyses, together with their decision rules about when to stop a trial. The algorithm was applied retrospectively to 3 clinical trials that investigated the efficacy of diaphragm pacing or ceftriaxone on survival in patients with ALS. Outcomes were additionally compared with conventional interim designs. RESULTS: We evaluated 183-1,351 unique interim analysis schemes for each trial. Application of the optimal designs correctly established lack of efficacy, would have concluded all studies 1.2-19.4 months earlier (reduction of 4.6%-57.7% in trial duration), and could have reduced the number of randomized patients by 1.7%-58.1%. By means of simulation, we illustrate the efficiency for other treatment scenarios. The optimized interim analysis schemes outperformed conventional interim designs in most scenarios. DISCUSSION: Our algorithm uses prior knowledge to determine the uncertainty of the expected treatment effect in ALS clinical trials and optimizes the planning of interim analyses. Improving futility monitoring in ALS could minimize the exposure of patients to ineffective or harmful treatments and result in significant ethical and efficiency gains.

Graphical tools for visualizing the results of network meta-analysis of multicomponent interventions.

Network meta-analysis (NMA) is an established method for assessing the comparative efficacy and safety of competing interventions. It is often the case that we deal with interventions that consist of multiple, possibly interacting, components. Examples of interventions' components include characteristics of the intervention, mode (face-to-face, remotely etc.), location (hospital, home etc.), provider (physician, nurse etc.), time of communication (synchronous, asynchronous etc.) and other context related components. Networks of multicomponent interventions are typically sparse and classical NMA inference is not straightforward and prone to confounding. Ideally, we would like to disentangle the effect of each component to find out what works (or does not work). To this aim, we propose novel ways of visualizing the NMA results, describe their use, and illustrate their application in real-life examples. We developed an R package viscomp to produce all the suggested figures.

Exploring the Effectiveness of Self-Management Interventions in Type 2 Diabetes: A Systematic Review and Network Meta-Analysis.

BACKGROUND: Chronic diseases are a leading cause of global morbidity and mortality. In response to this challenge, self-management interventions (SMIs) have emerged as an essential tool in improving patient outcomes. However, the diverse and complex nature of SMIs pose significant challenges in measuring their effectiveness. This work aims to investigate the comparative effectiveness of SMIs on Type 2 diabetes mellitus (T2DM) outcomes. METHODS: A rigorous analytical framework was employed to assess the relative effectiveness of different SMIs, encompassing both pairwise and network meta-analysis (NMA), as well as component network meta-analysis (CNMA). Various outcomes were considered, including glycated hemoglobin (HbA1c) control, body mass index (BMI) reduction and low-density lipoprotein (LDL) cholesterol. Visualization tools were also utilized to enhance the interpretation of results. RESULTS: SMIs were found promising in improving clinical outcomes and patient-reported measures. However, considerable heterogeneity and inconsistency across studies challenged the validity of NMA results. CNMA along with various visualization tools offered insights into the contributions of individual SMI components, highlighting the complexity of these interventions. DISCUSSION/CONCLUSIONS: SMIs represent a valuable approach to managing chronic conditions, but their effectiveness is context-dependent. Further research is needed to elucidate the contextual factors influencing SMI outcomes. This work contributes to a comprehensive understanding of SMIs' role in T2DM management, aiming to aid decision-makers, clinicians, and patients in selecting tailored interventions.

Modeling Multicomponent Interventions in Network Meta-Analysis.

There is a rapid increase in trials assessing healthcare interventions consisting of a combination of drugs (polytherapies) or multiple components. In the latter type of interventions (also known as complex interventions), the aspect of complexity is of paramount importance. For example, nonpharmacological interventions, such as psychological interventions or self-management interventions, usually share common components that relate to the nature of intervention, who delivers it, or where and how. In a network of trials, there is often the need to identify the most effective (or safest) component and/or combination of components. Four key meta-analytical approaches have been presented in the literature to handle complex interventions. These include (a) the single-effect model, (b) the full interaction model, (c) the additive main effects model, and (d) the two-way interaction model. In this chapter, we present and discuss the advantages and limitations of these approaches. We illustrate these methods using a network that assesses the relative effects of self-management interventions on waist size in patients with type 2 diabetes.

Innovating Clinical Trials for Amyotrophic Lateral Sclerosis: Challenging the Established Order.

Development of effective treatments for amyotrophic lateral sclerosis (ALS) has been hampered by disease heterogeneity, a limited understanding of underlying pathophysiology, and methodologic design challenges. We have evaluated 2 major themes in the design of pivotal, phase 3 clinical trials for ALS-(1) patient selection and (2) analytical strategy-and discussed potential solutions with the European Medicines Agency. Several design considerations were assessed using data from 5 placebo-controlled clinical trials (n = 988), 4 population-based cohorts (n = 5,100), and 2,436 placebo-allocated patients from the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) database. The validity of each proposed design modification was confirmed by means of simulation and illustrated for a hypothetical setting. Compared to classical trial design, the proposed design modifications reduce the sample size by 30.5% and placebo exposure time by 35.4%. By making use of prognostic survival models, one creates a potential to include a larger proportion of the population and maximize generalizability. We propose a flexible design framework that naturally adapts the trial duration when inaccurate assumptions are made at the design stage, such as enrollment or survival rate. In case of futility, the follow-up time is shortened and patient exposure to ineffective treatments or placebo is minimized. For diseases such as ALS, optimizing the use of resources, widening eligibility criteria, and minimizing exposure to futile treatments and placebo is critical to the development of effective treatments. Our proposed design modifications could circumvent important pitfalls and may serve as a blueprint for future clinical trials in this population.

Combined assessment of early and late-phase outcomes in orphan drug development.

In drug development programs, proof-of-concept Phase II clinical trials typically have a biomarker as a primary outcome, or an outcome that can be observed with relatively short follow-up. Subsequently, the Phase III clinical trials aim to demonstrate the treatment effect based on a clinical outcome that often needs a longer follow-up to be assessed. Early-phase outcomes or biomarkers are typically associated with late-phase outcomes and they are often included in Phase III trials. The decision to proceed to Phase III development is based on analysis of the early-Phase II outcome data. In rare diseases, it is likely that only one Phase II trial and one Phase III trial are available. In such cases and before drug marketing authorization requests, positive results of the early-phase outcome of Phase II trials are then likely seen as supporting (or even replicating) positive Phase III results on the late-phase outcome, without a formal retrospective combined assessment and without accounting for between-study differences. We used double-regression modeling applied to the Phase II and Phase III results to numerically mimic this informal retrospective assessment. We provide an analytical solution for the bias and mean square error of the overall effect that leads to a corrected double-regression. We further propose a flexible Bayesian double-regression approach that minimizes the bias by accounting for between-study differences via discounting the Phase II early-phase outcome when they are not in line with the Phase III biomarker outcome results. We illustrate all methods with an orphan drug example for Fabry disease.

An old friend who has overstayed their welcome: the ALSFRS-R total score as primary endpoint for ALS clinical trials.

Objective: The ALSFRS-R is limited by multidimensionality, which originates from the summation of various subscales. This prevents a direct comparison between patients with identical total scores. We aim to evaluate how multidimensionality affects the performance of the ALSFRS-R in clinical trials. Methods: We simulated clinical trial data with different treatment effects for the ALSFRS-R total score and its subscales (i.e. bulbar, fine motor, gross motor and respiratory). We considered scenarios where treatment reduced the rate of ALSFRS-R subscale decline either uniformly (i.e. all subscales respond identically to treatment) or non-uniformly (i.e. subscales respond differently to treatment). Two main analytical strategies were compared: (1) analyzing only the total score or (2) utilizing a subscale-based test (i.e. alternative strategy). For each analytical strategy, we calculated the empirical power and required sample size. Results: Both strategies are valid when there is no treatment benefit and provide adequate control of type 1 error. If all subscales respond identically to treatment, using the total score is the most powerful approach. As the differences in treatment responses between subscales increase, the more the total score becomes affected. For example, to detect a 40% reduction in the bulbar rate of decline with 80% power, the total score requires 1380 patients, whereas this is 336 when using the alternative strategy. Conclusions: Ignoring the multidimensional structure of the ALSFRS-R total score could have negative consequences for ALS clinical trials. We propose determining treatment benefit on a subscale level, prior to stating whether a treatment is generally effective.

Endovascular treatment for basilar artery occlusion: A systematic review and meta-analysis.

BACKGROUND AND PURPOSE: Independent randomized controlled clinical trials (RCTs) have provided robust evidence for endovascular treatment (EVT) as the standard of care treatment for acute large vessel occlusions in the anterior circulation. We examined available studies specific to posterior cerebral circulation ischemic strokes to see if any conclusions can be drawn regarding EVT options. METHODS: We performed a systematic literature search to identify studies evaluating the safety and efficacy of EVT versus standard medical treatment for patients with acute basilar artery occlusion (BAO). We extracted data for outcomes of interest and presented associations between the two groups with the use of risk ratios (RRs) or odds ratios (ORs), with corresponding 95% confidence intervals (CIs). We used a random-effects model to pool the effect estimates. RESULTS: We identified five studies (two RCTs, three observational cohorts) including a total of 1098 patients. Patients receiving EVT had a higher risk of symptomatic intracranial hemorrhage (sICH) compared to those receiving non-interventional medical management (RR 5.42, 95% CI 2.74-10.71). Nonsignificant trends towards modified Rankin Scale (mRS) scores 0-2 (RR 1.02, 95% CI 0.74-1.41), mRS scores 0-3 (RR = 0.97, 95% CI 0.64-1.47), overall functional improvement (OR 0.93, 95% CI 0.57-1.51), and all-cause mortality (RR 1.03, 95% CI 0.78-1.35) at 3 months were seen. CONCLUSION: Although EVT increases the probability of sICH, the available data do not exclude the possibility of improved functional outcomes over standard therapy. As larger studies are challenged by the perceived lack of equipoise in this vulnerable patient population, results of ongoing RCTs are expected to provide substantial input for future meta-analyses.

Prior distributions for variance parameters in a sparse-event meta-analysis of a few small trials.

In rare diseases, typically only a small number of patients are available for a randomized clinical trial. Nevertheless, it is not uncommon that more than one study is performed to evaluate a (new) treatment. Scarcity of available evidence makes it particularly valuable to pool the data in a meta-analysis. When the primary outcome is binary, the small sample sizes increase the chance of observing zero events. The frequentist random-effects model is known to induce bias and to result in improper interval estimation of the overall treatment effect in a meta-analysis with zero events. Bayesian hierarchical modeling could be a promising alternative. Bayesian models are known for being sensitive to the choice of prior distributions for between-study variance (heterogeneity) in sparse settings. In a rare disease setting, only limited data will be available to base the prior on, therefore, robustness of estimation is desirable. We performed an extensive and diverse simulation study, aiming to provide practitioners with advice on the choice of a sufficiently robust prior distribution shape for the heterogeneity parameter. Our results show that priors that place some concentrated mass on small τ values but do not restrict the density for example, the Uniform(-10, 10) heterogeneity prior on the log(τ2 ) scale, show robust 95% coverage combined with less overestimation of the overall treatment effect, across varying degrees of heterogeneity. We illustrate the results with meta-analyzes of a few small trials.

Misuse of the sign test in narrative synthesis of evidence.

In narrative synthesis of evidence, it can be the case that the only quantitative measures available concerning the efficacy of an intervention is the direction of the effect, that is, whether it is positive or negative. In such situations, the sign test has been proposed in the literature and in recent Cochrane guidelines as a way to test whether the proportion of positive effects is favorable. I argue that the sign test is inappropriate in this context as the data are not generated according to the binomial distribution it employs. I demonstrate possible consequences for both hypothesis testing and estimation via hypothetical examples.

Prevention of severe infectious complications after colorectal surgery using oral non-absorbable antimicrobial prophylaxis: results of a multicenter randomized placebo-controlled clinical trial.

BACKGROUND: Surgical site infections (SSIs) are common complications after colorectal surgery. Oral non-absorbable antibiotic prophylaxis (OAP) can be administered preoperatively to reduce the risk of SSIs. Its efficacy without simultaneous mechanical cleaning is unknown. METHODS: The Precaution trial was a double-blind, placebo-controlled randomized clinical trial conducted in six Dutch hospitals. Adult patients who underwent elective colorectal surgery were randomized to receive either a three-day course of preoperative OAP with tobramycin and colistin or placebo. The primary composite endpoint was the incidence of deep SSI or mortality within 30 days after surgery. Secondary endpoints included both infectious and non-infectious complications at 30 days and six months after surgery. RESULTS: The study was prematurely ended due to the loss of clinical equipoise. At that time, 39 patients had been randomized to active OAP and 39 to placebo, which reflected 8.1% of the initially pursued sample size. Nine (11.5%) patients developed the primary outcome, of whom four had been randomized to OAP (4/39; 10.3%) and five to placebo (5/39; 12.8%). This corresponds to a risk ratio in the intention-to-treat analysis of 0.80 (95% confidence interval (CI) 0.23-2.78). In the per-protocol analysis, the relative risk was 0.64 (95% CI 0.12-3.46). CONCLUSIONS: Observational data emerging during the study provided new evidence for the effectiveness of OAP that changed both the clinical and medical ethical landscape for infection prevention in colorectal surgery. We therefore consider it unethical to continue randomizing patients to placebo. We recommend the implementation of OAP in clinical practice and continuing monitoring of infection rates and antibiotic susceptibilities. TRIAL REGISTRATION: The PreCaution trial is registered in the Netherlands Trial Register under NL5932 (previously: NTR6113) as well as in the EudraCT register under 2015-005736-17.

Effectiveness of a school food aid programme in improving household food insecurity; a cluster randomized trial.

BACKGROUND: Aim of this cluster randomized trial was to examine the impact of a school feeding programme combining healthy meals provision and educational activities to reduce food insecurity. METHODS: Schools participating in the DIATROFI Program in Greece during the 2014-2015 school year were randomly allocated between a multicomponent intervention (MI: each student received a daily healthy meal along with educational actions; 28 schools) and an educational intervention (EI; 23 schools). A linear-mixed model was used to examine intervention effect on change from baseline in the food insecurity score, as measured via the Food Security Survey Module (FSSM). The analysis was based on 1442 pre-post intervention questionnaire pairs in the MI group and 986 in the EI group. RESULTS: The reduction in food insecurity score in the MI group was statistically significantly greater compared to the EI group, by 9.8% or -0.31 [95% confidence interval (CI) -0.61 to -0.01] FSSM units after adjusting for potential confounders. MI intervention was significantly more effective compared to EI, among students in food insecure households (mean -0.44, 95% CI -0.84 to -0.04), students in households facing hunger (mean -1.04, 95% CI -1.91 to -0.17) and overweight/obese students (mean -0.36, 95% CI -0.72 to -0.01). CONCLUSION: For interventions aiming to address childhood food insecurity, public health focus should be oriented towards school-based programmes combining food assistance with activities that promote healthy nutrition.

Pharmacogenetic interactions in amyotrophic lateral sclerosis: a step closer to a cure?

Genetic mutations related to amyotrophic lateral sclerosis (ALS) act through distinct pathophysiological pathways, which may lead to varying treatment responses. Here we assess the genetic interaction between C9orf72, UNC13A, and MOBP with creatine and valproic acid treatment in two clinical trials. Genotypic data was available for 309 of the 338 participants (91.4%). The UNC13A genotype affected mortality (p = 0.012), whereas C9orf72 repeat-expansion carriers exhibited a faster rate of decline in overall (p = 0.051) and bulbar functioning (p = 0.005). A dose-response pharmacogenetic interaction was identified between creatine and the A allele of the MOBP genotype (p = 0.027), suggesting a qualitative interaction in a recessive model (HR 3.96, p = 0.015). Not taking genetic information into account may mask evidence of response to treatment or be an unrecognized source of bias. Incorporating genetic data could help investigators to identify critical treatment clues in patients with ALS.

Neuropathy associated with immunoglobulin M monoclonal gammopathy: A combined sonographic and nerve conduction study.

INTRODUCTION: We assessed the specific sonographic pattern of structural nerve abnormalities in immunoglobulin M (IgM) neuropathy and disease controls. METHODS: We enrolled 106 incident patients-32 patients with IgM neuropathy, 42 treatment-naive patients with chronic inflammatory demyelinating polyneuropathy, and 32 patients with axonal neuropathies. All patients underwent standardized ancillary testing in addition to standardized sonography of the brachial plexus and the large arm and leg nerves bilaterally. RESULTS: We found widespread nerve enlargement in IgM neuropathy and chronic inflammatory demyelinating polyneuropathy (CIDP), with specific enlargement of brachial plexus and proximal segments of median nerve but not in axonal disease controls (P < .001). Sonographic nerve hypertrophy in IgM neuropathy was not associated with nerve conduction, clinical, or laboratory characteristics. DISCUSSION: Immunoglobulin M neuropathy is characterized by widespread nerve enlargement indistinguishable from CIDP. Our data provide evidence to confirm that the disease process is not confined to the more distal parts of nerves in either classical demyelinating or axonal variants of neuropathy with associated IgM.

Critical design considerations for time-to-event endpoints in amyotrophic lateral sclerosis clinical trials.

BACKGROUND: Funding and resources for low prevalent neurodegenerative disorders such as amyotrophic lateral sclerosis (ALS) are limited, and optimising their use is vital for efficient drug development. In this study, we review the design assumptions for pivotal ALS clinical trials with time-to-event endpoints and provide optimised settings for future trials. METHODS: We extracted design settings from 13 completed placebo-controlled trials. Optimal assumptions were estimated using parametric survival models in individual participant data (n=4991). Designs were compared in terms of sample size, trial duration, drug use and costs. RESULTS: Previous trials overestimated the hazard rate by 18.9% (95% CI 3.4% to 34.5%, p=0.021). The median expected HR was 0.56 (range 0.33-0.66). Additionally, we found evidence for an increasing mean hazard rate over time (Weibull shape parameter of 2.03, 95% CI 1.93 to 2.15, p<0.001), which affects the design and planning of future clinical trials. Incorporating accrual time and assuming an increasing hazard rate at the design stage reduced sample size by 33.2% (95% CI 27.9 to 39.4), trial duration by 17.4% (95% CI 11.6 to 23.3), drug use by 14.3% (95% CI 9.6 to 19.0) and follow-up costs by 21.2% (95% CI 15.6 to 26.8). CONCLUSIONS: Implementing distributional knowledge and incorporating accrual at the design stage could achieve large gains in the efficiency of ALS clinical trials with time-to-event endpoints. We provide an open-source platform that helps investigators to make more accurate sample size calculations and optimise the use of their available resources.

Refining eligibility criteria for amyotrophic lateral sclerosis clinical trials.

OBJECTIVE: To assess the effect of eligibility criteria on exclusion rates, generalizability, and outcome heterogeneity in amyotrophic lateral sclerosis (ALS) clinical trials and to assess the value of a risk-based inclusion criterion. METHODS: A literature search was performed to summarize the eligibility criteria of clinical trials. The extracted criteria were applied to an incidence cohort of 2,904 consecutive patients with ALS to quantify their effects on generalizability and outcome heterogeneity. We evaluated the effect of a risk-based selection approach on trial design using a personalized survival prediction model. RESULTS: We identified 38 trials. A large variability exists between trials in all patient characteristics for enrolled patients (p < 0.001), except for the proportion of men (p = 0.21). Exclusion rates varied widely (from 14% to 95%; mean 59.8%; 95% confidence interval 52.6%-66.7%). Stratification of the eligible populations into prognostic subgroups showed that eligibility criteria lead to exclusion of patients in all prognostic groups. Eligibility criteria neither reduce heterogeneity in survival time (from 22.0 to 20.5 months, p = 0.09) nor affect between-patient variability in functional decline (from 0.62 to 0.65, p = 0.25). In none of the 38 trials were the eligibility criteria found to be more efficient than the prediction model in optimizing sample size and eligibility rate. CONCLUSIONS: The majority of patients with ALS are excluded from trial participation, which questions the generalizability of trial results. Eligibility criteria only minimally improve homogeneity in trial endpoints. An individualized risk-based criterion could be used to balance the gains in trial design and loss in generalizability.

Nerve ultrasound: A reproducible diagnostic tool in peripheral neuropathy.

OBJECTIVE: To determine interobserver variability of nerve ultrasound in peripheral neuropathy in a prospective, systematic, multicenter study. METHODS: We enrolled 20 patients with an acquired chronic demyelinating or axonal polyneuropathy and 10 healthy controls in 3 different centers. All participants underwent an extensive nerve ultrasound protocol, including cross-sectional area measurements of median, ulnar, fibular, tibial, and sural nerves, and brachial plexus. Real-time image acquisition was performed blind by a local and a visiting investigator (reference). Five patients were investigated using different types of sonographic devices. Intraclass correlation coefficients were calculated, and a random-effects model was fitted to identify factors with significant effect on interobserver variability. RESULTS: Systematic differences between measurements made by different investigators were small (mean difference 0.11 mm2 [95% confidence interval 0.00-0.23 mm2]). Intraclass correlation coefficients were generally higher in arm nerves (0.48-0.96) than leg nerves (0.46-0.61). The hospital site and sonographic device did not contribute significantly to interobserver variability in the random-effects model. CONCLUSIONS: Interobserver variability of nerve ultrasound in peripheral neuropathy is generally limited, especially in arm nerves. Different devices and a multicenter setting have no effect on interobserver variability. Therefore, nerve ultrasound is a reproducible tool for diagnostics in routine clinical practice and (multicenter) research.