Combining randomized and non-randomized data to predict heterogeneous effects of competing treatments.

Some patients benefit from a treatment while others may do so less or do not benefit at all. We have previously developed a two-stage network meta-regression prediction model that synthesized randomized trials and evaluates how treatment effects vary across patient characteristics. In this article, we extended this model to combine different sources of types in different formats: aggregate data (AD) and individual participant data (IPD) from randomized and non-randomized evidence. In the first stage, a prognostic model is developed to predict the baseline risk of the outcome using a large cohort study. In the second stage, we recalibrated this prognostic model to improve our predictions for patients enrolled in randomized trials. In the third stage, we used the baseline risk as effect modifier in a network meta-regression model combining AD, IPD randomized clinical trial to estimate heterogeneous treatment effects. We illustrated the approach in the re-analysis of a network of studies comparing three drugs for relapsing-remitting multiple sclerosis. Several patient characteristics influence the baseline risk of relapse, which in turn modifies the effect of the drugs. The proposed model makes personalized predictions for health outcomes under several treatment options and encompasses all relevant randomized and non-randomized evidence.

Estimating Patient-Specific Relative Benefit of Adding Biologics to Conventional Rheumatoid Arthritis Treatment: An Individual Participant Data Meta-Analysis.

Importance: Current evidence remains ambiguous regarding whether biologics should be added to conventional treatment of rheumatoid arthritis for specific patients, which may cause potential overuse or treatment delay. Objectives: To estimate the benefit of adding biologics to conventional antirheumatic drugs for the treatment of rheumatoid arthritis given baseline characteristics. Data Sources: Cochrane CENTRAL, Scopus, MEDLINE, and the World Health Organization International Clinical Trials Registry Platform were searched for articles published from database inception to March 2, 2022. Study Selection: Randomized clinical trials comparing certolizumab plus conventional antirheumatic drugs with placebo plus conventional drugs were selected. Data Extraction and Synthesis: Individual participant data of the prespecified outcomes and covariates were acquired from the Vivli database. A 2-stage model was fitted to estimate patient-specific relative outcomes of adding certolizumab vs conventional drugs only. Stage 1 was a penalized logistic regression model to estimate the baseline expected probability of the outcome regardless of treatment using baseline characteristics. Stage 2 was a bayesian individual participant data meta-regression model to estimate the relative outcomes for a particular baseline expected probability. Patient-specific results were displayed interactively on an application based on a 2-stage model. Main Outcomes and Measures: The primary outcome was low disease activity or remission at 3 months, defined by 3 disease activity indexes (ie, Disease Activity Score based on the evaluation of 28 joints, Clinical Disease Activity Index, or Simplified Disease Activity Index). Results: Individual participant data were obtained from 3790 patients (2996 female [79.1%] and 794 male [20.9%]; mean [SD] age, 52.7 [12.3] years) from 5 large randomized clinical trials for moderate to high activity rheumatoid arthritis with usable data for 22 prespecified baseline covariates. Overall, adding certolizumab was associated with a higher probability of reaching low disease activity. The odds ratio for patients with an average baseline expected probability of the outcome was 6.31 (95% credible interval, 2.22-15.25). However, the benefits differed in patients with different baseline characteristics. For example, the estimated risk difference was smaller than 10% for patients with either low or high baseline expected probability. Conclusions and Relevance: In this individual participant data meta-analysis, adding certolizumab was associated with more effectiveness for rheumatoid arthritis in general. However, the benefit was uncertain for patients with low or high baseline expected probability, for whom other evaluations were necessary. The interactive application displaying individual estimates may help with treatment selection.

Synthesizing cross-design evidence and cross-format data using network meta-regression.

In network meta-analysis (NMA), we synthesize all relevant evidence about health outcomes with competing treatments. The evidence may come from randomized clinical trials (RCT) or non-randomized studies (NRS) as individual participant data (IPD) or as aggregate data (AD). We present a suite of Bayesian NMA and network meta-regression (NMR) models allowing for cross-design and cross-format synthesis. The models integrate a three-level hierarchical model for synthesizing IPD and AD into four approaches. The four approaches account for differences in the design and risk of bias (RoB) in the RCT and NRS evidence. These four approaches variously ignoring differences in RoB, using NRS to construct penalized treatment effect priors and bias-adjustment models that control the contribution of information from high RoB studies in two different ways. We illustrate the methods in a network of three pharmacological interventions and placebo for patients with relapsing-remitting multiple sclerosis. The estimated relative treatment effects do not change much when we accounted for differences in design and RoB. Conducting network meta-regression showed that intervention efficacy decreases with increasing participant age. We also re-analysed a network of 431 RCT comparing 21 antidepressants, and we did not observe material changes in intervention efficacy when adjusting for studies' high RoB. We re-analysed both case studies accounting for different study RoB. In summary, the described suite of NMA/NMR models enables the inclusion of all relevant evidence while incorporating information on the within-study bias in both observational and experimental data and enabling estimation of individualized treatment effects through the inclusion of participant characteristics.

Decision Curve Analysis for Personalized Treatment Choice between Multiple Options.

BACKGROUND: Decision curve analysis can be used to determine whether a personalized model for treatment benefit would lead to better clinical decisions. Decision curve analysis methods have been described to estimate treatment benefit using data from a single randomized controlled trial. OBJECTIVES: Our main objective is to extend the decision curve analysis methodology to the scenario in which several treatment options exist and evidence about their effects comes from a set of trials, synthesized using network meta-analysis (NMA). METHODS: We describe the steps needed to estimate the net benefit of a prediction model using evidence from studies synthesized in an NMA. We show how to compare personalized versus one-size-fit-all treatment decision-making strategies, such as "treat none" or "treat all patients with a specific treatment" strategies. First, threshold values for each included treatment need to be defined (i.e., the minimum risk difference compared with control that renders a treatment worth taking). The net benefit per strategy can then be plotted for a plausible range of threshold values to reveal the most clinically useful strategy. We applied our methodology to an NMA prediction model for relapsing-remitting multiple sclerosis, which can be used to choose between natalizumab, dimethyl fumarate, glatiramer acetate, and placebo. RESULTS: We illustrated the extended decision curve analysis methodology using several threshold value combinations for each available treatment. For the examined threshold values, the "treat patients according to the prediction model" strategy performs either better than or close to the one-size-fit-all treatment strategies. However, even small differences may be important in clinical decision making. As the advantage of the personalized model was not consistent across all thresholds, improving the existing model (by including, for example, predictors that will increase discrimination) is needed before advocating its clinical usefulness. CONCLUSIONS: This novel extension of decision curve analysis can be applied to NMA-based prediction models to evaluate their use to aid treatment decision making. HIGHLIGHTS: Decision curve analysis is extended into a (network) meta-analysis framework.Personalized models predicting treatment benefit are evaluated when several treatment options are available and evidence about their effects comes from a set of trials.Detailed steps to compare personalized versus one-size-fit-all treatment decision-making strategies are outlined.This extension of decision curve analysis can be applied to (network) meta-analysis-based prediction models to evaluate their use to aid treatment decision making.

Comparative efficacy and tolerability of 32 oral and long-acting injectable antipsychotics for the maintenance treatment of adults with schizophrenia: a systematic review and network meta-analysis.

BACKGROUND: Schizophrenia is a common, severe, and usually chronic disorder. Maintenance treatment with antipsychotic drugs can prevent relapse but also causes side-effects. We aimed to compare the efficacy and tolerability of antipsychotics as maintenance treatment for non-treatment resistant patients with schizophrenia. METHODS: In this systematic review and network meta-analysis, we searched, without language restrictions, the Cochrane Schizophrenia Group's specialised register between database inception and April 27, 2020, PubMed from April 1, 2020, to Jan 15, 2021, and the lists of included studies from related systematic reviews. We included randomised controlled trials (RCTs; ≥12 weeks of follow-up) that recruited adult participants with schizophrenia or schizoaffective disorder with stable symptoms who were treated with antipsychotics (monotherapy; oral or long-acting injectable) or placebo. We excluded RCTs of participants with specific comorbidities or treatment resistance. In duplicate, two authors independently selected eligible RCTs and extracted aggregate data. The primary outcome was the number of participants who relapsed and was analysed by random-effects, Bayesian network meta-analyses. The study was registered on PROSPERO, CRD42016049022. FINDINGS: We identified 4157 references through our search, from which 501 references on 127 RCTs of 32 antipsychotics (comprising 18 152 participants) were included. 100 studies including 16 812 participants and 30 antipsychotics contributed to our network meta-analysis of the primary outcome. All antipsychotics had risk ratios (RRs) less than 1·00 when compared with placebo for relapse prevention and almost all had 95% credible intervals (CrIs) excluding no effect. RRs ranged from 0·20 (95% CrI 0·05-0·41) for paliperidone oral to 0·65 (0·16-1·14) for cariprazine oral (moderate-to-low confidence in estimates). Generally, we interpret that there was no clear evidence for the superiority of specific antipsychotics in terms of relapse prevention because most comparisons between antipsychotics included a probability of no difference. INTERPRETATION: As we found no clear differences between antipsychotics for relapse prevention, we conclude that the choice of antipsychotic for maintenance treatment should be guided mainly by their tolerability. FUNDING: The German Ministry of Education and Research and Oxford Health Biomedical Research Centre.

Development, validation and clinical usefulness of a prognostic model for relapse in relapsing-remitting multiple sclerosis.

BACKGROUND: Prognosis for the occurrence of relapses in individuals with relapsing-remitting multiple sclerosis (RRMS), the most common subtype of multiple sclerosis (MS), could support individualized decisions and disease management and could be helpful for efficiently selecting patients for future randomized clinical trials. There are only three previously published prognostic models on this, all of them with important methodological shortcomings. OBJECTIVES: We aim to present the development, internal validation, and evaluation of the potential clinical benefit of a prognostic model for relapses for individuals with RRMS using real-world data. METHODS: We followed seven steps to develop and validate the prognostic model: (1) selection of prognostic factors via a review of the literature, (2) development of a generalized linear mixed-effects model in a Bayesian framework, (3) examination of sample size efficiency, (4) shrinkage of the coefficients, (5) dealing with missing data using multiple imputations, (6) internal validation of the model. Finally, we evaluated the potential clinical benefit of the developed prognostic model using decision curve analysis. For the development and the validation of our prognostic model, we followed the TRIPOD statement. RESULTS: We selected eight baseline prognostic factors: age, sex, prior MS treatment, months since last relapse, disease duration, number of prior relapses, expanded disability status scale (EDSS) score, and number of gadolinium-enhanced lesions. We also developed a web application that calculates an individual's probability of relapsing within the next 2 years. The optimism-corrected c-statistic is 0.65 and the optimism-corrected calibration slope is 0.92. For threshold probabilities between 15 and 30%, the "treat based on the prognostic model" strategy leads to the highest net benefit and hence is considered the most clinically useful strategy. CONCLUSIONS: The prognostic model we developed offers several advantages in comparison to previously published prognostic models on RRMS. Importantly, we assessed the potential clinical benefit to better quantify the clinical impact of the model. Our web application, once externally validated in the future, could be used by patients and doctors to calculate the individualized probability of relapsing within 2 years and to inform the management of their disease.

A two-stage prediction model for heterogeneous effects of treatments.

Treatment effects vary across different patients, and estimation of this variability is essential for clinical decision-making. We aimed to develop a model estimating the benefit of alternative treatment options for individual patients, extending a risk modeling approach in a network meta-analysis framework. We propose a two-stage prediction model for heterogeneous treatment effects by combining prognosis research and network meta-analysis methods where individual patient data are available. In the first stage, a prognostic model to predict the baseline risk of the outcome. In the second stage, we use the baseline risk score from the first stage as a single prognostic factor and effect modifier in a network meta-regression model. We apply the approach to a network meta-analysis of three randomized clinical trials comparing the relapses in Natalizumab, Glatiramer Acetate, and Dimethyl Fumarate, including 3590 patients diagnosed with relapsing-remitting multiple sclerosis. We find that the baseline risk score modifies the relative and absolute treatment effects. Several patient characteristics, such as age and disability status, impact the baseline risk of relapse, which in turn moderates the benefit expected for each of the treatments. For high-risk patients, the treatment that minimizes the risk of relapse in 2 years is Natalizumab, whereas Dimethyl Fumarate might be a better option for low-risk patients. Our approach can be easily extended to all outcomes of interest and has the potential to inform a personalized treatment approach.

Predicting the treatment response of certolizumab for individual adult patients with rheumatoid arthritis: protocol for an individual participant data meta-analysis.

BACKGROUND: A model that can predict treatment response for a patient with specific baseline characteristics would help decision-making in personalized medicine. The aim of the study is to develop such a model in the treatment of rheumatoid arthritis (RA) patients who receive certolizumab (CTZ) plus methotrexate (MTX) therapy, using individual participant data meta-analysis (IPD-MA). METHODS: We will search Cochrane CENTRAL, PubMed, and Scopus as well as clinical trial registries, drug regulatory agency reports, and the pharmaceutical company websites from their inception onwards to obtain randomized controlled trials (RCTs) investigating CTZ plus MTX compared with MTX alone in treating RA. We will request the individual-level data of these trials from an independent platform (http://vivli.org). The primary outcome is efficacy defined as achieving either remission (based on ACR-EULAR Boolean or index-based remission definition) or low disease activity (based on either of the validated composite disease activity measures). The secondary outcomes include ACR50 (50% improvement based on ACR core set variables) and adverse events. We will use a two-stage approach to develop the prediction model. First, we will construct a risk model for the outcomes via logistic regression to estimate the baseline risk scores. We will include baseline demographic, clinical, and biochemical features as covariates for this model. Next, we will develop a meta-regression model for treatment effects, in which the stage 1 risk score will be used both as a prognostic factor and as an effect modifier. We will calculate the probability of having the outcome for a new patient based on the model, which will allow estimation of the absolute and relative treatment effect. We will use R for our analyses, except for the second stage which will be performed in a Bayesian setting using R2Jags. DISCUSSION: This is a study protocol for developing a model to predict treatment response for RA patients receiving CTZ plus MTX in comparison with MTX alone, using a two-stage approach based on IPD-MA. The study will use a new modeling approach, which aims at retaining the statistical power. The model may help clinicians individualize treatment for particular patients. SYSTEMATIC REVIEW REGISTRATION: PROSPERO registration number pending (ID#157595).