Shortest path or random walks? A framework for path weights in network meta-analysis.

Quantifying the contributions, or weights, of comparisons or single studies to the estimates in a network meta-analysis (NMA) is an active area of research. We extend this work to include the contributions of paths of evidence. We present a general framework, based on the path-design matrix, that describes the problem of finding path contributions as a linear equation. The resulting solutions may have negative coefficients. We show that two known approaches, called shortestpath and randomwalk, are special solutions of this equation, and both meet an optimization criterion, as they minimize the sum of absolute path contributions. In general, there is an infinite set of solutions, which can be identified using the generalized inverse (Moore-Penrose pseudoinverse). We consider two further special approaches. For large networks we find that shortestpath is superior with respect to run time and variability, compared to the other approaches, and is thus recommended in practice. The path-weights framework also has the potential to answer more general research questions in NMA.

Evaluating agreement between evidence from randomised controlled trials and cohort studies in nutrition: a meta-research replication study.

This meta-research study aims to evaluate the agreement of effect estimates between bodies of evidence (BoE) from RCTs and cohort studies included in the same nutrition evidence synthesis, to identify factors associated with disagreement, and to replicate the findings of a previous study. We searched Medline, Epistemonikos and the Cochrane Database of Systematic Reviews for nutrition systematic reviews that included both RCTs and cohort studies for the same patient-relevant outcome or intermediate-disease marker. We rated similarity of PI/ECO (population, intervention/exposure, comparison, outcome) between BoE from RCTs and cohort studies. Agreement of effect estimates across BoE was analysed by pooling ratio of risk ratios (RRR) for binary outcomes and difference of standardised mean differences (DSMD) for continuous outcomes. We performed subgroup and sensitivity analyses to explore determinants associated with disagreements. We included 82 BoE-pairs from 51 systematic reviews. For binary outcomes, the RRR was 1.04 (95% confidence interval (CI) 0.99 to 1.10, I2 = 59%, τ2 = 0.02, prediction interval (PI) 0.77 to 1.41). For continuous outcomes, the pooled DSMD was - 0.09 (95% CI - 0.26 to 0.09, PI - 0.55 to 0.38). Subgroup analyses yielded that differences in type of intake/exposure were drivers towards disagreement. We replicated the findings of a previous study, where on average RCTs and cohort studies had similar effect estimates. Disagreement and wide prediction intervals were mainly driven by PI/ECO-dissimilarities. More research is needed to explore other potentially influencing factors (e.g. risk of bias) on the disagreement between effect estimates of both BoE.Trial registration: CRD42021278908.

Introducing the Treatment Hierarchy Question in Network Meta-Analysis.

Comparative effectiveness research using network meta-analysis can present a hierarchy of competing treatments, from the most to the least preferable option. However, in published reviews, the research question associated with the hierarchy of multiple interventions is typically not clearly defined. Here we introduce the novel notion of a treatment hierarchy question that describes the criterion for choosing a specific treatment over one or more competing alternatives. For example, stakeholders might ask which treatment is most likely to improve mean survival by at least 2 years, or which treatment is associated with the longest mean survival. We discuss the most commonly used ranking metrics (quantities that compare the estimated treatment-specific effects), how the ranking metrics produce a treatment hierarchy, and the type of treatment hierarchy question that each ranking metric can answer. We show that the ranking metrics encompass the uncertainty in the estimation of the treatment effects in different ways, which results in different treatment hierarchies. When using network meta-analyses that aim to rank treatments, investigators should state the treatment hierarchy question they aim to address and employ the appropriate ranking metric to answer it. Following this new proposal will avoid some controversies that have arisen in comparative effectiveness research.

Network meta-analysis and random walks.

Network meta-analysis (NMA) is a central tool for evidence synthesis in clinical research. The results of an NMA depend critically on the quality of evidence being pooled. In assessing the validity of an NMA, it is therefore important to know the proportion contributions of each direct treatment comparison to each network treatment effect. The construction of proportion contributions is based on the observation that each row of the hat matrix represents a so-called "evidence flow network" for each treatment comparison. However, the existing algorithm used to calculate these values is associated with ambiguity according to the selection of paths. In this article, we present a novel analogy between NMA and random walks. We use this analogy to derive closed-form expressions for the proportion contributions. A random walk on a graph is a stochastic process that describes a succession of random "hops" between vertices which are connected by an edge. The weight of an edge relates to the probability that the walker moves along that edge. We use the graph representation of NMA to construct the transition matrix for a random walk on the network of evidence. We show that the net number of times a walker crosses each edge of the network is related to the evidence flow network. By then defining a random walk on the directed evidence flow network, we derive analytically the matrix of proportion contributions. The random-walk approach has none of the associated ambiguity of the existing algorithm.

Answering complex hierarchy questions in network meta-analysis.

BACKGROUND: Network meta-analysis estimates all relative effects between competing treatments and can produce a treatment hierarchy from the most to the least desirable option according to a health outcome. While about half of the published network meta-analyses present such a hierarchy, it is rarely the case that it is related to a clinically relevant decision question. METHODS: We first define treatment hierarchy and treatment ranking in a network meta-analysis and suggest a simulation method to estimate the probability of each possible hierarchy to occur. We then propose a stepwise approach to express clinically relevant decision questions as hierarchy questions and quantify the uncertainty of the criteria that constitute them. The steps of the approach are summarized as follows: a) a question of clinical relevance is defined, b) the hierarchies that satisfy the defined question are collected and c) the frequencies of the respective hierarchies are added; the resulted sum expresses the certainty of the defined set of criteria to hold. We then show how the frequencies of all possible hierarchies relate to common ranking metrics. RESULTS: We exemplify the method and its implementation using two networks. The first is a network of four treatments for chronic obstructive pulmonary disease where the most probable hierarchy has a frequency of 28%. The second is a network of 18 antidepressants, among which Vortioxetine, Bupropion and Escitalopram occupy the first three ranks with frequency 19%. CONCLUSIONS: The developed method offers a generalised approach of producing treatment hierarchies in network meta-analysis, which moves towards attaching treatment ranking to a clear decision question, relevant to all or a subset of competing treatments.

ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis.

BACKGROUND: Selective outcome reporting and publication bias threaten the validity of systematic reviews and meta-analyses and can affect clinical decision-making. A rigorous method to evaluate the impact of this bias on the results of network meta-analyses of interventions is lacking. We present a tool to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN). METHODS: ROB-MEN first evaluates the risk of bias due to missing evidence for each of the possible pairwise comparison that can be made between the interventions in the network. This step considers possible bias due to the presence of studies with unavailable results (within-study assessment of bias) and the potential for unpublished studies (across-study assessment of bias). The second step combines the judgements about the risk of bias due to missing evidence in pairwise comparisons with (i) the contribution of direct comparisons to the network meta-analysis estimates, (ii) possible small-study effects evaluated by network meta-regression, and (iii) any bias from unobserved comparisons. Then, a level of "low risk", "some concerns", or "high risk" for the bias due to missing evidence is assigned to each estimate, which is our tool's final output. RESULTS: We describe the methodology of ROB-MEN step-by-step using an illustrative example from a published NMA of non-diagnostic modalities for the detection of coronary artery disease in patients with low risk acute coronary syndrome. We also report a full application of the tool on a larger and more complex published network of 18 drugs from head-to-head studies for the acute treatment of adults with major depressive disorder. CONCLUSIONS: ROB-MEN is the first tool for evaluating the risk of bias due to missing evidence in network meta-analysis and applies to networks of all sizes and geometry. The use of ROB-MEN is facilitated by an R Shiny web application that produces the Pairwise Comparisons and ROB-MEN Table and is incorporated in the reporting bias domain of the CINeMA framework and software.

CINeMA: An approach for assessing confidence in the results of a network meta-analysis.

BACKGROUND: The evaluation of the credibility of results from a meta-analysis has become an important part of the evidence synthesis process. We present a methodological framework to evaluate confidence in the results from network meta-analyses, Confidence in Network Meta-Analysis (CINeMA), when multiple interventions are compared. METHODOLOGY: CINeMA considers 6 domains: (i) within-study bias, (ii) reporting bias, (iii) indirectness, (iv) imprecision, (v) heterogeneity, and (vi) incoherence. Key to judgments about within-study bias and indirectness is the percentage contribution matrix, which shows how much information each study contributes to the results from network meta-analysis. The contribution matrix can easily be computed using a freely available web application. In evaluating imprecision, heterogeneity, and incoherence, we consider the impact of these components of variability in forming clinical decisions. CONCLUSIONS: Via 3 examples, we show that CINeMA improves transparency and avoids the selective use of evidence when forming judgments, thus limiting subjectivity in the process. CINeMA is easy to apply even in large and complicated networks.

Comparative efficacy and tolerability of 32 oral antipsychotics for the acute treatment of adults with multi-episode schizophrenia: a systematic review and network meta-analysis.

BACKGROUND: Schizophrenia is one of the most common, burdensome, and costly psychiatric disorders in adults worldwide. Antipsychotic drugs are its treatment of choice, but there is controversy about which agent should be used. We aimed to compare and rank antipsychotics by quantifying information from randomised controlled trials. METHODS: We did a network meta-analysis of placebo-controlled and head-to-head randomised controlled trials and compared 32 antipsychotics. We searched Embase, MEDLINE, PsycINFO, PubMed, BIOSIS, Cochrane Central Register of Controlled Trials (CENTRAL), WHO International Clinical Trials Registry Platform, and ClinicalTrials.gov from database inception to Jan 8, 2019. Two authors independently selected studies and extracted data. We included randomised controlled trials in adults with acute symptoms of schizophrenia or related disorders. We excluded studies in patients with treatment resistance, first episode, predominant negative or depressive symptoms, concomitant medical illnesses, and relapse-prevention studies. Our primary outcome was change in overall symptoms measured with standardised rating scales. We also extracted data for eight efficacy and eight safety outcomes. Differences in the findings of the studies were explored in metaregressions and sensitivity analyses. Effect size measures were standardised mean differences, mean differences, or risk ratios with 95% credible intervals (CrIs). Confidence in the evidence was assessed using CINeMA (Confidence in Network Meta-Analysis). The study protocol is registered with PROSPERO, number CRD42014014919. FINDINGS: We identified 54 417 citations and included 402 studies with data for 53 463 participants. Effect size estimates suggested all antipsychotics reduced overall symptoms more than placebo (although not statistically significant for six drugs), with standardised mean differences ranging from -0·89 (95% CrI -1·08 to -0·71) for clozapine to -0·03 (-0·59 to 0·52) for levomepromazine (40 815 participants). Standardised mean differences compared with placebo for reduction of positive symptoms (31 179 participants) varied from -0·69 (95% CrI -0·86 to -0·52) for amisulpride to -0·17 (-0·31 to -0·04) for brexpiprazole, for negative symptoms (32 015 participants) from -0·62 (-0·84 to -0·39; clozapine) to -0·10 (-0·45 to 0·25; flupentixol), for depressive symptoms (19 683 participants) from -0·90 (-1·36 to -0·44; sulpiride) to 0·04 (-0·39 to 0·47; flupentixol). Risk ratios compared with placebo for all-cause discontinuation (42 672 participants) ranged from 0·52 (0·12 to 0·95; clopenthixol) to 1·15 (0·36 to 1·47; pimozide), for sedation (30 770 participants) from 0·92 (0·17 to 2·03; pimozide) to 10·20 (4·72 to 29·41; zuclopenthixol), for use of antiparkinson medication (24 911 participants) from 0·46 (0·19 to 0·88; clozapine) to 6·14 (4·81 to 6·55; pimozide). Mean differences compared to placebo for weight gain (28 317 participants) ranged from -0·16 kg (-0·73 to 0·40; ziprasidone) to 3·21 kg (2·10 to 4·31; zotepine), for prolactin elevation (21 569 participants) from -77·05 ng/mL (-120·23 to -33·54; clozapine) to 48·51 ng/mL (43·52 to 53·51; paliperidone) and for QTc prolongation (15 467 participants) from -2·21 ms (-4·54 to 0·15; lurasidone) to 23·90 ms (20·56 to 27·33; sertindole). Conclusions for the primary outcome did not substantially change after adjusting for possible effect moderators or in sensitivity analyses (eg, when excluding placebo-controlled studies). The confidence in evidence was often low or very low. INTERPRETATION: There are some efficacy differences between antipsychotics, but most of them are gradual rather than discrete. Differences in side-effects are more marked. These findings will aid clinicians in balancing risks versus benefits of those drugs available in their countries. They should consider the importance of each outcome, the patients' medical problems, and preferences. FUNDING: German Ministry of Education and Research and National Institute for Health Research.

The statistical importance of a study for a network meta-analysis estimate.

BACKGROUND: In pairwise meta-analysis, the contribution of each study to the pooled estimate is given by its weight, which is based on the inverse variance of the estimate from that study. For network meta-analysis (NMA), the contribution of direct (and indirect) evidence is easily obtained from the diagonal elements of a hat matrix. It is, however, not fully clear how to generalize this to the percentage contribution of each study to a NMA estimate. METHODS: We define the importance of each study for a NMA estimate by the reduction of the estimate's variance when adding the given study to the others. An equivalent interpretation is the relative loss in precision when the study is left out. Importances are values between 0 and 1. An importance of 1 means that the study is an essential link of the pathway in the network connecting one of the treatments with another. RESULTS: Importances can be defined for two-stage and one-stage NMA. These numbers in general do not add to one and thus cannot be interpreted as 'percentage contributions'. After briefly discussing other available approaches, we question whether it is possible to obtain unique percentage contributions for NMA. CONCLUSIONS: Importances generalize the concept of weights in pairwise meta-analysis in a natural way. Moreover, they are uniquely defined, easily calculated, and have an intuitive interpretation. We give some real examples for illustration.

Transcatheter aortic valve implantation vs. surgical aortic valve replacement for treatment of symptomatic severe aortic stenosis: an updated meta-analysis.

AIMS: Owing to new evidence from randomized controlled trials (RCTs) in low-risk patients with severe aortic stenosis, we compared the collective safety and efficacy of transcatheter aortic valve implantation (TAVI) vs. surgical aortic valve replacement (SAVR) across the entire spectrum of surgical risk patients. METHODS AND RESULTS: The meta-analysis is registered with PROSPERO (CRD42016037273). We identified RCTs comparing TAVI with SAVR in patients with severe aortic stenosis reporting at different follow-up periods. We extracted trial, patient, intervention, and outcome characteristics following predefined criteria. The primary outcome was all-cause mortality up to 2 years for the main analysis. Seven trials that randomly assigned 8020 participants to TAVI (4014 patients) and SAVR (4006 patients) were included. The combined mean STS score in the TAVI arm was 9.4%, 5.1%, and 2.0% for high-, intermediate-, and low surgical risk trials, respectively. Transcatheter aortic valve implantation was associated with a significant reduction of all-cause mortality compared to SAVR {hazard ratio [HR] 0.88 [95% confidence interval (CI) 0.78-0.99], P = 0.030}; an effect that was consistent across the entire spectrum of surgical risk (P-for-interaction = 0.410) and irrespective of type of transcatheter heart valve (THV) system (P-for-interaction = 0.674). Transcatheter aortic valve implantation resulted in lower risk of strokes [HR 0.81 (95% CI 0.68-0.98), P = 0.028]. Surgical aortic valve replacement was associated with a lower risk of major vascular complications [HR 1.99 (95% CI 1.34-2.93), P = 0.001] and permanent pacemaker implantations [HR 2.27 (95% CI 1.47-3.64), P < 0.001] compared to TAVI. CONCLUSION: Compared with SAVR, TAVI is associated with reduction in all-cause mortality and stroke up to 2 years irrespective of baseline surgical risk and type of THV system.

Extensions of the probabilistic ranking metrics of competing treatments in network meta-analysis to reflect clinically important relative differences on many outcomes.

One of the key features of network meta-analysis is ranking of interventions according to outcomes of interest. Ranking metrics are prone to misinterpretation because of two limitations associated with the current ranking methods. First, differences in relative treatment effects might not be clinically important and this is not reflected in the ranking metrics. Second, there are no established methods to include several health outcomes in the ranking assessments. To address these two issues, we extended the P-score method to allow for multiple outcomes and modified it to measure the mean extent of certainty that a treatment is better than the competing treatments by a certain amount, for example, the minimum clinical important difference. We suggest to present the tradeoff between beneficial and harmful outcomes allowing stakeholders to consider how much adverse effect they are willing to tolerate for specific gains in efficacy. We used a published network of 212 trials comparing 15 antipsychotics and placebo using a random effects network meta-analysis model, focusing on three outcomes; reduction in symptoms of schizophrenia in a standardized scale, all-cause discontinuation, and weight gain.

Comparative efficacy and acceptability of pharmacological treatments for post-traumatic stress disorder in adults: a network meta-analysis.

BACKGROUND: Guidelines about post-traumatic stress disorder (PTSD) recommend broad categories of drugs, but uncertainty remains about what pharmacological treatment to select among all available compounds. METHODS: Cochrane Central Register of Controlled Trials register, MEDLINE, PsycINFO, National PTSD Center Pilots database, PubMed, trial registries, and databases of pharmaceutical companies were searched until February 2016 for double-blind randomised trials comparing any pharmacological intervention or placebo as oral therapy in adults with PTSD. Initially, we performed standard pairwise meta-analyses using a random effects model. We then carried out a network meta-analysis. The main outcome measures were mean change on a standardised scale and all-cause dropout rate. Acute treatment was defined as 8-week follow up. RESULTS: Desipramine, fluoxetine, paroxetine, phenelzine, risperidone, sertraline, and venlafaxine were more effective than placebo; phenelzine was better than many other active treatments and was the only drug, which was significantly better than placebo in terms of dropouts (odds ratio 7.50, 95% CI 1.72-32.80). Mirtazapine yielded a relatively high rank for efficacy, but the respective value for acceptability was not among the best treatments. Divalproex had overall the worst ranking. CONCLUSIONS: The efficacy and acceptability hierarchies generated by our study were robust against many sources of bias. The differences between drugs and placebo were small, with the only exception of phenelzine. Considering the small amount of available data, these results are probably not robust enough to suggest phenelzine as a drug of choice. However, findings from this review reinforce the idea that phenelzine should be prioritised in future trials in PTSD.

Antipsychotic drugs for patients with schizophrenia and predominant or prominent negative symptoms: a systematic review and meta-analysis.

BACKGROUND: Negative symptoms are the core of schizophrenia, but whether antipsychotics are efficacious for their treatment is unclear. Moreover, there is debate whether patients in relevant trials should have predominant negative symptoms or whether prominent negative symptoms are also acceptable. METHODS: We systematically reviewed randomised, blinded antipsychotic drug trials in patients with schizophrenia and either predominant or prominent negative symptoms (last search Dec 12, 2017). Separate pairwise meta-analyses were conducted in these two populations. The primary outcome was negative symptoms. Depressive, symptoms, positive symptoms, and extrapyramidal side-effects were analysed as causes of secondary negative symptoms. FINDINGS: We included 21 randomized-controlled trials with 3451 participants which revealed the following significant differences in the primary outcome: in patients with predominant negative symptoms amisulpride was superior to placebo (N = 4; n = 590, SMD 0.47, CI 0.23, 0.71), olanzapine was superior to haloperidol in a small trial (n = 35) and cariprazine outperformed risperidone (N = 1, n = 456, SMD - 0.29, CI - 0.48, - 0.11). In patients with prominent negative symptoms, olanzapine and quetiapine were superior to risperidone in single trials. Overall, studies in prominent negative symptoms were potentially more confounded by improvements of secondary negative symptoms. INTERPRETATION: Amisulpride is the only antipsychotic that outperformed placebo in the treatment of predominant negative symptoms, but there was a parallel reduction of depression. Cariprazine was better than risperidone in a large trial that was well-controlled for secondary negative symptoms, but the trial was sponsored by its manufacturer. Future trials should apply scientifically developed definitions such as the deficit syndrome and the persistent negative symptoms concept.

Client feedback in psychological therapy for children and adolescents with mental health problems.

BACKGROUND: Childhood and adolescent mental health problems are a serious and growing concern worldwide. Research suggests that psychotherapy can have a significant and positive impact on children and adolescents with mental health problems, such as anxiety disorders, depression and conduct disorders. Client feedback tools serve as a method of monitoring clients' progress and providing feedback from clients to therapists during the therapeutic process. These tools may help to enhance clinicians' decision-making by allowing them to adapt their treatment plans as the therapy progresses, resulting in a reduction of treatment failures. Research has shown that client feedback tools have a positive effect on adults' psychotherapy. This review addresses whether feedback tools in child and adolescent therapy could help therapists to better treat their young clients. OBJECTIVES: To assess the effects of client feedback in psychological therapy on child and adolescent mental health outcomes. SEARCH METHODS: We searched the Cochrane Common Mental Disorders Controlled Trials Register (CCMDCTR, Studies and References), the Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE (1946-), Embase (1974-) and PsycINFO (1967-) to 3 April 2018. We did not apply any restriction on date, language or publication status to the search. SELECTION CRITERIA: We included randomised controlled trials (RCTs) that compared client feedback to no client feedback in psychological therapies for children and adolescents. DATA COLLECTION AND ANALYSIS: Two review authors independently assessed references for inclusion eligibility and extracted outcome, risk of bias and study characteristics data into customised forms. We contacted study authors to obtain missing data. We analysed dichotomous data using risk ratios (RRs) and calculated their 95% confidence intervals (CIs). For continuous data, we calculated mean differences (MDs), or standardised mean differences (SMDs) if different scales were used to measure the same outcome. We used a random-effects model for all analyses. MAIN RESULTS: We included six published RCTs, conducted in the USA (5 RCTs) and Israel (1 RCT), with 1097 children and adolescents (11 to 18 years old), in the review.We are very uncertain about the effect of client feedback on improvement of symptoms, as reported by youth in the short term because we considered evidence to be of very low-certainty due to high risk of bias and very serious inconsistency in the effect estimates from the different studies. Similarly, we are very uncertain about the effect of client feedback on treatment acceptability, due to high risk of bias, imprecision in the results, and indirectness of measuring the outcome (RR 1.08, 95% CI 0.73 to 1.61; 2 studies, 237 participants; very low-certainty).Overall, most studies reported and carried out randomisation and allocation concealment adequately. None of the studies were blinded or attempted to blind participants and personnel and were at high risk of performance bias, and only one study had blind outcome assessors. All of the studies were at high or unclear risk of attrition bias mainly due to poor, non-transparent reporting of participants' flow through the studies. AUTHORS' CONCLUSIONS: Due to the paucity of high-quality data and considerable inconsistency in results from different studies, there is currently insufficient evidence to reach any firm conclusions regarding the role of client feedback in psychological therapies for children and adolescents with mental health problems, and further research on this important topic is needed.Future studies should avoid risks of performance, detection and attrition biases, as seen in the studies included in this review. Studies from countries other than the USA are needed, as well as studies including children younger than 10 years.

Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review.

BACKGROUND: Network meta-analysis (NMA) has become a popular method to compare more than two treatments. This scoping review aimed to explore the characteristics and methodological quality of knowledge synthesis approaches underlying the NMA process. We also aimed to assess the statistical methods applied using the Analysis subdomain of the ISPOR checklist. METHODS: Comprehensive literature searches were conducted in MEDLINE, PubMed, EMBASE, and Cochrane Database of Systematic Reviews from inception until April 14, 2015. References of relevant reviews were scanned. Eligible studies compared at least four different interventions from randomised controlled trials with an appropriate NMA approach. Two reviewers independently performed study selection and data abstraction of included articles. All discrepancies between reviewers were resolved by a third reviewer. Data analysis involved quantitative (frequencies) and qualitative (content analysis) methods. Quality was evaluated using the AMSTAR tool for the conduct of knowledge synthesis and the ISPOR tool for statistical analysis. RESULTS: After screening 3538 citations and 877 full-text papers, 456 NMAs were included. These were published between 1997 and 2015, with 95% published after 2006. Most were conducted in Europe (51%) or North America (31%), and approximately one-third reported public sources of funding. Overall, 84% searched two or more electronic databases, 62% searched for grey literature, 58% performed duplicate study selection and data abstraction (independently), and 62% assessed risk of bias. Seventy-eight (17%) NMAs relied on previously conducted systematic reviews to obtain studies for inclusion in their NMA. Based on the AMSTAR tool, almost half of the NMAs incorporated quality appraisal results to formulate conclusions, 36% assessed publication bias, and 16% reported the source of funding. Based on the ISPOR tool, half of the NMAs did not report if an assessment for consistency was conducted or whether they accounted for inconsistency when present. Only 13% reported heterogeneity assumptions for the random-effects model. CONCLUSIONS: The knowledge synthesis methods and analytical process for NMAs are poorly reported and need improvement.

Planning future studies based on the precision of network meta-analysis results.

When there are multiple competing interventions for a healthcare problem, the design of new studies could be based on the entire network of evidence as reflected in a network meta-analysis. There is a practical need to answer how many (if any) studies are needed, of which design (i.e., which treatments to compare), and with what sample size to infer conclusively about the relative treatment effects of a set of target or all competing treatments and their relative ranking. We consider the precision in the results obtained from network meta-analysis: the precision of the joint distribution of the estimated basic parameters of the model and the precision in the treatment ranking. We quantify the precision in the estimated effects by considering their variance-covariance matrix and estimate the precision in ranking by quantifying the dissimilarity of the density functions of summary effect estimates. Then, based on a desirable improvement in precision, we calculate the required sample size for each possible study design and number of study arms, and we present visual tools that can help trialists select the optimal study design. We use a published network of interventions for the treatment of hepatocellular carcinoma to illustrate the suggested methodology. The presented methodology can aid investigators making informed and evidence-based decisions about planning new studies.

Using conditional power of network meta-analysis (NMA) to inform the design of future clinical trials.

Clinical trials are typically designed with an aim to reach sufficient power to test a hypothesis about relative effectiveness of two or more interventions. Their role in informing evidence-based decision-making demands, however, that they are considered in the context of the existing evidence. Consequently, their planning can be informed by characteristics of relevant systematic reviews and meta-analyses. In the presence of multiple competing interventions the evidence base has the form of a network of trials, which provides information not only about the required sample size but also about the interventions that should be compared in a future trial. In this paper we present a methodology to evaluate the impact of new studies, their information size, the comparisons involved, and the anticipated heterogeneity on the conditional power (CP) of the updated network meta-analysis. The methods presented are an extension of the idea of CP initially suggested for a pairwise meta-analysis and we show how to estimate the required sample size using various combinations of direct and indirect evidence in future trials. We apply the methods to two previously published networks and we show that CP for a treatment comparison is dependent on the magnitude of heterogeneity and the ratio of direct to indirect information in existing and future trials for that comparison. Our methodology can help investigators calculate the required sample size under different assumptions about heterogeneity and make decisions about the number and design of future studies (set of treatments compared).

To adjust or not to adjust in living systematic reviews? It's all about the context.

With each update of meta-analyses from living systematic reviews, treatment effects and their confidence intervals are recalculated. This often raises the question whether or not multiplicity is an issue and whether a method to adjust for multiplicity is needed. It seems that answering these questions is not that straightforward. We approach this matter by considering the context of systematic reviews and pointing out existing methods for handling multiplicity in meta-analysis. We conclude that multiplicity is not a relevant issue in living systematic reviews when they are planned with the aim to provide up-to-date evidence, without any direct control on the decision over future research. Multiplicity might be an issue, though, in living systematic reviews designed under a protocol involving a "stopping decision", which can be the case in living guideline development or in reimbursement decisions. Several appropriate methods exist for handling multiplicity in meta-analysis. Existing methods, however, are also associated with several technical and conceptual limitations, and could be improved in future methodological projects. To better decide whether an adjustment for multiplicity is necessary at all, authors and users of living systematic reviews should be aware of the context of the work and question whether there is a dependency between the effect estimates of the living systematic review and its stopping/updating or an influence on future research.

When does the placebo effect have an impact on network meta-analysis results?

The placebo effect is the 'effect of the simulation of treatment that occurs due to a participant's belief or expectation that a treatment is effective'. Although the effect might be of little importance for some conditions, it can have a great role in others, mostly when the evaluated symptoms are subjective. Several characteristics that include informed consent, number of arms in a study, the occurrence of adverse events and quality of blinding may influence response to placebo and possibly bias the results of randomised controlled trials. Such a bias is inherited in systematic reviews of evidence and their quantitative components, pairwise meta-analysis (when two treatments are compared) and network meta-analysis (when more than two treatments are compared). In this paper, we aim to provide red flags as to when a placebo effect is likely to bias pairwise and network meta-analysis treatment effects. The classic paradigm has been that placebo-controlled randomised trials are focused on estimating the treatment effect. However, the magnitude of placebo effect itself may also in some instances be of interest and has also lately received attention. We use component network meta-analysis to estimate placebo effects. We apply these methods to a published network meta-analysis, examining the relative effectiveness of four psychotherapies and four control treatments for depression in 123 studies.

Association between substituting macronutrients and all-cause mortality: a network meta-analysis of prospective observational studies.

Background: Suboptimal diet quality is a key risk factor for premature death. Assuming relatively stable energy intake among individuals, changes in nutrient intakes occur by exchanging different nutrients. Therefore we aimed to examine the association of isocaloric substitution of dietary (macro)nutrients with all-cause mortality using network meta-analysis (NMA). Methods: For this systematic review and NMA of prospective observational studies MEDLINE, Embase, and Scopus were searched from inception to February 13th, 2024. Eligible studies reported substitution analyses for quantity and/or quality of macronutrients, including carbohydrates, proteins, and fatty acids on all-cause mortality. Random-effects NMA were used in order to evaluate the pooled hazard ratios (HR) and 95% confidence intervals (CI) of substituting each included nutrient with another. We assessed risk of bias with the ROBINS-E tool, and the certainty of evidence (CoE) using the Grading of Recommendations Assessment, Development and Evaluations (GRADE) approach. This study is registered with PROSPERO (CRD42023450706). Findings: Thirty-nine studies with 1,737,644 participants, 395,491 deaths, 297 direct comparisons, and seven nutrient-specific networks were included. Moderate CoE was found for an association with lower mortality risk when replacing 5% of energy intake from carbohydrates with polyunsaturated fatty acids (PUFA; HR: 0.90; 95%CI: 0.84, 0.95), n-6 PUFA (0.85; 0.77, 0.94), n-3 PUFA (0.72; 0.59, 0.86), and plant monounsaturated fatty acids (MUFA; 0.90; 0.85, 0.95), and when replacing 5% of energy from saturated fatty acids (SFA) and trans-fatty acids (TFA), with PUFA, MUFA, and plant-MUFA (HRrange: 0.75 to 0.91). A lower mortality risk was additionally found when 5% of animal-MUFA was replaced with plant-MUFA, and when replacing animal protein, and SFA with plant protein (HRrange: 0.81 to 0.87, moderate CoE). Interpretation: Our results provide practical knowledge for public health professionals and can inform upcoming dietary guidelines. The beneficial association of increasing PUFA (both n-3 and n-6) and (plant-) MUFA intake while reducing carbohydrates, SFA and TFA, along with replacing animal protein and animal-MUFA with plant-based sources of protein and fat (MUFA) on the all-cause mortality risk, underscores the importance of plant-based dietary recommendations. Funding: None.