Response to discussant comments on "NMA, the first 20 years".

We respond to discussant comments on our paper "Twenty years of network meta-analysis: Continuing controversies and recent developments" (https://doi.org/10.1002/jrsm.1700) and raise some additional points for consideration, including: the way in which methodological guidance is generated; integration of the estimand framework with evidence synthesis; and implications of the European Joint Clinical Assessment. We ask: what properties are required of population adjustment methods to enable transparent and consistent decision-making? We also ask why individual patient data is not routinely made available to re-imbursement authorities and clinical guideline developers.

Twenty years of network meta-analysis: Continuing controversies and recent developments.

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.