crossnma: An R package to synthesize cross-design evidence and cross-format data using network meta-analysis and network meta-regression.

BACKGROUND: Although aggregate data (AD) from randomised clinical trials (RCTs) are used in the majority of network meta-analyses (NMAs), other study designs (e.g., cohort studies and other non-randomised studies, NRS) can be informative about relative treatment effects. The individual participant data (IPD) of the study, when available, are preferred to AD for adjusting for important participant characteristics and to better handle heterogeneity and inconsistency in the network. RESULTS: We developed the R package crossnma to perform cross-format (IPD and AD) and cross-design (RCT and NRS) NMA and network meta-regression (NMR). The models are implemented as Bayesian three-level hierarchical models using Just Another Gibbs Sampler (JAGS) software within the R environment. The R package crossnma includes functions to automatically create the JAGS model, reformat the data (based on user input), assess convergence and summarize the results. We demonstrate the workflow within crossnma by using a network of six trials comparing four treatments. CONCLUSIONS: The R package crossnma enables the user to perform NMA and NMR with different data types in a Bayesian framework and facilitates the inclusion of all types of evidence recognising differences in risk of bias.

Treatment methods for cervical intraepithelial neoplasia in England: A cost-effectiveness analysis.

OBJECTIVE: To compare the cost-effectiveness of different treatments for cervical intraepithelial neoplasia (CIN). DESIGN: A cost-effectiveness analysis based on data available in the literature and expert opinion. SETTING: England. POPULATION: Women treated for CIN. METHODS: We developed a decision-analytic model to simulate the clinical course of 1000 women who received local treatment for CIN and were followed up for 10 years after treatment. In the model we considered surgical complications as well as oncological and reproductive outcomes over the 10-year period. The costs calculated were those incurred by the National Health Service (NHS) of England. MAIN OUTCOME MEASURES: Cost per one CIN2+ recurrence averted (oncological outcome); cost per one preterm birth averted (reproductive outcome); overall cost per one adverse oncological or reproductive outcome averted. RESULTS: For young women of reproductive age, large loop excision of the transformation zone (LLETZ) was the most cost-effective treatment overall at all willingness-to-pay thresholds. For postmenopausal women, LLETZ remained the most cost-effective treatment up to a threshold of £31,500, but laser conisation became the most cost-effective treatment above that threshold. CONCLUSIONS: LLETZ is the most cost-effective treatment for both younger and older women. However, for older women, more radical excision with laser conisation could also be considered if the NHS is willing to spend more than £31,500 to avert one CIN2+ recurrence.

Comparative efficacy and tolerability of 32 oral and long-acting injectable antipsychotics for the maintenance treatment of adults with schizophrenia: a systematic review and network meta-analysis.

BACKGROUND: Schizophrenia is a common, severe, and usually chronic disorder. Maintenance treatment with antipsychotic drugs can prevent relapse but also causes side-effects. We aimed to compare the efficacy and tolerability of antipsychotics as maintenance treatment for non-treatment resistant patients with schizophrenia. METHODS: In this systematic review and network meta-analysis, we searched, without language restrictions, the Cochrane Schizophrenia Group's specialised register between database inception and April 27, 2020, PubMed from April 1, 2020, to Jan 15, 2021, and the lists of included studies from related systematic reviews. We included randomised controlled trials (RCTs; ≥12 weeks of follow-up) that recruited adult participants with schizophrenia or schizoaffective disorder with stable symptoms who were treated with antipsychotics (monotherapy; oral or long-acting injectable) or placebo. We excluded RCTs of participants with specific comorbidities or treatment resistance. In duplicate, two authors independently selected eligible RCTs and extracted aggregate data. The primary outcome was the number of participants who relapsed and was analysed by random-effects, Bayesian network meta-analyses. The study was registered on PROSPERO, CRD42016049022. FINDINGS: We identified 4157 references through our search, from which 501 references on 127 RCTs of 32 antipsychotics (comprising 18 152 participants) were included. 100 studies including 16 812 participants and 30 antipsychotics contributed to our network meta-analysis of the primary outcome. All antipsychotics had risk ratios (RRs) less than 1·00 when compared with placebo for relapse prevention and almost all had 95% credible intervals (CrIs) excluding no effect. RRs ranged from 0·20 (95% CrI 0·05-0·41) for paliperidone oral to 0·65 (0·16-1·14) for cariprazine oral (moderate-to-low confidence in estimates). Generally, we interpret that there was no clear evidence for the superiority of specific antipsychotics in terms of relapse prevention because most comparisons between antipsychotics included a probability of no difference. INTERPRETATION: As we found no clear differences between antipsychotics for relapse prevention, we conclude that the choice of antipsychotic for maintenance treatment should be guided mainly by their tolerability. FUNDING: The German Ministry of Education and Research and Oxford Health Biomedical Research Centre.

Semi-automated assessment of the risk of bias due to missing evidence in network meta-analysis: a guidance paper for the ROB-MEN web-application.

Network meta-analysis compares multiple interventions and estimates the relative treatment effects between all interventions, combining both direct and indirect evidence. Recently, a framework was developed to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) which is part of the more comprehensive framework to evaluate the Confidence In the evidence for Network Meta-Analysis (CINeMA). To produce an overall risk of bias judgement for each network estimate, ROB-MEN: performs an assessment of the bias due to missing evidence in each possible pairwise comparison; combines the assessment with the contribution from the direct pairwise comparisons; considers the potential for small-study effects. To facilitate and semi-automate this process, ROB-MEN has been implemented in a user-friendly web-application ( https://cinema.ispm.unibe.ch/rob-men ). Here we provide a tutorial detailing the functionality and use of the application consisting of data upload, analysis configuration, output visualisation, and production of the tool's output tables for recording the risk of bias assessment. We also illustrate an example application using the demo dataset available for download on the application's homepage. The ROB-MEN web-application is open-source and freely available ( https://github.com/esm-ispm-unibe-ch/rob-men ).

Condom use and HIV testing among adults in Switzerland: repeated national cross-sectional surveys 2007, 2012, and 2017.

BACKGROUND: Monitoring of HIV and sexually transmitted infection (STI) prevention is important for guiding national sexual health programmes for both the general population and key populations. The objectives of this study were to examine trends and patterns of condom use at last intercourse and lifetime HIV testing in 2007, 2012 and 2017 in Switzerland, and to explore factors associated with these behaviours in men and women with opposite-sex partners and with same sex partners. METHODS: We analysed data from the 2007, 2012 and 2017 Swiss Health Survey. For each time point, outcome and population group, we conducted a descriptive analysis of weighted data and conducted multivariable logistic regression to obtain adjusted odds ratios (aOR) with 95% confidence intervals (CI) and compared outcomes between the timepoints. RESULTS: In total, 46,320 people were interviewed: 21,847 men and 23,141 women, who reported having sex only with partners of the opposite sex, 633 men who reported sex with a male partner and 699 women who reported sex with a female partner. Among the three surveys the prevalence of condom did not change but varied from 22 to 26% of men and 15 to 21% in women with only opposite-sex partners (aOR men, 0.93, 95% CI 0.82, 1.06; women 0.98, 95% CI 0.86 to 1.11). In men with any same sex partner the prevalence of condom use was 40% in 2007, 33% in 2012 and 54% in 2017 (aOR 1.80, 95% CI 0.97, 3.34). In multivariable analysis, the factor most strongly associated with condom use was sex with an occasional partner at last intercourse. HIV testing ever increased across all three survey years in people with opposite sex partners: 2017 vs. 2007, aOR men with only opposite-sex partners 1.64 (95% CI 1.49, 1.82), women with only opposite-sex partners 1.67 (1.51, 1.85), men with any same sex partner 0.98 (0.49, 1.96), women with any same sex partner 1.31 (0.74, 2.30). CONCLUSIONS: Monitoring of condom use, and HIV testing should continue and contribute to the development of the national sexual health programme. Stronger promotion of condoms for people with opposite-sex partners might be needed, since overall condom use at last intercourse has not changed since 2007.

The Impact of the COVID-19 Pandemic and Associated Control Measures on the Mental Health of the General Population : A Systematic Review and Dose-Response Meta-analysis.

BACKGROUND: To what extent the COVID-19 pandemic and its containment measures influenced mental health in the general population is still unclear. PURPOSE: To assess the trajectory of mental health symptoms during the first year of the pandemic and examine dose-response relations with characteristics of the pandemic and its containment. DATA SOURCES: Relevant articles were identified from the living evidence database of the COVID-19 Open Access Project, which indexes COVID-19-related publications from MEDLINE via PubMed, Embase via Ovid, and PsycInfo. Preprint publications were not considered. STUDY SELECTION: Longitudinal studies that reported data on the general population's mental health using validated scales and that were published before 31 March 2021 were eligible. DATA EXTRACTION: An international crowd of 109 trained reviewers screened references and extracted study characteristics, participant characteristics, and symptom scores at each timepoint. Data were also included for the following country-specific variables: days since the first case of SARS-CoV-2 infection, the stringency of governmental containment measures, and the cumulative numbers of cases and deaths. DATA SYNTHESIS: In a total of 43 studies (331 628 participants), changes in symptoms of psychological distress, sleep disturbances, and mental well-being varied substantially across studies. On average, depression and anxiety symptoms worsened in the first 2 months of the pandemic (standardized mean difference at 60 days, -0.39 [95% credible interval, -0.76 to -0.03]); thereafter, the trajectories were heterogeneous. There was a linear association of worsening depression and anxiety with increasing numbers of reported cases of SARS-CoV-2 infection and increasing stringency in governmental measures. Gender, age, country, deprivation, inequalities, risk of bias, and study design did not modify these associations. LIMITATIONS: The certainty of the evidence was low because of the high risk of bias in included studies and the large amount of heterogeneity. Stringency measures and surges in cases were strongly correlated and changed over time. The observed associations should not be interpreted as causal relationships. CONCLUSION: Although an initial increase in average symptoms of depression and anxiety and an association between higher numbers of reported cases and more stringent measures were found, changes in mental health symptoms varied substantially across studies after the first 2 months of the pandemic. This suggests that different populations responded differently to the psychological stress generated by the pandemic and its containment measures. PRIMARY FUNDING SOURCE: Swiss National Science Foundation. (PROSPERO: CRD42020180049).

Comparative effectiveness and risk of preterm birth of local treatments for cervical intraepithelial neoplasia and stage IA1 cervical cancer: a systematic review and network meta-analysis.

BACKGROUND: The trade-off between comparative effectiveness and reproductive morbidity of different treatment methods for cervical intraepithelial neoplasia (CIN) remains unclear. We aimed to determine the risks of treatment failure and preterm birth associated with various treatment techniques. METHODS: In this systematic review and network meta-analysis, we searched MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials database for randomised and non-randomised studies reporting on oncological or reproductive outcomes after CIN treatments from database inception until March 9, 2022, without language restrictions. We included studies of women with CIN, glandular intraepithelial neoplasia, or stage IA1 cervical cancer treated with excision (cold knife conisation [CKC], laser conisation, and large loop excision of the transformation zone [LLETZ]) or ablation (radical diathermy, laser ablation, cold coagulation, and cryotherapy). We excluded women treated with hysterectomy. The primary outcomes were any treatment failure (defined as any abnormal histology or cytology) and preterm birth (<37 weeks of gestation). The network for preterm birth also included women with untreated CIN (untreated colposcopy group). The main reference group was LLETZ for treatment failure and the untreated colposcopy group for preterm birth. For randomised controlled trials, we extracted group-level summary data, and for observational studies, we extracted relative treatment effect estimates adjusted for potential confounders, when available, and we did random-effects network meta-analyses to obtain odds ratios (ORs) with 95% CIs. We assessed within-study and across-study risk of bias using Cochrane tools. This systematic review is registered with PROSPERO, CRD42018115495 and CRD42018115508. FINDINGS: 7880 potential citations were identified for the outcome of treatment failure and 4107 for the outcome of preterm birth. After screening and removal of duplicates, the network for treatment failure included 19 240 participants across 71 studies (25 randomised) and the network for preterm birth included 68 817 participants across 29 studies (two randomised). Compared with LLETZ, risk of treatment failure was reduced for other excisional methods (laser conisation: OR 0·59 [95% CI 0·44-0·79] and CKC: 0·63 [0·50-0·81]) and increased for laser ablation (1·69 [1·27-2·24]) and cryotherapy (1·84 [1·33-2·56]). No differences were found for the comparison of cold coagulation versus LLETZ (1·09 [0·68-1·74]) but direct data were based on two small studies only. Compared with the untreated colposcopy group, risk of preterm birth was increased for all excisional techniques (CKC: 2·27 [1·70-3·02]; laser conisation: 1·77 [1·29-2·43]; and LLETZ: 1·37 [1·16-1·62]), whereas no differences were found for ablative methods (laser ablation: 1·05 [0·78-1·41]; cryotherapy: 1·01 [0·35-2·92]; and cold coagulation: 0·67 [0·02-29·15]). The evidence was based mostly on observational studies with their inherent risks of bias, and the credibility of many comparisons was low. INTERPRETATION: More radical excisional techniques reduce the risk of treatment failure but increase the risk of subsequent preterm birth. Although there is uncertainty, ablative treatments probably do not increase risk of preterm birth, but are associated with higher failure rates than excisional techniques. Although we found LLETZ to have balanced effectiveness and reproductive morbidity, treatment choice should rely on a woman's age, size and location of lesion, and future family planning. FUNDING: National Institute for Health and Care Research: Research for Patient Benefit.

Selective publication of antidepressant trials and its influence on apparent efficacy: Updated comparisons and meta-analyses of newer versus older trials.

BACKGROUND: Valid assessment of drug efficacy and safety requires an evidence base free of reporting bias. Using trial reports in Food and Drug Administration (FDA) drug approval packages as a gold standard, we previously found that the published literature inflated the apparent efficacy of antidepressant drugs. The objective of the current study was to determine whether this has improved with recently approved drugs. METHODS AND FINDINGS: Using medical and statistical reviews in FDA drug approval packages, we identified 30 Phase II/III double-blind placebo-controlled acute monotherapy trials, involving 13,747 patients, of desvenlafaxine, vilazodone, levomilnacipran, and vortioxetine; we then identified corresponding published reports. We compared the data from this newer cohort of antidepressants (approved February 2008 to September 2013) with the previously published dataset on 74 trials of 12 older antidepressants (approved December 1987 to August 2002). Using logistic regression, we examined the effects of trial outcome and trial cohort (newer versus older) on transparent reporting (whether published and FDA conclusions agreed). Among newer antidepressants, transparent publication occurred more with positive (15/15 = 100%) than negative (7/15 = 47%) trials (OR 35.1, CI95% 1.8 to 693). Controlling for trial outcome, transparent publication occurred more with newer than older trials (OR 6.6, CI95% 1.6 to 26.4). Within negative trials, transparent reporting increased from 11% to 47%. We also conducted and contrasted FDA- and journal-based meta-analyses. For newer antidepressants, FDA-based effect size (ESFDA) was 0.24 (CI95% 0.18 to 0.30), while journal-based effect size (ESJournals) was 0.29 (CI95% 0.23 to 0.36). Thus, effect size inflation, presumably due to reporting bias, was 0.05, less than for older antidepressants (0.10). Limitations of this study include a small number of trials and drugs-belonging to a single class-and a focus on efficacy (versus safety). CONCLUSIONS: Reporting bias persists but appears to have diminished for newer, compared to older, antidepressants. Continued efforts are needed to further improve transparency in the scientific literature.

Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: Update of a living systematic review and meta-analysis.

BACKGROUND: Debate about the level of asymptomatic Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection continues. The amount of evidence is increasing and study designs have changed over time. We updated a living systematic review to address 3 questions: (1) Among people who become infected with SARS-CoV-2, what proportion does not experience symptoms at all during their infection? (2) What is the infectiousness of asymptomatic and presymptomatic, compared with symptomatic, SARS-CoV-2 infection? (3) What proportion of SARS-CoV-2 transmission in a population is accounted for by people who are asymptomatic or presymptomatic? METHODS AND FINDINGS: The protocol was first published on 1 April 2020 and last updated on 18 June 2021. We searched PubMed, Embase, bioRxiv, and medRxiv, aggregated in a database of SARS-CoV-2 literature, most recently on 6 July 2021. Studies of people with PCR-diagnosed SARS-CoV-2, which documented symptom status at the beginning and end of follow-up, or mathematical modelling studies were included. Studies restricted to people already diagnosed, of single individuals or families, or without sufficient follow-up were excluded. One reviewer extracted data and a second verified the extraction, with disagreement resolved by discussion or a third reviewer. Risk of bias in empirical studies was assessed with a bespoke checklist and modelling studies with a published checklist. All data syntheses were done using random effects models. Review question (1): We included 130 studies. Heterogeneity was high so we did not estimate a mean proportion of asymptomatic infections overall (interquartile range (IQR) 14% to 50%, prediction interval 2% to 90%), or in 84 studies based on screening of defined populations (IQR 20% to 65%, prediction interval 4% to 94%). In 46 studies based on contact or outbreak investigations, the summary proportion asymptomatic was 19% (95% confidence interval (CI) 15% to 25%, prediction interval 2% to 70%). (2) The secondary attack rate in contacts of people with asymptomatic infection compared with symptomatic infection was 0.32 (95% CI 0.16 to 0.64, prediction interval 0.11 to 0.95, 8 studies). (3) In 13 modelling studies fit to data, the proportion of all SARS-CoV-2 transmission from presymptomatic individuals was higher than from asymptomatic individuals. Limitations of the evidence include high heterogeneity and high risks of selection and information bias in studies that were not designed to measure persistently asymptomatic infection, and limited information about variants of concern or in people who have been vaccinated. CONCLUSIONS: Based on studies published up to July 2021, most SARS-CoV-2 infections were not persistently asymptomatic, and asymptomatic infections were less infectious than symptomatic infections. Summary estimates from meta-analysis may be misleading when variability between studies is extreme and prediction intervals should be presented. Future studies should determine the asymptomatic proportion of SARS-CoV-2 infections caused by variants of concern and in people with immunity following vaccination or previous infection. Without prospective longitudinal studies with methods that minimise selection and measurement biases, further updates with the study types included in this living systematic review are unlikely to be able to provide a reliable summary estimate of the proportion of asymptomatic infections caused by SARS-CoV-2. REVIEW PROTOCOL: Open Science Framework (https://osf.io/9ewys/).

Introducing the Treatment Hierarchy Question in Network Meta-Analysis.

Comparative effectiveness research using network meta-analysis can present a hierarchy of competing treatments, from the most to the least preferable option. However, in published reviews, the research question associated with the hierarchy of multiple interventions is typically not clearly defined. Here we introduce the novel notion of a treatment hierarchy question that describes the criterion for choosing a specific treatment over one or more competing alternatives. For example, stakeholders might ask which treatment is most likely to improve mean survival by at least 2 years, or which treatment is associated with the longest mean survival. We discuss the most commonly used ranking metrics (quantities that compare the estimated treatment-specific effects), how the ranking metrics produce a treatment hierarchy, and the type of treatment hierarchy question that each ranking metric can answer. We show that the ranking metrics encompass the uncertainty in the estimation of the treatment effects in different ways, which results in different treatment hierarchies. When using network meta-analyses that aim to rank treatments, investigators should state the treatment hierarchy question they aim to address and employ the appropriate ranking metric to answer it. Following this new proposal will avoid some controversies that have arisen in comparative effectiveness research.

Optimal dose of aripiprazole for augmentation therapy of antidepressant-refractory depression: preliminary findings based on a systematic review and dose-effect meta-analysis.

BACKGROUND: Aripiprazole augmentation is proven effective for antidepressant-refractory depression, but its licensed dose range is wide and optimal dosage remains unclear. AIMS: To find the optimal dosage of aripiprazole augmentation. METHOD: Multiple electronic databases were searched (from inception to 16 February 2021) to identify all assessor-masked randomised controlled trials evaluating aripiprazole augmentation therapy in adults (≥18 years old, both genders) with major depressive disorder showing inadequate response to at least one antidepressant treatment. A random-effects, one-stage dose-effect meta-analysis with restricted cubic splines was conducted. Outcomes were efficacy (treatment response: ≥50% reduction in depression severity), tolerability (drop-out due to adverse effects) and acceptability (drop-out for any reason) after 8 weeks of treatment (range 4-12 weeks). RESULTS: Ten studies met the inclusion criteria. All were individually randomised, placebo-controlled, multi-centre, parallel studies including 2625 participants in total. The maximum target dose-efficacy curve showed an increase up to doses between 2 mg (odds ratio OR = 1.46, 95% CI 1.15-1.85) and 5 mg (OR = 1.93, 95% CI 1.33-2.81), and then a non-increasing trend through the higher licensed doses up to 20 mg (OR = 1.90, 95% CI 1.52-2.37). Tolerability showed a similar trend with greater uncertainty. Acceptability showed no significant difference through the examined dose range. Certainty of evidence was low to moderate. CONCLUSIONS: Low-dose aripiprazole as augmentation treatment might achieve the optimal balance between efficacy, tolerability and acceptability in the acute treatment of antidepressant-refractory depression. However, the small number of included studies and the overall moderate to high risk of bias seriously compromise the reliability of the results. Further research is required to investigate the benefits of low versus high dose.

Answering complex hierarchy questions in network meta-analysis.

BACKGROUND: Network meta-analysis estimates all relative effects between competing treatments and can produce a treatment hierarchy from the most to the least desirable option according to a health outcome. While about half of the published network meta-analyses present such a hierarchy, it is rarely the case that it is related to a clinically relevant decision question. METHODS: We first define treatment hierarchy and treatment ranking in a network meta-analysis and suggest a simulation method to estimate the probability of each possible hierarchy to occur. We then propose a stepwise approach to express clinically relevant decision questions as hierarchy questions and quantify the uncertainty of the criteria that constitute them. The steps of the approach are summarized as follows: a) a question of clinical relevance is defined, b) the hierarchies that satisfy the defined question are collected and c) the frequencies of the respective hierarchies are added; the resulted sum expresses the certainty of the defined set of criteria to hold. We then show how the frequencies of all possible hierarchies relate to common ranking metrics. RESULTS: We exemplify the method and its implementation using two networks. The first is a network of four treatments for chronic obstructive pulmonary disease where the most probable hierarchy has a frequency of 28%. The second is a network of 18 antidepressants, among which Vortioxetine, Bupropion and Escitalopram occupy the first three ranks with frequency 19%. CONCLUSIONS: The developed method offers a generalised approach of producing treatment hierarchies in network meta-analysis, which moves towards attaching treatment ranking to a clear decision question, relevant to all or a subset of competing treatments.

Development and validation of a meta-learner for combining statistical and machine learning prediction models in individuals with depression.

BACKGROUND: The debate of whether machine learning models offer advantages over standard statistical methods when making predictions is ongoing. We discuss the use of a meta-learner model combining both approaches as an alternative. METHODS: To illustrate the development of a meta-learner, we used a dataset of 187,757 people with depression. Using 31 variables, we aimed to predict two outcomes measured 60 days after initiation of antidepressant treatment: severity of depressive symptoms (continuous) and all-cause dropouts (binary). We fitted a ridge regression and a multi-layer perceptron (MLP) deep neural network as two separate prediction models ("base-learners"). We then developed two "meta-learners", combining predictions from the two base-learners. To compare the performance across the different methods, we calculated mean absolute error (MAE, for continuous outcome) and the area under the receiver operating characteristic curve (AUC, for binary outcome) using bootstrapping. RESULTS: Compared to the best performing base-learner (MLP base-learner, MAE at 4.63, AUC at 0.59), the best performing meta-learner showed a 2.49% decrease in MAE at 4.52 for the continuous outcome and a 6.47% increase in AUC at 0.60 for the binary outcome. CONCLUSIONS: A meta-learner approach may effectively combine multiple prediction models. Choosing between statistical and machine learning models may not be necessary in practice.

Generating comparative evidence on new drugs and devices before approval.

Fewer than half of new drugs have data on their comparative benefits and harms against existing treatment options at the time of regulatory approval in Europe and the USA. Even when active-comparator trials exist, they might not produce meaningful data to inform decisions in clinical practice and health policy. The uncertainty associated with the paucity of well designed active-comparator trials has been compounded by legal and regulatory changes in Europe and the USA that have created a complex mix of expedited programmes aimed at facilitating faster access to new drugs. Comparative evidence generation is even sparser for medical devices. Some have argued that the current process for regulatory approval needs to generate more evidence that is useful for patients, clinicians, and payers in health-care systems. We propose a set of five key principles relevant to the European Medicines Agency, European medical device regulatory agencies, US Food and Drug Administration, as well as payers, that we believe will provide the necessary incentives for pharmaceutical and device companies to generate comparative data on drugs and devices and assure timely availability of evidence that is useful for decision making. First, labelling should routinely inform patients and clinicians whether comparative data exist on new products. Second, regulators should be more selective in their use of programmes that facilitate drug and device approvals on the basis of incomplete benefit and harm data. Third, regulators should encourage the conduct of randomised trials with active comparators. Fourth, regulators should use prospectively designed network meta-analyses based on existing and future randomised trials. Last, payers should use their policy levers and negotiating power to incentivise the generation of comparative evidence on new and existing drugs and devices, for example, by explicitly considering proven added benefit in pricing and payment decisions.

ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis.

BACKGROUND: Selective outcome reporting and publication bias threaten the validity of systematic reviews and meta-analyses and can affect clinical decision-making. A rigorous method to evaluate the impact of this bias on the results of network meta-analyses of interventions is lacking. We present a tool to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN). METHODS: ROB-MEN first evaluates the risk of bias due to missing evidence for each of the possible pairwise comparison that can be made between the interventions in the network. This step considers possible bias due to the presence of studies with unavailable results (within-study assessment of bias) and the potential for unpublished studies (across-study assessment of bias). The second step combines the judgements about the risk of bias due to missing evidence in pairwise comparisons with (i) the contribution of direct comparisons to the network meta-analysis estimates, (ii) possible small-study effects evaluated by network meta-regression, and (iii) any bias from unobserved comparisons. Then, a level of "low risk", "some concerns", or "high risk" for the bias due to missing evidence is assigned to each estimate, which is our tool's final output. RESULTS: We describe the methodology of ROB-MEN step-by-step using an illustrative example from a published NMA of non-diagnostic modalities for the detection of coronary artery disease in patients with low risk acute coronary syndrome. We also report a full application of the tool on a larger and more complex published network of 18 drugs from head-to-head studies for the acute treatment of adults with major depressive disorder. CONCLUSIONS: ROB-MEN is the first tool for evaluating the risk of bias due to missing evidence in network meta-analysis and applies to networks of all sizes and geometry. The use of ROB-MEN is facilitated by an R Shiny web application that produces the Pairwise Comparisons and ROB-MEN Table and is incorporated in the reporting bias domain of the CINeMA framework and software.

A forward search algorithm for detecting extreme study effects in network meta-analysis.

In a quantitative synthesis of studies via meta-analysis, it is possible that some studies provide a markedly different relative treatment effect or have a large impact on the summary estimate and/or heterogeneity. Extreme study effects (outliers) can be detected visually with forest/funnel plots and by using statistical outlying detection methods. A forward search (FS) algorithm is a common outlying diagnostic tool recently extended to meta-analysis. FS starts by fitting the assumed model to a subset of the data which is gradually incremented by adding the remaining studies according to their closeness to the postulated data-generating model. At each step of the algorithm, parameter estimates, measures of fit (residuals, likelihood contributions), and test statistics are being monitored and their sharp changes are used as an indication for outliers. In this article, we extend the FS algorithm to network meta-analysis (NMA). In NMA, visualization of outliers is more challenging due to the multivariate nature of the data and the fact that studies contribute both directly and indirectly to the network estimates. Outliers are expected to contribute not only to heterogeneity but also to inconsistency, compromising the NMA results. The FS algorithm was applied to real and artificial networks of interventions that include outliers. We developed an R package (NMAoutlier) to allow replication and dissemination of the proposed method. We conclude that the FS algorithm is a visual diagnostic tool that helps to identify studies that are a potential source of heterogeneity and inconsistency.

A two-stage prediction model for heterogeneous effects of treatments.

Treatment effects vary across different patients, and estimation of this variability is essential for clinical decision-making. We aimed to develop a model estimating the benefit of alternative treatment options for individual patients, extending a risk modeling approach in a network meta-analysis framework. We propose a two-stage prediction model for heterogeneous treatment effects by combining prognosis research and network meta-analysis methods where individual patient data are available. In the first stage, a prognostic model to predict the baseline risk of the outcome. In the second stage, we use the baseline risk score from the first stage as a single prognostic factor and effect modifier in a network meta-regression model. We apply the approach to a network meta-analysis of three randomized clinical trials comparing the relapses in Natalizumab, Glatiramer Acetate, and Dimethyl Fumarate, including 3590 patients diagnosed with relapsing-remitting multiple sclerosis. We find that the baseline risk score modifies the relative and absolute treatment effects. Several patient characteristics, such as age and disability status, impact the baseline risk of relapse, which in turn moderates the benefit expected for each of the treatments. For high-risk patients, the treatment that minimizes the risk of relapse in 2 years is Natalizumab, whereas Dimethyl Fumarate might be a better option for low-risk patients. Our approach can be easily extended to all outcomes of interest and has the potential to inform a personalized treatment approach.

Prevalence of evidence of inconsistency and its association with network structural characteristics in 201 published networks of interventions.

BACKGROUND: Network meta-analysis (NMA) has attracted growing interest in evidence-based medicine. Consistency between different sources of evidence is fundamental to the reliability of the NMA results. The purpose of the present study was to estimate the prevalence of evidence of inconsistency and describe its association with different NMA characteristics. METHODS: We updated our collection of NMAs with articles published up to July 2018. We included networks with randomised clinical trials, at least four treatment nodes, at least one closed loop, a dichotomous primary outcome, and available arm-level data. We assessed consistency using the design-by-treatment interaction (DBT) model and testing all the inconsistency parameters globally through the Wald-type chi-squared test statistic. We estimated the prevalence of evidence of inconsistency and its association with different network characteristics (e.g., number of studies, interventions, intervention comparisons, loops). We evaluated the influence of the network characteristics on the DBT p-value via a multivariable regression analysis and the estimated Pearson correlation coefficients. We also evaluated heterogeneity in NMA (consistency) and DBT (inconsistency) random-effects models. RESULTS: We included 201 published NMAs. The p-value of the design-by-treatment interaction (DBT) model was lower than 0.05 in 14% of the networks and lower than 0.10 in 20% of the networks. Networks including many studies and comparing few interventions were more likely to have small DBT p-values (less than 0.10), which is probably because they yielded more precise estimates and power to detect differences between designs was higher. In the presence of inconsistency (DBT p-value lower than 0.10), the consistency model displayed higher heterogeneity than the DBT model. CONCLUSIONS: Our findings show that inconsistency was more frequent than what would be expected by chance, suggesting that researchers should devote more resources to exploring how to mitigate inconsistency. The results of this study highlight the need to develop strategies to detect inconsistency (because of the relatively high prevalence of evidence of inconsistency in published networks), and particularly in cases where the existing tests have low power.

Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: A living systematic review and meta-analysis.

BACKGROUND: There is disagreement about the level of asymptomatic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. We conducted a living systematic review and meta-analysis to address three questions: (1) Amongst people who become infected with SARS-CoV-2, what proportion does not experience symptoms at all during their infection? (2) Amongst people with SARS-CoV-2 infection who are asymptomatic when diagnosed, what proportion will develop symptoms later? (3) What proportion of SARS-CoV-2 transmission is accounted for by people who are either asymptomatic throughout infection or presymptomatic? METHODS AND FINDINGS: We searched PubMed, Embase, bioRxiv, and medRxiv using a database of SARS-CoV-2 literature that is updated daily, on 25 March 2020, 20 April 2020, and 10 June 2020. Studies of people with SARS-CoV-2 diagnosed by reverse transcriptase PCR (RT-PCR) that documented follow-up and symptom status at the beginning and end of follow-up or modelling studies were included. One reviewer extracted data and a second verified the extraction, with disagreement resolved by discussion or a third reviewer. Risk of bias in empirical studies was assessed with an adapted checklist for case series, and the relevance and credibility of modelling studies were assessed using a published checklist. We included a total of 94 studies. The overall estimate of the proportion of people who become infected with SARS-CoV-2 and remain asymptomatic throughout infection was 20% (95% confidence interval [CI] 17-25) with a prediction interval of 3%-67% in 79 studies that addressed this review question. There was some evidence that biases in the selection of participants influence the estimate. In seven studies of defined populations screened for SARS-CoV-2 and then followed, 31% (95% CI 26%-37%, prediction interval 24%-38%) remained asymptomatic. The proportion of people that is presymptomatic could not be summarised, owing to heterogeneity. The secondary attack rate was lower in contacts of people with asymptomatic infection than those with symptomatic infection (relative risk 0.35, 95% CI 0.10-1.27). Modelling studies fit to data found a higher proportion of all SARS-CoV-2 infections resulting from transmission from presymptomatic individuals than from asymptomatic individuals. Limitations of the review include that most included studies were not designed to estimate the proportion of asymptomatic SARS-CoV-2 infections and were at risk of selection biases; we did not consider the possible impact of false negative RT-PCR results, which would underestimate the proportion of asymptomatic infections; and the database does not include all sources. CONCLUSIONS: The findings of this living systematic review suggest that most people who become infected with SARS-CoV-2 will not remain asymptomatic throughout the course of the infection. The contribution of presymptomatic and asymptomatic infections to overall SARS-CoV-2 transmission means that combination prevention measures, with enhanced hand hygiene, masks, testing tracing, and isolation strategies and social distancing, will continue to be needed.

CINeMA: An approach for assessing confidence in the results of a network meta-analysis.

BACKGROUND: The evaluation of the credibility of results from a meta-analysis has become an important part of the evidence synthesis process. We present a methodological framework to evaluate confidence in the results from network meta-analyses, Confidence in Network Meta-Analysis (CINeMA), when multiple interventions are compared. METHODOLOGY: CINeMA considers 6 domains: (i) within-study bias, (ii) reporting bias, (iii) indirectness, (iv) imprecision, (v) heterogeneity, and (vi) incoherence. Key to judgments about within-study bias and indirectness is the percentage contribution matrix, which shows how much information each study contributes to the results from network meta-analysis. The contribution matrix can easily be computed using a freely available web application. In evaluating imprecision, heterogeneity, and incoherence, we consider the impact of these components of variability in forming clinical decisions. CONCLUSIONS: Via 3 examples, we show that CINeMA improves transparency and avoids the selective use of evidence when forming judgments, thus limiting subjectivity in the process. CINeMA is easy to apply even in large and complicated networks.