Twenty years of network meta-analysis: Continuing controversies and recent developments.

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

Spontaneous Clearance of Vertically Acquired Hepatitis C Infection: Implications for Testing and Treatment.

BACKGROUND: Current guidelines recommend that infants born to women with hepatitis C virus (HCV) viremia be screened for HCV antibody at age 18 months and, if positive, referred for RNA testing at 3 years to confirm chronic infection. This policy is based, in part, on analyses that suggest that 25%-40% of vertically acquired HCV infections clear spontaneously within 4-5 years. METHODS: Data on 179 infants with HCV RNA and/or anti-HCV evidence of vertically acquired infection in 3 prospective European cohorts were investigated. Ages at clearance of infection were estimated taking account of interval censoring and delayed entry. We also investigated clearance in initially HCV RNA-negative infants in whom RNA was not detectable until after 6 weeks. RESULTS: Clearance rates were initially high then declined slowly. Apparently, many infections clear before they can be confirmed. An estimated 65.9% (95% credible interval [CrI], 50.1-81.6) of confirmed infections cleared by 5 years, at a median 12.4 (CrI, 7.1-18.9) months. If treatment were to begin at age 6 months, 18 months, or 3 years, at least 59.0% (CrI, 42.0-76.9), 39.7% (CrI, 17.9-65.9), and 20.9% (CrI, 4.6-44.8) of those treated would clear without treatment. In 7 (6.6%) confirmed infections, RNA was not detectable until after 6 weeks and not until after 6 months in 2 (1.9%). However, all such cases subsequently cleared. CONCLUSIONS: Most confirmed infection cleared by age 3 years. Treatment before age 3, if it was available, would avoid loss to follow-up but would result in substantial overtreatment.

Validating the Assumptions of Population Adjustment: Application of Multilevel Network Meta-regression to a Network of Treatments for Plaque Psoriasis.

BACKGROUND: Network meta-analysis (NMA) and indirect comparisons combine aggregate data (AgD) from multiple studies on treatments of interest but may give biased estimates if study populations differ. Population adjustment methods such as multilevel network meta-regression (ML-NMR) aim to reduce bias by adjusting for differences in study populations using individual patient data (IPD) from 1 or more studies under the conditional constancy assumption. A shared effect modifier assumption may also be necessary for identifiability. This article aims to demonstrate how the assumptions made by ML-NMR can be assessed in practice to obtain reliable treatment effect estimates in a target population. METHODS: We apply ML-NMR to a network of evidence on treatments for plaque psoriasis with a mix of IPD and AgD trials reporting ordered categorical outcomes. Relative treatment effects are estimated for each trial population and for 3 external target populations represented by a registry and 2 cohort studies. We examine residual heterogeneity and inconsistency and relax the shared effect modifier assumption for each covariate in turn. RESULTS: Estimated population-average treatment effects were similar across study populations, as differences in the distributions of effect modifiers were small. Better fit was achieved with ML-NMR than with NMA, and uncertainty was reduced by explaining within- and between-study variation. We found little evidence that the conditional constancy or shared effect modifier assumptions were invalid. CONCLUSIONS: ML-NMR extends the NMA framework and addresses issues with previous population adjustment approaches. It coherently synthesizes evidence from IPD and AgD studies in networks of any size while avoiding aggregation bias and noncollapsibility bias, allows for key assumptions to be assessed or relaxed, and can produce estimates relevant to a target population for decision-making. HIGHLIGHTS: Multilevel network meta-regression (ML-NMR) extends the network meta-analysis framework to synthesize evidence from networks of studies providing individual patient data or aggregate data while adjusting for differences in effect modifiers between studies (population adjustment). We apply ML-NMR to a network of treatments for plaque psoriasis with ordered categorical outcomes.We demonstrate for the first time how ML-NMR allows key assumptions to be assessed. We check for violations of conditional constancy of relative effects (such as unobserved effect modifiers) through residual heterogeneity and inconsistency and the shared effect modifier assumption by relaxing this for each covariate in turn.Crucially for decision making, population-adjusted treatment effects can be produced in any relevant target population. We produce population-average estimates for 3 external target populations, represented by the PsoBest registry and the PROSPECT and Chiricozzi 2019 cohort studies.

Antenatal Seroprevalence of Zika and Chikungunya Viruses, Kingston Metropolitan Area, Jamaica, 2017-2019.

To determine the extent of exposure to Zika virus (ZIKV) and chikungunya virus (CHIKV) in Jamaica, we collected serum from 584 pregnant women during 2017-2019. We found that 15.6% had antibodies against ZIKV and 83.6% against CHIKV. These results indicate potential recirculation of ZIKV but not CHIKV in the near future.

A non-parametric approach for jointly combining evidence on progression free and overall survival time in network meta-analysis.

Randomised controlled trials of cancer treatments typically report progression free survival (PFS) and overall survival (OS) outcomes. Existing methods to synthesise evidence on PFS and OS either rely on the proportional hazards assumption or make parametric assumptions which may not capture the diverse survival curve shapes across studies and treatments. Furthermore, PFS and OS are not independent; OS is the sum of PFS and post-progression survival (PPS). Our aim was to develop a non-parametric approach for jointly synthesising evidence from published Kaplan-Meier survival curves of PFS and OS without assuming proportional hazards. Restricted mean survival times (RMST) are estimated by the area under the survival curves (AUCs) up to a restricted follow-up time. The correlation between AUCs due to the constraint that OS > PFS is estimated using bootstrap re-sampling. Network meta-analysis models are given for RMST for PFS and PPS and ensure that OS = PFS + PPS. Both additive and multiplicative network meta-analysis models are presented to obtain relative treatment effects as either differences or ratios of RMST. The methods are illustrated with a network meta-analysis of treatments for stage IIIA-N2 non-small cell lung cancer. The approach has implications for health economic models of cancer treatments, which require estimates of the mean time spent in the PFS and PPS health-states. The methods can be applied to a single time-to-event outcome, and so have wide applicability in any field where time-to-event outcomes are reported, the proportional hazards assumption is in doubt, and survival curve shapes differ across studies and interventions.

Accuracy of four lateral flow immunoassays for anti SARS-CoV-2 antibodies: a head-to-head comparative study.

BACKGROUND: SARS-CoV-2 antibody tests are used for population surveillance and might have a future role in individual risk assessment. Lateral flow immunoassays (LFIAs) can deliver results rapidly and at scale, but have widely varying accuracy. METHODS: In a laboratory setting, we performed head-to-head comparisons of four LFIAs: the Rapid Test Consortium's AbC-19TM Rapid Test, OrientGene COVID IgG/IgM Rapid Test Cassette, SureScreen COVID-19 Rapid Test Cassette, and Biomerica COVID-19 IgG/IgM Rapid Test. We analysed blood samples from 2,847 key workers and 1,995 pre-pandemic blood donors with all four devices. FINDINGS: We observed a clear trade-off between sensitivity and specificity: the IgG band of the SureScreen device and the AbC-19TM device had higher specificities but OrientGene and Biomerica higher sensitivities. Based on analysis of pre-pandemic samples, SureScreen IgG band had the highest specificity (98.9%, 95% confidence interval 98.3 to 99.3%), which translated to the highest positive predictive value across any pre-test probability: for example, 95.1% (95% uncertainty interval 92.6, 96.8%) at 20% pre-test probability. All four devices showed higher sensitivity at higher antibody concentrations ("spectrum effects"), but the extent of this varied by device. INTERPRETATION: The estimates of sensitivity and specificity can be used to adjust for test error rates when using these devices to estimate the prevalence of antibody. If tests were used to determine whether an individual has SARS-CoV-2 antibodies, in an example scenario in which 20% of individuals have antibodies we estimate around 5% of positive results on the most specific device would be false positives. FUNDING: Public Health England.

Association between self-reported signs and symptoms and SARS-CoV-2 antibody detection in UK key workers.

BACKGROUND: Screening for SARS-CoV-2 antibodies is under way in some key worker groups; how this adds to self-reported COVID-19 illness is unclear. In this study, we investigate the association between self-reported belief of COVID-19 illness and seropositivity. METHODS: Cross-sectional study of three key worker streams comprising (A) Police and Fire & Rescue (2 sites) (B) healthcare workers (1 site) and (C) healthcare workers with previously positive PCR result (5 sites). We collected self-reported signs and symptoms of COVID-19 and compared this with serology results from two SARS-CoV-2 immunoassays (Roche Elecsys® and EUROIMMUN). RESULTS: Between 01 and 26 June, we recruited 2847 individuals (Stream A: 1,247, Stream B: 1,546 and Stream C: 154). Amongst those without previous positive PCR tests, 687/2,579 (26%) reported belief they had COVID-19, having experienced compatible symptoms; however, only 208 (30.3%) of these were seropositive on both immunoassays. Both immunoassays had high sensitivities relative to previous PCR positivity (>93%); there was also limited decline in antibody titres up to 110 days post symptom onset. Symptomatic but seronegative individuals had differing symptom profiles and shorter illnesses than seropositive individuals. CONCLUSION: Non-COVID-19 respiratory illness may have been mistaken for COVID-19 during the outbreak; laboratory testing is more specific than self-reported key worker beliefs in ascertaining past COVID-19 disease.

Vertical transmission of Zika virus and its outcomes: a Bayesian synthesis of prospective studies.

BACKGROUND: Prospective studies of Zika virus in pregnancy have reported rates of congenital Zika syndrome and other adverse outcomes by trimester. However, Zika virus can infect and damage the fetus early in utero, but clear before delivery. The true vertical transmission rate is therefore unknown. We aimed to provide the first estimates of underlying vertical transmission rates and adverse outcomes due to congenital infection with Zika virus by trimester of exposure. METHODS: This was a Bayesian latent class analysis of data from seven prospective studies of Zika virus in pregnancy. We estimated vertical transmission rates, rates of Zika-virus-related and non-Zika-virus-related adverse outcomes, and the diagnostic sensitivity of markers of congenital infection. We allowed for variation between studies in these parameters and used information from women in comparison groups with no PCR-confirmed infection, where available. FINDINGS: The estimated mean risk of vertical transmission was 47% (95% credible interval 26 to 76) following maternal infection in the first trimester, 28% (15 to 46) in the second, and 25% (13 to 47) in the third. 9% (4 to 17) of deliveries following infections in the first trimester had symptoms consistent with congenital Zika syndrome, 3% (1 to 7) in the second, and 1% (0 to 3) in the third. We estimated that in infections during the first, second, and third trimester, respectively, 13% (2 to 27), 3% (-5 to 14), and 0% (-7 to 11) of pregnancies had adverse outcomes attributable to Zika virus infection. Diagnostic sensitivity of markers of congenital infection was lowest in the first trimester (42% [18 to 72]), but increased to 85% (51 to 99) in trimester two, and 80% (42 to 99) in trimester three. There was substantial between-study variation in the risks of vertical transmission and congenital Zika syndrome. INTERPRETATION: This preliminary analysis recovers the causal effects of Zika virus from disparate study designs. Higher transmission in the first trimester is unusual with congenital infections but accords with laboratory evidence of decreasing susceptibility of placental cells to infection during pregnancy. FUNDING: European Union Horizon 2020 programme.

Zika virus infection in pregnancy: a protocol for the joint analysis of the prospective cohort studies of the ZIKAlliance, ZikaPLAN and ZIKAction consortia.

INTRODUCTION: Zika virus (ZIKV) infection in pregnancy has been associated with microcephaly and severe neurological damage to the fetus. Our aim is to document the risks of adverse pregnancy and birth outcomes and the prevalence of laboratory markers of congenital infection in deliveries to women experiencing ZIKV infection during pregnancy, using data from European Commission-funded prospective cohort studies in 20 centres in 11 countries across Latin America and the Caribbean. METHODS AND ANALYSIS: We will carry out a centre-by-centre analysis of the risks of adverse pregnancy and birth outcomes, comparing women with confirmed and suspected ZIKV infection in pregnancy to those with no evidence of infection in pregnancy. We will document the proportion of deliveries in which laboratory markers of congenital infection were present. Finally, we will investigate the associations of trimester of maternal infection in pregnancy, presence or absence of maternal symptoms of acute ZIKV infection and previous flavivirus infections with adverse outcomes and with markers of congenital infection. Centre-specific estimates will be pooled using a two-stage approach. ETHICS AND DISSEMINATION: Ethical approval was obtained at each centre. Findings will be presented at international conferences and published in peer-reviewed open access journals and discussed with local public health officials and representatives of the national Ministries of Health, Pan American Health Organization and WHO involved with ZIKV prevention and control activities.

Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study.

Standard network meta-analysis and indirect comparisons combine aggregate data from multiple studies on treatments of interest, assuming that any factors that interact with treatment effects (effect modifiers) are balanced across populations. Population adjustment methods such as multilevel network meta-regression (ML-NMR), matching-adjusted indirect comparison (MAIC), and simulated treatment comparison (STC) relax this assumption using individual patient data from one or more studies, and are becoming increasingly prevalent in health technology appraisals and the applied literature. Motivated by an applied example and two recent reviews of applications, we undertook an extensive simulation study to assess the performance of these methods in a range of scenarios under various failures of assumptions. We investigated the impact of varying sample size, missing effect modifiers, strength of effect modification and validity of the shared effect modifier assumption, validity of extrapolation and varying between-study overlap, and different covariate distributions and correlations. ML-NMR and STC performed similarly, eliminating bias when the requisite assumptions were met. Serious concerns are raised for MAIC, which performed poorly in nearly all simulation scenarios and may even increase bias compared with standard indirect comparisons. All methods incur bias when an effect modifier is missing, highlighting the necessity of careful selection of potential effect modifiers prior to analysis. When all effect modifiers are included, ML-NMR and STC are robust techniques for population adjustment. ML-NMR offers additional advantages over MAIC and STC, including extending to larger treatment networks and producing estimates in any target population, making this an attractive choice in a variety of scenarios.

Multilevel network meta-regression for population-adjusted treatment comparisons.

Standard network meta-analysis (NMA) and indirect comparisons combine aggregate data from multiple studies on treatments of interest, assuming that any effect modifiers are balanced across populations. Population adjustment methods relax this assumption using individual patient data from one or more studies. However, current matching-adjusted indirect comparison and simulated treatment comparison methods are limited to pairwise indirect comparisons and cannot predict into a specified target population. Existing meta-regression approaches incur aggregation bias. We propose a new method extending the standard NMA framework. An individual level regression model is defined, and aggregate data are fitted by integrating over the covariate distribution to form the likelihood. Motivated by the complexity of the closed form integration, we propose a general numerical approach using quasi-Monte-Carlo integration. Covariate correlation structures are accounted for by using copulas. Crucially for decision making, comparisons may be provided in any target population with a given covariate distribution. We illustrate the method with a network of plaque psoriasis treatments. Estimated population-average treatment effects are similar across study populations, as differences in the distributions of effect modifiers are small. A better fit is achieved than a random effects NMA, uncertainty is substantially reduced by explaining within- and between-study variation, and estimates are more interpretable.

Equivalence of entropy balancing and the method of moments for matching-adjusted indirect comparison.

Indirect comparisons are used to obtain estimates of relative effectiveness between two treatments that have not been compared in the same randomized controlled trial, but have instead been compared against a common comparator in separate trials. Standard indirect comparisons use only aggregate data, under the assumption that there are no differences in effect-modifying variables between the trial populations. Population-adjusted indirect comparisons aim to relax this assumption by using individual patient data (IPD) from one trial to adjust for differences in effect modifiers between populations. At present, the most commonly used approach is matching-adjusted indirect comparison (MAIC), where weights are estimated that match the covariate distributions of the reweighted IPD to the aggregate trial. MAIC was originally proposed using the method of moments to estimate the weights, but more recently entropy balancing has been proposed as an alternative. Entropy balancing has an additional "optimality" property ensuring that the weights are as uniform as possible, reducing the standard error of the estimates. In this brief method note, we show that MAIC weights are mathematically identical whether estimated using entropy balancing or the method of moments. Importantly, this means that the standard MAIC (based on the method of moments) also enjoys the "optimality" property. Moreover, the additional flexibility of entropy balancing suggests several interesting avenues for further research, such as combining population adjustment via MAIC with adjustments for treatment switching or nonparametric covariate adjustment.

Estimating the prevalence of problem drug use from drug-related mortality data.

BACKGROUND AND AIMS: Indirect estimation methods are required for estimating the size of populations where only a proportion of individuals are observed directly, such as problem drug users (PDUs). Capture-recapture and multiplier methods are widely used, but have been criticized as subject to bias. We propose a new approach to estimating prevalence of PDU from numbers of fatal drug-related poisonings (fDRPs) using linked databases, addressing the key limitations of simplistic 'mortality multipliers'. METHODS: Our approach requires linkage of data on a large cohort of known PDUs to mortality registers and summary information concerning additional fDRPs observed outside this cohort. We model fDRP rates among the cohort and assume that rates in unobserved PDUs are equal to rates in the cohort during periods out of treatment. Prevalence is estimated in a Bayesian statistical framework, in which we simultaneously fit regression models to fDRP rates and prevalence, allowing both to vary by demographic factors and the former also by treatment status. RESULTS: We report a case study analysis, estimating the prevalence of opioid dependence in England in 2008/09, by gender, age group and geographical region. Overall prevalence was estimated as 0.82% (95% credible interval = 0.74-0.94%) of 15-64-year-olds, which is similar to a published estimate based on capture-recapture analysis. CONCLUSIONS: Our modelling approach estimates prevalence from drug-related mortality data, while addressing the main limitations of simplistic multipliers. This offers an alternative approach for the common situation where available data sources do not meet the strong assumptions required for valid capture-recapture estimation. In a case study analysis, prevalence estimates based on our approach were surprisingly similar to existing capture-recapture estimates but, we argue, are based on a much more objective and justifiable modelling approach.

Researching Zika in pregnancy: lessons for global preparedness.

Our understanding of congenital infections is based on prospective studies of women infected during pregnancy. The EU has funded three consortia to study Zika virus, each including a prospective study of pregnant women. Another multi-centre study has been funded by the US National Institutes of Health. This Personal View describes the study designs required to research Zika virus, and questions whether funding academics in the EU and USA to work with collaborators in outbreak areas is an effective strategy. 3 years after the 2015-16 Zika virus outbreaks, these collaborations have taught us little about vertical transmission of the virus. In the time taken to approve funding, agree contracts, secure ethics approval, and equip laboratories, Zika virus had largely disappeared. By contrast, prospective studies based on local surveillance and standard-of-care protocols have already provided valuable data. Threats to fetal and child health pose new challenges for global preparedness requiring support for the design and implementation of locally appropriate protocols. These protocols can answer the key questions earlier than externally designed studies and at lower cost. Local protocols can also provide a framework for recruitment of unexposed controls that are required to study less specific outcomes. Other priorities include accelerated development of non-invasive tests, and longer-term storage of neonatal and antenatal samples to facilitate retrospective reconstruction of cohort studies.

Joint synthesis of conditionally related multiple outcomes makes better use of data than separate meta-analyses.

BACKGROUND: When there are structural relationships between outcomes reported in different trials, separate analyses of each outcome do not provide a single coherent analysis, which is required for decision-making. For example, trials of intrapartum anti-bacterial prophylaxis (IAP) to prevent early onset group B streptococcal (EOGBS) disease can report three treatment effects: the effect on bacterial colonisation of the newborn, the effect on EOGBS, and the effect on EOGBS conditional on newborn colonisation. These outcomes are conditionally related, or nested, in a multi-state model. This paper shows how to exploit these structural relationships, providing a single coherent synthesis of all the available data, while checking to ensure that different sources of evidence are consistent. RESULTS: Overall, the use of IAP reduces the risk of EOGBS (RR: 0.03; 95% Credible Interval (CrI): 0.002-0.13). Most of the treatment effect is due to the prevention of colonisation in newborns of colonised mothers (RR: 0.08, 95% CrI: 0.04-0.14). Node-splitting demonstrated that the treatment effect calculated using only direct evidence was consistent with that predicted from the remaining evidence (p = 0.15). The findings accorded with previously published separate meta-analyses of the different outcomes, once these are re-analysed correctly accounting for zero cells. CONCLUSION: Multiple outcomes should be synthesised together where possible, taking account of their structural relationships. This generates an internally coherent analysis, suitable for decision making, in which estimates of each of the treatment effects are based on all available evidence (direct and indirect). Separate meta-analyses of each outcome have none of these properties.

Quantifying how diagnostic test accuracy depends on threshold in a meta-analysis.

Tests for disease often produce a continuous measure, such as the concentration of some biomarker in a blood sample. In clinical practice, a threshold C is selected such that results, say, greater than C are declared positive and those less than C negative. Measures of test accuracy such as sensitivity and specificity depend crucially on C, and the optimal value of this threshold is usually a key question for clinical practice. Standard methods for meta-analysis of test accuracy (i) do not provide summary estimates of accuracy at each threshold, precluding selection of the optimal threshold, and furthermore, (ii) do not make use of all available data. We describe a multinomial meta-analysis model that can take any number of pairs of sensitivity and specificity from each study and explicitly quantifies how accuracy depends on C. Our model assumes that some prespecified or Box-Cox transformation of test results in the diseased and disease-free populations has a logistic distribution. The Box-Cox transformation parameter can be estimated from the data, allowing for a flexible range of underlying distributions. We parameterise in terms of the means and scale parameters of the two logistic distributions. In addition to credible intervals for the pooled sensitivity and specificity across all thresholds, we produce prediction intervals, allowing for between-study heterogeneity in all parameters. We demonstrate the model using two case study meta-analyses, examining the accuracy of tests for acute heart failure and preeclampsia. We show how the model can be extended to explore reasons for heterogeneity using study-level covariates.

Population Adjustment Methods for Indirect Comparisons: A Review of National Institute for Health and Care Excellence Technology Appraisals.

OBJECTIVES: Indirect comparisons via a common comparator (anchored comparisons) are commonly used in health technology assessment. However, common comparators may not be available, or the comparison may be biased due to differences in effect modifiers between the included studies. Recently proposed population adjustment methods aim to adjust for differences between study populations in the situation where individual patient data are available from at least one study, but not all studies. They can also be used when there is no common comparator or for single-arm studies (unanchored comparisons). We aim to characterise the use of population adjustment methods in technology appraisals (TAs) submitted to the United Kingdom National Institute for Health and Care Excellence (NICE). METHODS: We reviewed NICE TAs published between 01/01/2010 and 20/04/2018. RESULTS: Population adjustment methods were used in 7 percent (18/268) of TAs. Most applications used unanchored comparisons (89 percent, 16/18), and were in oncology (83 percent, 15/18). Methods used included matching-adjusted indirect comparisons (89 percent, 16/18) and simulated treatment comparisons (17 percent, 3/18). Covariates were included based on: availability, expert opinion, effective sample size, statistical significance, or cross-validation. Larger treatment networks were commonplace (56 percent, 10/18), but current methods cannot account for this. Appraisal committees received results of population-adjusted analyses with caution and typically looked for greater cost effectiveness to minimise decision risk. CONCLUSIONS: Population adjustment methods are becoming increasingly common in NICE TAs, although their impact on decisions has been limited to date. Further research is needed to improve upon current methods, and to investigate their properties in simulation studies.

Has Chlamydia trachomatis prevalence in young women in England, Scotland and Wales changed? Evidence from national probability surveys.

We evaluate the utility of the National Surveys of Attitudes and Sexual Lifestyles (Natsal) undertaken in 2000 and 2010, before and after the introduction of the National Chlamydia Screening Programme, as an evidence source for estimating the change in prevalence of Chlamydia trachomatis (CT) in England, Scotland and Wales. Both the 2000 and 2010 surveys tested urine samples for CT by Nucleic Acid Amplification Tests (NAATs). We examined the sources of uncertainty in estimates of CT prevalence change, including sample size and adjustments for test sensitivity and specificity, survey non-response and informative non-response. In 2000, the unadjusted CT prevalence was 4.22% in women aged 18-24 years; in 2010, CT prevalence was 3.92%, a non-significant absolute difference of 0.30 percentage points (95% credible interval -2.8 to 2.0). In addition to uncertainty due to small sample size, estimates were sensitive to specificity, survey non-response or informative non-response, such that plausible changes in any one of these would be enough to either reverse or double any likely change in prevalence. Alternative ways of monitoring changes in CT incidence and prevalence over time are discussed.

Threshold Analysis as an Alternative to GRADE for Assessing Confidence in Guideline Recommendations Based on Network Meta-analyses.

Guideline development requires the synthesis of evidence on several treatments of interest, typically by using network meta-analysis (NMA). Because treatment effects may be estimated imprecisely or be based on evidence lacking internal or external validity, guideline developers must assess the robustness of recommendations made on the basis of the NMA to potential limitations in the evidence. Such limitations arise because the observed estimates differ from the true effects of interest, for example, because of study biases, sampling variation, or issues of relevance. The widely used GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework aims to assess the quality of evidence supporting a recommendation by using a structured series of qualitative judgments. This article argues that GRADE approaches proposed for NMA are insufficient for the purposes of guideline development, because the influence of the evidence on the final recommendation is not taken into account. It outlines threshold analysis as an alternative approach, demonstrating the method with 2 examples of clinical guidelines from the National Institute for Health and Care Excellence (NICE) in the United Kingdom. Threshold analysis quantifies precisely how much the evidence could change (for any reason, such as potential biases, or simply sampling variation) before the recommendation changes, and what the revised recommendation would be. If it is judged that the evidence could not plausibly change by more than this amount, then the recommendation is considered robust; otherwise, it is sensitive to plausible changes in the evidence. In this manner, threshold analysis directly informs decision makers and guideline developers of the robustness of treatment recommendations.