Comparison of statistical methods used to meta-analyse results from interrupted time series studies: an empirical study.

BACKGROUND: The Interrupted Time Series (ITS) is a robust design for evaluating public health and policy interventions or exposures when randomisation may be infeasible. Several statistical methods are available for the analysis and meta-analysis of ITS studies. We sought to empirically compare available methods when applied to real-world ITS data. METHODS: We sourced ITS data from published meta-analyses to create an online data repository. Each dataset was re-analysed using two ITS estimation methods. The level- and slope-change effect estimates (and standard errors) were calculated and combined using fixed-effect and four random-effects meta-analysis methods. We examined differences in meta-analytic level- and slope-change estimates, their 95% confidence intervals, p-values, and estimates of heterogeneity across the statistical methods. RESULTS: Of 40 eligible meta-analyses, data from 17 meta-analyses including 282 ITS studies were obtained (predominantly investigating the effects of public health interruptions (88%)) and analysed. We found that on average, the meta-analytic effect estimates, their standard errors and between-study variances were not sensitive to meta-analysis method choice, irrespective of the ITS analysis method. However, across ITS analysis methods, for any given meta-analysis, there could be small to moderate differences in meta-analytic effect estimates, and important differences in the meta-analytic standard errors. Furthermore, the confidence interval widths and p-values for the meta-analytic effect estimates varied depending on the choice of confidence interval method and ITS analysis method. CONCLUSIONS: Our empirical study showed that meta-analysis effect estimates, their standard errors, confidence interval widths and p-values can be affected by statistical method choice. These differences may importantly impact interpretations and conclusions of a meta-analysis and suggest that the statistical methods are not interchangeable in practice.

Interventions from pregnancy to two years after birth for parents experiencing complex post-traumatic stress disorder and/or with childhood experience of maltreatment.

BACKGROUND: Acceptable, effective and feasible support strategies (interventions) for parents experiencing complex post-traumatic stress disorder (CPTSD) symptoms or with a history of childhood maltreatment may offer an opportunity to support parental recovery, reduce the risk of intergenerational transmission of trauma and improve life-course trajectories for children and future generations. However, evidence relating to the effect of interventions has not been synthesised to provide a comprehensive review of available support strategies. This evidence synthesis is critical to inform further research, practice and policy approaches in this emerging area. OBJECTIVES: To assess the effects of interventions provided to support parents who were experiencing CPTSD symptoms or who had experienced childhood maltreatment (or both), on parenting capacity and parental psychological or socio-emotional wellbeing. SEARCH METHODS: In October 2021 we searched CENTRAL, MEDLINE, Embase, six other databases and two trials registers, together with checking references and contacting experts to identify additional studies. SELECTION CRITERIA: All variants of randomised controlled trials (RCTs) comparing any intervention delivered in the perinatal period designed to support parents experiencing CPTSD symptoms or with a history of childhood maltreatment (or both), to any active or inactive control. Primary outcomes were parental psychological or socio-emotional wellbeing and parenting capacity between pregnancy and up to two years postpartum. DATA COLLECTION AND ANALYSIS: Two review authors independently assessed the eligibility of trials for inclusion, extracted data using a pre-designed data extraction form, and assessed risk of bias and certainty of evidence. We contacted study authors for additional information as required. We analysed continuous data using mean difference (MD) for outcomes using a single measure, and standardised mean difference (SMD) for outcomes using multiple measures, and risk ratios (RR) for dichotomous data. All data are presented with 95% confidence intervals (CIs). We undertook meta-analyses using random-effects models. MAIN RESULTS: We included evidence from 1925 participants in 15 RCTs that investigated the effect of 17 interventions. All included studies were published after 2005. Interventions included seven parenting interventions, eight psychological interventions and two service system approaches. The studies were funded by major research councils, government departments and philanthropic/charitable organisations. All evidence was of low or very low certainty. Parenting interventions Evidence was very uncertain from a study (33 participants) assessing the effects of a parenting intervention compared to attention control on trauma-related symptoms, and psychological wellbeing symptoms (postpartum depression), in mothers who had experienced childhood maltreatment and were experiencing current parenting risk factors. Evidence suggested that parenting interventions may improve parent-child relationships slightly compared to usual service provision (SMD 0.45, 95% CI -0.06 to 0.96; I2 = 60%; 2 studies, 153 participants; low-certainty evidence). There may be little or no difference between parenting interventions and usual perinatal service in parenting skills including nurturance, supportive presence and reciprocity (SMD 0.25, 95% CI -0.07 to 0.58; I2 = 0%; 4 studies, 149 participants; low-certainty evidence). No studies assessed the effects of parenting interventions on parents' substance use, relationship quality or self-harm. Psychological interventions Psychological interventions may result in little or no difference in trauma-related symptoms compared to usual care (SMD -0.05, 95% CI -0.40 to 0.31; I2 = 39%; 4 studies, 247 participants; low-certainty evidence). Psychological interventions may make little or no difference compared to usual care to depression symptom severity (8 studies, 507 participants, low-certainty evidence, SMD -0.34, 95% CI -0.66 to -0.03; I2 = 63%). An interpersonally focused cognitive behavioural analysis system of psychotherapy may slightly increase the number of pregnant women who quit smoking compared to usual smoking cessation therapy and prenatal care (189 participants, low-certainty evidence). A psychological intervention may slightly improve parents' relationship quality compared to usual care (1 study, 67 participants, low-certainty evidence). Benefits for parent-child relationships were very uncertain (26 participants, very low-certainty evidence), while there may be a slight improvement in parenting skills compared to usual care (66 participants, low-certainty evidence). No studies assessed the effects of psychological interventions on parents' self-harm. Service system approaches One service system approach assessed the effect of a financial empowerment education programme, with and without trauma-informed peer support, compared to usual care for parents with low incomes. The interventions increased depression slightly (52 participants, low-certainty evidence). No studies assessed the effects of service system interventions on parents' trauma-related symptoms, substance use, relationship quality, self-harm, parent-child relationships or parenting skills. AUTHORS' CONCLUSIONS: There is currently a lack of high-quality evidence regarding the effectiveness of interventions to improve parenting capacity or parental psychological or socio-emotional wellbeing in parents experiencing CPTSD symptoms or who have experienced childhood maltreatment (or both). This lack of methodological rigour and high risk of bias made it difficult to interpret the findings of this review. Overall, results suggest that parenting interventions may slightly improve parent-child relationships but have a small, unimportant effect on parenting skills. Psychological interventions may help some women stop smoking in pregnancy, and may have small benefits on parents' relationships and parenting skills. A financial empowerment programme may slightly worsen depression symptoms. While potential beneficial effects were small, the importance of a positive effect in a small number of parents must be considered when making treatment and care decisions. There is a need for further high-quality research into effective strategies for this population.

Accuracy and Efficiency of Machine Learning-Assisted Risk-of-Bias Assessments in "Real-World" Systematic Reviews : A Noninferiority Randomized Controlled Trial.

BACKGROUND: Automation is a proposed solution for the increasing difficulty of maintaining up-to-date, high-quality health evidence. Evidence assessing the effectiveness of semiautomated data synthesis, such as risk-of-bias (RoB) assessments, is lacking. OBJECTIVE: To determine whether RobotReviewer-assisted RoB assessments are noninferior in accuracy and efficiency to assessments conducted with human effort only. DESIGN: Two-group, parallel, noninferiority, randomized trial. (Monash Research Office Project 11256). SETTING: Health-focused systematic reviews using Covidence. PARTICIPANTS: Systematic reviewers, who had not previously used RobotReviewer, completing Cochrane RoB assessments between February 2018 and May 2020. INTERVENTION: In the intervention group, reviewers received an RoB form prepopulated by RobotReviewer; in the comparison group, reviewers received a blank form. Studies were assigned in a 1:1 ratio via simple randomization to receive RobotReviewer assistance for either Reviewer 1 or Reviewer 2. Participants were blinded to study allocation before starting work on each RoB form. MEASUREMENTS: Co-primary outcomes were the accuracy of individual reviewer RoB assessments and the person-time required to complete individual assessments. Domain-level RoB accuracy was a secondary outcome. RESULTS: Of the 15 recruited review teams, 7 completed the trial (145 included studies). Integration of RobotReviewer resulted in noninferior overall RoB assessment accuracy (risk difference, -0.014 [95% CI, -0.093 to 0.065]; intervention group: 88.8% accurate assessments; control group: 90.2% accurate assessments). Data were inconclusive for the person-time outcome (RobotReviewer saved 1.40 minutes [CI, -5.20 to 2.41 minutes]). LIMITATION: Variability in user behavior and a limited number of assessable reviews led to an imprecise estimate of the time outcome. CONCLUSION: In health-related systematic reviews, RoB assessments conducted with RobotReviewer assistance are noninferior in accuracy to those conducted without RobotReviewer assistance. PRIMARY FUNDING SOURCE: University College London and Monash University.

Strengthening systematic reviews in public health: guidance in the Cochrane Handbook for Systematic Reviews of Interventions, 2nd edition.

AIMS: Decision makers in public health practice and policy rely on access to trustworthy, relevant, synthesized evidence. The second edition of the Cochrane Handbook for Systematic Reviews of Interventions ('the Handbook') reflects a major revision in guidance for authors of systematic reviews, incorporating a decade of methodological development and a number of significant changes to previous recommendations. This paper aims to highlight new guidance that addresses a number of key methodological challenges for authors of systematic reviews in public health. RESULTS: The revised Handbook includes guidance on framing public health research questions for synthesis, considering equity, intervention complexity, risk of bias assessment and synthesis methods other than meta-analysis. Reviews of public health interventions frequently encounter the types of methodological complexity addressed in this new guidance. CONCLUSION: We hope that readers will find that the Cochrane Handbook includes detailed and thoughtful guidance on both conceptualizing and executing systematic reviews relevant to public health questions. Considering the available methods guidance will, we hope, provide support for authors of public health reviews to tackle the challenges they encounter, strengthen their analysis and provide useful answers to the important questions asked by stakeholders and users of public health evidence.

[The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsDeclaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas]. / A declaração PRISMA 2020: diretriz atualizada para relatar revisões sistemáticas.

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

La declaración PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses), publicada en 2009, se diseñó para ayudar a los autores de revisiones sistemáticas a documentar de manera transparente el porqué de la revisión, qué hicieron los autores y qué encontraron. Durante la última década, ha habido muchos avances en la metodología y terminología de las revisiones sistemáticas, lo que ha requerido una actualización de esta guía. La declaración PRISMA 2020 sustituye a la declaración de 2009 e incluye una nueva guía de presentación de las publicaciones que refleja los avances en los métodos para identificar, seleccionar, evaluar y sintetizar estudios. La estructura y la presentación de los ítems ha sido modificada para facilitar su implementación. En este artículo, presentamos la lista de verificación PRISMA 2020 con 27 ítems, y una lista de verificación ampliada que detalla las recomendaciones en la publicación de cada ítem, la lista de verificación del resumen estructurado PRISMA 2020 y el diagrama de flujo revisado para revisiones sistemáticas.

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews.

Matthew Page and co-authors describe PRISMA 2020, an updated reporting guideline for systematic reviews and meta-analyses.

Comparison of six statistical methods for interrupted time series studies: empirical evaluation of 190 published series.

BACKGROUND: The Interrupted Time Series (ITS) is a quasi-experimental design commonly used in public health to evaluate the impact of interventions or exposures. Multiple statistical methods are available to analyse data from ITS studies, but no empirical investigation has examined how the different methods compare when applied to real-world datasets. METHODS: A random sample of 200 ITS studies identified in a previous methods review were included. Time series data from each of these studies was sought. Each dataset was re-analysed using six statistical methods. Point and confidence interval estimates for level and slope changes, standard errors, p-values and estimates of autocorrelation were compared between methods. RESULTS: From the 200 ITS studies, including 230 time series, 190 datasets were obtained. We found that the choice of statistical method can importantly affect the level and slope change point estimates, their standard errors, width of confidence intervals and p-values. Statistical significance (categorised at the 5% level) often differed across the pairwise comparisons of methods, ranging from 4 to 25% disagreement. Estimates of autocorrelation differed depending on the method used and the length of the series. CONCLUSIONS: The choice of statistical method in ITS studies can lead to substantially different conclusions about the impact of the interruption. Pre-specification of the statistical method is encouraged, and naive conclusions based on statistical significance should be avoided.

Evaluation of statistical methods used in the analysis of interrupted time series studies: a simulation study.

BACKGROUND: Interrupted time series (ITS) studies are frequently used to evaluate the effects of population-level interventions or exposures. However, examination of the performance of statistical methods for this design has received relatively little attention. METHODS: We simulated continuous data to compare the performance of a set of statistical methods under a range of scenarios which included different level and slope changes, varying lengths of series and magnitudes of lag-1 autocorrelation. We also examined the performance of the Durbin-Watson (DW) test for detecting autocorrelation. RESULTS: All methods yielded unbiased estimates of the level and slope changes over all scenarios. The magnitude of autocorrelation was underestimated by all methods, however, restricted maximum likelihood (REML) yielded the least biased estimates. Underestimation of autocorrelation led to standard errors that were too small and coverage less than the nominal 95%. All methods performed better with longer time series, except for ordinary least squares (OLS) in the presence of autocorrelation and Newey-West for high values of autocorrelation. The DW test for the presence of autocorrelation performed poorly except for long series and large autocorrelation. CONCLUSIONS: From the methods evaluated, OLS was the preferred method in series with fewer than 12 points, while in longer series, REML was preferred. The DW test should not be relied upon to detect autocorrelation, except when the series is long. Care is needed when interpreting results from all methods, given confidence intervals will generally be too narrow. Further research is required to develop better performing methods for ITS, especially for short series.

New generation antidepressants for depression in children and adolescents: a network meta-analysis.

BACKGROUND: Major depressive disorders have a significant impact on children and adolescents, including on educational and vocational outcomes, interpersonal relationships, and physical and mental health and well-being. There is an association between major depressive disorder and suicidal ideation, suicide attempts, and suicide. Antidepressant medication is used in moderate to severe depression; there is now a range of newer generations of these medications. OBJECTIVES: To investigate, via network meta-analysis (NMA), the comparative effectiveness and safety of different newer generation antidepressants in children and adolescents with a diagnosed major depressive disorder (MDD) in terms of depression, functioning, suicide-related outcomes and other adverse outcomes. The impact of age, treatment duration, baseline severity, and pharmaceutical industry funding was investigated on clinician-rated depression (CDRS-R) and suicide-related outcomes. SEARCH METHODS: We searched the Cochrane Common Mental Disorders Specialised Register, the Cochrane Library (Central Register of Controlled Trials (CENTRAL) and Cochrane Database of Systematic Reviews (CDSR)), together with Ovid Embase, MEDLINE and PsycINFO till March 2020. SELECTION CRITERIA: Randomised trials of six to 18 year olds of either sex and any ethnicity with clinically diagnosed major depressive disorder were included. Trials that compared the effectiveness of newer generation antidepressants with each other or with a placebo were included. Newer generation antidepressants included: selective serotonin reuptake inhibitors; selective norepinephrine reuptake inhibitors (SNRIs); norepinephrine reuptake inhibitors; norepinephrine dopamine reuptake inhibitors; norepinephrine dopamine disinhibitors (NDDIs); and tetracyclic antidepressants (TeCAs). DATA COLLECTION AND ANALYSIS: Two reviewers independently screened titles/abstracts and full texts, extracted data, and assessed risk of bias. We analysed dichotomous data as Odds Ratios (ORs), and continuous data as Mean Difference (MD) for the following outcomes: depression symptom severity (clinician rated), response or remission of depression symptoms, depression symptom severity (self-rated), functioning, suicide related outcomes and overall adverse outcomes. Random-effects network meta-analyses were conducted in a frequentist framework using multivariate meta-analysis. Certainty of evidence was assessed using Confidence in Network Meta-analysis (CINeMA). We used "informative statements" to standardise the interpretation and description of the results. MAIN RESULTS: Twenty-six studies were included. There were no data for the two primary outcomes (depressive disorder established via clinical diagnostic interview and suicide), therefore, the results comprise only secondary outcomes. Most antidepressants may be associated with a "small and unimportant" reduction in depression symptoms on the CDRS-R scale (range 17 to 113) compared with placebo (high certainty evidence: paroxetine: MD -1.43, 95% CI -3.90, 1.04; vilazodone: MD -0.84, 95% CI -3.03, 1.35; desvenlafaxine MD -0.07, 95% CI -3.51, 3.36; moderate certainty evidence: sertraline: MD -3.51, 95% CI -6.99, -0.04; fluoxetine: MD -2.84, 95% CI -4.12, -1.56; escitalopram: MD -2.62, 95% CI -5.29, 0.04; low certainty evidence: duloxetine: MD -2.70, 95% CI -5.03, -0.37; vortioxetine: MD 0.60, 95% CI -2.52, 3.72; very low certainty evidence for comparisons between other antidepressants and placebo). There were "small and unimportant" differences between most antidepressants in reduction of depression symptoms (high- or moderate-certainty evidence). Results were similar across other outcomes of benefit. In most studies risk of self-harm or suicide was an exclusion criterion for the study. Proportions of suicide-related outcomes were low for most included studies and 95% confidence intervals were wide for all comparisons. The evidence is very uncertain about the effects of mirtazapine (OR 0.50, 95% CI 0.03, 8.04), duloxetine (OR 1.15, 95% CI 0.72, 1.82), vilazodone (OR 1.01, 95% CI 0.68, 1.48), desvenlafaxine (OR 0.94, 95% CI 0.59, 1.52), citalopram (OR 1.72, 95% CI 0.76, 3.87) or vortioxetine (OR 1.58, 95% CI 0.29, 8.60) on suicide-related outcomes compared with placebo. There is low certainty evidence that escitalopram may "at least slightly" reduce odds of suicide-related outcomes compared with placebo (OR 0.89, 95% CI 0.43, 1.84). There is low certainty evidence that fluoxetine (OR 1.27, 95% CI 0.87, 1.86), paroxetine (OR 1.81, 95% CI 0.85, 3.86), sertraline (OR 3.03, 95% CI 0.60, 15.22), and venlafaxine (OR 13.84, 95% CI 1.79, 106.90) may "at least slightly" increase odds of suicide-related outcomes compared with placebo. There is moderate certainty evidence that venlafaxine probably results in an "at least slightly" increased odds of suicide-related outcomes compared with desvenlafaxine (OR 0.07, 95% CI 0.01, 0.56) and escitalopram (OR 0.06, 95% CI 0.01, 0.56). There was very low certainty evidence regarding other comparisons between antidepressants. AUTHORS' CONCLUSIONS: Overall, methodological shortcomings of the randomised trials make it difficult to interpret the findings with regard to the efficacy and safety of newer antidepressant medications. Findings suggest that most newer antidepressants may reduce depression symptoms in a small and unimportant way compared with placebo. Furthermore, there are likely to be small and unimportant differences in the reduction of depression symptoms between the majority of antidepressants. However, our findings reflect the average effects of the antidepressants, and given depression is a heterogeneous condition, some individuals may experience a greater response. Guideline developers and others making recommendations might therefore consider whether a recommendation for the use of newer generation antidepressants is warranted for some individuals in some circumstances. Our findings suggest sertraline, escitalopram, duloxetine, as well as fluoxetine (which is currently the only treatment recommended for first-line prescribing) could be considered as a first option. Children and adolescents considered at risk of suicide were frequently excluded from trials, so that we cannot be confident about the effects of these medications for these individuals. If an antidepressant is being considered for an individual, this should be done in consultation with the child/adolescent and their family/caregivers and it remains critical to ensure close monitoring of treatment effects and suicide-related outcomes (combined suicidal ideation and suicide attempt) in those treated with newer generation antidepressants, given findings that some of these medications may be associated with greater odds of these events. Consideration of psychotherapy, particularly cognitive behavioural therapy, as per guideline recommendations, remains important.

Which anticholinergic is best for people with overactive bladders? A network meta-analysis.

AIM: To carry out a network meta-analysis of randomised controlled trials (RCTs) of anticholinergic drug treatment for people with overactive bladders. METHODS: Comprehensive searches for relevant RCTs were carried out starting with RCTs included in previous systematic reviews with the last search in February 2017. Searches included terms for the anticholinergic drugs tolterodine, oxybutynin, trospium, propiverine, solifenacin, darifenacin, imidafenacin, and fesoterodine. Data was extracted from the systematic reviews or reports of studies for cure or improvement, voids per 24 hr, leakage episodes per 24 hr and dry mouth. Data was analysed using frequentist network meta-analysis. RESULTS: 128 studies were found. There was no clearly best treatment for cure or improvement. The differences between treatments for voids and leakages were small and unlikely to be of clinical importance. Transdermally delivered oxybutynin was clearly the best treatment for dry mouth but was still worse than placebo. CONCLUSIONS: All the anticholinergic drugs were better than placebo but apart from dry mouth were similar in effect. Transdermal oxybutynin caused less dry mouth than the other treatments, so may be worth considering as the first treatment.

Stakeholder views regarding ethical issues in the design and conduct of pragmatic trials: study protocol.

BACKGROUND: Randomized controlled trial (RCT) trial designs exist on an explanatory-pragmatic spectrum, depending on the degree to which a study aims to address a question of efficacy or effectiveness. As conceptualized by Schwartz and Lellouch in 1967, an explanatory approach to trial design emphasizes hypothesis testing about the mechanisms of action of treatments under ideal conditions (efficacy), whereas a pragmatic approach emphasizes testing effectiveness of two or more available treatments in real-world conditions. Interest in, and the number of, pragmatic trials has grown substantially in recent years, with increased recognition by funders and stakeholders worldwide of the need for credible evidence to inform clinical decision-making. This increase has been accompanied by the onset of learning healthcare systems, as well as an increasing focus on patient-oriented research. However, pragmatic trials have ethical challenges that have not yet been identified or adequately characterized. The present study aims to explore the views of key stakeholders with respect to ethical issues raised by the design and conduct of pragmatic trials. It is embedded within a large, four-year project that seeks to develop guidance for the ethical design and conduct of pragmatic trials. As a first step, this study will address important gaps in the current empirical literature with respect to identifying a comprehensive range of ethical issues arising from the design and conduct of pragmatic trials. By opening up a broad range of topics for consideration within our parallel ethical analysis, we will extend the current debate, which has largely emphasized issues of consent, to the range of ethical considerations that may flow from specific design choices. METHODS: Semi-structured interviews with key stakeholders (e.g. trialists, methodologists, lay members of study teams, bioethicists, and research ethics committee members), across multiple jurisdictions, identified based on their known experience and/or expertise with pragmatic trials. DISCUSSION: We expect that the study outputs will be of interest to a wide range of knowledge users including trialists, ethicists, research ethics committees, journal editors, regulators, healthcare policymakers, research funders and patient groups. All publications will adhere to the Tri-Agency Open Access Policy on Publications.

Core competencies for scientific editors of biomedical journals: consensus statement.

BACKGROUND: Scientific editors are responsible for deciding which articles to publish in their journals. However, we have not found documentation of their required knowledge, skills, and characteristics, or the existence of any formal core competencies for this role. METHODS: We describe the development of a minimum set of core competencies for scientific editors of biomedical journals. RESULTS: The 14 key core competencies are divided into three major areas, and each competency has a list of associated elements or descriptions of more specific knowledge, skills, and characteristics that contribute to its fulfillment. CONCLUSIONS: We believe that these core competencies are a baseline of the knowledge, skills, and characteristics needed to perform competently the duties of a scientific editor at a biomedical journal.

Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR).

BACKGROUND: Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews. METHODS: We developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool. RESULTS: The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions. CONCLUSIONS: The iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

Psychosocial interventions for supporting women to stop smoking in pregnancy.

BACKGROUND: Tobacco smoking remains one of the few preventable factors associated with complications in pregnancy, and has serious long-term implications for women and babies. Smoking in pregnancy is decreasing in high-income countries, but is strongly associated with poverty and is increasing in low- to middle-income countries. OBJECTIVES: To assess the effects of smoking cessation interventions during pregnancy on smoking behaviour and perinatal health outcomes. SEARCH METHODS: In this sixth update, we searched the Cochrane Pregnancy and Childbirth Group's Trials Register (13 November 2015), checked reference lists of retrieved studies and contacted trial authors. SELECTION CRITERIA: Randomised controlled trials, cluster-randomised trials, and quasi-randomised controlled trials of psychosocial smoking cessation interventions during pregnancy. DATA COLLECTION AND ANALYSIS: Two review authors independently assessed trials for inclusion and trial quality, and extracted data. Direct comparisons were conducted in RevMan, with meta-regression conducted in STATA 14. MAIN RESULTS: The overall quality of evidence was moderate to high, with reductions in confidence due to imprecision and heterogeneity for some outcomes. One hundred and two trials with 120 intervention arms (studies) were included, with 88 trials (involving over 28,000 women) providing data on smoking abstinence in late pregnancy. Interventions were categorised as counselling, health education, feedback, incentives, social support, exercise and dissemination.In separate comparisons, there is high-quality evidence that counselling increased smoking cessation in late pregnancy compared with usual care (30 studies; average risk ratio (RR) 1.44, 95% confidence interval (CI) 1.19 to 1.73) and less intensive interventions (18 studies; average RR 1.25, 95% CI 1.07 to 1.47). There was uncertainty whether counselling increased the chance of smoking cessation when provided as one component of a broader maternal health intervention or comparing one type of counselling with another. In studies comparing counselling and usual care (largest comparison), it was unclear whether interventions prevented smoking relapse among women who had stopped smoking spontaneously in early pregnancy. However, a clear effect was seen in smoking abstinence at zero to five months postpartum (11 studies; average RR 1.59, 95% CI 1.26 to 2.01) and 12 to 17 months (two studies, average RR 2.20, 95% CI 1.23 to 3.96), with a borderline effect at six to 11 months (six studies; average RR 1.33, 95% CI 1.00 to 1.77). In other comparisons, the effect was unclear for most secondary outcomes, but sample sizes were small.Evidence suggests a borderline effect of health education compared with usual care (five studies; average RR 1.59, 95% CI 0.99 to 2.55), but the quality was downgraded to moderate as the effect was unclear when compared with less intensive interventions (four studies; average RR 1.20, 95% CI 0.85 to 1.70), alternative interventions (one study; RR 1.88, 95% CI 0.19 to 18.60), or when smoking cessation health education was provided as one component of a broader maternal health intervention.There was evidence feedback increased smoking cessation when compared with usual care and provided in conjunction with other strategies, such as counselling (average RR 4.39, 95% CI 1.89 to 10.21), but the confidence in the quality of evidence was downgraded to moderate as this was based on only two studies and the effect was uncertain when feedback was compared to less intensive interventions (three studies; average RR 1.29, 95% CI 0.75 to 2.20).High-quality evidence suggests incentive-based interventions are effective when compared with an alternative (non-contingent incentive) intervention (four studies; RR 2.36, 95% CI 1.36 to 4.09). However pooled effects were not calculable for comparisons with usual care or less intensive interventions (substantial heterogeneity, I2 = 93%).High-quality evidence suggests the effect is unclear in social support interventions provided by peers (six studies; average RR 1.42, 95% CI 0.98 to 2.07), in a single trial of support provided by partners, or when social support for smoking cessation was provided as part of a broader intervention to improve maternal health.The effect was unclear in single interventions of exercise compared to usual care (RR 1.20, 95% CI 0.72 to 2.01) and dissemination of counselling (RR 1.63, 95% CI 0.62 to 4.32).Importantly, high-quality evidence from pooled results demonstrated that women who received psychosocial interventions had a 17% reduction in infants born with low birthweight, a significantly higher mean birthweight (mean difference (MD) 55.60 g, 95% CI 29.82 to 81.38 g higher) and a 22% reduction in neonatal intensive care admissions. However the difference in preterm births and stillbirths was unclear. There did not appear to be adverse psychological effects from the interventions.The intensity of support women received in both the intervention and comparison groups has increased over time, with higher-intensity interventions more likely to have higher-intensity comparisons, potentially explaining why no clear differences were seen with increasing intervention intensity in meta-regression analyses. Among meta-regression analyses: studies classified as having 'unclear' implementation and unequal baseline characteristics were less effective than other studies. There was no clear difference between trials implemented by researchers (efficacy studies), and those implemented by routine pregnancy staff (effectiveness studies), however there was uncertainty in the effectiveness of counselling in four dissemination trials where the focus on the intervention was at an organisational level. The pooled effects were similar in interventions provided for women classified as having predominantly low socio-economic status, compared to other women. The effect was significant in interventions among women from ethnic minority groups; however not among indigenous women. There were similar effect sizes in trials with biochemically validated smoking abstinence and those with self-reported abstinence. It was unclear whether incorporating use of self-help manuals or telephone support increased the effectiveness of interventions. AUTHORS' CONCLUSIONS: Psychosocial interventions to support women to stop smoking in pregnancy can increase the proportion of women who stop smoking in late pregnancy and the proportion of infants born low birthweight. Counselling, feedback and incentives appear to be effective, however the characteristics and context of the interventions should be carefully considered. The effect of health education and social support is less clear. New trials have been published during the preparation of this review and will be included in the next update.

Right choice, right time: Evaluation of an online decision aid for youth depression.

BACKGROUND: Appropriate treatment for youth depression is an important public health priority. Shared decision making has been recommended, yet no decision aids exist to facilitate this. OBJECTIVES: The main objective of this study was to evaluate an online decision aid for youth depression. DESIGN: An uncontrolled cohort study with pre-decision, immediately post-decision and follow-up measurements. SETTING AND PARTICIPANTS: Young people (n=66) aged 12-25 years with mild, mild-moderate or moderate-severe depression were recruited from two enhanced primary care services. INTERVENTION: Online decision aid with evidence communication, preference elicitation and decision support components. MAIN OUTCOME MEASURES: The main outcome measures were ability to make a decision; whether the decision was in line with clinical practice guidelines, personal preferences and values; decisional conflict; perceived involvement; satisfaction with decision; adherence; and depression scores at follow-up. RESULTS: After using the decision aid, clients were more likely to make a decision in line with guideline recommendations (93% vs 70%; P=.004), were more able to make a decision (97% vs 79%; P=.022), had significantly reduced decisional conflict (17.8 points lower (95% CI: 13.3-22.9 points lower) on the Decisional Conflict Scale (range 0-100)) and felt involved and satisfied with their decision. At follow-up, clients had significantly reduced depression symptoms (2.7 points lower (95% CI: 1.3-4.0 points lower) on the Patient Health Questionnaire nine-item scale (range 0-27)) and were adherent to 88% (95% CI: 82%-94%) of treatment courses. DISCUSSION AND CONCLUSIONS: A decision aid for youth depression can help ensure evidence-based, client-centred care, promoting collaboration in this often difficult to engage population.

Development and validation of SEER (Seeking, Engaging with and Evaluating Research): a measure of policymakers' capacity to engage with and use research.

BACKGROUND: Capacity building strategies are widely used to increase the use of research in policy development. However, a lack of well-validated measures for policy contexts has hampered efforts to identify priorities for capacity building and to evaluate the impact of strategies. We aimed to address this gap by developing SEER (Seeking, Engaging with and Evaluating Research), a self-report measure of individual policymakers' capacity to engage with and use research. METHODS: We used the SPIRIT Action Framework to identify pertinent domains and guide development of items for measuring each domain. Scales covered (1) individual capacity to use research (confidence in using research, value placed on research, individual perceptions of the value their organisation places on research, supporting tools and systems), (2) actions taken to engage with research and researchers, and (3) use of research to inform policy (extent and type of research use). A sample of policymakers engaged in health policy development provided data to examine scale reliability (internal consistency, test-retest) and validity (relation to measures of similar concepts, relation to a measure of intention to use research, internal structure of the individual capacity scales). RESULTS: Response rates were 55% (150/272 people, 12 agencies) for the validity and internal consistency analyses, and 54% (57/105 people, 9 agencies) for test-retest reliability. The individual capacity scales demonstrated adequate internal consistency reliability (alpha coefficients > 0.7, all four scales) and test-retest reliability (intra-class correlation coefficients > 0.7 for three scales and 0.59 for fourth scale). Scores on individual capacity scales converged as predicted with measures of similar concepts (moderate correlations of > 0.4), and confirmatory factor analysis provided evidence that the scales measured related but distinct concepts. Items in each of these four scales related as predicted to concepts in the measurement model derived from the SPIRIT Action Framework. Evidence about the reliability and validity of the research engagement actions and research use scales was equivocal. CONCLUSIONS: Initial testing of SEER suggests that the four individual capacity scales may be used in policy settings to examine current capacity and identify areas for capacity building. The relation between capacity, research engagement actions and research use requires further investigation.

Introduction to systematic reviews and meta-analysis.

Systematic reviews provide a method for collating and synthesizing research, and are used to inform healthcare decision making by clinicians, consumers and policy makers. A core component of many systematic reviews is a meta-analysis, which is a statistical synthesis of results across studies. In this review article, we introduce meta-analysis, focusing on the different meta-analysis models, their interpretation, how a model should be selected and discuss potential threats to the validity of meta-analyses. We illustrate the application of meta-analysis using data from a review examining the effects of early use of inhaled corticosteroids in the emergency department treatment of acute asthma.

Hand sanitiser provision for reducing illness absences in primary school children: a cluster randomised trial.

BACKGROUND: The potential for transmission of infectious diseases offered by the school environment are likely to be an important contributor to the rates of infectious disease experienced by children. This study aimed to test whether the addition of hand sanitiser in primary school classrooms compared with usual hand hygiene would reduce illness absences in primary school children in New Zealand. METHODS AND FINDINGS: This parallel-group cluster randomised trial took place in 68 primary schools, where schools were allocated using restricted randomisation (1:1 ratio) to the intervention or control group. All children (aged 5 to 11 y) in attendance at participating schools received an in-class hand hygiene education session. Schools in the intervention group were provided with alcohol-based hand sanitiser dispensers in classrooms for the winter school terms (27 April to 25 September 2009). Control schools received only the hand hygiene education session. The primary outcome was the number of absence episodes due to any illness among 2,443 follow-up children whose caregivers were telephoned after each absence from school. Secondary outcomes measured among follow-up children were the number of absence episodes due to specific illness (respiratory or gastrointestinal), length of illness and illness absence episodes, and number of episodes where at least one other member of the household became ill subsequently (child or adult). We also examined whether provision of sanitiser was associated with experience of a skin reaction. The number of absences for any reason and the length of the absence episode were measured in all primary school children enrolled at the schools. Children, school administrative staff, and the school liaison research assistants were not blind to group allocation. Outcome assessors of follow-up children were blind to group allocation. Of the 1,301 and 1,142 follow-up children in the hand sanitiser and control groups, respectively, the rate of absence episodes due to illness per 100 child-days was similar (1.21 and 1.16, respectively, incidence rate ratio 1.06, 95% CI 0.94 to 1.18). The provision of an alcohol-based hand sanitiser dispenser in classrooms was not effective in reducing rates of absence episodes due to respiratory or gastrointestinal illness, the length of illness or illness absence episodes, or the rate of subsequent infection for other members of the household in these children. The percentage of children experiencing a skin reaction was similar (10.4% hand sanitiser versus 10.3% control, risk ratio 1.01, 95% CI 0.78 to 1.30). The rate or length of absence episodes for any reason measured for all children also did not differ between groups. Limitations of the study include that the study was conducted during an influenza pandemic, with associated public health messaging about hand hygiene, which may have increased hand hygiene among all children and thereby reduced any additional effectiveness of sanitiser provision. We did not quite achieve the planned sample size of 1,350 follow-up children per group, although we still obtained precise estimates of the intervention effects. Also, it is possible that follow-up children were healthier than non-participating eligible children, with therefore less to gain from improved hand hygiene. However, lack of effectiveness of hand sanitiser provision on the rate of absences among all children suggests that this may not be the explanation. CONCLUSIONS: The provision of hand sanitiser in addition to usual hand hygiene in primary schools in New Zealand did not prevent disease of severity sufficient to cause school absence. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry ACTRN12609000478213. Please see later in the article for the Editors' Summary.

Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions.

BACKGROUND: Systematic reviews may be compromised by selective inclusion and reporting of outcomes and analyses. Selective inclusion occurs when there are multiple effect estimates in a trial report that could be included in a particular meta-analysis (e.g. from multiple measurement scales and time points) and the choice of effect estimate to include in the meta-analysis is based on the results (e.g. statistical significance, magnitude or direction of effect). Selective reporting occurs when the reporting of a subset of outcomes and analyses in the systematic review is based on the results (e.g. a protocol-defined outcome is omitted from the published systematic review). OBJECTIVES: To summarise the characteristics and synthesise the results of empirical studies that have investigated the prevalence of selective inclusion or reporting in systematic reviews of randomised controlled trials (RCTs), investigated the factors (e.g. statistical significance or direction of effect) associated with the prevalence and quantified the bias. SEARCH METHODS: We searched the Cochrane Methodology Register (to July 2012), Ovid MEDLINE, Ovid EMBASE, Ovid PsycINFO and ISI Web of Science (each up to May 2013), and the US Agency for Healthcare Research and Quality (AHRQ) Effective Healthcare Program's Scientific Resource Center (SRC) Methods Library (to June 2013). We also searched the abstract books of the 2011 and 2012 Cochrane Colloquia and the article alerts for methodological work in research synthesis published from 2009 to 2011 and compiled in Research Synthesis Methods. SELECTION CRITERIA: We included both published and unpublished empirical studies that investigated the prevalence and factors associated with selective inclusion or reporting, or both, in systematic reviews of RCTs of healthcare interventions. We included empirical studies assessing any type of selective inclusion or reporting, such as investigations of how frequently RCT outcome data is selectively included in systematic reviews based on the results, outcomes and analyses are discrepant between protocol and published review or non-significant outcomes are partially reported in the full text or summary within systematic reviews. DATA COLLECTION AND ANALYSIS: Two review authors independently selected empirical studies for inclusion, extracted the data and performed a risk of bias assessment. A third review author resolved any disagreements about inclusion or exclusion of empirical studies, data extraction and risk of bias. We contacted authors of included studies for additional unpublished data. Primary outcomes included overall prevalence of selective inclusion or reporting, association between selective inclusion or reporting and the statistical significance of the effect estimate, and association between selective inclusion or reporting and the direction of the effect estimate. We combined prevalence estimates and risk ratios (RRs) using a random-effects meta-analysis model. MAIN RESULTS: Seven studies met the inclusion criteria. No studies had investigated selective inclusion of results in systematic reviews, or discrepancies in outcomes and analyses between systematic review registry entries and published systematic reviews. Based on a meta-analysis of four studies (including 485 Cochrane Reviews), 38% (95% confidence interval (CI) 23% to 54%) of systematic reviews added, omitted, upgraded or downgraded at least one outcome between the protocol and published systematic review. The association between statistical significance and discrepant outcome reporting between protocol and published systematic review was uncertain. The meta-analytic estimate suggested an increased risk of adding or upgrading (i.e. changing a secondary outcome to primary) when the outcome was statistically significant, although the 95% CI included no association and a decreased risk as plausible estimates (RR 1.43, 95% CI 0.71 to 2.85; two studies, n = 552 meta-analyses). Also, the meta-analytic estimate suggested an increased risk of downgrading (i.e. changing a primary outcome to secondary) when the outcome was statistically significant, although the 95% CI included no association and a decreased risk as plausible estimates (RR 1.26, 95% CI 0.60 to 2.62; two studies, n = 484 meta-analyses). None of the included studies had investigated whether the association between statistical significance and adding, upgrading or downgrading of outcomes was modified by the type of comparison, direction of effect or type of outcome; or whether there is an association between direction of the effect estimate and discrepant outcome reporting.Several secondary outcomes were reported in the included studies. Two studies found that reasons for discrepant outcome reporting were infrequently reported in published systematic reviews (6% in one study and 22% in the other). One study (including 62 Cochrane Reviews) found that 32% (95% CI 21% to 45%) of systematic reviews did not report all primary outcomes in the abstract. Another study (including 64 Cochrane and 118 non-Cochrane reviews) found that statistically significant primary outcomes were more likely to be completely reported in the systematic review abstract than non-significant primary outcomes (RR 2.66, 95% CI 1.81 to 3.90). None of the studies included systematic reviews published after 2009 when reporting standards for systematic reviews (Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement, and Methodological Expectations of Cochrane Intervention Reviews (MECIR)) were disseminated, so the results might not be generalisable to more recent systematic reviews. AUTHORS' CONCLUSIONS: Discrepant outcome reporting between the protocol and published systematic review is fairly common, although the association between statistical significance and discrepant outcome reporting is uncertain. Complete reporting of outcomes in systematic review abstracts is associated with statistical significance of the results for those outcomes. Systematic review outcomes and analysis plans should be specified prior to seeing the results of included studies to minimise post-hoc decisions that may be based on the observed results. Modifications that occur once the review has commenced, along with their justification, should be clearly reported. Effect estimates and CIs should be reported for all systematic review outcomes regardless of the results. The lack of research on selective inclusion of results in systematic reviews needs to be addressed and studies that avoid the methodological weaknesses of existing research are also needed.