PROTOCOL: Abortion and mental health outcomes: A systematic review and meta-analysis.

This is a protocol for a systematic review and meta-analysis of research on mental health outcomes of abortion. Does abortion increase the risk of adverse mental health outcomes? That is the central question for this review. Our review aims to inform policy and practice by locating, critically appraising, and synthesizing empirical evidence on associations between abortion and subsequent mental health outcomes. Given the controversies surrounding this topic and the complex social, political, legal, and ideological contexts in which research and reviews on abortion are conducted, it is especially important to conduct this systematic review and meta-analysis with comprehensive, rigorous, unbiased, and transparent methods. We will include a variety of study designs to enhance understanding of studies' methodological strengths and weaknesses and to identify potential explanations for conflicting results. We will follow open science principles, providing access to our methods, measures, and results, and making data available for re-analysis.

The Logic of Generalization From Systematic Reviews and Meta-Analyses of Impact Evaluations.

Systematic reviews and meta-analyses are viewed as potent tools for generalized causal inference. These reviews are routinely used to inform decision makers about expected effects of interventions. However, the logic of generalization from research reviews to diverse policy and practice contexts is not well developed. Building on sampling theory, concerns about epistemic uncertainty, and principles of generalized causal inference, this article presents a pragmatic approach to generalizability assessment for use with systematic reviews and meta-analyses. This approach is applied to two systematic reviews and meta-analyses of effects of "evidence-based" psychosocial interventions for youth and families. Evaluations included in systematic reviews are not necessarily representative of populations and treatments of interest. Generalizability of results is limited by high risks of bias, uncertain estimates, and insufficient descriptive data from impact evaluations. Systematic reviews and meta-analyses can be used to test generalizability claims, explore heterogeneity, and identify potential moderators of effects. These reviews can also produce pooled estimates that are not representative of any larger sets of studies, programs, or people. Further work is needed to improve the conduct and reporting of impact evaluations and systematic reviews, and to develop practical approaches to generalizability assessment and guide applications of interventions in diverse policy and practice contexts.

Functional Family Therapy for families of youth (age 11-18) with behaviour problems: A systematic review and meta-analysis.

Background: Functional Family Therapy (FFT) is a short-term family-based intervention for youth with behaviour problems. FFT has been widely implemented in the USA and other high-income countries. It is often described as an evidence-based program with consistent, positive effects. Objectives: We aimed to synthesise the best available data to assess the effectiveness of FFT for families of youth with behaviour problems. Search Methods: Searches were performed in 2013-2014 and August 2020. We searched 22 bibliographic databases (including PsycINFO, ERIC, MEDLINE, Science Direct, Sociological Abstracts, Social Services Abstracts, World CAT dissertations and theses, and the Web of Science Core Collection), as well as government policy databanks and professional websites. Reference lists of articles were examined, and experts were contacted to search for missing information. Selection Criteria: We included randomised controlled trials (RCTs) and quasi-experimental designs (QEDs) with parallel cohorts and statistical controls for between-group differences at baseline. Participants were families of young people aged 11-18 with behaviour problems. FFT programmes were compared with usual services, alternative treatment, and no treatment. There were no publication, geographic, or language restrictions. Data Collection and Analysis: Two reviewers independently screened 1039 titles and abstracts, read all available study reports, assessed study eligibility, and extracted data onto structured electronic forms. We assessed risks of bias (ROB) using modified versions of the Cochrane ROB tool and the What Works Clearinghouse standards. Where possible, we used random effects models with inverse variance weights to pool results across studies. We used odds ratios for dichotomous outcomes and standardised mean differences for continuous outcomes. We used Hedges g to adjust for small sample sizes. We assessed the heterogeneity of effects with χ 2 and I 2. We produced separate forest plots for conceptually distinct outcomes and for different endpoints (<9, 9-14, 15-23, and 24-42 months after referral). We grouped studies by study design (RCT or QED), and then assessed differences between these two subgroups of studies with χ 2 tests. We generated robust variance estimates, using correlated effects (CE) models with small sample corrections to synthesise all available outcome data. Exploratory CE analyses assessed potential moderators of effects within these domains. We used GRADE guidelines to assess the certainty of evidence on six primary outcomes at 1 year after referral. Main Results: Twenty studies (14 RCTs and 6 QEDs) met our inclusion criteria. Fifteen of these studies provided some valid data for meta-analysis; these studies included 10,980 families in relevant FFT and comparison groups. All included studies had high risks of bias on at least one indicator. Half of the studies had high risks of bias on baseline equivalence, support for intent-to-treat analysis, selective reporting, and conflicts of interest. Fifteen studies had incomplete reporting of outcomes and endpoints. Using the GRADE rubric, we found that the certainty of evidence for FFT was very low for all of our primary outcomes. Using pairwise meta-analysis, we found no evidence of effects of FFT compared with other active treatments on any primary or secondary outcomes. Primary outcomes were: recidivism, out-of-home placement, internalising behaviour problems, external behaviour problems, self-reported delinquency, and drug or alcohol use. Secondary outcomes were: peer relations and prosocial behaviour, youth self esteem, parent symptoms and behaviour, family functioning, school attendance, and school performance. There were few studies in the pairwise meta-analysis (k < 7) and little heterogeneity of effects across studies in most of these analyses. There were few differences between effect estimates obtained in RCTs versus QEDs. More comprehensive CE models showed positive results of FFT in some domains and negative results in others, but these effects were small (standardised mean difference [SMD] <|0.20|) and not significantly different from no effect with one exception: Two studies found positive effects of FFT on youth substance abuse and two studies found null results in this domain, and the overall effect estimate for this outcome was statistically different from zero. Over all outcomes (15 studies and 293 effect sizes), small positive effects were detected (SMD = 0.19, SE = 0.09), but these were not significantly different from zero effect. Prediction intervals showed that future FFT evaluations are likely to produce a wide range of results, including moderate negative effects and strong positive results (-0.37 to 0.75). Authors' Conclusions: Results of 10 RCTs and five QEDs show that FFT does not produce consistent benefits or harms for youth with behavioural problems and their families. The positive or negative direction of results is inconsistent within and across studies. Most outcomes are not fully reported, the quality of available evidence is suboptimal, and the certainty of this evidence is very low. Overall estimates of effects of FFT may be inflated, due to selective reporting and publication biases.

PROTOCOL: Assessment of outcome reporting bias in studies included in Campbell systematic reviews.

This is the protocol for a Campbell systematic review. The objectives are as follows: To identify methods used to assess the risk of outcome reporting bias (ORB) in studies included in recent Campbell systematic reviews of intervention effects. The review will answer the following questions: What proportion of recent Campbell reviews included assessment of ORB? How did recent reviews define levels of risk of ORB (what categories, labels, and definitions did they use)? To what extent and how did these reviews use study protocols as sources of data on ORB? To what extent and how did reviews document reasons for judgments about risk of ORB? To what extent and how did reviews assess the inter-rater reliability of ORB ratings? To what extent and how were issues of ORB considered in the review's abstract, plain language summary, and conclusions?

The Campbell Collaboration's systematic review of school-based anti-bullying interventions does not meet mandatory methodological standards.

BACKGROUND: Many published reviews do not meet the widely accepted PRISMA standards for systematic reviews and meta-analysis. Campbell Collaboration and Cochrane reviews are expected to meet even more rigorous standards, but their adherence to these standards is uneven. For example, a newly updated Campbell systematic review of school-based anti-bullying interventions does not appear to meet many of the Campbell Collaboration's mandatory methodological standards. ISSUES: In this commentary, we document methodological problems in the Campbell Collaboration's new school-based anti-bullying interventions review, including (1) unexplained deviations from the protocol; (2) inadequate documentation of search strategies; (3) inconsistent reports on the number of included studies; (4) undocumented risk of bias ratings; (5) assessments of selective outcome reporting bias that are not transparent, not replicable, and appear to systematically underestimate risk of bias; (6) unreliable assessments of risk of publication bias; (7) use of a composite scale that conflates distinct risks of bias; and (8) failure to consider issues related to the strength of the evidence and risks of bias in interpreting results and drawing conclusions. Readers who are unaware of these problems may place more confidence in this review than is warranted. Campbell Collaboration editors declined to publish our comments and declined to issue a public statement of concern about this review. CONCLUSIONS: Systematic reviews are expected to use transparent methods and follow relevant methodological standards. Readers should be concerned when these expectations are not met, because transparency and rigor enhance the trustworthiness of results and conclusions. In the tradition of Donald T. Campbell, there is need for more public debate about the methods and conclusions of systematic reviews, and greater clarity regarding applications of (and adherence to) published standards for systematic reviews.

The methodological and reporting characteristics of Campbell reviews: A systematic review.

Background: The Campbell Collaboration undertakes systematic reviews of the effects of social and economic policies (interventions) to help policymakers, practitioners, and the public to make well-informed decisions about policy interventions. In 2010, the Cochrane Collaboration and the Campbell Collaboration developed a voluntary co-registration policy under the rationale to make full use of the shared interests and diverse expertise from different review groups within these two organizations. In order to promote the methodological quality and transparency of Campbell intervention reviews, the Methodological Expectations of Campbell Collaboration Intervention Reviews (MECCIR) were introduced in 2014 to guide Campbell reviewers. However, there has not been a comprehensive review of the methodological quality and reporting characteristics of Campbell reviews. Objectives: This review aimed to assess the methodological and reporting characteristics of Campbell intervention reviews and to compare the methodological quality and reporting completeness of Campbell reviews published before and after the implementation of MECCIR. A secondary aim was to compare the methodological quality and reporting completeness of reviews registered with Campbell only versus those co-registered with Cochrane and Campbell. Search Methods: We searched the Campbell Library to identify all the completed intervention reviews published between 1 January 2011 to 31 January 2018. Selection Criteria: One researcher downloaded and screened all the records to exclude non-intervention reviews based on reviews' title and abstract. A second researcher checked the full text of all the excluded records to confirm the exclusion. In case of discrepancies, the two researchers jointly agreed on the final decision. Data Collection and Analysis: We developed the abstraction form based on mandatory reporting items for methods, results, and discussion from the MECCIR reporting standards Version 1.1; and additional epidemiological characteristics identified in a similar study of systematic reviews in health. Additionally, we judged the methodological quality and completeness of reporting of each included review. For methodological quality, we used the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2) instrument; for reporting completeness we used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. We rated reporting as either complete/partial or not reported. We described characteristics of the included reviews with frequencies and percentages, and median with interquartile ranges (IQRs). We used Stata version 12.0 to conduct multiple linear regressions for continuous data and the ordered logistic regressions for ordered data to investigate associations between prespecified factors and both methodological quality and completeness of reporting. Main Results: We included 96 Campbell reviews, 46 were published between January 2011 and September 2014 (pre-MECCIR) and 50 between October 2014 and January 2018 (post-MECCIR). Twenty-two of 96 (23%) reviews were co-registered with Cochrane. For overall methodological quality, 16 (17%) reviews were rated as high, 40 (42%) as moderate, 24 (25%) as low and 16 (17%) as critical low using AMSTAR 2. Reviews published after the release of MECCIR had better methodological quality ratings than those published before MECCIR (odds ratio [OR] =6.61, 95% confidence interval [CI] [2.86, 15.27], p < .001). The percentages of reviews of high or moderate quality were 76% (post-MECCIR) and 39% (pre-MECCIR). Reviews co-registered with Cochrane were rated as having better methodological quality than those registered only with Campbell (OR = 5.57, 95% CI [2.13, 14.58], p < .001). The percentages of reviews of high or moderate quality were 77% versus 53% between co-registered and Campbell registered only reviews. Twenty-five of 96 reviews (26%) completely or partially reported all 27 PRISMA checklist items. The median number of items reported across reviews was 25 (IQR, 22-26). Reviews published after the release of MECCIR reported 2.80 more items than those published before MECCIR (95% CI [1.74, 3.88], p < .001); reviews co-registered on Campbell and Cochrane reported 1.98 more items than reviews only registered in Campbell (95% CI [0.72, 3.24], p = .003). An increasing trend over time was observed for both the percentage of high and moderate methodological quality of reviews and the median number of PRISMA items reported. Authors' Conclusions: Many features expected in systematic reviews were present in Campbell reviews most of the time. Methodological quality and reporting completeness were both significantly higher in reviews published after the introduction of MECCIR in 2014 compared with those published before. However, this may also reflect general improvement in the reporting the methodology of systematic reviews over time or associations with other characteristics which were not assessed such as funding or experience of teams. Reviews co-registered with Cochrane were of higher methodological quality and more complete reporting than reviews only registered in Campbell.

Evidence Synthesis International (ESI): Position Statement.

This paper is the initial Position Statement of Evidence Synthesis International, a new partnership of organizations that produce, support and use evidence synthesis around the world. The paper (i) argues for the importance of synthesis as a research exercise to clarify what is known from research evidence to inform policy, practice and personal decision making; (ii) discusses core issues for research synthesis such as the role of research evidence in decision making, the role of perspectives, participation and democracy in research and synthesis as a core component of evidence ecosystems; (iii) argues for 9 core principles for ESI on the nature and role of research synthesis; and (iv) lists the 5 main goals of ESI as a coordinating partnership for promoting and enabling the production and use of research synthesis.

Guidance for producing a Campbell evidence and gap map.

Evidence and Gap Maps (EGMs) are a systematic evidence synthesis product which display the available evidence relevant to a specific research question. EGMs are produced following the same principles as a systematic reviews, that is: specify a PICOS, a comprehensive search, screening against explicit inclusion and exclusion criteria, and systematic coding, analysis and reporting. This paper provides guidance on producing EGMs for publication in Campbell Systematic Reviews.

Do haphazard reviews provide sound directions for dissemination efforts?

Comments on The dissemination and implementation of evidence-based psychological treatments: A review of current efforts (see record 2010-02208-010) by Kathryn R. McHugh and David H. Barlow. The lead article in the February­March issue by McHugh and Barlow (2010) emphasized the need for "dissemination and implementation of evidence-based psychological treatments." The authors identified a number of intervention programs as evidence based and in need of dissemination. One is multisystemic therapy (MST). They claimed that this program is among "the most successful dissemination efforts...pursued by treatment developers" (p. 79). McHugh and Barlow's (2010) discussion of the implementation of MST in Hawaii is troubling, because it neglected to mention concerns about the perceived lack of cultural sensitivity of the MST program in that state.

Purification and characterization of enterocin 62-6, a two-peptide bacteriocin produced by a vaginal strain of Enterococcus faecium: Potential significance in bacterial vaginosis.

A bacteriocin produced by a vaginal isolate of Enterococcus faecium strain 62-6, designated enterocin 62-6, was characterized following purification and DNA sequence analysis and compared to previously described bacteriocins. Enterocin 62-6 was isolated from brain heart infusion (BHI) culture supernatants using ammonium sulfate precipitation followed by elution from a Sepharose cation exchange column using a continuous salt gradient (0.1-0.7 M NaCl). SDS-PAGE of an active column fraction resulted in an electrophoretically pure protein, which corresponded to the growth inhibition of the sensitive Lactobacillus indicator strain in the gel overlay assay. Purified enterocin 62-6 was shown to be heat- and pH-stable, and sensitive to the proteolytic enzymes alpha-chymotrypsin and pepsin. Results from mass spectrometry suggested that it comprised two peptides of 5206 and 5219+/-1 Da, which was confirmed by DNA sequence analysis. The characteristics of enterocin 62-6 as a small, heat- and pH-stable, cationic, hydrophobic, two-peptide, plasmid-borne bacteriocin, with an inhibitory spectrum against a broad range of Gram-positive but not Gram-negative bacteria, were consistent with its classification as a class IIc bacteriocin. Furthermore, its wide spectrum of growth inhibitory activity against Gram-positive bacteria of vaginal origin including lactobacilli, and stability under the acidic conditions of the vagina, are consistent with our hypothesis that it could have potential significance in disrupting the ecology of the vaginal tract and pave the way for the establishment of the abnormal microbiota associated with the vaginal syndrome bacterial vaginosis. This is the first class IIc bacteriocin produced by a strain of E. faecium of vaginal origin to be characterized.

Correlates of problem recognition and intentions to change among caregivers of abused and neglected children.

OBJECTIVE: To identify individual, family, and caseworker characteristics associated with problem recognition (PR) and intentions to change (ITC) in a sample of caregivers who received in-home child welfare services following substantiated reports of child abuse or neglect. METHODS: Caregivers were interviewed at 4 weeks, 16 weeks, and 1 year after referral for in-home services. In these interviews, the University of Rhode Island Change Assessment scale assessed PR and ITC in relation to caregiving practices. Additional data were obtained from administrative records and surveys of in-home services caseworkers. We used growth models to identify caregiver, family, and caseworker characteristics associated with initial levels of PR and ITC, and with changes in PR and ITC over time. RESULTS: Contrary to expectations, there were no overall increases in PR and ITC during the first 4 months of in-home services. PR and ITC scores fluctuated over time, in relation to some external events, case characteristics, and caseworker characteristics; however, we were able to account for small portions of the variance in PR and ITC. Controlling for social desirability bias and other variables in the analysis, negative life events, lack of network support, and the severity of caregiver depression were associated with greater PR; housing problems were associated with lower PR. Caregiver depression and age were associated with ITC. Caregivers whose children had been removed from their homes showed significant increases in ITC in the first few months of in-home services. The duration of caseworkers' child welfare experience predicted increases in their clients' PR in the first 4 months and more frequent contacts with an experienced caseworker predicted small, but significant increases in ITC over time. CONCLUSIONS: PR and ITC are associated with somewhat different case characteristics and may be affected by caseworkers' experience.

Caregivers' readiness for change: predictive validity in a child welfare sample.

OBJECTIVE: To assess the predictive validity of continuous measures of problem recognition (PR), intentions to change (ITC), and overall readiness for change (RFC) among primary caregivers who received in-home services following substantiated reports of child abuse or neglect. METHOD: A modified version of the University of Rhode Island Change Assessment scale was included in interviews with a sample of 353 primary caregivers at 4 weeks, 16 weeks, and 1 year after referral for in-home services. Additional data were obtained from administrative records and caseworker surveys. Hierarchical linear and nonlinear models were used to assess relationships between PR, ITC, RFC and changes over time in measures of individual and family functioning (e.g., parenting behaviors, children's behaviors, housing and economic problems, social support, and life events). Bivariate probit regression analysis was used to examine relationships between PR, ITC, RFC and the likelihood of subsequent reports of child maltreatment and out-of-home placements within 1 year after referral. RESULTS: Initial problem recognition and intentions to change predict a few improvements in individual and family functioning, along with significant reductions in the likelihood of additional reports of child maltreatment within 1 year. Initial intentions to change also predict reductions in the substantiation of subsequent reports of maltreatment. An overall measure of readiness for change predicts reductions in the likelihood of out-of-home placement. CONCLUSIONS: Problem recognition and intentions to change predict somewhat different outcomes; hence, there are few advantages of a combined measure of readiness for change. Further inquiry is needed to determine whether and how these associations are mediated by intervention processes or other factors in child welfare services populations.