How do cognitive behavioral therapy and interpersonal psychotherapy improve youth depression? Applying meta-analytic structural equation modeling to three decades of randomized trials.

Investigating the mechanisms through which psychotherapy brings about desired change can inform efforts to improve therapies. We applied meta-analytic structural equation modeling (MASEM) to assess putative change mechanisms for cognitive behavioral therapy (CBT) as mediators of youth depression treatment outcome. Then, we tested whether candidate mediators (CMs) showed evidence of treatment-specificity to CBT versus interpersonal psychotherapy (IPT). Literature searches identified 34 randomized trials (27 CBT, 6 IPT, 1 CBT/IPT, 3,868 participants, published 1982-2020) that measured seven CMs: negative cognition, social engagement, family functioning, pleasant activity engagement, problem solving, reframing, or avoidance. We assessed mediational pathways and whether posttreatment CMs mediated treatment effects on posttreatment depression symptoms, covarying pretreatment CMs, and symptoms. Treatment type was tested as a moderator of mediational pathways. Results show that negative cognition (24 trials) and pleasant activities (3 trials) mediated depression symptom outcome in CBT. Social engagement and family functioning showed stronger mediation in IPT (5 and 6 trials) than in CBT (14 and 13 trials). We conclude that negative cognition is a robust mediator of CBT but may not be treatment-specific; pleasant activities may also be a mediator of CBT. However, the lack of treatment or mediation effects involving problem solving and reframing contradicts CBT theory. In contrast, social and family mechanisms appear to be IPT-specific mediators. These conclusions are provisional due to small samples examining IPT and several CMs, limitations in CM measurement (i.e., posttreatment retrospective report), and assumptions of MASEM-and will need to be confirmed when more and better evidence accumulates. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Meta-analysis of Federally Funded Adolescent Pregnancy Prevention Program Evaluations.

Beginning in 2010, the U.S. Department of Health and Human Services (HHS) sponsored numerous studies testing the effectiveness of Adolescent Pregnancy Prevention programs on youths' risky sexual behaviors. This article presents results from a meta-analysis of such studies completed between 2015 and 2019 and provided to us by HHS. Studies were eligible for inclusion if they used an experimental/quasi-experimental design, included a comparison condition, assigned at least 10 study participants to each condition, and measured and reported at least one outcome related to sexual behavior or consequences. Data were extracted using a standardized coding protocol and outcomes were coded as, or transformed to, log odds ratio effect sizes for analysis and then transformed to odds ratio effect sizes for presentation in the text. All outcomes were coded such that log odds ratios greater than zero (odds ratios greater than 1) indicate beneficial effects (e.g., reductions in risky sexual behavior). A total of 52 studies met the inclusion criteria. Across 7 outcome types, pooled effects were generally small, the direction of effects was mixed, and none achieved statistical significance: recent pregnancy (OR = 1.30, 95% CI [1.00, 1.68]), ever had sex (OR = 1.07, 95% CI [0.99, 1.15]), recent sexual activity (OR = 0.95, 95% CI [0.84, 1.09]), recent unprotected sexual activity (OR = 1.05, 95% CI [0.96, 1.16]), ever pregnant (OR = 1.21, 95% CI [0.51, 2.89]), number of sexual partners (OR = 1.08, 95% CI = [0.28, 4.22]), and proportion of recent sexual experiences that were unprotected (OR = 0.74, 95% CI = [0.43, 1.31]). Moderator analyses pooling across all 7 outcome types indicated minor differences in program effectiveness between program settings, with those delivered in classroom settings being the least effective (b = - 0.17, 95% CI [- 0.33, - 0.01]). These findings do not lend meaningful support for the overall effectiveness of this group of pregnancy prevention programs. However, because the meta-analysis sample only included programs recently funded by HHS, these findings may have limited generalizability to programs funded by other entities or implemented in other countries.

Effects of a statewide pre-kindergarten program on children's achievement and behavior through sixth grade.

As state-funded pre-kindergarten (pre-K) programs expand, it is critical to investigate their short- and long-term effects. This article presents the results through sixth grade of a longitudinal randomized control study of the effects of a scaled-up, state-supported pre-K program. The analytic sample includes 2,990 children from low-income families who applied to oversubscribed pre-K program sites across the state and were randomly assigned to offers of admission or a wait list control. Data through sixth grade from state education records showed that the children randomly assigned to attend pre-K had lower state achievement test scores in third through sixth grades than control children, with the strongest negative effects in sixth grade. A negative effect was also found for disciplinary infractions, attendance, and receipt of special education services, with null effects on retention. The implications of these findings for pre-K policies and practices are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Adolescent Pregnancy Prevention: Meta-Analysis of Federally Funded Program Evaluations.

BACKGROUND: Beginning in 2010, the US Department of Health and Human Services (HHS) funded more than 40 evaluations of adolescent pregnancy prevention interventions. The government's emphasis on rigor and transparency, along with a requirement that grantees collect standardized behavioral outcomes, ensured that findings could be meaningfully compared across evaluations. OBJECTIVES: We used random and mixed-effects meta-analysis to analyze the findings generated by these evaluations to learn whether program elements, program implementation features, and participant demographics were associated with effects on adolescent sexual risk behavior. SEARCH METHODS: We screened all 43 independent evaluation reports, some of which included multiple studies, funded by HHS and completed before October 1, 2016. HHS released, and our team considered, all such studies regardless of favorability or statistical significance. SELECTION CRITERIA: Of these studies, we included those that used a randomized or high-quality quasi-experimental research design. We excluded studies that did not use statistical matching or provide pretest equivalence data on a measure of sexual behavior or a close proxy. We also excluded studies that compared 2 pregnancy prevention interventions without a control group. A total of 44 studies from 39 reports, comprising 51 150 youths, met the inclusion criteria. DATA COLLECTION AND ANALYSIS: Two researchers extracted data from each study by using standard systematic reviewing and meta-analysis procedures. In addition, study authors provided individual participant data for a subset of 34 studies. We used mixed-effects meta-regressions with aggregate data to examine whether program or participant characteristics were associated with program effects on adolescent sexual risk behaviors and consequences. To examine whether individual-level participant characteristics such as age, gender, and race/ethnicity were associated with program effects, we used a 1-stage meta-regression approach combining participant-level data (48 635 youths) with aggregate data from the 10 studies for which participant-level data were not available. MAIN RESULTS: Across all 44 studies, we found small but statistically insignificant mean effects favoring the programs and little variability around those means. Only 2 program characteristics showed statistically reliable relationships with program effects. First, gender-specific (girl-only) programs yielded a statistically significant average effect size (P < .05). Second, programs with individualized service delivery were more effective than programs delivering services to youths in small groups (P < .05). We found no other statistically significant associations between program effects and program or participant characteristics, or evaluation methods. Nor was there a statistically significant difference in the mean effect sizes for programs with previous evidence of effectiveness and previously untested programs. CONCLUSIONS: Although several individual studies reported positive impacts, the average effects were small and there was minimal variation in effect sizes across studies on all of the outcomes assessed. Thus, we were unable to confidently identify which individual program characteristics were associated with effects. However, these studies examined relatively short-term effects and it is an open question whether some programs, perhaps with distinctive characteristics, will show longer-term effects as more of the adolescent participants become sexually active. Public Health Implications. The success of a small number of individualized interventions designed specifically for girls in changing behavioral outcomes suggests the need to reexamine the assumptions that underlie coed group approaches. However, given the almost total absence of similar programs targeting male adolescents, it is likely to be some time before evidence to support or reject such an approach for boys is available.

Dealing with effect size multiplicity in systematic reviews and meta-analyses.

Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta-analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta-analyses of intervention and association studies, recommending a three-step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a convergent approach) or an interest in exploring heterogeneity (which we term a divergent approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a reductionist meta-analytic approach, which is characterized by inclusion of a single effect size per study, and an integrative approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta-analyses.

Can Propensity Score Analysis Approximate Randomized Experiments Using Pretest and Demographic Information in Pre-K Intervention Research?

BACKGROUND: It is unclear whether propensity score analysis (PSA) based on pretest and demographic covariates will meet the ignorability assumption for replicating the results of randomized experiments. PURPOSE: This study applies within-study comparisons to assess whether pre-Kindergarten (pre-K) treatment effects on achievement outcomes estimated using PSA based on a pretest and demographic covariates can approximate those found in a randomized experiment. METHODS: Data-Four studies with samples of pre-K children each provided data on two math achievement outcome measures with baseline pretests and child demographic variables that included race, gender, age, language spoken at home, and mother's highest education. Research Design and Data Analysis-A randomized study of a pre-K math curriculum provided benchmark estimates of effects on achievement measures. Comparison samples from other pre-K studies were then substituted for the original randomized control and the effects were reestimated using PSA. The correspondence was evaluated using multiple criteria. RESULTS AND CONCLUSIONS: The effect estimates using PSA were in the same direction as the benchmark estimates, had similar but not identical statistical significance, and did not differ from the benchmarks at statistically significant levels. However, the magnitude of the effect sizes differed and displayed both absolute and relative bias larger than required to show statistical equivalence with formal tests, but those results were not definitive because of the limited statistical power. We conclude that treatment effect estimates based on a single pretest and demographic covariates in PSA correspond to those from a randomized experiment on the most general criteria for equivalence.

Assessing meta-regression methods for examining moderator relationships with dependent effect sizes: A Monte Carlo simulation.

Dependent effect sizes are ubiquitous in meta-analysis. Using Monte Carlo simulation, we compared the performance of 2 methods for meta-regression with dependent effect sizes-robust variance estimation (RVE) and 3-level modeling-with the standard meta-analytic method for independent effect sizes. We further compared bias-reduced linearization and jackknife estimators as small-sample adjustments for RVE and Wald-type and likelihood ratio tests for 3-level models. The bias in the slope estimates, width of the confidence intervals around those estimates, and empirical type I error and statistical power rates of the hypothesis tests from these different methods were compared for mixed-effects meta-regression analysis with one moderator either at the study or at the effect size level. All methods yielded nearly unbiased slope estimates under most scenarios, but as expected, the standard method ignoring dependency provided inflated type I error rates when testing the significance of the moderators. Robust variance estimation methods yielded not only the best results in terms of type I error rate but also the widest confidence intervals and the lowest power rates, especially when using the jackknife adjustments. Three-level models showed a promising performance with a moderate to large number of studies, especially with the likelihood ratio test, and yielded narrower confidence intervals around the slope and higher power rates than those obtained with the RVE approach. All methods performed better when the moderator was at the effect size level, the number of studies was moderate to large, and the between-studies variance was small. Our results can help meta-analysts deal with dependency in their data.

Fitting meta-analytic structural equation models with complex datasets.

A modification of the first stage of the standard procedure for two-stage meta-analytic structural equation modeling for use with large complex datasets is presented. This modification addresses two common problems that arise in such meta-analyses: (a) primary studies that provide multiple measures of the same construct and (b) the correlation coefficients that exhibit substantial heterogeneity, some of which obscures the relationships between the constructs of interest or undermines the comparability of the correlations across the cells. One component of this approach is a three-level random effects model capable of synthesizing a pooled correlation matrix with dependent correlation coefficients. Another component is a meta-regression that can be used to generate covariate-adjusted correlation coefficients that reduce the influence of selected unevenly distributed moderator variables. A non-technical presentation of these techniques is given, along with an illustration of the procedures with a meta-analytic dataset. Copyright © 2016 John Wiley & Sons, Ltd.

Can brief alcohol interventions for youth also address concurrent illicit drug use? results from a meta-analysis.

Brief interventions aimed at reducing alcohol use among youth may interrupt a possible developmental progression to more serious substance use if they can also affect the use of other illicit drugs. This meta-analysis examined the findings of recent research on the effects of brief alcohol interventions for adolescents and young adults on both alcohol and illicit drug use. Eligible studies were those using randomized or controlled quasi-experimental designs to examine the effects of brief alcohol interventions on illicit drug use outcomes among youth. A comprehensive literature search identified 30 eligible study samples that, on average, included participants age 17, with 57 % male participants and 56 % White youth. Three-level random-effects meta-analyses were used to estimate mean effect sizes and explore variability in effects. Overall, brief interventions targeting both alcohol and other drugs were effective in reducing both of these substances. However, the brief interventions that targeted only alcohol had no significant secondary effects on untargeted illicit drug use. The evidence from current research, therefore, shows modest beneficial effects on outcomes that are targeted by brief interventions for youth, but does not show that those effects generalize to untargeted illicit drug use outcomes.

Brief alcohol interventions for adolescents and young adults: a systematic review and meta-analysis.

This study reports findings from a meta-analysis summarizing the effectiveness of brief alcohol interventions for adolescents (age 11-18) and young adults (age 19-30). We identified 185 eligible study samples using a comprehensive literature search and synthesized findings using random-effects meta-analyses with robust standard errors. Overall, brief alcohol interventions led to significant reductions in alcohol consumption and alcohol-related problems among adolescents (g = 0.27 and g = 0.19) and young adults (g = 0.17 and g = 0.11). These effects persisted for up to 1 year after intervention and did not vary across participant demographics, intervention length, or intervention format. However, certain intervention modalities (e.g., motivational interviewing) and components (e.g., decisional balance, goal-setting exercises) were associated with larger effects. We conclude that brief alcohol interventions yield beneficial effects on alcohol-related outcomes for adolescents and young adults that are modest but potentially worthwhile given their brevity and low cost.

Identifying Baseline Covariates for Use in Propensity Scores: A Novel Approach Illustrated for a Non-randomized Study of Recovery High Schools.

There are many situations where random assignment of participants to treatment and comparison conditions may be unethical or impractical. This article provides an overview of propensity score techniques that can be used for estimating treatment effects in non-randomized quasi-experimental studies. After reviewing the logic of propensity score methods, we call attention to the importance of the strong ignorability assumption and its implications. We then discuss the importance of identifying and measuring a sufficient set of baseline covariates upon which to base the propensity scores and illustrate approaches to that task in the design of a study of recovery high schools for adolescents treated for substance abuse. One novel approach for identifying important covariates that we suggest and demonstrate is to draw on the predictor-outcome correlations compiled in meta-analyses of prospective longitudinal correlations.

The effects of CenteringPregnancy group prenatal care on gestational age, birth weight, and fetal demise.

We examined the effects of CenteringPregnancy group prenatal care versus individually delivered prenatal care on gestational age, birth weight, and fetal demise. We conducted a retrospective chart review and used propensity score matching to form a sample of 6,155 women receiving prenatal care delivered in a group or individual format at five sites in Tennessee. Compared to the matched group of women receiving prenatal care in an individual format, women in CenteringPregnancy group prenatal care had longer weeks of gestation (b = .35, 95 % CI [.29, .41]), higher birth weight in grams (b = 28.6, 95 % CI [4.8, 52.3]), lower odds of very low birth weight (OR = .21, 95 % CI [.06, .70]), and lower odds of fetal demise (OR = .12, 95 % CI [.02, .92]). Results indicated no evidence of differences in the odds of preterm birth or low birth weight for participants in group versus individual prenatal care. CenteringPregnancy group prenatal care had statistically and clinically significant beneficial effects on very low birth weight and fetal demise outcomes relative to traditional individually delivered prenatal care. Group prenatal care had statistically significant beneficial effects on gestational age and birth weight, although the effects were relatively small in clinical magnitude.

Effects of CenteringPregnancy group prenatal care on breastfeeding outcomes.

INTRODUCTION: The purpose of this study was to examine the effects of CenteringPregnancy group prenatal care on breastfeeding outcomes, relative to traditional prenatal care delivered in an individual format. METHODS: A quasi-experimental research design was conducted with 794 women receiving prenatal care delivered in a group or individual format at 4 sites in Tennessee. Propensity scores were used to create groups of women statistically matched on background demographics and medical history. Outcomes included breastfeeding at discharge and breastfeeding at postpartum follow-up. RESULTS: Compared with the matched comparison group of women receiving prenatal care in an individual format, women in CenteringPregnancy group prenatal care had significantly higher odds of any breastfeeding at discharge (odds ratio [OR], 2.08; 95% confidence interval [CI], 1.32-3.26; P < .001). Across the 4 sites, there were no consistent differences in the odds of any breastfeeding at follow-up or exclusive breastfeeding at discharge or postpartum follow-up. DISCUSSION: CenteringPregnancy group prenatal care may have beneficial effects on initial rates of breastfeeding relative to individually delivered care. However, there is not sufficient evidence to conclude that CenteringPregnancy group prenatal care has robust effects on exclusive breastfeeding at discharge or postpartum follow-up.

The comparative effectiveness of outpatient treatment for adolescent substance abuse: a meta-analysis.

Meta-analysis was used to synthesize research on the effects of outpatient treatment on substance use outcomes for adolescents with substance use disorders. An extensive literature search located 45 eligible experimental or quasi-experimental studies reporting 73 treatment-comparison group pairs, with many of the comparison groups also receiving some treatment. The first analysis examined 250 effect sizes for the substance use outcomes of adolescents receiving different types of treatment relative to the respective comparison groups. As a category, family therapy programs were found to be more effective than their comparison conditions, whereas no treatment programs were less effective. However, not all treatment types were compared with each other in the available research, making it difficult to assess the comparative effectiveness of the different treatments. To provide a more differentiated picture of the relative improvement in substance use outcomes for different treatments, a second analysis examined 311 pre-post effect sizes measuring changes in substance use for adolescents in the separate treatment and comparison arms of the studies. The adolescents in almost all types of treatment showed reductions in substance use. The greatest improvements were found for family therapy and mixed and group counseling. Longer treatment duration was associated with smaller improvements, but other treatment characteristics and participant characteristics had little relationship to the pre-post changes in substance use. Based on these findings family therapy is the treatment with the strongest evidence of comparative effectiveness, although most types of treatment appear to be beneficial in helping adolescents reduce their substance use.

Language abilities of children who stutter: a meta-analytical review.

PURPOSE: To identify, integrate, and summarize evidence from empirical studies of the language abilities of children who stutter (CWS) and children who do not stutter (CWNS). METHOD: Candidate studies were identified through electronic databases, the tables of contents of speech-language journals, and reference lists of relevant articles and literature reviews. The 22 included studies met the following criteria: studied both children who did and did not stutter between ages 2;0 (years;months) and 8;0, and reported norm-referenced language measures and/or measures from spontaneous language samples amenable to effect size calculation. Data were extracted using a coding manual and were assessed by application of general and specialized analytical software. Mean difference effect size was estimated using Hedges's g (Hedges, 1982). RESULTS: Findings indicated that CWS scored significantly lower than CWNS on norm-referenced measures of overall language (Hedges's g = -0.48), receptive (Hedges's g = -0.52) and expressive vocabulary (Hedges's g = -0.41), and mean length of utterance (Hedges's g = -0.23). CONCLUSIONS: Present findings were taken to suggest that children's language abilities are potentially influential variables associated with childhood stuttering.

School-based interventions for aggressive and disruptive behavior: update of a meta-analysis.

BACKGROUND: Research about the effectiveness of school-based psychosocial prevention programs for reducing aggressive and disruptive behavior was synthesized using meta-analysis. This work updated previous work by the authors and further investigated which program and student characteristics were associated with the most positive outcomes. METHODS: Two hundred forty-nine experimental and quasi-experimental studies of school-based programs with outcomes representing aggressive and/or disruptive behavior were obtained. Effect sizes and study characteristics were coded from these studies and analyzed. RESULTS: Positive overall intervention effects were found on aggressive and disruptive behavior and other relevant outcomes. The most common and most effective approaches were universal programs and targeted programs for selected/indicated children. The mean effect sizes for these types of programs represent a decrease in aggressive/disruptive behavior that is likely to be of practical significance to schools. Multicomponent comprehensive programs did not show significant effects and those for special schools or classrooms were marginal. Different treatment modalities (e.g., behavioral, cognitive, social skills) produced largely similar effects. Effects were larger for better-implemented programs and those involving students at higher risk for aggressive behavior. CONCLUSIONS: Schools seeking prevention programs may choose from a range of effective programs with some confidence that whatever they pick will be effective. Without the researcher involvement that characterizes the great majority of programs in this meta-analysis, schools might be well-advised to give priority to those that will be easiest to implement well in their settings.