Meta-analysis of Federally Funded Adolescent Pregnancy Prevention Program Evaluations.

Beginning in 2010, the U.S. Department of Health and Human Services (HHS) sponsored numerous studies testing the effectiveness of Adolescent Pregnancy Prevention programs on youths' risky sexual behaviors. This article presents results from a meta-analysis of such studies completed between 2015 and 2019 and provided to us by HHS. Studies were eligible for inclusion if they used an experimental/quasi-experimental design, included a comparison condition, assigned at least 10 study participants to each condition, and measured and reported at least one outcome related to sexual behavior or consequences. Data were extracted using a standardized coding protocol and outcomes were coded as, or transformed to, log odds ratio effect sizes for analysis and then transformed to odds ratio effect sizes for presentation in the text. All outcomes were coded such that log odds ratios greater than zero (odds ratios greater than 1) indicate beneficial effects (e.g., reductions in risky sexual behavior). A total of 52 studies met the inclusion criteria. Across 7 outcome types, pooled effects were generally small, the direction of effects was mixed, and none achieved statistical significance: recent pregnancy (OR = 1.30, 95% CI [1.00, 1.68]), ever had sex (OR = 1.07, 95% CI [0.99, 1.15]), recent sexual activity (OR = 0.95, 95% CI [0.84, 1.09]), recent unprotected sexual activity (OR = 1.05, 95% CI [0.96, 1.16]), ever pregnant (OR = 1.21, 95% CI [0.51, 2.89]), number of sexual partners (OR = 1.08, 95% CI = [0.28, 4.22]), and proportion of recent sexual experiences that were unprotected (OR = 0.74, 95% CI = [0.43, 1.31]). Moderator analyses pooling across all 7 outcome types indicated minor differences in program effectiveness between program settings, with those delivered in classroom settings being the least effective (b = - 0.17, 95% CI [- 0.33, - 0.01]). These findings do not lend meaningful support for the overall effectiveness of this group of pregnancy prevention programs. However, because the meta-analysis sample only included programs recently funded by HHS, these findings may have limited generalizability to programs funded by other entities or implemented in other countries.

Adolescent Pregnancy Prevention: Meta-Analysis of Federally Funded Program Evaluations.

BACKGROUND: Beginning in 2010, the US Department of Health and Human Services (HHS) funded more than 40 evaluations of adolescent pregnancy prevention interventions. The government's emphasis on rigor and transparency, along with a requirement that grantees collect standardized behavioral outcomes, ensured that findings could be meaningfully compared across evaluations. OBJECTIVES: We used random and mixed-effects meta-analysis to analyze the findings generated by these evaluations to learn whether program elements, program implementation features, and participant demographics were associated with effects on adolescent sexual risk behavior. SEARCH METHODS: We screened all 43 independent evaluation reports, some of which included multiple studies, funded by HHS and completed before October 1, 2016. HHS released, and our team considered, all such studies regardless of favorability or statistical significance. SELECTION CRITERIA: Of these studies, we included those that used a randomized or high-quality quasi-experimental research design. We excluded studies that did not use statistical matching or provide pretest equivalence data on a measure of sexual behavior or a close proxy. We also excluded studies that compared 2 pregnancy prevention interventions without a control group. A total of 44 studies from 39 reports, comprising 51 150 youths, met the inclusion criteria. DATA COLLECTION AND ANALYSIS: Two researchers extracted data from each study by using standard systematic reviewing and meta-analysis procedures. In addition, study authors provided individual participant data for a subset of 34 studies. We used mixed-effects meta-regressions with aggregate data to examine whether program or participant characteristics were associated with program effects on adolescent sexual risk behaviors and consequences. To examine whether individual-level participant characteristics such as age, gender, and race/ethnicity were associated with program effects, we used a 1-stage meta-regression approach combining participant-level data (48 635 youths) with aggregate data from the 10 studies for which participant-level data were not available. MAIN RESULTS: Across all 44 studies, we found small but statistically insignificant mean effects favoring the programs and little variability around those means. Only 2 program characteristics showed statistically reliable relationships with program effects. First, gender-specific (girl-only) programs yielded a statistically significant average effect size (P < .05). Second, programs with individualized service delivery were more effective than programs delivering services to youths in small groups (P < .05). We found no other statistically significant associations between program effects and program or participant characteristics, or evaluation methods. Nor was there a statistically significant difference in the mean effect sizes for programs with previous evidence of effectiveness and previously untested programs. CONCLUSIONS: Although several individual studies reported positive impacts, the average effects were small and there was minimal variation in effect sizes across studies on all of the outcomes assessed. Thus, we were unable to confidently identify which individual program characteristics were associated with effects. However, these studies examined relatively short-term effects and it is an open question whether some programs, perhaps with distinctive characteristics, will show longer-term effects as more of the adolescent participants become sexually active. Public Health Implications. The success of a small number of individualized interventions designed specifically for girls in changing behavioral outcomes suggests the need to reexamine the assumptions that underlie coed group approaches. However, given the almost total absence of similar programs targeting male adolescents, it is likely to be some time before evidence to support or reject such an approach for boys is available.

Replicating Reducing the Risk: 12-Month Impacts of a Cluster Randomized Controlled Trial.

OBJECTIVES: To test the effectiveness of Reducing the Risk, an evidence-based sexual health curriculum designed to help prevent adolescent pregnancy and sexually transmitted infections, on youth sexual behavior and intermediate outcomes thought to lead to these behaviors. METHODS: Classes within schools in St. Louis, Missouri; Austin, Texas; and San Diego, California; were randomly assigned to receive Reducing the Risk or "business as usual." Youths completed Web-based surveys at baseline (preintervention, August 2012-January 2014) and 12 months later (August 2013-January 2015). Intent-to-treat analyses were conducted across sites; we tested for differences in impacts between sites and other subgroups. RESULTS: The program had no overall impact on sexual behaviors. However, at 1 site, program participants were significantly less likely to have engaged in recent sexual intercourse than were control group members. There were positive overall impacts on intermediate outcomes (e.g., knowledge, attitudes). CONCLUSIONS: After 12 months, Reducing the Risk was unsuccessful at changing sexual behaviors. Other results were mixed, but promising evidence (e.g., behavioral impacts at 1 site, impacts on intermediate outcomes) suggests potential for more widespread behavioral impacts over a longer term.

Replicating the Safer Sex Intervention: 9-Month Impact Findings of a Randomized Controlled Trial.

OBJECTIVES: To test the effects of the Safer Sex Intervention (SSI) on female adolescents' sexual behavior and possible antecedents of behavior such as sexual health attitudes, knowledge, motivation, intentions, and skills. METHODS: A randomized controlled trial compared SSI (n = 1196) with no intervention (n = 613) among female adolescents aged 13 to 20 years at 3 sites across the United States from 2012 to 2015. Intent-to-treat impacts were estimated at 9 months after baseline, overall, and for key subgroups. RESULTS: Compared with control participants, SSI participants were less likely to have sexual intercourse without birth control, more likely to report positive attitudes toward protection and intention to use condoms, and more confident of their ability to refuse sex. SSI did not affect sexual risk knowledge or motivation to delay childbearing. Positive impacts on sexual behavior and sexual risk were observed among key subgroups of youths who were aged 18 years or older, Hispanic, not sexually experienced at baseline, and enrolled at the Minnesota site. CONCLUSIONS: SSI produced meaningful changes in sexual behavior and sexual risk and successfully addressed some potential antecedents of sexual risk behavior.

Replicating ¡Cuídate!: 6-Month Impact Findings of a Randomized Controlled Trial.

OBJECTIVES: To test whether ¡Cuídate!, a program culturally adapted for Hispanic youths, affects sexual risk behavior. METHODS: We evaluated 3 replications of ¡Cuídate! in California, Arizona, and Massachusetts in a randomized controlled trial (registry no. NCT02540304) in which 2169 primarily Hispanic participants were randomly assigned to an intervention (n = 1326) or a control (n = 870) group. Youths were surveyed at baseline (September 2012-April 2014) and 6 months postbaseline (March 2013-October 2014). We estimated pooled and subgroup impacts using a regression framework with baseline covariates to increase statistical precision (1216 youths analyzed in the treatment group, 806 analyzed in the control group). RESULTS: We found no impacts on the study's primary outcomes of recent sexual activity or recent unprotected sexual activity. However, ¡Cuídate! improved knowledge (10%-20% increase; P < .001), attitudes (effect size = .24; P < .001), and skills (effect size = .14; P = .002). Exploratory subgroup analyses suggest potentially problematic effects for some groups. CONCLUSIONS: Findings suggest that ¡Cuídate! was effective in improving youths' knowledge and attitudes. However, after 6 months, these changes did not translate to improvements in reported sexual risk behaviors.

Implementing three evidence-based program models: early lessons from the Teen Pregnancy Prevention Replication Study.

PURPOSE: This article describes some of the early implementation challenges faced by nine grantees participating in the Teen Pregnancy Prevention Replication Study and their response to them. METHODS: The article draws on information collected as part of a comprehensive implementation study. Sources include site and program documents; program officer reports; notes from site investigation, selection and negotiation; ongoing communications with grantees as part of putting the study into place; and semi-structured interviews with program staff. RESULTS AND CONCLUSIONS: The issues faced by grantees in implementing evidence-based programs designed to prevent teen pregnancy varied by program model. Grantees implementing a classroom-based curriculum faced challenges in delivering the curriculum within the constraints of school schedules and calendars (program length and size of class). Grantees implementing a culturally tailored curriculum faced a series of challenges, including implementing the intervention as part of the regular school curriculum in schools with diverse populations; low attendance when delivered as an after-school program; and resistance on the part of schools to specific curriculum content. The third set of grantees, implementing a program in clinics, faced challenges in identifying and recruiting young women into the program and in retaining young women once they were in the program. The experiences of these grantees reflect some of the complexities that should be carefully considered when choosing to replicate evidence-based programs. The Teen Pregnancy Prevention replication study will provide important context for assessing the effectiveness of some of the more widely replicated evidence-based programs.

The "quality " of early care and education settings: definitional and measurement issues.

There is a widespread belief that high-quality early care and education can improve children's school readiness. However, debate continues about the essential elements of a high-quality experience, about whether quality means the same things across different types of care settings, about how to measure quality, and about the level of quality that might make a meaningful difference in outcomes for children. Are the aspects of the child care environment that researchers measure the ones that are most strongly related to children's development? This article argues that the ways in which researchers currently measure early care environments are flawed and that the conclusions drawn about the relationship between these measures and outcomes for children are frequently incorrect or overstated. The article addresses four questions: How is the quality of the child care environment commonly defined and measured? Do the most commonly used measures capture the child's experience? Do they work well across all settings? Are researchers drawing the correct conclusions from studies that relate the child care environment to child outcomes? Finally, the article discusses some possible directions for future research.