Meeting the Challenges of Longitudinal Cluster-Based Trials in Schools: Lessons From the Chicago Trial of Positive Action.

We describe challenges in the 6-year longitudinal cluster randomized controlled trial (CRCT) of Positive Action (PA), a social-emotional and character development (SECD) program, conducted in 14 low-income, urban Chicago Public Schools. Challenges pertained to logistics of study planning (school recruitment, retention of schools during the trial, consent rates, assessment of student outcomes, and confidentiality), study design (randomization of a small number of schools), fidelity (implementation of PA and control condition activities), and evaluation (restricted range of outcomes, measurement invariance, statistical power, student mobility, and moderators of program effects). Strategies used to address the challenges within each of these areas are discussed. Incorporation of lessons learned from this study may help to improve future evaluations of longitudinal CRCTs, especially those that involve evaluation of school-based interventions for minority populations and urban areas.

Can Universal SEL Programs Benefit Universally? Effects of the Positive Action Program on Multiple Trajectories of Social-Emotional and Misconduct Behaviors.

Behavioral trajectories during middle childhood are predictive of consequential outcomes later in life (e.g., substance abuse, violence). Social and emotional learning (SEL) programs are designed to promote trajectories that reflect both growth in positive behaviors and inhibited development of negative behaviors. The current study used growth mixture models to examine effects of the Positive Action (PA) program on behavioral trajectories of social-emotional and character development (SECD) and misconduct using data from a cluster-randomized trial that involved 14 schools and a sample of predominately low-income, urban youth followed from 3rd through 8th grade. For SECD, findings indicated that PA was similarly effective at improving trajectories within latent classes characterized as "high/declining" and "low/stable". Favorable program effects were likewise evident to a comparable degree for misconduct across observed latent classes that reflected "low/rising" and "high/rising" trajectories. These findings suggest that PA and perhaps other school-based universal SEL programs have the potential to yield comparable benefits across subgroups of youth with differing trajectories of positive and negative behaviors, making them promising strategies for achieving the intended goal of school-wide improvements in student outcomes.

Effects of the Positive Action Program on Indicators of Positive Youth Development Among Urban Youth.

This study evaluated effects of Positive Action, a school-based social-emotional and character development (SECD) intervention, on indicators of positive youth development (PYD) among a sample of low-income, ethnic minority youth attending 14 urban schools. The study used a matched-pair, cluster-randomized controlled design at the school level. A multiple-measure self-report protocol assessed four key strengths and resources for PYD: self-concept, peer affiliations, ethics, and social skills. Students (n=1170) were assessed from grades 3 to 8, the duration of the intervention, with drop-outs and late entrants included in analyses. Growth curve analyses revealed evidence of favorable program effects on each of the four types of resources. The study contributes to PYD research by providing evidence for school-based interventions in low-income, urban contexts for ethnic minority youth.

Effects of a School-Based Social-Emotional and Character Development Program on Health Behaviors: A Matched-Pair, Cluster-Randomized Controlled Trial.

There is considerable research that suggests that school-based social-emotional programs can foster improved mental health and reduce problem behaviors for participating youth; in contrast, much less is known about the impact of these programs on physical health, even though some of these programs also include at least limited direct attention to promoting physical health behaviors. We examined the effects of one such program, Positive Action (PA), on physical health behaviors and body mass index (BMI), and tested for mediation of program effects through a measure of social-emotional and character development (SECD). Participating schools in the matched-pair, cluster-randomized trial were 14 low-performing K-8 Chicago Public Schools. We followed a cohort of students in each school from grades 3 to 8 (eight waves of data collection; 1170 total students). Student self-reports of health behaviors served as the basis for measures of healthy eating and exercise, unhealthy eating, personal hygiene, consistent bedtime, and SECD. We collected height and weight measurements at endpoint to calculate age- and gender-adjusted BMI z-scores. Longitudinal multilevel modeling analyses revealed evidence of favorable program effects on personal hygiene [effect size (ES) = 0.48], healthy eating and exercise (ES = 0.21), and unhealthy eating (ES = -0.19); in addition, BMI z-scores were lower among students in PA schools at endpoint (ES = -0.21). Program effects were not moderated by either gender or student mobility. Longitudinal structural equation modeling demonstrated mediation through SECD for healthy eating and exercise, unhealthy eating, and personal hygiene. Findings suggest that a SECD program without a primary focus on health behavior promotion can have a modest impact on outcomes in this domain during the childhood to adolescence transition.

Teachers' Perceptions of School Organizational Climate as Predictors of Dosage and Quality of Implementation of a Social-Emotional and Character Development Program.

Organizational climate has been proposed as a factor that might influence a school's readiness to successfully implement school-wide prevention programs. The aim of this study was to evaluate the influence of teachers' perceptions of three dimensions of school organizational climate on the dosage and quality of teacher implementation of Positive Action, a social-emotional and character development (SECD) program. The dimensions measured were teachers' perceptions of (a) the school's openness to innovation, (b) the extent to which schools utilize participatory decision-making practices, and (c) the existence of supportive relationships among teachers (teacher-teacher affiliation). Data from 46 teachers in seven schools enrolled in the treatment arm of a longitudinal, cluster-randomized, controlled trial were analyzed. Teacher perceptions of a school's tendency to be innovative was associated with a greater number of lessons taught and self-reported quality of delivery, and teacher-teacher affiliation was associated with a higher use of supplementary activities. The findings suggest that perceptions of a school's organizational climate impact teachers' implementation of SECD programs and have implications for school administrators and technical assistance providers as they work to implement and sustain prevention programs in schools.

Two longterm studies of seasonal variation in depressive symptoms among community participants.

BACKGROUND: There is evidence that seasonal variation in depressive symptoms is common in the population. However, research is limited by a reliance on longterm retrospective methods. METHODS: Seasonal patterns were tested in two samples of community participants recruited in separate prospective studies in the Midwestern (n=556 males/females) and Pacific Northwestern (n=206 males) United States. Participants completed self-report measures of depressive symptoms 10-19 times from ages 14 to 36 years (n=8316 person observations). These data were compared with local meteorological conditions (e.g., solar radiation) recorded across the 2 weeks prior to each self-report. RESULTS: In within-subjects analyses, participants' depressive symptoms and the probability of clinically significant symptoms varied with the time of year, as hypothesized (highest in the weeks of early Winter; lowest in early Fall). However, effect sizes were modest and were not explained by recent sunlight or other meteorological conditions. LIMITATIONS: Samples were not nationally representative. Participants did not complete retrospective reports of seasonal depression or measures of current vegetative symptoms. CONCLUSIONS: Neither time of the year nor recent seasonally linked meteorological conditions were powerful influences on depressive symptoms experienced by community populations in relevant geographic regions. Prior studies may have overestimated the prevalence and significance of seasonal variation in depressive symptoms for the general population.

Using social-emotional and character development to improve academic outcomes: a matched-pair, cluster-randomized controlled trial in low-income, urban schools.

BACKGROUND: School-based social-emotional and character development (SECD) programs can influence not only SECD but also academic-related outcomes. This study evaluated the impact of one SECD program, Positive Action (PA), on educational outcomes among low-income, urban youth. METHODS: The longitudinal study used a matched-pair, cluster-randomized controlled design. Student-reported disaffection with learning and academic grades, and teacher ratings of academic ability and motivation were assessed for a cohort followed from grades 3 to 8. Aggregate school records were used to assess standardized test performance (for entire school, cohort, and demographic subgroups) and absenteeism (entire school). Multilevel growth-curve analyses tested program effects. RESULTS: PA significantly improved growth in academic motivation and mitigated disaffection with learning. There was a positive impact of PA on absenteeism and marginally significant impact on math performance of all students. There were favorable program effects on reading for African American boys and cohort students transitioning between grades 7 and 8, and on math for girls and low-income students. CONCLUSIONS: A school-based SECD program was found to influence academic outcomes among students living in low-income, urban communities. Future research should examine mechanisms by which changes in SECD influence changes in academic outcomes.

Effects of Positive Action on the emotional health of urban youth: a cluster-randomized trial.

PURPOSE: We examined the effects of Positive Action (PA), a school-based social-emotional learning and health promotion program, on the emotional health of predominately low-income and ethnic minority urban youth. METHODS: The study was a matched-pair, cluster-randomized controlled trial involving 14 Chicago public schools. Outcomes were assessed over a 6-year period of program implementation for a cohort of youth in each school, followed from grades 3 to 8. Youth reported on their emotional health (positive affect, life satisfaction, depression, anxiety) and social-emotional and character development. Growth-curve and structural-equation modeling analyses assessed overall program effects on the emotional health outcomes as well as mediation of these effects via the program's impact on youths' social-emotional and character development. RESULTS: Students in PA schools, compared with those in control schools, had more favorable change over the course of the study in positive affect (standardized mean difference effect size [ES] = .17) and life satisfaction (ES = .13) as well as significantly lower depression (ES = -.14) and anxiety (ES = -.26) at study end point. Program effects for positive affect, depression, and anxiety were mediated by more favorable change over time in social-emotional and character development for students in PA schools. CONCLUSIONS: Results suggest that universal, school-based programs can benefit the emotional health of youth in low-income, urban settings. The modest magnitude of effects over an extended period of program implementation, however, reflects the challenges of both mounting interventions and offsetting formidable risks for mental health problems in such environments.

Problem behavior and urban, low-income youth: a randomized controlled trial of positive action in Chicago.

BACKGROUND: Youth problem behaviors remain a public health issue. Youth in low-income, urban areas are particularly at risk for engaging in aggressive, violent, and disruptive behaviors. PURPOSE: To evaluate the effects of a school-based social-emotional learning and health promotion program on problem behaviors and related attitudes among low-income, urban youth. DESIGN: A matched-pair, cluster RCT. SETTING/PARTICIPANTS: Participants were drawn from 14 Chicago Public Schools over a 6-year period of program delivery with outcomes assessed for a cohort of youth followed from Grades 3 to 8. Data were collected from Fall 2004 to Spring 2010, and analyzed in Spring 2012. INTERVENTION: The Positive Action program includes a scoped and sequenced K-12 classroom curriculum with six components: self-concept, social and emotional positive actions for managing oneself responsibly, and positive actions directed toward physical and mental health, honesty, getting along with others, and continually improving oneself. The program also includes teacher, counselor, family, and community training as well as activities directed toward schoolwide climate development. MAIN OUTCOME MEASURES: Youth reported on their normative beliefs in support of aggression and on their bullying, disruptive, and violent behaviors; parents rated youths' bullying behaviors and conduct problems; schoolwide data on disciplinary referrals and suspensions were obtained from school records. RESULTS: Multilevel growth-curve modeling analyses conducted on completion of the trial indicated that Positive Action mitigated increases over time in (1) youth reports of normative beliefs supporting aggressive behaviors and of engaging in disruptive behavior and bullying (girls only) and (2) parent reports of youth bullying behaviors (boys only). At study end-point, students in Positive Action schools also reported a lower rate of violence-related behavior than students in control schools. Schoolwide findings indicated positive program effects on both disciplinary referrals and suspensions. Program effect sizes ranged from -0.26 to -0.68. CONCLUSIONS: These results extend evidence of the effectiveness of the Positive Action program to low-income, minority, urban school settings, and to middle school-aged youth.

Preventing negative behaviors among elementary-school students through enhancing students' social-emotional and character development.

PURPOSE: Examine the effects of a comprehensive, school-wide social-emotional and character development program using a positive youth development perspective. Specifically, we examined a mediation mechanism whereby positive academic-related behaviors mediated the intervention effects on substance use, violence, and sexual activity. DESIGN: Matched-pair, cluster-randomized, controlled design. SETTING: Twenty (10 intervention and 10 control) racially/ethnically diverse schools in Hawaii. SUBJECTS: Elementary-aged students (N = 1784) from grade 5. INTERVENTION: The Positive Action program. MEASURES: Students self-reported their academic behaviors, together with their substance use, violence, and voluntary sexual activity; teachers rated students' academic behaviors, substance use, and violence. ANALYSIS: Structural equation modeling. RESULTS: Students attending intervention schools reported significantly better academic behavior (B = .273, SE = .039, p < .001) and significantly less substance use (B = -.970, SE = .292, p < .01, incidence-rate ratio [IRR] = .379), violence (B = -1.410, SE = .296, p < .001, IRR = .244), and sexual activity (B = -2.415, SE = .608, p < .001, odds ratio = .089); boys reported more negative behaviors than girls. Intervention effects on student-reported substance use, violence, and sexual activity were mediated by positive academic behavior. Teacher reports corroborated these results, with rated academic behavior partially mediating the effects of the intervention on rated negative behaviors. CONCLUSION: This study (1) provides evidence that adds insight into one mechanism through which a social-emotional and character development program affects negative outcomes and (2) supports social-emotional and character development and positive youth development perspectives that posit that focusing on youths' assets may reduce negative behaviors.

Direct and Mediated Effects of a Social-Emotional and Character Development Program on Adolescent Substance Use.

Mitigating and preventing substance use among adolescents requires approaches that address the multitude of factors that influence this behavior. Such approaches must be tested, not only for evidence of empirical effectiveness, but also to determine the mechanisms by which they are successful. The aims of the present study were twofold: 1) To determine the effectiveness of a school-based social-emotional and character development (SECD) program, Positive Action (PA), in reducing substance use (SU) among a sample of U.S. youth living in a low-income, urban environment, and 2) to test one mechanism by which the program achieves its success. We used longitudinal mediation analysis to test the hypotheses that: 1) students attending PA intervention schools engage in significantly less SU than students attending control schools, 2) students attending PA intervention schools show significantly better change in SECD than students attending control schools, and 3) the effect of the PA intervention on SU is mediated by the change in SECD. Analyses revealed program effects on both SECD and SU, a relationship between SECD and SU, and the effects of PA on SU were completely mediated by changes in SECD. Future research directions and implications for school-based social-emotional and character development efforts and substance use prevention are addressed.

Improving elementary school quality through the use of a social-emotional and character development program: a matched-pair, cluster-randomized, controlled trial in Hawai'i.

BACKGROUND: School safety and quality affect student learning and success. This study examined the effects of a comprehensive elementary school-wide social-emotional and character education program, Positive Action, on teacher, parent, and student perceptions of school safety and quality utilizing a matched-pair, cluster-randomized, controlled design. The Positive Action Hawai'i trial included 20 racially/ethnically diverse schools and was conducted from 2002-2003 through 2005-2006. METHODS: School-level archival data, collected by the Hawai'i Department of Education, were used to examine program effects at 1-year post-trial. Teacher, parent, and student data were analyzed to examine indicators of school quality such as student safety and well-being, involvement, and satisfaction, as well as overall school quality. Matched-paired t-tests were used for the primary analysis, and sensitivity analyses included permutation tests and random-intercept growth curve models. RESULTS: Analyses comparing change from baseline to 1-year post-trial revealed that intervention schools demonstrated significantly improved school quality compared to control schools, with 21%, 13%, and 16% better overall school quality scores as reported by teachers, parents, and students, respectively. Teacher, parent, and student reports on individual school-quality indicators showed improvement in student safety and well-being, involvement, satisfaction, quality student support, focused and sustained action, standards-based learning, professionalism and system capacity, and coordinated team work. Teacher reports also showed an improvement in the responsiveness of the system. CONCLUSIONS: School quality was substantially improved, providing evidence that a school-wide social-emotional and character education program can enhance school quality and facilitate whole-school change.

Effects of the Positive Action programme on problem behaviours in elementary school students: a matched-pair randomised control trial in Chicago.

This study examined the effects of the Positive Action (PA) programme in Chicago Public Schools on problem behaviours among a cohort of elementary school students from grade three through grade five. Using a matched-pair, randomised control design with 14 elementary schools, approximately 510 fifth-graders self-reported lifetime substance use, serious violence-related behaviour, and current bullying and disruptive behaviours. Three-level (i.e. students nested within schools within school pairs) overdispersed Poisson models were used to examine programme effects on the number of items endorsed for each of the four outcomes. Findings indicated that students in the intervention endorsed 31% fewer substance use behaviours (incidence rate ratio [IRR] = 0.69), 37% fewer violence-related behaviours (IRR = 0.63) and 41% fewer bullying behaviours (IRR = 0.59), respectively, compared to students in the control schools. Reduction in reported disruptive behaviours was of a similar magnitude (27%, IRR = 0.73), but was not statistically significant. These results replicate findings of an earlier randomised trial of the PA programme and extend evidence of its effectiveness to youth attending large urban school systems.

Impact of a social-emotional and character development program on school-level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched-pair, cluster randomized, controlled trial.

This paper reports the effects of a comprehensive elementary school-based social-emotional and character education program on school-level achievement, absenteeism, and disciplinary outcomes utilizing a matched-pair, cluster randomized, controlled design. The Positive Action Hawai'i trial included 20 racially/ethnically diverse schools (mean enrollment = 544) and was conducted from the 2002-03 through the 2005-06 academic years. Using school-level archival data, analyses comparing change from baseline (2002) to one-year post trial (2007) revealed that intervention schools scored 9.8% better on the TerraNova (2nd ed.) test for reading and 8.8% on math; 20.7% better in Hawai'i Content and Performance Standards scores for reading and 51.4% better in math; and that intervention schools reported 15.2% lower absenteeism and fewer suspensions (72.6%) and retentions (72.7%). Overall, effect sizes were moderate to large (range 0.5-1.1) for all of the examined outcomes. Sensitivity analyses using permutation models and random-intercept growth curve models substantiated results. The results provide evidence that a comprehensive school-based program, specifically developed to target student behavior and character, can positively influence school-level achievement, attendance, and disciplinary outcomes concurrently.

Use of a social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary-school students in Hawaii.

OBJECTIVES: We assessed the effectiveness of a 5-year trial of a comprehensive school-based program designed to prevent substance use, violent behaviors, and sexual activity among elementary-school students. METHODS: We used a matched-pair, cluster-randomized, controlled design, with 10 intervention schools and 10 control schools. Fifth-graders (N = 1714) self-reported on lifetime substance use, violence, and voluntary sexual activity. Teachers of participant students reported on student (N = 1225) substance use and violence. RESULTS: Two-level random-effects count models (with students nested within schools) indicated that student-reported substance use (rate ratio [RR] = 0.41; 90% confidence interval [CI] = 0.25, 0.66) and violence (RR = 0.42; 90% CI = 0.24, 0.73) were significantly lower for students attending intervention schools. A 2-level random-effects binary model indicated that sexual activity was lower (odds ratio = 0.24; 90% CI = 0.08, 0.66) for intervention students. Teacher reports substantiated the effects seen for student-reported data. Dose-response analyses indicated that students exposed to the program for at least 3 years had significantly lower rates of all negative behaviors. CONCLUSIONS: Risk-related behaviors were substantially reduced for students who participated in the program, providing evidence that a comprehensive school-based program can have a strong beneficial effect on student behavior.

Longitudinal patterns of binge drinking among first year college students with a history of tobacco use.

BACKGROUND: Underage heavy episodic drinking is a major contributor to alcohol-related morbidity/mortality. Reports indicate underage binge drinking among college students is widespread and has remained stable over the past decade. This study describes individual characteristics and calendar-specific events associated with binge drinking episodes over the course of freshman college academic year (2002-2003). METHODS: Students (N=827, age 18 years), with a prior history of tobacco use, attending a large Midwest university completed weekly web-based surveys on the number of drinks consumed for each of the past 7 days over the duration of 35 consecutive weeks (avg. number of weeks reported 16.0+/-10.5). RESULTS: Average prevalence of binge episodes across the academic year was 17.2+/-14.4%, 23.6+/-8.3%, and 66.3+/-11.2% for weekdays, Thursdays, and weekend days, respectively. Two-level random effects logit survival models for repeated events indicated the prevalence of weekday and Thursday binge drinking was associated with specific university/community events (Local festival odds ratio [OR] 6.03, 95% confidence interval [CI] 4.34-8.36), holidays (New Year's Eve OR 18.48, CI 12.83-26.63), and academic breaks (Spring Break OR 6.45, CI 4.57-9.08). Expected associations of younger age of first heavy drinking, past 12-month drinking, and experiencing negative consequences from heavy drinking were observed. CONCLUSIONS: Although individual characteristics were related to engaging in a binge episode, binge episodes were strongly associated with time-specific calendar events. Effective interventions to prevent immediate and long-term health consequences associated with binge drinking should consider environmental and institutional policy-level controls to reduce high levels of binge drinking on college campuses connected with holidays and university/community events.

Health worry, physical activity participation, and walking difficulty among older adults: a mediation analysis.

This study examined the effect of health worry (i.e., cognitive aspect of anxiety resulting from concern for health) on walking difficulty in a nationally representative sample (N = 7,527) of older adults (M age = 76.83 years). The study further tested whether physical activity mediates the effect of health worry on walking difficulty in a 6-year follow-up design. Results of a mediation analysis using structural equation modeling showed that people with a high degree of health worry engaged in less physical activity (beta = -.24, p < .001), and people who participated in less physical activity were more likely to report walking difficulty at the 6-year follow-up (beta = -.22, p < .001). There was a significant indirect effect from health worry to walking difficulty through physical activity (beta = .05, p < .001), controlling for demographic, psychosocial, and health related factors. Results suggested that inducing threat and worry may not be effective for physical activity promotion in the older population. More promising coping and regulation strategies are discussed.

School climate and teachers' beliefs and attitudes associated with implementation of the positive action program: a diffusion of innovations model.

Teacher- and school-level factors influence the fidelity of implementation of school-based prevention and social character and development (SACD) programs. Using a diffusion of innovations framework, the relationships among teacher beliefs and attitudes towards a prevention/SACD program and the influence of a school's administrative support and perceptions of school connectedness, characteristics of a school's climate, were specified in two cross-sectional mediation models of program implementation. Implementation was defined as the amount of the programs' curriculum delivered (e.g., lessons taught), and use of program-specific materials in the classroom (e.g., ICU boxes and notes) and in relation to school-wide activities (e.g., participation in assemblies). Teachers from 10 elementary schools completed year-end process evaluation reports for year 2 (N = 171) and 3 (N = 191) of a multi-year trial. Classroom and school-wide material usage were each favorably associated with the amount of the curriculum delivered, which were associated with teachers' attitudes toward the program which, in turn, were related to teachers' beliefs about SACD. These, in turn, were associated with teachers' perceptions of school climate. Perceptions of school climate were indirectly related to classroom material usage and both indirectly and directly related to the use of school-wide activities. Program developers need to consider the importance of a supportive environment on program implementation and attempt to incorporate models of successful school leadership and collaboration among teachers that foster a climate promoting cohesiveness, shared visions, and support.