The Design of Making Pre-K count and High Fives: Two-Stage, Multiyear Random Assignment at Different Levels.

The Making Pre-K Count and High 5s studies represent a recent application of a phased two-stage, multi-level design, which was used to examine the effects of two aligned math programs implemented in early childhood settings. The purpose of this paper is to describe the challenges encountered in implementing this two-stage design and to describe approaches to resolving them. We then present a set of sensitivity analyses the study team used to examine the robustness of the findings. During the pre-K year, pre-K centers were randomly assigned either to receive an evidence-based early math curriculum and associated professional development (Making Pre-K Count) or to a pre-K-as-usual control condition. In the kindergarten year, students who had been in Making Pre-K Count program classrooms in pre-K were then individually randomly assigned within schools to small-group supplemental math clubs that were designed to sustain the gains from the pre-K program, or to a business-as-usual kindergarten experience. Making Pre-K Count took place in 69 pre-K sites, comprising 173 classrooms across New York City. High 5s took place in the 24 sites that were part of the public school treatment arm of the Making Pre-K Count study and included 613 students. The study focuses on the effect of the Making Pre-K Count and High 5s programs on children's math skills at the end of kindergarten as measured by two instruments, the Research-Based Early Math Assessment-Kindergarten (REMA-K) and the Woodcock-Johnson Applied Problems test. The multi-armed design, while logistically and analytically challenging, balanced multiple considerations of power, the number of research questions that could be answered, and efficiency of resources. Robustness checks suggest that the design created groups that were both meaningfully and statistically equivalent. Decisions to use a phased multi-armed design should consider both its strengths and weaknesses. While the design allows for a more flexible, expansive research study, it also introduces complexities that need to be addressed both logistically and analytically.

Promoting EF With Preschool Interventions: Lessons Learned From 15 Years of Conducting Large-Scale Studies.

In the past two decades, a growing number of early childhood interventions that aim to improve school readiness have also targeted children's executive function (EF), building on the theory that promoting EF skills in preschool may play a key role in reducing the substantial gaps in school readiness and later achievement associated with family income. Despite the expansion of school readiness interventions across preschool, research evidence is mixed regarding what works to promote EF development and the impact of these interventions on children's EF skills, and subsequently, their academic and behavioral outcomes. This paper reviews four intervention approaches designed to support school readiness that may also improve children's EF skills by: (a) encouraging adaptive classroom behaviors, (b) improving social-emotional learning, (c) promoting play and direct training of EF skills, and (d) improving cognitive skills related to EF. We describe program effects from rigorous trials testing these approaches, including summarizing the takeaways from four large-scale intervention research studies conducted by the authors, involving over 5,000 children. We conclude by exploring open questions for the field and future directions for research and intervention program development and refinement.

Designing Studies to Test Causal Questions About Early Math: The Development of Making Pre-K Count.

A growing literature has demonstrated that early math skills are associated with later outcomes for children. This research has generated interest in improving children's early math competencies as a pathway to improved outcomes for children in elementary school. The Making Pre-K Count study was designed to test the effects of an early math intervention for preschoolers. Its design was unique in that, in addition to causally testing the effects of early math skills, it also allowed for the examination of a number of additional questions about scale-up, the influence of contextual factors and the counterfactual environment, the mechanism of long-term fade-out, and the role of measurement in early childhood intervention findings. This chapter outlines some of the design considerations and decisions put in place to create a rigorous test of the causal effects of early math skills that is also able to answer these questions in early childhood mathematics and intervention. The study serves as a potential model for how to advance science in the fields of preschool intervention and early mathematics.