A Synthesis of the Conceptualization and Measurement of Implementation Fidelity in Mathematics Intervention Research.

In intervention studies, high rates of implementation fidelity are important markers of a study's success; however, the definition of implementation fidelity is both complex and dynamic. In this synthesis, we examined the dimensions of implementation fidelity measured and reported in 99 studies in which researchers utilized a mathematics intervention for elementary students (i.e., Grades 1-5). We examined implementation fidelity following recommendations made by Dane and Schneider (1998), O'Donnell (2008), and DeFouw et al. (2009) to capture a comprehensive representation of the implementation fidelity data collected and reported within mathematics intervention studies. We organized our conceptualization of implementation fidelity into four overarching categories and nine dimensions within those categories: intervention design (i.e., theories of change and logistics), fidelity of implementor (i.e., adherence, quality of delivery, dosage, and implementor knowledge or experience), student engagement, and treatment analysis (i.e., treatment differentiation and analysis of implementation fidelity). Overall, findings indicate many author teams reported adherence data and dosage data, but significantly fewer studies reported quality of delivery data, student engagement data, or treatment differentiation data. In addition, author teams were more likely to report at least one form of logistics, such as implementor support, than theories of change. Implications for research and practice are discussed.

Using Direct Observation to Document "Practice-Based Evidence" of Evidence-Based Mathematics Instruction.

Implementation of evidence-based practices (EBPs) is paramount to students' development of mathematics proficiency. This study investigated "practice-based evidence" of interventionists' actual use of explicit mathematics instruction, a well-established EBP. Specifically, this study analyzed direct observation data collected in a federally funded efficacy trial involving a Tier 2 first-grade mathematics intervention to examine whether the quantity and quality of explicit mathematics instruction was associated with the mathematics outcomes of 470 first-grade students with or at risk for mathematics learning disabilities. Associations between group-level pretreatment skill levels and the quality and quantity of explicit mathematics instructional practices used in the intervention were also explored. Findings suggested significant associations between positive gains in student mathematics outcomes and (a) lower rates of incorrectly answered mathematics-focused questions, and (b) the rate in which interventionists delivered group-level practice opportunities and offered academic feedback. Significant associations were also found between initial student mathematics performance and rates of student errors and the quality of explicit instruction. Implications for using direct observation to document enacted EBPs are discussed.

Kindergarteners at Risk for Severe Mathematics Difficulties: Investigating Tipping Points of Core Mathematics Instruction.

A concerning number of students enter kindergarten facing an intractable variation of mathematics difficulties (MD). This study investigated the impact of an explicit, core kindergarten mathematics program on the mathematical outcomes of kindergartners who demonstrated risk for severe MD at kindergarten entry and examined whether these students improved from a category of high MD risk (i.e., <10th percentile) to a lower risk of MD (i.e., norm-referenced performance at or above the 10th, 20th, 30th, and 40th percentiles) between the fall and spring of kindergarten. Differential response to the program based on the classroom-level proportion of students with severe MD was also explored. A total of 795 kindergarteners with severe MD from 122 classrooms were included in the analyses. Results suggested students with severe MD in treatment classrooms improved from fall to spring at a greater rate than their control peers. Treatment students also demonstrated higher rates of improvement from below the 10th percentile to a performance at or above the 20th, 30th, and 40th percentiles across the school year. No evidence of differential efficacy of the program by the classroom-level proportion of students with severe MD was found. Implications for using explicit mathematics programs to thwart the onset of severe MD among academically vulnerable students are discussed.

A Synthesis of Elementary Mathematics Interventions: Comparisons of Students With Mathematics Difficulty With and Without Comorbid Reading Difficulty.

In this synthesis, we reviewed 65 studies involving elementary students (i.e., grades 1-5) identified with mathematics difficulty (MD) in which authors implemented a mathematics intervention. Of these studies, we identified 33 group designs, 9 quasi-experimental designs, and 23 single-case designs. We aimed to synthesize performance differences between students with MD with and without reading difficulty (RD). We identified three categories of students for analysis: Students with MD+RD, MD-alone, or MD-nonspecified (i.e., no reading information provided). Overall, 80% of studies included students with MD-nonspecified, and the interventions for these students demonstrated strong effects. For the limited number of studies with students with MD+RD or MD-alone, intervention effects were strong for students with MD+RD and variable for students with MD-alone. In the three studies directly comparing the performance of students with MD+RD versus MD-alone, we noted differential patterns of performance. To tailor interventions to student need, more research must be conducted to understand whether students with MD with variable reading profiles respond differentially to mathematics intervention.