Bringing Assessment-to-Instruction (A2i) Technology to Scale: Exploring the Process from Development to Implementation.

Bringing effective, research-based literacy interventions into the classroom is challenging, especially given the cultural and linguistic diversity of today's classrooms. We examined the promise of Assessment-to-Instruction (A2i) technology redesigned to be used at scale to support teachers' implementation of the individualized student instruction (ISI) intervention from kindergarten through third grade. In seven randomized controlled trials, A2i and ISI have demonstrated efficacy. However, the research version of A2i was not scalable. In order to bring A2i to scale in schools serving linguistically diverse students, we carried out the current study across two phases. This study represents both an exploration of what it takes to bring an educational intervention to scale (Phase 1) and a quasi-experiment on the literacy outcomes of learners whose teachers used the technology (Phase 2). We integrated assessments of vocabulary, word decoding, and reading comprehension; revised the A2i algorithms to account for the constellation of skills English learners (ELs) bring to the classroom; updated the user interfaces and added new graphic features; and improved bandwidth and stability of the technology. Findings were mixed, including several non-significant results, a marginally significant intent-to-treat effect on word reading in kindergarten and first grade for English monolingual students and ELs, and one significant interaction effect, which suggested ELs and students with less developed reading skills in second and third grade benefitted most from the intervention. With some caution, we conclude that A2i demonstrates potential to be used at scale and promise of effectiveness for improving code-focused skills for diverse learners.

Observing individual children in early childhood classrooms using Optimizing Learning Opportunities for Students (OLOS): A feasibility study.

In this study, we present a newly developed observational system, Optimizing Learning Opportunities for Students (OLOS). OLOS is designed to elucidate the learning opportunities afforded to individual children within early childhood classrooms and as they transition to formal schooling (kindergarten through third grade). OLOS records the time spent in different types of learning opportunities (e.g., play, literacy, math) and the frequency of specific discourse moves children and teachers use (child talk and teacher talk). Importantly, it is being designed to be used validly and reliably by practitioners. Using OLOS, we explored individual children's experiences (n = 68 children in 12 classrooms) in four different types of early childhood programs; state-funded, state-funded PK serving children with disabilities, Head Start, and a tuition-based (non-profit) preschool. Results of our feasibility study revealed that we could feasibly and reliably use OLOS in these very different kinds of pre-kindergarten programs with some changes. OLOS provided data that aligned with our hypotheses and that our practitioner partners found useful. In analysing the observations, we found that individual children's learning opportunities varied significantly both within and between classrooms. In general, we observed that most of the PK day (or half day) was spent in language and literacy activities and non-instructional activities (e.g., transitions). Very little time in math and science was observed yet children were generally more likely to actively participate (i.e., more child talk) during academic learning opportunities (literacy, math, and science). The frequency of teacher talk also varied widely between classrooms and across programs. Plus, the more teacher talk we observed, the more likely we were to observe child talk. Our long-term aim is that OLOS can inform policy and provide information that supports practitioners in meeting the learning and social-behavioral needs of the children they serve.

Building Word Knowledge, Learning Strategies, and Metacognition with the Word-Knowledge E-Book.

Many children fail to comprehend what they read because they do not monitor their understanding, which requires making accurate judgements of what they know and then employing repair strategies when comprehension fails. Relying on research from learning science and cognitive and developmental psychology, we developed the Word Knowledge e-Book (WKe-Book) to improve children's calibration of their word knowledge, strategy use, and word knowledge overall; skills which are associated with reading comprehension. The WKe-Book, which is read on a tablet computer, is a choose-your-own adventure book where choices require choosing between two rare words (e.g., cogitate vs. procrastinate). Depending on the word chosen, the story follows a different plot. There are also embedded comprehension questions where students receive immediate feedback with consequences for incorrect answers, such as being sent back to reread a few pages. In a randomized controlled trial, we tested whether students (N = 603 in 25 third through fifth grade classrooms in Arizona in the US) reading the WKe-Book would demonstrate improved word knowledge, strategy use, and word knowledge calibration. Classrooms were randomly assigned to read the WKe-Book immediately (treatment) or later (delayed-treatment control), and within classrooms, students were randomly assigned to either participate in a 15-minute weekly book club (book club treatment) or to read the WKe-Book independently with no book club (no book club control). Results revealed a significant treatment effect of the WKe-Book on students' word knowledge, word knowledge calibration, and strategy use, which predicted student performance on standardized reading comprehension and vocabulary measures. The effects were greater for students who participated in weekly book clubs compared to students in the no book club control. These findings suggest that the affordances offered by technology, which are unavailable in paper-based books, can support students' development of metacognition, including word knowledge calibration, strategy use, and word learning skills.

Study of Telerobot Personalization for Children: Exploring Qualitative Coding of Artwork.

Social telepresence robots (i.e., telerobots) are used for social and learning experiences by children. However, most (if not all) commercially available telerobot bodies were designed for adults in corporate or healthcare settings. Due to an adult-focused market, telerobot design has typically not considered important factors such as age and physical aspect in the design of robot bodies. To better understand how peer interactants can facilitate the identities of remote children through personalization of robot bodies, we conducted an exploratory study to evaluate collaborative robot personalization. In this study, child participants (N=28) attended an interactive lesson on robots in our society. After the lesson, participants interacted with two telerobots for personalization activities and a robot fashion show. Finally, participants completed an artwork activity on robot design. Initial findings from this study will inform our continued work on telepresence robots for virtual inclusion and improved educational experiences of remote children and their peers.

Using assessment to individualize early mathematics instruction.

Accumulating evidence suggests that assessment-informed personalized instruction, tailored to students' individual skills and abilities, is more effective than more one-size-fits-all approaches. In this study, we evaluate the efficacy of Individualizing Student Instruction in Mathematics (ISI-Math) compared to Reading (ISI-Reading) where classrooms were randomly assigned to ISI-Math or ISI-Reading. The literature on child characteristics X instruction or skill X treatment interaction effects point to the complexities of tailoring instruction for individual students who present with constellations of skills. Second graders received mathematics instruction in small flexible learning groups based on their assessed learning needs. Results of the study (n=32 teachers, 370 students) revealed significant treatment effects on standardized mathematics assessments. With effect sizes (d) of 0.41-0.60, we show that we can significantly improve 2nd graders' mathematics achievement, including for children living in poverty, by using assessment data to individualize the mathematics instruction they receive. The instructional regime, ISI-Math, was implemented by regular classroom teachers and it led to about a 4-month achievement advantage on standardized mathematics tests when compared to students in control classrooms. These results were realized within one school year. Moreover, treatment effects were the same regardless of school-level poverty and students' gender, initial mathematics or vocabulary scores.