Bringing Assessment-to-Instruction (A2i) Technology to Scale: Exploring the Process from Development to Implementation.

Bringing effective, research-based literacy interventions into the classroom is challenging, especially given the cultural and linguistic diversity of today's classrooms. We examined the promise of Assessment-to-Instruction (A2i) technology redesigned to be used at scale to support teachers' implementation of the individualized student instruction (ISI) intervention from kindergarten through third grade. In seven randomized controlled trials, A2i and ISI have demonstrated efficacy. However, the research version of A2i was not scalable. In order to bring A2i to scale in schools serving linguistically diverse students, we carried out the current study across two phases. This study represents both an exploration of what it takes to bring an educational intervention to scale (Phase 1) and a quasi-experiment on the literacy outcomes of learners whose teachers used the technology (Phase 2). We integrated assessments of vocabulary, word decoding, and reading comprehension; revised the A2i algorithms to account for the constellation of skills English learners (ELs) bring to the classroom; updated the user interfaces and added new graphic features; and improved bandwidth and stability of the technology. Findings were mixed, including several non-significant results, a marginally significant intent-to-treat effect on word reading in kindergarten and first grade for English monolingual students and ELs, and one significant interaction effect, which suggested ELs and students with less developed reading skills in second and third grade benefitted most from the intervention. With some caution, we conclude that A2i demonstrates potential to be used at scale and promise of effectiveness for improving code-focused skills for diverse learners.

Using Assessment to Improve the Accuracy of Teachers' Perceptions of Students' Academic Competence.

Teachers' perceptions of their students' academic skills can affect students' achievement and may be influenced by unrelated student characteristics such as socioeconomic status (SES). In this ad hoc randomized controlled trial, teachers (n = 28) were randomly assigned to receive training on using assessment to guide literacy instruction, Assessment-to-Instruction (A2i), or on Math PALS (control). Teachers rated students' (n = 446) academic competence. A2i teachers' ratings did not vary by SES, and their ratings correlated more strongly with students' literacy and mathematics assessment scores compared with those of the control teachers. Control teachers generally underestimated lower SES students' academic competence; underestimation was greater at more affluent schools. Teachers' ratings of students' academic competence predicted reading and mathematics outcomes. Thoughtful use of assessments to guide instruction appeared to improve the precision of teachers' ratings of students' academic competence, improve student outcomes, and reduce potential teacher biases about children from higher-poverty families.

Predicting Second and Third Graders' Reading Comprehension Gains: Observing Students' and Classmates Talk during Literacy Instruction using COLT.

This paper introduces a new observation system that is designed to investigate students' and teachers' talk during literacy instruction, Creating Opportunities to Learn from Text (COLT). Using video-recorded observations of 2nd-3rd grade literacy instruction (N=51 classrooms, 337 students, 151 observations), we found that nine types of student talk ranged from using non-verbal gestures to generating new ideas. The more a student talked, the greater were his/her reading comprehension (RC) gains. Classmate talk also predicted RC outcomes (total effect size=0.27). We found that 11 types of teacher talk ranged from asking simple questions to encouraging students' thinking and reasoning. Teacher talk predicted student talk but did not predict students' RC gains directly. Findings highlight the importance of each student's discourse during literacy instruction, how classmates' talk contributes to the learning environments that each student experiences, and how this affects RC gains, with implications for improving the effectiveness of literacy instruction.

Observing individual children in early childhood classrooms using Optimizing Learning Opportunities for Students (OLOS): A feasibility study.

In this study, we present a newly developed observational system, Optimizing Learning Opportunities for Students (OLOS). OLOS is designed to elucidate the learning opportunities afforded to individual children within early childhood classrooms and as they transition to formal schooling (kindergarten through third grade). OLOS records the time spent in different types of learning opportunities (e.g., play, literacy, math) and the frequency of specific discourse moves children and teachers use (child talk and teacher talk). Importantly, it is being designed to be used validly and reliably by practitioners. Using OLOS, we explored individual children's experiences (n = 68 children in 12 classrooms) in four different types of early childhood programs; state-funded, state-funded PK serving children with disabilities, Head Start, and a tuition-based (non-profit) preschool. Results of our feasibility study revealed that we could feasibly and reliably use OLOS in these very different kinds of pre-kindergarten programs with some changes. OLOS provided data that aligned with our hypotheses and that our practitioner partners found useful. In analysing the observations, we found that individual children's learning opportunities varied significantly both within and between classrooms. In general, we observed that most of the PK day (or half day) was spent in language and literacy activities and non-instructional activities (e.g., transitions). Very little time in math and science was observed yet children were generally more likely to actively participate (i.e., more child talk) during academic learning opportunities (literacy, math, and science). The frequency of teacher talk also varied widely between classrooms and across programs. Plus, the more teacher talk we observed, the more likely we were to observe child talk. Our long-term aim is that OLOS can inform policy and provide information that supports practitioners in meeting the learning and social-behavioral needs of the children they serve.

The Relations Between Children's Comprehension Monitoring and Their Reading Comprehension and Vocabulary Knowledge: An Eye-Movement Study.

Poor reading comprehension may be due to having ineffective comprehension monitoring, the metacognitive process of evaluating and regulating comprehension. When comprehension breaks down due to an inconsistency either at the word-level (e.g., due to an unfamiliar word) or at the sentence-level (e.g., due to contradictory information), readers may identify the misunderstanding and take steps to regulate their comprehension. In the current study, we utilized two eye-movement tasks (one newly developed) to examine comprehension monitoring in third through fifth grade students (n = 123), when confronted with word- and sentence-level inconsistencies, by measuring the amount of time they read (gaze duration) and reread the target inconsistent words. We investigated how this skill may be associated with individual differences in age, reading comprehension ability, and vocabulary knowledge. The results showed that generally, all students detected the word-level inconsistencies, indicated by longer gaze durations, and attempted to regulate their comprehension after detecting both word- and sentence-level inconsistencies, as indicated by more time spent rereading. Students with stronger reading comprehension (when controlling for their vocabulary), and stronger vocabulary knowledge (when controlling for their reading comprehension) were more likely to attempt regulating their comprehension. In general, the difference between the control words and the inconsistent words was smaller for third graders and larger for fourth and fifth graders, which we argue indicates greater levels of comprehension monitoring - specifically employing repair strategies. With eye-tracking technology becoming more accessible, these tasks may be useful in assessing children's reading processes to better understand at which level of comprehension monitoring they may be struggling, which in return will allow us to develop more individualized instruction for all readers.

Modality and Interrelations Among Language, Reading, Spoken Phonological Awareness, and Fingerspelling.

Better understanding of the mechanisms underlying early reading skills can lead to improved interventions. Hence, the purpose of this study was to examine multivariate associations among reading, language, spoken phonological awareness, and fingerspelling abilities for three groups of deaf and hard-of-hearing (DHH) beginning readers: those who were acquiring only spoken English (n = 101), those who were visual learners and acquiring sign (n = 131), and those who were acquiring both (n = 104). Children were enrolled in kindergarten, first, or second grade. Within-group and between-group confirmatory factor analysis showed that there were both similarities and differences in the abilities that underlie reading in these three groups. For all groups, reading abilities related to both language and the ability to manipulate the sublexical features of words. However, the groups differed on whether these constructs were based on visual or spoken language. Our results suggest that there are alternative means to learning to read. Whereas all DHH children learning to read rely on the same fundamental abilities of language and phonological processing, the modality, levels, and relations among these abilities differ.

Building Word Knowledge, Learning Strategies, and Metacognition with the Word-Knowledge E-Book.

Many children fail to comprehend what they read because they do not monitor their understanding, which requires making accurate judgements of what they know and then employing repair strategies when comprehension fails. Relying on research from learning science and cognitive and developmental psychology, we developed the Word Knowledge e-Book (WKe-Book) to improve children's calibration of their word knowledge, strategy use, and word knowledge overall; skills which are associated with reading comprehension. The WKe-Book, which is read on a tablet computer, is a choose-your-own adventure book where choices require choosing between two rare words (e.g., cogitate vs. procrastinate). Depending on the word chosen, the story follows a different plot. There are also embedded comprehension questions where students receive immediate feedback with consequences for incorrect answers, such as being sent back to reread a few pages. In a randomized controlled trial, we tested whether students (N = 603 in 25 third through fifth grade classrooms in Arizona in the US) reading the WKe-Book would demonstrate improved word knowledge, strategy use, and word knowledge calibration. Classrooms were randomly assigned to read the WKe-Book immediately (treatment) or later (delayed-treatment control), and within classrooms, students were randomly assigned to either participate in a 15-minute weekly book club (book club treatment) or to read the WKe-Book independently with no book club (no book club control). Results revealed a significant treatment effect of the WKe-Book on students' word knowledge, word knowledge calibration, and strategy use, which predicted student performance on standardized reading comprehension and vocabulary measures. The effects were greater for students who participated in weekly book clubs compared to students in the no book club control. These findings suggest that the affordances offered by technology, which are unavailable in paper-based books, can support students' development of metacognition, including word knowledge calibration, strategy use, and word learning skills.

Profiles of Foundational Learning Components among First Graders.

School readiness includes a constellation of skills and behaviors, such as social and emotional development, language and literacy, and self-regulation that provide a basis critical for classroom participation and learning. Whereas it has been well-established that students who enter kindergarten with weaknesses in language and literacy are more likely to struggle academically, less research has focused on the variability and educational impact of other foundational learning components, including internalizing and externalizing behaviors, particularly in first grade. This study used latent profile analysis to identify the following four subgroups (profiles) of students, using foundational learning components, in a sample of first graders (n = 324): Emergent Hyperactive, Externalizing, Generally Good Students, and Internalizing. Latent class growth analysis illustrated significant differences in the average rate of growth in literacy skills from the beginning to the end of first grade across the four profiles, after controlling for gender and socioeconomic status. Findings indicated the greatest growth in literacy skills for students in the Externalizing profile and the least amount of vocabulary growth for students in the Emergent Hyperactive profile followed by the Internalizing profile. Educational implications of how researchers and educators might consider students' individual differences across profiles of foundational learning components to inform ways to support development and learning in the classroom are discussed.

Schooling Effects on Early Literacy Skills of Young Deaf and Hard of Hearing Children.

Already well documented for hearing children, schooling's effects on early literacy skills for young students who are deaf or hard of hearing (DHH) were examined for the first time in the present study. Piecewise growth curve modeling was used to describe 3-, 4-, and 5-year-old students' growth in phonological awareness, letter-word identification, and vocabulary during 2 years of schooling and the intervening summer (N = 56). Amplification mode was cochlear implants for 45% of the sample and hearing aids for 54%. Classroom communication mode was spoken language only (for 61%) or sign language (39%). Across all skills, significant growth occurred during the 2 years of schooling but not during the summer. These findings underscore early education's importance in promoting DHH children's critical early skills. Universal preschool intervention, including during summer, may be important in ensuring that DHH children have an adequate foundation when schooling begins.

Using Technology and Assessment to Personalize Instruction: Preventing Reading Problems.

Children who fail to learn to read proficiently are at serious risk of referral to special education, grade retention, dropping out of high school, and entering the juvenile justice system. Accumulating research suggests that instruction regimes that rely on assessment to inform instruction are effective in improving the implementation of personalized instruction and, in turn, student learning. However, teachers find it difficult to interpret assessment results in a way that optimizes learning opportunities for all of the students in their classrooms. This article focuses on the use of language, decoding, and comprehension assessments to develop personalized plans of literacy instruction for students from kindergarten through third grade, and A2i technology designed to support teachers' use of assessment to guide instruction. Results of seven randomized controlled trials demonstrate that personalized literacy instruction is more effective than traditional instruction, and that sustained implementation of personalized literacy instruction first through third grade may prevent the development of serious reading problems. We found effect sizes from .2 to .4 per school year, which translates into about a 2-month advantage. These effects accumulated from first through third grade with a large effect size (d = .7) equivalent to a full grade-equivalent advantage on standardize tests of literacy. These results demonstrate the efficacy of technology-supported personalized data-driven literacy instruction to prevent serious reading difficulties. Implications for translational prevention research in education and healthcare are discussed.

Associations among teachers' depressive symptoms and students' classroom instructional experiences in third grade.

Recent studies have established connections among teachers' mental health and student outcomes, however there is limited understanding of how these teacher characteristics manifest in the classroom to affect students. The present study informed this gap by examining the associations among third grade teachers' (N = 32) self-reported symptoms of clinical depression and their students' (N = 326) classroom instructional experiences. Eight student experiences described by the Individualizing Student Instruction framework were investigated, including academic instruction facilitated by the teacher in various student groupings, students' independent and group work, teachers' planning/organizing instruction, and students' time off-task and in transitions. Multilevel modeling revealed negative associations between teachers' depressive symptoms and (a) teacher-facilitated academic instruction provided to the whole class and (b) teachers' planning/organizing instruction. Results suggest that teachers experiencing more symptoms may under-utilize instructional approaches that require more effort on their part. We discussed the implications of our findings for students' academic and social-emotional learning, and the potential benefits of incorporating mental health support components into teacher training and professional development aimed at improving instructional practices.

Direct and Reciprocal Effects among Social Skills, Vocabulary, and Reading Comprehension in First Grade.

Social skills and vocabulary are important areas of development involved in early reading achievement, yet little attention has been given to understanding the dynamic associations among them during the elementary years. This study examined the relations among three dimensions of social skills-cooperation, assertion, and self-control-vocabulary and developing reading comprehension (RC) skills in a longitudinal sample of first graders (n = 468). Using Structural Equation Modeling, reciprocal effects were observed between vocabulary and RC as well as direct effects among social skills, vocabulary, and RC after controlling for the influence of problem behaviors. This study highlights the reciprocal nature of students' vocabulary and RC skills as well as provides preliminary evidence suggesting that social skills play a role in developing vocabulary and RC skills, and further, vocabulary and RC skills play a role in social development during middle childhood. Implications for policy and practice are discussed.

Using assessment to individualize early mathematics instruction.

Accumulating evidence suggests that assessment-informed personalized instruction, tailored to students' individual skills and abilities, is more effective than more one-size-fits-all approaches. In this study, we evaluate the efficacy of Individualizing Student Instruction in Mathematics (ISI-Math) compared to Reading (ISI-Reading) where classrooms were randomly assigned to ISI-Math or ISI-Reading. The literature on child characteristics X instruction or skill X treatment interaction effects point to the complexities of tailoring instruction for individual students who present with constellations of skills. Second graders received mathematics instruction in small flexible learning groups based on their assessed learning needs. Results of the study (n=32 teachers, 370 students) revealed significant treatment effects on standardized mathematics assessments. With effect sizes (d) of 0.41-0.60, we show that we can significantly improve 2nd graders' mathematics achievement, including for children living in poverty, by using assessment data to individualize the mathematics instruction they receive. The instructional regime, ISI-Math, was implemented by regular classroom teachers and it led to about a 4-month achievement advantage on standardized mathematics tests when compared to students in control classrooms. These results were realized within one school year. Moreover, treatment effects were the same regardless of school-level poverty and students' gender, initial mathematics or vocabulary scores.

Relations between third grade teachers' depressive symptoms and their feedback to students, with implications for student mathematics achievement.

Recent studies have observed connections among teachers' depressive symptoms and student outcomes; however, the specific mechanisms through which teachers' mental health characteristics operate in the classroom remain largely unknown. The present study used student-level observation methods to examine the relations between third-grade teachers' (N = 32) depressive symptoms and their academic feedback to students (N = 310) and sought to make inferences about how these factors might influence students' mathematics achievement. A novel observational tool, the Teacher Feedback Coding System-Academic (TFCS-A), was used that assesses feedback across 2 dimensions-teacher affect and instructional strategy, which have been shown to be important to student learning. Multilevel exploratory factor analysis of TFCS-A data suggested 2 primary factors: positive feedback and neutral/negative feedback. Hierarchical linear modeling revealed that positive feedback was related to higher math achievement among students who began the year with weaker math skills and that teachers who reported more depressive symptoms less frequently provided this positive feedback. Results offer new information about a type of instruction that may be affected by teachers' depressive symptoms and inform efforts aimed at improving teachers' instructional interactions with students. (PsycINFO Database Record

Examining the Efficacy of Targeted Component Interventions on Language and Literacy for Third and Fourth Graders Who are at Risk of Comprehension Difficulties.

Testing a component model of reading comprehension in a randomized controlled trial, we evaluated the efficacy of four different interventions that were designed to target components of language and metacognition that predict children's reading comprehension: vocabulary, listening comprehension, comprehension of literate language, academic knowledge, and comprehension monitoring. Third- and fourth-graders with language skills falling below age expectations participated (N = 645). Overall, the component interventions were only somewhat effective in improving the targeted skills, compared to a business-as-usual control (g ranged from -.14 to .33), and no main effects were significant after correcting for multiple comparisons. Effects did not generalize to other language skills or to students' reading comprehension. Moreover, there were child-characteristic-by-treatment interaction effects. For example, the intervention designed to build sensorimotor mental representations was more effective for children with weaker vocabulary skills. Implications for component models of reading and interventions for children at risk of reading comprehension difficulties are discussed.

A Lattice Model of the Development of Reading Comprehension.

In this article, I present a developmental model of how children learn to comprehend what they read, which builds on current models of reading comprehension and integrates findings from instructional research and evidence-based models of development in early and middle childhood. The lattice model holds that children's developing reading comprehension is a function of the interacting, reciprocal, and bootstrapping effects of developing text-specific, linguistic, and social-cognitive processes, which interact with instruction as child-characteristic-by-instruction (CXI) interaction effects. The processes develop over time and in the context of classroom, home, peer, community, and other influences to affect children's development of proficient reading comprehension. I first describe models of reading comprehension. I then review the basic processes in the model, the role of instruction, and CXI interactions in the context of the lattice model. I then discuss implications for instruction and research.

Classroom quality as a predictor of first graders' time in non-instructional activities and literacy achievement.

This study investigated how quality of the classroom learning environment influenced first grade students' (n=533) time spent in two non-instructional classroom activities (off-task and in transition) and their subsequent literacy outcomes. Hierarchical linear modeling revealed that higher classroom quality was related to higher student performance in reading comprehension and expressive vocabulary. Further, classroom quality predicted the amount of time students spent off-task and in transitions in the classroom, with slopes of change across the year particularly impacted. Mediation effects were detected in the case of expressive vocabulary such that the influence of classroom quality on students' achievement operated through students' time spent in these non-instructional activities. Results highlight the importance of overall classroom quality to how students navigate the classroom environment during learning opportunities, with subsequent literacy achievement impacted. Implications for policy and educational practices are discussed.

Reciprocal Effects of Self-Regulation, Semantic Knowledge, and Reading Comprehension in Early Elementary School.

Many assume that cognitive and linguistic processes, such as semantic knowledge (SK) and self-regulation (SR), subserve learned skills like reading. However, complex models of interacting and bootstrapping effects of SK, SR, instruction, and reading hypothesize reciprocal effects. Testing this "lattice" model with children (n = 852) followed from first to second grade (5.9-10.4 years of age) revealed reciprocal effects for reading and SR, and reading and SK, but not SR and SK. More effective literacy instruction reduced reading stability over time. Findings elucidate the synergistic and reciprocal effects of learning to read on other important linguistic, self-regulatory, and cognitive processes; the value of using complex models of development to inform intervention design; and how learned skills may influence development during middle childhood.

Dialect variation, dialect-shifting, and reading comprehension in second grade.

The purpose of this study was to examine second graders' (n=680) changing spoken nonmainstream American English (NMAE) use in relation to their oral language and reading comprehension achievement. Fall NMAE production was negatively associated with fall achievement scores. NMAE production generally decreased from fall to spring. Students who qualified for the US Free and Reduced Lunch program (FARL) and who had stronger language skills were more likely to decrease their NMAE use (i.e., dialect shifting) than their peers who did not qualify for FARL or their peers with weaker language skills. Dialect shifting for a sub-sample of 102 students who used substantial amounts of NMAE at the beginning of the school year was predicted by school context, controlling for reading and language skills - in general, students who attended more affluent schools dialect shifted to a greater extent than did their peers who attended higher poverty schools. Greater dialect shifting in this group predicted gains in reading comprehension from fall to spring.

Individualizing Student Instruction in Reading: Implications for Policy and Practice.

Despite three decades of scientific and public attention on efforts to improve literacy in America, little progress has been made in closing achievement gaps across racial, ethnic and socioeconomic lines. This article argues that one major reason is failure to take into account the mosaic of strengths and weaknesses individual children bring to school. With this comes the failure to develop personalized instruction for each child. We briefly review the research available, and then describe how research, ours and others, supports the efficacy of individualizing or personalizing student instructional (ISI) and illustrate how society might close achievement gaps. ISI, and other regimes, offer a systematic instructional program, incorporate child assessment, and personalized small-group instruction. In ISI, this is aided by computer-generated recommendations and planning tools, coupled with extensive, ongoing professional development. ISI has been shown to be highly effective from preschool through third grade in improving children's literacy skills. The practical and policy implications of implementing effective instruction are discussed.