Bringing Assessment-to-Instruction (A2i) Technology to Scale: Exploring the Process from Development to Implementation.

Bringing effective, research-based literacy interventions into the classroom is challenging, especially given the cultural and linguistic diversity of today's classrooms. We examined the promise of Assessment-to-Instruction (A2i) technology redesigned to be used at scale to support teachers' implementation of the individualized student instruction (ISI) intervention from kindergarten through third grade. In seven randomized controlled trials, A2i and ISI have demonstrated efficacy. However, the research version of A2i was not scalable. In order to bring A2i to scale in schools serving linguistically diverse students, we carried out the current study across two phases. This study represents both an exploration of what it takes to bring an educational intervention to scale (Phase 1) and a quasi-experiment on the literacy outcomes of learners whose teachers used the technology (Phase 2). We integrated assessments of vocabulary, word decoding, and reading comprehension; revised the A2i algorithms to account for the constellation of skills English learners (ELs) bring to the classroom; updated the user interfaces and added new graphic features; and improved bandwidth and stability of the technology. Findings were mixed, including several non-significant results, a marginally significant intent-to-treat effect on word reading in kindergarten and first grade for English monolingual students and ELs, and one significant interaction effect, which suggested ELs and students with less developed reading skills in second and third grade benefitted most from the intervention. With some caution, we conclude that A2i demonstrates potential to be used at scale and promise of effectiveness for improving code-focused skills for diverse learners.

Using Assessment to Improve the Accuracy of Teachers' Perceptions of Students' Academic Competence.

Teachers' perceptions of their students' academic skills can affect students' achievement and may be influenced by unrelated student characteristics such as socioeconomic status (SES). In this ad hoc randomized controlled trial, teachers (n = 28) were randomly assigned to receive training on using assessment to guide literacy instruction, Assessment-to-Instruction (A2i), or on Math PALS (control). Teachers rated students' (n = 446) academic competence. A2i teachers' ratings did not vary by SES, and their ratings correlated more strongly with students' literacy and mathematics assessment scores compared with those of the control teachers. Control teachers generally underestimated lower SES students' academic competence; underestimation was greater at more affluent schools. Teachers' ratings of students' academic competence predicted reading and mathematics outcomes. Thoughtful use of assessments to guide instruction appeared to improve the precision of teachers' ratings of students' academic competence, improve student outcomes, and reduce potential teacher biases about children from higher-poverty families.

Predicting Second and Third Graders' Reading Comprehension Gains: Observing Students' and Classmates Talk during Literacy Instruction using COLT.

This paper introduces a new observation system that is designed to investigate students' and teachers' talk during literacy instruction, Creating Opportunities to Learn from Text (COLT). Using video-recorded observations of 2nd-3rd grade literacy instruction (N=51 classrooms, 337 students, 151 observations), we found that nine types of student talk ranged from using non-verbal gestures to generating new ideas. The more a student talked, the greater were his/her reading comprehension (RC) gains. Classmate talk also predicted RC outcomes (total effect size=0.27). We found that 11 types of teacher talk ranged from asking simple questions to encouraging students' thinking and reasoning. Teacher talk predicted student talk but did not predict students' RC gains directly. Findings highlight the importance of each student's discourse during literacy instruction, how classmates' talk contributes to the learning environments that each student experiences, and how this affects RC gains, with implications for improving the effectiveness of literacy instruction.

Observing individual children in early childhood classrooms using Optimizing Learning Opportunities for Students (OLOS): A feasibility study.

In this study, we present a newly developed observational system, Optimizing Learning Opportunities for Students (OLOS). OLOS is designed to elucidate the learning opportunities afforded to individual children within early childhood classrooms and as they transition to formal schooling (kindergarten through third grade). OLOS records the time spent in different types of learning opportunities (e.g., play, literacy, math) and the frequency of specific discourse moves children and teachers use (child talk and teacher talk). Importantly, it is being designed to be used validly and reliably by practitioners. Using OLOS, we explored individual children's experiences (n = 68 children in 12 classrooms) in four different types of early childhood programs; state-funded, state-funded PK serving children with disabilities, Head Start, and a tuition-based (non-profit) preschool. Results of our feasibility study revealed that we could feasibly and reliably use OLOS in these very different kinds of pre-kindergarten programs with some changes. OLOS provided data that aligned with our hypotheses and that our practitioner partners found useful. In analysing the observations, we found that individual children's learning opportunities varied significantly both within and between classrooms. In general, we observed that most of the PK day (or half day) was spent in language and literacy activities and non-instructional activities (e.g., transitions). Very little time in math and science was observed yet children were generally more likely to actively participate (i.e., more child talk) during academic learning opportunities (literacy, math, and science). The frequency of teacher talk also varied widely between classrooms and across programs. Plus, the more teacher talk we observed, the more likely we were to observe child talk. Our long-term aim is that OLOS can inform policy and provide information that supports practitioners in meeting the learning and social-behavioral needs of the children they serve.

Using Technology and Assessment to Personalize Instruction: Preventing Reading Problems.

Children who fail to learn to read proficiently are at serious risk of referral to special education, grade retention, dropping out of high school, and entering the juvenile justice system. Accumulating research suggests that instruction regimes that rely on assessment to inform instruction are effective in improving the implementation of personalized instruction and, in turn, student learning. However, teachers find it difficult to interpret assessment results in a way that optimizes learning opportunities for all of the students in their classrooms. This article focuses on the use of language, decoding, and comprehension assessments to develop personalized plans of literacy instruction for students from kindergarten through third grade, and A2i technology designed to support teachers' use of assessment to guide instruction. Results of seven randomized controlled trials demonstrate that personalized literacy instruction is more effective than traditional instruction, and that sustained implementation of personalized literacy instruction first through third grade may prevent the development of serious reading problems. We found effect sizes from .2 to .4 per school year, which translates into about a 2-month advantage. These effects accumulated from first through third grade with a large effect size (d = .7) equivalent to a full grade-equivalent advantage on standardize tests of literacy. These results demonstrate the efficacy of technology-supported personalized data-driven literacy instruction to prevent serious reading difficulties. Implications for translational prevention research in education and healthcare are discussed.

Building Word Knowledge, Learning Strategies, and Metacognition with the Word-Knowledge E-Book.

Many children fail to comprehend what they read because they do not monitor their understanding, which requires making accurate judgements of what they know and then employing repair strategies when comprehension fails. Relying on research from learning science and cognitive and developmental psychology, we developed the Word Knowledge e-Book (WKe-Book) to improve children's calibration of their word knowledge, strategy use, and word knowledge overall; skills which are associated with reading comprehension. The WKe-Book, which is read on a tablet computer, is a choose-your-own adventure book where choices require choosing between two rare words (e.g., cogitate vs. procrastinate). Depending on the word chosen, the story follows a different plot. There are also embedded comprehension questions where students receive immediate feedback with consequences for incorrect answers, such as being sent back to reread a few pages. In a randomized controlled trial, we tested whether students (N = 603 in 25 third through fifth grade classrooms in Arizona in the US) reading the WKe-Book would demonstrate improved word knowledge, strategy use, and word knowledge calibration. Classrooms were randomly assigned to read the WKe-Book immediately (treatment) or later (delayed-treatment control), and within classrooms, students were randomly assigned to either participate in a 15-minute weekly book club (book club treatment) or to read the WKe-Book independently with no book club (no book club control). Results revealed a significant treatment effect of the WKe-Book on students' word knowledge, word knowledge calibration, and strategy use, which predicted student performance on standardized reading comprehension and vocabulary measures. The effects were greater for students who participated in weekly book clubs compared to students in the no book club control. These findings suggest that the affordances offered by technology, which are unavailable in paper-based books, can support students' development of metacognition, including word knowledge calibration, strategy use, and word learning skills.

Modality and Interrelations Among Language, Reading, Spoken Phonological Awareness, and Fingerspelling.

Better understanding of the mechanisms underlying early reading skills can lead to improved interventions. Hence, the purpose of this study was to examine multivariate associations among reading, language, spoken phonological awareness, and fingerspelling abilities for three groups of deaf and hard-of-hearing (DHH) beginning readers: those who were acquiring only spoken English (n = 101), those who were visual learners and acquiring sign (n = 131), and those who were acquiring both (n = 104). Children were enrolled in kindergarten, first, or second grade. Within-group and between-group confirmatory factor analysis showed that there were both similarities and differences in the abilities that underlie reading in these three groups. For all groups, reading abilities related to both language and the ability to manipulate the sublexical features of words. However, the groups differed on whether these constructs were based on visual or spoken language. Our results suggest that there are alternative means to learning to read. Whereas all DHH children learning to read rely on the same fundamental abilities of language and phonological processing, the modality, levels, and relations among these abilities differ.

Profiles of Foundational Learning Components among First Graders.

School readiness includes a constellation of skills and behaviors, such as social and emotional development, language and literacy, and self-regulation that provide a basis critical for classroom participation and learning. Whereas it has been well-established that students who enter kindergarten with weaknesses in language and literacy are more likely to struggle academically, less research has focused on the variability and educational impact of other foundational learning components, including internalizing and externalizing behaviors, particularly in first grade. This study used latent profile analysis to identify the following four subgroups (profiles) of students, using foundational learning components, in a sample of first graders (n = 324): Emergent Hyperactive, Externalizing, Generally Good Students, and Internalizing. Latent class growth analysis illustrated significant differences in the average rate of growth in literacy skills from the beginning to the end of first grade across the four profiles, after controlling for gender and socioeconomic status. Findings indicated the greatest growth in literacy skills for students in the Externalizing profile and the least amount of vocabulary growth for students in the Emergent Hyperactive profile followed by the Internalizing profile. Educational implications of how researchers and educators might consider students' individual differences across profiles of foundational learning components to inform ways to support development and learning in the classroom are discussed.

Examining the Efficacy of Targeted Component Interventions on Language and Literacy for Third and Fourth Graders Who are at Risk of Comprehension Difficulties.

Testing a component model of reading comprehension in a randomized controlled trial, we evaluated the efficacy of four different interventions that were designed to target components of language and metacognition that predict children's reading comprehension: vocabulary, listening comprehension, comprehension of literate language, academic knowledge, and comprehension monitoring. Third- and fourth-graders with language skills falling below age expectations participated (N = 645). Overall, the component interventions were only somewhat effective in improving the targeted skills, compared to a business-as-usual control (g ranged from -.14 to .33), and no main effects were significant after correcting for multiple comparisons. Effects did not generalize to other language skills or to students' reading comprehension. Moreover, there were child-characteristic-by-treatment interaction effects. For example, the intervention designed to build sensorimotor mental representations was more effective for children with weaker vocabulary skills. Implications for component models of reading and interventions for children at risk of reading comprehension difficulties are discussed.

Reciprocal Effects of Self-Regulation, Semantic Knowledge, and Reading Comprehension in Early Elementary School.

Many assume that cognitive and linguistic processes, such as semantic knowledge (SK) and self-regulation (SR), subserve learned skills like reading. However, complex models of interacting and bootstrapping effects of SK, SR, instruction, and reading hypothesize reciprocal effects. Testing this "lattice" model with children (n = 852) followed from first to second grade (5.9-10.4 years of age) revealed reciprocal effects for reading and SR, and reading and SK, but not SR and SK. More effective literacy instruction reduced reading stability over time. Findings elucidate the synergistic and reciprocal effects of learning to read on other important linguistic, self-regulatory, and cognitive processes; the value of using complex models of development to inform intervention design; and how learned skills may influence development during middle childhood.

Individual differences in fifth graders' reading and language predict their comprehension monitoring development: An eye-movement study.

In this study, we investigated fifth-graders' (n=52) fall literacy, academic language, and motivation, and how these skills predicted fall and spring comprehension monitoring on an eye movement task. Comprehension monitoring was defined as the identification and repair of misunderstandings when reading text. In the eye movement task, children read two sentences; the second included either a plausible or implausible word in the context of the first sentence. Stronger readers had shorter reading times overall suggesting faster processing of text. Generally fifth-graders reacted to the implausible word (i.e., longer gaze duration on the implausible v. the plausible word, which reflects lexical access). Students with stronger academic language, compared to those with weaker academic language, generally spent more time re-reading the implausible target compared to the plausible target. This difference increased from fall to spring. Results support the centrality of academic language for meaning integration, setting standards of coherence, and utilizing comprehension repair strategies.

Capturing the complexity: Content, type, and amount of instruction and quality of the classroom learning environment synergistically predict third graders' vocabulary and reading comprehension outcomes.

We examined classrooms as complex systems that affect students' literacy learning through interacting effects of content and amount of time individual students spent in literacy instruction along with the global quality of the classroom-learning environment. We observed 27 third grade classrooms serving 315 target students using two different observation systems. The first assessed instruction at a more micro-level; specifically, the amount of time individual students spent in literacy instruction defined by the type of instruction, role of the teacher, and content. The second assessed the quality of the classroom-learning environment at a more macro level focusing on classroom organization, teacher responsiveness, and support for vocabulary and language. Results revealed that both global quality of the classroom learning environment and time individual students spent in specific types of literacy instruction covering specific content interacted to predict students' comprehension and vocabulary gains whereas neither system alone did. These findings support a dynamic systems model of how individual children learn in the context of classroom literacy instruction and the classroom-learning environment, which can help to improve observations systems, advance research, elevate teacher evaluation and professional development, and enhance student achievement.

Foundations for literacy: An early literacy intervention for deaf and hard-of-hearing children.

The present study evaluated the efficacy of a new preschool early literacy intervention created specifically for deaf and hard-of-hearing (DHH) children with functional hearing. Teachers implemented Foundations for Literacy with 25 DHH children in 2 schools (intervention group). One school used only spoken language, and the other used sign with and without spoken language. A "business as usual" comparison group included 33 DHH children who were matched on key characteristics with the intervention children but attended schools that did not implement Foundations for Literacy. Children's hearing losses ranged from moderate to profound. Approximately half of the children had cochlear implants. All children had sufficient speech perception skills to identify referents of spoken words from closed sets of items. Teachers taught small groups of intervention children an hour a day, 4 days a week for the school year. From fall to spring, intervention children made significantly greater gains on tests of phonological awareness, letter-sound knowledge, and expressive vocabulary than did comparison children. In addition, intervention children showed significant increases in standard scores (based on hearing norms) on phonological awareness and vocabulary tests. This quasi-experimental study suggests that the intervention shows promise for improving early literacy skills of DHH children with functional hearing.

A longitudinal cluster-randomized controlled study on the accumulating effects of individualized literacy instruction on students' reading from first through third grade.

Using a longitudinal cluster-randomized controlled design, we examined whether students' reading outcomes differed when they received 1, 2, or 3 years of individualized reading instruction from first through third grade, compared with a treated control group. More than 45% of students came from families living in poverty. Following students, we randomly assigned their teachers each year to deliver individualized reading instruction or a treated control condition intervention focused on mathematics. Students who received individualized reading instruction in all three grades showed the strongest reading skills by the end of third grade compared with those who received fewer years of such instruction. There was inconsistent evidence supporting a sustained first-grade treatment effect: Individualized instruction in first grade was necessary but not sufficient for stronger third-grade reading outcomes. These effects were achieved by regular classroom teachers who received professional development, which indicates that policies that support the use of evidence-based reading instruction and teacher training can yield increased student achievement.

Schooling effects on preschoolers' self-regulation, early literacy, and language growth.

The present study examined the influence of schooling during children's first and second years of preschool for children who experienced different amounts of preschool (i.e., one or two years), but who were essentially the same chronological age. Children (n = 76) were tested in the fall and spring of the school year using measures of self-regulation, decoding, letter knowledge, and vocabulary. Using hierarchical linear modeling (HLM), preschool was not associated with children's development of self-regulation in either year. For decoding and letter knowledge, children finishing their second year of preschool had higher scores, although both groups of children grew similarly during the school year. Thus, our results suggest that the first and second years of preschool are both systematically associated with decoding and letter knowledge gains, and the effects are cumulative (two years predicted greater gains overall than did one year of preschool). Finally, children's chronological age, and not whether they experienced one versus two years of preschool, predicted children's vocabulary and self-regulation outcomes. Implications for preschool curricula and instruction are discussed, including the increasing emphasis on literacy learning prior to kindergarten entry and the need to address self-regulation development along with academic learning.

Testing the Impact of Child Characteristics × Instruction Interactions on Third Graders' Reading Comprehension by Differentiating Literacy Instruction.

There is accumulating correlational evidence that the effect of specific types of reading instruction depends on children's initial language and literacy skills, called child characteristics × instruction (C×I) interactions. There is, however, no experimental evidence beyond first grade. This randomized control study examined whether C×I interactions might present an underlying and predictable mechanism for explaining individual differences in how students respond to third-grade classroom literacy instruction. To this end, we designed and tested an instructional intervention (Individualizing Student Instruction [ISI]). Teachers (n = 33) and their students (n = 448) were randomly assigned to the ISI intervention or a vocabulary intervention, which was not individualized. Teachers in both conditions received professional development. Videotaped classroom observations conducted in the fall, winter, and spring documented the instruction that each student in the classroom received. Teachers in the ISI group were more likely to provide differentiated literacy instruction that considered C×I interactions than were the teachers in the vocabulary group. Students in the ISI intervention made greater gains on a standardized assessment of reading comprehension than did students in the vocabulary intervention. Results indicate that C×I interactions likely contribute to students' varying response to literacy instruction with regard to their reading comprehension achievement and that the association between students' profile of language and literacy skills and recommended instruction is nonlinear and dependent on a number of factors. Hence, dynamic and complex theories about classroom instruction and environment impacts on student learning appear to be warranted and should inform more effective literacy instruction in third grade.

The Relations Between Children's Comprehension Monitoring and Their Reading Comprehension and Vocabulary Knowledge: An Eye-Movement Study.

Poor reading comprehension may be due to having ineffective comprehension monitoring, the metacognitive process of evaluating and regulating comprehension. When comprehension breaks down due to an inconsistency either at the word-level (e.g., due to an unfamiliar word) or at the sentence-level (e.g., due to contradictory information), readers may identify the misunderstanding and take steps to regulate their comprehension. In the current study, we utilized two eye-movement tasks (one newly developed) to examine comprehension monitoring in third through fifth grade students (n = 123), when confronted with word- and sentence-level inconsistencies, by measuring the amount of time they read (gaze duration) and reread the target inconsistent words. We investigated how this skill may be associated with individual differences in age, reading comprehension ability, and vocabulary knowledge. The results showed that generally, all students detected the word-level inconsistencies, indicated by longer gaze durations, and attempted to regulate their comprehension after detecting both word- and sentence-level inconsistencies, as indicated by more time spent rereading. Students with stronger reading comprehension (when controlling for their vocabulary), and stronger vocabulary knowledge (when controlling for their reading comprehension) were more likely to attempt regulating their comprehension. In general, the difference between the control words and the inconsistent words was smaller for third graders and larger for fourth and fifth graders, which we argue indicates greater levels of comprehension monitoring - specifically employing repair strategies. With eye-tracking technology becoming more accessible, these tasks may be useful in assessing children's reading processes to better understand at which level of comprehension monitoring they may be struggling, which in return will allow us to develop more individualized instruction for all readers.

Individualizing student instruction precisely: effects of Child x Instruction interactions on first graders' literacy development.

Recent findings demonstrate that the most effective reading instruction may vary with children's language and literacy skills. These Child x Instruction interactions imply that individualizing instruction would be a potent strategy for improving students' literacy. A cluster-randomized control field trial, conducted in 10 high-moderate poverty schools, examined effects of individualizing literacy instruction. The instruction each first grader received (n = 461 in 47 classrooms, mean age = 6.7 years) during fall, winter, and spring was recorded. Comparing intervention-recommended amounts of instruction with observed amounts revealed that intervention teachers individualized instruction more precisely than did comparison teachers. Importantly, the more precisely the children received recommended amounts of instruction, the stronger was their literacy skill growth. Results provide strong evidence of Child x Instruction interaction effects on literacy outcomes.

Schooling Effects on Early Literacy Skills of Young Deaf and Hard of Hearing Children.

Already well documented for hearing children, schooling's effects on early literacy skills for young students who are deaf or hard of hearing (DHH) were examined for the first time in the present study. Piecewise growth curve modeling was used to describe 3-, 4-, and 5-year-old students' growth in phonological awareness, letter-word identification, and vocabulary during 2 years of schooling and the intervening summer (N = 56). Amplification mode was cochlear implants for 45% of the sample and hearing aids for 54%. Classroom communication mode was spoken language only (for 61%) or sign language (39%). Across all skills, significant growth occurred during the 2 years of schooling but not during the summer. These findings underscore early education's importance in promoting DHH children's critical early skills. Universal preschool intervention, including during summer, may be important in ensuring that DHH children have an adequate foundation when schooling begins.

Relations between third grade teachers' depressive symptoms and their feedback to students, with implications for student mathematics achievement.

Recent studies have observed connections among teachers' depressive symptoms and student outcomes; however, the specific mechanisms through which teachers' mental health characteristics operate in the classroom remain largely unknown. The present study used student-level observation methods to examine the relations between third-grade teachers' (N = 32) depressive symptoms and their academic feedback to students (N = 310) and sought to make inferences about how these factors might influence students' mathematics achievement. A novel observational tool, the Teacher Feedback Coding System-Academic (TFCS-A), was used that assesses feedback across 2 dimensions-teacher affect and instructional strategy, which have been shown to be important to student learning. Multilevel exploratory factor analysis of TFCS-A data suggested 2 primary factors: positive feedback and neutral/negative feedback. Hierarchical linear modeling revealed that positive feedback was related to higher math achievement among students who began the year with weaker math skills and that teachers who reported more depressive symptoms less frequently provided this positive feedback. Results offer new information about a type of instruction that may be affected by teachers' depressive symptoms and inform efforts aimed at improving teachers' instructional interactions with students. (PsycINFO Database Record