Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations.

Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relatedness. We tested eight DSMs that are popular in current cognitive and psycholinguistic research (positive pointwise mutual information; global vectors; and three variations each of Skip-gram and continuous bag of words (CBOW) using word, context, and mean embeddings) on a theoretically motivated, rich set of semantic relations involving words from multiple syntactic classes and spanning the abstract-concrete continuum (19 sets of ratings). We found that, overall, the DSMs are best at capturing overall semantic similarity and also can capture verb-noun thematic role relations and noun-noun event-based relations that play important roles in sentence comprehension. Interestingly, Skip-gram and CBOW performed the best in terms of capturing similarity, whereas GloVe dominated the thematic role and event-based relations. We discuss the theoretical and practical implications of our results, make recommendations for users of these models, and demonstrate significant differences in model performance on event-based relations.

Set for Variability as a Critical Predictor of Word Reading: Potential Implications for Early Identification and Treatment of Dyslexia.

Quasiregular orthographies such as English contain substantial ambiguities between orthography and phonology that force developing readers to acquire flexibility during decoding of unfamiliar words, a skill referred to as a "set for variability" (SfV). The ease with which a child can disambiguate the mismatch between the decoded form of a word and its actual lexical phonological form has been operationalized using the SfV mispronunciation task (e.g., the word wasp is pronounced to rhyme with clasp [i.e., /wæsp/] and the child must recognize the actual pronunciation of the word to be /wɒsp/). SfV has been shown to be a significant predictor of word reading variance. However, little is known about the relative strength of SfV as a predictor of word reading compared to other well-established predictors or the strength of this relationship in children with dyslexia. To address these questions, we administered the SfV task to a sample of grade 2-5 children (N=489) along with other reading related measures. SfV accounted for 15% unique variance in word reading above and beyond other predictors, whereas phonological awareness (PA) accounted for only 1%. Dominance analysis indicated SfV is the most powerful predictor, demonstrating complete statistical dominance over other variables including PA. Quantile regression revealed SfV is a stronger predictor at lower levels of reading skill, indicating it may be an important predictor in students with dyslexia. Results suggest that SfV is a powerful and potentially highly sensitive predictor of early reading difficulties and, therefore, may be important for early identification and treatment of dyslexia.

How you read affects what you gain: Individual differences in the functional organization of the reading system predict intervention gains in children with reading disabilities.

There is now considerable evidence regarding the types of interventions that are effective at remediating reading disabilities on average. It is generally unclear, however, what predicts the magnitude of individual-level change following a given intervention. We examine new predictors of intervention gains that are theoretically grounded in computational models of reading and focus on individual differences in the functional organization of the reading system. Specifically, we estimate the extent to which children with reading disabilities (n=118 3rd-4th graders) rely on two sources of information during an oral word reading task - print-speech correspondences and semantic imageability - before and after a phonologically-weighted intervention. We show that children who relied more on print-speech regularities and less on imageability pre-intervention had better intervention gains. In parallel, children who over the course of the intervention exhibited greater increases in their reliance on print-speech correspondences and greater decreases in their reliance on imageability had better intervention outcomes. Importantly, these two factors were differentially related to specific reading task outcomes, with greater reliance on print-speech correspondences associated with pseudoword naming, while (lesser) reliance on imageability related to word reading and comprehension. We discuss the implications of these findings for theoretical models of reading acquisition and educational practice.

Quantifying the regularities between orthography and semantics and their impact on group- and individual-level behavior.

Statistical views of reading highlight the link between proficient literacy and the assimilation of various regularities embedded in writing systems, including those in the mapping between print and meaning. Still, orthographic-semantic (O-S) regularities remain relatively understudied, with open questions regarding 3 issues: (a) how O-S regularities should be quantified, (b) how they impact the behavior of proficient readers, and (c) whether individual differences in sensitivity to these regularities predict reading skills. The goal of the current article is to address these questions. We start by reviewing previous studies estimating print-meaning regularities, where orthography-to-semantics consistency (OSC) is defined as the mean semantic similarity between a word and its orthographic neighbors. While we adopt this general strategy, we identify a potential confound in previous operational definitions. We therefore offer a modified measure, which we use to examine group-level OSC effects in available data sets of single word recognition and reading for comprehension. Our findings validate the existence of OSC effects but reveal variation across tasks, with OSC effects emerging more strongly in tasks involving a direct mapping of print to meaning. Next, we present a reanalysis of word naming data from 399 second through fifth graders, where we examine individual differences in reliance on O-S regularities and their relation to participants' reading skills. We show that early readers whose naming accuracy is more influenced by OSC (i.e., those who rely more on O-S) have better passage comprehension abilities. We conclude by discussing the role of O-S regularities in proficient reading and literacy acquisition. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Unpacking the Unique Relationship Between Set for Variability and Word Reading Development: Examining Word- and Child-Level Predictors of Performance.

Set for variability (SfV) is an oral language task which requires an individual to disambiguate the mismatch between the decoded form of an irregular word and its actual lexical pronunciation. For example, in the task, the word wasp is pronounced to rhyme with clasp (i.e., /wæsp/) and the individual must recognize the actual pronunciation of the word to be /wɒsp/. SfV has been shown to be a significant predictor of both item-specific and general word reading variance above and beyond that associated with phonemic awareness skill, letter-sound knowledge, and vocabulary skill. However, very little is known about the child characteristics and word features that affect SfV item performance. In this study we explored whether word features and child characteristics that involve phonology only are adequate to explain item-level variance in SfV performance or whether including predictors that involve the connection between phonology and orthography explain additional variance. To accomplish this we administered the SfV task (N=75 items) to a sample of grade 2-5 children (N=489) along with a battery of reading, reading related, and language measures. Results suggest that variance in SfV performance is uniquely accounted for by measures tapping phonological skill along with those capturing knowledge of phonology to orthography associations, but more so in children with better decoding skill. Additionally, word reading skill was found to moderate the influence of other predictors suggesting that how the task is approached may be impacted by word reading and decoding ability.

Theory-driven classification of reading difficulties from fMRI data using Bayesian latent-mixture models.

Decades of research have led to several competing theories regarding the neural contributors to impaired reading. But how can we know which theory (or theories) identifies the types of markers that indeed differentiate between individuals with reading disabilities (RD) and their typically developing (TD) peers? To answer this question, we propose a new analytical tool for theory evaluation and comparison, grounded in the Bayesian latent-mixture modeling framework. We start by constructing a series of latent-mixture classification models, each reflecting one existing theoretical claim regarding the neurofunctional markers of RD (highlighting network-level differences in either mean activation, inter-subject heterogeneity, inter-region variability, or connectivity). Then, we run each model on fMRI data alone (i.e., while models are blind to participants' behavioral status), which enables us to interpret the fit between a model's classification of participants and their behavioral (known) RD/TD status as an estimate of its explanatory power. Results from n=127 adolescents and young adults (RD: n=59; TD: n=68) show that models based on network-level differences in mean activation and heterogeneity failed to differentiate between TD and RD individuals. In contrast, classifications based on variability and connectivity were significantly associated with participants' behavioral status. These findings suggest that differences in inter-region variability and connectivity may be better network-level markers of RD than mean activation or heterogeneity (at least in some populations and tasks). More broadly, the results demonstrate the promise of latent-mixture modeling as a theory-driven tool for evaluating different theoretical claims regarding neural contributors to language disorders and other cognitive traits.

Tracking second language immersion across time: Evidence from a bi-directional longitudinal cross-linguistic fMRI study.

Parallel cohorts of Hebrew speakers learning English in the U.S., and American-English speakers learning Hebrew in Israel were tracked over the course of two years of immersion in their L2. We utilised a functional MRI semantic judgement task with print and speech tokens, as well as a battery of linguistic and cognitive behavioural measures prior to and after immersion, to track changes in both L1 and L2 processing. fMRI activation for print tokens produced a similar network of activation in both English and Hebrew, irrespective of L1 or L2 status. Significant convergence of print and speech processing was also observed in both languages across a network of left-hemisphere regions joint for both L1 and L2. Despite significant increases in behavioural measures of L2 proficiency, only a few signs of longitudinal change in L2 brain activation were found. In contrast, L1 showed widespread differences in processing across time, suggesting that the neurobiological footprint of reading is dynamic and plastic even in adults, with L2 immersion impacting L1 processing. Print/speech convergence showed little longitudinal change, suggesting that it is a stable marker of the differences in L1 and L2 processing across L2 proficiency.

Modeling and Visualizing the Codevelopment of Word and Nonword Reading in Children From First Through Fourth Grade: Informing Developmental Trajectories of Children With Dyslexia.

Developmental studies examining relations between word reading (WR) and decoding in typical and dyslexic populations routinely cut the reading distribution to form distinct groups. However, dichotomizing continuous variables to study development is problematic for multiple reasons. Instead, we modeled and visualized the parallel growth of WR and nonword reading (NWR) factor scores longitudinally in a Grade 1-4 developmental sample (N = 588). The results indicate that while WR and NWR growth factors are highly related (r = .71), the relation between WR and NWR trajectories change as a function of initial WR. Results are interpreted within computational models of dyslexia in which children with dyslexia overfit orthography â†’ phonology relations at the level of the word, limiting the development of sublexical representations needed to read nonwords.

Is that a pibu or a pibo? Children with reading and language deficits show difficulties in learning and overnight consolidation of phonologically similar pseudowords.

Word learning is critical for the development of reading and language comprehension skills. Although previous studies have indicated that word learning is compromised in children with reading disability (RD) or developmental language disorder (DLD), it is less clear how word learning difficulties manifest in children with comorbid RD and DLD. Furthermore, it is unclear whether word learning deficits in RD or DLD include difficulties with offline consolidation of newly learned words. In the current study, we employed an artificial lexicon learning paradigm with an overnight design to investigate how typically developing (TD) children (N = 25), children with only RD (N = 93), and children with both RD and DLD (N = 34) learned and remembered a set of phonologically similar pseudowords. Results showed that compared to TD children, children with RD exhibited: (i) slower growth in discrimination accuracy for cohort item pairs sharing an onset (e.g. pibu-pibo), but not for rhyming item pairs (e.g. pibu-dibu); and (ii) lower discrimination accuracy for both cohort and rhyme item pairs on Day 2, even when accounting for differences in Day 1 learning. Moreover, children with comorbid RD and DLD showed learning and retention deficits that extended to unrelated item pairs that were phonologically dissimilar (e.g. pibu-tupa), suggestive of broader impairments compared to children with only RD. These findings provide insights into the specific learning deficits underlying RD and DLD and motivate future research concerning how children use phonological similarity to guide the organization of new word knowledge.

Friends in Low-Entropy Places: Orthographic Neighbor Effects on Visual Word Identification Differ Across Letter Positions.

Visual word recognition is facilitated by the presence of orthographic neighbors that mismatch the target word by a single letter substitution. However, researchers typically do not consider where neighbors mismatch the target. In light of evidence that some letter positions are more informative than others, we investigate whether the influence of orthographic neighbors differs across letter positions. To do so, we quantify the number of enemies at each letter position (how many neighbors mismatch the target word at that position). Analyses of reaction time data from a visual word naming task indicate that the influence of enemies differs across letter positions, with the negative impacts of enemies being most pronounced at letter positions where readers have low prior uncertainty about which letters they will encounter (i.e., positions with low entropy). To understand the computational mechanisms that give rise to such positional entropy effects, we introduce a new computational model, VOISeR (Visual Orthographic Input Serial Reader), which receives orthographic inputs in parallel and produces an over-time sequence of phonemes as output. VOISeR produces a similar pattern of results as in the human data, suggesting that positional entropy effects may emerge even when letters are not sampled serially. Finally, we demonstrate that these effects also emerge in human subjects' data from a lexical decision task, illustrating the generalizability of positional entropy effects across visual word recognition paradigms. Taken together, such work suggests that research into orthographic neighbor effects in visual word recognition should also consider differences between letter positions.

Individual differences in learning the regularities between orthography, phonology and semantics predict early reading skills.

Statistical views of literacy development maintain that proficient reading requires the assimilation of myriad statistical regularities present in the writing system. Indeed, previous studies have tied statistical learning (SL) abilities to reading skills, establishing the existence of a link between the two. However, some issues are currently left unanswered, including questions regarding the underlying bases for these associations as well as the types of statistical regularities actually assimilated by developing readers. Here we present an alternative approach to study the role of SL in literacy development, focusing on individual differences among beginning readers. Instead of using an artificial task to estimate SL abilities, our approach identifies individual differences in children's reliance on statistical regularities as reflected by actual reading behavior. We specifically focus on individuals' reliance on regularities in the mapping between print and speech versus associations between print and meaning in a word naming task. We present data from 399 children, showing that those whose oral naming performance is impacted more by print-speech regularities and less by associations between print and meaning have better reading skills. These findings suggest that a key route by which SL mechanisms impact developing reading abilities is via their role in the assimilation of sub-lexical regularities between printed and spoken language -and more generally, in detecting regularities that are more reliable than others. We discuss the implications of our findings to both SL and reading theories.

EARSHOT: A Minimal Neural Network Model of Incremental Human Speech Recognition.

Despite the lack of invariance problem (the many-to-many mapping between acoustics and percepts), human listeners experience phonetic constancy and typically perceive what a speaker intends. Most models of human speech recognition (HSR) have side-stepped this problem, working with abstract, idealized inputs and deferring the challenge of working with real speech. In contrast, carefully engineered deep learning networks allow robust, real-world automatic speech recognition (ASR). However, the complexities of deep learning architectures and training regimens make it difficult to use them to provide direct insights into mechanisms that may support HSR. In this brief article, we report preliminary results from a two-layer network that borrows one element from ASR, long short-term memory nodes, which provide dynamic memory for a range of temporal spans. This allows the model to learn to map real speech from multiple talkers to semantic targets with high accuracy, with human-like timecourse of lexical access and phonological competition. Internal representations emerge that resemble phonetically organized responses in human superior temporal gyrus, suggesting that the model develops a distributed phonological code despite no explicit training on phonetic or phonemic targets. The ability to work with real speech is a major advance for cognitive models of HSR.

Using information-theoretic measures to characterize the structure of the writing system: the case of orthographic-phonological regularities in English.

It is generally well accepted that proficient reading requires the assimilation of myriad statistical regularities present in the writing system, including in particular the correspondences between words' orthographic and phonological forms. There is considerably less agreement, however, as to how to quantify these regularities. Here we present a comprehensive approach for this quantification using tools from Information Theory. We start by providing a glossary of the relevant information-theoretic metrics, with simplified examples showing their potential in assessing orthographic-phonological regularities. We specifically highlight the flexibility of our approach in quantifying information under different contexts (i.e., context-independent and dependent readings) and in different types of mappings (e.g., orthography-to-phonology and phonology-to-orthography). Then, we use these information-theoretic measures to assess real-world orthographic-phonological regularities of 10,093 mono-syllabic English words and examine whether these measures predict inter-item variability in accuracy and response times using available large-scale datasets of naming and lexical decision tasks. Together, the analyses demonstrate how information-theoretical measures can be used to quantify orthographical-phonological correspondences, and show that they capture variance in reading performance that is not accounted for by existing measures. We discuss the similarities and differences between the current framework and previous approaches as well as future directions towards understanding how the statistical regularities embedded in a writing system impact reading and reading acquisition.

Linking Behavioral and Computational Approaches to Better Understand Variant Vowel Pronunciations in Developing Readers.

The overarching goal of the new Florida State University/Haskins Laboratory/University of Connecticut Learning Disability (LD) Hub project is to align computational and behavioral theories of individual word reading development more closely with the challenges of learning to read a quasi-regular orthography (i.e., English) for both typically developing (TD) children and, more specifically, children with dyslexia. Our LD Hub adopts an integrated approach to better understand the neurocognitive bases of individual differences in word reading development by specifically examining the experiential (exogenous) and child-specific (endogenous) factors that determine acquisition of orthographic-phonological knowledge at different subword granularities using behavioral and computational modeling. Findings are intended to enrich understanding of the processes that influence individual differences in word reading development in TD and dyslexic children and significantly inform issues of practice (e.g., curriculum, instruction, diagnosis, and intervention). Here, we briefly provide the rationale for the Hub and present findings from the initial behavioral and computational modeling studies.

Reading and the Neurocognitive Bases of Statistical Learning1.

The processes underlying word reading are shaped by statistical properties of the writing system. According to some theoretical perspectives (e.g. Harm & Seidenberg, 2004) reading acquisition should be understood as an exercise in statistical learning. Statistical learning (SL) involves the extraction of organizing principles from a set of inputs. Several lines of research provide convergent evidence supporting the connection between SL and reading acquisition (e.g., Arciuli & Simpson, 2012; Frost et al., 2014; Bogaerts et al., 2015). An obstacle to fully appreciating the theoretical and educational implications of these findings is that SL is itself not well understood. In this paper, we review the current literature on SL with a particular focus on organizing this literature by grounding it in theories of learning and memory more generally. This approach can clarify the nature of SL and provide a framework for understanding its role in reading, reading acquisition, and reading disorders.

Common neural basis of motor sequence learning and word recognition and its relation with individual differences in reading skill.

To investigate the neural basis of a common statistical learning mechanism involved in motor sequence learning and decoding, we recorded same participants' brain activation in a serial reaction time (SRT) and word reading task using functional magnetic resonance imaging. In the SRT, a manual response was made depending on the location of a visual cue, and the order of the locations was either fixed or random. In the word reading task, visual words were passively presented. Compared to less skilled readers, more skilled readers showed greater differences in activation in the inferior frontal gyrus pars triangularis (IFGpTr) and the insula between the ordered and random condition in the SRT task and greater activation in those regions in the word reading task. It suggests that extraction of statistically predictable patterns in the IFGpTr and insula contributes to both motor sequence learning and orthographic learning, and therefore predicts individual differences in decoding skill.

Neurobiological signatures of L2 proficiency: Evidence from a bi-directional cross-linguistic study.

Recent evidence has shown that convergence of print and speech processing across a network of primarily left-hemisphere regions of the brain is a predictor of future reading skills in children, and a marker of fluent reading ability in adults. The present study extends these findings into the domain of second-language (L2) literacy, through brain imaging data of English and Hebrew L2 learners. Participants received an fMRI brain scan, while performing a semantic judgement task on spoken and written words and pseudowords in both their L1 and L2, alongside a battery of L1 and L2 behavioural measures. Imaging results show, overall, show a similar network of activation for reading across the two languages, alongside significant convergence of print and speech processing across a network of left-hemisphere regions in both L1 and L2 and in both cohorts. Importantly, convergence is greater for L1 in occipito-temporal regions tied to automatic skilled reading processes including the visual word-form area, but greater for L2 in frontal regions of the reading network, tied to more effortful, active processing. The main groupwise brain effects tell a similar story, with greater L2 than L1 activation across frontal, temporal and parietal regions, but greater L1 than L2 activation in parieto-occipital regions tied to automatic mapping processes in skilled reading. These results provide evidence for the shifting of the reading networks towards more automatic processing as reading proficiency rises and the mappings and statistics of the new orthography are learned and incorporated into the reading system.

Development and Prediction of Context-Dependent Vowel Pronunciation in Elementary Readers.

As children learn to read they become sensitive to context-dependent vowel pronunciations in words, considered a form of statistical learning. The work of Treiman and colleagues demonstrated that readers' vowel pronunciations depend on the consonantal context in which the vowel occurs and reading experience. We examined child- and nonword-factors associated with children's assignment of more vs. less frequent grapheme-phoneme correspondences (GPC) to vowel pronunciations as a function of rime coda in monosyllabic nonwords. Students (N=96) in grades 2-5 read nonwords in which more vs. less frequent vowel GPCs were wholly supported or partially favored by the rime unit. Two explanatory item-response models were developed using alternative nonword scoring procedures. Use of less frequent vowel GPCs was predicted by set for variability, word reading, and rime support for the context-dependent vowel pronunciation. We interpret the results within a developmental word reading model in which initially incomplete and oversimplified GPC representations become more context-dependent with reading experience.

The Role of Set for Variability in Irregular Word Reading: Word and Child Predictors in Typically Developing Readers and Students At-Risk for Reading Disabilities.

In a quasiregular orthography like English, children inevitably encounter irregular words during reading. Previous research suggests successful reading of an irregular word depends at least partially on a child's ability to address the mismatch between decoded form and stored word pronunciation, referred to as a child's set for variability, and the word's relative transparency, measured here using a spelling to pronunciation transparency rating. Item-level analyses were used to explore the relationship between general child performance on the set for variability mispronunciation task, word specific set for variability (predicting reading of that word), spelling to pronunciation transparency rating, and irregular word reading. Significant predictors included general word reading, general set for variability performance, and item-specific set for variability performance; word frequency and spelling to pronunciation transparency rating; and interaction between word reading and the transparency rating. Results underscore the importance of considering both general and item-specific factors affecting irregular word reading.

Neural representations for newly learned words are modulated by overnight consolidation, reading skill, and age.

Word learning depends not only on efficient online binding of phonological, orthographic and lexical information, but also on consolidation of new word representations into permanent lexical memory. Work on word learning under a variety of contexts indicates that reading and language skill impact facility of word learning in both print and speech. In addition, recent research finds that individuals with language impairments show deficits in both initial word form learning and in maintaining newly learned representations over time, implicating mechanisms associated with maintenance that may be driven by deficits in overnight consolidation. Although several recent studies have explored the neural bases of overnight consolidation of newly learned words, no extant work has examined individual differences in overnight consolidation at the neural level. The current study addresses this gap in the literature by investigating how individual differences in reading and language skills modulate patterns of neural activation associated with newly learned words following a period of overnight consolidation. Specifically, a community sample of adolescents and young adults with significant variability in reading and oral language (vocabulary) ability were trained on two spoken artificial lexicons, one in the evening on the day before fMRI scanning and one in the morning just prior to scanning. Comparisons of activation between words that were trained and consolidated vs. those that were trained but not consolidated revealed increased cortical activation in a number of language associated and memory associated regions. In addition, individual differences in age, reading skill and vocabulary modulated learning rate in our artificial lexicon learning task and the size of the cortical consolidation effect in the precuneus/posterior cingulate, such that older readers and more skilled readers had larger cortical consolidation effects in this learning-critical region. These findings suggest that age (even into late adolescence) and reading and language skills are important individual differences that affect overnight consolidation of newly learned words. These findings have significant implications for understanding reading and language disorders and should inform pedagogical models.