Encoding interference effects support self-organized sentence processing.

According to cue-based retrieval theories of sentence comprehension, establishing the syntactic dependency between a verb and the grammatical subject is susceptible to interference from other noun phrases in the sentence. At the verb, the subject must be retrieved from memory, but non-subject nouns that are similar on dimensions that are relevant to subject-verb agreement, like number marking, can make the retrieval more difficult. However, cue-based retrieval models fail to account for a class of interference effects, conventionally called "encoding interference," that cannot be due to retrieval interference. In this paper, we implement a self-organized sentence processing model that provides a more parsimonious explanation of encoding interference effects than otherwise reasonable extensions that could be made to the cue-based retrieval approach. We first also present new behavioral evidence for encoding interference using a semantic similarity manipulation in two self-paced reading studies of subject-verb number agreement. The results of these experiments are more compatible with the self-organizing account. We argue that self-organization, which reduces all parsing to fallible feature match optimization and makes no a priori distinction between encoding and retrieval, can provide a unifying approach to similarity-based interference in sentence comprehension.

Escape from fraught states in a coordination game.

Through a behavioural coordination game played by groups of humans and simulated with agent-based models, we investigated a social network dilemma that we call fraughtness. Seven players, connected to one another in various topologies via a computer network, each had to move a slider to the left or right along a horizontal bar on their screen. The goal was for all the players to move their slider to the same side. Players received feedback indicating the degree to which they and their neighbours agreed about the choice of side. When the topology had a hierarchical branching structure, the groups often got stuck in fraughtness: players on one branch favoured one side, while players on the other branch favoured the other; because all were receiving supportive local feedback, nobody wanted to change. Nevertheless, after being stuck in fraughtness for some time, most groups managed to escape it. Fraughtness is arguably an analog of generally negatively viewed social phenomena like polarization and echo chambers. Our analyses suggest that while fraughtness is problematic, it is closely linked to successful structure formation-it thus may be most effective to focus not on how to banish it, but on how to resolve it.

A self-organized sentence processing theory of gradience: The case of islands.

Formal theories of grammar and traditional parsing models, insofar as they presuppose a categorical notion of grammar, face the challenge of accounting for gradient effects (sentences receive gradient acceptability judgments, speakers report a gradient ability to comprehend sentences that deviate from idealized grammatical forms, and various online sentence processing measures yield gradient effects). This challenge is traditionally met by explaining gradient effects in terms of extra-grammatical factors, positing a purely categorical core for the language system. We present a new way of accounting for gradience in a self-organized sentence processing (SOSP) model. SOSP generates structures with a continuous range of grammaticality values by assuming a flexible structure-formation system in which the parses are formed even under sub-optimal circumstances by coercing elements to play roles that do not optimally suit them. We focus on islands, a family of syntactic domains out of which movement is generally prohibited. Islands are interesting because, although many linguistic theories treat them as fully ungrammatical and uninterpretable, experimental studies have revealed gradient patterns of acceptability and evidence for their interpretability. We describe the conceptual framework of SOSP, showing that it largely respects island constraints, but in certain cases, consistent with empirical data, coerces elements that block dependencies into elements that allow them.

Individual differences in subphonemic sensitivity and phonological skills.

Many studies have established a link between phonological abilities (indexed by phonological awareness and phonological memory tasks) and typical and atypical reading development. Individuals who perform poorly on phonological assessments have been mostly assumed to have underspecified (or "fuzzy") phonological representations, with typical phonemic categories, but with greater category overlap due to imprecise encoding. An alternative posits that poor readers have overspecified phonological representations, with speech sounds perceived allophonically (phonetically distinct variants of a single phonemic category). On both accounts, mismatch between phonological categories and orthography leads to reading difficulty. Here, we consider the implications of these accounts for online speech processing. We used eye tracking and an individual differences approach to assess sensitivity to subphonemic detail in a community sample of young adults with a wide range of reading-related skills. Subphonemic sensitivity inversely correlated with meta-phonological task performance, consistent with overspecification.

A Self-Organizing Approach to Subject-Verb Number Agreement.

We present a self-organizing approach to sentence processing that sheds new light on notional plurality effects in agreement attraction, using pseudopartitive subject noun phrases (e.g., a bottle of pills). We first show that notional plurality ratings (numerosity judgments for subject noun phrases) predict verb agreement choices in pseudopartitives, in line with the "Marking" component of the Marking and Morphing theory of agreement processing. However, no account to date has derived notional plurality values from independently needed principles of language processing. We argue on the basis of new experimental evidence and a dynamical systems model that the theoretical black box of notional plurality can be unpacked into objectively measurable semantic features. With these semantic features driving structure formation (and hence agreement choice), our model reproduces the human verb production patterns as a byproduct of normal processing. Finally, we discuss how the self-organizing approach might be extended to other agreement attraction phenomena.

Encoding and Retrieval Interference in Sentence Comprehension: Evidence from Agreement.

Long-distance verb-argument dependencies generally require the integration of a fronted argument when the verb is encountered for sentence interpretation. Under a parsing model that handles long-distance dependencies through a cue-based retrieval mechanism, retrieval is hampered when retrieval cues also resonate with non-target elements (retrieval interference). However, similarity-based interference may also stem from interference arising during the encoding of elements in memory (encoding interference), an effect that is not directly accountable for by a cue-based retrieval mechanism. Although encoding and retrieval interference are clearly distinct at the theoretical level, it is difficult to disentangle the two on empirical grounds, since encoding interference may also manifest at the retrieval region. We report two self-paced reading experiments aimed at teasing apart the role of each component in gender and number subject-verb agreement in Italian and English object relative clauses. In Italian, the verb does not agree in gender with the subject, thus providing no cue for retrieval. In English, although present tense verbs agree in number with the subject, past tense verbs do not, allowing us to test the role of number as a retrieval cue within the same language. Results from both experiments converge, showing similarity-based interference at encoding, and some evidence for an effect at retrieval. After having pointed out the non-negligible role of encoding in sentence comprehension, and noting that Lewis and Vasishth's (2005) ACT-R model of sentence processing, the most fully developed cue-based retrieval approach to sentence processing does not predict encoding effects, we propose an augmentation of this model that predicts these effects. We then also propose a self-organizing sentence processing model (SOSP), which has the advantage of accounting for retrieval and encoding interference with a single mechanism.

Individual differences in decoding skill, print exposure, and cortical structure in young adults.

This exploratory study investigated relations between individual differences in cortical grey matter structure and young adult readers' cognitive profiles. Whole-brain analyses revealed neuroanatomical correlations with word and nonword reading ability (decoding), and experience with printed matter. Decoding was positively correlated with grey matter volume (GMV) in left superior temporal sulcus, and thickness (GMT) in right superior temporal gyrus. Print exposure was negatively correlated with GMT in left inferior frontal gyrus (pars opercularis) and left fusiform gyrus (including the visual word form area). Both measures also correlated with supramarginal gyrus (SMG), but in spatially distinct subregions: decoding was positively associated with GMV in left anterior SMG, and print exposure was negatively associated with GMT in left posterior SMG. Our comprehensive approach to assessment both confirms and refines our understanding of the novel relation between the structure of pSMG and proficient reading, and unifies previous research relating cortical structure and reading skill.

Speaking up for vocabulary: reading skill differences in young adults.

This study is part of a broader project aimed at developing cognitive and neurocognitive profiles of adolescent and young adult readers whose educational and occupational prospects are constrained by their limited literacy skills. We explore the relationships among reading-related abilities in participants ages 16 to 24 years spanning a wide range of reading ability. Two specific questions are addressed: (a) Does the simple view of reading capture all nonrandom variation in reading comprehension? (b) Does orally assessed vocabulary knowledge account for variance in reading comprehension, as predicted by the lexical quality hypothesis? A comprehensive battery of cognitive and educational tests was employed to assess phonological awareness, decoding, verbal working memory, listening comprehension, reading comprehension, word knowledge, and experience with print. In this heterogeneous sample, decoding ability clearly played an important role in reading comprehension. The simple view of reading gave a reasonable fit to the data, although it did not capture all of the reliable variance in reading comprehension as predicted. Orally assessed vocabulary knowledge captured unique variance in reading comprehension even after listening comprehension and decoding skill were accounted for. We explore how a specific connectionist model of lexical representation and lexical access can account for these findings.

Discovery of a Recursive Principle: An Artificial Grammar Investigation of Human Learning of a Counting Recursion Language.

Learning is typically understood as a process in which the behavior of an organism is progressively shaped until it closely approximates a target form. It is easy to comprehend how a motor skill or a vocabulary can be progressively learned-in each case, one can conceptualize a series of intermediate steps which lead to the formation of a proficient behavior. With grammar, it is more difficult to think in these terms. For example, center embedding recursive structures seem to involve a complex interplay between multiple symbolic rules which have to be in place simultaneously for the system to work at all, so it is not obvious how the mechanism could gradually come into being. Here, we offer empirical evidence from a new artificial language (or "artificial grammar") learning paradigm, Locus Prediction, that, despite the conceptual conundrum, recursion acquisition occurs gradually, at least for a simple formal language. In particular, we focus on a variant of the simplest recursive language, a (n) b (n) , and find evidence that (i) participants trained on two levels of structure (essentially ab and aabb) generalize to the next higher level (aaabbb) more readily than participants trained on one level of structure (ab) combined with a filler sentence; nevertheless, they do not generalize immediately; (ii) participants trained up to three levels (ab, aabb, aaabbb) generalize more readily to four levels than participants trained on two levels generalize to three; (iii) when we present the levels in succession, starting with the lower levels and including more and more of the higher levels, participants show evidence of transitioning between the levels gradually, exhibiting intermediate patterns of behavior on which they were not trained; (iv) the intermediate patterns of behavior are associated with perturbations of an attractor in the sense of dynamical systems theory. We argue that all of these behaviors indicate a theory of mental representation in which recursive systems lie on a continuum of grammar systems which are organized so that grammars that produce similar behaviors are near one another, and that people learning a recursive system are navigating progressively through the space of these grammars.

The real-time prediction and inhibition of linguistic outcomes: Effects of language and literacy skill.

Recent studies have found considerable individual variation in language comprehenders' predictive behaviors, as revealed by their anticipatory eye movements during language comprehension. The current study investigated the relationship between these predictive behaviors and the language and literacy skills of a diverse, community-based sample of young adults. We found that rapid automatized naming (RAN) was a key determinant of comprehenders' prediction ability (e.g., as reflected in predictive eye movements to a white cake on hearing "The boy will eat the white…"). Simultaneously, comprehension-based measures predicted participants' ability to inhibit eye movements to objects that shared features with predictable referents but were implausible completions (e.g., as reflected in eye movements to a white but inedible white car). These findings suggest that the excitatory and inhibitory mechanisms that support prediction during language processing are closely linked with specific cognitive abilities that support literacy. We show that a self-organizing cognitive architecture captures this pattern of results.

Vocabulary does not complicate the simple view of reading.

Gough and Tunmer's (1986) simple view of reading (SVR) proposed that reading comprehension (RC) is a function of language comprehension (LC) and word recognition/decoding. Braze et al. (2007) presented data suggesting an extension of the SVR in which knowledge of vocabulary (V) affected RC over and above the effects of LC. Tunmer and Chapman (2012) found a similar independent contribution of V to RC when the data were analyzed by hierarchical regression. However, additional analysis by factor analysis and structural equation modeling indicated that the effect of V on RC was, in fact, completely captured by LC itself and there was no need to posit a separate direct effect of V on RC. In the present study, we present new data from young adults with sub-optimal reading skill (N = 286). Latent variable and regression analyses support Gough and Tunmer's original proposal and the conclusions of Tunmer and Chapman that V can be considered a component of LC and not an independent contributor to RC.

Behavioral and neurobiological effects of printed word repetition in lexical decision and naming.

A series of experiments studied the effects of repetition of printed words on (1) lexical decision (LD) and naming (NAM) behavior and (2) concomitant brain activation. It was hypothesized that subword phonological analysis (assembly) would decrease with increasing word familiarity and the greater decrease would occur in LD, a task that is believed to be less dependent on assembly than naming. As a behavioral marker of assembly, we utilized the regularity effect (the difference in response latency between words with regular versus irregular spelling-sound correspondences). In addition to repetition, stimulus familiarity was manipulated by word frequency and case alternation. Both experiments revealed an initial latency disadvantage for low frequency irregular words suggesting that assembly is the dominant process in both tasks when items are unfamiliar. As items become more familiar with repetition, the regularity effect disappeared in LD but persisted in NAM. Brain activation patterns for repeated words that were observed in fMRI paralleled the behavioral studies in showing greater reductions in activity under lexical decision than naming for regions previously identified as involved in assembly.

Evidence for self-organized sentence processing: digging-in effects.

Dynamical, self-organizing models of sentence processing predict "digging-in" effects: The more committed the parser becomes to a wrong syntactic choice, the harder it is to reanalyze. Experiment 1 replicates previous grammaticality judgment studies (F. Ferreira & J. M. Henderson, 1991b, 1993), revealing a deleterious effect of lengthening the ambiguous region of a garden-path sentence. The authors interpret this result as a digging-in effect. Experiment 2 finds a corresponding effect on reading times. Experiment 3 finds that making 2 wrong attachments is worse than making 1. Non-self-organizing models require multiple stipulations to predict both kinds of effects. The authors show that, under an appropriately formulated self-organizing account, both results stem from self-reinforcement of node and link activations, a feature that is needed independently. An implemented model is given.

Lexical interference effects in sentence processing: evidence from the visual world paradigm and self-organizing models.

Psycholinguistic research spanning a number of decades has produced diverging results with regard to the nature of constraint integration in online sentence processing. For example, evidence that language users anticipatorily fixate likely upcoming referents in advance of evidence in the speech signal supports rapid context integration. By contrast, evidence that language users activate representations that conflict with contextual constraints, or only indirectly satisfy them, supports nonintegration or late integration. Here we report on a self-organizing neural network framework that addresses 1 aspect of constraint integration: the integration of incoming lexical information (i.e., an incoming word) with sentence context information (i.e., from preceding words in an unfolding utterance). In 2 simulations, we show that the framework predicts both classic results concerned with lexical ambiguity resolution (Swinney, 1979; Tanenhaus, Leiman, & Seidenberg, 1979), which suggest late context integration, and results demonstrating anticipatory eye movements (e.g., Altmann & Kamide, 1999), which support rapid context integration. We also report 2 experiments using the visual world paradigm that confirm a new prediction of the framework. Listeners heard sentences like "The boy will eat the white …" while viewing visual displays with objects like a white cake (i.e., a predictable direct object of "eat"), white car (i.e., an object not predicted by "eat," but consistent with "white"), and distractors. In line with our simulation predictions, we found that while listeners fixated white cake most, they also fixated white car more than unrelated distractors in this highly constraining sentence (and visual) context.

Birth of an abstraction: a dynamical systems account of the discovery of an elsewhere principle in a category learning task.

Human participants and recurrent ("connectionist") neural networks were both trained on a categorization system abstractly similar to natural language systems involving irregular ("strong") classes and a default class. Both the humans and the networks exhibited staged learning and a generalization pattern reminiscent of the Elsewhere Condition (Kiparsky, 1973). Previous connectionist accounts of related phenomena have often been vague about the nature of the networks' encoding systems. We analyzed our network using dynamical systems theory, revealing topological and geometric properties that can be directly compared with the mechanisms of non-connectionist, rule-based accounts. The results reveal that the networks "contain" structures related to mechanisms posited by rule-based models, partly vindicating the insights of these models. On the other hand, they support the one mechanism (OM), as opposed to the more than one mechanism (MOM), view of symbolic abstraction by showing how the appearance of MOM behavior can arise emergently from one underlying set of principles. The key new contribution of this study is to show that dynamical systems theory can allow us to explicitly characterize the relationship between the two perspectives in implemented models.

Fractal analysis illuminates the form of connectionist structural gradualness.

We examine two connectionist networks-a fractal learning neural network (FLNN) and a Simple Recurrent Network (SRN)-that are trained to process center-embedded symbol sequences. Previous work provides evidence that connectionist networks trained on infinite-state languages tend to form fractal encodings. Most such work focuses on simple counting recursion cases (e.g., anbn), which are not comparable to the complex recursive patterns seen in natural language syntax. Here, we consider exponential state growth cases (including mirror recursion), describe a new training scheme that seems to facilitate learning, and note that the connectionist learning of these cases has a continuous metamorphosis property that looks very different from what is achievable with symbolic encodings. We identify a property-ragged progressive generalization-which helps make this difference clearer. We suggest two conclusions. First, the fractal analysis of these more complex learning cases reveals the possibility of comparing connectionist networks and symbolic models of grammatical structure in a principled way-this helps remove the black box character of connectionist networks and indicates how the theory they support is different from symbolic approaches. Second, the findings indicate the value of future, linked mathematical and empirical work on these models-something that is more possible now than it was 10 years ago.

Impulse processing: a dynamical systems model of incremental eye movements in the visual world paradigm.

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely on a dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP.

Unification of sentence processing via ear and eye: an fMRI study.

We present new evidence based on fMRI for the existence and neural architecture of an abstract supramodal language system that can integrate linguistic inputs arising from different modalities such that speech and print each activate a common code. Working with sentence material, our aim was to find out where the putative supramodal system is located and how it responds to comprehension challenges. To probe these questions we examined BOLD activity in experienced readers while they performed a semantic categorization task with matched written or spoken sentences that were either well-formed or contained anomalies of syntactic form or pragmatic content. On whole-brain scans, both anomalies increased net activity over non-anomalous baseline sentences, chiefly at left frontal and temporal regions of heteromodal cortex. The anomaly-sensitive sites correspond approximately to those that previous studies (Michael et al., 2001; Constable et al., 2004) have found to be sensitive to other differences in sentence complexity (object relative minus subject relative). Regions of interest (ROIs) were defined by peak response to anomaly averaging over modality conditions. Each anomaly-sensitive ROI showed the same pattern of response across sentence types in each modality. Voxel-by-voxel exploration over the whole brain based on a cosine similarity measure of common function confirmed the specificity of supramodal zones.

A dynamical systems perspective on the relationship between symbolic and non-symbolic computation.

It has been claimed that connectionist (artificial neural network) models of language processing, which do not appear to employ "rules", are doing something different in kind from classical symbol processing models, which treat "rules" as atoms (e.g., McClelland and Patterson in Trends Cogn Sci 6(11):465-472, 2002). This claim is hard to assess in the absence of careful, formal comparisons between the two approaches. This paper formally investigates the symbol-processing properties of simple dynamical systems called affine dynamical automata, which are close relatives of several recurrent connectionist models of language processing (e.g., Elman in Cogn Sci 14:179-211, 1990). In line with related work (Moore in Theor Comput Sci 201:99-136, 1998; Siegelmann in Neural networks and analog computation: beyond the Turing limit. Birkhäuser, Boston, 1999), the analysis shows that affine dynamical automata exhibit a range of symbol processing behaviors, some of which can be mirrored by various Turing machine devices, and others of which cannot be. On the assumption that the Turing machine framework is a good way to formalize the "computation" part of our understanding of classical symbol processing, this finding supports the view that there is a fundamental "incompatibility" between connectionist and classical models (see Fodor and Pylyshyn 1988; Smolensky in Behav Brain Sci 11(1):1-74, 1988; beim Graben in Mind Matter 2(2):29--51,2004b). Given the empirical successes of connectionist models, the more general, super-Turing framework is a preferable vantage point from which to consider cognitive phenomena. This vantage may give us insight into ill-formed as well as well-formed language behavior and shed light on important structural properties of learning processes.

Reading differences and brain: cortical integration of speech and print in sentence processing varies with reader skill.

Functional magnetic resonance imaging (fMRI) was used to investigate the impact of literacy skills in young adults on the distribution of cerebral activity during comprehension of sentences in spoken and printed form. The aim was to discover where speech and print streams merge, and whether their convergence is affected by the level of reading skill. The results from different analyses all point to the conclusion that neural integration of sentence processing across speech and print varies positively with the reader's skill. Further, they identify the inferior frontal region as the principal site of speech-print integration and a major focus of reading comprehension differences. The findings provide new evidence of the role of the inferior frontal region in supporting supramodal systems of linguistic representation.