Editors' Review and Introduction: Learning Grammatical Structures: Developmental, Cross-Species, and Computational Approaches.

Human languages all have a grammar, that is, rules that determine how symbols in a language can be combined to create complex meaningful expressions. Despite decades of research, the evolutionary, developmental, cognitive, and computational bases of grammatical abilities are still not fully understood. "Artificial Grammar Learning" (AGL) studies provide important insights into how rules and structured sequences are learned, the relevance of these processes to language in humans, and whether the cognitive systems involved are shared with other animals. AGL tasks can be used to study how human adults, infants, animals, or machines learn artificial grammars of various sorts, consisting of rules defined typically over syllables, sounds, or visual items. In this introduction, we distill some lessons from the nine other papers in this special issue, which review the advances made from this growing body of literature. We provide a critical synthesis, identify the questions that remain open, and recognize the challenges that lie ahead. A key observation across the disciplines is that the limits of human, animal, and machine capabilities have yet to be found. Thus, this interdisciplinary area of research firmly rooted in the cognitive sciences has unearthed exciting new questions and venues for research, along the way fostering impactful collaborations between traditionally disconnected disciplines that are breaking scientific ground.

Hierarchical Structure in Sequence Processing: How to Measure It and Determine Its Neural Implementation.

In many domains of human cognition, hierarchically structured representations are thought to play a key role. In this paper, we start with some foundational definitions of key phenomena like "sequence" and "hierarchy," and then outline potential signatures of hierarchical structure that can be observed in behavioral and neuroimaging data. Appropriate behavioral methods include classic ones from psycholinguistics along with some from the more recent artificial grammar learning and sentence processing literature. We then turn to neuroimaging evidence for hierarchical structure with a focus on the functional MRI literature. We conclude that, although a broad consensus exists about a role for a neural circuit incorporating the inferior frontal gyrus, the superior temporal sulcus, and the arcuate fasciculus, considerable uncertainty remains about the precise computational function(s) of this circuitry. An explicit theoretical framework, combined with an empirical approach focusing on distinguishing between plausible alternative hypotheses, will be necessary for further progress.

Five Ways in Which Computational Modeling Can Help Advance Cognitive Science: Lessons From Artificial Grammar Learning.

There is a rich tradition of building computational models in cognitive science, but modeling, theoretical, and experimental research are not as tightly integrated as they could be. In this paper, we show that computational techniques-even simple ones that are straightforward to use-can greatly facilitate designing, implementing, and analyzing experiments, and generally help lift research to a new level. We focus on the domain of artificial grammar learning, and we give five concrete examples in this domain for (a) formalizing and clarifying theories, (b) generating stimuli, (c) visualization, (d) model selection, and (e) exploring the hypothesis space.

A review of computational models of basic rule learning: The neural-symbolic debate and beyond.

We present a critical review of computational models of generalization of simple grammar-like rules, such as ABA and ABB. In particular, we focus on models attempting to account for the empirical results of Marcus et al. (Science, 283(5398), 77-80 1999). In that study, evidence is reported of generalization behavior by 7-month-old infants, using an Artificial Language Learning paradigm. The authors fail to replicate this behavior in neural network simulations, and claim that this failure reveals inherent limitations of a whole class of neural networks: those that do not incorporate symbolic operations. A great number of computational models were proposed in follow-up studies, fuelling a heated debate about what is required for a model to generalize. Twenty years later, this debate is still not settled. In this paper, we review a large number of the proposed models. We present a critical analysis of those models, in terms of how they contribute to answer the most relevant questions raised by the experiment. After identifying which aspects require further research, we propose a list of desiderata for advancing our understanding on generalization.

Selecting the model that best fits the data.

Leibovich et al. argue that that none of the experiments they review really establishes that human adults, infants, or nonhuman animals are sensitive to numerosity independent of a range of continuous quantities. We do not dispute their claim that the empirical record is inconclusive but argue that model-based data analysis does offer a way to make progress.

Five fundamental constraints on theories of the origins of music.

The diverse forms and functions of human music place obstacles in the way of an evolutionary reconstruction of its origins. In the absence of any obvious homologues of human music among our closest primate relatives, theorizing about its origins, in order to make progress, needs constraints from the nature of music, the capacities it engages, and the contexts in which it occurs. Here we propose and examine five fundamental constraints that bear on theories of how music and some of its features may have originated. First, cultural transmission, bringing the formal powers of cultural as contrasted with Darwinian evolution to bear on its contents. Second, generativity, i.e. the fact that music generates infinite pattern diversity by finite means. Third, vocal production learning, without which there can be no human singing. Fourth, entrainment with perfect synchrony, without which there is neither rhythmic ensemble music nor rhythmic dancing to music. And fifth, the universal propensity of humans to gather occasionally to sing and dance together in a group, which suggests a motivational basis endemic to our biology. We end by considering the evolutionary context within which these constraints had to be met in the genesis of human musicality.

Principles of structure building in music, language and animal song.

Human language, music and a variety of animal vocalizations constitute ways of sonic communication that exhibit remarkable structural complexity. While the complexities of language and possible parallels in animal communication have been discussed intensively, reflections on the complexity of music and animal song, and their comparisons, are underrepresented. In some ways, music and animal songs are more comparable to each other than to language as propositional semantics cannot be used as indicator of communicative success or wellformedness, and notions of grammaticality are less easily defined. This review brings together accounts of the principles of structure building in music and animal song. It relates them to corresponding models in formal language theory, the extended Chomsky hierarchy (CH), and their probabilistic counterparts. We further discuss common misunderstandings and shortcomings concerning the CH and suggest ways to move beyond. We discuss language, music and animal song in the context of their function and motivation and further integrate problems and issues that are less commonly addressed in the context of language, including continuous event spaces, features of sound and timbre, representation of temporality and interactions of multiple parallel feature streams. We discuss these aspects in the light of recent theoretical, cognitive, neuroscientific and modelling research in the domains of music, language and animal song.

Three design principles of language: the search for parsimony in redundancy.

In this paper we present three design principles of language - experience, heterogeneity and redundancy--and present recent developments in a family of models incorporating them, namely Data-Oriented Parsing/Unsupervised Data-Oriented Parsing. Although the idea of some form of redundant storage has become part and parcel of parsing technologies and usage-based linguistic approaches alike, the question how much of it is cognitively realistic and/or computationally optimally efficient is an open one. We argue that a segmentation-based approach (Bayesian Model Merging) combined with an all-subtrees approach reduces the number of rules needed to achieve an optimal performance, thus making the parser more efficient. At the same time, starting from unsegmented wholes comes closer to the acquisitional situation of a language learner, and thus adds to the cognitive plausibility of the model.

Simple rules can explain discrimination of putative recursive syntactic structures by a songbird species.

According to a controversial hypothesis, a characteristic unique to human language is recursion. Contradicting this hypothesis, it has been claimed that the starling, one of the two animal species tested for this ability to date, is able to distinguish acoustic stimuli based on the presence or absence of a center-embedded recursive structure. In our experiment we show that another songbird species, the zebra finch, can also discriminate between artificial song stimuli with these structures. Zebra finches are able to generalize this discrimination to new songs constructed using novel elements belonging to the same categories, similar to starlings. However, to demonstrate that this is based on the ability to detect the putative recursive structure, it is critical to test whether the birds can also distinguish songs with the same structure consisting of elements belonging to unfamiliar categories. We performed this test and show that seven out of eight zebra finches failed it. This suggests that the acquired discrimination was based on phonetic rather than syntactic generalization. The eighth bird, however, must have used more abstract, structural cues. Nevertheless, further probe testing showed that the results of this bird, as well as those of others, could be explained by simpler rules than recursive ones. Although our study casts doubts on whether the rules used by starlings and zebra finches really provide evidence for the ability to detect recursion as present in "context-free" syntax, it also provides evidence for abstract learning of vocal structure in a songbird.

Children's grammars grow more abstract with age--evidence from an automatic procedure for identifying the productive units of language.

We develop an approach to automatically identify the most probable multiword constructions used in children's utterances, given syntactically annotated utterances from the Brown corpus of CHILDES. The found constructions cover many interesting linguistic phenomena from the language acquisition literature and show a progression from very concrete toward abstract constructions. We show quantitatively that for all children of the Brown corpus grammatical abstraction, defined as the relative number of variable slots in the productive units of their grammar, increases globally with age.

Is evolvability involved in the origin of modular variation?

Lipson et al. (2002) presented an elegant linear algebraic formalism to define and study the evolution of modularity in an artificial evolving system. They employed simulation data to support their suggestion that modularity arises spontaneously in temporally fluctuating systems in response to selection for enhanced evolvability. We show analytically and by simulation that their correlate of modularity is itself under selection and so is not a reliable indicator of selection for modularity per se. In addition, we question the relation between modularity and evolvability in their simulations, suggesting that this modularity cannot confer enhanced evolvability.

Evolution of an optimal lexicon under constraints from embodiment.

Research in language evolution is concerned with the question of how complex linguistic structures can emerge from the interactions between many communicating individuals. Thus it complements psycholinguistics, which investigates the processes involved in individual adult language processing, and child language development studies, which investigate how children learn a given (fixed) language. We focus on the framework of language games and argue that they offer a fresh and formal perspective on many current debates in cognitive science, including those on the synchronic-versus-diachronic perspective on language, the embodiment and situatedness of language and cognition, and the self-organization of linguistic patterns. We present a measure for the quality of a lexicon in a population, and derive four characteristics of the optimal lexicon: specificity, coherence, distinctiveness, and regularity. We present a model of lexical dynamics that shows the spontaneous emergence of these characteristics in a distributed population of individuals that incorporate embodiment constraints. Finally, we discuss how research in cognitive science could contribute to improving existing language game models.