Infant-directed speech becomes less redundant as infants grow: Implications for language learning.

Do speakers use less redundant language with more proficient interlocutors? Both the communicative efficiency framework and the language development literature predict that speech directed to younger infants should be more redundant than speech directed to older infants. Here, we test this by quantifying redundancy in infant-directed speech using entropy rate - an information-theoretic measure reflecting average degree of repetitiveness. While IDS is often described as repetitive, entropy rate provides a novel holistic measure of redundancy in this speech genre. Using two developmental corpora, we compare entropy rates of samples taken from different ages. We find that parents use less redundant speech when talking to older children, illustrating an effect of perceived interlocutor proficiency on redundancy. The developmental decrease in redundancy reflects a decrease in lexical repetition, but also a decrease in repetitions of multi-word sequences, highlighting the importance of larger sequences in early language learning.

Cultural evolution creates the statistical structure of language.

Human language is unique in its structure: language is made up of parts that can be recombined in a productive way. The parts are not given but have to be discovered by learners exposed to unsegmented wholes. Across languages, the frequency distribution of those parts follows a power law. Both statistical properties-having parts and having them follow a particular distribution-facilitate learning, yet their origin is still poorly understood. Where do the parts come from and why do they follow a particular frequency distribution? Here, we show how these two core properties emerge from the process of cultural evolution with whole-to-part learning. We use an experimental analog of cultural transmission in which participants copy sets of non-linguistic sequences produced by a previous participant: This design allows us to ask if parts will emerge purely under pressure for the system to be learnable, even without meanings to convey. We show that parts emerge from initially unsegmented sequences, that their distribution becomes closer to a power law over generations, and, importantly, that these properties make the sets of sequences more learnable. We argue that these two core statistical properties of language emerge culturally both as a cause and effect of greater learnability.

Zipfian Distributions in Child-Directed Speech.

Across languages, word frequency and rank follow a power law relation, forming a distribution known as the Zipfian distribution. There is growing experimental evidence that this well-studied phenomenon may be beneficial for language learning. However, most investigations of word distributions in natural language have focused on adult-to-adult speech: Zipf's law has not been thoroughly evaluated in child-directed speech (CDS) across languages. If Zipfian distributions facilitate learning, they should also be found in CDS. At the same time, several unique properties of CDS may result in a less skewed distribution. Here, we examine the frequency distribution of words in CDS in three studies. We first show that CDS is Zipfian across 15 languages from seven language families. We then show that CDS is Zipfian from early on (six-months) and across development for five languages with sufficient longitudinal data. Finally, we show that the distribution holds across different parts of speech: Nouns, verbs, adjectives and prepositions follow a Zipfian distribution. Together, the results show that the input children hear is skewed in a particular way from early on, providing necessary (but not sufficient) support for the postulated learning advantage of such skew. They highlight the need to study skewed learning environments experimentally.

Do Children Use Multi-Word Information in Real-Time Sentence Comprehension?

Meaning in language emerges from multiple words, and children are sensitive to multi-word frequency from infancy. While children successfully use cues from single words to generate linguistic predictions, it is less clear whether and how they use multi-word sequences to guide real-time language processing and whether they form predictions on the basis of multi-word information or pairwise associations. We address these questions in two visual-world eye-tracking experiments with 5- to 8-year-old children. In Experiment 1, we asked whether children generate more robust predictions for the sentence-final object of highly frequent sequences (e.g., "Throw the ball"), compared to less frequent sequences (e.g., "Throw the book"). We further examined if gaze patterns reflect event knowledge or phrasal frequency by comparing the processing of phrases that have the same event structure but differ in multi-word content (e.g., "Brush your teeth" vs. "Brush her teeth"). In the second study, we employed a training paradigm to ask if children are capable of generating predictio.ns from novel multi-word associations while controlling for the overall frequency of the sequences. While the results of Experiment 1 suggested that children primarily relied on event associations to generate real-time predictions, those of Experiment 2 showed that the same children were able to use recurring novel multi-word sequences to generate real-time linguistic predictions. Together, these findings suggest that children can draw on multi-word information to generate linguistic predictions, in a context-dependent fashion, and highlight the need to account for the influence of multi-word sequences in models of language processing.

The Impact of Information Structure on the Emergence of Differential Object Marking: An Experimental Study.

Many languages exhibit differential object marking (DOM), where only certain types of grammatical objects are marked with morphological cases. Traditionally, it has been claimed that DOM arises as a way to prevent ambiguity by marking objects that might otherwise be mistaken for subjects (e.g., animate objects). While some recent experimental work supports this account, research on language typology suggests at least one alternative hypothesis. In particular, DOM may instead arise as a way of marking objects that are atypical from the point of view of information structure. According to this account, rather than being marked to avoid ambiguity, objects are marked when they are given (already familiar in the discourse) rather than new. Here, we experimentally investigate this hypothesis using two artificial language learning experiments. We find that information structure impacts participants' object marking, but in an indirect way: atypical information structure leads to a change in word order, which then triggers increased object marking. Interestingly, this staged process of change is compatible with documented cases of DOM emergence. We argue that this process is driven by two cognitive tendencies. First, a tendency to place discourse given information before new information, and second, a tendency to mark noncanonical word order. Taken together, our findings provide corroborating evidence for the role of information structure in the emergence of DOM systems.

The learnability consequences of Zipfian distributions in language.

While the languages of the world differ in many respects, they share certain commonalties, which can provide insight on our shared cognition. Here, we explore the learnability consequences of one of the striking commonalities between languages. Across languages, word frequencies follow a Zipfian distribution, showing a power law relation between a word's frequency and its rank. While their source in language has been studied extensively, less work has explored the learnability consequences of such distributions for language learners. We propose that the greater predictability of words in this distribution (relative to less skewed distributions) can facilitate word segmentation, a crucial aspect of early language acquisition. To explore this, we quantify word predictability using unigram entropy, assess it across languages using naturalistic corpora of child-directed speech and then ask whether similar unigram predictability facilitates word segmentation in the lab. We find similar unigram entropy in child-directed speech across 15 languages. We then use an auditory word segmentation task to show that the unigram predictability levels found in natural language are uniquely facilitative for word segmentation for both children and adults. These findings illustrate the facilitative impact of skewed input distributions on learning and raise questions about the possible role of cognitive pressures in the prevalence of Zipfian distributions in language.

Redundancy can benefit learning: Evidence from word order and case marking.

The prevalence of redundancy in the world languages has long puzzled language researchers. It is especially surprising in light of the growing evidence on speakers' tendency to avoid redundant elements in production (omitting or reducing more predictable elements). Here, we propose that redundancy can be functional for learning. In particular, we argue that redundant cues can facilitate learning, even when they make the language system more complicated. This prediction is further motivated by the Linguistic Niche Hypothesis (Lupyan & Dale, 2010), which suggests that morphological complexity can arise due to the advantage redundancy might confer for child learners. We test these hypotheses in an artificial language learning study with children and adults, where either word order alone or both word order and case marking serve as cues for thematic assignment in a novel construction. We predict, and find, that children learning the redundant language learn to produce it, and show better comprehension of the novel thematic assignment than children learning the non-redundant language, despite having to learn an additional morpheme. Children in both conditions were similarly accurate in producing the novel word order, suggesting redundancy might have a differential effect on comprehension and production. Adults did not show better learning in the redundant condition, most likely because they were at ceiling in both conditions. We discuss implications for theories of language learning and language change.

The Starting Big approach to language learning.

The study of language acquisition has a long and contentious history: researchers disagree on what drives this process, the relevant data, and the interesting questions. Here, I outline the Starting Big approach to language learning, which emphasizes the role of multiword units in language, and of coarse-to-fine processes in learning. I outline core predictions and supporting evidence. In short, the approach argues that multiword units are integral building blocks in language; that such units can facilitate mastery of semantically opaque relations between words; and that adults rely on them less than children, which can explain (some of) their difficulty in learning a second language. The Starting Big approach is a theory of how children learn language, how language is represented, and how to explain differences between first and second language learning. I discuss the learning and processing models at the heart of the approach and their cross-linguistic implications.

'Clap your hands' or 'take your hands'? One-year-olds distinguish between frequent and infrequent multiword phrases.

Although words are often described as the basic building blocks of language, there is growing evidence that multiword sequences also play an integral role in language learning and processing. It is not known, however, whether children become sensitive to multiword information at an age when they are still building knowledge of individual words. Using a central fixation paradigm, the present study examined whether infants between 11 and 12 months (N = 36) distinguish between three-word sequences (trigrams) with similar substring frequencies but different multiword frequency in infant-directed speech (e.g., high frequency: 'clap your hands' vs. low frequency: 'take your hands'). Infants looked significantly longer at frequent trigrams compared to infrequent ones. This provides the first evidence that infants at the cusp of one-word production are already sensitive to the frequency of multiword sequences, and suggests they represent linguistic units of varying sizes from early on, raising the need to evaluate knowledge of both words and larger sequences during development.

Testing a computational model of causative overgeneralizations: Child judgment and production data from English, Hebrew, Hindi, Japanese and K'iche'.

How do language learners avoid the production of verb argument structure overgeneralization errors ( *The clown laughed the man c.f. The clown made the man laugh), while retaining the ability to apply such generalizations productively when appropriate? This question has long been seen as one that is both particularly central to acquisition research and particularly challenging. Focussing on causative overgeneralization errors of this type, a previous study reported a computational model that learns, on the basis of corpus data and human-derived verb-semantic-feature ratings, to predict adults' by-verb preferences for less- versus more-transparent causative forms (e.g., * The clown laughed the man vs The clown made the man laugh) across English, Hebrew, Hindi, Japanese and K'iche Mayan. Here, we tested the ability of this model (and an expanded version with multiple hidden layers) to explain binary grammaticality judgment data from children aged 4;0-5;0, and elicited-production data from children aged 4;0-5;0 and 5;6-6;6 ( N=48 per language). In general, the model successfully simulated both children's judgment and production data, with correlations of r=0.5-0.6 and r=0.75-0.85, respectively, and also generalized to unseen verbs. Importantly, learners of all five languages showed some evidence of making the types of overgeneralization errors - in both judgments and production - previously observed in naturalistic studies of English (e.g., *I'm dancing it). Together with previous findings, the present study demonstrates that a simple learning model can explain (a) adults' continuous judgment data, (b) children's binary judgment data and (c) children's production data (with no training of these datasets), and therefore constitutes a plausible mechanistic account of the acquisition of verbs' argument structure restrictions.

Visual statistical learning is facilitated in Zipfian distributions.

Humans can extract co-occurrence regularities from their environment, and use them for learning. This statistical learning ability (SL) has been studied extensively as a way to explain how we learn the structure of our environment. These investigations have illustrated the impact of various distributional properties on learning. However, almost all SL studies present the regularities to be learned in uniform frequency distributions where each unit (e.g., image triplet) appears the same number of times: While the regularities themselves are informative, the appearance of the units cannot be predicted. In contrast, real-world learning environments, including the words children hear and the objects they see, are not uniform. Recent research shows that word segmentation is facilitated in a skewed (Zipfian) distribution. Here, we examine the domain-generality of the effect and ask if visual SL is also facilitated in a Zipfian distribution. We use an existing database to show that object combinations have a skewed distribution in children's environment. We then show that children and adults showed better learning in a Zipfian distribution compared to a uniform one, overall, and for low-frequency triplets. These results illustrate the facilitative impact of skewed distributions on learning across modality and age; suggest that the use of uniform distributions may underestimate performance; and point to the possible learnability advantage of such distributions in the real-world.

Starting Big: The Effect of Unit Size on Language Learning in Children and Adults.

Multiword units play an important role in language learning and use. It was proposed that learning from such units can facilitate mastery of certain grammatical relations, and that children and adults differ in their use of multiword units during learning, contributing to their varying language-learning trajectories. Accordingly, adults learn gender agreement better when encouraged to learn from multiword units. Previous work has not examined two core predictions of this proposal: (1) that children also benefit from initial exposure to multiword units, and (2) that their learning patterns reflect a greater reliance on multiword units compared to adults. We test both predictions using an artificial-language. As predicted, both children and adults benefit from early exposure to multiword units. In addition, when exposed to unsegmented input - adults show better learning of nouns compared to article-noun pairings, but children do not, a pattern consistent with adults' predicted tendency to focus less on multiword units.

Individual Differences in Learning Abilities Impact Structure Addition: Better Learners Create More Structured Languages.

Over the last decade, iterated learning studies have provided compelling evidence for the claim that linguistic structure can emerge from non-structured input, through the process of transmission. However, it is unclear whether individuals differ in their tendency to add structure, an issue with implications for understanding who are the agents of change. Here, we identify and test two contrasting predictions: The first sees learning as a pre-requisite for structure addition, and predicts a positive correlation between learning accuracy and structure addition, whereas the second maintains that it is those learners who struggle with learning and reproducing their input who add structure to it. This prediction is hard to test in standard iterated learning paradigms since each learner is exposed to a different input, and since structure and accuracy are computed using the same test items. Here, we test these contrasting predictions in two experiments using a one-generation artificial language learning paradigm designed to provide independent measures of learning accuracy and structure addition. Adults (N = 48 in each study) were exposed to a semi-regular language (with probabilistic structure) and had to learn it: Learning was assessed using seen items, whereas structure addition was calculated over unseen items. In both studies, we find a strong positive correlation between individuals' ability to learn the language and their tendency to add structure to it: Better learners also produced more structured languages. These findings suggest a strong link between learning and generalization. We discuss the implications of these findings for iterated language models and theories of language change more generally.

A learning bias for word order harmony: Evidence from speakers of non-harmonic languages.

Word order harmony describes the tendency, found across the world's languages, to consistently order syntactic heads relative to dependents. It is one of the most well-known and well-studied typological universals. Almost since it was first noted by Greenberg (1963), there has been disagreement about what role, if any, the cognitive system plays in driving harmony. Recently, a series of studies using artificial language learning experiments reported that harmonic noun phrase word orders were preferred over non-harmonic orders by English-speaking adults and children (Culbertson et al., 2012; Culbertson & Newport, 2015, 2017). However, this evidence is potentially confounded by the fact that English is itself a harmonic language (Goldberg, 2013). Here we sought to extend the results from these studies by exploring whether learners who have substantial experience with a non-harmonic language still showed a bias for harmonic patterns during learning. We found that monolingual French- and Hebrew-speaking children, whose language has a non-harmonic noun phrase order (N Adj, Num N) nevertheless preferred harmonic patterns when learning an artificial language. We also found evidence for a harmony bias across several populations of adult learners, although this interacted in complex ways with their L2 experience. Our results suggest that transfer from the L1 cannot explain the preference for harmony found in previous studies. Moreover, they provide the strongest evidence yet that a cognitive bias for harmony is a plausible candidate for shaping linguistic typology.

The crosslinguistic acquisition of sentence structure: Computational modeling and grammaticality judgments from adult and child speakers of English, Japanese, Hindi, Hebrew and K'iche'.

This preregistered study tested three theoretical proposals for how children form productive yet restricted linguistic generalizations, avoiding errors such as *The clown laughed the man, across three age groups (5-6 years, 9-10 years, adults) and five languages (English, Japanese, Hindi, Hebrew and K'iche'). Participants rated, on a five-point scale, correct and ungrammatical sentences describing events of causation (e.g., *Someone laughed the man; Someone made the man laugh; Someone broke the truck; ?Someone made the truck break). The verb-semantics hypothesis predicts that, for all languages, by-verb differences in acceptability ratings will be predicted by the extent to which the causing and caused event (e.g., amusing and laughing) merge conceptually into a single event (as rated by separate groups of adult participants). The entrenchment and preemption hypotheses predict, for all languages, that by-verb differences in acceptability ratings will be predicted by, respectively, the verb's relative overall frequency, and frequency in nearly-synonymous constructions (e.g., X made Y laugh for *Someone laughed the man). Analysis using mixed effects models revealed that entrenchment/preemption effects (which could not be distinguished due to collinearity) were observed for all age groups and all languages except K'iche', which suffered from a thin corpus and showed only preemption sporadically. All languages showed effects of event-merge semantics, except K'iche' which showed only effects of supplementary semantic predictors. We end by presenting a computational model which successfully simulates this pattern of results in a single discriminative-learning mechanism, achieving by-verb correlations of around r = 0.75 with human judgment data.

Do current statistical learning tasks capture stable individual differences in children? An investigation of task reliability across modality.

Do commonly used statistical-learning tasks capture stable individual differences in children? Infants, children, and adults are capable of using statistical learning (SL) to extract information about their environment. Although most studies have looked at group-level performance, a growing literature examines individual differences in SL and their relation to language-learning outcomes: Individuals who are better at SL are expected to show better linguistic abilities. Accordingly, studies have shown positive correlations between SL performance and language outcomes in both children and adults. However, these studies have often used tasks designed to explore group-level performance without modifying them, resulting in psychometric shortcomings that impact reliability in adults (Siegelman, Bogaerts, Christiansen, & Frost in Transactions of the Royal Society B, 372, 20160059, 2017a; Siegelman, Bogaerts, & Frost in Behavior Research Methods, 49, 418-432, 2017b). Even though similar measures are used to assess individual differences in children, no study to date has examined the reliability of these measures in development. This study examined the reliability of common SL measures in both children and adults. It assessed the reliability of three SL tasks (two auditory and one visual) twice (two months apart) in adults and children (mean age 8 years). Although the tasks showed moderate reliability in adults, they did not capture stable individual variation in children. None of the tasks were reliable across sessions, and all showed internal consistency measures well below psychometric standards. These findings raise significant concerns about the use of current SL measures to predict and explain individual differences in development. The article ends with a discussion of possible explanations for the difference in reliability between children and adults.

Statistical Learning, Implicit Learning, and First Language Acquisition: A Critical Evaluation of Two Developmental Predictions.

The role of distributional information in language learning, and learning more generally, has been studied extensively in both the statistical learning and the implicit learning literatures. Despite the similarity in research questions, the two literatures have remained largely separate. Here, we draw on findings from the two traditions to critically evaluate two developmental predictions that are central to both. The first is the question of age invariance: Does learning improve during development or is it fully developed in infancy? The combined findings suggest that both implicit and statistical learning improve during childhood, contra the age invariance prediction. This raises questions about the role of implicit statistical learning (ISL) in explaining the age-related deterioration in language learning: Children's better language learning abilities cannot be attributed to their improved distributional learning skills. The second issue we examine is the predictive relation to language outcomes: Does variation in learning predict variation in language outcomes? While there is evidence for such links, there is concern in both research traditions about the reliability of the tasks used with children. We present data suggesting that commonly used statistical learning measures may not capture stable individual differences in children, undermining their utility for assessing the link to language outcomes in developmental samples. The evaluation of both predictions highlights the empirical parallels between the implicit and statistical learning literatures, and the need to better integrate their developmental investigation. We go on to discuss several of the open challenges facing the study of ISL during development.

Statistical Learning Is Not Age-Invariant During Childhood: Performance Improves With Age Across Modality.

Humans are capable of extracting recurring patterns from their environment via statistical learning (SL), an ability thought to play an important role in language learning and learning more generally. While much work has examined statistical learning in infants and adults, less work has looked at the developmental trajectory of SL during childhood to see whether it is fully developed in infancy or improves with age, like many other cognitive abilities. A recent study showed modality-based differences in the effect of age during childhood: While visual SL improved with age, auditory SL did not. This finding was taken as evidence for modality-based differences in SL. However, since that study used auditory linguistic stimuli (syllables), the differential effect of age may have been driven by stimulus type (linguistic vs. non-linguistic) rather than modality. Here, we ask whether age will affect performance similarly in the two modalities when non-linguistic auditory stimuli are used (familiar sounds instead of syllables). We conduct a large-scale study of children's performance on visual and non-linguistic auditory SL during childhood (ages 5-12 years). The results show a similar effect of age in both modalities: Unlike previous findings, both visual and non-linguistic auditory SL improved with age. These findings highlight the stimuli-sensitive nature of SL and suggest that modality-based differences may be stimuli-dependent, and that age-invariance may be limited to linguistic stimuli.

Systematicity, but not compositionality: Examining the emergence of linguistic structure in children and adults using iterated learning.

Recent work suggests that cultural transmission can lead to the emergence of linguistic structure as speakers' weak individual biases become amplified through iterated learning. However, to date no published study has demonstrated a similar emergence of linguistic structure in children. The lack of evidence from child learners constitutes a problematic gap in the literature: if such learning biases impact the emergence of linguistic structure, they should also be found in children, who are the primary learners in real-life language transmission. However, children may differ from adults in their biases given age-related differences in general cognitive skills. Moreover, adults' performance on iterated learning tasks may reflect existing (and explicit) linguistic biases, partially undermining the generality of the results. Examining children's performance can also help evaluate contrasting predictions about their role in emerging languages: do children play a larger or smaller role than adults in the creation of structure? Here, we report a series of four iterated artificial language learning studies (based on Kirby, Cornish & Smith, 2008) with both children and adults, using a novel child-friendly paradigm. Our results show that linguistic structure does not emerge more readily in children compared to adults, and that adults are overall better in both language learning and in creating linguistic structure. When languages could become underspecified (by allowing homonyms), children and adults were similar in developing consistent mappings between meanings and signals in the form of structured ambiguities. However, when homonimity was not allowed, only adults created compositional structure. This study is a first step in using iterated language learning paradigms to explore child-adult differences. It provides the first demonstration that cultural transmission has a different effect on the languages produced by children and adults: While children were able to develop systematicity, their languages did not show compositionality. We focus on the relation between learning and structure creation as a possible explanation for our findings and discuss implications for children's role in the emergence of linguistic structure.

SES effects on the use of variation sets in child-directed speech.

Socio-economic status (SES) impacts the amount and type of input children hear in ways that have developmental consequences. Here, we examine the effect of SES on the use of variation sets (successive utterances with partial self-repetitions) in child-directed speech (CDS). Variation sets have been found to facilitate language learning, but have been studied only in higher-SES groups. Here, we examine their use in naturalistic speech in two languages (Hebrew and English) for both low- and high-SES caregivers. We find that variation sets are more frequent in the input of high-SES caregivers in both languages, indicating that SES also impacts structural properties of CDS.