Systematic testing of three Language Models reveals low language accuracy, absence of response stability, and a yes-response bias.

Humans are universally good in providing stable and accurate judgments about what forms part of their language and what not. Large Language Models (LMs) are claimed to possess human-like language abilities; hence, they are expected to emulate this behavior by providing both stable and accurate answers, when asked whether a string of words complies with or deviates from their next-word predictions. This work tests whether stability and accuracy are showcased by GPT-3/text-davinci-002, GPT-3/text-davinci-003, and ChatGPT, using a series of judgment tasks that tap on 8 linguistic phenomena: plural attraction, anaphora, center embedding, comparatives, intrusive resumption, negative polarity items, order of adjectives, and order of adverbs. For every phenomenon, 10 sentences (5 grammatical and 5 ungrammatical) are tested, each randomly repeated 10 times, totaling 800 elicited judgments per LM (total n = 2,400). Our results reveal variable above-chance accuracy in the grammatical condition, below-chance accuracy in the ungrammatical condition, a significant instability of answers across phenomena, and a yes-response bias for all the tested LMs. Furthermore, we found no evidence that repetition aids the Models to converge on a processing strategy that culminates in stable answers, either accurate or inaccurate. We demonstrate that the LMs' performance in identifying (un)grammatical word patterns is in stark contrast to what is observed in humans (n = 80, tested on the same tasks) and argue that adopting LMs as theories of human language is not motivated at their current stage of development.

The origins of music in (musi)language.

The view of music as a byproduct of other cognitive functions has been deemed incomplete or incorrect. Revisiting the six lines of evidence that support this conclusion, it is argued that it is unclear how the hypothesis that music has its origins in (musi)language is discarded. Two additional promising research lines that can support or discard the byproduct hypothesis are presented.

The role of minority language bilingualism in spotting agreement attraction errors: Evidence from Italian varieties.

Bilingual adaptations remain a subject of ongoing debate, with varying results reported across cognitive domains. A possible way to disentangle the apparent inconsistency of results is to focus on the domain of language processing, which is what the bilingual experience boils down to. This study delves into the role of the bilingual experience on the processing of agreement mismatches. Given the underrepresentation of minority bilingual speakers of non-standard varieties, we advance a unique comparative perspective that includes monolinguals, standard language bilinguals, and different groups of minority language bilinguals, taking advantage of the rich linguistic diversity of the Italian peninsula. This comparative approach can reveal the impact of various sociolinguistic aspects of the bilingual experience across different bilingual trajectories. We developed an auditory acceptability judgement task in Italian, featuring Subject-Verb agreement mismatches. Participants evaluated the stimuli on a 5-point Likert scale and reaction times were recorded. The results do not reveal significant differences between the speakers of standard languages: Italian monolinguals and Italian-Spanish bilinguals. Instead, significant differences are found between monolinguals and the two groups of minority language bidialectals, as well as between the bidialectal groups themselves: Italian-Pavese bidialectals were faster than both Italian-Agrigentino bidialectals and Italian monolinguals, while Italian-Agrigentino bidialectals were less accurate than both Italian-Pavese bidialectals and Italian monolinguals. This intricate picture is explained through variables associated with second language use and language switching. Our findings suggest that if bilingualism is viewed as a yes/no phenotype, it is unavoidable that the bilingual experience will remain a mystery linked to intensely debated results. If, however, one accepts that bilingual adaptations are shaped by the environmental ecology of each trajectory, variation across bilingual processing outcomes is unsurprising. Overall, we argue that specific sociolinguistic factors behind each bilingual experience can reveal where bilingual adaptations on language and cognition stem from.

Bilingual disadvantages are systematically compensated by bilingual advantages across tasks and populations.

Bilingualism is linked to both enhanced and hampered performance in various cognitive measures, yet the extent to which these bilingual advantages and disadvantages co-occur is unclear. To address this gap, we perform a systematic review and two quantitative analyses. First, we analyze results from 39 studies, obtained through the PRISMA method. Less than 50% of the studies that show up as results for the term "bilingual disadvantage" report exclusively a disadvantage, that shows bilinguals performing worse than monolinguals in a task. A Bayesian analysis reveals robust evidence for bilingual effects, but no evidence for differences in the proportion of advantages and disadvantages, suggesting that when results from different cognitive domains such as executive functions and verbal fluency are analyzed together, bilingual effects amount to a zero-sum game. This finding was replicated by repeating the analysis, using the datasets of two recent meta-analyses. We propose that the equilibrium we observe between positive and negative outcomes may not be accidental. Contrary to widespread belief, advantageous and disadvantageous effects are not stand-alone outcomes in free variation. We reframe them as the connatural components of a dynamic trade-off, whereby enhanced performance in one cognitive measure is offset by an incurred cost in another domain.

A Classification Bias and an Exclusion Bias Jointly Overinflated the Estimation of Publication Biases in Bilingualism Research.

A publication bias has been argued to affect the fate of results in bilingualism research. It was repeatedly suggested that studies presenting evidence for bilingual advantages are more likely to be published compared to studies that do not report results in favor of the bilingual advantage hypothesis. This work goes back to the original claim and re-examines both the dataset and the classification of the studies that were employed. We find that the exclusion of published works such as doctoral dissertations, book chapters, and conference proceedings from the original dataset significantly inflated the presumed publication bias. Moreover, the estimation of the publication bias was affected by a classification bias that uses a mega-category that consists of both null and negative outcomes. Yet finding evidence for a bilingual disadvantage is not synonymous with obtaining a result indistinguishable from zero. Consequently, grouping together null and negative findings in a mega-category has various ramifications, not only for the estimation of the presumed publication bias but also for the field's ability to appreciate the insofar hidden correlations between bilingual advantages and disadvantages. Tracking biases that inflate scientific results is important, but it is not enough. The next step is recognizing the nested Matryoshka doll effect of bias-within-bias, and this entails raising awareness for one's own bias blind spots in science.

Bilinguals are better than monolinguals in detecting manipulative discourse.

One of the most contentious topics in cognitive science concerns the impact of bilingualism on cognitive functions and neural resources. Research on executive functions has shown that bilinguals often perform better than monolinguals in tasks that require monitoring and inhibiting automatic responses. The robustness of this effect is a matter of an ongoing debate, with both sides approaching bilingual cognition mainly through measuring abilities that fall outside the core domain of language processing. However, the mental juggling that bilinguals perform daily involves language. This study takes a novel path to bilingual cognition by comparing the performance of monolinguals and bilinguals in a timed task that features a special category of stimulus, which has the peculiar ability to manipulate the cognitive parser into treating it as well-formed while it is not: grammatical illusions. The results reveal that bilinguals outperform monolinguals in detecting illusions, but they are also slower across the board in judging the stimuli, illusory or not. We capture this trade-off by proposing the Plurilingual Adaptive Trade-off Hypothesis (PATH), according to which the adaptation of bilinguals' cognitive abilities may (i) decrease fallibility to illusions by means of recruiting sharpened top-down control processes, but (ii) this is part of a larger bundle of effects, not all of which are necessarily advantageous.

Acceptable Ungrammatical Sentences, Unacceptable Grammatical Sentences, and the Role of the Cognitive Parser.

A search for the terms "acceptability judgment tasks" and "language" and "grammaticality judgment tasks" and "language" produces results which report findings that are based on the exact same elicitation technique. Although certain scholars have argued that acceptability and grammaticality are two separable notions that refer to different concepts, there are contexts in which the two terms are used interchangeably. The present work reaffirms that these two notions and their scales do not coincide: there are sentences that are acceptable, even though they are ungrammatical, and sentences that are unacceptable, despite being grammatical. First, we adduce a number of examples for both cases, including grammatical illusions, violations of Identity Avoidance, and sentences that involve a level of processing complexity that overloads the cognitive parser and tricks it into (un)acceptability. We then discuss whether the acceptability of grammatically ill-formed sentences entails that we assign a meaning to them. Last, it is shown that there are n ways of unacceptability, and two ways of ungrammaticality, in the absolute and the relative sense. Since the use of the terms "acceptable" and "grammatical" is often found in experiments that constitute the core of the evidential base of linguistics, disentangling their various uses is likely to aid the field reach a better level of terminological clarity.

Universal linguistic hierarchies are not innately wired. Evidence from multiple adjectives.

BACKGROUND: Linguists and psychologists have explained the remarkable similarities in the orderings of linguistic elements across languages by suggesting that our inborn ability for language makes available certain innately wired primitives. Different types of adjectives, adverbs, and other elements in the functional spine are considered to occupy fixed positions via innate hierarchies that determine orderings such as A>B>C, banning other permutations (*B>C>A). The goal of this research is to tap into the nature and rigidity of such hierarchies by comparing what happens when people process orderings that either comply with them or violate them. METHOD: N = 170 neurotypical, adult speakers completed a timed forced choice task that featured stimuli showing a combination of two adjectives and a Spelke-object (e.g., 'I bought a square black table'). Two types of responses were collected: (i) acceptability judgments on a 3-point Likert scale that featured the options 'correct', 'neither correct nor wrong', and 'wrong' and (ii) reaction times. The task featured three conditions: 1. size adjective > nationality adjective, 2. color adjective > shape adjective, 3. subjective comment adjective > material adjective. Each condition had two orders. In the congruent order, the adjective pair was ordered in agreement with what is traditionally accepted as dictated by the universal hierarchy. In the incongruent order, the ordering was reversed, thus the hierarchy was violated. RESULTS: In the first experiment, the results of n = 140 monolinguals showed that across conditions, both congruent and incongruent orders were generally accepted as correct. For 2/3 conditions, the difference in acceptability ratings between congruent and incongruent orders did not reach statistical significance. Using time as a window to processing, reaction times showed that incongruent orders do not take longer to process than congruent ones, as should be the case if the former were treated as being licensed under some type of special condition (e.g., contrastive focus) that reverses the unmarked order and legitimizes the violation of the hierarchy. In the second experiment, the results of n = 30 bidialectals, tested in both language varieties, corroborated the findings of the first experiment. CONCLUSIONS: Our findings do not provide evidence for an innate hierarchy for adjective ordering that imposes one rigid, unmarked order. We discuss the importance of notions such as subjectivity and inherentness, and show that for some conditions, not only is there no evidence for a hard constraint that bans incongruent orders, but even simple preferences of congruent orders over incongruent ones are hard to discern. Capturing the bigger picture, given that both the hierarchies and their legit permutations have been described as innate, our results reduce the amount of primitives that are cast as innate, eventually offering a deflationist approach to human linguistic cognition.

Eliciting Big Data From Small, Young, or Non-standard Languages: 10 Experimental Challenges.

The aim of this work is to identify and analyze a set of challenges that are likely to be encountered when one embarks on fieldwork in linguistic communities that feature small, young, and/or non-standard languages with a goal to elicit big sets of rich data. For each challenge, we (i) explain its nature and implications, (ii) offer one or more examples of how it is manifested in actual linguistic communities, and (iii) where possible, offer recommendations for addressing it effectively. Our list of challenges involves static characteristics (e.g., absence of orthographic conventions and how it affects data collection), dynamic processes (e.g., speed of language change in small languages and how it affects longitudinal collection of big amounts of data), and interactive relations between non-dynamic features that are nevertheless subject to potentially rapid change (e.g., absence of standardized assessment tools or estimates for psycholinguistic variables). The identified challenges represent the domains of data collection and handling, participant recruitment, and experimental design. Among other issues, we discuss population limits and degree of power, inter- and intraspeaker variation, absence of metalanguage and its implications for the process of eliciting acceptability judgments, and challenges that arise from absence of local funding, conflicting regulations in relation to privacy issues, and exporting large samples of data across countries. Finally, the ten experimental challenges presented are relevant to languages from a broad typological spectrum, encompassing both spoken and sign, extant and nearly extinct languages.

The Locus Preservation Hypothesis: Shared Linguistic Profiles across Developmental Disorders and the Resilient Part of the Human Language Faculty.

Grammatical markers are not uniformly impaired across speakers of different languages, even when speakers share a diagnosis and the marker in question is grammaticalized in a similar way in these languages. The aim of this work is to demarcate, from a cross-linguistic perspective, the linguistic phenotype of three genetically heterogeneous developmental disorders: specific language impairment, Down syndrome, and autism spectrum disorder. After a systematic review of linguistic profiles targeting mainly English-, Greek-, Catalan-, and Spanish-speaking populations with developmental disorders (n = 880), shared loci of impairment are identified and certain domains of grammar are shown to be more vulnerable than others. The distribution of impaired loci is captured by the Locus Preservation Hypothesis which suggests that specific parts of the language faculty are immune to impairment across developmental disorders. Through the Locus Preservation Hypothesis, a classical chicken and egg question can be addressed: Do poor conceptual resources and memory limitations result in an atypical grammar or does a grammatical breakdown lead to conceptual and memory limitations? Overall, certain morphological markers reveal themselves as highly susceptible to impairment, while syntactic operations are preserved, granting support to the first scenario. The origin of resilient syntax is explained from a phylogenetic perspective in connection to the "syntax-before-phonology" hypothesis.

Functionally Equivalent Variants in a Non-standard Variety and Their Implications for Universal Grammar: A Spontaneous Speech Corpus.

Findings from the field of experimental linguistics have shown that a native speaker may judge a variant that is part of her grammar as unacceptable, but still use it productively in spontaneous speech. The process of eliciting acceptability judgments from speakers of non-standard languages is sometimes clouded by factors akin to prescriptive notions of grammatical correctness. It has been argued that standardization enhances the ability to make clear-cut judgments, while non-standardization may result to grammatical hybridity, often manifested in the form of functionally equivalent variants in the repertoire of a single speaker. Recognizing the importance of working with corpora of spontaneous speech, this work investigates patterns of variation in the spontaneous production of five neurotypical, adult speakers of a non-standard variety in terms of three variants, each targeting one level of linguistic analysis: syntax, morphology, and phonology. The results reveal the existence of functionally equivalent variants across speakers and levels of analysis. We first discuss these findings in relation to the notions of competing, mixed, and fused grammars, and then we flesh out the implications that different values of the same variant carry for parametric approaches to Universal Grammar. We observe that intraspeaker realizations of different values of the same variant within the same syntactic environment are incompatible with the 'triggering-a-single-value' approach of parametric models, but we argue that they are compatible with the concept of Universal Grammar itself. Since the analysis of these variants is ultimately a way of investigating the status of Universal Grammar primitives, we conclude that claims about the alleged unfalsifiability of (the contents of) Universal Grammar are unfounded.

The Influence of Bilectalism and Non-standardization on the Perception of Native Grammatical Variants.

Research in speakers of closely related varieties has shown that bilectalism and non-standardization affect speakers' perception of the variants that exist in their native languages in a way that is absent from the performance of their monolingual peers. One possible explanation for this difference is that non-standardization blurs the boundaries of grammatical variants and increases grammatical fluidity. Affected by such factors, bilectals become less accurate in identifying the variety to which a grammatical variant pertains. Another explanation is that their differential performance derives from the fact that they are competent in two varieties. Under this scenario, the difference is due to the existence of two linguistic systems in the course of development, and not to how close or standardized these systems are. This study employs a novel variety-judgment task in order to elucidate which of the two explanations holds. Having administered the task to monolinguals, bilectals, and bilinguals, including heritage language learners and L1 attriters, we obtained a dataset of 16,245 sentences. The analysis shows differential performance between bilectal and bilingual speakers, granting support for the first explanation. We discuss the role of factors such as non-standardization and linguistic proximity in language development and flesh out the implications of the results in relation to different developmental trajectories.

Vision, language and a protective mechanism towards psychosis.

The absence of co-occurrence of schizophrenia with congenital/early blindness (CB) has led to the claim that CB confers protection against schizophrenia. It has recently been shown that the protective effects are particularly reinforced in cases of CB of cortical origin, since cases of CB of peripheral origin and schizophrenia in fact exist. The present work shows that the protection extends to psychosis more broadly and describes the brain basis of the protective mechanism and its relation to the language faculty and the language areas of the brain.

Schizophrenia and cortical blindness: protective effects and implications for language.

The repeatedly noted absence of case-reports of individuals with schizophrenia and congenital/early developed blindness has led several authors to argue that the latter can confer protective effects against the former. In this work, we present a number of relevant case-reports from different syndromes that show comorbidity of congenital and early blindness with schizophrenia. On the basis of these reports, we argue that a distinction between different types of blindness in terms of the origin of the visual deficit, cortical or peripheral, is crucial for understanding the observed patterns of comorbidity. We discuss the genetic underpinnings and the brain structures involved in schizophrenia and blindness, with insights from language processing, laying emphasis on the three structures that particularly stand out: the occipital cortex, the lateral geniculate nucleus (LGN), and the pulvinar. Last, we build on previous literature on the nature of the protective effects in order to offer novel insights into the nature of the protection mechanism from the perspective of the brain structures involved in each type of blindness.

Entangled parametric hierarchies: problems for an overspecified universal grammar.

This study addresses the feasibility of the classical notion of parameter in linguistic theory from the perspective of parametric hierarchies. A novel program-based analysis is implemented in order to show certain empirical problems related to these hierarchies. The program was developed on the basis of an enriched data base spanning 23 contemporary and 5 ancient languages. The empirical issues uncovered cast doubt on classical parametric models of language acquisition as well as on the conceptualization of an overspecified Universal Grammar that has parameters among its primitives. Pinpointing these issues leads to the proposal that (i) the (bio)logical problem of language acquisition does not amount to a process of triggering innately pre-wired values of parameters and (ii) it paves the way for viewing language, epigenetic ('parametric') variation as an externalization-related epiphenomenon, whose learning component may be more important than what sometimes is assumed.