Model-free and model-based reward prediction errors in EEG.

Learning theorists posit two reinforcement learning systems: model-free and model-based. Model-based learning incorporates knowledge about structure and contingencies in the world to assign candidate actions with an expected value. Model-free learning is ignorant of the world's structure; instead, actions hold a value based on prior reinforcement, with this value updated by expectancy violation in the form of a reward prediction error. Because they use such different learning mechanisms, it has been previously assumed that model-based and model-free learning are computationally dissociated in the brain. However, recent fMRI evidence suggests that the brain may compute reward prediction errors to both model-free and model-based estimates of value, signalling the possibility that these systems interact. Because of its poor temporal resolution, fMRI risks confounding reward prediction errors with other feedback-related neural activity. In the present study, EEG was used to show the presence of both model-based and model-free reward prediction errors and their place in a temporal sequence of events including state prediction errors and action value updates. This demonstration of model-based prediction errors questions a long-held assumption that model-free and model-based learning are dissociated in the brain.

I: INTRODUCTION.

The majority of the world's children grow up learning two or more languages. The study of early bilingualism is central to current psycholinguistics, offering insights into issues such as transfer and interference in development. From an applied perspective, it poses a universal challenge to language assessment practices throughout childhood, as typically developing bilingual children usually underperform relative to monolingual norms when assessed in one language only. We measured vocabulary with Communicative Development Inventories for 372 24-month-old toddlers learning British English and one Additional Language out of a diverse set of 13 (Bengali, Cantonese, Dutch, French, German, Greek, Hindi-Urdu, Italian, Mandarin, Polish, Portuguese, Spanish, and Welsh). We furthered theoretical understanding of bilingual development by showing, for the first time, that linguistic distance between the child's two languages predicts vocabulary outcome, with phonological overlap related to expressive vocabulary, and word order typology and morphological complexity related to receptive vocabulary, in the Additional Language. Our study also has crucial clinical implications: we have developed the first bilingual norms for expressive and receptive vocabulary for 24-month-olds learning British English and an Additional Language. These norms were derived from factors identified as uniquely predicting CDI vocabulary measures: the relative amount of English versus the Additional Language in child-directed input and parental overheard speech, and infant gender. The resulting UKBTAT tool was able to accurately predict the English vocabulary of an additional group of 58 bilinguals learning an Additional Language outside our target range. This offers a pragmatic method for the assessment of children in the majority language when no tool exists in the Additional Language. Our findings also suggest that the effect of linguistic distance might extend beyond bilinguals' acquisition of early vocabulary to encompass broader cognitive processes, and could constitute a key factor in the study of the debated bilingual advantage.

Comparing phoneme frequency, age of acquisition, and loss in aphasia: Implications for phonological universals.

Phonological complexity may be central to the nature of human language. It may shape the distribution of phonemes and phoneme sequences within languages, but also determine age of acquisition and susceptibility to loss in aphasia. We evaluated this claim using frequency statistics derived from a corpus of phonologically transcribed Italian words (phonitalia, available at phonitalia,org), rankings of phoneme age of acquisition (AoA) and rate of phoneme errors in patients with apraxia of speech (AoS) as an indication of articulatory complexity. These measures were related to cross-linguistically derived markedness rankings. We found strong correspondences. AoA, however, was predicted by both apraxic errors and frequency, suggesting independent contributions of these variables. Our results support the reality of universal principles of complexity. In addition they suggest that these complexity principles have articulatory underpinnings since they modulate the production of patients with AoS, but not the production of patients with more central phonological difficulties.

Principal components analysis of reward prediction errors in a reinforcement learning task.

Models of reinforcement learning represent reward and punishment in terms of reward prediction errors (RPEs), quantitative signed terms describing the degree to which outcomes are better than expected (positive RPEs) or worse (negative RPEs). An electrophysiological component known as feedback related negativity (FRN) occurs at frontocentral sites 240-340ms after feedback on whether a reward or punishment is obtained, and has been claimed to neurally encode an RPE. An outstanding question however, is whether the FRN is sensitive to the size of both positive RPEs and negative RPEs. Previous attempts to answer this question have examined the simple effects of RPE size for positive RPEs and negative RPEs separately. However, this methodology can be compromised by overlap from components coding for unsigned prediction error size, or "salience", which are sensitive to the absolute size of a prediction error but not its valence. In our study, positive and negative RPEs were parametrically modulated using both reward likelihood and magnitude, with principal components analysis used to separate out overlying components. This revealed a single RPE encoding component responsive to the size of positive RPEs, peaking at ~330ms, and occupying the delta frequency band. Other components responsive to unsigned prediction error size were shown, but no component sensitive to negative RPE size was found.

English-learning one- to two-year-olds do not show a consonant bias in word learning.

Following the proposal that consonants are more involved than vowels in coding the lexicon (Nespor, Peña & Mehler, 2003), an early lexical consonant bias was found from age 1;2 in French but an equal sensitivity to consonants and vowels from 1;0 to 2;0 in English. As different tasks were used in French and English, we sought to clarify this ambiguity by using an interactive word-learning study similar to that used in French, with British-English-learning toddlers aged 1;4 and 1;11. Children were taught two CVC labels differing on either a consonant or vowel and tested on their pairing of a third object named with one of the previously taught labels, or part of them. In concert with previous research on British-English toddlers, our results provided no evidence of a general consonant bias. The language-specific mechanisms explaining the differential status for consonants and vowels in lexical development are discussed.

PhonItalia: a phonological lexicon for Italian.

In this article, we present the first open-access lexical database that provides phonological representations for 120,000 Italian word forms. Each of these also includes syllable boundaries and stress markings and a comprehensive range of lexical statistics. Using data derived from this lexicon, we have also generated a set of derived databases and provided estimates of positional frequency use for Italian phonemes, syllables, syllable onsets and codas, and character and phoneme bigrams. These databases are freely available from phonitalia.org. This article describes the methods, content, and summarizing statistics for these databases. In a first application of this database, we also demonstrate how the distribution of phonological substitution errors made by Italian aphasic patients is related to phoneme frequency.

The irrepressible influence of vocal stereotypes on trust.

There is a reciprocal relationship between trust and vocal communication in human interactions. On one hand, a predisposition towards trust is necessary for communication to be meaningful and effective. On the other hand, we use vocal cues to signal our own trustworthiness and to infer it from the speech of others. Research on trustworthiness attributions to vocal characteristics is scarce and contradictory, however, being typically based on explicit judgements which may not predict actual trust-oriented behaviour. We use a game theory paradigm to examine the influence of speaker accent and prosody on trusting behaviour towards a simulated game partner, who responds either trustworthily or untrustworthily in an investment game. We found that speaking in a non-regional standard accent increases trust, as does relatively slow articulation rate. The effect of accent persists over time, despite the accumulation of clear evidence regarding the speaker's level of trustworthiness in a negotiated interaction. Accents perceived as positive for trust can maintain this benefit even in the face of behavioural evidence of untrustworthiness.

Electrophysiological examination of embodiment in vision and action.

A wealth of behavioral data has shown that the visual properties of objects automatically potentiate motor actions linked with them, but how deeply are these affordances embedded in visual processing? In the study reported here, we used electrophysiological measures to examine the time course of affordance resulting from the leftward or rightward orientation of the handles of common objects. Participants were asked to categorize those objects using a left- or right-handed motor response. Lateralized readiness potentials showed rapid motor preparation in the hand congruent with the affordance provided by the object only 100 to 200 ms after stimulus presentation and up to 400 ms before the actual response. Examination of event-related potentials also revealed an effect of handle orientation and response-hand congruency on the visual P1 and N1 components. Both of these results suggest that activity in the early sensory pathways is modulated by the action associations of objects and the intentions of the viewer.

Object affordance influences instruction span.

We measured memory span for assembly instructions involving objects with handles oriented to the left or right side. Right-handed participants remembered more instructions when objects' handles were spatially congruent with the hand used in forthcoming assembly actions. No such affordance-based memory benefit was found for left-handed participants. These results are discussed in terms of motor simulation as an embodied rehearsal mechanism.

Attention deployment during memorizing and executing complex instructions.

We investigated the mental rehearsal of complex action instructions by recording spontaneous eye movements of healthy adults as they looked at objects on a monitor. Participants heard consecutive instructions, each of the form "move [object] to [location]". Instructions were only to be executed after a go signal, by manipulating all objects successively with a mouse. Participants re-inspected previously mentioned objects already while listening to further instructions. This rehearsal behavior broke down after 4 instructions, coincident with participants' instruction span, as determined from subsequent execution accuracy. These results suggest that spontaneous eye movements while listening to instructions predict their successful execution.

Lexical stress and phonetic processing in word learning in 20- to 24-month-old English-learning children.

To investigate the interaction between segmental and supra-segmental stress-related information in early word learning, two experiments were conducted with 20- to 24-month-old English-learning children. In an adaptation of the object categorization study designed by Nazzi and Gopnik (2001), children were presented with pairs of novel objects whose labels differed by their initial consonant (Experiment 1) or their medial consonant (Experiment 2). Words were produced with a stress initial (trochaic) or a stress final (iambic) pattern. In both experiments successful word learning was established when the to-be-remembered contrast was embedded in a stressed syllable, but not when embedded in unstressed syllables. This was independent of the overall word pattern, trochaic or iambic, or the location of the phonemic contrast, word-initial or -medial. Results are discussed in light of the use of phonetic information in early lexical acquisition, highlighting the role of lexical stress and ambisyllabicity in early word processing.

Infants' Discrimination of Familiar and Unfamiliar Accents in Speech.

This study investigates infants' discrimination abilities for familiar and unfamiliar regional English accents. Using a variation of the head-turn preference procedure, 5-month-old infants demonstrated that they were able to distinguish between their own South-West English accent and an unfamiliar Welsh English accent. However, this distinction was not seen when two unfamiliar accents (Welsh English and Scottish English) were presented to the infants, indicating they had not acquired the general ability to distinguish between regional varieties, but only the distinction between their home accent and unfamiliar regional variations. This ability was also confirmed with 7-month-olds, challenging recent claims that infants lose their sensitivity to dialects at around that age. Taken together, our results argue in favor of an early sensitivity to the intonation system of languages, and to the early learning of accent-specific intonation and potentially segmental patterns. Implications for the development of accent normalization abilities are discussed.

Tool use and function knowledge shape visual object processing.

Perceiving the environment automatically informs how we can interact with it through affordance mechanisms. However, it remains unknown how our knowledge about the environment shapes how it is perceived. In this training study, we evaluated whether motor and function knowledge about novel objects affects visual object processing. Forty-three participants associated a usage or function to a novel object in interactive virtual reality while their EEG was recorded. Both usage and function influenced the mu-band (8-12 Hz) rhythms, suggesting that motor and function object information influence motor processing during object recognition. Learning the usage also prevented the reduction of the theta-band (4-8 Hz) rhythms recorded over the posterior cortical areas, suggesting a predominant top-down influence of tool use information on visuo-motor pathways. The modulation being specifically induced by learning an object usage, the results support further the embodied cognition approach rather than the reasoning-based approach of object processing.

The functional origin of the foreign accent: evidence from the syllable-frequency effect in bilingual speakers.

Individuals who speak more than one language often do so with a foreign accent in their second language. Previous investigations have focused on the acoustic phonetic properties of speech, showing how language-learning history shapes the occurrence of accent. By contrast, little is known about the phonological and phonetic representations that allow the production of each language within one speaker. We investigated this issue via the syllable-frequency effect, thought to index the retrieval of syllable-size representations during speech production. We tested French-Spanish early and late bilinguals in a task in which the materials' syllabic frequency in both languages was manipulated. The frequency of syllables in the nonspoken language affected performance only with late bilinguals. This is interpreted as evidence that syllabic representations are shared across languages in late bilinguals but are separate in early bilinguals. One of the functional origins of foreign accent in late bilinguals may be the retrieval of syllabic representations shared across languages.

Labels strengthen motor learning of new tools.

Recent reformulations of the Sapir-Whorf hypothesis have shown how labels can guide our thinking in situations of uncertainty, facilitating the identification of objects. We examine whether the effect of labels extends beyond perceptual processes, to help us learn the motoric manipulations required to use novel tools. Exploiting immersive virtual reality, we measured behavioural movement latencies and electrophysiological activity from participants learning to use a range of labeled and unlabeled novel tools. We found that providing a tool with a label reduced the time taken to reach for it, with participants also faster and more accurate when executing the manipulations required to use it. Conversely, labels did not confer any facilitation when the tool was simply moved to another location; participants were slower to grasp a labeled tool when asked to transport it. These findings were also supported by electrophysiological recordings, showing a reduction in sensorimotor beta-band (~30 Hz) power when participants were asked to use the labeled tools, but not move them. This modulation of beta activity indicates augmented learning of motor-activity related to tool use within somatosensory regions due to the activation of its lexical representation. These results suggest an extension of the Whorfian hypothesis, such that language not only modulates our thoughts and perceptual processes, but also affects our actions with objects and tools. We propose that labels tune our somatosensory experience and help to memorize body states related to tool use by creating an invariant lexical anchor on which we can build motor learning and experience.

Regional and foreign accent processing in English: can listeners adapt?

Recent data suggest that the first presentation of a foreign accent triggers a delay in word identification, followed by a subsequent adaptation. This study examines under what conditions the delay resumes to baseline level. The delay will be experimentally induced by the presentation of sentences spoken to listeners in a foreign or a regional accent as part of a lexical decision task for words placed at the end of sentences. Using a blocked design of accents presentation, Experiment 1 shows that accent changes cause a temporary perturbation in reaction times, followed by a smaller but long-lasting delay. Experiment 2 shows that the initial perturbation is dependent on participants' expectations about the task. Experiment 3 confirms that the subsequent long-lasting delay in word identification does not habituate after repeated exposure to the same accent. Results suggest that comprehensibility of accented speech, as measured by reaction times, does not benefit from accent exposure, contrary to intelligibility.