Variations in Pupil Size Related to Memory for Recently Presented Words and Event Related Potentials.

Pupillometry has been found to be correlated with activity of cholinergic and noradrenergic neuromodulator systems. These systems regulate the level of cortical arousal and therefore perception, attention, and memory. Here, we tested how different types of pupil size variance (prestimulus baseline and prestimulus hippus power) may correlate with behavioral and event-related potentials (ERPs). We recorded pupil size and ERPs while participants were presented with a series of words and then asked whether the words had been in the initial list when they were later presented intermixed with unpresented words. We found that a smaller prestimulus baseline pupil size during the study phase was associated with better memory performance. Study items also evoked a larger P3 response at presentation and a greater old/new memory ERP effect at test when prestimulus pupil size was small rather than large. Prestimulus hippus power was found to be a between-subjects factor affecting the robustness of memory encoding with less power being associated with a greater old/new memory ERP effect. These results provide evidence relating memory and ERPs to variables defined on pupil size that are thought to reflect varying states of parasympathetic and sympathetic arousal.

Episodic traces and statistical regularities: Paired associate learning in typical and dyslexic readers.

Learning visual-phonological associations is a key skill underlying successful reading acquisition. However, we are yet to understand the cognitive mechanisms that enable efficient learning in good readers, and those which are aberrant in individuals with developmental dyslexia. Here, we use a repeated cued-recall task to examine how typical and reading-impaired adults acquire novel associations between visual and phonological stimuli, incorporating a looking-at-nothing paradigm to probe implicit memory for target locations. Cued recall accuracy revealed that typical readers' recall of novel phonological associates was better than dyslexic readers' recall, and it also improved more with repetition. Eye fixation-contingent error analyses suggest that typical readers' greater improvement from repetition reflects their more robust encoding and/or retrieval of each instance in which a given pair was presented: whereas dyslexic readers tended to recall a phonological target better when fixating its most recent location, typical readers showed this pattern more strongly when the target location was consistent across multiple trials. Thus, typical readers' greater success in reading acquisition may derive from their better use of statistical contingencies to identify consistent stimulus features across multiple exposures. We discuss these findings in relation to the role of implicit memory in forming new visual-phonological associations as a foundational skill in reading, and areas of weakness in developmental dyslexia.

Abstract images and words can convey the same meaning.

Intuitively, deriving meaning from an abstract image is a uniquely human, idiosyncratic experience. Here we show that, despite having no universally recognised lexical association, abstract images spontaneously elicit specific concepts conveyed by words, with a consistency akin to that of concrete images. We presented a group of naïve participants with abstract picture-word pairs construed as 'related' or 'unrelated' according to a preliminary norming procedure conducted with different participants. Surprisingly, the naïve participants with no prior exposure to the abstract images or any hints regarding their possible meaning, displayed a reaction time priming effect for 'related' versus 'unrelated' picture-word pairs. Critically, this behavioural priming effect, and an associated decrease in N400 mean amplitude indexing semantic priming, both correlated significantly with the degree of relatedness established in the preliminary norming procedure. Given that ratings and electrophysiological measures were obtained in different groups of individuals, our results show that abstract images evoke consistent meaning across observers, as has been shown in the case of music.

Word meaning acquisition is reflected in brain potentials of isolated words.

Learning a new concept and corresponding word typically involves repeated exposure to the word in the same or a similar context until the link crystallizes in long term memory. Although electrophysiological indices of the result of learning are well documented, there is currently no measure of the process of conceptually-mediated learning itself. Here, we recorded event-related brain potentials from participants who read unfamiliar words presented in isolation followed by a definition that either explained the meaning of the word or was a true, but uninformative statement. Self-reported word knowledge ratings increased for those words that were followed by meaningful definitions and were correlated with a decrease in ERP amplitude of a late frontal negativity (LFN) elicited by the isolated word. Importantly, the rate of LFN amplitude change predicted post-hoc learning outcome measures. Therefore, the LFN is real-time measure that is not under conscious control and which reflects conceptually-mediated learning. We propose that the LFN provides for the first time the opportunity to assess learning during study.

ERPs Reveal the Time-Course of Aberrant Visual-Phonological Binding in Developmental Dyslexia.

New evidence is accumulating for a deficit in binding visual-orthographic information with the corresponding phonological code in developmental dyslexia. Here, we identify the mechanisms underpinning this deficit using event-related brain potentials (ERPs) in dyslexic and control adult readers performing a letter-matching task. In each trial, a printed letter was presented synchronously with an auditory letter name. Incongruent (mismatched), frequent trials were interleaved with congruent (matched) infrequent target pairs, which participants were asked to report by pressing a button. In critical trials, incongruent letter pairs were mismatched but confusable in terms of their visual or phonological features. Typical readers showed early detection of deviant trials, indicated by larger modulation in the range of the phonological mismatch negativity (PMN) compared with standard trials. This was followed by stronger modulation of the P3b wave for visually confusable deviants and an increased lateralized readiness potential (LRP) for phonological deviants, compared with standards. In contrast, dyslexic readers showed reduced sensitivity to deviancy in the PMN range. Responses to deviants in the P3b range indicated normal letter recognition processes, but the LRP calculation revealed a specific impairment for visual-orthographic information during response selection in dyslexia. In a follow-up experiment using an analogous non-lexical task in the same participants, we found no reading-group differences, indicating a degree of specificity to over-learnt visual-phonological binding. Our findings indicate early insensitivity to visual-phonological binding in developmental dyslexia, coupled with difficulty selecting the correct orthographic code.

Bilingualism and increased attention to speech: Evidence from event-related potentials.

A number of studies have shown that from an early age, bilinguals outperform their monolingual peers on executive control tasks. We previously found that bilingual children and adults also display greater attention to unexpected language switches within speech. Here, we investigated the effect of a bilingual upbringing on speech perception in one language. We recorded monolingual and bilingual toddlers' event-related potentials (ERPs) to spoken words preceded by pictures. Words matching the picture prime elicited an early frontal positivity in bilingual participants only, whereas later ERP amplitudes associated with semantic processing did not differ between groups. These results add to the growing body of evidence that bilingualism increases overall attention during speech perception whilst semantic integration is unaffected.

On the road to somewhere: Brain potentials reflect language effects on motion event perception.

Recent studies have identified neural correlates of language effects on perception in static domains of experience such as colour and objects. The generalization of such effects to dynamic domains like motion events remains elusive. Here, we focus on grammatical differences between languages relevant for the description of motion events and their impact on visual scene perception. Two groups of native speakers of German or English were presented with animated videos featuring a dot travelling along a trajectory towards a geometrical shape (endpoint). English is a language with grammatical aspect in which attention is drawn to trajectory and endpoint of motion events equally. German, in contrast, is a non-aspect language which highlights endpoints. We tested the comparative perceptual saliency of trajectory and endpoint of motion events by presenting motion event animations (primes) followed by a picture symbolising the event (target): In 75% of trials, the animation was followed by a mismatching picture (both trajectory and endpoint were different); in 10% of trials, only the trajectory depicted in the picture matched the prime; in 10% of trials, only the endpoint matched the prime; and in 5% of trials both trajectory and endpoint were matching, which was the condition requiring a response from the participant. In Experiment 1 we recorded event-related brain potentials elicited by the picture in native speakers of German and native speakers of English. German participants exhibited a larger P3 wave in the endpoint match than the trajectory match condition, whereas English speakers showed no P3 amplitude difference between conditions. In Experiment 2 participants performed a behavioural motion matching task using the same stimuli as those used in Experiment 1. German and English participants did not differ in response times showing that motion event verbalisation cannot readily account for the difference in P3 amplitude found in the first experiment. We argue that, even in a non-verbal context, the grammatical properties of the native language and associated sentence-level patterns of event encoding influence motion event perception, such that attention is automatically drawn towards aspects highlighted by the grammar.

Anomalous transfer of syntax between languages.

Each human language possesses a set of distinctive syntactic rules. Here, we show that balanced Welsh-English bilinguals reading in English unconsciously apply a morphosyntactic rule that only exists in Welsh. The Welsh soft mutation rule determines whether the initial consonant of a noun changes based on the grammatical context (e.g., the feminine noun cath--"cat" mutates into gath in the phrase y gath--"the cat"). Using event-related brain potentials, we establish that English nouns artificially mutated according to the Welsh mutation rule (e.g., "goncert" instead of "concert") require significantly less processing effort than the same nouns implicitly violating Welsh syntax. Crucially, this effect is found whether or not the mutation affects the same initial consonant in English and Welsh, showing that Welsh syntax is applied to English regardless of phonological overlap between the two languages. Overall, these results demonstrate for the first time that abstract syntactic rules transfer anomalously from one language to the other, even when such rules exist only in one language.

Semantic priming in the motor cortex: evidence from combined repetitive transcranial magnetic stimulation and event-related potential.

Reading action verbs is associated with activity in the motor cortices involved in performing the corresponding actions. Here, we present new evidence that the motor cortex is involved in semantic processing of bodily action verbs. In contrast to previous studies, we used a direct, nonbehavioural index of semantic processing after repetitive transcranial magnetic stimulation (rTMS). Participants saw pairs of hand-related (e.g. to grab-to point) or mouth-related (e.g. to speak-to sing) verbs, whereas semantic priming was assessed using event-related potentials. Presentation of the first verb coincided with rTMS over the participant's cortical-left hand area and event-related brain potentials were analysed time-locked to the presentation onset of the second verb. Semantic integration - indexed by the N400 brain potential - was impaired for hand-related but not for mouth-related verb pairs after rTMS. This finding provides strong evidence that the motor cortex is involved in semantic encoding of action verbs, and supports the 'embodied semantics' hypothesis.

ERP-pupil size correlations reveal how bilingualism enhances cognitive flexibility.

A bilingual upbringing has been shown to enhance executive control, but the neural mechanisms underpinning such effect are essentially unknown. Here, we investigated whether monolingual and bilingual toddlers differ in semantic processing efficiency and their allocation of attention to expected and unexpected visual stimuli. We simultaneously recorded event-related brain potentials (ERPs) and pupil size in monolingual and bilingual toddlers presented with (spoken) word-picture pairs. Although ERP effects elicited by semantic relatedness were indistinguishable between the two children groups, pictures unrelated to the preceding word evoked greater pupil dilation than related pictures in bilinguals, but not in monolinguals. Furthermore, increasing pupil dilation to unrelated pictures was associated with decreasing N400 amplitude in bilinguals, whereas the monolingual toddlers showed the opposite association. Hence, attention to unexpected stimuli seems to hamper semantic integration in monolinguals, but to facilitate semantic integration in bilinguals, suggesting that bilingual toddlers are more tolerant to variation in word-referent mappings. Given the link between pupil dilation and norepinephrine-driven cognitive efficiency, correlations between ERP amplitude and concurrent pupil dilation provide new insights into the neural bases of the bilingual cognitive advantage.

Event-related potential correlates of language change detection in bilingual toddlers.

Children raised in a bilingual environment are faced with the daunting task of learning to extract meaning from language input that can differ between caregivers but, depending on the social context, also within caregivers. Here, we investigated monolingual and bilingual toddlers' brain responses to an unexpected language change. We presented 2-3 year old children with picture-word pairs and occasionally changed the language of the spoken word while recording event-related potentials (ERPs). In line with previous results obtained in adults, bilingual children differentiated between the languages of input faster than their monolingual peers, i.e., within 200 ms of spoken word onset, a time range previously associated with lexical access. However, while adult bilinguals displayed a late stimulus re-evaluation ERP response to a language change, no such modulation was found in bilingual toddlers. These results suggest that although bilingual individuals are sensitive to phonemic language cues already from an early age, language awareness and language monitoring mechanisms probably develop later in life.

Congruency effects in conceptualizing for speech.

In naming a picture at the basic level, a semantically related distractor word induces interference in comparison to an unrelated word. When the task is changed from basic-level naming to categorization, however, this effect reverses to semantic facilitation. In previous studies, this semantic facilitation effect was attributed to "message congruency" at the conceptual level. The present study examines the nature of this message congruency effect: Is it due to competition between two activated category concepts in the incongruent condition or is it due to convergence of activity on a single category concept in the congruent condition? Two experiments show that neither the strength with which the context stimulus activates an incongruent category concept nor the semantic distance between the category concepts activated by target and distractor affect target categorization speed. We conclude that the message congruency effect is most likely due to convergence on a single category concept in the category-congruent condition.

N400 amplitude reduction correlates with an increase in pupil size.

Pupil dilation is classically associated with increase in cognitive load in humans. Here, we studied the potential link between human pupil dilation and meaning integration effort as indexed by event-related brain potentials (ERPs). We recorded pupil size variation and ERPs simultaneously while participants were presented with matching or unrelated picture-word pairs. Whilst relatedness in meaning between spoken words and pictures typically modulated ERPs, pupil size was also affected quickly after picture onset. Moreover, during the time-window associated with meaning integration, greater pupil dilation correlated with less negative N400 amplitudes elicited by unrelated pictures. Since pupil dilation has been linked to activity of the locus coeruleus-norepinephrine (LC-NE) system, these findings may provide new insights into the suggested link between human high-level cognitive function and activity of the LC-NE system.

Perceptual shift in bilingualism: brain potentials reveal plasticity in pre-attentive colour perception.

The validity of the linguistic relativity principle continues to stimulate vigorous debate and research. The debate has recently shifted from the behavioural investigation arena to a more biologically grounded field, in which tangible physiological evidence for language effects on perception can be obtained. Using brain potentials in a colour oddball detection task with Greek and English speakers, a recent study suggests that language effects may exist at early stages of perceptual integration [Thierry, G., Athanasopoulos, P., Wiggett, A., Dering, B., & Kuipers, J. (2009). Unconscious effects of language-specific terminology on pre-attentive colour perception. Proceedings of the National Academy of Sciences, 106, 4567-4570]. In this paper, we test whether in Greek speakers exposure to a new cultural environment (UK) with contrasting colour terminology from their native language affects early perceptual processing as indexed by an electrophysiological correlate of visual detection of colour luminance. We also report semantic mapping of native colour terms and colour similarity judgements. Results reveal convergence of linguistic descriptions, cognitive processing, and early perception of colour in bilinguals. This result demonstrates for the first time substantial plasticity in early, pre-attentive colour perception and has important implications for the mechanisms that are involved in perceptual changes during the processes of language learning and acculturation.

Event-related brain potentials reveal the time-course of language change detection in early bilinguals.

Using event-related brain potentials, we investigated the temporal course of language change detection in proficient bilinguals as compared to matched controls. Welsh-English bilingual participants and English controls were presented with a variant of the oddball paradigm involving picture-word pairs. The language of the spoken word was manipulated such that English was the frequent stimulus (75%) and Welsh the infrequent stimulus (25%). We also manipulated semantic relatedness between pictures and words, such that only half of the pictures were followed by a word that corresponded with the identity of the picture. The P2 wave was significantly modulated by language in the bilingual group only, suggesting that this group detected a language change as early as 200 ms after word onset. Monolinguals also reliably detected the language change, but at a later stage of semantic integration (N400 range), since Welsh words were perceived as meaningless. The early detection of a language change in bilinguals triggered stimulus re-evaluation mechanisms reflected by a significant P600 modulation by Welsh words. Furthermore, compared to English unrelated words, English words matching the picture identity elicited significantly greater P2 amplitudes in the bilingual group only, suggesting that proficient bilinguals validate an incoming word against their expectation based on the context. Overall, highly proficient bilinguals appear to detect language changes very early on during speech perception and to consciously monitor language changes when they occur.

The Whorfian mind: Electrophysiological evidence that language shapes perception.

Color perception has been a traditional test-case of the idea that the language we speak affects our perception of the world.1 It is now established that categorical perception of color is verbally mediated and varies with culture and language.2 However, it is unknown whether the well-demonstrated language effects on color discrimination really reach down to the level of visual perception, or whether they only reflect post-perceptual cognitive processes. Using brain potentials in a color oddball detection task with Greek and English speakers, we demonstrate that language effects may exist at a level that is literally perceptual, suggesting that speakers of different languages have differently structured minds.

Unconscious effects of language-specific terminology on preattentive color perception.

It is now established that native language affects one's perception of the world. However, it is unknown whether this effect is merely driven by conscious, language-based evaluation of the environment or whether it reflects fundamental differences in perceptual processing between individuals speaking different languages. Using brain potentials, we demonstrate that the existence in Greek of 2 color terms--ghalazio and ble--distinguishing light and dark blue leads to greater and faster perceptual discrimination of these colors in native speakers of Greek than in native speakers of English. The visual mismatch negativity, an index of automatic and preattentive change detection, was similar for blue and green deviant stimuli during a color oddball detection task in English participants, but it was significantly larger for blue than green deviant stimuli in native speakers of Greek. These findings establish an implicit effect of language-specific terminology on human color perception.

In defense of the lexical-competition account of picture-word interference: a comment on Finkbeiner and Caramazza (2006).

On the basis of two empirical observations, Finkbeiner and Caramazza (2006) take issue with the generally accepted interpretation of semantic interference in the picture-word interference task in terms of lexical competition. As an alternative, they propose a response-selection account, in which semantic interference is attributed to the time needed to remove the inappropriate (word-reading) response from an output buffer. In this comment we argue that the empirical work discussed provides an interesting challenge for current models of language production, but that the authors' alternative account is at variance with at least three robust empirical findings in the language production literature.