Pre-output Language Monitoring in Sign Production.

A domain-general monitoring mechanism is proposed to be involved in overt speech monitoring. This mechanism is reflected in a medial frontal component, the error negativity (Ne), present in both errors and correct trials (Ne-like wave) but larger in errors than correct trials. In overt speech production, this negativity starts to rise before speech onset and is therefore associated with inner speech monitoring. Here, we investigate whether the same monitoring mechanism is involved in sign language production. Twenty deaf signers (American Sign Language [ASL] dominant) and 16 hearing signers (English dominant) participated in a picture-word interference paradigm in ASL. As in previous studies, ASL naming latencies were measured using the keyboard release time. EEG results revealed a medial frontal negativity peaking within 15 msec after keyboard release in the deaf signers. This negativity was larger in errors than correct trials, as previously observed in spoken language production. No clear negativity was present in the hearing signers. In addition, the slope of the Ne was correlated with ASL proficiency (measured by the ASL Sentence Repetition Task) across signers. Our results indicate that a similar medial frontal mechanism is engaged in preoutput language monitoring in sign and spoken language production. These results suggest that the monitoring mechanism reflected by the Ne/Ne-like wave is independent of output modality (i.e., spoken or signed) and likely monitors prearticulatory representations of language. Differences between groups may be linked to several factors including differences in language proficiency or more variable lexical access to motor programming latencies for hearing than deaf signers.

Cross-language Neighborhood Effects in Learners Indicative of an Integrated Lexicon.

This study examined how acquisition of novel words from an unknown language (L2) is influenced by their orthographic similarity with existing native language (L1) words in beginning adult learners. Participants were tested in a two-alternative forced-choice recognition task and a typing production task as they learned to associate 80 L2 (pseudo)words with pictures depicting their meanings. There was no effect of L1 orthographic neighborhood density on accuracy in the two-alternative forced-choice task, but typing accuracy was higher for L2 words with many L1 neighbors in the earliest stages of learning. ERPs recorded during a language decision task before and after learning also showed differences as a function of L1 neighborhood density. Across sessions, L2 words with many L1 neighbors elicited slower responses and larger N400s than words with fewer L1 neighbors, suggesting that L1 neighbors continued to influence processing of the L2 words after learning (though to a lesser extent). Finally, ERPs recorded during a typing task after learning also revealed an effect of L1 neighborhood that began about 700 msec after picture onset, suggesting that the cross-language neighborhood effects cannot solely be attributed to bottom-up activation of L1 neighbors. Together, these results demonstrate that strategic associations between novel L2 words and existing L1 neighbors scaffold learning and result in interactions among cross-language neighbors, suggestive of an integrated L1-L2 lexicon.

Interactions in the neighborhood: Effects of orthographic and phonological neighbors on N400 amplitude.

The present study investigated effects of phonological and orthographic neighborhood density on event-related potentials, with an aim to better specify the factors that determine N400 amplitude in single word reading paradigms. We orthogonally manipulated the number of orthographic and phonological neighbors of words using the Levenshtein Distance metric (OLD20 and PLD20, respectively). The results showed opposite effects of phonological neighborhood density (PND) as a function of orthographic neighborhood density (OND). Larger N400 amplitudes were elicited by words with high PND compared with low PND when OND was high, and smaller N400 amplitudes were observed with high PND compared with low PND words when OND was low. We interpret these findings using the notion of cross-code consistency, according to which the compatibility of orthographic and phonological representations activated by a given word influences the process of recognizing that word. Words with similar numbers of orthographic and phonological neighbors have more consistent spellings and pronunciations across the neighborhood, and generate larger N400 amplitudes.

Neural changes underlying early stages of L2 vocabulary acquisition.

Research has shown neural changes following second language (L2) acquisition after weeks or months of instruction. But are such changes detectable even earlier than previously shown? The present study examines the electrophysiological changes underlying the earliest stages of second language vocabulary acquisition by recording event-related potentials (ERPs) within the first week of learning. Adult native English speakers with no previous Spanish experience completed less than four hours of Spanish vocabulary training, with pre- and post-training ERPs recorded to a backward translation task. Results indicate that beginning L2 learners show rapid neural changes following learning, manifested in changes to the N400 - an ERP component sensitive to lexicosemantic processing and degree of L2 proficiency. Specifically, learners in early stages of L2 acquisition show growth in N400 amplitude to L2 words following learning as well as a backward translation N400 priming effect that was absent pre-training. These results were shown within days of minimal L2 training, suggesting that the neural changes captured during adult second language acquisition are more rapid than previously shown. Such findings are consistent with models of early stages of bilingualism in adult learners of L2 (e.g. Kroll and Stewart's RHM) and reinforce the use of ERP measures to assess L2 learning.

A Thousand Words Are Worth a Picture: Snapshots of Printed-Word Processing in an Event-Related Potential Megastudy.

In the experiment reported here, approximately 1,000 words were presented to 75 participants in a go/no-go lexical decision task while event-related potentials (ERPs) were recorded. Partial correlations were computed for variables selected to reflect orthographic, lexical, and semantic processing, as well as for a novel measure of the visual complexity of written words. Correlations were based on the item-level ERPs at each electrode site and time slice while a false-discovery-rate correction was applied. Early effects of visual complexity were seen around 50 ms after word onset, followed by the earliest sustained orthographic effects around 100 to 150 ms, with the bulk of orthographic and lexical influences arising after 200 ms. Effects of a semantic variable (concreteness) emerged later, at around 300 ms. The overall time course of these ERP effects is in line with hierarchical, cascaded, interactive accounts of word recognition, in which fast feed-forward influences are consolidated by top-down feedback via recurrent processing loops.

An ERP investigation of perceptual vs motoric iconicity in sign production.

The type of form-meaning mapping for iconic signs can vary. For perceptually-iconic signs there is a correspondence between visual features of a referent (e.g., the beak of a bird) and the form of the sign (e.g., extended thumb and index finger at the mouth for the American Sign Language (ASL) sign BIRD). For motorically-iconic signs there is a correspondence between how an object is held/manipulated and the form of the sign (e.g., the ASL sign FLUTE depicts how a flute is played). Previous studies have found that iconic signs are retrieved faster in picture-naming tasks, but type of iconicity has not been manipulated. We conducted an ERP study in which deaf signers and a control group of English speakers named pictures that targeted perceptually-iconic, motorically-iconic, or non-iconic ASL signs. For signers (unlike the control group), naming latencies varied by iconicity type: perceptually-iconic < motorically-iconic < non-iconic signs. A reduction in the N400 amplitude was only found for the perceptually-iconic signs, compared to both non-iconic and motorically-iconic signs. No modulations of N400 amplitudes were observed for the control group. We suggest that this pattern of results arises because pictures eliciting perceptually-iconic signs can more effectively prime lexical access due to greater alignment between features of the picture and the semantic and phonological features of the sign. We speculate that naming latencies are facilitated for motorically-iconic signs due to later processes (e.g., faster phonological encoding via cascading activation from semantic features). Overall, the results indicate that type of iconicity plays role in sign production when elicited by picture-naming tasks.

Orthographic and Phonological Code Activation in Deaf and Hearing Readers.

Grainger et al. (2006) were the first to use ERP masked priming to explore the differing contributions of phonological and orthographic representations to visual word processing. Here we adapted their paradigm to examine word processing in deaf readers. We investigated whether reading-matched deaf and hearing readers (n = 36) exhibit different ERP effects associated with the activation of orthographic and phonological codes during word processing. In a visual masked priming paradigm, participants performed a go/no-go categorization task (detect an occasional animal word). Critical target words were preceded by orthographically-related (transposed letter - TL) or phonologically-related (pseudohomophone - PH) masked non-word primes were contrasted with the same target words preceded by letter substitution (control) non-words primes. Hearing readers exhibited typical N250 and N400 priming effects (greater negativity for control compared to TL or PH primed targets), and the TL and PH priming effects did not differ. For deaf readers, the N250 PH priming effect was later (250-350 ms), and they showed a reversed N250 priming effect for TL primes in this time window. The N400 TL and PH priming effects did not differ between groups. For hearing readers, those with better phonological and spelling skills showed larger early N250 PH and TL priming effects (150-250 ms). For deaf readers, those with better phonological skills showed a larger reversed TL priming effect in the late N250 window. We speculate that phonological knowledge modulates how strongly deaf readers rely on whole-word orthographic representations and/or the mapping from sublexical to lexical representations.

Different language control mechanisms in comprehension and production: Evidence from paragraph reading.

Chinese-English bilinguals read paragraphs with language switches using a rapid serial visual presentation paradigm silently while ERPs were measured (Experiment 1) or read them aloud (Experiment 2). Each paragraph was written in either Chinese or English with several function or content words switched to the other language. In Experiment 1, language switches elicited an early, long-lasting positivity when switching from the dominant language to the nondominant language, but when switching to the dominant language, the positivity started later, and was never larger than when switching to the nondominant language. In addition, switch effects on function words were not significantly larger than those on content words in any analyses. In Experiment 2, participants produced more cross-language intrusion errors when switching to the dominant than to the nondominant language, and more errors on function than content words. These results implicate different control mechanisms in bilingual language selection across comprehension and production.

ERPs Reveal How Semantic and Syntactic Processing Unfold across Parafoveal and Foveal Vision during Sentence Comprehension.

We examined how readers process content and function words in sentence comprehension with ERPs. Participants read simple declarative sentences using a rapid serial visual presentation (RSVP) with flankers paradigm. Sentences contained either an unexpected semantically anomalous content word, an unexpected syntactically anomalous function word or were well formed with no anomalies. ERPs were examined when target words were in the parafoveal or foveal vision. Unexpected content words elicited a typically distributed N400 when displayed in the parafovea, followed by a longer-lasting, widely distributed positivity starting around 300 ms once foveated. Unexpected function words elicited a left lateralized LAN-like component when presented in the parafovea, followed by a left lateralized, posteriorly distributed P600 when foveated. These results suggested that both semantic and syntactic processing involve two stages-the initial, fast process that can be completed in parafovea, followed by a more in depth attentionally mediated assessment that occurs with direct attention.

Asymetric Event-Related Potential Priming Effects Between English Letters and American Sign Language Fingerspelling Fonts.

Letter recognition plays an important role in reading and follows different phases of processing, from early visual feature detection to the access of abstract letter representations. Deaf ASL-English bilinguals experience orthography in two forms: English letters and fingerspelling. However, the neurobiological nature of fingerspelling representations, and the relationship between the two orthographies, remains unexplored. We examined the temporal dynamics of single English letter and ASL fingerspelling font processing in an unmasked priming paradigm with centrally presented targets for 200 ms preceded by 100 ms primes. Event-related brain potentials were recorded while participants performed a probe detection task. Experiment 1 examined English letter-to-letter priming in deaf signers and hearing non-signers. We found that English letter recognition is similar for deaf and hearing readers, extending previous findings with hearing readers to unmasked presentations. Experiment 2 examined priming effects between English letters and ASL fingerspelling fonts in deaf signers only. We found that fingerspelling fonts primed both fingerspelling fonts and English letters, but English letters did not prime fingerspelling fonts, indicating a priming asymmetry between letters and fingerspelling fonts. We also found an N400-like priming effect when the primes were fingerspelling fonts which might reflect strategic access to the lexical names of letters. The studies suggest that deaf ASL-English bilinguals process English letters and ASL fingerspelling differently and that the two systems may have distinct neural representations. However, the fact that fingerspelling fonts can prime English letters suggests that the two orthographies may share abstract representations to some extent.

How (and why) does iconicity effect lexical access: An electrophysiological study of American sign language.

Prior research has found that iconicity facilitates sign production in picture-naming paradigms and has effects on ERP components. These findings may be explained by two separate hypotheses: (1) a task-specific hypothesis that suggests these effects occur because visual features of the iconic sign form can map onto the visual features of the pictures, and (2) a semantic feature hypothesis that suggests that the retrieval of iconic signs results in greater semantic activation due to the robust representation of sensory-motor semantic features compared to non-iconic signs. To test these two hypotheses, iconic and non-iconic American Sign Language (ASL) signs were elicited from deaf native/early signers using a picture-naming task and an English-to-ASL translation task, while electrophysiological recordings were made. Behavioral facilitation (faster response times) and reduced negativities were observed for iconic signs (both prior to and within the N400 time window), but only in the picture-naming task. No ERP or behavioral differences were found between iconic and non-iconic signs in the translation task. This pattern of results supports the task-specific hypothesis and provides evidence that iconicity only facilitates sign production when the eliciting stimulus and the form of the sign can visually overlap (a picture-sign alignment effect).

Tracking the time course of sign recognition using ERP repetition priming.

Repetition priming and event-related potentials (ERPs) were used to investigate the time course of sign recognition in deaf users of American Sign Language. Signers performed a go/no-go semantic categorization task to rare probe signs referring to people; critical target items were repeated and unrelated signs. In Experiment 1, ERPs were time-locked either to the onset of the video or to sign onset within the video; in Experiment 2, the same full videos were clipped so that video and sign onset were aligned (removing transitional movements), and ERPs were time-locked to video/sign onset. All analyses revealed an N400 repetition priming effect (less negativity for repeated than unrelated signs) but differed in the timing and/or duration of the N400 effect. Results from Experiment 1 revealed that repetition priming effects began before sign onset within a video, suggesting that signers are sensitive to linguistic information within the transitional movement to sign onset. The timing and duration of the N400 for clipped videos were more parallel to that observed previously for auditorily presented words and was 200 ms shorter than either time-locking analysis from Experiment 1. We conclude that time-locking to full video onset is optimal when early ERP components or sensitivity to transitional movements are of interest and that time-locking to the onset of clipped videos is optimal for priming studies with fluent signers.

Sensitivity to orthographic vs. phonological constraints on word recognition: An ERP study with deaf and hearing readers.

The role of phonology in word recognition has previously been investigated using a masked lexical decision task and transposed letter (TL) nonwords that were either pronounceable (barve) or unpronounceable (brvae). We used event-related potentials (ERPs) to investigate these effects in skilled deaf readers, who may be more sensitive to orthotactic than phonotactic constraints, which are conflated in English. Twenty deaf and twenty hearing adults completed a masked lexical decision task while ERPs were recorded. The groups were matched in reading skill and IQ, but deaf readers had poorer phonological ability. Deaf readers were faster and more accurate at rejecting TL nonwords than hearing readers. Neither group exhibited an effect of nonword pronounceability in RTs or accuracy. For both groups, the N250 and N400 components were modulated by lexicality (more negative for nonwords). The N250 was not modulated by nonword pronounceability, but pronounceable nonwords elicited a larger amplitude N400 than unpronounceable nonwords. Because pronounceable nonwords are more word-like, they may incite activation that is unresolved when no lexical entry is found, leading to a larger N400 amplitude. Similar N400 pronounceability effects for deaf and hearing readers, despite differences in phonological sensitivity, suggest these TL effects arise from sensitivity to lexical-level orthotactic constraints. Deaf readers may have an advantage in processing TL nonwords because of enhanced early visual attention and/or tight orthographic-to-semantic connections, bypassing the phonologically mediated route to word recognition.

Are form priming effects phonological or perceptual? Electrophysiological evidence from American Sign Language.

Form priming has been used to identify and demarcate the processes that underlie word and sign recognition. The facilitation that results from the prime and target being related in form is typically interpreted in terms of pre-activation of linguistic representations, with little to no consideration for the potential contributions of increased perceptual overlap between related pairs. Indeed, isolating the contribution of perceptual similarity is impossible in spoken languages; there are no listeners who can perceive speech but have not acquired a sound-based phonological system. Here, we compared the electrophysiological indices of form priming effects in American Sign Language between hearing non-signers (i.e., who had no visual-manual phonological system) and deaf signers. We reasoned that similarities in priming effects between groups would most likely be perceptual in nature, whereas priming effects that are specific to the signer group would reflect pre-activation of phonological representations. Behavior in the go/no-go repetition detection task was remarkably similar between groups. Priming in a pre-N400 window was also largely similar across groups, consistent with an early effect of perceptual similarity. However, priming effects diverged between groups during the subsequent N400 and post-N400 windows. Signers had more typical form priming effects and were especially attuned to handshape overlap, whereas non-signers did not exhibit an N400 component and were more sensitive to location overlap. We attribute this pattern to an interplay between perceptual similarity and phonological knowledge. Perceptual similarity contributes to early phonological priming effects, while phonological knowledge tunes sensitivity to linguistically relevant dimensions of perceptual similarity.

Taxonomic and thematic semantic relationships in picture naming as revealed by Laplacian-transformed event-related potentials.

Semantically related concepts co-activate when we speak. Prior research reported both behavioral interference and facilitation due to co-activation during picture naming. Different word relationships may account for some of this discrepancy. Taxonomically related words (e.g., WOLF-DOG) have been associated with semantic interference; thematically related words (e.g., BONE-DOG) have been associated with facilitation. Although these different semantic relationships have been associated with opposite behavioral outcomes, electrophysiological studies have found inconsistent effects on event-related potentials. We conducted a picture-word interference electroencephalography experiment to examine word retrieval dynamics in these different semantic relationships. Importantly, we used traditional monopolar analysis as well as Laplacian transformation allowing us to examine spatially deblurred event-related components. Both analyses revealed greater negativity (150-250 ms) for unrelated than related taxonomic pairs, though more restricted in space for thematic pairs. Critically, Laplacian analyses revealed a larger negative-going component in the 300 to 500 ms time window in taxonomically related versus unrelated pairs which were restricted to a left frontal recording site. In parallel, an opposite effect was found in the same time window but localized to a left parietal site. Finding these opposite effects in the same time window was feasible thanks to the use of the Laplacian transformation and suggests that frontal control processes are concurrently engaged with cascading effects of the spread of activation through semantically related representations.

Effects of cognate status on word comprehension in second language learners: an ERP investigation.

ERPs were used to explore the different patterns of processing of cognate and noncognate words in the first (L1) and second (L2) language of a population of second language learners. L1 English students of French were presented with blocked lists of L1 and L2 words, and ERPs to cognates and noncognates were compared within each language block. For both languages, cognates had smaller amplitudes in the N400 component when compared with noncognates. L1 items that were cognates showed early differences in amplitude in the N400 epoch when compared with noncognates. L2 items showed later differences between cognates and noncognates than L1 items. The results are discussed in terms of how cognate status affects word recognition in second language learners.

Processing words in two languages: An event-related brain potential study of proficient bilinguals.

In a previous study of native-English speaking university learners of a second language (Spanish) we observed an asymmetric pattern of ERP translation priming effects in L1 and L2 (Alvarez et al., 2003, Brain & Language, 87, 290-304) with larger and earlier priming on the N400 component in the L2 to L1, compared with the L1 to L2 direction. In the current study 20 native-Russian speakers who were also highly proficient in English participated in a mixed language lexical decision task in which critical words were presented in Russian (L1) and English (L2) and repetitions of these words (within and between languages) were presented on subsequent trials. ERPs were recorded to all items allowing for comparisons of repetition effects within and between (translation) languages. The results revealed a symmetrical pattern of within-language repetition and between-language translation ERP priming effects, which in conjunction with Alvarez et al (2003), supports the hypothesis that L2 proficiency level rather than age or order of language acquisition is responsible for the observed patterns of translation priming. The ramifications of these results for models of bilingual word processing are discussed.

Masked ERP repetition priming in deaf and hearing readers.

Deaf readers provide unique insights into how the reading circuit is modified by altered linguistic and sensory input. We investigated whether reading-matched deaf and hearing readers (n = 62) exhibit different ERP effects associated with orthographic to phonological mapping (N250) or lexico-semantic processes (N400). In a visual masked priming paradigm, participants performed a go/no-go categorization task; target words were preceded by repeated or unrelated primes. Prime duration and word frequency were manipulated. Hearing readers exhibited typical N250 and N400 priming effects with 50 ms primes (greater negativity for unrelated primes) and smaller effects with 100 ms primes. Deaf readers showed a surprising reversed priming effect with 50 ms primes (greater negativity for related primes), and more typical N250 and N400 effects with 100 ms primes. Correlation results suggested deaf readers with poorer phonological skills drove this effect. We suggest that weak phonological activation may create orthographic "repetition enhancement" or form/lexical competition in deaf readers.

On the Connection Between Language Control and Executive Control-An ERP Study.

Models vary in the extent to which language control processes are domain general. Those that posit that language control is at least partially domain general insist on an overlap between language control and executive control at the goal level. To further probe whether or not language control is domain general, we conducted the first event-related potential (ERP) study that directly compares language-switch costs, as an index of language control, and task-switch costs, as an index of executive control. The language switching and task switching methodologies were identical, except that the former required switching between languages (English or Spanish) whereas the latter required switching between tasks (color naming or category naming). This design allowed us to directly compare control processes at the goal level (cue-locked ERPs) and at the task performance level (picture-locked ERPs). We found no significant differences in the switch-related cue-locked and picture-locked ERP patterns across the language and task switching paradigms. These results support models of domain-general language control.

Lexical selection in bimodal bilinguals: ERP evidence from picture-word interference.

The picture word interference (PWI) paradigm and ERPs were used to investigate whether lexical selection in deaf and hearing ASL-English bilinguals occurs via lexical competition or whether the response exclusion hypothesis (REH) for PWI effects is supported. The REH predicts that semantic interference should not occur for bimodal bilinguals because sign and word responses do not compete within an output buffer. Bimodal bilinguals named pictures in ASL, preceded by either a translation equivalent, semantically-related, or unrelated English written word. In both the translation and semantically-related conditions bimodal bilinguals showed facilitation effects: reduced RTs and N400 amplitudes for related compared to unrelated prime conditions. We also observed an unexpected focal left anterior positivity that was stronger in the translation condition, which we speculate may be due to articulatory priming. Overall, the results support the REH and models of bilingual language production that assume lexical selection occurs without competition between languages.