Decoding semantic relatedness and prediction from EEG: A classification method comparison.

Machine-learning (ML) decoding methods have become a valuable tool for analyzing information represented in electroencephalogram (EEG) data. However, a systematic quantitative comparison of the performance of major ML classifiers for the decoding of EEG data in neuroscience studies of cognition is lacking. Using EEG data from two visual word-priming experiments examining well-established N400 effects of prediction and semantic relatedness, we compared the performance of three major ML classifiers that each use different algorithms: support vector machine (SVM), linear discriminant analysis (LDA), and random forest (RF). We separately assessed the performance of each classifier in each experiment using EEG data averaged over cross-validation blocks and using single-trial EEG data by comparing them with analyses of raw decoding accuracy, effect size, and feature importance weights. The results of these analyses demonstrated that SVM outperformed the other ML methods on all measures and in both experiments.

Language Membership Identification Precedes Semantic Access: Suppression during Bilingual Word Recognition.

Previous research suggests that bilingual comprehenders access lexical representations of words in both languages nonselectively. However, it is unclear whether global language suppression plays a role in guiding attention to target language representations during ongoing lexico-semantic processing. To help clarify this issue, this study examined the relative timing of language membership and meaning activation during visual word recognition. Spanish-English bilinguals performed simultaneous semantic and language membership classification tasks on single words during EEG recording. Go/no-go ERP latencies provided evidence that language membership information was accessed before semantic information. Furthermore, N400 frequency effects indicated that the depth of processing of words in the nontarget language was reduced compared to the target language. These results suggest that the bilingual brain can rapidly identify the language to which a word belongs and subsequently use this information to selectively modulate the degree of processing in each language accordingly.

Sensitivity to Referential Ambiguity in Discourse: The Role of Attention, Working Memory, and Verbal Ability.

The establishment of reference is essential to language comprehension. The goal of this study was to examine listeners' sensitivity to referential ambiguity as a function of individual variation in attention, working memory capacity, and verbal ability. Participants listened to stories in which two entities were introduced that were either very similar (e.g., two oaks) or less similar (e.g., one oak and one elm). The manipulation rendered an anaphor in a subsequent sentence (e.g., oak) ambiguous or unambiguous. EEG was recorded as listeners comprehended the story, after which participants completed tasks to assess working memory, verbal ability, and the ability to use context in task performance. Power in the alpha and theta frequency bands when listeners received critical information about the discourse entities (e.g., oaks) was used to index attention and the involvement of the working memory system in processing the entities. These measures were then used to predict an ERP component that is sensitive to referential ambiguity, the Nref, which was recorded when listeners received the anaphor. Nref amplitude at the anaphor was predicted by alpha power during the earlier critical sentence: Individuals with increased alpha power in ambiguous compared with unambiguous stories were less sensitive to the anaphor's ambiguity. Verbal ability was also predictive of greater sensitivity to referential ambiguity. Finally, increased theta power in the ambiguous compared with unambiguous condition was associated with higher working-memory span. These results highlight the role of attention and working memory in referential processing during listening comprehension.

Deaf readers' response to syntactic complexity: evidence from self-paced reading.

This study was designed to determine the feasibility of using self-paced reading methods to study deaf readers and to assess how deaf readers respond to two syntactic manipulations. Three groups of participants read the test sentences: deaf readers, hearing monolingual English readers, and hearing bilingual readers whose second language was English. In Experiment 1, the participants read sentences containing subject-relative or object-relative clauses. The test sentences contained semantic information that would influence online processing outcomes (Traxler, Morris, & Seely Journal of Memory and Language 47: 69-90, 2002; Traxler, Williams, Blozis, & Morris Journal of Memory and Language 53: 204-224, 2005). All of the participant groups had greater difficulty processing sentences containing object-relative clauses. This difficulty was reduced when helpful semantic cues were present. In Experiment 2, participants read active-voice and passive-voice sentences. The sentences were processed similarly by all three groups. Comprehension accuracy was higher in hearing readers than in deaf readers. Within deaf readers, native signers read the sentences faster and comprehended them to a higher degree than did nonnative signers. These results indicate that self-paced reading is a useful method for studying sentence interpretation among deaf readers.

Feeling the past: the absence of experimental evidence for anomalous retroactive influences on text processing.

In two self-paced reading experiments, we investigated the hypothesis that information moves backward in time to influence prior behaviors (Bem Journal of Personality and Social Psychology 100:407-425, 2011a). In two of Bem's experiments, words were presented after target pictures in a pleasantness judgment task. In a condition in which the words were consistent with the emotional valence of the picture, reaction times to the pictures were significantly shorter , as compared with a condition in which the words were inconsistent with the emotional valence of the picture. Bem Journal of Personality and Social Psychology 100:407-425, (2011a) interpreted these results as showing a "retroactive priming" effect resulting from precognition. To test the precognition hypothesis, we adapted a standard repetition priming paradigm from psycholinguistics. In the experiments, participants read a set of texts. In one condition, the participants read the same text twice. In other conditions, participants read two different texts. The precognition hypothesis predicts that readers who encounter the same text twice will experience reductions in processing load during their first encounter with the text. Hence, these readers' average reading times should be shorter than those of readers who encounter the target text only once. Our results indicated that readers processed the target text faster the second time they read it. Also, their reading times decreased as their experience with the self-paced reading procedure increased. However, participants read the target text equally quickly during their initial encounter with the text, whether or not the text was subsequently repeated. Thus, the experiments demonstrated normal repetition priming and practice effects but offered no evidence for retroactive influences on text processing.

Bilinguals on the garden-path: Individual differences in syntactic ambiguity resolution.

Syntactic parsing plays a central role in the interpretation of sentences, but it is unclear to what extent non-native speakers can deploy native-like grammatical knowledge during online comprehension. The current eye-tracking study investigated how Chinese-English bilinguals and native English speakers respond to syntactic category and subcategorization information while reading sentences with OBJECT-SUBJECT ambiguities. We also obtained measures of English language experience, working memory capacity, and executive function to determine how these cognitive variables influence online parsing. During reading, monolinguals and bilinguals showed similar GARDEN-PATH EFFECTS related to syntactic reanalysis, but native English speakers responded more robustly to VERB SUBCATEGORIZATION cues. Readers with greater language experience and executive function showed increased sensitivity to verb subcategorization cues, but parsing was not influenced by working memory capacity. These results are consistent with exposure-based accounts of bilingual sentence processing, and they support a link between syntactic processing and domain-general cognitive control.

Cognitive control mediates age-related changes in flexible anticipatory processing during listening comprehension.

Effective listening comprehension not only requires processing local linguistic input, but also necessitates incorporating contextual cues available in the global communicative environment. Local sentence processing can be facilitated by pre-activation of likely upcoming input, or predictive processing. Recent evidence suggests that young adults can flexibly adapt local predictive processes based on cues provided by the global communicative environment, such as the reliability of specific speakers. Whether older comprehenders can also flexibly adapt to global contextual cues is currently unknown. Moreover, it is unclear whether the underlying mechanisms supporting local predictive processing differ from those supporting adaptation to global contextual cues. Critically, it is unclear whether these mechanisms change as a function of typical aging. We examined the flexibility of prediction in young and older adults by presenting sentences from speakers whose utterances were typically more or less predictable (i.e., reliable speakers who produced expected words 80% of the time, versus unreliable speakers who produced expected words 20% of the time). For young listeners, global speaker reliability cues modulated neural effects of local predictability on the N400. In contrast, older adults, on average, did not show global modulation of local processing. Importantly, however, cognitive control (i.e., Stroop interference effects) mediated age-related reductions in sensitivity to the reliability of the speaker. Both young and older adults with high cognitive control showed greater N400 effects of predictability during sentences produced by a reliable speaker, suggesting that cognitive control is required to regulate the strength of top-down predictions based on global contextual information. Critically, cognitive control predicted sensitivity to global speaker-specific information but not local predictability cues, suggesting that predictive processing in local sentence contexts may be supported by separable neural mechanisms from adaptation of prediction as a function of global context. These results have important implications for interpreting age-related change in predictive processing, and for drawing more generalized conclusions regarding domain-general versus language-specific accounts of prediction.

Zooming in on zooming out: Partial selectivity and dynamic tuning of bilingual language control during reading.

Prominent models of bilingual visual word recognition posit a bottom-up nonselective view of lexical processing with parallel access to lexical candidates of both languages. However, these accounts do not accommodate recent findings of top-down effects on the relative global activation level of each language during bilingual reading. We conducted two eye-tracking experiments to systematically assess the degree of accessibility of each language in different global language contexts. When critical words were presented overtly in Experiment 1, code switches disrupted reading early during lexical processing, but not as much as pseudowords did. Participants zoomed out of the target language with increasing exposure to language switches. In Experiment 2, a monolingual language context was created by presenting critical words covertly as parafoveal previews. Here, code-switched words were treated like pseudowords, and participants remained zoomed in to the target language throughout the experiment. Switch direction analyses confirmed and extended these interpretations to provide further support for the role of global language control on lexical access, above and beyond effects due to proficiency differences across languages. Together, these data provide strong evidence for dynamic top-down adjustment of the degree of language selectivity during bilingual reading.

A hierarchical linear modeling analysis of working memory and implicit prosody in the resolution of adjunct attachment ambiguity.

An eye-movement monitoring experiment investigated readers' response to temporarily ambiguous sentences. The sentences were ambiguous because a relative clause could attach to one of two preceding nouns. Semantic information disambiguated the sentences. Working memory considerations predict an overall preference for the second of the two nouns, as does the late closure principle (Frazier, On comprehending sentences: Syntactic parsing strategies. Ph.D. dissertation, University of Connecticut. West Bend, IN: Indiana University Linguistics Club, 1979). Previous studies assessing preferences for such items have obtained mixed results. On-line assessments show that working memory affects the degree of preference for the first noun, with lower capacity readers having a greater preference for the second noun (Felser et al., Language Acquisition: A Journal of Developmental Linguistics, 11, 127-163, 2003; Traxler, Memory & Cognition, 35, 1107-1121, 2007). Off-line assessments indicate the opposite pattern of preferences when the test sentences are displayed on a single line (Swets et al., Journal of Experimental Psychology: General, 136, 64-81, 2007). However, when implicit prosody is manipulated by displaying the sentences with a break between the second noun and the relative clause, the off-line assessments indicate that readers prefer to attach the relative clause to the first noun. In this experiment, readers' undertook a working memory assessment and then read test sentences that were displayed across two lines, with a break appearing after the second noun and before the relative clause. The eye-tracking data indicated an overall preference to attach the relative clause to the first noun, and there was little indication that working memory moderated the degree of preference for this configuration. Hence, it appears that readers' implicit prosodic contours rapidly affect resolution of adjunct attachment ambiguities.

Lexically-mediated syntactic priming effects in comprehension: Sources of facilitation.

The nature of the facilitation occurring when sentences share a verb and syntactic structure (i.e., lexically-mediated syntactic priming) has not been adequately addressed in comprehension. In four eye-tracking experiments, we investigated the degree to which lexical, syntactic, thematic, and verb form repetition contribute to facilitated target sentence processing. Lexically-mediated syntactic priming was observed when primes and targets shared a verb and abstract syntactic structure, regardless of the ambiguity of the prime. In addition, repeated thematic role assignment resulted in syntactic priming (to a lesser degree), and verb form repetition facilitated lexical rather than structural processing. We conclude that priming in comprehension involves lexically associated abstract syntactic representations, and facilitation of verb and thematic role processes. The results also indicate that syntactic computation errors during prime processing are not necessary for lexically-mediated priming to occur during target processing. This result is inconsistent with an error-driven learning account of lexically-mediated syntactic priming effects.

Flexible predictions during listening comprehension: Speaker reliability affects anticipatory processes.

During listening comprehension, the identification of individual words can be strongly influenced by properties of the preceding context. While sentence context can facilitate both behavioral and neural responses, it is unclear whether these effects can be attributed to the pre-activation of lexico-semantic features or the facilitated integration of contextually congruent words. Moreover, little is known about how statistics of the broader language environment, or information about the current speaker, might shape these facilitation effects. In the present study, we measured neural responses to predictable and unpredictable words as participants listened to sentences for comprehension. Critically, we manipulated the reliability of each speaker's utterances, such that individual speakers either tended to complete sentences with words that were highly predictable (reliable speaker) or with words that were unpredictable but still plausible (unreliable speaker). As expected, the amplitude of the N400 was reduced for locally predictable words, but, critically, these context effects were also modulated by speaker identity. Sentences from a reliable speaker showed larger facilitation effects with an earlier onset, suggesting that listeners engaged in enhanced anticipatory processing when a speaker's behavior was more predictable. This finding suggests that listeners can implicitly track the reliability of predictive cues in their environment and use these statistics to adaptively regulate predictive processing.

Lexically independent priming in online sentence comprehension.

Two eye-tracking experiments investigated what happens when people read pairs of sentences that have the same syntactic structure. Previous experiments have shown priming in online sentence processing only when critical lexical material overlaps between the prime and the target sentence. In the current study, participants were asked to read sentences containing modifier-goal ambiguities. Half of the target sentences were preceded by sentences with the same structure, and half were preceded by sentences with a different structure. In Experiment 1, the prime-target pairs had the same main verb. In Experiment 2, the prime-target pairs had different main verbs. Facilitated target sentence processing was observed in both Experiments 1 and 2 when the target sentences were preceded by a prime sentence with the same syntactic structure. These results provide the first evidence of lexically independent, between-sentence structural priming in online sentence comprehension.

Implicit learning of structure occurs in parallel with lexically-mediated syntactic priming effects in sentence comprehension.

The aim of this study was to determine whether cumulative structural priming effects and trial-to-trial lexically-mediated priming effects are produced by the same mechanism in comprehension. Participants took part in a five-session eye tracking study where they read reduced-relative prime-target pairs with the same initial verb. Half of the verbs in these sentences were repeated across the five sessions and half were novel to each session. Total fixation times on the syntactically challenging parts of prime sentences decreased across sessions, suggesting participants implicitly learned the structure. Additional priming was also observed at the critical regions of the target sentences, and the magnitude of this effect did not change over the five sessions. These finding suggests long-lived adaptation to structure and short-lived lexically-mediated priming effects are caused by separate mechanisms in comprehension. A dual mechanism account of syntactic priming effects can best reconcile these results.

Electrophysiological evidence for preserved primacy of lexical prediction in aging.

Young adults show consistent neural benefits of predictable contexts when processing upcoming words, but these benefits are less clear-cut in older adults. Here we disentangle the neural correlates of prediction accuracy and contextual support during word processing, in order to test current theories that suggest that neural mechanisms underlying predictive processing are specifically impaired in older adults. During a sentence comprehension task, older and younger readers were asked to predict passage-final words and report the accuracy of these predictions. Age-related reductions were observed for N250 and N400 effects of prediction accuracy, as well as for N400 effects of contextual support independent of prediction accuracy. Furthermore, temporal primacy of predictive processing (i.e., earlier facilitation for successful predictions) was preserved across the lifespan, suggesting that predictive mechanisms are unlikely to be uniquely impaired in older adults. In addition, older adults showed prediction effects on frontal post-N400 positivities (PNPs) that were similar in amplitude to PNPs in young adults. Previous research has shown correlations between verbal fluency and lexical prediction in older adult readers, suggesting that the production system may be linked to capacity for lexical prediction, especially in aging. The current study suggests that verbal fluency modulates PNP effects of contextual support, but not prediction accuracy. Taken together, our findings suggest that aging does not result in specific declines in lexical prediction.

Lexical mediation and context effects in sentence processing.

Studies of syntactic ambiguity resolution have played a central role in resolving questions about when and how contextual information affects parsing processes. These investigations are often couched in terms of modularity versus interaction, with demonstrations of rapid contextual effects being taken as evidence that the mechanisms responsible for structuring sentences are permeable to referential or semantic context, and therefore non-modular. In this paper, we will propose that argument relations are constructed on the basis of lexically stored syntactic representations (as in MacDonald, M.C., Pearlmutter, N.J., and Seidenberg, M.S. (1994). Lexical nature of syntactic ambiguity resolution. Psychological Review, 101, 676-703. Pickering, M.J., and Traxler, M.J. (2004). Grammatical repetition and garden path effects. Paper presented to the CUNY Sentence Processing Conference. College Park, MD., Pickering, M.J., and Traxler, M.J. (2006). Syntactic Priming in Comprehension. Manuscript in preparation. Traxler, M.J., and Pickering, M.J. (2005, March). Syntactic priming in comprehension. Paper presented to the CUNY Sentence Processing Conference. Tucson, AZ), but that other types of structural decisions are made on the basis of general processing principles. This formulation can be tested by looking at how the parser reacts to immediate intra- and inter-sentential factors (short-term context) and how it reacts to patterns of input over longer time scales (long-term context). We begin with a brief review of work on context effects in syntactic disambiguation, sketch our account of parsing, and then provide evidence from two eye-tracking experiments that illustrate some of the processing principles that govern parsing of argument relations.

Early processing of orthographic language membership information in bilingual visual word recognition: Evidence from ERPs.

For successful language comprehension, bilinguals often must exert top-down control to access and select lexical representations within a single language. These control processes may critically depend on identification of the language to which a word belongs, but it is currently unclear when different sources of such language membership information become available during word recognition. In the present study, we used event-related potentials to investigate the time course of influence of orthographic language membership cues. Using an oddball detection paradigm, we observed early neural effects of orthographic bias (Spanish vs. English orthography) that preceded effects of lexicality (word vs. pseudoword). This early orthographic pop-out effect was observed for both words and pseudowords, suggesting that this cue is available prior to full lexical access. We discuss the role of orthographic bias for models of bilingual word recognition and its potential role in the suppression of nontarget lexical information.

Language context processing deficits in schizophrenia: The role of attentional engagement.

Individuals with schizophrenia exhibit problems in language comprehension that are most evident during discourse processing. We hypothesized that deficits in cognitive control contribute to these comprehension deficits during discourse processing, and investigated the underlying cognitive-neural mechanisms using EEG (alpha power) and ERPs (N400). N400 amplitudes to globally supported or unsupported target words near the end of stories were used to index sensitivity to previous context. ERPs showed reduced sensitivity to context in patients versus controls. EEG alpha-band activity was used to index attentional engagement while participants listened to the stories. We found that context effects varied with attentional engagement in both groups, as well as with negative symptom severity in patients. Both groups demonstrated trial-to-trial fluctuations in alpha. Relatively high alpha power was associated with compromised discourse processing in participants with schizophrenia when it occurred during any early portion of the story. In contrast, discourse processing was only compromised in controls when alpha was relatively high for longer segments of the stories. Our results indicate that shifts in attention from the story context may be more detrimental to discourse processing for participants with schizophrenia than for controls, most likely due to an impaired ability to benefit from global context.

A time course analysis of enriched composition.

Linguistic analyses suggest that common and seemingly simple expressions, such as began the book, cannot be interpreted with simple compositional processes; rather, they require enriched composition to provide an interpretation, such as began reading the book (Jackendoff, 1997; Pustejovsky, 1995). Recent reading time studies have supported these accounts by providing evidence that these expressions are more costly to process than are minimally contrasting controls (e.g., McElree, Traxler, Pickering, Seely, and Jackendoff, 2001). We report a response signal speed-accuracy trade-off (SAT) study in which two types of expressions that are thought to require enriched composition were examined. Analyses of the full time course SAT data indicate that these expressions were interpreted less accurately and, most importantly, more slowly than control sentences. The latter finding suggests that enriched composition requires the online deployment of complex compositional operations.

Anticipating syntax during reading: Evidence from the boundary change paradigm.

Previous evidence suggests that grammatical constraints have a rapid influence during language comprehension, particularly at the level of word categories (noun, verb, preposition). These findings are in conflict with a recent study from Angele, Laishley, Rayner, and Liversedge (2014), in which sentential fit had no early influence on word skipping rates during reading. In the present study, we used a gaze-contingent boundary change paradigm to manipulate the syntactic congruity of an upcoming noun or verb outside of participants' awareness. Across 3 experiments (total N = 148), we observed higher skipping rates for syntactically valid previews (The admiral would not confess . . .), when compared with violation previews (The admiral would not surgeon . . .). Readers were less likely to skip an ungrammatical continuation, even when that word was repeated within the same sentence (The admiral would not admiral . . .), suggesting that word-class constraints can take precedence over lexical repetition effects. To our knowledge, these results provide the first evidence for an influence of syntactic context during parafoveal word recognition. On the basis of the early time-course of this effect, we argue that readers can use grammatical constraints to generate syntactic expectations for upcoming words. (PsycINFO Database Record

The difficulty of coercion: a response to de Almeida.

The sentence The secretary began the memo requires specifying what event the secretary began, because the memo does not refer to an event. McElree, Traxler, Pickering, Seely, and Jackendoff (2001) and Traxler, Pickering, and McElree (2002) found evidence from both self-paced reading and eye-tracking that such sentences caused processing difficulty, and thus argued that people "coerced" the object to refer to an event (e.g., writing the memo). de Almeida (2004) reports two self-paced reading experiments that failed to replicate some aspects of previous studies, and thereby argued against coercion during comprehension. A new experiment demonstrates coercion costs using new items, and provides evidence of coercion cost with de Almeida's stimuli. We conclude that coercion does cause processing difficulty.