A predictive coding model of the N400.

The N400 event-related component has been widely used to investigate the neural mechanisms underlying real-time language comprehension. However, despite decades of research, there is still no unifying theory that can explain both its temporal dynamics and functional properties. In this work, we show that predictive coding - a biologically plausible algorithm for approximating Bayesian inference - offers a promising framework for characterizing the N400. Using an implemented predictive coding computational model, we demonstrate how the N400 can be formalized as the lexico-semantic prediction error produced as the brain infers meaning from the linguistic form of incoming words. We show that the magnitude of lexico-semantic prediction error mirrors the functional sensitivity of the N400 to various lexical variables, priming, contextual effects, as well as their higher-order interactions. We further show that the dynamics of the predictive coding algorithm provides a natural explanation for the temporal dynamics of the N400, and a biologically plausible link to neural activity. Together, these findings directly situate the N400 within the broader context of predictive coding research. More generally, they raise the possibility that the brain may use the same computational mechanism for inference across linguistic and non-linguistic domains.

Dissociating the pre-activation of word meaning and form during sentence comprehension: Evidence from EEG representational similarity analysis.

During language comprehension, the processing of each incoming word is facilitated in proportion to its predictability. Here, we asked whether anticipated upcoming linguistic information is actually pre-activated before new bottom-up input becomes available, and if so, whether this pre-activation is limited to the level of semantic features, or whether extends to representations of individual word-forms (orthography/phonology). We carried out Representational Similarity Analysis on EEG data while participants read highly constraining sentences. Prior to the onset of the expected target words, sentence pairs predicting semantically related words (financial "bank" - "loan") and form-related words (financial "bank" - river "bank") produced more similar neural patterns than pairs predicting unrelated words ("bank" - "lesson"). This provides direct neural evidence for item-specific semantic and form predictive pre-activation. Moreover, the semantic pre-activation effect preceded the form pre-activation effect, suggesting that top-down pre-activation is propagated from higher to lower levels of the linguistic hierarchy over time.

Multiple predictions during language comprehension: Friends, foes, or indifferent companions?

To comprehend language, we continually use prior context to pre-activate expected upcoming information, resulting in facilitated processing of incoming words that confirm these predictions. But what are the consequences of disconfirming prior predictions? To address this question, most previous studies have examined unpredictable words appearing in contexts that constrain strongly for a single continuation. However, during natural language processing, it is far more common to encounter contexts that constrain for multiple potential continuations, each with some probability. Here, we ask whether and how pre-activating both higher and lower probability alternatives influences the processing of the lower probability incoming word. One possibility is that, similar to language production, there is continuous pressure to select the higher-probability pre-activated alternative through competitive inhibition. During comprehension, this would result in relative costs in processing the lower probability target. A second possibility is that if the two pre-activated alternatives share semantic features, they mutually enhance each other's pre-activation. This would result in greater facilitation in processing the lower probability target. To distinguish between these accounts, we recorded ERPs as participants read three-sentence scenarios that constrained either for a single word or for two potential continuations - a higher probability expected candidate and a lower probability second-best candidate. We found no evidence that competitive pre-activation between the expected and second-best candidates resulted in costs in processing the second-best target, either during lexico-semantic processing (indexed by the N400) or at later stages of processing (indexed by a later frontal positivity). Instead, we found only benefits of pre-activating multiple alternatives, with evidence of enhanced graded facilitation on lower-probability targets that were semantically related to a higher-probability pre-activated alternative. These findings are consistent with a previous eye-tracking study by Luke and Christianson (2016, Cogn Psychol) using corpus-based materials. They have significant theoretical implications for models of predictive language processing, indicating that routine graded prediction in language comprehension does not operate through the same competitive mechanisms that are engaged in language production. Instead, our results align more closely with hierarchical probabilistic accounts of language comprehension, such as predictive coding.

Predictive coding across the left fronto-temporal hierarchy during language comprehension.

We used magnetoencephalography (MEG) and event-related potentials (ERPs) to track the time-course and localization of evoked activity produced by expected, unexpected plausible, and implausible words during incremental language comprehension. We suggest that the full pattern of results can be explained within a hierarchical predictive coding framework in which increased evoked activity reflects the activation of residual information that was not already represented at a given level of the fronto-temporal hierarchy ("error" activity). Between 300 and 500 ms, the three conditions produced progressively larger responses within left temporal cortex (lexico-semantic prediction error), whereas implausible inputs produced a selectively enhanced response within inferior frontal cortex (prediction error at the level of the event model). Between 600 and 1,000 ms, unexpected plausible words activated left inferior frontal and middle temporal cortices (feedback activity that produced top-down error), whereas highly implausible inputs activated left inferior frontal cortex, posterior fusiform (unsuppressed orthographic prediction error/reprocessing), and medial temporal cortex (possibly supporting new learning). Therefore, predictive coding may provide a unifying theory that links language comprehension to other domains of cognition.

Capacity limits in sentence comprehension: Evidence from dual-task judgements and event-related potentials.

There is an ongoing controversy over whether readers can access the meaning of multiple words, simultaneously. To date, different experimental methods have generated seemingly contradictory evidence in support of serial or parallel processing accounts. For example, dual-task studies suggest that readers can process a maximum of one word at a time (White, Palmer & Boynton, 2018), while ERP studies have demonstrated neural priming effects that are more consistent with parallel activation (Wen, Snell & Grainger, 2019). To help reconcile these views, I measured neural responses and behavioral accuracy in a dual-task sentence comprehension paradigm. Participants saw masked sentences and two-word phrases and had to judge whether or not they were grammatical. Grammatically correct sentences (This girl is neat) produced smaller N400 responses compared to scrambled sentences (Those girl is fled): an N400 sentence superiority effect. Critically, participants' grammaticality judgements on the same trials showed striking capacity limitations, with dual-task deficits closely matching the predictions of a serial, all-or-none processing account. Together, these findings suggest that the N400 sentence superiority effect is fully compatible with serial word recognition, and that readers are unable to process multiple sentence positions simultaneously.

Domain-general conflict monitoring predicts neural and behavioral indices of linguistic error processing during reading comprehension.

The ability to detect and respond to linguistic errors is critical for successful reading comprehension, but these skills can vary considerably across readers. In the current study, healthy adults (age 18-35) read short discourse scenarios for comprehension while monitoring for the presence of semantic anomalies. Using a factor analytic approach, we examined if performance in nonlinguistic conflict monitoring tasks (Stroop, AX-CPT) would predict individual differences in neural and behavioral measures of linguistic error processing. Consistent with this hypothesis, domain-general conflict monitoring predicted both readers' end-of-trial acceptability judgments and the amplitude of a late neural response (the P600) evoked by linguistic anomalies. The influence on the P600 was nonlinear, suggesting that online neural responses to linguistic errors are influenced by both the effectiveness and efficiency of domain-general conflict monitoring. These relationships were also highly specific and remained after controlling for variability in working memory capacity and verbal knowledge. Finally, we found that domain-general conflict monitoring also predicted individual variability in measures of reading comprehension, and that this relationship was partially mediated by behavioral measures of linguistic error detection. These findings inform our understanding of the role of domain-general executive functions in reading comprehension, with potential implications for the diagnosis and treatment of language impairments. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Cognitive control mediates age-related changes in flexible anticipatory processing during listening comprehension.

Effective listening comprehension not only requires processing local linguistic input, but also necessitates incorporating contextual cues available in the global communicative environment. Local sentence processing can be facilitated by pre-activation of likely upcoming input, or predictive processing. Recent evidence suggests that young adults can flexibly adapt local predictive processes based on cues provided by the global communicative environment, such as the reliability of specific speakers. Whether older comprehenders can also flexibly adapt to global contextual cues is currently unknown. Moreover, it is unclear whether the underlying mechanisms supporting local predictive processing differ from those supporting adaptation to global contextual cues. Critically, it is unclear whether these mechanisms change as a function of typical aging. We examined the flexibility of prediction in young and older adults by presenting sentences from speakers whose utterances were typically more or less predictable (i.e., reliable speakers who produced expected words 80% of the time, versus unreliable speakers who produced expected words 20% of the time). For young listeners, global speaker reliability cues modulated neural effects of local predictability on the N400. In contrast, older adults, on average, did not show global modulation of local processing. Importantly, however, cognitive control (i.e., Stroop interference effects) mediated age-related reductions in sensitivity to the reliability of the speaker. Both young and older adults with high cognitive control showed greater N400 effects of predictability during sentences produced by a reliable speaker, suggesting that cognitive control is required to regulate the strength of top-down predictions based on global contextual information. Critically, cognitive control predicted sensitivity to global speaker-specific information but not local predictability cues, suggesting that predictive processing in local sentence contexts may be supported by separable neural mechanisms from adaptation of prediction as a function of global context. These results have important implications for interpreting age-related change in predictive processing, and for drawing more generalized conclusions regarding domain-general versus language-specific accounts of prediction.

Bilinguals on the garden-path: Individual differences in syntactic ambiguity resolution.

Syntactic parsing plays a central role in the interpretation of sentences, but it is unclear to what extent non-native speakers can deploy native-like grammatical knowledge during online comprehension. The current eye-tracking study investigated how Chinese-English bilinguals and native English speakers respond to syntactic category and subcategorization information while reading sentences with OBJECT-SUBJECT ambiguities. We also obtained measures of English language experience, working memory capacity, and executive function to determine how these cognitive variables influence online parsing. During reading, monolinguals and bilinguals showed similar GARDEN-PATH EFFECTS related to syntactic reanalysis, but native English speakers responded more robustly to VERB SUBCATEGORIZATION cues. Readers with greater language experience and executive function showed increased sensitivity to verb subcategorization cues, but parsing was not influenced by working memory capacity. These results are consistent with exposure-based accounts of bilingual sentence processing, and they support a link between syntactic processing and domain-general cognitive control.

Word predictability effects are linear, not logarithmic: Implications for probabilistic models of sentence comprehension.

During language comprehension, we routinely use information from the prior context to help identify the meaning of individual words. While measures of online processing difficulty, such as reading times, are strongly influenced by contextual predictability, there is disagreement about the mechanisms underlying this lexical predictability effect, with different models predicting different linking functions - linear (Reichle, Rayner & Pollatsek, 2003) or logarithmic (Levy, 2008). To help resolve this debate, we conducted two highly-powered experiments (self-paced reading, N = 216; cross-modal picture naming, N = 36), and a meta-analysis of prior eye-tracking while reading studies (total N = 218). We observed a robust linear relationship between lexical predictability and word processing times across all three studies. Beyond their methodological implications, these findings also place important constraints on predictive processing models of language comprehension. In particular, these results directly contradict the empirical predictions of surprisal theory, while supporting a proportional pre-activation account of lexical prediction effects in comprehension.

Lack of awareness despite complex visual processing: Evidence from event-related potentials in a case of selective metamorphopsia.

Visual awareness is thought to result from integration of low- and high-level processing; instances of integration failure provide a crucial window into the cognitive and neural bases of awareness. We present neurophysiological evidence of complex cognitive processing in the absence of awareness, raising questions about the conditions necessary for visual awareness. We describe an individual with a neurodegenerative disease who exhibits impaired visual awareness for the digits 2 to 9, and stimuli presented in close proximity to these digits, due to perceptual distortion. We identified robust event-related potential responses indicating 1) face detection with the N170 component and 2) task-dependent target-word detection with the P3b component, despite no awareness of the presence of faces or target words. These data force us to reconsider the relationship between neural processing and visual awareness; even stimuli processed by a workspace-like cognitive system can remain inaccessible to awareness. We discuss how this finding challenges and constrains theories of visual awareness.

Going the Extra Mile: Effects of Discourse Context on Two Late Positivities During Language Comprehension.

During language comprehension, online neural processing is strongly influenced by the constraints of the prior context. While the N400 ERP response (300-500ms) is known to be sensitive to a word's semantic predictability, less is known about a set of late positive-going ERP responses (600-1000ms) that can be elicited when an incoming word violates strong predictions about upcoming content (late frontal positivity) or about what is possible given the prior context (late posterior positivity/P600). Across three experiments, we systematically manipulated the length of the prior context and the source of lexical constraint to determine their influence on comprehenders' online neural responses to these two types of prediction violations. In Experiment 1, within minimal contexts, both lexical prediction violations and semantically anomalous words produced a larger N400 than expected continuations (James unlocked the door/laptop/gardener), but no late positive effects were observed. Critically, the late posterior positivity/P600 to semantic anomalies appeared when these same sentences were embedded within longer discourse contexts (Experiment 2a), and the late frontal positivity appeared to lexical prediction violations when the preceding context was rich and globally constraining (Experiment 2b). We interpret these findings within a hierarchical generative framework of language comprehension. This framework highlights the role of comprehension goals and broader linguistic context, and how these factors influence both top-down prediction and the decision to update or reanalyze the prior context when these predictions are violated.

A Tale of Two Positivities and the N400: Distinct Neural Signatures Are Evoked by Confirmed and Violated Predictions at Different Levels of Representation.

It has been proposed that hierarchical prediction is a fundamental computational principle underlying neurocognitive processing. Here, we ask whether the brain engages distinct neurocognitive mechanisms in response to inputs that fulfill versus violate strong predictions at different levels of representation during language comprehension. Participants read three-sentence scenarios in which the third sentence constrained for a broad event structure, for example, {Agent caution animate-Patient}. High constraint contexts additionally constrained for a specific event/lexical item, for example, a two-sentence context about a beach, lifeguards, and sharks constrained for the event, {Lifeguards cautioned Swimmers}, and the specific lexical item swimmers. Low constraint contexts did not constrain for any specific event/lexical item. We measured ERPs on critical nouns that fulfilled and/or violated each of these constraints. We found clear, dissociable effects to fulfilled semantic predictions (a reduced N400), to event/lexical prediction violations (an increased late frontal positivity), and to event structure/animacy prediction violations (an increased late posterior positivity/P600). We argue that the late frontal positivity reflects a large change in activity associated with successfully updating the comprehender's current situation model with new unpredicted information. We suggest that the late posterior positivity/P600 is triggered when the comprehender detects a conflict between the input and her model of the communicator and communicative environment. This leads to an initial failure to incorporate the unpredicted input into the situation model, which may be followed by second-pass attempts to make sense of the discourse through reanalysis, repair, or reinterpretation. Together, these findings provide strong evidence that confirmed and violated predictions at different levels of representation manifest as distinct spatiotemporal neural signatures.

Flexible predictions during listening comprehension: Speaker reliability affects anticipatory processes.

During listening comprehension, the identification of individual words can be strongly influenced by properties of the preceding context. While sentence context can facilitate both behavioral and neural responses, it is unclear whether these effects can be attributed to the pre-activation of lexico-semantic features or the facilitated integration of contextually congruent words. Moreover, little is known about how statistics of the broader language environment, or information about the current speaker, might shape these facilitation effects. In the present study, we measured neural responses to predictable and unpredictable words as participants listened to sentences for comprehension. Critically, we manipulated the reliability of each speaker's utterances, such that individual speakers either tended to complete sentences with words that were highly predictable (reliable speaker) or with words that were unpredictable but still plausible (unreliable speaker). As expected, the amplitude of the N400 was reduced for locally predictable words, but, critically, these context effects were also modulated by speaker identity. Sentences from a reliable speaker showed larger facilitation effects with an earlier onset, suggesting that listeners engaged in enhanced anticipatory processing when a speaker's behavior was more predictable. This finding suggests that listeners can implicitly track the reliability of predictive cues in their environment and use these statistics to adaptively regulate predictive processing.

Phonological versus semantic prediction in focus and repair constructions: No evidence for differential predictions.

Evidence suggests that the language processing system is predictive. Although past research has established prediction as a general tendency, it is not yet clear whether comprehenders can modulate their anticipatory strategies in response to cues based on sentence constructions. In two visual world eye-tracking experiments, we investigated whether focus constructions (not the hammer but rather the …) and repair disfluencies (the hammer uh I mean the …) would lead listeners to generate different patterns of predictions. In three offline tasks, we observed that participants preferred semantically related continuations (hammer - nail) following focus constructions and phonologically related continuations (hammer - hammock) following disfluencies. However, these offline preferences were not evident in participants' predictive eye-movements during online language processing: Semantically related (nail) and phonologically related words (hammock) received additional predictive looks regardless of whether the target word appeared in a disfluency or in a focus construction. However, significantly less semantic and phonological activation was observed in two "control" linguistic contexts in which predictive processing was discouraged. These findings suggest that although the prediction system is sensitive to sentence construction, is it not flexible enough to alter the type of prediction generated based on preceding context.

Electrophysiological evidence for preserved primacy of lexical prediction in aging.

Young adults show consistent neural benefits of predictable contexts when processing upcoming words, but these benefits are less clear-cut in older adults. Here we disentangle the neural correlates of prediction accuracy and contextual support during word processing, in order to test current theories that suggest that neural mechanisms underlying predictive processing are specifically impaired in older adults. During a sentence comprehension task, older and younger readers were asked to predict passage-final words and report the accuracy of these predictions. Age-related reductions were observed for N250 and N400 effects of prediction accuracy, as well as for N400 effects of contextual support independent of prediction accuracy. Furthermore, temporal primacy of predictive processing (i.e., earlier facilitation for successful predictions) was preserved across the lifespan, suggesting that predictive mechanisms are unlikely to be uniquely impaired in older adults. In addition, older adults showed prediction effects on frontal post-N400 positivities (PNPs) that were similar in amplitude to PNPs in young adults. Previous research has shown correlations between verbal fluency and lexical prediction in older adult readers, suggesting that the production system may be linked to capacity for lexical prediction, especially in aging. The current study suggests that verbal fluency modulates PNP effects of contextual support, but not prediction accuracy. Taken together, our findings suggest that aging does not result in specific declines in lexical prediction.

Early processing of orthographic language membership information in bilingual visual word recognition: Evidence from ERPs.

For successful language comprehension, bilinguals often must exert top-down control to access and select lexical representations within a single language. These control processes may critically depend on identification of the language to which a word belongs, but it is currently unclear when different sources of such language membership information become available during word recognition. In the present study, we used event-related potentials to investigate the time course of influence of orthographic language membership cues. Using an oddball detection paradigm, we observed early neural effects of orthographic bias (Spanish vs. English orthography) that preceded effects of lexicality (word vs. pseudoword). This early orthographic pop-out effect was observed for both words and pseudowords, suggesting that this cue is available prior to full lexical access. We discuss the role of orthographic bias for models of bilingual word recognition and its potential role in the suppression of nontarget lexical information.

Anticipating syntax during reading: Evidence from the boundary change paradigm.

Previous evidence suggests that grammatical constraints have a rapid influence during language comprehension, particularly at the level of word categories (noun, verb, preposition). These findings are in conflict with a recent study from Angele, Laishley, Rayner, and Liversedge (2014), in which sentential fit had no early influence on word skipping rates during reading. In the present study, we used a gaze-contingent boundary change paradigm to manipulate the syntactic congruity of an upcoming noun or verb outside of participants' awareness. Across 3 experiments (total N = 148), we observed higher skipping rates for syntactically valid previews (The admiral would not confess . . .), when compared with violation previews (The admiral would not surgeon . . .). Readers were less likely to skip an ungrammatical continuation, even when that word was repeated within the same sentence (The admiral would not admiral . . .), suggesting that word-class constraints can take precedence over lexical repetition effects. To our knowledge, these results provide the first evidence for an influence of syntactic context during parafoveal word recognition. On the basis of the early time-course of this effect, we argue that readers can use grammatical constraints to generate syntactic expectations for upcoming words. (PsycINFO Database Record

Language Membership Identification Precedes Semantic Access: Suppression during Bilingual Word Recognition.

Previous research suggests that bilingual comprehenders access lexical representations of words in both languages nonselectively. However, it is unclear whether global language suppression plays a role in guiding attention to target language representations during ongoing lexico-semantic processing. To help clarify this issue, this study examined the relative timing of language membership and meaning activation during visual word recognition. Spanish-English bilinguals performed simultaneous semantic and language membership classification tasks on single words during EEG recording. Go/no-go ERP latencies provided evidence that language membership information was accessed before semantic information. Furthermore, N400 frequency effects indicated that the depth of processing of words in the nontarget language was reduced compared to the target language. These results suggest that the bilingual brain can rapidly identify the language to which a word belongs and subsequently use this information to selectively modulate the degree of processing in each language accordingly.

Effects of prediction and contextual support on lexical processing: prediction takes precedence.

Readers may use contextual information to anticipate and pre-activate specific lexical items during reading. However, prior studies have not clearly dissociated the effects of accurate lexical prediction from other forms of contextual facilitation such as plausibility or semantic priming. In this study, we measured electrophysiological responses to predicted and unpredicted target words in passages providing varying levels of contextual support. This method was used to isolate the neural effects of prediction from other potential contextual influences on lexical processing. While both prediction and discourse context influenced ERP amplitudes within the time range of the N400, the effects of prediction occurred much more rapidly, preceding contextual facilitation by approximately 100 ms. In addition, a frontal, post-N400 positivity (PNP) was modulated by both prediction accuracy and the overall plausibility of the preceding passage. These results suggest a unique temporal primacy for prediction in facilitating lexical access. They also suggest that the frontal PNP may index the costs of revising discourse representations following an incorrect lexical prediction.

Two sensory channels mediate perception of fingertip force.

In two experiments we examined the ability of humans to exert forces accurately with the fingertips, and to perceive those forces. In experiment 1 participants used visual feedback to apply a range of fingertip forces with the distal pad of the thumb. Participants made magnitude discriminations regarding these forces, and their just noticeable differences were calculated at a series of standards by means of a two-interval, forced-choice tracking paradigm. As the standard increased, participants demonstrated a relative improvement in force discrimination; and the presence of a possible inflection point, at approximately 400 g, suggested that two sensory channels may contribute to performance. If this is the case, the operative channel at low forces is almost certainly the slowly adapting type I (SA-I) channel, while another mechanoreceptor class, the SA-II nail unit, is a plausible mediator of the more accurate performance seen at high force levels. To test this two-channel hypothesis in experiment 2, we hydrated participants' thumbnails in order to reduce nail rigidity and thus prevent stimulation of underlying SA-II mechanoreceptors. This technique was found to reduce sensory accuracy in a force-matching task at high forces (1000 g) while leaving low force matching (100 g) unimpaired. Taken together, these results suggest that two sensory channels mediate the perception of fingertip forces in humans: one channel predominating at low forces (below approximately 400 g) and another responsible for perceiving high forces which is likely mediated by the SA-II nail unit.