Predictive language processing: integrating comprehension and production, and what atypical populations can tell us.

Predictive processing, a crucial aspect of human cognition, is also relevant for language comprehension. In everyday situations, we exploit various sources of information to anticipate and therefore facilitate processing of upcoming linguistic input. In the literature, there are a variety of models that aim at accounting for such ability. One group of models propose a strict relationship between prediction and language production mechanisms. In this review, we first introduce very briefly the concept of predictive processing during language comprehension. Secondly, we focus on models that attribute a prominent role to language production and sensorimotor processing in language prediction ("prediction-by-production" models). Contextually, we provide a summary of studies that investigated the role of speech production and auditory perception on language comprehension/prediction tasks in healthy, typical participants. Then, we provide an overview of the limited existing literature on specific atypical/clinical populations that may represent suitable testing ground for such models-i.e., populations with impaired speech production and auditory perception mechanisms. Ultimately, we suggest a more widely and in-depth testing of prediction-by-production accounts, and the involvement of atypical populations both for model testing and as targets for possible novel speech/language treatment approaches.

I know how you'll say it: evidence of speaker-specific speech prediction.

Most models of language comprehension assume that the linguistic system is able to pre-activate phonological information. However, the evidence for phonological prediction is mixed and controversial. In this study, we implement a paradigm that capitalizes on the fact that foreign speakers usually make phonological errors. We investigate whether speaker identity (native vs. foreign) is used to make specific phonological predictions. Fifty-two participants were recruited to read sentence frames followed by a last spoken word which was uttered by either a native or a foreign speaker. They were required to perform a lexical decision on the last spoken word, which could be either semantically predictable or not. Speaker identity (native vs. foreign) may or may not be cued by the face of the speaker. We observed that the face cue is effective in speeding up the lexical decision when the word is predictable, but it is not effective when the word is not predictable. This result shows that speech prediction takes into account the phonological variability between speakers, suggesting that it is possible to pre-activate in a detailed and specific way the phonological representation of a predictable word.

Combining EEG and 3D-eye-tracking to study the prediction of upcoming speech in naturalistic virtual environments: A proof of principle.

EEG and eye-tracking provide complementary information when investigating language comprehension. Evidence that speech processing may be facilitated by speech prediction comes from the observation that a listener's eye gaze moves towards a referent before it is mentioned if the remainder of the spoken sentence is predictable. However, changes to the trajectory of anticipatory fixations could result from a change in prediction or an attention shift. Conversely, N400 amplitudes and concurrent spectral power provide information about the ease of word processing the moment the word is perceived. In a proof-of-principle investigation, we combined EEG and eye-tracking to study linguistic prediction in naturalistic, virtual environments. We observed increased processing, reflected in theta band power, either during verb processing - when the verb was predictive of the noun - or during noun processing - when the verb was not predictive of the noun. Alpha power was higher in response to the predictive verb and unpredictable nouns. We replicated typical effects of noun congruence but not predictability on the N400 in response to the noun. Thus, the rich visual context that accompanied speech in virtual reality influenced language processing compared to previous reports, where the visual context may have facilitated processing of unpredictable nouns. Finally, anticipatory fixations were predictive of spectral power during noun processing and the length of time fixating the target could be predicted by spectral power at verb onset, conditional on the object having been fixated. Overall, we show that combining EEG and eye-tracking provides a promising new method to answer novel research questions about the prediction of upcoming linguistic input, for example, regarding the role of extralinguistic cues in prediction during language comprehension.

Disruption to left inferior frontal cortex modulates semantic prediction effects in reading and subsequent memory: Evidence from simultaneous TMS-EEG.

Readers use prior context to predict features of upcoming words. When predictions are accurate, this increases the efficiency of comprehension. However, little is known about the fate of predictable and unpredictable words in memory or the neural systems governing these processes. Several theories suggest that the speech production system, including the left inferior frontal cortex (LIFC), is recruited for prediction but evidence that LIFC plays a causal role is lacking. We first examined the effects of predictability on memory and then tested the role of posterior LIFC using transcranial magnetic stimulation (TMS). In Experiment 1, participants read category cues, followed by a predictable, unpredictable, or incongruent target word for later recall. We observed a predictability benefit to memory, with predictable words remembered better than unpredictable words. In Experiment 2, participants performed the same task with electroencephalography (EEG) while undergoing event-related TMS over posterior LIFC using a protocol known to disrupt speech production, or over the right hemisphere homologue as an active control site. Under control stimulation, predictable words were better recalled than unpredictable words, replicating Experiment 1. This predictability benefit to memory was eliminated under LIFC stimulation. Moreover, while an a priori ROI-based analysis did not yield evidence for a reduction in the N400 predictability effect, mass-univariate analyses did suggest that the N400 predictability effect was reduced in spatial and temporal extent under LIFC stimulation. Collectively, these results provide causal evidence that the LIFC is recruited for prediction during silent reading, consistent with prediction-through-production accounts.

Inefficient speech-motor control affects predictive speech comprehension: atypical electrophysiological correlates in stuttering.

Listeners predict upcoming information during language comprehension. However, how this ability is implemented is still largely unknown. Here, we tested the hypothesis proposing that language production mechanisms have a role in prediction. We studied 2 electroencephalographic correlates of predictability during speech comprehension-pre-target alpha-beta (8-30 Hz) power decrease and the post-target N400 event-related potential effect-in a population with impaired speech-motor control, i.e. adults who stutter (AWS), compared to typically fluent adults (TFA). Participants listened to sentences that could either constrain towards a target word or not, modulating its predictability. As a complementary task, participants also performed context-driven word production. Compared to TFA, AWS not only displayed atypical neural responses in production, but, critically, they showed a different pattern also in comprehension. Specifically, while TFA showed the expected pre-target power decrease, AWS showed a power increase in frontal regions, associated with speech-motor control. In addition, the post-target N400 effect was reduced for AWS with respect to TFA. Finally, we found that production and comprehension power changes were positively correlated in TFA, but not in AWS. Overall, the results support the idea that processes and neural structures prominently devoted to speech planning also support prediction during speech comprehension.

Prediction of baseline expressive and receptive language function in children with focal epilepsy using diffusion tractography-based deep learning network.

PURPOSE: Focal epilepsy is a risk factor for language impairment in children. We investigated whether the current state-of-the-art deep learning network on diffusion tractography connectome can accurately predict expressive and receptive language scores of children with epilepsy. METHODS: We studied 37 children with a diagnosis of drug-resistant focal epilepsy (age: 11.8 ±â€¯3.1 years) using 3 T MRI and diffusion tractography connectome: G = (S, Ω), where S is an adjacency matrix of edges representing the connectivity strength (number of white-matter tract streamlines) between each pair of brain regions, and Ω reflects a set of brain regions. A convolutional neural network (CNN) was trained to learn the nonlinear relationship between 'S (input)' and 'language score (output)'. Repeated hold-out validation was then employed to measure the Pearson correlation and mean absolute error (MAE) between CNN-predicted and actual language scores. RESULTS: We found that CNN-predicted and actual scores were significantly correlated (i.e., Pearson's R/p-value: 0.82/<0.001 and 0.75/<0.001), yielding MAE: 7.77 and 7.40 for expressive and receptive scores, respectively. Specifically, sparse connectivity not only within the left cortico-cortical network but also involving the right subcortical structures was predictive of language impairment of expressive or receptive domain. Subsequent subgroup analyses inferred that the effectiveness of diffusion tractography-based prediction of language outcome was independent of clinical variables. Intrinsic diffusion tractography connectome properties may be useful for predicting the severity of baseline language dysfunction and possibly provide a better understanding of the biological mechanisms of epilepsy-related language impairment in children.

Everyday Language Exposure Shapes Prediction of Specific Words in Listening Comprehension: A Visual World Eye-Tracking Study.

We investigated the effects of everyday language exposure on the prediction of orthographic and phonological forms of a highly predictable word during listening comprehension. Native Japanese speakers in Tokyo (Experiment 1) and Berlin (Experiment 2) listened to sentences that contained a predictable word and viewed four objects. The critical object represented the target word (e.g., /sakana/; fish), an orthographic competitor (e.g., /tuno/; horn), a phonological competitor (e.g., /sakura/; cherry blossom), or an unrelated word (e.g., /hon/; book). The three other objects were distractors. The Tokyo group fixated the target and the orthographic competitor over the unrelated objects before the target word was mentioned, suggesting that they pre-activated the orthographic form of the target word. The Berlin group showed a weaker bias toward the target than the Tokyo group, and they showed a tendency to fixate the orthographic competitor only when the orthographic similarity was very high. Thus, prediction effects were weaker in the Berlin group than in the Tokyo group. We found no evidence for the prediction of phonological information. The obtained group differences support probabilistic models of prediction, which regard the built-up language experience as a basis of prediction.

Commonalities in alpha and beta neural desynchronizations during prediction in language comprehension and production.

The present study investigates whether predictions during language comprehension are generated by engaging the language production system. Previous studies investigating either prediction or production highlighted M/EEG desynchronization (power decrease) in the alpha (8-10 Hz) and beta (13-30 Hz) frequency bands preceding the target. However, it is unclear whether this electrophysiological modulation underlies common mechanisms. We recorded EEG from participants performing both a comprehension and a production task in two separate blocks. Participants listened to high and low constraint incomplete sentences and were asked either to name a picture to complete them (production) or to simply listen to the final word (comprehension). We found that in a silent gap before the final stimulus, predictable stimuli elicited alpha and beta desynchronization in both tasks, signaling the pre-activation of linguistic information. Source estimation highlighted the involvement of left-lateralized language areas and temporo-parietal areas in the right hemisphere. Furthermore, correlations between the desynchronizations in comprehension and production showed spatiotemporal commonalities in language-relevant areas of the left hemisphere. As proposed by prediction-by-production models, our results suggest that comprehenders engage the production system while predicting upcoming words.

The activation of contextually predictable words in syntactically illegal positions.

We present an eye-tracking study testing a hypothesis emerging from several theories of prediction during language processing, whereby predictable words should be skipped more than unpredictable words even in syntactically illegal positions. Participants read sentences in which a target word became predictable by a certain point (e.g., "bone" is 92% predictable given, "The dog buried his. . ."), with the next word actually being an intensifier (e.g., "really"), which a noun cannot follow. The target noun remained predictable to appear later in the sentence. We used the boundary paradigm to present the predictable noun or an alternative unpredictable noun (e.g., "food") directly after the intensifier, until participants moved beyond the intensifier, at which point the noun changed to a syntactically legal word. Participants also read sentences in which predictable or unpredictable nouns appeared in syntactically legal positions. A Bayesian linear-mixed model suggested a 5.7% predictability effect on skipping of nouns in syntactically legal positions, and a 3.1% predictability effect on skipping of nouns in illegal positions. We discuss our findings in relation to theories of lexical prediction during reading.

Prediction of orthographic information during listening comprehension: A printed-word visual world study.

Two visual world eye-tracking experiments examined the role of orthographic information in the visual context in pre-activation of orthographic and phonological information using Japanese. Participants heard sentences that contained a predictable target word and viewed a display showing four words in a logogram, kanji (Experiment 1) or in a phonogram, hiragana (Experiment 2). The four words included either the target word (e.g., /sakana/; fish), an orthographic competitor (e.g., /tuno/; horn), a phonological competitor (e.g., /sakura/; cherry blossom), or an unrelated word (e.g., /hon/; book), together with three distractor words. The orthographic competitor was orthographically or phonologically dissimilar to the target in hiragana. In Experiment 1, target and orthographic competitor words attracted more fixations than unrelated words before the target word was mentioned, suggesting that participants pre-activated orthographic form of the target word. In Experiment 2, target and phonological competitor words attracted more predictive fixations than unrelated words, but orthographic competitor words did not, suggesting a critical role of the visual context. This pre-activation pattern does not fit with the pattern of lexical activation in auditory word recognition, where orthography and phonology interact. However, it is compatible with the pattern of lexical activation in spoken word production, where orthographic information is not automatically activated, in line with production-based prediction accounts.

Perceptual facilitation of word recognition through motor activation during sentence comprehension.

Despite the growing literature on anticipatory language processing, the brain dynamics of this high-level predictive process are still unclear. In the present MEG study, we analyzed pre- and post-stimulus oscillatory activity time-locked to the reading of a target word. We experimentally contrasted the processing of the same target word following two highly constraining sentence contexts, in which the constraint was driven either by the semantic content or by the lexical association between words. Previous research suggests the presence of sensory facilitation for expected words in the latter condition but not in the former. We observed a dissociation between beta (∼20 Hz) and gamma (>50 Hz) band activity in pre- and post-stimulus time intervals respectively. Both the beta and gamma effects were evident in occipital brain regions, and only the pre-stimulus beta effect additionally involved left pre-articulatory motor regions. Lexically constrained (vs. semantically constrained) words elicited reduced beta power around 400 msec before the target word in motor regions and a functionally related gamma enhancement in occipital regions around 200 msec post-target. The present findings highlight the role of the motor network in word-form prediction and support proposals claiming that low-level perceptual representations can be pre-activated during language prediction.