Spoken language coding neurons in the Visual Word Form Area: Evidence from a TMS adaptation paradigm.

While part of the left ventral occipito-temporal cortex (left-vOT), known as the Visual Word Form Area, plays a central role in reading, the area also responds to speech. This cross-modal activation has been explained by three competing hypotheses. Firstly, speech is converted to orthographic representations that activate, in a top-down manner, written language coding neurons in the left-vOT. Secondly, the area contains multimodal neurons that respond to both language modalities. Thirdly, the area comprises functionally segregated neuronal populations that selectively encode different language modalities. A transcranial magnetic stimulation (TMS)-adaptation protocol was used to disentangle these hypotheses. During adaptation, participants were exposed to spoken or written words in order to tune the initial state of left-vOT neurons to one of the language modalities. After adaptation, they performed lexical decisions on spoken and written targets with TMS applied to the left-vOT. TMS showed selective facilitatory effects. It accelerated lexical decisions only when the adaptors and the targets shared the same modality, i.e., when left-vOT neurons had initially been adapted to the modality of the target stimuli. Since this within-modal adaptation was observed for both input modalities and no evidence for cross-modal adaptation was found, our findings suggest that the left-vOT contains neurons that selectively encode written and spoken language rather than purely written language coding neurons or multimodal neurons encoding language regardless of modality.

Top-down activation of the visuo-orthographic system during spoken sentence processing.

The left ventral occipitotemporal cortex (vOT) is considered the key area of the visuo-orthographic system. However, some studies reported that the area is also involved in speech processing tasks, especially those that require activation of orthographic knowledge. These findings suggest the existence of a top-down activation mechanism allowing such cross-modal activation. Yet, little is known about the involvement of the vOT in more natural speech processing situations like spoken sentence processing. Here, we addressed this issue in a functional Magnetic Resonance Imaging (fMRI) study while manipulating the impacts of two factors, i.e., task demands (semantic vs. low-level perceptual task) and the quality of speech signals (sentences presented against clear vs. noisy background). Analyses were performed at the levels of whole brain and region-of-interest (ROI) focusing on the vOT voxels individually identified through a reading task. Whole brain analysis showed that processing spoken sentences induced activity in a large network including the regions typically involved in phonological, articulatory, semantic and orthographic processing. ROI analysis further specified that a significant part of the vOT voxels that responded to written words also responded to spoken sentences, thus, suggesting that the same area within the left occipitotemporal pathway contributes to both reading and speech processing. Interestingly, both analyses provided converging evidence that vOT responses to speech were sensitive to both task demands and quality of speech signals: Compared to the low-level perceptual task, activity of the area increased when efforts on comprehension were required. The impact of background noise depended on task demands. It led to a decrease of vOT activity in the semantic task but not in the low-level perceptual task. Our results provide new insights into the function of this key area of the reading network, notably by showing that its speech-induced top-down activation also generalizes to ecological speech processing situations.

Automaticity of phonological and semantic processing during visual word recognition.

Reading involves activation of phonological and semantic knowledge. Yet, the automaticity of the activation of these representations remains subject to debate. The present study addressed this issue by examining how different brain areas involved in language processing responded to a manipulation of bottom-up (level of visibility) and top-down information (task demands) applied to written words. The analyses showed that the same brain areas were activated in response to written words whether the task was symbol detection, rime detection, or semantic judgment. This network included posterior, temporal and prefrontal regions, which clearly suggests the involvement of orthographic, semantic and phonological/articulatory processing in all tasks. However, we also found interactions between task and stimulus visibility, which reflected the fact that the strength of the neural responses to written words in several high-level language areas varied across tasks. Together, our findings suggest that the involvement of phonological and semantic processing in reading is supported by two complementary mechanisms. First, an automatic mechanism that results from a task-independent spread of activation throughout a network in which orthography is linked to phonology and semantics. Second, a mechanism that further fine-tunes the sensitivity of high-level language areas to the sensory input in a task-dependent manner.

Contribution of writing to reading: Dissociation between cognitive and motor process in the left dorsal premotor cortex.

Functional brain imaging studies reported activation of the left dorsal premotor cortex (PMd), that is, a main area in the writing network, in reading tasks. However, it remains unclear whether this area is causally relevant for written stimulus recognition or its activation simply results from a passive coactivation of reading and writing networks. Here, we used chronometric paired-pulse transcranial magnetic stimulation (TMS) to address this issue by disrupting the activity of the PMd, the so-called Exner's area, while participants performed a lexical decision task. Both words and pseudowords were presented in printed and handwritten characters. The latter was assumed to be closely associated with motor representations of handwriting gestures. We found that TMS over the PMd in relatively early time-windows, i.e., between 60 and 160 ms after the stimulus onset, increased reaction times to pseudoword without affecting word recognition. Interestingly, this result pattern was found for both printed and handwritten characters, that is, regardless of whether the characters evoked motor representations of writing actions. Our result showed that under some circumstances the activation of the PMd does not simply result from passive association between reading and writing networks but has a functional role in the reading process. At least, at an early stage of written stimuli recognition, this role seems to depend on a common sublexical and serial process underlying writing and pseudoword reading rather than on an implicit evocation of writing actions during reading as typically assumed.

Does orthographic training on a phonemic contrast absent in the listener's dialect influence word recognition?

This study examined whether the ability of southern French speakers to discriminate between standard French word forms such as /pike/ and /pikε/ can be improved by a training procedure in which participants were exposed to the orthographic representations of words forming /e/-/ε/ minimal pairs. The results of the training procedure showed that southern French speakers were able to perceive the /e/-/ε/ contrast in word final position when they associated words containing these vowels with their correct spelled form. Further, participants in a priming experiment, which was run immediately after training, no longer showed the priming effect on the trained minimal pairs that they had shown in the pre-test. However, a priming effect on the untrained minimal pairs was still observed immediately after training, showing that this training failed to transfer to untrained items. Finally, the benefits of the training procedure were no longer observed the day after training, since southern French speakers once again showed a priming effect on the trained minimal pair of words in a one day post-test. Implications of these findings for the locus of the difficulties of the southern French speakers with the word-final /e/-/ε/ contrast are discussed.

How Early Does the Brain Distinguish between Regular Words, Irregular Words, and Pseudowords during the Reading Process? Evidence from Neurochronometric TMS.

Cognitive theories on reading propose that the characteristics of written stimuli determine how they are processed in the brain. However, whether the brain distinguishes between regular words, irregular words, and pseudowords already at an early stage of the reading process is still subject to debate. Here we used chronometric TMS to address this issue. During the first 140 msec of regular word, irregular word, and pseudoword reading, TMS was used to disrupt the function of the ventral occipitotemporal, posterior middle temporal, and supramarginal gyri, which are key areas involved in orthographic, semantic, and phonological processing, respectively. Early TMS stimulation delivered on posterior middle temporal and supramarginal gyri affected regular and irregular word, but not pseudoword, reading. In contrast, ventral occipitotemporal disruption affected both word and pseudoword reading. We thus found evidence for an early distinction between word and pseudoword processing in the semantic and phonological systems, but not in the orthographic system.

How are visemes and graphemes integrated with speech sounds during spoken word recognition? ERP evidence for supra-additive responses during audiovisual compared to auditory speech processing.

Both visual articulatory gestures and orthography provide information on the phonological content of speech. This EEG study investigated the integration between speech and these two visual inputs. A comparison of skilled readers' brain responses elicited by a spoken word presented alone versus synchronously with a static image of a viseme or a grapheme of the spoken word's onset showed that while neither visual input induced audiovisual integration on N1 acoustic component, both led to a supra-additive integration on P2, with a stronger integration between speech and graphemes on left-anterior electrodes. This pattern persisted in P350 time-window and generalized to all electrodes. The finding suggests a strong impact of spelling knowledge on phonetic processing and lexical access. It also indirectly indicates that the dynamic and predictive value present in natural lip movements but not in static visemes is particularly critical to the contribution of visual articulatory gestures to speech processing.

The contribution of visual articulatory gestures and orthography to speech processing: Evidence from novel word learning.

Auditory speech appears to be linked to visual articulatory gestures and orthography through different mechanisms. Yet, both types of visual information have a strong influence on speech processing. The present study directly compared their contributions to speech processing using a novel word learning paradigm. Native speakers of French, who were familiar with English, learned minimal pairs of novel English words containing the English /θ/-/f/ phonemic contrast under one of three exposure conditions: (a) the auditory forms of novel words alone, (b) the auditory forms associated with articulatory gestures, or (c) the auditory forms associated with orthography. The benefits of the three methods were compared during training and at two posttraining time points where the visual cues were no longer available. We also assessed participants' auditory-only discrimination of the /θ/-/f/ contrast pretraining and posttraining. During training, the visual cues facilitated novel word learning beyond the benefit of the auditory input alone. However, these additional benefits did not persist when participants' discrimination and novel word learning performance were assessed immediately after training. Most interestingly, after a night's sleep, participants who were exposed to orthography during training showed significant improvement in both discrimination and novel word learning compared to the previous day. The findings are discussed in terms of online versus residual impacts of articulatory gestures and orthography on speech processing. While both visual cues are beneficial when they are simultaneously presented with speech, only orthography shows residual impacts leading to a sleep-dependent enhancement of lexical knowledge through memory consolidation and retuning of the second language /θ/-/f/ contrast. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Effective connectivity of the left-ventral occipito-temporal cortex during visual word processing: Direct causal evidence from TMS-EEG co-registration.

As an interface between the visual and language system, the left ventral occipito-temporal cortex (left-vOT) plays a key role in reading. This functional role is supported by anatomical and functional connections between the area and other brain regions within and outside the language network. Nevertheless, only a few studies have investigated how the functional state of this area, which is dependent upon the nature of the task demand and the stimulus being processed, could influence the activity of the connected brain regions. In the present combined TMS-EEG study, we studied the left-vOT effective connectivity by adopting a direct, causal intervention approach. Using TMS, we probed left-vOT activation in different processing contexts and measured the neural propagation of activity from this area to other brain regions. A comparison of neural propagation measured during low-level visual detection of language versus non-language stimuli showed that processing language stimuli reduced neural propagation from the left-vOT to the right occipital cortex. Additionally, compared to the low-level visual detection of language stimuli, performing semantic judgments on the same stimuli further reduced neural propagation to the posterior part of the corpus callosum, right superior parietal lobule and the right anterior temporal lobe. This reduction of cross-hemispheric neural propagation was accompanied by an increase in the collaboration between areas within the left-hemisphere language network. Together, this first evidence from a direct causal intervention approach suggests that processing language stimuli and performing a high-level language task reduce effective connectivity from the left-vOT to the right hemisphere, and may contribute to the left-hemisphere lateralization typically observed during language processing.

The effects of alphabetic literacy, linguistic-processing demand and tone type on the dichotic listening of lexical tones.

Brain lateralization of lexical tone processing remains a matter of debate. In this study we used a dichotic listening paradigm to examine the influences of the knowledge of Jyutping (a romanization writing system which provides explicit Cantonese tone markers), linguistic-processing demand and tone type on the ear preference pattern of native tone processing in Hong Kong Cantonese speakers. While participants with little knowledge of Jyutping showed a previously reported left-ear advantage (LEA), those with a good level of Jyutping expertise exhibited either a right-ear advantage or bilateral processing during lexical tone identification and contour tone discrimination, respectively. As for the effect of linguistic-processing demand, while an LEA was found in acoustic/phonetic perception situations, this advantage disappeared and was replaced by a bilateral pattern in conditions that involved a greater extent of linguistic processing, suggesting an increased involvement of the left hemisphere. Regarding the effect of tone type, both groups showed an LEA in level tone discrimination, but only the Jyutping group demonstrated a bilateral pattern in contour tone discrimination. Overall, knowledge of written codes of tones, greater degree of linguistic processing and contour tone processing seem to influence the brain lateralization of lexical tone processing in native listeners of Cantonese by increasing the recruitment of the left-hemisphere language network.

Graph theoretical analysis reveals the functional role of the left ventral occipito-temporal cortex in speech processing.

The left ventral occipito-temporal cortex (left-vOT) plays a key role in reading. Interestingly, the area also responds to speech input, suggesting that it may have other functions beyond written word recognition. Here, we adopt graph theoretical analysis to investigate the left-vOT's functional role in the whole-brain network while participants process spoken sentences in different contexts. Overall, different connectivity measures indicate that the left-vOT acts as an interface enabling the communication between distributed brain regions and sub-networks. During simple speech perception, the left-vOT is systematically part of the visual network and contributes to the communication between neighboring areas, remote areas, and sub-networks, by acting as a local bridge, a global bridge, and a connector, respectively. However, when speech comprehension is explicitly required, the specific functional role of the area and the sub-network to which the left-vOT belongs change and vary with the quality of speech signal and task difficulty. These connectivity patterns provide insightful information on the contribution of the left-vOT in various contexts of language processing beyond its role in reading. They advance our general understanding of the neural mechanisms underlying the flexibility of the language network that adjusts itself according to the processing context.

How does learning to read affect speech perception?

Behavioral studies have demonstrated that learning to read and write affects the processing of spoken language. The present study investigates the neural mechanism underlying the emergence of such orthographic effects during speech processing. Transcranial magnetic stimulation (TMS) was used to tease apart two competing hypotheses that consider this orthographic influence to be either a consequence of a change in the nature of the phonological representations during literacy acquisition or a consequence of online coactivation of the orthographic and phonological representations during speech processing. Participants performed an auditory lexical decision task in which the orthographic consistency of spoken words was manipulated and repetitive TMS was used to interfere with either phonological or orthographic processing by stimulating left supramarginal gyrus (SMG) or left ventral occipitotemporal cortex (vOTC), respectively. The advantage for consistently spelled words was removed only when the stimulation was delivered to SMG and not to vOTC, providing strong evidence that this effect arises at a phonological, rather than an orthographic, level. We propose a possible mechanistic explanation for the role of SMG in phonological processing and how this is affected by learning to read.

The Effect of Orthographic Transparency on Auditory Word Recognition Across the Development of Reading Proficiency.

A question under debate in psycholinguistics is the nature of the relationship between spoken and written languages. Although it has been extensively shown that orthographic transparency, which varies across writing systems, strongly affects reading performance, its role in speech processing is much less investigated. The present study addressed this issue in Persian, whose writing system provides a possibility to assess the impact of orthographic transparency on spoken word recognition in young children at different stages of reading acquisition. In Persian, the long vowels are systematically present in the script, whereas the spelling correspondence of short vowels is progressively omitted from the script in the course of reading acquisition, thus, turning transparent into opaque spelling. Based on this unique characteristic, we tested 144 monolingual Persian-speaking nonreaders (i.e., preschoolers) and readers (second graders to fifth graders and young adults) in an auditory lexical decision task using transparent and opaque words. Overall, the results showed that, in accordance with the fact that the diacritics of short vowels are progressively omitted during the second year of schooling, the stimuli containing short vowels (opaque words) were recognized more slowly than transparent ones in third graders. Interestingly, there is a hint that the emergence of the transparency effect in the third graders was associated with an overall slower recognition speed in this group compared to their younger peers. These findings indicate that learning opaque spelling-sound correspondence might not only generate interference between the two language codes but also induce a general processing cost in the entire spoken language system.

Early Auditory Event-Related Potentials Are Modulated by Alphabetic Literacy Skills in Logographic Chinese Readers.

The acquisition of an alphabetic orthography transforms speech processing in the human brain. Behavioral evidence shows that phonological awareness as assessed by meta-phonological tasks like phoneme judgment, is enhanced by alphabetic literacy acquisition. The current study investigates the time-course of the neuro-cognitive operations underlying this enhancement as revealed by event-related potentials (ERPs). Chinese readers with and without proficiency in Jyutping, a Romanization system of Cantonese, were recruited for an auditory onset phoneme judgment task; their behavioral responses and the elicited ERPs were examined. Proficient readers of Jyutping achieved higher response accuracy and exhibited more negative-going ERPs in three early ERP time-windows corresponding to the P1, N1, and P2 components. The phonological mismatch negativity component exhibited sensitivity to both onset and rhyme mismatch in the speech stimuli, but it was not modulated by alphabetic literacy skills. The sustained negativity in the P1-N1-P2 time-windows is interpreted as reflecting enhanced phonetic/phonological processing or attentional/awareness modulation associated with alphabetic literacy and phonological awareness skills.

Cerebellar and Cortical Correlates of Internal and External Speech Error Monitoring.

An event-related functional magnetic resonance imaging study examined how speakers inspect their own speech for errors. Concretely, we sought to assess 1) the role of the temporal cortex in monitoring speech errors, linked with comprehension-based monitoring; 2) the involvement of the cerebellum in internal and external monitoring, linked with forward modeling; and 3) the role of the medial frontal cortex for internal monitoring, linked with conflict-based monitoring. In a word production task priming speech errors, we observed enhanced involvement of the right posterior cerebellum for trials that were correct, but on which participants were more likely to make a word as compared with a nonword error (contrast of internal monitoring). Furthermore, comparing errors to correct utterances (contrast of external monitoring), we observed increased activation of the same cerebellar region, of the superior medial cerebellum, and of regions in temporal and medial frontal cortex. The presence of the cerebellum for both internal and external monitoring indicates the use of forward modeling across the planning and articulation of speech. Dissociations across internal and external monitoring in temporal and medial frontal cortex indicate that monitoring of overt errors is more reliant on vocal feedback control.

Investigating occipito-temporal contributions to reading with TMS.

The debate regarding the role of ventral occipito-temporal cortex (vOTC) in visual word recognition arises, in part, from difficulty delineating the functional contributions of vOTC as separate from other areas of the reading network. Here, we investigated the feasibility of using TMS to interfere with vOTC processing in order to explore its specific contributions to visual word recognition. Three visual lexical decision experiments were conducted using neuronavigated TMS. The first demonstrated that repetitive stimulation of vOTC successfully slowed word, but not nonword, responses. The second confirmed and extended these findings by demonstrating the effect was specific to vOTC and not present in the adjacent lateral occipital complex. The final experiment used paired-pulse TMS to investigate the time course of vOTC processing for words and revealed activation starting as early as 80-120 msec poststimulus onset-significantly earlier than that expected based on electrophysiological and magnetoencephalography studies. Taken together, these results clearly indicate that TMS can be successfully used to stimulate parts of vOTC previously believed to be inaccessible and provide a new tool for systematically investigating the information processing characteristics of vOTC. In addition, the findings provide strong evidence that lexical status and frequency significantly affect vOTC processing, findings difficult to reconcile with prelexical accounts of vOTC function.

Auditory word serial recall benefits from orthographic dissimilarity.

The influence of orthographic knowledge has been consistently observed in dissimilarity speech recognition and metaphonological tasks. The present study provides data suggesting that such influence also pervades other cognitive domains phonological related to language abilities, such as verbal working memory. Using serial similarity effect recall of auditory seven-word lists, we observed that inter-item orthographic dissimilarity assists verbal working memory by reducing or even avoiding verbal serial recall the detrimental effect of phonological similarity. However, this orthographic modulation of the phonological similarity effect only occurred at positions four to six of the word list. Performance at position seven benefited from a recency effect that may be assumed to result from a more surface-level (acoustic-phonetic) representation, while better performance at positions one to three is attributable to primacy effects, and can be accounted for in terms of consolidation through recapitulation. The beneficial influence of orthographic knowledge may, therefore, only be apparent when the item stored in short-term memory takes the form of an abstract but unconsolidated phonological representation.

Activation time-course of phonological code in silent word recognition in adult readers with and without dyslexia.

In skilled adult readers, reading words is generally assumed to rapidly and automatically activate the phonological code. In adults with dyslexia, despite the main consensus on their phonological processing deficits, little is known about the activation time course of this code. The present study investigated this issue in both populations. Participants' accuracy and eye movements were recorded while they performed a visual lexical decision task in which phonological consistency of written words was manipulated. Readers with dyslexia were affected by phonological consistency during second fixation duration of visual word recognition suggesting a late activation of the phonological code. Regarding skilled readers, no influence of phonological consistency was found when the participants were considered a homogeneous population. However, a different pattern emerged when they were divided into two subgroups according to their phonological and semantic abilities: Those who showed better decoding than semantic skills were affected by phonological consistency at the earliest stage of visual word recognition while those who showed better semantic than decoding skills were not affected by this factor at any processing stage. Overall, the findings suggest that the presence of phonological deficits in readers with dyslexia is associated with a delayed activation of phonological representations during reading. In skilled readers, the contribution of phonology varies with their reading profile, i.e., being phonologically or semantically oriented.

Consistency and variability in functional localisers.

A critical assumption underlying the use of functional localiser scans is that the voxels identified as the functional region-of-interest (fROI) are essentially the same as those activated by the main experimental manipulation. Intra-subject variability in the location of the fROI violates this assumption, reducing the sensitivity of the analysis and biasing the results. Here we investigated consistency and variability in fROIs in a set of 45 volunteers. They performed two functional localiser scans to identify word- and object-sensitive regions of ventral and lateral occipito-temporal cortex, respectively. In the main analyses, fROIs were defined as the category-selective voxels in each region and consistency was measured as the spatial overlap between scans. Consistency was greatest when minimally selective thresholds were used to define "active" voxels (p<0.05 uncorrected), revealing that approximately 65% of the voxels were commonly activated by both scans. In contrast, highly selective thresholds (p<10(-4) to 10(-6)) yielded the lowest consistency values with less than 25% overlap of the voxels active in both scans. In other words, intra-subject variability was surprisingly high, with between one third and three quarters of the voxels in a given fROI not corresponding to those activated in the main task. This level of variability stands in striking contrast to the consistency seen in retinotopically-defined areas and has important implications for designing robust but efficient functional localiser scans.

Morphological processing without semantics: An ERP study with spoken words.

The time-course of morphological processing during spoken word recognition was investigated using event-related brain potentials (ERPs) in an auditory lexical decision task. We compared three different types of French words: truly suffixed (e.g., pochette 'little pocket' = poche 'pocket' + diminutive suffix -ette), pseudo-suffixed (e.g., mouette 'seagull' = mou 'soft' + pseudo-suffix -ette) and non-suffixed target words (e.g., fortune 'fortune' = fort 'strong' + non-suffix -une). Suffixed (e.g., mouesse = mou + suffix -esse) and non-suffixed nonwords (e.g., mouipe = mou + non-suffix -ipe) were also tested. The behavioural results showed that participants responded more slowly to non-suffixed words than to truly suffixed and pseudo-suffixed words, but there was no difference between the two suffixed conditions. Moreover, participants made more errors rejecting pseudo-suffixed nonwords than non-suffixed nonwords. In the ERP analyses, T0 was shifted to the end of the embedded stem or pseudo-stem. The ERP results revealed enhanced N400 amplitudes for non-suffixed words compared to truly suffixed and pseudo-suffixed words. Again, there was no difference between the truly and pseudo-suffixed conditions. In addition, we found an increased N400 amplitude for both pseudo-suffixed and non-suffixed nonwords than for words. The latency of the onset of this N400 effect varied between the three experimental conditions: the word-nonword difference occurred earliest in the truly suffixed condition, slightly later in the pseudo-suffixed condition and latest in the non-suffixed condition. Both behavioural and EEG data jointly suggest that spoken words with a genuine morphological structure and words with a pseudo-morphological structure are decomposed into morphemic sub-units. Moreover, the earlier appearance of the N400 effects in the truly suffixed condition indicates that morphological information is more readily available in words with a semantically transparent morphological structure.