A theory of memory for binary sequences: Evidence for a mental compression algorithm in humans.

Working memory capacity can be improved by recoding the memorized information in a condensed form. Here, we tested the theory that human adults encode binary sequences of stimuli in memory using an abstract internal language and a recursive compression algorithm. The theory predicts that the psychological complexity of a given sequence should be proportional to the length of its shortest description in the proposed language, which can capture any nested pattern of repetitions and alternations using a limited number of instructions. Five experiments examine the capacity of the theory to predict human adults' memory for a variety of auditory and visual sequences. We probed memory using a sequence violation paradigm in which participants attempted to detect occasional violations in an otherwise fixed sequence. Both subjective complexity ratings and objective violation detection performance were well predicted by our theoretical measure of complexity, which simply reflects a weighted sum of the number of elementary instructions and digits in the shortest formula that captures the sequence in our language. While a simpler transition probability model, when tested as a single predictor in the statistical analyses, accounted for significant variance in the data, the goodness-of-fit with the data significantly improved when the language-based complexity measure was included in the statistical model, while the variance explained by the transition probability model largely decreased. Model comparison also showed that shortest description length in a recursive language provides a better fit than six alternative previously proposed models of sequence encoding. The data support the hypothesis that, beyond the extraction of statistical knowledge, human sequence coding relies on an internal compression using language-like nested structures.

Spoken language coding neurons in the Visual Word Form Area: Evidence from a TMS adaptation paradigm.

While part of the left ventral occipito-temporal cortex (left-vOT), known as the Visual Word Form Area, plays a central role in reading, the area also responds to speech. This cross-modal activation has been explained by three competing hypotheses. Firstly, speech is converted to orthographic representations that activate, in a top-down manner, written language coding neurons in the left-vOT. Secondly, the area contains multimodal neurons that respond to both language modalities. Thirdly, the area comprises functionally segregated neuronal populations that selectively encode different language modalities. A transcranial magnetic stimulation (TMS)-adaptation protocol was used to disentangle these hypotheses. During adaptation, participants were exposed to spoken or written words in order to tune the initial state of left-vOT neurons to one of the language modalities. After adaptation, they performed lexical decisions on spoken and written targets with TMS applied to the left-vOT. TMS showed selective facilitatory effects. It accelerated lexical decisions only when the adaptors and the targets shared the same modality, i.e., when left-vOT neurons had initially been adapted to the modality of the target stimuli. Since this within-modal adaptation was observed for both input modalities and no evidence for cross-modal adaptation was found, our findings suggest that the left-vOT contains neurons that selectively encode written and spoken language rather than purely written language coding neurons or multimodal neurons encoding language regardless of modality.

Top-down activation of the visuo-orthographic system during spoken sentence processing.

The left ventral occipitotemporal cortex (vOT) is considered the key area of the visuo-orthographic system. However, some studies reported that the area is also involved in speech processing tasks, especially those that require activation of orthographic knowledge. These findings suggest the existence of a top-down activation mechanism allowing such cross-modal activation. Yet, little is known about the involvement of the vOT in more natural speech processing situations like spoken sentence processing. Here, we addressed this issue in a functional Magnetic Resonance Imaging (fMRI) study while manipulating the impacts of two factors, i.e., task demands (semantic vs. low-level perceptual task) and the quality of speech signals (sentences presented against clear vs. noisy background). Analyses were performed at the levels of whole brain and region-of-interest (ROI) focusing on the vOT voxels individually identified through a reading task. Whole brain analysis showed that processing spoken sentences induced activity in a large network including the regions typically involved in phonological, articulatory, semantic and orthographic processing. ROI analysis further specified that a significant part of the vOT voxels that responded to written words also responded to spoken sentences, thus, suggesting that the same area within the left occipitotemporal pathway contributes to both reading and speech processing. Interestingly, both analyses provided converging evidence that vOT responses to speech were sensitive to both task demands and quality of speech signals: Compared to the low-level perceptual task, activity of the area increased when efforts on comprehension were required. The impact of background noise depended on task demands. It led to a decrease of vOT activity in the semantic task but not in the low-level perceptual task. Our results provide new insights into the function of this key area of the reading network, notably by showing that its speech-induced top-down activation also generalizes to ecological speech processing situations.

Contribution of writing to reading: Dissociation between cognitive and motor process in the left dorsal premotor cortex.

Functional brain imaging studies reported activation of the left dorsal premotor cortex (PMd), that is, a main area in the writing network, in reading tasks. However, it remains unclear whether this area is causally relevant for written stimulus recognition or its activation simply results from a passive coactivation of reading and writing networks. Here, we used chronometric paired-pulse transcranial magnetic stimulation (TMS) to address this issue by disrupting the activity of the PMd, the so-called Exner's area, while participants performed a lexical decision task. Both words and pseudowords were presented in printed and handwritten characters. The latter was assumed to be closely associated with motor representations of handwriting gestures. We found that TMS over the PMd in relatively early time-windows, i.e., between 60 and 160 ms after the stimulus onset, increased reaction times to pseudoword without affecting word recognition. Interestingly, this result pattern was found for both printed and handwritten characters, that is, regardless of whether the characters evoked motor representations of writing actions. Our result showed that under some circumstances the activation of the PMd does not simply result from passive association between reading and writing networks but has a functional role in the reading process. At least, at an early stage of written stimuli recognition, this role seems to depend on a common sublexical and serial process underlying writing and pseudoword reading rather than on an implicit evocation of writing actions during reading as typically assumed.

Brain-imaging evidence for compression of binary sound sequences in human memory.

According to the language-of-thought hypothesis, regular sequences are compressed in human memory using recursive loops akin to a mental program that predicts future items. We tested this theory by probing memory for 16-item sequences made of two sounds. We recorded brain activity with functional MRI and magneto-encephalography (MEG) while participants listened to a hierarchy of sequences of variable complexity, whose minimal description required transition probabilities, chunking, or nested structures. Occasional deviant sounds probed the participants' knowledge of the sequence. We predicted that task difficulty and brain activity would be proportional to the complexity derived from the minimal description length in our formal language. Furthermore, activity should increase with complexity for learned sequences, and decrease with complexity for deviants. These predictions were upheld in both fMRI and MEG, indicating that sequence predictions are highly dependent on sequence structure and become weaker and delayed as complexity increases. The proposed language recruited bilateral superior temporal, precentral, anterior intraparietal, and cerebellar cortices. These regions overlapped extensively with a localizer for mathematical calculation, and much less with spoken or written language processing. We propose that these areas collectively encode regular sequences as repetitions with variations and their recursive composition into nested structures.

Effective connectivity of the left-ventral occipito-temporal cortex during visual word processing: Direct causal evidence from TMS-EEG co-registration.

As an interface between the visual and language system, the left ventral occipito-temporal cortex (left-vOT) plays a key role in reading. This functional role is supported by anatomical and functional connections between the area and other brain regions within and outside the language network. Nevertheless, only a few studies have investigated how the functional state of this area, which is dependent upon the nature of the task demand and the stimulus being processed, could influence the activity of the connected brain regions. In the present combined TMS-EEG study, we studied the left-vOT effective connectivity by adopting a direct, causal intervention approach. Using TMS, we probed left-vOT activation in different processing contexts and measured the neural propagation of activity from this area to other brain regions. A comparison of neural propagation measured during low-level visual detection of language versus non-language stimuli showed that processing language stimuli reduced neural propagation from the left-vOT to the right occipital cortex. Additionally, compared to the low-level visual detection of language stimuli, performing semantic judgments on the same stimuli further reduced neural propagation to the posterior part of the corpus callosum, right superior parietal lobule and the right anterior temporal lobe. This reduction of cross-hemispheric neural propagation was accompanied by an increase in the collaboration between areas within the left-hemisphere language network. Together, this first evidence from a direct causal intervention approach suggests that processing language stimuli and performing a high-level language task reduce effective connectivity from the left-vOT to the right hemisphere, and may contribute to the left-hemisphere lateralization typically observed during language processing.

Symbols and mental programs: a hypothesis about human singularity.

Natural language is often seen as the single factor that explains the cognitive singularity of the human species. Instead, we propose that humans possess multiple internal languages of thought, akin to computer languages, which encode and compress structures in various domains (mathematics, music, shape…). These languages rely on cortical circuits distinct from classical language areas. Each is characterized by: (i) the discretization of a domain using a small set of symbols, and (ii) their recursive composition into mental programs that encode nested repetitions with variations. In various tasks of elementary shape or sequence perception, minimum description length in the proposed languages captures human behavior and brain activity, whereas non-human primate data are captured by simpler nonsymbolic models. Our research argues in favor of discrete symbolic models of human thought.

Graph theoretical analysis reveals the functional role of the left ventral occipito-temporal cortex in speech processing.

The left ventral occipito-temporal cortex (left-vOT) plays a key role in reading. Interestingly, the area also responds to speech input, suggesting that it may have other functions beyond written word recognition. Here, we adopt graph theoretical analysis to investigate the left-vOT's functional role in the whole-brain network while participants process spoken sentences in different contexts. Overall, different connectivity measures indicate that the left-vOT acts as an interface enabling the communication between distributed brain regions and sub-networks. During simple speech perception, the left-vOT is systematically part of the visual network and contributes to the communication between neighboring areas, remote areas, and sub-networks, by acting as a local bridge, a global bridge, and a connector, respectively. However, when speech comprehension is explicitly required, the specific functional role of the area and the sub-network to which the left-vOT belongs change and vary with the quality of speech signal and task difficulty. These connectivity patterns provide insightful information on the contribution of the left-vOT in various contexts of language processing beyond its role in reading. They advance our general understanding of the neural mechanisms underlying the flexibility of the language network that adjusts itself according to the processing context.

Cerebral representation of sequence patterns across multiple presentation formats.

The ability to detect the abstract pattern underlying a temporal sequence of events is crucial to many human activities, including language and mathematics, but its cortical correlates remain poorly understood. It is also unclear whether repeated exposure to the same sequence of sensory stimuli is sufficient to induce the encoding of an abstract amodal representation of the pattern. Using functional MRI, we probed the existence of such abstract codes for sequential patterns, their localization in the human brain, and their relation to existing language and math-responsive networks. We used a passive sequence violation paradigm, in which a given sequence is repeatedly presented before rare deviant sequences are introduced. We presented two binary patterns, AABB and ABAB, in four presentation formats, either visual or auditory, and either cued by the identity of the stimuli or by their spatial location. Regardless of the presentation format, a habituation to the repeated pattern and a response to pattern violations were seen in a set of inferior frontal, intraparietal and temporal areas. Within language areas, such pattern-violation responses were only found in the inferior frontal gyrus (IFG), whereas all math-responsive regions responded to pattern changes. Most of these regions also responded whenever the modality or the cue changed, suggesting a general sensitivity to violation detection. Thus, the representation of sequence patterns appears to be distributed, yet to include a core set of abstract amodal regions, particularly the IFG.

How specialized are writing-specific brain regions? An fMRI study of writing, drawing and oral spelling.

Several brain imaging studies identified brain regions that are consistently involved in writing tasks; the left premotor and superior parietal cortices have been associated with the peripheral components of writing performance as opposed to other regions that support the central, orthographic components. Based on a meta-analysis by Planton, Jucla, Roux, and Demonet (2013), we focused on five such writing areas and questioned the task-specificity and hemispheric lateralization profile of the brain response in an functional magnetic resonance imaging (fMRI) experiment where 16 right-handed participants wrote down, spelled out orally object names, and drew shapes from object pictures. All writing-related areas were activated by drawing, and some of them by oral spelling, thus questioning their specialization for written production. The graphemic/motor frontal area (GMFA), a subpart of the superior premotor cortex close to Exner's area (Roux et al., 2009), was the only area with a writing-specific lateralization profile, that is, clear left lateralization during handwriting, and bilateral activity during drawing. Furthermore, the relative lateralization and levels of activation in the superior parietal cortex, ventral premotor cortex, ventral occipitotemporal cortex and right cerebellum across the three tasks brought out new evidence regarding their respective contributions to the writing processes.

The neural basis for writing from dictation in the temporoparietal cortex.

Cortical electrical stimulation mapping was used to study neural substrates of the function of writing in the temporoparietal cortex. We identified the sites involved in oral language (sentence reading and naming) and writing from dictation, in order to spare these areas during removal of brain tumours in 30 patients (23 in the left, and 7 in the right hemisphere). Electrostimulation of the cortex impaired writing ability in 62 restricted cortical areas (.25 cm2). These were found in left temporoparietal lobes and were mostly located along the superior temporal gyrus (Brodmann's areas 22 and 42). Stimulation of right temporoparietal lobes in right-handed patients produced no writing impairments. However there was a high variability of location between individuals. Stimulation resulted in combined symptoms (affecting oral language and writing) in fourteen patients, whereas in eight other patients, stimulation-induced pure agraphia symptoms with no oral language disturbance in twelve of the identified areas. Each detected area affected writing in a different way. We detected the various different stages of the auditory-to-motor pathway of writing from dictation: either through comprehension of the dictated sentences (word deafness areas), lexico-semantic retrieval, or phonologic processing. In group analysis, barycentres of all different types of writing interferences reveal a hierarchical functional organization along the superior temporal gyrus from initial word recognition to lexico-semantic and phonologic processes along the ventral and the dorsal comprehension pathways, supporting the previously described auditory-to-motor process. The left posterior Sylvian region supports different aspects of writing function that are extremely specialized and localized, sometimes being segregated in a way that could account for the occurrence of pure agraphia that has long-been described in cases of damage to this region.

The "handwriting brain": a meta-analysis of neuroimaging studies of motor versus orthographic processes.

INTRODUCTION: Handwriting is a modality of language production whose cerebral substrates remain poorly known although the existence of specific regions is postulated. The description of brain damaged patients with agraphia and, more recently, several neuroimaging studies suggest the involvement of different brain regions. However, results vary with the methodological choices made and may not always discriminate between "writing-specific" and motor or linguistic processes shared with other abilities. METHODS: We used the "Activation Likelihood Estimate" (ALE) meta-analytical method to identify the cerebral network of areas commonly activated during handwriting in 18 neuroimaging studies published in the literature. Included contrasts were also classified according to the control tasks used, whether non-specific motor/output-control or linguistic/input-control. These data were included in two secondary meta-analyses in order to reveal the functional role of the different areas of this network. RESULTS: An extensive, mainly left-hemisphere network of 12 cortical and sub-cortical areas was obtained; three of which were considered as primarily writing-specific (left superior frontal sulcus/middle frontal gyrus area, left intraparietal sulcus/superior parietal area, right cerebellum) while others related rather to non-specific motor (primary motor and sensorimotor cortex, supplementary motor area, thalamus and putamen) or linguistic processes (ventral premotor cortex, posterior/inferior temporal cortex). CONCLUSIONS: This meta-analysis provides a description of the cerebral network of handwriting as revealed by various types of neuroimaging experiments and confirms the crucial involvement of the left frontal and superior parietal regions. These findings provide new insights into cognitive processes involved in handwriting and their cerebral substrates.