Emergence of a compositional neural code for written words: Recycling of a convolutional neural network for reading.

The visual word form area (VWFA) is a region of human inferotemporal cortex that emerges at a fixed location in the occipitotemporal cortex during reading acquisition and systematically responds to written words in literate individuals. According to the neuronal recycling hypothesis, this region arises through the repurposing, for letter recognition, of a subpart of the ventral visual pathway initially involved in face and object recognition. Furthermore, according to the biased connectivity hypothesis, its reproducible localization is due to preexisting connections from this subregion to areas involved in spoken-language processing. Here, we evaluate those hypotheses in an explicit computational model. We trained a deep convolutional neural network of the ventral visual pathway, first to categorize pictures and then to recognize written words invariantly for case, font, and size. We show that the model can account for many properties of the VWFA, particularly when a subset of units possesses a biased connectivity to word output units. The network develops a sparse, invariant representation of written words, based on a restricted set of reading-selective units. Their activation mimics several properties of the VWFA, and their lesioning causes a reading-specific deficit. The model predicts that, in literate brains, written words are encoded by a compositional neural code with neurons tuned either to individual letters and their ordinal position relative to word start or word ending or to pairs of letters (bigrams).

Spatially invariant coding of numerical information in functionally defined subregions of human parietal cortex.

Macaque electrophysiology has revealed neurons responsive to number in lateral (LIP) and ventral (VIP) intraparietal areas. Recently, fMRI pattern recognition revealed information discriminative of individual numbers in human parietal cortex but without precisely localizing the relevant sites or testing for subregions with different response profiles. Here, we defined the human functional equivalents of LIP (feLIP) and VIP (feVIP) using neurophysiologically motivated localizers. We applied multivariate pattern recognition to investigate whether both regions represent numerical information and whether number codes are position specific or invariant. In a delayed number comparison paradigm with laterally presented numerosities, parietal cortex discriminated between numerosities better than early visual cortex, and discrimination generalized across hemifields in parietal, but not early visual cortex. Activation patterns in the 2 parietal regions of interest did not differ in the coding of position-specific or position-independent number information, but in the expression of a numerical distance effect which was more pronounced in feLIP. Thus, the representation of number in parietal cortex is at least partially position invariant. Both feLIP and feVIP contain information about individual numerosities in humans, but feLIP hosts a coarser representation of numerosity than feVIP, compatible with either broader tuning or a summation code.

Interoperable atlases of the human brain.

The last two decades have seen an unprecedented development of human brain mapping approaches at various spatial and temporal scales. Together, these have provided a large fundus of information on many different aspects of the human brain including micro- and macrostructural segregation, regional specialization of function, connectivity, and temporal dynamics. Atlases are central in order to integrate such diverse information in a topographically meaningful way. It is noteworthy, that the brain mapping field has been developed along several major lines such as structure vs. function, postmortem vs. in vivo, individual features of the brain vs. population-based aspects, or slow vs. fast dynamics. In order to understand human brain organization, however, it seems inevitable that these different lines are integrated and combined into a multimodal human brain model. To this aim, we held a workshop to determine the constraints of a multi-modal human brain model that are needed to enable (i) an integration of different spatial and temporal scales and data modalities into a common reference system, and (ii) efficient data exchange and analysis. As detailed in this report, to arrive at fully interoperable atlases of the human brain will still require much work at the frontiers of data acquisition, analysis, and representation. Among them, the latter may provide the most challenging task, in particular when it comes to representing features of vastly different scales of space, time and abstraction. The potential benefits of such endeavor, however, clearly outweigh the problems, as only such kind of multi-modal human brain atlas may provide a starting point from which the complex relationships between structure, function, and connectivity may be explored.

Single-trial decoding of auditory novelty responses facilitates the detection of residual consciousness.

Detecting residual consciousness in unresponsive patients is a major clinical concern and a challenge for theoretical neuroscience. To tackle this issue, we recently designed a paradigm that dissociates two electro-encephalographic (EEG) responses to auditory novelty. Whereas a local change in pitch automatically elicits a mismatch negativity (MMN), a change in global sound sequence leads to a late P300b response. The latter component is thought to be present only when subjects consciously perceive the global novelty. Unfortunately, it can be difficult to detect because individual variability is high, especially in clinical recordings. Here, we show that multivariate pattern classifiers can extract subject-specific EEG patterns and predict single-trial local or global novelty responses. We first validate our method with 38 high-density EEG, MEG and intracranial EEG recordings. We empirically demonstrate that our approach circumvents the issues associated with multiple comparisons and individual variability while improving the statistics. Moreover, we confirm in control subjects that local responses are robust to distraction whereas global responses depend on attention. We then investigate 104 vegetative state (VS), minimally conscious state (MCS) and conscious state (CS) patients recorded with high-density EEG. For the local response, the proportion of significant decoding scores (M=60%) does not vary with the state of consciousness. By contrast, for the global response, only 14% of the VS patients' EEG recordings presented a significant effect, compared to 31% in MCS patients' and 52% in CS patients'. In conclusion, single-trial multivariate decoding of novelty responses provides valuable information in non-communicating patients and paves the way towards real-time monitoring of the state of consciousness.

Probing the cortical network underlying the psychological refractory period: a combined EEG-fMRI study.

Human performance exhibits strong multi-tasking limitations in simple response time tasks. In the psychological refractory period (PRP) paradigm, where two tasks have to be performed in brief succession, central processing of the second task is delayed when the two tasks are performed at short time intervals. Here, we aimed to probe the cortical network underlying this postponement of central processing by simultaneously recording electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data while 12 subjects performed two simple number-comparison tasks. Behavioral data showed a significant slowing of response times to the second target stimulus at short stimulus-onset asynchronies, together with significant correlations between response times to the first and second target stimulus, i.e., the hallmarks of the PRP effect. The analysis of EEG data showed a significant delay of the post-perceptual P3 component evoked by the second target, which was of similar magnitude as the effect on response times. fMRI data revealed an involvement of parietal and prefrontal regions in dual-task processing. The combined analysis of fMRI and EEG data-based on the trial-by-trial variability of the P3-revealed that BOLD signals in two bilateral regions in the inferior parietal lobe and precentral gyrus significantly covaried with P3 related activity. Our results show that combining neuroimaging methods of high spatial and temporal resolutions can help to identify cortical regions underlying the central bottleneck of information processing, and strengthen the conclusion that fronto-parietal cortical regions participate in a distributed "global neuronal workspace" system that underlies the generation of the P3 component and may be one of the key cerebral underpinnings of the PRP bottleneck.

Causal role of prefrontal cortex in the threshold for access to consciousness.

What neural mechanisms support our conscious perception of briefly presented stimuli? Some theories of conscious access postulate a key role of top-down amplification loops involving prefrontal cortex (PFC). To test this issue, we measured the visual backward masking threshold in patients with focal prefrontal lesions, using both objective and subjective measures while controlling for putative attention deficits. In all conditions of temporal or spatial attention cueing, the threshold for access to consciousness was systematically shifted in patients, particular after a lesion of the left anterior PFC. The deficit affected subjective reports more than objective performance, and objective performance conditioned on subjective visibility was essentially normal. We conclude that PFC makes a causal contribution to conscious visual perception of masked stimuli, and outline a dual-route signal detection theory of objective and subjective decision making.

Sources of mathematical thinking: behavioral and brain-imaging evidence.

Does the human capacity for mathematical intuition depend on linguistic competence or on visuo-spatial representations? A series of behavioral and brain-imaging experiments provides evidence for both sources. Exact arithmetic is acquired in a language-specific format, transfers poorly to a different language or to novel facts, and recruits networks involved in word-association processes. In contrast, approximate arithmetic shows language independence, relies on a sense of numerical magnitudes, and recruits bilateral areas of the parietal lobes involved in visuo-spatial processing. Mathematical intuition may emerge from the interplay of these brain systems.

Human consciousness is supported by dynamic complex patterns of brain signal coordination.

Adopting the framework of brain dynamics as a cornerstone of human consciousness, we determined whether dynamic signal coordination provides specific and generalizable patterns pertaining to conscious and unconscious states after brain damage. A dynamic pattern of coordinated and anticoordinated functional magnetic resonance imaging signals characterized healthy individuals and minimally conscious patients. The brains of unresponsive patients showed primarily a pattern of low interareal phase coherence mainly mediated by structural connectivity, and had smaller chances to transition between patterns. The complex pattern was further corroborated in patients with covert cognition, who could perform neuroimaging mental imagery tasks, validating this pattern's implication in consciousness. Anesthesia increased the probability of the less complex pattern to equal levels, validating its implication in unconsciousness. Our results establish that consciousness rests on the brain's ability to sustain rich brain dynamics and pave the way for determining specific and generalizable fingerprints of conscious and unconscious states.

Cerebral mechanisms of word masking and unconscious repetition priming.

We used functional magnetic resonance imaging (fMRI) and event-related potentials (ERPs) to visualize the cerebral processing of unseen masked words. Within the areas associated with conscious reading, masked words activated left extrastriate, fusiform and precentral areas. Furthermore, masked words reduced the amount of activation evoked by a subsequent conscious presentation of the same word. In the left fusiform gyrus, this repetition suppression phenomenon was independent of whether the prime and target shared the same case, indicating that case-independent information about letter strings was extracted unconsciously. In comparison to an unmasked situation, however, the activation evoked by masked words was drastically reduced and was undetectable in prefrontal and parietal areas, correlating with participants' inability to report the masked words.

Impaired conscious access and abnormal attentional amplification in schizophrenia.

Previous research suggests that the conscious perception of a masked stimulus is impaired in schizophrenia, while unconscious bottom-up processing of the same stimulus, as assessed by subliminal priming, can be preserved. Here, we test this postulated dissociation between intact bottom-up and impaired top-down processing and evaluate its brain mechanisms using high-density recordings of event-related potentials. Sixteen patients with schizophrenia and sixteen controls were exposed to peripheral digits with various degrees of visibility, under conditions of either focused attention or distraction by another task. In the distraction condition, the brain activity evoked by masked digits was drastically reduced in both groups, but early bottom-up visual activation could still be detected and did not differ between patients and controls. By contrast, under focused top-down attention, a major impairment was observed: in patients, contrary to controls, the late non-linear ignition associated with the P3 component was reduced. Interestingly, the patients showed an essentially normal attentional amplification of the P1 and N2 components. These results suggest that some but not all top-down attentional amplification processes are impaired in schizophrenia, while bottom-up processing seems to be preserved.

A lateral-to-mesial organization of human ventral visual cortex at birth.

In human adults, ventral extra-striate visual cortex contains a mosaic of functionally specialized areas, some responding preferentially to natural visual categories such as faces (fusiform face area) or places (parahippocampal place area) and others to cultural inventions such as written words and numbers (visual word form and number form areas). It has been hypothesized that this mosaic arises from innate biases in cortico-cortical connectivity. We tested this hypothesis by examining functional resting-state correlation at birth using fMRI data from full-term human newborns. The results revealed that ventral visual regions are functionally connected with their contra-lateral homologous regions and also exhibit distinct patterns of long-distance functional correlation with anterior associative regions. A mesial-to-lateral organization was observed, with the signal of the more lateral regions, including the sites of visual word and number form areas, exhibiting higher correlations with voxels of the prefrontal, inferior parietal and temporal cortices, including language areas. Finally, we observed hemispheric asymmetries in the functional correlation of key areas of the language network that may influence later adult hemispheric lateralization. We suggest that long-distance circuits present at birth constrain the subsequent functional differentiation of the ventral visual cortex.

A random-matrix theory of the number sense.

Number sense, a spontaneous ability to process approximate numbers, has been documented in human adults, infants and newborns, and many other animals. Species as distant as monkeys and crows exhibit very similar neurons tuned to specific numerosities. How number sense can emerge in the absence of learning or fine tuning is currently unknown. We introduce a random-matrix theory of self-organized neural states where numbers are coded by vectors of activation across multiple units, and where the vector codes for successive integers are obtained through multiplication by a fixed but random matrix. This cortical implementation of the 'von Mises' algorithm explains many otherwise disconnected observations ranging from neural tuning curves in monkeys to looking times in neonates and cortical numerotopy in adults. The theory clarifies the origin of Weber-Fechner's Law and yields a novel and empirically validated prediction of multi-peak number neurons. Random matrices constitute a novel mechanism for the emergence of brain states coding for quantity.This article is part of a discussion meeting issue 'The origins of numerical abilities'.

Attentional networks.

Recent brain-imaging and neurophysiological data indicate that attention is neither a property of a single brain area, nor of the entire brain. While attentional effects seem mediated by a relative amplification of blood flow and electrical activity in the cortical areas processing the attended computation, the details of how this is done through enhancement of attended or suppression of unattended items, or both, appear to depend on the task and brain-area studied. The origins of these amplification effects are to be found in specialized cortical areas of the frontal and parietal lobes that have been described as the anterior and posterior attention systems. These results represent substantial progress in the effort to determine how brain activity is regulated through attention. While many philosophical and practical issues remain in developing an understanding of attentional regulation, the new tools available should provide the basis for progress.

Abstract representations of numbers in the animal and human brain.

There is evidence to suggest that animals, young infants and adult humans possess a biologically determined, domain-specific representation of number and of elementary arithmetic operations. Behavioral studies in infants and animals reveal number perception, discrimination and elementary calculation abilities in non-verbal organisms. Lesion and brain-imaging studies in humans indicate that a specific neural substrate, located in the left and right intraparietal area, is associated with knowledge of numbers and their relations ('number sense'). The number domain is a prime example where strong evidence points to an evolutionary endowment of abstract domain-specific knowledge in the brain because there are parallels between number processing in animals and humans.The numerical distance effect, which refers to the finding that the ability to discriminate between two numbers improves as the numerical distance between them increases, has been demonstrated in humans and animals, as has the number size effect,which refers to the finding that for equal numerical distance,discrimination of two numbers worsens as their numerical size increases.

Computational models of association cortex.

Recent computational models, or mathematical realizations of neurobiological theories, are providing insights into the organization and workings of the association cortex. Such models concern the construction of cortical maps, the neural basis of cognitive functions such as visual perception, reward-motivated learning and some aspects of consciousness.

Linking morphological and functional variability in hand movement and silent reading.

It is generally accepted in neuroscience that anatomy and function go hand in hand. Accordingly, a local morphological variability could lead to a corresponding functional variability. In this study, we tested this hypothesis by linking the variability of the cortical folding pattern of 252 right-handed subjects to the localization or the pattern of functional activations induced by hand motion or silent reading. Three regions are selected: the central sulcus, the precentral sulcus and the superior temporal sulcus (STS). "Essential morphological variability traits" are identified using a method building upon multidimensional scaling. The link between variability in anatomy and function is confirmed by the perfect match between the central sulcus morphological "hand knob" and the corresponding motor activation: as the location of the hand knob moves more or less dorsally along the central sulcus, the motor hand activation moves accordingly. Furthermore, the size of the left hand activation in the right hemisphere is correlated with the knob location in the central sulcus. A new link between functional and morphological variability is discovered relative to the location of a premotor activation induced by silent reading. While this reading activation is located next to the wall of the central sulcus when the hand knob has a ventral positioning, it is pushed into a deep gyrus interrupting the precentral sulcus when the knob is more dorsal. Finally, it is shown that the size of the reading activation along the STS is larger when the posterior branches are less developed.

Two mental calculation systems: a case study of severe acalculia with preserved approximation.

We report the case of an aphasic and acalculic patient with selective preservation of approximation abilities. The patient's deficit was so severe that he judged 2 + 2 = 5 to be correct, illustrating a radical impairment in exact calculation. However, he easily rejected grossly false additions such as 2 + 2 = 9, therefore demonstrating a preserved knowledge of the approximate result. The dissociation between impaired exact processing and preserved approximation was identified in several numerical tasks: solving and verifying arithmetical operations, number reading, short-term memory, number comparison, parity judgement, and number knowledge. We suggest the existence of two distinct number-processing routes in the normal subject. One route permits exact number representation, memory and calculation using symbolic notation. The other route allows for approximate computations using an analog representation of quantities.

Impairments of number processing induced by prenatal alcohol exposure.

Prenatal alcohol exposure causes a variety of cognitive deficits, notably in mathematics and higher order processes such as abstraction. An exploratory battery was developed to examine specific types of number processing impairments in 29 adolescent and adult patients with Fetal Alcohol Syndrome (FAS) and Fetal Alcohol Effects (FAE) relative to controls matched for age, gender, and educational level. The battery included 11 tests: number reading and writing, exact calculation (addition, multiplication, subtraction), approximate calculation (selecting a plausible result for an operation), number comparison, proximity judgment, and cognitive estimation. The results indicated particular difficulties in calculation and estimation tests, with intact number reading and writing ability. The greatest impairment was found in the cognitive estimation test, which is sensitive to frontal lobe lesions. The patterns of deficit described may reflect either the diffuseness of brain damage incurred from prenatal alcohol exposure, or a cumulative deficit in comprehension which may be important for the acquisition of higher-order mathematical abilities.

Language and calculation within the parietal lobe: a combined cognitive, anatomical and fMRI study.

We report the case of a patient (ATH) who suffered from aphasia, deep dyslexia, and acalculia, following a lesion in her left perisylvian area. She showed a severe impairment in all tasks involving numbers in a verbal format, such as reading aloud, writing to dictation, or responding verbally to questions of numerical knowledge. In contrast, her ability to manipulate non-verbal representations of numbers, i.e., Arabic numerals and quantities, was comparatively well preserved, as evidenced for instance in number comparison or number bisection tasks. This dissociated impairment of verbal and non-verbal numerical abilities entailed a differential impairment of the four arithmetic operations. ATH performed much better with subtraction and addition, that can be solved on the basis of quantity manipulation, than with multiplication and division problems, that are commonly solved by retrieving stored verbal sequences. The brain lesion affected the classical language areas, but spared a subset of the left inferior parietal lobule that was active during calculation tasks, as demonstrated with functional MRI. Finally, the relative preservation of subtraction versus multiplication may be related to the fact that subtraction activated the intact right parietal lobe, while multiplication activated predominantly left-sided areas.

Cerebral activations during number multiplication and comparison: a PET study.

Positron emission tomography was used to examine the cerebral networks underlying number comparison and multiplication in eight normal volunteers. Cerebral blood flow was measured within anatomical regions of interest defined in each subject using magnetic resonance imaging. Three conditions were used: rest with eyes closed, mental multiplication of pairs of arabic digits and larger-smaller comparison of the same pairs. Both multiplication and comparison activated the left and right lateral occipital cortices, the left precentral gyrus, and the supplementary motor area. Beyond these common activations, multiplication activated also the left and right inferior parietal gyri, the left fusiform and lingual gyri, and the right cuneus. Relative to comparison, multiplication also yielded superior activity in the left lenticular nucleus and in Brodmann's area 8, and induced a hemispheric asymmetry in the activation of the precentral and inferior frontal gyri. Conversely, relative to multiplication, comparison yielded superior activity in the right superior temporal gyrus, the left and right middle temporal gyri, the right superior frontal gyrus, and the right inferior frontal gyrus. These results underline the role of bilateral inferior parietal regions in number processing and suggest that multiplication and comparison may rest on partially distinct networks.