Emergence of a compositional neural code for written words: Recycling of a convolutional neural network for reading.

The visual word form area (VWFA) is a region of human inferotemporal cortex that emerges at a fixed location in the occipitotemporal cortex during reading acquisition and systematically responds to written words in literate individuals. According to the neuronal recycling hypothesis, this region arises through the repurposing, for letter recognition, of a subpart of the ventral visual pathway initially involved in face and object recognition. Furthermore, according to the biased connectivity hypothesis, its reproducible localization is due to preexisting connections from this subregion to areas involved in spoken-language processing. Here, we evaluate those hypotheses in an explicit computational model. We trained a deep convolutional neural network of the ventral visual pathway, first to categorize pictures and then to recognize written words invariantly for case, font, and size. We show that the model can account for many properties of the VWFA, particularly when a subset of units possesses a biased connectivity to word output units. The network develops a sparse, invariant representation of written words, based on a restricted set of reading-selective units. Their activation mimics several properties of the VWFA, and their lesioning causes a reading-specific deficit. The model predicts that, in literate brains, written words are encoded by a compositional neural code with neurons tuned either to individual letters and their ordinal position relative to word start or word ending or to pairs of letters (bigrams).

Human consciousness is supported by dynamic complex patterns of brain signal coordination.

Adopting the framework of brain dynamics as a cornerstone of human consciousness, we determined whether dynamic signal coordination provides specific and generalizable patterns pertaining to conscious and unconscious states after brain damage. A dynamic pattern of coordinated and anticoordinated functional magnetic resonance imaging signals characterized healthy individuals and minimally conscious patients. The brains of unresponsive patients showed primarily a pattern of low interareal phase coherence mainly mediated by structural connectivity, and had smaller chances to transition between patterns. The complex pattern was further corroborated in patients with covert cognition, who could perform neuroimaging mental imagery tasks, validating this pattern's implication in consciousness. Anesthesia increased the probability of the less complex pattern to equal levels, validating its implication in unconsciousness. Our results establish that consciousness rests on the brain's ability to sustain rich brain dynamics and pave the way for determining specific and generalizable fingerprints of conscious and unconscious states.

Impaired conscious access and abnormal attentional amplification in schizophrenia.

Previous research suggests that the conscious perception of a masked stimulus is impaired in schizophrenia, while unconscious bottom-up processing of the same stimulus, as assessed by subliminal priming, can be preserved. Here, we test this postulated dissociation between intact bottom-up and impaired top-down processing and evaluate its brain mechanisms using high-density recordings of event-related potentials. Sixteen patients with schizophrenia and sixteen controls were exposed to peripheral digits with various degrees of visibility, under conditions of either focused attention or distraction by another task. In the distraction condition, the brain activity evoked by masked digits was drastically reduced in both groups, but early bottom-up visual activation could still be detected and did not differ between patients and controls. By contrast, under focused top-down attention, a major impairment was observed: in patients, contrary to controls, the late non-linear ignition associated with the P3 component was reduced. Interestingly, the patients showed an essentially normal attentional amplification of the P1 and N2 components. These results suggest that some but not all top-down attentional amplification processes are impaired in schizophrenia, while bottom-up processing seems to be preserved.

A lateral-to-mesial organization of human ventral visual cortex at birth.

In human adults, ventral extra-striate visual cortex contains a mosaic of functionally specialized areas, some responding preferentially to natural visual categories such as faces (fusiform face area) or places (parahippocampal place area) and others to cultural inventions such as written words and numbers (visual word form and number form areas). It has been hypothesized that this mosaic arises from innate biases in cortico-cortical connectivity. We tested this hypothesis by examining functional resting-state correlation at birth using fMRI data from full-term human newborns. The results revealed that ventral visual regions are functionally connected with their contra-lateral homologous regions and also exhibit distinct patterns of long-distance functional correlation with anterior associative regions. A mesial-to-lateral organization was observed, with the signal of the more lateral regions, including the sites of visual word and number form areas, exhibiting higher correlations with voxels of the prefrontal, inferior parietal and temporal cortices, including language areas. Finally, we observed hemispheric asymmetries in the functional correlation of key areas of the language network that may influence later adult hemispheric lateralization. We suggest that long-distance circuits present at birth constrain the subsequent functional differentiation of the ventral visual cortex.

A random-matrix theory of the number sense.

Number sense, a spontaneous ability to process approximate numbers, has been documented in human adults, infants and newborns, and many other animals. Species as distant as monkeys and crows exhibit very similar neurons tuned to specific numerosities. How number sense can emerge in the absence of learning or fine tuning is currently unknown. We introduce a random-matrix theory of self-organized neural states where numbers are coded by vectors of activation across multiple units, and where the vector codes for successive integers are obtained through multiplication by a fixed but random matrix. This cortical implementation of the 'von Mises' algorithm explains many otherwise disconnected observations ranging from neural tuning curves in monkeys to looking times in neonates and cortical numerotopy in adults. The theory clarifies the origin of Weber-Fechner's Law and yields a novel and empirically validated prediction of multi-peak number neurons. Random matrices constitute a novel mechanism for the emergence of brain states coding for quantity.This article is part of a discussion meeting issue 'The origins of numerical abilities'.

Linking morphological and functional variability in hand movement and silent reading.

It is generally accepted in neuroscience that anatomy and function go hand in hand. Accordingly, a local morphological variability could lead to a corresponding functional variability. In this study, we tested this hypothesis by linking the variability of the cortical folding pattern of 252 right-handed subjects to the localization or the pattern of functional activations induced by hand motion or silent reading. Three regions are selected: the central sulcus, the precentral sulcus and the superior temporal sulcus (STS). "Essential morphological variability traits" are identified using a method building upon multidimensional scaling. The link between variability in anatomy and function is confirmed by the perfect match between the central sulcus morphological "hand knob" and the corresponding motor activation: as the location of the hand knob moves more or less dorsally along the central sulcus, the motor hand activation moves accordingly. Furthermore, the size of the left hand activation in the right hemisphere is correlated with the knob location in the central sulcus. A new link between functional and morphological variability is discovered relative to the location of a premotor activation induced by silent reading. While this reading activation is located next to the wall of the central sulcus when the hand knob has a ventral positioning, it is pushed into a deep gyrus interrupting the precentral sulcus when the knob is more dorsal. Finally, it is shown that the size of the reading activation along the STS is larger when the posterior branches are less developed.

Spatially invariant coding of numerical information in functionally defined subregions of human parietal cortex.

Macaque electrophysiology has revealed neurons responsive to number in lateral (LIP) and ventral (VIP) intraparietal areas. Recently, fMRI pattern recognition revealed information discriminative of individual numbers in human parietal cortex but without precisely localizing the relevant sites or testing for subregions with different response profiles. Here, we defined the human functional equivalents of LIP (feLIP) and VIP (feVIP) using neurophysiologically motivated localizers. We applied multivariate pattern recognition to investigate whether both regions represent numerical information and whether number codes are position specific or invariant. In a delayed number comparison paradigm with laterally presented numerosities, parietal cortex discriminated between numerosities better than early visual cortex, and discrimination generalized across hemifields in parietal, but not early visual cortex. Activation patterns in the 2 parietal regions of interest did not differ in the coding of position-specific or position-independent number information, but in the expression of a numerical distance effect which was more pronounced in feLIP. Thus, the representation of number in parietal cortex is at least partially position invariant. Both feLIP and feVIP contain information about individual numerosities in humans, but feLIP hosts a coarser representation of numerosity than feVIP, compatible with either broader tuning or a summation code.

Interoperable atlases of the human brain.

The last two decades have seen an unprecedented development of human brain mapping approaches at various spatial and temporal scales. Together, these have provided a large fundus of information on many different aspects of the human brain including micro- and macrostructural segregation, regional specialization of function, connectivity, and temporal dynamics. Atlases are central in order to integrate such diverse information in a topographically meaningful way. It is noteworthy, that the brain mapping field has been developed along several major lines such as structure vs. function, postmortem vs. in vivo, individual features of the brain vs. population-based aspects, or slow vs. fast dynamics. In order to understand human brain organization, however, it seems inevitable that these different lines are integrated and combined into a multimodal human brain model. To this aim, we held a workshop to determine the constraints of a multi-modal human brain model that are needed to enable (i) an integration of different spatial and temporal scales and data modalities into a common reference system, and (ii) efficient data exchange and analysis. As detailed in this report, to arrive at fully interoperable atlases of the human brain will still require much work at the frontiers of data acquisition, analysis, and representation. Among them, the latter may provide the most challenging task, in particular when it comes to representing features of vastly different scales of space, time and abstraction. The potential benefits of such endeavor, however, clearly outweigh the problems, as only such kind of multi-modal human brain atlas may provide a starting point from which the complex relationships between structure, function, and connectivity may be explored.

Characterizing the dynamics of mental representations: the temporal generalization method.

Parsing a cognitive task into a sequence of operations is a central problem in cognitive neuroscience. We argue that a major advance is now possible owing to the application of pattern classifiers to time-resolved recordings of brain activity [electroencephalography (EEG), magnetoencephalography (MEG), or intracranial recordings]. By testing at which moment a specific mental content becomes decodable in brain activity, we can characterize the time course of cognitive codes. Most importantly, the manner in which the trained classifiers generalize across time, and from one experimental condition to another, sheds light on the temporal organization of information-processing stages. A repertoire of canonical dynamical patterns is observed across various experiments and brain regions. This method thus provides a novel way to understand how mental representations are manipulated and transformed.

A model of subjective report and objective discrimination as categorical decisions in a vast representational space.

Subliminal perception studies have shown that one can objectively discriminate a stimulus without subjectively perceiving it. We show how a minimalist framework based on Signal Detection Theory and Bayesian inference can account for this dissociation, by describing subjective and objective tasks with similar decision-theoretic mechanisms. Each of these tasks relies on distinct response classes, and therefore distinct priors and decision boundaries. As a result, they may reach different conclusions. By formalizing, within the same framework, forced-choice discrimination responses, subjective visibility reports and confidence ratings, we show that this decision model suffices to account for several classical characteristics of conscious and unconscious perception. Furthermore, the model provides a set of original predictions on the nonlinear profiles of discrimination performance obtained at various levels of visibility. We successfully test one such prediction in a novel experiment: when varying continuously the degree of perceptual ambiguity between two visual symbols presented at perceptual threshold, identification performance varies quasi-linearly when the stimulus is unseen and in an 'all-or-none' manner when it is seen. The present model highlights how conscious and non-conscious decisions may correspond to distinct categorizations of the same stimulus encoded by a high-dimensional neuronal population vector.

Cerebral mechanisms of general anesthesia.

How does general anesthesia (GA) work? Anesthetics are pharmacological agents that target specific central nervous system receptors. Once they bind to their brain receptors, anesthetics modulate remote brain areas and end up interfering with global neuronal networks, leading to a controlled and reversible loss of consciousness. This remarkable manipulation of consciousness allows millions of people every year to undergo surgery safely most of the time. However, despite all the progress that has been made, we still lack a clear and comprehensive insight into the specific neurophysiological mechanisms of GA, from the molecular level to the global brain propagation. During the last decade, the exponential progress in neuroscience and neuro-imaging led to a significant step in the understanding of the neural correlates of consciousness, with direct consequences for clinical anesthesia. Far from shutting down all brain activity, anesthetics lead to a shift in the brain state to a distinct, highly specific and complex state, which is being increasingly characterized by modern neuro-imaging techniques. There are several clinical consequences and challenges that are arising from the current efforts to dissect GA mechanisms: the improvement of anesthetic depth monitoring, the characterization and avoidance of intra-operative awareness and post-anesthesia cognitive disorders, and the development of future generations of anesthetics.

Single-trial decoding of auditory novelty responses facilitates the detection of residual consciousness.

Detecting residual consciousness in unresponsive patients is a major clinical concern and a challenge for theoretical neuroscience. To tackle this issue, we recently designed a paradigm that dissociates two electro-encephalographic (EEG) responses to auditory novelty. Whereas a local change in pitch automatically elicits a mismatch negativity (MMN), a change in global sound sequence leads to a late P300b response. The latter component is thought to be present only when subjects consciously perceive the global novelty. Unfortunately, it can be difficult to detect because individual variability is high, especially in clinical recordings. Here, we show that multivariate pattern classifiers can extract subject-specific EEG patterns and predict single-trial local or global novelty responses. We first validate our method with 38 high-density EEG, MEG and intracranial EEG recordings. We empirically demonstrate that our approach circumvents the issues associated with multiple comparisons and individual variability while improving the statistics. Moreover, we confirm in control subjects that local responses are robust to distraction whereas global responses depend on attention. We then investigate 104 vegetative state (VS), minimally conscious state (MCS) and conscious state (CS) patients recorded with high-density EEG. For the local response, the proportion of significant decoding scores (M=60%) does not vary with the state of consciousness. By contrast, for the global response, only 14% of the VS patients' EEG recordings presented a significant effect, compared to 31% in MCS patients' and 52% in CS patients'. In conclusion, single-trial multivariate decoding of novelty responses provides valuable information in non-communicating patients and paves the way towards real-time monitoring of the state of consciousness.

Probing consciousness with event-related potentials in the vegetative state.

OBJECTIVE: Probing consciousness in noncommunicating patients is a major medical and neuroscientific challenge. While standardized and expert behavioral assessment of patients constitutes a mandatory step, this clinical evaluation stage is often difficult and doubtful, and calls for complementary measures which may overcome its inherent limitations. Several functional brain imaging methods are currently being developed within this perspective, including fMRI and cognitive event-related potentials (ERPs). We recently designed an original rule extraction ERP test that is positive only in subjects who are conscious of the long-term regularity of auditory stimuli. METHODS: In the present work, we report the results of this test in a population of 22 patients who met clinical criteria for vegetative state. RESULTS: We identified 2 patients showing this neural signature of consciousness. Interestingly, these 2 patients showed unequivocal clinical signs of consciousness within the 3 to 4 days following ERP recording. CONCLUSIONS: Taken together, these results strengthen the relevance of bedside neurophysiological tools to improve diagnosis of consciousness in noncommunicating patients.

Probing the cortical network underlying the psychological refractory period: a combined EEG-fMRI study.

Human performance exhibits strong multi-tasking limitations in simple response time tasks. In the psychological refractory period (PRP) paradigm, where two tasks have to be performed in brief succession, central processing of the second task is delayed when the two tasks are performed at short time intervals. Here, we aimed to probe the cortical network underlying this postponement of central processing by simultaneously recording electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data while 12 subjects performed two simple number-comparison tasks. Behavioral data showed a significant slowing of response times to the second target stimulus at short stimulus-onset asynchronies, together with significant correlations between response times to the first and second target stimulus, i.e., the hallmarks of the PRP effect. The analysis of EEG data showed a significant delay of the post-perceptual P3 component evoked by the second target, which was of similar magnitude as the effect on response times. fMRI data revealed an involvement of parietal and prefrontal regions in dual-task processing. The combined analysis of fMRI and EEG data-based on the trial-by-trial variability of the P3-revealed that BOLD signals in two bilateral regions in the inferior parietal lobe and precentral gyrus significantly covaried with P3 related activity. Our results show that combining neuroimaging methods of high spatial and temporal resolutions can help to identify cortical regions underlying the central bottleneck of information processing, and strengthen the conclusion that fronto-parietal cortical regions participate in a distributed "global neuronal workspace" system that underlies the generation of the P3 component and may be one of the key cerebral underpinnings of the PRP bottleneck.

Language or music, mother or Mozart? Structural and environmental influences on infants' language networks.

Understanding how language emerged in our species calls for a detailed investigation of the initial specialization of the human brain for speech processing. Our earlier research demonstrated that an adult-like left-lateralized network of perisylvian areas is already active when infants listen to sentences in their native language, but did not address the issue of the specialization of this network for speech processing. Here we used fMRI to study the organization of brain activity in two-month-old infants when listening to speech or to music. We also explored how infants react to their mother's voice relative to an unknown voice. The results indicate that the well-known structural asymmetry already present in the infants' posterior temporal areas has a functional counterpart: there is a left-hemisphere advantage for speech relative to music at the level of the planum temporale. The posterior temporal regions are thus differently sensitive to the auditory environment very early on, channelling speech inputs preferentially to the left side. Furthermore, when listening to the mother's voice, activation was modulated in several areas, including areas involved in emotional processing (amygdala, orbito-frontal cortex), but also, crucially, a large extent of the left posterior temporal lobe, suggesting that the mother's voice plays a special role in the early shaping of posterior language areas. Both results underscore the joint contributions of genetic constraints and environmental inputs in the fast emergence of an efficient cortical network for language processing in humans.

Causal role of prefrontal cortex in the threshold for access to consciousness.

What neural mechanisms support our conscious perception of briefly presented stimuli? Some theories of conscious access postulate a key role of top-down amplification loops involving prefrontal cortex (PFC). To test this issue, we measured the visual backward masking threshold in patients with focal prefrontal lesions, using both objective and subjective measures while controlling for putative attention deficits. In all conditions of temporal or spatial attention cueing, the threshold for access to consciousness was systematically shifted in patients, particular after a lesion of the left anterior PFC. The deficit affected subjective reports more than objective performance, and objective performance conditioned on subjective visibility was essentially normal. We conclude that PFC makes a causal contribution to conscious visual perception of masked stimuli, and outline a dual-route signal detection theory of objective and subjective decision making.

Parsing a sequence of brain activations at psychological times using fMRI.

Identifying the sequence of computations which constitute a cognitive task is a fundamental problem in neuroscience. Here we show, using functional magnetic resonance imaging (fMRI), that we can parse, at the time scale of about 100 ms, the different stages of brain activations which compose a complex sequential task. To identify timing information from the slow blood oxygen level-dependent (BOLD) signal response, we use a simple analytic method, based on periodic stimulation and an analysis of covariation of the spectral parameters (phase and power spectrum at the stimulation frequency) with the different experimental conditions. We implement this strategy in a sequential task, where the onset and duration of different stages are under experimental control. We are able to detect changes in onset latency and in the duration of the response, in an invariant fashion across different brain regions, and reconstruct the stream of activations consistent with five distinct stages of processing of the task. Sensory and motor clusters activate in the expected order and for the expected duration. The timing of sensory activations is more precise than the timing of motor activation. We also parse in time the reading-verbal network: visual extrastriate and phonological access regions (supramarginal gyrus) activate at the time of word presentation, while the inferior frontal gyrus, the anterior cingulate and the supplementary motor area are activated during the rehearsal period.

Objective correlates of an unusual subjective experience: A single-case study of number-form synaesthesia.

There is a universal and often unconscious tendency to mentally associate the number sequence with a spatial continuum (the mental number line). Here we study one individual who reports a strong and vivid sense of space when processing numbers. For him, the number sequence has a precise spatial form: a curvilinear right-to-left oriented line. We used various tasks to demonstrate that this numerical-spatial association is not a mere figment of his imagination, but a constrained experiential phenomenon consistent across sessions and automatically triggered by the visual presentation of numbers. We also show that this idiosyncratic representation can coexist with another implicit association, the SNARC effect (Spatial-Numerical Association of Response Codes, where small numbers are associated with the left side of space). This effect is present in individuals without explicit number forms and is not affected in the present subject in spite of his reversed subjective representation.

Brain anatomy in Turner syndrome: evidence for impaired social and spatial-numerical networks.

Analysis of brain structure in Turner syndrome (TS) provides the opportunity to identify the consequences of the loss of one X chromosome on brain anatomy and to characterize the neural bases underlying the specific cognitive profile of TS subjects which includes deficits in spatial-numerical processing and social cognition. Fourteen subjects with TS and fourteen controls were investigated using voxel-based analysis of high resolution anatomical and diffusion tensor images and using sulcal morphometry. The analysis of anatomical images provided evidence for macroscopical changes in cortical regions involved in social cognition such as the left superior temporal sulcus and orbito-frontal cortex and in a region involved in spatial and numerical cognition such as the right intraparietal sulcus. Diffusion tensor images showed a displacement of the grey-white matter interface of the left and right superior temporal sulcus and revealed bilateral microstructural anomalies in the temporal white matter. The analysis of fiber orientation suggests specific alterations of fiber tracts connecting posterior to anterior temporal regions. Last, sulcal morphometry confirmed the anomalies of the left and right superior temporal sulci and of the right intraparietal sulcus. Our results thus provide converging evidence of regionally specific structural changes in TS that are highly consistent with the hallmark symptoms associated with TS.

Letter binding and invariant recognition of masked words: behavioral and neuroimaging evidence.

Fluent readers recognize visual words across changes in case and retinal location, while maintaining a high sensitivity to the arrangement of letters. To evaluate the automaticity and functional anatomy of invariant word recognition, we measured brain activity during subliminal masked priming. By preceding target words with an unrelated prime, a repeated prime, or an anagram made of the same letters, we separated letter-level and whole-word codes. By changing the case and the retinal location of primes and targets, we evaluated the invariance of those codes. Our results indicate that an invariant binding of letters into words is achieved unconsciously through a series of increasingly invariant stages in the left occipito-temporal pathway.