Beyond the word and image: II- Structural and functional connectivity of a common semantic system.

Understanding events requires interplaying cognitive processes arising in neural networks whose organisation and connectivity remain subjects of controversy in humans. In the present study, by combining diffusion tensor imaging and functional interaction analysis, we aim to provide new insights on the organisation of the structural and functional pathways connecting the multiple nodes of the identified semantic system -shared by vision and language (Jouen et al., 2015). We investigated a group of 19 healthy human subjects during experimental tasks of reading sentences or seeing pictures. The structural connectivity was realised by deterministic tractography using an algorithm to extract white matter fibers terminating in the selected regions of interest (ROIs) and the functional connectivity by independent component analysis to measure correlated activities among these ROIs. The major connections link ventral neural stuctures including the parietal and temporal cortices through inferior and middle longitudinal fasciculi, the retrosplenial and parahippocampal cortices through the cingulate bundle, and the temporal and prefrontal structures through the uncinate fasciculus. The imageability score provided when the subject was reading a sentence was significantly correlated with the factor of anisotropy of the left parieto-temporal connections of the middle longitudinal fasciculus. A large part of this ventrally localised structural connectivity corresponds to functional interactions between the main parietal, temporal and frontal nodes. More precisely, the strong coactivation both in the anterior temporal pole and in the region of the temporo-parietal cortex suggests dual and cooperating roles for these areas within the semantic system. These findings are discussed in terms of two semantics-related sub-systems responsible for conceptual representation.

Beyond the word and image: characteristics of a common meaning system for language and vision revealed by functional and structural imaging.

This research tests the hypothesis that comprehension of human events will engage an extended semantic representation system, independent of the input modality (sentence vs. picture). To investigate this, we examined brain activation and connectivity in 19 subjects who read sentences and viewed pictures depicting everyday events, in a combined fMRI and DTI study. Conjunction of activity in understanding sentences and pictures revealed a common fronto-temporo-parietal network that included the middle and inferior frontal gyri, the parahippocampal-retrosplenial complex, the anterior and middle temporal gyri, the inferior parietal lobe in particular the temporo-parietal cortex. DTI tractography seeded from this temporo-parietal cortex hub revealed a multi-component network reaching into the temporal pole, the ventral frontal pole and premotor cortex. A significant correlation was found between the relative pathway density issued from the temporo-parietal cortex and the imageability of sentences for individual subjects, suggesting a potential functional link between comprehension and the temporo-parietal connectivity strength. These data help to define a "meaning" network that includes components of recently characterized systems for semantic memory, embodied simulation, and visuo-spatial scene representation. The network substantially overlaps with the "default mode" network implicated as part of a core network of semantic representation, along with brain systems related to the formation of mental models, and reasoning. These data are consistent with a model of real-world situational understanding that is highly embodied. Crucially, the neural basis of this embodied understanding is not limited to sensorimotor systems, but extends to the highest levels of cognition, including autobiographical memory, scene analysis, mental model formation, reasoning and theory of mind.

Insights on embodiment induced by visuo-tactile stimulation during robotic telepresence.

Using a simple neuroscience-inspired procedure to beam human subjects into robots, we previously demonstrated by visuo-motor manipulations that embodiment into a robot can enhance the acceptability and closeness felt towards the robot. In that study, the feelings of likeability and closeness toward the robot were significantly related to the sense of agency, independently of the sensations of enfacement and location. Here, using the same paradigm we investigated the effect of a purely sensory manipulation on the sense of robotic embodiment associated to social cognition. Wearing a head-mounted display, participants saw the visual scene captured from the robot eyes. By positioning a mirror in front of the robot, subjects saw themselves as a robot. Tactile stimulation was provided by stroking synchronously or not with a paintbrush the same location of the subject and robot faces. In contrast to the previous motor induction of embodiment which particularly affected agency, tactile induction yields more generalized effects on the perception of ownership, location and agency. Interestingly, the links between positive social feelings towards the robot and the strength of the embodiment sensations were not observed. We conclude that the embodiment into a robot is not sufficient in itself to induce changes in social cognition.

Embodiment into a robot increases its acceptability.

Recent studies have shown how embodiment induced by multisensory bodily interactions between individuals can positively change social attitudes (closeness, empathy, racial biases). Here we use a simple neuroscience-inspired procedure to beam our human subjects into one of two distinct robots and demonstrate how this can readily increase acceptability and social closeness to that robot. Participants wore a Head Mounted Display tracking their head movements and displaying the 3D visual scene taken from the eyes of a robot which was positioned in front of a mirror and piloted by the subjects' head movements. As a result, participants saw themselves as a robot. When participant' and robot's head movements were correlated, participants felt that they were incorporated into the robot with a sense of agency. Critically, the robot they embodied was judged more likeable and socially closer. Remarkably, we found that the beaming experience with correlated head movements and corresponding sensation of embodiment and social proximity, was independent of robots' humanoid's appearance. These findings not only reveal the ease of body-swapping, via visual-motor synchrony, into robots that do not share any clear human resemblance, but they may also pave a new way to make our future robotic helpers socially acceptable.

Double dissociation in neural correlates of visual working memory: a PET study.

Using positron emission tomography (PET), we investigated the organisation of spatial versus object-based visual working memory in 11 normal human subjects. The paradigm involved a conditional colour-response association task embedded within two visual working memory tasks. The subject had to remember a position (spatial) or shape (object-based) and then use this to recover the colour of the matching element for the conditional association. Activation of the nucleus accumbens and the anterior cingulate cortex was observed during the conditional associative task, indicating a possible role of these limbic structures in associative memory. When the 2 memory tasks were contrasted, we observed activation of 2 distinct cortical networks: (1) The spatial task activated a dorsal stream network distributed in the right hemisphere in the parieto-occipital cortex and the dorsal prefrontal cortex, and (2) The non spatial task activated a ventral stream network distributed in the left hemisphere in the temporo- occipital cortex, the ventral prefrontal cortex and the striatum. These results support the existence of a domain-specific dissociation with dorsal and ventral cortical systems involved respectively in spatial and non spatial working memory functions.

Motor imagery in normal subjects and in asymmetrical Parkinson's disease: a PET study.

OBJECTIVE: To investigate, using PET and H2(15)O, brain activation abnormalities of patients with PD during motor imagery. To determine whether motor imagery activation patterns depend on the hand used to complete the task. BACKGROUND: Previous work in PD has shown that bradykinesia is associated with slowness of motor imagery. METHODS: The PET study was performed in eight patients with PD with predominantly right-sided akinesia, and in eight age-matched control subjects, all right-handed. Regional cerebral blood flow was measured by PET and H2(15)O while subjects imagined a predetermined unimanual externally cued sequential movement with a joystick with either the left or the right hand, and during a rest condition. RESULTS: In normal subjects, the prefrontal cortex, supplementary motor area (SMA), superior parietal lobe, inferior frontal gyrus, and cerebellum were activated during motor imagery with either the left or the right hand. Contralateral primary motor cortex activation was noted only when the task was imagined with the right (dominant) hand, whereas activation of the dorsolateral prefrontal cortex was observed only during imagery with the left hand. In patients with PD, motor imagery with the right ("akinetic") hand was characterized by lack of activation of the contralateral primary sensorimotor cortex and the cerebellum, persistent activation of the SMA, and bilateral activation of the superior parietal cortex. Motor imagery with the left ("non-akinetic") hand was also abnormal, with lack of activation of the SMA compared with controls. CONCLUSIONS: In patients with PD with predominantly right-sided akinesia, brain activation during motor imagery is abnormal and may appear even with the less affected hand. In normal subjects, brain activation during motor imagery depends on the hand used in the imagined movement.

Dissociable effects of dopaminergic therapy on spatial versus non-spatial working memory in Parkinson's disease.

There is now evidence for definite and early cognitive deficits in Parkinson's disease (PD), involving, in particular, executive functions and working memory. However, the distinction between visuo-spatial and non-spatial working memory deficits and the impact of dopamine on these deficits are still open to debate. The aim of this study was therefore to investigate cognitive and motor performance in PD patients in two conditional associative learning tasks requiring either spatial or non-spatial visual working memory. The subject had to point to visual targets according to the visual characteristics of memorised visual cues (colour, position and form). To assess the effect of L-dopa therapy, PD patients were studied over two consecutive days: one ON/OFF group of nine PD patients with treatment (ON condition) on the first day and without treatment (OFF condition) on the second day; and another OFF/ON group of nine PD patients tested on reverse. The PD groups were compared to a control group of nine age-matched healthy subjects. Our main data demonstrate that: (1) in PD patients with OFF treatment, the response time of manual pointing is increased mainly in the non-spatial working memory task; and (2) in PD patients with ON treatment, either the response time is normal (on the first day) or is increased in both visuo-spatial and non-spatial tasks. We suggest that this dissociation between spatial versus non-spatial working memory deficits in non-medicated PD might be related to compensatory mechanisms that occur following fronto-striatal dysfunction.

Analogical transfer is effective in a serial reaction time task in Parkinson's disease: evidence for a dissociable form of sequence learning.

Several studies of procedural learning in Parkinson's disease (PD) have demonstrated that these patients are impaired with respect to age-matched control subjects. In order to examine more closely the specific impairment, we considered three dimensions along which a procedural learning task could vary. These are: (1) implicit vs explicit learning, (2) instance vs rule learning, and (3) learning with internal vs external error correction. We consider two hypotheses that could explain the impairments observed in PD for different types of explicit motor learning: (H1) an impairment related to the acquisition of rules vs specific instances, and (H2) an impairment in learning when no explicit error feedback is provided. In order to examine the condition of rule learning with external error feedback, we developed a modified version of the serial reaction time (SRT) protocol that tests analogical transfer in sequence learning (ATSL). Reaction times are measured for responses to visual stimuli that appear in several different repeating sequences. While these isomorphic sequences are different, they share a common rule. Verbatim learning of a sequence would result in negative transfer from one sequence to a different one, while rule learning would result in positive transfer. Parkinson's patients and age-matched controls demonstrate significant acquisition and positive transfer of the rule between sequences. Our results demonstrate that PD patients are capable of learning and transferring rule or schema-based representations in an explicit learning format, and that this form of learning may be functionally distinct from learning mechanisms that rely on representations of the verbatim or statistical structure of sequences.

A shared system for learning serial and temporal structure of sensori-motor sequences? Evidence from simulation and human experiments.

This research investigates the influences of temporal structure on the representation of serial order. Experiments are performed in a neural network model of sequence learning and in human subjects. In the sequence learning model, a recurrent network of leaky integrator neurons encodes a succession of internal states that become associated, by reinforcement learning, with the correct sequential responses. First, the model is shown to learn a simple temporal discrimination task. The model is then exposed to two novel serial reaction time (SRT) experiments. In the standard SRT task (M.J. Nissen, P. Bullemer, Attentional requirements of learning: evidence from performance measures, Cogn. Psychol. 19 (1987) 1-32 [16]), reaction times for stimuli presented in a repeating sequence are reduced with respect to those for random stimuli, providing a measure of sequence learning. The novelty of the current experiments is that imbedded in the serial order of the sequences, there is a temporal structure of delays. The model is sensitive to both the serial structure and the temporal structure of the sequences. This observation is then confirmed in human subjects. These results demonstrate how a novel recurrent architecture encodes the interaction of temporal and serial structure and provide insight into related aspects of human sensori-motor sequence learning.

Encoding behavioral context in recurrent networks of the fronto-striatal system: a simulation study.

This research addresses the hypothesis that behavioral context is encoded in recurrent networks of the fronto-striatal system. Behavioral context influences the processing of subsequent brain events, including responses to sensory inputs, thus providing a basis for context-dependent behavior. We define context-dependent behavior as the adaptive ability to produce the appropriate response to a given stimulus, dependent upon the context in which it appears. Behavioral context can change with a time-scale on the order of seconds to tens of seconds or more. This suggests a flexible mechanism that encodes context via an ensemble of neural activation that will appropriately influence the processing of subsequent sensory stimuli. We present a functional model of context encoding in recurrent connections of the fronto-striatal system with simulation results that correspond closely to empirical data. Neuronal activity in monkeys that perform a context-dependent task indicate that the prefrontal cortex and striatum participate differentially in this kind of context encoding. Likewise, simulated neurons in our model of the fronto-striatal system, which performs the context-dependent task, display task-related activity remarkably similar to that found in monkey frontal cortex and striatum, supporting our hypothesis.

Schizophrenics learn surface but not abstract structure in a serial reaction time task.

We investigated the hypothesis that explicit cognitive processing is impaired in schizophrenia, while implicit processing is left largely intact, using a single sequence learning paradigm that simultaneously measures surface and abstract structure learning. Surface structure is the serial order of sequence elements. Abstract structure is defined in terms of the relationships between repeating elements. Sequences ABCBAC and DEFEDF thus share the same abstract structure, with different surface structures. Learning abstract structure requires explicit processing, while surface structure can be learned in implicit conditions. We predict however, that in explicit conditions, schizophrenics should learn surface structure but not abstract structure. Indeed, schizophrenic patients learned surface structure, but failed to learn abstract structure, demonstrating that implicit sequence processing is spared in schizophrenia while explicit sequence processing is impaired.

Contribution of frontostriatal function to sequence learning in Parkinson's disease: evidence for dissociable systems.

The frontostriatal system appears to play a crucial role in the organization and execution of sequential behaviour, but the precise nature of its contribution remains to be specified. From this perspective, relatively simple modifications of behavioural task parameters may invoke rather profound changes in the recruitment of appropriate neural mechanisms, including the frontostriatal system. This mini-review examines how variations in task requirements for sequence learning and related cognitive tasks can induce significant modifications in the performance of patients with Parkinson's disease. In particular, these observations are used to support a developing argument for neurophysiologically dissociable sequence learning systems in man. One of these mechanisms is sensitive to surface structure, or element-by-element sequence organization, and appears to rely on the frontostriatal system. Another sequence learning mechanism is sensitive to abstract structure, or relationships between repeating sequence elements, and appears to be largely independent of the frontostriatal system.

ERP analysis of cognitive sequencing: a left anterior negativity related to structural transformation processing.

A major objective of cognitive neuroscience is to identify those neurocomputational processes that may be shared by multiple cognitive functions vs those that are highly specific. This problem of identifying general vs specialized functions is of particular interest in the domain of language processing. Within this domain, event related brain potential (ERP) studies have demonstrated a left anterior negativity (LAN) in a range 300-700 ms, associated with syntactic processing, often linked to grammatical function words. These words have little or no semantic content, but rather play a role in encoding syntactic structure required for parsing. In the current study we test the hypothesis that the LAN reflects the operation of a more general sequence processing capability in which special symbols encode structural information that, when combined with past elements in the sequence, allows the prediction of successor elements. We recorded ERPs during a non-linguistic sequencing task that required subjects (n = 10) to process special symbols possessing the functional property defined above. When compared to ERPs in a control condition, function symbol processing elicits a left anterior negative shift between temporal and spatial characteristics quite similar to the LAN described during function word processing in language, supporting our hypothesis. These results are discussed in the context of related studies of syntactic and cognitive sequence processing.

Dissociable ERP profiles for processing rules vs instances in a cognitive sequencing task.

A hotly debated question in cognitive neuroscience is whether individual instances of perceptual sequences, and the rules that describe them, are processed by the same brain mechanisms. We tested the hypothesis that such rules and instances are processed by dissociable brain mechanisms. We analyzed event-related brain potentials (ERPs) evoked during cognitive sequencing tasks that assessed surface (instance) vs abstract (rule) structure learning. Sequence instances ABCBAC and DEFEDF have different serial order or surface structure, but share the same rule or abstract structure, 123213. Nine healthy subjects were first trained to learn a set of surface and abstract structures in sequences of visually presented stimuli. During the subsequent ERP recording, for surface and abstract structures, they then discriminated between acceptable and unacceptable sequences, based on the pre-learned regularities. Abstract structure processing evoked a late positivity around 500 ms, which was not seen in the surface structure processing, supporting our hypothesis of dissociable processes. We discuss implications for the rule vs instance debate, and similarities between this late positivity and the P600 observed in previous studies of syntactic processing.

Extraocular proprioception is required for spatial localization in man.

We studied the effect of extraocular muscle deafferentation on the performance of manual pointing to foveated visual targets at different craniotopic locations. Five patients who underwent unilateral thermocoagulation of the trigeminal nerve (V) for trigeminal neuralgia were tested before and after surgery. Only the group of patients with postsurgical corneal anaesthesia, attesting to involvement of the ophthalmic branch (which carries the ocular proprioceptive fibres), presented deficits in accuracy of pointing. The position of initial and final pointing was shifted toward the lesioned side predominantly in the ipsilateral hemispace. These data support a model in which balanced ocular proprioceptive inputs are required for accuracy of visually guided movements in egocentric space.

Cognitive sequence processing and syntactic comprehension in schizophrenia.

It has been repeatedly demonstrated that schizophrenic patients are impaired in the comprehension of sentences with complex syntax. We investigated the hypothesis that this syntactic comprehension impairment in schizophrenia is not a purely linguistic dysfunction, but rather the reflection of a cognitive sequence processing impairment that is revealed as task complexity increases. We tested 10 schizophrenic patients using a standard measure of syntactic comprehension, and a non-linguistic sequence processing task, both of which required simple and complex transformation processing. Patients' performance impairment on the two tasks was highly correlated (r2 = 0.84), and there was a significant effect for complexity, independent of the task. These results are quite similar to those of aphasic patients with left hemisphere lesions. This suggests that syntactic comprehension deficits in schizophrenia reveal the dysfunction of cognitive sequence processing mechanisms that can be expressed both in linguistic and non-linguistic sequence tasks.

Semantic priming in major depressive state.

We studied semantic priming in 20 major depressive subjects. The methodology used was a visual lexical decision task. Semantic priming is the facilitation of target word recognition (shortening of response time) by the prior presentation of a semantically related context (a prime word). It relies on semantic processing of words and context, facilitating early cognitive stages of response. Varying the temporal interval between prime and target words onset allows us to distinguish between two priming mechanisms, relying on more automatic (test 1) or more controlled (i.e. attention dependent) (test 2) information processing. We observe a significant retardation for words and pseudo-words in depressives (in relation to controls) in both tests. In spite of a general retardation and increase of response times in depressives, semantic priming is evident in both groups and both tests, and does not differ significantly between depressive and control groups in either automatic or controlled conditions. Theses results confirm that semantic processing is not impaired in depression, and are discussed with regard to the hypothesis of an effortful processing impairment in depression, and to depressive retardation.

Anomia in major depressive state.

Anomia, or word finding difficulty, is a frequent clinical symptom of the depressive state. This study investigates naming and lexicalization processes (or word production processes) in 11 depressive patients (major depressive state), through a picture naming task of 53 images corresponding to low frequency words. Depressives showed significantly more anomia and made more naming errors (semantically related substitution words) than control subjects. Tip-of-the-tongue (TOT) states, which correspond to an impairment at a later stage of phonological encoding with partial activation of phonological shape, remained rare in depressives despite the increase of lexicalization difficulties observed. Anomia observed in depressives could thus be related to an impairment at the early stage of lexicalization or word production processes (pre-phonological item selection and access, or storage of the semantic lexical item in Working Memory for further phonological encoding), without lexical-semantic disorganization. We discuss the relationship between such an elementary speech production disorder and cognitive impairments demonstrated in the depressive state (deficit of effortful and attentional processes, impairment in activation or initiation of cognitive processes and responses).

An anatomically structured sensory-motor sequence learning system displays some general linguistic capacities.

The capacity in primates to master temporal-sequential constraints from the external world might provide a basis for accommodating similar constraints in language. While the neural specialization required for language clearly distinguishes man from the other primates, it is less clear to what extent this specialization constitutes a drastic neurophysiological divergence versus a variation on an existing sequencing capability. In an effort to address this issue, an anatomically structured neural network model, previously developed to reproduce complex sensory-motor sequences and the corresponding single-unit recordings from primate prefrontal cortex, is studied in a simple linguistic context. The model is presented sentences from a small language and demonstrates a simple capacity to "understand" and generalize at different levels. Interactions between variations on (a) the model architecture and (b) the target language structure agree with data from crosslinguistic aphasia studies. These results support the hypothesis that a brain architecture for nonlinguistic cognitive functions (in this case sensory-motor sequencing) can provide a basis for a general sequence processing component of linguistic function.