Neural correlates of an illusionary sense of agency caused by virtual reality.

Sense of agency (SoA) is the sensation that self-actions lead to ensuing perceptual consequences. The prospective mechanism emphasizes that SoA arises from motor prediction and its comparison with actual action outcomes, while the reconstructive mechanism stresses that SoA emerges from retrospective causal processing about the action outcomes. Consistent with the prospective mechanism, motor planning regions were identified by neuroimaging studies using the temporal binding (TB) effect, a behavioral measure often linked to implicit SoA. Yet, TB also occurs during passive observation of another's action, lending support to the reconstructive mechanism, but its neural correlates remain unexplored. Here, we employed virtual reality (VR) to modulate such observation-based SoA and examined it with functional magnetic resonance imaging (fMRI). After manipulating an avatar hand in VR, participants passively observed an avatar's "action" and showed a significant increase in TB. The binding effect was associated with the right angular gyrus and inferior parietal lobule, which are critical nodes for inferential and agency processing. These results suggest that the experience of controlling an avatar may potentiate inferential processing within the right inferior parietal cortex and give rise to the illusionary SoA without voluntary action.

Separable amygdala activation patterns in the evaluations of robots.

Given the increasing presence of robots in everyday environments and the significant challenge posed by social interactions with robots, it is crucial to gain a deeper understanding into the social evaluations of robots. One potentially effective approach to comprehend the fundamental processes underlying controlled and automatic evaluations of robots is to probe brain response to different perception levels of robot-related stimuli. Here, we investigate controlled and automatic evaluations of robots based on brain responses during viewing of suprathreshold (duration: 200 ms) and subthreshold (duration: 17 ms) humanoid robot stimuli. Our behavioral analysis revealed that despite participants' self-reported positive attitudes, they held negative implicit attitudes toward humanoid robots. Neuroimaging analysis indicated that subthreshold presentation of humanoid robot stimuli elicited significant activation in the left amygdala, which was associated with negative implicit attitudes. Conversely, no significant left amygdala activation was observed during suprathreshold presentation. Following successful attenuation of negative attitudes, the left amygdala response to subthreshold presentation of humanoid robot stimuli decreased, and this decrease correlated positively with the reduction in negative attitudes. These findings provide evidence for separable patterns of amygdala activation between controlled and automatic processing of robots, suggesting that controlled evaluations may influence automatic evaluations of robots.

Pulvinar Response Profiles and Connectivity Patterns to Object Domains.

Distributed cortical regions show differential responses to visual objects belonging to different domains varying by animacy (e.g., animals vs tools), yet it remains unclear whether this is an organization principle also applying to the subcortical structures. Combining multiple fMRI activation experiments (two main experiments and six validation datasets; 12 females and 9 males in the main Experiment 1; 10 females and 10 males in the main Experiment 2), resting-state functional connectivity, and task-based dynamic causal modeling analysis in human subjects, we found that visual processing of images of animals and tools elicited different patterns of response in the pulvinar, with robust left lateralization for tools, and distinct, bilateral (with rightward tendency) clusters for animals. Such domain-preferring activity distribution in the pulvinar was associated with the magnitude with which the voxels were intrinsically connected with the corresponding domain-preferring regions in the cortex. The pulvinar-to-right-amygdala path showed a one-way shortcut supporting the perception of animals, and the modulation connection from pulvinar to parietal showed an advantage to the perception of tools. These results incorporate the subcortical regions into the object processing network and highlight that domain organization appears to be an overarching principle across various processing stages in the brain.SIGNIFICANCE STATEMENT Viewing objects belonging to different domains elicited different cortical regions, but whether the domain organization applied to the subcortical structures (e.g., pulvinar) was unknown. Multiple fMRI activation experiments revealed that object pictures belonging to different domains elicited differential patterns of response in the pulvinar, with robust left lateralization for tool pictures, and distinct, bilateral (with rightward tendency) clusters for animals. Combining the resting-state functional connectivity and dynamic causal modeling analysis on task-based fMRI data, we found domain-preferring activity distribution in the pulvinar aligned with that in cortical regions. These results highlight the need for coherent visual theories that explain the mechanisms underlying the domain organization across various processing stages.

Gravity matters for the neural representations of action semantics.

The dynamic relationship between the neural representation of action word semantics and specific sensorimotor experience remains controversial. Here, we temporarily altered human subjects' sensorimotor experience in a 15-day head-down tilt bed rest setting, a ground-based analog of microgravity that disproportionally affects sensorimotor experiences of the lower limbs, and examined whether such effector-dependent activity deprivation specifically affected the neural processes of comprehending verbs of lower-limb actions (e.g. to kick) relative to upper-limb ones (e.g. to pinch). Using functional magnetic resonance imaging, we compared the multivoxel neural patterns for such action words prior to and after bed rest. We found an effector-specific (lower vs. upper limb) experience modulation in subcortical sensorimotor-related and anterior temporal regions. The neural action semantic representations in other effector-specific verb semantic regions (e.g. left lateral posterior temporal cortex) and motor execution regions were robust against such experience alterations. These effector-specific, sensorimotor-experience-sensitive and experience-independent patterns of verb neural representation highlight the multidimensional and dynamic nature of semantic neural representation, and the broad influence of microgravity (hence gravity) environment on cognition.

Simple shape feature computation across modalities: convergence and divergence between the ventral and dorsal visual streams.

Shape processing, whether by seeing or touching, is pivotal to object recognition and manipulation. Although the low-level signals are initially processed by different modality-specific neural circuits, multimodal responses to object shapes have been reported along both ventral and dorsal visual pathways. To understand this transitional process, we conducted visual and haptic shape perception fMRI experiments to test basic shape features (i.e. curvature and rectilinear) across the visual pathways. Using a combination of region-of-interest-based support vector machine decoding analysis and voxel selection method, we found that the top visual-discriminative voxels in the left occipital cortex (OC) could also classify haptic shape features, and the top haptic-discriminative voxels in the left posterior parietal cortex (PPC) could also classify visual shape features. Furthermore, these voxels could decode shape features in a cross-modal manner, suggesting shared neural computation across visual and haptic modalities. In the univariate analysis, the top haptic-discriminative voxels in the left PPC showed haptic rectilinear feature preference, whereas the top visual-discriminative voxels in the left OC showed no significant shape feature preference in either of the two modalities. Together, these results suggest that mid-level shape features are represented in a modality-independent manner in both the ventral and dorsal streams.

Different computational relations in language are captured by distinct brain systems.

A critical way for humans to acquire information is through language, yet whether and how language experience drives specific neural semantic representations is still poorly understood. We considered statistical properties captured by 3 different computational principles of language (simple co-occurrence, network-(graph)-topological relations, and neural-network-vector-embedding relations) and tested the extent to which they can explain the neural patterns of semantic representations, measured by 2 functional magnetic resonance imaging experiments that shared common semantic processes. Distinct graph-topological word relations, and not simple co-occurrence or neural-network-vector-embedding relations, had unique explanatory power for the neural patterns in the anterior temporal lobe (capturing graph-common-neighbors), inferior frontal gyrus, and posterior middle/inferior temporal gyrus (capturing graph-shortest-path). These results were relatively specific to language: they were not explained by sensory-motor similarities and the same computational relations of visual objects (based on visual image database) showed effects in the visual cortex in the picture naming experiment. That is, different topological properties within language and the same topological computations (common-neighbors) for language and visual inputs are captured by different brain regions. These findings reveal the specific neural semantic representations along graph-topological properties of language, highlighting the information type-specific and statistical property-specific manner of semantic representations in the human brain.

Neural substrate underlying the learning of a passage with unfamiliar vocabulary and syntax.

Speech comprehension is a complex process involving multiple stages, such as decoding of phonetic units, recognizing words, and understanding sentences and passages. In this study, we identify cortical networks beyond basic phonetic processing using a novel passage learning paradigm. Participants learn to comprehend a story composed of syllables of their native language, but containing unfamiliar vocabulary and syntax. Three learning methods are employed, each resulting in some degree of learning within a 12-min learning session. Functional magnetic resonance imaging results reveal that, when listening to the same story, the classic temporal-frontal language network is significantly enhanced by learning. Critically, activation of the left anterior and posterior temporal lobe correlates with the learning outcome that is assessed behaviorally through, e.g. word recognition and passage comprehension tests. This study demonstrates that a brief learning session is sufficient to induce neural plasticity in the left temporal lobe, which underlies the transformation from phonetic units to the units of meaning, such as words and sentences.

The Role of Visual Experience in Individual Differences of Brain Connectivity.

Visual cortex organization is highly consistent across individuals. But to what degree does this consistency depend on life experience, in particular sensory experience? In this study, we asked whether visual cortex reorganization in congenital blindness results in connectivity patterns that are particularly variable across individuals, focusing on resting-state functional connectivity (RSFC) patterns from the primary visual cortex. We show that the absence of shared visual experience results in more variable RSFC patterns across blind individuals than sighted controls. Increased variability is specifically found in areas that show a group difference between the blind and sighted in their RSFC. These findings reveal a relationship between brain plasticity and individual variability; reorganization manifests variably across individuals. We further investigated the different patterns of reorganization in the blind, showing that the connectivity to frontal regions, proposed to have a role in the reorganization of the visual cortex of the blind toward higher cognitive roles, is highly variable. Further, we link some of the variability in visual-to-frontal connectivity to another environmental factor-duration of formal education. Together, these findings show a role of postnatal sensory and socioeconomic experience in imposing consistency on brain organization. By revealing the idiosyncratic nature of neural reorganization, these findings highlight the importance of considering individual differences in fitting sensory aids and restoration approaches for vision loss.SIGNIFICANCE STATEMENT The typical visual system is highly consistent across individuals. What are the origins of this consistency? Comparing the consistency of visual cortex connectivity between people born blind and sighted people, we showed that blindness results in higher variability, suggesting a key impact of postnatal individual experience on brain organization. Further, connectivity patterns that changed following blindness were particularly variable, resulting in diverse patterns of brain reorganization. Individual differences in reorganization were also directly affected by nonvisual experiences in the blind (years of formal education). Together, these findings show a role of sensory and socioeconomic experiences in creating individual differences in brain organization and endorse the use of individual profiles for rehabilitation and restoration of vision loss.

Neuroplastic changes in functional wiring in sensory cortices of the congenitally deaf: A network analysis.

Congenital sensory deprivation induces significant changes in the structural and functional organisation of the brain. These are well-characterised by cross-modal plasticity, in which deprived cortical areas are recruited to process information from non-affected sensory modalities, as well as by other neuroplastic alterations within regions dedicated to the remaining senses. Here, we analysed visual and auditory networks of congenitally deaf and hearing individuals during different visual tasks to assess changes in network community structure and connectivity patterns due to congenital deafness. In the hearing group, the nodes are clearly divided into three communities (visual, auditory and subcortical), whereas in the deaf group a fourth community consisting mainly of bilateral superior temporal sulcus and temporo-insular regions is present. Perhaps more importantly, the right lateral geniculate body, as well as bilateral thalamus and pulvinar joined the auditory community of the deaf. Moreover, there is stronger connectivity between bilateral thalamic and pulvinar and auditory areas in the deaf group, when compared to the hearing group. No differences were found in the number of connections of these nodes to visual areas. Our findings reveal substantial neuroplastic changes occurring within the auditory and visual networks caused by deafness, emphasising the dynamic nature of the sensory systems in response to congenital deafness. Specifically, these results indicate that in the deaf but not the hearing group, subcortical thalamic nuclei are highly connected to auditory areas during processing of visual information, suggesting that these relay areas may be responsible for rerouting visual information to the auditory cortex under congenital deafness.

Preference for animate domain sounds in the fusiform gyrus of blind individuals is modulated by shape-action mapping.

In high-level visual areas in the human brain, preference for inanimate objects is observed regardless of stimulation modality (visual/auditory/tactile) and individual's visual experience (sighted/blind) whereas preference for animate entities seems robust mainly in the visual modality. Here, we test a hypothesis explaining this domain difference: Object representations can be activated through nonvisual stimulation when their shapes are systematically related to action system representations, a quality typical of most inanimate objects but of only specific animate entities. We studied functional magnetic resonance imaging activations in congenitally blind and sighted individuals listening to animal, object, and human sounds. In blind individuals, the typical location of the fusiform face area preferentially responded to human facial expression sounds clearly related to specific facial actions and resulting face shapes but not to speech or animal sounds. No univariate preference for any sound category was observed in the fusiform gyrus in sighted individuals, but the expected multivoxel effects were present. We conclude that nonvisual signals can activate shape representations of those stimuli-inanimate or animate-for which shape and action computations are transparently related. However, absence of potentially competing visual inputs seems necessary for this effect to be clearly detectable in the case of animate representation.

Brain intrinsic connection patterns underlying tool processing in human adults are present in neonates and not in macaques.

Tool understanding and use are supported by a dedicated left-lateralized, intrinsically connected network in the human adult brain. To examine this network's phylogenetic and ontogenetic origins, we compared resting-state functional connectivity (rsFC) among regions subserving tool processing in human adults to rsFC among homologous regions in human neonates and macaque monkeys (adolescent and mature). These homologous regions formed an intrinsic network in human neonates, but not in macaques. Network topological patterns were highly similar between human adults and neonates, and significantly less so between humans and macaques. The premotor-parietal rsFC had most significant contribution to the formation of the neonatal tool network. These results suggest that an intrinsic brain network potentially supporting tool processing exists in the human brain prior to individual tool use experiences, and that the premotor-parietal functional connection in particular offers a brain basis for complex tool behaviors specific to humans.

Idiosyncratic Tower of Babel: Individual Differences in Word-Meaning Representation Increase as Word Abstractness Increases.

Humans primarily rely on language to communicate, on the basis of a shared understanding of the basic building blocks of communication: words. Do we mean the same things when we use the same words? Although cognitive neural research on semantics has revealed the common principles of word-meaning representation, the factors underlying the potential individual variations in word meanings are unknown. Here, we empirically characterized the intersubject consistency of 90 words across 20 adult subjects (10 female) using both behavioral measures (rating-based semantic-relationship patterns) and neuroimaging measures (word-evoked brain activity patterns). Across both the behavioral and neuroimaging experiments, we showed that the magnitude of individual disagreements on word meanings could be modeled on the basis of how much language or sensory experience is associated with a word and that this variation increases with word abstractness. Uncovering the cognitive and neural origins of word-meaning disagreements across individuals has implications for potential mechanisms to modulate such disagreements.

Semantic representation in the white matter pathway.

Object conceptual processing has been localized to distributed cortical regions that represent specific attributes. A challenging question is how object semantic space is formed. We tested a novel framework of representing semantic space in the pattern of white matter (WM) connections by extending the representational similarity analysis (RSA) to structural lesion pattern and behavioral data in 80 brain-damaged patients. For each WM connection, a neural representational dissimilarity matrix (RDM) was computed by first building machine-learning models with the voxel-wise WM lesion patterns as features to predict naming performance of a particular item and then computing the correlation between the predicted naming score and the actual naming score of another item in the testing patients. This correlation was used to build the neural RDM based on the assumption that if the connection pattern contains certain aspects of information shared by the naming processes of these two items, models trained with one item should also predict naming accuracy of the other. Correlating the neural RDM with various cognitive RDMs revealed that neural patterns in several WM connections that connect left occipital/middle temporal regions and anterior temporal regions associated with the object semantic space. Such associations were not attributable to modality-specific attributes (shape, manipulation, color, and motion), to peripheral picture-naming processes (picture visual similarity, phonological similarity), to broad semantic categories, or to the properties of the cortical regions that they connected, which tended to represent multiple modality-specific attributes. That is, the semantic space could be represented through WM connection patterns across cortical regions representing modality-specific attributes.

Doctor, Teacher, and Stethoscope: Neural Representation of Different Types of Semantic Relations.

Concepts can be related in many ways. They can belong to the same taxonomic category (e.g., "doctor" and "teacher," both in the category of people) or be associated with the same event context (e.g., "doctor" and "stethoscope," both associated with medical scenarios). How are these two major types of semantic relations coded in the brain? We constructed stimuli from three taxonomic categories (people, manmade objects, and locations) and three thematic categories (school, medicine, and sports) and investigated the neural representations of these two dimensions using representational similarity analyses in human participants (10 men and nine women). In specific regions of interest, the left anterior temporal lobe (ATL) and the left temporoparietal junction (TPJ), we found that, whereas both areas had significant effects of taxonomic information, the taxonomic relations had stronger effects in the ATL than in the TPJ ("doctor" and "teacher" closer in ATL neural activity), with the reverse being true for thematic relations ("doctor" and "stethoscope" closer in TPJ neural activity). A whole-brain searchlight analysis revealed that widely distributed regions, mainly in the left hemisphere, represented the taxonomic dimension. Interestingly, the significant effects of the thematic relations were only observed after the taxonomic differences were controlled for in the left TPJ, the right superior lateral occipital cortex, and other frontal, temporal, and parietal regions. In summary, taxonomic grouping is a primary organizational dimension across distributed brain regions, with thematic grouping further embedded within such taxonomic structures.SIGNIFICANCE STATEMENT How are concepts organized in the brain? It is well established that concepts belonging to the same taxonomic categories (e.g., "doctor" and "teacher") share neural representations in specific brain regions. How concepts are associated in other manners (e.g., "doctor" and "stethoscope," which are thematically related) remains poorly understood. We used representational similarity analyses to unravel the neural representations of these different types of semantic relations by testing the same set of words that could be differently grouped by taxonomic categories or by thematic categories. We found that widely distributed brain areas primarily represented taxonomic categories, with the thematic categories further embedded within the taxonomic structure.

Close yet independent: Dissociation of social from valence and abstract semantic dimensions in the left anterior temporal lobe.

The anterior temporal lobe (ATL) is engaged in various types of semantic dimensions. One consistently reported dimension is social information, with abstract words describing social behaviors inducing stronger activations in the ATL than nonsocial words. One potential factor that has been systematically confounded in this finding is emotional valence, given that abstract social words tend to be associated with emotional feelings. We investigated which factors drove the ATL sensitivity using a 2 (social/nonsocial) × 2 (valenced/neutral) factorial design in an fMRI study with relatively high spatial resolutions. We found that sociality and valence were processed in different ATL regions without significant interactions: The social effect was found in the left anterior superior temporal sulcus (aSTS), whereas the valence effect activated small clusters in the bilateral temporal poles (TP). In the left ATL, the social- and valence-related clusters were distinct from another superior ATL area that exhibited a general "abstractness" effect with little modulation of sociality or valence. These subregions exhibited distinct whole-brain functional connectivity patterns during the resting state, with the social cluster functionally connected to the default mode network, the valence cluster connected to the adjacent temporal regions and amygdala, and the abstractness cluster connected to a distributed network including a set of language-related regions. These results of activation profiles and connectivity patterns together indicate that the way in which the left ATL supports semantic processing is highly fine-grained, with the neural substrate for social semantic effects dissociated from those for emotional valence and abstractness.

Organizational Principles of Abstract Words in the Human Brain.

words constitute nearly half of the human lexicon and are critically associated with human abstract thoughts, yet little is known about how they are represented in the brain. We tested the neural basis of 2 classical cognitive notions of abstract meaning representation: by linguistic contexts and by semantic features. We collected fMRI BOLD responses for 360 abstract words and built theoretical representational models from state-of-the-art corpus-based natural language processing models and behavioral ratings of semantic features. Representational similarity analyses revealed that both linguistic contextual and semantic feature similarity affected the representation of abstract concepts, but in distinct neural levels. The corpus-based similarity was coded in the high-level linguistic processing system, whereas semantic feature information was reflected in distributed brain regions and in the principal component space derived from whole-brain activation patterns. These findings highlight the multidimensional organization and the neural dissociation between linguistic contextual and featural aspects of abstract concepts.

[The cognitive and neural bases of abstract concepts].

Words denoting abstract concepts constitute nearly half of human lexicon and serve as building blocks of the human culture. Since the advent of non-invasive neuroimaging techniques, great progress has been made in revealing the neurobiological foundation of concrete object and action concepts, yet it remains unclear how abstract concepts are stored and processed in the brain. Here we review recent development in this field, focusing on both theoretical perspectives and neuroimaging findings. We found that abstract concepts can be represented via linguistic and experiential information; the neural correlates of abstract concepts are partly in line with such a theoretical framework. Future studies are warranted to uncover the cognitive and neural mechanisms of language and experience in abstract word representation, which will help to deepen our understanding of general computational principles of the human conceptual system and to promote the development of the brain-like artificial intelligence.

Domain Selectivity in the Parahippocampal Gyrus Is Predicted by the Same Structural Connectivity Patterns in Blind and Sighted Individuals.

Human ventral occipital temporal cortex contains clusters of neurons that show domain-preferring responses during visual perception. Recent studies have reported that some of these clusters show surprisingly similar domain selectivity in congenitally blind participants performing nonvisual tasks. An important open question is whether these functional similarities are driven by similar innate connections in blind and sighted groups. Here we addressed this question focusing on the parahippocampal gyrus (PHG), a region that is selective for large objects and scenes. Based on the assumption that patterns of long-range connectivity shape local computation, we examined whether domain selectivity in PHG is driven by similar structural connectivity patterns in the two populations. Multiple regression models were built to predict the selectivity of PHG voxels for large human-made objects from white matter (WM) connectivity patterns in both groups. These models were then tested using independent data from participants with similar visual experience (two sighted groups) and using data from participants with different visual experience (blind and sighted groups). Strikingly, the WM-based predictions between blind and sighted groups were as successful as predictions between two independent sighted groups. That is, the functional selectivity for large objects of a PHG voxel in a blind participant could be accurately predicted by its WM pattern using the connection-to-function model built from the sighted group data, and vice versa. Regions that significantly predicted PHG selectivity were located in temporal and frontal cortices in both sighted and blind populations. These results show that the large-scale network driving domain selectivity in PHG is independent of vision.SIGNIFICANCE STATEMENT Recent studies have reported intriguingly similar domain selectivity in sighted and congenitally blind individuals in regions within the ventral visual cortex. To examine whether these similarities originate from similar innate connectional roots, we investigated whether the domain selectivity in one population could be predicted by the structural connectivity pattern of the other. We found that the selectivity for large objects of a PHG voxel in a blind participant could be predicted by its structural connectivity pattern using the connection-to-function model built from the sighted group data, and vice versa. These results reveal that the structural connectivity underlying domain selectivity in the PHG is independent of visual experience, providing evidence for nonvisual representations in this region.

Connectivity of the ventral visual cortex is necessary for object recognition in patients.

The functional profiles of regions in the ventral occipital-temporal cortex (VTC), a critical region for object visual recognition, are associated with the VTC connectivity patterns to nonvisual regions relevant to the corresponding object domain. However, whether and how whole-brain connections affect recognition behavior remains untested. We directly examined the necessity of VTC connectivity in object recognition behavior by testing 82 patients whose lesion spared relevant VTC regions but affected various white matter (WM) tracts and other regions. In these patients, we extracted the whole-brain anatomical connections of two VTC domain-selective (large manmade objects and animals) clusters with probabilistic tractography, and examined whether such connectivity pattern can predict recognition performance of the corresponding domains with support vector regression (SVR) analysis. We found that the whole-brain anatomical connectivity of large manmade object-specific cluster successfully predicted patients' large object recognition performance but not animal recognition or control tasks, even after we excluded connections with early visual regions. The contributing connections to large object recognition included tracts between VTC-large object cluster and distributed regions both within and beyond the visual cortex (e.g., putamen, superior, and middle temporal gyrus). These results provide causal evidence that the VTC whole-brain anatomical connectivity is necessary for at least certain domains of object recognition behavior.

Intrinsic Brain Hub Connectivity Underlies Individual Differences in Spatial Working Memory.

Spatial working memory (SWM) is an important component of working memory and plays an essential role in driving high-level cognitive abilities. Recent studies have demonstrated that individual SWM is associated with global brain communication. However, whether specific network nodal connectivity, such as brain hub connectivity, is involved in individual SWM performances remains largely unknown. Here, we collected resting-state fMRI (R-fMRI) data from a large group of 130 young healthy participants and evaluated their SWM performances. A voxel-wise whole-brain network analysis approach was employed to study the relationship between the nodal functional connectivity strength (FCS) and the SWM behavioral scores and to further estimate the participation of brain hubs in individual SWM. We showed significant associations between nodal FCS and SWM performance primarily in the default mode, visual, dorsal attention, and fronto-parietal systems. Moreover, over 41% of these nodal regions were identified as brain network hubs, and these hubs' FCS values contributed to 57% of the variance of the individual SWM performances that all SWM-related regions could explain. Collectively, our findings highlight the cognitive significance of the brain network hubs in SWM, which furthers our understanding of how intrinsic brain network architectures underlie individual differences in SWM processing.