Vision matters for shape representation: Evidence from sculpturing and drawing in the blind.

Shape is a property that could be perceived by vision and touch, and is classically considered to be supramodal. While there is mounting evidence for the shared cognitive and neural representation space between visual and tactile shape, previous research tended to rely on dissimilarity structures between objects and had not examined the detailed properties of shape representation in the absence of vision. To address this gap, we conducted three explicit object shape knowledge production experiments with congenitally blind and sighted participants, who were asked to produce verbal features, 3D clay models, and 2D drawings of familiar objects with varying levels of tactile exposure, including tools, large nonmanipulable objects, and animals. We found that the absence of visual experience (i.e., in the blind group) led to stronger differences in animals than in tools and large objects, suggesting that direct tactile experience of objects is essential for shape representation when vision is unavailable. For tools with rich tactile/manipulation experiences, the blind produced overall good shapes comparable to the sighted, yet also showed intriguing differences. The blind group had more variations and a systematic bias in the geometric property of tools (making them stubbier than the sighted), indicating that visual experience contributes to aligning internal representations and calibrating overall object configurations, at least for tools. Taken together, the object shape representation reflects the intricate orchestration of vision, touch and language.

Neural correlates of an illusionary sense of agency caused by virtual reality.

Sense of agency (SoA) is the sensation that self-actions lead to ensuing perceptual consequences. The prospective mechanism emphasizes that SoA arises from motor prediction and its comparison with actual action outcomes, while the reconstructive mechanism stresses that SoA emerges from retrospective causal processing about the action outcomes. Consistent with the prospective mechanism, motor planning regions were identified by neuroimaging studies using the temporal binding (TB) effect, a behavioral measure often linked to implicit SoA. Yet, TB also occurs during passive observation of another's action, lending support to the reconstructive mechanism, but its neural correlates remain unexplored. Here, we employed virtual reality (VR) to modulate such observation-based SoA and examined it with functional magnetic resonance imaging (fMRI). After manipulating an avatar hand in VR, participants passively observed an avatar's "action" and showed a significant increase in TB. The binding effect was associated with the right angular gyrus and inferior parietal lobule, which are critical nodes for inferential and agency processing. These results suggest that the experience of controlling an avatar may potentiate inferential processing within the right inferior parietal cortex and give rise to the illusionary SoA without voluntary action.

Separable amygdala activation patterns in the evaluations of robots.

Given the increasing presence of robots in everyday environments and the significant challenge posed by social interactions with robots, it is crucial to gain a deeper understanding into the social evaluations of robots. One potentially effective approach to comprehend the fundamental processes underlying controlled and automatic evaluations of robots is to probe brain response to different perception levels of robot-related stimuli. Here, we investigate controlled and automatic evaluations of robots based on brain responses during viewing of suprathreshold (duration: 200 ms) and subthreshold (duration: 17 ms) humanoid robot stimuli. Our behavioral analysis revealed that despite participants' self-reported positive attitudes, they held negative implicit attitudes toward humanoid robots. Neuroimaging analysis indicated that subthreshold presentation of humanoid robot stimuli elicited significant activation in the left amygdala, which was associated with negative implicit attitudes. Conversely, no significant left amygdala activation was observed during suprathreshold presentation. Following successful attenuation of negative attitudes, the left amygdala response to subthreshold presentation of humanoid robot stimuli decreased, and this decrease correlated positively with the reduction in negative attitudes. These findings provide evidence for separable patterns of amygdala activation between controlled and automatic processing of robots, suggesting that controlled evaluations may influence automatic evaluations of robots.

Neuroplastic changes in functional wiring in sensory cortices of the congenitally deaf: A network analysis.

Congenital sensory deprivation induces significant changes in the structural and functional organisation of the brain. These are well-characterised by cross-modal plasticity, in which deprived cortical areas are recruited to process information from non-affected sensory modalities, as well as by other neuroplastic alterations within regions dedicated to the remaining senses. Here, we analysed visual and auditory networks of congenitally deaf and hearing individuals during different visual tasks to assess changes in network community structure and connectivity patterns due to congenital deafness. In the hearing group, the nodes are clearly divided into three communities (visual, auditory and subcortical), whereas in the deaf group a fourth community consisting mainly of bilateral superior temporal sulcus and temporo-insular regions is present. Perhaps more importantly, the right lateral geniculate body, as well as bilateral thalamus and pulvinar joined the auditory community of the deaf. Moreover, there is stronger connectivity between bilateral thalamic and pulvinar and auditory areas in the deaf group, when compared to the hearing group. No differences were found in the number of connections of these nodes to visual areas. Our findings reveal substantial neuroplastic changes occurring within the auditory and visual networks caused by deafness, emphasising the dynamic nature of the sensory systems in response to congenital deafness. Specifically, these results indicate that in the deaf but not the hearing group, subcortical thalamic nuclei are highly connected to auditory areas during processing of visual information, suggesting that these relay areas may be responsible for rerouting visual information to the auditory cortex under congenital deafness.

Functional connectome through the human life span.

The functional connectome of the human brain represents the fundamental network architecture of functional interdependence in brain activity, but its normative growth trajectory across the life course remains unknown. Here, we aggregate the largest, quality-controlled multimodal neuroimaging dataset from 119 global sites, including 33,809 task-free fMRI and structural MRI scans from 32,328 individuals ranging in age from 32 postmenstrual weeks to 80 years. Lifespan growth charts of the connectome are quantified at the whole cortex, system, and regional levels using generalized additive models for location, scale, and shape. We report critical inflection points in the non-linear growth trajectories of the whole-brain functional connectome, particularly peaking in the fourth decade of life. Having established the first fine-grained, lifespan-spanning suite of system-level brain atlases, we generate person-specific parcellation maps and further show distinct maturation timelines for functional segregation within different subsystems. We identify a spatiotemporal gradient axis that governs the life-course growth of regional connectivity, transitioning from primary sensory cortices to higher-order association regions. Using the connectome-based normative model, we demonstrate substantial individual heterogeneities at the network level in patients with autism spectrum disorder and patients with major depressive disorder. Our findings shed light on the life-course evolution of the functional connectome and serve as a normative reference for quantifying individual variation in patients with neurological and psychiatric disorders.

A social-semantic working-memory account for two canonical language areas.

Language and social cognition are traditionally studied as separate cognitive domains, yet accumulative studies reveal overlapping neural correlates at the left ventral temporoparietal junction (vTPJ) and the left lateral anterior temporal lobe (lATL), which have been attributed to sentence processing and social concept activation. We propose a common cognitive component underlying both effects: social-semantic working memory. We confirmed two key predictions of our hypothesis using functional MRI. First, the left vTPJ and lATL showed sensitivity to sentences only when the sentences conveyed social meaning; second, these regions showed persistent social-semantic-selective activity after the linguistic stimuli disappeared. We additionally found that both regions were sensitive to the socialness of non-linguistic stimuli and were more tightly connected with the social-semantic-processing areas than with the sentence-processing areas. The converging evidence indicates the social-semantic working-memory function of the left vTPJ and lATL and challenges the general-semantic and/or syntactic accounts for the neural activity of these regions.

Spoken language processing activates the primary visual cortex.

Primary visual cortex (V1) is generally thought of as a low-level sensory area that primarily processes basic visual features. Although there is evidence for multisensory effects on its activity, these are typically found for the processing of simple sounds and their properties, for example spatially or temporally-congruent simple sounds. However, in congenitally blind individuals, V1 is involved in language processing, with no evidence of major changes in anatomical connectivity that could explain this seemingly drastic functional change. This is at odds with current accounts of neural plasticity, which emphasize the role of connectivity and conserved function in determining a neural tissue's role even after atypical early experiences. To reconcile what appears to be unprecedented functional reorganization with known accounts of plasticity limitations, we tested whether V1's multisensory roles include responses to spoken language in sighted individuals. Using fMRI, we found that V1 in normally sighted individuals was indeed activated by comprehensible spoken sentences as compared to an incomprehensible reversed speech control condition, and more strongly so in the left compared to the right hemisphere. Activation in V1 for language was also significant and comparable for abstract and concrete words, suggesting it was not driven by visual imagery. Last, this activation did not stem from increased attention to the auditory onset of words, nor was it correlated with attentional arousal ratings, making general attention accounts an unlikely explanation. Together these findings suggest that V1 responds to spoken language even in sighted individuals, reflecting the binding of multisensory high-level signals, potentially to predict visual input. This capability might be the basis for the strong V1 language activation observed in people born blind, re-affirming the notion that plasticity is guided by pre-existing connectivity and abilities in the typically developed brain.

Neural substrate underlying the learning of a passage with unfamiliar vocabulary and syntax.

Speech comprehension is a complex process involving multiple stages, such as decoding of phonetic units, recognizing words, and understanding sentences and passages. In this study, we identify cortical networks beyond basic phonetic processing using a novel passage learning paradigm. Participants learn to comprehend a story composed of syllables of their native language, but containing unfamiliar vocabulary and syntax. Three learning methods are employed, each resulting in some degree of learning within a 12-min learning session. Functional magnetic resonance imaging results reveal that, when listening to the same story, the classic temporal-frontal language network is significantly enhanced by learning. Critically, activation of the left anterior and posterior temporal lobe correlates with the learning outcome that is assessed behaviorally through, e.g. word recognition and passage comprehension tests. This study demonstrates that a brief learning session is sufficient to induce neural plasticity in the left temporal lobe, which underlies the transformation from phonetic units to the units of meaning, such as words and sentences.

Simple shape feature computation across modalities: convergence and divergence between the ventral and dorsal visual streams.

Shape processing, whether by seeing or touching, is pivotal to object recognition and manipulation. Although the low-level signals are initially processed by different modality-specific neural circuits, multimodal responses to object shapes have been reported along both ventral and dorsal visual pathways. To understand this transitional process, we conducted visual and haptic shape perception fMRI experiments to test basic shape features (i.e. curvature and rectilinear) across the visual pathways. Using a combination of region-of-interest-based support vector machine decoding analysis and voxel selection method, we found that the top visual-discriminative voxels in the left occipital cortex (OC) could also classify haptic shape features, and the top haptic-discriminative voxels in the left posterior parietal cortex (PPC) could also classify visual shape features. Furthermore, these voxels could decode shape features in a cross-modal manner, suggesting shared neural computation across visual and haptic modalities. In the univariate analysis, the top haptic-discriminative voxels in the left PPC showed haptic rectilinear feature preference, whereas the top visual-discriminative voxels in the left OC showed no significant shape feature preference in either of the two modalities. Together, these results suggest that mid-level shape features are represented in a modality-independent manner in both the ventral and dorsal streams.

Early language exposure affects neural mechanisms of semantic representations.

One signature of the human brain is its ability to derive knowledge from language inputs, in addition to nonlinguistic sensory channels such as vision and touch. How does human language experience modulate the mechanism by which semantic knowledge is stored in the human brain? We investigated this question using a unique human model with varying amounts and qualities of early language exposure: early deaf adults who were born to hearing parents and had reduced early exposure and delayed acquisition of any natural human language (speech or sign), with early deaf adults who acquired sign language from birth as the control group that matches on nonlinguistic sensory experiences. Neural responses in a semantic judgment task with 90 written words that were familiar to both groups were measured using fMRI. The deaf group with reduced early language exposure, compared with the deaf control group, showed reduced semantic sensitivity, in both multivariate pattern (semantic structure encoding) and univariate (abstractness effect) analyses, in the left dorsal anterior temporal lobe (dATL). These results provide positive, causal evidence that language experience drives the neural semantic representation in the dATL, highlighting the roles of language in forming human neural semantic structures beyond nonverbal sensory experiences.

Humans are the only known species where much of knowledge learning happens symbolically through language, in addition to information received directly from the senses. For example, humans can learn about the color of some rose flowers from the popular expression "roses are red" without needing to see any red roses ­ allowing them to accumulate knowledge beyond the constraints of their own senses. Recent work suggests that a region of the brain known as the dorsal anterior temporal lobe represents knowledge acquired from language instead of sensory experiences. However, these studies were based on volunteers deprived of sensory experiences rather than those with reduced language exposure. Therefore, it was not clear whether this brain structure represents knowledge derived specifically from language and the importance of language in shaping non-sensory knowledge. To address this question, Wang et al. studied the brain activity of deaf adult volunteers in a word meaning judgement task. Volunteers were either born deaf or lost their hearing as toddlers, and all primarily used Chinese Sign Language for communication. One group of volunteers had been exposed to sign language from birth, giving them similar exposure to language as hearing individuals. The other group had less exposure to language in their early years and only learned sign language later in childhood. The task included 90 written words that were familiar to the volunteers. They included a mixture of object words ­ related to material objects ­ such as "shoulder" and "hammer" and abstract words ­ which are not linked to physical objects ­ such as "cause" and "violence". The volunteers were shown each word in turn and asked to think about the word's meaning. Brain scans revealed that the left dorsal anterior temporal lobes of the volunteers with reduced early language exposure were less sensitive to the meaning of the words compared with those of the other volunteers. The findings demonstrate that the dorsal anterior temporal lobe specifically supports meaning derived from a person's experience of language as opposed to sensory experience, providing a new angle to understand the mechanism of knowledge representations. Increased understanding of how language supports knowledge will help to uncover the human-specific ways of representing and creating knowledge in the brain.

Gravity matters for the neural representations of action semantics.

The dynamic relationship between the neural representation of action word semantics and specific sensorimotor experience remains controversial. Here, we temporarily altered human subjects' sensorimotor experience in a 15-day head-down tilt bed rest setting, a ground-based analog of microgravity that disproportionally affects sensorimotor experiences of the lower limbs, and examined whether such effector-dependent activity deprivation specifically affected the neural processes of comprehending verbs of lower-limb actions (e.g. to kick) relative to upper-limb ones (e.g. to pinch). Using functional magnetic resonance imaging, we compared the multivoxel neural patterns for such action words prior to and after bed rest. We found an effector-specific (lower vs. upper limb) experience modulation in subcortical sensorimotor-related and anterior temporal regions. The neural action semantic representations in other effector-specific verb semantic regions (e.g. left lateral posterior temporal cortex) and motor execution regions were robust against such experience alterations. These effector-specific, sensorimotor-experience-sensitive and experience-independent patterns of verb neural representation highlight the multidimensional and dynamic nature of semantic neural representation, and the broad influence of microgravity (hence gravity) environment on cognition.

Pulvinar Response Profiles and Connectivity Patterns to Object Domains.

Distributed cortical regions show differential responses to visual objects belonging to different domains varying by animacy (e.g., animals vs tools), yet it remains unclear whether this is an organization principle also applying to the subcortical structures. Combining multiple fMRI activation experiments (two main experiments and six validation datasets; 12 females and 9 males in the main Experiment 1; 10 females and 10 males in the main Experiment 2), resting-state functional connectivity, and task-based dynamic causal modeling analysis in human subjects, we found that visual processing of images of animals and tools elicited different patterns of response in the pulvinar, with robust left lateralization for tools, and distinct, bilateral (with rightward tendency) clusters for animals. Such domain-preferring activity distribution in the pulvinar was associated with the magnitude with which the voxels were intrinsically connected with the corresponding domain-preferring regions in the cortex. The pulvinar-to-right-amygdala path showed a one-way shortcut supporting the perception of animals, and the modulation connection from pulvinar to parietal showed an advantage to the perception of tools. These results incorporate the subcortical regions into the object processing network and highlight that domain organization appears to be an overarching principle across various processing stages in the brain.SIGNIFICANCE STATEMENT Viewing objects belonging to different domains elicited different cortical regions, but whether the domain organization applied to the subcortical structures (e.g., pulvinar) was unknown. Multiple fMRI activation experiments revealed that object pictures belonging to different domains elicited differential patterns of response in the pulvinar, with robust left lateralization for tool pictures, and distinct, bilateral (with rightward tendency) clusters for animals. Combining the resting-state functional connectivity and dynamic causal modeling analysis on task-based fMRI data, we found domain-preferring activity distribution in the pulvinar aligned with that in cortical regions. These results highlight the need for coherent visual theories that explain the mechanisms underlying the domain organization across various processing stages.

Different computational relations in language are captured by distinct brain systems.

A critical way for humans to acquire information is through language, yet whether and how language experience drives specific neural semantic representations is still poorly understood. We considered statistical properties captured by 3 different computational principles of language (simple co-occurrence, network-(graph)-topological relations, and neural-network-vector-embedding relations) and tested the extent to which they can explain the neural patterns of semantic representations, measured by 2 functional magnetic resonance imaging experiments that shared common semantic processes. Distinct graph-topological word relations, and not simple co-occurrence or neural-network-vector-embedding relations, had unique explanatory power for the neural patterns in the anterior temporal lobe (capturing graph-common-neighbors), inferior frontal gyrus, and posterior middle/inferior temporal gyrus (capturing graph-shortest-path). These results were relatively specific to language: they were not explained by sensory-motor similarities and the same computational relations of visual objects (based on visual image database) showed effects in the visual cortex in the picture naming experiment. That is, different topological properties within language and the same topological computations (common-neighbors) for language and visual inputs are captured by different brain regions. These findings reveal the specific neural semantic representations along graph-topological properties of language, highlighting the information type-specific and statistical property-specific manner of semantic representations in the human brain.

Brain intrinsic connection patterns underlying tool processing in human adults are present in neonates and not in macaques.

Tool understanding and use are supported by a dedicated left-lateralized, intrinsically connected network in the human adult brain. To examine this network's phylogenetic and ontogenetic origins, we compared resting-state functional connectivity (rsFC) among regions subserving tool processing in human adults to rsFC among homologous regions in human neonates and macaque monkeys (adolescent and mature). These homologous regions formed an intrinsic network in human neonates, but not in macaques. Network topological patterns were highly similar between human adults and neonates, and significantly less so between humans and macaques. The premotor-parietal rsFC had most significant contribution to the formation of the neonatal tool network. These results suggest that an intrinsic brain network potentially supporting tool processing exists in the human brain prior to individual tool use experiences, and that the premotor-parietal functional connection in particular offers a brain basis for complex tool behaviors specific to humans.

The Role of Visual Experience in Individual Differences of Brain Connectivity.

Visual cortex organization is highly consistent across individuals. But to what degree does this consistency depend on life experience, in particular sensory experience? In this study, we asked whether visual cortex reorganization in congenital blindness results in connectivity patterns that are particularly variable across individuals, focusing on resting-state functional connectivity (RSFC) patterns from the primary visual cortex. We show that the absence of shared visual experience results in more variable RSFC patterns across blind individuals than sighted controls. Increased variability is specifically found in areas that show a group difference between the blind and sighted in their RSFC. These findings reveal a relationship between brain plasticity and individual variability; reorganization manifests variably across individuals. We further investigated the different patterns of reorganization in the blind, showing that the connectivity to frontal regions, proposed to have a role in the reorganization of the visual cortex of the blind toward higher cognitive roles, is highly variable. Further, we link some of the variability in visual-to-frontal connectivity to another environmental factor-duration of formal education. Together, these findings show a role of postnatal sensory and socioeconomic experience in imposing consistency on brain organization. By revealing the idiosyncratic nature of neural reorganization, these findings highlight the importance of considering individual differences in fitting sensory aids and restoration approaches for vision loss.SIGNIFICANCE STATEMENT The typical visual system is highly consistent across individuals. What are the origins of this consistency? Comparing the consistency of visual cortex connectivity between people born blind and sighted people, we showed that blindness results in higher variability, suggesting a key impact of postnatal individual experience on brain organization. Further, connectivity patterns that changed following blindness were particularly variable, resulting in diverse patterns of brain reorganization. Individual differences in reorganization were also directly affected by nonvisual experiences in the blind (years of formal education). Together, these findings show a role of sensory and socioeconomic experiences in creating individual differences in brain organization and endorse the use of individual profiles for rehabilitation and restoration of vision loss.

From words to phrases: neural basis of social event semantic composition.

Events are typically composed of at least actions and entities. Both actions and entities have been shown to be represented by neural structures respecting domain organizations in the brain, including those of social/animate (face and body; person-directed action) versus inanimate (man-made object or tool; object-directed action) concepts. It is unclear whether the brain combines actions and entities into events in a (relative) domain-specific fashion or via domain-general mechanisms in regions that have been shown to support semantic and syntactic composition. We tested these hypotheses in a functional magnetic resonance imaging experiment where two domains of verb-noun event phrases (social-person versus manipulation-artifact, e.g., "hug mother" versus "fold napkin") and their component words were contrasted. We found a series of brain region supporting social-composition effects more strongly than the manipulation phrase composition-the bilateral inferior occipital gyrus (IOG), inferior temporal gyrus (ITG) and anterior temporal lobe (ATL)-which either showed stronger activation strength tested by univariate contrast, stronger content representation tested by representation similarity analysis, or stronger relationship between the neural activation patterns of phrases and synthesis (additive and multiplication) of the neural activity patterns of the word constituents. No regions were observed showing evidence of phrase composition for both domains or stronger effects of manipulation phrases. These findings highlight the roles of the visual cortex and ATL in social event compositions, suggesting a domain-preferring, rather than domain-general, mechanisms of verbal event composition.

Preference for animate domain sounds in the fusiform gyrus of blind individuals is modulated by shape-action mapping.

In high-level visual areas in the human brain, preference for inanimate objects is observed regardless of stimulation modality (visual/auditory/tactile) and individual's visual experience (sighted/blind) whereas preference for animate entities seems robust mainly in the visual modality. Here, we test a hypothesis explaining this domain difference: Object representations can be activated through nonvisual stimulation when their shapes are systematically related to action system representations, a quality typical of most inanimate objects but of only specific animate entities. We studied functional magnetic resonance imaging activations in congenitally blind and sighted individuals listening to animal, object, and human sounds. In blind individuals, the typical location of the fusiform face area preferentially responded to human facial expression sounds clearly related to specific facial actions and resulting face shapes but not to speech or animal sounds. No univariate preference for any sound category was observed in the fusiform gyrus in sighted individuals, but the expected multivoxel effects were present. We conclude that nonvisual signals can activate shape representations of those stimuli-inanimate or animate-for which shape and action computations are transparently related. However, absence of potentially competing visual inputs seems necessary for this effect to be clearly detectable in the case of animate representation.

Social and emotion dimensional organizations in the abstract semantic space: the neuropsychological evidence.

An essential aspect of human cognition is supported by a rich reservoir of abstract concepts without tangible external referents (e.g., "honor", "relationship", "direction"). While decades of research showed that the neural organization of conceptual knowledge referring to concrete words respects domains of evolutionary salience and sensorimotor attributes, the organization principles of abstract word meanings are poorly understood. Here, we provide neuropsychological evidence for a domain (sociality) and attribute (emotion) structure in abstract word processing. Testing 34 brain-damaged patients on a word-semantic judgment task, we observed double dissociations between social and nonsocial words and a single dissociation of sparing of emotional (relative to non-emotional) words. The lesion profiles of patients with specific dissociations suggest potential neural correlates positively or negatively associated with each dimension. These results unravel a general domain-attribute architecture of word meanings and highlight the roles of the social domain and the emotional attribute in the non-object semantic space.

Dual coding of knowledge in the human brain.

How does the human brain code knowledge about the world? While disciplines such as artificial intelligence represent world knowledge based on human language, neurocognitive models of knowledge have been dominated by sensory embodiment, in which knowledge is derived from sensory/motor experience and supported by high-level sensory/motor and association cortices. The neural correlates of an alternative disembodied symbolic system had previously been difficult to establish. A recent line of studies exploring knowledge about visual properties, such as color, in visually deprived individuals converge to provide positive, compelling evidence for non-sensory, language-derived, knowledge representation in dorsal anterior temporal lobe and extended language network, in addition to the sensory-derived representations, leading to a sketch of a dual-coding knowledge neural framework.

Idiosyncratic Tower of Babel: Individual Differences in Word-Meaning Representation Increase as Word Abstractness Increases.

Humans primarily rely on language to communicate, on the basis of a shared understanding of the basic building blocks of communication: words. Do we mean the same things when we use the same words? Although cognitive neural research on semantics has revealed the common principles of word-meaning representation, the factors underlying the potential individual variations in word meanings are unknown. Here, we empirically characterized the intersubject consistency of 90 words across 20 adult subjects (10 female) using both behavioral measures (rating-based semantic-relationship patterns) and neuroimaging measures (word-evoked brain activity patterns). Across both the behavioral and neuroimaging experiments, we showed that the magnitude of individual disagreements on word meanings could be modeled on the basis of how much language or sensory experience is associated with a word and that this variation increases with word abstractness. Uncovering the cognitive and neural origins of word-meaning disagreements across individuals has implications for potential mechanisms to modulate such disagreements.