The Role of Agentive and Physical Forces in the Neural Representation of Motion Events.

How does the brain represent information about motion events in relation to agentive and physical forces? In this study, we investigated the neural activity patterns associated with observing animated actions of agents (e.g., an agent hitting a chair) in comparison to similar movements of inanimate objects that were either shaped solely by the physics of the scene (e.g., gravity causing an object to fall down a hill and hit a chair) or initiated by agents (e.g., a visible agent causing an object to hit a chair). Using an fMRI-based multivariate pattern analysis (MVPA), this design allowed testing where in the brain the neural activity patterns associated with motion events change as a function of, or are invariant to, agentive versus physical forces behind them. A total of 29 human participants (nine male) participated in the study. Cross-decoding revealed a shared neural representation of animate and inanimate motion events that is invariant to agentive or physical forces in regions spanning frontoparietal and posterior temporal cortices. In contrast, the right lateral occipitotemporal cortex showed a higher sensitivity to agentive events, while the left dorsal premotor cortex was more sensitive to information about inanimate object events that were solely shaped by the physics of the scene.

From intermediate shape-centered representations to the perception of oriented shapes: response to commentaries.

In this response paper, we start by addressing the main points made by the commentators on the target article's main theoretical conclusions: the existence and characteristics of the intermediate shape-centered representations (ISCRs) in the visual system, their emergence from edge detection mechanisms operating on different types of visual properties, and how they are eventually reunited in higher order frames of reference underlying conscious visual perception. We also address the much-commented issue of the possible neural mechanisms of the ISCRs. In the final section, we address more specific and general comments, questions, and suggestions which, albeit very interesting, were less directly focused on the main conclusions of the target paper.

Preference for animate domain sounds in the fusiform gyrus of blind individuals is modulated by shape-action mapping.

In high-level visual areas in the human brain, preference for inanimate objects is observed regardless of stimulation modality (visual/auditory/tactile) and individual's visual experience (sighted/blind) whereas preference for animate entities seems robust mainly in the visual modality. Here, we test a hypothesis explaining this domain difference: Object representations can be activated through nonvisual stimulation when their shapes are systematically related to action system representations, a quality typical of most inanimate objects but of only specific animate entities. We studied functional magnetic resonance imaging activations in congenitally blind and sighted individuals listening to animal, object, and human sounds. In blind individuals, the typical location of the fusiform face area preferentially responded to human facial expression sounds clearly related to specific facial actions and resulting face shapes but not to speech or animal sounds. No univariate preference for any sound category was observed in the fusiform gyrus in sighted individuals, but the expected multivoxel effects were present. We conclude that nonvisual signals can activate shape representations of those stimuli-inanimate or animate-for which shape and action computations are transparently related. However, absence of potentially competing visual inputs seems necessary for this effect to be clearly detectable in the case of animate representation.

Evidence for an effector-independent action system from people born without hands.

Many parts of the visuomotor system guide daily hand actions, like reaching for and grasping objects. Do these regions depend exclusively on the hand as a specific body part whose movement they guide, or are they organized for the reaching task per se, for any body part used as an effector? To address this question, we conducted a neuroimaging study with people born without upper limbs-individuals with dysplasia-who use the feet to act, as they and typically developed controls performed reaching and grasping actions with their dominant effector. Individuals with dysplasia have no prior experience acting with hands, allowing us to control for hand motor imagery when acting with another effector (i.e., foot). Primary sensorimotor cortices showed selectivity for the hand in controls and foot in individuals with dysplasia. Importantly, we found a preference based on action type (reaching/grasping) regardless of the effector used in the association sensorimotor cortex, in the left intraparietal sulcus and dorsal premotor cortex, as well as in the basal ganglia and anterior cerebellum. These areas also showed differential response patterns between action types for both groups. Intermediate areas along a posterior-anterior gradient in the left dorsal premotor cortex gradually transitioned from selectivity based on the body part to selectivity based on the action type. These findings indicate that some visuomotor association areas are organized based on abstract action functions independent of specific sensorimotor parameters, paralleling sensory feature-independence in visual and auditory cortices in people born blind and deaf. Together, they suggest association cortices across action and perception may support specific computations, abstracted from low-level sensorimotor elements.

Embodied cognition and mirror neurons: a critical assessment.

According to embodied cognition theories, higher cognitive abilities depend on the reenactment of sensory and motor representations. In the first part of this review, we critically analyze the central claims of embodied theories and argue that the existing behavioral and neuroimaging data do not allow investigators to discriminate between embodied cognition and classical cognitive accounts, which assume that conceptual representations are amodal and symbolic. In the second part, we review the main claims and the core electrophysiological findings typically cited in support of the mirror neuron theory of action understanding, one of the most influential examples of embodied cognition theories. In the final part, we analyze the claim that mirror neurons subserve action understanding by mapping visual representations of observed actions on motor representations, trying to clarify in what sense the representations carried by these neurons can be claimed motor.

Shape-centered representations of bounded regions of space mediate the perception of objects.

We report the study of a woman who perceives 2D bounded regions of space ("shapes") defined by sharp edges of medium to high contrast as if they were rotated by 90, 180 degrees around their centre, mirrored across their own axes, or both. In contrast, her perception of 3D, strongly blurred or very low contrast shapes, and of stimuli emerging from a collection of shapes, is intact. This suggests that a stage in the process of constructing the conscious visual representation of a scene consists of representing mutually exclusive bounded regions extracted from the initial retinotopic space in "shape-centered" frames of reference. The selectivity of the disorder to shapes originally biased toward the parvocellular subcortical pathway, and the absence of any other type of error, additionally invite new hypotheses about the operations involved in computing these "intermediate shape-centered representations" and in mapping them onto higher frames for perception and action.

Typically Efficient Lipreading without Motor Simulation.

All it takes is a face-to-face conversation in a noisy environment to realize that viewing a speaker's lip movements contributes to speech comprehension. What are the processes underlying the perception and interpretation of visual speech? Brain areas that control speech production are also recruited during lipreading. This finding raises the possibility that lipreading may be supported, at least to some extent, by a covert unconscious imitation of the observed speech movements in the observer's own speech motor system-a motor simulation. However, whether, and if so to what extent, motor simulation contributes to visual speech interpretation remains unclear. In two experiments, we found that several participants with congenital facial paralysis were as good at lipreading as the control population and performed these tasks in a way that is qualitatively similar to the controls despite severely reduced or even completely absent lip motor representations. Although it remains an open question whether this conclusion generalizes to other experimental conditions and to typically developed participants, these findings considerably narrow the space of hypothesis for a role of motor simulation in lipreading. Beyond its theoretical significance in the field of speech perception, this finding also calls for a re-examination of the more general hypothesis that motor simulation underlies action perception and interpretation developed in the frameworks of motor simulation and mirror neuron hypotheses.

The contribution of object size, manipulability, and stability on neural responses to inanimate objects.

In human occipitotemporal cortex, brain responses to depicted inanimate objects have a large-scale organization by real-world object size. Critically, the size of objects in the world is systematically related to behaviorally-relevant properties: small objects are often grasped and manipulated (e.g., forks), while large objects tend to be less motor-relevant (e.g., tables), though this relationship does not always have to be true (e.g., picture frames and wheelbarrows). To determine how these two dimensions interact, we measured brain activity with functional magnetic resonance imaging while participants viewed a stimulus set of small and large objects with either low or high motor-relevance. The results revealed that the size organization was evident for objects with both low and high motor-relevance; further, a motor-relevance map was also evident across both large and small objects. Targeted contrasts revealed that typical combinations (small motor-relevant vs. large non-motor-relevant) yielded more robust topographies than the atypical covariance contrast (small non-motor-relevant vs. large motor-relevant). In subsequent exploratory analyses, a factor analysis revealed that the construct of motor-relevance was better explained by two underlying factors: one more related to manipulability, and the other to whether an object moves or is stable. The factor related to manipulability better explained responses in lateral small-object preferring regions, while the factor related to object stability (lack of movement) better explained responses in ventromedial large-object preferring regions. Taken together, these results reveal that the structure of neural responses to objects of different sizes further reflect behavior-relevant properties of manipulability and stability, and contribute to a deeper understanding of some of the factors that help the large-scale organization of object representation in high-level visual cortex.

Plasticity based on compensatory effector use in the association but not primary sensorimotor cortex of people born without hands.

What forces direct brain organization and its plasticity? When brain regions are deprived of their input, which regions reorganize based on compensation for the disability and experience, and which regions show topographically constrained plasticity? People born without hands activate their primary sensorimotor hand region while moving body parts used to compensate for this disability (e.g., their feet). This was taken to suggest a neural organization based on functions, such as performing manual-like dexterous actions, rather than on body parts, in primary sensorimotor cortex. We tested the selectivity for the compensatory body parts in the primary and association sensorimotor cortex of people born without hands (dysplasic individuals). Despite clear compensatory foot use, the primary sensorimotor hand area in the dysplasic subjects showed preference for adjacent body parts that are not compensatorily used as effectors. This suggests that function-based organization, proposed for congenital blindness and deafness, does not apply to the primary sensorimotor cortex deprivation in dysplasia. These findings stress the roles of neuroanatomical constraints like topographical proximity and connectivity in determining the functional development of primary cortex even in extreme, congenital deprivation. In contrast, increased and selective foot movement preference was found in dysplasics' association cortex in the inferior parietal lobule. This suggests that the typical motor selectivity of this region for manual actions may correspond to high-level action representations that are effector-invariant. These findings reveal limitations to compensatory plasticity and experience in modifying brain organization of early topographical cortex compared with association cortices driven by function-based organization.

Large-Scale Organization of the Hand Action Observation Network in Individuals Born Without Hands.

The human high-level visual cortex comprises regions specialized for the processing of distinct types of stimuli, such as objects, animals, and human actions. How does this specialization emerge? Here, we investigated the role of effector-specific visuomotor coupling experience in shaping the organization of the action observation network (AON) as a window on this question. Observed body movements are frequently coupled with corresponding motor codes, e.g., during monitoring one's own movements and imitation, resulting in bidirectionally connected circuits between areas involved in body movements observation (e.g., of the hand) and the motor codes involved in their execution. If the organization of the AON is shaped by this effector-specific visuomotor coupling, then, it should not form for body movements that do not belong to individuals' motor repertoire. To test this prediction, we used fMRI to investigate the spatial arrangement and functional properties of the hand and foot action observation circuits in individuals born without upper limbs. Multivoxel pattern decoding, pattern similarity, and univariate analyses revealed an intact hand AON in the individuals born without upper limbs. This suggests that the organization of the AON does not require effector-specific visuomotor coupling.

Sensorimotor-independent development of hands and tools selectivity in the visual cortex.

The visual occipito-temporal cortex is composed of several distinct regions specialized in the identification of different object kinds such as tools and bodies. Its organization appears to reflect not only the visual characteristics of the inputs but also the behavior that can be achieved with them. For example, there are spatially overlapping responses for viewing hands and tools, which is likely due to their common role in object-directed actions. How dependent is occipito-temporal cortex organization on object manipulation and motor experience? To investigate this question, we studied five individuals born without hands (individuals with upper limb dysplasia), who use tools with their feet. Using fMRI, we found the typical selective hand-tool overlap (HTO) not only in typically developed control participants but also in four of the five dysplasics. Functional connectivity of the HTO in the dysplasics also showed a largely similar pattern as in the controls. The preservation of functional organization in the dysplasics suggests that occipito-temporal cortex specialization is driven largely by inherited connectivity constraints that do not require sensorimotor experience. These findings complement discoveries of intact functional organization of the occipito-temporal cortex in people born blind, supporting an organization largely independent of any one specific sensory or motor experience.

Lateral occipitotemporal cortex encodes perceptual components of social actions rather than abstract representations of sociality.

Neuroimaging studies suggest that areas in the lateral occipitotemporal cortex (LOTC) play an important role in the perception of social actions. However, it is unclear what precisely about social actions these areas represent: perceptual features that may be indicative of social actions - such as the presence of persons in a scene, their orientation toward each other, and in particular the directedness of action movements toward persons or other targets - or more abstract representations that capture whether an action is meant to be social. In two fMRI experiments, we used representational similarity analysis (RSA) to test whether LOTC is sensitive to perceptual action components important for social interpretation and/or more general representations of sociality (Experiment 1) and implied person-directedness (Experiment 2). We found that LOTC is sensitive to perceptual action components (person presence, person orientation, and action directedness toward different types of recipients). By contrast, more general levels of sociality and implied person-directedness were not captured by LOTC. Our findings suggest that regions in LOTC provide the perceptual basis for social action interpretation but challenge accounts that posit specialization at more general levels sensitive to social actions and sociality as such. We propose that the interpretation of an action - in terms of sociality or other intentional aspects - arises from the interaction of multiple areas in processing relevant action components in a situation-dependent manner.

Directional tuning for eye and arm movements in overlapping regions in human posterior parietal cortex.

A network of frontal and parietal regions is known to be recruited during the planning and execution of arm and eye movements. While movements of the two effectors are typically coupled with each other, it remains unresolved how information is shared between them. Here we aimed to identify regions containing neuronal populations that show directional tuning for both arm and eye movements. In two separate fMRI experiments, the same participants were scanned while performing a center-out arm or eye movement task. Using a whole-brain searchlight-based representational similarity analysis (RSA), we found that a bilateral region in the posterior superior parietal lobule represents both arm and eye movement direction, thus extending previous findings in monkeys.

Conceptual processing of action verbs with and without motor representations.

Reading an action verb activates its corresponding motor representation in the reader's motor cortex, but whether this activation is relevant for comprehension remains unclear. To quantify the contribution of motor representations to the conceptual processing of action verbs, we measured the efficiency of two participants with atypical motor experience due to congenitally severely reduced upper limbs in processing verbs referring to actions that they had previously executed (e.g., writing) or not (e.g., shoveling) and compared the efficiency difference between the two verb categories to that found in typical participants, who had previously executed all these actions. This allowed measuring the contribution of motor representations unbiased by confounded low-level, lexical and semantic variables. Although the task was sensitive and the participants' performance was positively influenced by the richness of the words' conceptual representations, we found no detectable advantage for words associated with motor representations.

Typical action perception and interpretation without motor simulation.

Every day, we interact with people synchronously, immediately understand what they are doing, and easily infer their mental state and the likely outcome of their actions from their kinematics. According to various motor simulation theories of perception, such efficient perceptual processing of others' actions cannot be achieved by visual analysis of the movements alone but requires a process of motor simulation--an unconscious, covert imitation of the observed movements. According to this hypothesis, individuals incapable of simulating observed movements in their motor system should have difficulty perceiving and interpreting observed actions. Contrary to this prediction, we found across eight sensitive experiments that individuals born with absent or severely shortened upper limbs (upper limb dysplasia), despite some variability, could perceive, anticipate, predict, comprehend, and memorize upper limb actions, which they cannot simulate, as efficiently as typically developed participants. We also found that, like the typically developed participants, the dysplasic participants systematically perceived the position of moving upper limbs slightly ahead of their real position but only when the anticipated position was not biomechanically awkward. Such anticipatory bias and its modulation by implicit knowledge of the body biomechanical constraints were previously considered as indexes of the crucial role of motor simulation in action perception. Our findings undermine this assumption and the theories that place the locus of action perception and comprehension in the motor system and invite a shift in the focus of future research to the question of how the visuo-perceptual system represents and processes observed body movements and actions.

Action Categories in Lateral Occipitotemporal Cortex Are Organized Along Sociality and Transitivity.

How neural specificity for distinct conceptual knowledge categories arises is central for understanding the organization of semantic memory in the human brain. Although there is a large body of research on the neural processing of distinct object categories, the organization of action categories remains largely unknown. In particular, it is unknown whether different action categories follow a specific topographical organization on the cortical surface analogously to the category-specific organization of object knowledge. Here, we tested whether the neural representation of action knowledge is organized in terms of nonsocial versus social and object-unrelated versus object-related actions (sociality and transitivity, respectively, hereafter). We hypothesized a major distinction of sociality and transitivity along dorsal and ventral lateral occipitotemporal cortex (LOTC), respectively. Using fMRI-based multivoxel pattern analysis, we identified neural representations of action information associated with sociality and transitivity in bilateral LOTC. Representational similarity analysis revealed a dissociation between dorsal and ventral LOTC. We found that action representations in dorsal LOTC are segregated along features of sociality, whereas action representations in ventral LOTC are segregated along features of transitivity. In addition, representations of sociality and transitivity features were found more anteriorly in LOTC than representations of specific subtypes of actions, suggesting a posterior-anterior gradient from concrete to abstract action features. These findings elucidate how the neural representations of perceptually and conceptually diverse actions are organized in distinct subsystems in the LOTC. SIGNIFICANCE STATEMENT: The lateral occipitotemporal cortex (LOTC) is critically involved in the recognition of objects and actions, but our knowledge about the underlying organizing principles is limited. Here, we discovered a dorsal-ventral distinction of actions in LOTC: dorsal LOTC represents actions based on sociality (how much an action is directed to another person) in proximity to person knowledge. In contrast, ventral LOTC represents actions based on transitivity (how much an action involves the interaction with inanimate objects) in proximity to tools/artifacts in ventral LOTC, suggesting a mutually dependent organization of actions and objects. In addition, we found a posterior-to-anterior organization of the LOTC for concrete and abstract representations, respectively. Our findings provide important insights about the organization of actions in LOTC.

Domain Selectivity in the Parahippocampal Gyrus Is Predicted by the Same Structural Connectivity Patterns in Blind and Sighted Individuals.

Human ventral occipital temporal cortex contains clusters of neurons that show domain-preferring responses during visual perception. Recent studies have reported that some of these clusters show surprisingly similar domain selectivity in congenitally blind participants performing nonvisual tasks. An important open question is whether these functional similarities are driven by similar innate connections in blind and sighted groups. Here we addressed this question focusing on the parahippocampal gyrus (PHG), a region that is selective for large objects and scenes. Based on the assumption that patterns of long-range connectivity shape local computation, we examined whether domain selectivity in PHG is driven by similar structural connectivity patterns in the two populations. Multiple regression models were built to predict the selectivity of PHG voxels for large human-made objects from white matter (WM) connectivity patterns in both groups. These models were then tested using independent data from participants with similar visual experience (two sighted groups) and using data from participants with different visual experience (blind and sighted groups). Strikingly, the WM-based predictions between blind and sighted groups were as successful as predictions between two independent sighted groups. That is, the functional selectivity for large objects of a PHG voxel in a blind participant could be accurately predicted by its WM pattern using the connection-to-function model built from the sighted group data, and vice versa. Regions that significantly predicted PHG selectivity were located in temporal and frontal cortices in both sighted and blind populations. These results show that the large-scale network driving domain selectivity in PHG is independent of vision.SIGNIFICANCE STATEMENT Recent studies have reported intriguingly similar domain selectivity in sighted and congenitally blind individuals in regions within the ventral visual cortex. To examine whether these similarities originate from similar innate connectional roots, we investigated whether the domain selectivity in one population could be predicted by the structural connectivity pattern of the other. We found that the selectivity for large objects of a PHG voxel in a blind participant could be predicted by its structural connectivity pattern using the connection-to-function model built from the sighted group data, and vice versa. These results reveal that the structural connectivity underlying domain selectivity in the PHG is independent of visual experience, providing evidence for nonvisual representations in this region.

View-invariant representation of hand postures in the human lateral occipitotemporal cortex.

Understanding other people's actions and mental states includes the interpretation of body postures and movements. In particular, hand postures are an important channel to signal both action and communicative intentions. Recognizing hand postures is computationally challenging because hand postures often differ only in the subtle configuration of relative finger positions and because visual characteristics of hand postures change across viewpoints. To allow for accurate interpretation, the brain needs to represent hand postures in a view-invariant but posture-specific manner. Here we test for such representations in hand-, body-, and object-selective regions of the lateral occipitotemporal cortex (LOTC). We used multivariate pattern analysis of fMRI data to test for view-specific and view-invariant representations of individual hand postures, separately for two domains: action-related postures (e.g., a precision grasp) and communicative postures (e.g., thumbs up). Results showed that hand-selective LOTC, but not nearby body- and object-selective LOTC, represented hand postures in a view-invariant manner, with relatively similar activity patterns to the same hand posture seen from different viewpoints. View invariance was equally strong for action and communicative postures. By contrast, object-selective cortex represented hand postures in a view-specific manner. These results indicate a role for hand-selective LOTC in solving the view-invariance problem for individual hand postures. View-invariant representations of hand postures in this region may then be accessed and further interpreted by multiple downstream systems to inform high-level judgments related to action understanding, emotion recognition, and non-verbal communication.

Multivariate pattern dependence.

When we perform a cognitive task, multiple brain regions are engaged. Understanding how these regions interact is a fundamental step to uncover the neural bases of behavior. Most research on the interactions between brain regions has focused on the univariate responses in the regions. However, fine grained patterns of response encode important information, as shown by multivariate pattern analysis. In the present article, we introduce and apply multivariate pattern dependence (MVPD): a technique to study the statistical dependence between brain regions in humans in terms of the multivariate relations between their patterns of responses. MVPD characterizes the responses in each brain region as trajectories in region-specific multidimensional spaces, and models the multivariate relationship between these trajectories. We applied MVPD to the posterior superior temporal sulcus (pSTS) and to the fusiform face area (FFA), using a searchlight approach to reveal interactions between these seed regions and the rest of the brain. Across two different experiments, MVPD identified significant statistical dependence not detected by standard functional connectivity. Additionally, MVPD outperformed univariate connectivity in its ability to explain independent variance in the responses of individual voxels. In the end, MVPD uncovered different connectivity profiles associated with different representational subspaces of FFA: the first principal component of FFA shows differential connectivity with occipital and parietal regions implicated in the processing of low-level properties of faces, while the second and third components show differential connectivity with anterior temporal regions implicated in the processing of invariant representations of face identity.

The Large-Scale Organization of Object-Responsive Cortex Is Reflected in Resting-State Network Architecture.

Neural responses to visually presented objects have a large-scale spatial organization across the cortex, related to the dimensions of animacy and object size. Most proposals about the origins of this organization point to the influence of differential connectivity with other cortical regions as the key organizing force that drives distinctions in object-responsive cortex. To explore this possibility, we used resting-state functional connectivity to examine the relationship between stimulus-evoked organization of objects, and distinctions in functional network architecture. Using a data-driven analysis, we found evidence for three distinct whole-brain resting-state networks that route through object-responsive cortex, and these naturally manifest the tripartite structure of the stimulus-evoked organization. However, object-responsive regions were also highly correlated with each other at rest. Together, these results point to a nested network architecture, with a local interconnected network across object-responsive cortex and distinctive subnetworks that specifically route these key object distinctions to distinct long-range regions. Broadly, these results point to the viability that long-range connections are a driving force of the large-scale organization of object-responsive cortex.