Preserved recognition of basic visual features despite lack of awareness of shape: Evidence from a case of neglect.

Human visual experience of objects comprises a combination of visual features, such as color, position, and shape. Spatial attention is thought to play a role in creating a coherent perceptual experience, integrating visual information coming from a given location, but the mechanisms underlying this process are not fully understood. Deficits of spatial attention in which this integration process does not occur normally, such as neglect, can provide insights regarding the mechanisms of spatial attention in visual object recognition. In this study, we describe a series of experiments conducted with an individual with neglect, DH. DH presents characteristic lack of awareness of the left side of individual objects, evidenced by poor object and face recognition, and impaired word reading. However, he exhibits intact recognition of color within the boundaries of the same objects he fails to recognize. Furthermore, he can also report the orientation and location of a colored region on the neglected left side despite lack of awareness of the shape of the region. Overall, DH shows selective lack of awareness of shape despite intact processing of basic visual features in the same spatial location. DH's performance raises intriguing questions and challenges about the role of spatial attention in the formation of coherent object percepts and visual awareness.

The Role of Agentive and Physical Forces in the Neural Representation of Motion Events.

How does the brain represent information about motion events in relation to agentive and physical forces? In this study, we investigated the neural activity patterns associated with observing animated actions of agents (e.g., an agent hitting a chair) in comparison to similar movements of inanimate objects that were either shaped solely by the physics of the scene (e.g., gravity causing an object to fall down a hill and hit a chair) or initiated by agents (e.g., a visible agent causing an object to hit a chair). Using an fMRI-based multivariate pattern analysis (MVPA), this design allowed testing where in the brain the neural activity patterns associated with motion events change as a function of, or are invariant to, agentive versus physical forces behind them. A total of 29 human participants (nine male) participated in the study. Cross-decoding revealed a shared neural representation of animate and inanimate motion events that is invariant to agentive or physical forces in regions spanning frontoparietal and posterior temporal cortices. In contrast, the right lateral occipitotemporal cortex showed a higher sensitivity to agentive events, while the left dorsal premotor cortex was more sensitive to information about inanimate object events that were solely shaped by the physics of the scene.

Effector-specific motor simulation supplements core action recognition processes in adverse conditions.

Observing other people acting activates imitative motor plans in the observer. Whether, and if so when and how, such 'effector-specific motor simulation' contributes to action recognition remains unclear. We report that individuals born without upper limbs (IDs)-who cannot covertly imitate upper-limb movements-are significantly less accurate at recognizing degraded (but not intact) upper-limb than lower-limb actions (i.e. point-light animations). This finding emphasizes the need to reframe the current controversy regarding the role of effector-specific motor simulation in action recognition: instead of focusing on the dichotomy between motor and non-motor theories, the field would benefit from new hypotheses specifying when and how effector-specific motor simulation may supplement core action recognition processes to accommodate the full variety of action stimuli that humans can recognize.

From intermediate shape-centered representations to the perception of oriented shapes: response to commentaries.

In this response paper, we start by addressing the main points made by the commentators on the target article's main theoretical conclusions: the existence and characteristics of the intermediate shape-centered representations (ISCRs) in the visual system, their emergence from edge detection mechanisms operating on different types of visual properties, and how they are eventually reunited in higher order frames of reference underlying conscious visual perception. We also address the much-commented issue of the possible neural mechanisms of the ISCRs. In the final section, we address more specific and general comments, questions, and suggestions which, albeit very interesting, were less directly focused on the main conclusions of the target paper.

A shared neural code for the physics of actions and object events.

Observing others' actions recruits frontoparietal and posterior temporal brain regions - also called the action observation network. It is typically assumed that these regions support recognizing actions of animate entities (e.g., person jumping over a box). However, objects can also participate in events with rich meaning and structure (e.g., ball bouncing over a box). So far, it has not been clarified which brain regions encode information specific to goal-directed actions or more general information that also defines object events. Here, we show a shared neural code for visually presented actions and object events throughout the action observation network. We argue that this neural representation captures the structure and physics of events regardless of animacy. We find that lateral occipitotemporal cortex encodes information about events that is also invariant to stimulus modality. Our results shed light onto the representational profiles of posterior temporal and frontoparietal cortices, and their roles in encoding event information.

Preference for animate domain sounds in the fusiform gyrus of blind individuals is modulated by shape-action mapping.

In high-level visual areas in the human brain, preference for inanimate objects is observed regardless of stimulation modality (visual/auditory/tactile) and individual's visual experience (sighted/blind) whereas preference for animate entities seems robust mainly in the visual modality. Here, we test a hypothesis explaining this domain difference: Object representations can be activated through nonvisual stimulation when their shapes are systematically related to action system representations, a quality typical of most inanimate objects but of only specific animate entities. We studied functional magnetic resonance imaging activations in congenitally blind and sighted individuals listening to animal, object, and human sounds. In blind individuals, the typical location of the fusiform face area preferentially responded to human facial expression sounds clearly related to specific facial actions and resulting face shapes but not to speech or animal sounds. No univariate preference for any sound category was observed in the fusiform gyrus in sighted individuals, but the expected multivoxel effects were present. We conclude that nonvisual signals can activate shape representations of those stimuli-inanimate or animate-for which shape and action computations are transparently related. However, absence of potentially competing visual inputs seems necessary for this effect to be clearly detectable in the case of animate representation.

Shape-centered representations of bounded regions of space mediate the perception of objects.

We report the study of a woman who perceives 2D bounded regions of space ("shapes") defined by sharp edges of medium to high contrast as if they were rotated by 90, 180 degrees around their centre, mirrored across their own axes, or both. In contrast, her perception of 3D, strongly blurred or very low contrast shapes, and of stimuli emerging from a collection of shapes, is intact. This suggests that a stage in the process of constructing the conscious visual representation of a scene consists of representing mutually exclusive bounded regions extracted from the initial retinotopic space in "shape-centered" frames of reference. The selectivity of the disorder to shapes originally biased toward the parvocellular subcortical pathway, and the absence of any other type of error, additionally invite new hypotheses about the operations involved in computing these "intermediate shape-centered representations" and in mapping them onto higher frames for perception and action.

Two 'what' pathways for action and object recognition.

The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition critically depends on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are typically found in ventral occipitotemporal cortex. We argue that occipitotemporal cortex contains similarly organized lateral and ventral 'what' pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and ventral cortex.

The contribution of object size, manipulability, and stability on neural responses to inanimate objects.

In human occipitotemporal cortex, brain responses to depicted inanimate objects have a large-scale organization by real-world object size. Critically, the size of objects in the world is systematically related to behaviorally-relevant properties: small objects are often grasped and manipulated (e.g., forks), while large objects tend to be less motor-relevant (e.g., tables), though this relationship does not always have to be true (e.g., picture frames and wheelbarrows). To determine how these two dimensions interact, we measured brain activity with functional magnetic resonance imaging while participants viewed a stimulus set of small and large objects with either low or high motor-relevance. The results revealed that the size organization was evident for objects with both low and high motor-relevance; further, a motor-relevance map was also evident across both large and small objects. Targeted contrasts revealed that typical combinations (small motor-relevant vs. large non-motor-relevant) yielded more robust topographies than the atypical covariance contrast (small non-motor-relevant vs. large motor-relevant). In subsequent exploratory analyses, a factor analysis revealed that the construct of motor-relevance was better explained by two underlying factors: one more related to manipulability, and the other to whether an object moves or is stable. The factor related to manipulability better explained responses in lateral small-object preferring regions, while the factor related to object stability (lack of movement) better explained responses in ventromedial large-object preferring regions. Taken together, these results reveal that the structure of neural responses to objects of different sizes further reflect behavior-relevant properties of manipulability and stability, and contribute to a deeper understanding of some of the factors that help the large-scale organization of object representation in high-level visual cortex.

A common selection mechanism at each linguistic level in bilingual and monolingual language production.

The primary goal of research on the functional and neural architecture of bilingualism is to elucidate how bilingual individuals' language architecture is organized such that they can both speak in a single language without accidental insertions of the other, but also flexibly switch between their two languages if the context allows/demands them to. Here we review the principles under which any proposed architecture could operate, and present a framework where the selection mechanism for individual elements strictly operates on the basis of the highest level of activation and does not require suppressing representations in the non-target language. We specify the conjunction of parameters and factors that jointly determine these levels of activation and develop a theory of bilingual language organization that extends beyond the lexical level to other levels of representation (i.e., semantics, morphology, syntax and phonology). The proposed architecture assumes a common selection principle at each linguistic level to account for attested features of bilingual speech in, but crucially also out, of experimental settings.

Typically Efficient Lipreading without Motor Simulation.

All it takes is a face-to-face conversation in a noisy environment to realize that viewing a speaker's lip movements contributes to speech comprehension. What are the processes underlying the perception and interpretation of visual speech? Brain areas that control speech production are also recruited during lipreading. This finding raises the possibility that lipreading may be supported, at least to some extent, by a covert unconscious imitation of the observed speech movements in the observer's own speech motor system-a motor simulation. However, whether, and if so to what extent, motor simulation contributes to visual speech interpretation remains unclear. In two experiments, we found that several participants with congenital facial paralysis were as good at lipreading as the control population and performed these tasks in a way that is qualitatively similar to the controls despite severely reduced or even completely absent lip motor representations. Although it remains an open question whether this conclusion generalizes to other experimental conditions and to typically developed participants, these findings considerably narrow the space of hypothesis for a role of motor simulation in lipreading. Beyond its theoretical significance in the field of speech perception, this finding also calls for a re-examination of the more general hypothesis that motor simulation underlies action perception and interpretation developed in the frameworks of motor simulation and mirror neuron hypotheses.

On the Need for Theoretically Guided Approaches to Possible Bilingual Advantages: An Evaluation of the Potential Loci in the Language and Executive Control Systems.

Whether a cognitive advantage exists for bilingual individuals has been the source of heated debate in the last decade. While empirical evidence putatively in favor of or against this alleged advantage has been frequently discussed, the potential sources of enhanced cognitive control in bilinguals have only been broadly declared, with no mechanistic elaboration of where, why, and how this purported link between bilingualism and enhanced language control develops, and how this enhancement transfers to, and subsequently improves, general executive function. Here, we evaluate different potential sources for a bilingual advantage and develop the assumptions one would have to make about the language processing system to be consistent with each of these notions. Subsequently, we delineate the limitations in the generalizations from language to overall executive function, and characterize where these advantages could be identified if there were to be any. Ultimately, we conclude that in order to make significant progress in this area, it is necessary to look for advantages in theoretically motivated areas, and that in the absence of clear theories as to the source, transfer, and target processes that could lead to potential advantages, an inconsistent body of results will follow, making the whole pursuit of a bilingual advantage moot.

Evidence for an effector-independent action system from people born without hands.

Many parts of the visuomotor system guide daily hand actions, like reaching for and grasping objects. Do these regions depend exclusively on the hand as a specific body part whose movement they guide, or are they organized for the reaching task per se, for any body part used as an effector? To address this question, we conducted a neuroimaging study with people born without upper limbs-individuals with dysplasia-who use the feet to act, as they and typically developed controls performed reaching and grasping actions with their dominant effector. Individuals with dysplasia have no prior experience acting with hands, allowing us to control for hand motor imagery when acting with another effector (i.e., foot). Primary sensorimotor cortices showed selectivity for the hand in controls and foot in individuals with dysplasia. Importantly, we found a preference based on action type (reaching/grasping) regardless of the effector used in the association sensorimotor cortex, in the left intraparietal sulcus and dorsal premotor cortex, as well as in the basal ganglia and anterior cerebellum. These areas also showed differential response patterns between action types for both groups. Intermediate areas along a posterior-anterior gradient in the left dorsal premotor cortex gradually transitioned from selectivity based on the body part to selectivity based on the action type. These findings indicate that some visuomotor association areas are organized based on abstract action functions independent of specific sensorimotor parameters, paralleling sensory feature-independence in visual and auditory cortices in people born blind and deaf. Together, they suggest association cortices across action and perception may support specific computations, abstracted from low-level sensorimotor elements.

Efficient recognition of facial expressions does not require motor simulation.

What mechanisms underlie facial expression recognition? A popular hypothesis holds that efficient facial expression recognition cannot be achieved by visual analysis alone but additionally requires a mechanism of motor simulation - an unconscious, covert imitation of the observed facial postures and movements. Here, we first discuss why this hypothesis does not necessarily follow from extant empirical evidence. Next, we report experimental evidence against the central premise of this view: we demonstrate that individuals can achieve normotypical efficient facial expression recognition despite a congenital absence of relevant facial motor representations and, therefore, unaided by motor simulation. This underscores the need to reconsider the role of motor simulation in facial expression recognition.

Two Forms of Knowledge Representations in the Human Brain.

Sensory experience shapes what and how knowledge is stored in the brain-our knowledge about the color of roses depends in part on the activity of color-responsive neurons based on experiences of seeing roses. We compared the brain basis of color knowledge in congenitally (or early) blind individuals, whose color knowledge can only be obtained through language descriptions and/or cognitive inference, to that of sighted individuals whose color-knowledge benefits from both sensory experience and language. We found that some regions support color knowledge only in the sighted, whereas a region in the left dorsal anterior temporal lobe supports object-color knowledge in both the blind and sighted groups, indicating the existence of a sensory-independent knowledge coding system in both groups. Thus, there are (at least) two forms of object knowledge representations in the human brain: sensory-derived and language- and cognition-derived knowledge, supported by different brain systems.

Individuation of parts of a single object and multiple distinct objects relies on a common neural mechanism in inferior intraparietal sulcus.

Object identification and enumeration rely on the ability to distinguish, or individuate, objects from the background. Does multiple object individuation operate only over bounded, separable objects or does it operate equally over connected features within a single object? While previous fMRI experiments suggest that connectedness affects the processing and enumeration of objects, recent behavioral and EEG studies demonstrated that parallel individuation occurs over both object parts and distinct objects. However, it is unclear whether individuation of object parts and distinct objects relies on a common or independent neural mechanisms. Using fMRI-based multivariate pattern analyses, we here demonstrate that activity patterns in inferior and superior intraparietal sulci (IPS) encode numerosity independently of whether the individuated items are connected parts of a single object or distinct objects. Lateral occipital cortex is more sensitive to perceptual aspects of the two stimulus types and the targets of the stimuli, suggesting a dissociation between ventral and dorsal areas in representing perceptual object properties and more general information about numerosity, respectively. Our results suggest that objecthood is not a necessary prerequisite for parallel individuation in IPS. Rather, our results point toward a common individuation mechanism that selects targets over a flexible object hierarchy, independently of whether the targets are distinct separable objects or parts of a single object.

Lateral occipitotemporal cortex encodes perceptual components of social actions rather than abstract representations of sociality.

Neuroimaging studies suggest that areas in the lateral occipitotemporal cortex (LOTC) play an important role in the perception of social actions. However, it is unclear what precisely about social actions these areas represent: perceptual features that may be indicative of social actions - such as the presence of persons in a scene, their orientation toward each other, and in particular the directedness of action movements toward persons or other targets - or more abstract representations that capture whether an action is meant to be social. In two fMRI experiments, we used representational similarity analysis (RSA) to test whether LOTC is sensitive to perceptual action components important for social interpretation and/or more general representations of sociality (Experiment 1) and implied person-directedness (Experiment 2). We found that LOTC is sensitive to perceptual action components (person presence, person orientation, and action directedness toward different types of recipients). By contrast, more general levels of sociality and implied person-directedness were not captured by LOTC. Our findings suggest that regions in LOTC provide the perceptual basis for social action interpretation but challenge accounts that posit specialization at more general levels sensitive to social actions and sociality as such. We propose that the interpretation of an action - in terms of sociality or other intentional aspects - arises from the interaction of multiple areas in processing relevant action components in a situation-dependent manner.

Directional tuning for eye and arm movements in overlapping regions in human posterior parietal cortex.

A network of frontal and parietal regions is known to be recruited during the planning and execution of arm and eye movements. While movements of the two effectors are typically coupled with each other, it remains unresolved how information is shared between them. Here we aimed to identify regions containing neuronal populations that show directional tuning for both arm and eye movements. In two separate fMRI experiments, the same participants were scanned while performing a center-out arm or eye movement task. Using a whole-brain searchlight-based representational similarity analysis (RSA), we found that a bilateral region in the posterior superior parietal lobule represents both arm and eye movement direction, thus extending previous findings in monkeys.

Distinct roles of temporal and frontoparietal cortex in representing actions across vision and language.

Both temporal and frontoparietal brain areas are associated with the representation of knowledge about the world, in particular about actions. However, what these brain regions represent and precisely how they differ remains unknown. Here, we reveal distinct functional profiles of lateral temporal and frontoparietal cortex using fMRI-based MVPA. Frontoparietal areas encode representations of observed actions and corresponding written sentences in an overlapping way, but these representations do not generalize across stimulus type. By contrast, only left lateral posterior temporal cortex (LPTC) encodes action representations that generalize across observed action scenes and written descriptions. The representational organization of stimulus-general action information in LPTC can be predicted from models that describe basic agent-patient relations (object- and person-directedness) and the general semantic similarity between actions. Our results suggest that LPTC encodes general, conceptual aspects of actions whereas frontoparietal representations appear to be tied to specific stimulus types.

Conceptual processing of action verbs with and without motor representations.

Reading an action verb activates its corresponding motor representation in the reader's motor cortex, but whether this activation is relevant for comprehension remains unclear. To quantify the contribution of motor representations to the conceptual processing of action verbs, we measured the efficiency of two participants with atypical motor experience due to congenitally severely reduced upper limbs in processing verbs referring to actions that they had previously executed (e.g., writing) or not (e.g., shoveling) and compared the efficiency difference between the two verb categories to that found in typical participants, who had previously executed all these actions. This allowed measuring the contribution of motor representations unbiased by confounded low-level, lexical and semantic variables. Although the task was sensitive and the participants' performance was positively influenced by the richness of the words' conceptual representations, we found no detectable advantage for words associated with motor representations.