Stable readout of observed actions from format-dependent activity of monkey's anterior intraparietal neurons.

Humans accurately identify observed actions despite large dynamic changes in their retinal images and a variety of visual presentation formats. A large network of brain regions in primates participates in the processing of others' actions, with the anterior intraparietal area (AIP) playing a major role in routing information about observed manipulative actions (OMAs) to the other nodes of the network. This study investigated whether the AIP also contributes to invariant coding of OMAs across different visual formats. We recorded AIP neuronal activity from two macaques while they observed videos portraying seven manipulative actions (drag, drop, grasp, push, roll, rotate, squeeze) in four visual formats. Each format resulted from the combination of two actor's body postures (standing, sitting) and two viewpoints (lateral, frontal). Out of 297 recorded units, 38% were OMA-selective in at least one format. Robust population code for viewpoint and actor's body posture emerged shortly after stimulus presentation, followed by OMA selectivity. Although we found no fully invariant OMA-selective neuron, we discovered a population code that allowed us to classify action exemplars irrespective of the visual format. This code depends on a multiplicative mixing of signals about OMA identity and visual format, particularly evidenced by a set of units maintaining a relatively stable OMA selectivity across formats despite considerable rescaling of their firing rate depending on the visual specificities of each format. These findings suggest that the AIP integrates format-dependent information and the visual features of others' actions, leading to a stable readout of observed manipulative action identity.

The unique role of parietal cortex in action observation: Functional organization for communicative and manipulative actions.

Action observation is supported by a network of regions in occipito-temporal, parietal, and premotor cortex in primates. Recent research suggests that the parietal node has regions dedicated to different action classes including manipulation, interpersonal interactions, skin displacement, locomotion, and climbing. The goals of the current study consist of: 1) extending this work with new classes of actions that are communicative and specific to humans, 2) investigating how parietal cortex differs from the occipito-temporal and premotor cortex in representing action classes. Human subjects underwent fMRI scanning while observing three action classes: indirect communication, direct communication, and manipulation, plus two types of control stimuli, static controls which were static frames from the video clips, and dynamic controls consisting of temporally-scrambled optic flow information. Using univariate analysis, MVPA, and representational similarity analysis, our study presents several novel findings. First, we provide further evidence for the anatomical segregation in parietal cortex of different action classes: We have found a new site that is specific for representing human-specific indirect communicative actions in cytoarchitectonic parietal area PFt. Second, we found that the discriminability between action classes was higher in parietal cortex than the other two levels suggesting the coding of action identity information at this level. Finally, our results advocate the use of the control stimuli not just for univariate analysis of complex action videos but also when using multivariate techniques.

A parietal region processing numerosity of observed actions: An FMRI study.

When observing others' behavior, it is important to perceive not only the identity of the observed actions (OAs), but also the number of times they were performed. Given the mounting evidence implicating posterior parietal cortex in action observation, and in particular that of manipulative actions, the aim of this study was to identify the parietal region, if any, that contributes to the processing of observed manipulative action (OMA) numerosity, using the functional magnetic resonance imaging technique. Twenty-one right-handed healthy volunteers performed two discrimination tasks while in the scanner, responding to video stimuli in which an actor performed manipulative actions on colored target balls that appeared four times consecutively. The subjects discriminated between two small numerosities of either OMAs ("Action" condition) or colors of balls ("Ball" condition). A significant difference between the "Action" and "Ball" conditions was observed in occipito-temporal cortex and the putative human anterior intraparietal sulcus (phAIP) area as well as the third topographic map of numerosity-selective neurons at the post-central sulcus (NPC3) of the left parietal cortex. A further region of interest analysis of the group-average data showed that at the single voxel level the latter area, more than any other parietal or occipito-temporal numerosity map, favored numerosity of OAs. These results suggest that phAIP processes the identity of OMAs, while neighboring NPC3 likely processes the numerosity of the identified OAs.

Fast Compensatory Functional Network Changes Caused by Reversible Inactivation of Monkey Parietal Cortex.

The brain has a remarkable capacity to recover after lesions. However, little is known about compensatory neural adaptations at the systems level. We addressed this question by investigating behavioral and (correlated) functional changes throughout the cortex that are induced by focal, reversible inactivations. Specifically, monkeys performed a demanding covert spatial attention task while the lateral intraparietal area (LIP) was inactivated with muscimol and whole-brain fMRI activity was recorded. The inactivation caused LIP-specific decreases in task-related fMRI activity. In addition, these local effects triggered large-scale network changes. Unlike most studies in which animals were mainly passive relative to the stimuli, we observed heterogeneous effects with more profound muscimol-induced increases of task-related fMRI activity in areas connected to LIP, especially FEF. Furthermore, in areas such as FEF and V4, muscimol-induced changes in fMRI activity correlated with changes in behavioral performance. Notably, the activity changes in remote areas did not correlate with the decreased activity at the site of the inactivation, suggesting that such changes arise via neuronal mechanisms lying in the intact portion of the functional task network, with FEF a likely key player. The excitation-inhibition dynamics unmasking existing excitatory connections across the functional network might initiate these rapid adaptive changes.

Anterior Intraparietal Area: A Hub in the Observed Manipulative Action Network.

Current knowledge regarding the processing of observed manipulative actions (OMAs) (e.g., grasping, dragging, or dropping) is limited to grasping and underlying neural circuitry remains controversial. Here, we addressed these issues by combining chronic neuronal recordings along the anteroposterior extent of monkeys' anterior intraparietal (AIP) area with tracer injections into the recorded sites. We found robust neural selectivity for 7 distinct OMAs, particularly in the posterior part of AIP (pAIP), where it was associated with motor coding of grip type and own-hand visual feedback. This cluster of functional properties appears to be specifically grounded in stronger direct connections of pAIP with the temporal regions of the ventral visual stream and the prefrontal cortex, as connections with skeletomotor related areas and regions of the dorsal visual stream exhibited opposite or no rostrocaudal gradients. Temporal and prefrontal areas may provide visual and contextual information relevant for manipulative action processing. These results revise existing models of the action observation network, suggesting that pAIP constitutes a parietal hub for routing information about OMA identity to the other nodes of the network.

The extraction of 3D shape in the visual system of human and nonhuman primates.

Depth structure, the third dimension of object shape, is extracted from disparity, motion, texture, and shading in the optic array. Gradient-selective neurons play a key role in this process. Such neurons occur in CIP, AIP, TEs, and F5 (for first- or second-order disparity gradients), in MT/V5, in FST (for speed gradients), and in CIP and TEs (for texture gradients). Most of these regions are activated during magnetic resonance scanning in alert monkeys by comparing 3D conditions with the 2D controls for the different cues. Similarities in activation patterns of monkeys and humans tested with identical paradigms suggest that like gradient-selective neurons are found in corresponding human cortical areas. This view gains credence as the homologies between such areas become more evident. Furthermore, 3D shape-processing networks are similar in the two species, with the exception of the greater involvement of human posterior parietal cortex in the extraction of 3D shape from motion. Thus we can begin to understand how depth structure is extracted from motion, disparity, and texture in the primate brain, but the extraction of depth structure from shading and that of wire-like objects requires further scrutiny.

Four-dimensional maps of the human somatosensory system.

A fine-grained description of the spatiotemporal dynamics of human brain activity is a major goal of neuroscientific research. Limitations in spatial and temporal resolution of available noninvasive recording and imaging techniques have hindered so far the acquisition of precise, comprehensive four-dimensional maps of human neural activity. The present study combines anatomical and functional data from intracerebral recordings of nearly 100 patients, to generate highly resolved four-dimensional maps of human cortical processing of nonpainful somatosensory stimuli. These maps indicate that the human somatosensory system devoted to the hand encompasses a widespread network covering more than 10% of the cortical surface of both hemispheres. This network includes phasic components, centered on primary somatosensory cortex and neighboring motor, premotor, and inferior parietal regions, and tonic components, centered on opercular and insular areas, and involving human parietal rostroventral area and ventral medial-superior-temporal area. The technique described opens new avenues for investigating the neural basis of all levels of cortical processing in humans.

Characterization of network structure in stereoEEG data using consensus-based partial coherence.

Coherence is a widely used measure to determine the frequency-resolved functional connectivity between pairs of recording sites, but this measure is confounded by shared inputs to the pair. To remove shared inputs, the 'partial coherence' can be computed by conditioning the spectral matrices of the pair on all other recorded channels, which involves the calculation of a matrix (pseudo-) inverse. It has so far remained a challenge to use the time-resolved partial coherence to analyze intracranial recordings with a large number of recording sites. For instance, calculating the partial coherence using a pseudoinverse method produces a high number of false positives when it is applied to a large number of channels. To address this challenge, we developed a new method that randomly aggregated channels into a smaller number of effective channels on which the calculation of partial coherence was based. We obtained a 'consensus' partial coherence (cPCOH) by repeating this approach for several random aggregations of channels (permutations) and only accepting those activations in time and frequency with a high enough consensus. Using model data we show that the cPCOH method effectively filters out the effect of shared inputs and performs substantially better than the pseudo-inverse. We successfully applied the cPCOH procedure to human stereotactic EEG data and demonstrated three key advantages of this method relative to alternative procedures. First, it reduces the number of false positives relative to the pseudo-inverse method. Second, it allows for titration of the amount of false positives relative to the false negatives by adjusting the consensus threshold, thus allowing the data-analyst to prioritize one over the other to meet specific analysis demands. Third, it substantially reduced the number of identified interactions compared to coherence, providing a sparser network of connections from which clear spatial patterns emerged. These patterns can serve as a starting point of further analyses that provide insight into network dynamics during cognitive processes. These advantages likely generalize to other modalities in which shared inputs introduce confounds, such as electroencephalography (EEG) and magneto-encephalography (MEG).

Observing Others Speak or Sing Activates Spt and Neighboring Parietal Cortex.

To obtain further evidence that action observation can serve as a proxy for action execution and planning in posterior parietal cortex, we scanned participants while they were (1) observing two classes of action: vocal communication and oral manipulation, which share the same effector but differ in nature, and (2) rehearsing and listening to nonsense sentences to localize area Spt, thought to be involved in audio-motor transformation during speech. Using this localizer, we found that Spt is specifically activated by vocal communication, indicating that Spt is not only involved in planning speech but also in observing vocal communication actions. In addition, we observed that Spt is distinct from the parietal region most specialized for observing vocal communication, revealed by an interaction contrast and located in PFm. The latter region, unlike Spt, processes the visual and auditory signals related to other's vocal communication independently. Our findings are consistent with the view that several small regions in the temporoparietal cortex near the ventral part of the supramarginal/angular gyrus border are involved in the planning of vocal communication actions and are also concerned with observation of these actions, though involvements in those two aspects are unequal.

3D Shape Perception in Posterior Cortical Atrophy: A Visual Neuroscience Perspective.

Posterior cortical atrophy (PCA) is a rare focal neurodegenerative syndrome characterized by progressive visuoperceptual and visuospatial deficits, most often due to atypical Alzheimer's disease (AD). We applied insights from basic visual neuroscience to analyze 3D shape perception in humans affected by PCA. Thirteen PCA patients and 30 matched healthy controls participated, together with two patient control groups with diffuse Lewy body dementia (DLBD) and an amnestic-dominant phenotype of AD, respectively. The hierarchical study design consisted of 3D shape processing for 4 cues (shading, motion, texture, and binocular disparity) with corresponding 2D and elementary feature extraction control conditions. PCA and DLBD exhibited severe 3D shape-processing deficits and AD to a lesser degree. In PCA, deficient 3D shape-from-shading was associated with volume loss in the right posterior inferior temporal cortex. This region coincided with a region of functional activation during 3D shape-from-shading in healthy controls. In PCA patients who performed the same fMRI paradigm, response amplitude during 3D shape-from-shading was reduced in this region. Gray matter volume in this region also correlated with 3D shape-from-shading in AD. 3D shape-from-disparity in PCA was associated with volume loss slightly more anteriorly in posterior inferior temporal cortex as well as in ventral premotor cortex. The findings in right posterior inferior temporal cortex and right premotor cortex are consistent with neurophysiologically based models of the functional anatomy of 3D shape processing. However, in DLBD, 3D shape deficits rely on mechanisms distinct from inferior temporal structural integrity. SIGNIFICANCE STATEMENT: Posterior cortical atrophy (PCA) is a neurodegenerative syndrome characterized by progressive visuoperceptual dysfunction and most often an atypical presentation of Alzheimer's disease (AD) affecting the ventral and dorsal visual streams rather than the medial temporal system. We applied insights from fundamental visual neuroscience to analyze 3D shape perception in PCA. 3D shape-processing deficits were affected beyond what could be accounted for by lower-order processing deficits. For shading and disparity, this was related to volume loss in regions previously implicated in 3D shape processing in the intact human and nonhuman primate brain. Typical amnestic-dominant AD patients also exhibited 3D shape deficits. Advanced visual neuroscience provides insight into the pathogenesis of PCA that also bears relevance for vision in typical AD.

Seeing biological actions in 3D: An fMRI study.

Precise kinematics or body configuration cannot be recovered from visual input without disparity information. Yet, no imaging study has investigated the role of disparity on action observation. Here, we investigated the interaction between disparity and the main cues of biological motion, kinematics and configuration, in two fMRI experiments. Stimuli were presented as point-light figures, depicting complex action sequences lasting 21 s. We hypothesized that interactions could occur at any of the three levels of the action observation network, comprising occipitotemporal, parietal and premotor cortex, with premotor cortex being the most likely location. The main effects of kinematics and configuration confirmed that the biological motion sequences activated all three levels of the action observation network, validating our approach. The interaction between configuration and disparity activated only premotor cortex, whereas interactions between kinematics and disparity occurred at all levels of the action observation network but were strongest at the premotor level. Control experiments demonstrated that these interactions could not be accounted for by low level motion in depth, task effects, spatial attention, or eye movements, including vergence. These results underscore the role of premotor cortex in action observation, and in imitating others or responding to their actions.

Functional definitions of parietal areas in human and non-human primates.

Establishing homologies between cortical areas in animal models and humans lies at the heart of translational neuroscience, as it demonstrates how knowledge obtained from these models can be applied to the human brain. Here, we review progress in using parallel functional imaging to ascertain homologies between parietal areas of human and non-human primates, species sharing similar behavioural repertoires. The human homologues of several areas along monkey IPS involved in action planning and observation, such as AIP, LIP and CIP, as well as those of opercular areas (SII complex), have been defined. In addition, uniquely human areas, such as the tool-use area in left anterior supramarginal gyrus, have also been identified.

The retinotopic organization of macaque occipitotemporal cortex anterior to V4 and caudoventral to the middle temporal (MT) cluster.

The retinotopic organization of macaque occipitotemporal cortex rostral to area V4 and caudorostral to the recently described middle temporal (MT) cluster of the monkey (Kolster et al., 2009) is not well established. The proposed number of areas within this region varies from one to four, underscoring the ambiguity concerning the functional organization in this region of extrastriate cortex. We used phase-encoded retinotopic functional MRI mapping methods to reveal the functional topography of this cortical domain. Polar-angle maps showed one complete hemifield representation bordering area V4 anteriorly, split into dorsal and ventral counterparts corresponding to the lower and upper visual field quadrants, respectively. The location of this hemifield representation corresponds to area V4A. More rostroventrally, we identified three other complete hemifield representations. Two of these correspond to the dorsal and the ventral posterior inferotemporal areas (PITd and PITv, respectively) as identified in the Felleman and Van Essen (1991) scheme. The third representation has been tentatively named dorsal occipitotemporal area (OTd). Areas V4A, PITd, PITv, and OTd share a central visual field representation, similar to the areas constituting the MT cluster. Furthermore, they vary widely in size and represent the complete contralateral visual field. Functionally, these four areas show little motion sensitivity, unlike those of the MT cluster, and two of them, OTd and PITd, displayed pronounced two-dimensional shape sensitivity. In general, these results suggest that retinotopically organized tissue extends farther into rostral occipitotemporal cortex of the monkey than generally assumed.

The organization of the posterior parietal cortex devoted to upper limb actions: An fMRI study.

The present fMRI study examined whether upper-limb action classes differing in their motor goal are encoded by different PPC sectors. Action observation was used as a proxy for action execution. Subjects viewed actors performing object-related (e.g., grasping), skin-displacing (e.g., rubbing the skin), and interpersonal upper limb actions (e.g., pushing someone). Observation of the three action classes activated a three-level network including occipito-temporal, parietal, and premotor cortex. The parietal region common to observing all three action classes was located dorsally to the left intraparietal sulcus (DIPSM/DIPSA border). Regions specific for observing an action class were obtained by combining the interaction between observing action classes and stimulus types with exclusive masking for observing the other classes, while for regions considered preferentially active for a class the interaction was exclusively masked with the regions common to all observed actions. Left putative human anterior intraparietal was specific for observing manipulative actions, and left parietal operculum including putative human SII region, specific for observing skin-displacing actions. Control experiments demonstrated that this latter activation depended on seeing the skin being moved and not simply on seeing touch. Psychophysiological interactions showed that the two specific parietal regions had similar connectivities. Finally, observing interpersonal actions preferentially activated a dorsal sector of left DIPSA, possibly the homologue of ventral intraparietal coding the impingement of the target person's body into the peripersonal space of the actor. These results support the importance of segregation according to the action class as principle of posterior parietal cortex organization for action observation and by implication for action execution.

Interspecies activity correlations reveal functional correspondence between monkey and human brain areas.

Evolution-driven functional changes in the primate brain are typically assessed by aligning monkey and human activation maps using cortical surface expansion models. These models use putative homologous areas as registration landmarks, assuming they are functionally correspondent. For cases in which functional changes have occurred in an area, this assumption prohibits to reveal whether other areas may have assumed lost functions. Here we describe a method to examine functional correspondences across species. Without making spatial assumptions, we assessed similarities in sensory-driven functional magnetic resonance imaging responses between monkey (Macaca mulatta) and human brain areas by temporal correlation. Using natural vision data, we revealed regions for which functional processing has shifted to topologically divergent locations during evolution. We conclude that substantial evolution-driven functional reorganizations have occurred, not always consistent with cortical expansion processes. This framework for evaluating changes in functional architecture is crucial to building more accurate evolutionary models.

Evolutionarily novel functional networks in the human brain?

Primate evolution has been accompanied by complex reorganizations in brain anatomy and function. Little is known, however, about the relationship between anatomical and functional changes induced through primate evolution. Using functional magnetic resonance imaging, we assessed spatial and temporal correspondences of cortical networks in humans and monkeys. We provided evidence for topologically and functionally correspondent networks in sensory-motor and attention regions. More specifically, we revealed a possible monkey equivalent of the human ventral attention network. For other human networks, such as the language and the default-mode networks, we detected topological correspondent networks in the monkey, but with different functional signatures. Furthermore, we observed two lateralized human frontoparietal networks in the cortical regions displaying the greatest evolutionary expansion, having neither topological nor functional monkey correspondents. This finding may indicate that these two human networks are evolutionarily novel. Thus, our findings confirm the existence of networks where evolution has conserved both topology and function but also suggest that functions of structurally preserved networks can diverge over time and that novel, hence human-specific networks, have emerged during human evolution.

Differences in neural activation for object-directed grasping in chimpanzees and humans.

The human faculty for object-mediated action, including tool use and imitation, exceeds that of even our closest primate relatives and is a key foundation of human cognitive and cultural uniqueness. In humans and macaques, observing object-directed grasping actions activates a network of frontal, parietal, and occipitotemporal brain regions, but differences in human and macaque activation suggest that this system has been a focus of selection in the primate lineage. To study the evolution of this system, we performed functional neuroimaging in humans' closest living relatives, chimpanzees. We compare activations during performance of an object-directed manual grasping action, observation of the same action, and observation of a mimed version of the action that consisted of only movements without results. Performance and observation of the same action activated a distributed frontoparietal network similar to that reported in macaques and humans. Like humans and unlike macaques, these regions were also activated by observing movements without results. However, in a direct chimpanzee/human comparison, we also identified unique aspects of human neural responses to observed grasping. Chimpanzee activation showed a prefrontal bias, including significantly more activity in ventrolateral prefrontal cortex, whereas human activation was more evenly distributed across more posterior regions, including significantly more activation in ventral premotor cortex, inferior parietal cortex, and inferotemporal cortex. This indicates a more "bottom-up" representation of observed action in the human brain and suggests that the evolution of tool use, social learning, and cumulative culture may have involved modifications of frontoparietal interactions.

Fine-grained stimulus representations in body selective areas of human occipito-temporal cortex.

Neurophysiological and functional imaging studies have investigated the representation of animate and inanimate stimulus classes in monkey inferior temporal (IT) and human occipito-temporal cortex (OTC). These studies proposed a distributed representation of stimulus categories across IT and OTC and at the same time highlighted category specific modules for the processing of bodies, faces and objects. Here, we investigated whether the stimulus representation within the extrastriate (EBA) and the fusiform (FBA) body areas differed from the representation across OTC. To address this question, we performed an event-related fMRI experiment, evaluating the pattern of activation elicited by 200 individual stimuli that had already been extensively tested in our earlier monkey imaging and single cell studies (Popivanov et al., 2012, 2014). The set contained achromatic images of headless monkey and human bodies, two sets of man-made objects, monkey and human faces, four-legged mammals, birds, fruits, and sculptures. The fMRI response patterns within EBA and FBA primarily distinguished bodies from non-body stimuli, with subtle differences between the areas. However, despite responding on average stronger to bodies than to other categories, classification performance for preferred and non-preferred categories was comparable. OTC primarily distinguished animate from inanimate stimuli. However, cluster analysis revealed a much more fine-grained representation with several homogeneous clusters consisting entirely of stimuli of individual categories. Overall, our data suggest that category representation varies with location within OTC. Nevertheless, body modules contain information to discriminate also non-preferred stimuli and show an increasing specificity in a posterior to anterior gradient.