Natural scenes reveal diverse representations of 2D and 3D body pose in the human brain.

Human pose, defined as the spatial relationships between body parts, carries instrumental information supporting the understanding of motion and action of a person. A substantial body of previous work has identified cortical areas responsive to images of bodies and different body parts. However, the neural basis underlying the visual perception of body part relationships has received less attention. To broaden our understanding of body perception, we analyzed high-resolution fMRI responses to a wide range of poses from over 4,000 complex natural scenes. Using ground-truth annotations and an application of three-dimensional (3D) pose reconstruction algorithms, we compared similarity patterns of cortical activity with similarity patterns built from human pose models with different levels of depth availability and viewpoint dependency. Targeting the challenge of explaining variance in complex natural image responses with interpretable models, we achieved statistically significant correlations between pose models and cortical activity patterns (though performance levels are substantially lower than the noise ceiling). We found that the 3D view-independent pose model, compared with two-dimensional models, better captures the activation from distinct cortical areas, including the right posterior superior temporal sulcus (pSTS). These areas, together with other pose-selective regions in the LOTC, form a broader, distributed cortical network with greater view-tolerance in more anterior patches. We interpret these findings in light of the computational complexity of natural body images, the wide range of visual tasks supported by pose structures, and possible shared principles for view-invariant processing between articulated objects and ordinary, rigid objects.

Visual adaptation selective for individual limbs reveals hierarchical human body representation.

The spatial relationships between body parts are a rich source of information for person perception, with even simple pairs of parts providing highly valuable information. Computation of these relationships would benefit from a hierarchical representation, where body parts are represented individually. We hypothesized that the human visual system makes use of such representations. To test this hypothesis, we used adaptation to determine whether observers were sensitive to changes in the length of one body part relative to another. Observers viewed forearm/upper arm pairs where the forearm had been either lengthened or shortened, judging the perceived length of the forearm. Observers then adapted to a variety of different stimuli (e.g., arms, objects, etc.) in different orientations and visual field locations. We found that following adaptation to distorted limbs, but not non-limb objects, observers experienced a shift in perceived forearm length. Furthermore, this effect partially transferred across different orientations and visual field locations. Taken together, these results suggest the effect arises in high level mechanisms specialized for specific body parts, providing evidence for a representation of bodies based on parts and their relationships.

Temporal multivariate pattern analysis (tMVPA): A single trial approach exploring the temporal dynamics of the BOLD signal.

BACKGROUND: fMRI provides spatial resolution that is unmatched by non-invasive neuroimaging techniques. Its temporal dynamics however are typically neglected due to the sluggishness of the hemodynamic signal. NEW METHODS: We present temporal multivariate pattern analysis (tMVPA), a method for investigating the temporal evolution of neural representations in fMRI data, computed on single-trial BOLD time-courses, leveraging both spatial and temporal components of the fMRI signal. We implemented an expanding sliding window approach that allows identifying the time-window of an effect. RESULTS: We demonstrate that tMVPA can successfully detect condition-specific multivariate modulations over time, in the absence of mean BOLD amplitude differences. Using Monte-Carlo simulations and synthetic data, we quantified family-wise error rate (FWER) and statistical power. Both at the group and single-subject levels, FWER was either at or significantly below 5%. We reached the desired power with 18 subjects and 12 trials for the group level, and with 14 trials in the single-subject scenario. COMPARISON WITH EXISTING METHODS: We compare the tMVPA statistical evaluation to that of a linear support vector machine (SVM). SVM outperformed tMVPA with large N and trial numbers. Conversely, tMVPA, leveraging on single trials analyses, outperformed SVM in low N and trials and in a single-subject scenario. CONCLUSION: Recent evidence suggesting that the BOLD signal carries finer-grained temporal information than previously thought, advocates the need for analytical tools, such as tMVPA, tailored to investigate BOLD temporal dynamics. The comparable performance between tMVPA and SVM, a powerful and reliable tool for fMRI, supports the validity of our technique.

The impact of configural superiority on the processing of spatial information.

The impact of context on perception has been well documented for over a century. In some cases, the introduction of context to a set of target features may produce a unified percept, leading to a quicker and more accurate classification; a configural superiority effect (Pomerantz, Sager, & Stoever, 1977). Although this effect has been well characterized in terms of the stimulus features that produce the effect, the specific impact context has on the spatial strategies adopted by observers when making perceptual judgments remains unclear. Here, we sought to address this question by using the methods of response classification and ideal observer analysis. In our main experiment, we used a stimulus set known to produce the configural superiority effect and found that although observers were faster in the presence of context, they were actually less efficient at extracting stimulus information. This surprising result was attributable to the use of a spatial strategy in which observers relied on redundant, noninformative features in the presence of context. A control experiment ruled out the possibility that the mere presence of added context led to these strategic shifts. Our results support previous notions about the nature of the perceptual shifts that are induced by the configural superiority effect. However, they also show that configural processing is more nuanced than originally thought: Although observers may be faster at making judgments when context induces the percept of a configural whole, there appears to be a hidden cost in terms of the efficiency with which information is used. (PsycINFO Database Record

Working Memory Systems in the Rat.

A fundamental feature of memory in humans is the ability to simultaneously work with multiple types of information using independent memory systems. Working memory is conceptualized as two independent memory systems under executive control [1, 2]. Although there is a long history of using the term "working memory" to describe short-term memory in animals, it is not known whether multiple, independent memory systems exist in nonhumans. Here, we used two established short-term memory approaches to test the hypothesis that spatial and olfactory memory operate as independent working memory resources in the rat. In the olfactory memory task, rats chose a novel odor from a gradually incrementing set of old odors [3]. In the spatial memory task, rats searched for a depleting food source at multiple locations [4]. We presented rats with information to hold in memory in one domain (e.g., olfactory) while adding a memory load in the other domain (e.g., spatial). Control conditions equated the retention interval delay without adding a second memory load. In a further experiment, we used proactive interference [5-7] in the spatial domain to compromise spatial memory and evaluated the impact of adding an olfactory memory load. Olfactory and spatial memory are resistant to interference from the addition of a memory load in the other domain. Our data suggest that olfactory and spatial memory draw on independent working memory systems in the rat.

The perception of a familiar face is no more than the sum of its parts.

Why do faces become easier to recognize with repeated exposure? Previous research has suggested that familiarity may induce a qualitative shift in visual processing from an independent analysis of individual facial features to analysis that includes information about the relationships among features (Farah, Wilson, Drain, & Tanaka Psychological Review, 105, 482-498, 1998; Maurer, Grand, & Mondloch Trends in Cognitive Science, 6, 255-260, 2002). We tested this idea by using a "summation-at-threshold" technique (Gold, Mundy, & Tjan Psychological Science, 23, 427-434, 2012; Nandy & Tjan Journal of Vision, 8, 3.1-20, 2008), in which an observer's ability to recognize each individual facial feature shown independently is used to predict their ability to recognize all of the features shown in combination. We find that, although people are better overall at recognizing familiar as opposed to unfamiliar faces, their ability to integrate information across features is similar for unfamiliar and highly familiar faces and is well predicted by their ability to recognize each of the facial features shown in isolation. These results are consistent with the idea that familiarity has a quantitative effect on the efficiency with which information is extracted from individual features, rather than a qualitative effect on the process by which features are combined.