Distributed representations of behaviorally-relevant object dimensions in the human visual system.

Object vision is commonly thought to involve a hierarchy of brain regions processing increasingly complex image features, with high-level visual cortex supporting object recognition and categorization. However, object vision supports diverse behavioral goals, suggesting basic limitations of this category-centric framework. To address these limitations, we mapped a series of behaviorally-relevant dimensions derived from a large-scale analysis of human similarity judgments directly onto the brain. Our results reveal broadly distributed representations of behaviorally-relevant information, demonstrating selectivity to a wide variety of novel dimensions while capturing known selectivities for visual features and categories. Behaviorally-relevant dimensions were superior to categories at predicting brain responses, yielding mixed selectivity in much of visual cortex and sparse selectivity in category-selective clusters. This framework reconciles seemingly disparate findings regarding regional specialization, explaining category selectivity as a special case of sparse response profiles among representational dimensions, suggesting a more expansive view on visual processing in the human brain.

The temporal evolution of conceptual object representations revealed through models of behavior, semantics and deep neural networks.

Visual object representations are commonly thought to emerge rapidly, yet it has remained unclear to what extent early brain responses reflect purely low-level visual features of these objects and how strongly those features contribute to later categorical or conceptual representations. Here, we aimed to estimate a lower temporal bound for the emergence of conceptual representations by defining two criteria that characterize such representations: 1) conceptual object representations should generalize across different exemplars of the same object, and 2) these representations should reflect high-level behavioral judgments. To test these criteria, we compared magnetoencephalography (MEG) recordings between two groups of participants (n = 16 per group) exposed to different exemplar images of the same object concepts. Further, we disentangled low-level from high-level MEG responses by estimating the unique and shared contribution of models of behavioral judgments, semantics, and different layers of deep neural networks of visual object processing. We find that 1) both generalization across exemplars as well as generalization of object-related signals across time increase after 150 ms, peaking around 230 ms; 2) representations specific to behavioral judgments emerged rapidly, peaking around 160 ms. Collectively, these results suggest a lower bound for the emergence of conceptual object representations around 150 ms following stimulus onset.

Neuronal representation of disappearing and hidden objects in temporal cortex of the macaque.

Neurons in the anterior regions of the banks of the superior temporal sulcus (STSa) of the macaque monkey respond to the sight of biologically significant stimuli such as faces, bodies and their motion. In this study the responses of STSa neurons were recorded during the gradual occlusion of the experimenter and other mobile objects behind screens at distances of 0.5-4 m from the monkeys. The experimenter or other object remained out of sight for 3-15 s before emerging back in to view. We describe a population of neurons (n=33) showing increased activity during the occlusion of objects that was maintained for up to 11 s following complete occlusion (when only the occluder itself was visible). This increase in activity was selective for the position of the occlusion within the testing room. Many neurons showed little or no change in activity prior to occlusion when the object or experimenter was completely in view. By coding for the presence and location of recently occluded objects, these responses may contribute to the perceptual capacity for object permanence.

Neural representation for the perception of the intentionality of actions.

A novel population of cells is described, located in the anterior part of the superior temporal sulcus (STSa, sometimes called STPa) of the temporal lobe in the macaque monkey. These cells respond selectively to the sight of reaching but only when the agent performing the action is seen to be attending to the target position of the reaching. We describe how such conditional selectivity can be generated from the properties of distinct cell populations within STSa. One cell population responds selectively to faces, eye gaze, and body posture, and we argue that subsets of these cells code for the direction of attention of others. A second cell population is selectively responsive to limb movement in certain directions (e.g., responding to an arm movement to the left but not to an equivalent leg movement or vice versa). The responses of a subset of cells sensitive to limb movement are modulated by the direction of attention (indicated by head and body posture of the agent performing the action). We conclude that this combined analysis of direction of attention and body movements supports the detection of intentional actions.

Gaze following and joint attention in rhesus monkeys (Macaca mulatta).

Gaze and attention direction provide important sources of social information for primates. Behavioral studies show that chimpanzees spontaneously follow human gaze direction. By contrast, non-ape species such as macaques fail to follow gaze cues. The authors investigated the reactions of rhesus macaques (Macaca mulatta) to attention cues of conspecifics. Two subjects were presented with videotaped images of a stimulus monkey with its attention directed to 1 of 2 identical objects. Analysis of eye movements revealed that both subjects inspected the target (object or position attended by the stimulus monkey) more often than the distractor (nonattended object or position). These results provide evidence that rhesus monkeys follow gaze and use the attention cues of other monkeys to orient their own attention to objects.