The anterior hippocampus is associated with spatial memory encoding.

There are many hypotheses regarding specialization of the anterior versus posterior hippocampus including memory encoding versus retrieval and other cognitive processes versus spatial memory. In the present functional magnetic resonance imaging study, we distinguished between the hypothesis linking encoding to the anterior hippocampus and the hypothesis linking spatial memory to the posterior hippocampus by evaluating whether spatial memory encoding involved the anterior hippocampus or the posterior hippocampus. During encoding, participants viewed abstract shapes in each of four visual field quadrants while instructed to maintain central fixation. During retrieval, old shapes were presented at fixation and participants identified the previous quadrant of each shape. A general linear model analysis did not reveal encoding activations in the anterior or posterior hippocampus. These results motivated a multi-voxel pattern analysis to assess whether there were distinct patterns of activity associated with encoding shapes in each quadrant within the anterior or posterior hippocampus. For each participant, patterns of activity associated with each quadrant were split by run (i.e., odd runs versus even runs) and the patterns in half the data were used to classify patterns in the other half of the data. Classification accuracy for items at encoding, collapsed over subsequent accuracy, was significantly above chance in the anterior but not posterior hippocampus. The present findings indicate that spatial memory encoding is associated with patterns of activity in the anterior hippocampus.

Distinct regions of the hippocampus are associated with memory for different spatial locations.

In the present functional magnetic resonance imaging (fMRI) study, we aimed to evaluate whether distinct regions of the hippocampus were associated with spatial memory for items presented in different locations of the visual field. In Experiment 1, during the study phase, participants viewed abstract shapes in the left or right visual field while maintaining central fixation. At test, old shapes were presented at fixation and participants classified each shape as previously in the "left" or "right" visual field followed by an "unsure"-"sure"-"very sure" confidence rating. Accurate spatial memory for shapes in the left visual field was isolated by contrasting accurate versus inaccurate spatial location responses. This contrast produced one hippocampal activation in which the interaction between item type and accuracy was significant. The analogous contrast for right visual field shapes did not produce activity in the hippocampus; however, the contrast of high confidence versus low confidence right-hits produced one hippocampal activation in which the interaction between item type and confidence was significant. In Experiment 2, the same paradigm was used but shapes were presented in each quadrant of the visual field during the study phase. Accurate memory for shapes in each quadrant, exclusively masked by accurate memory for shapes in the other quadrants, produced a distinct activation in the hippocampus. A multi-voxel pattern analysis (MVPA) of hippocampal activity revealed a significant correlation between behavioral spatial location accuracy and hippocampal MVPA accuracy across participants. The findings of both experiments indicate that distinct hippocampal regions are associated with memory for different visual field locations.

Encoding-Stage Crosstalk Between Object- and Spatial Property-Based Scene Processing Pathways.

Scene categorization draws on 2 information sources: The identities of objects scenes contain and scenes' intrinsic spatial properties. Because these resources are formally independent, it is possible for them to leads to conflicting judgments of scene category. We tested the hypothesis that the potential for such conflicts is mitigated by a system of "crosstalk" between object- and spatial layout-processing pathways, under which the encoded spatial properties of scenes are biased by scenes' object contents. Specifically, we show that the presence of objects strongly associated with a given scene category can bias the encoded spatial properties of scenes containing them toward the average of that category, an effect which is evident both in behavioral measures of scenes' perceived spatial properties and in scene-evoked multivoxel patterns recorded with functional magnetic resonance imaging from the parahippocampal place area (PPA), a region associated with the processing of scenes' spatial properties. These results indicate that harmonization of object- and spatial property-based estimates of scene identity begins when spatial properties are encoded, and that the PPA plays a central role in this process.

Do simultaneously viewed objects influence scene recognition individually or as groups? Two perceptual studies.

The ability to quickly categorize visual scenes is critical to daily life, allowing us to identify our whereabouts and to navigate from one place to another. Rapid scene categorization relies heavily on the kinds of objects scenes contain; for instance, studies have shown that recognition is less accurate for scenes to which incongruent objects have been added, an effect usually interpreted as evidence of objects' general capacity to activate semantic networks for scene categories they are statistically associated with. Essentially all real-world scenes contain multiple objects, however, and it is unclear whether scene recognition draws on the scene associations of individual objects or of object groups. To test the hypothesis that scene recognition is steered, at least in part, by associations between object groups and scene categories, we asked observers to categorize briefly-viewed scenes appearing with object pairs that were semantically consistent or inconsistent with the scenes. In line with previous results, scenes were less accurately recognized when viewed with inconsistent versus consistent pairs. To understand whether this reflected individual or group-level object associations, we compared the impact of pairs composed of mutually related versus unrelated objects; i.e., pairs, which, as groups, had clear associations to particular scene categories versus those that did not. Although related and unrelated object pairs equally reduced scene recognition accuracy, unrelated pairs were consistently less capable of drawing erroneous scene judgments towards scene categories associated with their individual objects. This suggests that scene judgments were influenced by the scene associations of object groups, beyond the influence of individual objects. More generally, the fact that unrelated objects were as capable of degrading categorization accuracy as related objects, while less capable of generating specific alternative judgments, indicates that the process by which objects interfere with scene recognition is separate from the one through which they inform it.

Evidence for participation by object-selective visual cortex in scene category judgments.

Scene recognition is a core function of the visual system, drawing both on scenes' intrinsic global features, prominently their spatial properties, and on the identities of the objects scenes contain. Neuroimaging and neuropsychological studies have associated spatial property-based scene categorization with parahippocampal cortex, while processing of scene-relevant object information is associated with the lateral occipital complex (LOC), wherein activity patterns distinguish between categories of standalone objects and those embedded in scenes. However, despite the importance of objects to scene categorization and the role of LOC in processing them, damage or disruption to LOC that hampers object recognition has been shown to improve scene categorization. To address this paradox, we used functional magnetic resonance imaging (fMRI) to directly assess the contributions of LOC and the parahippocampal place area (PPA) to category judgments of indoor scenes that were devoid of objective identity signals. Observers were alternately cued to base judgments on scenes' objects or spatial properties. In both LOC and PPA, multivoxel activity patterns better decoded judgments based on their typically associated features: LOC more accurately decoded object-based judgments, while PPA more accurately decoded spatial property-based judgments. The cue contingency of LOC decoding accuracy indicates that it was not an outcome of feedback from judgments and is instead consistent with dependency of judgments on the output of object processing pathways in which LOC participates.

"What?" and "where?" versus "what is where?": the impact of task on coding of object form and position in the lateral occipital complex.

Fast and accurate recognition of both the identities and positions of objects in visual space is critical to deciphering visual environments. Studies in both humans and nonhuman primates have demonstrated that neural populations in ventral temporal visual areas are jointly tuned to both the form and position of objects, allowing information about the identities of objects to be "tagged" with their positions. Because not all behaviors demand that the identities of objects be associated with position information with equal precision, however, the present study asked whether the spatial tuning of form-encoding populations in the human lateral occipital complex (LOC) is sculpted by task demands. Subjects were scanned using functional magnetic resonance imaging while viewing matches of the game Rock, Paper, Scissors played with exemplar pairs from those categories. Subjects first performed a repetition-detection task that depended on object form but not position; subsequently, subjects viewed the same stimuli while determining the position of each pair's "winner," a task that depended upon the conjunction of object form and position. Compared to data from the initial scan, multivoxel activity patterns evoked in the lateral occipital (LO) subdivision of LOC while subjects judged winners showed enhanced sensitivity to the relative positions of objects in pairs. Although superficially consistent with dynamic position tuning, this effect appears to be attributable to an accompanying task-dependent improvement in the sensitivity of LO populations to object form. The results thus suggest that the spatial tuning of form-encoding populations in LO does not depend upon the precision of spatial information demanded by a task.

Joint neuronal tuning for object form and position in the human lateral occipital complex.

A long-standing heuristic in visual neuroscience holds that extrastriate visual cortex is parceled into a dorsal "where" pathway concerned with stimulus position and motion and a ventral "what" pathway concerned with stimulus form. Several recent studies using functional magnetic resonance imaging (fMRI), however, have shown that small changes in the position of a single object can produce reliable changes in activity patterns in object-selective lateral occipital complex (LOC). Although these data demonstrate that information about both object form and position is present at the region level in LOC, the extent to which they reflect joint neuronal tuning to these dimensions is unclear. To measure joint tuning for form and position, we used fMRI to record patterns of activity evoked in LOC and other visual areas while subjects viewed pairs of objects that varied in category content, overall position, and relative object position. Consistent with previous results, multivoxel activity patterns in LOC varied reliably with the category content and position of object pairs. Moreover, activity patterns in the lateral occipital (LO) subregion of LOC varied significantly with the relative positions of objects within pairs, even when absolute pair position was constant. This result provides strong evidence for the existence of neuronal populations in LO which are jointly tuned for both object form (i.e., category) and position.

Constructing scenes from objects in human occipitotemporal cortex.

We used functional magnetic resonance imaging (fMRI) to demonstrate the existence of a mechanism in the human lateral occipital (LO) cortex that supports recognition of real-world visual scenes through parallel analysis of within-scene objects. Neural activity was recorded while subjects viewed four categories of scenes and eight categories of 'signature' objects strongly associated with the scenes in three experiments. Multivoxel patterns evoked by scenes in the LO cortex were well predicted by the average of the patterns elicited by their signature objects. By contrast, there was no relationship between scene and object patterns in the parahippocampal place area (PPA), even though this region responds strongly to scenes and is believed to be crucial for scene identification. By combining information about multiple objects within a scene, the LO cortex may support an object-based channel for scene recognition that complements the processing of global scene properties in the PPA.

Distances between real-world locations are represented in the human hippocampus.

Spatial navigation is believed to be guided in part by reference to an internal map of the environment. We used functional magnetic resonance imaging (fMRI) to test for a key aspect of a cognitive map: preservation of real-world distance relationships. University students were scanned while viewing photographs of familiar campus landmarks. fMRI response levels in the left hippocampus corresponded to real-world distances between landmarks shown on successive trials, indicating that this region considered closer landmarks to be more representationally similar and more distant landmarks to be more representationally distinct. In contrast, posterior visually responsive regions such as retrosplenial complex and the parahippocampal place area were sensitive to landmark repetition and encoded landmark identity in their multivoxel activity patterns but did not show a distance-related response. These data suggest the existence of a map-like representation in the human medial temporal lobe that encodes the coordinates of familiar locations in large-scale, real-world environments.

Eye-centered encoding of visual space in scene-selective regions.

We used functional magnetic resonance imaging (fMRI) to investigate the reference frames used to encode visual information in scene-responsive cortical regions. At early levels of the cortical visual hierarchy, neurons possess spatially selective receptive fields (RFs) that are yoked to specific locations on the retina. In lieu of this eye-centered organization, we speculated that visual areas implicated in scene processing, such as the parahippocampal place area (PPA), the retrosplenial complex (RSC), and transverse occipital sulcus (TOS) might instead possess RFs defined in head-, body-, or world-centered reference frames. To test this, we scanned subjects while they viewed objects and scenes presented at four screen locations while they maintained fixation at one of three possible gaze positions. We then examined response profiles as a function of either fixation-referenced or screen-referenced position. Contrary to our prediction, the PPA and TOS exhibited position-response curves that moved with the fixation point rather than being anchored to the screen, a pattern indicative of eye-centered encoding. RSC, on the other hand, did not exhibit a position-response curve in either reference frame. By showing an important commonality between the PPA/TOS and other visually responsive regions, the results emphasize the critical involvement of these regions in the visual analysis of scenes.

Decoding the representation of multiple simultaneous objects in human occipitotemporal cortex.

Previous work using functional magnetic resonance imaging has shown that the identities of isolated objects viewed by human subjects can be extracted from distributed patterns of brain activity. Outside the laboratory, however, objects almost never appear in isolation; thus it is important to understand how multiple simultaneously occurring objects are encoded by the visual system. We used multivoxel pattern analysis to examine this issue, testing whether activity patterns in the lateral occipital complex (LOC) evoked by object pairs showed an ordered relationship to patterns evoked by their constituent objects. Applying a searchlight analysis to identify voxels with the highest signal-to-noise ratios, we found that responses to object pairs within these informative voxels were well predicted by the averages of responses to their constituent objects. Consistent with this result, we were able to classify object pairs by using synthetic patterns created by averaging single-object patterns. These results indicate that the representation of multiple objects in LOC is governed by a response normalization mechanism similar to that reported in visual areas of several nonhuman species. They also suggest a population coding scheme that preserves information about multiple objects under conditions of distributed attention, facilitating fast object and scene recognition during natural vision.

A precise form of divisive suppression supports population coding in the primary visual cortex.

The responses of neurons in the primary visual cortex (V1) to an optimally oriented grating are suppressed when a non-optimal grating is superimposed. Although cross-orientation suppression is thought to reflect mechanisms that maintain a distributed code for orientation, the effect of superimposed gratings on V1 population responses is unknown. Using intrinsic signal optical imaging, we found that patterns of tree shrew V1 activity evoked by superimposed equal-contrast gratings were predicted by the averages of patterns evoked by individual component gratings. This prediction held across contrasts, for summed sinusoidal gratings or nonsumming square-wave gratings, and was evident in single-unit extracellular recordings. Intracellular recordings revealed consistent levels of suppression throughout the time course of subthreshold responses. These results indicate that divisive suppression powerfully governs population responses to multiple orientations. Moreover, the specific form of suppression that we observed appears to support independent population codes for stimulus orientation and strength and calls for a reassessment of mechanisms that underlie cross-orientation suppression.

Macaque V1 activity during natural vision: effects of natural scenes and saccades.

In the present study, we examined the way that scene complexity and saccades combine to sculpt the temporal response patterns of V1 neurons. To bridge the gap between conventional and free viewing experiments, we compared responses of neurons across four paradigms ranging from less to more natural. An optimal bar stimulus was either flashed into a receptive field (RF) or brought into it via saccade and was embedded in either a natural scene or a uniform gray background. Responses to a flashed bar tended to be higher with a uniform rather than natural background. The most novel result reported here is that responses evoked by stimuli brought into the RF via saccades were enhanced compared with the same stimuli flashed during steady fixation. No single factor appears to account entirely for this surprising effect, but there were small contributions from fixational saccades and residual activity carried over from the previous fixation. We also found a negative correlation with cells' response "history" in that a larger response on one fixation was associated with a lower response on the subsequent fixation. The effects of the natural background and saccades exhibited a significant nonlinear interaction with the suppressive effects of the natural background less for stimuli entering RFs with saccades. Together, these results suggest that even responses to standard optimal stimuli are difficult to predict under conditions similar to natural vision, and further demonstrate the importance of naturalistic experimental paradigms to the study of visual processing in V1.

Position selectivity in scene- and object-responsive occipitotemporal regions.

Complex visual scenes preferentially activate several areas of the human brain, including the parahippocampal place area (PPA), the retrosplenial complex (RSC), and the transverse occipital sulcus (TOS). The sensitivity of neurons in these regions to the retinal position of stimuli is unknown, but could provide insight into their roles in scene perception and navigation. To address this issue, we used functional magnetic resonance imaging (fMRI) to measure neural responses evoked by sequences of scenes and objects confined to either the left or right visual hemifields. We also measured the level of adaptation produced when stimuli were either presented first in one hemifield and then repeated in the opposite hemifield or repeated in the same hemifield. Although overall responses in the PPA, RSC, and TOS tended to be higher for contralateral stimuli than for ipsilateral stimuli, all three regions exhibited position-invariant adaptation, insofar as the magnitude of adaptation did not depend on whether stimuli were repeated in the same or opposite hemifields. In contrast, object-selective regions showed significantly greater adaptation when objects were repeated in the same hemifield. These results suggest that neuronal receptive fields (RFs) in scene-selective regions span the vertical meridian, whereas RFs in object-selective regions do not. The PPA, RSC, and TOS may support scene perception and navigation by maintaining stable representations of large-scale features of the visual environment that are insensitive to the shifts in retinal stimulation that occur frequently during natural vision.

Lightness, filling-in, and the fundamental role of context in visual perception.

Visual perception is defined by the unique spatial interactions that distinguish it from the point-to-point precision of a photometer. Over several decades, Lothar Spillmann has made key observations about the nature of these interactions and the role of context in perception. Our lab has explored the perceptual properties of spatial interactions and more generally the importance of visual context for neuronal responses and perception. Our investigations into the spatiotemporal dynamics of lightness provide insight into underlying mechanisms. For example, backward masking and luminance modulation experiments suggest that the representation of a uniformly luminous object develops first at the borders and, in some manner, the center fills in. The temporal dynamics of lightness induction are also consistent with a filling-in process. There is a slow cutoff temporal frequency above which surround luminance modulation will not elicit perceptual induction of a central area. The larger the central area, the lower the cutoff frequency for induction, perhaps indicating that an edge-based process requires more time to "complete" the larger area. In recordings from primary visual cortex we find that neurons respond in a manner surprisingly consistent with lightness perception and the spatial and temporal properties of induction. For example, the activity of V1 neurons can be modulated by light outside the receptive field and as the modulation rate is increased response modulation falls off more rapidly for large uniform areas than smaller areas. The conclusion we draw from these experiments is that lightness appears to be computed slowly on the basis of edge and context information. A possible role for the spatial interactions is lightness constancy, which is thought to depend on extensive spatial integration. We find not only that V1 responses are strongly context dependent, but that this dependence makes V1 lightness constant on average. The dependence of constancy on surround interactions underscores the fundamental role that context plays in perception. In more recent studies, further support has been found for the importance of context in experiments using natural scene stimuli.

Visual physiology: perceived size looms large.

Visual illusions tell us that size perception depends heavily upon complex contextual cues, often thought to be extracted by brain areas high in the visual hierarchy. Now, a new study shows that perceived size is reflected in activity as early as the primary visual cortex.

The importance of modulatory input for V1 activity and perception.

To conduct well-controlled studies of visual processing in the laboratory, deviations from natural visual situations must generally be employed. In some regards, the reduced visual paradigms typically used are adequate for providing an accurate description of visual representations. However, the use of fixation paradigms and stimuli isolated within a receptive field may underestimate the richness of visual processing in area V1. Experiments ranging from lightness encoding and perception to paradigms involving natural scenes and saccades used to examine the relationship between V1 activity and perception are reviewed in this chapter. Using more complex and natural visual stimulation, V1 responses have been detected that are significantly different from responses obtained in more reduced paradigms. A feature common to the findings of different experiments is that the scale of the activated neural population and circuitry appears to play a key role in the correlation between V1 activity and perception. More complex and natural visual stimulation brings into play extra-receptive field modulatory input not involved with stimulation localized to the receptive field. The results suggest that rather than subtly sculpting the response, modulatory input coming from intra- and/or intercortical sources is fundamental in establishing perceptual response patterns in natural visual situations.

Perception of brightness and brightness illusions in the macaque monkey.

Recent physiological studies show that neural responses correlated with the perception of brightness are found in cortical area V1 but not earlier in the visual pathway (Kayama et al., 1979; Reid and Shapley, 1989; Squatrito et al., 1990; Komatsu et al., 1996; Rossi et al., 1996; MacEvoy et al., 1998; Rossi and Paradiso, 1999; Hung et al., 2001; Kinoshita and Komatsu, 2001; MacEvoy and Paradiso, 2001). However, these studies are based on comparisons of neural responses in animals with brightness perception in humans. Very little is known about the perception of brightness in animals typically used in physiological experiments. In this study, we quantify brightness discrimination, brightness induction, and White's effect in macaque monkeys. The results show that, qualitatively and quantitatively, the perception of brightness in macaques and humans is quite similar. This similarity may be an indication of common underlying neural computations in the two species.