Spatial pattern of BOLD fMRI activation reveals cross-modal information in auditory cortex.

Recent findings suggest that neural representations in early auditory cortex reflect not only the physical properties of a stimulus, but also high-level, top-down, and even cross-modal information. However, the nature of cross-modal information in auditory cortex remains poorly understood. Here, we used pattern analyses of fMRI data to ask whether early auditory cortex contains information about the visual environment. Our data show that 1) early auditory cortex contained information about a visual stimulus when there was no bottom-up auditory signal, and that 2) no influence of visual stimulation was observed in auditory cortex when visual stimuli did not provide a context relevant to audition. Our findings attest to the capacity of auditory cortex to reflect high-level, top-down, and cross-modal information and indicate that the spatial patterns of activation in auditory cortex reflect contextual/implied auditory information but not visual information per se.

Recognition alters the spatial pattern of FMRI activation in early retinotopic cortex.

Early retinotopic cortex has traditionally been viewed as containing a veridical representation of the low-level properties of the image, not imbued by high-level interpretation and meaning. Yet several recent results indicate that neural representations in early retinotopic cortex reflect not just the sensory properties of the image, but also the perceived size and brightness of image regions. Here we used functional magnetic resonance imaging pattern analyses to ask whether the representation of an object in early retinotopic cortex changes when the object is recognized compared with when the same stimulus is presented but not recognized. Our data confirmed this hypothesis: the pattern of response in early retinotopic visual cortex to a two-tone "Mooney" image of an object was more similar to the response to the full grayscale photo version of the same image when observers knew what the two-tone image represented than when they did not. Further, in a second experiment, high-level interpretations actually overrode bottom-up stimulus information, such that the pattern of response in early retinotopic cortex to an identified two-tone image was more similar to the response to the photographic version of that stimulus than it was to the response to the identical two-tone image when it was not identified. Our findings are consistent with prior results indicating that perceived size and brightness affect representations in early retinotopic visual cortex and, further, show that even higher-level information--knowledge of object identity--also affects the representation of an object in early retinotopic cortex.

Representation of perceived object shape by the human lateral occipital complex.

The human lateral occipital complex (LOC) has been implicated in object recognition, but it is unknown whether this region represents low-level image features or perceived object shape. We used an event-related functional magnetic resonance imaging adaptation paradigm in which the response to pairs of successively presented stimuli is lower when they are identical than when they are different. Adaptation across a change between the two stimuli in a pair provides evidence for a common neural representation invariant to that change. We found adaptation in the LOC when perceived shape was identical but contours differed, but not when contours were identical but perceived shape differed. These data indicate that the LOC represents not simple image features, but rather higher level shape information.

Free and forced diving in birds.

Heart rates were measured during free and forced diving on each of two species of aquatic birds: the double-crested cormorant (Phalacrocorax auritus), a true diver, and the Canada goose (Branta candensis), a bottom feeder in shallow water. When they immersed voluntarily they showed no bradycardia, but when the same birds were forcibly held under water there was a rapid drop in heart rate to well below that at rest. This decrease indicates that ther may be a large component of emotional stress in the heart rate records from previous diving studies where restrained animals were forcibly submerged.

A cortical area selective for visual processing of the human body.

Despite extensive evidence for regions of human visual cortex that respond selectively to faces, few studies have considered the cortical representation of the appearance of the rest of the human body. We present a series of functional magnetic resonance imaging (fMRI) studies revealing substantial evidence for a distinct cortical region in humans that responds selectively to images of the human body, as compared with a wide range of control stimuli. This region was found in the lateral occipitotemporal cortex in all subjects tested and apparently reflects a specialized neural system for the visual perception of the human body.

The generality of parietal involvement in visual attention.

Functional magnetic resonance imaging (fMRI) was used to determine whether different kinds of visual attention rely on a common neural substrate. Within one session, subjects performed three different attention experiments (each comparing an attentionally demanding task with an easier task using identical stimuli): (1) peripheral shifting, (2) object matching, and (3) a nonspatial conjunction task. Two areas were activated in all three experiments: one at the junction of intraparietal and transverse occipital sulci (IPTO), and another in the anterior intraparietal sulcus (AIPS). These regions are not simply involved in any effortful task, because they were not activated in a fourth experiment comparing a difficult language task with an easier control task. Thus, activity in IPTO and AIPS generalizes across a wide variety of attention-requiring tasks, supporting the existence of a common neural substrate underlying multiple modes of visual selection.

Binocular rivalry and visual awareness in human extrastriate cortex.

We used functional magnetic resonance imaging (fMRI) to monitor stimulus-selective responses of the human fusiform face area (FFA) and parahippocampal place area (PPA) during binocular rivalry in which a face and a house stimulus were presented to different eyes. Though retinal stimulation remained constant, subjects perceived changes from house to face that were accompanied by increasing FFA and decreasing PPA activity; perceived changes from face to house led to the opposite pattern of responses. These responses during rivalry were equal in magnitude to those evoked by nonrivalrous stimulus alternation, suggesting that activity in the FFA and PPA reflects the perceived rather than the retinal stimulus, and that neural competition during binocular rivalry has been resolved by these stages of visual processing.

The parahippocampal place area: recognition, navigation, or encoding?

The parahippocampal place area (PPA) has been demonstrated to respond more strongly in fMRI to scenes depicting places than to other kinds of visual stimuli. Here, we test several hypotheses about the function of the PPA. We find that PPA activity (1) is not affected by the subjects' familiarity with the place depicted, (2) does not increase when subjects experience a sense of motion through the scene, and (3) is greater when viewing novel versus repeated scenes but not novel versus repeated faces. Thus, we find no evidence that the PPA is involved in matching perceptual information to stored representations in memory, in planning routes, or in monitoring locomotion through the local or distal environment but some evidence that it is involved in encoding new perceptual information about the appearance and layout of scenes.

Attention response functions: characterizing brain areas using fMRI activation during parametric variations of attentional load.

We derived attention response functions for different cortical areas by plotting neural activity (measured by fMRI) as a function of attentional load in a visual tracking task. In many parietal and frontal cortical areas, activation increased with load over the entire range of loads tested, suggesting that these areas are directly involved in attentional processes. However, in other areas (FEF and parietal area 7), strong activation was observed even at the lowest attentional load (compared to a passive baseline using identical stimuli), but little or no additional activation was seen with increasing load. These latter areas appear to play a different role, perhaps supporting task-relevant functions that do not vary with load, such as the suppression of eye movements.

Neuroimaging of cognitive functions in human parietal cortex.

Functional neuroimaging has proven highly valuable in mapping human sensory regions, particularly visual areas in occipital cortex. Recent evidence suggests that human parietal cortex may also consist of numerous specialized subregions similar to those reported in neurophysiological studies of non-human primates. However, parietal activation generalizes across a wide variety of cognitive tasks and the extension of human brain mapping into higher-order "association cortex" may prove to be a challenge.

Perceiving visually presented objects: recognition, awareness, and modularity.

Object perception may involve seeing, recognition, preparation of actions, and emotional responses--functions that human brain imaging and neuropsychology suggest are localized separately. Perhaps because of this specialization, object perception is remarkably rapid and efficient. Representations of componential structure and interpolation from view-dependent images both play a part in object recognition. Unattended objects may be implicitly registered, but recent experiments suggest that attention is required to bind features, to represent three-dimensional structure, and to mediate awareness.

Cortical regions involved in perceiving object shape.

The studies described here use functional magnetic resonance imaging to test whether common or distinct cognitive and/or neural mechanisms are involved in extracting object structure from the different image cues defining an object's shape, such as contours, shading, and monocular depth cues. We found overlapping activations in the lateral and ventral occipital cortex [known as the lateral occipital complex (LOC)] for objects defined by different visual cues (e.g., grayscale photographs and line drawings) when each was compared with its own scrambled-object control. In a second experiment we found a reduced response when objects were repeated, independent of whether they appeared in the same or a different format (i.e., grayscale images vs line drawings). A third experiment showed that activation in the LOC was no stronger for three-dimensional shapes defined by contours or monocular depth cues, such as occlusion, than for two-dimensional shapes, suggesting that these regions are not selectively involved in processing three-dimensional shape information. These results suggest that common regions in the LOC are involved in extracting and/or representing information about object structure from different image cues.

Testing cognitive models of visual attention with fMRI and MEG.

Neuroimaging techniques can be used not only to identify the neural substrates of attention, but also to test cognitive theories of attention. Here we consider four classic questions in the psychology of visual attention: (i) Are some 'special' classes of stimuli (e.g. faces) immune to attentional modulation?; (ii) What are the information units on which attention operates?; (iii) How early in stimulus processing are attentional effects observed?; and (iv) Are common mechanisms involved in different modes of attentional selection (e.g. spatial and non-spatial selection)? We describe studies from our laboratory that illustrate the ways in which fMRI and MEG can provide key evidence in answering these questions. A central methodological theme in many of our fMRI studies is the use of analyses in which the activity in certain functionally-defined regions of interest (ROIs) is used to test specific cognitive hypotheses. An analogous sensor-of-interest (SOI) approach is applied to MEG. Our results include: evidence for the modulation of face representations by attention; confirmation of the independent contributions of object-based and location-based selection; evidence for modulation of face representations by non-spatial selection within the first 170 ms of processing; and implication of the intraparietal sulcus in functions general to spatial and non-spatial visual selection.

A region of right posterior superior temporal sulcus responds to observed intentional actions.

Human adults and infants identify the actions of another agent based not only on its intrinsic perceptual features, but critically on the contingent relationship between its motion path and the environmental context [Trends Cogn. Sci. 7 (1995) 287; Cognition 72 (2003) 237]. Functional neuroimaging studies of the perception of agents and intentional actions, on the other hand, have mostly focussed on the perception of intrinsic cues to agency, like a face or articulated body motion (e.g. [J. Neurosci. 17 (1997) 4302; Neuroimage 8 (1998) 221; Trends Cogn. Sci. 4 (2000) 267; Nat. Neurosci. 3 (2000) 80; Neuroimage 13 (2001) 775; Proc. Natl. Acad. Sci. U.S.A. 98 (2001) 11656; Neuron 35 (2002) 1167; Neuron 34 (2002) 149, Neuroscience 15 (2003) 991; J. Neurosci. 23 (2003) 6819; Philos. Trans. R Soc. Lond. B. Biol. Sci. 358 (2003) 435]. Here we describe a region of the right posterior superior temporal sulcus that is sensitive not to articulated body motion per se, but to the relationship between the observed motion and the structure of the surrounding environment. From this and other aspects of the region's response, we hypothesize that this region is involved in the representation of observed intentional actions.

Neural events and perceptual awareness.

Neural correlates of perceptual awareness, until very recently an elusive quarry, are now almost commonplace findings. This article first describes a variety of neural correlates of perceptual awareness based on fMRI, ERPs, and single-unit recordings. It is then argued that our quest should ultimately focus not on mere correlates of awareness, but rather on the neural events that are both necessary and sufficient for perceptual awareness. Indeed, preliminary evidence suggests that although many of the neural correlates already reported may be necessary for the corresponding state of awareness, it is unlikely that they are sufficient for it. The final section considers three hypotheses concerning the possible sufficiency conditions for perceptual awareness.

The effect of face inversion on the human fusiform face area.

Inversion severely impairs the recognition of greyscale faces and the ability to see the stimulus as a face in two-tone Mooney images. We used functional magnetic resonance imaging to study the effect of face inversion on the human fusiform face area (FFA). MR signal intensity from the FFA was reduced when greyscale faces were presented upside-down, but this effect was small and inconsistent across subjects when subjects were required to attend to both upright and inverted faces. However when two-tone faces were inverted, the MR signal from the FFA was substantially reduced for all subjects. We conclude that (i) the FFA responds to faces per se, rather than to the low-level visual features present in faces, and (ii) inverted greyscale faces can strongly activate this face-specific mechanism.

The fusiform face area is selective for faces not animals.

To test whether the human fusiform face area (FFA) responds not only to faces but to anything human or animate, we used fMRI to measure the response of the FFA to six new stimulus categories. The strongest responses were to stimuli containing faces: human faces (2.0% signal increase from fixation baseline) and human heads (1.7%), with weaker but still strong responses to whole humans (1.5%) and animal heads (1.3%). Responses to whole animals (1.0%) and human bodies without heads (1.0%) were significantly stronger than responses to inanimate objects (0.7%), but responses to animal bodies without heads (0.8%) were not. These results demonstrate that the FFA is selective for faces, not for animals.

Early brain potentials link repetition blindness, priming and novelty detection.

To study mechanisms of visual object identification in humans, event-related potentials (ERPs) were recorded during successful or unsuccessful identification of rapid, serially presented words (unrepeated or repeated). We observed 'repetition blindness' (RB): more repeated than unrepeated words were incorrectly reported. ERPs from repetition-blinded words exhibited little or none of the enhanced positivity found for correctly reported repeated words, resembling instead ERPs from any unrepeated sequence initially, but only incorrectly reported unrepeated sequences later. Thus it appears that in RB an early (220 ms) neural operation that normally initiates facilitated processing from immediate repetition priming erroneously processes a repeated item as novel. This operation (possibly in basotemporal neocortex) appears to induce differential subsequent processing of novel vs repeated information.

The selectivity of the occipitotemporal M170 for faces.

Evidence from fMRI, ERPs and intracranial recordings suggests the existence of face-specific mechanisms in the primate occipitotemporal cortex. The present study used a 64-channel MEG system to monitor neural activity while normal subjects viewed a sequence of grayscale photographs of a variety of unfamiliar faces and non-face stimuli. In 14 of 15 subjects, face stimuli evoked a larger response than non-face stimuli at a latency of 160 ms after stimulus onset at bilateral occipitotemporal sensors. Inverted face stimuli elicited responses that were no different in amplitude but 13 ms later in latency than upright faces. The profile of this M170 response across stimulus conditions is largely consistent with prior results using scalp and subdural ERPs.