Familiarity enhances functional connectivity between visual and nonvisual regions of the brain during natural viewing.

We explored the neural correlates of familiarity with people and places using a naturalistic viewing paradigm. Neural responses were measured using functional magnetic resonance imaging, while participants viewed a movie taken from Game of Thrones. We compared inter-subject correlations and functional connectivity in participants who were either familiar or unfamiliar with the TV series. Higher inter-subject correlations were found between familiar participants in regions, beyond the visual brain, that are typically associated with the processing of semantic, episodic, and affective information. However, familiarity also increased functional connectivity between face and scene regions in the visual brain and the nonvisual regions of the familiarity network. To determine whether these regions play an important role in face recognition, we measured responses in participants with developmental prosopagnosia (DP). Consistent with a deficit in face recognition, the effect of familiarity was significantly attenuated across the familiarity network in DP. The effect of familiarity on functional connectivity between face regions and the familiarity network was also attenuated in DP. These results show that the neural response to familiarity involves an extended network of brain regions and that functional connectivity between visual and nonvisual regions of the brain plays an important role in the recognition of people and places during natural viewing.

Mapping the functional and structural connectivity of the scene network.

The recognition and perception of places has been linked to a network of scene-selective regions in the human brain. While previous studies have focussed on functional connectivity between scene-selective regions themselves, less is known about their connectivity with other cortical and subcortical regions in the brain. Here, we determine the functional and structural connectivity profile of the scene network. We used fMRI to examine functional connectivity between scene regions and across the whole brain during rest and movie-watching. Connectivity within the scene network revealed a bias between posterior and anterior scene regions implicated in perceptual and mnemonic aspects of scene perception respectively. Differences between posterior and anterior scene regions were also evident in the connectivity with cortical and subcortical regions across the brain. For example, the Occipital Place Area (OPA) and posterior Parahippocampal Place Area (PPA) showed greater connectivity with visual and dorsal attention networks, while anterior PPA and Retrosplenial Complex showed preferential connectivity with default mode and frontoparietal control networks and the hippocampus. We further measured the structural connectivity of the scene network using diffusion tractography. This indicated both similarities and differences with the functional connectivity, highlighting biases between posterior and anterior regions, but also between ventral and dorsal scene regions. Finally, we quantified the structural connectivity between the scene network and major white matter tracts throughout the brain. These findings provide a map of the functional and structural connectivity of scene-selective regions to each other and the rest of the brain.

No evidence for an other-race effect in dominance and trustworthy judgements from faces.

A variety of evidence shows that social categorization of people based on their race can lead to stereotypical judgements and prejudicial behaviour. Here, we explore the extent to which trait judgements of faces are influenced by race. To address this issue, we measured the reliability of first impressions for own-race and other-race faces in Asian and White participants. Participants viewed pairs of faces and were asked to indicate which of the two faces was more dominant or which of the two faces was more trustworthy. We measured the consistency (or reliability) of these judgements across participants for own-race and other-races faces. We found that judgements of dominance or trustworthiness showed similar levels of reliability for own-race and other-race faces. Moreover, an item analysis showed that the judgements on individual trials were very similar across participants from different races. Next, participants made overall ratings of dominance and trustworthiness from own-race and other-race faces. Again, we found that there was no evidence for an ORE. Together, these results provide a new approach to measuring trait judgements of faces and show that in these conditions there is no ORE for the perception of dominance and trustworthiness.

Connectopic mapping techniques do not reflect functional gradients in the brain.

Functional gradients, in which response properties change gradually across a brain region, have been proposed as a key organising principle of the brain. Recent studies using both resting-state and natural viewing paradigms have indicated that these gradients may be reconstructed from functional connectivity patterns via "connectopic mapping" analyses. However, local connectivity patterns may be confounded by spatial autocorrelations artificially introduced during data analysis, for instance by spatial smoothing or interpolation between coordinate spaces. Here, we investigate whether such confounds can produce illusory connectopic gradients. We generated datasets comprising random white noise in subjects' functional volume spaces, then optionally applied spatial smoothing and/or interpolated the data to a different volume or surface space. Both smoothing and interpolation induced spatial autocorrelations sufficient for connectopic mapping to produce both volume- and surface-based local gradients in numerous brain regions. Furthermore, these gradients appeared highly similar to those obtained from real natural viewing data, although gradients generated from real and random data were statistically different in certain scenarios. We also reconstructed global gradients across the whole-brain - while these appeared less susceptible to artificial spatial autocorrelations, the ability to reproduce previously reported gradients was closely linked to specific features of the analysis pipeline. These results indicate that previously reported gradients identified by connectopic mapping techniques may be confounded by artificial spatial autocorrelations introduced during the analysis, and in some cases may reproduce poorly across different analysis pipelines. These findings imply that connectopic gradients need to be interpreted with caution.

The representation of shape and texture in category-selective regions of ventral-temporal cortex.

Neuroimaging studies using univariate and multivariate approaches have shown that the fusiform face area (FFA) and parahippocampal place area (PPA) respond selectively to images of faces and places. The aim of this study was to determine the extent to which this selectivity to faces or places is based on the shape or texture properties of the images. Faces and houses were filtered to manipulate their texture properties, while preserving the shape properties (spatial envelope) of the images. In Experiment 1, multivariate pattern analysis (MVPA) showed that patterns of fMRI response to faces and houses in FFA and PPA were predicted by the shape properties, but not by the texture properties of the image. In Experiment 2, a univariate analysis (fMR-adaptation) showed that responses in the FFA and PPA were sensitive to changes in both the shape and texture properties of the image. These findings can be explained by the spatial scale of the representation of images in the FFA and PPA. At a coarser scale (revealed by MVPA), the neural selectivity to faces and houses is sensitive to variation in the shape properties of the image. However, at a finer scale (revealed by fMR-adaptation), the neural selectivity is sensitive to the texture properties of the image. By combining these neuroimaging paradigms, our results provide insights into the spatial scale of the neural representation of faces and places in the ventral-temporal cortex.

Symmetrical Viewpoint Representations in Face-Selective Regions Convey an Advantage in the Perception and Recognition of Faces.

Learning new identities is crucial for effective social interaction. A critical aspect of this process is the integration of different images from the same face into a view-invariant representation that can be used for recognition. The representation of symmetrical viewpoints has been proposed to be a key computational step in achieving view-invariance. The aim of this study was to determine whether the representation of symmetrical viewpoints in face-selective regions is directly linked to the perception and recognition of face identity. In Experiment 1, we measured fMRI responses while male and female human participants viewed images of real faces from different viewpoints (-90, -45, 0, 45, and 90° from full-face view). Within the face regions, patterns of neural response to symmetrical views (-45 and 45° or -90 and 90°) were more similar than responses to nonsymmetrical views in the fusiform face area and superior temporal sulcus, but not in the occipital face area. In Experiment 2, participants made perceptual similarity judgements to pairs of face images. Images with symmetrical viewpoints were reported as being more similar than nonsymmetric views. In Experiment 3, we asked whether symmetrical views also convey an advantage when learning new faces. We found that recognition was best when participants were tested with novel face images that were symmetrical to the learning viewpoint. Critically, the pattern of perceptual similarity and recognition across different viewpoints predicted the pattern of neural response in face-selective regions. Together, our results provide support for the functional value of symmetry as an intermediate step in generating view-invariant representations.SIGNIFICANCE STATEMENT The recognition of identity from faces is crucial for successful social interactions. A critical step in this process is the integration of different views into a unified, view-invariant representation. The representation of symmetrical views (e.g., left profile and right profile) has been proposed as an important intermediate step in computing view-invariant representations. We found view symmetric representations were specific to some face-selective regions, but not others. We also show that these neural representations influence the perception of faces. Symmetric views were perceived to be more similar and were recognized more accurately than nonsymmetric views. Moreover, the perception and recognition of faces at different viewpoints predicted patterns of response in those face regions with view symmetric representations.

Neural Correlates of Group Bias During Natural Viewing.

Individuals from different social groups interpret the world in different ways. This study explores the neural basis of these group differences using a paradigm that simulates natural viewing conditions. Our aim was to determine if group differences could be found in sensory regions involved in the perception of the world or were evident in higher-level regions that are important for the interpretation of sensory information. We measured brain responses from 2 groups of football supporters, while they watched a video of matches between their teams. The time-course of response was then compared between individuals supporting the same (within-group) or the different (between-group) team. We found high intersubject correlations in low-level and high-level regions of the visual brain. However, these regions of the brain did not show any group differences. Regions that showed higher correlations for individuals from the same group were found in a network of frontal and subcortical brain regions. The interplay between these regions suggests a range of cognitive processes from motor control to social cognition and reward are important in the establishment of social groups. These results suggest that group differences are primarily reflected in regions involved in the evaluation and interpretation of the sensory input.

Selectivity for mid-level properties of faces and places in the fusiform face area and parahippocampal place area.

Regions in the ventral visual pathway, such as the fusiform face area (FFA) and parahippocampal place area (PPA) are selective for images from specific object categories. Yet images from different object categories differ in their image properties. To investigate how these image properties are represented in the FFA and PPA, we compared neural responses to locally-SCRAMBLED images (in which mid-level, spatial properties are preserved) and globally-SCRAMBLED images (in which mid-level, spatial properties are not preserved). There was a greater response in the FFA and PPA to images from the preferred CATEGORY relative to their non-preferred category for the scrambled conditions. However, there was a greater selectivity for locally-scrambled compared to globally-scrambled images. Next, we compared the magnitude of fMR-adaptation to intact and scrambled images. fMR-adaptation was evident to locally-scrambled images from the preferred category. However, there was no adaptation to globally-scrambled images from the preferred category. These results show that the selectivity to faces and places in the FFA and PPA is dependent on mid-level properties of the image that are preserved by local-scrambling.

A data-driven approach to stimulus selection reveals an image-based representation of objects in high-level visual areas.

The ventral visual pathway is directly involved in the perception and recognition of objects. However, the extent to which the neural representation of objects in this region reflects low-level or high-level properties remains unresolved. A problem in resolving this issue is that only a small proportion of the objects experienced during natural viewing can be shown during a typical experiment. This can lead to an uneven sampling of objects that biases our understanding of how they are represented. To address this issue, we developed a data-driven approach to stimulus selection that involved describing a large number objects in terms of their image properties. In the first experiment, clusters of objects were evenly selected from this multi-dimensional image space. Although the clusters did not have any consistent semantic features, each elicited a distinct pattern of neural response. In the second experiment, we asked whether high-level, category-selective patterns of response could be elicited by objects from other categories, but with similar image properties. Object clusters were selected based on the similarity of their image properties to objects from five different categories (bottle, chair, face, house, and shoe). The pattern of response to each metameric object cluster was similar to the pattern elicited by objects from the corresponding category. For example, the pattern for bottles was similar to the pattern for objects with similar image properties to bottles. In both experiments, the patterns of response were consistent across participants providing evidence for common organising principles. This study provides a more ecological approach to understanding the perceptual representations of objects and reveals the importance of image properties.

Distinct but Overlapping Patterns of Response to Words and Faces in the Fusiform Gyrus.

Converging evidence suggests that the fusiform gyrus is involved in the processing of both faces and words. We used fMRI to investigate the extent to which the representation of words and faces in this region of the brain is based on a common neural representation. In Experiment 1, a univariate analysis revealed regions in the fusiform gyrus that were only selective for faces and other regions that were only selective for words. However, we also found regions that showed both word-selective and face-selective responses, particularly in the left hemisphere. We then used a multivariate analysis to measure the pattern of response to faces and words. Despite the overlap in regional responses, we found distinct patterns of response to both faces and words in the left and right fusiform gyrus. In Experiment 2, fMR adaptation was used to determine whether information about familiar faces and names is integrated in the fusiform gyrus. Distinct regions of the fusiform gyrus showed adaptation to either familiar faces or familiar names. However, there was no adaptation to sequences of faces and names with the same identity. Taken together, these results provide evidence for distinct, but overlapping, neural representations for words and faces in the fusiform gyrus.

Patterns of neural response in scene-selective regions of the human brain are affected by low-level manipulations of spatial frequency.

Neuroimaging studies have found distinct patterns of response to different categories of scenes. However, the relative importance of low-level image properties in generating these response patterns is not fully understood. To address this issue, we directly manipulated the low level properties of scenes in a way that preserved the ability to perceive the category. We then measured the effect of these manipulations on category-selective patterns of fMRI response in the PPA, RSC and OPA. In Experiment 1, a horizontal-pass or vertical-pass orientation filter was applied to images of indoor and natural scenes. The image filter did not have a large effect on the patterns of response. For example, vertical- and horizontal-pass filtered indoor images generated similar patterns of response. Similarly, vertical- and horizontal-pass filtered natural scenes generated similar patterns of response. In Experiment 2, low-pass or high-pass spatial frequency filters were applied to the images. We found that image filter had a marked effect on the patterns of response in scene-selective regions. For example, low-pass indoor images generated similar patterns of response to low-pass natural images. The effect of filter varied across different scene-selective regions, suggesting differences in the way that scenes are represented in these regions. These results indicate that patterns of response in scene-selective regions are sensitive to the low-level properties of the image, particularly the spatial frequency content.

Spatial properties of objects predict patterns of neural response in the ventral visual pathway.

Neuroimaging studies have revealed topographically organised patterns of response to different objects in the ventral visual pathway. These patterns are thought to be based on the form of the object. However, it is not clear what dimensions of object form are important. Here, we determined the extent to which spatial properties (energy across the image) could explain patterns of response in these regions. We compared patterns of fMRI response to images from different object categories presented at different retinal sizes. Although distinct neural patterns were evident to different object categories, changing the size (and thus the spatial properties) of the images had a significant effect on these patterns. Next, we used a computational approach to determine whether more fine-grained differences in the spatial properties can explain the patterns of neural response to different objects. We found that the spatial properties of the image were able to predict patterns of neural response, even when categorical factors were removed from the analysis. We also found that the effect of spatial properties on the patterns of response varies across the ventral visual pathway. These results show how spatial properties can be an important organising principle in the topography of the ventral visual pathway.

Category-selective patterns of neural response in the ventral visual pathway in the absence of categorical information.

Neuroimaging studies have revealed distinct patterns of response to different object categories in the ventral visual pathway. These findings imply that object category is an important organizing principle in this region of visual cortex. However, object categories also differ systematically in their image properties. So, it is possible that these patterns of neural response could reflect differences in image properties rather than object category. To differentiate between these alternative explanations, we used images of objects that had been phase-scrambled at a local or global level. Both scrambling processes preserved many of the lower-level image properties, but rendered the images unrecognizable. We then measured the effect of image scrambling on the patterns of neural response within the ventral pathway. We found that intact and scrambled images evoked distinct category-selective patterns of activity in the ventral stream. Moreover, intact and scrambled images of the same object category produced highly similar patterns of response. These results suggest that the neural representation in the ventral visual pathway is tightly linked to the statistical properties of the image.

Modelling the perceptual similarity of facial expressions from image statistics and neural responses.

The ability to perceive facial expressions of emotion is essential for effective social communication. We investigated how the perception of facial expression emerges from the image properties that convey this important social signal, and how neural responses in face-selective brain regions might track these properties. To do this, we measured the perceptual similarity between expressions of basic emotions, and investigated how this is reflected in image measures and in the neural response of different face-selective regions. We show that the perceptual similarity of different facial expressions (fear, anger, disgust, sadness, happiness) can be predicted by both surface and feature shape information in the image. Using block design fMRI, we found that the perceptual similarity of expressions could also be predicted from the patterns of neural response in the face-selective posterior superior temporal sulcus (STS), but not in the fusiform face area (FFA). These results show that the perception of facial expression is dependent on the shape and surface properties of the image and on the activity of specific face-selective regions.

Low-level image properties of visual objects predict patterns of neural response across category-selective regions of the ventral visual pathway.

Neuroimaging studies have revealed strong selectivity for object categories in high-level regions of the human visual system. However, it is unknown whether this selectivity is truly based on object category, or whether it reflects tuning for low-level features that are common to images from a particular category. To address this issue, we measured the neural response to different object categories across the ventral visual pathway. Each object category elicited a distinct neural pattern of response. Next, we compared the patterns of neural response between object categories. We found a strong positive correlation between the neural patterns and the underlying low-level image properties. Importantly, this correlation was still evident when the within-category correlations were removed from the analysis. Next, we asked whether basic image properties could also explain variation in the pattern of response to different exemplars from one object category (faces). A significant correlation was also evident between the similarity of neural patterns of response and the low-level properties of different faces, particularly in regions associated with face processing. These results suggest that the appearance of category-selective regions at this coarse scale of representation may be explained by the systematic convergence of responses to low-level features that are characteristic of each category.

Neural responses to expression and gaze in the posterior superior temporal sulcus interact with facial identity.

Neural models of human face perception propose parallel pathways. One pathway (including posterior superior temporal sulcus, pSTS) is responsible for processing changeable aspects of faces such as gaze and expression, and the other pathway (including the fusiform face area, FFA) is responsible for relatively invariant aspects such as identity. However, to be socially meaningful, changes in expression and gaze must be tracked across an individual face. Our aim was to investigate how this is achieved. Using functional magnetic resonance imaging, we found a region in pSTS that responded more to sequences of faces varying in gaze and expression in which the identity was constant compared with sequences in which the identity varied. To determine whether this preferential response to same identity faces was due to the processing of identity in the pSTS or was a result of interactions between pSTS and other regions thought to code face identity, we measured the functional connectivity between face-selective regions. We found increased functional connectivity between the pSTS and FFA when participants viewed same identity faces compared with different identity faces. Together, these results suggest that distinct neural pathways involved in expression and identity interact to process the changeable features of the face in a socially meaningful way.

Morphing between expressions dissociates continuous from categorical representations of facial expression in the human brain.

Whether the brain represents facial expressions as perceptual continua or as emotion categories remains controversial. Here, we measured the neural response to morphed images to directly address how facial expressions of emotion are represented in the brain. We found that face-selective regions in the posterior superior temporal sulcus and the amygdala responded selectively to changes in facial expression, independent of changes in identity. We then asked whether the responses in these regions reflected categorical or continuous neural representations of facial expression. Participants viewed images from continua generated by morphing between faces posing different expressions such that the expression could be the same, could involve a physical change but convey the same emotion, or could differ by the same physical amount but be perceived as two different emotions. We found that the posterior superior temporal sulcus was equally sensitive to all changes in facial expression, consistent with a continuous representation. In contrast, the amygdala was only sensitive to changes in expression that altered the perceived emotion, demonstrating a more categorical representation. These results offer a resolution to the controversy about how facial expression is processed in the brain by showing that both continuous and categorical representations underlie our ability to extract this important social cue.

Low-level properties of natural images predict topographic patterns of neural response in the ventral visual pathway.

Neuroimaging research over the past 20 years has begun to reveal a picture of how the human visual system is organized. A key distinction that has arisen from these studies is the difference in the organization of low-level and high-level visual regions. Low-level regions contain topographic maps that are tightly linked to properties of the image. In contrast, high-level visual areas are thought to be arranged in modules that are tightly linked to categorical or semantic information in the image. To date, an unresolved question has been how the strong functional selectivity for object categories in high-level visual regions might arise from the image-based representations found in low-level visual regions. Here, we review recent evidence suggesting that patterns of response in high-level visual areas may be better explained by response to image properties that are characteristic of different object categories.

Patterns of response to visual scenes are linked to the low-level properties of the image.

Scene-selective regions in the brain play an important role in the way that we navigate through our visual environment. However, the principles that govern the organization of these regions are not fully understood. For example, it is not clear whether patterns of response in scene-selective regions are linked to high-level semantic category or to low-level spatial structure in scenes. To address this issue, we used multivariate pattern analysis with fMRI to compare patterns of response to different categories of scenes. Although we found distinct patterns of neural response to each category of scene, the magnitude of the within-category similarity varied across different scenes. To determine whether this variation in the categorical response to scenes could reflect variation in the low-level image properties, we measured the similarity of images from each category of scene. Although we found that the low-level properties of images from each category were more similar to each other than to other categories of scenes, we also found that the magnitude of the within-category similarity varied across different scenes. Finally, we compared variation in the neural response to different categories of scenes with corresponding variation in the low-level image properties. We found a strong positive correlation between the similarity in the patterns of neural response to different scenes and the similarity in the image properties. Together, these results suggest that categorical patterns of response to scenes are linked to the low-level properties of the images.

Brain regions involved in processing facial identity and expression are differentially selective for surface and edge information.

Although different brain regions are widely considered to be involved in the recognition of facial identity and expression, it remains unclear how these regions process different properties of the visual image. Here, we ask how surface-based reflectance information and edge-based shape cues contribute to the perception and neural representation of facial identity and expression. Contrast-reversal was used to generate images in which normal contrast relationships across the surface of the image were disrupted, but edge information was preserved. In a behavioural experiment, contrast-reversal significantly attenuated judgements of facial identity, but only had a marginal effect on judgements of expression. An fMR-adaptation paradigm was then used to ask how brain regions involved in the processing of identity and expression responded to blocks comprising all normal, all contrast-reversed, or a mixture of normal and contrast-reversed faces. Adaptation in the posterior superior temporal sulcus--a region directly linked with processing facial expression--was relatively unaffected by mixing normal with contrast-reversed faces. In contrast, the response of the fusiform face area--a region linked with processing facial identity--was significantly affected by contrast-reversal. These results offer a new perspective on the reasons underlying the neural segregation of facial identity and expression in which brain regions involved in processing invariant aspects of faces, such as identity, are very sensitive to surface-based cues, whereas regions involved in processing changes in faces, such as expression, are relatively dependent on edge-based cues.