Differential sustained and transient temporal processing across visual streams.

How do high-level visual regions process the temporal aspects of our visual experience? While the temporal sensitivity of early visual cortex has been studied with fMRI in humans, temporal processing in high-level visual cortex is largely unknown. By modeling neural responses with millisecond precision in separate sustained and transient channels, and introducing a flexible encoding framework that captures differences in neural temporal integration time windows and response nonlinearities, we predict fMRI responses across visual cortex for stimuli ranging from 33 ms to 20 s. Using this innovative approach, we discovered that lateral category-selective regions respond to visual transients associated with stimulus onsets and offsets but not sustained visual information. Thus, lateral category-selective regions compute moment-to-moment visual transitions, but not stable features of the visual input. In contrast, ventral category-selective regions process both sustained and transient components of the visual input. Our model revealed that sustained channel responses to prolonged stimuli exhibit adaptation, whereas transient channel responses to stimulus offsets are surprisingly larger than for stimulus onsets. This large offset transient response may reflect a memory trace of the stimulus when it is no longer visible, whereas the onset transient response may reflect rapid processing of new items. Together, these findings reveal previously unconsidered, fundamental temporal mechanisms that distinguish visual streams in the human brain. Importantly, our results underscore the promise of modeling brain responses with millisecond precision to understand the underlying neural computations.

The functional neuroanatomy of face perception: from brain measurements to deep neural networks.

A central goal in neuroscience is to understand how processing within the ventral visual stream enables rapid and robust perception and recognition. Recent neuroscientific discoveries have significantly advanced understanding of the function, structure and computations along the ventral visual stream that serve as the infrastructure supporting this behaviour. In parallel, significant advances in computational models, such as hierarchical deep neural networks (DNNs), have brought machine performance to a level that is commensurate with human performance. Here, we propose a new framework using the ventral face network as a model system to illustrate how increasing the neural accuracy of present DNNs may allow researchers to test the computational benefits of the functional architecture of the human brain. Thus, the review (i) considers specific neural implementational features of the ventral face network, (ii) describes similarities and differences between the functional architecture of the brain and DNNs, and (iii) provides a hypothesis for the computational value of implementational features within the brain that may improve DNN performance. Importantly, this new framework promotes the incorporation of neuroscientific findings into DNNs in order to test the computational benefits of fundamental organizational features of the visual system.

Defining the most probable location of the parahippocampal place area using cortex-based alignment and cross-validation.

The parahippocampal place area (PPA) is a widely studied high-level visual region in the human brain involved in place and scene processing. The goal of the present study was to identify the most probable location of place-selective voxels in medial ventral temporal cortex. To achieve this goal, we first used cortex-based alignment (CBA) to create a probabilistic place-selective region of interest (ROI) from one group of 12 participants. We then tested how well this ROI could predict place selectivity in each hemisphere within a new group of 12 participants. Our results reveal that a probabilistic ROI (pROI) generated from one group of 12 participants accurately predicts the location and functional selectivity in individual brains from a new group of 12 participants, despite between subject variability in the exact location of place-selective voxels relative to the folding of parahippocampal cortex. Additionally, the prediction accuracy of our pROI is significantly higher than that achieved by volume-based Talairach alignment. Comparing the location of the pROI of the PPA relative to published data from over 500 participants, including data from the Human Connectome Project, shows a striking convergence of the predicted location of the PPA and the cortical location of voxels exhibiting the highest place selectivity across studies using various methods and stimuli. Specifically, the most predictive anatomical location of voxels exhibiting the highest place selectivity in medial ventral temporal cortex is the junction of the collateral and anterior lingual sulci. Methodologically, we make this pROI freely available (vpnl.stanford.edu/PlaceSelectivity), which provides a means to accurately identify a functional region from anatomical MRI data when fMRI data are not available (for example, in patient populations). Theoretically, we consider different anatomical and functional factors that may contribute to the consistent anatomical location of place selectivity relative to the folding of high-level visual cortex.

Encoding model of temporal processing in human visual cortex.

How is temporal information processed in human visual cortex? Visual input is relayed to V1 through segregated transient and sustained channels in the retina and lateral geniculate nucleus (LGN). However, there is intense debate as to how sustained and transient temporal channels contribute to visual processing beyond V1. The prevailing view associates transient processing predominately with motion-sensitive regions and sustained processing with ventral stream regions, while the opposing view suggests that both temporal channels contribute to neural processing beyond V1. Using fMRI, we measured cortical responses to time-varying stimuli and then implemented a two temporal channel-encoding model to evaluate the contributions of each channel. Different from the general linear model of fMRI that predicts responses directly from the stimulus, the encoding approach first models neural responses to the stimulus from which fMRI responses are derived. This encoding approach not only predicts cortical responses to time-varying stimuli from milliseconds to seconds but also, reveals differential contributions of temporal channels across visual cortex. Consistent with the prevailing view, motion-sensitive regions and adjacent lateral occipitotemporal regions are dominated by transient responses. However, ventral occipitotemporal regions are driven by both sustained and transient channels, with transient responses exceeding the sustained. These findings propose a rethinking of temporal processing in the ventral stream and suggest that transient processing may contribute to rapid extraction of the content of the visual input. Importantly, our encoding approach has vast implications, because it can be applied with fMRI to decipher neural computations in millisecond resolution in any part of the brain.

The Cytoarchitecture of Domain-specific Regions in Human High-level Visual Cortex.

A fundamental hypothesis in neuroscience proposes that underlying cellular architecture (cytoarchitecture) contributes to the functionality of a brain area. However, this hypothesis has not been tested in human ventral temporal cortex (VTC) that contains domain-specific regions causally involved in perception. To fill this gap in knowledge, we used cortex-based alignment to register functional regions from living participants to cytoarchitectonic areas in ex vivo brains. This novel approach reveals 3 findings. First, there is a consistent relationship between domain-specific regions and cytoarchitectonic areas: each functional region is largely restricted to 1 cytoarchitectonic area. Second, extracting cytoarchitectonic profiles from face- and place-selective regions after back-projecting each region to 20-µm thick histological sections indicates that cytoarchitectonic properties distinguish these regions from each other. Third, some cytoarchitectonic areas contain more than 1 domain-specific region. For example, face-, body-, and character-selective regions are located within the same cytoarchitectonic area. We summarize these findings with a parsimonious hypothesis incorporating how cellular properties may contribute to functional specialization in human VTC. Specifically, we link computational principles to correlated axes of functional and cytoarchitectonic segregation in human VTC, in which parallel processing across domains occurs along a lateral-medial axis while transformations of information within domain occur along an anterior-posterior axis.

Development of Neural Sensitivity to Face Identity Correlates with Perceptual Discriminability.

Face perception is subserved by a series of face-selective regions in the human ventral stream, which undergo prolonged development from childhood to adulthood. However, it is unknown how neural development of these regions relates to the development of face-perception abilities. Here, we used functional magnetic resonance imaging (fMRI) to measure brain responses of ventral occipitotemporal regions in children (ages, 5-12 years) and adults (ages, 19-34 years) when they viewed faces that parametrically varied in dissimilarity. Since similar faces generate lower responses than dissimilar faces due to fMRI adaptation, this design objectively evaluates neural sensitivity to face identity across development. Additionally, a subset of subjects participated in a behavioral experiment to assess perceptual discriminability of face identity. Our data reveal three main findings: (1) neural sensitivity to face identity increases with age in face-selective but not object-selective regions; (2) the amplitude of responses to faces increases with age in both face-selective and object-selective regions; and (3) perceptual discriminability of face identity is correlated with the neural sensitivity to face identity of face-selective regions. In contrast, perceptual discriminability is not correlated with the amplitude of response in face-selective regions or of responses of object-selective regions. These data suggest that developmental increases in neural sensitivity to face identity in face-selective regions improve perceptual discriminability of faces. Our findings significantly advance the understanding of the neural mechanisms of development of face perception and open new avenues for using fMRI adaptation to study the neural development of high-level visual and cognitive functions more broadly. SIGNIFICANCE STATEMENT: Face perception, which is critical for daily social interactions, develops from childhood to adulthood. However, it is unknown what developmental changes in the brain lead to improved performance. Using fMRI in children and adults, we find that from childhood to adulthood, neural sensitivity to changes in face identity increases in face-selective regions. Critically, subjects' perceptual discriminability among faces is linked to neural sensitivity: participants with higher neural sensitivity in face-selective regions demonstrate higher perceptual discriminability. Thus, our results suggest that developmental increases in face-selective regions' sensitivity to face identity improve perceptual discrimination of faces. These findings significantly advance understanding of the neural mechanisms underlying the development of face perception and have important implications for assessing both typical and atypical development.

The Face-Processing Network Is Resilient to Focal Resection of Human Visual Cortex.

UNLABELLED: Human face perception requires a network of brain regions distributed throughout the occipital and temporal lobes with a right hemisphere advantage. Present theories consider this network as either a processing hierarchy beginning with the inferior occipital gyrus (occipital face area; IOG-faces/OFA) or a multiple-route network with nonhierarchical components. The former predicts that removing IOG-faces/OFA will detrimentally affect downstream stages, whereas the latter does not. We tested this prediction in a human patient (Patient S.P.) requiring removal of the right inferior occipital cortex, including IOG-faces/OFA. We acquired multiple fMRI measurements in Patient S.P. before and after a preplanned surgery and multiple measurements in typical controls, enabling both within-subject/across-session comparisons (Patient S.P. before resection vs Patient S.P. after resection) and between-subject/across-session comparisons (Patient S.P. vs controls). We found that the spatial topology and selectivity of downstream ipsilateral face-selective regions were stable 1 and 8 month(s) after surgery. Additionally, the reliability of distributed patterns of face selectivity in Patient S.P. before versus after resection was not different from across-session reliability in controls. Nevertheless, postoperatively, representations of visual space were typical in dorsal face-selective regions but atypical in ventral face-selective regions and V1 of the resected hemisphere. Diffusion weighted imaging in Patient S.P. and controls identifies white matter tracts connecting retinotopic areas to downstream face-selective regions, which may contribute to the stable and plastic features of the face network in Patient S.P. after surgery. Together, our results support a multiple-route network of face processing with nonhierarchical components and shed light on stable and plastic features of high-level visual cortex following focal brain damage. SIGNIFICANCE STATEMENT: Brain networks consist of interconnected functional regions commonly organized in processing hierarchies. Prevailing theories predict that damage to the input of the hierarchy will detrimentally affect later stages. We tested this prediction with multiple brain measurements in a rare human patient requiring surgical removal of the putative input to a network processing faces. Surprisingly, the spatial topology and selectivity of downstream face-selective regions are stable after surgery. Nevertheless, representations of visual space were typical in dorsal face-selective regions but atypical in ventral face-selective regions and V1. White matter connections from outside the face network may support these stable and plastic features. As processing hierarchies are ubiquitous in biological and nonbiological systems, our results have pervasive implications for understanding the construction of resilient networks.

Temporal Processing Capacity in High-Level Visual Cortex Is Domain Specific.

Prevailing hierarchical models propose that temporal processing capacity--the amount of information that a brain region processes in a unit time--decreases at higher stages in the ventral stream regardless of domain. However, it is unknown if temporal processing capacities are domain general or domain specific in human high-level visual cortex. Using a novel fMRI paradigm, we measured temporal capacities of functional regions in high-level visual cortex. Contrary to hierarchical models, our data reveal domain-specific processing capacities as follows: (1) regions processing information from different domains have differential temporal capacities within each stage of the visual hierarchy and (2) domain-specific regions display the same temporal capacity regardless of their position in the processing hierarchy. In general, character-selective regions have the lowest capacity, face- and place-selective regions have an intermediate capacity, and body-selective regions have the highest capacity. Notably, domain-specific temporal processing capacities are not apparent in V1 and have perceptual implications. Behavioral testing revealed that the encoding capacity of body images is higher than that of characters, faces, and places, and there is a correspondence between peak encoding rates and cortical capacities for characters and bodies. The present evidence supports a model in which the natural statistics of temporal information in the visual world may affect domain-specific temporal processing and encoding capacities. These findings suggest that the functional organization of high-level visual cortex may be constrained by temporal characteristics of stimuli in the natural world, and this temporal capacity is a characteristic of domain-specific networks in high-level visual cortex. Significance statement: Visual stimuli bombard us at different rates every day. For example, words and scenes are typically stationary and vary at slow rates. In contrast, bodies are dynamic and typically change at faster rates. Using a novel fMRI paradigm, we measured temporal processing capacities of functional regions in human high-level visual cortex. Contrary to prevailing theories, we find that different regions have different processing capacities, which have behavioral implications. In general, character-selective regions have the lowest capacity, face- and place-selective regions have an intermediate capacity, and body-selective regions have the highest capacity. These results suggest that temporal processing capacity is a characteristic of domain-specific networks in high-level visual cortex and contributes to the segregation of cortical regions.

Multiple object properties drive scene-selective regions.

Neuroimaging studies have identified brain regions that respond preferentially to specific stimulus categories, including 3 areas that activate maximally during viewing of real-world scenes: The parahippocampal place area (PPA), retrosplenial complex (RSC), and transverse occipital sulcus (TOS). Although these findings suggest the existence of regions specialized for scene processing, this interpretation is challenged by recent reports that activity in scene-preferring regions is modulated by properties of isolated single objects. To understand the mechanisms underlying these object-related responses, we collected functional magnetic resonance imaging data while subjects viewed objects rated along 7 dimensions, shown both in isolation and on a scenic background. Consistent with previous reports, we find that scene-preferring regions are sensitive to multiple object properties; however, results of an item analysis suggested just 2 independent factors--visual size and the landmark suitability of the objects--sufficed to explain most of the response. This object-based modulation was found in PPA and RSC irrespective of the presence or absence of a scenic background, but was only observed in TOS for isolated objects. We hypothesize that scene-preferring regions might process both visual qualities unique to scenes and spatial qualities that can appertain to either scenes or objects.

Humans have precise knowledge of familiar geographical slants.

Whereas maps primarily represent the 2-dimensional layout of the environment, people are also aware of the 3-dimensional layout of their environment. An experiment conducted on a small college campus tested whether the remembered slants of familiar paths were precisely represented. Three measures of slant (verbal, manual, and pictorial) were collected in 2 different between-subject conditions (perception and memory) for 5 familiar paths on the campus of Swarthmore College, ranging in slant from 0.5° to 8.6°. Estimates from memory and from perception did not differ for any of the measures. Moreover, estimates from all measures, though different in mean value, were correlated within participant, suggesting a common underlying representation was consulted in all cases.

An imputed dissociation might be an artifact: Further evidence for the generalizability of the observations of Durgin et al. 2010.

We recently showed that palm board measures are systematically inaccurate for full-cue surfaces within reach of one's hand, whereas free-hand gestures and reaching actions are quite accurate for such surfaces (Durgin, Hajnal, Li, Tonge, & Stigliani, 2010). Proffitt and Zadra (2010) claim that our demonstration that palm boards are highly inaccurate is irrelevant to interpreting past and present findings concerning dissociations between verbal reports and palm board estimates. In their paper they offer a theoretical representation of the findings of Bhalla and Proffitt (1999) and argue that our analysis is incompatible with their account. We offer here an alternative account of the findings of Bhalla and Proffitt, based on their actual data (which are fully compatible with our original analysis). We further show how our account generalizes to more recent studies that continue (1) to mistakenly describe null statistical effects on (insensitive) palm boards as evidence of a "dissociation" from (more sensitive) verbal measures that show a similar relative magnitude of change and (2) to introduce uncontrolled demand characteristics.

Palm boards are not action measures: an alternative to the two-systems theory of geographical slant perception.

Whereas most reports of the perception of outdoor hills demonstrate dramatic overestimation, estimates made by adjusting a palm board are much closer to the true hill orientation. We test the dominant hypothesis that palm board accuracy is related to the need for motor action to be accurately guided and conclude instead that the perceptual experience of palm-board orientation is biased and variable due to poorly calibrated proprioception of wrist flexion. Experiments 1 and 3 show that wrist-flexion palm boards grossly underestimate the orientations of near, reachable surfaces whereas gesturing with a free hand is fairly accurate. Experiment 2 shows that palm board estimates are much lower than free hand estimates for an outdoor hill as well. Experiments 4 shows that wrist flexion is biased and noisy compared to elbow flexion, while Experiment 5 shows that small changes in palm board height produce large changes in palm board estimates. Together, these studies suggest that palm boards are biased and insensitive measures. The existing literature arguing that there are two systems in the perception of geographical slant is re-evaluated, and a new theoretical framework is proposed in which a single exaggerated representation of ground-surface orientation guides both action and perception.