Effects of unilateral cortical resection of the visual cortex on bilateral human white matter.

Children with unilateral resections of ventral occipito-temporal cortex (VOTC) typically do not evince visual perceptual impairments, even when relatively large swathes of VOTC are resected. In search of possible explanations for this behavioral competence, we evaluated white matter microstructure and connectivity in eight pediatric epilepsy patients following unilateral cortical resection and 15 age-matched controls. To uncover both local and broader resection-induced effects, we analyzed tractography data using two complementary approaches. First, the microstructural properties were measured in the inferior longitudinal and the inferior fronto-occipital fasciculi, the major VOTC association tracts. Group differences were only evident in the ipsilesional, and not in the contralesional, hemisphere, and single-subject analyses revealed that these differences were limited to the site of the resection. Second, graph theory was used to characterize the connectivity of the contralesional occipito-temporal regions. There were no changes to the network properties in patients with left VOTC resections nor in patients with resections outside the VOTC, but altered network efficiency was observed in two cases with right VOTC resections. These results suggest that, in many, although perhaps not all, cases of unilateral VOTC resections in childhood, the white matter profile in the preserved contralesional hemisphere along with residual neural activity might be sufficient for normal visual perception.

Dissociations and Associations between Shape and Category Representations in the Two Visual Pathways.

The dorsal and ventral visual pathways represent both visual and conceptual object properties. Yet the relative contribution of these two factors in the representational content of visual areas is unclear. Indeed, research investigating brain category representations rarely dissociate visual and semantic properties of objects. We present a human event-related fMRI study with a two-factorial stimulus set with 54 images that explicitly dissociates shape from category to investigate their independent contribution as well as their interactions through representational similarity analyses. Results reveal a contribution from each dimension in both streams, with a transition from shape to category along the posterior-to-anterior anatomical axis. The nature of category representations differs in the two pathways: ventral areas represent object animacy and dorsal areas represent object action properties. Furthermore, information about shape evolved from low-level pixel-based to high-level perceived shape following a posterior-to-anterior gradient similar to the shape-to-category emergence. To conclude, results show that representations of shape and category independently coexist, but at the same time they are closely related throughout the visual hierarchy. SIGNIFICANCE STATEMENT: Research investigating visual cortex conceptual category representations rarely takes into account visual properties of objects. In this report, we explicitly dissociate shape from category and investigate independent contributions and interactions of these two highly correlated dimensions.

Being BOLD: The neural dynamics of face perception.

According to a non-hierarchical view of human cortical face processing, selective responses to faces may emerge in a higher-order area of the hierarchy, in the lateral part of the middle fusiform gyrus (fusiform face area [FFA]) independently from face-selective responses in the lateral inferior occipital gyrus (occipital face area [OFA]), a lower order area. Here we provide a stringent test of this hypothesis by gradually revealing segmented face stimuli throughout strict linear descrambling of phase information [Ales et al., 2012]. Using a short sampling rate (500 ms) of fMRI acquisition and single subject statistical analysis, we show a face-selective responses emerging earlier, that is, at a lower level of structural (i.e., phase) information, in the FFA compared with the OFA. In both regions, a face detection response emerging at a lower level of structural information for upright than inverted faces, both in the FFA and OFA, in line with behavioral responses and with previous findings of delayed responses to inverted faces with direct recordings of neural activity were also reported. Overall, these results support the non-hierarchical view of human cortical face processing and open new perspectives for time-resolved analysis at the single subject level of fMRI data obtained during continuously evolving visual stimulation. Hum Brain Mapp 38:120-139, 2017. © 2016 Wiley Periodicals, Inc.

A Predominantly Visual Subdivision of The Right Temporo-Parietal Junction (vTPJ).

A multiplicity of sensory and cognitive functions has been attributed to the large cortical region at the temporo-parietal junction (TPJ). Using functional MRI, we report that a small region lateralized within the right TPJ responds robustly to certain simple visual stimuli ("vTPJ"). The vTPJ was found in all right hemispheres (n = 7), posterior to the auditory cortex. To manipulate stimuli and attention, subjects were presented with a mixture of visual and auditory stimuli in a concurrent block design in 2 experiments: (1) A simple visual stimulus (a grating pattern modulating in mean luminance) elicited robust responses in the vTPJ, whether or not the subject attended to vision and(2) a drifting low-contrast dartboard pattern of constant mean luminance evoked robust responses in the vTPJ when it was task-relevant (visual task), and smaller responses when it was not (auditory task). The results suggest a focal, visually responsive region within the right TPJ that is powerfully driven by certain visual stimuli (luminance fluctuations), and that can be driven by other visual stimuli when the subject is attending. The precise localization of this visually responsive region is helpful in segmenting the TPJ and to better understand its role in visual awareness and related disorders such as extinction and neglect.

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream.

Converging evidence suggests that the primate ventral visual pathway encodes increasingly complex stimulus features in downstream areas. We quantitatively show that there indeed exists an explicit gradient for feature complexity in the ventral pathway of the human brain. This was achieved by mapping thousands of stimulus features of increasing complexity across the cortical sheet using a deep neural network. Our approach also revealed a fine-grained functional specialization of downstream areas of the ventral stream. Furthermore, it allowed decoding of representations from human brain activity at an unsurpassed degree of accuracy, confirming the quality of the developed approach. Stimulus features that successfully explained neural responses indicate that population receptive fields were explicitly tuned for object categorization. This provides strong support for the hypothesis that object categorization is a guiding principle in the functional organization of the primate ventral stream.

There Is a "U" in Clutter: Evidence for Robust Sparse Codes Underlying Clutter Tolerance in Human Vision.

The ability to recognize objects in clutter is crucial for human vision, yet the underlying neural computations remain poorly understood. Previous single-unit electrophysiology recordings in inferotemporal cortex in monkeys and fMRI studies of object-selective cortex in humans have shown that the responses to pairs of objects can sometimes be well described as a weighted average of the responses to the constituent objects. Yet, from a computational standpoint, it is not clear how the challenge of object recognition in clutter can be solved if downstream areas must disentangle the identity of an unknown number of individual objects from the confounded average neuronal responses. An alternative idea is that recognition is based on a subpopulation of neurons that are robust to clutter, i.e., that do not show response averaging, but rather robust object-selective responses in the presence of clutter. Here we show that simulations using the HMAX model of object recognition in cortex can fit the aforementioned single-unit and fMRI data, showing that the averaging-like responses can be understood as the result of responses of object-selective neurons to suboptimal stimuli. Moreover, the model shows how object recognition can be achieved by a sparse readout of neurons whose selectivity is robust to clutter. Finally, the model provides a novel prediction about human object recognition performance, namely, that target recognition ability should show a U-shaped dependency on the similarity of simultaneously presented clutter objects. This prediction is confirmed experimentally, supporting a simple, unifying model of how the brain performs object recognition in clutter. SIGNIFICANCE STATEMENT: The neural mechanisms underlying object recognition in cluttered scenes (i.e., containing more than one object) remain poorly understood. Studies have suggested that neural responses to multiple objects correspond to an average of the responses to the constituent objects. Yet, it is unclear how the identities of an unknown number of objects could be disentangled from a confounded average response. Here, we use a popular computational biological vision model to show that averaging-like responses can result from responses of clutter-tolerant neurons to suboptimal stimuli. The model also provides a novel prediction, that human detection ability should show a U-shaped dependency on target-clutter similarity, which is confirmed experimentally, supporting a simple, unifying account of how the brain performs object recognition in clutter.

Mapping the macaque superior temporal sulcus: functional delineation of vergence and version eye-movement-related activity.

It is currently thought that the primate oculomotor system has evolved distinct but interrelated subsystems to generate different types of visually guided eye movements (e.g., saccades/smooth pursuit/vergence). Although progress has been made in elucidating the neural basis of these movement types, no study to date has investigated all three movement types on a large scale and within the same animals. Here, we used fMRI in rhesus macaque monkeys to map the superior temporal sulcus (STS) for BOLD modulation associated with visually guided eye movements. Further, we ascertained whether modulation in a given area was movement type specific and, if not, the modulation each movement type elicited relative to the others (i.e., dominance). Our results show that multiple areas within STS modulate during all movement types studied, including the middle temporal, medial superior temporal, fundus of the superior temporal, lower superior temporal, and dorsal posterior inferotemporal areas. Our results also reveal an area in dorsomedial STS that is modulated almost exclusively by vergence movements. In contrast, we found that ventrolateral STS is driven preferentially during versional movements. These results illuminate an STS network involved in processes associated with multiple eye movement types, illustrate unique patterns of modulation within said network as a function of movement type, and provide evidence for a vergence-specific area within dorsomedial STS. We conclude that producing categorically different eye movement types requires access to a common STS network and that individual network nodes are recruited differentially based upon the type of movement generated.

Population Receptive Field Estimation Reveals New Retinotopic Maps in Human Subcortex.

The human subcortex contains multiple nuclei that govern the transmission of information to and among cortical areas. In the visual domain, these nuclei are organized into retinotopic maps. Because of their small size, these maps have been difficult to precisely measure using phase-encoded functional magnetic resonance imaging, particularly in the eccentricity dimension. Using instead the population receptive field model to estimate the response properties of individual voxels, we were able to resolve two previously unreported retinotopic maps in the thalamic reticular nucleus and the substantia nigra. We measured both the polar angle and eccentricity components, receptive field size and hemodynamic response function delay, in the these nuclei and in the lateral geniculate nucleus, the superior colliculus, and the lateral and intergeniculate pulvinars. The anatomical boundaries of these nuclei were delineated using multiple averaged proton density-weighted images and were used to constrain and confirm the functional activations. Deriving the retinotopic organization of these small, subcortical nuclei is the first step in exploring their response properties and their roles in neural dynamics.

Functional architecture for disparity in macaque inferior temporal cortex and its relationship to the architecture for faces, color, scenes, and visual field.

Binocular disparity is a powerful depth cue for object perception. The computations for object vision culminate in inferior temporal cortex (IT), but the functional organization for disparity in IT is unknown. Here we addressed this question by measuring fMRI responses in alert monkeys to stimuli that appeared in front of (near), behind (far), or at the fixation plane. We discovered three regions that showed preferential responses for near and far stimuli, relative to zero-disparity stimuli at the fixation plane. These "near/far" disparity-biased regions were located within dorsal IT, as predicted by microelectrode studies, and on the posterior inferotemporal gyrus. In a second analysis, we instead compared responses to near stimuli with responses to far stimuli and discovered a separate network of "near" disparity-biased regions that extended along the crest of the superior temporal sulcus. We also measured in the same animals fMRI responses to faces, scenes, color, and checkerboard annuli at different visual field eccentricities. Disparity-biased regions defined in either analysis did not show a color bias, suggesting that disparity and color contribute to different computations within IT. Scene-biased regions responded preferentially to near and far stimuli (compared with stimuli without disparity) and had a peripheral visual field bias, whereas face patches had a marked near bias and a central visual field bias. These results support the idea that IT is organized by a coarse eccentricity map, and show that disparity likely contributes to computations associated with both central (face processing) and peripheral (scene processing) visual field biases, but likely does not contribute much to computations within IT that are implicated in processing color.

The Anatomical and Functional Organization of the Human Visual Pulvinar.

The pulvinar is the largest nucleus in the primate thalamus and contains extensive, reciprocal connections with visual cortex. Although the anatomical and functional organization of the pulvinar has been extensively studied in old and new world monkeys, little is known about the organization of the human pulvinar. Using high-resolution functional magnetic resonance imaging at 3 T, we identified two visual field maps within the ventral pulvinar, referred to as vPul1 and vPul2. Both maps contain an inversion of contralateral visual space with the upper visual field represented ventrally and the lower visual field represented dorsally. vPul1 and vPul2 border each other at the vertical meridian and share a representation of foveal space with iso-eccentricity lines extending across areal borders. Additional, coarse representations of contralateral visual space were identified within ventral medial and dorsal lateral portions of the pulvinar. Connectivity analyses on functional and diffusion imaging data revealed a strong distinction in thalamocortical connectivity between the dorsal and ventral pulvinar. The two maps in the ventral pulvinar were most strongly connected with early and extrastriate visual areas. Given the shared eccentricity representation and similarity in cortical connectivity, we propose that these two maps form a distinct visual field map cluster and perform related functions. The dorsal pulvinar was most strongly connected with parietal and frontal areas. The functional and anatomical organization observed within the human pulvinar was similar to the organization of the pulvinar in other primate species. SIGNIFICANCE STATEMENT: The anatomical organization and basic response properties of the visual pulvinar have been extensively studied in nonhuman primates. Yet, relatively little is known about the functional and anatomical organization of the human pulvinar. Using neuroimaging, we found multiple representations of visual space within the ventral human pulvinar and extensive topographically organized connectivity with visual cortex. This organization is similar to other nonhuman primates and provides additional support that the general organization of the pulvinar is consistent across the primate phylogenetic tree. These results suggest that the human pulvinar, like other primates, is well positioned to regulate corticocortical communication.

Neural Mechanism for Mirrored Self-face Recognition.

Self-face recognition in the mirror is considered to involve multiple processes that integrate 2 perceptual cues: temporal contingency of the visual feedback on one's action (contingency cue) and matching with self-face representation in long-term memory (figurative cue). The aim of this study was to examine the neural bases of these processes by manipulating 2 perceptual cues using a "virtual mirror" system. This system allowed online dynamic presentations of real-time and delayed self- or other facial actions. Perception-level processes were identified as responses to only a single perceptual cue. The effect of the contingency cue was identified in the cuneus. The regions sensitive to the figurative cue were subdivided by the response to a static self-face, which was identified in the right temporal, parietal, and frontal regions, but not in the bilateral occipitoparietal regions. Semantic- or integration-level processes, including amodal self-representation and belief validation, which allow modality-independent self-recognition and the resolution of potential conflicts between perceptual cues, respectively, were identified in distinct regions in the right frontal and insular cortices. The results are supportive of the multicomponent notion of self-recognition and suggest a critical role for contingency detection in the co-emergence of self-recognition and empathy in infants.

The Impact of Density and Ratio on Object-Ensemble Representation in Human Anterior-Medial Ventral Visual Cortex.

Behavioral research has demonstrated that observers can extract summary statistics from ensembles of multiple objects. We recently showed that a region of anterior-medial ventral visual cortex, overlapping largely with the scene-sensitive parahippocampal place area (PPA), participates in object-ensemble representation. Here we investigated the encoding of ensemble density in this brain region using fMRI-adaptation. In Experiment 1, we varied density by changing the spacing between objects and found no sensitivity in PPA to such density changes. Thus, density may not be encoded in PPA, possibly because object spacing is not perceived as an intrinsic ensemble property. In Experiment 2, we varied relative density by changing the ratio of 2 types of objects comprising an ensemble, and observed significant sensitivity in PPA to such ratio change. Although colorful ensembles were shown in Experiment 2, Experiment 3 demonstrated that sensitivity to object ratio change was not driven mainly by a change in the ratio of colors. Thus, while anterior-medial ventral visual cortex is insensitive to density (object spacing) changes, it does code relative density (object ratio) within an ensemble. Object-ensemble processing in this region may thus depend on high-level visual information, such as object ratio, rather than low-level information, such as spacing/spatial frequency.

Gaze-dependent topography in human posterior parietal cortex.

The brain must convert retinal coordinates into those required for directing an effector. One prominent theory holds that, through a combination of visual and motor/proprioceptive information, head-/body-centered representations are computed within the posterior parietal cortex (PPC). An alternative theory, supported by recent visual and saccade functional magnetic resonance imaging (fMRI) topographic mapping studies, suggests that PPC neurons provide a retinal/eye-centered coordinate system, in which the coding of a visual stimulus location and/or intended saccade endpoints should remain unaffected by changes in gaze position. To distinguish between a retinal/eye-centered and a head-/body-centered coordinate system, we measured how gaze direction affected the representation of visual space in the parietal cortex using fMRI. Subjects performed memory-guided saccades from a central starting point to locations "around the clock." Starting points varied between left, central, and right gaze relative to the head-/body midline. We found that memory-guided saccadotopic maps throughout the PPC showed spatial reorganization with very subtle changes in starting gaze position, despite constant retinal input and eye movement metrics. Such a systematic shift is inconsistent with models arguing for a retinal/eye-centered coordinate system in the PPC, but it is consistent with head-/body-centered coordinate representations.

Probabilistic and single-subject retinotopic maps reveal the topographic organization of face patches in the macaque cortex.

Face perception is crucial to survival among social primates. It has been suggested that a group of extrastriate cortical regions responding more strongly to faces than to nonface objects is critical for face processing in primates. It is generally assumed that these regions are not retinotopically organized, as with human face-processing areas, showing foveal bias but lacking any organization with respect to polar angle. Despite many electrophysiological studies targeting monkey face patches, the retinotopic organization of these patches remains largely unclear. We have examined the relationship between cortical face patches and the topographic organization of extrastriate cortex using biologically relevant, phase-encoded retinotopic mapping stimuli in macaques. Single-subject fMRI results indicated a gradual shift from highly retinotopic to no topographic organization from posterior to anterior face patches in inferotemporal cortex. We also constructed a probabilistic retinotopic atlas of occipital and ventral extrastriate visual cortex. By comparing this probabilistic map to the locations of face patches at the group level, we showed that a previously identified posterior lateral temporal face patch (PL) is located within the posterior inferotemporal dorsal (PITd) retinotopic area. Furthermore, we identified a novel face patch posterior PL, which is located in retinotopically organized transitional area V4 (V4t). Previously published coordinates of human PITd coincide with the group-level occipital face area (OFA), according to a probabilistic map derived from a large population, implying a potential correspondence between monkey PL/PITd and human OFA/PITd. Furthermore, the monkey middle lateral temporal face patch (ML) shows consistent foveal biases but no obvious polar-angle structure. In contrast, middle fundus temporal (MF), anterior temporal and prefrontal monkey face patches lacked topographic organization.

Object-specific semantic coding in human perirhinal cortex.

Category-specificity has been demonstrated in the human posterior ventral temporal cortex for a variety of object categories. Although object representations within the ventral visual pathway must be sufficiently rich and complex to support the recognition of individual objects, little is known about how specific objects are represented. Here, we used representational similarity analysis to determine what different kinds of object information are reflected in fMRI activation patterns and uncover the relationship between categorical and object-specific semantic representations. Our results show a gradient of informational specificity along the ventral stream from representations of image-based visual properties in early visual cortex, to categorical representations in the posterior ventral stream. A key finding showed that object-specific semantic information is uniquely represented in the perirhinal cortex, which was also increasingly engaged for objects that are more semantically confusable. These findings suggest a key role for the perirhinal cortex in representing and processing object-specific semantic information that is more critical for highly confusable objects. Our findings extend current distributed models by showing coarse dissociations between objects in posterior ventral cortex, and fine-grained distinctions between objects supported by the anterior medial temporal lobes, including the perirhinal cortex, which serve to integrate complex object information.

Attention-dependent early cortical suppression contributes to crowding.

Crowding, the identification difficulty for a target in the presence of nearby flankers, is ubiquitous in spatial vision and is considered a bottleneck of object recognition and visual awareness. Despite its significance, the neural mechanisms of crowding are still unclear. Here, we performed event-related potential and fMRI experiments to measure the cortical interaction between the target and flankers in human subjects. We found that the magnitude of the crowding effect was closely associated with an early suppressive cortical interaction. The cortical suppression was reflected in the earliest event-related potential component (C1), which originated in V1, and in the BOLD signal in V1, but not other higher cortical areas. Intriguingly, spatial attention played a critical role in the manifestation of the suppression. These findings provide direct and converging evidence that attention-dependent V1 suppression contributes to crowding at a very early stage of visual processing.

The animacy continuum in the human ventral vision pathway.

Major theories for explaining the organization of semantic memory in the human brain are premised on the often-observed dichotomous dissociation between living and nonliving objects. Evidence from neuroimaging has been interpreted to suggest that this distinction is reflected in the functional topography of the ventral vision pathway as lateral-to-medial activation gradients. Recently, we observed that similar activation gradients also reflect differences among living stimuli consistent with the semantic dimension of graded animacy. Here, we address whether the salient dichotomous distinction between living and nonliving objects is actually reflected in observable measured brain activity or whether previous observations of a dichotomous dissociation were the illusory result of stimulus sampling biases. Using fMRI, we measured neural responses while participants viewed 10 animal species with high to low animacy and two inanimate categories. Representational similarity analysis of the activity in ventral vision cortex revealed a main axis of variation with high-animacy species maximally different from artifacts and with the least animate species closest to artifacts. Although the associated functional topography mirrored activation gradients observed for animate-inanimate contrasts, we found no evidence for a dichotomous dissociation. We conclude that a central organizing principle of human object vision corresponds to the graded psychological property of animacy with no clear distinction between living and nonliving stimuli. The lack of evidence for a dichotomous dissociation in the measured brain activity challenges theories based on this premise.

Visual Awareness Is Limited by the Representational Architecture of the Visual System.

Visual perception and awareness have strict limitations. We suggest that one source of these limitations is the representational architecture of the visual system. Under this view, the extent to which items activate the same neural channels constrains the amount of information that can be processed by the visual system and ultimately reach awareness. Here, we measured how well stimuli from different categories (e.g., faces and cars) blocked one another from reaching awareness using two distinct paradigms that render stimuli invisible: visual masking and continuous flash suppression. Next, we used fMRI to measure the similarity of the neural responses elicited by these categories across the entire visual hierarchy. Overall, we found strong brain-behavior correlations within the ventral pathway, weaker correlations in the dorsal pathway, and no correlations in early visual cortex (V1-V3). These results suggest that the organization of higher level visual cortex constrains visual awareness and the overall processing capacity of visual cognition.

Structural similarity exerts opposing effects on perceptual differentiation and categorization: an FMRI study.

We manipulated the degree of structural similarity between objects that had to be matched either according to whether they represented the same object (perceptual matching) or belonged to the same category (conceptual matching). Behaviorally, performance improved as a linear function of increased structural similarity during conceptual matching but deteriorated as a linear function of increased structural similarity during perceptual matching. These effects were mirrored in fMRI recordings where activation in several ventral posterior areas exhibited a similar interaction between match type and structural similarity. Our findings provide direct support for the notion that structural similarity exerts opposing effects on classification depending on whether objects are to be perceptually differentiated or categorized-a notion that has been based on rather circumstantial evidence. In particular, the finding that structural similarity plays a major role in categorization of instances according to taxonomy challenges the view that the organization of superordinate categories is not driven by shared structural features.

Decoding the future from past experience: learning shapes predictions in early visual cortex.

Learning the structure of the environment is critical for interpreting the current scene and predicting upcoming events. However, the brain mechanisms that support our ability to translate knowledge about scene statistics to sensory predictions remain largely unknown. Here we provide evidence that learning of temporal regularities shapes representations in early visual cortex that relate to our ability to predict sensory events. We tested the participants' ability to predict the orientation of a test stimulus after exposure to sequences of leftward- or rightward-oriented gratings. Using fMRI decoding, we identified brain patterns related to the observers' visual predictions rather than stimulus-driven activity. Decoding of predicted orientations following structured sequences was enhanced after training, while decoding of cued orientations following exposure to random sequences did not change. These predictive representations appear to be driven by the same large-scale neural populations that encode actual stimulus orientation and to be specific to the learned sequence structure. Thus our findings provide evidence that learning temporal structures supports our ability to predict future events by reactivating selective sensory representations as early as in primary visual cortex.