Normalization by orientation-tuned surround in human V1-V3.

An influential account of neuronal responses in primary visual cortex is the normalized energy model. This model is often implemented as a multi-stage computation. The first stage is linear filtering. The second stage is the extraction of contrast energy, whereby a complex cell computes the squared and summed outputs of a pair of the linear filters in quadrature phase. The third stage is normalization, in which a local population of complex cells mutually inhibit one another. Because the population includes cells tuned to a range of orientations and spatial frequencies, the result is that the responses are effectively normalized by the local stimulus contrast. Here, using evidence from human functional MRI, we show that the classical model fails to account for the relative responses to two classes of stimuli: straight, parallel, band-passed contours (gratings), and curved, band-passed contours (snakes). The snakes elicit fMRI responses that are about twice as large as the gratings, yet a traditional divisive normalization model predicts responses that are about the same. Motivated by these observations and others from the literature, we implement a divisive normalization model in which cells matched in orientation tuning ("tuned normalization") preferentially inhibit each other. We first show that this model accounts for differential responses to these two classes of stimuli. We then show that the model successfully generalizes to other band-pass textures, both in V1 and in extrastriate cortex (V2 and V3). We conclude that even in primary visual cortex, complex features of images such as the degree of heterogeneity, can have large effects on neural responses.

Attention preserves the selectivity of feature-tuned normalization.

Attention and divisive normalization both contribute to making visual processing more efficient. Attention selectively increases the neural gain of relevant information in the early visual cortex, resulting in stronger perceived salience for attended regions or features. Divisive normalization improves processing efficiency by suppressing responses to homogeneous inputs and highlighting salient boundaries, facilitating sparse coding of inputs. Theoretical and empirical research suggest a tight link between attention and normalization, wherein attending to a stimulus results in a release from normalization, thereby allowing for an increase in neural response gain. In the present study, we address whether attention alters the qualitative properties of normalization. Specifically, we examine how attention influences the feature-tuned nature of normalization, whereby suppression is stronger between visual stimuli whose orientation contents are similar, and weaker when the orientations are different. Ten human observers viewed stimuli that varied in orientation content while we acquired fMRI BOLD responses under two attentional states: attending toward or attending away from the stimulus. Our results indicate that attention does not alter the specificity of feature-tuned normalization. Instead, attention seems to enhance visuocortical responses evenly, regardless of the degree of orientation similarity within the stimulus. Since visuocortical responses exhibit adaptation to statistical regularities in natural scenes, we conclude that while attention can selectively increase the gain of responses to attended items, it does not appear to alter the ecologically relevant correspondence between orientation differences and strength of tuned normalization.NEW & NOTEWORTHY The magnitude of visuocortical BOLD responses scales with orientation differences in visual stimuli, with the strongest response suppression for collinear stimuli and least suppression for orthogonal, in a way that appears to match natural scene statistics. We examined the effects of attention on this feature-tuned property of suppression and found that while attending to a stimulus increases the overall gain of visuocortical responses, the qualitative properties of feature-tuning remain unchanged, suggesting attention preserves tuned normalization properties.

Saturating Nonlinearities of Contrast Response in Human Visual Cortex.

Response nonlinearities are ubiquitous throughout the brain, especially within sensory cortices where changes in stimulus intensity typically produce compressed responses. Although this relationship is well established in electrophysiological measurements, it remains controversial whether the same nonlinearities hold for population-based measurements obtained with human fMRI. We propose that these purported disparities are not contingent on measurement type and are instead largely dependent on the visual system state at the time of interrogation. We show that deploying a contrast adaptation paradigm permits reliable measurements of saturating sigmoidal contrast response functions (10 participants, 7 female). When not controlling the adaptation state, our results coincide with previous fMRI studies, yielding nonsaturating, largely linear contrast responses. These findings highlight the important role of adaptation in manifesting measurable nonlinear responses within human visual cortex, reconciling discrepancies reported in vision neuroscience, re-establishing the qualitative relationship between stimulus intensity and response across different neural measures and the concerted study of cortical gain control.SIGNIFICANCE STATEMENT Nonlinear stimulus-response relationships govern many essential brain functions, ranging from the sensory to cognitive level. Certain core response properties previously shown to be nonlinear with nonhuman electrophysiology recordings have yet to be reliably measured with human neuroimaging, prompting uncertainty and reconsideration. The results of this study stand to reconcile these incongruencies in the vision neurosciences, demonstrating the profound impact adaptation can have on brain activation throughout the early visual cortex. Moving forward, these findings facilitate the study of modulatory influences on sensory processing (i.e., arousal and attention) and help establish a closer link between neural recordings in animals and hemodynamic measurements from human fMRI, resuming a concerted effort to understand operations in the mammalian cortex.

The specificity of orientation-tuned normalization within human early visual cortex.

Normalization within visual cortex is modulated by contextual influences; stimuli sharing similar features suppress each other more than dissimilar stimuli. This feature-tuned component of suppression depends on multiple factors, including the orientation content of stimuli. Indeed, pairs of stimuli arranged in a center-surround configuration attenuate each other's response to a greater degree when oriented collinearly than when oriented orthogonally. Although numerous studies have examined the nature of surround suppression at these two extremes, far less is known about how the strength of tuned normalization varies as a function of continuous changes in orientation similarity, particularly in humans. In this study, we used functional magnetic resonance imaging (fMRI) to examine the bandwidth of orientation-tuned suppression within human visual cortex. Blood-oxygen-level-dependent (BOLD) responses were acquired as participants viewed a full-field circular stimulus composed of wedges of orientation-bandpass filtered noise. This stimulus configuration allowed us to parametrically vary orientation differences between neighboring wedges in gradual steps between collinear and orthogonal. We found the greatest suppression for collinearly arranged stimuli with a gradual increase in BOLD response as the orientation content became more dissimilar. We quantified the tuning width of orientation-tuned suppression, finding that the voxel-wise bandwidth of orientation tuned normalization was between 20° and 30°, and did not differ substantially between early visual areas. Voxel-wise analyses revealed that suppression width covaried with retinotopic preference, with the tightest bandwidths at outer eccentricities. Having an estimate of orientation-tuned suppression bandwidth can serve to constrain models of tuned normalization, establishing the precise degree to which suppression strength depends on similarity between visual stimulus components.NEW & NOTEWORTHY Neurons in the early visual cortex are subject to divisive normalization, but the feature-tuning aspect of this computation remains understudied, particularly in humans. We investigated orientation tuning of normalization in human early visual cortex using fMRI and estimated the bandwidth of the tuned normalization function across observers. Our findings provide a characterization of tuned normalization in early visual cortex that could help constrain models of divisive normalization in vision.

Normalization governs attentional modulation within human visual cortex.

Although attention is known to increase the gain of visuocortical responses, its underlying neural computations remain unclear. Here, we use fMRI to test the hypothesis that a neural population's ability to be modulated by attention is dependent on divisive normalization. To do so, we leverage the feature-tuned properties of normalization and find that visuocortical responses to stimuli sharing features normalize each other more strongly. Comparing these normalization measures to measures of attentional modulation, we demonstrate that subpopulations which exhibit stronger normalization also exhibit larger attentional benefits. In a converging experiment, we reveal that attentional benefits are greatest when a subpopulation is forced into a state of stronger normalization. Taken together, these results suggest that the degree to which a subpopulation exhibits normalization plays a role in dictating its potential for attentional benefits.

Visual Memories Bypass Normalization.

How distinct are visual memory representations from visual perception? Although evidence suggests that briefly remembered stimuli are represented within early visual cortices, the degree to which these memory traces resemble true visual representations remains something of a mystery. Here, we tested whether both visual memory and perception succumb to a seemingly ubiquitous neural computation: normalization. Observers were asked to remember the contrast of visual stimuli, which were pitted against each other to promote normalization either in perception or in visual memory. Our results revealed robust normalization between visual representations in perception, yet no signature of normalization occurring between working memory stores-neither between representations in memory nor between memory representations and visual inputs. These results provide unique insight into the nature of visual memory representations, illustrating that visual memory representations follow a different set of computational rules, bypassing normalization, a canonical visual computation.

Attentional modulation interacts with orientation anisotropies in contrast perception.

Orientation perception is not comparable across all orientations-a phenomenon commonly referred to as the oblique effect. Here, we first assessed the interaction between stimulus contrast and the oblique effect. Specifically, we examined whether the impairment in behavioral performance for oblique versus cardinal orientations is best explained by a contrast or a response gain modulation of the contrast psychometric function. Results revealed a robust oblique effect, whereby asymptotic performance for oblique orientations was substantially lower than for cardinal orientations, which we interpret as the result of multiplicative attenuation of contrast responses for oblique orientations. Next, we assessed how orientation anisotropies interact with attention by measuring psychometric functions for orientations under low or high attentional load. Interestingly, attentional load affects the performance for cardinal and oblique orientations differently: While it multiplicatively attenuates contrast psychometric functions for both cardinal and oblique orientation conditions, the magnitude of this effect is greater for the obliques. Thus, having less attentional resources available seems to impair the response for oblique orientations to a larger degree than for cardinal orientations.

The impact of interference on short-term memory for visual orientation.

Visual short-term memory serves as an efficient buffer for maintaining no longer directly accessible information. How robust are visual memories against interference? Memory for simple visual features has proven vulnerable to distractors containing conflicting information along the relevant stimulus dimension, leading to the idea that interacting feature-specific channels at an early stage of visual processing support memory for simple visual features. Here we showed that memory for a single randomly orientated grating was susceptible to interference from a to-be-ignored distractor grating presented midway through a 3-s delay period. Memory for the initially presented orientation became noisier when it differed from the distractor orientation, and response distributions were shifted toward the distractor orientation (by ∼3°). Interestingly, when the distractor was rendered task-relevant by making it a second memory target, memory for both retained orientations showed reduced reliability as a function of increased orientation differences between them. However, the degree to which responses to the first grating shifted toward the orientation of the task-relevant second grating was much reduced. Finally, using a dichoptic display, we demonstrated that these systematic biases caused by a consciously perceived distractor disappeared once the distractor was presented outside of participants' awareness. Together, our results show that visual short-term memory for orientation can be systematically biased by interfering information that is consciously perceived.

Scrutinizing visual images: the role of gaze in mental imagery and memory.

Gaze was monitored by use of an infrared remote eye-tracker during perception and imagery of geometric forms and figures of animals. Based on the idea that gaze prioritizes locations where features with high information content are visible, we hypothesized that eye fixations should focus on regions that contain one or more local features that are relevant for object recognition. Most importantly, we predicted that when observers looked at an empty screen and at the same time generated a detailed visual image of what they had previously seen, their gaze would probabilistically dwell within regions corresponding to the original positions of salient features or parts. Correlation analyses showed positive relations between gaze's dwell time within locations visited during perception and those in which gaze dwelled during the imagery generation task. Moreover, the more faithful an observer's gaze enactment, the more accurate was the observer's memory, in a separate test, of the dimension or size in which the forms had been perceived. In another experiment, observers saw a series of pictures of animals and were requested to memorize them. They were then asked later, in a recall phase, to answer a question about a property of one of the encoded forms; it was found that, when retrieving from long-term memory a previously seen picture, gaze returned to the location of the part probed by the question. In another experimental condition, the observers were asked to maintain fixation away from the original location of the shape while thinking about the answer, so as to interfere with the gaze enactment process; such a manipulation resulted in measurable costs in the quality of memory. We conclude that the generation of mental images relies upon a process of enactment of gaze that can be beneficial to visual memory.

Intensive tool-practice and skillfulness facilitate the extension of body representations in humans.

The brain׳s representation of the body can be extended to include objects that are not originally part of the body. Various studies have found both extremely rapid extensions that occur as soon as an object is held, as well as extremely slow extensions that require weeks of training. Due to species and methodological differences, it is unclear whether the studies were probing different representations, or revealing multiple aspects of the same representation. Here, we present evidence that objects (cotton balls) held by a tool (chopsticks) are rapidly integrated into the body representation, as indexed by fading of the cotton balls (or 'second-order extensions') from a positive afterimage. Skillfulness with chopsticks was predictive of more rapid integration of the second-order cotton balls held by this tool. We also found that extensive training over a period of weeks augmented the level of integration. Together, our findings demonstrate integration of second-order objects held by tools, and reveal that the body representation probed by positive afterimages is subject to both rapid and slow processes of adaptive change.