Transformations of sensory information in the brain suggest changing criteria for optimality.

Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities-the slight differences in the retinal images of the two eyes-we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes.

Pattern Motion Direction Is Encoded in the Population Activity of Macaque Area MT.

Direction selective neurons in macaque primary visual cortex are narrowly tuned for orientation, and are thus afflicted by the aperture problem. At the next stage of motion processing, in the middle temporal (MT) area, some cells appear to solve this problem, responding to the pattern motion direction of plaids. Models have been proposed to account for this computation, but they do not replicate the diversity of responses observed in MT. We recorded from 386 cells in area MT of two male macaques, while presenting a wide range of random-line stimuli and their compositions into noise plaids. As we broadened the range of stimuli used to probe the cells, yielding ever more challenging conditions for extracting pattern motion, the diversity of the responses observed increased, and the fraction of cells that faithfully encoded pattern motion direction shrank. However, we show here that a pattern motion signal is present at the population level. We identified four mechanisms, one never proposed before, that together might account for the observed diversity in single-cell responses. Pattern motion is thus extracted in area MT, but it is encoded across the population, and not in a small subset of pattern neurons.SIGNIFICANCE STATEMENT Some neurons in the middle temporal area of macaques solve the aperture problem, signaling the direction of motion of complex patterns. As the number of pattern types used to probe this mechanism is increased, fewer and fewer cells retain this capability. We show here that different cells fail in different ways, and that simply summing their responses averages away their failures, yielding a clear pattern motion signal. Similar encodings, which unequivocally violate the "neuron as a feature detector" hypothesis that has dominated sensory processing theories for the past 50 years, might apply throughout the brain.

Model-based characterization of the selectivity of neurons in primary visual cortex.

Statistical models are increasingly being used to understand the complexity of stimulus selectivity in primary visual cortex (V1) in the context of complex time-varying stimuli, replacing averaging responses to simple parametric stimuli. Although such models often can more accurately reflect the computations performed by V1 neurons in more natural visual environments, they do not by themselves provide insight into V1 neural selectivity to basic stimulus features such as receptive field size, spatial frequency tuning, and phase invariance. Here, we present a battery of analyses that can be directly applied to encoding models to link complex encoding models to more interpretable aspects of stimulus selectivity. We apply this battery to nonlinear models of V1 neurons recorded in awake macaque during random bar stimuli. In linking model properties to more classical measurements, we demonstrate several novel aspects of V1 selectivity not available to simpler experimental measurements. For example, this approach reveals that individual spatiotemporal elements of the V1 models often have a smaller spatial scale than the neuron as a whole, resulting in nontrivial tuning to spatial frequencies. In addition, we propose measures of nonlinear integration that suggest that classical classifications of V1 neurons into simple versus complex cells will be spatial-frequency dependent. In total, rather than obfuscate classical characterizations of V1 neurons, model-based characterizations offer a means to more fully understand their selectivity, and link their classical tuning properties to their roles in more complex, natural, visual processing.NEW & NOTEWORTHY Visual neurons are increasingly being studied with more complex, natural visual stimuli, and increasingly complex models are necessary to characterize their response properties. Here, we describe a battery of analyses that relate these more complex models to classical characterizations. Using such model-based characterizations of V1 neurons furthermore yields several new insights into V1 processing not possible to capture in more classical means to measure their visual selectivity.

Weighted summation and contrast normalization account for short-latency disparity vergence responses to white noise stimuli in humans.

Natural images are typically broadband, whereas detectors in early visual processing are selective for narrow ranges of spatial frequency. White noise patterns are widely used in laboratory settings to investigate how responses are derived from Fourier components in the image. Here, we report disparity vergence responses (DVRs) to white noise stimuli in human subjects and compare these with responses to white noise patterns filtered with bandpass filters and notch filters and to sinusoidal gratings. Although the contribution of these short-latency eye movements to the overall vergence response to a given stimulus is generally small, they have proven to be a valuable tool for the study of the early mechanisms that process disparity stimuli in human subjects. Removing lower spatial frequency (SF) components reduced DVR amplitude, whereas removing higher SF components led to an increase in DVR amplitude. For larger disparities, the transition occurred at lower SFs. All of these effects were quantitatively well described by a model that combined two factors: (a) an excitatory drive determined by a weighted sum of stimulus Fourier components, which was scaled by (b) a contrast normalization mechanism.

Short-latency ocular following responses to motion stimuli are strongly affected by temporal modulations of the visual content during the initial fixation period.

Neuronal and psychophysical responses to a visual stimulus are known to depend on the preceding history of visual stimulation, but the effect of stimulation history on reflexive eye movements has received less attention. Here, we quantify these effects using short-latency ocular following responses (OFRs), a valuable tool for studying early motion processing. We recorded, in human subjects, the horizontal OFRs induced by drifting vertical 1D pink noise. The stimulus was preceded by 600 to 1000 ms of maintained fixation (on a visible cross), and we explored the effect of different stimuli ("fixation patterns") presented during the fixation period. We found that any temporal modulation present during the fixation period reduced the magnitude of the subsequent OFRs. Even changes in the overall luminance during the fixation period induced significant suppression. The magnitude of the effect was a function of both spatial and temporal structure of the fixation pattern. Suppression that was selective for both relative orientation and relative spatial frequency accounted for a considerable fraction of total suppression. Finally, changes in stimulus temporal structure alone (i.e. "flicker" versus "transparent motion") led to changes in the spatial frequency tuning of suppression. In the time domain, the suppression developed quickly: 100 ms of temporal modulation in the fixation pattern produced up to 80% of maximal suppression. Recovery from suppression was instead more gradual, taking up to several seconds. By presenting transparent motion during the fixation period, with opposite motion signals having different spatial frequency content, we also discovered a direction-selective component of suppression, which depended on both the frequency and the direction of the moving stimulus.

Decision-related activity in sensory neurons: correlations among neurons and with behavior.

Neurons in early sensory cortex show weak but systematic correlations with perceptual decisions when trained animals perform at psychophysical threshold. These correlations are observed across repeated presentations of identical stimuli and cannot be explained by variation in external factors. The relationship between the activity of individual sensory neurons and the animal's behavioral choice means that even neurons in early sensory cortex carry information about an upcoming decision. This relationship, termed choice probability, may reflect the effect of fluctuations in neuronal firing rate on the animal's decision, but it can also reflect modulation of sensory responses by cognitive factors, or network properties such as variability that is shared among populations of neurons. Here, we review recent work clarifying the relationship among fluctuations in the responses of individual neurons, correlated variability, and behavior in a variety of tasks and cortical areas. We also discuss the possibility that choice probability may in part reflect the influence of cognitive factors on sensory neurons and explore the situations in which choice probability can be used to make inferences about the role of particular sensory neurons in the decision-making process.

Short-latency ocular-following responses: Weighted nonlinear summation predicts the outcome of a competition between two sine wave gratings moving in opposite directions.

We recorded horizontal ocular-following responses to pairs of superimposed vertical sine wave gratings moving in opposite directions in human subjects. This configuration elicits a nonlinear interaction: when the relative contrast of the gratings is changed, the response transitions abruptly between the responses elicited by either grating alone. We explore this interaction in pairs of gratings that differ in spatial and temporal frequency and show that all cases can be described as a weighted sum of the responses to each grating presented alone, where the weights are a nonlinear function of stimulus contrast: a nonlinear weighed summation model. The weights depended on the spatial and temporal frequency of the component grating. In many cases the dominant component was not the one that produced the strongest response when presented alone, implying that the neuronal circuits assigning weights precede the stages at which motor responses to visual motion are generated. When the stimulus area was reduced, the relationship between spatial frequency and weight shifted to higher frequencies. This finding may reflect a contribution from surround suppression. The nonlinear interaction is strongest when the two components have similar spatial frequencies, suggesting that the nonlinearity may reflect interactions within single spatial frequency channels. This framework can be extended to stimuli composed of more than two components: our model was able to predict the responses to stimuli composed of three gratings. That this relatively simple model successfully captures the ocular-following responses over a wide range of spatial/temporal frequency and contrast parameters suggests that these interactions reflect a simple mechanism.

The psychophysics of stereopsis can be explained without invoking independent ON and OFF channels.

Early vision proceeds through distinct ON and OFF channels, which encode luminance increments and decrements respectively. It has been argued that these channels also contribute separately to stereoscopic vision. This is based on the fact that observers perform better on a noisy disparity discrimination task when the stimulus is a random-dot pattern consisting of equal numbers of black and white dots (a "mixed-polarity stimulus," argued to activate both ON and OFF stereo channels), than when it consists of all-white or all-black dots ("same-polarity," argued to activate only one). However, it is not clear how this theory can be reconciled with our current understanding of disparity encoding. Recently, a binocular convolutional neural network was able to replicate the mixed-polarity advantage shown by human observers, even though it was based on linear filters and contained no mechanisms which would respond separately to black or white dots. Here, we show that a subtle feature of the way the stimuli were constructed in all these experiments can explain the results. The interocular correlation between left and right images is actually lower for the same-polarity stimuli than for mixed-polarity stimuli with the same amount of disparity noise applied to the dots. Because our current theories suggest stereopsis is based on a correlation-like computation in primary visual cortex, this postulate can explain why performance was better for the mixed-polarity stimuli. We conclude that there is currently no evidence supporting separate ON and OFF channels in stereopsis.

Decision-Related Activity in Macaque V2 for Fine Disparity Discrimination Is Not Compatible with Optimal Linear Readout.

Fine judgments of stereoscopic depth rely mainly on relative judgments of depth (relative binocular disparity) between objects, rather than judgments of the distance to where the eyes are fixating (absolute disparity). In macaques, visual area V2 is the earliest site in the visual processing hierarchy for which neurons selective for relative disparity have been observed (Thomas et al., 2002). Here, we found that, in macaques trained to perform a fine disparity discrimination task, disparity-selective neurons in V2 were highly selective for the task, and their activity correlated with the animals' perceptual decisions (unexplained by the stimulus). This may partially explain similar correlations reported in downstream areas. Although compatible with a perceptual role of these neurons for the task, the interpretation of such decision-related activity is complicated by the effects of interneuronal "noise" correlations between sensory neurons. Recent work has developed simple predictions to differentiate decoding schemes (Pitkow et al., 2015) without needing measures of noise correlations, and found that data from early sensory areas were compatible with optimal linear readout of populations with information-limiting correlations. In contrast, our data here deviated significantly from these predictions. We additionally tested this prediction for previously reported results of decision-related activity in V2 for a related task, coarse disparity discrimination (Nienborg and Cumming, 2006), thought to rely on absolute disparity. Although these data followed the predicted pattern, they violated the prediction quantitatively. This suggests that optimal linear decoding of sensory signals is not generally a good predictor of behavior in simple perceptual tasks. SIGNIFICANCE STATEMENT: Activity in sensory neurons that correlates with an animal's decision is widely believed to provide insights into how the brain uses information from sensory neurons. Recent theoretical work developed simple predictions to differentiate decoding schemes, and found support for optimal linear readout of early sensory populations with information-limiting correlations. Here, we observed decision-related activity for neurons in visual area V2 of macaques performing fine disparity discrimination, as yet the earliest site for this task. These findings, and previously reported results from V2 in a different task, deviated from the predictions for optimal linear readout of a population with information-limiting correlations. Our results suggest that optimal linear decoding of early sensory information is not a general decoding strategy used by the brain.

Visual Responses in FEF, Unlike V1, Primarily Reflect When the Visual Context Renders a Receptive Field Salient.

When light falls within a neuronal visual receptive field (RF) the resulting activity is referred to as the visual response. Recent work suggests this activity is in response to both the visual stimulation and the abrupt appearance, or salience, of the presentation. Here we present a novel method for distinguishing the two, based on the timing of random and nonrandom presentations. We examined these contributions in frontal eye field (FEF; N = 51) and as a comparison, an early stage in the primary visual cortex (V1; N = 15) of male monkeys (Macaca mulatta). An array of identical stimuli was presented within and outside the neuronal RF while we manipulated salience by varying the time between stimulus presentations. We hypothesized that the rapid presentation would reduce salience (the sudden appearance within the visual field) of a stimulus at any one location, and thus decrease responses driven by salience in the RF. We found that when the interstimulus interval decreased from 500 to 16 ms there was an approximate 79% reduction in the FEF response compared with an estimated 17% decrease in V1. This reduction in FEF response for rapid presentation was evident even when the random sequence preceding a stimulus did not stimulate the RF for 500 ms. The time course of these response changes in FEF suggest that salience is represented much earlier (<100 ms following stimulus onset) than previously estimated. Our results suggest that the contribution of salience dominates at higher levels of the visual system.SIGNIFICANCE STATEMENT The neuronal responses in early visual processing [e.g., primary visual cortex (V1)] reflect primarily the retinal stimulus. Processing in higher visual areas is modulated by a combination of the visual stimulation and contextual factors, such as salience, but identifying these components separately has been difficult. Here we quantified these contributions at a late stage of visual processing [frontal eye field (FEF)] and as a comparison, an early stage in V1. Our results suggest that as visual information continues through higher levels of processing the neural responses are no longer driven primarily by the visual stimulus in the receptive field, but by the broader context that stimulus defines-very different from current views about visual signals in FEF.

Suppression and Contrast Normalization in Motion Processing.

Sensory neurons are activated by a range of stimuli to which they are said to be tuned. Usually, they are also suppressed by another set of stimuli that have little effect when presented in isolation. The interactions between preferred and suppressive stimuli are often quite complex and vary across neurons, even within a single area, making it difficult to infer their collective effect on behavioral responses mediated by activity across populations of neurons. Here, we investigated this issue by measuring, in human subjects (three males), the suppressive effect of static masks on the ocular following responses induced by moving stimuli. We found a wide range of effects, which depend in a nonlinear and nonseparable manner on the spatial frequency, contrast, and spatial location of both stimulus and mask. Under some conditions, the presence of the mask can be seen as scaling the contrast of the driving stimulus. Under other conditions, the effect is more complex, involving also a direct scaling of the behavioral response. All of this complexity at the behavioral level can be captured by a simple model in which stimulus and mask interact nonlinearly at two stages, one monocular and one binocular. The nature of the interactions is compatible with those observed at the level of single neurons in primates, usually broadly described as divisive normalization, without having to invoke any scaling mechanism.SIGNIFICANCE STATEMENT The response of sensory neurons to their preferred stimulus is often modulated by stimuli that are not effective when presented alone. Individual neurons can exhibit multiple modulatory effects, with considerable variability across neurons even in a single area. Such diversity has made it difficult to infer the impact of these modulatory mechanisms on behavioral responses. Here, we report the effects of a stationary mask on the reflexive eye movements induced by a moving stimulus. A model with two stages, each incorporating a divisive modulatory mechanism, reproduces our experimental results and suggests that qualitative variability of masking effects in cortical neurons might arise from differences in the extent to which such effects are inherited from earlier stages.

Human infants can generate vergence responses to retinal disparity by 5 to 10 weeks of age.

Vergence is defined as a binocular eye movement during which the two eyes move in opposite directions to align to a target in depth. In adults, fine vergence control is driven primarily by interocular retinal image disparity. Although infants have not typically been shown to respond to disparity until 3 to 5 months postpartum, they have been shown to align their eyes from hours after birth. It remains unclear what drives these responses in young infants. In this experiment, 5- to 10-week-old human infants were presented with a dynamic random noise stimulus oscillating in disparity at 0.1 Hz over an amplitude of 2° for 30 s. Fourier transforms of the horizontal eye movements revealed significant disparity-driven responses at the frequency of the stimulus in over half of the tested infants. Because the stimulus updated dynamically, this experiment precluded the possibility of independent monocular fixations to a sustained target. These data demonstrate cortical binocular function in humans by five weeks, the youngest age tested here, which is as much as two months younger than previously believed.

Binocular summation for reflexive eye movements.

Psychophysical studies and our own subjective experience suggest that, in natural viewing conditions (i.e., at medium to high contrasts), monocularly and binocularly viewed scenes appear very similar, with the exception of the improved depth perception provided by stereopsis. This phenomenon is usually described as a lack of binocular summation. We show here that there is an exception to this rule: Ocular following eye movements induced by the sudden motion of a large stimulus, which we recorded from three human subjects, are much larger when both eyes see the moving stimulus, than when only one eye does. We further discovered that this binocular advantage is a function of the interocular correlation between the two monocular images: It is maximal when they are identical, and reduced when the two eyes are presented with different images. This is possible only if the neurons that underlie ocular following are sensitive to binocular disparity.

Variability and Correlations in Primary Visual Cortical Neurons Driven by Fixational Eye Movements.

UNLABELLED: The ability to distinguish between elements of a sensory neuron's activity that are stimulus independent versus driven by the stimulus is critical for addressing many questions in systems neuroscience. This is typically accomplished by measuring neural responses to repeated presentations of identical stimuli and identifying the trial-variable components of the response as noise. In awake primates, however, small "fixational" eye movements (FEMs) introduce uncontrolled trial-to-trial differences in the visual stimulus itself, potentially confounding this distinction. Here, we describe novel analytical methods that directly quantify the stimulus-driven and stimulus-independent components of visual neuron responses in the presence of FEMs. We apply this approach, combined with precise model-based eye tracking, to recordings from primary visual cortex (V1), finding that standard approaches that ignore FEMs typically miss more than half of the stimulus-driven neural response variance, creating substantial biases in measures of response reliability. We show that these effects are likely not isolated to the particular experimental conditions used here, such as the choice of visual stimulus or spike measurement time window, and thus will be a more general problem for V1 recordings in awake primates. We also demonstrate that measurements of the stimulus-driven and stimulus-independent correlations among pairs of V1 neurons can be greatly biased by FEMs. These results thus illustrate the potentially dramatic impact of FEMs on measures of signal and noise in visual neuron activity and also demonstrate a novel approach for controlling for these eye-movement-induced effects. SIGNIFICANCE STATEMENT: Distinguishing between the signal and noise in a sensory neuron's activity is typically accomplished by measuring neural responses to repeated presentations of an identical stimulus. For recordings from the visual cortex of awake animals, small "fixational" eye movements (FEMs) inevitably introduce trial-to-trial variability in the visual stimulus, potentially confounding such measures. Here, we show that FEMs often have a dramatic impact on several important measures of response variability for neurons in primary visual cortex. We also present an analytical approach for quantifying signal and noise in visual neuron activity in the presence of FEMs. These results thus highlight the importance of controlling for FEMs in studies of visual neuron function, and demonstrate novel methods for doing so.

A Motion-from-Form Mechanism Contributes to Extracting Pattern Motion from Plaids.

Since the discovery of neurons selective for pattern motion direction in primate middle temporal area MT (Albright, 1984; Movshon et al., 1985), the neural computation of this signal has been the subject of intense study. The bulk of this work has explored responses to plaids obtained by summing two drifting sinusoidal gratings. Unfortunately, with these stimuli, many different mechanisms are similarly effective at extracting pattern motion. We devised a new set of stimuli, obtained by summing two random line stimuli with different orientations. This allowed several novel manipulations, including generating plaids that do not contain rigid 2D motion. Importantly, these stimuli do not engage most of the previously proposed mechanisms. We then recorded the ocular following responses that such stimuli induce in human subjects. We found that pattern motion is computed even with stimuli that do not cohere perceptually, including those without rigid motion, and even when the two gratings are presented separately to the two eyes. Moderate temporal and/or spatial separation of the gratings impairs the computation. We show that, of the models proposed so far, only those based on the intersection-of-constraints rule, embedding a motion-from-form mechanism (in which orientation signals are used in the computation of motion direction signals), can account for our results. At least for the eye movements reported here, a motion-from-form mechanism is thus involved in one of the most basic functions of the visual motion system: extracting motion direction from complex scenes. SIGNIFICANCE STATEMENT: Anatomical considerations led to the proposal that visual function is organized in separate processing streams: one (ventral) devoted to form and one (dorsal) devoted to motion. Several experimental results have challenged this view, arguing in favor of a more integrated view of visual processing. Here we add to this body of work, supporting a role for form information even in a function--extracting pattern motion direction from complex scenes--for which decisive evidence for the involvement of form signals has been lacking.

Neurons in Striate Cortex Signal Disparity in Half-Matched Random-Dot Stereograms.

UNLABELLED: Human stereopsis can operate in dense "cyclopean" images containing no monocular objects. This is believed to depend on the computation of binocular correlation by neurons in primary visual cortex (V1). The observation that humans perceive depth in half-matched random-dot stereograms, although these stimuli have no net correlation, has led to the proposition that human depth perception in these stimuli depends on a distinct "matching" computation possibly performed in extrastriate cortex. However, recording from disparity-selective neurons in V1 of fixating monkeys, we found that they are in fact able to signal disparity in half-matched stimuli. We present a simple model that explains these results. This reinstates the view that disparity-selective neurons in V1 provide the initial substrate for perception in dense cyclopean stimuli, and strongly suggests that separate correlation and matching computations are not necessary to explain existing data on mixed correlation stereograms. SIGNIFICANCE STATEMENT: The initial step in stereoscopic 3D vision is generally thought to be a correlation-based computation that takes place in striate cortex. Recent research has argued that there must be an additional matching computation involved in extracting stereoscopic depth in random-dot stereograms. This is based on the observation that humans can perceive depth in stimuli with a mean binocular correlation of zero (where a correlation-based mechanism should not signal depth). We show that correlation-based cells in striate cortex do in fact signal depth here because they convert fluctuations in the correlation level into a mean change in the firing rate. Our results reinstate the view that these cells provide a sufficient substrate for the perception of stereoscopic depth.

A Single Mechanism Can Account for Human Perception of Depth in Mixed Correlation Random Dot Stereograms.

In order to extract retinal disparity from a visual scene, the brain must match corresponding points in the left and right retinae. This computationally demanding task is known as the stereo correspondence problem. The initial stage of the solution to the correspondence problem is generally thought to consist of a correlation-based computation. However, recent work by Doi et al suggests that human observers can see depth in a class of stimuli where the mean binocular correlation is 0 (half-matched random dot stereograms). Half-matched random dot stereograms are made up of an equal number of correlated and anticorrelated dots, and the binocular energy model-a well-known model of V1 binocular complex cells-fails to signal disparity here. This has led to the proposition that a second, match-based computation must be extracting disparity in these stimuli. Here we show that a straightforward modification to the binocular energy model-adding a point output nonlinearity-is by itself sufficient to produce cells that are disparity-tuned to half-matched random dot stereograms. We then show that a simple decision model using this single mechanism can reproduce psychometric functions generated by human observers, including reduced performance to large disparities and rapidly updating dot patterns. The model makes predictions about how performance should change with dot size in half-matched stereograms and temporal alternation in correlation, which we test in human observers. We conclude that a single correlation-based computation, based directly on already-known properties of V1 neurons, can account for the literature on mixed correlation random dot stereograms.

Combining 1-D components to extract pattern information: It is about more than component similarity.

At least under some conditions, plaid stimuli are processed by combining information first extracted in orientation and scale-selective channels. The rules that govern this combination across channels are only partially understood. Although the available data suggests that only components having similar spatial frequency and contrast are combined, the extent to which this holds has not been firmly established. To address this question, we measured, in human subjects, the short-latency reflexive vergence eye movements induced by stereo plaids in which spatial frequency and contrast of the components are independently varied. We found that, although similarity in component spatial frequency and contrast matter, they interact in a nonseparable way. One way in which this relationship might arise is if the internal estimate of contrast is not a faithful representation of stimulus contrast but is instead spatial frequency-dependent (with higher spatial frequencies being boosted). We propose that such weighting might have been put in place by a mechanism that, in an effort of achieve contrast constancy and/or coding efficiency, regulates the gain of detectors in early visual cortex to equalize their long-term average response to natural images.

Ocular-following responses to white noise stimuli in humans reveal a novel nonlinearity that results from temporal sampling.

White noise stimuli are frequently used to study the visual processing of broadband images in the laboratory. A common goal is to describe how responses are derived from Fourier components in the image. We investigated this issue by recording the ocular-following responses (OFRs) to white noise stimuli in human subjects. For a given speed we compared OFRs to unfiltered white noise with those to noise filtered with band-pass filters and notch filters. Removing components with low spatial frequency (SF) reduced OFR magnitudes, and the SF associated with the greatest reduction matched the SF that produced the maximal response when presented alone. This reduction declined rapidly with SF, compatible with a winner-take-all operation. Removing higher SF components increased OFR magnitudes. For higher speeds this effect became larger and propagated toward lower SFs. All of these effects were quantitatively well described by a model that combined two factors: (a) an excitatory drive that reflected the OFRs to individual Fourier components and (b) a suppression by higher SF channels where the temporal sampling of the display led to flicker. This nonlinear interaction has an important practical implication: Even with high refresh rates (150 Hz), the temporal sampling introduced by visual displays has a significant impact on visual processing. For instance, we show that this distorts speed tuning curves, shifting the peak to lower speeds. Careful attention to spectral content, in the light of this nonlinearity, is necessary to minimize the resulting artifact when using white noise patterns undergoing apparent motion.

Decision-related activity in sensory neurons may depend on the columnar architecture of cerebral cortex.

Many studies have reported correlations between the activity of sensory neurons and animals' judgments in discrimination tasks. Here, we suggest that such neuron-behavior correlations may require a cortical map for the task relevant features. This would explain why studies using discrimination tasks based on disparity in area V1 have not found these correlations: V1 contains no map for disparity. This scheme predicts that activity of V1 neurons correlates with decisions in an orientation-discrimination task. To test this prediction, we trained two macaque monkeys in a coarse orientation discrimination task using band-pass-filtered dynamic noise. The two orientations were always 90° apart and task difficulty was controlled by varying the orientation bandwidth of the filter. While the trained animals performed this task, we recorded from orientation-selective V1 neurons (n = 82, n = 31 for Monkey 1, n = 51 for Monkey 2). For both monkeys, we observed significant correlation (quantified as "choice probabilities") of the V1 activity with the monkeys' perceptual judgments (mean choice probability 0.54, p = 10(-5)). In one of these animals, we had previously measured choice probabilities in a disparity discrimination task in V1, which had been at chance (0.49, not significantly different from 0.5). The choice probabilities in this monkey for the orientation discrimination task were significantly larger than those for the disparity discrimination task (p = 0.032). These results are predicted by our suggestion that choice probabilities are only observed for cortical sensory neurons that are organized in maps for the task-relevant feature.