Mice Discriminate Stereoscopic Surfaces Without Fixating in Depth.

Stereopsis is a ubiquitous feature of primate mammalian vision, but little is known about if and how rodents such as mice use stereoscopic vision. We used random dot stereograms to test for stereopsis in male and female mice, and they were able to discriminate near from far surfaces over a range of disparities, with diminishing performance for small and large binocular disparities. Based on two-photon measurements of disparity tuning, the range of disparities represented in the visual cortex aligns with the behavior and covers a broad range of disparities. When we examined their binocular eye movements, we found that, unlike primates, mice did not systematically vary relative eye positions or use vergence eye movements when presented with different disparities. Nonetheless, the representation of disparity tuning was wide enough to capture stereoscopic information over a range of potential vergence angles. Although mice share fundamental characteristics of stereoscopic vision with primates and carnivores, their lack of disparity-dependent vergence eye movements and wide neuronal representation suggests that they may use a distinct strategy for stereopsis.SIGNIFICANCE STATEMENT Binocular vision allows us to derive depth information by comparing right and left eye information. We characterized binocular integration in mice because tools exist in these animals to dissect the underlying neural circuitry for binocular vision. Using random dot stereograms, we find that behavior and disparity tuning in the visual cortex share fundamental characteristics with primates, but we did not observe any evidence of disparity-dependent changes in vergence angle. We propose that mice use a distinct strategy of stereopsis compared with primates by using a broad range of disparities to encode depth over a large field of view and to compensate for nonstereoscopic changes in vergence angle that arise during natural behavior.

Evidence of Stereoscopic Surface Disambiguation in the Responses of V1 Neurons.

For the important task of binocular depth perception from complex natural-image stimuli, the neurophysiological basis for disambiguating multiple matches between the eyes across similar features has remained a long-standing problem. Recurrent interactions among binocular disparity-tuned neurons in the primary visual cortex (V1) could play a role in stereoscopic computations by altering responses to favor the most likely depth interpretation for a given image pair. Psychophysical research has shown that binocular disparity stimuli displayed in 1 region of the visual field can be extrapolated into neighboring regions that contain ambiguous depth information. We tested whether neurons in macaque V1 interact in a similar manner and found that unambiguous binocular disparity stimuli displayed in the surrounding visual fields of disparity-selective V1 neurons indeed modified their responses when either bistable stereoscopic or uniform featureless stimuli were presented within their receptive field centers. The delayed timing of the response behavior compared with the timing of classical surround suppression and multiple control experiments suggests that these modulations are carried out by slower disparity-specific recurrent connections among V1 neurons. These results provide explicit evidence that the spatial interactions that are predicted by cooperative algorithms play an important role in solving the stereo correspondence problem.

Nonuniform surround suppression of visual responses in mouse V1.

Complex receptive field characteristics, distributed across a population of neurons, are thought to be critical for solving perceptual inference problems that arise during motion and image segmentation. For example, in a class of neurons referred to as "end-stopped," increasing the length of stimuli outside of the bar-responsive region into the surround suppresses responsiveness. It is unknown whether these properties exist for receptive field surrounds in the mouse. We examined surround modulation in layer 2/3 neurons of the primary visual cortex in mice using two-photon calcium imaging. We found that surround suppression was significantly asymmetric in 17% of the visually responsive neurons examined. Furthermore, the magnitude of asymmetry was correlated with orientation selectivity. Our results demonstrate that neurons in mouse primary visual cortex are differentially sensitive to the addition of elements in the surround and that individual neurons can be described as being either uniformly suppressed by the surround, end-stopped, or side-stopped. NEW & NOTEWORTHY Perception of visual scenes requires active integration of both local and global features to successfully segment objects from the background. Although the underlying circuitry and development of perceptual inference is not well understood, converging evidence indicates that asymmetry and diversity in surround modulation are likely fundamental for these computations. We determined that these key features are present in the mouse. Our results support the mouse as a model to explore the neural basis and development of surround modulation as it relates to perceptual inference.

Relative luminance and binocular disparity preferences are correlated in macaque primary visual cortex, matching natural scene statistics.

Humans excel at inferring information about 3D scenes from their 2D images projected on the retinas, using a wide range of depth cues. One example of such inference is the tendency for observers to perceive lighter image regions as closer. This psychophysical behavior could have an ecological basis because nearer regions tend to be lighter in natural 3D scenes. Here, we show that an analogous association exists between the relative luminance and binocular disparity preferences of neurons in macaque primary visual cortex. The joint coding of relative luminance and binocular disparity at the neuronal population level may be an integral part of the neural mechanisms for perceptual inference of depth from images.

Recurrent connectivity can account for the dynamics of disparity processing in V1.

Disparity tuning measured in the primary visual cortex (V1) is described well by the disparity energy model, but not all aspects of disparity tuning are fully explained by the model. Such deviations from the disparity energy model provide us with insight into how network interactions may play a role in disparity processing and help to solve the stereo correspondence problem. Here, we propose a neuronal circuit model with recurrent connections that provides a simple account of the observed deviations. The model is based on recurrent connections inferred from neurophysiological observations on spike timing correlations, and is in good accord with existing data on disparity tuning dynamics. We further performed two additional experiments to test predictions of the model. First, we increased the size of stimuli to drive more neurons and provide a stronger recurrent input. Our model predicted sharper disparity tuning for larger stimuli. Second, we displayed anticorrelated stereograms, where dots of opposite luminance polarity are matched between the left- and right-eye images and result in inverted disparity tuning in the disparity energy model. In this case, our model predicted reduced sharpening and strength of inverted disparity tuning. For both experiments, the dynamics of disparity tuning observed from the neurophysiological recordings in macaque V1 matched model simulation predictions. Overall, the results of this study support the notion that, while the disparity energy model provides a primary account of disparity tuning in V1 neurons, neural disparity processing in V1 neurons is refined by recurrent interactions among elements in the neural circuit.

Sample skewness as a statistical measurement of neuronal tuning sharpness.

We propose using the statistical measurement of the sample skewness of the distribution of mean firing rates of a tuning curve to quantify sharpness of tuning. For some features, like binocular disparity, tuning curves are best described by relatively complex and sometimes diverse functions, making it difficult to quantify sharpness with a single function and parameter. Skewness provides a robust nonparametric measure of tuning curve sharpness that is invariant with respect to the mean and variance of the tuning curve and is straightforward to apply to a wide range of tuning, including simple orientation tuning curves and complex object tuning curves that often cannot even be described parametrically. Because skewness does not depend on a specific model or function of tuning, it is especially appealing to cases of sharpening where recurrent interactions among neurons produce sharper tuning curves that deviate in a complex manner from the feedforward function of tuning. Since tuning curves for all neurons are not typically well described by a single parametric function, this model independence additionally allows skewness to be applied to all recorded neurons, maximizing the statistical power of a set of data. We also compare skewness with other nonparametric measures of tuning curve sharpness and selectivity. Compared to these other nonparametric measures tested, skewness is best used for capturing the sharpness of multimodal tuning curves defined by narrow peaks (maximum) and broad valleys (minima). Finally, we provide a more formal definition of sharpness using a shape-based information gain measure and derive and show that skewness is correlated with this definition.

Mammals Achieve Common Neural Coverage of Visual Scenes Using Distinct Sampling Behaviors.

Most vertebrates use head and eye movements to quickly change gaze orientation and sample different portions of the environment with periods of stable fixation. Visual information must be integrated across fixations to construct a complete perspective of the visual environment. In concert with this sampling strategy, neurons adapt to unchanging input to conserve energy and ensure that only novel information from each fixation is processed. We demonstrate how adaptation recovery times and saccade properties interact and thus shape spatiotemporal tradeoffs observed in the motor and visual systems of mice, cats, marmosets, macaques, and humans. These tradeoffs predict that in order to achieve similar visual coverage over time, animals with smaller receptive field sizes require faster saccade rates. Indeed, we find comparable sampling of the visual environment by neuronal populations across mammals when integrating measurements of saccadic behavior with receptive field sizes and V1 neuronal density. We propose that these mammals share a common statistically driven strategy of maintaining coverage of their visual environment over time calibrated to their respective visual system characteristics.

Natural binocular depth discrimination behavior in mice explained by visual cortical activity.

In mice and other mammals, forebrain neurons integrate right and left eye information to generate a three-dimensional representation of the visual environment. Neurons in the visual cortex of mice are sensitive to binocular disparity,1-3 yet it is unclear whether that sensitivity is linked to the perception of depth.4-8 We developed a natural task based on the classic visual cliff and pole descent tasks to estimate the psychophysical range of mouse depth discrimination.5,9 Mice with binocular vision descended to a near (shallow) surface more often when surrounding far (deep) surfaces were progressively more distant. Occlusion of one eye severely impaired their ability to target the near surface. We quantified the distance at which animals make their decisions to estimate the binocular image displacement of the checkerboard pattern on the near and far surfaces. Then, we assayed the disparity sensitivity of large populations of binocular neurons in primary visual cortex (V1) using two-photon microscopy2 and quantitatively compared this information available in V1 to their behavioral sensitivity. Disparity information in V1 matches the behavioral performance over the range of depths examined and was resistant to changes in binocular alignment. These findings reveal that mice naturally use stereoscopic cues to guide their behavior and indicate a neural basis for this depth discrimination task.

Cooperative and competitive interactions facilitate stereo computations in macaque primary visual cortex.

Inferring depth from binocular disparities is a difficult problem for the visual system because local features in the left- and right-eye images must be matched correctly to solve this "stereo correspondence problem." Cortical architecture and computational studies suggest that lateral interactions among neurons could help resolve local uncertainty about disparity encoded in individual neurons by incorporating contextual constraints. We found that correlated activity among pairs of neurons in primary visual cortex depended both on disparity-tuning relationships and the stimuli displayed within the receptive fields of the neurons. Nearby pairs of neurons with distinct disparity tuning exhibited a decrease in spike correlation at competing disparities soon after response onset. Distant neuronal pairs of similar disparity tuning exhibited an increase in spike correlation at mutually preferred disparities. The observed correlated activity and response dynamics suggests that local competitive and distant cooperative interactions improve disparity tuning of individual neurons over time. Such interactions could represent a neural substrate for the principal constraints underlying cooperative stereo algorithms.

Motion Discrimination and the Motion Aftereffect in Mouse Vision.

Prolonged exposure to motion in one direction often leads to the illusion of motion in the opposite direction for stationary objects. This motion aftereffect likely arises across several visual areas from adaptive changes in the balance of activity and competitive interactions. We examined whether or not the mouse was susceptible to this same illusion to determine whether it would be a suitable model for learning about the neural representation of the motion aftereffect. Under a classical conditioning paradigm, mice learned to lick when presented with motion in one direction and not the opposite direction. When the mice were adapted to motion preceding this test, their lick behavior for zero coherence motion was biased for motion in the opposite direction of the adapting stimulus. Overall, lick count versus motion coherence shifted in the opposite direction of the adapting stimulus. This suggests that although the mouse has a simpler visual system compared with primates, it still is subject to the motion aftereffect and may elucidate the underlying circuitry.

Natural image and receptive field statistics predict saccade sizes.

Humans and other primates sample the visual environment using saccadic eye movements that shift a high-resolution fovea toward regions of interest to create a clear perception of a scene across fixations. Many mammals, however, like mice, lack a fovea, which raises the question of why they make saccades. Here we describe and test the hypothesis that saccades are matched to natural scene statistics and to the receptive field sizes and adaptive properties of neural populations. Specifically, we determined the minimum amplitude of saccades in natural scenes necessary to provide uncorrelated inputs to model neural populations. This analysis predicts the distributions of observed saccade sizes during passive viewing for nonhuman primates, cats, and mice. Furthermore, disrupting the development of receptive field properties by monocular deprivation changed saccade sizes consistent with this hypothesis. Therefore, natural-scene statistics and the neural representation of natural images appear to be critical factors guiding saccadic eye movements.

Relating functional connectivity in V1 neural circuits and 3D natural scenes using Boltzmann machines.

Bayesian theory has provided a compelling conceptualization for perceptual inference in the brain. Central to Bayesian inference is the notion of statistical priors. To understand the neural mechanisms of Bayesian inference, we need to understand the neural representation of statistical regularities in the natural environment. In this paper, we investigated empirically how statistical regularities in natural 3D scenes are represented in the functional connectivity of disparity-tuned neurons in the primary visual cortex of primates. We applied a Boltzmann machine model to learn from 3D natural scenes, and found that the units in the model exhibited cooperative and competitive interactions, forming a "disparity association field", analogous to the contour association field. The cooperative and competitive interactions in the disparity association field are consistent with constraints of computational models for stereo matching. In addition, we simulated neurophysiological experiments on the model, and found the results to be consistent with neurophysiological data in terms of the functional connectivity measurements between disparity-tuned neurons in the macaque primary visual cortex. These findings demonstrate that there is a relationship between the functional connectivity observed in the visual cortex and the statistics of natural scenes. They also suggest that the Boltzmann machine can be a viable model for conceptualizing computations in the visual cortex and, as such, can be used to predict neural circuits in the visual cortex from natural scene statistics.

Cooperation between area 17 neuron pairs enhances fine discrimination of orientation.

We examined 66 complex cells in area 17 of cats that were paralyzed and anesthetized with propofol and N2O. We studied changes in ensemble responses for small (<10 degrees ) and large (>10 degrees ) differences in orientation. Examination of temporal resolution and discharge history revealed advantages in discrimination from both dependent (e.g., synchronization) and independent (e.g., bursting) interspike interval properties. For 27 pairs of neurons, we found that the average cooperation (the advantage gained from the joint activity) was 57.6% for fine discrimination of orientation but <5% for gross discrimination. Dependency (probabilistic quantification of the interaction between the cells) was measured between 29 pairs of neurons while varying orientation. On average, the dependency tuning for orientation was 35.5% narrower than the average firing rate tuning. The changes in dependency around the peak orientation (at which the firing rate remains relatively constant) lead to substantial cooperation that can improve discrimination in this region. The narrow tuning of dependency and the cooperation provide evidence to support a population-encoding scheme that is based on biologically plausible mechanisms and that could account for hyperacuities.

Real-time visualization of neural synchrony for identifying coordinated cell assemblies.

We introduce a synchrony map that translates the fine temporal organization of multi-unit responses in the visual cortex into an easily interpreted spatial display. We test the synchrony map on microelectrode array recordings in Area 17 of anesthetized and paralyzed cats. We first examine the synchrony map using averaged data and probability calculations to demonstrate orientation-dependent changes in synchrony. We then demonstrate how the synchrony map can be implemented for real-time visualization of synchrony among neural assemblies.

Synchronous activity in cat visual cortex encodes collinear and cocircular contours.

We explored how contour information in primary visual cortex might be embedded in the simultaneous activity of multiple cells recorded with a 100-electrode array. Synchronous activity in cat visual cortex was more selective and predictable in discriminating between drifting grating and concentric ring stimuli than changes in firing rate. Synchrony was found even between cells with wholly different orientation preferences when their receptive fields were circularly aligned, and membership in synchronous groups was orientation and curvature dependent. The existence of synchrony between cocircular cells reinforces its role as a general mechanism for contour integration and shape detection as predicted by association field concepts. Our data suggest that cortical synchrony results from common and synchronous input from earlier visual areas and that it could serve to shape extrastriate response selectivity.

Gamma oscillation maintains stimulus structure-dependent synchronization in cat visual cortex.

Visual cortical cells demonstrate both oscillation and synchronization, although the underlying causes and functional significance of these behaviors remain uncertain. We simultaneously recorded single-unit activity with microelectrode arrays in supragranular layers of area 17 of cats paralyzed and anesthetized with propofol and N(2)O. Rate-normalized autocorrelograms of 24 cells reveal bursting (100%) and gamma oscillation (63%). Renewal density analysis, used to explore the source of oscillation, suggests a contribution from extrinsic influences such as feedback. However, a bursting refractory period, presumably membrane-based, could also encourage oscillatory firing. When we investigated the source of synchronization for 60 cell pairs we found only moderate correlation of synchrony with bursts and oscillation. We did, nonetheless, discover a possible functional role for oscillation. In all cases of cross-correlograms that exhibited oscillation, the strength of the synchrony was maintained throughout the stimulation period. When no oscillation was apparent, 75% of the cell pairs showed decay in their synchronization. The synchrony between cells is strongly dependent on similar response onset latencies. We therefore propose that structured input, which yields tight organization of latency, is a more likely candidate for the source of synchronization than oscillation. The reliable synchrony at response onset could be driven by spatial and temporal correlation of the stimulus that is preserved through the earlier stages of the visual system. Oscillation then contributes to maintenance of the synchrony to enhance reliable transmission of the information for higher cognitive processing.

From another angle: Differences in cortical coding between fine and coarse discrimination of orientation.

We measured the information available for orientation discrimination from metric distances for 24 cells in area 17 of cats that were paralyzed and anesthetized with Propofol and N(2)O. The metric distance information confirms fundamental coding differences for discrimination between fine (<10 degrees ) and coarse (>10 degrees ) orientation differences. The information for discriminating larger orientation differences is contained mainly in the firing rate, with minor enhancements from the coarse (30-70 ms) temporal structure in the firing rate. Both precise spike timing (9.2 ms) and intervals (6.8 ms) sustained over the stimulus presentation provide information for fine discrimination of orientation, where almost no reliable information is provided by the spike count. We compare and confirm the results (using the same data set) to vector distances based on classification theory. The results support a dynamic spiking mechanism where coordinated activity could provide fast and reliable information about detailed angle and/or direction information in the region of the preferred orientation.

Cooperative synchronized assemblies enhance orientation discrimination.

There is no clear link between the broad tuning of single neurons and the fine behavioral capabilities of orientation discrimination. We recorded from populations of cells in the cat visual cortex (area 17) to examine whether the joint activity of cells can support finer discrimination than found in individual responses. Analysis of joint firing yields a substantial advantage (i.e., cooperation) in fine-angle discrimination. This cooperation increases to more considerable levels as the population of an assembly is increased. The cooperation in a population of six cells provides encoding of orientation with an information advantage that is at least 2-fold in terms of requiring either fewer cells or less time than independent coding. This cooperation suggests that correlated or synchronized activity can increase information.