Variability of visual field maps in human early extrastriate cortex challenges the canonical model of organization of V2 and V3.

Visual field maps in human early extrastriate areas (V2 and V3) are traditionally thought to form mirror-image representations which surround the primary visual cortex (V1). According to this scheme, V2 and V3 form nearly symmetrical halves with respect to the calcarine sulcus, with the dorsal halves representing lower contralateral quadrants, and the ventral halves representing upper contralateral quadrants. This arrangement is considered to be consistent across individuals, and thus predictable with reasonable accuracy using templates. However, data that deviate from this expected pattern have been observed, but mainly treated as artifactual. Here, we systematically investigate individual variability in the visual field maps of human early visual cortex using the 7T Human Connectome Project (HCP) retinotopy dataset. Our results demonstrate substantial and principled inter-individual variability. Visual field representation in the dorsal portions of V2 and V3 was more variable than in their ventral counterparts, including substantial departures from the expected mirror-symmetrical patterns. In addition, left hemisphere retinotopic maps were more variable than those in the right hemisphere. Surprisingly, only one-third of individuals had maps that conformed to the expected pattern in the left hemisphere. Visual field sign analysis further revealed that in many individuals the area conventionally identified as dorsal V3 shows a discontinuity in the mirror-image representation of the retina, associated with a Y-shaped lower vertical representation. Our findings challenge the current view that inter-individual variability in early extrastriate cortex is negligible, and that the dorsal portions of V2 and V3 are roughly mirror images of their ventral counterparts.

Visual responses in the dorsolateral frontal cortex of marmoset monkeys.

The marmoset monkey (Callithrix jacchus) has gained attention in neurophysiology research as a new primate model for visual processing and behavior. In particular, marmosets have a lissencephalic cortex, making multielectrode, optogenetic, and calcium-imaging techniques more accessible than other primate models. However, the degree of homology of brain circuits for visual behavior with those identified in macaques and humans is still being ascertained. For example, whereas the location of the frontal eye fields (FEF) within the dorsolateral frontal cortex has been proposed, it remains unclear whether neurons in the corresponding areas show visual responses-an important characteristic of FEF neurons in other species. Here, we provide the first description of receptive field properties and neural response latencies in the marmoset dorsolateral frontal cortex, based on recordings using Utah arrays in anesthetized animals. We find brisk visual responses in specific regions of the dorsolateral prefrontal cortex, particularly in areas 8aV, 8C, and 6DR. As in macaque FEF, the receptive fields were typically large (10°-30° in diameter) and the median responses latency was brisk (60 ms). These results constrain the possible interpretations about the location of the marmoset FEF and suggest that the marmoset model's significant advantages for the use of physiological techniques may be leveraged in the study of visuomotor cognition.NEW & NOTEWORTHY Behavior and cognition in humans and other primates rely on networks of brain areas guided by the frontal cortex. The marmoset offers exciting new opportunities to study links between brain physiology and behavior, but the functions of frontal cortex areas are still being identified in this species. Here, we provide the first evidence of visual receptive fields in the marmoset dorsolateral frontal cortex, an important step toward future studies of visual cognitive behavior.

A twisted visual field map in the primate dorsomedial cortex predicted by topographic continuity.

Adjacent neurons in visual cortex have overlapping receptive fields within and across area boundaries, an arrangement theorized to minimize wiring cost. This constraint is traditionally thought to create retinotopic maps of opposing field signs (mirror and nonmirror visual field representations) in adjacent areas, a concept that has become central in current attempts to subdivide the extrastriate cortex. We simulated the formation of retinotopic maps using a model that balances constraints imposed by smoothness in the representation within an area and by congruence between areas. As in the primate cortex, this model usually leads to alternating mirror and nonmirror maps. However, we found that it can also produce a more complex type of map, consisting of sectors with opposing field sign within a single area. Using fully quantitative electrode array recordings, we then demonstrate that this type of inhomogeneous map exists in the controversial dorsomedial region of the primate extrastriate cortex.

Weighting neurons by selectivity produces near-optimal population codes.

Perception is produced by "reading out" the representation of a sensory stimulus contained in the activity of a population of neurons. To examine experimentally how populations code information, a common approach is to decode a linearly weighted sum of the neurons' spike counts. This approach is popular because of the biological plausibility of weighted, nonlinear integration. For neurons recorded in vivo, weights are highly variable when derived through optimization methods, but it is unclear how the variability affects decoding performance in practice. To address this, we recorded from neurons in the middle temporal area (MT) of anesthetized marmosets (Callithrix jacchus) viewing stimuli comprising a sheet of dots that moved coherently in 1 of 12 different directions. We found that high peak response and direction selectivity both predicted that a neuron would be weighted more highly in an optimized decoding model. Although learned weights differed markedly from weights chosen according to a priori rules based on a neuron's tuning profile, decoding performance was only marginally better for the learned weights. In the models with a priori rules, selectivity is the best predictor of weighting, and defining weights according to a neuron's preferred direction and selectivity improves decoding performance to very near the maximum level possible, as defined by the learned weights. NEW & NOTEWORTHY We examined which aspects of a neuron's tuning account for its contribution to sensory coding. Strongly direction-selective neurons are weighted most highly by optimal decoders trained to discriminate motion direction. Models with predefined decoding weights demonstrate that this weighting scheme causally improved direction representation by a neuronal population. Optimizing decoders (using a generalized linear model or Fisher's linear discriminant) led to only marginally better performance than decoders based purely on a neuron's preferred direction and selectivity.

Contrast and luminance adaptation alter neuronal coding and perception of stimulus orientation.

Sensory systems face a barrage of stimulation that continually changes along multiple dimensions. These simultaneous changes create a formidable problem for the nervous system, as neurons must dynamically encode each stimulus dimension, despite changes in other dimensions. Here, we measured how neurons in visual cortex encode orientation following changes in luminance and contrast, which are critical for visual processing, but nuisance variables in the context of orientation coding. Using information theoretic analysis and population decoding approaches, we find that orientation discriminability is luminance and contrast dependent, changing over time due to firing rate adaptation. We also show that orientation discrimination in human observers changes during adaptation, in a manner consistent with the neuronal data. Our results suggest that adaptation does not maintain information rates per se, but instead acts to keep sensory systems operating within the limited dynamic range afforded by spiking activity, despite a wide range of possible inputs.

Correlated Variability in the Neurons With the Strongest Tuning Improves Direction Coding.

Sensory perception depends on neuronal populations creating an accurate representation of the external world. The amount of information that a population can represent depends on the tuning of individual neurons and the trial-by-trial variability shared among neurons. Although on average, pairwise spike-count correlations between neurons are positive, the distribution is wide, and the relationship between correlations and encoding is not straightforward. Here, we examine how single-neuron and population-level factors impact the efficacy of the neural code. We recorded responses to moving visual stimuli from motion-sensitive neurons in the middle temporal area of anesthetized marmosets (Callithrix jacchus) and trained decoders to assess how correlated and uncorrelated populations encoded stimulus motion direction. We found that the most responsive, direction-selective, and least variable neurons are the most relied-upon neurons in an uncorrelated population. In correlated populations, the same neurons do the most to shape the shared variability across the population in a way that facilitates decoding, and decoding is improved by the presence of temporally stable correlations. This suggests that the least variable neurons with the strongest stimulus representations enhance the population code by providing a strong signal and shaping correlations in variability orthogonally to the locus defined by the mean response.

Understanding Sensory Information Processing Through Simultaneous Multi-area Population Recordings.

The goal of sensory neuroscience is to understand how the brain creates its myriad of representations of the world, and uses these representations to produce perception and behavior. Circuits of neurons in spatially segregated regions of brain tissue have distinct functional specializations, and these regions are connected to form a functional processing hierarchy. Advances in technology for recording neuronal activity from multiple sites in multiple cortical areas mean that we are now able to collect data that reflects how information is transformed within and between connected members of this hierarchy. This advance is an important step in understanding the brain because, after the sensory organs have transduced a physical signal, every processing stage takes the activity of other neurons as its input, not measurements of the physical world. However, as we explore the potential of studying how populations of neurons in multiple areas respond in concert, we must also expand both the analytical tools that we use to make sense of these data and the scope of the theories that we attempt to define. In this article, we present an overview of some of the most promising analytical approaches for making inferences from population recordings in multiple brain areas, such as dimensionality reduction and measuring changes in correlated variability, and examine how they may be used to address longstanding questions in sensory neuroscience.

Rapid Adaptation Induces Persistent Biases in Population Codes for Visual Motion.

Each visual experience changes the neural response to subsequent stimuli. If the brain is unable to incorporate these encoding changes, the decoding, or perception, of subsequent stimuli is biased. Although the phenomenon of adaptation pervades the nervous system, its effects have been studied mainly in isolation, based on neuronal encoding changes induced by an isolated, prolonged stimulus. To understand how adaptation-induced biases arise and persist under continuous, naturalistic stimulation, we simultaneously recorded the responses of up to 61 neurons in the marmoset (Callithrix jacchus) middle temporal area to a sequence of directions that changed every 500 ms. We found that direction-specific adaptation following only 0.5 s of stimulation strongly affected encoding for up to 2 s by reducing both the gain and the spike count correlations between pairs of neurons with preferred directions close to the adapting direction. In addition, smaller changes in bandwidth and preferred direction were observed in some animals. Decoding individual trials of adaptation-affected activity in simultaneously recorded neurons predicted repulsive biases that are consistent with the direction aftereffect. Surprisingly, removing spike count correlations by trial shuffling did not impact decoding performance or bias. When adaptation had the largest effect on encoding, the decoder made the most errors. This suggests that neural and perceptual repulsion is not a mechanism to enhance perceptual performance but is instead a necessary consequence of optimizing neural encoding for the identification of a wide range of stimulus properties in diverse temporal contexts. SIGNIFICANCE STATEMENT: Although perception depends upon decoding the pattern of activity across a neuronal population, the encoding properties of individual neurons are unreliable: a single neuron's response to repetitions of the same stimulus is variable, and depends on both its spatial and temporal context. In this manuscript, we describe the complete cascade of adaptation-induced effects in sensory encoding and show how they predict population decoding errors consistent with perceptual biases. We measure the time course of adaptation-induced changes to the response properties of neurons in isolation, and to the correlation structure across pairs of simultaneously recorded neurons. These results provide novel insight into how and for how long adaptation affects the neural code, particularly during continuous, naturalistic vision.

Higher order image structure enables boundary segmentation in the absence of luminance or contrast cues.

Lower order image statistics, which can be described by an image's Fourier energy content, enable segmentation when they are different on either side of a boundary. We have previously demonstrated that the spatial distribution of the energy in an image (described by its higher order statistics or structure) could influence segmentation thresholds for contrast- and orientation-defined boundaries, even though it was the same on either side of the boundary and thus task irrelevant (Zavitz & Baker, 2013). Here we examined whether higher order statistics can also enable segmentation when boundaries are defined by differences in structure or density of texture elements. We used micropattern-based naturalistic synthetic textures to manipulate the sparseness, global phase alignment, and local phase alignment of carrier textures and measured segmentation thresholds based on forced-choice judgments of boundary orientation. We found that both global phase structure and sparseness, but not local phase alignment, enable segmentation and that local structure also has a small effect on segmentation thresholds in both cases. Simulations of a two-stage filter model with a compressive intermediate nonlinearity can reproduce the major features of the experimental data, segmenting boundaries defined by higher order statistics alone while capturing the influence of global image structure on segmentation thresholds.

Texture sparseness, but not local phase structure, impairs second-order segmentation.

Texture boundary segmentation is typically thought to reflect a comparison of differences in Fourier energy (i.e. low-order texture statistics) on either side of a boundary. However in a previous study (Arsenault, Yoonessi, & Baker, 2011) we showed that the distribution of energy within a natural texture (i.e. its higher-order statistical structure) also influences segmentation of contrast boundaries. Here we examine the influence of specific higher-order texture statistics on segmentation of contrast- and orientation-defined boundaries. Using naturalistic synthetic textures to manipulate the sparseness, global phase structure, and local phase alignments of carrier textures, we measure segmentation thresholds based on forced-choice judgments of boundary orientation. We find a similar pattern of results for both contrast and orientation boundaries: (1) randomizing all structure by globally phase scrambling the texture reduces segmentation thresholds substantially, (2) decreasing sparseness also reduces thresholds, and (3) removing local phase alignments has little or no effect on segmentation thresholds. We show that a two-stage filter model with an intermediate compressive nonlinearity and expansive output nonlinearity can account for these data using synthetic textures. Furthermore, the model parameter fits obtained using synthetic textures also predict the segmentation thresholds presented in Arsenault, Yoonessi, and Baker (2011) for natural and phase-scrambled natural texture carriers.