High-precision coding in visual cortex.

Individual neurons in visual cortex provide the brain with unreliable estimates of visual features. It is not known whether the single-neuron variability is correlated across large neural populations, thus impairing the global encoding of stimuli. We recorded simultaneously from up to 50,000 neurons in mouse primary visual cortex (V1) and in higher order visual areas and measured stimulus discrimination thresholds of 0.35° and 0.37°, respectively, in an orientation decoding task. These neural thresholds were almost 100 times smaller than the behavioral discrimination thresholds reported in mice. This discrepancy could not be explained by stimulus properties or arousal states. Furthermore, behavioral variability during a sensory discrimination task could not be explained by neural variability in V1. Instead, behavior-related neural activity arose dynamically across a network of non-sensory brain areas. These results imply that perceptual discrimination in mice is limited by downstream decoders, not by neural noise in sensory representations.

Common population codes produce extremely nonlinear neural manifolds.

Populations of neurons represent sensory, motor, and cognitive variables via patterns of activity distributed across the population. The size of the population used to encode a variable is typically much greater than the dimension of the variable itself, and thus, the corresponding neural population activity occupies lower-dimensional subsets of the full set of possible activity states. Given population activity data with such lower-dimensional structure, a fundamental question asks how close the low-dimensional data lie to a linear subspace. The linearity or nonlinearity of the low-dimensional structure reflects important computational features of the encoding, such as robustness and generalizability. Moreover, identifying such linear structure underlies common data analysis methods such as Principal Component Analysis (PCA). Here, we show that for data drawn from many common population codes the resulting point clouds and manifolds are exceedingly nonlinear, with the dimension of the best-fitting linear subspace growing at least exponentially with the true dimension of the data. Consequently, linear methods like PCA fail dramatically at identifying the true underlying structure, even in the limit of arbitrarily many data points and no noise.

Neural representational geometry underlies few-shot concept learning.

Understanding the neural basis of the remarkable human cognitive capacity to learn novel concepts from just one or a few sensory experiences constitutes a fundamental problem. We propose a simple, biologically plausible, mathematically tractable, and computationally powerful neural mechanism for few-shot learning of naturalistic concepts. We posit that the concepts that can be learned from few examples are defined by tightly circumscribed manifolds in the neural firing-rate space of higher-order sensory areas. We further posit that a single plastic downstream readout neuron learns to discriminate new concepts based on few examples using a simple plasticity rule. We demonstrate the computational power of our proposal by showing that it can achieve high few-shot learning accuracy on natural visual concepts using both macaque inferotemporal cortex representations and deep neural network (DNN) models of these representations and can even learn novel visual concepts specified only through linguistic descriptors. Moreover, we develop a mathematical theory of few-shot learning that links neurophysiology to predictions about behavioral outcomes by delineating several fundamental and measurable geometric properties of neural representations that can accurately predict the few-shot learning performance of naturalistic concepts across all our numerical simulations. This theory reveals, for instance, that high-dimensional manifolds enhance the ability to learn new concepts from few examples. Intriguingly, we observe striking mismatches between the geometry of manifolds in the primate visual pathway and in trained DNNs. We discuss testable predictions of our theory for psychophysics and neurophysiological experiments.

Methylphenidate as a causal test of translational and basic neural coding hypotheses.

Most systems neuroscience studies fall into one of two categories: basic science work aimed at understanding the relationship between neurons and behavior, or translational work aimed at developing treatments for neuropsychiatric disorders. Here we use these two approaches to inform and enhance each other. Our study both tests hypotheses about basic science neural coding principles and elucidates the neuronal mechanisms underlying clinically relevant behavioral effects of systemically administered methylphenidate (Ritalin). We discovered that orally administered methylphenidate, used clinically to treat attention deficit hyperactivity disorder (ADHD) and generally to enhance cognition, increases spatially selective visual attention, enhancing visual performance at only the attended location. Further, we found that this causal manipulation enhances vision in rhesus macaques specifically when it decreases the mean correlated variability of neurons in visual area V4. Our findings demonstrate that the visual system is a platform for understanding the neural underpinnings of both complex cognitive processes (basic science) and neuropsychiatric disorders (translation). Addressing basic science hypotheses, our results are consistent with a scenario in which methylphenidate has cognitively specific effects by working through naturally selective cognitive mechanisms. Clinically, our findings suggest that the often staggeringly specific symptoms of neuropsychiatric disorders may be caused and treated by leveraging general mechanisms.

Contrast gain control is a reparameterization of a population response curve.

Neurons in primary visual cortex (area V1) adapt in varying degrees to the average contrast of the environment, suggesting that the representation of visual stimuli may interact with the state of cortical gain control in complex ways. To investigate this possibility, we measured and analyzed the responses of neural populations in mouse V1 to visual stimuli as a function of contrast in different environments, each characterized by a unique distribution of contrast values. Our findings reveal that, for a fixed stimulus, the population response can be described by a vector function r(gec), where the gain ge is a decreasing function of the mean contrast of the environment. Thus, gain control can be viewed as a reparameterization of a population response curve, which is invariant across environments. Different stimuli are mapped to distinct curves, all originating from a common origin, corresponding to a zero-contrast response. Altogether, our findings provide a straightforward, geometric interpretation of contrast gain control at the population level and show that changes in gain are well matched among members of a population.NEW & NOTEWORTHY The authors study the responses of neural populations in mouse primary visual cortex as a function of stimulus contrast. Measurements are performed in different "environments," each characterized by a different distribution of contrast values. They find that responses across environments can be viewed as a reparameterization of a single contrast-response curve, offering a simple, geometric account of contrast gain control in neural populations.

On the contrast response function of adapted neural populations.

The magnitude of neural responses in sensory cortex depends on the intensity of a stimulus and its probability of being observed within the environment. How these two variables combine to influence the overall response of cortical populations remains unknown. Here we show that, in primary visual cortex, the vector magnitude of the population response is described by a separable power law that factors the intensity of a stimulus and its probability. Moreover, the discriminability between two contrast levels in a cortical population is proportional to the logarithm of the contrast ratio.NEW & NOTEWORTHY The magnitude of neural responses in sensory cortex depends on the intensity of a stimulus and its probability of being observed within the environment. The authors show that, in primary visual cortex, the vector magnitude of the population response is described by a separable power law that factors the intensity of a stimulus and its probability.

A robust and compact population code for competing sounds in auditory cortex.

Cortical circuits encoding sensory information consist of populations of neurons, yet how information aggregates via pooling individual cells remains poorly understood. Such pooling may be particularly important in noisy settings where single-neuron encoding is degraded. One example is the cocktail party problem, with competing sounds from multiple spatial locations. How populations of neurons in auditory cortex code competing sounds have not been previously investigated. Here, we apply a novel information-theoretic approach to estimate information in populations of neurons in mouse auditory cortex about competing sounds from multiple spatial locations, including both summed population (SP) and labeled line (LL) codes. We find that a small subset of neurons is sufficient to nearly maximize mutual information over different spatial configurations, with the labeled line code outperforming the summed population code and approaching information levels attained in the absence of competing stimuli. Finally, information in the labeled line code increases with spatial separation between target and masker, in correspondence with behavioral results on spatial release from masking in humans and animals. Taken together, our results reveal that a compact population of neurons in auditory cortex provides a robust code for competing sounds from different spatial locations.NEW & NOTEWORTHY Little is known about how populations of neurons within cortical circuits encode sensory stimuli in the presence of competing stimuli at other spatial locations. Here, we investigate this problem in auditory cortex using a recently proposed information-theoretic approach. We find a small subset of neurons nearly maximizes information about target sounds in the presence of competing maskers, approaching information levels for isolated stimuli, and provides a noise-robust code for sounds in a complex auditory scene.

Gated transformations from egocentric to allocentric reference frames involving retrosplenial cortex, entorhinal cortex, and hippocampus.

This paper reviews the recent experimental finding that neurons in behaving rodents show egocentric coding of the environment in a number of structures associated with the hippocampus. Many animals generating behavior on the basis of sensory input must deal with the transformation of coordinates from the egocentric position of sensory input relative to the animal, into an allocentric framework concerning the position of multiple goals and objects relative to each other in the environment. Neurons in retrosplenial cortex show egocentric coding of the position of boundaries in relation to an animal. These neuronal responses are discussed in relation to existing models of the transformation from egocentric to allocentric coordinates using gain fields and a new model proposing transformations of phase coding that differ from current models. The same type of transformations could allow hierarchical representations of complex scenes. The responses in rodents are also discussed in comparison to work on coordinate transformations in humans and non-human primates.

Understanding the computation of time using neural network models.

To maximize future rewards in this ever-changing world, animals must be able to discover the temporal structure of stimuli and then anticipate or act correctly at the right time. How do animals perceive, maintain, and use time intervals ranging from hundreds of milliseconds to multiseconds in working memory? How is temporal information processed concurrently with spatial information and decision making? Why are there strong neuronal temporal signals in tasks in which temporal information is not required? A systematic understanding of the underlying neural mechanisms is still lacking. Here, we addressed these problems using supervised training of recurrent neural network models. We revealed that neural networks perceive elapsed time through state evolution along stereotypical trajectory, maintain time intervals in working memory in the monotonic increase or decrease of the firing rates of interval-tuned neurons, and compare or produce time intervals by scaling state evolution speed. Temporal and nontemporal information is coded in subspaces orthogonal with each other, and the state trajectories with time at different nontemporal information are quasiparallel and isomorphic. Such coding geometry facilitates the decoding generalizability of temporal and nontemporal information across each other. The network structure exhibits multiple feedforward sequences that mutually excite or inhibit depending on whether their preferences of nontemporal information are similar or not. We identified four factors that facilitate strong temporal signals in nontiming tasks, including the anticipation of coming events. Our work discloses fundamental computational principles of temporal processing, and it is supported by and gives predictions to a number of experimental phenomena.

Stochastic sampling provides a unifying account of visual working memory limits.

Research into human working memory limits has been shaped by the competition between different formal models, with a central point of contention being whether internal representations are continuous or discrete. Here we describe a sampling approach derived from principles of neural coding as a framework to understand working memory limits. Reconceptualizing existing models in these terms reveals strong commonalities between seemingly opposing accounts, but also allows us to identify specific points of difference. We show that the discrete versus continuous nature of sampling is not critical to model fits, but that, instead, random variability in sample counts is the key to reproducing human performance in both single- and whole-report tasks. A probabilistic limit on the number of items successfully retrieved is an emergent property of stochastic sampling, requiring no explicit mechanism to enforce it. These findings resolve discrepancies between previous accounts and establish a unified computational framework for working memory that is compatible with neural principles.

An Emergent Population Code in Primary Auditory Cortex Supports Selective Attention to Spectral and Temporal Sound Features.

Textbook descriptions of primary sensory cortex (PSC) revolve around single neurons' representation of low-dimensional sensory features, such as visual object orientation in primary visual cortex (V1), location of somatic touch in primary somatosensory cortex (S1), and sound frequency in primary auditory cortex (A1). Typically, studies of PSC measure neurons' responses along few (one or two) stimulus and/or behavioral dimensions. However, real-world stimuli usually vary along many feature dimensions and behavioral demands change constantly. In order to illuminate how A1 supports flexible perception in rich acoustic environments, we recorded from A1 neurons while rhesus macaques (one male, one female) performed a feature-selective attention task. We presented sounds that varied along spectral and temporal feature dimensions (carrier bandwidth and temporal envelope, respectively). Within a block, subjects attended to one feature of the sound in a selective change detection task. We found that single neurons tend to be high-dimensional, in that they exhibit substantial mixed selectivity for both sound features, as well as task context. We found no overall enhancement of single-neuron coding of the attended feature, as attention could either diminish or enhance this coding. However, a population-level analysis reveals that ensembles of neurons exhibit enhanced encoding of attended sound features, and this population code tracks subjects' performance. Importantly, surrogate neural populations with intact single-neuron tuning but shuffled higher-order correlations among neurons fail to yield attention- related effects observed in the intact data. These results suggest that an emergent population code not measurable at the single-neuron level might constitute the functional unit of sensory representation in PSC.SIGNIFICANCE STATEMENT The ability to adapt to a dynamic sensory environment promotes a range of important natural behaviors. We recorded from single neurons in monkey primary auditory cortex (A1), while subjects attended to either the spectral or temporal features of complex sounds. Surprisingly, we found no average increase in responsiveness to, or encoding of, the attended feature across single neurons. However, when we pooled the activity of the sampled neurons via targeted dimensionality reduction (TDR), we found enhanced population-level representation of the attended feature and suppression of the distractor feature. This dissociation of the effects of attention at the level of single neurons versus the population highlights the synergistic nature of cortical sound encoding and enriches our understanding of sensory cortical function.

Population Coding of Natural Electrosensory Stimuli by Midbrain Neurons.

Natural stimuli display spatiotemporal characteristics that typically vary over orders of magnitude, and their encoding by sensory neurons remains poorly understood. We investigated population coding of highly heterogeneous natural electrocommunication stimuli in Apteronotus leptorhynchus of either sex. Neuronal activities were positively correlated with one another in the absence of stimulation, and correlation magnitude decayed with increasing distance between recording sites. Under stimulation, we found that correlations between trial-averaged neuronal responses (i.e., signal correlations) were positive and higher in magnitude for neurons located close to another, but that correlations between the trial-to-trial variability (i.e., noise correlations) were independent of physical distance. Overall, signal and noise correlations were independent of stimulus waveform as well as of one another. To investigate how neuronal populations encoded natural electrocommunication stimuli, we considered a nonlinear decoder for which the activities were combined. Decoding performance was best for a timescale of 6 ms, indicating that midbrain neurons transmit information via precise spike timing. A simple summation of neuronal activities (equally weighted sum) revealed that noise correlations limited decoding performance by introducing redundancy. Using an evolution algorithm to optimize performance when considering instead unequally weighted sums of neuronal activities revealed much greater performance values, indicating that midbrain neuron populations transmit information that reliably enable discrimination between different stimulus waveforms. Interestingly, we found that different weight combinations gave rise to similar discriminability, suggesting robustness. Our results have important implications for understanding how natural stimuli are integrated by downstream brain areas to give rise to behavioral responses.SIGNIFICANCE STATEMENT We show that midbrain electrosensory neurons display correlations between their activities and that these can significantly impact performance of decoders. While noise correlations limited discrimination performance by introducing redundancy, considering unequally weighted sums of neuronal activities gave rise to much improved performance and mitigated the deleterious effects of noise correlations. Further analysis revealed that increased discriminability was achieved by making trial-averaged responses more separable, as well as by reducing trial-to-trial variability by eliminating noise correlations. We further found that multiple combinations of weights could give rise to similar discrimination performances, which suggests that such combinatorial codes could be achieved in the brain. We conclude that the activities of midbrain neuronal populations can be used to reliably discriminate between highly heterogeneous stimulus waveforms.

Multiplexed Representation of Itch and Mechanical and Thermal Sensation in the Primary Somatosensory Cortex.

The primary somatosensory cortex (S1) plays a critical role in processing multiple somatosensations, but the mechanism underlying the representation of different submodalities of somatosensation in S1 remains unclear. Using in vivo two-photon calcium imaging that simultaneously monitors hundreds of layer 2/3 pyramidal S1 neurons of awake male mice, we examined neuronal responses triggered by mechanical, thermal, or pruritic stimuli. We found that mechanical, thermal, and pruritic stimuli activated largely overlapping neuronal populations in the same somatotopic S1 subregion. Population decoding analysis revealed that the local neuronal population in S1 encoded sufficient information to distinguish different somatosensory submodalities. Although multimodal S1 neurons responding to multiple types of stimuli exhibited no spatial clustering, S1 neurons preferring mechanical and thermal stimuli tended to show local clustering. These findings demonstrated the coding scheme of different submodalities of somatosensation in S1, paving the way for a deeper understanding of the processing and integration of multimodal somatosensory information in the cortex.SIGNIFICANCE STATEMENT Cortical processing of somatosensory information is one of the most fundamental aspects in cognitive neuroscience. Previous studies mainly focused on mechanical sensory processing within the rodent whisking system, but mechanisms underlying the coding of multiple somatosensations remain largely unknown. In this study, we examined the representation of mechanical, thermal, and pruritic stimuli in S1 by in vivo two-photon calcium imaging of awake mice. We revealed a multiplexed representation for multiple somatosensory stimuli in S1 and demonstrated that the activity of a small population of S1 neurons is capable of decoding different somatosensory submodalities. Our results elucidate the coding mechanism for multiple somatosensations in S1 and provide new insights that improve the present understanding of how the brain processes multimodal sensory information.

Integrated neural dynamics for behavioural decisions and attentional competition in the prefrontal cortex.

In the behaving monkey, complex neural dynamics in the prefrontal cortex contribute to context-dependent decisions and attentional competition. We used demixed principal component analysis to track prefrontal activity dynamics in a cued target detection task. In this task, the animal combined identity of a visual object with a prior instruction cue to determine a target/nontarget decision. From population activity, we extracted principal components for each task feature and examined their time course and sensitivity to stimulus and task variations. For displays containing a single choice object in left or right hemifield, object identity, cue identity and decision were all encoded in population activity, with different dynamics and lateralisation. Object information peaked at 100-200 ms from display onset and was largely confined to the contralateral hemisphere. Cue information was weaker and present even prior to display onset. Integrating information from cue and object, decision information arose more slowly and was bilateral. Individual neurons contributed independently to coding of the three task features. The analysis was then extended to displays with a target in one hemifield and a competing distractor in the other. In this case, the data suggest that each hemisphere initially encoded the identity of the contralateral object. The distractor representation was then rapidly suppressed, with the final target decision again encoded bilaterally. The results show how information is coded along task-related dimensions while competition is resolved and suggest how information flows within and across frontal lobes to implement a learned behavioural decision.

Dual-Factor Representation of the Environmental Context in the Retrosplenial Cortex.

The retrosplenial cortex (RSC) is thought to be involved in a variety of spatial and contextual memory processes. However, we do not know how contextual information might be encoded in the RSC or whether the RSC representations may be distinct from context representations seen in other brain regions such as the hippocampus. We recorded RSC neuronal responses while rats explored different environments and discovered 2 kinds of context representations: one involving a novel rate code in which neurons reliably fire at a higher rate in the preferred context regardless of spatial location, and a second involving context-dependent spatial firing patterns similar to those seen in the hippocampus. This suggests that the RSC employs a unique dual-factor representational mechanism to support contextual memory.

Decoding of Attentional State Using High-Frequency Local Field Potential Is As Accurate As Using Spikes.

Local field potentials (LFPs) in visual cortex are reliably modulated when the subject's focus of attention is cued into versus out of the receptive field of the recorded sites, similar to modulation of spikes. However, human psychophysics studies have used an additional attention condition, neutral cueing, for decades. The effect of neutral cueing on spikes was examined recently and found to be intermediate between cued and uncued conditions. However, whether LFPs are also precise enough to represent graded states of attention is unknown. We found in rhesus monkeys that LFPs during neutral cueing were also intermediate between cued and uncued conditions. For a single electrode, attention was more discriminable using high frequency (>30 Hz) LFP power than spikes, which is expected because LFP represents a population signal and therefore is expected to be less noisy than spikes. However, previous studies have shown that when multiple electrodes are used, spikes can outperform LFPs. Surprisingly, in our study, spikes did not outperform LFPs when discriminability was computed using multiple electrodes, even though the LFP activity was highly correlated across electrodes compared with spikes. These results constrain the spatial scale over which attention operates and highlight the usefulness of LFPs in studying attention.

Concomitant Processing of Choice and Outcome in Frontal Corticostriatal Ensembles Correlates with Performance of Rats.

The frontal cortex-basal ganglia network plays a pivotal role in adaptive goal-directed behaviors. Medial frontal cortex (MFC) encodes information about choices and outcomes into sequential activation of neural population, or neural trajectory. While MFC projects to the dorsal striatum (DS), whether DS also displays temporally coordinated activity remains unknown. We studied this question by simultaneously recording neural ensembles in the MFC and DS of rodents performing an outcome-based alternative choice task. We found that the two regions exhibited highly parallel evolution of neural trajectories, transforming choice information into outcome-related information. When the two trajectories were highly correlated, spike synchrony was task-dependently modulated in some MFC-DS neuron pairs. Our results suggest that neural trajectories concomitantly process decision-relevant information in MFC and DS with increased spike synchrony between these regions.

Optimal Population Coding for Dynamic Input by Nonequilibrium Networks.

The efficient coding hypothesis states that neural response should maximize its information about the external input. Theoretical studies focus on optimal response in single neuron and population code in networks with weak pairwise interactions. However, more biological settings with asymmetric connectivity and the encoding for dynamical stimuli have not been well-characterized. Here, we study the collective response in a kinetic Ising model that encodes the dynamic input. We apply gradient-based method and mean-field approximation to reconstruct networks given the neural code that encodes dynamic input patterns. We measure network asymmetry, decoding performance, and entropy production from networks that generate optimal population code. We analyze how stimulus correlation, time scale, and reliability of the network affect optimal encoding networks. Specifically, we find network dynamics altered by statistics of the dynamic input, identify stimulus encoding strategies, and show optimal effective temperature in the asymmetric networks. We further discuss how this approach connects to the Bayesian framework and continuous recurrent neural networks. Together, these results bridge concepts of nonequilibrium physics with the analyses of dynamics and coding in networks.

Information-Limiting Correlations in Large Neural Populations.

Understanding the neural code requires understanding how populations of neurons code information. Theoretical models predict that information may be limited by correlated noise in large neural populations. Nevertheless, analyses based on tens of neurons have failed to find evidence of saturation. Moreover, some studies have shown that noise correlations can be very small, and therefore may not affect information coding. To determine whether information-limiting correlations exist, we implanted eight Utah arrays in prefrontal cortex (PFC; area 46) of two male macaque monkeys, recording >500 neurons simultaneously. We estimated information in PFC about saccades as a function of ensemble size. Noise correlations were, on average, small (∼10-3). However, information scaled strongly sublinearly with ensemble size. After shuffling trials, destroying noise correlations, information was a linear function of ensemble size. Thus, we provide evidence for the existence of information-limiting noise correlations in large populations of PFC neurons.SIGNIFICANCE STATEMENT Recent theoretical work has shown that even small correlations can limit information if they are "differential correlations," which are difficult to measure directly. However, they can be detected through decoding analyses on recordings from a large number of neurons over a large number of trials. We have achieved both by collecting neural activity in dorsal-lateral prefrontal cortex of macaques using eight microelectrode arrays (768 electrodes), from which we were able to compute accurate information estimates. We show, for the first time, strong evidence for information-limiting correlations. Despite pairwise correlations being small (on the order of 10-3), they affect information coding in populations on the order of 100 s of neurons.

Temporally precise population coding of dynamic sounds by auditory cortex.

Fluctuations in the amplitude envelope of complex sounds provide critical cues for hearing, particularly for speech and animal vocalizations. Responses to amplitude modulation (AM) in the ascending auditory pathway have chiefly been described for single neurons. How neural populations might collectively encode and represent information about AM remains poorly characterized, even in primary auditory cortex (A1). We modeled population responses to AM based on data recorded from A1 neurons in awake squirrel monkeys and evaluated how accurately single trial responses to modulation frequencies from 4 to 512 Hz could be decoded as functions of population size, composition, and correlation structure. We found that a population-based decoding model that simulated convergent, equally weighted inputs was highly accurate and remarkably robust to the inclusion of neurons that were individually poor decoders. By contrast, average rate codes based on convergence performed poorly; effective decoding using average rates was only possible when the responses of individual neurons were segregated, as in classical population decoding models using labeled lines. The relative effectiveness of dynamic rate coding in auditory cortex was explained by shared modulation phase preferences among cortical neurons, despite heterogeneity in rate-based modulation frequency tuning. Our results indicate significant population-based synchrony in primary auditory cortex and suggest that robust population coding of the sound envelope information present in animal vocalizations and speech can be reliably achieved even with indiscriminate pooling of cortical responses. These findings highlight the importance of firing rate dynamics in population-based sensory coding.NEW & NOTEWORTHY Fundamental questions remain about population coding in primary auditory cortex (A1). In particular, issues of spike timing in models of neural populations have been largely ignored. We find that spike-timing in response to sound envelope fluctuations is highly similar across neuron populations in A1. This property of shared envelope phase preference allows for a simple population model involving unweighted convergence of neuronal responses to classify amplitude modulation frequencies with high accuracy.