Brightness illusions drive a neuronal response in the primary visual cortex under top-down modulation.

Brightness illusions are a powerful tool in studying vision, yet their neural correlates are poorly understood. Based on a human paradigm, we presented illusory drifting gratings to mice. Primary visual cortex (V1) neurons responded to illusory gratings, matching their direction selectivity for real gratings, and they tracked the spatial phase offset between illusory and real gratings. Illusion responses were delayed compared to real gratings, in line with the theory that processing illusions requires feedback from higher visual areas (HVAs). We provide support for this theory by showing a reduced V1 response to illusions, but not real gratings, following HVAs optogenetic inhibition. Finally, we used the pupil response (PR) as an indirect perceptual report and showed that the mouse PR matches the human PR to perceived luminance changes. Our findings resolve debates over whether V1 neurons are involved in processing illusions and highlight the involvement of feedback from HVAs.

Visual Corticotectal Neurons in Awake Rabbits: Receptive Fields and Driving Monosynaptic Thalamocortical Inputs.

The superior colliculus receives powerful synaptic inputs from corticotectal neurons in the visual cortex. The function of these corticotectal neurons remains largely unknown due to a limited understanding of their response properties and connectivity. Here, we use antidromic methods to identify corticotectal neurons in awake male and female rabbits, and measure their axonal conduction times, thalamic inputs and receptive field properties. All corticotectal neurons responded to sinusoidal drifting gratings with a nonlinear (nonsinusoidal) increase in mean firing rate but showed pronounced differences in their ON-OFF receptive field structures that we classified into three groups, Cx, S2, and S1. Cx receptive fields had highly overlapping ON and OFF subfields as classical complex cells, S2 had largely separated ON and OFF subfields as classical simple cells, and S1 had a single ON or OFF subfield. Thus, all corticotectal neurons are homogeneous in their nonlinear spatial summation but very heterogeneous in their spatial integration of ON and OFF inputs. The Cx type had the fastest conducting axons, the highest spontaneous activity, and the strongest and fastest visual responses. The S2 type had the highest orientation selectivity, and the S1 type had the slowest conducting axons. Moreover, our cross-correlation analyses found that a subpopulation of corticotectal neurons with very fast conducting axons and high spontaneous firing rates (largely "Cx" type) receives monosynaptic input from retinotopically aligned thalamic neurons. This previously unrecognized fast-conducting thalamic-mediated corticotectal pathway may provide specialized information to superior colliculus and prime recipient neurons for subsequent corticotectal or retinal synaptic input.

A multicolor suite for deciphering population coding of calcium and cAMP in vivo.

cAMP is a universal second messenger regulated by various upstream pathways including Ca2+ and G-protein-coupled receptors (GPCRs). To decipher in vivo cAMP dynamics, we rationally designed cAMPinG1, a sensitive genetically encoded green cAMP indicator that outperformed its predecessors in both dynamic range and cAMP affinity. Two-photon cAMPinG1 imaging detected cAMP transients in the somata and dendritic spines of neurons in the mouse visual cortex on the order of tens of seconds. In addition, multicolor imaging with a sensitive red Ca2+ indicator RCaMP3 allowed simultaneous measurement of population patterns in Ca2+ and cAMP in hundreds of neurons. We found Ca2+-related cAMP responses that represented specific information, such as direction selectivity in vision and locomotion, as well as GPCR-related cAMP responses. Overall, our multicolor suite will facilitate analysis of the interaction between the Ca2+, GPCR and cAMP signaling at single-cell resolution both in vitro and in vivo.

Visuo-frontal interactions during social learning in freely moving macaques.

Social interactions represent a ubiquitous aspect of our everyday life that we acquire by interpreting and responding to visual cues from conspecifics1. However, despite the general acceptance of this view, how visual information is used to guide the decision to cooperate is unknown. Here, we wirelessly recorded the spiking activity of populations of neurons in the visual and prefrontal cortex in conjunction with wireless recordings of oculomotor events while freely moving macaques engaged in social cooperation. As animals learned to cooperate, visual and executive areas refined the representation of social variables, such as the conspecific or reward, by distributing socially relevant information among neurons in each area. Decoding population activity showed that viewing social cues influences the decision to cooperate. Learning social events increased coordinated spiking between visual and prefrontal cortical neurons, which was associated with improved accuracy of neural populations to encode social cues and the decision to cooperate. These results indicate that the visual-frontal cortical network prioritizes relevant sensory information to facilitate learning social interactions while freely moving macaques interact in a naturalistic environment.

Cortical reactivations predict future sensory responses.

Many theories of offline memory consolidation posit that the pattern of neurons activated during a salient sensory experience will be faithfully reactivated, thereby stabilizing the pattern1,2. However, sensory-evoked patterns are not stable but, instead, drift across repeated experiences3-6. Here, to investigate the relationship between reactivations and the drift of sensory representations, we imaged the calcium activity of thousands of excitatory neurons in the mouse lateral visual cortex. During the minute after a visual stimulus, we observed transient, stimulus-specific reactivations, often coupled with hippocampal sharp-wave ripples. Stimulus-specific reactivations were abolished by local cortical silencing during the preceding stimulus. Reactivations early in a session systematically differed from the pattern evoked by the previous stimulus-they were more similar to future stimulus response patterns, thereby predicting both within-day and across-day representational drift. In particular, neurons that participated proportionally more or less in early stimulus reactivations than in stimulus response patterns gradually increased or decreased their future stimulus responses, respectively. Indeed, we could accurately predict future changes in stimulus responses and the separation of responses to distinct stimuli using only the rate and content of reactivations. Thus, reactivations may contribute to a gradual drift and separation in sensory cortical response patterns, thereby enhancing sensory discrimination7.

Learnable latent embeddings for joint behavioural and neural analysis.

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations1-3. In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics3-5. Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

Cortico-cortical feedback engages active dendrites in visual cortex.

Sensory processing in the neocortex requires both feedforward and feedback information flow between cortical areas1. In feedback processing, higher-level representations provide contextual information to lower levels, and facilitate perceptual functions such as contour integration and figure-ground segmentation2,3. However, we have limited understanding of the circuit and cellular mechanisms that mediate feedback influence. Here we use long-range all-optical connectivity mapping in mice to show that feedback influence from the lateromedial higher visual area (LM) to the primary visual cortex (V1) is spatially organized. When the source and target of feedback represent the same area of visual space, feedback is relatively suppressive. By contrast, when the source is offset from the target in visual space, feedback is relatively facilitating. Two-photon calcium imaging data show that this facilitating feedback is nonlinearly integrated in the apical tuft dendrites of V1 pyramidal neurons: retinotopically offset (surround) visual stimuli drive local dendritic calcium signals indicative of regenerative events, and two-photon optogenetic activation of LM neurons projecting to identified feedback-recipient spines in V1 can drive similar branch-specific local calcium signals. Our results show how neocortical feedback connectivity and nonlinear dendritic integration can together form a substrate to support both predictive and cooperative contextual interactions.

Dual-polarity voltage imaging of the concurrent dynamics of multiple neuron types.

Genetically encoded fluorescent voltage indicators are ideally suited to reveal the millisecond-scale interactions among and between targeted cell populations. However, current indicators lack the requisite sensitivity for in vivo multipopulation imaging. We describe next-generation green and red voltage sensors, Ace-mNeon2 and VARNAM2, and their reverse response-polarity variants pAce and pAceR. Our indicators enable 0.4- to 1-kilohertz voltage recordings from >50 spiking neurons per field of view in awake mice and ~30-minute continuous imaging in flies. Using dual-polarity multiplexed imaging, we uncovered brain state-dependent antagonism between neocortical somatostatin-expressing (SST+) and vasoactive intestinal peptide-expressing (VIP+) interneurons and contributions to hippocampal field potentials from cell ensembles with distinct axonal projections. By combining three mutually compatible indicators, we performed simultaneous triple-population imaging. These approaches will empower investigations of the dynamic interplay between neuronal subclasses at single-spike resolution.

Modular strategy for development of the hierarchical visual network in mice.

Hierarchical and parallel networks are fundamental structures of the mammalian brain1-8. During development, lower- and higher-order thalamic nuclei and many cortical areas in the visual system form interareal connections and build hierarchical dorsal and ventral streams9-13. One hypothesis for the development of visual network wiring involves a sequential strategy wherein neural connections are sequentially formed alongside hierarchical structures from lower to higher areas14-17. However, this sequential strategy would be inefficient for building the entire visual network comprising numerous interareal connections. We show that neural pathways from the mouse retina to primary visual cortex (V1) or dorsal/ventral higher visual areas (HVAs) through lower- or higher-order thalamic nuclei form as parallel modules before corticocortical connections. Subsequently, corticocortical connections among V1 and HVAs emerge to combine these modules. Retina-derived activity propagating the initial parallel modules is necessary to establish retinotopic inter-module connections. Thus, the visual network develops in a modular manner involving initial establishment of parallel modules and their subsequent concatenation. Findings in this study raise the possibility that parallel modules from higher-order thalamic nuclei to HVAs act as templates for cortical ventral and dorsal streams and suggest that the brain has an efficient strategy for the development of a hierarchical network comprising numerous areas.

Cortical feedback loops bind distributed representations of working memory.

Working memory-the brain's ability to internalize information and use it flexibly to guide behaviour-is an essential component of cognition. Although activity related to working memory has been observed in several brain regions1-3, how neural populations actually represent working memory4-7 and the mechanisms by which this activity is maintained8-12 remain unclear13-15. Here we describe the neural implementation of visual working memory in mice alternating between a delayed non-match-to-sample task and a simple discrimination task that does not require working memory but has identical stimulus, movement and reward statistics. Transient optogenetic inactivations revealed that distributed areas of the neocortex were required selectively for the maintenance of working memory. Population activity in visual area AM and premotor area M2 during the delay period was dominated by orderly low-dimensional dynamics16,17 that were, however, independent of working memory. Instead, working memory representations were embedded in high-dimensional population activity, present in both cortical areas, persisted throughout the inter-stimulus delay period, and predicted behavioural responses during the working memory task. To test whether the distributed nature of working memory was dependent on reciprocal interactions between cortical regions18-20, we silenced one cortical area (AM or M2) while recording the feedback it received from the other. Transient inactivation of either area led to the selective disruption of inter-areal communication of working memory. Therefore, reciprocally interconnected cortical areas maintain bound high-dimensional representations of working memory.

A transcriptomic axis predicts state modulation of cortical interneurons.

Transcriptomics has revealed that cortical inhibitory neurons exhibit a great diversity of fine molecular subtypes1-6, but it is not known whether these subtypes have correspondingly diverse patterns of activity in the living brain. Here we show that inhibitory subtypes in primary visual cortex (V1) have diverse correlates with brain state, which are organized by a single factor: position along the main axis of transcriptomic variation. We combined in vivo two-photon calcium imaging of mouse V1 with a transcriptomic method to identify mRNA for 72 selected genes in ex vivo slices. We classified inhibitory neurons imaged in layers 1-3 into a three-level hierarchy of 5 subclasses, 11 types and 35 subtypes using previously defined transcriptomic clusters3. Responses to visual stimuli differed significantly only between subclasses, with cells in the Sncg subclass uniformly suppressed, and cells in the other subclasses predominantly excited. Modulation by brain state differed at all hierarchical levels but could be largely predicted from the first transcriptomic principal component, which also predicted correlations with simultaneously recorded cells. Inhibitory subtypes that fired more in resting, oscillatory brain states had a smaller fraction of their axonal projections in layer 1, narrower spikes, lower input resistance and weaker adaptation as determined in vitro7, and expressed more inhibitory cholinergic receptors. Subtypes that fired more during arousal had the opposite properties. Thus, a simple principle may largely explain how diverse inhibitory V1 subtypes shape state-dependent cortical processing.

Face neurons encode nonsemantic features.

The primate inferior temporal cortex contains neurons that respond more strongly to faces than to other objects. Termed "face neurons," these neurons are thought to be selective for faces as a semantic category. However, face neurons also partly respond to clocks, fruits, and single eyes, raising the question of whether face neurons are better described as selective for visual features related to faces but dissociable from them. We used a recently described algorithm, XDream, to evolve stimuli that strongly activated face neurons. XDream leverages a generative neural network that is not limited to realistic objects. Human participants assessed images evolved for face neurons and for nonface neurons and natural images depicting faces, cars, fruits, etc. Evolved images were consistently judged to be distinct from real faces. Images evolved for face neurons were rated as slightly more similar to faces than images evolved for nonface neurons. There was a correlation among natural images between face neuron activity and subjective "faceness" ratings, but this relationship did not hold for face neuron­evolved images, which triggered high activity but were rated low in faceness. Our results suggest that so-called face neurons are better described as tuned to visual features rather than semantic categories.

Local connectivity and synaptic dynamics in mouse and human neocortex.

We present a unique, extensive, and open synaptic physiology analysis platform and dataset. Through its application, we reveal principles that relate cell type to synaptic properties and intralaminar circuit organization in the mouse and human cortex. The dynamics of excitatory synapses align with the postsynaptic cell subclass, whereas inhibitory synapse dynamics partly align with presynaptic cell subclass but with considerable overlap. Synaptic properties are heterogeneous in most subclass-to-subclass connections. The two main axes of heterogeneity are strength and variability. Cell subclasses divide along the variability axis, whereas the strength axis accounts for substantial heterogeneity within the subclass. In the human cortex, excitatory-to-excitatory synaptic dynamics are distinct from those in the mouse cortex and vary with depth across layers 2 and 3.

Three-dimensional multi-site random access photostimulation (3D-MAP).

Optical control of neural ensemble activity is crucial for understanding brain function and disease, yet no technology can achieve optogenetic control of very large numbers of neurons at an extremely fast rate over a large volume. State-of-the-art multiphoton holographic optogenetics requires high-power illumination that only addresses relatively small populations of neurons in parallel. Conversely, one-photon holographic techniques can stimulate more neurons with two to three orders lower power, but with limited resolution or addressable volume. Perhaps most problematically, two-photon holographic optogenetic systems are extremely expensive and sophisticated which has precluded their broader adoption in the neuroscience community. To address this technical gap, we introduce a new one-photon light sculpting technique, three-dimensional multi-site random access photostimulation (3D-MAP), that overcomes these limitations by modulating light dynamically, both in the spatial and in the angular domain at multi-kHz rates. We use 3D-MAP to interrogate neural circuits in 3D and demonstrate simultaneous photostimulation and imaging of dozens of user-selected neurons in the intact mouse brain in vivo with high spatio-temporal resolution. 3D-MAP can be broadly adopted for high-throughput all-optical interrogation of brain circuits owing to its powerful combination of scale, speed, simplicity, and cost.

Moving bar of light evokes vectorial spatial selectivity in the immobile rat hippocampus.

Visual cortical neurons encode the position and motion direction of specific stimuli retrospectively, without any locomotion or task demand1. The hippocampus, which is a part of the visual system, is hypothesized to require self-motion or a cognitive task to generate allocentric spatial selectivity that is scalar, abstract2,3 and prospective4-7. Here we measured rodent hippocampal selectivity to a moving bar of light in a body-fixed rat to bridge these seeming disparities. About 70% of dorsal CA1 neurons showed stable activity modulation as a function of the angular position of the bar, independent of behaviour and rewards. One-third of tuned cells also encoded the direction of revolution. In other experiments, neurons encoded the distance of the bar, with preference for approaching motion. Collectively, these demonstrate visually evoked vectorial selectivity (VEVS). Unlike place cells, VEVS was retrospective. Changes in the visual stimulus or its predictability did not cause remapping but only caused gradual changes. Most VEVS-tuned neurons behaved like place cells during spatial exploration and the two selectivities were correlated. Thus, VEVS could form the basic building block of hippocampal activity. When combined with self-motion, reward or multisensory stimuli8, it can generate the complexity of prospective representations including allocentric space9, time10,11 and episodes12.

Temporal dynamics of the neural representation of hue and luminance polarity.

Hue and luminance contrast are basic visual features. Here we use multivariate analyses of magnetoencephalography data to investigate the timing of the neural computations that extract them, and whether they depend on common neural circuits. We show that hue and luminance-contrast polarity can be decoded from MEG data and, with lower accuracy, both features can be decoded across changes in the other feature. These results are consistent with the existence of both common and separable neural mechanisms. The decoding time course is earlier and more temporally precise for luminance polarity than hue, a result that does not depend on task, suggesting that luminance contrast is an updating signal that separates visual events. Meanwhile, cross-temporal generalization is slightly greater for representations of hue compared to luminance polarity, providing a neural correlate of the preeminence of hue in perceptual grouping and memory. Finally, decoding of luminance polarity varies depending on the hues used to obtain training and testing data. The pattern of results is consistent with observations that luminance contrast is mediated by both L-M and S cone sub-cortical mechanisms.

Synaptic connectivity to L2/3 of primary visual cortex measured by two-photon optogenetic stimulation.

Understanding cortical microcircuits requires thorough measurement of physiological properties of synaptic connections formed within and between diverse subclasses of neurons. Towards this goal, we combined spatially precise optogenetic stimulation with multicellular recording to deeply characterize intralaminar and translaminar monosynaptic connections to supragranular (L2/3) neurons in the mouse visual cortex. The reliability and specificity of multiphoton optogenetic stimulation were measured across multiple Cre lines, and measurements of connectivity were verified by comparison to paired recordings and targeted patching of optically identified presynaptic cells. With a focus on translaminar pathways, excitatory and inhibitory synaptic connections from genetically defined presynaptic populations were characterized by their relative abundance, spatial profiles, strength, and short-term dynamics. Consistent with the canonical cortical microcircuit, layer 4 excitatory neurons and interneurons within L2/3 represented the most common sources of input to L2/3 pyramidal cells. More surprisingly, we also observed strong excitatory connections from layer 5 intratelencephalic neurons and potent translaminar inhibition from multiple interneuron subclasses. The hybrid approach revealed convergence to and divergence from excitatory and inhibitory neurons within and across cortical layers. Divergent excitatory connections often spanned hundreds of microns of horizontal space. In contrast, divergent inhibitory connections were more frequently measured from postsynaptic targets near each other.

Constrained brain volume in an efficient coding model explains the fraction of excitatory and inhibitory neurons in sensory cortices.

The number of neurons in mammalian cortex varies by multiple orders of magnitude across different species. In contrast, the ratio of excitatory to inhibitory neurons (E:I ratio) varies in a much smaller range, from 3:1 to 9:1 and remains roughly constant for different sensory areas within a species. Despite this structure being important for understanding the function of neural circuits, the reason for this consistency is not yet understood. While recent models of vision based on the efficient coding hypothesis show that increasing the number of both excitatory and inhibitory cells improves stimulus representation, the two cannot increase simultaneously due to constraints on brain volume. In this work, we implement an efficient coding model of vision under a constraint on the volume (using number of neurons as a surrogate) while varying the E:I ratio. We show that the performance of the model is optimal at biologically observed E:I ratios under several metrics. We argue that this happens due to trade-offs between the computational accuracy and the representation capacity for natural stimuli. Further, we make experimentally testable predictions that 1) the optimal E:I ratio should be higher for species with a higher sparsity in the neural activity and 2) the character of inhibitory synaptic distributions and firing rates should change depending on E:I ratio. Our findings, which are supported by our new preliminary analyses of publicly available data, provide the first quantitative and testable hypothesis based on optimal coding models for the distribution of excitatory and inhibitory neural types in the mammalian sensory cortices.

Vision-dependent specification of cell types and function in the developing cortex.

The role of postnatal experience in sculpting cortical circuitry, while long appreciated, is poorly understood at the level of cell types. We explore this in the mouse primary visual cortex (V1) using single-nucleus RNA sequencing, visual deprivation, genetics, and functional imaging. We find that vision selectively drives the specification of glutamatergic cell types in upper layers (L) (L2/3/4), while deeper-layer glutamatergic, GABAergic, and non-neuronal cell types are established prior to eye opening. L2/3 cell types form an experience-dependent spatial continuum defined by the graded expression of ∼200 genes, including regulators of cell adhesion and synapse formation. One of these genes, Igsf9b, a vision-dependent gene encoding an inhibitory synaptic cell adhesion molecule, is required for the normal development of binocular responses in L2/3. In summary, vision preferentially regulates the development of upper-layer glutamatergic cell types through the regulation of cell-type-specific gene expression programs.

Fast optical recording of neuronal activity by three-dimensional custom-access serial holography.

Optical recording of neuronal activity in three-dimensional (3D) brain circuits at cellular and millisecond resolution in vivo is essential for probing information flow in the brain. While random-access multiphoton microscopy permits fast optical access to neuronal targets in three dimensions, the method is challenged by motion artifacts when recording from behaving animals. Therefore, we developed three-dimensional custom-access serial holography (3D-CASH). Built on a fast acousto-optic light modulator, 3D-CASH performs serial sampling at 40 kHz from neurons at freely selectable 3D locations. Motion artifacts are eliminated by targeting each neuron with a size-optimized pattern of excitation light covering the cell body and its anticipated displacement field. Spike rates inferred from GCaMP6f recordings in visual cortex of awake mice tracked the phase of a moving bar stimulus with higher spike correlation between intra compared to interlaminar neuron pairs. 3D-CASH offers access to the millisecond correlation structure of in vivo neuronal activity in 3D microcircuits.