Dynamic Stimulation of Visual Cortex Produces Form Vision in Sighted and Blind Humans.

A visual cortical prosthesis (VCP) has long been proposed as a strategy for restoring useful vision to the blind, under the assumption that visual percepts of small spots of light produced with electrical stimulation of visual cortex (phosphenes) will combine into coherent percepts of visual forms, like pixels on a video screen. We tested an alternative strategy in which shapes were traced on the surface of visual cortex by stimulating electrodes in dynamic sequence. In both sighted and blind participants, dynamic stimulation enabled accurate recognition of letter shapes predicted by the brain's spatial map of the visual world. Forms were presented and recognized rapidly by blind participants, up to 86 forms per minute. These findings demonstrate that a brain prosthetic can produce coherent percepts of visual forms.

Controlling Visually Guided Behavior by Holographic Recalling of Cortical Ensembles.

Neurons in cortical circuits are often coactivated as ensembles, yet it is unclear whether ensembles play a functional role in behavior. Some ensemble neurons have pattern completion properties, triggering the entire ensemble when activated. Using two-photon holographic optogenetics in mouse primary visual cortex, we tested whether recalling ensembles by activating pattern completion neurons alters behavioral performance in a visual task. Disruption of behaviorally relevant ensembles by activation of non-selective neurons decreased performance, whereas activation of only two pattern completion neurons from behaviorally relevant ensembles improved performance, by reliably recalling the whole ensemble. Also, inappropriate behavioral choices were evoked by the mistaken activation of behaviorally relevant ensembles. Finally, in absence of visual stimuli, optogenetic activation of two pattern completion neurons could trigger behaviorally relevant ensembles and correct behavioral responses. Our results demonstrate a causal role of neuronal ensembles in a visually guided behavior and suggest that ensembles implement internal representations of perceptual states.

Visual Projection Neurons Mediating Directed Courtship in Drosophila.

Many animals rely on vision to detect, locate, and track moving objects. In Drosophila courtship, males primarily use visual cues to orient toward and follow females and to select the ipsilateral wing for courtship song. Here, we show that the LC10 visual projection neurons convey essential visual information during courtship. Males with LC10 neurons silenced are unable to orient toward or maintain proximity to the female and do not predominantly use the ipsilateral wing when singing. LC10 neurons preferentially respond to small moving objects using an antagonistic motion-based center-surround mechanism. Unilateral activation of LC10 neurons recapitulates the orienting and ipsilateral wing extension normally elicited by females, and the potency with which LC10 induces wing extension is enhanced in a state of courtship arousal controlled by male-specific P1 neurons. These data suggest that LC10 is a major pathway relaying visual input to the courtship circuits in the male brain.

Visuomotor Coupling Shapes the Functional Development of Mouse Visual Cortex.

The emergence of sensory-guided behavior depends on sensorimotor coupling during development. How sensorimotor experience shapes neural processing is unclear. Here, we show that the coupling between motor output and visual feedback is necessary for the functional development of visual processing in layer 2/3 (L2/3) of primary visual cortex (V1) of the mouse. Using a virtual reality system, we reared mice in conditions of normal or random visuomotor coupling. We recorded the activity of identified excitatory and inhibitory L2/3 neurons in response to transient visuomotor mismatches in both groups of mice. Mismatch responses in excitatory neurons were strongly experience dependent and driven by a transient release from inhibition mediated by somatostatin-positive interneurons. These data are consistent with a model in which L2/3 of V1 computes a difference between an inhibitory visual input and an excitatory locomotion-related input, where the balance between these two inputs is finely tuned by visuomotor experience.

Integration of Feedforward and Feedback Information Streams in the Modular Architecture of Mouse Visual Cortex.

Radial cell columns are a hallmark feature of cortical architecture in many mammalian species. It has long been held, based on the lack of orientation columns, that such functional units are absent in rodent primary visual cortex (V1). These observations led to the view that rodent visual cortex has a fundamentally different network architecture than that of carnivores and primates. While columns may be lacking in rodent V1, we describe in this review that modular clusters of inputs to layer 1 and projection neurons in the layers below are prominent features of the mouse visual cortex. We propose that modules organize thalamocortical inputs, intracortical processing streams, and transthalamic communications that underlie distinct sensory and sensorimotor functions.

Neuronal Firing Rate Homeostasis Is Inhibited by Sleep and Promoted by Wake.

Homeostatic mechanisms stabilize neural circuit function by keeping firing rates within a set-point range, but whether this process is gated by brain state is unknown. Here, we monitored firing rate homeostasis in individual visual cortical neurons in freely behaving rats as they cycled between sleep and wake states. When neuronal firing rates were perturbed by visual deprivation, they gradually returned to a precise, cell-autonomous set point during periods of active wake, with lengthening of the wake period enhancing firing rate rebound. Unexpectedly, this resetting of neuronal firing was suppressed during sleep. This raises the possibility that memory consolidation or other sleep-dependent processes are vulnerable to interference from homeostatic plasticity mechanisms. PAPERCLIP.

Cortical reactivations predict future sensory responses.

Many theories of offline memory consolidation posit that the pattern of neurons activated during a salient sensory experience will be faithfully reactivated, thereby stabilizing the pattern1,2. However, sensory-evoked patterns are not stable but, instead, drift across repeated experiences3-6. Here, to investigate the relationship between reactivations and the drift of sensory representations, we imaged the calcium activity of thousands of excitatory neurons in the mouse lateral visual cortex. During the minute after a visual stimulus, we observed transient, stimulus-specific reactivations, often coupled with hippocampal sharp-wave ripples. Stimulus-specific reactivations were abolished by local cortical silencing during the preceding stimulus. Reactivations early in a session systematically differed from the pattern evoked by the previous stimulus-they were more similar to future stimulus response patterns, thereby predicting both within-day and across-day representational drift. In particular, neurons that participated proportionally more or less in early stimulus reactivations than in stimulus response patterns gradually increased or decreased their future stimulus responses, respectively. Indeed, we could accurately predict future changes in stimulus responses and the separation of responses to distinct stimuli using only the rate and content of reactivations. Thus, reactivations may contribute to a gradual drift and separation in sensory cortical response patterns, thereby enhancing sensory discrimination7.

Minute-scale oscillatory sequences in medial entorhinal cortex.

The medial entorhinal cortex (MEC) hosts many of the brain's circuit elements for spatial navigation and episodic memory, operations that require neural activity to be organized across long durations of experience1. Whereas location is known to be encoded by spatially tuned cell types in this brain region2,3, little is known about how the activity of entorhinal cells is tied together over time at behaviourally relevant time scales, in the second-to-minute regime. Here we show that MEC neuronal activity has the capacity to be organized into ultraslow oscillations, with periods ranging from tens of seconds to minutes. During these oscillations, the activity is further organized into periodic sequences. Oscillatory sequences manifested while mice ran at free pace on a rotating wheel in darkness, with no change in location or running direction and no scheduled rewards. The sequences involved nearly the entire cell population, and transcended epochs of immobility. Similar sequences were not observed in neighbouring parasubiculum or in visual cortex. Ultraslow oscillatory sequences in MEC may have the potential to couple neurons and circuits across extended time scales and serve as a template for new sequence formation during navigation and episodic memory formation.

Visuo-frontal interactions during social learning in freely moving macaques.

Social interactions represent a ubiquitous aspect of our everyday life that we acquire by interpreting and responding to visual cues from conspecifics1. However, despite the general acceptance of this view, how visual information is used to guide the decision to cooperate is unknown. Here, we wirelessly recorded the spiking activity of populations of neurons in the visual and prefrontal cortex in conjunction with wireless recordings of oculomotor events while freely moving macaques engaged in social cooperation. As animals learned to cooperate, visual and executive areas refined the representation of social variables, such as the conspecific or reward, by distributing socially relevant information among neurons in each area. Decoding population activity showed that viewing social cues influences the decision to cooperate. Learning social events increased coordinated spiking between visual and prefrontal cortical neurons, which was associated with improved accuracy of neural populations to encode social cues and the decision to cooperate. These results indicate that the visual-frontal cortical network prioritizes relevant sensory information to facilitate learning social interactions while freely moving macaques interact in a naturalistic environment.

Response sub-additivity and variability quenching in visual cortex.

Sub-additivity and variability are ubiquitous response motifs in the primary visual cortex (V1). Response sub-additivity enables the construction of useful interpretations of the visual environment, whereas response variability indicates the factors that limit the precision with which the brain can do this. There is increasing evidence that experimental manipulations that elicit response sub-additivity often also quench response variability. Here, we provide an overview of these phenomena and suggest that they may have common origins. We discuss empirical findings and recent model-based insights into the functional operations, computational objectives and circuit mechanisms underlying V1 activity. These different modelling approaches all predict that response sub-additivity and variability quenching often co-occur. The phenomenology of these two response motifs, as well as many of the insights obtained about them in V1, generalize to other cortical areas. Thus, the connection between response sub-additivity and variability quenching may be a canonical motif across the cortex.

Learnable latent embeddings for joint behavioural and neural analysis.

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations1-3. In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics3-5. Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

Cortico-cortical feedback engages active dendrites in visual cortex.

Sensory processing in the neocortex requires both feedforward and feedback information flow between cortical areas1. In feedback processing, higher-level representations provide contextual information to lower levels, and facilitate perceptual functions such as contour integration and figure-ground segmentation2,3. However, we have limited understanding of the circuit and cellular mechanisms that mediate feedback influence. Here we use long-range all-optical connectivity mapping in mice to show that feedback influence from the lateromedial higher visual area (LM) to the primary visual cortex (V1) is spatially organized. When the source and target of feedback represent the same area of visual space, feedback is relatively suppressive. By contrast, when the source is offset from the target in visual space, feedback is relatively facilitating. Two-photon calcium imaging data show that this facilitating feedback is nonlinearly integrated in the apical tuft dendrites of V1 pyramidal neurons: retinotopically offset (surround) visual stimuli drive local dendritic calcium signals indicative of regenerative events, and two-photon optogenetic activation of LM neurons projecting to identified feedback-recipient spines in V1 can drive similar branch-specific local calcium signals. Our results show how neocortical feedback connectivity and nonlinear dendritic integration can together form a substrate to support both predictive and cooperative contextual interactions.

Distinguishing externally from saccade-induced motion in visual cortex.

Distinguishing sensory stimuli caused by changes in the environment from those caused by an animal's own actions is a hallmark of sensory processing1. Saccades are rapid eye movements that shift the image on the retina. How visual systems differentiate motion of the image induced by saccades from actual motion in the environment is not fully understood2. Here we discovered that in mouse primary visual cortex (V1) the two types of motion evoke distinct activity patterns. This is because, during saccades, V1 combines the visual input with a strong non-visual input arriving from the thalamic pulvinar nucleus. The non-visual input triggers responses that are specific to the direction of the saccade and the visual input triggers responses that are specific to the direction of the shift of the stimulus on the retina, yet the preferred directions of these two responses are uncorrelated. Thus, the pulvinar input ensures differential V1 responses to external and self-generated motion. Integration of external sensory information with information about body movement may be a general mechanism for sensory cortices to distinguish between self-generated and external stimuli.

Cortical feedback loops bind distributed representations of working memory.

Working memory-the brain's ability to internalize information and use it flexibly to guide behaviour-is an essential component of cognition. Although activity related to working memory has been observed in several brain regions1-3, how neural populations actually represent working memory4-7 and the mechanisms by which this activity is maintained8-12 remain unclear13-15. Here we describe the neural implementation of visual working memory in mice alternating between a delayed non-match-to-sample task and a simple discrimination task that does not require working memory but has identical stimulus, movement and reward statistics. Transient optogenetic inactivations revealed that distributed areas of the neocortex were required selectively for the maintenance of working memory. Population activity in visual area AM and premotor area M2 during the delay period was dominated by orderly low-dimensional dynamics16,17 that were, however, independent of working memory. Instead, working memory representations were embedded in high-dimensional population activity, present in both cortical areas, persisted throughout the inter-stimulus delay period, and predicted behavioural responses during the working memory task. To test whether the distributed nature of working memory was dependent on reciprocal interactions between cortical regions18-20, we silenced one cortical area (AM or M2) while recording the feedback it received from the other. Transient inactivation of either area led to the selective disruption of inter-areal communication of working memory. Therefore, reciprocally interconnected cortical areas maintain bound high-dimensional representations of working memory.

A transcriptomic axis predicts state modulation of cortical interneurons.

Transcriptomics has revealed that cortical inhibitory neurons exhibit a great diversity of fine molecular subtypes1-6, but it is not known whether these subtypes have correspondingly diverse patterns of activity in the living brain. Here we show that inhibitory subtypes in primary visual cortex (V1) have diverse correlates with brain state, which are organized by a single factor: position along the main axis of transcriptomic variation. We combined in vivo two-photon calcium imaging of mouse V1 with a transcriptomic method to identify mRNA for 72 selected genes in ex vivo slices. We classified inhibitory neurons imaged in layers 1-3 into a three-level hierarchy of 5 subclasses, 11 types and 35 subtypes using previously defined transcriptomic clusters3. Responses to visual stimuli differed significantly only between subclasses, with cells in the Sncg subclass uniformly suppressed, and cells in the other subclasses predominantly excited. Modulation by brain state differed at all hierarchical levels but could be largely predicted from the first transcriptomic principal component, which also predicted correlations with simultaneously recorded cells. Inhibitory subtypes that fired more in resting, oscillatory brain states had a smaller fraction of their axonal projections in layer 1, narrower spikes, lower input resistance and weaker adaptation as determined in vitro7, and expressed more inhibitory cholinergic receptors. Subtypes that fired more during arousal had the opposite properties. Thus, a simple principle may largely explain how diverse inhibitory V1 subtypes shape state-dependent cortical processing.

Moving bar of light evokes vectorial spatial selectivity in the immobile rat hippocampus.

Visual cortical neurons encode the position and motion direction of specific stimuli retrospectively, without any locomotion or task demand1. The hippocampus, which is a part of the visual system, is hypothesized to require self-motion or a cognitive task to generate allocentric spatial selectivity that is scalar, abstract2,3 and prospective4-7. Here we measured rodent hippocampal selectivity to a moving bar of light in a body-fixed rat to bridge these seeming disparities. About 70% of dorsal CA1 neurons showed stable activity modulation as a function of the angular position of the bar, independent of behaviour and rewards. One-third of tuned cells also encoded the direction of revolution. In other experiments, neurons encoded the distance of the bar, with preference for approaching motion. Collectively, these demonstrate visually evoked vectorial selectivity (VEVS). Unlike place cells, VEVS was retrospective. Changes in the visual stimulus or its predictability did not cause remapping but only caused gradual changes. Most VEVS-tuned neurons behaved like place cells during spatial exploration and the two selectivities were correlated. Thus, VEVS could form the basic building block of hippocampal activity. When combined with self-motion, reward or multisensory stimuli8, it can generate the complexity of prospective representations including allocentric space9, time10,11 and episodes12.

Modular strategy for development of the hierarchical visual network in mice.

Hierarchical and parallel networks are fundamental structures of the mammalian brain1-8. During development, lower- and higher-order thalamic nuclei and many cortical areas in the visual system form interareal connections and build hierarchical dorsal and ventral streams9-13. One hypothesis for the development of visual network wiring involves a sequential strategy wherein neural connections are sequentially formed alongside hierarchical structures from lower to higher areas14-17. However, this sequential strategy would be inefficient for building the entire visual network comprising numerous interareal connections. We show that neural pathways from the mouse retina to primary visual cortex (V1) or dorsal/ventral higher visual areas (HVAs) through lower- or higher-order thalamic nuclei form as parallel modules before corticocortical connections. Subsequently, corticocortical connections among V1 and HVAs emerge to combine these modules. Retina-derived activity propagating the initial parallel modules is necessary to establish retinotopic inter-module connections. Thus, the visual network develops in a modular manner involving initial establishment of parallel modules and their subsequent concatenation. Findings in this study raise the possibility that parallel modules from higher-order thalamic nuclei to HVAs act as templates for cortical ventral and dorsal streams and suggest that the brain has an efficient strategy for the development of a hierarchical network comprising numerous areas.

State-dependent pupil dilation rapidly shifts visual feature selectivity.

To increase computational flexibility, the processing of sensory inputs changes with behavioural context. In the visual system, active behavioural states characterized by motor activity and pupil dilation1,2 enhance sensory responses, but typically leave the preferred stimuli of neurons unchanged2-9. Here we find that behavioural state also modulates stimulus selectivity in the mouse visual cortex in the context of coloured natural scenes. Using population imaging in behaving mice, pharmacology and deep neural network modelling, we identified a rapid shift in colour selectivity towards ultraviolet stimuli during an active behavioural state. This was exclusively caused by state-dependent pupil dilation, which resulted in a dynamic switch from rod to cone photoreceptors, thereby extending their role beyond night and day vision. The change in tuning facilitated the decoding of ethological stimuli, such as aerial predators against the twilight sky10. For decades, studies in neuroscience and cognitive science have used pupil dilation as an indirect measure of brain state. Our data suggest that, in addition, state-dependent pupil dilation itself tunes visual representations to behavioural demands by differentially recruiting rods and cones on fast timescales.

A multicolor suite for deciphering population coding of calcium and cAMP in vivo.

cAMP is a universal second messenger regulated by various upstream pathways including Ca2+ and G-protein-coupled receptors (GPCRs). To decipher in vivo cAMP dynamics, we rationally designed cAMPinG1, a sensitive genetically encoded green cAMP indicator that outperformed its predecessors in both dynamic range and cAMP affinity. Two-photon cAMPinG1 imaging detected cAMP transients in the somata and dendritic spines of neurons in the mouse visual cortex on the order of tens of seconds. In addition, multicolor imaging with a sensitive red Ca2+ indicator RCaMP3 allowed simultaneous measurement of population patterns in Ca2+ and cAMP in hundreds of neurons. We found Ca2+-related cAMP responses that represented specific information, such as direction selectivity in vision and locomotion, as well as GPCR-related cAMP responses. Overall, our multicolor suite will facilitate analysis of the interaction between the Ca2+, GPCR and cAMP signaling at single-cell resolution both in vitro and in vivo.

Cortical response selectivity derives from strength in numbers of synapses.

Single neocortical neurons are driven by populations of excitatory inputs, which form the basis of neuronal selectivity to features of sensory input. Excitatory connections are thought to mature during development through activity-dependent Hebbian plasticity1, whereby similarity between presynaptic and postsynaptic activity selectively strengthens some synapses and weakens others2. Evidence in support of this process includes measurements of synaptic ultrastructure and in vitro and in vivo physiology and imaging studies3-8. These corroborating lines of evidence lead to the prediction that a small number of strong synaptic inputs drive neuronal selectivity, whereas weak synaptic inputs are less correlated with the somatic output and modulate activity overall6,7. Supporting evidence from cortical circuits, however, has been limited to measurements of neighbouring, connected cell pairs, raising the question of whether this prediction holds for a broad range of synapses converging onto cortical neurons. Here we measure the strengths of functionally characterized excitatory inputs contacting single pyramidal neurons in ferret primary visual cortex (V1) by combining in vivo two-photon synaptic imaging and post hoc electron microscopy. Using electron microscopy reconstruction of individual synapses as a metric of strength, we find no evidence that strong synapses have a predominant role in the selectivity of cortical neuron responses to visual stimuli. Instead, selectivity appears to arise from the total number of synapses activated by different stimuli. Moreover, spatial clustering of co-active inputs appears to be reserved for weaker synapses, enhancing the contribution of weak synapses to somatic responses. Our results challenge the role of Hebbian mechanisms in shaping neuronal selectivity in cortical circuits, and suggest that selectivity reflects the co-activation of large populations of presynaptic neurons with similar properties and a mixture of strengths.