Simple synaptic modulations implement diverse novelty computations.

Detecting novelty is ethologically useful for an organism's survival. Recent experiments characterize how different types of novelty over timescales from seconds to weeks are reflected in the activity of excitatory and inhibitory neuron types. Here, we introduce a learning mechanism, familiarity-modulated synapses (FMSs), consisting of multiplicative modulations dependent on presynaptic or pre/postsynaptic neuron activity. With FMSs, network responses that encode novelty emerge under unsupervised continual learning and minimal connectivity constraints. Implementing FMSs within an experimentally constrained model of a visual cortical circuit, we demonstrate the generalizability of FMSs by simultaneously fitting absolute, contextual, and omission novelty effects. Our model also reproduces functional diversity within cell subpopulations, leading to experimentally testable predictions about connectivity and synaptic dynamics that can produce both population-level novelty responses and heterogeneous individual neuron signals. Altogether, our findings demonstrate how simple plasticity mechanisms within a cortical circuit structure can produce qualitatively distinct and complex novelty responses.

Behavioral strategy shapes activation of the Vip-Sst disinhibitory circuit in visual cortex.

In complex environments, animals can adopt diverse strategies to find rewards. How distinct strategies differentially engage brain circuits is not well understood. Here, we investigate this question, focusing on the cortical Vip-Sst disinhibitory circuit between vasoactive intestinal peptide-postive (Vip) interneurons and somatostatin-positive (Sst) interneurons. We characterize the behavioral strategies used by mice during a visual change detection task. Using a dynamic logistic regression model, we find that individual mice use mixtures of a visual comparison strategy and a statistical timing strategy. Separately, mice also have periods of task engagement and disengagement. Two-photon calcium imaging shows large strategy-dependent differences in neural activity in excitatory, Sst inhibitory, and Vip inhibitory cells in response to both image changes and image omissions. In contrast, task engagement has limited effects on neural population activity. We find that the diversity of neural correlates of strategy can be understood parsimoniously as the increased activation of the Vip-Sst disinhibitory circuit during the visual comparison strategy, which facilitates task-appropriate responses.

Coordinated changes in a cortical circuit sculpt effects of novelty on neural dynamics.

Recent studies have found dramatic cell-type specific responses to stimulus novelty, highlighting the importance of analyzing the cortical circuitry at the cell-type specific level of granularity to understand brain function. Although initial work classified and characterized activity for each cell type, the specific alterations in cortical circuitry-particularly when multiple novelty effects interact-remain unclear. To address this gap, we employed a large-scale public dataset of electrophysiological recordings in the visual cortex of awake, behaving mice using Neuropixels probes and designed population network models to investigate the observed changes in neural dynamics in response to a combination of distinct forms of novelty. The model parameters were rigorously constrained by publicly available structural datasets, including multi-patch synaptic physiology and electron microscopy data. Our systematic optimization approach identified tens of thousands of model parameter sets that replicate the observed neural activity. Analysis of these solutions revealed generally weaker connections under novel stimuli, as well as a shift in the balance e between SST and VIP populations. Along with this, PV and SST populations experienced overall more excitatory influences compared to excitatory and VIP populations. Our results also highlight the role of VIP neurons in multiple aspects of visual stimulus processing and altering gain and saturation dynamics under novel conditions. In sum, our findings provide a systematic characterization of how the cortical circuit adapts to stimulus novelty by combining multiple rich public datasets.

Familiarity modulated synapses model visual cortical circuit novelty responses.

Since environments are constantly in flux, the brain's ability to identify novel stimuli that fall outside its own internal representation of the world is crucial for an organism's survival. Within the mammalian neocortex, inhibitory microcircuits are proposed to regulate activity in an experience-dependent manner and different inhibitory neuron subtypes exhibit distinct novelty responses. Discerning the function of diverse neural circuits and their modulation by experience can be daunting unless one has a biologically plausible mechanism to detect and learn from novel experiences that is both understandable and flexible. Here we introduce a learning mechanism, familiarity modulated synapses (FMSs), through which a network response that encodes novelty emerges from unsupervised synaptic modifications depending only on the presynaptic or both the pre- and postsynaptic activity. FMSs stand apart from other familiarity mechanisms in their simplicity: they operate under continual learning, do not require specialized architecture, and can distinguish novelty rapidly without requiring feedback. Implementing FMSs within a model of a visual cortical circuit that includes multiple inhibitory populations, we simultaneously reproduce three distinct novelty effects recently observed in experimental data from visual cortical circuits in mice: absolute, contextual, and omission novelty. Additionally, our model results in a set of diverse physiological responses across cell subpopulations, allowing us to analyze how their connectivity and synaptic dynamics influences their distinct behavior, leading to predictions that can be tested in experiment. Altogether, our findings demonstrate how experimentally-constrained cortical circuit structure can give rise to qualitatively distinct novelty responses using simple plasticity mechanisms. The flexibility of FMSs opens the door to computationally and theoretically investigating how distinct synapse modulations can lead to a variety of experience-dependent responses in a simple, understandable, and biologically plausible setup.

Differential encoding of temporal context and expectation under representational drift across hierarchically connected areas.

The classic view that neural populations in sensory cortices preferentially encode responses to incoming stimuli has been strongly challenged by recent experimental studies. Despite the fact that a large fraction of variance of visual responses in rodents can be attributed to behavioral state and movements, trial-history, and salience, the effects of contextual modulations and expectations on sensory-evoked responses in visual and association areas remain elusive. Here, we present a comprehensive experimental and theoretical study showing that hierarchically connected visual and association areas differentially encode the temporal context and expectation of naturalistic visual stimuli, consistent with the theory of hierarchical predictive coding. We measured neural responses to expected and unexpected sequences of natural scenes in the primary visual cortex (V1), the posterior medial higher order visual area (PM), and retrosplenial cortex (RSP) using 2-photon imaging in behaving mice collected through the Allen Institute Mindscope's OpenScope program. We found that information about image identity in neural population activity depended on the temporal context of transitions preceding each scene, and decreased along the hierarchy. Furthermore, our analyses revealed that the conjunctive encoding of temporal context and image identity was modulated by expectations of sequential events. In V1 and PM, we found enhanced and specific responses to unexpected oddball images, signaling stimulus-specific expectation violation. In contrast, in RSP the population response to oddball presentation recapitulated the missing expected image rather than the oddball image. These differential responses along the hierarchy are consistent with classic theories of hierarchical predictive coding whereby higher areas encode predictions and lower areas encode deviations from expectation. We further found evidence for drift in visual responses on the timescale of minutes. Although activity drift was present in all areas, population responses in V1 and PM, but not in RSP, maintained stable encoding of visual information and representational geometry. Instead we found that RSP drift was independent of stimulus information, suggesting a role in generating an internal model of the environment in the temporal domain. Overall, our results establish temporal context and expectation as substantial encoding dimensions in the visual cortex subject to fast representational drift and suggest that hierarchically connected areas instantiate a predictive coding mechanism.

The geometry of representational drift in natural and artificial neural networks.

Neurons in sensory areas encode/represent stimuli. Surprisingly, recent studies have suggested that, even during persistent performance, these representations are not stable and change over the course of days and weeks. We examine stimulus representations from fluorescence recordings across hundreds of neurons in the visual cortex using in vivo two-photon calcium imaging and we corroborate previous studies finding that such representations change as experimental trials are repeated across days. This phenomenon has been termed "representational drift". In this study we geometrically characterize the properties of representational drift in the primary visual cortex of mice in two open datasets from the Allen Institute and propose a potential mechanism behind such drift. We observe representational drift both for passively presented stimuli, as well as for stimuli which are behaviorally relevant. Across experiments, the drift differs from in-session variance and most often occurs along directions that have the most in-class variance, leading to a significant turnover in the neurons used for a given representation. Interestingly, despite this significant change due to drift, linear classifiers trained to distinguish neuronal representations show little to no degradation in performance across days. The features we observe in the neural data are similar to properties of artificial neural networks where representations are updated by continual learning in the presence of dropout, i.e. a random masking of nodes/weights, but not other types of noise. Therefore, we conclude that a potential reason for the representational drift in biological networks is driven by an underlying dropout-like noise while continuously learning and that such a mechanism may be computational advantageous for the brain in the same way it is for artificial neural networks, e.g. preventing overfitting.

Adaptation supports short-term memory in a visual change detection task.

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.

Survey of spiking in the mouse visual system reveals functional hierarchy.

The anatomy of the mammalian visual system, from the retina to the neocortex, is organized hierarchically1. However, direct observation of cellular-level functional interactions across this hierarchy is lacking due to the challenge of simultaneously recording activity across numerous regions. Here we describe a large, open dataset-part of the Allen Brain Observatory2-that surveys spiking from tens of thousands of units in six cortical and two thalamic regions in the brains of mice responding to a battery of visual stimuli. Using cross-correlation analysis, we reveal that the organization of inter-area functional connectivity during visual stimulation mirrors the anatomical hierarchy from the Allen Mouse Brain Connectivity Atlas3. We find that four classical hierarchical measures-response latency, receptive-field size, phase-locking to drifting gratings and response decay timescale-are all correlated with the hierarchy. Moreover, recordings obtained during a visual task reveal that the correlation between neural activity and behavioural choice also increases along the hierarchy. Our study provides a foundation for understanding coding and signal propagation across hierarchically organized cortical and thalamic visual areas.

Characterization of Learning, Motivation, and Visual Perception in Five Transgenic Mouse Lines Expressing GCaMP in Distinct Cell Populations.

To study the mechanisms of perception and cognition, neural measurements must be made during behavior. A goal of the Allen Brain Observatory is to map the activity of distinct cortical cell classes underlying visual and behavioral processing. Here we describe standardized methodology for training head-fixed mice on a visual change detection task, and we use our paradigm to characterize learning and behavior of five GCaMP6-expressing transgenic lines. We used automated training procedures to facilitate comparisons across mice. Training times varied, but most transgenic mice learned the behavioral task. Motivation levels also varied across mice. To compare mice in similar motivational states we subdivided sessions into over-, under-, and optimally motivated periods. When motivated, the pattern of perceptual decisions were highly correlated across transgenic lines, although overall performance (d-prime) was lower in one line labeling somatostatin inhibitory cells. These results provide important context for using these mice to map neural activity underlying perception and behavior.

Experience shapes activity dynamics and stimulus coding of VIP inhibitory cells.

Cortical circuits can flexibly change with experience and learning, but the effects on specific cell types, including distinct inhibitory types, are not well understood. Here we investigated how excitatory and VIP inhibitory cells in layer 2/3 of mouse visual cortex were impacted by visual experience in the context of a behavioral task. Mice learned a visual change detection task with a set of eight natural scene images. Subsequently, during 2-photon imaging experiments, mice performed the task with these familiar images and three sets of novel images. Strikingly, the temporal dynamics of VIP activity differed markedly between novel and familiar images: VIP cells were stimulus-driven by novel images but were suppressed by familiar stimuli and showed ramping activity when expected stimuli were omitted from a temporally predictable sequence. This prominent change in VIP activity suggests that these cells may adopt different modes of processing under novel versus familiar conditions.

A large-scale standardized physiological survey reveals functional organization of the mouse visual cortex.

To understand how the brain processes sensory information to guide behavior, we must know how stimulus representations are transformed throughout the visual cortex. Here we report an open, large-scale physiological survey of activity in the awake mouse visual cortex: the Allen Brain Observatory Visual Coding dataset. This publicly available dataset includes the cortical activity of nearly 60,000 neurons from six visual areas, four layers, and 12 transgenic mouse lines in a total of 243 adult mice, in response to a systematic set of visual stimuli. We classify neurons on the basis of joint reliabilities to multiple stimuli and validate this functional classification with models of visual responses. While most classes are characterized by responses to specific subsets of the stimuli, the largest class is not reliably responsive to any of the stimuli and becomes progressively larger in higher visual areas. These classes reveal a functional organization wherein putative dorsal areas show specialization for visual motion signals.

Biological variation in the sizes, shapes and locations of visual cortical areas in the mouse.

Visual cortex is organized into discrete sub-regions or areas that are arranged into a hierarchy and serves different functions in the processing of visual information. In retinotopic maps of mouse cortex, there appear to be substantial mouse-to-mouse differences in visual area location, size and shape. Here we quantify the biological variation in the size, shape and locations of 11 visual areas in the mouse, after separating biological variation and measurement noise. We find that there is biological variation in the locations and sizes of visual areas.

Higher-Order Thalamic Circuits Channel Parallel Streams of Visual Information in Mice.

Higher-order thalamic nuclei, such as the visual pulvinar, play essential roles in cortical function by connecting functionally related cortical and subcortical brain regions. A coherent framework describing pulvinar function remains elusive because of its anatomical complexity and involvement in diverse cognitive processes. We combined large-scale anatomical circuit mapping with high-density electrophysiological recordings to dissect a homolog of the pulvinar in mice, the lateral posterior thalamic nucleus (LP). We define three broad LP subregions based on correspondence between connectivity and functional properties. These subregions form corticothalamic loops biased toward ventral or dorsal stream cortical areas and contain separate representations of visual space. Silencing the visual cortex or superior colliculus revealed that they drive visual tuning properties in separate LP subregions. Thus, by specifying the driving input sources, functional properties, and downstream targets of LP circuits, our data provide a roadmap for understanding the mechanisms of higher-order thalamic function in vision.

Aberrant Cortical Activity in Multiple GCaMP6-Expressing Transgenic Mouse Lines.

Transgenic mouse lines are invaluable tools for neuroscience but, as with any technique, care must be taken to ensure that the tool itself does not unduly affect the system under study. Here we report aberrant electrical activity, similar to interictal spikes, and accompanying fluorescence events in some genotypes of transgenic mice expressing GCaMP6 genetically encoded calcium sensors. These epileptiform events have been observed particularly, but not exclusively, in mice with Emx1-Cre and Ai93 transgenes, of either sex, across multiple laboratories. The events occur at >0.1 Hz, are very large in amplitude (>1.0 mV local field potentials, >10% df/f widefield imaging signals), and typically cover large regions of cortex. Many properties of neuronal responses and behavior seem normal despite these events, although rare subjects exhibit overt generalized seizures. The underlying mechanisms of this phenomenon remain unclear, but we speculate about possible causes on the basis of diverse observations. We encourage researchers to be aware of these activity patterns while interpreting neuronal recordings from affected mouse lines and when considering which lines to study.

An extended retinotopic map of mouse cortex.

Visual perception and behavior are mediated by cortical areas that have been distinguished using architectonic and retinotopic criteria. We employed fluorescence imaging and GCaMP6 reporter mice to generate retinotopic maps, revealing additional regions of retinotopic organization that extend into barrel and retrosplenial cortices. Aligning retinotopic maps to architectonic borders, we found a mismatch in border location, indicating that architectonic borders are not aligned with the retinotopic transition at the vertical meridian. We also assessed the representation of visual space within each region, finding that four visual areas bordering V1 (LM, P, PM and RL) display complementary representations, with overlap primarily at the central hemifield. Our results extend our understanding of the organization of mouse cortex to include up to 16 distinct retinotopically organized regions.

Automated identification of mouse visual areas with intrinsic signal imaging.

Intrinsic signal optical imaging (ISI) is a rapid and noninvasive method for observing brain activity in vivo over a large area of the cortex. Here we describe our protocol for mapping retinotopy to identify mouse visual cortical areas using ISI. First, surgery is performed to attach a head frame to the mouse skull (∼1 h). The next day, intrinsic activity across the visual cortex is recorded during the presentation of a full-field drifting bar in the horizontal and vertical directions (∼2 h). Horizontal and vertical retinotopic maps are generated by analyzing the response of each pixel during the period of the stimulus. Last, an algorithm uses these retinotopic maps to compute the visual field sign and coverage, and automatically construct visual borders without human input. Compared with conventional retinotopic mapping with episodic presentation of adjacent stimuli, a continuous, periodic stimulus is more resistant to biological artifacts. Furthermore, unlike manual hand-drawn approaches, we present a method for automatically segmenting visual areas, even in the small mouse cortex. This relatively simple procedure and accompanying open-source code can be implemented with minimal surgical and computational experience, and is useful to any laboratory wishing to target visual cortical areas in this increasingly valuable model system.

Topography and areal organization of mouse visual cortex.

To guide future experiments aimed at understanding the mouse visual system, it is essential that we have a solid handle on the global topography of visual cortical areas. Ideally, the method used to measure cortical topography is objective, robust, and simple enough to guide subsequent targeting of visual areas in each subject. We developed an automated method that uses retinotopic maps of mouse visual cortex obtained with intrinsic signal imaging (Schuett et al., 2002; Kalatsky and Stryker, 2003; Marshel et al., 2011) and applies an algorithm to automatically identify cortical regions that satisfy a set of quantifiable criteria for what constitutes a visual area. This approach facilitated detailed parcellation of mouse visual cortex, delineating nine known areas (primary visual cortex, lateromedial area, anterolateral area, rostrolateral area, anteromedial area, posteromedial area, laterointermediate area, posterior area, and postrhinal area), and revealing two additional areas that have not been previously described as visuotopically mapped in mice (laterolateral anterior area and medial area). Using the topographic maps and defined area boundaries from each animal, we characterized several features of map organization, including variability in area position, area size, visual field coverage, and cortical magnification. We demonstrate that higher areas in mice often have representations that are incomplete or biased toward particular regions of visual space, suggestive of specializations for processing specific types of information about the environment. This work provides a comprehensive description of mouse visuotopic organization and describes essential tools for accurate functional localization of visual areas.

Functional specialization of seven mouse visual cortical areas.

To establish the mouse as a genetically tractable model for high-order visual processing, we characterized fine-scale retinotopic organization of visual cortex and determined functional specialization of layer 2/3 neuronal populations in seven retinotopically identified areas. Each area contains a distinct visuotopic representation and encodes a unique combination of spatiotemporal features. Areas LM, AL, RL, and AM prefer up to three times faster temporal frequencies and significantly lower spatial frequencies than V1, while V1 and PM prefer high spatial and low temporal frequencies. LI prefers both high spatial and temporal frequencies. All extrastriate areas except LI increase orientation selectivity compared to V1, and three areas are significantly more direction selective (AL, RL, and AM). Specific combinations of spatiotemporal representations further distinguish areas. These results reveal that mouse higher visual areas are functionally distinct, and separate groups of areas may be specialized for motion-related versus pattern-related computations, perhaps forming pathways analogous to dorsal and ventral streams in other species.