Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos.

Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the SENSORIUM 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net.

The sloppy relationship between neural circuit structure and function.

Investigating and describing the relationships between the structure of a circuit and its function has a long tradition in neuroscience. Since neural circuits acquire their structure through sophisticated developmental programmes, and memories and experiences are maintained through synaptic modification, it is to be expected that structure is closely linked to function. Recent findings challenge this hypothesis from three different angles: function does not strongly constrain circuit parameters, many parameters in neural circuits are irrelevant and contribute little to function, and circuit parameters are unstable and subject to constant random drift. At the same time, however, recent work also showed that dynamics in neural circuit activity that is related to function are robust over time and across individuals. Here this apparent contradiction is addressed by considering the properties of neural manifolds that restrict circuit activity to functionally relevant subspaces, and it will be suggested that degenerate, anisotropic and unstable parameter spaces are closely related to the structure and implementation of functionally relevant neural manifolds.

Movement-specific signaling is differentially distributed across motor cortex layer 5 projection neuron classes.

Motor cortex generates descending output necessary for executing a wide range of limb movements. Although movement-related activity has been described throughout motor cortex, the spatiotemporal organization of movement-specific signaling in deep layers remains largely unknown. Here we record layer 5B population dynamics in the caudal forelimb area of motor cortex while mice perform a forelimb push/pull task and find that most neurons show movement-invariant responses, with a minority displaying movement specificity. Using cell-type-specific imaging, we identify that invariant responses dominate pyramidal tract (PT) neuron activity, with a small subpopulation representing movement type, whereas a larger proportion of intratelencephalic (IT) neurons display movement-type-specific signaling. The proportion of IT neurons decoding movement-type peaks prior to movement initiation, whereas for PT neurons, this occurs during movement execution. Our data suggest that layer 5B population dynamics largely reflect movement-invariant signaling, with information related to movement-type being routed through relatively small, distributed subpopulations of projection neurons.

Receptive field estimation in large visual neuron assemblies using a super-resolution approach.

Computing the spike-triggered average (STA) is a simple method to estimate linear receptive fields (RFs) in sensory neurons. For random, uncorrelated stimuli, the STA provides an unbiased RF estimate, but in practice, white noise at high resolution is not an optimal stimulus choice as it usually evokes only weak responses. Therefore, for a visual stimulus, images of randomly modulated blocks of pixels are often used. This solution naturally limits the resolution at which an RF can be measured. Here, we present a simple super-resolution technique that can overcome these limitations. We define a novel stimulus type, the shifted white noise (SWN), by introducing random spatial shifts in the usual stimulus to increase the resolution of the measurements. In simulated data, we show that the average error using the SWN was 1.7 times smaller than when using the classical stimulus, with successful mapping of 2.3 times more neurons, covering a broader range of RF sizes. Moreover, successful RF mapping was achieved with brief recordings of light responses, lasting only about 1 min of activity, which is more than 10 times more efficient than the classical white noise stimulus. In recordings from mouse retinal ganglion cells with large scale multielectrode arrays, we successfully mapped 21 times more RFs than when using the traditional white noise stimuli. In summary, randomly shifting the usual white noise stimulus significantly improves RFs estimation, and requires only short recordings.NEW & NOTEWORTHY We present a novel approach to measure receptive fields in large and heterogeneous populations of sensory neurons recorded with large-scale, high-density multielectrode arrays. Our approach leverages super-resolution principles to improve the yield of the spike-triggered average method. By simply designing a new stimulus, we provide experimentalists with a new and fast technique to simultaneously detect more receptive fields at higher resolution in population of hundreds to thousands of neurons.

The information theory of developmental pruning: Optimizing global network architectures using local synaptic rules.

During development, biological neural networks produce more synapses and neurons than needed. Many of these synapses and neurons are later removed in a process known as neural pruning. Why networks should initially be over-populated, and the processes that determine which synapses and neurons are ultimately pruned, remains unclear. We study the mechanisms and significance of neural pruning in model neural networks. In a deep Boltzmann machine model of sensory encoding, we find that (1) synaptic pruning is necessary to learn efficient network architectures that retain computationally-relevant connections, (2) pruning by synaptic weight alone does not optimize network size and (3) pruning based on a locally-available measure of importance based on Fisher information allows the network to identify structurally important vs. unimportant connections and neurons. This locally-available measure of importance has a biological interpretation in terms of the correlations between presynaptic and postsynaptic neurons, and implies an efficient activity-driven pruning rule. Overall, we show how local activity-dependent synaptic pruning can solve the global problem of optimizing a network architecture. We relate these findings to biology as follows: (I) Synaptic over-production is necessary for activity-dependent connectivity optimization. (II) In networks that have more neurons than needed, cells compete for activity, and only the most important and selective neurons are retained. (III) Cells may also be pruned due to a loss of synapses on their axons. This occurs when the information they convey is not relevant to the target population.

Building population models for large-scale neural recordings: Opportunities and pitfalls.

Modern recording technologies now enable simultaneous recording from large numbers of neurons. This has driven the development of new statistical models for analyzing and interpreting neural population activity. Here, we provide a broad overview of recent developments in this area. We compare and contrast different approaches, highlight strengths and limitations, and discuss biological and mechanistic insights that these methods provide.

Optimal Encoding in Stochastic Latent-Variable Models.

In this work we explore encoding strategies learned by statistical models of sensory coding in noisy spiking networks. Early stages of sensory communication in neural systems can be viewed as encoding channels in the information-theoretic sense. However, neural populations face constraints not commonly considered in communications theory. Using restricted Boltzmann machines as a model of sensory encoding, we find that networks with sufficient capacity learn to balance precision and noise-robustness in order to adaptively communicate stimuli with varying information content. Mirroring variability suppression observed in sensory systems, informative stimuli are encoded with high precision, at the cost of more variable responses to frequent, hence less informative stimuli. Curiously, we also find that statistical criticality in the neural population code emerges at model sizes where the input statistics are well captured. These phenomena have well-defined thermodynamic interpretations, and we discuss their connection to prevailing theories of coding and statistical criticality in neural populations.

SpikeInterface, a unified framework for spike sorting.

Much development has been directed toward improving the performance and automation of spike sorting. This continuous development, while essential, has contributed to an over-saturation of new, incompatible tools that hinders rigorous benchmarking and complicates reproducible analysis. To address these limitations, we developed SpikeInterface, a Python framework designed to unify preexisting spike sorting technologies into a single codebase and to facilitate straightforward comparison and adoption of different approaches. With a few lines of code, researchers can reproducibly run, compare, and benchmark most modern spike sorting algorithms; pre-process, post-process, and visualize extracellular datasets; validate, curate, and export sorting outputs; and more. In this paper, we provide an overview of SpikeInterface and, with applications to real and simulated datasets, demonstrate how it can be utilized to reduce the burden of manual curation and to more comprehensively benchmark automated spike sorters.

Neural field models for latent state inference: Application to large-scale neuronal recordings.

Large-scale neural recording methods now allow us to observe large populations of identified single neurons simultaneously, opening a window into neural population dynamics in living organisms. However, distilling such large-scale recordings to build theories of emergent collective dynamics remains a fundamental statistical challenge. The neural field models of Wilson, Cowan, and colleagues remain the mainstay of mathematical population modeling owing to their interpretable, mechanistic parameters and amenability to mathematical analysis. Inspired by recent advances in biochemical modeling, we develop a method based on moment closure to interpret neural field models as latent state-space point-process models, making them amenable to statistical inference. With this approach we can infer the intrinsic states of neurons, such as active and refractory, solely from spiking activity in large populations. After validating this approach with synthetic data, we apply it to high-density recordings of spiking activity in the developing mouse retina. This confirms the essential role of a long lasting refractory state in shaping spatiotemporal properties of neonatal retinal waves. This conceptual and methodological advance opens up new theoretical connections between mathematical theory and point-process state-space models in neural data analysis.

Slow NMDA-Mediated Excitation Accelerates Offset-Response Latencies Generated via a Post-Inhibitory Rebound Mechanism.

In neural circuits, action potentials (spikes) are conventionally caused by excitatory inputs whereas inhibitory inputs reduce or modulate neuronal excitability. We previously showed that neurons in the superior paraolivary nucleus (SPN) require solely synaptic inhibition to generate their hallmark offset response, a burst of spikes at the end of a sound stimulus, via a post-inhibitory rebound mechanism. In addition SPN neurons receive excitatory inputs, but their functional significance is not yet known. Here we used mice of both sexes to demonstrate that in SPN neurons, the classical roles for excitation and inhibition are switched, with inhibitory inputs driving spike firing and excitatory inputs modulating this response. Hodgkin-Huxley modeling suggests that a slow, NMDA receptor (NMDAR)-mediated excitation would accelerate the offset response. We find corroborating evidence from in vitro and in vivo recordings that lack of excitation prolonged offset-response latencies and rendered them more variable to changing sound intensity levels. Our results reveal an unsuspected function for slow excitation in improving the timing of post-inhibitory rebound firing even when the firing itself does not depend on excitation. This shows the auditory system employs highly specialized mechanisms to encode timing-sensitive features of sound offsets which are crucial for sound-duration encoding and have profound biological importance for encoding the temporal structure of speech.

Scaling Spike Detection and Sorting for Next-Generation Electrophysiology.

Reliable spike detection and sorting, the process of assigning each detected spike to its originating neuron, are essential steps in the analysis of extracellular electrical recordings from neurons. The volume and complexity of the data from recently developed large-scale, high-density microelectrode arrays and probes, which allow recording from thousands of channels simultaneously, substantially complicate this task conceptually and computationally. This chapter provides a summary and discussion of recently developed methods to tackle these challenges and discusses the important aspect of algorithm validation, and assessment of detection and sorting quality.

Non-parametric Physiological Classification of Retinal Ganglion Cells in the Mouse Retina.

Retinal ganglion cells, the sole output neurons of the retina, exhibit surprising diversity. A recent study reported over 30 distinct types in the mouse retina, indicating that the processing of visual information is highly parallelised in the brain. The advent of high density multi-electrode arrays now enables recording from many hundreds to thousands of neurons from a single retina. Here we describe a method for the automatic classification of large-scale retinal recordings using a simple stimulus paradigm and a spike train distance measure as a clustering metric. We evaluate our approach using synthetic spike trains, and demonstrate that major known cell types are identified in high-density recording sessions from the mouse retina with around 1,000 retinal ganglion cells. A comparison across different retinas reveals substantial variability between preparations, suggesting pooling data across retinas should be approached with caution. As a parameter-free method, our approach is broadly applicable for cellular physiological classification in all sensory modalities.

Glucose and lactate as metabolic constraints on presynaptic transmission at an excitatory synapse.

KEY POINTS: Synapses have high energy demands which increase during intense activity. We show that presynaptic terminals can utilise extracellular glucose or lactate to generate energy to maintain synaptic transmission. Reducing energy substrates induces a metabolic stress: presynaptic ATP depletion impaired synaptic transmission through a reduction in the number of functional synaptic vesicle release sites and a slowing of vesicle pool replenishment, without a consistent change in release probability. Metabolic function is compromised in many pathological conditions (e.g. stroke, traumatic brain injury and neurodegeneration). Knowledge of how synaptic transmission is constrained by metabolic stress, especially during intense brain activity, will provide insights to improve cognition following pathological insults. ABSTRACT: The synapse has high energy demands, which increase during intense activity. Presynaptic ATP production depends on substrate availability and usage will increase during activity, which in turn could influence transmitter release and information transmission. We investigated transmitter release at the mouse calyx of Held synapse using glucose or lactate (10, 1 or 0 mm) as the extracellular substrates while inducing metabolic stress. High-frequency stimulation (HFS) and recovery paradigms evoked trains of EPSCs monitored under voltage-clamp. Whilst postsynaptic intracellular ATP was stabilised by diffusion from the patch pipette, depletion of glucose increased EPSC depression during HFS and impaired subsequent recovery. Computational modelling of these data demonstrated a reduction in the number of functional release sites and slowed vesicle pool replenishment during metabolic stress, with little change in release probability. Directly depleting presynaptic terminal ATP impaired transmitter release in an analogous manner to glucose depletion. In the absence of glucose, presynaptic terminal metabolism could utilise lactate from the aCSF and this was blocked by inhibition of monocarboxylate transporters (MCTs). MCT inhibitors significantly suppressed transmission in low glucose, implying that lactate is a presynaptic substrate. Additionally, block of glycogenolysis accelerated synaptic transmission failure in the absence of extracellular glucose, consistent with supplemental supply of lactate by local astrocytes. We conclude that both glucose and lactate support presynaptic metabolism and that limited availability, exacerbated by high-intensity firing, constrains presynaptic ATP, impeding transmission through a reduction in functional presynaptic release sites as vesicle recycling slows when ATP levels are low.

Pan-retinal characterisation of Light Responses from Ganglion Cells in the Developing Mouse Retina.

We have investigated the ontogeny of light-driven responses in mouse retinal ganglion cells (RGCs). Using a large-scale, high-density multielectrode array, we recorded from hundreds to thousands of RGCs simultaneously at pan-retinal level, including dorsal and ventral locations. Responses to different contrasts not only revealed a complex developmental profile for ON, OFF and ON-OFF responses, but also unveiled differences between dorsal and ventral RGC responses. At eye-opening, dorsal RGCs of all types were more responsive to light, perhaps indicating an environmental priority to nest viewing for pre-weaning pups. The developmental profile of ON and OFF responses exhibited antagonistic behaviour, with the strongest ON responses shortly after eye-opening, followed by an increase in the strength of OFF responses later on. Further, we found that with maturation receptive field (RF) center sizes decrease, spike-triggered averaged responses to white noise become stronger, and centers become more circular while maintaining differences between RGC types. We conclude that the maturation of retinal functionality is not spatially homogeneous, likely reflecting ecological requirements that favour earlier maturation of the dorsal retina.

Spike-Based Bayesian-Hebbian Learning of Temporal Sequences.

Many cognitive and motor functions are enabled by the temporal representation and processing of stimuli, but it remains an open issue how neocortical microcircuits can reliably encode and replay such sequences of information. To better understand this, a modular attractor memory network is proposed in which meta-stable sequential attractor transitions are learned through changes to synaptic weights and intrinsic excitabilities via the spike-based Bayesian Confidence Propagation Neural Network (BCPNN) learning rule. We find that the formation of distributed memories, embodied by increased periods of firing in pools of excitatory neurons, together with asymmetrical associations between these distinct network states, can be acquired through plasticity. The model's feasibility is demonstrated using simulations of adaptive exponential integrate-and-fire model neurons (AdEx). We show that the learning and speed of sequence replay depends on a confluence of biophysically relevant parameters including stimulus duration, level of background noise, ratio of synaptic currents, and strengths of short-term depression and adaptation. Moreover, sequence elements are shown to flexibly participate multiple times in the sequence, suggesting that spiking attractor networks of this type can support an efficient combinatorial code. The model provides a principled approach towards understanding how multiple interacting plasticity mechanisms can coordinate hetero-associative learning in unison.

Acoustic trauma slows AMPA receptor-mediated EPSCs in the auditory brainstem, reducing GluA4 subunit expression as a mechanism to rescue binaural function.

KEY POINTS: Lateral superior olive (LSO) principal neurons receive AMPA receptor (AMPAR) - and NMDA receptor (NMDAR)-mediated EPSCs and glycinergic IPSCs. Both EPSCs and IPSCs have slow kinetics in prehearing animals, which during developmental maturation accelerate to sub-millisecond decay time-constants. This correlates with a change in glutamate and glycine receptor subunit composition quantified via mRNA levels. The NMDAR-EPSCs accelerate over development to achieve decay time-constants of 2.5 ms. This is the fastest NMDAR-mediated EPSC reported. Acoustic trauma (AT, loud sounds) slow AMPAR-EPSC decay times, increasing GluA1 and decreasing GluA4 mRNA. Modelling of interaural intensity difference suggests that the increased EPSC duration after AT shifts interaural level difference to the right and compensates for hearing loss. Two months after AT the EPSC decay times recovered to control values. Synaptic transmission in the LSO matures by postnatal day 20, with EPSCs and IPSCs having fast kinetics. AT changes the AMPAR subunits expressed and slows the EPSC time-course at synapses in the central auditory system. ABSTRACT: Damaging levels of sound (acoustic trauma, AT) diminish peripheral synapses, but what is the impact on the central auditory pathway? Developmental maturation of synaptic function and hearing were characterized in the mouse lateral superior olive (LSO) from postnatal day 7 (P7) to P96 using voltage-clamp and auditory brainstem responses. IPSCs and EPSCs show rapid acceleration during development, so that decay kinetics converge to similar sub-millisecond time-constants (τ, 0.87 ± 0.11 and 0.77 ± 0.08 ms, respectively) in adult mice. This correlated with LSO mRNA levels for glycinergic and glutamatergic ionotropic receptor subunits, confirming a switch from Glyα2 to Glyα1 for IPSCs and increased expression of GluA3 and GluA4 subunits for EPSCs. The NMDA receptor (NMDAR)-EPSC decay τ accelerated from >40 ms in prehearing animals to 2.6 ± 0.4 ms in adults, as GluN2C expression increased. In vivo induction of AT at around P20 disrupted IPSC and EPSC integration in the LSO, so that 1 week later the AMPA receptor (AMPAR)-EPSC decay was slowed and mRNA for GluA1 increased while GluA4 decreased. In contrast, GlyR IPSC and NMDAR-EPSC decay times were unchanged. Computational modelling confirmed that matched IPSC and EPSC kinetics are required to generate mature interaural level difference functions, and that longer-lasting EPSCs compensate to maintain binaural function with raised auditory thresholds after AT. We conclude that LSO excitatory and inhibitory synaptic drive matures to identical time-courses, that AT changes synaptic AMPARs by expression of subunits with slow kinetics (which recover over 2 months) and that loud sounds reversibly modify excitatory synapses in the brain, changing synaptic function for several weeks after exposure.

A Diffusive Homeostatic Signal Maintains Neural Heterogeneity and Responsiveness in Cortical Networks.

Gaseous neurotransmitters such as nitric oxide (NO) provide a unique and often overlooked mechanism for neurons to communicate through diffusion within a network, independent of synaptic connectivity. NO provides homeostatic control of intrinsic excitability. Here we conduct a theoretical investigation of the distinguishing roles of NO-mediated diffusive homeostasis in comparison with canonical non-diffusive homeostasis in cortical networks. We find that both forms of homeostasis provide a robust mechanism for maintaining stable activity following perturbations. However, the resulting networks differ, with diffusive homeostasis maintaining substantial heterogeneity in activity levels of individual neurons, a feature disrupted in networks with non-diffusive homeostasis. This results in networks capable of representing input heterogeneity, and linearly responding over a broader range of inputs than those undergoing non-diffusive homeostasis. We further show that these properties are preserved when homeostatic and Hebbian plasticity are combined. These results suggest a mechanism for dynamically maintaining neural heterogeneity, and expose computational advantages of non-local homeostatic processes.

Sloppiness in spontaneously active neuronal networks.

Various plasticity mechanisms, including experience-dependent, spontaneous, as well as homeostatic ones, continuously remodel neural circuits. Yet, despite fluctuations in the properties of single neurons and synapses, the behavior and function of neuronal assemblies are generally found to be very stable over time. This raises the important question of how plasticity is coordinated across the network. To address this, we investigated the stability of network activity in cultured rat hippocampal neurons recorded with high-density multielectrode arrays over several days. We used parametric models to characterize multineuron activity patterns and analyzed their sensitivity to changes. We found that the models exhibited sloppiness, a property where the model behavior is insensitive to changes in many parameter combinations, but very sensitive to a few. The activity of neurons with sloppy parameters showed faster and larger fluctuations than the activity of a small subset of neurons associated with sensitive parameters. Furthermore, parameter sensitivity was highly correlated with firing rates. Finally, we tested our observations from cell cultures on an in vivo recording from monkey visual cortex and we confirm that spontaneous cortical activity also shows hallmarks of sloppy behavior and firing rate dependence. Our findings suggest that a small subnetwork of highly active and stable neurons supports group stability, and that this endows neuronal networks with the flexibility to continuously remodel without compromising stability and function.

Spike Detection for Large Neural Populations Using High Density Multielectrode Arrays.

An emerging generation of high-density microelectrode arrays (MEAs) is now capable of recording spiking activity simultaneously from thousands of neurons with closely spaced electrodes. Reliable spike detection and analysis in such recordings is challenging due to the large amount of raw data and the dense sampling of spikes with closely spaced electrodes. Here, we present a highly efficient, online capable spike detection algorithm, and an offline method with improved detection rates, which enables estimation of spatial event locations at a resolution higher than that provided by the array by combining information from multiple electrodes. Data acquired with a 4096 channel MEA from neuronal cultures and the neonatal retina, as well as synthetic data, was used to test and validate these methods. We demonstrate that these algorithms outperform conventional methods due to a better noise estimate and an improved signal-to-noise ratio (SNR) through combining information from multiple electrodes. Finally, we present a new approach for analyzing population activity based on the characterization of the spatio-temporal event profile, which does not require the isolation of single units. Overall, we show how the improved spatial resolution provided by high density, large scale MEAs can be reliably exploited to characterize activity from large neural populations and brain circuits.

Synaptic and nonsynaptic plasticity approximating probabilistic inference.

Learning and memory operations in neural circuits are believed to involve molecular cascades of synaptic and nonsynaptic changes that lead to a diverse repertoire of dynamical phenomena at higher levels of processing. Hebbian and homeostatic plasticity, neuromodulation, and intrinsic excitability all conspire to form and maintain memories. But it is still unclear how these seemingly redundant mechanisms could jointly orchestrate learning in a more unified system. To this end, a Hebbian learning rule for spiking neurons inspired by Bayesian statistics is proposed. In this model, synaptic weights and intrinsic currents are adapted on-line upon arrival of single spikes, which initiate a cascade of temporally interacting memory traces that locally estimate probabilities associated with relative neuronal activation levels. Trace dynamics enable synaptic learning to readily demonstrate a spike-timing dependence, stably return to a set-point over long time scales, and remain competitive despite this stability. Beyond unsupervised learning, linking the traces with an external plasticity-modulating signal enables spike-based reinforcement learning. At the postsynaptic neuron, the traces are represented by an activity-dependent ion channel that is shown to regulate the input received by a postsynaptic cell and generate intrinsic graded persistent firing levels. We show how spike-based Hebbian-Bayesian learning can be performed in a simulated inference task using integrate-and-fire (IAF) neurons that are Poisson-firing and background-driven, similar to the preferred regime of cortical neurons. Our results support the view that neurons can represent information in the form of probability distributions, and that probabilistic inference could be a functional by-product of coupled synaptic and nonsynaptic mechanisms operating over several timescales. The model provides a biophysical realization of Bayesian computation by reconciling several observed neural phenomena whose functional effects are only partially understood in concert.