High-precision coding in visual cortex.

Individual neurons in visual cortex provide the brain with unreliable estimates of visual features. It is not known whether the single-neuron variability is correlated across large neural populations, thus impairing the global encoding of stimuli. We recorded simultaneously from up to 50,000 neurons in mouse primary visual cortex (V1) and in higher order visual areas and measured stimulus discrimination thresholds of 0.35° and 0.37°, respectively, in an orientation decoding task. These neural thresholds were almost 100 times smaller than the behavioral discrimination thresholds reported in mice. This discrepancy could not be explained by stimulus properties or arousal states. Furthermore, behavioral variability during a sensory discrimination task could not be explained by neural variability in V1. Instead, behavior-related neural activity arose dynamically across a network of non-sensory brain areas. These results imply that perceptual discrimination in mice is limited by downstream decoders, not by neural noise in sensory representations.

Vagal sensory neurons mediate the Bezold-Jarisch reflex and induce syncope.

Visceral sensory pathways mediate homeostatic reflexes, the dysfunction of which leads to many neurological disorders1. The Bezold-Jarisch reflex (BJR), first described2,3 in 1867, is a cardioinhibitory reflex that is speculated to be mediated by vagal sensory neurons (VSNs) that also triggers syncope. However, the molecular identity, anatomical organization, physiological characteristics and behavioural influence of cardiac VSNs remain mostly unknown. Here we leveraged single-cell RNA-sequencing data and HYBRiD tissue clearing4 to show that VSNs that express neuropeptide Y receptor Y2 (NPY2R) predominately connect the heart ventricular wall to the area postrema. Optogenetic activation of NPY2R VSNs elicits the classic triad of BJR responses-hypotension, bradycardia and suppressed respiration-and causes an animal to faint. Photostimulation during high-resolution echocardiography and laser Doppler flowmetry with behavioural observation revealed a range of phenotypes reflected in clinical syncope, including reduced cardiac output, cerebral hypoperfusion, pupil dilation and eye-roll. Large-scale Neuropixels brain recordings and machine-learning-based modelling showed that this manipulation causes the suppression of activity across a large distributed neuronal population that is not explained by changes in spontaneous behavioural movements. Additionally, bidirectional manipulation of the periventricular zone had a push-pull effect, with inhibition leading to longer syncope periods and activation inducing arousal. Finally, ablating NPY2R VSNs specifically abolished the BJR. Combined, these results demonstrate a genetically defined cardiac reflex that recapitulates characteristics of human syncope at physiological, behavioural and neural network levels.

Spike sorting with Kilosort4.

Spike sorting is the computational process of extracting the firing times of single neurons from recordings of local electrical fields. This is an important but hard problem in neuroscience, made complicated by the nonstationarity of the recordings and the dense overlap in electrical fields between nearby neurons. To address the spike-sorting problem, we have been openly developing the Kilosort framework. Here we describe the various algorithmic steps introduced in different versions of Kilosort. We also report the development of Kilosort4, a version with substantially improved performance due to clustering algorithms inspired by graph-based approaches. To test the performance of Kilosort, we developed a realistic simulation framework that uses densely sampled electrical fields from real experiments to generate nonstationary spike waveforms and realistic noise. We found that nearly all versions of Kilosort outperformed other algorithms on a variety of simulated conditions and that Kilosort4 performed best in all cases, correctly identifying even neurons with low amplitudes and small spatial extents in high drift conditions.

Cellpose 2.0: how to train your own model.

Pretrained neural network models for biological segmentation can provide good out-of-the-box results for many image types. However, such models do not allow users to adapt the segmentation style to their specific needs and can perform suboptimally for test images that are very different from the training images. Here we introduce Cellpose 2.0, a new package that includes an ensemble of diverse pretrained models as well as a human-in-the-loop pipeline for rapid prototyping of new custom models. We show that models pretrained on the Cellpose dataset can be fine-tuned with only 500-1,000 user-annotated regions of interest (ROI) to perform nearly as well as models trained on entire datasets with up to 200,000 ROI. A human-in-the-loop approach further reduced the required user annotation to 100-200 ROI, while maintaining high-quality segmentations. We provide software tools such as an annotation graphical user interface, a model zoo and a human-in-the-loop pipeline to facilitate the adoption of Cellpose 2.0.

Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation.

Advances in microscopy hold great promise for allowing quantitative and precise measurement of morphological and molecular phenomena at the single-cell level in bacteria; however, the potential of this approach is ultimately limited by the availability of methods to faithfully segment cells independent of their morphological or optical characteristics. Here, we present Omnipose, a deep neural network image-segmentation algorithm. Unique network outputs such as the gradient of the distance field allow Omnipose to accurately segment cells on which current algorithms, including its predecessor, Cellpose, produce errors. We show that Omnipose achieves unprecedented segmentation performance on mixed bacterial cultures, antibiotic-treated cells and cells of elongated or branched morphology. Furthermore, the benefits of Omnipose extend to non-bacterial subjects, varied imaging modalities and three-dimensional objects. Finally, we demonstrate the utility of Omnipose in the characterization of extreme morphological phenotypes that arise during interbacterial antagonism. Our results distinguish Omnipose as a powerful tool for characterizing diverse and arbitrarily shaped cell types from imaging data.

High-dimensional geometry of population responses in visual cortex.

A neuronal population encodes information most efficiently when its stimulus responses are high-dimensional and uncorrelated, and most robustly when they are lower-dimensional and correlated. Here we analysed the dimensionality of the encoding of natural images by large populations of neurons in the visual cortex of awake mice. The evoked population activity was high-dimensional, and correlations obeyed an unexpected power law: the nth principal component variance scaled as 1/n. This scaling was not inherited from the power law spectrum of natural images, because it persisted after stimulus whitening. We proved mathematically that if the variance spectrum was to decay more slowly then the population code could not be smooth, allowing small changes in input to dominate population activity. The theory also predicts larger power-law exponents for lower-dimensional stimulus ensembles, which we validated experimentally. These results suggest that coding smoothness may represent a fundamental constraint that determines correlations in neural population codes.

Cellpose: a generalist algorithm for cellular segmentation.

Many biological applications require the segmentation of cell bodies, membranes and nuclei from microscopy images. Deep learning has enabled great progress on this problem, but current methods are specialized for images that have large training datasets. Here we introduce a generalist, deep learning-based segmentation method called Cellpose, which can precisely segment cells from a wide range of image types and does not require model retraining or parameter adjustments. Cellpose was trained on a new dataset of highly varied images of cells, containing over 70,000 segmented objects. We also demonstrate a three-dimensional (3D) extension of Cellpose that reuses the two-dimensional (2D) model and does not require 3D-labeled data. To support community contributions to the training data, we developed software for manual labeling and for curation of the automated results. Periodically retraining the model on the community-contributed data will ensure that Cellpose improves constantly.

The Importance of Accounting for Movement When Relating Neuronal Activity to Sensory and Cognitive Processes.

A surprising finding of recent studies in mouse is the dominance of widespread movement-related activity throughout the brain, including in early sensory areas. In awake subjects, failing to account for movement risks misattributing movement-related activity to other (e.g., sensory or cognitive) processes. In this article, we (1) review task designs for separating task-related and movement-related activity, (2) review three "case studies" in which not considering movement would have resulted in critically different interpretations of neuronal function, and (3) discuss functional couplings that may prevent us from ever fully isolating sensory, motor, and cognitive-related activity. Our main thesis is that neural signals related to movement are ubiquitous, and therefore ought to be considered first and foremost when attempting to correlate neuronal activity with task-related processes.

Robustness of Spike Deconvolution for Neuronal Calcium Imaging.

Calcium imaging is a powerful method to record the activity of neural populations in many species, but inferring spike times from calcium signals is a challenging problem. We compared multiple approaches using multiple datasets with ground truth electrophysiology and found that simple non-negative deconvolution (NND) outperformed all other algorithms on out-of-sample test data. We introduce a novel benchmark applicable to recordings without electrophysiological ground truth, based on the correlation of responses to two stimulus repeats, and used this to show that unconstrained NND also outperformed the other algorithms when run on "zoomed out" datasets of ∼10,000 cell recordings from the visual cortex of mice of either sex. Finally, we show that NND-based methods match the performance of a supervised method based on convolutional neural networks while avoiding some of the biases of such methods, and at much faster running times. We therefore recommend that spikes be inferred from calcium traces using simple NND because of its simplicity, efficiency, and accuracy.SIGNIFICANCE STATEMENT The experimental method that currently allows for recordings of the largest numbers of cells simultaneously is two-photon calcium imaging. However, use of this powerful method requires that neuronal firing times be inferred correctly from the large resulting datasets. Previous studies have claimed that complex supervised learning algorithms outperform simple deconvolution methods at this task. Unfortunately, these studies suffered from several problems and biases. When we repeated the analysis, using the same data and correcting these problems, we found that simpler spike inference methods perform better. Even more importantly, we found that supervised learning methods can introduce artifactual structure into spike trains, which can in turn lead to erroneous scientific conclusions. Of the algorithms we evaluated, we found that an extremely simple method performed best in all circumstances tested, was much faster to run, and was insensitive to parameter choices, making incorrect scientific conclusions much less likely.

Facemap: a framework for modeling neural activity based on orofacial tracking.

Recent studies in mice have shown that orofacial behaviors drive a large fraction of neural activity across the brain. To understand the nature and function of these signals, we need better computational models to characterize the behaviors and relate them to neural activity. Here we developed Facemap, a framework consisting of a keypoint tracker and a deep neural network encoder for predicting neural activity. Our algorithm for tracking mouse orofacial behaviors was more accurate than existing pose estimation tools, while the processing speed was several times faster, making it a powerful tool for real-time experimental interventions. The Facemap tracker was easy to adapt to data from new labs, requiring as few as 10 annotated frames for near-optimal performance. We used the keypoints as inputs to a deep neural network which predicts the activity of ~50,000 simultaneously-recorded neurons and, in visual cortex, we doubled the amount of explained variance compared to previous methods. Using this model, we found that the neuronal activity clusters that were well predicted from behavior were more spatially spread out across cortex. We also found that the deep behavioral features from the model had stereotypical, sequential dynamics that were not reversible in time. In summary, Facemap provides a stepping stone toward understanding the function of the brain-wide neural signals and their relation to behavior.

Distributed fMRI dynamics predict distinct EEG rhythms across sleep and wakefulness.

The brain exhibits rich oscillatory dynamics that vary across tasks and states, such as the EEG oscillations that define sleep. These oscillations play critical roles in cognition and arousal, but the brainwide mechanisms underlying them are not yet described. Using simultaneous EEG and fast fMRI in subjects drifting between sleep and wakefulness, we developed a machine learning approach to investigate which brainwide fMRI dynamics predict alpha (8-12 Hz) and delta (1-4 Hz) rhythms. We predicted moment-by-moment EEG power from fMRI activity in held-out subjects, and found that information about alpha power was represented by a remarkably small set of regions, segregated in two distinct networks linked to arousal and visual systems. Conversely, delta rhythms were diffusely represented on a large spatial scale across the cortex. These results identify distributed networks that predict delta and alpha rhythms, and establish a computational framework for investigating fMRI brainwide dynamics underlying EEG oscillations.

Spatial Single-cell Analysis Decodes Cortical Layer and Area Specification.

The human cerebral cortex, pivotal for advanced cognitive functions, is composed of six distinct layers and dozens of functionally specialized areas1,2. The layers and areas are distinguished both molecularly, by diverse neuronal and glial cell subtypes, and structurally, through intricate spatial organization3,4. While single-cell transcriptomics studies have advanced molecular characterization of human cortical development, a critical gap exists due to the loss of spatial context during cell dissociation5,6,7,8. Here, we utilized multiplexed error-robust fluorescence in situ hybridization (MERFISH)9, augmented with deep-learning-based cell segmentation, to examine the molecular, cellular, and cytoarchitectural development of human fetal cortex with spatially resolved single-cell resolution. Our extensive spatial atlas, encompassing 16 million single cells, spans eight cortical areas across four time points in the second and third trimesters. We uncovered an early establishment of the six-layer structure, identifiable in the laminar distribution of excitatory neuronal subtypes by mid-gestation, long before the emergence of cytoarchitectural layers. Notably, while anterior-posterior gradients of neuronal subtypes were generally observed in most cortical areas, a striking exception was the sharp molecular border between primary (V1) and secondary visual cortices (V2) at gestational week 20. Here we discovered an abrupt binary shift in neuronal subtype specification at the earliest stages, challenging the notion that continuous morphogen gradients dictate mid-gestation cortical arealization6,10. Moreover, integrating single-nuclei RNA-sequencing and in situ whole transcriptomics revealed an early upregulation of synaptogenesis in V1-specific Layer 4 neurons, suggesting a role of synaptogenesis in this discrete border formation. Collectively, our findings underscore the crucial role of spatial relationships in determining the molecular specification of cortical layers and areas. This work not only provides a valuable resource for the field, but also establishes a spatially resolved single-cell analysis paradigm that paves the way for a comprehensive developmental atlas of the human brain.

A modular chemigenetic calcium indicator enables in vivo functional imaging with near-infrared light.

Genetically encoded fluorescent calcium indicators have revolutionized neuroscience and other biological fields by allowing cellular-resolution recording of physiology during behavior. However, we currently lack bright, genetically targetable indicators in the near infrared that can be used in animals. Here, we describe WHaloCaMP, a modular chemigenetic calcium indicator built from bright dye-ligands and protein sensor domains that can be genetically targeted to specific cell populations. Fluorescence change in WHaloCaMP results from reversible quenching of the bound dye via a strategically placed tryptophan. WHaloCaMP is compatible with rhodamine dye-ligands that fluoresce from green to near-infrared, including several dye-ligands that efficiently label the central nervous system in animals. When bound to a near-infrared dye-ligand, WHaloCaMP1a is more than twice as bright as jGCaMP8s, and shows a 7× increase in fluorescence intensity and a 2.1 ns increase in fluorescence lifetime upon calcium binding. We use WHaloCaMP1a with near-infrared fluorescence emission to image Ca2+ responses in flies and mice, to perform three-color multiplexed functional imaging of hundreds of neurons and astrocytes in zebrafish larvae, and to quantitate calcium concentration using fluorescence lifetime imaging microscopy (FLIM).

Not so spontaneous: Multi-dimensional representations of behaviors and context in sensory areas.

Sensory areas are spontaneously active in the absence of sensory stimuli. This spontaneous activity has long been studied; however, its functional role remains largely unknown. Recent advances in technology, allowing large-scale neural recordings in the awake and behaving animal, have transformed our understanding of spontaneous activity. Studies using these recordings have discovered high-dimensional spontaneous activity patterns, correlation between spontaneous activity and behavior, and dissimilarity between spontaneous and sensory-driven activity patterns. These findings are supported by evidence from developing animals, where a transition toward these characteristics is observed as the circuit matures, as well as by evidence from mature animals across species. These newly revealed characteristics call for the formulation of a new role for spontaneous activity in neural sensory computation.

Computational processing of neural recordings from calcium imaging data.

Electrophysiology has long been the workhorse of neuroscience, allowing scientists to record with millisecond precision the action potentials generated by neurons in vivo. Recently, calcium imaging of fluorescent indicators has emerged as a powerful alternative. This technique has its own strengths and weaknesses and unique data processing problems and interpretation confounds. Here we review the computational methods that convert raw calcium movies to estimates of single neuron spike times with minimal human supervision. By computationally addressing the weaknesses of calcium imaging, these methods hold the promise of significantly improving data quality. We also introduce a new metric to evaluate the output of these processing pipelines, which is based on the cluster isolation distance routinely used in electrophysiology.

Spontaneous behaviors drive multidimensional, brainwide activity.

Neuronal populations in sensory cortex produce variable responses to sensory stimuli and exhibit intricate spontaneous activity even without external sensory input. Cortical variability and spontaneous activity have been variously proposed to represent random noise, recall of prior experience, or encoding of ongoing behavioral and cognitive variables. Recording more than 10,000 neurons in mouse visual cortex, we observed that spontaneous activity reliably encoded a high-dimensional latent state, which was partially related to the mouse's ongoing behavior and was represented not just in visual cortex but also across the forebrain. Sensory inputs did not interrupt this ongoing signal but added onto it a representation of external stimuli in orthogonal dimensions. Thus, visual cortical population activity, despite its apparently noisy structure, reliably encodes an orthogonal fusion of sensory and multidimensional behavioral information.

Inhibitory control of correlated intrinsic variability in cortical networks.

Cortical networks exhibit intrinsic dynamics that drive coordinated, large-scale fluctuations across neuronal populations and create noise correlations that impact sensory coding. To investigate the network-level mechanisms that underlie these dynamics, we developed novel computational techniques to fit a deterministic spiking network model directly to multi-neuron recordings from different rodent species, sensory modalities, and behavioral states. The model generated correlated variability without external noise and accurately reproduced the diverse activity patterns in our recordings. Analysis of the model parameters suggested that differences in noise correlations across recordings were due primarily to differences in the strength of feedback inhibition. Further analysis of our recordings confirmed that putative inhibitory neurons were indeed more active during desynchronized cortical states with weak noise correlations. Our results demonstrate that network models with intrinsically-generated variability can accurately reproduce the activity patterns observed in multi-neuron recordings and suggest that inhibition modulates the interactions between intrinsic dynamics and sensory inputs to control the strength of noise correlations.

Simultaneous Computation of Dynamical and Equilibrium Information Using a Weighted Ensemble of Trajectories.

Equilibrium formally can be represented as an ensemble of uncoupled systems undergoing unbiased dynamics in which detailed balance is maintained. Many nonequilibrium processes can be described by suitable subsets of the equilibrium ensemble. Here, we employ the "weighted ensemble" (WE) simulation protocol [Huber and Kim, Biophys. J.1996, 70, 97-110] to generate equilibrium trajectory ensembles and extract nonequilibrium subsets for computing kinetic quantities. States do not need to be chosen in advance. The procedure formally allows estimation of kinetic rates between arbitrary states chosen after the simulation, along with their equilibrium populations. We also describe a related history-dependent matrix procedure for estimating equilibrium and nonequilibrium observables when phase space has been divided into arbitrary non-Markovian regions, whether in WE or ordinary simulation. In this proof-of-principle study, these methods are successfully applied and validated on two molecular systems: explicitly solvated methane association and the implicitly solvated Ala4 peptide. We comment on challenges remaining in WE calculations.