Smart lattice light-sheet microscopy for imaging rare and complex cellular events.

Light-sheet microscopes enable rapid high-resolution imaging of biological specimens; however, biological processes span spatiotemporal scales. Moreover, long-term phenotypes are often instigated by rare or fleeting biological events that are difficult to capture with a single imaging modality. Here, to overcome this limitation, we present smartLLSM, a microscope that incorporates artificial intelligence-based instrument control to autonomously switch between epifluorescent inverted imaging and lattice light-sheet microscopy (LLSM). We apply this approach to two unique processes: cell division and immune synapse formation. In each context, smartLLSM provides population-level statistics across thousands of cells and autonomously captures multicolor three-dimensional datasets or four-dimensional time-lapse movies of rare events at rates that dramatically exceed human capabilities. From this, we quantify the effects of Taxol dose on spindle structure and kinetochore dynamics in dividing cells and of antigen strength on cytotoxic T lymphocyte engagement and lytic granule polarization at the immune synapse. Overall, smartLLSM efficiently detects rare events within heterogeneous cell populations and records these processes with high spatiotemporal four-dimensional imaging over statistically significant replicates.

FIOLA: an accelerated pipeline for fluorescence imaging online analysis.

Optical microscopy methods such as calcium and voltage imaging enable fast activity readout of large neuronal populations using light. However, the lack of corresponding advances in online algorithms has slowed progress in retrieving information about neural activity during or shortly after an experiment. This gap not only prevents the execution of real-time closed-loop experiments, but also hampers fast experiment-analysis-theory turnover for high-throughput imaging modalities. Reliable extraction of neural activity from fluorescence imaging frames at speeds compatible with indicator dynamics and imaging modalities poses a challenge. We therefore developed FIOLA, a framework for fluorescence imaging online analysis that extracts neuronal activity from calcium and voltage imaging movies at speeds one order of magnitude faster than state-of-the-art methods. FIOLA exploits algorithms optimized for parallel processing on GPUs and CPUs. We demonstrate reliable and scalable performance of FIOLA on both simulated and real calcium and voltage imaging datasets. Finally, we present an online experimental scenario to provide guidance in setting FIOLA parameters and to highlight the trade-offs of our approach.

Smart Lattice Light Sheet Microscopy for imaging rare and complex cellular events.

Light sheet microscopes enable rapid, high-resolution imaging of biological specimens; however, biological processes span a variety of spatiotemporal scales. Moreover, long-term phenotypes are often instigated by rare or fleeting biological events that are difficult to capture with a single imaging modality and constant imaging parameters. To overcome this limitation, we present smartLLSM, a microscope that incorporates AI-based instrument control to autonomously switch between epifluorescent inverted imaging and lattice light sheet microscopy. We apply this technology to two major scenarios. First, we demonstrate that the instrument provides population-level statistics of cell cycle states across thousands of cells on a coverslip. Second, we show that by using real-time image feedback to switch between imaging modes, the instrument autonomously captures multicolor 3D datasets or 4D time-lapse movies of dividing cells at rates that dramatically exceed human capabilities. Quantitative image analysis on high-content + high-throughput datasets reveal kinetochore and chromosome dynamics in dividing cells and determine the effects of drug perturbation on cells in specific mitotic stages. This new methodology enables efficient detection of rare events within a heterogeneous cell population and records these processes with high spatiotemporal 4D imaging over statistically significant replicates.

A deep learning framework for inference of single-trial neural population dynamics from calcium imaging with subframe temporal resolution.

In many areas of the brain, neural populations act as a coordinated network whose state is tied to behavior on a millisecond timescale. Two-photon (2p) calcium imaging is a powerful tool to probe such network-scale phenomena. However, estimating the network state and dynamics from 2p measurements has proven challenging because of noise, inherent nonlinearities and limitations on temporal resolution. Here we describe Recurrent Autoencoder for Discovering Imaged Calcium Latents (RADICaL), a deep learning method to overcome these limitations at the population level. RADICaL extends methods that exploit dynamics in spiking activity for application to deconvolved calcium signals, whose statistics and temporal dynamics are quite distinct from electrophysiologically recorded spikes. It incorporates a new network training strategy that capitalizes on the timing of 2p sampling to recover network dynamics with high temporal precision. In synthetic tests, RADICaL infers the network state more accurately than previous methods, particularly for high-frequency components. In 2p recordings from sensorimotor areas in mice performing a forelimb reach task, RADICaL infers network state with close correspondence to single-trial variations in behavior and maintains high-quality inference even when neuronal populations are substantially reduced.

Sustained deep-tissue voltage recording using a fast indicator evolved for two-photon microscopy.

Genetically encoded voltage indicators are emerging tools for monitoring voltage dynamics with cell-type specificity. However, current indicators enable a narrow range of applications due to poor performance under two-photon microscopy, a method of choice for deep-tissue recording. To improve indicators, we developed a multiparameter high-throughput platform to optimize voltage indicators for two-photon microscopy. Using this system, we identified JEDI-2P, an indicator that is faster, brighter, and more sensitive and photostable than its predecessors. We demonstrate that JEDI-2P can report light-evoked responses in axonal termini of Drosophila interneurons and the dendrites and somata of amacrine cells of isolated mouse retina. JEDI-2P can also optically record the voltage dynamics of individual cortical neurons in awake behaving mice for more than 30 min using both resonant-scanning and ULoVE random-access microscopy. Finally, ULoVE recording of JEDI-2P can robustly detect spikes at depths exceeding 400 µm and report voltage correlations in pairs of neurons.

Two-photon calcium imaging of neuronal activity.

In vivo two-photon calcium imaging (2PCI) is a technique used for recording neuronal activity in the intact brain. It is based on the principle that, when neurons fire action potentials, intracellular calcium levels rise, which can be detected using fluorescent molecules that bind to calcium. This Primer is designed for scientists who are considering embarking on experiments with 2PCI. We provide the reader with a background on the basic concepts behind calcium imaging and on the reasons why 2PCI is an increasingly powerful and versatile technique in neuroscience. The Primer explains the different steps involved in experiments with 2PCI, provides examples of what ideal preparations should look like and explains how data are analysed. We also discuss some of the current limitations of the technique, and the types of solutions to circumvent them. Finally, we conclude by anticipating what the future of 2PCI might look like, emphasizing some of the analysis pipelines that are being developed and international efforts for data sharing.

Inhibitory gating of coincidence-dependent sensory binding in secondary auditory cortex.

Integration of multi-frequency sounds into a unified perceptual object is critical for recognizing syllables in speech. This "feature binding" relies on the precise synchrony of each component's onset timing, but little is known regarding its neural correlates. We find that multi-frequency sounds prevalent in vocalizations, specifically harmonics, preferentially activate the mouse secondary auditory cortex (A2), whose response deteriorates with shifts in component onset timings. The temporal window for harmonics integration in A2 was broadened by inactivation of somatostatin-expressing interneurons (SOM cells), but not parvalbumin-expressing interneurons (PV cells). Importantly, A2 has functionally connected subnetworks of neurons preferentially encoding harmonic over inharmonic sounds. These subnetworks are stable across days and exist prior to experimental harmonics exposure, suggesting their formation during development. Furthermore, A2 inactivation impairs performance in a discrimination task for coincident harmonics. Together, we propose A2 as a locus for multi-frequency integration, which may form the circuit basis for vocal processing.

VolPy: Automated and scalable analysis pipelines for voltage imaging datasets.

Voltage imaging enables monitoring neural activity at sub-millisecond and sub-cellular scale, unlocking the study of subthreshold activity, synchrony, and network dynamics with unprecedented spatio-temporal resolution. However, high data rates (>800MB/s) and low signal-to-noise ratios create bottlenecks for analyzing such datasets. Here we present VolPy, an automated and scalable pipeline to pre-process voltage imaging datasets. VolPy features motion correction, memory mapping, automated segmentation, denoising and spike extraction, all built on a highly parallelizable, modular, and extensible framework optimized for memory and speed. To aid automated segmentation, we introduce a corpus of 24 manually annotated datasets from different preparations, brain areas and voltage indicators. We benchmark VolPy against ground truth segmentation, simulations and electrophysiology recordings, and we compare its performance with existing algorithms in detecting spikes. Our results indicate that VolPy's performance in spike extraction and scalability are state-of-the-art.

Online analysis of microendoscopic 1-photon calcium imaging data streams.

In vivo calcium imaging through microendoscopic lenses enables imaging of neuronal populations deep within the brains of freely moving animals. Previously, a constrained matrix factorization approach (CNMF-E) has been suggested to extract single-neuronal activity from microendoscopic data. However, this approach relies on offline batch processing of the entire video data and is demanding both in terms of computing and memory requirements. These drawbacks prevent its applicability to the analysis of large datasets and closed-loop experimental settings. Here we address both issues by introducing two different online algorithms for extracting neuronal activity from streaming microendoscopic data. Our first algorithm, OnACID-E, presents an online adaptation of the CNMF-E algorithm, which dramatically reduces its memory and computation requirements. Our second algorithm proposes a convolution-based background model for microendoscopic data that enables even faster (real time) processing. Our approach is modular and can be combined with existing online motion artifact correction and activity deconvolution methods to provide a highly scalable pipeline for microendoscopic data analysis. We apply our algorithms on four previously published typical experimental datasets and show that they yield similar high-quality results as the popular offline approach, but outperform it with regard to computing time and memory requirements. They can be used instead of CNMF-E to process pre-recorded data with boosted speeds and dramatically reduced memory requirements. Further, they newly enable online analysis of live-streaming data even on a laptop.

CaImAn an open source tool for scalable calcium imaging data analysis.

Advances in fluorescence microscopy enable monitoring larger brain areas in-vivo with finer time resolution. The resulting data rates require reproducible analysis pipelines that are reliable, fully automated, and scalable to datasets generated over the course of months. We present CaImAn, an open-source library for calcium imaging data analysis. CaImAn provides automatic and scalable methods to address problems common to pre-processing, including motion correction, neural activity identification, and registration across different sessions of data collection. It does this while requiring minimal user intervention, with good scalability on computers ranging from laptops to high-performance computing clusters. CaImAn is suitable for two-photon and one-photon imaging, and also enables real-time analysis on streaming data. To benchmark the performance of CaImAn we collected and combined a corpus of manual annotations from multiple labelers on nine mouse two-photon datasets. We demonstrate that CaImAn achieves near-human performance in detecting locations of active neurons.

Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data.

In vivo calcium imaging through microendoscopic lenses enables imaging of previously inaccessible neuronal populations deep within the brains of freely moving animals. However, it is computationally challenging to extract single-neuronal activity from microendoscopic data, because of the very large background fluctuations and high spatial overlaps intrinsic to this recording modality. Here, we describe a new constrained matrix factorization approach to accurately separate the background and then demix and denoise the neuronal signals of interest. We compared the proposed method against previous independent components analysis and constrained nonnegative matrix factorization approaches. On both simulated and experimental data recorded from mice, our method substantially improved the quality of extracted cellular signals and detected more well-isolated neural signals, especially in noisy data regimes. These advances can in turn significantly enhance the statistical power of downstream analyses, and ultimately improve scientific conclusions derived from microendoscopic data.

NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data.

BACKGROUND: Motion correction is a challenging pre-processing problem that arises early in the analysis pipeline of calcium imaging data sequences. The motion artifacts in two-photon microscopy recordings can be non-rigid, arising from the finite time of raster scanning and non-uniform deformations of the brain medium. NEW METHOD: We introduce an algorithm for fast Non-Rigid Motion Correction (NoRMCorre) based on template matching. NoRMCorre operates by splitting the field of view (FOV) into overlapping spatial patches along all directions. The patches are registered at a sub-pixel resolution for rigid translation against a regularly updated template. The estimated alignments are subsequently up-sampled to create a smooth motion field for each frame that can efficiently approximate non-rigid artifacts in a piecewise-rigid manner. EXISTING METHODS: Existing approaches either do not scale well in terms of computational performance or are targeted to non-rigid artifacts arising just from the finite speed of raster scanning, and thus cannot correct for non-rigid motion observable in datasets from a large FOV. RESULTS: NoRMCorre can be run in an online mode resulting in comparable to or even faster than real time motion registration of streaming data. We evaluate its performance with simple yet intuitive metrics and compare against other non-rigid registration methods on simulated data and in vivo two-photon calcium imaging datasets. Open source Matlab and Python code is also made available. CONCLUSIONS: The proposed method and accompanying code can be useful for solving large scale image registration problems in calcium imaging, especially in the presence of non-rigid deformations.

Cerebellar granule cells acquire a widespread predictive feedback signal during motor learning.

Cerebellar granule cells, which constitute half the brain's neurons, supply Purkinje cells with contextual information necessary for motor learning, but how they encode this information is unknown. Here we show, using two-photon microscopy to track neural activity over multiple days of cerebellum-dependent eyeblink conditioning in mice, that granule cell populations acquire a dense representation of the anticipatory eyelid movement. Initially, granule cells responded to neutral visual and somatosensory stimuli as well as periorbital airpuffs used for training. As learning progressed, two-thirds of monitored granule cells acquired a conditional response whose timing matched or preceded the learned eyelid movements. Granule cell activity covaried trial by trial to form a redundant code. Many granule cells were also active during movements of nearby body structures. Thus, a predictive signal about the upcoming movement is widely available at the input stage of the cerebellar cortex, as required by forward models of cerebellar control.

Cerebellar associative sensory learning defects in five mouse autism models.

Sensory integration difficulties have been reported in autism, but their underlying brain-circuit mechanisms are underexplored. Using five autism-related mouse models, Shank3+/ΔC, Mecp2(R308/Y), Cntnap2-/-, L7-Tsc1 (L7/Pcp2(Cre)::Tsc1(flox/+)), and patDp(15q11-13)/+, we report specific perturbations in delay eyeblink conditioning, a form of associative sensory learning requiring cerebellar plasticity. By distinguishing perturbations in the probability and characteristics of learned responses, we found that probability was reduced in Cntnap2-/-, patDp(15q11-13)/+, and L7/Pcp2(Cre)::Tsc1(flox/+), which are associated with Purkinje-cell/deep-nuclear gene expression, along with Shank3+/ΔC. Amplitudes were smaller in L7/Pcp2(Cre)::Tsc1(flox/+) as well as Shank3+/ΔC and Mecp2(R308/Y), which are associated with granule cell pathway expression. Shank3+/ΔC and Mecp2(R308/Y) also showed aberrant response timing and reduced Purkinje-cell dendritic spine density. Overall, our observations are potentially accounted for by defects in instructed learning in the olivocerebellar loop and response representation in the granule cell pathway. Our findings indicate that defects in associative temporal binding of sensory events are widespread in autism mouse models.

Coding of stimulus strength via analog calcium signals in Purkinje cell dendrites of awake mice.

The climbing fiber input to Purkinje cells acts as a teaching signal by triggering a massive influx of dendritic calcium that marks the occurrence of instructive stimuli during cerebellar learning. Here, we challenge the view that these calcium spikes are all-or-none and only signal whether the instructive stimulus has occurred, without providing parametric information about its features. We imaged ensembles of Purkinje cell dendrites in awake mice and measured their calcium responses to periocular airpuffs that serve as instructive stimuli during cerebellar-dependent eyeblink conditioning. Information about airpuff duration and pressure was encoded probabilistically across repeated trials, and in two additional signals in single trials: the synchrony of calcium spikes in the Purkinje cell population, and the amplitude of the calcium spikes, which was modulated by a non-climbing fiber pathway. These results indicate that calcium-based teaching signals in Purkinje cells contain analog information that encodes the strength of instructive stimuli trial-by-trial.

A Cerebellar Neuroprosthetic System: Computational Architecture and in vivo Test.

Emulating the input-output functions performed by a brain structure opens the possibility for developing neuroprosthetic systems that replace damaged neuronal circuits. Here, we demonstrate the feasibility of this approach by replacing the cerebellar circuit responsible for the acquisition and extinction of motor memories. Specifically, we show that a rat can undergo acquisition, retention, and extinction of the eye-blink reflex even though the biological circuit responsible for this task has been chemically inactivated via anesthesia. This is achieved by first developing a computational model of the cerebellar microcircuit involved in the acquisition of conditioned reflexes and training it with synthetic data generated based on physiological recordings. Secondly, the cerebellar model is interfaced with the brain of an anesthetized rat, connecting the model's inputs and outputs to afferent and efferent cerebellar structures. As a result, we show that the anesthetized rat, equipped with our neuroprosthetic system, can be classically conditioned to the acquisition of an eye-blink response. However, non-stationarities in the recorded biological signals limit the performance of the cerebellar model. Thus, we introduce an updated cerebellar model and validate it with physiological recordings showing that learning becomes stable and reliable. The resulting system represents an important step toward replacing lost functions of the central nervous system via neuroprosthetics, obtained by integrating a synthetic circuit with the afferent and efferent pathways of a damaged brain region. These results also embody an early example of science-based medicine, where on the one hand the neuroprosthetic system directly validates a theory of cerebellar learning that informed the design of the system, and on the other one it takes a step toward the development of neuro-prostheses that could recover lost learning functions in animals and, in the longer term, humans.

Sensory-driven enhancement of calcium signals in individual Purkinje cell dendrites of awake mice.

Climbing fibers (CFs) are thought to contribute to cerebellar plasticity and learning by triggering a large influx of dendritic calcium in the postsynaptic Purkinje cell (PC) to signal the occurrence of an unexpected sensory event. However, CFs fire about once per second whether or not an event occurs, raising the question of how sensory-driven signals might be distinguished from a background of ongoing spontaneous activity. Here, we report that in PC dendrites of awake mice, CF-triggered calcium signals are enhanced when the trigger is a sensory event. In addition, we show that a large fraction of the total enhancement in each PC dendrite can be accounted for by an additional boost of calcium provided by sensory activation of a non-CF input. We suggest that sensory stimulation may modulate dendritic voltage and calcium concentration in PCs to increase the strength of plasticity signals during cerebellar learning.

Fast calcium sensor proteins for monitoring neural activity.

A major goal of the BRAIN Initiative is the development of technologies to monitor neuronal network activity during active information processing. Toward this goal, genetically encoded calcium indicator proteins have become widely used for reporting activity in preparations ranging from invertebrates to awake mammals. However, slow response times, the narrow sensitivity range of Ca2+ and in some cases, poor signal-to-noise ratio still limit their usefulness. Here, we review recent improvements in the field of neural activity-sensitive probe design with a focus on the GCaMP family of calcium indicator proteins. In this context, we present our newly developed Fast-GCaMPs, which have up to 4-fold accelerated off-responses compared with the next-fastest GCaMP, GCaMP6f. Fast-GCaMPs were designed by destabilizing the association of the hydrophobic pocket of calcium-bound calmodulin with the RS20 binding domain, an intramolecular interaction that protects the green fluorescent protein chromophore. Fast-GCaMP6f-RS06 and Fast-GCaMP6f-RS09 have rapid off-responses in stopped-flow fluorimetry, in neocortical brain slices, and in the intact cerebellum in vivo. Fast-GCaMP6f variants should be useful for tracking action potentials closely spaced in time, and for following neural activity in fast-changing compartments, such as axons and dendrites. Finally, we discuss strategies that may allow tracking of a wider range of neuronal firing rates and improve spike detection.

A VLSI field-programmable mixed-signal array to perform neural signal processing and neural modeling in a prosthetic system.

A very-large-scale integration field-programmable mixed-signal array specialized for neural signal processing and neural modeling has been designed. This has been fabricated as a core on a chip prototype intended for use in an implantable closed-loop prosthetic system aimed at rehabilitation of the learning of a discrete motor response. The chosen experimental context is cerebellar classical conditioning of the eye-blink response. The programmable system is based on the intimate mixing of switched capacitor analog techniques with low speed digital computation; power saving innovations within this framework are presented. The utility of the system is demonstrated by the implementation of a motor classical conditioning model applied to eye-blink conditioning in real time with associated neural signal processing. Paired conditioned and unconditioned stimuli were repeatedly presented to an anesthetized rat and recordings were taken simultaneously from two precerebellar nuclei. These paired stimuli were detected in real time from this multichannel data. This resulted in the acquisition of a trigger for a well-timed conditioned eye-blink response, and repetition of unpaired trials constructed from the same data led to the extinction of the conditioned response trigger, compatible with natural cerebellar learning in awake animals.