Fast and robust active neuron segmentation in two-photon calcium imaging using spatiotemporal deep learning.

Calcium imaging records large-scale neuronal activity with cellular resolution in vivo. Automated, fast, and reliable active neuron segmentation is a critical step in the analysis workflow of utilizing neuronal signals in real-time behavioral studies for discovery of neuronal coding properties. Here, to exploit the full spatiotemporal information in two-photon calcium imaging movies, we propose a 3D convolutional neural network to identify and segment active neurons. By utilizing a variety of two-photon microscopy datasets, we show that our method outperforms state-of-the-art techniques and is on a par with manual segmentation. Furthermore, we demonstrate that the network trained on data recorded at a specific cortical layer can be used to accurately segment active neurons from another layer with different neuron density. Finally, our work documents significant tabulation flaws in one of the most cited and active online scientific challenges in neuron segmentation. As our computationally fast method is an invaluable tool for a large spectrum of real-time optogenetic experiments, we have made our open-source software and carefully annotated dataset freely available online.

BiconNet: An Edge-preserved Connectivity-based Approach for Salient Object Detection.

Salient object detection (SOD) is viewed as a pixel-wise saliency modeling task by traditional deep learning-based methods. A limitation of current SOD models is insufficient utilization of inter-pixel information, which usually results in imperfect segmentation near edge regions and low spatial coherence. As we demonstrate, using a saliency mask as the only label is suboptimal. To address this limitation, we propose a connectivity-based approach called bilateral connectivity network (BiconNet), which uses connectivity masks together with saliency masks as labels for effective modeling of inter-pixel relationships and object saliency. Moreover, we propose a bilateral voting module to enhance the output connectivity map, and a novel edge feature enhancement method that efficiently utilizes edge-specific features. Through comprehensive experiments on five benchmark datasets, we demonstrate that our proposed method can be plugged into any existing state-of-the-art saliency-based SOD framework to improve its performance with negligible parameter increase.

Deep learning-enabled volumetric cone photoreceptor segmentation in adaptive optics optical coherence tomography images of normal and diseased eyes.

Objective quantification of photoreceptor cell morphology, such as cell diameter and outer segment length, is crucial for early, accurate, and sensitive diagnosis and prognosis of retinal neurodegenerative diseases. Adaptive optics optical coherence tomography (AO-OCT) provides three-dimensional (3-D) visualization of photoreceptor cells in the living human eye. The current gold standard for extracting cell morphology from AO-OCT images involves the tedious process of 2-D manual marking. To automate this process and extend to 3-D analysis of the volumetric data, we propose a comprehensive deep learning framework to segment individual cone cells in AO-OCT scans. Our automated method achieved human-level performance in assessing cone photoreceptors of healthy and diseased participants captured with three different AO-OCT systems representing two different types of point scanning OCT: spectral domain and swept source.

Segmentation of Neurons from Fluorescence Calcium Recordings Beyond Real-time.

Fluorescent genetically encoded calcium indicators and two-photon microscopy help understand brain function by generating large-scale in vivo recordings in multiple animal models. Automatic, fast, and accurate active neuron segmentation is critical when processing these videos. In this work, we developed and characterized a novel method, Shallow U-Net Neuron Segmentation (SUNS), to quickly and accurately segment active neurons from two-photon fluorescence imaging videos. We used temporal filtering and whitening schemes to extract temporal features associated with active neurons, and used a compact shallow U-Net to extract spatial features of neurons. Our method was both more accurate and an order of magnitude faster than state-of-the-art techniques when processing multiple datasets acquired by independent experimental groups; the difference in accuracy was enlarged when processing datasets containing few manually marked ground truths. We also developed an online version, potentially enabling real-time feedback neuroscience experiments.

Connectivity-based deep learning approach for segmentation of the epithelium in in vivo human esophageal OCT images.

Optical coherence tomography (OCT) is used for diagnosis of esophageal diseases such as Barrett's esophagus. Given the large volume of OCT data acquired, automated analysis is needed. Here we propose a bilateral connectivity-based neural network for in vivo human esophageal OCT layer segmentation. Our method, connectivity-based CE-Net (Bicon-CE), defines layer segmentation as a combination of pixel connectivity modeling and pixel-wise tissue classification. Bicon-CE outperformed other widely used neural networks and reduced common topological prediction issues in tissues from healthy patients and from patients with Barrett's esophagus. This is the first end-to-end learning method developed for automatic segmentation of the epithelium in in vivo human esophageal OCT images.

Weakly supervised individual ganglion cell segmentation from adaptive optics OCT images for glaucomatous damage assessment.

Cell-level quantitative features of retinal ganglion cells (GCs) are potentially important biomarkers for improved diagnosis and treatment monitoring of neurodegenerative diseases such as glaucoma, Parkinson's disease, and Alzheimer's disease. Yet, due to limited resolution, individual GCs cannot be visualized by commonly used ophthalmic imaging systems, including optical coherence tomography (OCT), and assessment is limited to gross layer thickness analysis. Adaptive optics OCT (AO-OCT) enables in vivo imaging of individual retinal GCs. We present an automated segmentation of GC layer (GCL) somas from AO-OCT volumes based on weakly supervised deep learning (named WeakGCSeg), which effectively utilizes weak annotations in the training process. Experimental results show that WeakGCSeg is on par with or superior to human experts and is superior to other state-of-the-art networks. The automated quantitative features of individual GCLs show an increase in structure-function correlation in glaucoma subjects compared to using thickness measures from OCT images. Our results suggest that by automatic quantification of GC morphology, WeakGCSeg can potentially alleviate a major bottleneck in using AO-OCT for vision research.

Hybrid light-sheet and light-field microscope for high resolution and large volume neuroimaging.

Large scale simultaneous recording of fast patterns of neural activity remains challenging. Volumetric imaging modalities such as scanning-beam light-sheet microscopy (LSM) and wide-field light-field microscopy (WFLFM) fall short of the goal due to their complex calibration procedure, low spatial resolution, or high-photobleaching. Here, we demonstrate a hybrid light-sheet light-field microscopy (LSLFM) modality that yields high spatial resolution with simplified alignment of the imaging plane and the excitation plane. This new modality combines the selective excitation of light-sheet illumination with volumetric light-field imaging. This modality overcomes the current limitations of the scanning-beam LSM and WFLFM implementations. Compared with LSM, LSLFM captures volumetric data at a frame rate 50× lower than the rate of LSM and requires no dynamic calibration. Compared with WFLFM, LSLFM produces moderate improvements in spatial resolutions, 10 times improvement in the contrast when imaging fluorescent beads, and 3.2× the signal-to-noise ratio in the detection of neural activity when imaging live zebrafish expressing a genetically encoded calcium sensor.

Information-Theoretic Approach and Fundamental Limits of Resolving Two Closely Timed Neuronal Spikes in Mouse Brain Calcium Imaging.

OBJECTIVE: Although optical imaging of neurons using fluorescent genetically encoded calcium sensors has enabled large-scale in vivo experiments, the sensors' slow dynamics often blur closely timed action potentials into indistinguishable transients. While several previous approaches have been proposed to estimate the timing of individual spikes, they have overlooked the important and practical problem of estimating interspike interval (ISI) for overlapping transients. METHODS: We use statistical detection theory to find the minimum detectable ISI under different levels of signal-to-noise ratio (SNR), model complexity, and recording speed. We also derive the Cramer-Rao lower bounds (CRBs) for the problem of ISI estimation. We use Monte-Carlo simulations with biologically derived parameters to numerically obtain the minimum detectable ISI and evaluate the performance of our estimators. Furthermore, we apply our detector to distinguish overlapping transients from experimentally obtained calcium imaging data. RESULTS: Experiments based on simulated and real data across different SNR levels and recording speeds show that our algorithms can accurately distinguish two fluorescence signals with ISI on the order of tens of milliseconds, shorter than the waveform's rise time. Our study shows that the statistically optimal ISI estimators closely approached the CRBs. CONCLUSION: Our work suggests that full analysis using recording speed, sensor kinetics, SNR, and the sensor's stochastically distributed response to action potentials can accurately resolve ISIs much smaller than the fluorescence waveform's rise time in modern calcium imaging experiments. SIGNIFICANCE: Such analysis aids not only in future spike detection methods, but also in future experimental design when choosing sensors of neuronal activity.