Model-Based Explainable Deep Learning for Light-Field Microscopy Imaging.

In modern neuroscience, observing the dynamics of large populations of neurons is a critical step of understanding how networks of neurons process information. Light-field microscopy (LFM) has emerged as a type of scanless, high-speed, three-dimensional (3D) imaging tool, particularly attractive for this purpose. Imaging neuronal activity using LFM calls for the development of novel computational approaches that fully exploit domain knowledge embedded in physics and optics models, as well as enabling high interpretability and transparency. To this end, we propose a model-based explainable deep learning approach for LFM. Different from purely data-driven methods, the proposed approach integrates wave-optics theory, sparse representation and non-linear optimization with the artificial neural network. In particular, the architecture of the proposed neural network is designed following precise signal and optimization models. Moreover, the network's parameters are learned from a training dataset using a novel training strategy that integrates layer-wise training with tailored knowledge distillation. Such design allows the network to take advantage of domain knowledge and learned new features. It combines the benefit of both model-based and learning-based methods, thereby contributing to superior interpretability, transparency and performance. By evaluating on both structural and functional LFM data obtained from scattering mammalian brain tissues, we demonstrate the capabilities of the proposed approach to achieve fast, robust 3D localization of neuron sources and accurate neural activity identification.

Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields.

Significance: Light-field microscopy (LFM) enables fast, light-efficient, volumetric imaging of neuronal activity with calcium indicators. Calcium transients differ in temporal signal-to-noise ratio (tSNR) and spatial confinement when extracted from volumes reconstructed by different algorithms. Aim: We evaluated the capabilities and limitations of two light-field reconstruction algorithms for calcium fluorescence imaging. Approach: We acquired light-field image series from neurons either bulk-labeled or filled intracellularly with the red-emitting calcium dye CaSiR-1 in acute mouse brain slices. We compared the tSNR and spatial confinement of calcium signals extracted from volumes reconstructed with synthetic refocusing and Richardson-Lucy three-dimensional deconvolution with and without total variation regularization. Results: Both synthetic refocusing and Richardson-Lucy deconvolution resolved calcium signals from single cells and neuronal dendrites in three dimensions. Increasing deconvolution iteration number improved spatial confinement but reduced tSNR compared with synthetic refocusing. Volumetric light-field imaging did not decrease calcium signal tSNR compared with interleaved, widefield image series acquired in matched planes. Conclusions: LFM enables high-volume rate, volumetric imaging of calcium transients in single cell somata (bulk-labeled) and dendrites (intracellularly loaded). The trade-offs identified for tSNR, spatial confinement, and computational cost indicate which of synthetic refocusing or deconvolution can better realize the scientific requirements of future LFM calcium imaging applications.

Light-Field Microscopy for Optical Imaging of Neuronal Activity: When Model-Based Methods Meet Data-Driven Approaches.

Understanding how networks of neurons process information is one of the key challenges in modern neuroscience. A necessary step to achieve this goal is to be able to observe the dynamics of large populations of neurons over a large area of the brain. Light-field microscopy (LFM), a type of scanless microscope, is a particularly attractive candidate for high-speed three-dimensional (3D) imaging. It captures volumetric information in a single snapshot, allowing volumetric imaging at video frame-rates. Specific features of imaging neuronal activity using LFM call for the development of novel machine learning approaches that fully exploit priors embedded in physics and optics models. Signal processing theory and wave-optics theory could play a key role in filling this gap, and contribute to novel computational methods with enhanced interpretability and generalization by integrating model-driven and data-driven approaches. This paper is devoted to a comprehensive survey to state-of-the-art of computational methods for LFM, with a focus on model-based and data-driven approaches.

Subcellular resolution three-dimensional light-field imaging with genetically encoded voltage indicators.

Significance: Light-field microscopy (LFM) enables high signal-to-noise ratio (SNR) and light efficient volume imaging at fast frame rates. Voltage imaging with genetically encoded voltage indicators (GEVIs) stands to particularly benefit from LFM's volumetric imaging capability due to high required sampling rates and limited probe brightness and functional sensitivity. Aim: We demonstrate subcellular resolution GEVI light-field imaging in acute mouse brain slices resolving dendritic voltage signals in three spatial dimensions. Approach: We imaged action potential-induced fluorescence transients in mouse brain slices sparsely expressing the GEVI VSFP-Butterfly 1.2 in wide-field microscopy (WFM) and LFM modes. We compared functional signal SNR and localization between different LFM reconstruction approaches and between LFM and WFM. Results: LFM enabled three-dimensional (3-D) localization of action potential-induced fluorescence transients in neuronal somata and dendrites. Nonregularized deconvolution decreased SNR with increased iteration number compared to synthetic refocusing but increased axial and lateral signal localization. SNR was unaffected for LFM compared to WFM. Conclusions: LFM enables 3-D localization of fluorescence transients, therefore eliminating the need for structures to lie in a single focal plane. These results demonstrate LFM's potential for studying dendritic integration and action potential propagation in three spatial dimensions.

3D Localization for Light-Field Microscopy via Convolutional Sparse Coding on Epipolar Images.

Light-field microscopy (LFM) is a type of all-optical imaging system that is able to capture 4D geometric information of light rays and can reconstruct a 3D model from a single snapshot. In this paper, we propose a new 3D localization approach to effectively detect 3D positions of neuronal cells from a single light-field image with high accuracy and outstanding robustness to light scattering. This is achieved by constructing a depth-aware dictionary and by combining it with convolutional sparse coding. Specifically, our approach includes 3 key parts: light-field calibration, depth-aware dictionary construction, and localization based on convolutional sparse coding (CSC). In the first part, an observed raw light-field image is calibrated and then decoded into a two-plane parameterized 4D format which leads to the epi-polar plane image (EPI). The second part involves simulating a set of light-fields using a wave-optics forward model for a ball-shaped volume that is located at different depths. Then, a depth-aware dictionary is constructed where each element is a synthetic EPI associated to a specific depth. Finally, by taking full advantage of the sparsity prior and shift-invariance property of EPI, 3D localization is achieved via convolutional sparse coding on an observed EPI with respect to the depth-aware EPI dictionary. We evaluate our approach on both non-scattering specimen (fluorescent beads suspended in agarose gel) and scattering media (brain tissues of genetically encoded mice). Extensive experiments demonstrate that our approach can reliably detect the 3D positions of granular targets with small Root Mean Square Error (RMSE), high robustness to optical aberration and light scattering in mammalian brain tissues.

Dedicated Setup for the Photoconversion of Fluorescent Dyes for Functional Electron Microscopy.

Here, we describe a cost-effective setup for targeted photoconversion of fluorescent signals into electron dense ones. This approach has offered invaluable insights in the morphology and function of fine neuronal structures. The technique relies on the localized oxidation of diaminobenzidine (DAB) mediated by excited fluorophores. This paper includes a detailed description of how to build a simple photoconversion setup that can increase reliability and throughput of this well-established technique. The system described here, is particularly well-suited for thick neuronal tissue, where light penetration and oxygen diffusion may be limiting DAB oxidation. To demonstrate the system, we use Correlative Light and Electron Microscopy (CLEM) to visualize functionally-labeled individual synaptic vesicles released onto an identified layer 5 neuron in an acute cortical slice. The setup significantly simplifies the photoconversion workflow, increasing the depth of photoillumination, improving the targeting of the region of interest and reducing the time required to process each individual sample. We have tested this setup extensively for the photoconversion of FM 1-43FX and Lucifer Yellow both excited at 473 nm. In principle, the system can be adapted to any dye or nanoparticle able to oxidize DAB when excited by a specific wavelength of light.

All-optical crosstalk-free manipulation and readout of Chronos-expressing neurons.

All optical neurophysiology allows manipulation and readout of neural network activity with single-cell spatial resolution and millisecond temporal resolution. Neurons can be made to express proteins that actuate transmembrane currents upon light absorption, enabling optical control of membrane potential and action potential signalling. In addition, neurons can be genetically or synthetically labelled with fluorescent reporters of changes in intracellular calcium concentration or membrane potential. Thus, to optically manipulate and readout neural activity in parallel, two spectra are involved: the action spectrum of the actuator, and the absorption spectrum of the fluorescent reporter. Due to overlap in these spectra, previous all-optical neurophysiology paradigms have been hindered by spurious activation of neuronal activity caused by the readout light. Here, we pair the blue-green absorbing optogenetic actuator, Chronos, with a deep red-emitting fluorescent calcium reporter CaSiR-1. We show that cultured Chinese hamster ovary cells transfected with Chronos do not exhibit transmembrane currents when illuminated with wavelengths and intensities suitable for exciting one-photon CaSiR-1 fluorescence. We then demonstrate crosstalk-free, high signal-to-noise ratio CaSiR-1 red fluorescence imaging at 100 frames s-1 of Chronos-mediated calcium transients evoked in neurons with blue light pulses at rates up to 20 Hz. These results indicate that the spectral separation between red light excited fluorophores, excited efficiently at or above 640 nm, with blue-green absorbing opsins such as Chronos, is sufficient to avoid spurious opsin actuation by the imaging wavelengths and therefore enable crosstalk-free all-optical neuronal manipulation and readout.

Corrigendum: Single-Neuron Level One-Photon Voltage Imaging With Sparsely Targeted Genetically Encoded Voltage Indicators.

[This corrects the article DOI: 10.3389/fncel.2019.00039.].

Single-Neuron Level One-Photon Voltage Imaging With Sparsely Targeted Genetically Encoded Voltage Indicators.

Voltage imaging of many neurons simultaneously at single-cell resolution is hampered by the difficulty of detecting small voltage signals from overlapping neuronal processes in neural tissue. Recent advances in genetically encoded voltage indicator (GEVI) imaging have shown single-cell resolution optical voltage recordings in intact tissue through imaging naturally sparse cell classes, sparse viral expression, soma restricted expression, advanced optical systems, or a combination of these. Widespread sparse and strong transgenic GEVI expression would enable straightforward optical access to a densely occurring cell type, such as cortical pyramidal cells. Here we demonstrate that a recently described sparse transgenic expression strategy can enable single-cell resolution voltage imaging of cortical pyramidal cells in intact brain tissue without restricting expression to the soma. We also quantify the functional crosstalk in brain tissue and discuss optimal imaging rates to inform future GEVI experimental design.

Surface plasmon resonance imaging of excitable cells.

Surface plasmons (SPs) are surface charge density oscillations occuring at a metal/dieletric interface and are highly sensitive to refractive index variations adjacent to the surface. This sensitivity has been exploited successfully for chemical and biological assays. In these systems, a surface plasmon resonance (SPR)-based sensor detects temporal variations in the refractive index at a point. SPR has also been used in imaging systems where the spatial variations of refractive index in the sample provide the contrast mechanism. SPR imaging systems using high numerical aperture (NA) objective lenses have been designed to image adherent live cells with high magnification and near-diffraction limited spatial resolution. Addressing research questions in cell physiology and pharmacology often requires the development of a multimodal microscope where complementary information can be obtained. In this paper, we present the development of a multimodal microscope that combines SPR imaging with a number of additional imaging modalities including bright-field, epifluorescence, total internal reflection microscopy and SPR fluorescence microscopy. We used a high NA objective lens for SPR and TIR microscopy and the platform has been used to image live cell cultures demonstrating both fluorescent and label-free techniques. Both the SPR and TIR imaging systems feature a wide field of view (~300 µm) that allows measurements from multiple cells whilst maintaining a resolution sufficient to image fine cellular processes. The capability of the platform to perform label-free functional imaging of living cells was demonstrated by imaging the spatial variations in contractions from stem cell-derived cardiomyocytes. This technique shows promise for non-invasive imaging of cultured cells over very long periods of time during development.