Model-Based Explainable Deep Learning for Light-Field Microscopy Imaging.

In modern neuroscience, observing the dynamics of large populations of neurons is a critical step of understanding how networks of neurons process information. Light-field microscopy (LFM) has emerged as a type of scanless, high-speed, three-dimensional (3D) imaging tool, particularly attractive for this purpose. Imaging neuronal activity using LFM calls for the development of novel computational approaches that fully exploit domain knowledge embedded in physics and optics models, as well as enabling high interpretability and transparency. To this end, we propose a model-based explainable deep learning approach for LFM. Different from purely data-driven methods, the proposed approach integrates wave-optics theory, sparse representation and non-linear optimization with the artificial neural network. In particular, the architecture of the proposed neural network is designed following precise signal and optimization models. Moreover, the network's parameters are learned from a training dataset using a novel training strategy that integrates layer-wise training with tailored knowledge distillation. Such design allows the network to take advantage of domain knowledge and learned new features. It combines the benefit of both model-based and learning-based methods, thereby contributing to superior interpretability, transparency and performance. By evaluating on both structural and functional LFM data obtained from scattering mammalian brain tissues, we demonstrate the capabilities of the proposed approach to achieve fast, robust 3D localization of neuron sources and accurate neural activity identification.

Comparing synthetic refocusing to deconvolution for the extraction of neuronal calcium transients from light fields.

Significance: Light-field microscopy (LFM) enables fast, light-efficient, volumetric imaging of neuronal activity with calcium indicators. Calcium transients differ in temporal signal-to-noise ratio (tSNR) and spatial confinement when extracted from volumes reconstructed by different algorithms. Aim: We evaluated the capabilities and limitations of two light-field reconstruction algorithms for calcium fluorescence imaging. Approach: We acquired light-field image series from neurons either bulk-labeled or filled intracellularly with the red-emitting calcium dye CaSiR-1 in acute mouse brain slices. We compared the tSNR and spatial confinement of calcium signals extracted from volumes reconstructed with synthetic refocusing and Richardson-Lucy three-dimensional deconvolution with and without total variation regularization. Results: Both synthetic refocusing and Richardson-Lucy deconvolution resolved calcium signals from single cells and neuronal dendrites in three dimensions. Increasing deconvolution iteration number improved spatial confinement but reduced tSNR compared with synthetic refocusing. Volumetric light-field imaging did not decrease calcium signal tSNR compared with interleaved, widefield image series acquired in matched planes. Conclusions: LFM enables high-volume rate, volumetric imaging of calcium transients in single cell somata (bulk-labeled) and dendrites (intracellularly loaded). The trade-offs identified for tSNR, spatial confinement, and computational cost indicate which of synthetic refocusing or deconvolution can better realize the scientific requirements of future LFM calcium imaging applications.

Light-Field Microscopy for Optical Imaging of Neuronal Activity: When Model-Based Methods Meet Data-Driven Approaches.

Understanding how networks of neurons process information is one of the key challenges in modern neuroscience. A necessary step to achieve this goal is to be able to observe the dynamics of large populations of neurons over a large area of the brain. Light-field microscopy (LFM), a type of scanless microscope, is a particularly attractive candidate for high-speed three-dimensional (3D) imaging. It captures volumetric information in a single snapshot, allowing volumetric imaging at video frame-rates. Specific features of imaging neuronal activity using LFM call for the development of novel machine learning approaches that fully exploit priors embedded in physics and optics models. Signal processing theory and wave-optics theory could play a key role in filling this gap, and contribute to novel computational methods with enhanced interpretability and generalization by integrating model-driven and data-driven approaches. This paper is devoted to a comprehensive survey to state-of-the-art of computational methods for LFM, with a focus on model-based and data-driven approaches.

Subcellular resolution three-dimensional light-field imaging with genetically encoded voltage indicators.

Significance: Light-field microscopy (LFM) enables high signal-to-noise ratio (SNR) and light efficient volume imaging at fast frame rates. Voltage imaging with genetically encoded voltage indicators (GEVIs) stands to particularly benefit from LFM's volumetric imaging capability due to high required sampling rates and limited probe brightness and functional sensitivity. Aim: We demonstrate subcellular resolution GEVI light-field imaging in acute mouse brain slices resolving dendritic voltage signals in three spatial dimensions. Approach: We imaged action potential-induced fluorescence transients in mouse brain slices sparsely expressing the GEVI VSFP-Butterfly 1.2 in wide-field microscopy (WFM) and LFM modes. We compared functional signal SNR and localization between different LFM reconstruction approaches and between LFM and WFM. Results: LFM enabled three-dimensional (3-D) localization of action potential-induced fluorescence transients in neuronal somata and dendrites. Nonregularized deconvolution decreased SNR with increased iteration number compared to synthetic refocusing but increased axial and lateral signal localization. SNR was unaffected for LFM compared to WFM. Conclusions: LFM enables 3-D localization of fluorescence transients, therefore eliminating the need for structures to lie in a single focal plane. These results demonstrate LFM's potential for studying dendritic integration and action potential propagation in three spatial dimensions.

3D Localization for Light-Field Microscopy via Convolutional Sparse Coding on Epipolar Images.

Light-field microscopy (LFM) is a type of all-optical imaging system that is able to capture 4D geometric information of light rays and can reconstruct a 3D model from a single snapshot. In this paper, we propose a new 3D localization approach to effectively detect 3D positions of neuronal cells from a single light-field image with high accuracy and outstanding robustness to light scattering. This is achieved by constructing a depth-aware dictionary and by combining it with convolutional sparse coding. Specifically, our approach includes 3 key parts: light-field calibration, depth-aware dictionary construction, and localization based on convolutional sparse coding (CSC). In the first part, an observed raw light-field image is calibrated and then decoded into a two-plane parameterized 4D format which leads to the epi-polar plane image (EPI). The second part involves simulating a set of light-fields using a wave-optics forward model for a ball-shaped volume that is located at different depths. Then, a depth-aware dictionary is constructed where each element is a synthetic EPI associated to a specific depth. Finally, by taking full advantage of the sparsity prior and shift-invariance property of EPI, 3D localization is achieved via convolutional sparse coding on an observed EPI with respect to the depth-aware EPI dictionary. We evaluate our approach on both non-scattering specimen (fluorescent beads suspended in agarose gel) and scattering media (brain tissues of genetically encoded mice). Extensive experiments demonstrate that our approach can reliably detect the 3D positions of granular targets with small Root Mean Square Error (RMSE), high robustness to optical aberration and light scattering in mammalian brain tissues.

China's urban air quality evaluation with streaming data: A DEA window analysis.

After the promotion of urbanization in the past decades, air pollution has become one of the bottlenecks of China's urban development. Kinds of real-time air pollution indicators are recorded by using environmental detection system in China's urban area that produces precious streaming data. The present paper constructs window DEA model to compute the dynamic air quality index after applying hierarchy analysis to resolve the heterogeneity of time varying data. In the section of empirical study, we select the daily data of CO, NO, SO2, PM 2.5 and PM 10 since January 2018 to August 2019 for 360 cities in China which includes 1,092,600 streaming data. Our empirical findings indicate that air pollution is heavily serious in most China' cities, in which more than 95% cities have over emitted air pollution by 30% at least. Chinese urban air quality is significantly affected by the change of month and shows an inverse U-shaped curve relationship in year, while the orders of weeks within month or order of days within week is irrelevant. The provinces with the most urban air pollution concentrate in middle China and from a continuous pollution zone with Shanxi as the center.

Coupled Dictionary Learning for Multi-Contrast MRI Reconstruction.

Magnetic resonance (MR) imaging tasks often involve multiple contrasts, such as T1-weighted, T2-weighted and fluid-attenuated inversion recovery (FLAIR) data. These contrasts capture information associated with the same underlying anatomy and thus exhibit similarities in either structure level or gray level. In this paper, we propose a coupled dictionary learning based multi-contrast MRI reconstruction (CDLMRI) approach to leverage the dependency correlation between different contrasts for guided or joint reconstruction from their under-sampled k -space data. Our approach iterates between three stages: coupled dictionary learning, coupled sparse denoising, and enforcing k -space consistency. The first stage learns a set of dictionaries that not only are adaptive to the contrasts, but also capture correlations among multiple contrasts in a sparse transform domain. By capitalizing on the learned dictionaries, the second stage performs coupled sparse coding to remove the aliasing and noise in the corrupted contrasts. The third stage enforces consistency between the denoised contrasts and the measurements in the k -space domain. Numerical experiments, consisting of retrospective under-sampling of various MRI contrasts with a variety of sampling schemes, demonstrate that CDLMRI is capable of capturing structural dependencies between different contrasts. The learned priors indicate notable advantages in multi-contrast MR imaging and promising applications in quantitative MR imaging such as MR fingerprinting.

HYDRA: Hybrid deep magnetic resonance fingerprinting.

PURPOSE: Magnetic resonance fingerprinting (MRF) methods typically rely on dictionary matching to map the temporal MRF signals to quantitative tissue parameters. Such approaches suffer from inherent discretization errors, as well as high computational complexity as the dictionary size grows. To alleviate these issues, we propose a HYbrid Deep magnetic ResonAnce fingerprinting (HYDRA) approach, referred to as HYDRA. METHODS: HYDRA involves two stages: a model-based signature restoration phase and a learning-based parameter restoration phase. Signal restoration is implemented using low-rank based de-aliasing techniques while parameter restoration is performed using a deep nonlocal residual convolutional neural network. The designed network is trained on synthesized MRF data simulated with the Bloch equations and fast imaging with steady-state precession (FISP) sequences. In test mode, it takes a temporal MRF signal as input and produces the corresponding tissue parameters. RESULTS: We validated our approach on both synthetic data and anatomical data generated from a healthy subject. The results demonstrate that, in contrast to conventional dictionary matching-based MRF techniques, our approach significantly improves inference speed by eliminating the time-consuming dictionary matching operation, and alleviates discretization errors by outputting continuous-valued parameters. We further avoid the need to store a large dictionary, thus reducing memory requirements. CONCLUSIONS: Our approach demonstrates advantages in terms of inference speed, accuracy, and storage requirements over competing MRF methods.