Diverse role of NMDA receptors for dendritic integration of neural dynamics.

Neurons, represented as a tree structure of morphology, have various distinguished branches of dendrites. Different types of synaptic receptors distributed over dendrites are responsible for receiving inputs from other neurons. NMDA receptors (NMDARs) are expressed as excitatory units, and play a key physiological role in synaptic function. Although NMDARs are widely expressed in most types of neurons, they play a different role in the cerebellar Purkinje cells (PCs). Utilizing a computational PC model with detailed dendritic morphology, we explored the role of NMDARs at different parts of dendritic branches and regions. We found somatic responses can switch from silent, to simple spikes and complex spikes, depending on specific dendritic branches. Detailed examination of the dendrites regarding their diameters and distance to soma revealed diverse response patterns, yet explain two firing modes, simple and complex spike. Taken together, these results suggest that NMDARs play an important role in controlling excitability sensitivity while taking into account the factor of dendritic properties. Given the complexity of neural morphology varying in cell types, our work suggests that the functional role of NMDARs is not stereotyped but highly interwoven with local properties of neuronal structure.

Decoding Pixel-Level Image Features From Two-Photon Calcium Signals of Macaque Visual Cortex.

Images of visual scenes comprise essential features important for visual cognition of the brain. The complexity of visual features lies at different levels, from simple artificial patterns to natural images with different scenes. It has been a focus of using stimulus images to predict neural responses. However, it remains unclear how to extract features from neuronal responses. Here we address this question by leveraging two-photon calcium neural data recorded from the visual cortex of awake macaque monkeys. With stimuli including various categories of artificial patterns and diverse scenes of natural images, we employed a deep neural network decoder inspired by image segmentation technique. Consistent with the notation of sparse coding for natural images, a few neurons with stronger responses dominated the decoding performance, whereas decoding of ar tificial patterns needs a large number of neurons. When natural images using the model pretrained on artificial patterns are decoded, salient features of natural scenes can be extracted, as well as the conventional category information. Altogether, our results give a new perspective on studying neural encoding principles using reverse-engineering decoding strategies.

Dissecting cascade computational components in spiking neural networks.

Finding out the physical structure of neuronal circuits that governs neuronal responses is an important goal for brain research. With fast advances for large-scale recording techniques, identification of a neuronal circuit with multiple neurons and stages or layers becomes possible and highly demanding. Although methods for mapping the connection structure of circuits have been greatly developed in recent years, they are mostly limited to simple scenarios of a few neurons in a pairwise fashion; and dissecting dynamical circuits, particularly mapping out a complete functional circuit that converges to a single neuron, is still a challenging question. Here, we show that a recent method, termed spike-triggered non-negative matrix factorization (STNMF), can address these issues. By simulating different scenarios of spiking neural networks with various connections between neurons and stages, we demonstrate that STNMF is a persuasive method to dissect functional connections within a circuit. Using spiking activities recorded at neurons of the output layer, STNMF can obtain a complete circuit consisting of all cascade computational components of presynaptic neurons, as well as their spiking activities. For simulated simple and complex cells of the primary visual cortex, STNMF allows us to dissect the pathway of visual computation. Taken together, these results suggest that STNMF could provide a useful approach for investigating neuronal systems leveraging recorded functional neuronal activity.

Bandwidth-Aware Traffic Sensing in Vehicular Networks with Mobile Edge Computing.

Traffic sensing is one of the promising applications to guarantee safe and efficient traffic systems in vehicular networks. However, due to the unique characteristics of vehicular networks, such as limited wireless bandwidth and dynamic mobility of vehicles, traffic sensing always faces high estimation error based on collected traffic data with missing elements and over-high communication cost between terminal users and central server. Hence, this paper investigates the traffic sensing system in vehicular networks with mobile edge computing (MEC), where each MEC server enables traffic data collection and recovery in its local server. On this basis, we formulate the bandwidth-constrained traffic sensing (BCTS) problem, aiming at minimizing the estimation error based on the collected traffic data. To tackle the BCTS problem, we first propose the bandwidth-aware data collection (BDC) algorithm to select the optimal uploaded traffic data by evaluating the priority of each road segment covered by the MEC server. Then, we propose the convex-based data recovery (CDR) algorithm to minimize estimation error by transforming the BCTS into an l 2 -norm minimization problem. Last but not the least, we implement the simulation model and conduct performance evaluation. The comprehensive simulation results verify the superiority of the proposed algorithm.

Phase unwrapping with graph cuts optimization and dual decomposition acceleration for 3D high-resolution MRI data.

PURPOSE: Existence of low SNR regions and rapid-phase variations pose challenges to spatial phase unwrapping algorithms. Global optimization-based phase unwrapping methods are widely used, but are significantly slower than greedy methods. In this paper, dual decomposition acceleration is introduced to speed up a three-dimensional graph cut-based phase unwrapping algorithm. METHODS: The phase unwrapping problem is formulated as a global discrete energy minimization problem, whereas the technique of dual decomposition is used to increase the computational efficiency by splitting the full problem into overlapping subproblems and enforcing the congruence of overlapping variables. Using three dimensional (3D) multiecho gradient echo images from an agarose phantom and five brain hemorrhage patients, we compared this proposed method with an unaccelerated graph cut-based method. RESULTS: Experimental results show up to 18-fold acceleration in computation time. CONCLUSIONS: Dual decomposition significantly improves the computational efficiency of 3D graph cut-based phase unwrapping algorithms. Magn Reson Med 77:1353-1358, 2017. © 2016 International Society for Magnetic Resonance in Medicine.

Robust Decoding of Rich Dynamical Visual Scenes With Retinal Spikes.

Sensory information transmitted to the brain activates neurons to create a series of coping behaviors. Understanding the mechanisms of neural computation and reverse engineering the brain to build intelligent machines requires establishing a robust relationship between stimuli and neural responses. Neural decoding aims to reconstruct the original stimuli that trigger neural responses. With the recent upsurge of artificial intelligence, neural decoding provides an insightful perspective for designing novel algorithms of brain-machine interface. For humans, vision is the dominant contributor to the interaction between the external environment and the brain. In this study, utilizing the retinal neural spike data collected over multi trials with visual stimuli of two movies with different levels of scene complexity, we used a neural network decoder to quantify the decoded visual stimuli with six different metrics for image quality assessment establishing comprehensive inspection of decoding. With the detailed and systematical study of the effect and single and multiple trials of data, different noise in spikes, and blurred images, our results provide an in-depth investigation of decoding dynamical visual scenes using retinal spikes. These results provide insights into the neural coding of visual scenes and services as a guideline for designing next-generation decoding algorithms of neuroprosthesis and other devices of brain-machine interface.

Revealing the mechanisms of semantic satiation with deep learning models.

The phenomenon of semantic satiation, which refers to the loss of meaning of a word or phrase after being repeated many times, is a well-known psychological phenomenon. However, the microscopic neural computational principles responsible for these mechanisms remain unknown. In this study, we use a deep learning model of continuous coupled neural networks to investigate the mechanism underlying semantic satiation and precisely describe this process with neuronal components. Our results suggest that, from a mesoscopic perspective, semantic satiation may be a bottom-up process. Unlike existing macroscopic psychological studies that suggest that semantic satiation is a top-down process, our simulations use a similar experimental paradigm as classical psychology experiments and observe similar results. Satiation of semantic objectives, similar to the learning process of our network model used for object recognition, relies on continuous learning and switching between objects. The underlying neural coupling strengthens or weakens satiation. Taken together, both neural and network mechanisms play a role in controlling semantic satiation.

Deep Spike Learning With Local Classifiers.

Backpropagation has been successfully generalized to optimize deep spiking neural networks (SNNs), where, nevertheless, gradients need to be propagated back through all layers, resulting in a massive consumption of computing resources and an obstacle to the parallelization of training. A biologically motivated scheme of local learning provides an alternative to efficiently train deep networks but often suffers a low performance of accuracy on practical tasks. Thus, how to train deep SNNs with the local learning scheme to achieve both efficient and accurate performance still remains an important challenge. In this study, we focus on a supervised local learning scheme where each layer is independently optimized with an auxiliary classifier. Accordingly, we first propose a spike-based efficient local learning rule by only considering the direct dependencies in the current time. We then propose two variants that additionally incorporate temporal dependencies through a backward and forward process, respectively. The effectiveness and performance of our proposed methods are extensively evaluated with six mainstream datasets. Experimental results show that our methods can successfully scale up to large networks and substantially outperform the spike-based local learning baselines on all studied benchmarks. Our results also reveal that gradients with temporal dependencies are essential for high performance on temporal tasks, while they have negligible effects on rate-based tasks. Our work is significant as it brings the performance of spike-based local learning to a new level with the computational benefits being retained.

Protocol for dissecting cascade computational components in neural networks of a visual system.

Finding the complete functional circuits of neurons is a challenging problem in brain research. Here, we present a protocol, based on visual stimuli and spikes, for obtaining the complete circuit of recorded neurons using spike-triggered nonnegative matrix factorization. We describe steps for data preprocessing, inferring the spatial receptive field of the subunits, and analyzing the module matrix. This approach identifies computational components of the feedforward network of retinal ganglion cells and dissects the network structure based on natural image stimuli. For complete details on the use and execution of this protocol, please refer to Jia et al. (2021).1.

Asynchronous Spatiotemporal Spike Metric for Event Cameras.

Event cameras as bioinspired vision sensors have shown great advantages in high dynamic range and high temporal resolution in vision tasks. Asynchronous spikes from event cameras can be depicted using the marked spatiotemporal point processes (MSTPPs). However, how to measure the distance between asynchronous spikes in the MSTPPs still remains an open issue. To address this problem, we propose a general asynchronous spatiotemporal spike metric considering both spatiotemporal structural properties and polarity attributes for event cameras. Technically, the conditional probability density function is first introduced to describe the spatiotemporal distribution and polarity prior in the MSTPPs. Besides, a spatiotemporal Gaussian kernel is defined to capture the spatiotemporal structure, which transforms discrete spikes into the continuous function in a reproducing kernel Hilbert space (RKHS). Finally, the distance between asynchronous spikes can be quantified by the inner product in the RKHS. The experimental results demonstrate that the proposed approach outperforms the state-of-the-art methods and achieves significant improvement in computational efficiency. Especially, it is able to better depict the changes involving spatiotemporal structural properties and polarity attributes.

Capture the Moment: High-Speed Imaging With Spiking Cameras Through Short-Term Plasticity.

High-speed imaging can help us understand some phenomena that are too fast to be captured by our eyes. Although ultra-fast frame-based cameras (e.g., Phantom) can record millions of fps at reduced resolution, they are too expensive to be widely used. Recently, a retina-inspired vision sensor, spiking camera, has been developed to record external information at 40, 000 Hz. The spiking camera uses the asynchronous binary spike streams to represent visual information. Despite this, how to reconstruct dynamic scenes from asynchronous spikes remains challenging. In this paper, we introduce novel high-speed image reconstruction models based on the short-term plasticity (STP) mechanism of the brain, termed TFSTP and TFMDSTP. We first derive the relationship between states of STP and spike patterns. Then, in TFSTP, by setting up the STP model at each pixel, the scene radiance can be inferred by the states of the models. In TFMDSTP, we use the STP to distinguish the moving and stationary regions, and then use two sets of STP models to reconstruct them respectively. In addition, we present a strategy for correcting error spikes. Experimental results show that the STP-based reconstruction methods can effectively reduce noise with less computing time, and achieve the best performances on both real-world and simulated datasets.

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence.

Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency by introducing neural dynamics and spike properties. As the emerging spiking deep learning paradigm attracts increasing interest, traditional programming frameworks cannot meet the demands of the automatic differentiation, parallel computation acceleration, and high integration of processing neuromorphic datasets and deployment. In this work, we present the SpikingJelly framework to address the aforementioned dilemma. We contribute a full-stack toolkit for preprocessing neuromorphic datasets, building deep SNNs, optimizing their parameters, and deploying SNNs on neuromorphic chips. Compared to existing methods, the training of deep SNNs can be accelerated 11×, and the superior extensibility and flexibility of SpikingJelly enable users to accelerate custom models at low costs through multilevel inheritance and semiautomatic code generation. SpikingJelly paves the way for synthesizing truly energy-efficient SNN-based machine intelligence systems, which will enrich the ecology of neuromorphic computing.

Assessment of visual function in blind mice and monkeys with subretinally implanted nanowire arrays as artificial photoreceptors.

Retinal prostheses could restore image-forming vision in conditions of photoreceptor degeneration. However, contrast sensitivity and visual acuity are often insufficient. Here we report the performance, in mice and monkeys with induced photoreceptor degeneration, of subretinally implanted gold-nanoparticle-coated titania nanowire arrays providing a spatial resolution of 77.5 µm and a temporal resolution of 3.92 Hz in ex vivo retinas (as determined by patch-clamp recording of retinal ganglion cells). In blind mice, the arrays allowed for the detection of drifting gratings and flashing objects at light-intensity thresholds of 15.70-18.09 µW mm-2, and offered visual acuities of 0.3-0.4 cycles per degree, as determined by recordings of visually evoked potentials and optomotor-response tests. In monkeys, the arrays were stable for 54 weeks, allowed for the detection of a 10-µW mm-2 beam of light (0.5° in beam angle) in visually guided saccade experiments, and induced plastic changes in the primary visual cortex, as indicated by long-term in vivo calcium imaging. Nanomaterials as artificial photoreceptors may ameliorate visual deficits in patients with photoreceptor degeneration.

Zooming in on the brain via data science.

Zhaofei Yu, a young principal investigator (PI), and Shanshan Jia, a PhD student, explain the importance of data science in their research. They have developed a method to extract information from high-dimensional data based on wavelet decompositions using simulated and experimental neuroscience data. This method is not only beneficial for neuroscientists, but it might also be helpful to anyone who deals with high-dimensional data.

Spike-based Motion Estimation for Object Tracking through Bio-inspired Unsupervised Learning.

Neuromorphic vision sensors, whose pixels output events/spikes asynchronously with a high temporal resolution according to the scene radiance change, are naturally appropriate for capturing high-speed motion in the scenes. However, how to utilize the events/spikes to smoothly track high-speed moving objects is still a challenging problem. Existing approaches either employ time-consuming iterative optimization, or require large amounts of labeled data to train the object detector. To this end, we propose a bio-inspired unsupervised learning framework, which takes advantage of the spatiotemporal information of events/spikes generated by neuromorphic vision sensors to capture the intrinsic motion patterns. Without off-line training, our models can filter the redundant signals with dynamic adaption module based on short-term plasticity, and extract the motion patterns with motion estimation module based on the spike-timing-dependent plasticity. Combined with the spatiotemporal and motion information of the filtered spike stream, the traditional DBSCAN clustering algorithm and Kalman filter can effectively track multiple targets in extreme scenes. We evaluate the proposed unsupervised framework for object detection and tracking tasks on synthetic data, publicly available event-based datasets, and spiking camera datasets. The experiment results show that the proposed model can robustly detect and smoothly track the moving targets on various challenging scenarios and outperforms state-of-the-art approaches.

Representing the dynamics of high-dimensional data with non-redundant wavelets.

A crucial question in data science is to extract meaningful information embedded in high-dimensional data into a low-dimensional set of features that can represent the original data at different levels. Wavelet analysis is a pervasive method for decomposing time-series signals into a few levels with detailed temporal resolution. However, obtained wavelets are intertwined and over-represented across levels for each sample and across different samples within one population. Here, using neuroscience data of simulated spikes, experimental spikes, calcium imaging signals, and human electrocorticography signals, we leveraged conditional mutual information between wavelets for feature selection. The meaningfulness of selected features was verified to decode stimulus or condition with high accuracy yet using only a small set of features. These results provide a new way of wavelet analysis for extracting essential features of the dynamics of spatiotemporal neural data, which then enables to support novel model design of machine learning with representative features.

Structure inference of networked system with the synergy of deep residual network and fully connected layer network.

The networked systems are booming in multi-disciplines, including the industrial engineering system, the social system, and so on. The network structure is a prerequisite for the understanding and exploration of networked systems. However, the network structure is always unknown in practice, thus, it is significant yet challenging to investigate the inference of network structure. Although some model-based methods and data-driven methods, such as the phase-space based method and the compressive sensing based method, have investigated the structure inference tasks, they were time-consuming due to the greedy iterative optimization procedure, which makes them difficult to satisfy real-time structure inference requirements. Although the reconstruction time of L1 and other methods is short, the reconstruction accuracy is very low. Inspired by the powerful representation ability and time efficiency for the structure inference with the deep learning framework, a novel synergy method combines the deep residual network and fully connected layer network to solve the network structure inference task efficiently and accurately. This method perfectly solves the problems of long reconstruction time and low accuracy of traditional methods. Moreover, the proposed method can also fulfill the inference task of large scale complex network, which further indicates the scalability of the proposed method.

Neural System Identification With Spike-Triggered Non-Negative Matrix Factorization.

Neuronal circuits formed in the brain are complex with intricate connection patterns. Such complexity is also observed in the retina with a relatively simple neuronal circuit. A retinal ganglion cell (GC) receives excitatory inputs from neurons in previous layers as driving forces to fire spikes. Analytical methods are required to decipher these components in a systematic manner. Recently a method called spike-triggered non-negative matrix factorization (STNMF) has been proposed for this purpose. In this study, we extend the scope of the STNMF method. By using retinal GCs as a model system, we show that STNMF can detect various computational properties of upstream bipolar cells (BCs), including spatial receptive field, temporal filter, and transfer nonlinearity. In addition, we recover synaptic connection strengths from the weight matrix of STNMF. Furthermore, we show that STNMF can separate spikes of a GC into a few subsets of spikes, where each subset is contributed by one presynaptic BC. Taken together, these results corroborate that STNMF is a useful method for deciphering the structure of neuronal circuits.

Revealing Fine Structures of the Retinal Receptive Field by Deep-Learning Networks.

Deep convolutional neural networks (CNNs) have demonstrated impressive performance on many visual tasks. Recently, they became useful models for the visual system in neuroscience. However, it is still not clear what is learned by CNNs in terms of neuronal circuits. When a deep CNN with many layers is used for the visual system, it is not easy to compare the structure components of CNNs with possible neuroscience underpinnings due to highly complex circuits from the retina to the higher visual cortex. Here, we address this issue by focusing on single retinal ganglion cells with biophysical models and recording data from animals. By training CNNs with white noise images to predict neuronal responses, we found that fine structures of the retinal receptive field can be revealed. Specifically, convolutional filters learned are resembling biological components of the retinal circuit. This suggests that a CNN learning from one single retinal cell reveals a minimal neural network carried out in this cell. Furthermore, when CNNs learned from different cells are transferred between cells, there is a diversity of transfer learning performance, which indicates that CNNs are cell specific. Moreover, when CNNs are transferred between different types of input images, here white noise versus natural images, transfer learning shows a good performance, which implies that CNNs indeed capture the full computational ability of a single retinal cell for different inputs. Taken together, these results suggest that CNNs could be used to reveal structure components of neuronal circuits, and provide a powerful model for neural system identification.

Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks.

Traditional models of retinal system identification analyze the neural response to artificial stimuli using models consisting of predefined components. The model design is limited to prior knowledge, and the artificial stimuli are too simple to be compared with stimuli processed by the retina. To fill in this gap with an explainable model that reveals how a population of neurons work together to encode the larger field of natural scenes, here we used a deep-learning model for identifying the computational elements of the retinal circuit that contribute to learning the dynamics of natural scenes. Experimental results verify that the recurrent connection plays a key role in encoding complex dynamic visual scenes while learning biological computational underpinnings of the retinal circuit. In addition, the proposed models reveal both the shapes and the locations of the spatiotemporal receptive fields of ganglion cells.