Brain topology improved spiking neural network for efficient reinforcement learning of continuous control.

The brain topology highly reflects the complex cognitive functions of the biological brain after million-years of evolution. Learning from these biological topologies is a smarter and easier way to achieve brain-like intelligence with features of efficiency, robustness, and flexibility. Here we proposed a brain topology-improved spiking neural network (BT-SNN) for efficient reinforcement learning. First, hundreds of biological topologies are generated and selected as subsets of the Allen mouse brain topology with the help of the Tanimoto hierarchical clustering algorithm, which has been widely used in analyzing key features of the brain connectome. Second, a few biological constraints are used to filter out three key topology candidates, including but not limited to the proportion of node functions (e.g., sensation, memory, and motor types) and network sparsity. Third, the network topology is integrated with the hybrid numerical solver-improved leaky-integrated and fire neurons. Fourth, the algorithm is then tuned with an evolutionary algorithm named adaptive random search instead of backpropagation to guide synaptic modifications without affecting raw key features of the topology. Fifth, under the test of four animal-survival-like RL tasks (i.e., dynamic controlling in Mujoco), the BT-SNN can achieve higher scores than not only counterpart SNN using random topology but also some classical ANNs (i.e., long-short-term memory and multi-layer perception). This result indicates that the research effort of incorporating biological topology and evolutionary learning rules has much in store for the future.

Spiking Neural Network for Ultralow-Latency and High-Accurate Object Detection.

Spiking Neural Networks (SNNs) have attracted significant attention for their energy-efficient and brain-inspired event-driven properties. Recent advancements, notably Spiking-YOLO, have enabled SNNs to undertake advanced object detection tasks. Nevertheless, these methods often suffer from increased latency and diminished detection accuracy, rendering them less suitable for latency-sensitive mobile platforms. Additionally, the conversion of artificial neural networks (ANNs) to SNNs frequently compromises the integrity of the ANNs' structure, resulting in poor feature representation and heightened conversion errors. To address the issues of high latency and low detection accuracy, we introduce two solutions: timestep compression and spike-time-dependent integrated (STDI) coding. Timestep compression effectively reduces the number of timesteps required in the ANN-to-SNN conversion by condensing information. The STDI coding employs a time-varying threshold to augment information capacity. Furthermore, we have developed an SNN-based spatial pyramid pooling (SPP) structure, optimized to preserve the network's structural efficacy during conversion. Utilizing these approaches, we present the ultralow latency and highly accurate object detection model, SUHD. SUHD exhibits exceptional performance on challenging datasets like PASCAL VOC and MS COCO, achieving a remarkable reduction of approximately 750 times in timesteps and a 30% enhancement in mean average precision (mAP) compared to Spiking-YOLO on MS COCO. To the best of our knowledge, SUHD is currently the deepest spike-based object detection model, achieving ultralow timesteps for lossless conversion.

Learning and Controlling Multiscale Dynamics in Spiking Neural Networks Using Recursive Least Square Modifications.

Invasive brain-computer interfaces (BCIs) have the capability to simultaneously record discrete signals across multiple scales, but how to effectively process and analyze these potentially related signals remains an open challenge. This article introduces an innovative approach that merges modern control theory with spiking neural networks (SNNs) to bridge the gap among multiscale discrete information. Specifically, the macroscopic point-to-point trajectory is formulated as an optimal control problem with fixed terminal time and state, and it is iteratively solved using the direct dynamic programming (DDP) algorithm. Additionally, SNN is utilized to simulate microscale neural activities in the premotor cortex, employing the product of the weighted adjacency matrix and the mesoscale firing rate to approximate the macroscopic trajectory. The error between actual macroscale behavior and the preceding approximation is then used to update the weighted adjacency matrix through the recursive least square (RLS) method. Analysis and simulation of various tasks, including low-dimensional point-to-point tasks, high-dimensional complex Lorenz systems, and center-out-and-back tasks, verify the feasibility and interpretability of our method in processing multiscale signals ranging from spiking neurons to motion trajectory through the integration of SNN and control theory.

A brain-inspired algorithm that mitigates catastrophic forgetting of artificial and spiking neural networks with low computational cost.

Neuromodulators in the brain act globally at many forms of synaptic plasticity, represented as metaplasticity, which is rarely considered by existing spiking (SNNs) and nonspiking artificial neural networks (ANNs). Here, we report an efficient brain-inspired computing algorithm for SNNs and ANNs, referred to here as neuromodulation-assisted credit assignment (NACA), which uses expectation signals to induce defined levels of neuromodulators to selective synapses, whereby the long-term synaptic potentiation and depression are modified in a nonlinear manner depending on the neuromodulator level. The NACA algorithm achieved high recognition accuracy with substantially reduced computational cost in learning spatial and temporal classification tasks. Notably, NACA was also verified as efficient for learning five different class continuous learning tasks with varying degrees of complexity, exhibiting a markedly mitigated catastrophic forgetting at low computational cost. Mapping synaptic weight changes showed that these benefits could be explained by the sparse and targeted synaptic modifications attributed to expectation-based global neuromodulation.

Mixture of personality improved spiking actor network for efficient multi-agent cooperation.

Adaptive multi-agent cooperation with especially unseen partners is becoming more challenging in multi-agent reinforcement learning (MARL) research, whereby conventional deep-learning-based algorithms suffer from the poor new-player-generalization problem, possibly caused by not considering theory-of-mind theory (ToM). Inspired by the ToM personality in cognitive psychology, where a human can easily resolve this problem by predicting others' intuitive personality first before complex actions, we propose a biologically-plausible algorithm named the mixture of personality (MoP) improved spiking actor network (SAN). The MoP module contains a determinantal point process to simulate the formation and integration of different personality types, and the SAN module contains spiking neurons for efficient reinforcement learning. The experimental results on the benchmark cooperative overcooked task showed that the proposed MoP-SAN algorithm could achieve higher performance for the paradigms with (learning) and without (generalization) unseen partners. Furthermore, ablation experiments highlighted the contribution of MoP in SAN learning, and some visualization analysis explained why the proposed algorithm is superior to some counterpart deep actor networks.

Self-Lateral Propagation Elevates Synaptic Modifications in Spiking Neural Networks for the Efficient Spatial and Temporal Classification.

The brain's mystery for efficient and intelligent computation hides in the neuronal encoding, functional circuits, and plasticity principles in natural neural networks. However, many plasticity principles have not been fully incorporated into artificial or spiking neural networks (SNNs). Here, we report that incorporating a novel feature of synaptic plasticity found in natural networks, whereby synaptic modifications self-propagate to nearby synapses, named self-lateral propagation (SLP), could further improve the accuracy of SNNs in three benchmark spatial and temporal classification tasks. The SLP contains lateral pre ( SLPpre ) and lateral post ( SLPpost ) synaptic propagation, describing the spread of synaptic modifications among output synapses made by axon collaterals or among converging synapses on the postsynaptic neuron, respectively. The SLP is biologically plausible and can lead to a coordinated synaptic modification within layers that endow higher efficiency without losing much accuracy. Furthermore, the experimental results showed the impressive role of SLP in sharpening the normal distribution of synaptic weights and broadening the more uniform distribution of misclassified samples, which are both considered essential for understanding the learning convergence and network generalization of neural networks.

Explaining cocktail party effect and McGurk effect with a spiking neural network improved by Motif-topology.

Network architectures and learning principles have been critical in developing complex cognitive capabilities in artificial neural networks (ANNs). Spiking neural networks (SNNs) are a subset of ANNs that incorporate additional biological features such as dynamic spiking neurons, biologically specified architectures, and efficient and useful paradigms. Here we focus more on network architectures in SNNs, such as the meta operator called 3-node network motifs, which is borrowed from the biological network. We proposed a Motif-topology improved SNN (M-SNN), which is further verified efficient in explaining key cognitive phenomenon such as the cocktail party effect (a typical noise-robust speech-recognition task) and McGurk effect (a typical multi-sensory integration task). For M-SNN, the Motif topology is obtained by integrating the spatial and temporal motifs. These spatial and temporal motifs are first generated from the pre-training of spatial (e.g., MNIST) and temporal (e.g., TIDigits) datasets, respectively, and then applied to the previously introduced two cognitive effect tasks. The experimental results showed a lower computational cost and higher accuracy and a better explanation of some key phenomena of these two effects, such as new concept generation and anti-background noise. This mesoscale network motifs topology has much room for the future.

Origin of the efficiency of spike timing-based neural computation for processing temporal information.

Although the advantage of spike timing-based over rate-based network computation has been recognized, the underlying mechanism remains unclear. Using Tempotron and Perceptron as elementary neural models, we examined the intrinsic difference between spike timing-based and rate-based computations. For more direct comparison, we modified Tempotron computation into rate-based computation with the retention of some temporal information. Previous studies have shown that spike timing-based computation are computationally more powerful than rate-based computation in terms of the number of computational units required and the capability in classifying random patterns. Our study showed that spike timing-based and rate-based Tempotron computations provided similar capability in classifying random spike patterns, as well as in text sentiment classification and spam text detection. However, spike timing-based computation is superior in performing a task involving discriminating forward vs. reverse sequence of events, i.e., information mainly temporal in nature. Further studies revealed that this superiority required the asymmetry in the profile of the postsynaptic potential (PSP), and that temporal sequence information was converted to biased spatial distribution of synaptic weight modifications during learning. Thus, the intrinsic PSP asymmetry is a mechanistic basis for the high efficiency of spike timing-based computation for processing temporal information.

A Brain-Inspired Approach for Probabilistic Estimation and Efficient Planning in Precision Physical Interaction.

This article presents a novel structure of spiking neural networks (SNNs) to simulate the joint function of multiple brain regions in handling precision physical interactions. This task desires efficient movement planning while considering contact prediction and fast radial compensation. Contact prediction demands the cognitive memory of the interaction model, and we novelly propose a double recurrent network to imitate the hippocampus, addressing the spatiotemporal property of the distribution. Radial contact response needs rich spatial information, and we use a cerebellum-inspired module to achieve temporally dynamic prediction. We also use a block-based feedforward network to plan movements, behaving like the prefrontal cortex. These modules are integrated to realize the joint cognitive function of multiple brain regions in prediction, controlling, and planning. We present an appropriate controller and planner to generate teaching signals and provide a feasible network initialization for reinforcement learning, which modifies synapses in accordance with reality. The experimental results demonstrate the validity of the proposed method.

A Brain-Inspired Approach for Collision-Free Movement Planning in the Small Operational Space.

In a small operational space, e.g., mesoscale or microscale, we need to control movements carefully because of fragile objects. This article proposes a novel structure based on spiking neural networks to imitate the joint function of multiple brain regions in visual guiding in the small operational space and offers two channels to achieve collision-free movements. For the state sensation, we simulate the primary visual cortex to directly extract features from multiple input images and the high-level visual cortex to obtain the object distance, which is indirectly measurable, in the Cartesian coordinates. Our approach emulates the prefrontal cortex from two aspects: multiple liquid state machines to predict distances of the next several steps based on the preceding trajectory and a block-based excitation-inhibition feedforward network to plan movements considering the target and prediction. Responding to "too close" states needs rich temporal information, and we leverage a cerebellar network for the subconscious reaction. From the viewpoint of the inner pathway, they also form two channels. One channel starts from state extraction to attraction movement planning, both in the camera coordinates, behaving visual-servo control. The other is the collision-avoidance channel, which calculates distances, predicts trajectories, and reacts to the repulsion, all in the Cartesian coordinates. We provide appropriate supervised signals for coarse training and apply reinforcement learning to modify synapses in accordance with reality. Simulation and experiment results validate the proposed method.

Tuning Convolutional Spiking Neural Network With Biologically Plausible Reward Propagation.

Spiking neural networks (SNNs) contain more biologically realistic structures and biologically inspired learning principles than those in standard artificial neural networks (ANNs). SNNs are considered the third generation of ANNs, powerful on the robust computation with a low computational cost. The neurons in SNNs are nondifferential, containing decayed historical states and generating event-based spikes after their states reaching the firing threshold. These dynamic characteristics of SNNs make it difficult to be directly trained with the standard backpropagation (BP), which is also considered not biologically plausible. In this article, a biologically plausible reward propagation (BRP) algorithm is proposed and applied to the SNN architecture with both spiking-convolution (with both 1-D and 2-D convolutional kernels) and full-connection layers. Unlike the standard BP that propagates error signals from postsynaptic to presynaptic neurons layer by layer, the BRP propagates target labels instead of errors directly from the output layer to all prehidden layers. This effort is more consistent with the top-down reward-guiding learning in cortical columns of the neocortex. Synaptic modifications with only local gradient differences are induced with pseudo-BP that might also be replaced with the spike-timing-dependent plasticity (STDP). The performance of the proposed BRP-SNN is further verified on the spatial (including MNIST and Cifar-10) and temporal (including TIDigits and DvsGesture) tasks, where the SNN using BRP has reached a similar accuracy compared to other state-of-the-art (SOTA) BP-based SNNs and saved 50% more computational cost than ANNs. We think that the introduction of biologically plausible learning rules to the training procedure of biologically realistic SNNs will give us more hints and inspiration toward a better understanding of the biological system's intelligent nature.

Spiking Adaptive Dynamic Programming Based on Poisson Process for Discrete-Time Nonlinear Systems.

In this article, a new iterative spiking adaptive dynamic programming (SADP) method based on the Poisson process is developed to solve optimal impulsive control problems. For a fixed time interval, combining the Poisson process and the maximum likelihood estimation (MLE), the three-tuple of state, spiking interval, and probability of Poisson distribution can be computed, and then, the iterative value functions and iterative control laws can be obtained. A property analysis method is developed to show that the value functions converge to optimal performance index function as the iterative index increases from zero to infinity. Finally, two simulation examples are given to verify the effectiveness of the developed algorithm.

Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks.

Many synaptic plasticity rules found in natural circuits have not been incorporated into artificial neural networks (ANNs). We showed that incorporating a nonlocal feature of synaptic plasticity found in natural neural networks, whereby synaptic modification at output synapses of a neuron backpropagates to its input synapses made by upstream neurons, markedly reduced the computational cost without affecting the accuracy of spiking neural networks (SNNs) and ANNs in supervised learning for three benchmark tasks. For SNNs, synaptic modification at output neurons generated by spike timing­dependent plasticity was allowed to self-propagate to limited upstream synapses. For ANNs, modified synaptic weights via conventional backpropagation algorithm at output neurons self-backpropagated to limited upstream synapses. Such self-propagating plasticity may produce coordinated synaptic modifications across neuronal layers that reduce computational cost.

Comparison Between Human and Rodent Neurons for Persistent Activity Performance: A Biologically Plausible Computational Investigation.

Elucidating the multi-scale detailed differences between the human brain and other brains will help shed light on what makes us unique as a species. Computational models help link biochemical and anatomical properties to cognitive functions and predict key properties of the cortex. Here, we present a detailed human neocortex network, with all human neuron parameters derived from the newest Allen Brain human brain cell database. Compared with that of rodents, the human neural network maintains more complete and accurate information under the same graphic input. Unique membrane properties in human neocortical neurons enhance the human brain's capacity for signal processing.

The Interactions Between Antibiotic Resistance Genes and Heavy Metal Pollution Under Co-Selective Pressure Influenced the Bio-Enzyme Activity.

The spread of antibiotic resistance genes (ARGs) has brought potential risks to public health. However, the interactions between heavy metals and ARGs, as well as their potential effect on bio-enzyme activity under the pressure of co-selectivity in soil still remain poorly understood. In this work, the distribution characteristics and the co-selective relationship of 28 ARGs and eight heavy metals in soil in a dairy farm were visualized via the geographic information system (GIS) technique. Eight kinds of heavy metals were detected by an atomic fluorescence spectrometer and atomic absorption spectrophotometer, which were further evaluated via the single factor pollution index value. The GIS analysis showed that arsenic (As) was the key element responsible for soil pollution, which was found to be positively related to soil depths. The top three comprehensive scores of ARGs ranked the orders of sul2 > tetX > blaTEM, indicating the high potential of risk caused by these genes in the soil environment. In addition, the functional predications performed with the 16 SrDNA sequencing data based on the KEGG database indicated that the sulfonamides in soil involved multiple pathways, especially the metabolism, transport and catabolism, and membrane transport processes. This suggested that most bio-enzymes were found to be expressed in low activities in different pathways. Significant correlations were observed between the heavy metals and ARGs (p < 0.05), particularly between the ARGs and As, Cu, Ni, Pb, and Zn (p < 0.01). This study offers deep insights into the potential interactions between heavy metals and ARGs in soil and provides guidance for the fabrication of enzyme-based smart materials for soil remediation in dairy farms.

Quantum superposition inspired spiking neural network.

Despite advances in artificial intelligence models, neural networks still cannot achieve human performance, partly due to differences in how information is encoded and processed compared with human brain. Information in an artificial neural network (ANN) is represented using a statistical method and processed as a fitting function, enabling handling of structural patterns in image, text, and speech processing. However, substantial changes to the statistical characteristics of the data, for example, reversing the background of an image, dramatically reduce the performance. Here, we propose a quantum superposition spiking neural network (QS-SNN) inspired by quantum mechanisms and phenomena in the brain, which can handle reversal of image background color. The QS-SNN incorporates quantum theory with brain-inspired spiking neural network models from a computational perspective, resulting in more robust performance compared with traditional ANN models, especially when processing noisy inputs. The results presented here will inform future efforts to develop brain-inspired artificial intelligence.

Neuron type classification in rat brain based on integrative convolutional and tree-based recurrent neural networks.

The study of cellular complexity in the nervous system based on anatomy has shown more practical and objective advantages in morphology than other perspectives on molecular, physiological, and evolutionary aspects. However, morphology-based neuron type classification in the whole rat brain is challenging, given the significant number of neuron types, limited reconstructed neuron samples, and diverse data formats. Here, we report that different types of deep neural network modules may well process different kinds of features and that the integration of these submodules will show power on the representation and classification of neuron types. For SWC-format data, which are compressed but unstructured, we construct a tree-based recurrent neural network (Tree-RNN) module. For 2D or 3D slice-format data, which are structured but with large volumes of pixels, we construct a convolutional neural network (CNN) module. We also generate a virtually simulated dataset with two classes, reconstruct a CASIA rat-neuron dataset with 2.6 million neurons without labels, and select the NeuroMorpho-rat dataset with 35,000 neurons containing hierarchical labels. In the twelve-class classification task, the proposed model achieves state-of-the-art performance compared with other models, e.g., the CNN, RNN, and support vector machine based on hand-designed features.

Neuronal-Plasticity and Reward-Propagation Improved Recurrent Spiking Neural Networks.

Different types of dynamics and plasticity principles found through natural neural networks have been well-applied on Spiking neural networks (SNNs) because of their biologically-plausible efficient and robust computations compared to their counterpart deep neural networks (DNNs). Here, we further propose a special Neuronal-plasticity and Reward-propagation improved Recurrent SNN (NRR-SNN). The historically-related adaptive threshold with two channels is highlighted as important neuronal plasticity for increasing the neuronal dynamics, and then global labels instead of errors are used as a reward for the paralleling gradient propagation. Besides, a recurrent loop with proper sparseness is designed for robust computation. Higher accuracy and stronger robust computation are achieved on two sequential datasets (i.e., TIDigits and TIMIT datasets), which to some extent, shows the power of the proposed NRR-SNN with biologically-plausible improvements.

GLSNN: A Multi-Layer Spiking Neural Network Based on Global Feedback Alignment and Local STDP Plasticity.

Spiking Neural Networks (SNNs) are considered as the third generation of artificial neural networks, which are more closely with information processing in biological brains. However, it is still a challenge for how to train the non-differential SNN efficiently and robustly with the form of spikes. Here we give an alternative method to train SNNs by biologically-plausible structural and functional inspirations from the brain. Firstly, inspired by the significant top-down structural connections, a global random feedback alignment is designed to help the SNN propagate the error target from the output layer directly to the previous few layers. Then inspired by the local plasticity of the biological system in which the synapses are more tuned by the neighborhood neurons, a differential STDP is used to optimize local plasticity. Extensive experimental results on the benchmark MNIST (98.62%) and Fashion MNIST (89.05%) have shown that the proposed algorithm performs favorably against several state-of-the-art SNNs trained with backpropagation.