Direct training high-performance deep spiking neural networks: a review of theories and methods.

Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks (ANNs), in virtue of their high biological plausibility, rich spatial-temporal dynamics, and event-driven computation. The direct training algorithms based on the surrogate gradient method provide sufficient flexibility to design novel SNN architectures and explore the spatial-temporal dynamics of SNNs. According to previous studies, the performance of models is highly dependent on their sizes. Recently, direct training deep SNNs have achieved great progress on both neuromorphic datasets and large-scale static datasets. Notably, transformer-based SNNs show comparable performance with their ANN counterparts. In this paper, we provide a new perspective to summarize the theories and methods for training deep SNNs with high performance in a systematic and comprehensive way, including theory fundamentals, spiking neuron models, advanced SNN models and residual architectures, software frameworks and neuromorphic hardware, applications, and future trends.

Giant Second Harmonic Generation in Supertwisted WS2 Spirals Grown in Step-Edge Particle-Induced Non-Euclidean Surfaces.

In moiré crystals resulting from the stacking of twisted two-dimensional (2D) layered materials, a subtle adjustment in the twist angle surprisingly gives rise to a wide range of correlated optical and electrical properties. Herein, we report the synthesis of supertwisted WS2 spirals and the observation of giant second harmonic generation (SHG) in these spirals. Supertwisted WS2 spirals featuring different twist angles are synthesized on a Euclidean or step-edge particle-induced non-Euclidean surface using carefully designed water-assisted chemical vapor deposition. We observed an oscillatory dependence of SHG intensity on layer number, attributed to atomically phase-matched nonlinear dipoles within layers of supertwisted spiral crystals where inversion symmetry is restored. Through an investigation into the twist angle evolution of SHG intensity, we discovered that the stacking model between layers plays a crucial role in determining the nonlinearity, and the SHG signals in supertwisted spirals exhibit enhancements by a factor of 2 to 136 when compared with the SHG of the single-layer structure. These findings provide helpful perspectives on the rational growth of 2D twisted structures and the implementation of twist angle adjustable endowing them great potential for exploring strong coupling correlation physics and applications in the field of twistronics.

Spiking Tucker Fusion Transformer for Audio-Visual Zero-Shot Learning.

The spiking neural networks (SNNs) that efficiently encode temporal sequences have shown great potential in extracting audio-visual joint feature representations. However, coupling SNNs (binary spike sequences) with transformers (float-point sequences) to jointly explore the temporal-semantic information still facing challenges. In this paper, we introduce a novel Spiking Tucker Fusion Transformer (STFT) for audio-visual zero-shot learning (ZSL). The STFT leverage the temporal and semantic information from different time steps to generate robust representations. The time-step factor (TSF) is introduced to dynamically synthesis the subsequent inference information. To guide the formation of input membrane potentials and reduce the spike noise, we propose a global-local pooling (GLP) which combines the max and average pooling operations. Furthermore, the thresholds of the spiking neurons are dynamically adjusted based on semantic and temporal cues. Integrating the temporal and semantic information extracted by SNNs and Transformers are difficult due to the increased number of parameters in a straightforward bilinear model. To address this, we introduce a temporal-semantic Tucker fusion module, which achieves multi-scale fusion of SNN and Transformer outputs while maintaining full second-order interactions. Our experimental results demonstrate the effectiveness of the proposed approach in achieving state-of-the-art performance in three benchmark datasets. The harmonic mean (HM) improvement of VGGSound, UCF101 and ActivityNet are around 15.4%, 3.9%, and 14.9%, respectively.

Preparation of chitosan-hyaluronic acid microcapsules and its dynamic release behavior analysis in a 3D-printed microchannel system: Exploration and verification.

This research focuses on the challenges of efficiently constructing drug carriers and evaluating their dynamic release in vitro simulation. By using pickering emulsion and layer-by-layer self-assembly methods. The microcapsules had tea tree oil as the core material, SiO2 nanoparticles as stabilizers, and chitosan and hyaluronic acid as shell materials. The microencapsulation mechanism, as well as the effects of core-shell mass ratio and stirring, were discussed. Specifically, a dynamic circulation simulation microchannel system was designed and manufactured based on 3D printing technology. In this simulation system, the release rate of microcapsules is accelerated and the trend changes, with its behavior aligning with the Boltzmann model. The study demonstrates the advantages of self-assembled inorganic-organic drug-loaded microcapsules in terms of controllable fabrication and ease of functional modification, and shows the potential of 3D printed cyclic microchannel systems in terms of operability and simulation fidelity in drug and physiological analysis.

GeneWorker: An end-to-end robotic reinforcement learning approach with collaborative generator and worker networks.

Reinforcement learning aided by the skill conception exhibits potent capabilities in guiding autonomous agents toward acquiring meaningful behaviors. However, in the current landscape of reinforcement learning, a skill is often merely a rudimentary abstraction of a sequence of primitive actions, serving as a component of the input to policy networks with fixed network parameters. This rigid methodology presents obstacles when attempting to integrate with burgeoning techniques such as meta-learning and large language models. To address this issue, we introduce a unique neural skill representation that abstracts the activation of neurons in each neural layer. Based on this, a novel end-to-end robotic reinforcement learning algorithm is proposed, in which two sub-networks, i.e., generator and worker networks, implement collaborative inferences via neural skills. Specifically, the generator produces a series of multi-spatial neural skills, providing efficient guidance for subsequent decision-making; by integrating these skills, the worker can determine its own network weights and biases to cope with various environmental conditions. Therefore, actions can be sampled with flexibly changeable network parameters through the collaboration between generator and worker networks. The experiments demonstrate that GeneWorker can achieve a mean success rate of over 90.67% on continuous robotic tasks and outperforms previous state-of-the-art methods by a minimum of 54% on the pick-and-place task.

Learning a Deep Demosaicing Network for Spike Camera With Color Filter Array.

For capturing dynamic scenes with ultra-fast motion, neuromorphic cameras with extremely high temporal resolution have demonstrated their great capability and potential. Different from the event cameras that only record relative changes in light intensity, spike camera fires a stream of spikes according to a full-time accumulation of photons so that it can recover the texture details for both static areas and dynamic areas. Recently, color spike camera has been invented to record color information of dynamic scenes using a color filter array (CFA). However, demosaicing for color spike cameras is an open and challenging problem. In this paper, we develop a demosaicing network, called CSpkNet, to reconstruct dynamic color visual signals from the spike stream captured by the color spike camera. Firstly, we develop a light inference module to convert binary spike streams to intensity estimates. In particular, a feature-based channel attention module is proposed to reduce the noises caused by quantization errors. Secondly, considering both the Bayer configuration and object motion, we propose a motion-guided filtering module to estimate the missing pixels of each color channel, without undesired motion blur. Finally, we design a refinement module to improve the intensity and details, utilizing the color correlation. Experimental results demonstrate that CSpkNet can reconstruct color images from the Bayer-pattern spike stream with promising visual quality.

Anomalous layer-dependent photoluminescence spectra of supertwisted spiral WS2.

Twisted stacking of two-dimensional materials with broken inversion symmetry, such as spiral MoTe2 nanopyramids and supertwisted spiral WS2, emerge extremely strong second- and third-harmonic generation. Unlike well-studied nonlinear optical effects in these newly synthesized layered materials, photoluminescence (PL) spectra and exciton information involving their optoelectronic applications remain unknown. Here, we report layer- and power-dependent PL spectra of the supertwisted spiral WS2. The anomalous layer-dependent PL evolutions that PL intensity almost linearly increases with the rise of layer thickness have been determined. Furthermore, from the power-dependent spectra, we find the power exponents of the supertwisted spiral WS2 are smaller than 1, while those of the conventional multilayer WS2 are bigger than 1. These two abnormal phenomena indicate the enlarged interlayer spacing and the decoupling interlayer interaction in the supertwisted spiral WS2. These observations provide insight into PL features in the supertwisted spiral materials and may pave the way for further optoelectronic devices based on the twisted stacking materials.

SGLFormer: Spiking Global-Local-Fusion Transformer with high performance.

Introduction: Spiking Neural Networks (SNNs), inspired by brain science, offer low energy consumption and high biological plausibility with their event-driven nature. However, the current SNNs are still suffering from insufficient performance. Methods: Recognizing the brain's adeptness at information processing for various scenarios with complex neuronal connections within and across regions, as well as specialized neuronal architectures for specific functions, we propose a Spiking Global-Local-Fusion Transformer (SGLFormer), that significantly improves the performance of SNNs. This novel architecture enables efficient information processing on both global and local scales, by integrating transformer and convolution structures in SNNs. In addition, we uncover the problem of inaccurate gradient backpropagation caused by Maxpooling in SNNs and address it by developing a new Maxpooling module. Furthermore, we adopt spatio-temporal block (STB) in the classification head instead of global average pooling, facilitating the aggregation of spatial and temporal features. Results: SGLFormer demonstrates its superior performance on static datasets such as CIFAR10/CIFAR100, and ImageNet, as well as dynamic vision sensor (DVS) datasets including CIFAR10-DVS and DVS128-Gesture. Notably, on ImageNet, SGLFormer achieves a top-1 accuracy of 83.73% with 64 M parameters, outperforming the current SOTA directly trained SNNs by a margin of 6.66%. Discussion: With its high performance, SGLFormer can support more computer vision tasks in the future. The codes for this study can be found in https://github.com/ZhangHanN1/SGLFormer.

Identification of important gene signatures in schizophrenia through feature fusion and genetic algorithm.

Schizophrenia is a debilitating psychiatric disorder that can significantly affect a patient's quality of life and lead to permanent brain damage. Although medical research has identified certain genetic risk factors, the specific pathogenesis of the disorder remains unclear. Despite the prevalence of research employing magnetic resonance imaging, few studies have focused on the gene level and gene expression profile involving a large number of screened genes. However, the high dimensionality of genetic data presents a great challenge to accurately modeling the data. To tackle the current challenges, this study presents a novel feature selection strategy that utilizes heuristic feature fusion and a multi-objective optimization genetic algorithm. The goal is to improve classification performance and identify the key gene subset for schizophrenia diagnostics. Traditional gene screening techniques are inadequate for accurately determining the precise number of key genes associated with schizophrenia. Our innovative approach integrates a filter-based feature selection method to reduce data dimensionality and a multi-objective optimization genetic algorithm for improved classification tasks. By combining the filtering and wrapper methods, our strategy leverages their respective strengths in a deliberate manner, leading to superior classification accuracy and a more efficient selection of relevant genes. This approach has demonstrated significant improvements in classification results across 11 out of 14 relevant datasets. The performance on the remaining three datasets is comparable to the existing methods. Furthermore, visual and enrichment analyses have confirmed the practicality of our proposed method as a promising tool for the early detection of schizophrenia.

Mesoporous and Encapsulated In2O3/Ti3C2Tx Schottky Heterojunctions for Rapid and ppb-Level NO2 Detection at Room Temperature.

Rapid and ultrasensitive detection of toxic gases at room temperature is highly desired in health protection but presents grand challenges in the sensing materials reported so far. Here, we present a gas sensor based on novel zero dimensional (0D)/two dimensional (2D) indium oxide (In2O3)/titanium carbide (Ti3C2Tx) Schottky heterostructures with a high surface area and rich oxygen vacancies for parts per billion (ppb) level nitrogen dioxide (NO2) detection at room temperature. The In2O3/Ti3C2Tx gas sensor exhibits a fast response time (4 s), good response (193.45% to 250 ppb NO2), high selectivity, and excellent cycling stability. The rich surface oxygen vacancies play the role of active sites for the adsorption of NO2 molecules, and the Schottky junctions effectively adjust the charge-transfer behavior through the conduction tunnel in the sensing material. Furthermore, In2O3 nanoparticles almost fully cover the Ti3C2Tx nanosheets which can avoid the oxidation of Ti3C2Tx, thus contributing to the good cycling stability of the sensing materials. This work sheds light on the sensing mechanism of heterojunction nanostructures and provides an efficient pathway to construct high-performance gas sensors through the rational design of active sites.