Fixed-time synchronization of complex-valued neural networks for image protection and 3D point cloud information protection.

This paper studies the fixed-time synchronization (FDTS) of complex-valued neural networks (CVNNs) based on quantized intermittent control (QIC) and applies it to image protection and 3D point cloud information protection. A new controller was designed which achieved FDTS of the CVNNs, with the estimation of the convergence time not dependent on the initial state. Our approach divides the neural network into two real-valued systems and then combines the framework of the Lyapunov method to give criteria for FDTS. Applying synchronization to image protection, the image will be encrypted with a drive system sequence and decrypted with a response system sequence. The quality of image encryption and decryption depends on the synchronization error. Meanwhile, the depth image of the object is encrypted and then the 3D point cloud is reconstructed based on the decrypted depth image. This means that the 3D point cloud information is protected. Finally, simulation examples verify the efficacy of the controller and the synchronization criterion, giving results for applications in image protection and 3D point cloud information protection.

Self-paced regularized adaptive multi-view unsupervised feature selection.

Multi-view unsupervised feature selection (MUFS) is an efficient approach for dimensional reduction of heterogeneous data. However, existing MUFS approaches mostly assign the samples the same weight, thus the diversity of samples is not utilized efficiently. Additionally, due to the presence of various regularizations, the resulting MUFS problems are often non-convex, making it difficult to find the optimal solutions. To address this issue, a novel MUFS method named Self-paced Regularized Adaptive Multi-view Unsupervised Feature Selection (SPAMUFS) is proposed. Specifically, the proposed approach firstly trains the MUFS model with simple samples, and gradually learns complex samples by using self-paced regularizer. l2,p-norm (0

Pruning and quantization algorithm with applications in memristor-based convolutional neural network.

The human brain's ultra-low power consumption and highly parallel computational capabilities can be accomplished by memristor-based convolutional neural networks. However, with the rapid development of memristor-based convolutional neural networks in various fields, more complex applications and heavier computations lead to the need for a large number of memristors, which makes power consumption increase significantly and the network model larger. To mitigate this problem, this paper proposes an SBT-memristor-based convolutional neural network architecture and a hybrid optimization method combining pruning and quantization. Firstly, SBT-memristor-based convolutional neural network is constructed by using the good thresholding property of the SBT memristor. The memristive in-memory computing unit, activation unit and max-pooling unit are designed. Then, the hybrid optimization method combining pruning and quantization is used to improve the SBT-memristor-based convolutional neural network architecture. This hybrid method can simplify the memristor-based neural network and represent the weights at the memristive synapses better. Finally, the results show that the SBT-memristor-based convolutional neural network reduces a large number of memristors, decreases the power consumption and compresses the network model at the expense of a little precision loss. The SBT-memristor-based convolutional neural network obtains faster recognition speed and lower power consumption in MNIST recognition. It provides new insights for the complex application of convolutional neural networks.

Fault-Tolerant Synchronization for Memristive Neural Networks With Multiple Actuator Failures.

By using the fault-tolerant control method, the synchronization of memristive neural networks (MNNs) subjected to multiple actuator failures is investigated in this article. The considered actuator failures include the effectiveness failure and the lock-in-place failure, which are different from previous results. First of all, the mathematical expression of the control inputs in the considered system is given by introducing the models of the above two types of actuator failures. Following, two classes of synchronization strategies, which are state feedback control strategies and event-triggered control strategies, are proposed by using some inequality techniques and Lyapunov stability theories. The designed controllers can, respectively, guarantee the realization of synchronizations of the global exponential, the finite-time and the fixed-time for the MNNs by selecting different parameter conditions. Then the estimations of settling times of provided synchronization schemes are computed and the Zeno phenomenon of proposed event-triggered strategies is explicitly excluded. Finally, two experiments are conducted to confirm the availability of given synchronization strategies.

Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions.

Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly (2m + 3)n equilibria, where (m + 2)n equilibria are locally exponentially stable, the remainder (2m + 3)n - (m + 2)n equilibria are unstable, and a positive integer m is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.

Finite-Time Stabilization of Inertial Memristive Neural Networks via Nonreduced Order Method.

This article investigates the finite-time stabilization problem of inertial memristive neural networks (IMNNs) with bounded and unbounded time-varying delays, respectively. To simplify the theoretical derivation, the nonreduced order method is utilized for constructing appropriate comparison functions and designing a discontinuous state feedback controller. Then, based on the controller, the state of IMNNs can directly converge to 0 in finite time. Several criteria for finite-time stabilization of IMNNs are obtained and the setting time is estimated. Compared with previous studies, the requirement of differentiability of time delay is eliminated. Finally, numerical examples illustrate the usefulness of the analysis results in this article.

Neural Network Circuits for Bionic Associative Memory and Temporal Order Memory Based on DNA Strand Displacement.

Pavlovian associative memory plays an important role in our daily life and work. The realization of Pavlovian associative memory at the deoxyribonucleic acid (DNA) molecular level will promote the development of biological computing and broaden the application scenarios of neural networks. In this article, bionic associative memory and temporal order memory circuits are constructed by DNA strand displacement (DSD) reactions. First, a temporal logic gate is constructed on the basis of DSD circuit and extended to a three-input temporal logic gate. The output of temporal logic gate is used for the weight species of associative memory. Second, the forgetting module and output module based on the DSD circuit are constructed to realize some functions of associative memory, including associative memory with simultaneous stimulus, associative memory with interstimulus interval effect, and the facilitation by intermittent stimulus. In addition, the coding, storage, and retrieval modules are designed based on the analysis and memory capabilities of temporal logic gate for temporal information. The temporal order memory circuit is constructed, demonstrating the temporal order memory ability of DNA circuit. Finally, the reliability of the circuit is verified through Visual DSD software simulation. Our work provides ideas and inspiration to construct more complex DNA bionic circuits and intelligent circuits by using DSD technology.

Operant Conditioning Neuromorphic Circuit With Addictiveness and Time Memory for Automatic Learning.

Most operant conditioning circuits predominantly focus on simple feedback process, few studies consider the intricacies of feedback outcomes and the uncertainty of feedback time. This paper proposes a neuromorphic circuit based on operant conditioning with addictiveness and time memory for automatic learning. The circuit is mainly composed of hunger output module, neuron module, excitement output module, memristor-based decision module, and memory and feedback generation module. In the circuit, the process of output excitement and addiction in stochastic feedback is achieved. The memory of interval between the two rewards is formed. The circuit can adapt to complex scenarios with these functions. In addition, hunger and satiety are introduced to realize the interaction between biological behavior and exploration desire, which enables the circuit to continuously reshape its memories and actions. The process of operant conditioning theory for automatic learning is accomplished. The study of operant conditioning can serve as a reference for more intelligent brain-inspired neural systems.

Unified analysis on multistablity of fraction-order multidimensional-valued memristive neural networks.

This article provides a unified analysis of the multistability of fraction-order multidimensional-valued memristive neural networks (FOMVMNNs) with unbounded time-varying delays. Firstly, based on the knowledge of fractional differentiation and memristors, a unified model is established. This model is a unified form of real-valued, complex-valued, and quaternion-valued systems. Then, based on a unified method, the number of equilibrium points for FOMVMNNs is discussed. The sufficient conditions for determining the number of equilibrium points have been obtained. By using 1-norm to construct Lyapunov functions, the unified criteria for multistability of FOMVMNNs are obtained, these criteria are less conservative and easier to verify. Moreover, the attraction basins of the stable equilibrium points are estimated. Finally, two numerical simulation examples are provided to verify the correctness of the results.

Input-to-state stability of delayed memristor-based inertial neural networks via non-reduced order method.

This paper is concerned with the input-to-state stability (ISS) for a kind of delayed memristor-based inertial neural networks (DMINNs). Based on the nonsmooth analysis and stability theory, novel delay-dependent and delay-independent criteria on the ISS of DMINNs are obtained by constructing different Lyapunov functions. Moreover, compared with the reduced order approach used in the previous works, this paper consider the ISS of DMINNs via non-reduced order approach. Directly analysis the model of DMINNs can better maintain its physical backgrounds, which reduces the complexity of calculations and is more rigorous in practical application. Additionally, the novel proposed results on the ISS of DMINNs here incorporate and complement the existing studies on memristive neural network dynamical systems. Lastly, a numerical example is provided to show that the obtained criteria are reliable.

Residual Temporal Convolutional Network With Dual Attention Mechanism for Multilead-Time Interpretable Runoff Forecasting.

As a pivotal subfield within the domain of time series forecasting, runoff forecasting plays a crucial role in water resource management and scheduling. Recent advancements in the application of artificial neural networks (ANNs) and attention mechanisms have markedly enhanced the accuracy of runoff forecasting models. This article introduces an innovative hybrid model, ResTCN-DAM, which synergizes the strengths of deep residual network (ResNet), temporal convolutional networks (TCNs), and dual attention mechanisms (DAMs). The proposed ResTCN-DAM is designed to leverage the unique attributes of these three modules: TCN has outstanding capability to process time series data in parallel. By combining with modified ResNet, multiple TCN layers can be densely stacked to capture more hidden information in the temporal dimension. DAM module adeptly captures the interdependencies within both temporal and feature dimensions, adeptly accentuating relevant time steps/features while diminishing less significant ones with minimal computational cost. Furthermore, the snapshot ensemble method is able to obtain the effect of training multiple models through one single training process, which ensures the accuracy and robustness of the forecasts. The deep integration and collaborative cooperation of these modules comprehensively enhance the model's forecasting capability from various perspectives. Ablation studies conducted validate the efficacy of each module, and through multiple sets of comparative experiments, it is shown that the proposed ResTCN-DAM has exceptional and consistent performance across varying lead times. We also employ visualization techniques to display heatmaps of the model's weights, thereby enhancing the interpretability of the model. When compared with the prevailing neural network-based runoff forecasting models, ResTCN-DAM exhibits state-of-the-art accuracy, temporal robustness, and interpretability, positioning it at the forefront of contemporary research.

A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures.

In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel analog memristive platforms are prominent for their ability to generate multiple feature maps in a single processing cycle. However, a notable limitation is that they are specifically tailored for neural networks with fixed structures. As an orthogonal direction, recent research reveals that neural architecture should be specialized for tasks and deployment platforms. Building upon this, the neural architecture search (NAS) methods effectively explore promising architectures in a large design space. However, these NAS-based architectures are generally heterogeneous and diversified, making it challenging for deployment on current single-prototype, customized, parallel analog memristive hardware circuits. Therefore, investigating memristive analog deployment that overrides the full search space is a promising and challenging problem. Inspired by this, and beginning with the DARTS search space, we study the memristive hardware design of primitive operations and propose the memristive all-inclusive hypernetwork that covers 2×1025 network architectures. Our computational simulation results on 3 representative architectures (DARTS-V1, DARTS-V2, PDARTS) show that our memristive all-inclusive hypernetwork achieves promising results on the CIFAR10 dataset (89.2% of PDARTS with 8-bit quantization precision), and is compatible with all architectures in the DARTS full-space. The hardware performance simulation indicates that the memristive all-inclusive hypernetwork costs slightly more resource consumption (nearly the same in power, 22%∼25% increase in Latency, 1.5× in Area) relative to the individual deployment, which is reasonable and may reach a tolerable trade-off deployment scheme for industrial scenarios.