A Novel Adaptive Deskewing Algorithm for Document Images.

Document scanning often suffers from skewing, which may seriously influence the efficiency of Optical Character Recognition (OCR). Therefore, it is necessary to correct the skewed document before document image information analysis. In this article, we propose a novel adaptive deskewing algorithm for document images, which mainly includes Skeleton Line Detection (SKLD), Piecewise Projection Profile (PPP), Morphological Clustering (MC), and the image classification method. The image type is determined firstly based on the image's layout feature. Thus, adaptive correcting is applied to deskew the image according to its type. Our method maintains high accuracy on the Document Image Skew Estimation Contest (DISEC'2013) and PubLayNet datasets, which achieved 97.6% and 80.1% accuracy, respectively. Meanwhile, extensive experiments show the superiority of the proposed algorithm.

Saturation function-based continuous control on fixed-time synchronization of competitive neural networks.

Currently, through proposing discontinuous control strategies with the signum function and discussing separately short-term memory (STM) and long-term memory (LTM) of competitive artificial neural networks (ANNs), the fixed-time (FXT) synchronization of competitive ANNs has been explored. Note that the method of separate analysis usually leads to complicated theoretical derivation and synchronization conditions, and the signum function inevitably causes the chattering to reduce the performance of the control schemes. To try to solve these challenging problems, the FXT synchronization issue is concerned in this paper for competitive ANNs by establishing a theorem of FXT stability with switching type and developing continuous control schemes based on a kind of saturation functions. Firstly, different from the traditional method of studying separately STM and LTM of competitive ANNs, the models of STM and LTM are compressed into a high-dimensional system so as to reduce the complexity of theoretical analysis. Additionally, as an important theoretical preliminary, a FXT stability theorem with switching differential conditions is established and some high-precision estimates for the convergence time are explicitly presented by means of several special functions. To achieve FXT synchronization of the addressed competitive ANNs, a type of continuous pure power-law control scheme is developed via introducing the saturation function instead of the signum function, and some synchronization criteria are further derived by the established FXT stability theorem. These theoretical results are further illustrated lastly via a numerical example and are applied to image encryption.

Self-paced regularized adaptive multi-view unsupervised feature selection.

Multi-view unsupervised feature selection (MUFS) is an efficient approach for dimensional reduction of heterogeneous data. However, existing MUFS approaches mostly assign the samples the same weight, thus the diversity of samples is not utilized efficiently. Additionally, due to the presence of various regularizations, the resulting MUFS problems are often non-convex, making it difficult to find the optimal solutions. To address this issue, a novel MUFS method named Self-paced Regularized Adaptive Multi-view Unsupervised Feature Selection (SPAMUFS) is proposed. Specifically, the proposed approach firstly trains the MUFS model with simple samples, and gradually learns complex samples by using self-paced regularizer. l2,p-norm (0

Fixed-time synchronization of complex-valued neural networks for image protection and 3D point cloud information protection.

This paper studies the fixed-time synchronization (FDTS) of complex-valued neural networks (CVNNs) based on quantized intermittent control (QIC) and applies it to image protection and 3D point cloud information protection. A new controller was designed which achieved FDTS of the CVNNs, with the estimation of the convergence time not dependent on the initial state. Our approach divides the neural network into two real-valued systems and then combines the framework of the Lyapunov method to give criteria for FDTS. Applying synchronization to image protection, the image will be encrypted with a drive system sequence and decrypted with a response system sequence. The quality of image encryption and decryption depends on the synchronization error. Meanwhile, the depth image of the object is encrypted and then the 3D point cloud is reconstructed based on the decrypted depth image. This means that the 3D point cloud information is protected. Finally, simulation examples verify the efficacy of the controller and the synchronization criterion, giving results for applications in image protection and 3D point cloud information protection.

A Distributed k-Winners-Take-All Model With Binary Consensus Protocols.

This article concentrates on solving the k -winners-take-all (k WTA) problem with large-scale inputs in a distributed setting. We propose a multiagent system with a relatively simple structure, in which each agent is equipped with a 1-D system and interacts with others via binary consensus protocols. That is, only the signs of the relative state information between neighbors are required. By virtue of differential inclusion theory, we prove that the system converges from arbitrary initial states. In addition, we derive the convergence rate as O(1/t) . Furthermore, in comparison to the existing models, we introduce a novel comparison filter to eliminate the resolution ratio requirement on the input signal, that is, the difference between the k th and (k+1) th largest inputs must be larger than a positive threshold. As a result, the proposed distributed k WTA model is capable of solving the k WTA problem, even when more than two elements of the input signal share the same value. Finally, we validate the effectiveness of the theoretical results through two simulation examples.

Unified analysis on multistablity of fraction-order multidimensional-valued memristive neural networks.

This article provides a unified analysis of the multistability of fraction-order multidimensional-valued memristive neural networks (FOMVMNNs) with unbounded time-varying delays. Firstly, based on the knowledge of fractional differentiation and memristors, a unified model is established. This model is a unified form of real-valued, complex-valued, and quaternion-valued systems. Then, based on a unified method, the number of equilibrium points for FOMVMNNs is discussed. The sufficient conditions for determining the number of equilibrium points have been obtained. By using 1-norm to construct Lyapunov functions, the unified criteria for multistability of FOMVMNNs are obtained, these criteria are less conservative and easier to verify. Moreover, the attraction basins of the stable equilibrium points are estimated. Finally, two numerical simulation examples are provided to verify the correctness of the results.

Finite-Time Stabilization of Inertial Memristive Neural Networks via Nonreduced Order Method.

This article investigates the finite-time stabilization problem of inertial memristive neural networks (IMNNs) with bounded and unbounded time-varying delays, respectively. To simplify the theoretical derivation, the nonreduced order method is utilized for constructing appropriate comparison functions and designing a discontinuous state feedback controller. Then, based on the controller, the state of IMNNs can directly converge to 0 in finite time. Several criteria for finite-time stabilization of IMNNs are obtained and the setting time is estimated. Compared with previous studies, the requirement of differentiability of time delay is eliminated. Finally, numerical examples illustrate the usefulness of the analysis results in this article.

Input-to-state stability of delayed memristor-based inertial neural networks via non-reduced order method.

This paper is concerned with the input-to-state stability (ISS) for a kind of delayed memristor-based inertial neural networks (DMINNs). Based on the nonsmooth analysis and stability theory, novel delay-dependent and delay-independent criteria on the ISS of DMINNs are obtained by constructing different Lyapunov functions. Moreover, compared with the reduced order approach used in the previous works, this paper consider the ISS of DMINNs via non-reduced order approach. Directly analysis the model of DMINNs can better maintain its physical backgrounds, which reduces the complexity of calculations and is more rigorous in practical application. Additionally, the novel proposed results on the ISS of DMINNs here incorporate and complement the existing studies on memristive neural network dynamical systems. Lastly, a numerical example is provided to show that the obtained criteria are reliable.

Fault-Tolerant Synchronization for Memristive Neural Networks With Multiple Actuator Failures.

By using the fault-tolerant control method, the synchronization of memristive neural networks (MNNs) subjected to multiple actuator failures is investigated in this article. The considered actuator failures include the effectiveness failure and the lock-in-place failure, which are different from previous results. First of all, the mathematical expression of the control inputs in the considered system is given by introducing the models of the above two types of actuator failures. Following, two classes of synchronization strategies, which are state feedback control strategies and event-triggered control strategies, are proposed by using some inequality techniques and Lyapunov stability theories. The designed controllers can, respectively, guarantee the realization of synchronizations of the global exponential, the finite-time and the fixed-time for the MNNs by selecting different parameter conditions. Then the estimations of settling times of provided synchronization schemes are computed and the Zeno phenomenon of proposed event-triggered strategies is explicitly excluded. Finally, two experiments are conducted to confirm the availability of given synchronization strategies.

Neural Network Circuits for Bionic Associative Memory and Temporal Order Memory Based on DNA Strand Displacement.

Pavlovian associative memory plays an important role in our daily life and work. The realization of Pavlovian associative memory at the deoxyribonucleic acid (DNA) molecular level will promote the development of biological computing and broaden the application scenarios of neural networks. In this article, bionic associative memory and temporal order memory circuits are constructed by DNA strand displacement (DSD) reactions. First, a temporal logic gate is constructed on the basis of DSD circuit and extended to a three-input temporal logic gate. The output of temporal logic gate is used for the weight species of associative memory. Second, the forgetting module and output module based on the DSD circuit are constructed to realize some functions of associative memory, including associative memory with simultaneous stimulus, associative memory with interstimulus interval effect, and the facilitation by intermittent stimulus. In addition, the coding, storage, and retrieval modules are designed based on the analysis and memory capabilities of temporal logic gate for temporal information. The temporal order memory circuit is constructed, demonstrating the temporal order memory ability of DNA circuit. Finally, the reliability of the circuit is verified through Visual DSD software simulation. Our work provides ideas and inspiration to construct more complex DNA bionic circuits and intelligent circuits by using DSD technology.

Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions.

Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly (2m + 3)n equilibria, where (m + 2)n equilibria are locally exponentially stable, the remainder (2m + 3)n - (m + 2)n equilibria are unstable, and a positive integer m is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.

Pruning and quantization algorithm with applications in memristor-based convolutional neural network.

The human brain's ultra-low power consumption and highly parallel computational capabilities can be accomplished by memristor-based convolutional neural networks. However, with the rapid development of memristor-based convolutional neural networks in various fields, more complex applications and heavier computations lead to the need for a large number of memristors, which makes power consumption increase significantly and the network model larger. To mitigate this problem, this paper proposes an SBT-memristor-based convolutional neural network architecture and a hybrid optimization method combining pruning and quantization. Firstly, SBT-memristor-based convolutional neural network is constructed by using the good thresholding property of the SBT memristor. The memristive in-memory computing unit, activation unit and max-pooling unit are designed. Then, the hybrid optimization method combining pruning and quantization is used to improve the SBT-memristor-based convolutional neural network architecture. This hybrid method can simplify the memristor-based neural network and represent the weights at the memristive synapses better. Finally, the results show that the SBT-memristor-based convolutional neural network reduces a large number of memristors, decreases the power consumption and compresses the network model at the expense of a little precision loss. The SBT-memristor-based convolutional neural network obtains faster recognition speed and lower power consumption in MNIST recognition. It provides new insights for the complex application of convolutional neural networks.

Operant Conditioning Neuromorphic Circuit With Addictiveness and Time Memory for Automatic Learning.

Most operant conditioning circuits predominantly focus on simple feedback process, few studies consider the intricacies of feedback outcomes and the uncertainty of feedback time. This paper proposes a neuromorphic circuit based on operant conditioning with addictiveness and time memory for automatic learning. The circuit is mainly composed of hunger output module, neuron module, excitement output module, memristor-based decision module, and memory and feedback generation module. In the circuit, the process of output excitement and addiction in stochastic feedback is achieved. The memory of interval between the two rewards is formed. The circuit can adapt to complex scenarios with these functions. In addition, hunger and satiety are introduced to realize the interaction between biological behavior and exploration desire, which enables the circuit to continuously reshape its memories and actions. The process of operant conditioning theory for automatic learning is accomplished. The study of operant conditioning can serve as a reference for more intelligent brain-inspired neural systems.

Residual Temporal Convolutional Network With Dual Attention Mechanism for Multilead-Time Interpretable Runoff Forecasting.

As a pivotal subfield within the domain of time series forecasting, runoff forecasting plays a crucial role in water resource management and scheduling. Recent advancements in the application of artificial neural networks (ANNs) and attention mechanisms have markedly enhanced the accuracy of runoff forecasting models. This article introduces an innovative hybrid model, ResTCN-DAM, which synergizes the strengths of deep residual network (ResNet), temporal convolutional networks (TCNs), and dual attention mechanisms (DAMs). The proposed ResTCN-DAM is designed to leverage the unique attributes of these three modules: TCN has outstanding capability to process time series data in parallel. By combining with modified ResNet, multiple TCN layers can be densely stacked to capture more hidden information in the temporal dimension. DAM module adeptly captures the interdependencies within both temporal and feature dimensions, adeptly accentuating relevant time steps/features while diminishing less significant ones with minimal computational cost. Furthermore, the snapshot ensemble method is able to obtain the effect of training multiple models through one single training process, which ensures the accuracy and robustness of the forecasts. The deep integration and collaborative cooperation of these modules comprehensively enhance the model's forecasting capability from various perspectives. Ablation studies conducted validate the efficacy of each module, and through multiple sets of comparative experiments, it is shown that the proposed ResTCN-DAM has exceptional and consistent performance across varying lead times. We also employ visualization techniques to display heatmaps of the model's weights, thereby enhancing the interpretability of the model. When compared with the prevailing neural network-based runoff forecasting models, ResTCN-DAM exhibits state-of-the-art accuracy, temporal robustness, and interpretability, positioning it at the forefront of contemporary research.

A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures.

In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel analog memristive platforms are prominent for their ability to generate multiple feature maps in a single processing cycle. However, a notable limitation is that they are specifically tailored for neural networks with fixed structures. As an orthogonal direction, recent research reveals that neural architecture should be specialized for tasks and deployment platforms. Building upon this, the neural architecture search (NAS) methods effectively explore promising architectures in a large design space. However, these NAS-based architectures are generally heterogeneous and diversified, making it challenging for deployment on current single-prototype, customized, parallel analog memristive hardware circuits. Therefore, investigating memristive analog deployment that overrides the full search space is a promising and challenging problem. Inspired by this, and beginning with the DARTS search space, we study the memristive hardware design of primitive operations and propose the memristive all-inclusive hypernetwork that covers 2×1025 network architectures. Our computational simulation results on 3 representative architectures (DARTS-V1, DARTS-V2, PDARTS) show that our memristive all-inclusive hypernetwork achieves promising results on the CIFAR10 dataset (89.2% of PDARTS with 8-bit quantization precision), and is compatible with all architectures in the DARTS full-space. The hardware performance simulation indicates that the memristive all-inclusive hypernetwork costs slightly more resource consumption (nearly the same in power, 22%∼25% increase in Latency, 1.5× in Area) relative to the individual deployment, which is reasonable and may reach a tolerable trade-off deployment scheme for industrial scenarios.

Multiple Mittag-Leffler Stability of Fractional-Order Complex-Valued Memristive Neural Networks With Delays.

This article discusses the coexistence and dynamical behaviors of multiple equilibrium points (Eps) for fractional-order complex-valued memristive neural networks (FCVMNNs) with delays. First, based on the state space partition method, some sufficient conditions are proposed to guarantee that there are multiple Eps in one FCVMNN. Then, the Mittag-Leffler stability of those multiple Eps is proved by using the Lyapunov function. Simultaneously, the enlarged attraction basins are obtained to improve and extend the existing theoretical results in the previous literature. In addition, some existing stability results in the literature are special cases of a new result herein. Finally, two illustrative examples with computer simulations are presented to verify the effectiveness of theoretical analysis.

Cluster Synchronization of Multiple Fractional-Order Recurrent Neural Networks With Time-Varying Delays.

This article focuses on the cluster synchronization of multiple fractional-order recurrent neural networks (FNNs) with time-varying delays. Sufficient criteria are deduced for realizing cluster synchronization of multiple FNNs via a pinning control by applying an extended Halanay inequality applicable for time-delayed fractional-order differential equations. Moreover, an adaptive control applicable for the synchronization of fractional-order systems with time-varying delays is proposed, under which sufficient criteria are derived for realizing cluster synchronization of multiple FNNs with time-varying delays. Finally, two examples are presented to illustrate the effectiveness of the theoretical results.

Multiobjective Reinforcement Learning-Based Neural Architecture Search for Efficient Portrait Parsing.

This article dedicates to automatically explore efficient portrait parsing models that are easily deployed in edge computing or terminal devices. In the interest of the tradeoff between the resource cost and performance, we design the multiobjective reinforcement learning (RL)-based neural architecture search (NAS) scheme, which comprehensively balances the accuracy, parameters, FLOPs, and inference latency. Finally, under varying hyperparameter configurations, the search procedure emits a bunch of excellent objective-oriented architectures. The combination of two-stage training with precomputing and memory-resident feature maps effectively reduces the time consumption of the RL-based NAS method, so that we complete approximately 1000 search iterations in two GPU days. To accelerate the convergence of the lightweight candidate architecture, we incorporate knowledge distillation into the training of the search process. This also provides a reasonable evaluation signal to the RL controller that enables it to converge well. In the end, we conduct full training with outstanding Pareto-optimal architectures, so that a series of excellent portrait parsing models (with only approximately 0.3M parameters) is received. Furthermore, we directly transfer the architectures searched on CelebAMask-HQ (Portrait Parsing) to other portrait and face segmentation tasks. Finally, we achieve the state-of-the-art performance of 96.5% MIOU on EG1800 (portrait segmentation) and 91.6% overall F1 -score on HELEN (face labeling). That is, our models significantly surpass the artificial network on the accuracy, but with lower resource consumption and higher real-time performance.

Mittag-Leffler stability of fractional-order quaternion-valued memristive neural networks with generalized piecewise constant argument.

This paper studies the global Mittag-Leffler (M-L) stability problem for fractional-order quaternion-valued memristive neural networks (FQVMNNs) with generalized piecewise constant argument (GPCA). First, a novel lemma is established, which is used to investigate the dynamic behaviors of quaternion-valued memristive neural networks (QVMNNs). Second, by using the theories of differential inclusion, set-valued mapping, and Banach fixed point, several sufficient criteria are derived to ensure the existence and uniqueness (EU) of the solution and equilibrium point for the associated systems. Then, by constructing Lyapunov functions and employing some inequality techniques, a set of criteria are proposed to ensure the global M-L stability of the considered systems. The obtained results in this paper not only extends previous works, but also provides new algebraic criteria with a larger feasible range. Finally, two numerical examples are introduced to illustrate the effectiveness of the obtained results.

Suboptimal Leader-to-Coordination Control for Nonlinear Systems With Switching Topologies: A Learning-Based Method.

In the cooperative control for multiagent systems (MASs), the key issues of distributed interaction, nonlinear characteristics, and optimization should be considered simultaneously, which, however, remain intractable theoretically even to this day. Considering these factors, this article investigates leader-to-formation control and optimization for nonlinear MASs using a learning-based method. Under time-varying switching topology, a fully distributed state observer based on neural networks is designed to reconstruct the dynamics and the state trajectory of the leader signal with arbitrary precision under jointly connected topology assumption. Benefitted from the observers, formation for MASs under switching topologies is transformed into tracking control for each subsystem with continuous state generated by the observers. An augmented system with discounted infinite LQR performance index is considered to optimize the control effect. Due to the complexity of solving the Hamilton-Jacobi-Bellman equation, the optimal value function is approximated by a critic network via the integral reinforcement learning method without the knowledge of drift dynamics. Meanwhile, an actor network is also presented to assure stability. The tracking errors and estimation weighted matrices are proven to be uniformly ultimately bounded. Finally, two illustrative examples are given to show the effectiveness of this method.