A double-cycle echo state network topology for time series prediction.

Echo state network (ESN) has gained wide acceptance in the field of time series prediction, relying on sufficiently complex reservoir connections to remember the historical features of the data and using these features to obtain the outputs by a simple linear readout. However, the randomness of its input and reservoir connections pose negative impacts on the prediction performance and performance stability of the models, the complexity of reservoir connections brings high time consumption during network computing, and the presence of randomness and complexity makes the hardware implementation of the ESN difficult. In response, we propose a double-cycle ESN (DCESN) based on the Li-ESN model, which has fixed weights to improve prediction performance and performance stability and simpler reservoir connections compared to the classical ESN to reduce the time consumption. The existence of both greatly reduces the difficulty of hardware implementation of the ESN and provides many conveniences for the future application of the ESN. Experimental results on many widely used time series datasets show that the DCESN has comparable or even better prediction performance than the ESN and good robustness against noise and parameter fluctuations.

A novel conservative system with hidden flows evolved from the simplest memristive circuit.

Over the past few decades, the research of dissipative chaotic systems has yielded many achievements in both theory and application. However, attractors in dissipative systems are easily reconstructed by the attacker, which leads to information security problems. Compared with dissipative systems, conservative ones can effectively avoid these reconstructing attacks due to the absence of attractors. Therefore, conservative systems have advantages in chaos-based applications. Currently, there are still relatively few studies on conservative systems. For this purpose, based on the simplest memristor circuit in this paper, a non-Hamiltonian 3D conservative system without equilibria is proposed. The phase volume conservatism is analyzed by calculating the divergence of the system. Furthermore, a Kolmogorov-type transformation suggests that the Hamiltonian energy is not conservative. The most prominent property in the conservative system is that it exhibits quasi-periodic 3D tori with heterogeneous coexisting and different amplitude rescaling trajectories triggered by initial values. In addition, the results of Spectral Entropy analysis and NIST test show that the system can produce pseudo-random numbers with high randomness. To the best of our knowledge, there is no 3D conservative system with such complex dynamics, especially in a memristive conservative system. Finally, the analog circuit of the system is designed and implemented to test its feasibility as well.

Convolution Kernel Operations on a Two-Dimensional Spin Memristor Cross Array.

In recent years, convolution operations often consume a lot of time and energy in deep learning algorithms, and convolution is usually used to remove noise or extract the edges of an image. However, under data-intensive conditions, frequent operations of the above algorithms will cause a significant memory/communication burden to the computing system. This paper proposes a circuit based on spin memristor cross array to solve the problems mentioned above. First, a logic switch based on spin memristors is proposed, which realizes the control of the memristor cross array. Secondly, a new type of spin memristor cross array and peripheral circuits is proposed, which realizes the multiplication and addition operation in the convolution operation and significantly alleviates the computational memory bottleneck. At last, the color image filtering and edge extraction simulation are carried out. By calculating the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the image result, the processing effects of different operators are compared, and the correctness of the circuit is verified.

A Robust Diffusion Minimum Kernel Risk-Sensitive Loss Algorithm over Multitask Sensor Networks.

Distributed estimation over sensor networks has attracted much attention due to its various applications. The mean-square error (MSE) criterion is one of the most popular cost functions used in distributed estimation, which achieves its optimality only under Gaussian noise. However, impulsive noise also widely exists in real-world sensor networks. Thus, the distributed estimation algorithm based on the minimum kernel risk-sensitive loss (MKRSL) criterion is proposed in this paper to deal with non-Gaussian noise, particularly for impulsive noise. Furthermore, multiple tasks estimation problems in sensor networks are considered. Differing from a conventional single-task, the unknown parameters (tasks) can be different for different nodes in the multitask problem. Another important issue we focus on is the impact of the task similarity among nodes on multitask estimation performance. Besides, the performance of mean and mean square are analyzed theoretically. Simulation results verify a superior performance of the proposed algorithm compared with other related algorithms.

Diffusion Logarithm-Correntropy Algorithm for Parameter Estimation in Non-Stationary Environments over Sensor Networks.

This paper considers the parameter estimation problem under non-stationary environments in sensor networks. The unknown parameter vector is considered to be a time-varying sequence. To further promote estimation performance, this paper suggests a novel diffusion logarithm-correntropy algorithm for each node in the network. Such an algorithm can adopt both the logarithm operation and correntropy criterion to the estimation error. Moreover, if the error gets larger due to the non-stationary environments, the algorithm can respond immediately by taking relatively steeper steps. Thus, the proposed algorithm achieves smaller error in time. The tracking performance of the proposed logarithm-correntropy algorithm is analyzed. Finally, experiments verify the validity of the proposed algorithmic schemes, which are compared to other recent algorithms that have been proposed for parameter estimation.

Highly Sensitive Humidity Sensor Based on Oblique Carbon Nanoplumes.

In this work, we fabricated three carbon nanoplume structured samples under different temperatures using a simple hot filament physical vapor deposition (HFPVD) process, and investigated the role of surface morphology, defects, and graphitic content on relative humidity (RH) sensing performances. The Van der Drift growth model and oblique angle deposition (OAD) technique of growing a large area of uniformly aligned and inclined oblique arrays of carbon nanoplumes (CNPs) on a catalyst-free silicon substrate was demonstrated. The optimal growing temperature of 800 °C was suitable for the formation of nanoplumes with larger surface area, more defect sites, and less graphitic content, compared to the other samples that were prepared. As expected, a low detection limit, high response, capability of reversible behavior, and rapid response/recovery speed with respect to RH variation, was achieved without additional surface modification or chemical functionalization. The holes' depletion has been described as a RH sensing mechanism that leads to the increase of the conduction of the CNPs with increasing RH levels.

A Robust Diffusion Estimation Algorithm with Self-Adjusting Step-Size in WSNs.

In wireless sensor networks (WSNs), each sensor node can estimate the global parameter from the local data in a distributed manner. This paper proposed a robust diffusion estimation algorithm based on a minimum error entropy criterion with a self-adjusting step-size, which are referred to as the diffusion MEE-SAS (DMEE-SAS) algorithm. The DMEE-SAS algorithm has a fast speed of convergence and is robust against non-Gaussian noise in the measurements. The detailed performance analysis of the DMEE-SAS algorithm is performed. By combining the DMEE-SAS algorithm with the diffusion minimum error entropy (DMEE) algorithm, an Improving DMEE-SAS algorithm is proposed for a non-stationary environment where tracking is very important. The Improving DMEE-SAS algorithm can avoid insensitivity of the DMEE-SAS algorithm due to the small effective step-size near the optimal estimator and obtain a fast convergence speed. Numerical simulations are given to verify the effectiveness and advantages of these proposed algorithms.

Self-Taught Learning Based on Sparse Autoencoder for E-Nose in Wound Infection Detection.

For an electronic nose (E-nose) in wound infection distinguishing, traditional learning methods have always needed large quantities of labeled wound infection samples, which are both limited and expensive; thus, we introduce self-taught learning combined with sparse autoencoder and radial basis function (RBF) into the field. Self-taught learning is a kind of transfer learning that can transfer knowledge from other fields to target fields, can solve such problems that labeled data (target fields) and unlabeled data (other fields) do not share the same class labels, even if they are from entirely different distribution. In our paper, we obtain numerous cheap unlabeled pollutant gas samples (benzene, formaldehyde, acetone and ethylalcohol); however, labeled wound infection samples are hard to gain. Thus, we pose self-taught learning to utilize these gas samples, obtaining a basis vector θ. Then, using the basis vector θ, we reconstruct the new representation of wound infection samples under sparsity constraint, which is the input of classifiers. We compare RBF with partial least squares discriminant analysis (PLSDA), and reach a conclusion that the performance of RBF is superior to others. We also change the dimension of our data set and the quantity of unlabeled data to search the input matrix that produces the highest accuracy.

A Novel Semi-Supervised Electronic Nose Learning Technique: M-Training.

When an electronic nose (E-nose) is used to distinguish different kinds of gases, the label information of the target gas could be lost due to some fault of the operators or some other reason, although this is not expected. Another fact is that the cost of getting the labeled samples is usually higher than for unlabeled ones. In most cases, the classification accuracy of an E-nose trained using labeled samples is higher than that of the E-nose trained by unlabeled ones, so gases without label information should not be used to train an E-nose, however, this wastes resources and can even delay the progress of research. In this work a novel multi-class semi-supervised learning technique called M-training is proposed to train E-noses with both labeled and unlabeled samples. We employ M-training to train the E-nose which is used to distinguish three indoor pollutant gases (benzene, toluene and formaldehyde). Data processing results prove that the classification accuracy of E-nose trained by semi-supervised techniques (tri-training and M-training) is higher than that of an E-nose trained only with labeled samples, and the performance of M-training is better than that of tri-training because more base classifiers can be employed by M-training.

Enhancing Electronic Nose Performance Based on a Novel QPSO-KELM Model.

A novel multi-class classification method for bacteria detection termed quantum-behaved particle swarm optimization-based kernel extreme learning machine (QPSO-KELM) based on an electronic nose (E-nose) technology is proposed in this paper. Time and frequency domain features are extracted from E-nose signals used for detecting four different classes of wounds (uninfected and infected with Staphylococcu aureus, Escherichia coli and Pseudomonas aeruginosa) in this experiment. In addition, KELM is compared with five existing classification methods: Linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), extreme learning machine (ELM), k-nearest neighbor (KNN) and support vector machine (SVM). Meanwhile, three traditional optimization methods including particle swarm optimization algorithm (PSO), genetic algorithm (GA) and grid search algorithm (GS) and four kernel functions (Gaussian kernel, linear kernel, polynomial kernel and wavelet kernel) for KELM are discussed in this experiment. Finally, the QPSO-KELM model is also used to deal with another two experimental E-nose datasets in the previous experiments. The experimental results demonstrate the superiority of QPSO-KELM in various E-nose applications.

A Novel Pre-Processing Technique for Original Feature Matrix of Electronic Nose Based on Supervised Locality Preserving Projections.

An electronic nose (E-nose) consisting of 14 metal oxide gas sensors and one electronic chemical gas sensor has been constructed to identify four different classes of wound infection. However, the classification results of the E-nose are not ideal if the original feature matrix containing the maximum steady-state response value of sensors is processed by the classifier directly, so a novel pre-processing technique based on supervised locality preserving projections (SLPP) is proposed in this paper to process the original feature matrix before it is put into the classifier to improve the performance of the E-nose. SLPP is good at finding and keeping the nonlinear structure of data; furthermore, it can provide an explicit mapping expression which is unreachable by the traditional manifold learning methods. Additionally, some effective optimization methods are found by us to optimize the parameters of SLPP and the classifier. Experimental results prove that the classification accuracy of support vector machine (SVM combined with the data pre-processed by SLPP outperforms other considered methods. All results make it clear that SLPP has a better performance in processing the original feature matrix of the E-nose.

A Novel Optimization Technique to Improve Gas Recognition by Electronic Noses Based on the Enhanced Krill Herd Algorithm.

An electronic nose (E-nose) is an intelligent system that we will use in this paper to distinguish three indoor pollutant gases (benzene (C6H6), toluene (C7H8), formaldehyde (CH2O)) and carbon monoxide (CO). The algorithm is a key part of an E-nose system mainly composed of data processing and pattern recognition. In this paper, we employ support vector machine (SVM) to distinguish indoor pollutant gases and two of its parameters need to be optimized, so in order to improve the performance of SVM, in other words, to get a higher gas recognition rate, an effective enhanced krill herd algorithm (EKH) based on a novel decision weighting factor computing method is proposed to optimize the two SVM parameters. Krill herd (KH) is an effective method in practice, however, on occasion, it cannot avoid the influence of some local best solutions so it cannot always find the global optimization value. In addition its search ability relies fully on randomness, so it cannot always converge rapidly. To address these issues we propose an enhanced KH (EKH) to improve the global searching and convergence speed performance of KH. To obtain a more accurate model of the krill behavior, an updated crossover operator is added to the approach. We can guarantee the krill group are diversiform at the early stage of iterations, and have a good performance in local searching ability at the later stage of iterations. The recognition results of EKH are compared with those of other optimization algorithms (including KH, chaotic KH (CKH), quantum-behaved particle swarm optimization (QPSO), particle swarm optimization (PSO) and genetic algorithm (GA)), and we can find that EKH is better than the other considered methods. The research results verify that EKH not only significantly improves the performance of our E-nose system, but also provides a good beginning and theoretical basis for further study about other improved krill algorithms' applications in all E-nose application areas.

A Novel Semi-Supervised Method of Electronic Nose for Indoor Pollution Detection Trained by M-S4VMs.

Electronic nose (E-nose), as a device intended to detect odors or flavors, has been widely used in many fields. Many labeled samples are needed to gain an ideal E-nose classification model. However, the labeled samples are not easy to obtain and there are some cases where the gas samples in the real world are complex and unlabeled. As a result, it is necessary to make an E-nose that cannot only classify unlabeled samples, but also use these samples to modify its classification model. In this paper, we first introduce a semi-supervised learning algorithm called S4VMs and improve its use within a multi-classification algorithm to classify the samples for an E-nose. Then, we enhance its performance by adding the unlabeled samples that it has classified to modify its model and by using an optimization algorithm called quantum-behaved particle swarm optimization (QPSO) to find the optimal parameters for classification. The results of comparing this with other semi-supervised learning algorithms show that our multi-classification algorithm performs well in the classification system of an E-nose after learning from unlabeled samples.

A Novel Feature Extraction Approach Using Window Function Capturing and QPSO-SVM for Enhancing Electronic Nose Performance.

In this paper, a novel feature extraction approach which can be referred to as moving window function capturing (MWFC) has been proposed to analyze signals of an electronic nose (E-nose) used for detecting types of infectious pathogens in rat wounds. Meanwhile, a quantum-behaved particle swarm optimization (QPSO) algorithm is implemented in conjunction with support vector machine (SVM) for realizing a synchronization optimization of the sensor array and SVM model parameters. The results prove the efficacy of the proposed method for E-nose feature extraction, which can lead to a higher classification accuracy rate compared to other established techniques. Meanwhile it is interesting to note that different classification results can be obtained by changing the types, widths or positions of windows. By selecting the optimum window function for the sensor response, the performance of an E-nose can be enhanced.

Electronic Nose Feature Extraction Methods: A Review.

Many research groups in academia and industry are focusing on the performance improvement of electronic nose (E-nose) systems mainly involving three optimizations, which are sensitive material selection and sensor array optimization, enhanced feature extraction methods and pattern recognition method selection. For a specific application, the feature extraction method is a basic part of these three optimizations and a key point in E-nose system performance improvement. The aim of a feature extraction method is to extract robust information from the sensor response with less redundancy to ensure the effectiveness of the subsequent pattern recognition algorithm. Many kinds of feature extraction methods have been used in E-nose applications, such as extraction from the original response curves, curve fitting parameters, transform domains, phase space (PS) and dynamic moments (DM), parallel factor analysis (PARAFAC), energy vector (EV), power density spectrum (PSD), window time slicing (WTS) and moving window time slicing (MWTS), moving window function capture (MWFC), etc. The object of this review is to provide a summary of the various feature extraction methods used in E-noses in recent years, as well as to give some suggestions and new inspiration to propose more effective feature extraction methods for the development of E-nose technology.

A novel memristive multilayer feedforward small-world neural network with its applications in PID control.

In this paper, we present an implementation scheme of memristor-based multilayer feedforward small-world neural network (MFSNN) inspirited by the lack of the hardware realization of the MFSNN on account of the need of a large number of electronic neurons and synapses. More specially, a mathematical closed-form charge-governed memristor model is presented with derivation procedures and the corresponding Simulink model is presented, which is an essential block for realizing the memristive synapse and the activation function in electronic neurons. Furthermore, we investigate a more intelligent memristive PID controller by incorporating the proposed MFSNN into intelligent PID control based on the advantages of the memristive MFSNN on computation speed and accuracy. Finally, numerical simulations have demonstrated the effectiveness of the proposed scheme.

Memristive patch attention neural network for facial expression recognition and edge computing.

Facial expression recognition has made a significant progress as a result of the advent of more and more convolutional neural networks (CNN). However, with the improvement of CNN, the models continues to get deeper and larger so as to a greater focus on the high-level features of the image and the low-level features tend to be lost. Because of the reason above, the dependence of low-level features between different areas of the face often cannot be summarized. In response to this problem, we propose a novel network based on the CNN model. To extract long-range dependencies of low-level features, multiple attention mechanisms has been introduced into the network. In this paper, the patch attention mechanism is designed to obtain the dependence between low-level features of facial expressions firstly. After fusion, the feature maps are input to the backbone network incorporating convolutional block attention module (CBAM) to enhance the feature extraction ability and improve the accuracy of facial expression recognition, and achieve competitive results on three datasets CK+ (98.10%), JAFFE (95.12%) and FER2013 (73.50%). Further, according to the PA Net designed in this paper, a hardware friendly implementation scheme is designed based on memristor crossbars, which is expected to provide a software and hardware co-design scheme for edge computing of personal and wearable electronic products.

High-Performance Method and Architecture for Attention Computation in DNN Inference.

In recent years, The combination of Attention mechanism and deep learning has a wide range of applications in the field of medical imaging. However, due to its complex computational processes, existing hardware architectures have high resource consumption or low accuracy, and deploying them efficiently to DNN accelerators is a challenge. This paper proposes an online-programmable Attention hardware architecture based on compute-in-memory (CIM) marco, which reduces the complexity of Attention in hardware and improves integration density, energy efficiency, and calculation accuracy. First, the Attention computation process is decomposed into multiple cascaded combinatorial matrix operations to reduce the complexity of its implementation on the hardware side; second, in order to reduce the influence of the non-ideal characteristics of the hardware, an online-programmable CIM architecture is designed to improve calculation accuracy by dynamically adjusting the weights; and lastly, it is verified that the proposed Attention hardware architecture can be applied for the inference of deep neural networks through Spice simulation. Based on the 100nm CMOS process, compared with the traditional Attention hardware architectures, the integrated density and energy efficiency are increased by at least 91.38 times, and latency and computing efficiency are improved by about 12.5 times.

Fixed-Time Passivity and Synchronization of Multiweighted Coupled Memristive Neural Networks With Adaptive Couplings.

Two types of multiweighted coupled memristive neural networks (CMNNs) with adaptive couplings are introduced in this article, and the fixed-time passivity (FXTP) and fixed-time synchronization (FXTS) of such networks are considered. First, under the developed adaptive scheme, a sufficient condition to guarantee the FXTP for multiweighted CMNNs with adaptive couplings is obtained. Second, the FXTP, fixed-time input-strict passivity and fixed-time output-strict passivity for multiweighted CMNNs with adaptive couplings and coupling delays are investigated by devising an appropriate state feedback controller. Third, applying the Lyapunov functional method, it establishes the FXTS criteria for the two kinds of networks presented. Finally, numerical examples are provided to demonstrate the effectiveness of the derived results.

A White-Box Testing for Deep Neural Networks Based on Neuron Coverage.

With the introduction of neuron coverage as a testing criterion for deep neural networks (DNNs), covering more neurons to detect more internal logic of DNNs became the main goal of many research studies. While some works had made progress, some new challenges for testing methods based on neuron coverage had been proposed, mainly as establishing better neuron selection and activation strategies influenced not only obtaining higher neuron coverage, but also more testing efficiency, validating testing results automatically, labeling generated test cases to extricate manual work, and so on. In this article, we put forward Test4Deep, an effective white-box testing DNN approach based on neuron coverage. It is based on a differential testing framework to automatically verify inconsistent DNNs' behavior. We designed a strategy that can track inactive neurons and constantly triggered them in each iteration to maximize neuron coverage. Furthermore, we devised an optimization function that guided the DNN under testing to deviate predictions between the original input and generated test data and dominated unobservable generation perturbations to avoid manually checking test oracles. We conducted comparative experiments with two state-of-the-art white-box testing methods DLFuzz and DeepXplore. Empirical results on three popular datasets with nine DNNs demonstrated that compared to DLFuzz and DeepXplore, Test4Deep, on average, exceeded by 32.87% and 35.69% in neuron coverage, while reducing 58.37% and 53.24% testing time, respectively. In the meantime, Test4Deep also produced 58.37% and 53.24% more test cases with 23.81% and 98.40% fewer perturbations. Even compared with the two highest neuron coverage strategies of DLFuzz, Test4Deep still enhanced neuron coverage by 4.34% and 23.23% and achieved 94.48% and 85.67% higher generation time efficiency. Furthermore, Test4Deep could improve the accuracy and robustness of DNNs by merging generated test cases and retraining.