Developing a distributed bipartite optimal consensus scheme while ensuring user-predefined performance is essential in practical applications. Existing approaches to this problem typically require a complex controller structure due to adopting an identifier-actor-critic framework and prescribed performance cannot be guaranteed. In this work, an adaptive critic learning (ACL)-based optimal bipartite consensus scheme is developed to bridge the gap. A newly designed error scaling function, which defines the user-predefined settling time and steady accuracy without relying on the initial conditions, is then integrated into a cost function. The backstepping framework combines the ACL and integral reinforcement learning (IRL) algorithm to develop the adaptive optimal bipartite consensus scheme, which contributes a critic-only controller structure by removing the identifier and actor networks in the existing methods. The adaptive law of the critic network is derived by the gradient descent algorithm and experience replay to minimize the IRL-based residual error. It is shown that a compute-saving learning mechanism can achieve the optimal consensus, and the error variables of the closed-loop system are uniformly ultimately bounded (UUB). Besides, in any bounded initial condition, the evolution of bipartite consensus is limited to a user-prescribed boundary under bounded initial conditions. The illustrative simulation results validate the efficacy of the approach.
Temporal link prediction is one of the most important tasks for predicting time-varying links by capturing dynamics within complex networks. However, it suffers from difficulties such as vulnerability to adversarial attacks and inadaptation to distinct evolutionary patterns. In this article, we propose a robust temporal link prediction architecture via stable gated models with reinforcement learning (SAGE-RL) consisting of a state encoding network (SEN) and a self-adaptive policy network (SPN). The former is utilized to capture network dynamics, while the latter helps the former adapt to distinct evolutionary patterns across various time periods. Within the SEN, a novel stable gate is introduced to ensure multiple spatiotemporal dependency paths and defend against adversarial attacks. An SPN is proposed to select different SEN instances by approximating the optimal action function, thereby adapting to various evolutionary patterns to learn the robust temporal and structural features from dynamic complex networks. It is proven that SAGE-LR with integral Lipschitz graph convolution is stable to relative perturbations in dynamic complex networks. With the aid of extensive experiments on five real-world graph benchmarks, SAGE-LR is shown to substantially outperform current state-of-the-art approaches in terms of precision and stability of temporal link prediction and ability to successfully defend against various attacks. We also implement the temporal link prediction in shipping transaction networks, which forecast effectively its potential transaction risks.
Due to its marvelous performance and remarkable scalability, a broad learning system (BLS) has aroused a wide range of attention. However, its incremental learning suffers from low accuracy and long training time, especially when dealing with unstable data streams, making it difficult to apply in real-world scenarios. To overcome these issues and enrich its relevant research, a robust incremental BLS (RI-BLS) is proposed. In this method, the proposed weight update strategy introduces two memory matrices to store the learned information, thus the computational procedure of ridge regression is decomposed, resulting in precomputed ridge regression. During incremental learning, RI-BLS updates two memory matrices and renews weights via precomputed ridge regression efficiently. In addition, this update strategy is theoretically analyzed in error, time complexity, and space complexity compared with existing incremental BLSs. Different from Greville's method used in the original incremental BLS, its results are closer to the solution of one-shot calculation. Compared with the existing incremental BLSs, the proposed method exhibits more stable time complexity and superior space complexity. The experiments prove that RI-BLS outperforms other incremental BLSs when handling both stable and unstable data streams. Furthermore, experiments demonstrate that the proposed weight update strategy applies to other random neural networks as well.
Broad learning system (BLS) with semi-supervised learning relieves label dependence and expands application. Despite some efforts and progress, the semi-supervised BLS still needs improvement, especially in handling imbalanced data or concept drift scenarios for self-training-based methods. To this extent, this article proposes a robust semi-supervised BLS guided by ensemble-based self-training (ESTSS-BLS). Distinctive to self-training that assigns the pseudo-label via a single classifier and confidence, the advocated ensemble-based self-training determines the pseudo-label according to the turnout of multiple BLSs. In addition, label purity is proposed to ensure the correctness and credibility of the auxiliary training data, which is a comprehensive evaluation of the voting. During iterative learning, a small portion of labeled data first trains multiple BLSs in parallel. Then, the system recursively updates its data, structure, and meta-parameters using label purity and a data-driven dynamic nodes mechanism that dynamically guides the network's structural adjustments to solve the concept drift problem caused by a large amount of auxiliary training data. The experimental results demonstrate that ESTSS-BLS exhibits exceptional performance compared to existing methods, with the lowest-time consumption and the highest accuracy, precision, recall, F1 score, and AUC. Exhilaratingly, it achieves an accuracy of 87.84% with only 0.1% labeled data on MNIST, and with just 2% labeled data, it matches the performance of supervised learning using all training data on NORB. In addition, ESTSS-BLS also performs stably on medical or biological data, verifying its high adaptability.
Low-rank tensor representation with the tensor nuclear norm has been rising in popularity in multi-view subspace clustering (MVSC), in which the tensor nuclear norm is commonly implemented using discrete Fourier transform (DFT). Unfortunately, existing DFT-oriented MVSC methods may provide unsatisfactory results since (1) DFT exploits complex arithmetic in the Fourier domain, usually resulting in high tubal tensor rank, and (2) local structural information is rarely considered. To solve these problems, in this paper, we propose a novel double discrete cosine transform (DCT)-oriented multi-view subspace clustering (D2CTMSC) method, in which the first DCT aims to derive the tensor nuclear norm without complex arithmetic while the second DCT aims to explore the local structure of the self-representation tensor, such that the essential low-rankness and sparsity embedding in multi-view features can be thoroughly exploited. Moreover, we design an effective alternating iteration strategy to solve the proposed model. Experimental results on four types of multi-view datasets (News stories, Face images, Scene images, and Generic objects) demonstrate the superiority of the D2CTMSC method compared with DFT-based methods and other state-of-the-art clustering methods.
In light of the expanding cyber-space applications, the imperative consideration of cyber-attack ramifications on system security is evident. This paper presents a resilient dynamic event-triggered fault detection scheme for a class of nonlinear interconnected systems subjected to denial of service (DoS) attacks. To counteract multifaceted threats, the co-design challenge involving switched-type fault detection filters and a resilient dynamic event-triggered transmission mechanism is addressed. In the design phase of the filters, the frequency information of the signal is considered comprehensively and linear solvable conditions ensuring desired augment system performance are delineated. Through a series of comparative simulation experiments, the findings support the conclusion that the proposed attack-tolerant fault detection mechanism not only conserves network resources but also demonstrates superior detection capabilities for specific frequency fault signals.
With the rapid increase of economic globalization, the significant expansion of shipping volume has resulted in shipping route congestion, causing the necessity of trajectory prediction for effective service and efficient management. While trajectory prediction can achieve a relatively high level of accuracy, the performance and generalization of prediction models remain critical bottlenecks. Therefore, this article proposes a dual-attention (DA) based end-to-end (E2E) neural network (DAE2ENet) for trajectory prediction. In the E2E structure, long short-term memory (LSTM) units are included for the task of pursuing sequential trajectory data from the encoder layer to the decoder layer. In DA mechanisms, global attention is introduced between the encoder and decoder layers to facilitate interactions between input and output trajectory sequences, and multi-head self-attention is utilized to extract sequential features from the input trajectory. In experiments, we use a ro-ro ship with a fixed navigation route as a case study. Compared with baseline models and benchmark neural networks, DAE2ENet can obtain higher performance on trajectory prediction, and better validation of environmental factors on ship navigation.
The study of nicotine addiction mechanism is of great significance in both nicotine withdrawal and brain science. The detection of addiction-related brain connectivity using functional magnetic resonance imaging (fMRI) is a critical step in study of this mechanism. However, it is challenging to accurately estimate addiction-related brain connectivity due to the low-signal-to-noise ratio of fMRI and the issue of small sample size. In this work, a prior-embedding graph generative adversarial network (PG-GAN) is proposed to capture addiction-related brain connectivity accurately. By designing a dual-generator-based scheme, the addiction-related connectivity generator is employed to learn the feature map of addiction connection, while the reconstruction generator is used for sample reconstruction. Moreover, a bidirectional mapping mechanism is designed to maintain the consistency of sample distribution in the latent space so that addiction-related brain connectivity can be estimated more accurately. The proposed model utilizes prior knowledge embeddings to reduce the search space so that the model can better understand the latent distribution for the issue of small sample size. Experimental results demonstrate the effectiveness of the proposed PG-GAN.
Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation.
Metabolic syndrome (MetS) as a multifactorial disease is highly prevalent in countries and individuals. Monitoring the conventional risk factors (CRFs) would be a cost-effective strategy to target the increasing prevalence of MetS and the potential of noninvasive CRF for precisely detection of MetS in the early stage remains to be explored. From large-scale multicenter MetS clinical dataset, we discover 15 non-invasive CRFs which have strong relevance with MetS and first propose a broad learning-based approach named Genetic Programming Collaborative-competitive Broad Learning System (GP-CCBLS) with noninvasive CRF for early detection of MetS. The proposed GP-CCBLS model can significantly boost the detection performance and achieve the accuracy of 80.54%. This study supports the potential clinical validity of noninvasive CRF to complement general diagnostic criteria for early detecting the MetS and also illustrates possible strength of broad learning in disease diagnosis comparing with other machine learning approaches.
As an effective alternative to deep neural networks, broad learning system (BLS) has attracted more attention due to its efficient and outstanding performance and shorter training process in classification and regression tasks. Nevertheless, the performance of BLS will not continue to increase, but even decrease, as the number of nodes reaches the saturation point and continues to increase. In addition, the previous research on neural networks usually ignored the reason for the good generalization of neural networks. To solve these problems, this article first proposes the Extreme Fuzzy BLS (E-FBLS), a novel cascaded fuzzy BLS, in which multiple fuzzy BLS blocks are grouped or cascaded together. Moreover, the original data is input to each FBLS block rather than the previous blocks. In addition, we use residual learning to illustrate the effectiveness of E-FBLS. From the frequency domain perspective, we also discover the existence of the frequency principle in E-FBLS, which can provide good interpretability for the generalization of the neural network. Experimental results on classical classification and regression datasets show that the accuracy of the proposed E-FBLS is superior to traditional BLS in handling classification and regression tasks. The accuracy improves when the number of blocks increases to some extent. Moreover, we verify the frequency principle of E-FBLS that E-FBLS can obtain the low-frequency components quickly, while the high-frequency components are gradually adjusted as the number of FBLS blocks increases.
Alzheimer's disease (AD) is characterized by alterations of the brain's structural and functional connectivity during its progressive degenerative processes. Existing auxiliary diagnostic methods have accomplished the classification task, but few of them can accurately evaluate the changing characteristics of brain connectivity. In this work, a prior-guided adversarial learning with hypergraph (PALH) model is proposed to predict abnormal brain connections using triple-modality medical images. Concretely, a prior distribution from anatomical knowledge is estimated to guide multimodal representation learning using an adversarial strategy. Also, the pairwise collaborative discriminator structure is further utilized to narrow the difference in representation distribution. Moreover, the hypergraph perceptual network is developed to effectively fuse the learned representations while establishing high-order relations within and between multimodal images. Experimental results demonstrate that the proposed model outperforms other related methods in analyzing and predicting AD progression. More importantly, the identified abnormal connections are partly consistent with previous neuroscience discoveries. The proposed model can evaluate the characteristics of abnormal brain connections at different stages of AD, which is helpful for cognitive disease study and early treatment.
Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG µ and ß rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model.
Algorithms , Brain-Computer Interfaces , Humans , Imagination , Electroencephalography/methods , Neural Networks, Computer
Aiming at simplifying the network structure of broad learning system (BLS), this article proposes a novel simplification method called compact BLS (CBLS). Groups of nodes play an important role in the modeling process of BLS, and it means that there may be a correlation between nodes. The proposed CBLS not only focuses on the compactness of network structure but also pays closer attention to the correlation between nodes. Learning from the idea of Fused Lasso and Smooth Lasso, it uses the L1 -regularization term and the fusion term to penalize each output weight and the difference between adjacent output weights, respectively. The L1 -regularization term determines the correlation between the nodes and the outputs, whereas the fusion term captures the correlation between nodes. By optimizing the output weights iteratively, the correlation between the nodes and the outputs and the correlation between nodes are attempted to be considered in the simplification process simultaneously. Without reducing the prediction accuracy, finally, the network structure is simplified more reasonably and a sparse and smooth output weights solution is provided, which can reflect the characteristic of group learning of BLS. Furthermore, according to the fusion terms used in Fused Lasso and Smooth Lasso, two different simplification strategies are developed and compared. Multiple experiments based on public datasets are used to demonstrate the feasibility and effectiveness of the proposed methods.
In this article, a fuzzy adaptive fixed-time asymptotic consistent control scheme is developed for a class of nonlinear multiagent systems (NMASs) with a nonstrict-feedback (NSF) structure. In the control process, a fixed-time consistency control method without control singularity is proposed by combining fuzzy logic systems (FLSs) with good approximation capability, fixed-time stability theory, and plus power integration techniques. Then, by using Barbalat's Lemma, the asymptotic stability of tracking errors and the boundedness of the controlled systems are successfully achieved, which means that the tracking errors can converge to zero in a fixed time. Finally, the effectiveness of the designed control scheme is demonstrated by a simulation example.
Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits.
Alzheimer Disease , Diffusion Tensor Imaging , Humans , Diffusion Tensor Imaging/methods , Alzheimer Disease/diagnostic imaging , Brain/diagnostic imaging , Magnetic Resonance Imaging/methods , Learning
Broad learning system (BLS) is a novel neural network with efficient learning and expansion capacity, but it is sensitive to noise. Accordingly, the existing robust broad models try to suppress noise by assigning each sample an appropriate scalar weight to tune down the contribution of noisy samples in network training. However, they disregard the useful information of the noncorrupted elements hidden in the noisy samples, leading to unsatisfactory performance. To this end, a novel BLS with adaptive reweighting (BLS-AR) strategy is proposed in this article for the classification of data with label noise. Different from the previous works, the BLS-AR learns for each sample a weight vector rather than a scalar weight to indicate the noise degree of each element in the sample, which extends the reweighting strategy from sample level to element level. This enables the proposed network to precisely identify noisy elements and thus highlight the contribution of informative ones to train a more accurate representation model. Thanks to the separability of the model, the proposed network can be divided into several subnetworks, each of which can be trained efficiently. In addition, three corresponding incremental learning algorithms of the BLS-AR are developed for adding new samples or expanding the network. Substantial experiments are conducted to explicate the effectiveness and robustness of the proposed BLS-AR model.
To meet requirements for real-time trajectory scheduling and distributed coordination, underwater target hunting task is challenging in terms of turbulent ocean environments and dynamic adversarial environment. Despite the existing research in game-based target hunting area, few approaches have considered dynamic environmental factors, such as sea currents, winds, and communication delay. In this article, we focus on a target hunting system consisted of multiple unmanned underwater vehicles (UUVs) and a target with high maneuverability. Besides, differential game theory is leveraged to analyze adversarial behaviors between hunters and the escapee. However, it is intractable that UUVs have to deploy an adaptive scheme to guarantee the consistency and avoid the escape of the target without collision. Therefore, we conceive the Hamiltonian function with Leibniz's formula to obtain feedback control policies. In addition, it proves that the target hunting system is asymptotically stable in the mean, and the system can satisfy Nash equilibrium relying on the proposed control policies. Furthermore, we design a modified multiagent reinforcement learning (MARL) to facilitate the underwater target hunting task under the constraints of energetic flows and acoustic propagation delay. Simulation results show that the proposed scheme is superior to the typical MARL algorithm in terms of reward and success rate.
Introduction: Alzheimer's disease (AD) is a chronic neurodegenerative disease of the brain that has attracted wide attention in the world. The diagnosis of Alzheimer's disease is faced with the difficulties of insufficient manpower and great difficulty. With the intervention of artificial intelligence, deep learning methods are widely used to assist clinicians in the early recognition of Alzheimer's disease. And a series of methods based on data input with different dimensions have been proposed. However, traditional deep learning models rely on expensive hardware resources and consume a lot of training time, and may fall into the dilemma of local optima. Methods: In recent years, broad learning system (BLS) has provided researchers with new research ideas. Based on the three-dimensional residual convolution module and BLS, a novel broad-deep ensemble model based on BLS is proposed for the early detection of Alzheimer's disease. The Alzheimer's Disease Neuroimaging Initiative (ADNI) MRI image dataset is used to train the model and then we compare the performance of proposed model with previous work and clinicians' diagnosis. Results: The result of experiments demonstrate that the broad-deep ensemble model is superior to previously proposed related works, including 3D-ResNet and VoxCNN, in accuracy, sensitivity, specificity and F1. Discussion: The proposed broad-deep ensemble model is effective for early detection of Alzheimer's disease. In addition, the proposed model does not need the pre-training process of its depth module, which greatly reduces the training time and hardware dependence.
This article investigates the adaptive optimal tracking problem for a class of nonlinear affine systems with asymmetric Prandtl-Ishlinskii (PI) hysteresis nonlinearities based on actor-critic (A-C) learning mechanisms. Considering the huge obstacles arising from the uncertainty of hysteresis nonlinearity in actuators, we develop a scheme for the conflict between the construction of Hamilton functions and hysteresis nonlinearity. The actuator hysteresis forces the input into a hysteresis delay, thus preventing the Hamilton function from getting the current moment's input instantly and thus making optimization impossible. In the first step, an inverse model is constructed to compensate for the hysteresis model with a shift factor. In the second step, we compensate for the control input by designing a feedback controller and incorporating the estimation and approximation errors into the Hamilton error. Optimal control, the other part of the actual control input, is obtained by taking partial derivatives of the Hamiltonian function after the nonlinearities have been circumvented. At the end, a simulation is given to validate the developed solution.
