An Incremental-Self-Training-Guided Semi-Supervised Broad Learning System.

The broad learning system (BLS) has recently been applied in numerous fields. However, it is mainly a supervised learning system and thus not suitable for specific practical applications with a mixture of labeled and unlabeled data. Despite a manifold regularization-based semi-supervised BLS, its performance still requires improvement, because its assumption is not always applicable. Therefore, this article proposes an incremental-self-training-guided semi-supervised BLS (ISTSS-BLS). Distinctive to traditional self-training, where all unlabeled data are labeled simultaneously, incremental self-training (IST) obtains unlabeled data incrementally from an established sorted list based on the distance between the data and their cluster center. During iterative learning, a small portion of labeled data is first used to train BLS. The system recursively self-updates its structure and meta-parameters using: 1) the double-restricted mechanism and 2) the dynamic neuron-incremental mechanism. The double-restricted mechanism is beneficial to preventing the introduction of incorrect pseudo-labeled samples, and the dynamic neuron-incremental mechanism guides the self-updating of the network structure effectively based on the training accuracy of the labeled data. These strategies guarantee a parsimonious model during the update. Besides, a novel metric, the accuracy-time ratio (A/T), is proposed to evaluate the model's performance comprehensively regarding time and accuracy. In experimental verifications, ISTSS-BLS performs outstandingly on 11 datasets. Specifically, the IST is compared with the traditional one on three scales data, saving up to 52.02% learning time. In addition, ISTSS-BLS is compared with different state-of-the-art alternatives, and all results indicate that it possesses significant advantages in performance.

High-Quality Fusion and Visualization for MR-PET Brain Tumor Images via Multi-Dimensional Features.

The fusion of magnetic resonance imaging and positron emission tomography can combine biological anatomical information and physiological metabolic information, which is of great significance for the clinical diagnosis and localization of lesions. In this paper, we propose a novel adaptive linear fusion method for multi-dimensional features of brain magnetic resonance and positron emission tomography images based on a convolutional neural network, termed as MdAFuse. First, in the feature extraction stage, three-dimensional feature extraction modules are constructed to extract coarse, fine, and multi-scale information features from the source image. Second, at the fusion stage, the affine mapping function of multi-dimensional features is established to maintain a constant geometric relationship between the features, which can effectively utilize structural information from a feature map to achieve a better reconstruction effect. Furthermore, our MdAFuse comprises a key feature visualization enhancement algorithm designed to observe the dynamic growth of brain lesions, which can facilitate the early diagnosis and treatment of brain tumors. Extensive experimental results demonstrate that our method is superior to existing fusion methods in terms of visual perception and nine kinds of objective image fusion metrics. Specifically, in the results of MR-PET fusion, the SSIM (Structural Similarity) and VIF (Visual Information Fidelity) metrics show improvements of 5.61% and 13.76%, respectively, compared to the current state-of-the-art algorithm. Our project is publicly available at: https://github.com/22385wjy/MdAFuse.

A Robust Semi-Supervised Broad Learning System Guided by Ensemble-Based Self-Training.

Broad learning system (BLS) with semi-supervised learning relieves label dependence and expands application. Despite some efforts and progress, the semi-supervised BLS still needs improvement, especially in handling imbalanced data or concept drift scenarios for self-training-based methods. To this extent, this article proposes a robust semi-supervised BLS guided by ensemble-based self-training (ESTSS-BLS). Distinctive to self-training that assigns the pseudo-label via a single classifier and confidence, the advocated ensemble-based self-training determines the pseudo-label according to the turnout of multiple BLSs. In addition, label purity is proposed to ensure the correctness and credibility of the auxiliary training data, which is a comprehensive evaluation of the voting. During iterative learning, a small portion of labeled data first trains multiple BLSs in parallel. Then, the system recursively updates its data, structure, and meta-parameters using label purity and a data-driven dynamic nodes mechanism that dynamically guides the network's structural adjustments to solve the concept drift problem caused by a large amount of auxiliary training data. The experimental results demonstrate that ESTSS-BLS exhibits exceptional performance compared to existing methods, with the lowest-time consumption and the highest accuracy, precision, recall, F1 score, and AUC. Exhilaratingly, it achieves an accuracy of 87.84% with only 0.1% labeled data on MNIST, and with just 2% labeled data, it matches the performance of supervised learning using all training data on NORB. In addition, ESTSS-BLS also performs stably on medical or biological data, verifying its high adaptability.

Robust Incremental Broad Learning System for Data Streams of Uncertain Scale.

Due to its marvelous performance and remarkable scalability, a broad learning system (BLS) has aroused a wide range of attention. However, its incremental learning suffers from low accuracy and long training time, especially when dealing with unstable data streams, making it difficult to apply in real-world scenarios. To overcome these issues and enrich its relevant research, a robust incremental BLS (RI-BLS) is proposed. In this method, the proposed weight update strategy introduces two memory matrices to store the learned information, thus the computational procedure of ridge regression is decomposed, resulting in precomputed ridge regression. During incremental learning, RI-BLS updates two memory matrices and renews weights via precomputed ridge regression efficiently. In addition, this update strategy is theoretically analyzed in error, time complexity, and space complexity compared with existing incremental BLSs. Different from Greville's method used in the original incremental BLS, its results are closer to the solution of one-shot calculation. Compared with the existing incremental BLSs, the proposed method exhibits more stable time complexity and superior space complexity. The experiments prove that RI-BLS outperforms other incremental BLSs when handling both stable and unstable data streams. Furthermore, experiments demonstrate that the proposed weight update strategy applies to other random neural networks as well.

Robust Temporal Link Prediction in Dynamic Complex Networks via Stable Gated Models With Reinforcement Learning.

Temporal link prediction is one of the most important tasks for predicting time-varying links by capturing dynamics within complex networks. However, it suffers from difficulties such as vulnerability to adversarial attacks and inadaptation to distinct evolutionary patterns. In this article, we propose a robust temporal link prediction architecture via stable gated models with reinforcement learning (SAGE-RL) consisting of a state encoding network (SEN) and a self-adaptive policy network (SPN). The former is utilized to capture network dynamics, while the latter helps the former adapt to distinct evolutionary patterns across various time periods. Within the SEN, a novel stable gate is introduced to ensure multiple spatiotemporal dependency paths and defend against adversarial attacks. An SPN is proposed to select different SEN instances by approximating the optimal action function, thereby adapting to various evolutionary patterns to learn the robust temporal and structural features from dynamic complex networks. It is proven that SAGE-LR with integral Lipschitz graph convolution is stable to relative perturbations in dynamic complex networks. With the aid of extensive experiments on five real-world graph benchmarks, SAGE-LR is shown to substantially outperform current state-of-the-art approaches in terms of precision and stability of temporal link prediction and ability to successfully defend against various attacks. We also implement the temporal link prediction in shipping transaction networks, which forecast effectively its potential transaction risks.

IFKMHC: Implicit Fuzzy K-Means Model for High-Dimensional Data Clustering.

The graph-information-based fuzzy clustering has shown promising results in various datasets. However, its performance is hindered when dealing with high-dimensional data due to challenges related to redundant information and sensitivity to the similarity matrix design. To address these limitations, this article proposes an implicit fuzzy k-means (FKMs) model that enhances graph-based fuzzy clustering for high-dimensional data. Instead of explicitly designing a similarity matrix, our approach leverages the fuzzy partition result obtained from the implicit FKMs model to generate an effective similarity matrix. We employ a projection-based technique to handle redundant information, eliminating the need for specific feature extraction methods. By formulating the fuzzy clustering model solely based on the similarity matrix derived from the membership matrix, we mitigate issues, such as dependence on initial values and random fluctuations in clustering results. This innovative approach significantly improves the competitiveness of graph-enhanced fuzzy clustering for high-dimensional data. We present an efficient iterative optimization algorithm for our model and demonstrate its effectiveness through theoretical analysis and experimental comparisons with other state-of-the-art methods, showcasing its superior performance.

Adaptive Critic Learning-Based Optimal Bipartite Consensus for Multiagent Systems With Prescribed Performance.

Developing a distributed bipartite optimal consensus scheme while ensuring user-predefined performance is essential in practical applications. Existing approaches to this problem typically require a complex controller structure due to adopting an identifier-actor-critic framework and prescribed performance cannot be guaranteed. In this work, an adaptive critic learning (ACL)-based optimal bipartite consensus scheme is developed to bridge the gap. A newly designed error scaling function, which defines the user-predefined settling time and steady accuracy without relying on the initial conditions, is then integrated into a cost function. The backstepping framework combines the ACL and integral reinforcement learning (IRL) algorithm to develop the adaptive optimal bipartite consensus scheme, which contributes a critic-only controller structure by removing the identifier and actor networks in the existing methods. The adaptive law of the critic network is derived by the gradient descent algorithm and experience replay to minimize the IRL-based residual error. It is shown that a compute-saving learning mechanism can achieve the optimal consensus, and the error variables of the closed-loop system are uniformly ultimately bounded (UUB). Besides, in any bounded initial condition, the evolution of bipartite consensus is limited to a user-prescribed boundary under bounded initial conditions. The illustrative simulation results validate the efficacy of the approach.

Attack-resilient fault detection for interconnected systems under DoS attack.

In light of the expanding cyber-space applications, the imperative consideration of cyber-attack ramifications on system security is evident. This paper presents a resilient dynamic event-triggered fault detection scheme for a class of nonlinear interconnected systems subjected to denial of service (DoS) attacks. To counteract multifaceted threats, the co-design challenge involving switched-type fault detection filters and a resilient dynamic event-triggered transmission mechanism is addressed. In the design phase of the filters, the frequency information of the signal is considered comprehensively and linear solvable conditions ensuring desired augment system performance are delineated. Through a series of comparative simulation experiments, the findings support the conclusion that the proposed attack-tolerant fault detection mechanism not only conserves network resources but also demonstrates superior detection capabilities for specific frequency fault signals.

End-to-end model-based trajectory prediction for ro-ro ship route using dual-attention mechanism.

With the rapid increase of economic globalization, the significant expansion of shipping volume has resulted in shipping route congestion, causing the necessity of trajectory prediction for effective service and efficient management. While trajectory prediction can achieve a relatively high level of accuracy, the performance and generalization of prediction models remain critical bottlenecks. Therefore, this article proposes a dual-attention (DA) based end-to-end (E2E) neural network (DAE2ENet) for trajectory prediction. In the E2E structure, long short-term memory (LSTM) units are included for the task of pursuing sequential trajectory data from the encoder layer to the decoder layer. In DA mechanisms, global attention is introduced between the encoder and decoder layers to facilitate interactions between input and output trajectory sequences, and multi-head self-attention is utilized to extract sequential features from the input trajectory. In experiments, we use a ro-ro ship with a fixed navigation route as a case study. Compared with baseline models and benchmark neural networks, DAE2ENet can obtain higher performance on trajectory prediction, and better validation of environmental factors on ship navigation.

Double Discrete Cosine Transform-Oriented Multi-View Subspace Clustering.

Low-rank tensor representation with the tensor nuclear norm has been rising in popularity in multi-view subspace clustering (MVSC), in which the tensor nuclear norm is commonly implemented using discrete Fourier transform (DFT). Unfortunately, existing DFT-oriented MVSC methods may provide unsatisfactory results since (1) DFT exploits complex arithmetic in the Fourier domain, usually resulting in high tubal tensor rank, and (2) local structural information is rarely considered. To solve these problems, in this paper, we propose a novel double discrete cosine transform (DCT)-oriented multi-view subspace clustering (D2CTMSC) method, in which the first DCT aims to derive the tensor nuclear norm without complex arithmetic while the second DCT aims to explore the local structure of the self-representation tensor, such that the essential low-rankness and sparsity embedding in multi-view features can be thoroughly exploited. Moreover, we design an effective alternating iteration strategy to solve the proposed model. Experimental results on four types of multi-view datasets (News stories, Face images, Scene images, and Generic objects) demonstrate the superiority of the D2CTMSC method compared with DFT-based methods and other state-of-the-art clustering methods.

Estimating Addiction-Related Brain Connectivity by Prior-Embedding Graph Generative Adversarial Networks.

The study of nicotine addiction mechanism is of great significance in both nicotine withdrawal and brain science. The detection of addiction-related brain connectivity using functional magnetic resonance imaging (fMRI) is a critical step in study of this mechanism. However, it is challenging to accurately estimate addiction-related brain connectivity due to the low-signal-to-noise ratio of fMRI and the issue of small sample size. In this work, a prior-embedding graph generative adversarial network (PG-GAN) is proposed to capture addiction-related brain connectivity accurately. By designing a dual-generator-based scheme, the addiction-related connectivity generator is employed to learn the feature map of addiction connection, while the reconstruction generator is used for sample reconstruction. Moreover, a bidirectional mapping mechanism is designed to maintain the consistency of sample distribution in the latent space so that addiction-related brain connectivity can be estimated more accurately. The proposed model utilizes prior knowledge embeddings to reduce the search space so that the model can better understand the latent distribution for the issue of small sample size. Experimental results demonstrate the effectiveness of the proposed PG-GAN.

Prior-Guided Adversarial Learning With Hypergraph for Predicting Abnormal Connections in Alzheimer's Disease.

Alzheimer's disease (AD) is characterized by alterations of the brain's structural and functional connectivity during its progressive degenerative processes. Existing auxiliary diagnostic methods have accomplished the classification task, but few of them can accurately evaluate the changing characteristics of brain connectivity. In this work, a prior-guided adversarial learning with hypergraph (PALH) model is proposed to predict abnormal brain connections using triple-modality medical images. Concretely, a prior distribution from anatomical knowledge is estimated to guide multimodal representation learning using an adversarial strategy. Also, the pairwise collaborative discriminator structure is further utilized to narrow the difference in representation distribution. Moreover, the hypergraph perceptual network is developed to effectively fuse the learned representations while establishing high-order relations within and between multimodal images. Experimental results demonstrate that the proposed model outperforms other related methods in analyzing and predicting AD progression. More importantly, the identified abnormal connections are partly consistent with previous neuroscience discoveries. The proposed model can evaluate the characteristics of abnormal brain connections at different stages of AD, which is helpful for cognitive disease study and early treatment.

Employing broad learning and non-invasive risk factor to improve the early diagnosis of metabolic syndrome.

Metabolic syndrome (MetS) as a multifactorial disease is highly prevalent in countries and individuals. Monitoring the conventional risk factors (CRFs) would be a cost-effective strategy to target the increasing prevalence of MetS and the potential of noninvasive CRF for precisely detection of MetS in the early stage remains to be explored. From large-scale multicenter MetS clinical dataset, we discover 15 non-invasive CRFs which have strong relevance with MetS and first propose a broad learning-based approach named Genetic Programming Collaborative-competitive Broad Learning System (GP-CCBLS) with noninvasive CRF for early detection of MetS. The proposed GP-CCBLS model can significantly boost the detection performance and achieve the accuracy of 80.54%. This study supports the potential clinical validity of noninvasive CRF to complement general diagnostic criteria for early detecting the MetS and also illustrates possible strength of broad learning in disease diagnosis comparing with other machine learning approaches.

Extreme Fuzzy Broad Learning System: Algorithm, Frequency Principle, and Applications in Classification and Regression.

As an effective alternative to deep neural networks, broad learning system (BLS) has attracted more attention due to its efficient and outstanding performance and shorter training process in classification and regression tasks. Nevertheless, the performance of BLS will not continue to increase, but even decrease, as the number of nodes reaches the saturation point and continues to increase. In addition, the previous research on neural networks usually ignored the reason for the good generalization of neural networks. To solve these problems, this article first proposes the Extreme Fuzzy BLS (E-FBLS), a novel cascaded fuzzy BLS, in which multiple fuzzy BLS blocks are grouped or cascaded together. Moreover, the original data is input to each FBLS block rather than the previous blocks. In addition, we use residual learning to illustrate the effectiveness of E-FBLS. From the frequency domain perspective, we also discover the existence of the frequency principle in E-FBLS, which can provide good interpretability for the generalization of the neural network. Experimental results on classical classification and regression datasets show that the accuracy of the proposed E-FBLS is superior to traditional BLS in handling classification and regression tasks. The accuracy improves when the number of blocks increases to some extent. Moreover, we verify the frequency principle of E-FBLS that E-FBLS can obtain the low-frequency components quickly, while the high-frequency components are gradually adjusted as the number of FBLS blocks increases.

High-Similarity-Pass Attention for Single Image Super-Resolution.

Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation.

ADFCNN: Attention-Based Dual-Scale Fusion Convolutional Neural Network for Motor Imagery Brain-Computer Interface.

Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG µ and ß rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model.

Compact Broad Learning System Based on Fused Lasso and Smooth Lasso.

Aiming at simplifying the network structure of broad learning system (BLS), this article proposes a novel simplification method called compact BLS (CBLS). Groups of nodes play an important role in the modeling process of BLS, and it means that there may be a correlation between nodes. The proposed CBLS not only focuses on the compactness of network structure but also pays closer attention to the correlation between nodes. Learning from the idea of Fused Lasso and Smooth Lasso, it uses the L1 -regularization term and the fusion term to penalize each output weight and the difference between adjacent output weights, respectively. The L1 -regularization term determines the correlation between the nodes and the outputs, whereas the fusion term captures the correlation between nodes. By optimizing the output weights iteratively, the correlation between the nodes and the outputs and the correlation between nodes are attempted to be considered in the simplification process simultaneously. Without reducing the prediction accuracy, finally, the network structure is simplified more reasonably and a sparse and smooth output weights solution is provided, which can reflect the characteristic of group learning of BLS. Furthermore, according to the fusion terms used in Fused Lasso and Smooth Lasso, two different simplification strategies are developed and compared. Multiple experiments based on public datasets are used to demonstrate the feasibility and effectiveness of the proposed methods.

Nonsingular Fixed-Time Asymptotic Consensus for Nonlinear Nonstrict-Feedback MASs.

In this article, a fuzzy adaptive fixed-time asymptotic consistent control scheme is developed for a class of nonlinear multiagent systems (NMASs) with a nonstrict-feedback (NSF) structure. In the control process, a fixed-time consistency control method without control singularity is proposed by combining fuzzy logic systems (FLSs) with good approximation capability, fixed-time stability theory, and plus power integration techniques. Then, by using Barbalat's Lemma, the asymptotic stability of tracking errors and the boundedness of the controlled systems are successfully achieved, which means that the tracking errors can converge to zero in a fixed time. Finally, the effectiveness of the designed control scheme is demonstrated by a simulation example.

Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network.

Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits.

When Broad Learning System Meets Label Noise Learning: A Reweighting Learning Framework.

Broad learning system (BLS) is a novel neural network with efficient learning and expansion capacity, but it is sensitive to noise. Accordingly, the existing robust broad models try to suppress noise by assigning each sample an appropriate scalar weight to tune down the contribution of noisy samples in network training. However, they disregard the useful information of the noncorrupted elements hidden in the noisy samples, leading to unsatisfactory performance. To this end, a novel BLS with adaptive reweighting (BLS-AR) strategy is proposed in this article for the classification of data with label noise. Different from the previous works, the BLS-AR learns for each sample a weight vector rather than a scalar weight to indicate the noise degree of each element in the sample, which extends the reweighting strategy from sample level to element level. This enables the proposed network to precisely identify noisy elements and thus highlight the contribution of informative ones to train a more accurate representation model. Thanks to the separability of the model, the proposed network can be divided into several subnetworks, each of which can be trained efficiently. In addition, three corresponding incremental learning algorithms of the BLS-AR are developed for adding new samples or expanding the network. Substantial experiments are conducted to explicate the effectiveness and robustness of the proposed BLS-AR model.