RESUMEN
Time series anomaly detection is the process of identifying anomalies within time series data. The primary challenge of this task lies in the necessity for the model to comprehend the characteristics of time-independent and abnormal data patterns. In this study, a novel algorithm called adaptive memory broad learning system (AdaMemBLS) is proposed for time series anomaly detection. This algorithm leverages the rapid inference capabilities of the broad learning algorithm and the memory bank's capacity to differentiate between normal and abnormal data. Furthermore, an incremental algorithm based on multiple data augmentation techniques is introduced and applied to multiple ensemble learners, thereby enhancing the model's effectiveness in learning the characteristics of time series data. To bolster the model's anomaly detection capabilities, a more diverse ensemble approach and a discriminative anomaly score are recommended. Extensive experiments conducted on various real-world datasets demonstrate that the proposed method exhibits superior inference speed and more accurate anomaly detection compared to the existing competitors. A detailed experimental investigation is presented to elucidate the effectiveness of the proposed method and the underlying reasons for its efficacy.
RESUMEN
Histopathological tissue classification is a fundamental task in computational pathology. Deep learning (DL)-based models have achieved superior performance but centralized training suffers from the privacy leakage problem. Federated learning (FL) can safeguard privacy by keeping training samples locally, while existing FL-based frameworks require a large number of well-annotated training samples and numerous rounds of communication which hinder their viability in real-world clinical scenarios. In this article, we propose a lightweight and universal FL framework, named federated deep-broad learning (FedDBL), to achieve superior classification performance with limited training samples and only one-round communication. By simply integrating a pretrained DL feature extractor, a fast and lightweight broad learning inference system with a classical federated aggregation approach, FedDBL can dramatically reduce data dependency and improve communication efficiency. Five-fold cross-validation demonstrates that FedDBL greatly outperforms the competitors with only one-round communication and limited training samples, while it even achieves comparable performance with the ones under multiple-round communications. Furthermore, due to the lightweight design and one-round communication, FedDBL reduces the communication burden from 4.6 GB to only 138.4 KB per client using the ResNet-50 backbone at 50-round training. Extensive experiments also show the scalability of FedDBL on model generalization to the unseen dataset, various client numbers, model personalization and other image modalities. Since no data or deep model sharing across different clients, the privacy issue is well-solved and the model security is guaranteed with no model inversion attack risk. Code is available at https://github.com/tianpeng-deng/FedDBL.
RESUMEN
The broad learning system (BLS) has recently been applied in numerous fields. However, it is mainly a supervised learning system and thus not suitable for specific practical applications with a mixture of labeled and unlabeled data. Despite a manifold regularization-based semi-supervised BLS, its performance still requires improvement, because its assumption is not always applicable. Therefore, this article proposes an incremental-self-training-guided semi-supervised BLS (ISTSS-BLS). Distinctive to traditional self-training, where all unlabeled data are labeled simultaneously, incremental self-training (IST) obtains unlabeled data incrementally from an established sorted list based on the distance between the data and their cluster center. During iterative learning, a small portion of labeled data is first used to train BLS. The system recursively self-updates its structure and meta-parameters using: 1) the double-restricted mechanism and 2) the dynamic neuron-incremental mechanism. The double-restricted mechanism is beneficial to preventing the introduction of incorrect pseudo-labeled samples, and the dynamic neuron-incremental mechanism guides the self-updating of the network structure effectively based on the training accuracy of the labeled data. These strategies guarantee a parsimonious model during the update. Besides, a novel metric, the accuracy-time ratio (A/T), is proposed to evaluate the model's performance comprehensively regarding time and accuracy. In experimental verifications, ISTSS-BLS performs outstandingly on 11 datasets. Specifically, the IST is compared with the traditional one on three scales data, saving up to 52.02% learning time. In addition, ISTSS-BLS is compared with different state-of-the-art alternatives, and all results indicate that it possesses significant advantages in performance.
RESUMEN
Developing a distributed bipartite optimal consensus scheme while ensuring user-predefined performance is essential in practical applications. Existing approaches to this problem typically require a complex controller structure due to adopting an identifier-actor-critic framework and prescribed performance cannot be guaranteed. In this work, an adaptive critic learning (ACL)-based optimal bipartite consensus scheme is developed to bridge the gap. A newly designed error scaling function, which defines the user-predefined settling time and steady accuracy without relying on the initial conditions, is then integrated into a cost function. The backstepping framework combines the ACL and integral reinforcement learning (IRL) algorithm to develop the adaptive optimal bipartite consensus scheme, which contributes a critic-only controller structure by removing the identifier and actor networks in the existing methods. The adaptive law of the critic network is derived by the gradient descent algorithm and experience replay to minimize the IRL-based residual error. It is shown that a compute-saving learning mechanism can achieve the optimal consensus, and the error variables of the closed-loop system are uniformly ultimately bounded (UUB). Besides, in any bounded initial condition, the evolution of bipartite consensus is limited to a user-prescribed boundary under bounded initial conditions. The illustrative simulation results validate the efficacy of the approach.
RESUMEN
Temporal link prediction is one of the most important tasks for predicting time-varying links by capturing dynamics within complex networks. However, it suffers from difficulties such as vulnerability to adversarial attacks and inadaptation to distinct evolutionary patterns. In this article, we propose a robust temporal link prediction architecture via stable gated models with reinforcement learning (SAGE-RL) consisting of a state encoding network (SEN) and a self-adaptive policy network (SPN). The former is utilized to capture network dynamics, while the latter helps the former adapt to distinct evolutionary patterns across various time periods. Within the SEN, a novel stable gate is introduced to ensure multiple spatiotemporal dependency paths and defend against adversarial attacks. An SPN is proposed to select different SEN instances by approximating the optimal action function, thereby adapting to various evolutionary patterns to learn the robust temporal and structural features from dynamic complex networks. It is proven that SAGE-LR with integral Lipschitz graph convolution is stable to relative perturbations in dynamic complex networks. With the aid of extensive experiments on five real-world graph benchmarks, SAGE-LR is shown to substantially outperform current state-of-the-art approaches in terms of precision and stability of temporal link prediction and ability to successfully defend against various attacks. We also implement the temporal link prediction in shipping transaction networks, which forecast effectively its potential transaction risks.
RESUMEN
Broad learning system (BLS) with semi-supervised learning relieves label dependence and expands application. Despite some efforts and progress, the semi-supervised BLS still needs improvement, especially in handling imbalanced data or concept drift scenarios for self-training-based methods. To this extent, this article proposes a robust semi-supervised BLS guided by ensemble-based self-training (ESTSS-BLS). Distinctive to self-training that assigns the pseudo-label via a single classifier and confidence, the advocated ensemble-based self-training determines the pseudo-label according to the turnout of multiple BLSs. In addition, label purity is proposed to ensure the correctness and credibility of the auxiliary training data, which is a comprehensive evaluation of the voting. During iterative learning, a small portion of labeled data first trains multiple BLSs in parallel. Then, the system recursively updates its data, structure, and meta-parameters using label purity and a data-driven dynamic nodes mechanism that dynamically guides the network's structural adjustments to solve the concept drift problem caused by a large amount of auxiliary training data. The experimental results demonstrate that ESTSS-BLS exhibits exceptional performance compared to existing methods, with the lowest-time consumption and the highest accuracy, precision, recall, F1 score, and AUC. Exhilaratingly, it achieves an accuracy of 87.84% with only 0.1% labeled data on MNIST, and with just 2% labeled data, it matches the performance of supervised learning using all training data on NORB. In addition, ESTSS-BLS also performs stably on medical or biological data, verifying its high adaptability.
RESUMEN
The graph-information-based fuzzy clustering has shown promising results in various datasets. However, its performance is hindered when dealing with high-dimensional data due to challenges related to redundant information and sensitivity to the similarity matrix design. To address these limitations, this article proposes an implicit fuzzy k-means (FKMs) model that enhances graph-based fuzzy clustering for high-dimensional data. Instead of explicitly designing a similarity matrix, our approach leverages the fuzzy partition result obtained from the implicit FKMs model to generate an effective similarity matrix. We employ a projection-based technique to handle redundant information, eliminating the need for specific feature extraction methods. By formulating the fuzzy clustering model solely based on the similarity matrix derived from the membership matrix, we mitigate issues, such as dependence on initial values and random fluctuations in clustering results. This innovative approach significantly improves the competitiveness of graph-enhanced fuzzy clustering for high-dimensional data. We present an efficient iterative optimization algorithm for our model and demonstrate its effectiveness through theoretical analysis and experimental comparisons with other state-of-the-art methods, showcasing its superior performance.
RESUMEN
Due to its marvelous performance and remarkable scalability, a broad learning system (BLS) has aroused a wide range of attention. However, its incremental learning suffers from low accuracy and long training time, especially when dealing with unstable data streams, making it difficult to apply in real-world scenarios. To overcome these issues and enrich its relevant research, a robust incremental BLS (RI-BLS) is proposed. In this method, the proposed weight update strategy introduces two memory matrices to store the learned information, thus the computational procedure of ridge regression is decomposed, resulting in precomputed ridge regression. During incremental learning, RI-BLS updates two memory matrices and renews weights via precomputed ridge regression efficiently. In addition, this update strategy is theoretically analyzed in error, time complexity, and space complexity compared with existing incremental BLSs. Different from Greville's method used in the original incremental BLS, its results are closer to the solution of one-shot calculation. Compared with the existing incremental BLSs, the proposed method exhibits more stable time complexity and superior space complexity. The experiments prove that RI-BLS outperforms other incremental BLSs when handling both stable and unstable data streams. Furthermore, experiments demonstrate that the proposed weight update strategy applies to other random neural networks as well.
RESUMEN
In light of the expanding cyber-space applications, the imperative consideration of cyber-attack ramifications on system security is evident. This paper presents a resilient dynamic event-triggered fault detection scheme for a class of nonlinear interconnected systems subjected to denial of service (DoS) attacks. To counteract multifaceted threats, the co-design challenge involving switched-type fault detection filters and a resilient dynamic event-triggered transmission mechanism is addressed. In the design phase of the filters, the frequency information of the signal is considered comprehensively and linear solvable conditions ensuring desired augment system performance are delineated. Through a series of comparative simulation experiments, the findings support the conclusion that the proposed attack-tolerant fault detection mechanism not only conserves network resources but also demonstrates superior detection capabilities for specific frequency fault signals.
RESUMEN
With the rapid increase of economic globalization, the significant expansion of shipping volume has resulted in shipping route congestion, causing the necessity of trajectory prediction for effective service and efficient management. While trajectory prediction can achieve a relatively high level of accuracy, the performance and generalization of prediction models remain critical bottlenecks. Therefore, this article proposes a dual-attention (DA) based end-to-end (E2E) neural network (DAE2ENet) for trajectory prediction. In the E2E structure, long short-term memory (LSTM) units are included for the task of pursuing sequential trajectory data from the encoder layer to the decoder layer. In DA mechanisms, global attention is introduced between the encoder and decoder layers to facilitate interactions between input and output trajectory sequences, and multi-head self-attention is utilized to extract sequential features from the input trajectory. In experiments, we use a ro-ro ship with a fixed navigation route as a case study. Compared with baseline models and benchmark neural networks, DAE2ENet can obtain higher performance on trajectory prediction, and better validation of environmental factors on ship navigation.
RESUMEN
Low-rank tensor representation with the tensor nuclear norm has been rising in popularity in multi-view subspace clustering (MVSC), in which the tensor nuclear norm is commonly implemented using discrete Fourier transform (DFT). Unfortunately, existing DFT-oriented MVSC methods may provide unsatisfactory results since (1) DFT exploits complex arithmetic in the Fourier domain, usually resulting in high tubal tensor rank, and (2) local structural information is rarely considered. To solve these problems, in this paper, we propose a novel double discrete cosine transform (DCT)-oriented multi-view subspace clustering (D2CTMSC) method, in which the first DCT aims to derive the tensor nuclear norm without complex arithmetic while the second DCT aims to explore the local structure of the self-representation tensor, such that the essential low-rankness and sparsity embedding in multi-view features can be thoroughly exploited. Moreover, we design an effective alternating iteration strategy to solve the proposed model. Experimental results on four types of multi-view datasets (News stories, Face images, Scene images, and Generic objects) demonstrate the superiority of the D2CTMSC method compared with DFT-based methods and other state-of-the-art clustering methods.
RESUMEN
The study of nicotine addiction mechanism is of great significance in both nicotine withdrawal and brain science. The detection of addiction-related brain connectivity using functional magnetic resonance imaging (fMRI) is a critical step in study of this mechanism. However, it is challenging to accurately estimate addiction-related brain connectivity due to the low-signal-to-noise ratio of fMRI and the issue of small sample size. In this work, a prior-embedding graph generative adversarial network (PG-GAN) is proposed to capture addiction-related brain connectivity accurately. By designing a dual-generator-based scheme, the addiction-related connectivity generator is employed to learn the feature map of addiction connection, while the reconstruction generator is used for sample reconstruction. Moreover, a bidirectional mapping mechanism is designed to maintain the consistency of sample distribution in the latent space so that addiction-related brain connectivity can be estimated more accurately. The proposed model utilizes prior knowledge embeddings to reduce the search space so that the model can better understand the latent distribution for the issue of small sample size. Experimental results demonstrate the effectiveness of the proposed PG-GAN.
RESUMEN
Alzheimer's disease (AD) is characterized by alterations of the brain's structural and functional connectivity during its progressive degenerative processes. Existing auxiliary diagnostic methods have accomplished the classification task, but few of them can accurately evaluate the changing characteristics of brain connectivity. In this work, a prior-guided adversarial learning with hypergraph (PALH) model is proposed to predict abnormal brain connections using triple-modality medical images. Concretely, a prior distribution from anatomical knowledge is estimated to guide multimodal representation learning using an adversarial strategy. Also, the pairwise collaborative discriminator structure is further utilized to narrow the difference in representation distribution. Moreover, the hypergraph perceptual network is developed to effectively fuse the learned representations while establishing high-order relations within and between multimodal images. Experimental results demonstrate that the proposed model outperforms other related methods in analyzing and predicting AD progression. More importantly, the identified abnormal connections are partly consistent with previous neuroscience discoveries. The proposed model can evaluate the characteristics of abnormal brain connections at different stages of AD, which is helpful for cognitive disease study and early treatment.
Asunto(s)
Enfermedad de Alzheimer , Encéfalo , Enfermedad de Alzheimer/diagnóstico por imagen , Enfermedad de Alzheimer/fisiopatología , Humanos , Encéfalo/diagnóstico por imagen , Interpretación de Imagen Asistida por Computador/métodos , Imagen por Resonancia Magnética/métodos , Algoritmos , Aprendizaje Automático , Redes Neurales de la Computación , AncianoRESUMEN
Metabolic syndrome (MetS) as a multifactorial disease is highly prevalent in countries and individuals. Monitoring the conventional risk factors (CRFs) would be a cost-effective strategy to target the increasing prevalence of MetS and the potential of noninvasive CRF for precisely detection of MetS in the early stage remains to be explored. From large-scale multicenter MetS clinical dataset, we discover 15 non-invasive CRFs which have strong relevance with MetS and first propose a broad learning-based approach named Genetic Programming Collaborative-competitive Broad Learning System (GP-CCBLS) with noninvasive CRF for early detection of MetS. The proposed GP-CCBLS model can significantly boost the detection performance and achieve the accuracy of 80.54%. This study supports the potential clinical validity of noninvasive CRF to complement general diagnostic criteria for early detecting the MetS and also illustrates possible strength of broad learning in disease diagnosis comparing with other machine learning approaches.
RESUMEN
As an effective alternative to deep neural networks, broad learning system (BLS) has attracted more attention due to its efficient and outstanding performance and shorter training process in classification and regression tasks. Nevertheless, the performance of BLS will not continue to increase, but even decrease, as the number of nodes reaches the saturation point and continues to increase. In addition, the previous research on neural networks usually ignored the reason for the good generalization of neural networks. To solve these problems, this article first proposes the Extreme Fuzzy BLS (E-FBLS), a novel cascaded fuzzy BLS, in which multiple fuzzy BLS blocks are grouped or cascaded together. Moreover, the original data is input to each FBLS block rather than the previous blocks. In addition, we use residual learning to illustrate the effectiveness of E-FBLS. From the frequency domain perspective, we also discover the existence of the frequency principle in E-FBLS, which can provide good interpretability for the generalization of the neural network. Experimental results on classical classification and regression datasets show that the accuracy of the proposed E-FBLS is superior to traditional BLS in handling classification and regression tasks. The accuracy improves when the number of blocks increases to some extent. Moreover, we verify the frequency principle of E-FBLS that E-FBLS can obtain the low-frequency components quickly, while the high-frequency components are gradually adjusted as the number of FBLS blocks increases.
RESUMEN
Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation.
RESUMEN
Aiming at simplifying the network structure of broad learning system (BLS), this article proposes a novel simplification method called compact BLS (CBLS). Groups of nodes play an important role in the modeling process of BLS, and it means that there may be a correlation between nodes. The proposed CBLS not only focuses on the compactness of network structure but also pays closer attention to the correlation between nodes. Learning from the idea of Fused Lasso and Smooth Lasso, it uses the L1 -regularization term and the fusion term to penalize each output weight and the difference between adjacent output weights, respectively. The L1 -regularization term determines the correlation between the nodes and the outputs, whereas the fusion term captures the correlation between nodes. By optimizing the output weights iteratively, the correlation between the nodes and the outputs and the correlation between nodes are attempted to be considered in the simplification process simultaneously. Without reducing the prediction accuracy, finally, the network structure is simplified more reasonably and a sparse and smooth output weights solution is provided, which can reflect the characteristic of group learning of BLS. Furthermore, according to the fusion terms used in Fused Lasso and Smooth Lasso, two different simplification strategies are developed and compared. Multiple experiments based on public datasets are used to demonstrate the feasibility and effectiveness of the proposed methods.
RESUMEN
Convolutional neural networks (CNNs) have been successfully applied to motor imagery (MI)-based brain-computer interface (BCI). Nevertheless, single-scale CNN fail to extract abundant information over a wide spectrum from EEG signals, while typical multi-scale CNNs cannot effectively fuse information from different scales with concatenation-based methods. To overcome these challenges, we propose a new scheme equipped with attention-based dual-scale fusion convolutional neural network (ADFCNN), which jointly extracts and fuses EEG spectral and spatial information at different scales. This scheme also provides novel insight through self-attention for effective information fusion from different scales. Specifically, temporal convolutions with two different kernel sizes identify EEG µ and ß rhythms, while spatial convolutions at two different scales generate global and detailed spatial information, respectively, and the self-attention mechanism performs feature fusion based on the internal similarity of the concatenated features extracted by the dual-scale CNN. The proposed scheme achieves the superior performance compared with state-of-the-art methods in subject-specific motor imagery recognition on BCI Competition IV dataset 2a, 2b and OpenBMI dataset, with the cross-session average classification accuracies of 79.39% and significant improvements of 9.14% on BCI-IV2a, 87.81% and 7.66% on BCI-IV2b, 65.26% and 7.2% on OpenBMI dataset, and the within-session average classification accuracies of 86.87% and significant improvements of 10.89% on BCI-IV2a, 87.26% and 8.07% on BCI-IV2b, 84.29% and 5.17% on OpenBMI dataset, respectively. What is more, ablation experiments are conducted to investigate the mechanism and demonstrate the effectiveness of the dual-scale joint temporal-spatial CNN and self-attention modules. Visualization is also used to reveal the learning process and feature distribution of the model.
Asunto(s)
Algoritmos , Interfaces Cerebro-Computador , Humanos , Imaginación , Electroencefalografía/métodos , Redes Neurales de la ComputaciónRESUMEN
In this article, a fuzzy adaptive fixed-time asymptotic consistent control scheme is developed for a class of nonlinear multiagent systems (NMASs) with a nonstrict-feedback (NSF) structure. In the control process, a fixed-time consistency control method without control singularity is proposed by combining fuzzy logic systems (FLSs) with good approximation capability, fixed-time stability theory, and plus power integration techniques. Then, by using Barbalat's Lemma, the asymptotic stability of tracking errors and the boundedness of the controlled systems are successfully achieved, which means that the tracking errors can converge to zero in a fixed time. Finally, the effectiveness of the designed control scheme is demonstrated by a simulation example.
RESUMEN
Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits.