Transformer-Based Weakly Supervised Learning for Whole Slide Lung Cancer Image Classification.

Image analysis can play an important role in supporting histopathological diagnoses of lung cancer, with deep learning methods already achieving remarkable results. However, due to the large scale of whole-slide images (WSIs), creating manual pixel-wise annotations from expert pathologists is expensive and time-consuming. In addition, the heterogeneity of tumors and similarities in the morphological phenotype of tumor subtypes have caused inter-observer variability in annotations, which limits optimal performance. Effective use of weak labels could potentially alleviate these issues. In this paper, we propose a two-stage transformer-based weakly supervised learning framework called Simple Shuffle-Remix Vision Transformer (SSRViT). Firstly, we introduce a Shuffle-Remix Vision Transformer (SRViT) to retrieve discriminative local tokens and extract effective representative features. Then, the token features are selected and aggregated to generate sparse representations of WSIs, which are fed into a simple transformer-based classifier (SViT) for slide-level prediction. Experimental results demonstrate that the performance of our proposed SSRViT is significantly improved compared with other state-of-the-art methods in discriminating between adenocarcinoma, pulmonary sclerosing pneumocytoma and normal lung tissue (accuracy of 96.9% and AUC of 99.6%).

Self-supervised contrastive learning for EEG-based cross-subject motor imagery recognition.

Objective. The extensive application of electroencephalography (EEG) in brain-computer interfaces (BCIs) can be attributed to its non-invasive nature and capability to offer high-resolution data. The acquisition of EEG signals is a straightforward process, but the datasets associated with these signals frequently exhibit data scarcity and require substantial resources for proper labeling. Furthermore, there is a significant limitation in the generalization performance of EEG models due to the substantial inter-individual variability observed in EEG signals.Approach. To address these issues, we propose a novel self-supervised contrastive learning framework for decoding motor imagery (MI) signals in cross-subject scenarios. Specifically, we design an encoder combining convolutional neural network and attention mechanism. In the contrastive learning training stage, the network undergoes training with the pretext task of data augmentation to minimize the distance between pairs of homologous transformations while simultaneously maximizing the distance between pairs of heterologous transformations. It enhances the amount of data utilized for training and improves the network's ability to extract deep features from original signals without relying on the true labels of the data.Main results. To evaluate our framework's efficacy, we conduct extensive experiments on three public MI datasets: BCI IV IIa, BCI IV IIb, and HGD datasets. The proposed method achieves cross-subject classification accuracies of 67.32%, 82.34%, and 81.13%on the three datasets, demonstrating superior performance compared to existing methods.Significance. Therefore, this method has great promise for improving the performance of cross-subject transfer learning in MI-based BCI systems.

Motion states identification of underwater glider based on complex networks and graph convolutional networks.

Underwater glider (UG) plays an important role in ocean observation and exploration for a more efficient and deeper understanding of complex ocean environment. Timely identifying the motion states of UG is conducive for timely attitude adjustment and detection of potential anomalies, thereby improving the working reliability of UG. Combining limited penetrable visibility graph (LPVG) and graph convolutional networks (GCN) with self-attention mechanisms, we propose a novel method for motion states identification of UG, which is called as visibility graph and self-attention mechanism-based graph convolutional network (VGSA-GCN). Based on the actual sea trial data of UG, we chose the attitude angle signals of motion states related sensors collected by the control system of UG as the research object and constructed complex networks based on the LPVG method from pitch angle, roll angle, and heading angle data in diving and climbing states. Then, we build a self-attention mechanism-based GCN framework and classify the graphs under different motion states constructed by a complex network. Compared with support vector machines, convolutional neural network, and GCN without self-attention pooling layer, the proposed VGSA-GCN method can more accurately distinguish the diving and climbing states of UG. Subsequently, we analyze the variation of the transitivity coefficient corresponding to these two motion states. The results suggest that the coordination of the various sensors in the attitude adjustment unit during diving becomes closer and more efficient, which corresponds to the higher network measure of the diving state compared to the climbing state.

Masked autoencoders with handcrafted feature predictions: Transformer for weakly supervised esophageal cancer classification.

BACKGROUND AND OBJECTIVE: Esophageal cancer is a serious disease with a high prevalence in Eastern Asia. Histopathology tissue analysis stands as the gold standard in diagnosing esophageal cancer. In recent years, there has been a shift towards digitizing histopathological images into whole slide images (WSIs), progressively integrating them into cancer diagnostics. However, the gigapixel sizes of WSIs present significant storage and processing challenges, and they often lack localized annotations. To address this issue, multi-instance learning (MIL) has been introduced for WSI classification, utilizing weakly supervised learning for diagnosis analysis. By applying the principles of MIL to WSI analysis, it is possible to reduce the workload of pathologists by facilitating the generation of localized annotations. Nevertheless, the approach's effectiveness is hindered by the traditional simple aggregation operation and the domain shift resulting from the prevalent use of convolutional feature extractors pretrained on ImageNet. METHODS: We propose a MIL-based framework for WSI analysis and cancer classification. Concurrently, we introduce employing self-supervised learning, which obviates the need for manual annotation and demonstrates versatility in various tasks, to pretrain feature extractors. This method enhances the extraction of representative features from esophageal WSI for MIL, ensuring more robust and accurate performance. RESULTS: We build a comprehensive dataset of whole esophageal slide images and conduct extensive experiments utilizing this dataset. The performance on our dataset demonstrates the efficiency of our proposed MIL framework and the pretraining process, with our framework outperforming existing methods, achieving an accuracy of 93.07% and AUC (area under the curve) of 95.31%. CONCLUSION: This work proposes an effective MIL method to classify WSI of esophageal cancer. The promising results indicate that our cancer classification framework holds great potential in promoting the automatic whole esophageal slide image analysis.

WS-MTST: Weakly Supervised Multi-Label Brain Tumor Segmentation With Transformers.

Brain tumor segmentation is a key step in brain cancer diagnosis. Segmentation of brain tumor sub-regions, including necrotic, enhancing, and edematous regions, can provide more detailed guidance for clinical diagnosis. Weakly supervised brain tumor segmentation methods have received much attention because they do not require time-consuming pixel-level annotations. However, existing weakly supervised methods focus on the segmentation of the entire tumor region while ignoring the challenging task of multi-label segmentation for the tumor sub-regions. In this article, we propose a weakly supervised approach to solve the multi-label brain tumor segmentation problem. To the best of our knowledge, it's the first end-to-end multi-label weakly supervised segmentation model applied to brain tumor segmentation. With well-designed loss functions and a contrastive learning pre-training process, our proposed Transformer-based segmentation method (WS-MTST) has the ability to perform segmentation of brain tumor sub-regions. We conduct comprehensive experiments and demonstrate that our method reaches the state-of-the-art on the popular brain tumor dataset BraTS (from 2018 to 2020).

Nuclei segmentation with point annotations from pathology images via self-supervised learning and co-training.

Nuclei segmentation is a crucial task for whole slide image analysis in digital pathology. Generally, the segmentation performance of fully-supervised learning heavily depends on the amount and quality of the annotated data. However, it is time-consuming and expensive for professional pathologists to provide accurate pixel-level ground truth, while it is much easier to get coarse labels such as point annotations. In this paper, we propose a weakly-supervised learning method for nuclei segmentation that only requires point annotations for training. First, coarse pixel-level labels are derived from the point annotations based on the Voronoi diagram and the k-means clustering method to avoid overfitting. Second, a co-training strategy with an exponential moving average method is designed to refine the incomplete supervision of the coarse labels. Third, a self-supervised visual representation learning method is tailored for nuclei segmentation of pathology images that transforms the hematoxylin component images into the H&E stained images to gain better understanding of the relationship between the nuclei and cytoplasm. We comprehensively evaluate the proposed method using two public datasets. Both visual and quantitative results demonstrate the superiority of our method to the state-of-the-art methods, and its competitive performance compared to the fully-supervised methods. Codes are available at https://github.com/hust-linyi/SC-Net.

Criticality in transient behavior of coupled oscillator system toward chimera and synchronization.

Chimera states in spatiotemporal dynamical systems have been investigated in physical, chemical, and biological systems, while how the system is steering toward different final destinies upon spatially localized perturbation is still unknown. Through a systematic numerical analysis of the evolution of the spatiotemporal patterns of multi-chimera states, we uncover a critical behavior of the system in transient time toward either chimera or synchronization as the final stable state. We measure the critical values and the transient time of chimeras with different numbers of clusters. Then, based on an adequate verification, we fit and analyze the distribution of the transient time, which obeys power-law variation process with the increase in perturbation strengths. Moreover, the comparison between different clusters exhibits an interesting phenomenon, thus we find that the critical value of odd and even clusters will alternatively converge into a certain value from two sides, respectively, implying that this critical behavior can be modeled and enabling the articulation of a phenomenological model.

Assisting schizophrenia diagnosis using clinical electroencephalography and interpretable graph neural networks: a real-world and cross-site study.

Schizophrenia (SCZ) is a chronic and serious mental disorder with a high mortality rate. At present, there is a lack of objective, cost-effective and widely disseminated diagnosis tools to address this mental health crisis globally. Clinical electroencephalogram (EEG) is a noninvasive technique to measure brain activity with high temporal resolution, and accumulating evidence demonstrates that clinical EEG is capable of capturing abnormal SCZ neuropathology. Although EEG-based automated diagnostic tools have obtained impressive performance on individual datasets, the transportability of potential EEG biomarkers in cross-site real-world application is still an open question. To address the challenges of small sample sizes and population heterogeneity, we develop an advanced interpretable deep learning model using multimodal clinical EEG features and demographic information as inputs to graph neural networks, and further propose different transfer learning strategies to adapt to different clinical scenarios. Taking the disease discrimination of health control (HC) and SCZ with 1030 participants as a use case, our model is trained on a small clinical dataset (N = 188, Chinese) and enhanced using a large-scale public dataset (N = 508, American) of adult participants. Cross-site validation from an independent dataset of adult participants (N = 157, Chinese) produced stable performance, with AUCs of 0.793-0.852 and accuracies of 0.786-0.858 for different SCZ prevalence, respectively. In addition, cross-site validation from another dataset of adolescent boys (N = 84, Russian) yielded an AUC of 0.702 and an accuracy of 0.690. Moreover, feature visualization further revealed that the ranking of feature importance varied significantly among different datasets, and that EEG theta and alpha band power appeared to be the most significant and translational biomarkers of SCZ pathology. Overall, our promising results demonstrate the feasibility of SCZ discrimination using EEG biomarkers in multiple clinical settings.

Interconnected ordinal pattern complex network for characterizing the spatial coupling behavior of gas-liquid two-phase flow.

The complex phase interactions of the two-phase flow are a key factor in understanding the flow pattern evolutional mechanisms, yet these complex flow behaviors have not been well understood. In this paper, we employ a series of gas-liquid two-phase flow multivariate fluctuation signals as observations and propose a novel interconnected ordinal pattern network to investigate the spatial coupling behaviors of the gas-liquid two-phase flow patterns. In addition, we use two network indices, which are the global subnetwork mutual information (I) and the global subnetwork clustering coefficient (C), to quantitatively measure the spatial coupling strength of different gas-liquid flow patterns. The gas-liquid two-phase flow pattern evolutionary behaviors are further characterized by calculating the two proposed coupling indices under different flow conditions. The proposed interconnected ordinal pattern network provides a novel tool for a deeper understanding of the evolutional mechanisms of the multi-phase flow system, and it can also be used to investigate the coupling behaviors of other complex systems with multiple observations.

EEG-Based Motor Imagery Recognition Framework via Multisubject Dynamic Transfer and Iterative Self-Training.

A robust decoding model that can efficiently deal with the subject and period variation is urgently needed to apply the brain-computer interface (BCI) system. The performance of most electroencephalogram (EEG) decoding models depends on the characteristics of specific subjects and periods, which require calibration and training with annotated data prior to application. However, this situation will become unacceptable as it would be difficult for subjects to collect data for an extended period, especially in the rehabilitation process of disability based on motor imagery (MI). To address this issue, we propose an unsupervised domain adaptation framework called iterative self-training multisubject domain adaptation (ISMDA) that focuses on the offline MI task. First, the feature extractor is purposefully designed to map the EEG to a latent space of discriminative representations. Second, the attention module based on dynamic transfer matches the source domain and target domain samples with a higher coincidence degree in latent space. Then, an independent classifier oriented to the target domain is employed in the first stage of the iterative training process to cluster the samples of the target domain through similarity. Finally, a pseudolabel algorithm based on certainty and confidence is employed in the second stage of the iterative training process to adequately calibrate the error between prediction and empirical probabilities. To evaluate the effectiveness of the model, extensive testing has been performed on three publicly available MI datasets, the BCI IV IIa, the High gamma dataset, and Kwon et al. datasets. The proposed method achieved 69.51%, 82.38%, and 90.98% cross-subject classification accuracy on the three datasets, which outperforms the current state-of-the-art offline algorithms. Meanwhile, all results demonstrated that the proposed method could address the main challenges of the offline MI paradigm.

A Brain Network Analysis-Based Double Way Deep Neural Network for Emotion Recognition.

Constructing reliable and effective models to recognize human emotional states has become an important issue in recent years. In this article, we propose a double way deep residual neural network combined with brain network analysis, which enables the classification of multiple emotional states. To begin with, we transform the emotional EEG signals into five frequency bands by wavelet transform and construct brain networks by inter-channel correlation coefficients. These brain networks are then fed into a subsequent deep neural network block which contains several modules with residual connection and enhanced by channel attention mechanism and spatial attention mechanism. In the second way of the model, we feed the emotional EEG signals directly into another deep neural network block to extract temporal features. At the end of the two ways, the features are concatenated for classification. To verify the effectiveness of our proposed model, we carried out a series of experiments to collect emotional EEG from eight subjects. The average accuracy of the proposed model on our emotional dataset is 94.57%. In addition, the evaluation results on public databases SEED and SEED-IV are 94.55% and 78.91%, respectively, demonstrating the superiority of our model in emotion recognition tasks.

Characterizing gas-liquid two-phase flow behavior using complex network and deep learning.

Gas-liquid two-phase flow is polymorphic and unstable, and characterizing its flow behavior is a major challenge in the study of multiphase flow. We first conduct dynamic experiments on gas-liquid two-phase flow in a vertical tube and obtain multi-channel signals using a self-designed four-sector distributed conductivity sensor. In order to characterize the evolution of gas-liquid two-phase flow, we transform the obtained signals using the adaptive optimal kernel time-frequency representation and build a complex network based on the time-frequency energy distribution. As quantitative indicators, global clustering coefficients of the complex network at various sparsity levels are computed to analyze the dynamic behavior of various flow structures. The results demonstrate that the proposed approach enables effective analysis of multi-channel measurement information for revealing the evolutionary mechanisms of gas-liquid two-phase flow. Furthermore, for the purpose of flow structure recognition, we propose a temporal-spatio convolutional neural network and achieve a classification accuracy of 95.83%.

Reinforcement learning for robust stabilization of nonlinear systems with asymmetric saturating actuators.

We study the robust stabilization problem of a class of nonlinear systems with asymmetric saturating actuators and mismatched disturbances. Initially, we convert such a robust stabilization problem into a nonlinear-constrained optimal control problem by constructing a discounted cost function for the auxiliary system. Then, for the purpose of solving the nonlinear-constrained optimal control problem, we develop a simultaneous policy iteration (PI) in the reinforcement learning framework. The implementation of the simultaneous PI relies on an actor-critic architecture, which employs actor and critic neural networks (NNs) to separately approximate the control policy and the value function. To determine the actor and critic NNs' weights, we use the approach of weighted residuals together with the typical Monte-Carlo integration technique. Finally, we perform simulations of two nonlinear plants to validate the established theoretical claims.

Transfer Learning With Optimal Transportation and Frequency Mixup for EEG-Based Motor Imagery Recognition.

Electroencephalography-based Brain Computer Interfaces (BCIs) invariably have a degenerate performance due to the considerable individual variability. To address this problem, we develop a novel domain adaptation method with optimal transport and frequency mixup for cross-subject transfer learning in motor imagery BCIs. Specifically, the preprocessed EEG signals from source and target domain are mapped into latent space with an embedding module, where the representation distributions and label distributions across domains have a large discrepancy. We assume that there exists a non-linear coupling matrix between both domains, which can be utilized to estimate the distance of joint distributions for different domains. Depending on the optimal transport, the Wasserstein distance between source and target domains is minimized, yielding the alignment of joint distributions. Moreover, a new mixup strategy is also introduced to generalize the model, where the inputs trials are mixed in frequency domain rather than in raw space. The extensive experiments on three evaluation benchmarks are conducted to validate the proposed framework. All the results demonstrate that our method achieves a superior performance than previous state-of-the-art domain adaptation approaches.

Cross-subject emotion recognition using visibility graph and genetic algorithm-based convolution neural network.

An efficient emotion recognition model is an important research branch in electroencephalogram (EEG)-based brain-computer interfaces. However, the input of the emotion recognition model is often a whole set of EEG channels obtained by electrodes placed on subjects. The unnecessary information produced by redundant channels affects the recognition rate and depletes computing resources, thereby hindering the practical applications of emotion recognition. In this work, we aim to optimize the input of EEG channels using a visibility graph (VG) and genetic algorithm-based convolutional neural network (GA-CNN). First, we design an experiment to evoke three types of emotion states using movies and collect the multi-channel EEG signals of each subject under different emotion states. Then, we construct VGs for each EEG channel and derive nonlinear features representing each EEG channel. We employ the genetic algorithm (GA) to find the optimal subset of EEG channels for emotion recognition and use the recognition results of the CNN as fitness values. The experimental results show that the recognition performance of the proposed method using a subset of EEG channels is superior to that of the CNN using all channels for each subject. Last, based on the subset of EEG channels searched by the GA-CNN, we perform cross-subject emotion recognition tasks employing leave-one-subject-out cross-validation. These results demonstrate the effectiveness of the proposed method in recognizing emotion states using fewer EEG channels and further enrich the methods of EEG classification using nonlinear features.

Decentralized Neurocontroller Design With Critic Learning for Nonlinear-Interconnected Systems.

We consider the decentralized control problem of a class of continuous-time nonlinear systems with mismatched interconnections. Initially, with the discounted cost functions being introduced to auxiliary subsystems, we have the decentralized control problem converted into a set of optimal control problems. To derive solutions to these optimal control problems, we first present the related Hamilton-Jacobi-Bellman equations (HJBEs). Then, we develop a novel critic learning method to solve these HJBEs. To implement the newly developed critic learning approach, we only use critic neural networks (NNs) and tune their weight vectors via the combination of a modified gradient descent method and concurrent learning. By using the present critic learning method, we not only remove the restriction of initial admissible control but also relax the persistence-of-excitation condition. After that, we employ Lyapunov's direct method to demonstrate that the critic NNs' weight estimation error and the states of closed-loop auxiliary systems are stable in the sense of uniform ultimate boundedness. Finally, we separately provide a nonlinear-interconnected plant and an unstable interconnected power system to validate the present critic learning approach.

A gradient-based automatic optimization CNN framework for EEG state recognition.

Objective. The electroencephalogram (EEG) signal, as a data carrier that can contain a large amount of information about the human brain in different states, is one of the most widely used metrics for assessing human psychophysiological states. Among a variety of analysis methods, deep learning, especially convolutional neural network (CNN), has achieved remarkable results in recent years as a method to effectively extract features from EEG signals. Although deep learning has the advantages of automatic feature extraction and effective classification, it also faces difficulties in network structure design and requires an army of prior knowledge. Automating the design of these hyperparameters can therefore save experts' time and manpower. Neural architecture search techniques have thus emerged.Approach. In this paper, based on an existing gradient-based neural architecture search (NAS) algorithm, partially-connected differentiable architecture search (PC-DARTS), with targeted improvements and optimizations for the characteristics of EEG signals. Specifically, we establish the model architecture step by step based on the manually designed deep learning models for EEG discrimination by retaining the framework of the search algorithm and performing targeted optimization of the model search space. Corresponding features are extracted separately according to the frequency domain, time domain characteristics of the EEG signal and the spatial position of the EEG electrode. The architecture was applied to EEG-based emotion recognition and driver drowsiness assessment tasks.Main results. The results illustrate that compared with the existing methods, the model architecture obtained in this paper can achieve competitive overall accuracy and better standard deviation in both tasks.Significance. Therefore, this approach is an effective migration of NAS technology into the field of EEG analysis and has great potential to provide high-performance results for other types of classification and prediction tasks. This can effectively reduce the time cost for researchers and facilitate the application of CNN in more areas.

Howling Detection and Suppression Based on Segmented Notch Filtering.

The existing adaptive echo cancellation based howling (typically in hearing aids) removal methods have several drawbacks such as insufficient attenuation of the howling component, slow response and nonlinear distortion. To solve these problems, we propose a segmented notch filtering based scheme. Specifically, firstly, it is proved that the attenuation value can reach -330 dB at any detected howling frequency; secondly, the filter coefficients can be readily calculated by a closed-form formula, yielding a fast response to the sudden howling accident; thirdly, the closed-form formula of this filter is theoretically an even function, indicating that this filter possesses a linear transfer characteristic. In combination with proper segmentation and precisely removing these transient samples arising from FIR (Finite Impulsive Response) filtering, nonlinear distortion can be entirely avoided. Experimental results show that our proposed scheme can not only accurately estimate the howling frequency, but can also completely remove it, which yields a high-quality output waveform with a recovery SNR of about 22 dB. Therefore, the proposed segmented notching based scheme possesses vast potential for hearing aid development and other relevant applications.

COVID-19 Screening in Chest X-Ray Images Using Lung Region Priors.

Early screening of COVID-19 is essential for pandemic control, and thus to relieve stress on the health care system. Lung segmentation from chest X-ray (CXR) is a promising method for early diagnoses of pulmonary diseases. Recently, deep learning has achieved great success in supervised lung segmentation. However, how to effectively utilize the lung region in screening COVID-19 still remains a challenge due to domain shift and lack of manual pixel-level annotations. We hereby propose a multi-appearance COVID-19 screening framework by using lung region priors derived from CXR images. Firstly, we propose a multi-scale adversarial domain adaptation network (MS-AdaNet) to boost the cross-domain lung segmentation task as the prior knowledge to the classification network. Then, we construct a multi-appearance network (MA-Net), which is composed of three sub-networks to realize multi-appearance feature extraction and fusion using lung region priors. At last, we can obtain prediction results from normal, viral pneumonia, and COVID-19 using the proposed MA-Net. We extend the proposed MS-AdaNet for lung segmentation task on three different public CXR datasets. The results suggest that the MS-AdaNet outperforms contrastive methods in cross-domain lung segmentation. Moreover, experiments reveal that the proposed MA-Net achieves accuracy of 98.83 % and F1-score of 98.71 % on COVID-19 screening. The results indicate that the proposed MA-Net can obtain significant performance on COVID-19 screening.