EEG-Oriented Self-Supervised Learning With Triple Information Pathways Network.

Recently, deep learning-based electroencephalogram (EEG) analysis and decoding have attracted widespread attention for monitoring the clinical condition of users and identifying their intention/emotion. Nevertheless, the existing methods generally model EEG signals with limited viewpoints or restricted concerns about the characteristics of the EEG signals, and thus represent complex spectro-/spatiotemporal patterns and suffer from high variability. In this work, we propose the novel EEG-oriented self-supervised learning methods and a novel deep architecture to learn rich representation, including information about the diverse spectral characteristics of neural oscillations, the spatial properties of electrode sensor distribution, and the temporal patterns of both the global and local viewpoints. Along with the proposed self-supervision strategies and deep architectures, we devise a feature normalization strategy to resolve the intra-/inter-subject variability problem. We demonstrate the validity of our proposed deep learning framework on the four publicly available datasets by conducting comparisons with the state of the art baselines. It is also noteworthy that we exploit the same network architecture for the four different EEG paradigms and outperform the comparison methods, thereby meeting the challenge of the task-dependent network architecture engineering in EEG-based applications.

Mutual Information-Driven Subject-Invariant and Class-Relevant Deep Representation Learning in BCI.

In recent years, deep learning-based feature representation methods have shown a promising impact on electroencephalography (EEG)-based brain-computer interface (BCI). Nonetheless, owing to high intra- and inter-subject variabilities, many studies on decoding EEG were designed in a subject-specific manner by using calibration samples, with no concern of its practical use, hampered by time-consuming steps and a large data requirement. To this end, recent studies adopted a transfer learning strategy, especially domain adaptation techniques. Among those, we have witnessed the potential of adversarial learning-based transfer learning in BCIs. In the meantime, it is known that adversarial learning-based domain adaptation methods are prone to negative transfer that disrupts learning generalized feature representations, applicable to diverse domains, for example, subjects or sessions in BCIs. In this article, we propose a novel framework that learns class-relevant and subject-invariant feature representations in an information-theoretic manner, without using adversarial learning. To be specific, we devise two operational components in a deep network that explicitly estimate mutual information between feature representations: 1) to decompose features in an intermediate layer into class-relevant and class-irrelevant ones and 2) to enrich class-discriminative feature representation. On two large EEG datasets, we validated the effectiveness of our proposed framework by comparing with several comparative methods in performance. Furthermore, we conducted rigorous analyses by performing an ablation study in regard to the components in our network, explaining our model's decision on input EEG signals via layer-wise relevance propagation, and visualizing the distribution of learned features via t-SNE.

TransSleep: Transitioning-Aware Attention-Based Deep Neural Network for Sleep Staging.

Sleep staging is essential for sleep assessment and plays a vital role as a health indicator. Many recent studies have devised various machine/deep learning methods for sleep staging. However, two key challenges hinder the practical use of those methods: 1) effectively capturing salient waveforms in sleep signals and 2) correctly classifying confusing stages in transitioning epochs. In this study, we propose a novel deep neural-network structure, TransSleep, that captures distinctive local temporal patterns and distinguishes confusing stages using two auxiliary tasks. In particular, TransSleep captures salient waveforms in sleep signals by an attention-based multiscale feature extractor and correctly classifies confusing stages in transitioning epochs, while modeling contextual relationships with two auxiliary tasks. Results show that TransSleep achieves promising performance in automatic sleep staging. The validity of TransSleep is demonstrated by its state-of-the-art performance on two publicly available datasets: 1) Sleep-EDF and 2) MASS. Furthermore, we performed ablations to analyze our results from different perspectives. Based on our overall results, we believe that TransSleep has immense potential to provide new insights into deep-learning-based sleep staging.

A Deep Generative-Discriminative Learning for Multimodal Representation in Imaging Genetics.

Imaging genetics, one of the foremost emerging topics in the medical imaging field, analyzes the inherent relations between neuroimaging and genetic data. As deep learning has gained widespread acceptance in many applications, pioneering studies employed deep learning frameworks for imaging genetics. However, existing approaches suffer from some limitations. First, they often adopt a simple strategy for joint learning of phenotypic and genotypic features. Second, their findings have not been extended to biomedical applications, e.g., degenerative brain disease diagnosis and cognitive score prediction. Finally, existing studies perform insufficient and inappropriate analyses from the perspective of data science and neuroscience. In this work, we propose a novel deep learning framework to simultaneously tackle the aforementioned issues. Our proposed framework learns to effectively represent the neuroimaging and the genetic data jointly, and achieves state-of-the-art performance when used for Alzheimer's disease and mild cognitive impairment identification. Furthermore, unlike the existing methods, the framework enables learning the relation between imaging phenotypes and genotypes in a nonlinear way without any prior neuroscientific knowledge. To demonstrate the validity of our proposed framework, we conducted experiments on a publicly available dataset and analyzed the results from diverse perspectives. Based on our experimental results, we believe that the proposed framework has immense potential to provide new insights and perspectives in deep learning-based imaging genetics studies.

Semi-supervised generative and discriminative adversarial learning for motor imagery-based brain-computer interface.

Convolutional neural networks (CNNs), which can recognize structural/configuration patterns in data with different architectures, have been studied for feature extraction. However, challenges remain regarding leveraging advanced deep learning methods in BCIs. We focus on problems of small-sized training samples and interpretability of the learned parameters and leverages a semi-supervised generative and discriminative learning framework that effectively utilizes synthesized samples with real samples to discover class-discriminative features. Our framework learns the distributional characteristics of EEG signals in an embedding space using a generative model. By using artificially generated and real EEG signals, our framework finds class-discriminative spatio-temporal feature representations that help to correctly discriminate input EEG signals. It is noteworthy that the framework facilitates the exploitation of real, unlabeled samples to better uncover the underlying patterns inherent in a user's EEG signals. To validate our framework, we conducted experiments comparing our method with conventional linear models by utilizing variants of three existing CNN architectures as generator networks and measuring the performance on three public datasets. Our framework exhibited statistically significant improvements over the competing methods. We investigated the learned network via activation pattern maps and visualized generated artificial samples to empirically justify the stability and neurophysiological plausibility of our model.

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain-Computer Interfaces.

Brain-computer interfaces (BCIs) utilizing machine learning techniques are an emerging technology that enables a communication pathway between a user and an external system, such as a computer. Owing to its practicality, electroencephalography (EEG) is one of the most widely used measurements for BCI. However, EEG has complex patterns and EEG-based BCIs mostly involve a cost/time-consuming calibration phase; thus, acquiring sufficient EEG data is rarely possible. Recently, deep learning (DL) has had a theoretical/practical impact on BCI research because of its use in learning representations of complex patterns inherent in EEG. Moreover, algorithmic advances in DL facilitate short/zero-calibration in BCI, thereby suppressing the data acquisition phase. Those advancements include data augmentation (DA), increasing the number of training samples without acquiring additional data, and transfer learning (TL), taking advantage of representative knowledge obtained from one dataset to address the so-called data insufficiency problem in other datasets. In this study, we review DL-based short/zero-calibration methods for BCI. Further, we elaborate methodological/algorithmic trends, highlight intriguing approaches in the literature, and discuss directions for further research. In particular, we search for generative model-based and geometric manipulation-based DA methods. Additionally, we categorize TL techniques in DL-based BCIs into explicit and implicit methods. Our systematization reveals advances in the DA and TL methods. Among the studies reviewed herein, ~45% of DA studies used generative model-based techniques, whereas ~45% of TL studies used explicit knowledge transferring strategy. Moreover, based on our literature review, we recommend an appropriate DA strategy for DL-based BCIs and discuss trends of TLs used in DL-based BCIs.

Electrophysiological Decoding of Spatial and Color Processing in Human Prefrontal Cortex.

The prefrontal cortex (PFC) plays a pivotal role in goal-directed cognition, yet its representational code remains an open problem with decoding techniques ineffective in disentangling task-relevant variables from PFC. Here we applied regularized linear discriminant analysis to human scalp EEG data and were able to distinguish a mental-rotation task versus a color-perception task with 87% decoding accuracy. Dorsal and ventral areas in lateral PFC provided the dominant features dissociating the two tasks. Our findings show that EEG can reliably decode two independent task states from PFC and emphasize the PFC dorsal/ventral functional specificity in processing the where rotation task versus the what color task.

A unified framework for personalized regions selection and functional relation modeling for early MCI identification.

Resting-state functional magnetic resonance imaging (rs-fMRI) has been widely adopted to investigate functional abnormalities in brain diseases. Rs-fMRI data is unsupervised in nature because the psychological and neurological labels are coarse-grained, and no accurate region-wise label is provided along with the complex co-activities of multiple regions. To the best of our knowledge, most studies regarding univariate group analysis or multivariate pattern recognition for brain disease identification have focused on discovering functional characteristics shared across subjects; however, they have paid less attention to individual properties of neural activities that result from different symptoms or degrees of abnormality. In this work, we propose a novel framework that can identify subjects with early-stage mild cognitive impairment (eMCI) and consider individual variability by learning functional relations from automatically selected regions of interest (ROIs) for each subject concurrently. In particular, we devise a deep neural network composed of a temporal embedding module, an ROI selection module, and a disease-identification module. Notably, the ROI selection module is equipped with a reinforcement learning mechanism so it adaptively selects ROIs to facilitate the learning of discriminative feature representations from a temporally embedded blood-oxygen-level-dependent signals. Furthermore, our method allows us to capture the functional relations of a subject-specific ROI subset through the use of a graph-based neural network. Our method considers individual characteristics for diagnosis, as opposed to most conventional methods that identify the same biomarkers across subjects within a group. Based on the ADNI cohort, we validate the effectiveness of our method by presenting the superior performance of our network in eMCI identification. Furthermore, we provide insightful neuroscientific interpretations by analyzing the regions selected for the eMCI classification.