Revealing the spatiotemporal brain dynamics of covert speech compared with overt speech: A simultaneous EEG-fMRI study.

Covert speech (CS) refers to speaking internally to oneself without producing any sound or movement. CS is involved in multiple cognitive functions and disorders. Reconstructing CS content by brain-computer interface (BCI) is also an emerging technique. However, it is still controversial whether CS is a truncated neural process of overt speech (OS) or involves independent patterns. Here, we performed a word-speaking experiment with simultaneous EEG-fMRI. It involved 32 participants, who generated words both overtly and covertly. By integrating spatial constraints from fMRI into EEG source localization, we precisely estimated the spatiotemporal dynamics of neural activity. During CS, EEG source activity was localized in three regions: the left precentral gyrus, the left supplementary motor area, and the left putamen. Although OS involved more brain regions with stronger activations, CS was characterized by an earlier event-locked activation in the left putamen (peak at 262 ms versus 1170 ms). The left putamen was also identified as the only hub node within the functional connectivity (FC) networks of both OS and CS, while showing weaker FC strength towards speech-related regions in the dominant hemisphere during CS. Path analysis revealed significant multivariate associations, indicating an indirect association between the earlier activation in the left putamen and CS, which was mediated by reduced FC towards speech-related regions. These findings revealed the specific spatiotemporal dynamics of CS, offering insights into CS mechanisms that are potentially relevant for future treatment of self-regulation deficits, speech disorders, and development of BCI speech applications.

µ-STAR: A novel framework for spatio-temporal M/EEG source imaging optimized by microstates.

Source imaging of Electroencephalography (EEG) and Magnetoencephalography (MEG) provides a noninvasive way of monitoring brain activities with high spatial and temporal resolution. In order to address this highly ill-posed problem, conventional source imaging models adopted spatio-temporal constraints that assume spatial stability of the source activities, neglecting the transient characteristics of M/EEG. In this work, a novel source imaging method µ-STAR that includes a microstate analysis and a spatio-temporal Bayesian model was introduced to address this problem. Specifically, the microstate analysis was applied to achieve automatic determination of time window length with quasi-stable source activity pattern for optimal reconstruction of source dynamics. Then a user-specific spatial prior and data-driven temporal basis functions were utilized to characterize the spatio-temporal information of sources within each state. The solution of the source reconstruction was obtained through a computationally efficient algorithm based upon variational Bayesian and convex analysis. The performance of the µ-STAR was first assessed through numerical simulations, where we found that the determination and inclusion of optimal temporal length in the spatio-temporal prior significantly improved the performance of source reconstruction. More importantly, the µ-STAR model achieved robust performance under various settings (i.e., source numbers/areas, SNR levels, and source depth) with fast convergence speed compared with five widely-used benchmark models (including wMNE, STV, SBL, BESTIES, & SI-STBF). Additional validations on real data were then performed on two publicly-available datasets (including block-design face-processing ERP and continuous resting-state EEG). The reconstructed source activities exhibited spatial and temporal neurophysiologically plausible results consistent with previously-revealed neural substrates, thereby further proving the feasibility of the µ-STAR model for source imaging in various applications.

Machine Learning Driven Synthesis of Few-Layered WTe2 with Geometrical Control.

Reducing the lateral scale of two-dimensional (2D) materials to one-dimensional (1D) has attracted substantial research interest not only to achieve competitive electronic applications but also for the exploration of fundamental physical properties. Controllable synthesis of high-quality 1D nanoribbons (NRs) is thus highly desirable and essential for further study. Here, we report the implementation of supervised machine learning (ML) for the chemical vapor deposition (CVD) synthesis of high-quality quasi-1D few-layered WTe2 NRs. Feature importance analysis indicates that H2 gas flow rate has a profound influence on the formation of WTe2, and the source ratio governs the sample morphology. Notably, the growth mechanism of 1T' few-layered WTe2 NRs is further proposed, which provides new insights for the growth of intriguing 2D and 1D tellurides and may inspire the growth strategies for other 1D nanostructures. Our findings suggest the effectiveness and capability of ML in guiding the synthesis of 1D nanostructures, opening up new opportunities for intelligent materials development.

Actigraphy studies and clinical and biobehavioural correlates in schizophrenia: a systematic review.

Actigraphy is a non-invasive method of monitoring circadian rhythms and motor activity. We systematically reviewed extant evidence until September 2018 pertaining to actigraphy use in schizophrenia, its clinical/biological correlates and posit future research directions. Within 38 included studies involving 2700 subjects, patients with schizophrenia generally have lower motor activity levels, poorer sleep quality and efficiency, increased sleep fragmentation and duration compared with healthy controls. Lowered motor activity and longer sleep duration in patients were associated with greater severity of negative symptoms. Less structured motor activity and decreased sleep quality were associated with greater severity of positive symptoms, worse cognitive functioning involving attention and processing speed, illness chronicity, higher antipsychotic dose, and poorer quality of life. Correlations of actigraphic measures with biological factors are sparse with inconclusive results. Future studies with larger sample sets may adopt a multimodal, longitudinal approach which examines both motor and sleep activity, triangulates clinical, actigraphic and biological measures to clarify their inter-relationships and inform risk prediction of illness onset, course, and treatment response over time.

Facilitating effects of transcranial direct current stimulation on motor imagery brain-computer interface with robotic feedback for stroke rehabilitation.

OBJECTIVE: To investigate the efficacy and effects of transcranial direct current stimulation (tDCS) on motor imagery brain-computer interface (MI-BCI) with robotic feedback for stroke rehabilitation. DESIGN: A sham-controlled, randomized controlled trial. SETTING: Patients recruited through a hospital stroke rehabilitation program. PARTICIPANTS: Subjects (N=19) who incurred a stroke 0.8 to 4.3 years prior, with moderate to severe upper extremity functional impairment, and passed BCI screening. INTERVENTIONS: Ten sessions of 20 minutes of tDCS or sham before 1 hour of MI-BCI with robotic feedback upper limb stroke rehabilitation for 2 weeks. Each rehabilitation session comprised 8 minutes of evaluation and 1 hour of therapy. MAIN OUTCOME MEASURES: Upper extremity Fugl-Meyer Motor Assessment (FMMA) scores measured end-intervention at week 2 and follow-up at week 4, online BCI accuracies from the evaluation part, and laterality coefficients of the electroencephalogram (EEG) from the therapy part of the 10 rehabilitation sessions. RESULTS: FMMA score improved in both groups at week 4, but no intergroup differences were found at any time points. Online accuracies of the evaluation part from the tDCS group were significantly higher than those from the sham group. The EEG laterality coefficients from the therapy part of the tDCS group were significantly higher than those of the sham group. CONCLUSIONS: The results suggest a role for tDCS in facilitating motor imagery in stroke.

Subject-independent meta-learning framework towards optimal training of EEG-based classifiers.

Advances in deep learning have shown great promise towards the application of performing high-accuracy Electroencephalography (EEG) signal classification in a variety of tasks. However, many EEG-based datasets are often plagued by the issue of high inter-subject signal variability. Robust deep learning models are notoriously difficult to train under such scenarios, often leading to subpar or widely varying performance across subjects under the leave-one-subject-out paradigm. Recently, the model agnostic meta-learning framework was introduced as a way to increase the model's ability to generalize towards new tasks. While the original framework focused on task-based meta-learning, this research aims to show that the meta-learning methodology can be modified towards subject-based signal classification while maintaining the same task objectives and achieve state-of-the-art performance. Namely, we propose the novel implementation of a few/zero-shot subject-independent meta-learning framework towards multi-class inner speech and binary class motor imagery classification. Compared to current subject-adaptive methods which utilize large number of labels from the target, the proposed framework shows its effectiveness in training zero-calibration and few-shot models for subject-independent EEG classification. The proposed few/zero-shot subject-independent meta-learning mechanism performs well on both small and large datasets and achieves robust, generalized performance across subjects. The results obtained shows a significant improvement over the current state-of-the-art, with the binary class motor imagery achieving 88.70% and the accuracy of multi-class inner speech achieving an average of 31.15%. Codes will be made available to public upon publication.

MASA-TCN: Multi-Anchor Space-Aware Temporal Convolutional Neural Networks for Continuous and Discrete EEG Emotion Recognition.

Emotion recognition from electroencephalogram (EEG) signals is a critical domain in biomedical research with applications ranging from mental disorder regulation to human-computer interaction. In this paper, we address two fundamental aspects of EEG emotion recognition: continuous regression of emotional states and discrete classification of emotions. While classification methods have garnered significant attention, regression methods remain relatively under-explored. To bridge this gap, we introduce MASA-TCN, a novel unified model that leverages the spatial learning capabilities of Temporal Convolutional Networks (TCNs) for EEG emotion regression and classification tasks. The key innovation lies in the introduction of a space-aware temporal layer, which empowers TCN to capture spatial relationships among EEG electrodes, enhancing its ability to discern nuanced emotional states. Additionally, we design a multi-anchor block with attentive fusion, enabling the model to adaptively learn dynamic temporal dependencies within the EEG signals. Experiments on two publicly available datasets show that MASA-TCN achieves higher results than the state-of-the-art methods for both EEG emotion regression and classification tasks.

Aggregating intrinsic information to enhance BCI performance through federated learning.

Insufficient data is a long-standing challenge for Brain-Computer Interface (BCI) to build a high-performance deep learning model. Though numerous research groups and institutes collect a multitude of EEG datasets for the same BCI task, sharing EEG data from multiple sites is still challenging due to the heterogeneity of devices. The significance of this challenge cannot be overstated, given the critical role of data diversity in fostering model robustness. However, existing works rarely discuss this issue, predominantly centering their attention on model training within a single dataset, often in the context of inter-subject or inter-session settings. In this work, we propose a hierarchical personalized Federated Learning EEG decoding (FLEEG) framework to surmount this challenge. This innovative framework heralds a new learning paradigm for BCI, enabling datasets with disparate data formats to collaborate in the model training process. Each client is assigned a specific dataset and trains a hierarchical personalized model to manage diverse data formats and facilitate information exchange. Meanwhile, the server coordinates the training procedure to harness knowledge gleaned from all datasets, thus elevating overall performance. The framework has been evaluated in Motor Imagery (MI) classification with nine EEG datasets collected by different devices but implementing the same MI task. Results demonstrate that the proposed framework can boost classification performance up to 8.4% by enabling knowledge sharing between multiple datasets, especially for smaller datasets. Visualization results also indicate that the proposed framework can empower the local models to put a stable focus on task-related areas, yielding better performance. To the best of our knowledge, this is the first end-to-end solution to address this important challenge.

Improving SSVEP-BCI Performance Through Repetitive Anodal tDCS-Based Neuromodulation: Insights From Fractal EEG and Brain Functional Connectivity.

This study embarks on a comprehensive investigation of the effectiveness of repetitive transcranial direct current stimulation (tDCS)-based neuromodulation in augmenting steady-state visual evoked potential (SSVEP) brain-computer interfaces (BCIs), alongside exploring pertinent electroencephalography (EEG) biomarkers for assessing brain states and evaluating tDCS efficacy. EEG data were garnered across three distinct task modes (eyes open, eyes closed, and SSVEP stimulation) and two neuromodulation patterns (sham-tDCS and anodal-tDCS). Brain arousal and brain functional connectivity were measured by extracting features of fractal EEG and information flow gain, respectively. Anodal-tDCS led to diminished offsets and enhanced information flow gains, indicating improvements in both brain arousal and brain information transmission capacity. Additionally, anodal-tDCS markedly enhanced SSVEP-BCIs performance as evidenced by increased amplitudes and accuracies, whereas sham-tDCS exhibited lesser efficacy. This study proffers invaluable insights into the application of neuromodulation methods for bolstering BCI performance, and concurrently authenticates two potent electrophysiological markers for multifaceted characterization of brain states.

Federated Graph Neural Networks: Overview, Techniques, and Challenges.

Graph neural networks (GNNs) have attracted extensive research attention in recent years due to their capability to progress with graph data and have been widely used in practical applications. As societies become increasingly concerned with the need for data privacy protection, GNNs face the need to adapt to this new normal. Besides, as clients in federated learning (FL) may have relationships, more powerful tools are required to utilize such implicit information to boost performance. This has led to the rapid development of the emerging research field of federated GNNs (FedGNNs). This promising interdisciplinary field is highly challenging for interested researchers to grasp. The lack of an insightful survey on this topic further exacerbates the entry difficulty. In this article, we bridge this gap by offering a comprehensive survey of this emerging field. We propose a 2-D taxonomy of the FedGNN literature: 1) the main taxonomy provides a clear perspective on the integration of GNNs and FL by analyzing how GNNs enhance FL training as well as how FL assists GNN training and 2) the auxiliary taxonomy provides a view on how FedGNNs deal with heterogeneity across FL clients. Through discussions of key ideas, challenges, and limitations of existing works, we envision future research directions that can help build more robust, explainable, efficient, fair, inductive, and comprehensive FedGNNs.

Transferring a deep learning model from healthy subjects to stroke patients in a motor imagery brain-computer interface.

Objective.Motor imagery (MI) brain-computer interfaces (BCIs) based on electroencephalogram (EEG) have been developed primarily for stroke rehabilitation, however, due to limited stroke data, current deep learning methods for cross-subject classification rely on healthy data. This study aims to assess the feasibility of applying MI-BCI models pre-trained using data from healthy individuals to detect MI in stroke patients.Approach.We introduce a new transfer learning approach where features from two-class MI data of healthy individuals are used to detect MI in stroke patients. We compare the results of the proposed method with those obtained from analyses within stroke data. Experiments were conducted using Deep ConvNet and state-of-the-art subject-specific machine learning MI classifiers, evaluated on OpenBMI two-class MI-EEG data from healthy subjects and two-class MI versus rest data from stroke patients.Main results.Results of our study indicate that through domain adaptation of a model pre-trained using healthy subjects' data, an average MI detection accuracy of 71.15% (±12.46%) can be achieved across 71 stroke patients. We demonstrate that the accuracy of the pre-trained model increased by 18.15% after transfer learning (p<0.001). Additionally, the proposed transfer learning method outperforms the subject-specific results achieved by Deep ConvNet and FBCSP, with significant enhancements of 7.64% (p<0.001) and 5.55% (p<0.001) in performance, respectively. Notably, the healthy-to-stroke transfer learning approach achieved similar performance to stroke-to-stroke transfer learning, with no significant difference (p>0.05). Explainable AI analyses using transfer models determined channel relevance patterns that indicate contributions from the bilateral motor, frontal, and parietal regions of the cortex towards MI detection in stroke patients.Significance.Transfer learning from healthy to stroke can enhance the clinical use of BCI algorithms by overcoming the challenge of insufficient clinical data for optimal training.

Leveraging temporal dependency for cross-subject-MI BCIs by contrastive learning and self-attention.

Brain-computer interfaces (BCIs) built based on motor imagery paradigm have found extensive utilization in motor rehabilitation and the control of assistive applications. However, traditional MI-BCI systems often exhibit suboptimal classification performance and require significant time for new users to collect subject-specific training data. This limitation diminishes the user-friendliness of BCIs and presents significant challenges in developing effective subject-independent models. In response to these challenges, we propose a novel subject-independent framework for learning temporal dependency for motor imagery BCIs by Contrastive Learning and Self-attention (CLS). In CLS model, we incorporate self-attention mechanism and supervised contrastive learning into a deep neural network to extract important information from electroencephalography (EEG) signals as features. We evaluate the CLS model using two large public datasets encompassing numerous subjects in a subject-independent experiment condition. The results demonstrate that CLS outperforms six baseline algorithms, achieving a mean classification accuracy improvement of 1.3 % and 4.71 % than the best algorithm on the Giga dataset and OpenBMI dataset, respectively. Our findings demonstrate that CLS can effectively learn invariant discriminative features from training data obtained from non-target subjects, thus showcasing its potential for building models for new users without the need for calibration.

TASA: Temporal Attention With Spatial Autoencoder Network for Odor-Induced Emotion Classification Using EEG.

The olfactory system enables humans to smell different odors, which are closely related to emotions. The high temporal resolution and non-invasiveness of Electroencephalogram (EEG) make it suitable to objectively study human preferences for odors. Effectively learning the temporal dynamics and spatial information from EEG is crucial for detecting odor-induced emotional valence. In this paper, we propose a deep learning architecture called Temporal Attention with Spatial Autoencoder Network (TASA) for predicting odor-induced emotions using EEG. TASA consists of a filter-bank layer, a spatial encoder, a time segmentation layer, a Long Short-Term Memory (LSTM) module, a multi-head self-attention (MSA) layer, and a fully connected layer. We improve upon the previous work by utilizing a two-phase learning framework, using the autoencoder module to learn the spatial information among electrodes by reconstructing the given input with a latent representation in the spatial dimension, which aims to minimize information loss compared to spatial filtering with CNN. The second improvement is inspired by the continuous nature of the olfactory process; we propose to use LSTM-MSA in TASA to capture its temporal dynamics by learning the intercorrelation among the time segments of the EEG. TASA is evaluated on an existing olfactory EEG dataset and compared with several existing deep learning architectures to demonstrate its effectiveness in predicting olfactory-triggered emotional responses. Interpretability analyses with DeepLIFT also suggest that TASA learns spatial-spectral features that are relevant to olfactory-induced emotion recognition.

ECGAN-Assisted ResT-Net Based on Fuzziness for OSA Detection.

OBJECTIVE: Growing attention has been paid recently to electrocardiogram (ECG) based obstructive sleep apnea (OSA) detection, with some progresses been made on this topic. However, the lack of data, low data quality, and incomplete data labeling hinder the application of deep learning to OSA detection, which in turn affects the overall generalization capacity of the network. METHODS: To address these issues, we propose the ResT-ECGAN framework. It uses a one-dimensional generative adversarial network (ECGAN) for sample generation, and integrates it into ResTNet for OSA detection. ECGAN filters the generated ECG signals by incorporating the concept of fuzziness, effectively increasing the amount of high-quality data. ResT-Net not only alleviates the problems caused by deepening the network but also utilizes multihead attention mechanisms to parallelize sequence processing and extract more valuable OSA detection features by leveraging contextual information. RESULTS: Through extensive experiments, we verify that ECGAN can effectively improve the OSA detection performance of ResT-Net. Using only ResT-Net for detection, the accuracy on the Apnea-ECG and private databases is 0.885 and 0.837, respectively. By adding ECGAN-generated data augmentation, the accuracy is increased to 0.893 and 0.848, respectively. CONCLUSION AND SIGNIFICANCE: Comparing with the state-of-the-art deep learning methods, our method outperforms them in terms of accuracy. This study provides a new approach and solution to improve OSA detection in situations with limited labeled samples.

Machine learning-guided realization of full-color high-quantum-yield carbon quantum dots.

Carbon quantum dots (CQDs) have versatile applications in luminescence, whereas identifying optimal synthesis conditions has been challenging due to numerous synthesis parameters and multiple desired outcomes, creating an enormous search space. In this study, we present a novel multi-objective optimization strategy utilizing a machine learning (ML) algorithm to intelligently guide the hydrothermal synthesis of CQDs. Our closed-loop approach learns from limited and sparse data, greatly reducing the research cycle and surpassing traditional trial-and-error methods. Moreover, it also reveals the intricate links between synthesis parameters and target properties and unifies the objective function to optimize multiple desired properties like full-color photoluminescence (PL) wavelength and high PL quantum yields (PLQY). With only 63 experiments, we achieve the synthesis of full-color fluorescent CQDs with high PLQY exceeding 60% across all colors. Our study represents a significant advancement in ML-guided CQDs synthesis, setting the stage for developing new materials with multiple desired properties.

EEG data space adaptation to reduce intersession nonstationarity in brain-computer interface.

A major challenge in EEG-based brain-computer interfaces (BCIs) is the intersession nonstationarity in the EEG data that often leads to deteriorated BCI performances. To address this issue, this letter proposes a novel data space adaptation technique, EEG data space adaptation (EEG-DSA), to linearly transform the EEG data from the target space (evaluation session), such that the distribution difference to the source space (training session) is minimized. Using the Kullback-Leibler (KL) divergence criterion, we propose two versions of the EEG-DSA algorithm: the supervised version, when labeled data are available in the evaluation session, and the unsupervised version, when labeled data are not available. The performance of the proposed EEG-DSA algorithm is evaluated on the publicly available BCI Competition IV data set IIa and a data set recorded from 16 subjects performing motor imagery tasks on different days. The results show that the proposed EEG-DSA algorithm in both the supervised and unsupervised versions significantly outperforms the results without adaptation in terms of classification accuracy. The results also show that for subjects with poor BCI performances when no adaptation is applied, the proposed EEG-DSA algorithm in both the supervised and unsupervised versions significantly outperforms the unsupervised bias adaptation algorithm (PMean).

Discriminative learning of propagation and spatial pattern for motor imagery EEG analysis.

Effective learning and recovery of relevant source brain activity patterns is a major challenge to brain-computer interface using scalp EEG. Various spatial filtering solutions have been developed. Most current methods estimate an instantaneous demixing with the assumption of uncorrelatedness of the source signals. However, recent evidence in neuroscience suggests that multiple brain regions cooperate, especially during motor imagery, a major modality of brain activity for brain-computer interface. In this sense, methods that assume uncorrelatedness of the sources become inaccurate. Therefore, we are promoting a new methodology that considers both volume conduction effect and signal propagation between multiple brain regions. Specifically, we propose a novel discriminative algorithm for joint learning of propagation and spatial pattern with an iterative optimization solution. To validate the new methodology, we conduct experiments involving 16 healthy subjects and perform numerical analysis of the proposed algorithm for EEG classification in motor imagery brain-computer interface. Results from extensive analysis validate the effectiveness of the new methodology with high statistical significance.

Tensor-CSPNet: A Novel Geometric Deep Learning Framework for Motor Imagery Classification.

Deep learning (DL) has been widely investigated in a vast majority of applications in electroencephalography (EEG)-based brain-computer interfaces (BCIs), especially for motor imagery (MI) classification in the past five years. The mainstream DL methodology for the MI-EEG classification exploits the temporospatial patterns of EEG signals using convolutional neural networks (CNNs), which have been particularly successful in visual images. However, since the statistical characteristics of visual images depart radically from EEG signals, a natural question arises whether an alternative network architecture exists apart from CNNs. To address this question, we propose a novel geometric DL (GDL) framework called Tensor-CSPNet, which characterizes spatial covariance matrices derived from EEG signals on symmetric positive definite (SPD) manifolds and fully captures the temporospatiofrequency patterns using existing deep neural networks on SPD manifolds, integrating with experiences from many successful MI-EEG classifiers to optimize the framework. In the experiments, Tensor-CSPNet attains or slightly outperforms the current state-of-the-art performance on the cross-validation and holdout scenarios in two commonly used MI-EEG datasets. Moreover, the visualization and interpretability analyses also exhibit the validity of Tensor-CSPNet for the MI-EEG classification. To conclude, in this study, we provide a feasible answer to the question by generalizing the DL methodologies on SPD manifolds, which indicates the start of a specific GDL methodology for the MI-EEG classification.

Self reward design with fine-grained interpretability.

The black-box nature of deep neural networks (DNN) has brought to attention the issues of transparency and fairness. Deep Reinforcement Learning (Deep RL or DRL), which uses DNN to learn its policy, value functions etc, is thus also subject to similar concerns. This paper proposes a way to circumvent the issues through the bottom-up design of neural networks with detailed interpretability, where each neuron or layer has its own meaning and utility that corresponds to humanly understandable concept. The framework introduced in this paper is called the Self Reward Design (SRD), inspired by the Inverse Reward Design, and this interpretable design can (1) solve the problem by pure design (although imperfectly) and (2) be optimized like a standard DNN. With deliberate human designs, we show that some RL problems such as lavaland and MuJoCo can be solved using a model constructed with standard NN components with few parameters. Furthermore, with our fish sale auction example, we demonstrate how SRD is used to address situations that will not make sense if black-box models are used, where humanly-understandable semantic-based decision is required.

Gait Pattern Recognition Based on Supervised Contrastive Learning Between EEG and EMG.

Electroencephalography (EEG) and lower-limb electromyography (EMG) signals are widely used in lower-limb kinematic classification and regression tasks. Since it directly measures muscle responses, EMG usually works better. However, due to the susceptibility of EMG signals to muscle fatigue, insufficient residual myoelectric activity, and the difficulty of precise localization, it is difficult to acquire EMG signals in practice. In contrast, EEG signals are stable and easy to sample. Therefore, in this work, we propose a multimodal training strategy based on supervised contrastive learning. With this training strategy, we can effectively use the guiding role of EMG in the training phase to help the model fit the gait with EEG signal while using only EEG signal in the testing phase to obtain better results than using any single modal signal to train and test the model. Finally, we compared the models trained with the strategy proposed in this paper with other models trained with EEG signals. The obtained Pearson's Correlation Coefficient value exceeds those of all baseline models.