Dynamical changes of interaction across functional brain communities during propofol-induced sedation.

It is crucial to understand how anesthetics disrupt information transmission within the whole-brain network and its hub structure to gain insight into the network-level mechanisms underlying propofol-induced sedation. However, the influence of propofol on functional integration, segregation, and community structure of whole-brain networks were still unclear. We recruited 12 healthy subjects and acquired resting-state functional magnetic resonance imaging data during 5 different propofol-induced effect-site concentrations (CEs): 0, 0.5, 1.0, 1.5, and 2.0 µg/ml. We constructed whole-brain functional networks for each subject under different conditions and identify community structures. Subsequently, we calculated the global and local topological properties of whole-brain network to investigate the alterations in functional integration and segregation with deepening propofol sedation. Additionally, we assessed the alteration of key nodes within the whole-brain community structure at each effect-site concentrations level. We found that global participation was significantly increased at high effect-site concentrations, which was mediated by bilateral postcentral gyrus. Meanwhile, connector hubs appeared and were located in posterior cingulate cortex and precentral gyrus at high effect-site concentrations. Finally, nodal participation coefficients of connector hubs were closely associated to the level of sedation. These findings provide valuable insights into the relationship between increasing propofol dosage and enhanced functional interaction within the whole-brain networks.

EEG-FRM: a neural network based familiar and unfamiliar face EEG recognition method.

Recognizing familiar faces holds great value in various fields such as medicine, criminal investigation, and lie detection. In this paper, we designed a Complex Trial Protocol-based familiar and unfamiliar face recognition experiment that using self-face information, and collected EEG data from 147 subjects. A novel neural network-based method, the EEG-based Face Recognition Model (EEG-FRM), is proposed in this paper for cross-subject familiar/unfamiliar face recognition, which combines a multi-scale convolutional classification network with the maximum probability mechanism to realize individual face recognition. The multi-scale convolutional neural network extracts temporal information and spatial features from the EEG data, the attention module and supervised contrastive learning module are employed to promote the classification performance. Experimental results on the dataset reveal that familiar face stimuli could evoke significant P300 responses, mainly concentrated in the parietal lobe and nearby regions. Our proposed model achieved impressive results, with a balanced accuracy of 85.64%, a true positive rate of 73.23%, and a false positive rate of 1.96% on the collected dataset, outperforming other compared methods. The experimental results demonstrate the effectiveness and superiority of our proposed model.

Tracking the Immediate and Short-Term Effects of Continuous Theta Burst Stimulation on Dynamic Brain States.

Continuous Theta Burst Stimulation (cTBS) has been shown to modulate cortical oscillations and induce cortical inhibitory effects. Electroencephalography (EEG) studies have shown some immediate effects of cTBS on brain activity. To investigate both immediate effects and short-term effects of cTBS on dynamic brain changes, cTBS was applied to 22 healthy participants over their left motor cortex. We recorded eyes-open, resting-state EEG and performance in the Nine-Hole Peg Test (NHPT) before cTBS, immediately after cTBS, and 80 minutes after cTBS. We identified nine states using a Hidden Markov Model (HMM)-based approach to describe the process of dynamic brain changes. The spatial activation, temporal profiles of HMM states and behavioral performance of NHPT were assessed and compared. cTBS altered the temporal profiles of S1-S5 immediately after cTBS and the temporal profiles of S5, S6 and S7 80 min after cTBS. Moreover, cTBS improved motor function of the left hand. State 1 was characterized as the activation of right occipito-temporal area, and NHPT behavioral performance of the left hand positively correlated with the occurrence of state 1, and negatively correlated with the interval time of state 1 after cTBS. The transitions between S1 or S7 and other states showed dynamic reconfiguration during after-effect sustained time after cTBS. These results suggest that the dynamic characteristics of state 1 are potential biomarkers for characterizing the aftereffect changes of cTBS.

Growing Like a Tree: Finding Trunks From Graph Skeleton Trees.

The message-passing paradigm has served as the foundation of graph neural networks (GNNs) for years, making them achieve great success in a wide range of applications. Despite its elegance, this paradigm presents several unexpected challenges for graph-level tasks, such as the long-range problem, information bottleneck, over-squashing phenomenon, and limited expressivity. In this study, we aim to overcome these major challenges and break the conventional "node- and edge-centric" mindset in graph-level tasks. To this end, we provide an in-depth theoretical analysis of the causes of the information bottleneck from the perspective of information influence. Building on the theoretical results, we offer unique insights to break this bottleneck and suggest extracting a skeleton tree from the original graph, followed by propagating information in a distinctive manner on this tree. Drawing inspiration from natural trees, we further propose to find trunks from graph skeleton trees to create powerful graph representations and develop the corresponding framework for graph-level tasks. Extensive experiments on multiple real-world datasets demonstrate the superiority of our model. Comprehensive experimental analyses further highlight its capability of capturing long-range dependencies and alleviating the over-squashing problem, thereby providing novel insights into graph-level tasks.

High-dimensional topographic organization of visual features in the primate temporal lobe.

The inferotemporal cortex supports our supreme object recognition ability. Numerous studies have been conducted to elucidate the functional organization of this brain area, but there are still important questions that remain unanswered, including how this organization differs between humans and non-human primates. Here, we use deep neural networks trained on object categorization to construct a 25-dimensional space of visual features, and systematically measure the spatial organization of feature preference in both male monkey brains and human brains using fMRI. These feature maps allow us to predict the selectivity of a previously unknown region in monkey brains, which is corroborated by additional fMRI and electrophysiology experiments. These maps also enable quantitative analyses of the topographic organization of the temporal lobe, demonstrating the existence of a pair of orthogonal gradients that differ in spatial scale and revealing significant differences in the functional organization of high-level visual areas between monkey and human brains.

Topographic representation of visually evoked emotional experiences in the human cerebral cortex.

Affective neuroscience seeks to uncover the neural underpinnings of emotions that humans experience. However, it remains unclear whether an affective space underlies the discrete emotion categories in the human brain, and how it relates to the hypothesized affective dimensions. To address this question, we developed a voxel-wise encoding model to investigate the cortical organization of human emotions. Results revealed that the distributed emotion representations are constructed through a fundamental affective space. We further compared each dimension of this space to 14 hypothesized affective dimensions, and found that many affective dimensions are captured by the fundamental affective space. Our results suggest that emotional experiences are represented by broadly spatial overlapping cortical patterns and form smooth gradients across large areas of the cortex. This finding reveals the specific structure of the affective space and its relationship to hypothesized affective dimensions, while highlighting the distributed nature of emotional representations in the cortex.

Investigating EEG-based cross-session and cross-task vigilance estimation in BCI systems.

Objective. The state of vigilance is crucial for effective performance in brain-computer interface (BCI) tasks, and therefore, it is essential to investigate vigilance levels in BCI tasks. Despite this, most studies have focused on vigilance levels in driving tasks rather than on BCI tasks, and the electroencephalogram (EEG) patterns of vigilance states in different BCI tasks remain unclear. This study aimed to identify similarities and differences in EEG patterns and performances of vigilance estimation in different BCI tasks and sessions.Approach.To achieve this, we built a steady-state visual evoked potential-based BCI system and a rapid serial visual presentation-based BCI system and recruited 18 participants to carry out four BCI experimental sessions over four days.Main results. Our findings demonstrate that specific neural patterns for high and low vigilance levels are relatively stable across sessions. Differential entropy features significantly differ between different vigilance levels in all frequency bands and between BCI tasks in the delta and theta frequency bands, with the theta frequency band features playing a critical role in vigilance estimation. Additionally, prefrontal, temporal, and occipital regions are more relevant to the vigilance state in BCI tasks. Our results suggest that cross-session vigilance estimation is more accurate than cross-task estimation.Significance.Our study clarifies the underlying mechanisms of vigilance state in two BCI tasks and provides a foundation for further research in vigilance estimation in BCI applications.

Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features.

Decoding human visual neural representations is a challenging task with great scientific significance in revealing vision-processing mechanisms and developing brain-like intelligent machines. Most existing methods are difficult to generalize to novel categories that have no corresponding neural data for training. The two main reasons are 1) the under-exploitation of the multimodal semantic knowledge underlying the neural data and 2) the small number of paired (stimuli-responses) training data. To overcome these limitations, this paper presents a generic neural decoding method called BraVL that uses multimodal learning of brain-visual-linguistic features. We focus on modeling the relationships between brain, visual and linguistic features via multimodal deep generative models. Specifically, we leverage the mixture-of-product-of-experts formulation to infer a latent code that enables a coherent joint generation of all three modalities. To learn a more consistent joint representation and improve the data efficiency in the case of limited brain activity data, we exploit both intra- and inter-modality mutual information maximization regularization terms. In particular, our BraVL model can be trained under various semi-supervised scenarios to incorporate the visual and textual features obtained from the extra categories. Finally, we construct three trimodal matching datasets, and the extensive experiments lead to some interesting conclusions and cognitive insights: 1) decoding novel visual categories from human brain activity is practically possible with good accuracy; 2) decoding models using the combination of visual and linguistic features perform much better than those using either of them alone; 3) visual perception may be accompanied by linguistic influences to represent the semantics of visual stimuli.

Graph-Enhanced Emotion Neural Decoding.

Brain signal-based emotion recognition has recently attracted considerable attention since it has powerful potential to be applied in human-computer interaction. To realize the emotional interaction of intelligent systems with humans, researchers have made efforts to decode human emotions from brain imaging data. The majority of current efforts use emotion similarities (e.g., emotion graphs) or brain region similarities (e.g., brain networks) to learn emotion and brain representations. However, the relationships between emotions and brain regions are not explicitly incorporated into the representation learning process. As a result, the learned representations may not be informative enough to benefit specific tasks, e.g., emotion decoding. In this work, we propose a novel idea of graph-enhanced emotion neural decoding, which takes advantage of a bipartite graph structure to integrate the relationships between emotions and brain regions into the neural decoding process, thus helping learn better representations. Theoretical analyses conclude that the suggested emotion-brain bipartite graph inherits and generalizes the conventional emotion graphs and brain networks. Comprehensive experiments on visually evoked emotion datasets demonstrate the effectiveness and superiority of our approach.

Temporal dynamics of electroencephalographic microstates during sustained pain.

Brain dynamics can be modeled by a sequence of transient, nonoverlapping patterns of quasi-stable electrical potentials named "microstates." While electroencephalographic (EEG) microstates among patients with chronic pain remained inconsistent in the literature, this study characterizes the temporal dynamics of EEG microstates among healthy individuals during experimental sustained pain. We applied capsaicin (pain condition) or control (no-pain condition) cream to 58 healthy participants in different sessions and recorded resting-state EEG 15 min after application. We identified 4 canonical microstates (A-D) that are related to auditory, visual, salience, and attentional networks. Microstate C had less occurrence, as were bidirectional transitions between microstate C and microstates A and B during sustained pain. In contrast, sustained pain was associated with more frequent and longer duration of microsite D, as well as more bidirectional transitions between microstate D and microstates A and B. Microstate D duration positively correlated with intensity of ongoing pain. Sustained pain improved global integration within microstate C functional network, but weakened global integration and efficiency within microstate D functional network. These results suggest that sustained pain leads to an imbalance between processes that load on saliency (microstate C) and processes related to switching and reorientation of attention (microstate D).

Multimodal motor imagery decoding method based on temporal spatial feature alignment and fusion.

Objective. A motor imagery-based brain-computer interface (MI-BCI) translates spontaneous movement intention from the brain to outside devices. Multimodal MI-BCI that uses multiple neural signals contains rich common and complementary information and is promising for enhancing the decoding accuracy of MI-BCI. However, the heterogeneity of different modalities makes the multimodal decoding task difficult. How to effectively utilize multimodal information remains to be further studied.Approach. In this study, a multimodal MI decoding neural network was proposed. Spatial feature alignment losses were designed to enhance the feature representations extracted from the heterogeneous data and guide the fusion of features from different modalities. An attention-based modality fusion module was built to align and fuse the features in the temporal dimension. To evaluate the proposed decoding method, a five-class MI electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) dataset were constructed.Main results and significance. The comparison experimental results showed that the proposed decoding method achieved higher decoding accuracy than the compared methods on both the self-collected dataset and a public dataset. The ablation results verified the effectiveness of each part of the proposed method. Feature distribution visualization results showed that the proposed losses enhance the feature representation of EEG and fNIRS modalities. The proposed method based on EEG and fNIRS modalities has significant potential for improving decoding performance of MI tasks.

Cross-modal guiding and reweighting network for multi-modal RSVP-based target detection.

Rapid Serial Visual Presentation (RSVP) based Brain-Computer Interface (BCI) facilities the high-throughput detection of rare target images by detecting evoked event-related potentials (ERPs). At present, the decoding accuracy of the RSVP-based BCI system limits its practical applications. This study introduces eye movements (gaze and pupil information), referred to as EYE modality, as another useful source of information to combine with EEG-based BCI and forms a novel target detection system to detect target images in RSVP tasks. We performed an RSVP experiment, recorded the EEG signals and eye movements simultaneously during a target detection task, and constructed a multi-modal dataset including 20 subjects. Also, we proposed a cross-modal guiding and fusion network to fully utilize EEG and EYE modalities and fuse them for better RSVP decoding performance. In this network, a two-branch backbone was built to extract features from these two modalities. A Cross-Modal Feature Guiding (CMFG) module was proposed to guide EYE modality features to complement the EEG modality for better feature extraction. A Multi-scale Multi-modal Reweighting (MMR) module was proposed to enhance the multi-modal features by exploring intra- and inter-modal interactions. And, a Dual Activation Fusion (DAF) was proposed to modulate the enhanced multi-modal features for effective fusion. Our proposed network achieved a balanced accuracy of 88.00% (±2.29) on the collected dataset. The ablation studies and visualizations revealed the effectiveness of the proposed modules. This work implies the effectiveness of introducing the EYE modality in RSVP tasks. And, our proposed network is a promising method for RSVP decoding and further improves the performance of RSVP-based target detection systems.

Multi-View Multi-Label Fine-Grained Emotion Decoding From Human Brain Activity.

Decoding emotional states from human brain activity play an important role in the brain-computer interfaces. Existing emotion decoding methods still have two main limitations: one is only decoding a single emotion category from a brain activity pattern and the decoded emotion categories are coarse-grained, which is inconsistent with the complex emotional expression of humans; the other is ignoring the discrepancy of emotion expression between the left and right hemispheres of the human brain. In this article, we propose a novel multi-view multi-label hybrid model for fine-grained emotion decoding (up to 80 emotion categories) which can learn the expressive neural representations and predict multiple emotional states simultaneously. Specifically, the generative component of our hybrid model is parameterized by a multi-view variational autoencoder, in which we regard the brain activity of left and right hemispheres and their difference as three distinct views and use the product of expert mechanism in its inference network. The discriminative component of our hybrid model is implemented by a multi-label classification network with an asymmetric focal loss. For more accurate emotion decoding, we first adopt a label-aware module for emotion-specific neural representation learning and then model the dependency of emotional states by a masked self-attention mechanism. Extensive experiments on two visually evoked emotional datasets show the superiority of our method.

Dynamic Domain Adaptation for Class-Aware Cross-Subject and Cross-Session EEG Emotion Recognition.

It is vital to develop general models that can be shared across subjects and sessions in the real-world deployment of electroencephalogram (EEG) emotion recognition systems. Many prior studies have exploited domain adaptation algorithms to alleviate the inter-subject and inter-session discrepancies of EEG distributions. However, these methods only aligned the global domain divergence, but overlooked the local domain divergence with respect to each emotion category. This degenerates the emotion-discriminating ability of the domain invariant features. In this paper, we argue that aligning the EEG data within the same emotion categories is important for generalizable and discriminative features. Hence, we propose the dynamic domain adaptation (DDA) algorithm where the global and local divergences are disposed by minimizing the global domain discrepancy and local subdomain discrepancy, respectively. To tackle the absence of emotion labels in the target domain, we introduce a dynamic training strategy where the model focuses on optimizing the global domain discrepancy in the early training steps, and then gradually switches to the local subdomain discrepancy. The DDA algorithm is formally implemented as an unsupervised version and a semi-supervised version for different experimental settings. Based on the coarse-to-fine alignment, our model achieves the average peak accuracy of 91.08%, 92.89% on SEED, and 81.58%, 80.82% on SEED-IV in the cross-subject and cross-session scenarios, respectively.

Driving brain state transitions in major depressive disorder through external stimulation.

Major depressive disorder (MDD) as a dysfunction of neural circuits and brain networks has been established in modern neuroimaging sciences. However, the brain state transitions between MDD and health through external stimulation remain unclear, which limits translation to clinical contexts and demonstrable clinical utility. We propose a framework of the large-scale whole-brain network model for MDD linking the underlying anatomical connectivity with functional dynamics obtained from diffusion tensor imaging (DTI) and functional magnetic resonance imaging (fMRI). Then, we further explored the optimal brain regions to promote the transition of brain states between MDD and health through external stimulation of the model. Based on the whole-brain model successfully fitting the brain state space in MDD and the health, we demonstrated that the transition from MDD to health is achieved by the excitatory activation of the limbic system and from health to MDD by the inhibitory stimulation of the reward circuit. Our finding provides novel biophysical evidence for the neural mechanism of MDD and its recovery and allows the discovery of new stimulation targets for MDD recovery.

Bidirectional Siamese correlation analysis method for enhancing the detection of SSVEPs.

Objective.Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have attracted increasing attention due to their high information transfer rate. To improve the performance of SSVEP detection, we propose a bidirectional Siamese correlation analysis (bi-SiamCA) model.Approach. In this model, an long short-term memory-based Siamese architecture is designed to measure the similarity between the SSVEP signal and the template in each frequency and obtain the probability that the SSVEP signal belongs to each frequency. Additionally, a maximize agreement module with a designed contrastive loss is adopted in the Siamese architecture to increase the similarity between the SSVEP signal and the reference signal in the same frequency. Moreover, a two-way signal processing mechanism is built to effectively integrate complementary information from two temporal directions of the input signals. Our model uses raw SSVEPs as inputs and can be trained end-to-end.Main results.Experimental results on a 40-class dataset and a 12-class dataset indicate that bi-SiamCA can significantly improve the classification accuracy compared with the prominent traditional and deep learning methods, especially under short data lengths. Feature visualizations show that the similarity between the SSVEP signal and the reference signal in the same frequency gradually improved in our model.Conclusion.The proposed bi-SiamCA model enhances the performance of SSVEP detection and outperforms the compared methods.Significance.Due to its high decoding accuracy under short signals, our approach has great potential to implement a high-speed SSVEP-based BCI.

Continuous theta-burst stimulation modulates resting-state EEG microstates in healthy subjects.

Continuous theta-burst stimulation (cTBS) induces long-lasting inhibitory effects on cortical excitability. Although cTBS has been reported to modulate neural oscillations and functional connectivity, it is still unclear how cTBS affects brain dynamics that could be captured by the resting-sate EEG microstate sequences. This study aims to investigate how cTBS over the left motor cortex affects brain dynamics. We applied 40 s-long cTBS over the left motor cortex of 28 healthy participants. Before and in multi-sessions up to 90 min after cTBS, their performance in a Nine-Hole Peg Test (NHPT), that measures the hand dexterity, and resting state EEG were recorded. Resting-sate EEG data were clustered into four microstates (namely A, B, C, and D) using k-means clustering algorithms. cTBS-induced changes in NHPT performance, microstate dynamics and functional connectivity networks were comprehensively assessed. As compared with baseline, the completion time of NHPT became shorter immediately after cTBS, suggesting cTBS-induced motor function improvement. After cTBS, the topography of microstate B revealed a greater change compared with other three topographies. Importantly, cTBS-induced decrease in completion time of NHPT correlated with cTBS-induced decrease of the mean occurrence of microstate B. Functional connectivity analysis further revealed that cTBS led to an increase of the node efficiency at C4 electrode in microstate B. These results indicated the specific modulation of cTBS over the motor cortex on the dynamics of microstate B. This work provided the evidence of the association between B and motor function, and it also implies the modulation of cTBS over the motor network. Supplementary Information: The online version contains supplementary material available at 10.1007/s11571-021-09726-6.

ERP prototypical matching net: a meta-learning method for zero-calibration RSVP-based image retrieval.

Objective.A rapid serial visual presentation (RSVP)-based brain-computer interface (BCI) is an efficient information detection technology through detecting event-related potentials (ERPs) evoked by target visual stimuli. The BCI system requires a time-consuming calibration process to build a reliable decoding model for a new user. Therefore, zero-calibration has become an important topic in BCI research.Approach.In this paper, we construct an RSVP dataset that includes 31 subjects, and propose a zero-calibration method based on a metric-based meta-learning: ERP prototypical matching net (EPMN). EPMN learns a metric space where the distance between electroencephalography (EEG) features and ERP prototypes belonging to the same category is smaller than that of different categories. Here, we employ prototype learning to learn a common representation from ERP templates of different subjects as ERP prototypes. Additionally, a metric-learning loss function is proposed for maximizing the distance between different classes of EEG and ERP prototypes and minimizing the distance between the same classes of EEG and ERP prototypes in the metric space.Main results.The experimental results showed that EPMN achieved a balanced-accuracy of 86.34% and outperformed the comparable methods.Significance.Our EPMN can realize zero-calibration for an RSVP-based BCI system.

Time-Distributed Attention Network for EEG-Based Motor Imagery Decoding From the Same Limb.

A brain-computer interface (BCI) based on motor imagery (MI) from the same limb can provide an intuitive control pathway but has received limited attention. It is still a challenge to classify multiple MI tasks from the same limb. The goal of this study is to propose a novel decoding method to classify the MI tasks of four joints of the same upper limb and the resting state. EEG signals were collected from 20 participants. A time-distributed attention network (TD-Atten) was proposed to adaptively assign different weights to different classes and frequency bands of the input multiband Common Spatial Pattern (CSP) features. The long short-term memory (LSTM) and dense layers were then used to learn sequential information from the reweight features and perform the classification. Our proposed method outperformed other baseline and deep learning-based methods and obtained the accuracies of 46.8% in the 5-class scenario and 53.4% in the 4-class scenario. The visualization results of attention weights indicated that the proposed framework can adaptively pay attention to alpha-band related features in MI tasks, which was consistent with the analysis of brain activation patterns. These results demonstrated the feasibility and interpretability of the attention mechanism in MI decoding and the potential of this fine MI paradigm to be applied for the control of a robotic arm or a neural prosthesis.

Adaptive and Maladaptive Brain Functional Network Reorganization After Stroke in Hemianopia Patients: An Electroencephalogram-Tracking Study.

Objective: Hemianopia after occipital stroke is believed to be mainly due to local damage at or near the lesion site. However, magnetic resonance imaging studies suggest functional connectivity network (FCN) reorganization also in distant brain regions. Because it is unclear whether reorganization is adaptive or maladaptive, compensating for, or aggravating vision loss, we characterized FCNs electrophysiologically to explore local and global brain plasticity and correlated FCN reorganization with visual performance. Methods: Resting-state electroencephalography (EEG) was recorded in chronic, unilateral stroke patients and healthy age-matched controls (n = 24 each). This study was approved by the local ethics committee. The correlation of oscillating EEG activity was calculated with the imaginary part of coherence between pairs of regions of interest, and FCN graph theory metrics (degree, strength, clustering coefficient) were correlated with stimulus detection and reaction time. Results: Stroke brains showed altered FCNs in the alpha- and low beta-band in numerous occipital, temporal brain structures. On a global level, FCN had a less efficient network organization whereas on the local level node networks were reorganized especially in the intact hemisphere. Here, the occipital network was 58% more rigid (with a more "regular" network structure) whereas the temporal network was 32% more efficient (showing greater "small-worldness"), both of which correlated with worse or better visual processing, respectively. Conclusions: Occipital stroke is associated with both local and global FCN reorganization, but this can be both adaptive and maladaptive. We propose that the more "regular" FCN structure in the intact visual cortex indicates maladaptive plasticity, where less processing efficacy with reduced signal/noise ratio may cause the perceptual deficits in the intact visual field (VF). In contrast, reorganization in intact temporal brain regions is presumably adaptive, possibly supporting enhanced peripheral movement perception.