EEG-based emotion recognition using a temporal-difference minimizing neural network.

Electroencephalogram (EEG) emotion recognition plays an important role in human-computer interaction. An increasing number of algorithms for emotion recognition have been proposed recently. However, it is still challenging to make efficient use of emotional activity knowledge. In this paper, based on prior knowledge that emotion varies slowly across time, we propose a temporal-difference minimizing neural network (TDMNN) for EEG emotion recognition. We use maximum mean discrepancy (MMD) technology to evaluate the difference in EEG features across time and minimize the difference by a multibranch convolutional recurrent network. State-of-the-art performances are achieved using the proposed method on the SEED, SEED-IV, DEAP and DREAMER datasets, demonstrating the effectiveness of including prior knowledge in EEG emotion recognition.

Effects of long-term closed and socially isolating spaceflight analog environment on default mode network connectivity as indicated by fMRI.

Long-term manned spaceflight and extraterrestrial planet settlement become the focus of space powers. However, the potential influence of closed and socially isolating spaceflight on the brain function remains unclear. A 180-day controlled ecological life support system integrated experiment was conducted, establishing a spaceflight analog environment to explore the effect of long-term socially isolating living. Three crewmembers were enrolled and underwent resting-state fMRI scanning before and after the experiment. We performed both seed-based and network-based analyses to investigate the functional connectivity (FC) changes of the default mode network (DMN), considering its key role in multiple higher-order cognitive functions. Compared with normal controls, the leader of crewmembers exhibited significantly reduced within-DMN and between-DMN FC after the experiment, while two others exhibited opposite trends. Moreover, individual differences of FC changes were further supported by evidence from behavioral analyses. The findings may shed new light on the development of psychological protection for space exploration.

An SSVEP-based BCI with 112 targets using frequency spatial multiplexing.

Objective.Brain-computer interface (BCI) systems with large directly accessible instruction sets are one of the difficulties in BCI research. Research to achieve high target resolution (⩾100) has not yet entered a rapid development stage, which contradicts the application requirements. Steady-state visual evoked potential (SSVEP) based BCIs have an advantage in terms of the number of targets, but the competitive mechanism between the target stimulus and its neighboring stimuli is a key challenge that prevents the target resolution from being improved significantly.Approach.In this paper, we reverse the competitive mechanism and propose a frequency spatial multiplexing method to produce more targets with limited frequencies. In the proposed paradigm, we replicated each flicker stimulus as a 2 × 2 matrix and arrange the matrices of all frequencies in a tiled fashion to form the interaction interface. With different arrangements, we designed and tested three example paradigms with different layouts. Further we designed a graph neural network that distinguishes between targets of the same frequency by recognizing the different electroencephalography (EEG) response distribution patterns evoked by each target and its neighboring targets.Main results.Extensive experiment studies employing eleven subjects have been performed to verify the validity of the proposed method. The average classification accuracies in the offline validation experiments for the three paradigms are 89.16%, 91.38%, and 87.90%, with information transfer rates (ITR) of 51.66, 53.96, and 50.55 bits/min, respectively.Significance.This study utilized the positional relationship between stimuli and did not circumvent the competing response problem. Therefore, other state-of-the-art methods focusing on enhancing the efficiency of SSVEP detection can be used as a basis for the present method to achieve very promising improvements.

Sleep deprivation changes frequency-specific functional organization of the resting human brain.

Previous resting-state functional magnetic resonance imaging (rs-fMRI) studies have widely explored the temporal connection changes in the human brain following long-term sleep deprivation (SD). However, the frequency-specific topological properties of sleep-deprived functional networks remain virtually unclear. In this study, thirty-seven healthy male subjects underwent resting-state fMRI during rested wakefulness (RW) and after 36 hours of SD, and we examined frequency-specific spectral connection changes (0.01-0.08 Hz, interval = 0.01 Hz) caused by SD. First, we conducted a multivariate pattern analysis combining linear SVM classifiers with a robust feature selection algorithm, and the results revealed that accuracies of 74.29%-84.29% could be achieved in the classification between RW and SD states in leave-one-out cross-validation at different frequency bands, moreover, the spectral connection at the lowest and highest frequency bands exhibited higher discriminative power. Connection involving the cingulo-opercular network increased most, while connection involving the default-mode network decreased most following SD. Then we performed a graph-theoretic analysis and observed reduced low-frequency modularity and high-frequency global efficiency in the SD state. Moreover, hub regions, which were primarily situated in the cerebellum and the cingulo-opercular network after SD, exhibited high discriminative power in the aforementioned classification consistently. The findings may indicate the frequency-dependent effects of SD on the functional network topology and its efficiency of information exchange, providing new insights into the impact of SD on the human brain.

Frequency-specific segregation and integration of human cerebral cortex: An intrinsic functional atlas.

The cognitive and behavioral functions of the human brain are supported by its frequency multiplexing mechanism. However, there is limited understanding of the dynamics of the functional network topology. This study aims to investigate the frequency-specific topology of the functional human brain using 7T rs-fMRI data. Frequency-specific parcellations were first performed, revealing frequency-dependent dynamics within the frontoparietal control, parietal memory, and visual networks. An intrinsic functional atlas containing 456 parcels was proposed and validated using stereo-EEG. Graph theory analysis suggested that, in addition to the task-positive vs. task-negative organization observed in static networks, there was a cognitive control system additionally from a frequency perspective. The reproducibility and plausibility of the identified hub sets were confirmed through 3T fMRI analysis, and their artificial removal had distinct effects on network topology. These results indicate a more intricate and subtle dynamics of the functional human brain and emphasize the significance of accurate topography.

Cognitive Load Prediction from Multimodal Physiological Signals using Multiview Learning.

Predicting cognitive load is a crucial issue in the emerging field of human-computer interaction and holds significant practical value, particularly in flight scenarios. Although previous studies have realized efficient cognitive load classification, new research is still needed to adapt the current state-of-the-art multimodal fusion methods. Here, we proposed a feature selection framework based on multiview learning to address the challenges of information redundancy and reveal the common physiological mechanisms underlying cognitive load. Specifically, the multimodal signal features (EEG, EDA, ECG, EOG, & eye movements) at three cognitive load levels were estimated during multiattribute task battery (MATB) tasks performed by 22 healthy participants and fed into a feature selection-multiview classification with cohesion and diversity (FS-MCCD) framework. The optimized feature set was extracted from the original feature set by integrating the weight of each view and the feature weights to formulate the ranking criteria. The cognitive load prediction model, evaluated using real-time classification results, achieved an average accuracy of 81.08% and an average F1-score of 80.94% for three-class classification among 22 participants. Furthermore, the weights of the physiological signal features revealed the physiological mechanisms related to cognitive load. Specifically, heightened cognitive load was linked to amplified δ and θ power in the frontal lobe, reduced α power in the parietal lobe, and an increase in pupil diameter. Thus, the proposed multimodal feature fusion framework emphasizes the effectiveness and efficiency of using these features to predict cognitive load.

The decoupling between hemodynamic parameters and neural activity implies a complex origin of spontaneous brain oscillations.

Introduction: Spontaneous low-frequency oscillations play a key role in brain activity. However, the underlying mechanism and origin of low-frequency oscillations remain under debate. Methods: Optical imaging and an electrophysiological recording system were combined to investigate spontaneous oscillations in the hemodynamic parameters and neuronal activity of awake and anesthetized mice after Nω-nitro-L-arginine methyl ester (L-NAME) administration. Results: The spectrum of local field potential (LFP) signals was significantly changed by L-NAME, which was further corroborated by the increase in energy and spatial synchronization. The important finding was that L-NAME triggered regular oscillations in both LFP signals and hemodynamic signals. Notably, the frequency peak of hemodynamic signals can be different from that of LFP oscillations in awake mice. Discussion: A model of the neurovascular system was proposed to interpret this mismatch of peak frequencies, supporting the view that spontaneous low-frequency oscillations arise from multiple sources.

Incomplete Multi-View Learning Under Label Shift.

In image processing, images are usually composed of partial views due to the uncertainty of collection and how to efficiently process these images, which is called incomplete multi-view learning, has attracted widespread attention. The incompleteness and diversity of multi-view data enlarges the difficulty of annotation, resulting in the divergence of label distribution between the training and testing data, named as label shift. However, existing incomplete multi-view methods generally assume that the label distribution is consistent and rarely consider the label shift scenario. To address this new but important challenge, we propose a novel framework termed as Incomplete Multi-view Learning under Label Shift (IMLLS). In this framework, we first give the formal definitions of IMLLS and the bidirectional complete representation which describes the intrinsic and common structure. Then, a multilayer perceptron which combines the reconstruction and classification loss is employed to learn the latent representation, whose existence, consistency and universality are proved with the theoretical satisfaction of label shift assumption. After that, to align the label distribution, the learned representation and trained source classifier are used to estimate the importance weight by designing a new estimation scheme which balances the error generated by finite samples in theory. Finally, the trained classifier reweighted by the estimated weight is fine-tuned to reduce the gap between the source and target representations. Extensive experimental results validate the effectiveness of our algorithm over existing state-of-the-arts methods in various aspects, together with its effectiveness in discriminating schizophrenic patients from healthy controls.

RayMVSNet++: Learning Ray-Based 1D Implicit Fields for Accurate Multi-View Stereo.

Learning-based multi-view stereo (MVS) has by far centered around 3D convolution on cost volumes. Due to the high computation and memory consumption of 3D CNN, the resolution of output depth is often considerably limited. Different from most existing works dedicated to adaptive refinement of cost volumes, we opt to directly optimize the depth value along each camera ray, mimicking the range (depth) finding of a laser scanner. This reduces the MVS problem to ray-based depth optimization which is much more light-weight than full cost volume optimization. In particular, we propose RayMVSNet which learns sequential prediction of a 1D implicit field along each camera ray with the zero-crossing point indicating scene depth. This sequential modeling, conducted based on transformer features, essentially learns the epipolar line search in traditional multi-view stereo. We devise a multi-task learning for better optimization convergence and depth accuracy. We found the monotonicity property of the SDFs along each ray greatly benefits the depth estimation. Our method ranks top on both the DTU and the Tanks & Temples datasets over all previous learning-based methods, achieving an overall reconstruction score of 0.33 mm on DTU and an F-score of 59.48% on Tanks & Temples. It is able to produce high-quality depth estimation and point cloud reconstruction in challenging scenarios such as objects/scenes with non-textured surface, severe occlusion, and highly varying depth range. Further, we propose RayMVSNet++ to enhance contextual feature aggregation for each ray through designing an attentional gating unit to select semantically relevant neighboring rays within the local frustum around that ray. This improves the performance on datasets with more challenging examples (e.g., low-quality images caused by poor lighting conditions or motion blur). RayMVSNet++ achieves state-of-the-art performance on the ScanNet dataset. In particular, it attains an AbsRel of 0.058m and produces accurate results on the two subsets of textureless regions and large depth variation.

Hippocampus Parcellation via Discriminative Embedded Clustering of fMRI Functional Connectivity.

Dividing a pre-defined brain region into several heterogenous subregions is crucial for understanding its functional segregation and integration. Due to the high dimensionality of brain functional features, clustering is often postponed until dimensionality reduction in traditional parcellation frameworks occurs. However, under such stepwise parcellation, it is very easy to fall into the dilemma of local optimum since dimensionality reduction could not take into account the requirement of clustering. In this study, we developed a new parcellation framework based on the discriminative embedded clustering (DEC), combining subspace learning and clustering in a common procedure with alternative minimization adopted to approach global optimum. We tested the proposed framework in functional connectivity-based parcellation of the hippocampus. The hippocampus was parcellated into three spatial coherent subregions along the anteroventral-posterodorsal axis; the three subregions exhibited distinct functional connectivity changes in taxi drivers relative to non-driver controls. Moreover, compared with traditional stepwise methods, the proposed DEC-based framework demonstrated higher parcellation consistency across different scans within individuals. The study proposed a new brain parcellation framework with joint dimensionality reduction and clustering; the findings might shed new light on the functional plasticity of hippocampal subregions related to long-term navigation experience.

Adaptive Feature Selection With Augmented Attributes.

In many dynamic environment applications, with the evolution of data collection ways, the data attributes are incremental and the samples are stored with accumulated feature spaces gradually. For instance, in the neuroimaging-based diagnosis of neuropsychiatric disorders, with emerging of diverse testing ways, we get more brain image features over time. The accumulation of different types of features will unavoidably bring difficulties in manipulating the high-dimensional data. It is challenging to design an algorithm to select valuable features in this feature incremental scenario. To address this important but rarely studied problem, we propose a novel Adaptive Feature Selection method (AFS). It enables the reusability of the feature selection model trained on previous features and adapts it to fit the feature selection requirements on all features automatically. Besides, an ideal l0-norm sparse constraint for feature selection is imposed with a proposed effective solving strategy. We present the theoretical analyses about the generalization bound and convergence behavior. After tackling this problem in a one-shot case, we extend it to the multi-shot scenario. Plenty of experimental results demonstrate the effectiveness of reusing previous features and the superior of l0-norm constraint in various aspects, together with its effectiveness in discriminating schizophrenic patients from healthy controls.

GeoTransformer: Fast and Robust Point Cloud Registration With Geometric Transformer.

We study the problem of extracting accurate correspondences for point cloud registration. Recent keypoint-free methods have shown great potential through bypassing the detection of repeatable keypoints which is difficult to do especially in low-overlap scenarios. They seek correspondences over downsampled superpoints, which are then propagated to dense points. Superpoints are matched based on whether their neighboring patches overlap. Such sparse and loose matching requires contextual features capturing the geometric structure of the point clouds. We propose Geometric Transformer, or GeoTransformer for short, to learn geometric feature for robust superpoint matching. It encodes pair-wise distances and triplet-wise angles, making it invariant to rigid transformation and robust in low-overlap cases. The simplistic design attains surprisingly high matching accuracy such that no RANSAC is required in the estimation of alignment transformation, leading to 100 times acceleration. Extensive experiments on rich benchmarks encompassing indoor, outdoor, synthetic, multiway and non-rigid demonstrate the efficacy of GeoTransformer. Notably, our method improves the inlier ratio by 18 âˆ¼ 31 percentage points and the registration recall by over 7 points on the challenging 3DLoMatch benchmark.

SS-TBN: A Semi-Supervised Tri-Branch Network for COVID-19 Screening and Lesion Segmentation.

Insufficient annotated data and minor lung lesions pose big challenges for computed tomography (CT)-aided automatic COVID-19 diagnosis at an early outbreak stage. To address this issue, we propose a Semi-Supervised Tri-Branch Network (SS-TBN). First, we develop a joint TBN model for dual-task application scenarios of image segmentation and classification such as CT-based COVID-19 diagnosis, in which pixel-level lesion segmentation and slice-level infection classification branches are simultaneously trained via lesion attention, and individual-level diagnosis branch aggregates slice-level outputs for COVID-19 screening. Second, we propose a novel hybrid semi-supervised learning method to make full use of unlabeled data, combining a new double-threshold pseudo labeling method specifically designed to the joint model and a new inter-slice consistency regularization method specifically tailored to CT images. Besides two publicly available external datasets, we collect internal and our own external datasets including 210,395 images (1,420 cases versus 498 controls) from ten hospitals. Experimental results show that the proposed method achieves state-of-the-art performance in COVID-19 classification with limited annotated data even if lesions are subtle, and that segmentation results promote interpretability for diagnosis, suggesting the potential of the SS-TBN in early screening in insufficient labeled data situations at the early stage of a pandemic outbreak like COVID-19.

Learning to Detect 3D Symmetry From Single-View RGB-D Images With Weak Supervision.

3D symmetry detection is a fundamental problem in computer vision and graphics. Most prior works detect symmetry when the object model is fully known, few studies symmetry detection on objects with partial observation, such as single RGB-D images. Recent work addresses the problem of detecting symmetries from incomplete data with a deep neural network by leveraging the dense and accurate symmetry annotations. However, due to the tedious labeling process, full symmetry annotations are not always practically available. In this work, we present a 3D symmetry detection approach to detect symmetry from single-view RGB-D images without using symmetry supervision. The key idea is to train the network in a weakly-supervised learning manner to complete the shape based on the predicted symmetry such that the completed shape be similar to existing plausible shapes. To achieve this, we first propose a discriminative variational autoencoder to learn the shape prior in order to determine whether a 3D shape is plausible or not. Based on the learned shape prior, a symmetry detection network is present to predict symmetries that produce shapes with high shape plausibility when completed based on those symmetries. Moreover, to facilitate end-to-end network training and multiple symmetry detection, we introduce a new symmetry parametrization for the learning-based symmetry estimation of both reflectional and rotational symmetry. The proposed approach, coupled symmetry detection with shape completion, essentially learns the symmetry-aware shape prior, facilitating more accurate and robust symmetry detection. Experiments demonstrate that the proposed method is capable of detecting reflectional and rotational symmetries accurately, and shows good generality in challenging scenarios, such as objects with heavy occlusion and scanning noise. Moreover, it achieves state-of-the-art performance, improving the F1-score over the existing supervised learning method by 2%-11% on the ShapeNet and ScanNet datasets.

A Pose-Only Solution to Visual Reconstruction and Navigation.

Visual navigation and three-dimensional (3D) scene reconstruction are essential for robotics to interact with the surrounding environment. Large-scale scenarios and computational robustness are great challenges facing the research community to achieve this goal. This paper raises a pose-only imaging geometry representation and algorithms that might help solve these challenges. The pose-only representation, equivalent to the classical multiple-view geometry, is discovered to be linearly related to camera global translations, which allows for efficient and robust camera motion estimation. As a result, the spatial feature coordinates can be analytically reconstructed and do not require nonlinear optimization. Comprehensive experiments demonstrate that the computational efficiency of recovering the scene and associated camera poses is significantly improved by 2-4 orders of magnitude.

Aberrant temporal-spatial complexity of intrinsic fluctuations in major depression.

Accumulating evidence suggests that the brain is highly dynamic; thus, investigation of brain dynamics especially in brain connectivity would provide crucial information that stationary functional connectivity could miss. This study investigated temporal expressions of spatial modes within the default mode network (DMN), salience network (SN) and cognitive control network (CCN) using a reliable data-driven co-activation pattern (CAP) analysis in two independent data sets. We found enhanced CAP-to-CAP transitions of the SN in patients with MDD. Results suggested enhanced flexibility of this network in the patients. By contrast, we also found reduced spatial consistency and persistence of the DMN in the patients, indicating reduced variability and stability in individuals with MDD. In addition, the patients were characterized by prominent activation of mPFC. Moreover, further correlation analysis revealed that persistence and transitions of RCCN were associated with the severity of depression. Our findings suggest that functional connectivity in the patients may not be simply attenuated or potentiated, but just alternating faster or slower among more complex patterns. The aberrant temporal-spatial complexity of intrinsic fluctuations reflects functional diaschisis of resting-state networks as characteristic of patients with MDD.

Parcellating the human brain using resting-state dynamic functional connectivity.

Brain cartography has expanded substantially over the past decade. In this regard, resting-state functional connectivity (FC) plays a key role in identifying the locations of putative functional borders. However, scant attention has been paid to the dynamic nature of functional interactions in the human brain. Indeed, FC is typically assumed to be stationary across time, which may obscure potential or subtle functional boundaries, particularly in regions with high flexibility and adaptability. In this study, we developed a dynamic FC (dFC)-based parcellation framework, established a new functional human brain atlas termed D-BFA (DFC-based Brain Functional Atlas), and verified its neurophysiological plausibility by stereo-EEG data. As the first dFC-based whole-brain atlas, the proposed D-BFA delineates finer functional boundaries that cannot be captured by static FC, and is further supported by good correspondence with cytoarchitectonic areas and task activation maps. Moreover, the D-BFA reveals the spatial distribution of dynamic variability across the brain and generates more homogenous parcels compared with most alternative parcellations. Our results demonstrate the superiority and practicability of dFC in brain parcellation, providing a new template to exploit brain topographic organization from a dynamic perspective. The D-BFA will be publicly available for download at https://github.com/sliderplm/D-BFA-618.

Gradient Matching Federated Domain Adaptation for Brain Image Classification.

Federated learning has shown its unique advantages in many different tasks, including brain image analysis. It provides a new way to train deep learning models while protecting the privacy of medical image data from multiple sites. However, previous studies suggest that domain shift across different sites may influence the performance of federated models. As a solution, we propose a gradient matching federated domain adaptation (GM-FedDA) method for brain image classification, aiming to reduce domain discrepancy with the assistance of a public image dataset and train robust local federated models for target sites. It mainly includes two stages: 1) pretraining stage; we propose a one-common-source adversarial domain adaptation (OCS-ADA) strategy, i.e., adopting ADA with gradient matching loss to pretrain encoders for reducing domain shift at each target site (private data) with the assistance of a common source domain (public data) and 2) fine-tuning stage; we develop a gradient matching federated (GM-Fed) fine-tuning method for updating local federated models pretrained with the OCS-ADA strategy, i.e., pushing the optimization direction of a local federated model toward its specific local minimum by minimizing gradient matching loss between sites. Using fully connected networks as local models, we validate our method with the diagnostic classification tasks of schizophrenia and major depressive disorder based on multisite resting-state functional MRI (fMRI), respectively. Results show that the proposed GM-FedDA method outperforms other commonly used methods, suggesting the potential of our method in brain imaging analysis and other fields, which need to utilize multisite data while preserving data privacy.

Functional orderly topography of brain networks associated with gene expression heterogeneity.

The human cerebral cortex is vastly expanded relative to nonhuman primates and rodents, leading to a functional orderly topography of brain networks. Here, we show that functional topography may be associated with gene expression heterogeneity. The neocortex exhibits greater heterogeneity in gene expression, with a lower expression of housekeeping genes, a longer mean path length, fewer clusters, and a lower degree of ordering in networks than archicortical and subcortical areas in human, rhesus macaque, and mouse brains. In particular, the cerebellar cortex displays greater heterogeneity in gene expression than cerebellar deep nuclei in the human brain, but not in the mouse brain, corresponding to the emergence of novel functions in the human cerebellar cortex. Moreover, the cortical areas with greater heterogeneity, primarily located in the multimodal association cortex, tend to express genes with higher evolutionary rates and exhibit a higher degree of functional connectivity measured by resting-state fMRI, implying that such a spatial distribution of gene expression may be shaped by evolution and is favourable for the specialization of higher cognitive functions. Together, the cross-species imaging and genetic findings may provide convergent evidence to support the association between the orderly topography of brain function networks and gene expression.

Altered cerebellar-motor loop in benign adult familial myoclonic epilepsy type 1: The structural basis of cortical tremor.

OBJECTIVE: Cortical tremor/myoclonus is the hallmark feature of benign adult familial myoclonic epilepsy (BAFME), the mechanism of which remains elusive. A hypothesis is that a defective control in the preexisting cerebellar-motor loop drives cortical tremor. Meanwhile, the basal ganglia system might also participate in BAFME. This study aimed to discover the structural basis of cortical tremor/myoclonus in BAFME. METHODS: Nineteen patients with BAFME type 1 (BAFME1) and 30 matched healthy controls underwent T1-weighted and diffusion tensor imaging scans. FreeSurfer and spatially unbiased infratentorial template (SUIT) toolboxes were utilized to assess the motor cortex and the cerebellum. Probabilistic tractography was generated for two fibers to test the hypothesis: the dentato-thalamo-(M1) (primary motor cortex) and globus pallidus internus (GPi)-thalamic projections. Average fractional anisotropy (FA), axial diffusivity (AD), mean diffusivity (MD), and radial diffusivity (RD) of each tract were extracted. RESULTS: Cerebellar atrophy and dentate nucleus alteration were observed in the patients. In addition, patients with BAFME1 exhibited reduced AD and FA in the left and right dentato-thalamo-M1 nondecussating fibers, respectively false discovery rate (FDR) correction q < .05. Cerebellar projections showed negative correlations with somatosensory-evoked potential P25-N33 amplitude and were independent of disease duration and medication. BAFME1 patients also had increased FA and decreased MD in the left GPi-thalamic projection. Higher FA and lower RD in the right GPi-thalamic projection were also observed (FDR q < .05). SIGNIFICANCE: The present findings support the hypothesis that the cerebello-thalamo-M1 loop might be the structural basis of cortical tremor in BAFME1. The basal ganglia system also participates in BAFME1 and probably serves a regulatory role.