Brain Structural Connectivity Guided Vision Transformers for Identification of Functional Connectivity Characteristics in Preterm Neonates.

Preterm birth is the leading cause of death in children under five years old, and is associated with a wide sequence of complications in both short and long term. In view of rapid neurodevelopment during the neonatal period, preterm neonates may exhibit considerable functional alterations compared to term ones. However, the identified functional alterations in previous studies merely achieve moderate classification performance, while more accurate functional characteristics with satisfying discrimination ability for better diagnosis and therapeutic treatment is underexplored. To address this problem, we propose a novel brain structural connectivity (SC) guided Vision Transformer (SCG-ViT) to identify functional connectivity (FC) differences among three neonatal groups: preterm, preterm with early postnatal experience, and term. Particularly, inspired by the neuroscience-derived information, a novel patch token of SC/FC matrix is defined, and the SC matrix is then adopted as an effective mask into the ViT model to screen out input FC patch embeddings with weaker SC, and to focus on stronger ones for better classification and identification of FC differences among the three groups. The experimental results on multi-modal MRI data of 437 neonatal brains from publicly released Developing Human Connectome Project (dHCP) demonstrate that SCG-ViT achieves superior classification ability compared to baseline models, and successfully identifies holistically different FC patterns among the three groups. Moreover, these different FCs are significantly correlated with the differential gene expressions of the three groups. In summary, SCG-ViT provides a powerfully brain-guided pipeline of adopting large-scale and data-intensive deep learning models for medical imaging-based diagnosis.

An explainable deep learning framework for characterizing and interpreting human brain states.

Deep learning approaches have been widely adopted in the medical image analysis field. However, a most of existing deep learning approaches focus on achieving promising performances such as classification, detection, and segmentation, and much less effort is devoted to the explanation of the designed models. Similarly, in the brain imaging field, many deep learning approaches have been designed and applied to characterize and predict human brain states. However, these models lack interpretation. In response, we propose a novel domain knowledge informed self-attention graph pooling-based (SAGPool) graph convolutional neural network to study human brain states. Specifically, the dense individualized and common connectivity-based cortical landmarks system (DICCCOL, structural brain connectivity profiles) and holistic atlases of functional networks and interactions system (HAFNI, functional brain connectivity profiles) are integrated with the SAGPool model to better characterize and interpret the brain states. Extensive experiments are designed and carried out on the large-scale human connectome project (HCP) Q1 and S1200 dataset. Promising brain state classification performances are observed (e.g., an average of 93.7% for seven-task classification and 100% for binary classification). In addition, the importance of the brain regions, which contributes most to the accurate classification, is successfully quantified and visualized. A thorough neuroscientific interpretation suggests that these extracted brain regions and their importance calculated from self-attention graph pooling layer offer substantial explainability.

Modeling spatio-temporal patterns of holistic functional brain networks via multi-head guided attention graph neural networks (Multi-Head GAGNNs).

Mounting evidence has demonstrated that complex brain function processes are realized by the interaction of holistic functional brain networks which are spatially distributed across specific brain regions in a temporally dynamic fashion. Therefore, modeling spatio-temporal patterns of holistic functional brain networks plays an important role in understanding brain function. Compared to traditional modeling methods such as principal component analysis, independent component analysis, and sparse coding, superior performance has been achieved by recent deep learning methodologies. However, there are still two limitations of existing deep learning approaches for functional brain network modeling. They either (1) merely modeled a single targeted network and ignored holistic ones at one time, or (2) underutilized both spatial and temporal features of fMRI during network modeling, and the spatial/temporal accuracy was thus not warranted. To address these limitations, we proposed a novel Multi-Head Guided Attention Graph Neural Network (Multi-Head GAGNN) to simultaneously model both spatial and temporal patterns of holistic functional brain networks. Specifically, a spatial Multi-Head Attention Graph U-Net was first adopted to model the spatial patterns of multiple brain networks, and a temporal Multi-Head Guided Attention Network was then introduced to model the corresponding temporal patterns under the guidance of modeled spatial patterns. Based on seven task fMRI datasets from the public Human Connectome Project and resting state fMRI datasets from the public Autism Brain Imaging Data Exchange I of 1448 subjects, the proposed Multi-Head GAGNN showed superior ability and generalizability in modeling both spatial and temporal patterns of holistic functional brain networks in individual brains compared to other state-of-the-art (SOTA) models. Furthermore, the modeled spatio-temporal patterns of functional brain networks via the proposed Multi-Head GAGNN can better predict the individual cognitive behavioral measures compared to the other SOTA models. This study provided a novel and powerful tool for brain function modeling as well as for understanding the brain-cognitive behavior associations.

NAS-optimized topology-preserving transfer learning for differentiating cortical folding patterns.

Increasing evidence suggests that cortical folding patterns of human cerebral cortex manifest overt structural and functional differences. However, for interpretability, few studies leverage advanced techniques (e.g., deep learning) to investigate the difference among cortical folds, resulting in more differences yet to be extensively explored. To this end, we proposed an effective topology-preserving transfer learning framework to differentiate cortical fMRI time series extracted from cortical folds. Our framework consists of three main parts: (1) Neural architecture search (NAS), which is used to devise a well-performing network structure based on an initialized hand-designed super-graph in an image dataset; (2) Topology-preserving transfer, which takes the model searched by NAS as the source network, keeping the topological connectivity in the network unchanged, while transforming all 2D operations including convolution and pooling into 1D, therefore resulting in a topology-preserving network, named TPNAS-Net; (3) Classification and correlation analysis, which involves using the TPNAS-Net to classify 1D cortical fMRI time series for each individual brain, and performing a group difference analysis between autism spectrum disorder (ASD) and healthy control (HC) and correlation analysis with clinical information (i.e., age). Extensive experiments on two ASD datasets obtain consistent results, demonstrating that the TPNAS-Net not only discriminates cortical folding patterns at high classification accuracy, but also captures subtle differences between ASD and HC (p-value = 0.042). In addition, we discover that there is a positive correlation between the classification accuracy and age in ASD (r = 0.39, p-value = 0.04). These findings together suggest that structural and functional differences in cortical folding patterns between ASD and HC may provide a potentially useful biomarker for the diagnosis of ASD.

The effects of lutein and zeaxanthin on resting state functional connectivity in older Caucasian adults: a randomized controlled trial.

The carotenoids lutein (L) and zeaxanthin (Z) accumulate in retinal regions of the eye and have long been shown to benefit visual health. A growing literature suggests cognitive benefits as well, particularly in older adults. The present randomized controlled trial sought to investigate the effects of L and Z on brain function using resting state functional magnetic resonance imaging (fMRI). It was hypothesized that L and Z supplementation would (1) improve intra-network integrity of default mode network (DMN) and (2) reduce inter-network connectivity between DMN and other resting state networks. 48 community-dwelling older adults (mean age = 72 years) were randomly assigned to receive a daily L (10 mg) and Z (2 mg) supplement or a placebo for 1 year. Resting state fMRI data were acquired at baseline and post-intervention. A dictionary learning and sparse coding computational framework, based on machine learning principles, was used to investigate intervention-related changes in functional connectivity. DMN integrity was evaluated by calculating spatial overlap rate with a well-established DMN template provided in the neuroscience literature. Inter-network connectivity was evaluated via time series correlations between DMN and nine other resting state networks. Contrary to expectation, results indicated that L and Z significantly increased rather than decreased inter-network connectivity (Cohen's d = 0.89). A significant intra-network effect on DMN integrity was not observed. Rather than restoring what has been described in the available literature as a "youth-like" pattern of intrinsic brain activity, L and Z may facilitate the aging brain's capacity for compensation by enhancing integration between networks that tend to be functionally segregated earlier in the lifespan.

Joint representation of connectome-scale structural and functional profiles for identification of consistent cortical landmarks in macaque brain.

Discovery and representation of common structural and functional cortical architectures has been a significant yet challenging problem for years. Due to the remarkable variability of structural and functional cortical architectures in human brain, it is challenging to jointly represent a common cortical architecture which can comprehensively encode both structure and function characteristics. In order to better understand this challenge and considering that macaque monkey brain has much less variability in structure and function compared with human brain, in this paper, we propose a novel computational framework to apply our DICCCOL (Dense Individualized and Common Connectivity-based Cortical Landmarks) and HAFNI (Holistic Atlases of Functional Networks and Interactions) frameworks on macaque brains, in order to jointly represent structural and functional connectome-scale profiles for identification of a set of consistent and common cortical landmarks across different macaque brains based on multimodal DTI and resting state fMRI (rsfMRI) data. Experimental results demonstrate that 100 consistent and common cortical landmarks are successfully identified via the proposed framework, each of which has reasonably accurate anatomical, structural fiber connection pattern, and functional correspondences across different macaque brains. This set of 100 landmarks offer novel insights into the structural and functional cortical architectures in macaque brains.

Dynamic Time-frequency Feature Extraction for Brain Activity Recognition.

The biomedical signal classification accuracy on motor imagery is not always satisfactory, partially because not all the important features have been effectively extracted. This paper proposes an improved dynamic feature extraction approach based on a time-frequency representation and an optimal sequence similarity measurement. Since the wavelet packet decomposition (WPD) generates more detailed signal variation information and the dynamic time warping (DTW) helps optimally measure the sequence similarity, more important features are kept for classification. We apply the extracted features from our proposed method to Electroencephalogram (EEG) based motor imagery through the OpenBCI device and obtain higher classification accuracy. Compared with traditional feature extraction methods, there is a significant classification accuracy improvement from 83.53% to 90.89%. Our work demonstrates the importance of the advanced feature extraction in time series data analysis, e.g. biomedical signal.

Automatic recognition of holistic functional brain networks using iteratively optimized convolutional neural networks (IO-CNN) with weak label initialization.

fMRI data decomposition techniques have advanced significantly from shallow models such as Independent Component Analysis (ICA) and Sparse Coding and Dictionary Learning (SCDL) to deep learning models such Deep Belief Networks (DBN) and Convolutional Autoencoder (DCAE). However, interpretations of those decomposed networks are still open questions due to the lack of functional brain atlases, no correspondence across decomposed or reconstructed networks across different subjects, and significant individual variabilities. Recent studies showed that deep learning, especially deep convolutional neural networks (CNN), has extraordinary ability of accommodating spatial object patterns, e.g., our recent works using 3D CNN for fMRI-derived network classifications achieved high accuracy with a remarkable tolerance for mistakenly labelled training brain networks. However, the training data preparation is one of the biggest obstacles in these supervised deep learning models for functional brain network map recognitions, since manual labelling requires tedious and time-consuming labours which will sometimes even introduce label mistakes. Especially for mapping functional networks in large scale datasets such as hundreds of thousands of brain networks used in this paper, the manual labelling method will become almost infeasible. In response, in this work, we tackled both the network recognition and training data labelling tasks by proposing a new iteratively optimized deep learning CNN (IO-CNN) framework with an automatic weak label initialization, which enables the functional brain networks recognition task to a fully automatic large-scale classification procedure. Our extensive experiments based on ABIDE-II 1099 brains' fMRI data showed the great promise of our IO-CNN framework.

Decoding Auditory Saliency from Brain Activity Patterns during Free Listening to Naturalistic Audio Excerpts.

In recent years, natural stimuli such as audio excerpts or video streams have received increasing attention in neuroimaging studies. Compared with conventional simple, idealized and repeated artificial stimuli, natural stimuli contain more unrepeated, dynamic and complex information that are more close to real-life. However, there is no direct correspondence between the stimuli and any sensory or cognitive functions of the brain, which makes it difficult to apply traditional hypothesis-driven analysis methods (e.g., the general linear model (GLM)). Moreover, traditional data-driven methods (e.g., independent component analysis (ICA)) lack quantitative modeling of stimuli, which may limit the power of analysis models. In this paper, we propose a sparse representation based decoding framework to explore the neural correlates between the computational audio features and functional brain activities under free listening conditions. First, we adopt a biologically-plausible auditory saliency feature to quantitatively model the audio excerpts and meanwhile develop sparse representation/dictionary learning method to learn an over-complete dictionary basis of brain activity patterns. Then, we reconstruct the auditory saliency features from the learned fMRI-derived dictionaries. After that, a group-wise analysis procedure is conducted to identify the associated brain regions and networks. Experiments showed that the auditory saliency feature can be well decoded from brain activity patterns by our methods, and the identified brain regions and networks are consistent and meaningful. At last, our method is evaluated and compared with ICA method and experimental results demonstrated the superiority of our methods.

Joint representation of consistent structural and functional profiles for identification of common cortical landmarks.

In the brain mapping field, there have been significant interests in representation of structural/functional profiles to establish structural/functional landmark correspondences across individuals and populations. For example, from the structural perspective, our previous studies have identified hundreds of consistent DICCCOL (dense individualized and common connectivity-based cortical landmarks) landmarks across individuals and populations, each of which possess consistent DTI-derived fiber connection patterns. From the functional perspective, a large collection of well-characterized HAFNI (holistic atlases of functional networks and interactions) networks based on sparse representation of whole-brain fMRI signals have been identified in our prior studies. However, due to the remarkable variability of structural and functional architectures in the human brain, it is challenging for earlier studies to jointly represent the connectome-scale structural and functional profiles for establishing a common cortical architecture which can comprehensively encode both structural and functional characteristics across individuals. To address this challenge, we propose an effective computational framework to jointly represent the structural and functional profiles for identification of consistent and common cortical landmarks with both structural and functional correspondences across different brains based on DTI and fMRI data. Experimental results demonstrate that 55 structurally and functionally common cortical landmarks can be successfully identified.

Decoding power-spectral profiles from FMRI brain activities during naturalistic auditory experience.

Recent studies have demonstrated a close relationship between computational acoustic features and neural brain activities, and have largely advanced our understanding of auditory information processing in the human brain. Along this line, we proposed a multidisciplinary study to examine whether power spectral density (PSD) profiles can be decoded from brain activities during naturalistic auditory experience. The study was performed on a high resolution functional magnetic resonance imaging (fMRI) dataset acquired when participants freely listened to the audio-description of the movie "Forrest Gump". Representative PSD profiles existing in the audio-movie were identified by clustering the audio samples according to their PSD descriptors. Support vector machine (SVM) classifiers were trained to differentiate the representative PSD profiles using corresponding fMRI brain activities. Based on PSD profile decoding, we explored how the neural decodability correlated to power intensity and frequency deviants. Our experimental results demonstrated that PSD profiles can be reliably decoded from brain activities. We also suggested a sigmoidal relationship between the neural decodability and power intensity deviants of PSD profiles. Our study in addition substantiates the feasibility and advantage of naturalistic paradigm for studying neural encoding of complex auditory information.

Holistic atlases of functional networks and interactions reveal reciprocal organizational architecture of cortical function.

For decades, it has been largely unknown to what extent multiple functional networks spatially overlap/interact with each other and jointly realize the total cortical function. Here, by developing novel sparse representation of whole-brain fMRI signals and by using the recently publicly released large-scale Human Connectome Project high-quality fMRI data, we show that a number of reproducible and robust functional networks, including both task-evoked and resting state networks, are simultaneously distributed in distant neuroanatomic areas and substantially spatially overlapping with each other, thus forming an initial collection of holistic atlases of functional networks and interactions (HAFNIs). More interestingly, the HAFNIs revealed two distinct patterns of highly overlapped regions and highly specialized regions and exhibited that these two patterns of areas are reciprocally localized, revealing a novel organizational principle of cortical function.

Data-driven analysis of functional brain interactions during free listening to music and speech.

Natural stimulus functional magnetic resonance imaging (N-fMRI) such as fMRI acquired when participants were watching video streams or listening to audio streams has been increasingly used to investigate functional mechanisms of the human brain in recent years. One of the fundamental challenges in functional brain mapping based on N-fMRI is to model the brain's functional responses to continuous, naturalistic and dynamic natural stimuli. To address this challenge, in this paper we present a data-driven approach to exploring functional interactions in the human brain during free listening to music and speech streams. Specifically, we model the brain responses using N-fMRI by measuring the functional interactions on large-scale brain networks with intrinsically established structural correspondence, and perform music and speech classification tasks to guide the systematic identification of consistent and discriminative functional interactions when multiple subjects were listening music and speech in multiple categories. The underlying premise is that the functional interactions derived from N-fMRI data of multiple subjects should exhibit both consistency and discriminability. Our experimental results show that a variety of brain systems including attention, memory, auditory/language, emotion, and action networks are among the most relevant brain systems involved in classic music, pop music and speech differentiation. Our study provides an alternative approach to investigating the human brain's mechanism in comprehension of complex natural music and speech.

Increased cortico-subcortical functional connectivity in schizophrenia.

It has been widely reported that structural and functional connectivities are disturbed in cortical networks in schizophrenia (SZ). However, much less is known about the structural and functional connectivities between cortical and subcortical regions in SZ. Here, diffusion tensor imaging (DTI) data was used to identify consistent cortico-subcortical structural connection patterns across SZ patients and controls, and thus 13 common cortical Regions of Interest (ROIs) were determined. DTI and resting state fMRI (R-fMRI) datasets were used to assess the structural and functional connectivities between the 13 cortical ROIs and 12 subcortical regions in 8 SZ patients and 10 normal controls. It was found that there are significantly increased functional connectivities for 7 cortico-subcortical connections between the 13 cortical ROIs and 12 subcortical regions. Among most of these connections, the functional connectivity strength was doubled in SZ in comparison to controls. The cortical ROIs with functional hyper-connectivities to subcortical regions are localized in frontal and parietal lobes. However, no significant difference in the structural connectivity between these cortical and subcortical regions was found between SZ and controls. Additional analysis results showed 4 significantly increased and 2 significantly decreased cortico-cortical connections. Our study results suggest the functional hyper-connectivity between cortical and subcortical regions, adding further evidence to literature findings that SZ is a disorder of connectivity between components of large-scale brain networks. The result of either increased or decreased functional connectivities among cortical ROIs exhibits the complex pattern of disturbance of brain networks in SZ.