Sparse Graph Representation Learning based on Reinforcement Learning for Personalized Mild Cognitive Impairment (MCI) Diagnosis.

Resting-state functional magnetic resonance imaging (rs-fMRI) has gained attention as a reliable technique for investigating the intrinsic function patterns of the brain. It facilitates the extraction of functional connectivity networks (FCNs) that capture synchronized activity patterns among regions of interest (ROIs). Analyzing FCNs enables the identification of distinctive connectivity patterns associated with mild cognitive impairment (MCI). For MCI diagnosis, various sparse representation techniques have been introduced, including statistical- and deep learningbased methods. However, these methods face limitations due to their reliance on supervised learning schemes, which restrict the exploration necessary for probing novel solutions. To overcome such limitation, prior work has incorporated reinforcement learning (RL) to dynamically select ROIs, but effective exploration remains challenging due to the vast search space during training. To tackle this issue, in this study, we propose an advanced RL-based framework that utilizes a divide-and-conquer approach to decompose the FCN construction task into smaller subproblems in a subject-specific manner, enabling efficient exploration under each sub-problem condition. Additionally, we leverage the learned value function to determine the sparsity level of FCNs, considering individual characteristics of FCNs. We validate the effectiveness of our proposed framework by demonstrating its superior performance in MCI diagnosis on publicly available cohort datasets.

FTMMR: Fusion Transformer for Integrating Multiple Molecular Representations.

Molecular property prediction has gained substantial attention due to its potential for various bio-chemical applications. Numerous attempts have been made to enhance the performance by combining multiple molecular representations (1D, 2D, and 3D). However, most prior works only merged a limited number of representations or tried to embed multiple representations through a single network without using representation-specific networks. Furthermore, the heterogeneous characteristics of each representation made the fusion more challenging. Addressing these challenges, we introduce the Fusion Transformer for Multiple Molecular Representations (FTMMR) framework. Our strategy employs three distinct representation-specific networks and integrates information from each network using a fusion transformer architecture to generate fused representations. Additionally, we use self-supervised learning methods to align heterogeneous representations and to effectively utilize the limited chemical data available. In particular, we adopt a combinatorial loss function to leverage the contrastive loss for all three representations. We evaluate the performance of FTMMR using seven benchmark datasets, demonstrating that our framework outperforms existing fusion and self-supervised methods.

DeepHealthNet: Adolescent Obesity Prediction System Based on a Deep Learning Framework.

The global prevalence of childhood and adolescent obesity is a major concern due to its association with chronic diseases and long-term health risks. Artificial intelligence technology has been identified as a potential solution to accurately predict obesity rates and provide personalized feedback to adolescents. This study highlights the importance of early identification and prevention of obesity-related health issues. To develop effective algorithms for the prediction of obesity rates and provide personalized feedback, factors such as height, weight, waist circumference, calorie intake, physical activity levels, and other relevant health information must be taken into account. Therefore, by collecting health datasets from 321 adolescents who participated in Would You Do It! application, we proposed an adolescent obesity prediction system that provides personalized predictions and assists individuals in making informed health decisions. Our proposed deep learning framework, DeepHealthNet, effectively trains the model using data augmentation techniques, even when daily health data are limited, resulting in improved prediction accuracy (acc: 0.8842). Additionally, the study revealed variations in the prediction of the obesity rate between boys (acc: 0.9320) and girls (acc: 0.9163), allowing the identification of disparities and the determination of the optimal time to provide feedback. Statistical analysis revealed that the performance of the proposed deep learning framework was more statistically significant (p 0.001) compared to the other general models. The proposed system has the potential to effectively address childhood and adolescent obesity.

Spectral Graph Neural Network-Based Multi-Atlas Brain Network Fusion for Major Depressive Disorder Diagnosis.

Major Depressive Disorder (MDD) imposes a substantial burden within the healthcare domain, impacting millions of individuals worldwide. Functional Magnetic Resonance Imaging (fMRI) has emerged as a promising tool for the objective diagnosis of MDD, enabling the investigation of functional connectivity patterns in the brain associated with this disorder. However, most existing methods focus on a single brain atlas, which limits their ability to capture the complex, multi-scale nature of functional brain networks. To address these limitations, we propose a novel multi-atlas fusion method that incorporates early and late fusion in a unified framework. Our method introduces the concept of the holistic Functional Connectivity Network (FCN), which captures both intra-atlas relationships within individual atlases and inter-regional relationships between atlases with different brain parcellation scales. This comprehensive representation enables the identification of potential disease-related patterns associated with MDD in the early stage of our framework. Moreover, by decoding the holistic FCN from various perspectives through multiple spectral Graph Convolutional Neural Networks and fusing their results with decision-level ensembles, we further improve the performance of MDD diagnosis. Our approach is easily implemented with minimal modifications to existing model structures and demonstrates a robust performance across different baseline models. Our method, evaluated on public resting-state fMRI datasets, surpasses the current multi-atlas fusion methods, enhancing the accuracy of MDD diagnosis. The proposed novel multi-atlas fusion framework provides a more reliable MDD diagnostic technique. Experimental results show our approach outperforms both single- and multi-atlas-based methods, demonstrating its effectiveness in advancing MDD diagnosis.

A Unified Multi-Modality Fusion Framework for Deep Spatio-Temporal-Spectral Feature Learning in Resting-State fMRI Denoising.

Resting-state functional magnetic resonance imaging (rs-fMRI) is a commonly used functional neuroimaging technique to investigate the functional brain networks. However, rs-fMRI data are often contaminated with noise and artifacts that adversely affect the results of rs-fMRI studies. Several machine/deep learning methods have achieved impressive performance to automatically regress the noise-related components decomposed from rs-fMRI data, which are expressed as the pairs of a spatial map and its associated time series. However, most of the previous methods individually analyze each modality of the noise-related components and simply aggregate the decision-level information (or knowledge) extracted from each modality to make a final decision. Moreover, these approaches consider only the limited modalities making it difficult to explore class-discriminative spectral information of noise-related components. To overcome these limitations, we propose a unified deep attentive spatio-spectral-temporal feature fusion framework. We first adopt a learnable wavelet transform module at the input-level of the framework to elaborately explore the spectral information in subsequent processes. We then construct a feature-level multi-modality fusion module to efficiently exchange the information from multi-modality inputs in the feature space. Finally, we design confidence-based voting strategies for decision-level fusion at the end of the framework to make a robust final decision. In our experiments, the proposed method achieved remarkable performance for noise-related component detection on various rs-fMRI datasets.

Graph-Based Conditional Generative Adversarial Networks for Major Depressive Disorder Diagnosis With Synthetic Functional Brain Network Generation.

Major Depressive Disorder (MDD) is a pervasive disorder affecting millions of individuals, presenting a significant global health concern. Functional connectivity (FC) derived from resting-state functional Magnetic Resonance Imaging (rs-fMRI) serves as a crucial tool in revealing functional connectivity patterns associated with MDD, playing an essential role in precise diagnosis. However, the limited data availability of FC poses challenges for robust MDD diagnosis. To tackle this, some studies have employed Deep Neural Networks (DNN) architectures to construct Generative Adversarial Networks (GAN) for synthetic FC generation, but this tends to overlook the inherent topology characteristics of FC. To overcome this challenge, we propose a novel Graph Convolutional Networks (GCN)-based Conditional GAN with Class-Aware Discriminator (GC-GAN). GC-GAN utilizes GCN in both the generator and discriminator to capture intricate FC patterns among brain regions, and the class-aware discriminator ensures the diversity and quality of the generated synthetic FC. Additionally, we introduce a topology refinement technique to enhance MDD diagnosis performance by optimizing the topology using the augmented FC dataset. Our framework was evaluated on publicly available rs-fMRI datasets, and the results demonstrate that GC-GAN outperforms existing methods. This indicates the superior potential of GCN in capturing intricate topology characteristics and generating high-fidelity synthetic FC, thus contributing to a more robust MDD diagnosis.

The effects of layer-wise relevance propagation-based feature selection for EEG classification: a comparative study on multiple datasets.

Introduction: The brain-computer interface (BCI) allows individuals to control external devices using their neural signals. One popular BCI paradigm is motor imagery (MI), which involves imagining movements to induce neural signals that can be decoded to control devices according to the user's intention. Electroencephalography (EEG) is frequently used for acquiring neural signals from the brain in the fields of MI-BCI due to its non-invasiveness and high temporal resolution. However, EEG signals can be affected by noise and artifacts, and patterns of EEG signals vary across different subjects. Therefore, selecting the most informative features is one of the essential processes to enhance classification performance in MI-BCI. Methods: In this study, we design a layer-wise relevance propagation (LRP)-based feature selection method which can be easily integrated into deep learning (DL)-based models. We assess its effectiveness for reliable class-discriminative EEG feature selection on two different publicly available EEG datasets with various DL-based backbone models in the subject-dependent scenario. Results and discussion: The results show that LRP-based feature selection enhances the performance for MI classification on both datasets for all DL-based backbone models. Based on our analysis, we believe that it can broad its capability to different research domains.

Bridging the BCI illiteracy gap: a subject-to-subject semantic style transfer for EEG-based motor imagery classification.

Introduction: Brain-computer interfaces (BCIs) facilitate direct interaction between the human brain and computers, enabling individuals to control external devices through cognitive processes. Despite its potential, the problem of BCI illiteracy remains one of the major challenges due to inter-subject EEG variability, which hinders many users from effectively utilizing BCI systems. In this study, we propose a subject-to-subject semantic style transfer network (SSSTN) at the feature-level to address the BCI illiteracy problem in electroencephalogram (EEG)-based motor imagery (MI) classification tasks. Methods: Our approach uses the continuous wavelet transform method to convert high-dimensional EEG data into images as input data. The SSSTN 1) trains a classifier for each subject, 2) transfers the distribution of class discrimination styles from the source subject (the best-performing subject for the classifier, i.e., BCI expert) to each subject of the target domain (the remaining subjects except the source subject, specifically BCI illiterates) through the proposed style loss, and applies a modified content loss to preserve the class-relevant semantic information of the target domain, and 3) finally merges the classifier predictions of both source and target subject using an ensemble technique. Results and discussion: We evaluate the proposed method on the BCI Competition IV-2a and IV-2b datasets and demonstrate improved classification performance over existing methods, especially for BCI illiterate users. The ablation experiments and t-SNE visualizations further highlight the effectiveness of the proposed method in achieving meaningful feature-level semantic style transfer.

Deep attentive spatio-temporal feature learning for automatic resting-state fMRI denoising.

Resting-state functional magnetic resonance imaging (rs-fMRI) is a non-invasive functional neuroimaging modality that has been widely used to investigate functional connectomes in the brain. Since noise and artifacts generated by non-neuronal physiological activities are predominant in raw rs-fMRI data, effective noise removal is one of the most important preprocessing steps prior to any subsequent analysis. For rs-fMRI denoising, a common trend is to decompose rs-fMRI data into multiple components and then regress out noise-related components. Therefore, various machine learning techniques have been used in such analyses with predefined procedures and manually engineered features. However, the lack of a universal definition of a noise-related source or artifact complicates manual feature engineering. Manual feature selection can result in the failure to capture unknown types of noise. Furthermore, the possibility that the hand-crafted features will only work for the broader population (e.g., healthy adults) but not for "outliers" (e.g., infants or subjects that belong to a disease cohort) is quite high. In practice, we have limited knowledge of which features should be extracted; thus, multi-classifier assembly must be implemented to improve performance, although this process is quite time-consuming. However, in real rs-fMRI applications, fast and accurate automatic identification of noise-related components on different datasets is critical. To solve this problem, we propose a novel, automatic, and end-to-end deep learning framework dedicated to noise-related component identification via a faster and more effective multi-layer feature extraction strategy that learns deeply embedded spatio-temporal features of the components. In this study, we achieved remarkable performance on various rs-fMRI datasets, including multiple adult rs-fMRI datasets from different rs-fMRI studies and an infant rs-fMRI dataset, which is quite heterogeneous and differs from that of adults. Our proposed framework also dramatically increases the noise detection speed owing to its inherent ability for deep learning (< 1s for single-component classification). It can be easily integrated into any preprocessing pipeline, even those that do not use standard procedures but depend on alternative toolboxes.

A Computational Framework for Dissociating Development-Related from Individually Variable Flexibility in Regional Modularity Assignment in Early Infancy.

Functional brain development in early infancy is a highly dynamic and complex process. Understanding each brain region's topological role and its development in the brain functional connectivity (FC) networks is essential for early disorder detection. A handful of previous studies have mostly focused on how FC network is changing regarding age. These approaches inevitably overlook the effect of individual variability for those at the same age that could shape unique cognitive capabilities and personalities among infants. With that in mind, we propose a novel computational framework based on across-subject across-age multilayer network analysis with a fully automatic (for parameter optimization), robust community detection algorithm. By detecting group consistent modules without losing individual information, this method allows a first-ever dissociation analysis of the two variability sources - age dependency and individual specificity - that greatly shape early brain development. This method is applied to a large cohort of 0-2 years old infants' functional MRI data during natural sleep. We not only detected the brain regions with greatest flexibility in this early developmental period but also identified five categories of brain regions with distinct development-related and individually variable flexibility changes. Our method is highly valuable for more thorough understanding of the early brain functional organizations and sheds light on early developmental abnormality detection.

Deep Learning of Static and Dynamic Brain Functional Networks for Early MCI Detection.

While convolutional neural network (CNN) has been demonstrating powerful ability to learn hierarchical spatial features from medical images, it is still difficult to apply it directly to resting-state functional MRI (rs-fMRI) and the derived brain functional networks (BFNs). We propose a novel CNN framework to simultaneously learn embedded features from BFNs for brain disease diagnosis. Since BFNs can be built by considering both static and dynamic functional connectivity (FC), we first decompose rs-fMRI into multiple static BFNs with modified independent component analysis. Then, the voxel-wise variability in dynamic FC is used to quantify BFN dynamics. A set of paired 3D images representing static/dynamic BFNs can be fed into 3D CNNs, from which we can hierarchically and simultaneously learn static/dynamic BFN features. As a result, the dynamic BFN features can complement static BFN features and, at the meantime, different BFNs can help each other toward a joint and better classification. We validate our method with a publicly accessible, large cohort of rs-fMRI dataset in early-stage mild cognitive impairment (eMCI) diagnosis, which is one of the most challenging problems to the clinicians. By comparing with a conventional method, our method shows significant diagnostic performance improvement by almost 10%. This result demonstrates the effectiveness of deep learning in preclinical Alzheimer's disease diagnosis, based on the complex and high-dimensional voxel-wise spatiotemporal patterns of the resting-state brain functional connectomics. The framework provides a new but intuitive way to fully exploit deeply embedded diagnostic features from rs-fMRI for a better-individualized diagnosis of various neurological diseases.

Dynamic Routing Capsule Networks for Mild Cognitive Impairment Diagnosis.

Alzheimer's disease (AD) is a chronic neurodegenerative disease that could cause severe cognitive damage to the patients. Diagnosis of AD at its preclinical stage, i.e., mild cognitive impairment (MCI), could help to prevent or slow down AD progression. With machine learning, automatic MCI diagnosis could be achieved. Most of the previous studies mainly share a similar framework, i.e., building a classifier based on the features extracted from static or dynamic functional connectivity. Recently, inspired by the great successes achieved by deep learning in other areas of medical image analysis, researchers have introduced neural network models for MCI diagnosis. In this paper, we propose dynamic routing capsule networks for MCI diagnosis. Our proposed methods are based on a novel neural network fashion of capsule net. Two variants of capsule net are designed and discussed, which respectively uses the intra-ROIs and inter-ROIs dynamic routing to obtain functional representation. More importantly, we design a learnable dynamic functional connectivity metric in our inter-ROIs dynamic model, in which the functional connectivity is dynamically learned during network training. To the best of our knowledge, it's the first time to propose dynamic routing capsule networks for MCI diagnosis. Compared with other machine learning methods and deep learning model, our method can achieve superior performance from various aspects of evaluations.

A Novel Deep Learning Framework on Brain Functional Networks for Early MCI Diagnosis.

Although alternations of brain functional networks (BFNs) derived from resting-state functional magnetic resonance imaging (rs-fMRI) have been considered as promising biomarkers for early Alzheimer's disease (AD) diagnosis, it is still challenging to perform individualized diagnosis, especially at the very early stage of preclinical stage of AD, i.e., early mild cognitive impairment (eMCI). Recently, convolutional neural networks (CNNs) show powerful ability in computer vision and image analysis applications, but there is still a gap for directly applying CNNs to rs-fMRI-based disease diagnosis. In this paper, we propose a novel multiple-BFN-based 3D CNN framework that can automatically and deeply learn complex, high-level, hierarchical diagnostic features from various independent component analysis-derived BFNs. More importantly, the embedded features of different BFNs could comprehensively support each other towards a more accurate eMCI diagnosis in a unified model. The performance of the proposed method is validated by a large-sample, multisite, rigorously controlled publicly accessible dataset. The proposed framework can also be conveniently and straightforwardly applied to individualized diagnosis of various neurological and psychiatric diseases.

Multiple functional networks modeling for autism spectrum disorder diagnosis.

Despite countless studies on autism spectrum disorder (ASD), diagnosis relies on specific behavioral criteria and neuroimaging biomarkers for the disorder are still relatively scarce and irrelevant for diagnostic workup. Many researchers have focused on functional networks of brain activities using resting-state functional magnetic resonance imaging (rsfMRI) to diagnose brain diseases, including ASD. Although some existing methods are able to reveal the abnormalities in functional networks, they are either highly dependent on prior assumptions for modeling these networks or do not focus on latent functional connectivities (FCs) by considering discriminative relations among FCs in a nonlinear way. In this article, we propose a novel framework to model multiple networks of rsfMRI with data-driven approaches. Specifically, we construct large-scale functional networks with hierarchical clustering and find discriminative connectivity patterns between ASD and normal controls (NC). We then learn features and classifiers for each cluster through discriminative restricted Boltzmann machines (DRBMs). In the testing phase, each DRBM determines whether a test sample is ASD or NC, based on which we make a final decision with a majority voting strategy. We assess the diagnostic performance of the proposed method using public datasets and describe the effectiveness of our method by comparing it to competing methods. We also rigorously analyze FCs learned by DRBMs on each cluster and discover dominant FCs that play a major role in discriminating between ASD and NC. Hum Brain Mapp 38:5804-5821, 2017. © 2017 Wiley Periodicals, Inc.

Human visual cortical responses to specular and matte motion flows.

Determining the compositional properties of surfaces in the environment is an important visual capacity. One such property is specular reflectance, which encompasses the range from matte to shiny surfaces. Visual estimation of specular reflectance can be informed by characteristic motion profiles; a surface with a specular reflectance that is difficult to determine while static can be confidently disambiguated when set in motion. Here, we used fMRI to trace the sensitivity of human visual cortex to such motion cues, both with and without photometric cues to specular reflectance. Participants viewed rotating blob-like objects that were rendered as images (photometric) or dots (kinematic) with either matte-consistent or shiny-consistent specular reflectance profiles. We were unable to identify any areas in low and mid-level human visual cortex that responded preferentially to surface specular reflectance from motion. However, univariate and multivariate analyses identified several visual areas; V1, V2, V3, V3A/B, and hMT+, capable of differentiating shiny from matte surface flows. These results indicate that the machinery for extracting kinematic cues is present in human visual cortex, but the areas involved in integrating such information with the photometric cues necessary for surface specular reflectance remain unclear.