Mapping dynamic spatial patterns of brain function with spatial-wise attention.

Objective: Using functional magnetic resonance imaging (fMRI) and deep learning to discover the spatial pattern of brain function, or functional brain networks (FBNs) has been attracted many reseachers. Most existing works focus on static FBNs or dynamic functional connectivity among fixed spatial network nodes, but ignore the potential dynamic/time-varying characteristics of the spatial networks themselves. And most of works based on the assumption of linearity and independence, that oversimplify the relationship between blood-oxygen level dependence signal changes and the heterogeneity of neuronal activity within voxels.Approach: To overcome these problems, we proposed a novel spatial-wise attention (SA) based method called Spatial and Channel-wise Attention Autoencoder (SCAAE) to discover the dynamic FBNs without the assumptions of linearity or independence. The core idea of SCAAE is to apply the SA to generate FBNs directly, relying solely on the spatial information present in fMRI volumes. Specifically, we trained the SCAAE in a self-supervised manner, using the autoencoder to guide the SA to focus on the activation regions. Experimental results show that the SA can generate multiple meaningful FBNs at each fMRI time point, which spatial similarity are close to the FBNs derived by known classical methods, such as independent component analysis.Main results: To validate the generalization of the method, we evaluate the approach on HCP-rest, HCP-task and ADHD-200 dataset. The results demonstrate that SA mechanism can be used to discover time-varying FBNs, and the identified dynamic FBNs over time clearly show the process of time-varying spatial patterns fading in and out.Significance: Thus we provide a novel method to understand human brain better. Code is available athttps://github.com/WhatAboutMyStar/SCAAE.

Cerebral cortical regions always connect with each other via the shortest paths.

In human society, the choice of transportation mode between two cities is largely influenced by the distance between the regions. Similarly, when neurons communicate with each other within the cerebral cortex, do they establish their connections based on their physical distance? In this study, we employed a data-driven approach to explore the relationships between fiber length and corresponding geodesic distance between the fiber's two endpoints on brain surface. Diffusion-MRI-derived fiber streamlines were used to represent extra-cortical axonal connections between neurons or cortical regions, while geodesic paths between cortical points were employed to simulate intra-cortical connections. The results demonstrated that the geodesic distance between two cortical regions connected by a fiber streamline was greater than the fiber length most of the time, indicating that cortical regions tend to choose the shortest path for connection; whether it be an intra-cortical or extra-cortical route, especially when intra-cortical routes within cortical regions are longer than potential extrinsic fiber routes, there is an increased probability to establish fiber routes to connect the both regions. These findings were validated in a group of human brains and may provide insights into the underlying mechanisms of neuronal growth, connection, and wiring.

Multi-head attention-based masked sequence model for mapping functional brain networks.

The investigation of functional brain networks (FBNs) using task-based functional magnetic resonance imaging (tfMRI) has gained significant attention in the field of neuroimaging. Despite the availability of several methods for constructing FBNs, including traditional methods like GLM and deep learning methods such as spatiotemporal self-attention mechanism (STAAE), these methods have design and training limitations. Specifically, they do not consider the intrinsic characteristics of fMRI data, such as the possibility that the same signal value at different time points could represent different brain states and meanings. Furthermore, they overlook prior knowledge, such as task designs, during training. This study aims to overcome these limitations and develop a more efficient model by drawing inspiration from techniques in the field of natural language processing (NLP). The proposed model, called the Multi-head Attention-based Masked Sequence Model (MAMSM), uses a multi-headed attention mechanism and mask training approach to learn different states corresponding to the same voxel values. Additionally, it combines cosine similarity and task design curves to construct a novel loss function. The MAMSM was applied to seven task state datasets from the Human Connectome Project (HCP) tfMRI dataset. Experimental results showed that the features acquired by the MAMSM model exhibit a Pearson correlation coefficient with the task design curves above 0.95 on average. Moreover, the model can extract more meaningful networks beyond the known task-related brain networks. The experimental results demonstrated that MAMSM has great potential in advancing the understanding of functional brain networks.

Accurate corresponding fiber tract segmentation via FiberGeoMap learner with application to autism.

Fiber tract segmentation is a prerequisite for tract-based statistical analysis. Brain fiber streamlines obtained by diffusion magnetic resonance imaging and tractography technology are usually difficult to be leveraged directly, thus need to be segmented into fiber tracts. Previous research mainly consists of two steps: defining and computing the similarity features of fiber streamlines, then adopting machine learning algorithms for fiber clustering or classification. Defining the similarity feature is the basic premise and determines its potential reliability and application. In this study, we adopt geometric features for fiber tract segmentation and develop a novel descriptor (FiberGeoMap) for the corresponding representation, which can effectively depict fiber streamlines' shapes and positions. FiberGeoMap can differentiate fiber tracts within the same subject, meanwhile preserving the shape and position consistency across subjects, thus can identify common fiber tracts across brains. We also proposed a Transformer-based encoder network called FiberGeoMap Learner, to perform segmentation based on the geometric features. Experimental results showed that the proposed method can differentiate the 103 various fiber tracts, which outperformed the existing methods in both the number of categories and segmentation accuracy. Furthermore, the proposed method identified some fiber tracts that were statistically different on fractional anisotropy (FA), mean diffusion (MD), and fiber number ration in autism.

Survey on natural language processing in medical image analysis. / 自然语言处理在医学影像分析中的应用.

Recent advancement in natural language processing (NLP) and medical imaging empowers the wide applicability of deep learning models. These developments have increased not only data understanding, but also knowledge of state-of-the-art architectures and their real-world potentials. Medical imaging researchers have recognized the limitations of only targeting images, as well as the importance of integrating multimodal inputs into medical image analysis. The lack of comprehensive surveys of the current literature, however, impedes the progress of this domain. Existing research perspectives, as well as the architectures, tasks, datasets, and performance measures examined in the present literature, are reviewed in this work, and we also provide a brief description of possible future directions in the field, aiming to provide researchers and healthcare professionals with a detailed summary of existing academic research and to provide rational insights to facilitate future research.