Metric learning with spectral graph convolutions on brain connectivity networks.

Graph representations are often used to model structured data at an individual or population level and have numerous applications in pattern recognition problems. In the field of neuroscience, where such representations are commonly used to model structural or functional connectivity between a set of brain regions, graphs have proven to be of great importance. This is mainly due to the capability of revealing patterns related to brain development and disease, which were previously unknown. Evaluating similarity between these brain connectivity networks in a manner that accounts for the graph structure and is tailored for a particular application is, however, non-trivial. Most existing methods fail to accommodate the graph structure, discarding information that could be beneficial for further classification or regression analyses based on these similarities. We propose to learn a graph similarity metric using a siamese graph convolutional neural network (s-GCN) in a supervised setting. The proposed framework takes into consideration the graph structure for the evaluation of similarity between a pair of graphs, by employing spectral graph convolutions that allow the generalisation of traditional convolutions to irregular graphs and operates in the graph spectral domain. We apply the proposed model on two datasets: the challenging ABIDE database, which comprises functional MRI data of 403 patients with autism spectrum disorder (ASD) and 468 healthy controls aggregated from multiple acquisition sites, and a set of 2500 subjects from UK Biobank. We demonstrate the performance of the method for the tasks of classification between matching and non-matching graphs, as well as individual subject classification and manifold learning, showing that it leads to significantly improved results compared to traditional methods.

Human brain mapping: A systematic comparison of parcellation methods for the human cerebral cortex.

The macro-connectome elucidates the pathways through which brain regions are structurally connected or functionally coupled to perform a specific cognitive task. It embodies the notion of representing and understanding all connections within the brain as a network, while the subdivision of the brain into interacting functional units is inherent in its architecture. As a result, the definition of network nodes is one of the most critical steps in connectivity network analysis. Although brain atlases obtained from cytoarchitecture or anatomy have long been used for this task, connectivity-driven methods have arisen only recently, aiming to delineate more homogeneous and functionally coherent regions. This study provides a systematic comparison between anatomical, connectivity-driven and random parcellation methods proposed in the thriving field of brain parcellation. Using resting-state functional MRI data from the Human Connectome Project and a plethora of quantitative evaluation techniques investigated in the literature, we evaluate 10 subject-level and 24 groupwise parcellation methods at different resolutions. We assess the accuracy of parcellations from four different aspects: (1) reproducibility across different acquisitions and groups, (2) fidelity to the underlying connectivity data, (3) agreement with fMRI task activation, myelin maps, and cytoarchitectural areas, and (4) network analysis. This extensive evaluation of different parcellations generated at the subject and group level highlights the strengths and shortcomings of the various methods and aims to provide a guideline for the choice of parcellation technique and resolution according to the task at hand. The results obtained in this study suggest that there is no optimal method able to address all the challenges faced in this endeavour simultaneously.

A flexible graphical model for multi-modal parcellation of the cortex.

Advances in neuroimaging have provided a tremendous amount of in-vivo information on the brain's organisation. Its anatomy and cortical organisation can be investigated from the point of view of several imaging modalities, many of which have been studied for mapping functionally specialised cortical areas. There is strong evidence that a single modality is not sufficient to fully identify the brain's cortical organisation. Combining multiple modalities in the same parcellation task has the potential to provide more accurate and robust subdivisions of the cortex. Nonetheless, existing brain parcellation methods are typically developed and tested on single modalities using a specific type of information. In this paper, we propose Graph-based Multi-modal Parcellation (GraMPa), an iterative framework designed to handle the large variety of available input modalities to tackle the multi-modal parcellation task. At each iteration, we compute a set of parcellations from different modalities and fuse them based on their local reliabilities. The fused parcellation is used to initialise the next iteration, forcing the parcellations to converge towards a set of mutually informed modality specific parcellations, where correspondences are established. We explore two different multi-modal configurations for group-wise parcellation using resting-state fMRI, diffusion MRI tractography, myelin maps and task fMRI. Quantitative and qualitative results on the Human Connectome Project database show that integrating multi-modal information yields a stronger agreement with well established atlases and more robust connectivity networks that provide a better representation of the population.

Group-wise parcellation of the cortex through multi-scale spectral clustering.

The delineation of functionally and structurally distinct regions as well as their connectivity can provide key knowledge towards understanding the brain's behaviour and function. Cytoarchitecture has long been the gold standard for such parcellation tasks, but has poor scalability and cannot be mapped in vivo. Functional and diffusion magnetic resonance imaging allow in vivo mapping of brain's connectivity and the parcellation of the brain based on local connectivity information. Several methods have been developed for single subject connectivity driven parcellation, but very few have tackled the task of group-wise parcellation, which is essential for uncovering group specific behaviours. In this paper, we propose a group-wise connectivity-driven parcellation method based on spectral clustering that captures local connectivity information at multiple scales and directly enforces correspondences between subjects. The method is applied to diffusion Magnetic Resonance Imaging driven parcellation on two independent groups of 50 subjects from the Human Connectome Project. Promising quantitative and qualitative results in terms of information loss, modality comparisons, group consistency and inter-group similarities demonstrate the potential of the method.

CLAD: A realistic Continual Learning benchmark for Autonomous Driving.

In this paper we describe the design and the ideas motivating a new Continual Learning benchmark for Autonomous Driving (CLAD), that focuses on the problems of object classification and object detection. The benchmark utilises SODA10M, a recently released large-scale dataset that concerns autonomous driving related problems. First, we review and discuss existing continual learning benchmarks, how they are related, and show that most are extreme cases of continual learning. To this end, we survey the benchmarks used in continual learning papers at three highly ranked computer vision conferences. Next, we introduce CLAD-C, an online classification benchmark realised through a chronological data stream that poses both class and domain incremental challenges; and CLAD-D, a domain incremental continual object detection benchmark. We examine the inherent difficulties and challenges posed by the benchmark, through a survey of the techniques and methods used by the top-3 participants in a CLAD-challenge workshop at ICCV 2021. We conclude with possible pathways to improve the current continual learning state of the art, and which directions we deem promising for future research.

A Continual Learning Survey: Defying Forgetting in Classification Tasks.

Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase. The resulting network resembles a static entity of knowledge, with endeavours to extend this knowledge without targeting the original task resulting in a catastrophic forgetting. Continual learning shifts this paradigm towards networks that can continually accumulate knowledge over different tasks without the need to retrain from scratch. We focus on task incremental classification, where tasks arrive sequentially and are delineated by clear boundaries. Our main contributions concern: (1) a taxonomy and extensive overview of the state-of-the-art; (2) a novel framework to continually determine the stability-plasticity trade-off of the continual learner; (3) a comprehensive experimental comparison of 11 state-of-the-art continual learning methods; and (4) baselines. We empirically scrutinize method strengths and weaknesses on three benchmarks, considering Tiny Imagenet and large-scale unbalanced iNaturalist and a sequence of recognition datasets. We study the influence of model capacity, weight decay and dropout regularization, and the order in which the tasks are presented, and qualitatively compare methods in terms of required memory, computation time, and storage.

Disease prediction using graph convolutional networks: Application to Autism Spectrum Disorder and Alzheimer's disease.

Graphs are widely used as a natural framework that captures interactions between individual elements represented as nodes in a graph. In medical applications, specifically, nodes can represent individuals within a potentially large population (patients or healthy controls) accompanied by a set of features, while the graph edges incorporate associations between subjects in an intuitive manner. This representation allows to incorporate the wealth of imaging and non-imaging information as well as individual subject features simultaneously in disease classification tasks. Previous graph-based approaches for supervised or unsupervised learning in the context of disease prediction solely focus on pairwise similarities between subjects, disregarding individual characteristics and features, or rather rely on subject-specific imaging feature vectors and fail to model interactions between them. In this paper, we present a thorough evaluation of a generic framework that leverages both imaging and non-imaging information and can be used for brain analysis in large populations. This framework exploits Graph Convolutional Networks (GCNs) and involves representing populations as a sparse graph, where its nodes are associated with imaging-based feature vectors, while phenotypic information is integrated as edge weights. The extensive evaluation explores the effect of each individual component of this framework on disease prediction performance and further compares it to different baselines. The framework performance is tested on two large datasets with diverse underlying data, ABIDE and ADNI, for the prediction of Autism Spectrum Disorder and conversion to Alzheimer's disease, respectively. Our analysis shows that our novel framework can improve over state-of-the-art results on both databases, with 70.4% classification accuracy for ABIDE and 80.0% for ADNI.

Reproducible Large-Scale Neuroimaging Studies with the OpenMOLE Workflow Management System.

OpenMOLE is a scientific workflow engine with a strong emphasis on workload distribution. Workflows are designed using a high level Domain Specific Language (DSL) built on top of Scala. It exposes natural parallelism constructs to easily delegate the workload resulting from a workflow to a wide range of distributed computing environments. OpenMOLE hides the complexity of designing complex experiments thanks to its DSL. Users can embed their own applications and scale their pipelines from a small prototype running on their desktop computer to a large-scale study harnessing distributed computing infrastructures, simply by changing a single line in the pipeline definition. The construction of the pipeline itself is decoupled from the execution context. The high-level DSL abstracts the underlying execution environment, contrary to classic shell-script based pipelines. These two aspects allow pipelines to be shared and studies to be replicated across different computing environments. Workflows can be run as traditional batch pipelines or coupled with OpenMOLE's advanced exploration methods in order to study the behavior of an application, or perform automatic parameter tuning. In this work, we briefly present the strong assets of OpenMOLE and detail recent improvements targeting re-executability of workflows across various Linux platforms. We have tightly coupled OpenMOLE with CARE, a standalone containerization solution that allows re-executing on a Linux host any application that has been packaged on another Linux host previously. The solution is evaluated against a Python-based pipeline involving packages such as scikit-learn as well as binary dependencies. All were packaged and re-executed successfully on various HPC environments, with identical numerical results (here prediction scores) obtained on each environment. Our results show that the pair formed by OpenMOLE and CARE is a reliable solution to generate reproducible results and re-executable pipelines. A demonstration of the flexibility of our solution showcases three neuroimaging pipelines harnessing distributed computing environments as heterogeneous as local clusters or the European Grid Infrastructure (EGI).

(Hyper)-graphical models in biomedical image analysis.

Computational vision, visual computing and biomedical image analysis have made tremendous progress over the past two decades. This is mostly due the development of efficient learning and inference algorithms which allow better and richer modeling of image and visual understanding tasks. Hyper-graph representations are among the most prominent tools to address such perception through the casting of perception as a graph optimization problem. In this paper, we briefly introduce the importance of such representations, discuss their strength and limitations, provide appropriate strategies for their inference and present their application to address a variety of problems in biomedical image analysis.

A Probabilistic Atlas of Diffuse WHO Grade II Glioma Locations in the Brain.

Diffuse WHO grade II gliomas are diffusively infiltrative brain tumors characterized by an unavoidable anaplastic transformation. Their management is strongly dependent on their location in the brain due to interactions with functional regions and potential differences in molecular biology. In this paper, we present the construction of a probabilistic atlas mapping the preferential locations of diffuse WHO grade II gliomas in the brain. This is carried out through a sparse graph whose nodes correspond to clusters of tumors clustered together based on their spatial proximity. The interest of such an atlas is illustrated via two applications. The first one correlates tumor location with the patient's age via a statistical analysis, highlighting the interest of the atlas for studying the origins and behavior of the tumors. The second exploits the fact that the tumors have preferential locations for automatic segmentation. Through a coupled decomposed Markov Random Field model, the atlas guides the segmentation process, and characterizes which preferential location the tumor belongs to and consequently which behavior it could be associated to. Leave-one-out cross validation experiments on a large database highlight the robustness of the graph, and yield promising segmentation results.

Joint Spectral Decomposition for the Parcellation of the Human Cerebral Cortex Using Resting-State fMRI.

Identification of functional connections within the human brain has gained a lot of attention due to its potential to reveal neural mechanisms. In a whole-brain connectivity analysis, a critical stage is the computation of a set of network nodes that can effectively represent cortical regions. To address this problem, we present a robust cerebral cortex parcellation method based on spectral graph theory and resting-state fMRI correlations that generates reliable parcellations at the single-subject level and across multiple subjects. Our method models the cortical surface in each hemisphere as a mesh graph represented in the spectral domain with its eigenvectors. We connect cortices of different subjects with each other based on the similarity of their connectivity profiles and construct a multi-layer graph, which effectively captures the fundamental properties of the whole group as well as preserves individual subject characteristics. Spectral decomposition of this joint graph is used to cluster each cortical vertex into a subregion in order to obtain whole-brain parcellations. Using rs-fMRI data collected from 40 healthy subjects, we show that our proposed algorithm computes highly reproducible parcellations across different groups of subjects and at varying levels of detail with an average Dice score of 0.78, achieving up to 9% better reproducibility compared to existing approaches. We also report that our group-wise parcellations are functionally more consistent, thus, can be reliably used to represent the population in network analyses.

Tractography-Driven Groupwise Multi-scale Parcellation of the Cortex.

The analysis of the connectome of the human brain provides key insight into the brain's organisation and function, and its evolution in disease or ageing. Parcellation of the cortical surface into distinct regions in terms of structural connectivity is an essential step that can enable such analysis. The estimation of a stable connectome across a population of healthy subjects requires the estimation of a groupwise parcellation that can capture the variability of the connectome across the population. This problem has solely been addressed in the literature via averaging of connectivity profiles or finding correspondences between individual parcellations a posteriori. In this paper, we propose a groupwise parcellation method of the cortex based on diffusion MR images (dMRI). We borrow ideas from the area of cosegmentation in computer vision and directly estimate a consistent parcellation across different subjects and scales through a spectral clustering approach. The parcellation is driven by the tractography connectivity profiles, and information between subjects and across scales. Promising qualitative and quantitative results on a sizeable data-set demonstrate the strong potential of the method.

Concurrent tumor segmentation and registration with uncertainty-based sparse non-uniform graphs.

In this paper, we present a graph-based concurrent brain tumor segmentation and atlas to diseased patient registration framework. Both segmentation and registration problems are modeled using a unified pairwise discrete Markov Random Field model on a sparse grid superimposed to the image domain. Segmentation is addressed based on pattern classification techniques, while registration is performed by maximizing the similarity between volumes and is modular with respect to the matching criterion. The two problems are coupled by relaxing the registration term in the tumor area, corresponding to areas of high classification score and high dissimilarity between volumes. In order to overcome the main shortcomings of discrete approaches regarding appropriate sampling of the solution space as well as important memory requirements, content driven samplings of the discrete displacement set and the sparse grid are considered, based on the local segmentation and registration uncertainties recovered by the min marginal energies. State of the art results on a substantial low-grade glioma database demonstrate the potential of our method, while our proposed approach shows maintained performance and strongly reduced complexity of the model.

Joint tumor segmentation and dense deformable registration of brain MR images.

In this paper we propose a novel graph-based concurrent registration and segmentation framework. Registration is modeled with a pairwise graphical model formulation that is modular with respect to the data and regularization term. Segmentation is addressed by adopting a similar graphical model, using image-based classification techniques while producing a smooth solution. The two problems are coupled via a relaxation of the registration criterion in the presence of tumors as well as a segmentation through a registration term aiming the separation between healthy and diseased tissues. Efficient linear programming is used to solve both problems simultaneously. State of the art results demonstrate the potential of our method on a large and challenging low-grade glioma data set.

Graph based spatial position mapping of low-grade gliomas.

Low-grade gliomas (WHO grade II) are diffusively infiltrative brain tumors arising from glial cells. Spatial classification that is usually based on cerebral lobes lacks accuracy and is far from being able to provide some pattern or statistical interpretation of their appearance. In this paper, we propose a novel approach to understand and infer position of low-grade gliomas using a graphical model. The problem is formulated as a graph topology optimization problem. Graph nodes correspond to extracted tumors and graph connections to the spatial and content dependencies among them. The task of spatial position mapping is then expressed as an unsupervised clustering problem, where cluster centers correspond to centers with position appearance prior, and cluster samples to nodes with strong statistical dependencies on their position with respect to the cluster center. Promising results using leave-one-out cross-validation outperform conventional dimensionality reduction methods and seem to coincide with conclusions drawn in physiological studies regarding the expected tumor spatial distributions and interactions.

Estimation of intra-operator variability in perfusion parameter measurements using DCE-US.

AIM: To investigate intra-operator variability of semi-quantitative perfusion parameters using dynamic contrast-enhanced ultrasonography (DCE-US), following bolus injections of SonoVue(®). METHODS: The in vitro experiments were conducted using three in-house sets up based on pumping a fluid through a phantom placed in a water tank. In the in vivo experiments, B16F10 melanoma cells were xenografted to five nude mice. Both in vitro and in vivo, images were acquired following bolus injections of the ultrasound contrast agent SonoVue(®) (Bracco, Milan, Italy) and using a Toshiba Aplio(®) ultrasound scanner connected to a 2.9-5.8 MHz linear transducer (PZT, PLT 604AT probe) (Toshiba, Japan) allowing harmonic imaging ("Vascular Recognition Imaging") involving linear raw data. A mathematical model based on the dye-dilution theory was developed by the Gustave Roussy Institute, Villejuif, France and used to evaluate seven perfusion parameters from time-intensity curves. Intra-operator variability analyses were based on determining perfusion parameter coefficients of variation (CV). RESULTS: In vitro, different volumes of SonoVue(®) were tested with the three phantoms: intra-operator variability was found to range from 2.33% to 23.72%. In vivo, experiments were performed on tumor tissues and perfusion parameters exhibited values ranging from 1.48% to 29.97%. In addition, the area under the curve (AUC) and the area under the wash-out (AUWO) were two of the parameters of great interest since throughout in vitro and in vivo experiments their variability was lower than 15.79%. CONCLUSION: AUC and AUWO appear to be the most reliable parameters for assessing tumor perfusion using DCE-US as they exhibited the lowest CV values.