Generating multi-pathological and multi-modal images and labels for brain MRI.

The last few years have seen a boom in using generative models to augment real datasets, as synthetic data can effectively model real data distributions and provide privacy-preserving, shareable datasets that can be used to train deep learning models. However, most of these methods are 2D and provide synthetic datasets that come, at most, with categorical annotations. The generation of paired images and segmentation samples that can be used in downstream, supervised segmentation tasks remains fairly uncharted territory. This work proposes a two-stage generative model capable of producing 2D and 3D semantic label maps and corresponding multi-modal images. We use a latent diffusion model for label synthesis and a VAE-GAN for semantic image synthesis. Synthetic datasets provided by this model are shown to work in a wide variety of segmentation tasks, supporting small, real datasets or fully replacing them while maintaining good performance. We also demonstrate its ability to improve downstream performance on out-of-distribution data.

Latent Transformer Models for out-of-distribution detection.

Any clinically-deployed image-processing pipeline must be robust to the full range of inputs it may be presented with. One popular approach to this challenge is to develop predictive models that can provide a measure of their uncertainty. Another approach is to use generative modelling to quantify the likelihood of inputs. Inputs with a low enough likelihood are deemed to be out-of-distribution and are not presented to the downstream predictive model. In this work, we evaluate several approaches to segmentation with uncertainty for the task of segmenting bleeds in 3D CT of the head. We show that these models can fail catastrophically when operating in the far out-of-distribution domain, often providing predictions that are both highly confident and wrong. We propose to instead perform out-of-distribution detection using the Latent Transformer Model: a VQ-GAN is used to provide a highly compressed latent representation of the input volume, and a transformer is then used to estimate the likelihood of this compressed representation of the input. We demonstrate this approach can identify images that are both far- and near- out-of-distribution, as well as provide spatial maps that highlight the regions considered to be out-of-distribution. Furthermore, we find a strong relationship between an image's likelihood and the quality of a model's segmentation on it, demonstrating that this approach is viable for filtering out unsuitable images.

Generalizability of 3D CNN models for age estimation in diverse youth populations using structural MRI.

Recently, several studies have investigated the neurodevelopment of psychiatric disorders using brain data acquired via structural magnetic resonance imaging (sMRI). These analyses have shown the potential of sMRI data to provide a relatively precise characterization of brain structural biomarkers. Despite these advances, a relatively unexplored question is how reliable and consistent a model is when assessing subjects from other independent datasets. In this study, we investigate the performance and generalizability of the same model architecture trained from distinct datasets comprising youths in diverse stages of neurodevelopment and with different mental health conditions. We employed models with the same 3D convolutional neural network (CNN) architecture to assess autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), brain age, and a measure of dimensional psychopathology, the Child Behavior Checklist (CBCL) total score. The investigated datasets include the Autism Brain Imaging Data Exchange II (ABIDE-II, N = 580), Attention Deficit Hyperactivity Disorder (ADHD-200, N = 922), Brazilian High-Risk Cohort Study (BHRCS, N = 737), and Adolescent Brain Cognitive Development (ABCD, N = 11,031). Models' performance and interpretability were assessed within each dataset (for diagnosis tasks) and inter-datasets (for age estimation). Despite the demographic and phenotypic differences of the subjects, all models presented significant estimations for age (p value < 0.001) within and between datasets. In addition, most models showed a moderate to high correlation in age estimation. The results, including the models' brain regions of interest (ROI), were analyzed and discussed in light of the youth neurodevelopmental structural changes. Among other interesting discoveries, we found that less confounded training datasets produce models with higher generalization capacity.

Geometry-invariant abnormality detection.

Cancer is a highly heterogeneous condition best visualised in positron emission tomography. Due to this heterogeneity, a general-purpose cancer detection model can be built using unsupervised learning anomaly detection models. While prior work in this field has showcased the efficacy of abnormality detection methods (e.g. Transformer-based), these have shown significant vulnerabilities to differences in data geometry. Changes in image resolution or observed field of view can result in inaccurate predictions, even with significant data pre-processing and augmentation. We propose a new spatial conditioning mechanism that enables models to adapt and learn from varying data geometries, and apply it to a state-of-the-art Vector-Quantized Variational Autoencoder + Transformer abnormality detection model. We showcase that this spatial conditioning mechanism statistically-significantly improves model performance on whole-body data compared to the same model without conditioning, while allowing the model to perform inference at varying data geometries.

Sleep-Energy: An Energy Optimization Method to Sleep Stage Scoring.

Sleep is essential for physical and mental health. Polysomnography (PSG) procedures are labour-intensive and time-consuming, making diagnosing sleep disorders difficult. Automatic sleep staging using Machine Learning (ML) - based methods has been studied extensively, but frequently provides noisier predictions incompatible with typical manually annotated hypnograms. We propose an energy optimization method to improve the quality of hypnograms generated by automatic sleep staging procedures. The method evaluates the system's total energy based on conditional probabilities for each epoch's stage and employs an energy minimisation procedure. It can be used as a meta-optimisation layer over the sleep stage sequences generated by any classifier that generates prediction probabilities. The method improved the accuracy of state-of-the-art Deep Learning models in the Sleep EDFx dataset by 4.0% and in the DRM-SUB dataset by 2.8%.

Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection.

Cancers can have highly heterogeneous uptake patterns best visualised in positron emission tomography. These patterns are essential to detect, diagnose, stage and predict the evolution of cancer. Due to this heterogeneity, a general-purpose cancer detection model can be built using unsupervised learning anomaly detection models; these models learn a healthy representation of tissue and detect cancer by predicting deviations from healthy appearances. This task alone requires models capable of accurately learning long-range interactions between organs, imaging patterns, and other abstract features with high levels of expressivity. Such characteristics are suitably satisfied by transformers, and have been shown to generate state-of-the-art results in unsupervised anomaly detection by training on healthy data. This work expands upon such approaches by introducing multi-modal conditioning of the transformer via cross-attention, i.e. supplying anatomical reference information from paired CT images to aid the PET anomaly detection task. Using 83 whole-body PET/CT samples containing various cancer types, we show that our anomaly detection method is robust and capable of achieving accurate cancer localisation results even in cases where healthy training data is unavailable. Furthermore, the proposed model uncertainty, in conjunction with a kernel density estimation approach, is shown to provide a statistically robust alternative to residual-based anomaly maps. Overall, a superior performance is demonstrated against leading alternatives, drawing attention to the potential of these approaches.

Estimating Gender and Age from Brain Structural MRI of Children and Adolescents: A 3D Convolutional Neural Network Multitask Learning Model.

Despite recent advances, assessing biological measurements for neuropsychiatric disorders is still a challenge, where confounding variables such as gender and age (as a proxy for neurodevelopment) play an important role. This study explores brain structural magnetic resonance imaging (sMRI) from two public data sets (ABIDE-II and ADHD-200) with healthy control (HC, N = 894), autism spectrum disorder (ASD, N = 251), and attention deficit hyperactivity disorder (ADHD, N = 357) individuals. We used gray and white matter preprocessed via voxel-based morphometry (VBM) to train a 3D convolutional neural network with a multitask learning strategy to estimate gender, age, and mental health status from structural brain differences. Gradient-based methods were employed to generate attention maps, providing clinically relevant identification of most representative brain regions for models' decision-making. This approach resulted in satisfactory predictions for gender and age. ADHD-200-trained models, evaluated in 10-fold cross-validation procedures on test set, obtained a mean absolute error (MAE) of 1.43 years (±0.22 SD) for age prediction and an area under the curve (AUC) of 0.85 (±0.04 SD) for gender classification. In out-of-sample validation, the best-performing ADHD-200 models satisfactorily predicted age (MAE = 1.57 years) and gender (AUC = 0.89) in the ABIDE-II data set. The models' accuracy was in line with the current state-of-the-art machine learning applications in neuroimaging. Key regions for models' accuracy were presented as a meaningful graphical output. New implementations, such as the use of VBM along with a 3D convolutional neural network multitask learning model and a brain imaging graphical output, reinforce the relevance of the proposed workflow.

Papez Circuit Gray Matter and Episodic Memory in Amyotrophic Lateral Sclerosis and Behavioural Variant Frontotemporal Dementia.

Amyotrophic lateral sclerosis and behavioural variant frontotemporal dementia are two different diseases recognized to overlap at clinical, pathological and genetic characteristics. Both conditions are traditionally known for relative sparing of episodic memory. However, recent studies have disputed that with the report of patients presenting with marked episodic memory impairment. Besides that, structural and functional changes in temporal lobe regions responsible for episodic memory processing are often detected in neuroimaging studies of both conditions. In this study, we investigated the gray matter features associated with the Papez circuit in amyotrophic lateral sclerosis, behavioural variant frontotemporal dementia and healthy controls to further explore similarities and differences between the two conditions. Our non-demented amyotrophic lateral sclerosis patients showed no episodic memory deficits measured by a short-term delayed recall test while no changes in gray matter of the Papez circuit were found. Compared with the amyotrophic lateral sclerosis group, the behavioural variant frontotemporal dementia group had lower performance on the short-term delayed recall test and marked atrophy in gray matter of the Papez circuit. Bilateral atrophy of entorhinal cortex and mammillary bodies distinguished behavioural variant frontotemporal dementia from amyotrophic lateral sclerosis patients as well as atrophy in left cingulate, left hippocampus and right parahippocampal gyrus. Taken together, our results suggest that sub-regions of the Papez circuit could be differently affected in amyotrophic lateral sclerosis and behavioural variant frontotemporal dementia.

Regional Dynamics of the Resting Brain in Amyotrophic Lateral Sclerosis Using Fractional Amplitude of Low-Frequency Fluctuations and Regional Homogeneity Analyses.

Resting-state functional magnetic resonance imaging has been playing an important role in the study of amyotrophic lateral sclerosis (ALS). Although functional connectivity is widely studied, the patterns of spontaneous neural activity of the resting brain are important mechanisms that have been used recently to study a variety of conditions but remain less explored in ALS. Here we have used fractional amplitude of low-frequency fluctuation (fALFF) and regional homogeneity (ReHo) to study the regional dynamics of the resting brain of nondemented ALS patients compared with healthy controls. As expected, we found the sensorimotor network with changes in fALFF and ReHo, and also found the default mode network (DMN), frontoparietal network (FPN), and salience network (SN) altered and the cerebellum, although no structural changes between ALS patients and controls were reported in the regions with fALFF and ReHo changes. We show an altered pattern in the spontaneous low-frequency oscillations that is not confined to the motor areas and reveal a more widespread involvement of nonmotor regions, including those responsible for cognition.

Investigating brain structural patterns in first episode psychosis and schizophrenia using MRI and a machine learning approach.

In this study, we employed the Maximum Uncertainty Linear Discriminant Analysis (MLDA) to investigate whether the structural brain patterns in first episode psychosis (FEP) patients would be more similar to patients with chronic schizophrenia (SCZ) or healthy controls (HC), from a schizophrenia model perspective. Brain regions volumetric data were estimated by using MRI images of SCZ and FEP patients and HC. First, we evaluated the MLDA performance in discriminating SCZ from controls, which provided a score based on a model for changes in brain structure in SCZ. In the following, we compared the volumetric patterns of FEP patients with patterns of SCZ and healthy controls using these scores. The FEP group had a score distribution more similar to patients with schizophrenia (p-value = .461; Cohen's d=-.15) in comparison with healthy subjects (p-value=.003; Cohen's d = .62). Structures related to the limbic system and the circuitry involved in goal-directed behaviours were the most discriminant regions. There is a distinct pattern of volumetric changes in patients with schizophrenia in contrast to healthy controls, and this pattern seem to be detectable already in FEP.