Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets.

Every year, millions of brain MRI scans are acquired in hospitals, which is a figure considerably larger than the size of any research dataset. Therefore, the ability to analyze such scans could transform neuroimaging research. Yet, their potential remains untapped since no automated algorithm is robust enough to cope with the high variability in clinical acquisitions (MR contrasts, resolutions, orientations, artifacts, and subject populations). Here, we present SynthSeg+, an AI segmentation suite that enables robust analysis of heterogeneous clinical datasets. In addition to whole-brain segmentation, SynthSeg+ also performs cortical parcellation, intracranial volume estimation, and automated detection of faulty segmentations (mainly caused by scans of very low quality). We demonstrate SynthSeg+ in seven experiments, including an aging study on 14,000 scans, where it accurately replicates atrophy patterns observed on data of much higher quality. SynthSeg+ is publicly released as a ready-to-use tool to unlock the potential of quantitative morphometry.

Quantitative Brain Morphometry of Portable Low-Field-Strength MRI Using Super-Resolution Machine Learning.

Background Portable, low-field-strength (0.064-T) MRI has the potential to transform neuroimaging but is limited by low spatial resolution and low signal-to-noise ratio. Purpose To implement a machine learning super-resolution algorithm that synthesizes higher spatial resolution images (1-mm isotropic) from lower resolution T1-weighted and T2-weighted portable brain MRI scans, making them amenable to automated quantitative morphometry. Materials and Methods An external high-field-strength MRI data set (1-mm isotropic scans from the Open Access Series of Imaging Studies data set) and segmentations for 39 regions of interest (ROIs) in the brain were used to train a super-resolution convolutional neural network (CNN). Secondary analysis of an internal test set of 24 paired low- and high-field-strength clinical MRI scans in participants with neurologic symptoms was performed. These were part of a prospective observational study (August 2020 to December 2021) at Massachusetts General Hospital (exclusion criteria: inability to lay flat, body habitus preventing low-field-strength MRI, presence of MRI contraindications). Three well-established automated segmentation tools were applied to three sets of scans: high-field-strength (1.5-3 T, reference standard), low-field-strength (0.064 T), and synthetic high-field-strength images generated from the low-field-strength data with the CNN. Statistical significance of correlations was assessed with Student t tests. Correlation coefficients were compared with Steiger Z tests. Results Eleven participants (mean age, 50 years ± 14; seven men) had full cerebrum coverage in the images without motion artifacts or large stroke lesion with distortion from mass effect. Direct segmentation of low-field-strength MRI yielded nonsignificant correlations with volumetric measurements from high field strength for most ROIs (P > .05). Correlations largely improved when segmenting the synthetic images: P values were less than .05 for all ROIs (eg, for the hippocampus [r = 0.85; P < .001], thalamus [r = 0.84; P = .001], and whole cerebrum [r = 0.92; P < .001]). Deviations from the model (z score maps) visually correlated with pathologic abnormalities. Conclusion This work demonstrated proof-of-principle augmentation of portable MRI with a machine learning super-resolution algorithm, which yielded highly correlated brain morphometric measurements to real higher resolution images. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Ertl-Wagner amd Wagner in this issue. An earlier incorrect version appeared online. This article was corrected on February 1, 2023.

Brain Growth Charts for Quantitative Analysis of Pediatric Clinical Brain MRI Scans with Limited Imaging Pathology.

Background Clinically acquired brain MRI scans represent a valuable but underused resource for investigating neurodevelopment due to their technical heterogeneity and lack of appropriate controls. These barriers have curtailed retrospective studies of clinical brain MRI scans compared with more costly prospectively acquired research-quality brain MRI scans. Purpose To provide a benchmark for neuroanatomic variability in clinically acquired brain MRI scans with limited imaging pathology (SLIPs) and to evaluate if growth charts from curated clinical MRI scans differed from research-quality MRI scans or were influenced by clinical indication for the scan. Materials and Methods In this secondary analysis of preexisting data, clinical brain MRI SLIPs from an urban pediatric health care system (individuals aged ≤22 years) were scanned across nine 3.0-T MRI scanners. The curation process included manual review of signed radiology reports and automated and manual quality review of images without gross pathology. Global and regional volumetric imaging phenotypes were measured using two image segmentation pipelines, and clinical brain growth charts were quantitatively compared with charts derived from a large set of research controls in the same age range by means of Pearson correlation and age at peak volume. Results The curated clinical data set included 532 patients (277 male; median age, 10 years [IQR, 5-14 years]; age range, 28 days after birth to 22 years) scanned between 2005 and 2020. Clinical brain growth charts were highly correlated with growth charts derived from research data sets (22 studies, 8346 individuals [4947 male]; age range, 152 days after birth to 22 years) in terms of normative developmental trajectories predicted by the models (median r = 0.979). Conclusion The clinical indication of the scans did not significantly bias the output of clinical brain charts. Brain growth charts derived from clinical controls with limited imaging pathology were highly correlated with brain charts from research controls, suggesting the potential of curated clinical MRI scans to supplement research data sets. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Ertl-Wagner and Pai in this issue.

A deep learning toolbox for automatic segmentation of subcortical limbic structures from MRI images.

A tool was developed to automatically segment several subcortical limbic structures (nucleus accumbens, basal forebrain, septal nuclei, hypothalamus without mammillary bodies, the mammillary bodies, and fornix) using only a T1-weighted MRI as input. This tool fills an unmet need as there are few, if any, publicly available tools to segment these clinically relevant structures. A U-Net with spatial, intensity, contrast, and noise augmentation was trained using 39 manually labeled MRI data sets. In general, the Dice scores, true positive rates, false discovery rates, and manual-automatic volume correlation were very good relative to comparable tools for other structures. A diverse data set of 698 subjects were segmented using the tool; evaluation of the resulting labelings showed that the tool failed in less than 1% of cases. Test-retest reliability of the tool was excellent. The automatically segmented volume of all structures except mammillary bodies showed effectiveness at detecting either clinical AD effects, age effects, or both. This tool will be publicly released with FreeSurfer (surfer.nmr.mgh.harvard.edu/fswiki/ScLimbic). Together with the other cortical and subcortical limbic segmentations, this tool will allow FreeSurfer to provide a comprehensive view of the limbic system in an automated way.

Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast.

Most existing algorithms for automatic 3D morphometry of human brain MRI scans are designed for data with near-isotropic voxels at approximately 1 mm resolution, and frequently have contrast constraints as well-typically requiring T1-weighted images (e.g., MP-RAGE scans). This limitation prevents the analysis of millions of MRI scans acquired with large inter-slice spacing in clinical settings every year. In turn, the inability to quantitatively analyze these scans hinders the adoption of quantitative neuro imaging in healthcare, and also precludes research studies that could attain huge sample sizes and hence greatly improve our understanding of the human brain. Recent advances in convolutional neural networks (CNNs) are producing outstanding results in super-resolution and contrast synthesis of MRI. However, these approaches are very sensitive to the specific combination of contrast, resolution and orientation of the input images, and thus do not generalize to diverse clinical acquisition protocols - even within sites. In this article, we present SynthSR, a method to train a CNN that receives one or more scans with spaced slices, acquired with different contrast, resolution and orientation, and produces an isotropic scan of canonical contrast (typically a 1 mm MP-RAGE). The presented method does not require any preprocessing, beyond rigid coregistration of the input scans. Crucially, SynthSR trains on synthetic input images generated from 3D segmentations, and can thus be used to train CNNs for any combination of contrasts, resolutions and orientations without high-resolution real images of the input contrasts. We test the images generated with SynthSR in an array of common downstream analyses, and show that they can be reliably used for subcortical segmentation and volumetry, image registration (e.g., for tensor-based morphometry), and, if some image quality requirements are met, even cortical thickness morphometry. The source code is publicly available at https://github.com/BBillot/SynthSR.

Automated segmentation of the hypothalamus and associated subunits in brain MRI.

Despite the crucial role of the hypothalamus in the regulation of the human body, neuroimaging studies of this structure and its nuclei are scarce. Such scarcity partially stems from the lack of automated segmentation tools, since manual delineation suffers from scalability and reproducibility issues. Due to the small size of the hypothalamus and the lack of image contrast in its vicinity, automated segmentation is difficult and has been long neglected by widespread neuroimaging packages like FreeSurfer or FSL. Nonetheless, recent advances in deep machine learning are enabling us to tackle difficult segmentation problems with high accuracy. In this paper we present a fully automated tool based on a deep convolutional neural network, for the segmentation of the whole hypothalamus and its subregions from T1-weighted MRI scans. We use aggressive data augmentation in order to make the model robust to T1-weighted MR scans from a wide array of different sources, without any need for preprocessing. We rigorously assess the performance of the presented tool through extensive analyses, including: inter- and intra-rater variability experiments between human observers; comparison of our tool with manual segmentation; comparison with an automated method based on multi-atlas segmentation; assessment of robustness by quality control analysis of a larger, heterogeneous dataset (ADNI); and indirect evaluation with a volumetric study performed on ADNI. The presented model outperforms multi-atlas segmentation scores as well as inter-rater accuracy level, and approaches intra-rater precision. Our method does not require any preprocessing and runs in less than a second on a GPU, and approximately 10 seconds on a CPU. The source code as well as the trained model are publicly available at https://github.com/BBillot/hypothalamus_seg, and will also be distributed with FreeSurfer.

SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI.

Rigid motion tracking is paramount in many medical imaging applications where movements need to be detected, corrected, or accounted for. Modern strategies rely on convolutional neural networks (CNN) and pose this problem as rigid registration. Yet, CNNs do not exploit natural symmetries in this task, as they are equivariant to translations (their outputs shift with their inputs) but not to rotations. Here we propose EquiTrack, the first method that uses recent steerable SE(3)-equivariant CNNs (E-CNN) for motion tracking. While steerable E-CNNs can extract corresponding features across different poses, testing them on noisy medical images reveals that they do not have enough learning capacity to learn noise invariance. Thus, we introduce a hybrid architecture that pairs a denoiser with an E-CNN to decouple the processing of anatomically irrelevant intensity features from the extraction of equivariant spatial features. Rigid transforms are then estimated in closed-form. EquiTrack outperforms state-of-the-art learning and optimisation methods for motion tracking in adult brain MRI and fetal MRI time series. Our code is available at https://github.com/BBillot/EquiTrack.

Linking brain structure, cognition, and sleep: insights from clinical data.

STUDY OBJECTIVES: To use relatively noisy routinely collected clinical data (brain magnetic resonance imaging (MRI) data, clinical polysomnography (PSG) recordings, and neuropsychological testing), to investigate hypothesis-driven and data-driven relationships between brain physiology, structure, and cognition. METHODS: We analyzed data from patients with clinical PSG, brain MRI, and neuropsychological evaluations. SynthSeg, a neural network-based tool, provided high-quality segmentations despite noise. A priori hypotheses explored associations between brain function (measured by PSG) and brain structure (measured by MRI). Associations with cognitive scores and dementia status were studied. An exploratory data-driven approach investigated age-structure-physiology-cognition links. RESULTS: Six hundred and twenty-three patients with sleep PSG and brain MRI data were included in this study; 160 with cognitive evaluations. Three hundred and forty-two participants (55%) were female, and age interquartile range was 52 to 69 years. Thirty-six individuals were diagnosed with dementia, 71 with mild cognitive impairment, and 326 with major depression. One hundred and fifteen individuals were evaluated for insomnia and 138 participants had an apnea-hypopnea index equal to or greater than 15. Total PSG delta power correlated positively with frontal lobe/thalamic volumes, and sleep spindle density with thalamic volume. rapid eye movement (REM) duration and amygdala volume were positively associated with cognition. Patients with dementia showed significant differences in five brain structure volumes. REM duration, spindle, and slow-oscillation features had strong associations with cognition and brain structure volumes. PSG and MRI features in combination predicted chronological age (R2 = 0.67) and cognition (R2 = 0.40). CONCLUSIONS: Routine clinical data holds extended value in understanding and even clinically using brain-sleep-cognition relationships.

A next-generation, histological atlas of the human brain and its application to automated brain MRI segmentation.

Magnetic resonance imaging (MRI) is the standard tool to image the human brain in vivo. In this domain, digital brain atlases are essential for subject-specific segmentation of anatomical regions of interest (ROIs) and spatial comparison of neuroanatomy from different subjects in a common coordinate frame. High-resolution, digital atlases derived from histology (e.g., Allen atlas [7], BigBrain [13], Julich [15]), are currently the state of the art and provide exquisite 3D cytoarchitectural maps, but lack probabilistic labels throughout the whole brain. Here we present NextBrain, a next-generation probabilistic atlas of human brain anatomy built from serial 3D histology and corresponding highly granular delineations of five whole brain hemispheres. We developed AI techniques to align and reconstruct ~10,000 histological sections into coherent 3D volumes with joint geometric constraints (no overlap or gaps between sections), as well as to semi-automatically trace the boundaries of 333 distinct anatomical ROIs on all these sections. Comprehensive delineation on multiple cases enabled us to build the first probabilistic histological atlas of the whole human brain. Further, we created a companion Bayesian tool for automated segmentation of the 333 ROIs in any in vivo or ex vivo brain MRI scan using the NextBrain atlas. We showcase two applications of the atlas: automated segmentation of ultra-high-resolution ex vivo MRI and volumetric analysis of Alzheimer's disease and healthy brain ageing based on ~4,000 publicly available in vivo MRI scans. We publicly release: the raw and aligned data (including an online visualisation tool); the probabilistic atlas; the segmentation tool; and ground truth delineations for a 100 µm isotropic ex vivo hemisphere (that we use for quantitative evaluation of our segmentation method in this paper). By enabling researchers worldwide to analyse brain MRI scans at a superior level of granularity without manual effort or highly specific neuroanatomical knowledge, NextBrain holds promise to increase the specificity of MRI findings and ultimately accelerate our quest to understand the human brain in health and disease.

Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology.

We present open-source tools for 3D analysis of photographs of dissected slices of human brains, which are routinely acquired in brain banks but seldom used for quantitative analysis. Our tools can: (i) 3D reconstruct a volume from the photographs and, optionally, a surface scan; and (ii) produce a high-resolution 3D segmentation into 11 brain regions per hemisphere (22 in total), independently of the slice thickness. Our tools can be used as a substitute for ex vivo magnetic resonance imaging (MRI), which requires access to an MRI scanner, ex vivo scanning expertise, and considerable financial resources. We tested our tools on synthetic and real data from two NIH Alzheimer's Disease Research Centers. The results show that our methodology yields accurate 3D reconstructions, segmentations, and volumetric measurements that are highly correlated to those from MRI. Our method also detects expected differences between post mortem confirmed Alzheimer's disease cases and controls. The tools are available in our widespread neuroimaging suite "FreeSurfer" ( https://surfer.nmr.mgh.harvard.edu/fswiki/PhotoTools ).

Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology.

We present open-source tools for three-dimensional (3D) analysis of photographs of dissected slices of human brains, which are routinely acquired in brain banks but seldom used for quantitative analysis. Our tools can: (1) 3D reconstruct a volume from the photographs and, optionally, a surface scan; and (2) produce a high-resolution 3D segmentation into 11 brain regions per hemisphere (22 in total), independently of the slice thickness. Our tools can be used as a substitute for ex vivo magnetic resonance imaging (MRI), which requires access to an MRI scanner, ex vivo scanning expertise, and considerable financial resources. We tested our tools on synthetic and real data from two NIH Alzheimer's Disease Research Centers. The results show that our methodology yields accurate 3D reconstructions, segmentations, and volumetric measurements that are highly correlated to those from MRI. Our method also detects expected differences between post mortem confirmed Alzheimer's disease cases and controls. The tools are available in our widespread neuroimaging suite 'FreeSurfer' (https://surfer.nmr.mgh.harvard.edu/fswiki/PhotoTools).

Every year, thousands of human brains are donated to science. These brains are used to study normal aging, as well as neurological diseases like Alzheimer's or Parkinson's. Donated brains usually go to 'brain banks', institutions where the brains are dissected to extract tissues relevant to different diseases. During this process, it is routine to take photographs of brain slices for archiving purposes. Often, studies of dead brains rely on qualitative observations, such as 'the hippocampus displays some atrophy', rather than concrete 'numerical' measurements. This is because the gold standard to take three-dimensional measurements of the brain is magnetic resonance imaging (MRI), which is an expensive technique that requires high expertise ­ especially with dead brains. The lack of quantitative data means it is not always straightforward to study certain conditions. To bridge this gap, Gazula et al. have developed an openly available software that can build three-dimensional reconstructions of dead brains based on photographs of brain slices. The software can also use machine learning methods to automatically extract different brain regions from the three-dimensional reconstructions and measure their size. These data can be used to take precise quantitative measurements that can be used to better describe how different conditions lead to changes in the brain, such as atrophy (reduced volume of one or more brain regions). The researchers assessed the accuracy of the method in two ways. First, they digitally sliced MRI-scanned brains and used the software to compute the sizes of different structures based on these synthetic data, comparing the results to the known sizes. Second, they used brains for which both MRI data and dissection photographs existed and compared the measurements taken by the software to the measurements obtained with MRI images. Gazula et al. show that, as long as the photographs satisfy some basic conditions, they can provide good estimates of the sizes of many brain structures. The tools developed by Gazula et al. are publicly available as part of FreeSurfer, a widespread neuroimaging software that can be used by any researcher working at a brain bank. This will allow brain banks to obtain accurate measurements of dead brains, allowing them to cheaply perform quantitative studies of brain structures, which could lead to new findings relating to neurodegenerative diseases.

SynthSR: A public AI tool to turn heterogeneous clinical brain scans into high-resolution T1-weighted images for 3D morphometry.

Every year, millions of brain magnetic resonance imaging (MRI) scans are acquired in hospitals across the world. These have the potential to revolutionize our understanding of many neurological diseases, but their morphometric analysis has not yet been possible due to their anisotropic resolution. We present an artificial intelligence technique, "SynthSR," that takes clinical brain MRI scans with any MR contrast (T1, T2, etc.), orientation (axial/coronal/sagittal), and resolution and turns them into high-resolution T1 scans that are usable by virtually all existing human neuroimaging tools. We present results on segmentation, registration, and atlasing of >10,000 scans of controls and patients with brain tumors, strokes, and Alzheimer's disease. SynthSR yields morphometric results that are very highly correlated with what one would have obtained with high-resolution T1 scans. SynthSR allows sample sizes that have the potential to overcome the power limitations of prospective research studies and shed new light on the healthy and diseased human brain.

SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining.

Despite advances in data augmentation and transfer learning, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain scans, CNNs are highly sensitive to changes in resolution and contrast: even within the same MRI modality, performance can decrease across datasets. Here we introduce SynthSeg, the first segmentation CNN robust against changes in contrast and resolution. SynthSeg is trained with synthetic data sampled from a generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the contrast and resolution of the synthetic training data. Consequently, SynthSeg can segment real scans from a wide range of target domains without retraining or fine-tuning, which enables straightforward analysis of huge amounts of heterogeneous clinical data. Because SynthSeg only requires segmentations to be trained (no images), it can learn from labels obtained by automated methods on diverse populations (e.g., ageing and diseased), thus achieving robustness to a wide range of morphological variability. We demonstrate SynthSeg on 5,000 scans of six modalities (including CT) and ten resolutions, where it exhibits unparallelled generalisation compared with supervised CNNs, state-of-the-art domain adaptation, and Bayesian segmentation. Finally, we demonstrate the generalisability of SynthSeg by applying it to cardiac MRI and CT scans.

Domain-agnostic segmentation of thalamic nuclei from joint structural and diffusion MRI.

The human thalamus is a highly connected subcortical grey-matter structure within the brain. It comprises dozens of nuclei with different function and connectivity, which are affected differently by disease. For this reason, there is growing interest in studying the thalamic nuclei in vivo with MRI. Tools are available to segment the thalamus from 1 mm T1 scans, but the contrast of the lateral and internal boundaries is too faint to produce reliable segmentations. Some tools have attempted to incorporate information from diffusion MRI in the segmentation to refine these boundaries, but do not generalise well across diffusion MRI acquisitions. Here we present the first CNN that can segment thalamic nuclei from T1 and diffusion data of any resolution without retraining or fine tuning. Our method builds on a public histological atlas of the thalamic nuclei and silver standard segmentations on high-quality diffusion data obtained with a recent Bayesian adaptive segmentation tool. We combine these with an approximate degradation model for fast domain randomisation during training. Our CNN produces a segmentation at 0.7 mm isotropic resolution, irrespective of the resolution of the input. Moreover, it uses a parsimonious model of the diffusion signal at each voxel (fractional anisotropy and principal eigenvector) that is compatible with virtually any set of directions and b-values, including huge amounts of legacy data. We show results of our proposed method on three heterogeneous datasets acquired on dozens of different scanners. An implementation of the method is publicly available at https://freesurfer.net/fswiki/ThalamicNucleiDTI.

SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images.

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to contrast introduced by magnetic resonance imaging (MRI). While classical registration methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning-based techniques are fast at test time but limited to registering images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency on training data by leveraging a generative strategy for diverse synthetic label maps and images that exposes networks to a wide range of variability, forcing them to learn more invariant features. This approach results in powerful networks that accurately generalize to a broad array of MRI contrasts. We present extensive experiments with a focus on 3D neuroimaging, showing that this strategy enables robust and accurate registration of arbitrary MRI contrasts even if the target contrast is not seen by the networks during training. We demonstrate registration accuracy surpassing the state of the art both within and across contrasts, using a single model. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images. Our code is available at doic https://w3id.org/synthmorph.

In vivo hypothalamic regional volumetry across the frontotemporal dementia spectrum.

BACKGROUND: Frontotemporal dementia (FTD) is a spectrum of diseases characterised by language, behavioural and motor symptoms. Among the different subcortical regions implicated in the FTD symptomatology, the hypothalamus regulates various bodily functions, including eating behaviours which are commonly present across the FTD spectrum. The pattern of specific hypothalamic involvement across the clinical, pathological, and genetic forms of FTD has yet to be fully investigated, and its possible associations with abnormal eating behaviours have yet to be fully explored. METHODS: Using an automated segmentation tool for volumetric T1-weighted MR images, we measured hypothalamic regional volumes in a cohort of 439 patients with FTD (197 behavioural variant FTD [bvFTD]; 7 FTD with associated motor neurone disease [FTD-MND]; 99 semantic variant primary progressive aphasia [svPPA]; 117 non-fluent variant PPA [nfvPPA]; 19 PPA not otherwise specified [PPA-NOS]) and 118 age-matched controls. We compared volumes across the clinical, genetic (29 MAPT, 32 C9orf72, 23 GRN), and pathological diagnoses (61 tauopathy, 40 TDP-43opathy, 4 FUSopathy). We correlated the volumes with presence of abnormal eating behaviours assessed with the revised version of the Cambridge Behavioural Inventory (CBI-R). RESULTS: On average, FTD patients showed 14% smaller hypothalamic volumes than controls. The groups with the smallest hypothalamic regions were FTD-MND (20%), MAPT (25%) and FUS (33%), with differences mainly localised in the anterior and posterior regions. The inferior tuberal region was only significantly smaller in tauopathies (MAPT and Pick's disease) and in TDP-43 type C compared to controls and was the only regions that did not correlate with eating symptoms. PPA-NOS and nfvPPA were the groups with the least frequent eating behaviours and the least hypothalamic involvement. CONCLUSIONS: Abnormal hypothalamic volumes are present in all the FTD forms, but different hypothalamic regions might play a different role in the development of abnormal eating behavioural and metabolic symptoms. These findings might therefore help in the identification of different underlying pathological mechanisms, suggesting the potential use of hypothalamic imaging biomarkers and the research of potential therapeutic targets within the hypothalamic neuropeptides.

JOINT SEGMENTATION OF MULTIPLE SCLEROSIS LESIONS AND BRAIN ANATOMY IN MRI SCANS OF ANY CONTRAST AND RESOLUTION WITH CNNs.

We present the first deep learning method to segment Multiple Sclerosis lesions and brain structures from MRI scans of any (possibly multimodal) contrast and resolution. Our method only requires segmentations to be trained (no images), as it leverages the generative model of Bayesian segmentation to generate synthetic scans with simulated lesions, which are then used to train a CNN. Our method can be retrained to segment at any resolution by adjusting the amount of synthesised partial volume. By construction, the synthetic scans are perfectly aligned with their labels, which enables training with noisy labels obtained with automatic methods. The training data are generated on the fly, and aggressive augmentation (including artefacts) is applied for improved generalisation. We demonstrate our method on two public datasets, comparing it with a state-of-the-art Bayesian approach implemented in FreeSurfer, and dataset specific CNNs trained on real data. The code is available at https://github.com/BBillot/SynthSeg.

LEARNING MRI CONTRAST-AGNOSTIC REGISTRATION.

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to magnetic resonance imaging (MRI) contrast. While classical methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning methods are fast at test time but limited to images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency using a generative strategy that exposes networks to a wide range of images synthesized from segmentations during training, forcing them to generalize across contrasts. We show that networks trained within this framework generalize to a broad array of unseen MRI contrasts and surpass classical state-of-the-art brain registration accuracy by up to 12.4 Dice points for a variety of tested contrast combinations. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images during training.