CarveMix: A simple data augmentation method for brain lesion segmentation.

Brain lesion segmentation provides a valuable tool for clinical diagnosis and research, and convolutional neural networks (CNNs) have achieved unprecedented success in the segmentation task. Data augmentation is a widely used strategy to improve the training of CNNs. In particular, data augmentation approaches that mix pairs of annotated training images have been developed. These methods are easy to implement and have achieved promising results in various image processing tasks. However, existing data augmentation approaches based on image mixing are not designed for brain lesions and may not perform well for brain lesion segmentation. Thus, the design of this type of simple data augmentation method for brain lesion segmentation is still an open problem. In this work, we propose a simple yet effective data augmentation approach, dubbed as CarveMix, for CNN-based brain lesion segmentation. Like other mixing-based methods, CarveMix stochastically combines two existing annotated images (annotated for brain lesions only) to obtain new labeled samples. To make our method more suitable for brain lesion segmentation, CarveMix is lesion-aware, where the image combination is performed with a focus on the lesions and preserves the lesion information. Specifically, from one annotated image we carve a region of interest (ROI) according to the lesion location and geometry with a variable ROI size. The carved ROI then replaces the corresponding voxels in a second annotated image to synthesize new labeled images for network training, and additional harmonization steps are applied for heterogeneous data where the two annotated images can originate from different sources. Besides, we further propose to model the mass effect that is unique to whole brain tumor segmentation during image mixing. To evaluate the proposed method, experiments were performed on multiple publicly available or private datasets, and the results show that our method improves the accuracy of brain lesion segmentation. The code of the proposed method is available at https://github.com/ZhangxinruBIT/CarveMix.git.

Prediction of H3K27M Alteration Status in Brainstem Glioma Using Multi-Shell Diffusion MRI Metrics.

BACKGROUND: Multi-shell diffusion characteristics may help characterize brainstem gliomas (BSGs) and predict H3K27M status. PURPOSE: To identify the diffusion characteristics of BSG patients and investigate the predictive values of various diffusion metrics for H3K27M status in BSG. STUDY TYPE: Prospective. POPULATION: Eighty-four BSG patients (median age 10.5 years [IQR 6.8-30.0 years]) were included, of whom 56 were pediatric and 28 were adult patients. FIELD STRENGTH/SEQUENCE: 3 T, multi-shell diffusion imaging. ASSESSMENT: Diffusion kurtosis imaging and neurite orientation dispersion and density imaging analyses were performed. Age, gender, and diffusion metrics, including fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity, radial diffusivity (RD), mean kurtosis (MK), axial kurtosis (AK), radial kurtosis, intracellular volume fraction (ICVF), orientation dispersion index, and isotropic volume fraction (ISOVF), were compared between H3K27M-altered and wildtype BSG patients. STATISTICAL TESTS: Chi-square test, Mann-Whitney U test, multivariate analysis of variance (MANOVA), step-wise multivariable logistic regression. P-values <0.05 were considered significant. RESULTS: 82.4% pediatric and 57.1% adult patients carried H3K27M alteration. In the whole group, the H3K27M-altered BSGs demonstrated higher FA, AK and lower RD, ISOVF. The combination of age and median ISOVF showed fair performance for H3K27M prediction (AUC = 0.78). In the pediatric group, H3K27M-altered BSGs showed higher FA, AK, MK, ICVF and lower RD, MD, ISOVF. The combinations of median ISOVF, 5th percentile of FA, median MK and median MD showed excellent predictive power (AUC = 0.91). In the adult group, H3K27M-altered BSGs showed higher ICVF and lower RD, MD. The 75th percentile of RD demonstrated fair performance for H3K27M status prediction (AUC = 0.75). DATA CONCLUSION: Different alteration patterns of diffusion measures were identified between H3K27M-altered and wildtype BSGs, which collectively had fair to excellent predictive value for H3K27M alteration status, especially in pediatric patients. EVIDENCE LEVEL: 2 TECHNICAL EFFICACY: Stage 3.

Deep Learning for Noninvasive Assessment of H3 K27M Mutation Status in Diffuse Midline Gliomas Using MR Imaging.

BACKGROUND: Determination of H3 K27M mutation in diffuse midline glioma (DMG) is key for prognostic assessment and stratifying patient subgroups for clinical trials. MRI can noninvasively depict morphological and metabolic characteristics of H3 K27M mutant DMG. PURPOSE: This study aimed to develop a deep learning (DL) approach to noninvasively predict H3 K27M mutation in DMG using T2-weighted images. STUDY TYPE: Retrospective and prospective. POPULATION: For diffuse midline brain gliomas, 341 patients from Center-1 (27 ± 19 years, 184 males), 42 patients from Center-2 (33 ± 19 years, 27 males) and 35 patients (37 ± 18 years, 24 males). For diffuse spinal cord gliomas, 133 patients from Center-1 (30 ± 15 years, 80 males). FIELD STRENGTH/SEQUENCE: 5T and 3T, T2-weighted turbo spin echo imaging. ASSESSMENT: Conventional radiological features were independently reviewed by two neuroradiologists. H3 K27M status was determined by histopathological examination. The Dice coefficient was used to evaluate segmentation performance. Classification performance was evaluated using accuracy, sensitivity, specificity, and area under the curve. STATISTICAL TESTS: Pearson's Chi-squared test, Fisher's exact test, two-sample Student's t-test and Mann-Whitney U test. A two-sided P value <0.05 was considered statistically significant. RESULTS: In the testing cohort, Dice coefficients of tumor segmentation using DL were 0.87 for diffuse midline brain and 0.81 for spinal cord gliomas. In the internal prospective testing dataset, the predictive accuracies, sensitivities, and specificities of H3 K27M mutation status were 92.1%, 98.2%, 82.9% in diffuse midline brain gliomas and 85.4%, 88.9%, 82.6% in spinal cord gliomas. Furthermore, this study showed that the performance generalizes to external institutions, with predictive accuracies of 85.7%-90.5%, sensitivities of 90.9%-96.0%, and specificities of 82.4%-83.3%. DATA CONCLUSION: In this study, an automatic DL framework was developed and validated for accurately predicting H3 K27M mutation using T2-weighted images, which could contribute to the noninvasive determination of H3 K27M status for clinical decision-making. EVIDENCE LEVEL: 2 Technical Efficacy: Stage 2.

Volumetric segmentation of white matter tracts with label embedding.

Convolutional neural networks have achieved state-of-the-art performance for white matter (WM) tract segmentation based on diffusion magnetic resonance imaging (dMRI). However, the segmentation can still be difficult for challenging WM tracts with thin bodies or complicated shapes; the segmentation is even more problematic in challenging scenarios with reduced data quality or domain shift between training and test data, which can be easily encountered in clinical settings. In this work, we seek to improve the segmentation of WM tracts, especially for challenging WM tracts in challenging scenarios. In particular, our method is based on volumetric WM tract segmentation, where voxels are directly labeled without performing tractography. To improve the segmentation, we exploit the characteristics of WM tracts that different tracts can cross or overlap and revise the network design accordingly. Specifically, because multiple tracts can co-exist in a voxel, we hypothesize that the different tract labels can be correlated. The tract labels at a single voxel are concatenated as a label vector, the length of which is the number of tract labels. Due to the tract correlation, this label vector can be projected into a lower-dimensional space-referred to as the embedded space-for each voxel, which allows the segmentation network to solve a simpler problem. By predicting the coordinate in the embedded space for the tracts at each voxel and subsequently mapping the coordinate to the label vector with a reconstruction module, the segmentation result can be achieved. To facilitate the learning of the embedded space, an auxiliary label reconstruction loss is integrated with the segmentation accuracy loss during network training, and network training and inference are end-to-end. Our method was validated on two dMRI datasets under various settings. The results show that the proposed method improves the accuracy of WM tract segmentation, and the improvement is more prominent for challenging tracts in challenging scenarios.

Cerebellar Contributions to Motor and Cognitive Control in Multiple Sclerosis✰✰✰.

OBJECTIVE: To evaluate relationships between specific cerebellar regions and common clinical measures of motor and cognitive function in persons with multiple sclerosis (PwMS). DESIGN: Cross-sectional. SETTING: Laboratory. PARTICIPANTS: Twenty-nine PwMS and 28 age- and sex-matched controls without multiple sclerosis (MS) (N=57). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Both diffusion and lobule magnetic resonance imaging analyses and common clinical measures of motor and cognitive function were used to examine structure-function relationships in the cerebellum. RESULTS: PwMS demonstrate significantly worse motor and cognitive function than controls, including weaker strength, slower walking, and poorer performance on the Symbol Digit Modalities Test, but demonstrate no differences in cerebellar volume. However, PwMS demonstrate significantly worse diffusivity (mean diffusivity: P=.0003; axial diffusivity: P=.0015; radial diffusivity: P=.0005; fractional anisotropy: P=.016) of the superior cerebellar peduncle, the primary output of the cerebellum. Increased volume of the motor lobules (I-V, VIII) was significantly related to better motor (P<.022) and cognitive (P=.046) performance, and increased volume of the cognitive lobules (VI-VII) was also related to better motor (P<.032) and cognitive (P=.008) performance, supporting the role of the cerebellum in both motor and cognitive functioning. CONCLUSIONS: These data highlight the contributions of the cerebellum to both motor and cognitive function in PwMS. Using novel neuroimaging techniques to examine structure-function relationships in PwMS improves our understanding of individualized differences in this heterogeneous group and may provide an avenue for targeted, individualized rehabilitation aimed at improving cerebellar dysfunction in MS.

Automated cerebellar lobule segmentation with application to cerebellar structural analysis in cerebellar disease.

The cerebellum plays an important role in both motor control and cognitive function. Cerebellar function is topographically organized and diseases that affect specific parts of the cerebellum are associated with specific patterns of symptoms. Accordingly, delineation and quantification of cerebellar sub-regions from magnetic resonance images are important in the study of cerebellar atrophy and associated functional losses. This paper describes an automated cerebellar lobule segmentation method based on a graph cut segmentation framework. Results from multi-atlas labeling and tissue classification contribute to the region terms in the graph cut energy function and boundary classification contributes to the boundary term in the energy function. A cerebellar parcellation is achieved by minimizing the energy function using the α-expansion technique. The proposed method was evaluated using a leave-one-out cross-validation on 15 subjects including both healthy controls and patients with cerebellar diseases. Based on reported Dice coefficients, the proposed method outperforms two state-of-the-art methods. The proposed method was then applied to 77 subjects to study the region-specific cerebellar structural differences in three spinocerebellar ataxia (SCA) genetic subtypes. Quantitative analysis of the lobule volumes shows distinct patterns of volume changes associated with different SCA subtypes consistent with known patterns of atrophy in these genetic subtypes.

Reconstruction of the human cerebral cortex robust to white matter lesions: method and validation.

Cortical atrophy has been reported in a number of diseases, such as multiple sclerosis and Alzheimer's disease, that are also associated with white matter (WM) lesions. However, most cortical reconstruction techniques do not account for these pathologies, thereby requiring additional processing to correct for the effect of WM lesions. In this work, we introduce CRUISE(+), an automated process for cortical reconstruction from magnetic resonance brain images with WM lesions. The process extends previously well validated methods to allow for multichannel input images and to accommodate for the presence of WM lesions. We provide new validation data and tools for measuring the accuracy of cortical reconstruction methods on healthy brains as well as brains with multiple sclerosis lesions. Using this data, we validate the accuracy of CRUISE(+) and compare it to another state-of-the-art cortical reconstruction tool. Our results demonstrate that CRUISE(+) has superior performance in the cortical regions near WM lesions, and similar performance in other regions.

Multi-task Collaborative Pre-training and Adaptive Token Selection: A Unified Framework for Brain Representation Learning.

Structural magnetic resonance imaging (sMRI) reveals the structural organization of the brain. Learning general brain representations from sMRI is an enduring topic in neuroscience. Previous deep learning models neglect that the brain, as the core of cognition, is distinct from other organs whose primary attribute is anatomy. Capturing the high-level representation associated with inter-individual cognitive variability is key to appropriately represent the brain. Given that this cognition-related information is subtle, mixed, and distributed in the brain structure, sMRI-based models need to both capture fine-grained details and understand how they relate to the overall global structure. Additionally, it is also necessary to explicitly express the cognitive information that implicitly embedded in local-global image features. Therefore, we propose MCPATS, a brain representation learning framework that combines Multi-task Collaborative Pre-training (MCP) and Adaptive Token Selection (ATS). First, we develop MCP, including mask-reconstruction to understand global context, distort-restoration to capture fine-grained local details, adversarial learning to integrate features at different granularities, and age-prediction, using age as a surrogate for cognition to explicitly encode cognition-related information from local-global image features. This co-training allows progressive learning of implicit and explicit cognition-related representations. Then, we develop ATS based on mutual attention for downstream use of the learned representation. During fine-tuning, the ATS highlights discriminative features and reduces the impact of irrelevant information. MCPATS was validated on three different public datasets for brain disease diagnosis, outperforming competing methods and achieving accurate diagnosis. Further, we performed detailed analysis to confirm that the MCPATS-learned representation captures cognition-related information.

One-shot segmentation of novel white matter tracts via extensive data augmentation and adaptive knowledge transfer.

The use of convolutional neural networks (CNNs) has allowed accurate white matter (WM) tract segmentation on diffusion magnetic resonance imaging (dMRI). To train the CNN-based segmentation models, a large number of scans on which WM tracts are annotated need to be collected, and these annotated scans can be accumulated over a long period of time. However, when novel WM tracts that are different from existing annotated WM tracts are of interest, additional annotations are required for their segmentation. Due to the cost of manual annotations, methods have been developed for few-shot segmentation of novel WM tracts, where the segmentation knowledge is transferred from existing WM tracts to novel WM tracts and the amount of annotated data for novel WM tracts is reduced. Despite these developments, it is desirable to further reduce the amount of annotated data to the one-shot setting with a single annotated image. To address this problem, we develop an approach to one-shot segmentation of novel WM tracts. Our method follows the existing pretraining/fine-tuning framework that transfers segmentation knowledge from existing to novel WM tracts. First, as there is extremely scarce annotated data in the one-shot setting, we design several different data augmentation strategies so that extensive data augmentation can be performed to obtain extra synthetic training data. The data augmentation strategies are based on image masking and thus applicable to the one-shot setting. Second, to address overfitting and knowledge forgetting in the fine-tuning stage that can be more severe given limited training data, we propose an adaptive knowledge transfer strategy that selects the network weights to be updated. The data augmentation and adaptive knowledge transfer strategies are combined to train the segmentation model. Considering that the different data augmentation strategies can generate synthetic data that contain potentially conflicting information, we apply the data augmentation strategies separately, each leading to a different segmentation model. The results predicted by the different models are fused to produce the final segmentation. We validated our method on two brain dMRI datasets, including a public dataset and an in-house dataset. Different settings were considered for the validation, and the results show that the proposed method improves the one-shot segmentation of novel WM tracts.

A microstructure estimation Transformer inspired by sparse representation for diffusion MRI.

Diffusion magnetic resonance imaging (dMRI) is an important tool in characterizing tissue microstructure based on biophysical models, which are typically multi-compartmental models with mathematically complex and highly non-linear forms. Resolving microstructures from these models with conventional optimization techniques is prone to estimation errors and requires dense sampling in the q-space with a long scan time. Deep learning based approaches have been proposed to overcome these limitations. Motivated by the superior performance of the Transformer in feature extraction than the convolutional structure, in this work, we present a learning-based framework based on Transformer, namely, a Microstructure Estimation Transformer with Sparse Coding (METSC) for dMRI-based microstructural parameter estimation. To take advantage of the Transformer while addressing its limitation in large training data requirement, we explicitly introduce an inductive bias-model bias into the Transformer using a sparse coding technique to facilitate the training process. Thus, the METSC is composed with three stages, an embedding stage, a sparse representation stage, and a mapping stage. The embedding stage is a Transformer-based structure that encodes the signal in a high-level space to ensure the core voxel of a patch is represented effectively. In the sparse representation stage, a dictionary is constructed by solving a sparse reconstruction problem that unfolds the Iterative Hard Thresholding (IHT) process. The mapping stage is essentially a decoder that computes the microstructural parameters from the output of the second stage, based on the weighted sum of normalized dictionary coefficients where the weights are also learned. We tested our framework on two dMRI models with downsampled q-space data, including the intravoxel incoherent motion (IVIM) model and the neurite orientation dispersion and density imaging (NODDI) model. The proposed method achieved up to 11.25 folds of acceleration while retaining high fitting accuracy for NODDI fitting, reducing the mean squared error (MSE) up to 70% compared with the previous q-space learning approach. METSC outperformed the other state-of-the-art learning-based methods, including the model-free and model-based methods. The network also showed robustness against noise and generalizability across different datasets. The superior performance of METSC indicates its potential to improve dMRI acquisition and model fitting in clinical applications.

The combination of HT-ac and HBET improves the cognitive and learning abilities of heat-stressed mice by maintaining mitochondrial function through the PKA-CREB-BDNF pathway.

The aim was to investigate whether the combination of hydroxytyrosol acetate (HT-ac) and ethyl ß-hydroxybutyrate (HBET) can improve the cognition of heat-stressed mice, meanwhile exploring the mechanism of action. Mice were divided into 5 groups: control, heat-stressed, HT-ac, HBET, and HT-ac + HBET. Mice were gavaged for 21 days and exposed to heat (42.5 ± 0.5 °C, RH 60 ± 10%, 1 h day-1) on days 15-21, except for the control group. Results showed that the combination of HT-ac + HBET improved the cognitive and learning abilities of heat-stressed mice, which were tested by Morris water maze, shuttle box, and jumping stage tests. The combination of HT-ac + HBET maintained the integrity of neurons and mitochondria of heat-stressed mice. Likewise, this combination increased the mitochondrial membrane potential, the ATP content, the expression of phosphorylated PKA, BDNF, phosphorylated CREB and Bcl-2, and decreased the expression of Bax, caspase-3, and intracytoplasmic Cyt C in heat-stressed mice.

A transfer learning approach to few-shot segmentation of novel white matter tracts.

Convolutional neural networks (CNNs) have achieved state-of-the-art performance for white matter (WM) tract segmentation based on diffusion magnetic resonance imaging (dMRI). The training of the CNN-based segmentation model generally requires a large number of manual delineations of WM tracts, which can be expensive and time-consuming. Although it is possible to carefully curate abundant training data for a set of WM tracts of interest, there can also be novel WM tracts-i.e., WM tracts that are not included in the existing annotated WM tracts-that are specific to a new scientific problem, and it is desired that the novel WM tracts can be segmented without repeating the laborious collection of a large number of manual delineations for these tracts. One possible solution to the problem is to transfer the knowledge learned for segmenting existing WM tracts to the segmentation of novel WM tracts with a fine-tuning strategy, where a CNN pretrained for segmenting existing WM tracts is fine-tuned with only a few annotated scans to segment the novel WM tracts. However, in classic fine-tuning, the information in the last task-specific layer for segmenting existing WM tracts is completely discarded. In this work, based on the pretraining and fine-tuning framework, we propose an improved transfer learning approach to the segmentation of novel WM tracts in the few-shot setting, where all knowledge in the pretrained model is incorporated into the fine-tuning procedure. Specifically, from the weights of the pretrained task-specific layer for segmenting existing WM tracts, we derive a better initialization of the last task-specific layer for the target model that segments novel WM tracts. In addition, to allow further improvement of the initialization of the last layer and thus the segmentation performance in the few-shot setting, we develop a simple yet effective data augmentation strategy that generates synthetic annotated images with tract-aware image mixing. To validate the proposed method, we performed experiments on brain dMRI scans from public and private datasets under various experimental settings, and the results indicate that our method improves the performance of few-shot segmentation of novel WM tracts.

miR-182 mediated the inhibitory effects of NF-κB on the GPR39/CREB/BDNF pathway in the hippocampus of mice with depressive-like behaviors.

BACKGROUND: Chronic stress is one of the most important causes of depression, accompanied by neuroinflammation and hippocampal injuries. Long-term elevation of glucocorticoid leads to activation of NF-κB and inhibition of GPR39/CREB/BDNF pathway, which is pivotal for neuroprotection and neurogenesis. The present study thus was designed to determine the relationship between NF-κB and GPR39/CREB/BDNF pathway. METHODS: Depressive-like behaviors were induced by chronic unpredictable mild stress (CUMS) and chronic restraint stress (CRS) in mice. Corticosterone, inflammatory cytokines, and GPR39/CREB/BDNF pathway were determined by ELISA and Western Blot assays. The activation of NF-κB and inhibition of GPR39 were connected by bioinformatic analysis and experimentally validated in hippocampus cells by knock-in and knock-down techniques. RESULTS: CUMS and CRS led to an elevation of serum corticosterone and depressive-like behaviors in mice, with activation of NF-κB subunit p65 in the hippocampus and elevations of TNFα and IL-6. The expression of GPR39/CREB/BDNF pathway in the hippocampus was inhibited. Bioinformatic analysis revealed that four miRNAs, miR-96, miR-143, miR-150, and miR-182, were potentially transcribed by NF-κB and bound with GPR39 mRNA. NF-κB overexpression increased miR-182 expression and decreased GPR39 expression in hippocampus cells. Its inhibitor led to reverse effects. miR-182 mimics or inhibitors also regulated GPR39 expression in hippocampus cells and more importantly, blocked the regulation of NF-κB on GPR39. CONCLUSIONS: The results suggested that activation of NF-κB inhibited GPR39/CREB/BDNF pathway through increasing miR-182 in chronic stress-induced depressive-like behaviors. The negative-regulation features of miRNAs might be important for neuroinflammation-induced inhibition of neurofunction in depression.

Nutritional supplements formulated to prevent cognitive impairment in animals.

Heat stress will cause a series of response in the living system and the most significant impact is on brain functions. The aim of this article is to develop nutritional supplements that can alleviate cognitive decline caused by heat stress. In this article, we screen functional food factors which can prevent or relieve effects on heat stress injury based on bioinformatics. 129 function factors related to the crossover targets were obtained, and a food database related to the prevention of high-temperature impairment was constructed. After a series of scoring standards combined with food classification, two formulas-nutrition fortifier formula (tyrosine and multivitamin B) and plant compound formula (quercetin, proanthocyanidin, and naringin) were investigated using animal experiments to determine their ability to prevent cognitive impairment of heat-stressed animals. Our results demonstrated that certain functional food factors and our two designed formulations significantly prevent cognitive impairment of heat-stressed animals. Further mechanism was carried out by cell viability assay, reactive oxygen species assay, real-time quantitative PCR and Western blot. The results showed that the plant compound formula diluted 4000 times had the best relieving effect on HT22 after heat stress, and this concentration formula can significantly alleviate the elevated levels of reactive oxygen species caused by heat stress. This formula also can significantly down-regulate IL-1ß, IL-6, TNF-α, IL-10, iNOS and COX-2 expression. Likewise, Western blot results showed that the formula could activate the cAMP pathway and increase the expression of phosphorylated PKA and BDNF in hippocampal cells.

Automated Classification of Intramedullary Spinal Cord Tumors and Inflammatory Demyelinating Lesions Using Deep Learning.

Accurate differentiation of intramedullary spinal cord tumors and inflammatory demyelinating lesions and their subtypes are warranted because of their overlapping characteristics at MRI but with different treatments and prognosis. The authors aimed to develop a pipeline for spinal cord lesion segmentation and classification using two-dimensional MultiResUNet and DenseNet121 networks based on T2-weighted images. A retrospective cohort of 490 patients (118 patients with astrocytoma, 130 with ependymoma, 101 with multiple sclerosis [MS], and 141 with neuromyelitis optica spectrum disorders [NMOSD]) was used for model development, and a prospective cohort of 157 patients (34 patients with astrocytoma, 45 with ependymoma, 33 with MS, and 45 with NMOSD) was used for model testing. In the test cohort, the model achieved Dice scores of 0.77, 0.80, 0.50, and 0.58 for segmentation of astrocytoma, ependymoma, MS, and NMOSD, respectively, against manual labeling. Accuracies of 96% (area under the receiver operating characteristic curve [AUC], 0.99), 82% (AUC, 0.90), and 79% (AUC, 0.85) were achieved for the classifications of tumor versus demyelinating lesion, astrocytoma versus ependymoma, and MS versus NMOSD, respectively. In a subset of radiologically difficult cases, the classifier showed an accuracy of 79%-95% (AUC, 0.78-0.97). The established deep learning pipeline for segmentation and classification of spinal cord lesions can support an accurate radiologic diagnosis. Supplemental material is available for this article. © RSNA, 2022 Keywords: Spinal Cord MRI, Astrocytoma, Ependymoma, Multiple Sclerosis, Neuromyelitis Optica Spectrum Disorder, Deep Learning.

Direct segmentation of the major white matter tracts in diffusion tensor images.

Diffusion-weighted images of the human brain are acquired more and more routinely in clinical research settings, yet segmenting and labeling white matter tracts in these images is still challenging. We present in this paper a fully automated method to extract many anatomical tracts at once on diffusion tensor images, based on a Markov random field model and anatomical priors. The approach provides a direct voxel labeling, models explicitly fiber crossings and can handle white matter lesions. Experiments on simulations and repeatability studies show robustness to noise and reproducibility of the algorithm, which has been made publicly available.

Volumetric white matter tract segmentation with nested self-supervised learning using sequential pretext tasks.

White matter (WM) tract segmentation based on diffusion magnetic resonance imaging (dMRI) provides an important tool for the analysis of brain development, function, and disease. Deep learning based methods of WM tract segmentation have been proposed, which greatly improve the accuracy of the segmentation. However, the training of the deep networks usually requires a large number of manual delineations of WM tracts, which can be especially difficult to obtain and unavailable in many scenarios. Therefore, in this work, we explore how to perform deep learning based WM tract segmentation when annotated training data is scarce. To this end, we seek to exploit the abundant unannotated dMRI data in the self-supervised learning framework. From the unannotated data, knowledge about image context can be learned with pretext tasks that do not require manual annotations. Specifically, a deep network can be pretrained for the pretext task, and the knowledge learned from the pretext task is then transferred to the subsequent WM tract segmentation task with only a small number of annotated scans via fine-tuning. We explore two designs of pretext tasks that are related to WM tracts. The first pretext task predicts the density map of fiber streamlines, which are representations of generic WM pathways, and the training data can be obtained automatically with tractography. The second pretext task learns to mimic the results of registration-based WM tract segmentation, which, although inaccurate, is more relevant to WM tract segmentation and provides a good target for learning context knowledge. Then, we combine the two pretext tasks and develop a nested self-supervised learning strategy. In the nested self-supervised learning strategy, the first pretext task provides initial knowledge for the second pretext task, and the knowledge learned from the second pretext task with the initial knowledge is transferred to the target WM tract segmentation task via fine-tuning. To evaluate the proposed method, experiments were performed on brain dMRI scans from the Human Connectome Project dataset with various experimental settings. The results show that the proposed method improves the performance of WM tract segmentation when tract annotations are scarce.

Knowledge transfer between brain lesion segmentation tasks with increased model capacity.

Convolutional neural networks (CNNs) have become an increasingly popular tool for brain lesion segmentation in recent years due to its accuracy and efficiency. However, CNN-based brain lesion segmentation generally requires a large amount of annotated training data, which can be costly for medical imaging. In many scenarios, only a few annotations of brain lesions are available. One common strategy to address the issue of limited annotated data is to transfer knowledge from a different yet relevant source task, where training data is abundant, to the target task of interest. Typically, a model can be pretrained for the source task, and then fine-tuned with the scarce training data associated with the target task. However, classic fine-tuning tends to make small modifications to the pretrained model, which could hinder its adaptation to the target task. Fine-tuning with increased model capacity has been shown to alleviate this negative impact in image classification problems. In this work, we extend the strategy of fine-tuning with increased model capacity to the problem of brain lesion segmentation, and then develop an advanced version that is better suitable for segmentation problems. First, we propose a vanilla strategy of increasing the capacity, where, like in the classification problem, the width of the network is augmented during fine-tuning. Second, because unlike image classification, in segmentation problems each voxel is associated with a labeling result, we further develop a spatially adaptive augmentation strategy during fine-tuning. Specifically, in addition to the vanilla width augmentation, we incorporate a module that computes a spatial map of the contribution of the information given by width augmentation in the final segmentation. For demonstration, the proposed method was applied to ischemic stroke lesion segmentation, where a model pretrained for brain tumor segmentation was fine-tuned, and the experimental results indicate the benefit of our method.

Multimodal super-resolved q-space deep learning.

Super-resolvedq-space deep learning (SR-q-DL) has been developed to estimate high-resolution (HR) tissue microstructure maps from low-quality diffusion magnetic resonance imaging (dMRI) scans acquired with a reduced number of diffusion gradients and low spatial resolution, where deep networks are designed for the estimation. However, existing methods do not exploit HR information from other modalities, which are generally acquired together with dMRI and could provide additional useful information for HR tissue microstructure estimation. In this work, we extend SR-q-DL and propose multimodal SR-q-DL, where information in low-resolution (LR) dMRI is combined with HR information from another modality for HR tissue microstructure estimation. Because the HR modality may not be as sensitive to tissue microstructure as dMRI, direct concatenation of multimodal information does not necessarily lead to improved estimation performance. Since existing deep networks for HR tissue microstructure estimation are patch-based and use redundant information in the spatial domain to enhance the spatial resolution, the HR information in the other modality could inform the deep networks about what input voxels are relevant for the computation of tissue microstructure. Thus, we propose to incorporate the HR information from the HR modality by designing an attention module that guides the computation of HR tissue microstructure from LR dMRI. Specifically, the attention module is integrated with the patch-based SR-q-DL framework that exploits the sparsity of diffusion signals. The sparse representation of the LR diffusion signals in the input patch is first computed with a network component that unrolls an iterative process for sparse reconstruction. Then, the proposed attention module computes a relevance map from the HR modality with sequential convolutional layers. The relevance map indicates the relevance of the LR sparse representation at each voxel for computing the patch of HR tissue microstructure. The relevance is applied to the LR sparse representation with voxelwise multiplication, and the weighted LR sparse representation is used to compute HR tissue microstructure with another network component that allows resolution enhancement. All weights in the proposed network for multimodal SR-q-DL are jointly learned and the estimation is end-to-end. To evaluate the proposed method, we performed experiments on brain dMRI scans together with images of additional HR modalities. In the experiments, the proposed method was applied to the estimation of tissue microstructure measures for different datasets and advanced biophysical models, where the benefit of incorporating multimodal information using the proposed method is shown.