Multifaceted atlases of the human brain in its infancy.

Brain atlases are spatial references for integrating, processing, and analyzing brain features gathered from different individuals, sources, and scales. Here we introduce a collection of joint surface-volume atlases that chart postnatal development of the human brain in a spatiotemporally dense manner from two weeks to two years of age. Our month-specific atlases chart normative patterns and capture key traits of early brain development and are therefore conducive to identifying aberrations from normal developmental trajectories. These atlases will enhance our understanding of early structural and functional development by facilitating the mapping of diverse features of the infant brain to a common reference frame for precise multifaceted quantification of cortical and subcortical changes.

Dynamic neural circuit disruptions associated with antisocial behaviors.

Antisocial behavior (ASB) is believed to have neural substrates; however, the association between ASB and functional brain networks remains unclear. The temporal variability of the functional connectivity (or dynamic FC) derived from resting-state functional MRI has been suggested as a useful metric for studying abnormal behaviors including ASB. This is the first study using low-frequency fluctuations of the dynamic FC to unravel potential system-level neural correlates with ASB. Specifically, we individually associated the dynamic FC patterns with the ASB scores (measured by Antisocial Process Screening Device) of the male offenders (age: 23.29 ± 3.36 years) based on machine learning. Results showed that the dynamic FCs were associated with individual ASB scores. Moreover, we found that it was mainly the inter-network dynamic FCs that were negatively associated with the ASB severity. Three major high-order cognitive functional networks and the sensorimotor network were found to be more associated with ASB. We further found that impaired behavior in the ASB subjects was mainly associated with decreased FC dynamics in these networks, which may explain why ASB subjects usually have impaired executive control and emotional processing functions. Our study shows that temporal variation of the FC could be a promising tool for ASB assessment, treatment, and prevention.

Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis.

In this article, the authors aim to maximally utilize multimodality neuroimaging and genetic data for identifying Alzheimer's disease (AD) and its prodromal status, Mild Cognitive Impairment (MCI), from normal aging subjects. Multimodality neuroimaging data such as MRI and PET provide valuable insights into brain abnormalities, while genetic data such as single nucleotide polymorphism (SNP) provide information about a patient's AD risk factors. When these data are used together, the accuracy of AD diagnosis may be improved. However, these data are heterogeneous (e.g., with different data distributions), and have different number of samples (e.g., with far less number of PET samples than the number of MRI or SNPs). Thus, learning an effective model using these data is challenging. To this end, we present a novel three-stage deep feature learning and fusion framework, where deep neural network is trained stage-wise. Each stage of the network learns feature representations for different combinations of modalities, via effective training using the maximum number of available samples. Specifically, in the first stage, we learn latent representations (i.e., high-level features) for each modality independently, so that the heterogeneity among modalities can be partially addressed, and high-level features from different modalities can be combined in the next stage. In the second stage, we learn joint latent features for each pair of modality combination by using the high-level features learned from the first stage. In the third stage, we learn the diagnostic labels by fusing the learned joint latent features from the second stage. To further increase the number of samples during training, we also use data at multiple scanning time points for each training subject in the dataset. We evaluate the proposed framework using Alzheimer's disease neuroimaging initiative (ADNI) dataset for AD diagnosis, and the experimental results show that the proposed framework outperforms other state-of-the-art methods.

Identification of infants at high-risk for autism spectrum disorder using multiparameter multiscale white matter connectivity networks.

Autism spectrum disorder (ASD) is a wide range of disabilities that cause life-long cognitive impairment and social, communication, and behavioral challenges. Early diagnosis and medical intervention are important for improving the life quality of autistic patients. However, in the current practice, diagnosis often has to be delayed until the behavioral symptoms become evident during childhood. In this study, we demonstrate the feasibility of using machine learning techniques for identifying high-risk ASD infants at as early as six months after birth. This is based on the observation that ASD-induced abnormalities in white matter (WM) tracts and whole-brain connectivity have already started to appear within 24 months after birth. In particular, we propose a novel multikernel support vector machine classification framework by using the connectivity features gathered from WM connectivity networks, which are generated via multiscale regions of interest (ROIs) and multiple diffusion statistics such as fractional anisotropy, mean diffusivity, and average fiber length. Our proposed framework achieves an accuracy of 76% and an area of 0.80 under the receiver operating characteristic curve (AUC), in comparison to the accuracy of 70% and the AUC of 70% provided by the best single-parameter single-scale network. The improvement in accuracy is mainly due to the complementary information provided by multiparameter multiscale networks. In addition, our framework also provides the potential imaging connectomic markers and an objective means for early ASD diagnosis.

Neurodegenerative disease diagnosis using incomplete multi-modality data via matrix shrinkage and completion.

In this work, we are interested in predicting the diagnostic statuses of potentially neurodegenerated patients using feature values derived from multi-modality neuroimaging data and biological data, which might be incomplete. Collecting the feature values into a matrix, with each row containing a feature vector of a sample, we propose a framework to predict the corresponding associated multiple target outputs (e.g., diagnosis label and clinical scores) from this feature matrix by performing matrix shrinkage following matrix completion. Specifically, we first combine the feature and target output matrices into a large matrix and then partition this large incomplete matrix into smaller submatrices, each consisting of samples with complete feature values (corresponding to a certain combination of modalities) and target outputs. Treating each target output as the outcome of a prediction task, we apply a 2-step multi-task learning algorithm to select the most discriminative features and samples in each submatrix. Features and samples that are not selected in any of the submatrices are discarded, resulting in a shrunk version of the original large matrix. The missing feature values and unknown target outputs of the shrunk matrix is then completed simultaneously. Experimental results using the ADNI dataset indicate that our proposed framework achieves higher classification accuracy at a greater speed when compared with conventional imputation-based classification methods and also yields competitive performance when compared with the state-of-the-art methods.

Functional Hierarchy of the Human Neocortex from Cradle to Grave.

Recent evidence indicates that the organization of the human neocortex is underpinned by smooth spatial gradients of functional connectivity (FC). These gradients provide crucial insight into the relationship between the brain's topographic organization and the texture of human cognition. However, no studies to date have charted how intrinsic FC gradient architecture develops across the entire human lifespan. In this work, we model developmental trajectories of the three primary gradients of FC using a large, high-quality, and temporally-dense functional MRI dataset spanning from birth to 100 years of age. The gradient axes, denoted as sensorimotor-association (SA), visual-somatosensory (VS), and modulation-representation (MR), encode crucial hierarchical organizing principles of the brain in development and aging. By tracking their evolution throughout the human lifespan, we provide the first ever comprehensive low-dimensional normative reference of global FC hierarchical architecture. We observe significant age-related changes in global network features, with global markers of hierarchical organization increasing from birth to early adulthood and decreasing thereafter. During infancy and early childhood, FC organization is shaped by primary sensory processing, dense short-range connectivity, and immature association and control hierarchies. Functional differentiation of transmodal systems supported by long-range coupling drives a convergence toward adult-like FC organization during late childhood, while adolescence and early adulthood are marked by the expansion and refinement of SA and MR hierarchies. While gradient topographies remain stable during late adulthood and aging, we observe decreases in global gradient measures of FC differentiation and complexity from 30 to 100 years. Examining cortical microstructure gradients alongside our functional gradients, we observed that structure-function gradient coupling undergoes differential lifespan trajectories across multiple gradient axes.

Constructing Multi-View High-Order Functional Connectivity Networks for Diagnosis of Autism Spectrum Disorder.

Brain functional connectivity network (FCN) based on resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used to identify neuropsychiatric disorders such as autism spectrum disorder (ASD). Most existing FCN-based methods only estimate the correlation between brain regions of interest (ROIs), without exploring more informative higher-level interactions among multiple ROIs which could be beneficial to disease diagnosis. To fully explore the discriminative information provided by different brain networks, a cluster-based multi-view high-order FCN (Ho-FCN) framework is proposed in this paper. Specifically, we first group the functional connectivity (FC) time series into different clusters and compute the multi-order central moment series for the FC time series in each cluster. Then we utilize the correlation of central moment series between different clusters to reveal the high-order FC relationships among multiple ROIs. In addition, to address the phase mismatch issue in conventional FCNs, we also adopt the central moments of the correlation time series as the temporal-invariance features to capture the dynamic characteristics of low-order dynamic FCN (Lo-D-FCN). Experimentalresults on the ABIDE dataset validate that: 1) the proposed multi-view Ho-FCNs is able to explore rich discriminative information for ASD diagnosis; 2) the phase mismatch issue can be well circumvented by using central moments; and 3) the combination of different types of FCNs can significantly improve the diagnostic accuracy of ASD (86.2%).

Localization of Craniomaxillofacial Landmarks on CBCT Images Using 3D Mask R-CNN and Local Dependency Learning.

Cephalometric analysis relies on accurate detection of craniomaxillofacial (CMF) landmarks from cone-beam computed tomography (CBCT) images. However, due to the complexity of CMF bony structures, it is difficult to localize landmarks efficiently and accurately. In this paper, we propose a deep learning framework to tackle this challenge by jointly digitalizing 105 CMF landmarks on CBCT images. By explicitly learning the local geometrical relationships between the landmarks, our approach extends Mask R-CNN for end-to-end prediction of landmark locations. Specifically, we first apply a detection network on a down-sampled 3D image to leverage global contextual information to predict the approximate locations of the landmarks. We subsequently leverage local information provided by higher-resolution image patches to refine the landmark locations. On patients with varying non-syndromic jaw deformities, our method achieves an average detection accuracy of 1.38± 0.95mm, outperforming a related state-of-the-art method.

Learning-Based Computer-Aided Prescription Model for Parkinson's Disease: A Data-Driven Perspective.

In this article, we study a novel problem: "automatic prescription recommendation for PD patients." To realize this goal, we first build a dataset by collecting 1) symptoms of PD patients, and 2) their prescription drug provided by neurologists. Then, we build a novel computer-aided prescription model by learning the relation between observed symptoms and prescription drug. Finally, for the new coming patients, we could recommend (predict) suitable prescription drug on their observed symptoms by our prescription model. From the methodology part, our proposed model, namely Prescription viA Learning lAtent Symptoms (PALAS), could recommend prescription using the multi-modality representation of the data. In PALAS, a latent symptom space is learned to better model the relationship between symptoms and prescription drug, as there is a large semantic gap between them. Moreover, we present an efficient alternating optimization method for PALAS. We evaluated our method using the data collected from 136 PD patients at Nanjing Brain Hospital, which can be regarded as a large dataset in PD research community. The experimental results demonstrate the effectiveness and clinical potential of our method in this recommendation task, if compared with other competing methods.

Estimating Reference Shape Model for Personalized Surgical Reconstruction of Craniomaxillofacial Defects.

OBJECTIVE: To estimate a patient-specific reference bone shape model for a patient with craniomaxillofacial (CMF) defects due to facial trauma. METHODS: We proposed an automatic facial bone shape estimation framework using pre-traumatic conventional portrait photos and post-traumatic head computed tomography (CT) scans via a 3D face reconstruction and a deformable shape model. Specifically, a three-dimensional (3D) face was first reconstructed from the patient's pre-traumatic portrait photos. Second, a correlation model between the skin and bone surfaces was constructed using a sparse representation based on the CT images of training normal subjects. Third, by feeding the reconstructed 3D face into the correlation model, an initial reference shape model was generated. In addition, we refined the initial estimation by applying non-rigid surface matching between the initially estimated shape and the patient's post-traumatic bone based on the adaptive-focus deformable shape model (AFDSM). Furthermore, a statistical shape model, built from the training normal subjects, was utilized to constrain the deformation process to avoid overfitting. RESULTS AND CONCLUSION: The proposed method was evaluated using both synthetic and real patient data. Experimental results show that the patient's abnormal facial bony structure can be recovered using our method, and the estimated reference shape model is considered clinically acceptable by an experienced CMF surgeon. SIGNIFICANCE: The proposed method is more suitable to the complex CMF defects for CMF reconstructive surgical planning.

Hierarchical Nonlocal Residual Networks for Image Quality Assessment of Pediatric Diffusion MRI With Limited and Noisy Annotations.

Fast and automated image quality assessment (IQA) of diffusion MR images is crucial for making timely decisions for rescans. However, learning a model for this task is challenging as the number of annotated data is limited and the annotation labels might not always be correct. As a remedy, we will introduce in this paper an automatic image quality assessment (IQA) method based on hierarchical non-local residual networks for pediatric diffusion MR images. Our IQA is performed in three sequential stages, i.e., 1) slice-wise IQA, where a nonlocal residual network is first pre-trained to annotate each slice with an initial quality rating (i.e., pass/questionable/fail), which is subsequently refined via iterative semi-supervised learning and slice self-training; 2) volume-wise IQA, which agglomerates the features extracted from the slices of a volume, and uses a nonlocal network to annotate the quality rating for each volume via iterative volume self-training; and 3) subject-wise IQA, which ensembles the volumetric IQA results to determine the overall image quality pertaining to a subject. Experimental results demonstrate that our method, trained using only samples of modest size, exhibits great generalizability, and is capable of conducting rapid hierarchical IQA with near-perfect accuracy.

Multi-View Spatial Aggregation Framework for Joint Localization and Segmentation of Organs at Risk in Head and Neck CT Images.

Accurate segmentation of organs at risk (OARs) from head and neck (H&N) CT images is crucial for effective H&N cancer radiotherapy. However, the existing deep learning methods are often not trained in an end-to-end fashion, i.e., they independently predetermine the regions of target organs before organ segmentation, causing limited information sharing between related tasks and thus leading to suboptimal segmentation results. Furthermore, when conventional segmentation network is used to segment all the OARs simultaneously, the results often favor big OARs over small OARs. Thus, the existing methods often train a specific model for each OAR, ignoring the correlation between different segmentation tasks. To address these issues, we propose a new multi-view spatial aggregation framework for joint localization and segmentation of multiple OARs using H&N CT images. The core of our framework is a proposed region-of-interest (ROI)-based fine-grained representation convolutional neural network (CNN), which is used to generate multi-OAR probability maps from each 2D view (i.e., axial, coronal, and sagittal view) of CT images. Specifically, our ROI-based fine-grained representation CNN (1) unifies the OARs localization and segmentation tasks and trains them in an end-to-end fashion, and (2) improves the segmentation results of various-sized OARs via a novel ROI-based fine-grained representation. Our multi-view spatial aggregation framework then spatially aggregates and assembles the generated multi-view multi-OAR probability maps to segment all the OARs simultaneously. We evaluate our framework using two sets of H&N CT images and achieve competitive and highly robust segmentation performance for OARs of various sizes.

Real-Time Quality Assessment of Pediatric MRI via Semi-Supervised Deep Nonlocal Residual Neural Networks.

In this paper, we introduce an image quality assessment (IQA) method for pediatric T1- and T2-weighted MR images. IQA is first performed slice-wise using a nonlocal residual neural network (NR-Net) and then volume-wise by agglomerating the slice QA results using random forest. Our method requires only a small amount of quality-annotated images for training and is designed to be robust to annotation noise that might occur due to rater errors and the inevitable mix of good and bad slices in an image volume. Using a small set of quality-assessed images, we pre-train NR-Net to annotate each image slice with an initial quality rating (i.e., pass, questionable, fail), which we then refine by semi-supervised learning and iterative self-training. Experimental results demonstrate that our method, trained using only samples of modest size, exhibit great generalizability, capable of real-time (milliseconds per volume) large-scale IQA with nearperfect accuracy.

Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data.

Fusing multi-modality data is crucial for accurate identification of brain disorder as different modalities can provide complementary perspectives of complex neurodegenerative disease. However, there are at least four common issues associated with the existing fusion methods. First, many existing fusion methods simply concatenate features from each modality without considering the correlations among different modalities. Second, most existing methods often make prediction based on a single classifier, which might not be able to address the heterogeneity of the Alzheimer's disease (AD) progression. Third, many existing methods often employ feature selection (or reduction) and classifier training in two independent steps, without considering the fact that the two pipelined steps are highly related to each other. Forth, there are missing neuroimaging data for some of the participants (e.g., missing PET data), due to the participants' "no-show" or dropout. In this paper, to address the above issues, we propose an early AD diagnosis framework via novel multi-modality latent space inducing ensemble SVM classifier. Specifically, we first project the neuroimaging data from different modalities into a latent space, and then map the learned latent representations into the label space to learn multiple diversified classifiers. Finally, we obtain the more reliable classification results by using an ensemble strategy. More importantly, we present a Complete Multi-modality Latent Space (CMLS) learning model for complete multi-modality data and also an Incomplete Multi-modality Latent Space (IMLS) learning model for incomplete multi-modality data. Extensive experiments using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset have demonstrated that our proposed models outperform other state-of-the-art methods.

Characterizing Intra-soma Diffusion with Spherical Mean Spectrum Imaging.

Most brain microstructure models are dedicated to the quantification of white matter microstructure, using for example sticks, cylinders, and zeppelins to model intra- and extra-axonal environments. Gray matter presents unique micro-architecture with cell bodies (somas) exhibiting diffusion characteristics that differ from axons in white matter. In this paper, we introduce a method to quantify soma microstructure, giving measures such as volume fraction, diffusivity, and kurtosis. Our method captures a spectrum of diffusion patterns and scales and does not rely on restrictive model assumptions. We show that our method yields unique and meaningful contrasts that are in agreement with histological data. We demonstrate its application in the mapping of the distinct spatial patterns of soma density in the cortex.

Fast Correction of Eddy-Current and Susceptibility-Induced Distortions Using Rotation-Invariant Contrasts.

Diffusion MRI (dMRI) is typically time consuming as it involves acquiring a series of 3D volumes, each associated with a wave-vector in q-space that determines the diffusion direction and strength. The acquisition time is further increased when "blip-up blip-down" scans are acquired with opposite phase encoding directions (PEDs) to facilitate distortion correction. In this work, we show that geometric distortions can be corrected without acquiring with opposite PEDs for each wave-vector, and hence the acquisition time can be halved. Our method uses complimentary rotation-invariant contrasts across shells of different diffusion weightings. Distortion-free structural T1-/T2-weighted MRI is used as reference for nonlinear registration in correcting the distortions. Signal dropout and pileup are corrected with the help of spherical harmonics. To demonstrate that our method is robust to changes in image appearance, we show that distortion correction with good structural alignment can be achieved within minutes for dMRI data of infants between 1 to 24 months of age.

Probing Tissue Microarchitecture of the Baby Brain via Spherical Mean Spectrum Imaging.

During the first years of life, the human brain undergoes dynamic spatially-heterogeneous changes, invo- lving differentiation of neuronal types, dendritic arbori- zation, axonal ingrowth, outgrowth and retraction, synaptogenesis, and myelination. To better quantify these changes, this article presents a method for probing tissue microarchitecture by characterizing water diffusion in a spectrum of length scales, factoring out the effects of intra-voxel orientation heterogeneity. Our method is based on the spherical means of the diffusion signal, computed over gradient directions for a set of diffusion weightings (i.e., b -values). We decompose the spherical mean profile at each voxel into a spherical mean spectrum (SMS), which essentially encodes the fractions of spin packets undergoing fine- to coarse-scale diffusion proce- sses, characterizing restricted and hindered diffusion stemming respectively from intra- and extra-cellular water compartments. From the SMS, multiple orientation distribution invariant indices can be computed, allowing for example the quantification of neurite density, microscopic fractional anisotropy ( µ FA), per-axon axial/radial diffusivity, and free/restricted isotropic diffusivity. We show that these indices can be computed for the developing brain for greater sensitivity and specificity to development related changes in tissue microstructure. Also, we demonstrate that our method, called spherical mean spectrum imaging (SMSI), is fast, accurate, and can overcome the biases associated with other state-of-the-art microstructure models.

One-Shot Generative Adversarial Learning for MRI Segmentation of Craniomaxillofacial Bony Structures.

Compared to computed tomography (CT), magnetic resonance imaging (MRI) delineation of craniomaxillofacial (CMF) bony structures can avoid harmful radiation exposure. However, bony boundaries are blurry in MRI, and structural information needs to be borrowed from CT during the training. This is challenging since paired MRI-CT data are typically scarce. In this paper, we propose to make full use of unpaired data, which are typically abundant, along with a single paired MRI-CT data to construct a one-shot generative adversarial model for automated MRI segmentation of CMF bony structures. Our model consists of a cross-modality image synthesis sub-network, which learns the mapping between CT and MRI, and an MRI segmentation sub-network. These two sub-networks are trained jointly in an end-to-end manner. Moreover, in the training phase, a neighbor-based anchoring method is proposed to reduce the ambiguity problem inherent in cross-modality synthesis, and a feature-matching-based semantic consistency constraint is proposed to encourage segmentation-oriented MRI synthesis. Experimental results demonstrate the superiority of our method both qualitatively and quantitatively in comparison with the state-of-the-art MRI segmentation methods.

Brain-Wide Genome-Wide Association Study for Alzheimer's Disease via Joint Projection Learning and Sparse Regression Model.

Brain-wide and genome-wide association (BW-GWA) study is presented in this paper to identify the associations between the brain imaging phenotypes (i.e., regional volumetric measures) and the genetic variants [i.e., single nucleotide polymorphism (SNP)] in Alzheimer's disease (AD). The main challenges of this study include the data heterogeneity, complex phenotype-genotype associations, high-dimensional data (e.g., thousands of SNPs), and the existence of phenotype outliers. Previous BW-GWA studies, while addressing some of these challenges, did not consider the diagnostic label information in their formulations, thus limiting their clinical applicability. To address these issues, we present a novel joint projection and sparse regression model to discover the associations between the phenotypes and genotypes. Specifically, to alleviate the negative influence of data heterogeneity, we first map the genotypes into an intermediate imaging-phenotype-like space. Then, to better reveal the complex phenotype-genotype associations, we project both the mapped genotypes and the original imaging phenotypes into a diagnostic-label-guided joint feature space, where the intraclass projected points are constrained to be close to each other. In addition, we use l2,1-norm minimization on both the regression loss function and the transformation coefficient matrices, to reduce the effect of phenotype outliers and also to encourage sparse feature selections of both the genotypes and phenotypes. We evaluate our method using AD neuroimaging initiative dataset, and the results show that our proposed method outperforms several state-of-the-art methods in term of the average root-mean-square error of genome-to-phenotype predictions. Besides, the associated SNPs and brain regions identified in this study have also been shown in the previous AD-related studies, thus verifying the effectiveness and potential of our proposed method in AD pathogenesis study.

Latent Representation Learning for Alzheimer's Disease Diagnosis With Incomplete Multi-Modality Neuroimaging and Genetic Data.

The fusion of complementary information contained in multi-modality data [e.g., magnetic resonance imaging (MRI), positron emission tomography (PET), and genetic data] has advanced the progress of automated Alzheimer's disease (AD) diagnosis. However, multi-modality based AD diagnostic models are often hindered by the missing data, i.e., not all the subjects have complete multi-modality data. One simple solution used by many previous studies is to discard samples with missing modalities. However, this significantly reduces the number of training samples, thus leading to a sub-optimal classification model. Furthermore, when building the classification model, most existing methods simply concatenate features from different modalities into a single feature vector without considering their underlying associations. As features from different modalities are often closely related (e.g., MRI and PET features are extracted from the same brain region), utilizing their inter-modality associations may improve the robustness of the diagnostic model. To this end, we propose a novel latent representation learning method for multi-modality based AD diagnosis. Specifically, we use all the available samples (including samples with incomplete modality data) to learn a latent representation space. Within this space, we not only use samples with complete multi-modality data to learn a common latent representation, but also use samples with incomplete multi-modality data to learn independent modality-specific latent representations. We then project the latent representations to the label space for AD diagnosis. We perform experiments using 737 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, and the experimental results verify the effectiveness of our proposed method.