MISPEL: A supervised deep learning harmonization method for multi-scanner neuroimaging data.

Large-scale data obtained from aggregation of already collected multi-site neuroimaging datasets has brought benefits such as higher statistical power, reliability, and robustness to the studies. Despite these promises from growth in sample size, substantial technical variability stemming from differences in scanner specifications exists in the aggregated data and could inadvertently bias any downstream analyses on it. Such a challenge calls for data normalization and/or harmonization frameworks, in addition to comprehensive criteria to estimate the scanner-related variability and evaluate the harmonization frameworks. In this study, we propose MISPEL (Multi-scanner Image harmonization via Structure Preserving Embedding Learning), a supervised multi-scanner harmonization method that is naturally extendable to more than two scanners. We also designed a set of criteria to investigate the scanner-related technical variability and evaluate the harmonization techniques. As an essential requirement of our criteria, we introduced a multi-scanner matched dataset of 3T T1 images across four scanners, which, to the best of our knowledge is one of the few datasets of this kind. We also investigated our evaluations using two popular segmentation frameworks: FSL and segmentation in statistical parametric mapping (SPM). Lastly, we compared MISPEL to popular methods of normalization and harmonization, namely White Stripe, RAVEL, and CALAMITI. MISPEL outperformed these methods and is promising for many other neuroimaging modalities.

Anatomic development of the upper airway during the first five years of life: A three-dimensional imaging study.

PURPOSE: Normative data on the growth and development of the upper airway across the sexes is needed for the diagnosis and treatment of congenital and acquired respiratory anomalies and to gain insight on developmental changes in speech acoustics and disorders with craniofacial anomalies. METHODS: The growth of the upper airway in children ages birth to 5 years, as compared to adults, was quantified using an imaging database with computed tomography studies from typically developing individuals. Methodological criteria for scan inclusion and airway measurements included: head position, histogram-based airway segmentation, anatomic landmark placement, and development of a semi-automatic centerline for data extraction. A comprehensive set of 2D and 3D supra- and sub-glottal measurements from the choanae to tracheal opening were obtained including: naso-oro-laryngo-pharynx subregion volume and length, each subregion's superior and inferior cross-sectional-area, and antero-posterior and transverse/width distances. RESULTS: Growth of the upper airway during the first 5 years of life was more pronounced in the vertical and transverse/lateral dimensions than in the antero-posterior dimension. By age 5 years, females have larger pharyngeal measurement than males. Prepubertal sex-differences were identified in the subglottal region. CONCLUSIONS: Our findings demonstrate the importance of studying the growth of the upper airway in 3D. As the lumen length increases, its shape changes, becoming increasingly elliptical during the first 5 years of life. This study also emphasizes the importance of methodological considerations for both image acquisition and data extraction, as well as the use of consistent anatomic structures in defining pharyngeal regions.

MULTI-DOMAIN LEARNING BY META-LEARNING: TAKING OPTIMAL STEPS IN MULTI-DOMAIN LOSS LANDSCAPES BY INNER-LOOP LEARNING.

We consider a model-agnostic solution to the problem of Multi-Domain Learning (MDL) for multi-modal applications. Many existing MDL techniques are model-dependent solutions which explicitly require nontrivial architectural changes to construct domain-specific modules. Thus, properly applying these MDL techniques for new problems with well-established models, e.g. U-Net for semantic segmentation, may demand various low-level implementation efforts. In this paper, given emerging multi-modal data (e.g., various structural neuroimaging modalities), we aim to enable MDL purely algorithmically so that widely used neural networks can trivially achieve MDL in a model-independent manner. To this end, we consider a weighted loss function and extend it to an effective procedure by employing techniques from the recently active area of learning-to-learn (meta-learning). Specifically, we take inner-loop gradient steps to dynamically estimate posterior distributions over the hyperparameters of our loss function. Thus, our method is model-agnostic, requiring no additional model parameters and no network architecture changes; instead, only a few efficient algorithmic modifications are needed to improve performance in MDL. We demonstrate our solution to a fitting problem in medical imaging, specifically, in the automatic segmentation of white matter hyperintensity (WMH). We look at two neuroimaging modalities (T1-MR and FLAIR) with complementary information fitting for our problem.

ROBUST WHITE MATTER HYPERINTENSITY SEGMENTATION ON UNSEEN DOMAIN.

Typical machine learning frameworks heavily rely on an underlying assumption that training and test data follow the same distribution. In medical imaging which increasingly begun acquiring datasets from multiple sites or scanners, this identical distribution assumption often fails to hold due to systematic variability induced by site or scanner dependent factors. Therefore, we cannot simply expect a model trained on a given dataset to consistently work well, or generalize, on a dataset from another distribution. In this work, we address this problem, investigating the application of machine learning models to unseen medical imaging data. Specifically, we consider the challenging case of Domain Generalization (DG) where we train a model without any knowledge about the testing distribution. That is, we train on samples from a set of distributions (sources) and test on samples from a new, unseen distribution (target). We focus on the task of white matter hyperintensity (WMH) prediction using the multi-site WMH Segmentation Challenge dataset and our local in-house dataset. We identify how two mechanically distinct DG approaches, namely domain adversarial learning and mix-up, have theoretical synergy. Then, we show drastic improvements of WMH prediction on an unseen target domain.

Multi-scanner Harmonization of Paired Neuroimaging Data via Structure Preserving Embedding Learning.

Combining datasets from multiple sites/scanners has been becoming increasingly more prevalent in modern neuroimaging studies. Despite numerous benefits from the growth in sample size, substantial technical variability associated with site/scanner-related effects exists which may inadvertently bias subsequent downstream analyses. Such a challenge calls for a data harmonization procedure which reduces the scanner effects and allows the scans to be combined for pooled analyses. In this work, we present MISPEL (Multi-scanner Image harmonization via Structure Preserving Embedding Learning), a multi-scanner harmonization framework. Unlike existing techniques, MISPEL does not assume a perfect coregistration across the scans, and the framework is naturally extendable to more than two scanners. Importantly, we incorporate our multi-scanner dataset where each subject is scanned on four different scanners. This unique paired dataset allows us to define and aim for an ideal harmonization (e.g., each subject with identical brain tissue volumes on all scanners). We extensively view scanner effects under varying metrics and demonstrate how MISPEL significantly improves them.

PAC Bayesian Performance Guarantees for Deep (Stochastic) Networks in Medical Imaging.

Application of deep neural networks to medical imaging tasks has in some sense become commonplace. Still, a "thorn in the side" of the deep learning movement is the argument that deep networks are prone to overfitting and are thus unable to generalize well when datasets are small (as is common in medical imaging tasks). One way to bolster confidence is to provide mathematical guarantees, or bounds, on network performance after training which explicitly quantify the possibility of overfitting. In this work, we explore recent advances using the PAC-Bayesian framework to provide bounds on generalization error for large (stochastic) networks. While previous efforts focus on classification in larger natural image datasets (e.g., MNIST and CIFAR-10), we apply these techniques to both classification and segmentation in a smaller medical imagining dataset: the ISIC 2018 challenge set. We observe the resultant bounds are competitive compared to a simpler baseline, while also being more explainable and alleviating the need for holdout sets.

A multi-scanner neuroimaging data harmonization using RAVEL and ComBat.

Modern neuroimaging studies frequently combine data collected from multiple scanners and experimental conditions. Such data often contain substantial technical variability associated with image intensity scale (image intensity scales are not the same in different images) and scanner effects (images obtained from different scanners contain substantial technical biases). Here we evaluate and compare results of data analysis methods without any data transformation (RAW), with intensity normalization using RAVEL, with regional harmonization methods using ComBat, and a combination of RAVEL and ComBat. Methods are evaluated on a unique sample of 16 study participants who were scanned on both 1.5T and 3T scanners a few months apart. Neuroradiological evaluation was conducted for 7 different regions of interest (ROI's) pertinent to Alzheimer's disease (AD). Cortical measures and results indicate that: (1) RAVEL substantially improved the reproducibility of image intensities; (2) ComBat is preferred over RAVEL and the RAVEL-ComBat combination in terms of regional level harmonization due to more consistent harmonization across subjects and image-derived measures; (3) RAVEL and ComBat substantially reduced bias compared to analysis of RAW images, but RAVEL also resulted in larger variance; and (4) the larger root mean square deviation (RMSD) of RAVEL compared to ComBat is due mainly to its larger variance.

Developmental morphology of the cervical vertebrae and the emergence of sexual dimorphism in size and shape: A computed tomography study.

Cervical vertebral bodies undergo substantial morphological development during the first two decades of life that are used clinically to visually determine skeletal maturation with the cervical vertebral maturation index (CVMI). CVMI defines six stages that capture the morphological transformations from 6 years to 18 years. However, CVMI has poor reproducibility given its qualitative nature and does not account for sexual dimorphism. This study aims to quantify the morphological development of the cervical vertebral bodies C2-C7 in size (height and depth) and shape and examine the emergence of sexual dimorphism. Using 115 (70 M;45F) computed tomography studies from typically developing individuals ages 6 months to 20 years, landmarks were placed at the margins of the C2-C7 cervical vertebral bodies in the midsagittal plane for size and shape analysis. Findings revealed a dichotomy in the growth trends of height versus depth. The C2-C7 growth in depth gained the majority of the adult size by age 5 years, while the C3-C7 growth in height displayed two periods of accelerated growth during early childhood and puberty. Significant sex differences were found in height and depth growth trends and the form-space ontogenetic trajectories during puberty, with minor but evident differences emerging at age 3 years. Female C2-C7 depth measures were smaller than males at all ages. However, sex differences in height became evident due to males continuing to grow after females reach maturity. Findings quantify the morphological developmental stages of CVMI and emphasize the need to account for sex differences when assessing skeletal maturation.

Cervical vertebral body growth and emergence of sexual dimorphism: a developmental study using computed tomography.

The size and shape of human cervical vertebral bodies serve as a reference for measurement or treatment planning in multiple disciplines. It is therefore necessary to understand thoroughly the developmental changes in the cervical vertebrae in relation to the changing biomechanical demands on the neck during the first two decades of life. To delineate sex-specific changes in human cervical vertebral bodies, 23 landmarks were placed in the midsagittal plane to define the boundaries of C2 to C7 in 123 (73 M; 50 F) computed tomography scans from individuals, ages 6 months to 19 years. Size was calculated as the geometric area, from which sex-specific growth trend, rate, and type for each vertebral body were determined, as well as length measures of local deformation-based morphometry vectors from the centroid to each landmark. Additionally, for each of the four pubertal-staged age cohorts, sex-specific vertebral body wireframes were superimposed using generalized Procrustes analysis to determine sex-specific changes in form (size and shape) and shape alone. Our findings reveal that C2 was unique in achieving more of its adult size by 5 years, particularly in females. In contrast, C3-C7 had a second period of accelerated growth during puberty. The vertebrae of males and females were significantly different in size, particularly after puberty, when males had larger cervical vertebral bodies. Male growth outpaced female growth around age 10 years and persisted until around age 19-20 years, whereas females completed growth earlier, around age 17-18 years. The greatest shape differences between males and females occurred during puberty. Both sexes had similar growth in the superoinferior height, but males also displayed more growth in anteroposterior depth. Such prominent sex differences in size, shape, and form are likely the result of differences in growth rate and growth duration. Female vertebrae are thus not simply smaller versions of the male vertebrae. Additional research is needed to further quantify growth and help improve age- and sex-specific guidance in clinical practice.

Cerebrospinal fluid biomarkers of neurofibrillary tangles and synaptic dysfunction are associated with longitudinal decline in white matter connectivity: A multi-resolution graph analysis.

In addition to the development of beta amyloid plaques and neurofibrillary tangles, Alzheimer's disease (AD) involves the loss of connecting structures including degeneration of myelinated axons and synaptic connections. However, the extent to which white matter tracts change longitudinally, particularly in the asymptomatic, preclinical stage of AD, remains poorly characterized. In this study we used a novel graph wavelet algorithm to determine the extent to which microstructural brain changes evolve in concert with the development of AD neuropathology as observed using CSF biomarkers. A total of 118 participants with at least two diffusion tensor imaging (DTI) scans and one lumbar puncture for CSF were selected from two observational and longitudinally followed cohorts. CSF was assayed for pathology specific to AD (Aß42 and phosphorylated-tau), neurodegeneration (total-tau), axonal degeneration (neurofilament light chain protein; NFL), and synaptic degeneration (neurogranin). Tractography was performed on DTI scans to obtain structural connectivity networks with 160 nodes where the nodes correspond to specific brain regions of interest (ROIs) and their connections were defined by DTI metrics (i.e., fractional anisotropy (FA) and mean diffusivity (MD)). For the analysis, we adopted a multi-resolution graph wavelet technique called Wavelet Connectivity Signature (WaCS) which derives higher order representations from DTI metrics at each brain connection. Our statistical analysis showed interactions between the CSF measures and the MRI time interval, such that elevated CSF biomarkers and longer time were associated with greater longitudinal changes in white matter microstructure (decreasing FA and increasing MD). Specifically, we detected a total of 17 fiber tracts whose WaCS representations showed an association between longitudinal decline in white matter microstructure and both CSF p-tau and neurogranin. While development of neurofibrillary tangles and synaptic degeneration are cortical phenomena, the results show that they are also associated with degeneration of underlying white matter tracts, a process which may eventually play a role in the development of cognitive decline and dementia.

Associations Between Positron Emission Tomography Amyloid Pathology and Diffusion Tensor Imaging Brain Connectivity in Pre-Clinical Alzheimer's Disease.

Characterizing Alzheimer's disease (AD) at pre-clinical stages is crucial for initiating early treatment strategies. It is widely accepted that amyloid accumulation is a primary pathological event in AD. Also, loss of connectivity between brain regions is suspected of contributing to cognitive decline, but studies that test these associations using either local (i.e., individual edges) or global (i.e., modularity) connectivity measures may be limited. In this study, we utilized data acquired from 139 cognitively unimpaired participants. Sixteen gray matter (GM) regions known to be affected by AD were selected for analysis. For each of the 16 regions, the effect of amyloid burden, measured using Pittsburgh Compound B (PiB) positron emission tomography, on each of the 1761 brain network connections derived from diffusion tensor imaging (DTI) connecting 162 GM regions, was investigated. Applying our unique multiresolution statistical analysis called the Wavelet Connectivity Signature (WaCS), this study demonstrates the relationship between amyloid burden and structural brain connectivity as assessed with DTI. Our statistical analysis using WaCS shows that in 15 of 16 GM regions, statistically significant relationships between amyloid burden in those regions and structural connectivity networks were observed. After applying multiple testing correction, 10 unique structural brain connections were found to be significantly associated with amyloid accumulation. For 7 of those 10 network connections, the decrease in their network connection strength indexed by fractional anisotropy was, in turn, associated with lower cognitive function, providing evidence that AD-related structural connectivity loss is a correlate of cognitive decline.

Sampling-free Uncertainty Estimation in Gated Recurrent Units with Applications to Normative Modeling in Neuroimaging.

There has recently been a concerted effort to derive mechanisms in vision and machine learning systems to offer uncertainty estimates of the predictions they make. Clearly, there are benefits to a system that is not only accurate but also has a sense for when it is not. Existing proposals center around Bayesian interpretations of modern deep architectures - these are effective but can often be computationally demanding. We show how classical ideas in the literature on exponential families on probabilistic networks provide an excellent starting point to derive uncertainty estimates in Gated Recurrent Units (GRU). Our proposal directly quantifies uncertainty deterministically, without the need for costly sampling-based estimation. We show that while uncertainty is quite useful by itself in computer vision and machine learning, we also demonstrate that it can play a key role in enabling statistical analysis with deep networks in neuroimaging studies with normative modeling methods. To our knowledge, this is the first result describing sampling-free uncertainty estimation for powerful sequential models such as GRUs.

Conditional Recurrent Flow: Conditional Generation of Longitudinal Samples with Applications to Neuroimaging.

We develop a conditional generative model for longitudinal image datasets based on sequential invertible neural networks. Longitudinal image acquisitions are common in various scientific and biomedical studies where often each image sequence sample may also come together with various secondary (fixed or temporally dependent) measurements. The key goal is not only to estimate the parameters of a deep generative model for the given longitudinal data, but also to enable evaluation of how the temporal course of the generated longitudinal samples are influenced as a function of induced changes in the (secondary) temporal measurements (or events). Our proposed formulation incorporates recurrent subnetworks and temporal context gating, which provide a smooth transition in a temporal sequence of generated data that can be easily informed or modulated by secondary temporal conditioning variables. We show that the formulation works well despite the smaller sample sizes common in these applications. Our model is validated on two video datasets and a longitudinal Alzheimer's disease (AD) dataset for both quantitative and qualitative evaluations of the generated samples. Further, using our generated longitudinal image samples, we show that we can capture the pathological progressions in the brain that turn out to be consistent with the existing literature, and could facilitate various types of downstream statistical analysis.

Adaptive Signal Recovery on Graphs via Harmonic Analysis for Experimental Design in Neuroimaging.

Consider an experimental design of a neuroimaging study, where we need to obtain p measurements for each participant in a setting where p' (< p) are cheaper and easier to acquire while the remaining (p - p') are expensive. For example, the p' measurements may include demographics, cognitive scores or routinely offered imaging scans while the (p - p') measurements may correspond to more expensive types of brain image scans with a higher participant burden. In this scenario, it seems reasonable to seek an "adaptive" design for data acquisition so as to minimize the cost of the study without compromising statistical power. We show how this problem can be solved via harmonic analysis of a band-limited graph whose vertices correspond to participants and our goal is to fully recover a multi-variate signal on the nodes, given the full set of cheaper features and a partial set of more expensive measurements. This is accomplished using an adaptive query strategy derived from probing the properties of the graph in the frequency space. To demonstrate the benefits that this framework can provide, we present experimental evaluations on two independent neuroimaging studies and show that our proposed method can reliably recover the true signal with only partial observations directly yielding substantial financial savings.

Coupled Harmonic Bases for Longitudinal Characterization of Brain Networks.

There is a great deal of interest in using large scale brain imaging studies to understand how brain connectivity evolves over time for an individual and how it varies over different levels/quantiles of cognitive function. To do so, one typically performs so-called tractography procedures on diffusion MR brain images and derives measures of brain connectivity expressed as graphs. The nodes correspond to distinct brain regions and the edges encode the strength of the connection. The scientific interest is in characterizing the evolution of these graphs over time or from healthy individuals to diseased. We pose this important question in terms of the Laplacian of the connectivity graphs derived from various longitudinal or disease time points - quantifying its progression is then expressed in terms of coupling the harmonic bases of a full set of Laplacians. We derive a coupled system of generalized eigenvalue problems (and corresponding numerical optimization schemes) whose solution helps characterize the full life cycle of brain connectivity evolution in a given dataset. Finally, we show a set of results on a diffusion MR imaging dataset of middle aged people at risk for Alzheimer's disease (AD), who are cognitively healthy. In such asymptomatic adults, we find that a framework for characterizing brain connectivity evolution provides the ability to predict cognitive scores for individual subjects, and for estimating the progression of participant's brain connectivity into the future.

A Projection free method for Generalized Eigenvalue Problem with a nonsmooth Regularizer.

Eigenvalue problems are ubiquitous in computer vision, covering a very broad spectrum of applications ranging from estimation problems in multi-view geometry to image segmentation. Few other linear algebra problems have a more mature set of numerical routines available and many computer vision libraries leverage such tools extensively. However, the ability to call the underlying solver only as a "black box" can often become restrictive. Many 'human in the loop' settings in vision frequently exploit supervision from an expert, to the extent that the user can be considered a subroutine in the overall system. In other cases, there is additional domain knowledge, side or even partial information that one may want to incorporate within the formulation. In general, regularizing a (generalized) eigenvalue problem with such side information remains difficult. Motivated by these needs, this paper presents an optimization scheme to solve generalized eigenvalue problems (GEP) involving a (nonsmooth) regularizer. We start from an alternative formulation of GEP where the feasibility set of the model involves the Stiefel manifold. The core of this paper presents an end to end stochastic optimization scheme for the resultant problem. We show how this general algorithm enables improved statistical analysis of brain imaging data where the regularizer is derived from other 'views' of the disease pathology, involving clinical measurements and other image-derived representations.