Neuropsychiatric disease classification using functional connectomics - results of the connectomics in neuroimaging transfer learning challenge.

Large, open-source datasets, such as the Human Connectome Project and the Autism Brain Imaging Data Exchange, have spurred the development of new and increasingly powerful machine learning approaches for brain connectomics. However, one key question remains: are we capturing biologically relevant and generalizable information about the brain, or are we simply overfitting to the data? To answer this, we organized a scientific challenge, the Connectomics in NeuroImaging Transfer Learning Challenge (CNI-TLC), held in conjunction with MICCAI 2019. CNI-TLC included two classification tasks: (1) diagnosis of Attention-Deficit/Hyperactivity Disorder (ADHD) within a pre-adolescent cohort; and (2) transference of the ADHD model to a related cohort of Autism Spectrum Disorder (ASD) patients with an ADHD comorbidity. In total, 240 resting-state fMRI (rsfMRI) time series averaged according to three standard parcellation atlases, along with clinical diagnosis, were released for training and validation (120 neurotypical controls and 120 ADHD). We also provided Challenge participants with demographic information of age, sex, IQ, and handedness. The second set of 100 subjects (50 neurotypical controls, 25 ADHD, and 25 ASD with ADHD comorbidity) was used for testing. Classification methodologies were submitted in a standardized format as containerized Docker images through ChRIS, an open-source image analysis platform. Utilizing an inclusive approach, we ranked the methods based on 16 metrics: accuracy, area under the curve, F1-score, false discovery rate, false negative rate, false omission rate, false positive rate, geometric mean, informedness, markedness, Matthew's correlation coefficient, negative predictive value, optimized precision, precision, sensitivity, and specificity. The final rank was calculated using the rank product for each participant across all measures. Furthermore, we assessed the calibration curves of each methodology. Five participants submitted their method for evaluation, with one outperforming all other methods in both ADHD and ASD classification. However, further improvements are still needed to reach the clinical translation of functional connectomics. We have kept the CNI-TLC open as a publicly available resource for developing and validating new classification methodologies in the field of connectomics.

Reusable Client-Side JavaScript Modules for Immersive Web-Based Real-Time Collaborative Neuroimage Visualization.

In this paper we present a web-based software solution to the problem of implementing real-time collaborative neuroimage visualization. In both clinical and research settings, simple and powerful access to imaging technologies across multiple devices is becoming increasingly useful. Prior technical solutions have used a server-side rendering and push-to-client model wherein only the server has the full image dataset. We propose a rich client solution in which each client has all the data and uses the Google Drive Realtime API for state synchronization. We have developed a small set of reusable client-side object-oriented JavaScript modules that make use of the XTK toolkit, a popular open-source JavaScript library also developed by our team, for the in-browser rendering and visualization of brain image volumes. Efficient realtime communication among the remote instances is achieved by using just a small JSON object, comprising a representation of the XTK image renderers' state, as the Google Drive Realtime collaborative data model. The developed open-source JavaScript modules have already been instantiated in a web-app called MedView, a distributed collaborative neuroimage visualization application that is delivered to the users over the web without requiring the installation of any extra software or browser plugin. This responsive application allows multiple physically distant physicians or researchers to cooperate in real time to reach a diagnosis or scientific conclusion. It also serves as a proof of concept for the capabilities of the presented technological solution.

Event time analysis of longitudinal neuroimage data.

This paper presents a method for the statistical analysis of the associations between longitudinal neuroimaging measurements, e.g., of cortical thickness, and the timing of a clinical event of interest, e.g., disease onset. The proposed approach consists of two steps, the first of which employs a linear mixed effects (LME) model to capture temporal variation in serial imaging data. The second step utilizes the extended Cox regression model to examine the relationship between time-dependent imaging measurements and the timing of the event of interest. We demonstrate the proposed method both for the univariate analysis of image-derived biomarkers, e.g., the volume of a structure of interest, and the exploratory mass-univariate analysis of measurements contained in maps, such as cortical thickness and gray matter density. The mass-univariate method employs a recently developed spatial extension of the LME model. We applied our method to analyze structural measurements computed using FreeSurfer, a widely used brain Magnetic Resonance Image (MRI) analysis software package. We provide a quantitative and objective empirical evaluation of the statistical performance of the proposed method on longitudinal data from subjects suffering from Mild Cognitive Impairment (MCI) at baseline.

Spatiotemporal linear mixed effects modeling for the mass-univariate analysis of longitudinal neuroimage data.

We present an extension of the Linear Mixed Effects (LME) modeling approach to be applied to the mass-univariate analysis of longitudinal neuroimaging (LNI) data. The proposed method, called spatiotemporal LME or ST-LME, builds on the flexible LME framework and exploits the spatial structure in image data. We instantiated ST-LME for the analysis of cortical surface measurements (e.g. thickness) computed by FreeSurfer, a widely-used brain Magnetic Resonance Image (MRI) analysis software package. We validate the proposed ST-LME method and provide a quantitative and objective empirical comparison with two popular alternative methods, using two brain MRI datasets obtained from the Alzheimer's disease neuroimaging initiative (ADNI) and Open Access Series of Imaging Studies (OASIS). Our experiments revealed that ST-LME offers a dramatic gain in statistical power and repeatability of findings, while providing good control of the false positive rate.

Statistical analysis of longitudinal neuroimage data with Linear Mixed Effects models.

Longitudinal neuroimaging (LNI) studies are rapidly becoming more prevalent and growing in size. Today, no standardized computational tools exist for the analysis of LNI data and widely used methods are sub-optimal for the types of data encountered in real-life studies. Linear Mixed Effects (LME) modeling, a mature approach well known in the statistics community, offers a powerful and versatile framework for analyzing real-life LNI data. This article presents the theory behind LME models, contrasts it with other popular approaches in the context of LNI, and is accompanied with an array of computational tools that will be made freely available through FreeSurfer - a popular Magnetic Resonance Image (MRI) analysis software package. Our core contribution is to provide a quantitative empirical evaluation of the performance of LME and competing alternatives popularly used in prior longitudinal structural MRI studies, namely repeated measures ANOVA and the analysis of annualized longitudinal change measures (e.g. atrophy rate). In our experiments, we analyzed MRI-derived longitudinal hippocampal volume and entorhinal cortex thickness measurements from a public dataset consisting of Alzheimer's patients, subjects with mild cognitive impairment and healthy controls. Our results suggest that the LME approach offers superior statistical power in detecting longitudinal group differences.

Determining the optimal level of smoothing in cortical thickness analysis: a hierarchical approach based on sequential statistical thresholding.

The extent of smoothing applied to cortical thickness maps critically influences sensitivity, anatomical precision and resolution of statistical change detection. Theoretically, it could be optimized by increasing the trade-off between vertex-wise sensitivity and specificity across several levels of smoothing. But to date neither parametric nor nonparametric methods are able to control the error at the vertex level if the null hypothesis is rejected after smoothing of cortical thickness maps. To overcome these drawbacks, we applied sequential statistical thresholding based on a simple hierarchical model. This methodology aims at controlling erroneous detections; firstly at the level of clusters, over smoothed statistical maps; and secondly at the vertex level, over unsmoothed statistical maps, by applying an adaptive false discovery rate (FDR) procedure to clusters previously detected. The superior performance of the proposed methodology over other conventional procedures was demonstrated in simulation studies. As expected, only the hierarchical method yielded a predictable false discovery proportion near the predefined FDR q-value for any smoothing level at the same time as being as sensitive as the others at the optimal setting. It was therefore the only method able to approximate the optimal size of spatial smoothing when the true change was assumed unknown. The hierarchical method was further validated in a cross-sectional study comparing moderate Alzheimer's disease (AD) patients with healthy elderly subjects. Results suggest that the extent of cortical thinning reported in previous AD studies might be artificially inflated by the choice of inadequate smoothing. In these cases, interpretation should be based on the location of local maxima of suprathreshold regions rather than on the spatial extent of the detected signal in the statistical parametric map.

Detection of focal changes in human cortical thickness: spherical wavelets versus Gaussian smoothing.

Subtle but progressive variations in human cortical thickness have been associated with the initial phases of prevalent neurological and psychiatric conditions. But slight changes in cortical thickness at preclinical stages are typically masked by effects of the Gaussian kernel smoothing on the cortical surface shape descriptors. Here we present the first study aimed at detecting changes in human cortical thickness maps by applying soft-thresholding to multiresolution spherical wavelet coefficients. In order to make Gaussian and wavelet smoothing methods comparable, the trade-off between sensitivity and specificity was optimized to detect simulated thickness changes in various cortical areas of healthy elderly subjects. Results revealed a better sensitivity-specificity trade-off when using wavelet-based methods as compared to Gaussian smoothing in both the whole neocortex (p<10(-7)) and cortical region-based statistical analyses (p<10(-9)), which was mainly due to the higher specificity obtained with the wavelet approach. The lower smoothing introduced by wavelets and their adaptive properties may account for the enhanced specificity and sensitivity when compared with Gaussian spatial filters. These results strongly support the use of spherical wavelet methods to detect subtle variations in cortical thickness maps, which may be crucial in better understanding the course of neuronal loss in normal aging and in finding early markers of cortical degeneration.