Psychiatric neuroimaging designs for individualised, cohort, and population studies.

Psychiatric neuroimaging faces challenges to rigour and reproducibility that prompt reconsideration of the relative strengths and limitations of study designs. Owing to high resource demands and varying inferential goals, current designs differentially emphasise sample size, measurement breadth, and longitudinal assessments. In this overview and perspective, we provide a guide to the current landscape of psychiatric neuroimaging study designs with respect to this balance of scientific goals and resource constraints. Through a heuristic data cube contrasting key design features, we discuss a resulting trade-off among small sample, precision longitudinal studies (e.g., individualised studies and cohorts) and large sample, minimally longitudinal, population studies. Precision studies support tests of within-person mechanisms, via intervention and tracking of longitudinal course. Population studies support tests of generalisation across multifaceted individual differences. A proposed reciprocal validation model (RVM) aims to recursively leverage these complementary designs in sequence to accumulate evidence, optimise relative strengths, and build towards improved long-term clinical utility.

Power and reproducibility in the external validation of brain-phenotype predictions.

Brain-phenotype predictive models seek to identify reproducible and generalizable brain-phenotype associations. External validation, or the evaluation of a model in external datasets, is the gold standard in evaluating the generalizability of models in neuroimaging. Unlike typical studies, external validation involves two sample sizes: the training and the external sample sizes. Thus, traditional power calculations may not be appropriate. Here we ran over 900 million resampling-based simulations in functional and structural connectivity data to investigate the relationship between training sample size, external sample size, phenotype effect size, theoretical power and simulated power. Our analysis included a wide range of datasets: the Healthy Brain Network, the Adolescent Brain Cognitive Development Study, the Human Connectome Project (Development and Young Adult), the Philadelphia Neurodevelopmental Cohort, the Queensland Twin Adolescent Brain Project, and the Chinese Human Connectome Project; and phenotypes: age, body mass index, matrix reasoning, working memory, attention problems, anxiety/depression symptoms and relational processing. High effect size predictions achieved adequate power with training and external sample sizes of a few hundred individuals, whereas low and medium effect size predictions required hundreds to thousands of training and external samples. In addition, most previous external validation studies used sample sizes prone to low power, and theoretical power curves should be adjusted for the training sample size. Furthermore, model performance in internal validation often informed subsequent external validation performance (Pearson's r difference <0.2), particularly for well-harmonized datasets. These results could help decide how to power future external validation studies.

The brain structure, inflammatory, and genetic mechanisms mediate the association between physical frailty and depression.

Cross-sectional studies have demonstrated strong associations between physical frailty and depression. However, the evidence from prospective studies is limited. Here, we analyze data of 352,277 participants from UK Biobank with 12.25-year follow-up. Compared with non-frail individuals, pre-frail and frail individuals have increased risk for incident depression independent of many putative confounds. Altogether, pre-frail and frail individuals account for 20.58% and 13.16% of depression cases by population attributable fraction analyses. Higher risks are observed in males and individuals younger than 65 years than their counterparts. Mendelian randomization analyses support a potential causal effect of frailty on depression. Associations are also observed between inflammatory markers, brain volumes, and incident depression. Moreover, these regional brain volumes and three inflammatory markers-C-reactive protein, neutrophils, and leukocytes-significantly mediate associations between frailty and depression. Given the scarcity of curative treatment for depression and the high disease burden, identifying potential modifiable risk factors of depression, such as frailty, is needed.

Connectome caricatures: removing large-amplitude co-activation patterns in resting-state fMRI emphasizes individual differences.

High-amplitude co-activation patterns are sparsely present during resting-state fMRI but drive functional connectivity1-5. Further, they resemble task activation patterns and are well-studied3,5-10. However, little research has characterized the remaining majority of the resting-state signal. In this work, we introduced caricaturing-a method to project resting-state data to a subspace orthogonal to a manifold of co-activation patterns estimated from the task fMRI data. Projecting to this subspace removes linear combinations of these co-activation patterns from the resting-state data to create Caricatured connectomes. We used rich task data from the Human Connectome Project (HCP)11 and the UCLA Consortium for Neuropsychiatric Phenomics12 to construct a manifold of task co-activation patterns. Caricatured connectomes were created by projecting resting-state data from the HCP and the Yale Test-Retest13 datasets away from this manifold. Like caricatures, these connectomes emphasized individual differences by reducing between-individual similarity and increasing individual identification14. They also improved predictive modeling of brain-phenotype associations. As caricaturing removes group-relevant task variance, it is an initial attempt to remove task-like co-activations from rest. Therefore, our results suggest that there is a useful signal beyond the dominating co-activations that drive resting-state functional connectivity, which may better characterize the brain's intrinsic functional architecture.

Data leakage inflates prediction performance in connectome-based machine learning models.

Predictive modeling is a central technique in neuroimaging to identify brain-behavior relationships and test their generalizability to unseen data. However, data leakage undermines the validity of predictive models by breaching the separation between training and test data. Leakage is always an incorrect practice but still pervasive in machine learning. Understanding its effects on neuroimaging predictive models can inform how leakage affects existing literature. Here, we investigate the effects of five forms of leakage-involving feature selection, covariate correction, and dependence between subjects-on functional and structural connectome-based machine learning models across four datasets and three phenotypes. Leakage via feature selection and repeated subjects drastically inflates prediction performance, whereas other forms of leakage have minor effects. Furthermore, small datasets exacerbate the effects of leakage. Overall, our results illustrate the variable effects of leakage and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Brain-phenotype predictions can survive across diverse real-world data.

Recent work suggests that machine learning models predicting psychiatric treatment outcomes based on clinical data may fail when applied to unharmonized samples. Neuroimaging predictive models offer the opportunity to incorporate neurobiological information, which may be more robust to dataset shifts. Yet, among the minority of neuroimaging studies that undertake any form of external validation, there is a notable lack of attention to generalization across dataset-specific idiosyncrasies. Research settings, by design, remove the between-site variations that real-world and, eventually, clinical applications demand. Here, we rigorously test the ability of a range of predictive models to generalize across three diverse, unharmonized samples: the Philadelphia Neurodevelopmental Cohort (n=1291), the Healthy Brain Network (n=1110), and the Human Connectome Project in Development (n=428). These datasets have high inter-dataset heterogeneity, encompassing substantial variations in age distribution, sex, racial and ethnic minority representation, recruitment geography, clinical symptom burdens, fMRI tasks, sequences, and behavioral measures. We demonstrate that reproducible and generalizable brain-behavior associations can be realized across diverse dataset features with sample sizes in the hundreds. Results indicate the potential of functional connectivity-based predictive models to be robust despite substantial inter-dataset variability. Notably, for the HCPD and HBN datasets, the best predictions were not from training and testing in the same dataset (i.e., cross-validation) but across datasets. This result suggests that training on diverse data may improve prediction in specific cases. Overall, this work provides a critical foundation for future work evaluating the generalizability of neuroimaging predictive models in real-world scenarios and clinical settings.

Test-Retest Reliability of Functional Connectivity in Adolescents With Depression.

BACKGROUND: The test-retest reliability of functional magnetic resonance imaging is critical to identifying reproducible biomarkers for psychiatric illness. Recent work has shown how reliability limits the observable effect size of brain-behavior associations, hindering detection of these effects. However, while a fast-growing literature has explored both univariate and multivariate reliability in healthy individuals, relatively few studies have explored reliability in populations with psychiatric illnesses or how this interacts with age. METHODS: Here, we investigated functional connectivity reliability over the course of 1 year in a longitudinal cohort of 88 adolescents (age at baseline = 15.63 ± 1.29 years; 64 female) with major depressive disorder (MDD) and without MDD (healthy volunteers [HVs]). We compared a univariate metric, intraclass correlation coefficient, and 2 multivariate metrics, fingerprinting and discriminability. RESULTS: Adolescents with MDD had marginally higher mean intraclass correlation coefficient (µMDD = 0.34, 95% CI, 0.12-0.54; µHV = 0.27, 95% CI, 0.05-0.52), but both groups had poor average intraclass correlation coefficients (<0.4). Fingerprinting index was greater than chance and did not differ between groups (fingerprinting indexMDD = 0.75; fingerprinting indexHV = 0.91; Poisson tests p < .001). Discriminability indicated high multivariate reliability in both groups (discriminabilityMDD = 0.80; discriminabilityHV = 0.82; permutation tests p < .01). Neither univariate nor multivariate reliability was associated with symptom severity or edge-level effect size of group differences. CONCLUSIONS: Overall, we found little evidence for a relationship between depression and reliability of functional connectivity during adolescence. These findings suggest that biomarker identification in depression is not limited due to reliability compared with healthy samples and support the shift toward multivariate analysis for improved power and reliability.

Power and reproducibility in the external validation of brain-phenotype predictions.

Identifying reproducible and generalizable brain-phenotype associations is a central goal of neuroimaging. Consistent with this goal, prediction frameworks evaluate brain-phenotype models in unseen data. Most prediction studies train and evaluate a model in the same dataset. However, external validation, or the evaluation of a model in an external dataset, provides a better assessment of robustness and generalizability. Despite the promise of external validation and calls for its usage, the statistical power of such studies has yet to be investigated. In this work, we ran over 60 million simulations across several datasets, phenotypes, and sample sizes to better understand how the sizes of the training and external datasets affect statistical power. We found that prior external validation studies used sample sizes prone to low power, which may lead to false negatives and effect size inflation. Furthermore, increases in the external sample size led to increased simulated power directly following theoretical power curves, whereas changes in the training dataset size offset the simulated power curves. Finally, we compared the performance of a model within a dataset to the external performance. The within-dataset performance was typically within r=0.2 of the cross-dataset performance, which could help decide how to power future external validation studies. Overall, our results illustrate the importance of considering the sample sizes of both the training and external datasets when performing external validation.

Network controllability of structural connectomes in the neonatal brain.

White matter connectivity supports diverse cognitive demands by efficiently constraining dynamic brain activity. This efficiency can be inferred from network controllability, which represents the ease with which the brain moves between distinct mental states based on white matter connectivity. However, it remains unclear how brain networks support diverse functions at birth, a time of rapid changes in connectivity. Here, we investigate the development of network controllability during the perinatal period and the effect of preterm birth in 521 neonates. We provide evidence that elements of controllability are exhibited in the infant's brain as early as the third trimester and develop rapidly across the perinatal period. Preterm birth disrupts the development of brain networks and altered the energy required to drive state transitions at different levels. In addition, controllability at birth is associated with cognitive ability at 18 months. Our results suggest network controllability develops rapidly during the perinatal period to support cognitive demands but could be altered by environmental impacts like preterm birth.

Connectome-based machine learning models are vulnerable to subtle data manipulations.

Neuroimaging-based predictive models continue to improve in performance, yet a widely overlooked aspect of these models is "trustworthiness," or robustness to data manipulations. High trustworthiness is imperative for researchers to have confidence in their findings and interpretations. In this work, we used functional connectomes to explore how minor data manipulations influence machine learning predictions. These manipulations included a method to falsely enhance prediction performance and adversarial noise attacks designed to degrade performance. Although these data manipulations drastically changed model performance, the original and manipulated data were extremely similar (r = 0.99) and did not affect other downstream analysis. Essentially, connectome data could be inconspicuously modified to achieve any desired prediction performance. Overall, our enhancement attacks and evaluation of existing adversarial noise attacks in connectome-based models highlight the need for counter-measures that improve the trustworthiness to preserve the integrity of academic research and any potential translational applications.

Cross Atlas Remapping via Optimal Transport (CAROT): Creating connectomes for different atlases when raw data is not available.

Open-source, publicly available neuroimaging datasets - whether from large-scale data collection efforts or pooled from multiple smaller studies - offer unprecedented sample sizes and promote generalization efforts. Releasing data can democratize science, increase the replicability of findings, and lead to discoveries. Partly due to patient privacy, computational, and data storage concerns, researchers typically release preprocessed data with the voxelwise time series parcellated into a map of predefined regions, known as an atlas. However, releasing preprocessed data also limits the choices available to the end-user. This is especially true for connectomics, as connectomes created from different atlases are not directly comparable. Since there exist several atlases with no gold standards, it is unrealistic to have processed, open-source data available from all atlases. Together, these limitations directly inhibit the potential benefits of open-source neuroimaging data. To address these limitations, we introduce Cross Atlas Remapping via Optimal Transport (CAROT) to find a mapping between two atlases. This approach allows data processed from one atlas to be directly transformed into a connectome based on another atlas without the need for raw data access. To validate CAROT, we compare reconstructed connectomes against their original counterparts (i.e., connectomes generated directly from an atlas), demonstrate the utility of transformed connectomes in downstream analyses, and show how a connectome-based predictive model can generalize to publicly available data that was processed with different atlases. Overall, CAROT can reconstruct connectomes from an extensive set of atlases - without needing the raw data - allowing already processed connectomes to be easily reused in a wide range of analyses while eliminating redundant processing efforts. We share this tool as both source code and as a stand-alone web application (http://carotproject.com/).

Altered Brain Dynamics Across Bipolar Disorder and Schizophrenia During Rest and Task Switching Revealed by Overlapping Brain States.

BACKGROUND: Individuals with bipolar disorder (BD) and schizophrenia (SCZ) show aberrant brain dynamics (i.e., altered recruitment or traversal through different brain states over time). Existing investigations of brain dynamics typically assume that one dominant brain state characterizes each time point. However, as multiple brain states likely are engaged at any given moment, this approach can obscure alterations in less prominent but critical brain states. Here, we examined brain dynamics in BD and SCZ by implementing a novel framework that simultaneously assessed the engagement of multiple brain states. METHODS: Four recurring brain states were identified by applying nonlinear manifold learning and k-means clustering to the Human Connectome Project task-based functional magnetic resonance imaging data. We then assessed moment-to-moment state engagement in 2 independent samples of healthy control participants and patients with BD or SCZ using resting-state (N = 336) or task-based (N = 217) functional magnetic resonance imaging data. Relative state engagement and state engagement variability were extracted and compared across groups using multivariate analysis of covariance, controlling for site, medication, age, and sex. RESULTS: Our framework identified dynamic alterations in BD and SCZ, while a state discretization approach revealed no significant group differences. Participants with BD or SCZ showed reduced state engagement variability, but not relative state engagement, across multiple brain states during resting-state and task-based functional magnetic resonance imaging. We found decreased state engagement variability in older participants and preliminary evidence suggesting an association with avolition. CONCLUSIONS: Assessing multiple brain states simultaneously can reflect the complexity of aberrant brain dynamics in BD and SCZ, providing a more comprehensive understanding of the neural mechanisms underpinning these conditions.

Associations of physical frailty with health outcomes and brain structure in 483 033 middle-aged and older adults: a population-based study from the UK Biobank.

BACKGROUND: Physical frailty is a state of increased vulnerability to stressors and is associated with serious health issues. However, how frailty affects and is affected by numerous other factors, including mental health and brain structure, remains underexplored. We aimed to investigate the mutual effects of frailty and health using large, multidimensional data. METHODS: For this population-based study, we used data from the UK Biobank to examine the pattern and direction of association between physical frailty and 325 health-related measures across multiple domains, using linear mixed-effect models and adjusting for numerous confounders. Participants were included if complete data were available for all five indicators of frailty, all covariates, and at least one health measure. We further examined the association between frailty and brain structure and the role of this association in mediating the relationship between frailty and health outcomes. FINDINGS: 483 033 participants aged 38-73 years were included in the study at baseline (between Dec 19, 2006, and Oct 1, 2010); at a median follow-up of 9 years (IQR 8-10), behavioural data were available for 46 501 participants and neuroimaging data for 40 210 participants. The severity of physical frailty was significantly associated with decreased cognitive performance (Cohen's d=0·025-0·162), increased early-life risks (d=0·026-0·111), unhealthy lifestyle (d=0·013-0·394), poor physical fitness (d=0·007-0·668), increased symptoms of poor mental health (d=0·032-0·607), severe environmental pollution (d=0·013-0·064), and adverse biochemical markers (d=0·025-0·198). Some associations were bidirectional, with the strongest effects on mental health measures. The severity of frailty correlated with increased total white matter hyperintensity and lower grey matter volume, particularly in subcortical regions (d=0·027-0·082), which significantly mediated the association between frailty and health-related outcomes, although the mediated effects were small. INTERPRETATION: Physical frailty is associated with diverse unfavourable health-related outcomes, which can be mediated by differences in brain structure. Our findings offer a framework for guiding preventative strategies targeting both frailty and psychiatric disorders. FUNDING: National Institute of Mental Health, National Science Foundation.

Functional brain networks reflect spatial and temporal autocorrelation.

High-throughput experimental methods in neuroscience have led to an explosion of techniques for measuring complex interactions and multi-dimensional patterns. However, whether sophisticated measures of emergent phenomena can be traced back to simpler, low-dimensional statistics is largely unknown. To explore this question, we examined resting-state functional magnetic resonance imaging (rs-fMRI) data using complex topology measures from network neuroscience. Here we show that spatial and temporal autocorrelation are reliable statistics that explain numerous measures of network topology. Surrogate time series with subject-matched spatial and temporal autocorrelation capture nearly all reliable individual and regional variation in these topology measures. Network topology changes during aging are driven by spatial autocorrelation, and multiple serotonergic drugs causally induce the same topographic change in temporal autocorrelation. This reductionistic interpretation of widely used complexity measures may help link them to neurobiology.

Machine Learning and Prediction in Fetal, Infant, and Toddler Neuroimaging: A Review and Primer.

Predictive models in neuroimaging are increasingly designed with the intent to improve risk stratification and support interventional efforts in psychiatry. Many of these models have been developed in samples of children school-aged or older. Nevertheless, despite growing evidence that altered brain maturation during the fetal, infant, and toddler (FIT) period modulates risk for poor mental health outcomes in childhood, these models are rarely implemented in FIT samples. Applications of predictive modeling in children of these ages provide an opportunity to develop powerful tools for improved characterization of the neural mechanisms underlying development. To facilitate the broader use of predictive models in FIT neuroimaging, we present a brief primer and systematic review on the methods used in current predictive modeling FIT studies. Reflecting on current practices in more than 100 studies conducted over the past decade, we provide an overview of topics, modalities, and methods commonly used in the field and under-researched areas. We then outline ethical and future considerations for neuroimaging researchers interested in predicting health outcomes in early life, including researchers who may be relatively new to either advanced machine learning methods or using FIT data. Altogether, the last decade of FIT research in machine learning has provided a foundation for accelerating the prediction of early-life trajectories across the full spectrum of illness and health.

Global diversity in individualized cortical network topography.

Individualized cortical network topography (ICNT) varies between people and exhibits great variability in the association networks in the human brain. However, these findings were mainly discovered in Western populations. It remains unclear whether and how ICNT is shaped by the non-Western populations. Here, we leveraged a multisession hierarchical Bayesian model to define individualized functional networks in White American and Han Chinese populations with data from both US and Chinese Human Connectome Projects. We found that both the size and spatial topography of individualized functional networks differed between White American and Han Chinese groups, especially in the heteromodal association cortex (including the ventral attention, control, language, dorsal attention, and default mode networks). Employing a support vector machine, we then demonstrated that ethnicity-related ICNT diversity can be used to identify an individual's ethnicity with high accuracy (74%, pperm < 0.0001), with heteromodal networks contributing most to the classification. This finding was further validated through mass-univariate analyses with generalized additive models. Moreover, we reveal that the spatial heterogeneity of ethnic diversity in ICNT correlated with fundamental properties of cortical organization, including evolutionary cortical expansion, brain myelination, and cerebral blood flow. Altogether, this case study highlights a need for more globally diverse and publicly available neuroimaging datasets.

A functional connectome signature of blood pressure in >30 000 participants from the UK biobank.

AIMS: Elevated blood pressure (BP) is a prevalent modifiable risk factor for cardiovascular diseases and contributes to cognitive decline in late life. Despite the fact that functional changes may precede irreversible structural damage and emerge in an ongoing manner, studies have been predominantly informed by brain structure and group-level inferences. Here, we aim to delineate neurobiological correlates of BP at an individual level using machine learning and functional connectivity. METHODS AND RESULTS: Based on whole-brain functional connectivity from the UK Biobank, we built a machine learning model to identify neural representations for individuals' past (∼8.9 years before scanning, N = 35 882), current (N = 31 367), and future (∼2.4 years follow-up, N = 3 138) BP levels within a repeated cross-validation framework. We examined the impact of multiple potential covariates, as well as assessed these models' generalizability across various contexts.The predictive models achieved significant correlations between predicted and actual systolic/diastolic BP and pulse pressure while controlling for multiple confounders. Predictions for participants not on antihypertensive medication were more accurate than for currently medicated patients. Moreover, the models demonstrated robust generalizability across contexts in terms of ethnicities, imaging centres, medication status, participant visits, gender, age, and body mass index. The identified connectivity patterns primarily involved the cerebellum, prefrontal, anterior insula, anterior cingulate cortex, supramarginal gyrus, and precuneus, which are key regions of the central autonomic network, and involved in cognition processing and susceptible to neurodegeneration in Alzheimer's disease. Results also showed more involvement of default mode and frontoparietal networks in predicting future BP levels and in medicated participants. CONCLUSION: This study, based on the largest neuroimaging sample currently available and using machine learning, identifies brain signatures underlying BP, providing evidence for meaningful BP-associated neural representations in connectivity profiles.

The effects of data leakage on connectome-based machine learning models.

Predictive modeling has now become a central technique in neuroimaging to identify complex brain-behavior relationships and test their generalizability to unseen data. However, data leakage, which unintentionally breaches the separation between data used to train and test the model, undermines the validity of predictive models. Previous literature suggests that leakage is generally pervasive in machine learning, but few studies have empirically evaluated the effects of leakage in neuroimaging data. Although leakage is always an incorrect practice, understanding the effects of leakage on neuroimaging predictive models provides insight into the extent to which leakage may affect the literature. Here, we investigated the effects of leakage on machine learning models in two common neuroimaging modalities, functional and structural connectomes. Using over 400 different pipelines spanning four large datasets and three phenotypes, we evaluated five forms of leakage fitting into three broad categories: feature selection, covariate correction, and lack of independence between subjects. As expected, leakage via feature selection and repeated subjects drastically inflated prediction performance. Notably, other forms of leakage had only minor effects (e.g., leaky site correction) or even decreased prediction performance (e.g., leaky covariate regression). In some cases, leakage affected not only prediction performance, but also model coefficients, and thus neurobiological interpretations. Finally, we found that predictive models using small datasets were more sensitive to leakage. Overall, our results illustrate the variable effects of leakage on prediction pipelines and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Leveraging edge-centric networks complements existing network-level inference for functional connectomes.

The human connectome is modular with distinct brain regions clustering together to form large-scale communities, or networks. This concept has recently been leveraged in novel inferencing procedures by averaging the edge-level statistics within networks to induce more powerful inferencing at the network level. However, these networks are constructed based on the similarity between pairs of nodes. Emerging work has described novel edge-centric networks, which instead use the similarity between pairs of edges to construct networks. In this work, we use these edge-centric networks in a network-level inferencing procedure and compare this novel method to traditional inferential procedures and the network-level procedure using node-centric networks. We use data from the Human Connectome Project, the Healthy Brain Network, and the Philadelphia Neurodevelopmental Cohort and use a resampling technique with various sample sizes (n=40, 80, 120) to probe the power and specificity of each method. Across datasets and sample sizes, using the edge-centric networks outperforms using node-centric networks for inference as well as edge-level FDR correction and NBS. Additionally, the edge-centric networks were found to be more consistent in clustering effect sizes of similar values as compared to node-centric networks, although node-centric networks often had a lower average within-network effect size variability. Together, these findings suggest that using edge-centric networks for network-level inference can procure relatively powerful results while remaining similarly accurate to the underlying edge-level effects across networks, complementing previous inferential methods.