Brain-phenotype predictions can survive across diverse real-world data.

Recent work suggests that machine learning models predicting psychiatric treatment outcomes based on clinical data may fail when applied to unharmonized samples. Neuroimaging predictive models offer the opportunity to incorporate neurobiological information, which may be more robust to dataset shifts. Yet, among the minority of neuroimaging studies that undertake any form of external validation, there is a notable lack of attention to generalization across dataset-specific idiosyncrasies. Research settings, by design, remove the between-site variations that real-world and, eventually, clinical applications demand. Here, we rigorously test the ability of a range of predictive models to generalize across three diverse, unharmonized samples: the Philadelphia Neurodevelopmental Cohort (n=1291), the Healthy Brain Network (n=1110), and the Human Connectome Project in Development (n=428). These datasets have high inter-dataset heterogeneity, encompassing substantial variations in age distribution, sex, racial and ethnic minority representation, recruitment geography, clinical symptom burdens, fMRI tasks, sequences, and behavioral measures. We demonstrate that reproducible and generalizable brain-behavior associations can be realized across diverse dataset features with sample sizes in the hundreds. Results indicate the potential of functional connectivity-based predictive models to be robust despite substantial inter-dataset variability. Notably, for the HCPD and HBN datasets, the best predictions were not from training and testing in the same dataset (i.e., cross-validation) but across datasets. This result suggests that training on diverse data may improve prediction in specific cases. Overall, this work provides a critical foundation for future work evaluating the generalizability of neuroimaging predictive models in real-world scenarios and clinical settings.

Power and reproducibility in the external validation of brain-phenotype predictions.

Brain-phenotype predictive models seek to identify reproducible and generalizable brain-phenotype associations. External validation, or the evaluation of a model in external datasets, is the gold standard in evaluating the generalizability of models in neuroimaging. Unlike typical studies, external validation involves two sample sizes: the training and the external sample sizes. Thus, traditional power calculations may not be appropriate. Here we ran over 900 million resampling-based simulations in functional and structural connectivity data to investigate the relationship between training sample size, external sample size, phenotype effect size, theoretical power and simulated power. Our analysis included a wide range of datasets: the Healthy Brain Network, the Adolescent Brain Cognitive Development Study, the Human Connectome Project (Development and Young Adult), the Philadelphia Neurodevelopmental Cohort, the Queensland Twin Adolescent Brain Project, and the Chinese Human Connectome Project; and phenotypes: age, body mass index, matrix reasoning, working memory, attention problems, anxiety/depression symptoms and relational processing. High effect size predictions achieved adequate power with training and external sample sizes of a few hundred individuals, whereas low and medium effect size predictions required hundreds to thousands of training and external samples. In addition, most previous external validation studies used sample sizes prone to low power, and theoretical power curves should be adjusted for the training sample size. Furthermore, model performance in internal validation often informed subsequent external validation performance (Pearson's r difference <0.2), particularly for well-harmonized datasets. These results could help decide how to power future external validation studies.

Ketamine induces multiple individually distinct whole-brain functional connectivity signatures.

Background: Ketamine has emerged as one of the most promising therapies for treatment-resistant depression. However, inter-individual variability in response to ketamine is still not well understood and it is unclear how ketamine's molecular mechanisms connect to its neural and behavioral effects. Methods: We conducted a single-blind placebo-controlled study, with participants blinded to their treatment condition. 40 healthy participants received acute ketamine (initial bolus 0.23 mg/kg, continuous infusion 0.58 mg/kg/hr). We quantified resting-state functional connectivity via data-driven global brain connectivity and related it to individual ketamine-induced symptom variation and cortical gene expression targets. Results: We found that: (i) both the neural and behavioral effects of acute ketamine are multi-dimensional, reflecting robust inter-individual variability; (ii) ketamine's data-driven principal neural gradient effect matched somatostatin (SST) and parvalbumin (PVALB) cortical gene expression patterns in humans, while the mean effect did not; and (iii) behavioral data-driven individual symptom variation mapped onto distinct neural gradients of ketamine, which were resolvable at the single-subject level. Conclusions: These results highlight the importance of considering individual behavioral and neural variation in response to ketamine. They also have implications for the development of individually precise pharmacological biomarkers for treatment selection in psychiatry. Funding: This study was supported by NIH grants DP5OD012109-01 (A.A.), 1U01MH121766 (A.A.), R01MH112746 (J.D.M.), 5R01MH112189 (A.A.), 5R01MH108590 (A.A.), NIAAA grant 2P50AA012870-11 (A.A.); NSF NeuroNex grant 2015276 (J.D.M.); Brain and Behavior Research Foundation Young Investigator Award (A.A.); SFARI Pilot Award (J.D.M., A.A.); Heffter Research Institute (Grant No. 1-190420) (FXV, KHP); Swiss Neuromatrix Foundation (Grant No. 2016-0111) (FXV, KHP); Swiss National Science Foundation under the framework of Neuron Cofund (Grant No. 01EW1908) (KHP); Usona Institute (2015 - 2056) (FXV). Clinical trial number: NCT03842800.

Ketamine is a widely used anesthetic as well as a popular illegal recreational drug. Recently, it has also gained attention as a potential treatment for depression, particularly in cases that don't respond to conventional therapies. However, individuals can vary in their response to ketamine. For example, the drug can alter some people's perception, such as seeing objects as larger or small than they are, while other individuals are unaffected. Although a single dose of ketamine was shown to improve depression symptoms in approximately 65% of patients, the treatment does not work for a significant portion of patients. Understanding why ketamine does not work for everyone could help to identify which patients would benefit most from the treatment. Previous studies investigating ketamine as a treatment for depression have typically included a group of individuals given ketamine and a group given a placebo drug. Assuming people respond similarly to ketamine, the responses in each group were averaged and compared to one another. However, this averaging of results may have masked any individual differences in response to ketamine. As a result, Moujaes et al. set out to investigate whether individuals show differences in brain activity and behavior in response to ketamine. Moujaes et al. monitored the brain activity and behavior of 40 healthy individuals that were first given a placebo drug and then ketamine. The results showed that brain activity and behavior varied significantly between individuals after ketamine administration. Genetic analysis revealed that different gene expression patterns paired with differences in ketamine response in individuals ­ an effect that was hidden when the results were averaged. Ketamine also caused greater differences in brain activity and behavior between individuals than other drugs, such as psychedelics, suggesting ketamine generates a particularly complex response in people. In the future, extending these findings in healthy individuals to those with depression will be crucial for determining whether differences in response to ketamine align with how effective ketamine treatment is for an individual.