Brain-phenotype models fail for individuals who defy sample stereotypes.

Individual differences in brain functional organization track a range of traits, symptoms and behaviours1-12. So far, work modelling linear brain-phenotype relationships has assumed that a single such relationship generalizes across all individuals, but models do not work equally well in all participants13,14. A better understanding of in whom models fail and why is crucial to revealing robust, useful and unbiased brain-phenotype relationships. To this end, here we related brain activity to phenotype using predictive models-trained and tested on independent data to ensure generalizability15-and examined model failure. We applied this data-driven approach to a range of neurocognitive measures in a new, clinically and demographically heterogeneous dataset, with the results replicated in two independent, publicly available datasets16,17. Across all three datasets, we find that models reflect not unitary cognitive constructs, but rather neurocognitive scores intertwined with sociodemographic and clinical covariates; that is, models reflect stereotypical profiles, and fail when applied to individuals who defy them. Model failure is reliable, phenotype specific and generalizable across datasets. Together, these results highlight the pitfalls of a one-size-fits-all modelling approach and the effect of biased phenotypic measures18-20 on the interpretation and utility of resulting brain-phenotype models. We present a framework to address these issues so that such models may reveal the neural circuits that underlie specific phenotypes and ultimately identify individualized neural targets for clinical intervention.

Improving power in functional magnetic resonance imaging by moving beyond cluster-level inference.

Inference in neuroimaging typically occurs at the level of focal brain areas or circuits. Yet, increasingly, well-powered studies paint a much richer picture of broad-scale effects distributed throughout the brain, suggesting that many focal reports may only reflect the tip of the iceberg of underlying effects. How focal versus broad-scale perspectives influence the inferences we make has not yet been comprehensively evaluated using real data. Here, we compare sensitivity and specificity across procedures representing multiple levels of inference using an empirical benchmarking procedure that resamples task-based connectomes from the Human Connectome Project dataset (∼1,000 subjects, 7 tasks, 3 resampling group sizes, 7 inferential procedures). Only broad-scale (network and whole brain) procedures obtained the traditional 80% statistical power level to detect an average effect, reflecting >20% more statistical power than focal (edge and cluster) procedures. Power also increased substantially for false discovery rate- compared with familywise error rate-controlling procedures. The downsides are fairly limited; the loss in specificity for broad-scale and FDR procedures was relatively modest compared to the gains in power. Furthermore, the broad-scale methods we introduce are simple, fast, and easy to use, providing a straightforward starting point for researchers. This also points to the promise of more sophisticated broad-scale methods for not only functional connectivity but also related fields, including task-based activation. Altogether, this work demonstrates that shifting the scale of inference and choosing FDR control are both immediately attainable and can help remedy the issues with statistical power plaguing typical studies in the field.

Global diversity in individualized cortical network topography.

Individualized cortical network topography (ICNT) varies between people and exhibits great variability in the association networks in the human brain. However, these findings were mainly discovered in Western populations. It remains unclear whether and how ICNT is shaped by the non-Western populations. Here, we leveraged a multisession hierarchical Bayesian model to define individualized functional networks in White American and Han Chinese populations with data from both US and Chinese Human Connectome Projects. We found that both the size and spatial topography of individualized functional networks differed between White American and Han Chinese groups, especially in the heteromodal association cortex (including the ventral attention, control, language, dorsal attention, and default mode networks). Employing a support vector machine, we then demonstrated that ethnicity-related ICNT diversity can be used to identify an individual's ethnicity with high accuracy (74%, pperm < 0.0001), with heteromodal networks contributing most to the classification. This finding was further validated through mass-univariate analyses with generalized additive models. Moreover, we reveal that the spatial heterogeneity of ethnic diversity in ICNT correlated with fundamental properties of cortical organization, including evolutionary cortical expansion, brain myelination, and cerebral blood flow. Altogether, this case study highlights a need for more globally diverse and publicly available neuroimaging datasets.

Large-scale functional brain networks of maladaptive childhood aggression identified by connectome-based predictive modeling.

Disruptions in frontoparietal networks supporting emotion regulation have been long implicated in maladaptive childhood aggression. However, the association of connectivity between large-scale functional networks with aggressive behavior has not been tested. The present study examined whether the functional organization of the connectome predicts severity of aggression in children. This cross-sectional study included a transdiagnostic sample of 100 children with aggressive behavior (27 females) and 29 healthy controls without aggression or psychiatric disorders (13 females). Severity of aggression was indexed by the total score on the parent-rated Reactive-Proactive Aggression Questionnaire. During fMRI, participants completed a face emotion perception task of fearful and calm faces. Connectome-based predictive modeling with internal cross-validation was conducted to identify brain networks that predicted aggression severity. The replication and generalizability of the aggression predictive model was then tested in an independent sample of children from the Adolescent Brain Cognitive Development (ABCD) study. Connectivity predictive of aggression was identified within and between networks implicated in cognitive control (medial-frontal, frontoparietal), social functioning (default mode, salience), and emotion processing (subcortical, sensorimotor) (r = 0.31, RMSE = 9.05, p = 0.005). Out-of-sample replication (p < 0.002) and generalization (p = 0.007) of findings predicting aggression from the functional connectome was demonstrated in an independent sample of children from the ABCD study (n = 1791; n = 1701). Individual differences in large-scale functional networks contribute to variability in maladaptive aggression in children with psychiatric disorders. Linking these individual differences in the connectome to variation in behavioral phenotypes will advance identification of neural biomarkers of maladaptive childhood aggression to inform targeted treatments.

Predicting the future of neuroimaging predictive models in mental health.

Predictive modeling using neuroimaging data has the potential to improve our understanding of the neurobiology underlying psychiatric disorders and putatively information interventions. Accordingly, there is a plethora of literature reviewing published studies, the mathematics underlying machine learning, and the best practices for using these approaches. As our knowledge of mental health and machine learning continue to evolve, we instead aim to look forward and "predict" topics that we believe will be important in current and future studies. Some of the most discussed topics in machine learning, such as bias and fairness, the handling of dirty data, and interpretable models, may be less familiar to the broader community using neuroimaging-based predictive modeling in psychiatry. In a similar vein, transdiagnostic research and targeting brain-based features for psychiatric intervention are modern topics in psychiatry that predictive models are well-suited to tackle. In this work, we target an audience who is a researcher familiar with the fundamental procedures of machine learning and who wishes to increase their knowledge of ongoing topics in the field. We aim to accelerate the utility and applications of neuroimaging-based predictive models for psychiatric research by highlighting and considering these topics. Furthermore, though not a focus, these ideas generalize to neuroimaging-based predictive modeling in other clinical neurosciences and predictive modeling with different data types (e.g., digital health data).

Leveraging edge-centric networks complements existing network-level inference for functional connectomes.

The human connectome is modular with distinct brain regions clustering together to form large-scale communities, or networks. This concept has recently been leveraged in novel inferencing procedures by averaging the edge-level statistics within networks to induce more powerful inferencing at the network level. However, these networks are constructed based on the similarity between pairs of nodes. Emerging work has described novel edge-centric networks, which instead use the similarity between pairs of edges to construct networks. In this work, we use these edge-centric networks in a network-level inferencing procedure and compare this novel method to traditional inferential procedures and the network-level procedure using node-centric networks. We use data from the Human Connectome Project, the Healthy Brain Network, and the Philadelphia Neurodevelopmental Cohort and use a resampling technique with various sample sizes (n=40, 80, 120) to probe the power and specificity of each method. Across datasets and sample sizes, using the edge-centric networks outperforms using node-centric networks for inference as well as edge-level FDR correction and NBS. Additionally, the edge-centric networks were found to be more consistent in clustering effect sizes of similar values as compared to node-centric networks, although node-centric networks often had a lower average within-network effect size variability. Together, these findings suggest that using edge-centric networks for network-level inference can procure relatively powerful results while remaining similarly accurate to the underlying edge-level effects across networks, complementing previous inferential methods.

Large-scale differences in functional organization of left- and right-handed individuals using whole-brain, data-driven analysis of connectivity.

Handedness influences differences in lateralization of language areas as well as dominance of motor and somatosensory cortices. However, differences in whole-brain functional connectivity (i.e., functional connectomes) due to handedness have been relatively understudied beyond pre-specified networks of interest. Here, we compared functional connectomes of left- and right-handed individuals at the whole brain level. We explored differences in functional connectivity of previously established regions of interest, and showed differences between primarily left- and primarily right-handed individuals in the motor, somatosensory, and language areas using functional connectivity. We then proceeded to investigate these differences in the whole brain and found that the functional connectivity of left- and right-handed individuals are not specific to networks of interest, but extend across every region of the brain. In particular, we found that connections between and within the cerebellum show distinct patterns of connectivity. To put these effects into context, we show that the effect sizes associated with handedness differences account for a similar amount of individual differences in the connectome as sex differences. Together these results shed light on regions of the brain beyond those traditionally explored that contribute to differences in the functional organization of left- and right-handed individuals and underscore that handedness effects are neurobiologically meaningful in addition to being statistically significant.

Cetuximab and Irinotecan With or Without Bevacizumab in Refractory Metastatic Colorectal Cancer: BOND-3, an ACCRU Network Randomized Clinical Trial.

BACKGROUND: Combination irinotecan and cetuximab is approved for irinotecan-refractory metastatic colorectal cancer (mCRC). It is unknown if adding bevacizumab improves outcomes. PATIENTS AND METHODS: In this multicenter, randomized, double-blind, placebo-controlled phase II trial, patients with irinotecan-refractory RAS-wildtype mCRC and no prior anti-EGFR therapy were randomized to cetuximab 500 mg/m2, bevacizumab 5 mg/kg, and irinotecan 180 mg/m2 (or previously tolerated dose) (CBI) versus cetuximab, irinotecan, and placebo (CI) every 2 weeks until disease progression or intolerable toxicity. The primary endpoint was progression-free survival (PFS). Secondary endpoints included overall survival (OS), objective response rate (ORR), and adverse events (AEs). RESULTS: The study closed early after the accrual of 36 out of a planned 120 patients due to changes in funding. Nineteen patients were randomized to CBI and 17 to CI. Baseline characteristics were similar between arms. Median PFS was 9.7 versus 5.5 months for CBI and CI, respectively (1-sided log-rank P = .38; adjusted hazard ratio [HR] = 0.64; 95% confidence interval [CI], 0.25-1.66). Median OS was 19.7 versus 10.2 months for CBI and CI (1-sided log-rank P = .02; adjusted HR = 0.41; 95% CI, 0.15-1.09). ORR was 36.8% for CBI versus 11.8% for CI (P = .13). Grade 3 or higher AEs occurred in 47% of patients receiving CBI versus 35% for CI (P = .46). CONCLUSION: In this prematurely discontinued trial, there was no significant difference in the primary endpoint of PFS between CBI and CI. There was a statistically significant improvement in OS in favor of CBI compared with CI. Further investigation of CBI for the treatment of irinotecan-refractory mCRC is warranted.Clinical Trial Registration: NCT02292758.

Associations between grip strength, brain structure, and mental health in > 40,000 participants from the UK Biobank.

BACKGROUND: Grip strength is a widely used and well-validated measure of overall health that is increasingly understood to index risk for psychiatric illness and neurodegeneration in older adults. However, existing work has not examined how grip strength relates to a comprehensive set of mental health outcomes, which can detect early signs of cognitive decline. Furthermore, whether brain structure mediates associations between grip strength and cognition remains unknown. METHODS: Based on cross-sectional and longitudinal data from over 40,000 participants in the UK Biobank, this study investigated the behavioral and neural correlates of handgrip strength using a linear mixed effect model and mediation analysis. RESULTS: In cross-sectional analysis, we found that greater grip strength was associated with better cognitive functioning, higher life satisfaction, greater subjective well-being, and reduced depression and anxiety symptoms while controlling for numerous demographic, anthropometric, and socioeconomic confounders. Further, grip strength of females showed stronger associations with most behavioral outcomes than males. In longitudinal analysis, baseline grip strength was related to cognitive performance at ~9 years follow-up, while the reverse effect was much weaker. Further, baseline neuroticism, health, and financial satisfaction were longitudinally associated with subsequent grip strength. The results revealed widespread associations between stronger grip strength and increased grey matter volume, especially in subcortical regions and temporal cortices. Moreover, grey matter volume of these regions also correlated with better mental health and considerably mediated their relationship with grip strength. CONCLUSIONS: Overall, using the largest population-scale neuroimaging dataset currently available, our findings provide the most well-powered characterization of interplay between grip strength, mental health, and brain structure, which may facilitate the discovery of possible interventions to mitigate cognitive decline during aging.

Eliminating accidental deviations to minimize generalization error and maximize replicability: Applications in connectomics and genomics.

Replicability, the ability to replicate scientific findings, is a prerequisite for scientific discovery and clinical utility. Troublingly, we are in the midst of a replicability crisis. A key to replicability is that multiple measurements of the same item (e.g., experimental sample or clinical participant) under fixed experimental constraints are relatively similar to one another. Thus, statistics that quantify the relative contributions of accidental deviations-such as measurement error-as compared to systematic deviations-such as individual differences-are critical. We demonstrate that existing replicability statistics, such as intra-class correlation coefficient and fingerprinting, fail to adequately differentiate between accidental and systematic deviations in very simple settings. We therefore propose a novel statistic, discriminability, which quantifies the degree to which an individual's samples are relatively similar to one another, without restricting the data to be univariate, Gaussian, or even Euclidean. Using this statistic, we introduce the possibility of optimizing experimental design via increasing discriminability and prove that optimizing discriminability improves performance bounds in subsequent inference tasks. In extensive simulated and real datasets (focusing on brain imaging and demonstrating on genomics), only optimizing data discriminability improves performance on all subsequent inference tasks for each dataset. We therefore suggest that designing experiments and analyses to optimize discriminability may be a crucial step in solving the replicability crisis, and more generally, mitigating accidental measurement error.

Transdiagnostic, Connectome-Based Prediction of Memory Constructs Across Psychiatric Disorders.

Memory deficits are observed in a range of psychiatric disorders, but it is unclear whether memory deficits arise from a shared brain correlate across disorders or from various dysfunctions unique to each disorder. Connectome-based predictive modeling is a computational method that captures individual differences in functional connectomes associated with behavioral phenotypes such as memory. We used publicly available task-based functional MRI data from patients with schizophrenia (n = 33), bipolar disorder (n = 34), attention deficit hyper-activity disorder (n = 32), and healthy controls (n = 73) to model the macroscale brain networks associated with working, short- and long-term memory. First, we use 10-fold and leave-group-out analyses to demonstrate that the same macroscale brain networks subserve memory across diagnostic groups and that individual differences in memory performance are related to individual differences within networks distributed throughout the brain, including the subcortex, default mode network, limbic network, and cerebellum. Next, we show that diagnostic groups are associated with significant differences in whole-brain functional connectivity that are distinct from the predictive models of memory. Finally, we show that models trained on the transdiagnostic sample generalize to novel, healthy participants (n = 515) from the Human Connectome Project. These results suggest that despite significant differences in whole-brain patterns of functional connectivity between diagnostic groups, the core macroscale brain networks that subserve memory are shared.

Cluster failure or power failure? Evaluating sensitivity in cluster-level inference.

Pioneering work in human neuroscience has relied on the ability to map brain function using task-based fMRI, but the empirical validity of these inferential methods is still being characterized. A recent landmark study by Eklund and colleagues showed that popular multiple comparison corrections based on cluster extent suffer from unexpectedly low specificity (i.e., high false positive rate). Yet that study's focus on specificity, while important, is incomplete. The validity of a method depends also on its sensitivity (i.e., true positive rate or power), yet the sensitivity of cluster correction remains poorly understood. Here, we assessed the sensitivity of gold-standard nonparametric cluster correction by resampling real data from five tasks in the Human Connectome Project and comparing results with those from the full "ground truth" datasets (n â€‹= â€‹480-493). Critically, we found that sensitivity after correction is lower than may be practical for many fMRI applications. In particular, sensitivity to medium-sized effects (|Cohen's d| â€‹= â€‹0.5) was less than 20% across tasks on average, about three times smaller than without any correction. Furthermore, cluster extent correction exhibited a spatial bias in sensitivity that was independent of effect size. In comparison, correction based on the Threshold-Free Cluster Enhancement (TFCE) statistic approximately doubled sensitivity across tasks but increased spatial bias. These results suggest that we have, until now, only measured the tip of the iceberg in the activation-mapping literature due to our goal of limiting the familywise error rate through cluster extent-based inference. There is a need to revise our practices to improve sensitivity; we therefore conclude with a list of modern strategies to boost sensitivity while maintaining respectable specificity in future investigations.

A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis.

BACKGROUND: Once considered mere noise, fMRI-based functional connectivity has become a major neuroscience tool in part due to early studies demonstrating its reliability. These fundamental studies revealed only the tip of the iceberg; over the past decade, many test-retest reliability studies have continued to add nuance to our understanding of this complex topic. A summary of these diverse and at times contradictory perspectives is needed. OBJECTIVES: We aimed to summarize the existing knowledge regarding test-retest reliability of functional connectivity at the most basic unit of analysis: the individual edge level. This entailed (1) a meta-analytic estimate of reliability and (2) a review of factors influencing reliability. METHODS: A search of Scopus was conducted to identify studies that estimated edge-level test-retest reliability. To facilitate comparisons across studies, eligibility was restricted to studies measuring reliability via the intraclass correlation coefficient (ICC). The meta-analysis included a random effects pooled estimate of mean edge-level ICC, with studies nested within datasets. The review included a narrative summary of factors influencing edge-level ICC. RESULTS: From an initial pool of 212 studies, 44 studies were identified for the qualitative review and 25 studies for quantitative meta-analysis. On average, individual edges exhibited a "poor" ICC of 0.29 (95% CI = 0.23 to 0.36). The most reliable measurements tended to involve: (1) stronger, within-network, cortical edges, (2) eyes open, awake, and active recordings, (3) more within-subject data, (4) shorter test-retest intervals, (5) no artifact correction (likely due in part to reliable artifact), and (6) full correlation-based connectivity with shrinkage. CONCLUSION: This study represents the first meta-analysis and systematic review investigating test-retest reliability of edge-level functional connectivity. Key findings suggest there is room for improvement, but care should be taken to avoid promoting reliability at the expense of validity. By pooling existing knowledge regarding this key facet of accuracy, this study supports broader efforts to improve inferences in the field.

Multivariate approaches improve the reliability and validity of functional connectivity and prediction of individual behaviors.

Brain functional connectivity features can predict cognition and behavior at the level of the individual. Most studies measure univariate signals, correlating timecourses from the average of constituent voxels in each node. While straightforward, this approach overlooks the spatial patterns of voxel-wise signals within individual nodes. Given that multivariate spatial activity patterns across voxels can improve fMRI measures of mental representations, here we asked whether using voxel-wise timecourses can better characterize region-by-region interactions relative to univariate approaches. Using two fMRI datasets, the Human Connectome Project sample and a local test-retest sample, we measured multivariate functional connectivity with multivariate distance correlation and univariate connectivity with Pearson's correlation. We compared multivariate and univariate connectivity estimates, demonstrating that relative to univariate estimates, multivariate estimates exhibited higher reliability at both the edge-level and connectome-level, stronger prediction of individual differences, and greater sensitivity to brain states within individuals. Our findings suggest that multivariate estimates reliably provide more powerful information about an individual's functional brain organization and its relation to cognitive skills.

Ten simple rules for predictive modeling of individual differences in neuroimaging.

Establishing brain-behavior associations that map brain organization to phenotypic measures and generalize to novel individuals remains a challenge in neuroimaging. Predictive modeling approaches that define and validate models with independent datasets offer a solution to this problem. While these methods can detect novel and generalizable brain-behavior associations, they can be daunting, which has limited their use by the wider connectivity community. Here, we offer practical advice and examples based on functional magnetic resonance imaging (fMRI) functional connectivity data for implementing these approaches. We hope these ten rules will increase the use of predictive models with neuroimaging data.

Considering factors affecting the connectome-based identification process: Comment on Waller et al.

A recent study by Waller and colleagues evaluated the reliability, specificity, and generalizability of using functional connectivity data to identify individuals from a group. The authors note they were able to replicate identification rates in a larger version of the original Human Connectome Project (HCP) dataset. However, they also report lower identification accuracies when using historical neuroimaging acquisitions with low spatial and temporal resolution. The authors suggest that their results indicate connectomes derived from historical imaging data may be similar across individuals, to the extent that this connectome-based approach may be inappropriate for precision psychiatry and the goal of drawing inferences based on subject-level data. Here we note that the authors did not take into account factors affecting data quality and hence identification rates, independent of whether a low spatiotemporal resolution acquisition or a high spatiotemporal resolution acquisition is used. Specifically, we show here that the amount of data collected per subject and in-scanner motion are the predominant factors influencing identification rates, not the spatiotemporal resolution of the acquisition. To do this, we investigated identification rates in the HCP dataset as a function of the amount of data and motion. Using a dataset from the Consortium for Reliability and Reproducibility (CoRR), we investigated the impact of multiband versus non-multiband imaging parameters; that is, high spatiotemporal resolution versus low spatiotemporal resolution acquisitions. We show scan length and motion affect identification, whereas the imaging protocol does not affect these rates. Our results suggest that motion and amount of data per subject are the primary factors impacting individual connectivity profiles, but that within these constraints, individual differences in the connectome are readily observable.

Influences on the Test-Retest Reliability of Functional Connectivity MRI and its Relationship with Behavioral Utility.

Best practices are currently being developed for the acquisition and processing of resting-state magnetic resonance imaging data used to estimate brain functional organization-or "functional connectivity." Standards have been proposed based on test-retest reliability, but open questions remain. These include how amount of data per subject influences whole-brain reliability, the influence of increasing runs versus sessions, the spatial distribution of reliability, the reliability of multivariate methods, and, crucially, how reliability maps onto prediction of behavior. We collected a dataset of 12 extensively sampled individuals (144 min data each across 2 identically configured scanners) to assess test-retest reliability of whole-brain connectivity within the generalizability theory framework. We used Human Connectome Project data to replicate these analyses and relate reliability to behavioral prediction. Overall, the historical 5-min scan produced poor reliability averaged across connections. Increasing the number of sessions was more beneficial than increasing runs. Reliability was lowest for subcortical connections and highest for within-network cortical connections. Multivariate reliability was greater than univariate. Finally, reliability could not be used to improve prediction; these findings are among the first to underscore this distinction for functional connectivity. A comprehensive understanding of test-retest reliability, including its limitations, supports the development of best practices in the field.

Multisite reliability of MR-based functional connectivity.

Recent years have witnessed an increasing number of multisite MRI functional connectivity (fcMRI) studies. While multisite studies provide an efficient way to accelerate data collection and increase sample sizes, especially for rare clinical populations, any effects of site or MRI scanner could ultimately limit power and weaken results. Little data exists on the stability of functional connectivity measurements across sites and sessions. In this study, we assess the influence of site and session on resting state functional connectivity measurements in a healthy cohort of traveling subjects (8 subjects scanned twice at each of 8 sites) scanned as part of the North American Prodrome Longitudinal Study (NAPLS). Reliability was investigated in three types of connectivity analyses: (1) seed-based connectivity with posterior cingulate cortex (PCC), right motor cortex (RMC), and left thalamus (LT) as seeds; (2) the intrinsic connectivity distribution (ICD), a voxel-wise connectivity measure; and (3) matrix connectivity, a whole-brain, atlas-based approach to assessing connectivity between nodes. Contributions to variability in connectivity due to subject, site, and day-of-scan were quantified and used to assess between-session (test-retest) reliability in accordance with Generalizability Theory. Overall, no major site, scanner manufacturer, or day-of-scan effects were found for the univariate connectivity analyses; instead, subject effects dominated relative to the other measured factors. However, summaries of voxel-wise connectivity were found to be sensitive to site and scanner manufacturer effects. For all connectivity measures, although subject variance was three times the site variance, the residual represented 60-80% of the variance, indicating that connectivity differed greatly from scan to scan independent of any of the measured factors (i.e., subject, site, and day-of-scan). Thus, for a single 5min scan, reliability across connectivity measures was poor (ICC=0.07-0.17), but increased with increasing scan duration (ICC=0.21-0.36 at 25min). The limited effects of site and scanner manufacturer support the use of multisite studies, such as NAPLS, as a viable means of collecting data on rare populations and increasing power in univariate functional connectivity studies. However, the results indicate that aggregation of fcMRI data across longer scan durations is necessary to increase the reliability of connectivity estimates at the single-subject level.

Presurgical language fMRI: Mapping of six critical regions.

Language mapping is a key goal in neurosurgical planning. fMRI mapping typically proceeds with a focus on Broca's and Wernicke's areas, although multiple other language-critical areas are now well-known. We evaluated whether clinicians could use a novel approach, including clinician-driven individualized thresholding, to reliably identify six language regions, including Broca's Area, Wernicke's Area (inferior, superior), Exner's Area, Supplementary Speech Area, Angular Gyrus, and Basal Temporal Language Area. We studied 22 epilepsy and tumor patients who received Wada and fMRI (age 36.4[12.5]; Wada language left/right/mixed in 18/3/1). fMRI tasks (two × three tasks) were analyzed by two clinical neuropsychologists who flexibly thresholded and combined these to identify the six regions. The resulting maps were compared to fixed threshold maps. Clinicians generated maps that overlapped significantly, and were highly consistent, when at least one task came from the same set. Cases diverged when clinicians prioritized different language regions or addressed noise differently. Language laterality closely mirrored Wada data (85% accuracy). Activation consistent with all six language regions was consistently identified. In blind review, three external, independent clinicians rated the individualized fMRI language maps as superior to fixed threshold maps; identified the majority of regions significantly more frequently; and judged language laterality to mirror Wada lateralization more often. These data provide initial validation of a novel, clinician-based approach to localizing language cortex. They also demonstrate clinical fMRI is superior when analyzed by an experienced clinician and that when fMRI data is of low quality judgments of laterality are unreliable and should be withheld. Hum Brain Mapp 38:4239-4255, 2017. © 2017 Wiley Periodicals, Inc.