The neuroconnectionist research programme.

Artificial neural networks (ANNs) inspired by biology are beginning to be widely used to model behavioural and neural data, an approach we call 'neuroconnectionism'. ANNs have been not only lauded as the current best models of information processing in the brain but also criticized for failing to account for basic cognitive functions. In this Perspective article, we propose that arguing about the successes and failures of a restricted set of current ANNs is the wrong approach to assess the promise of neuroconnectionism for brain science. Instead, we take inspiration from the philosophy of science, and in particular from Lakatos, who showed that the core of a scientific research programme is often not directly falsifiable but should be assessed by its capacity to generate novel insights. Following this view, we present neuroconnectionism as a general research programme centred around ANNs as a computational language for expressing falsifiable theories about brain computation. We describe the core of the programme, the underlying computational framework and its tools for testing specific neuroscientific hypotheses and deriving novel understanding. Taking a longitudinal view, we review past and present neuroconnectionist projects and their responses to challenges and argue that the research programme is highly progressive, generating new and otherwise unreachable insights into the workings of the brain.

Phenotype based prediction of exome sequencing outcome using machine learning for neurodevelopmental disorders.

PURPOSE: Although the introduction of exome sequencing (ES) has led to the diagnosis of a significant portion of patients with neurodevelopmental disorders (NDDs), the diagnostic yield in actual clinical practice has remained stable at approximately 30%. We hypothesized that improving the selection of patients to test on the basis of their phenotypic presentation will increase diagnostic yield and therefore reduce unnecessary genetic testing. METHODS: We tested 4 machine learning methods and developed PredWES from these: a statistical model predicting the probability of a positive ES result solely on the basis of the phenotype of the patient. RESULTS: We first trained the tool on 1663 patients with NDDs and subsequently showed that diagnostic ES on the top 10% of patients with the highest probability of a positive ES result would provide a diagnostic yield of 56%, leading to a notable 114% increase. Inspection of our model revealed that for patients with NDDs, comorbid abnormal (lower) muscle tone and microcephaly positively correlated with a conclusive ES diagnosis, whereas autism was negatively associated with a molecular diagnosis. CONCLUSION: In conclusion, PredWES allows prioritizing patients with NDDs eligible for diagnostic ES on the basis of their phenotypic presentation to increase the diagnostic yield, making a more efficient use of health care resources.

Brain-optimized extraction of complex sound features that drive continuous auditory perception.

Understanding how the human brain processes auditory input remains a challenge. Traditionally, a distinction between lower- and higher-level sound features is made, but their definition depends on a specific theoretical framework and might not match the neural representation of sound. Here, we postulate that constructing a data-driven neural model of auditory perception, with a minimum of theoretical assumptions about the relevant sound features, could provide an alternative approach and possibly a better match to the neural responses. We collected electrocorticography recordings from six patients who watched a long-duration feature film. The raw movie soundtrack was used to train an artificial neural network model to predict the associated neural responses. The model achieved high prediction accuracy and generalized well to a second dataset, where new participants watched a different film. The extracted bottom-up features captured acoustic properties that were specific to the type of sound and were associated with various response latency profiles and distinct cortical distributions. Specifically, several features encoded speech-related acoustic properties with some features exhibiting shorter latency profiles (associated with responses in posterior perisylvian cortex) and others exhibiting longer latency profiles (associated with responses in anterior perisylvian cortex). Our results support and extend the current view on speech perception by demonstrating the presence of temporal hierarchies in the perisylvian cortex and involvement of cortical sites outside of this region during audiovisual speech perception.

Validation of structural brain connectivity networks: The impact of scanning parameters.

Evaluation of the structural connectivity (SC) of the brain based on tractography has mainly focused on the choice of diffusion model, tractography algorithm, and their respective parameter settings. Here, we systematically validate SC derived from a post mortem monkey brain, while varying key acquisition parameters such as the b-value, gradient angular resolution and image resolution. As gold standard we use the connectivity matrix obtained invasively with histological tracers by Markov et al. (2014). As performance metric, we use cross entropy as a measure that enables comparison of the relative tracer labeled neuron counts to the streamline counts from tractography. We find that high angular resolution and high signal-to-noise ratio are important to estimate SC, and that SC derived from low image resolution (1.03 mm3) are in better agreement with the tracer network, than those derived from high image resolution (0.53 mm3) or at an even lower image resolution (2.03 mm3). In contradiction, sensitivity and specificity analyses suggest that if the angular resolution is sufficient, the balanced compromise in which sensitivity and specificity are identical remains 60-64% regardless of the other scanning parameters. Interestingly, the tracer graph is assumed to be the gold standard but by thresholding, the balanced compromise increases to 70-75%. Hence, by using performance metrics based on binarized tracer graphs, one risks losing important information, changing the performance of SC graphs derived by tractography and their dependence of different scanning parameters.

Distinct directional couplings between slow and fast gamma power to the phase of theta oscillations in the rat hippocampus.

It is well-established that theta (~4-10 Hz) and gamma (~25-100 Hz) oscillations interact in the rat hippocampus. This cross-frequency coupling might facilitate neuronal coordination both within and between brain areas. However, it remains unclear whether the phase of theta oscillations controls the power of slow and fast gamma activity or vice versa. We here applied spectral Granger causality, phase slope index and a newly developed cross-frequency directionality (CFD) measure to investigate directional interactions between local field potentials recorded within and across hippocampal subregions of CA1 and CA3 of freely exploring rats. Given the well-known CA3 to CA1 anatomical connection, we hypothesized that interregional directional interactions were constrained by anatomical connection, and within-frequency and cross-frequency directional interactions were always from CA3 to CA1. As expected, we found that CA3 drove CA1 in the theta band, and theta phase-to-gamma power coupling was prominent both within and between CA3 and CA1 regions. The CFD measure further demonstrated that distinct directional couplings with respect to theta phase was different between slow and fast gamma activity. Importantly, CA3 slow gamma power phase-adjusted CA1 theta oscillations, suggesting that slow gamma activity in CA3 entrains theta oscillations in CA1. In contrast, CA3 theta phase controls CA1 fast gamma activity, indicating that communication at CA1 fast gamma is coordinated by CA3 theta phase. Overall, these findings demonstrate dynamic directional interactions between theta and slow/fast gamma oscillations in the hippocampal network, suggesting that anatomical connections constrain the directional interactions.

Eye movements explain decodability during perception and cued attention in MEG.

Eye movements are an integral part of human perception, but can induce artifacts in many magneto-encephalography (MEG) and electroencephalography (EEG) studies. For this reason, investigators try to minimize eye movements and remove these artifacts from their data using different techniques. When these artifacts are not purely random, but consistent regarding certain stimuli or conditions, the possibility arises that eye movements are actually inducing effects in the MEG signal. It remains unclear how much of an influence eye movements can have on observed effects in MEG, since most MEG studies lack a control analysis to verify whether an effect found in the MEG signal is induced by eye movements. Here, we find that we can decode stimulus location from eye movements in two different stages of a working memory match-to-sample task that encompass different areas of research typically done with MEG. This means that the observed MEG effect might be (partly) due to eye movements instead of any true neural correlate. We suggest how to check for eye movement effects in the data and make suggestions on how to minimize eye movement artifacts from occurring in the first place.

Vividness of Visual Imagery Depends on the Neural Overlap with Perception in Visual Areas.

Research into the neural correlates of individual differences in imagery vividness point to an important role of the early visual cortex. However, there is also great fluctuation of vividness within individuals, such that only looking at differences between people necessarily obscures the picture. In this study, we show that variation in moment-to-moment experienced vividness of visual imagery, within human subjects, depends on the activity of a large network of brain areas, including frontal, parietal, and visual areas. Furthermore, using a novel multivariate analysis technique, we show that the neural overlap between imagery and perception in the entire visual system correlates with experienced imagery vividness. This shows that the neural basis of imagery vividness is much more complicated than studies of individual differences seemed to suggest. SIGNIFICANCE STATEMENT: Visual imagery is the ability to visualize objects that are not in our direct line of sight: something that is important for memory, spatial reasoning, and many other tasks. It is known that the better people are at visual imagery, the better they can perform these tasks. However, the neural correlates of moment-to-moment variation in visual imagery remain unclear. In this study, we show that the more the neural response during imagery is similar to the neural response during perception, the more vivid or perception-like the imagery experience is.

Neural Tuning to Low-Level Features of Speech throughout the Perisylvian Cortex.

Despite a large body of research, we continue to lack a detailed account of how auditory processing of continuous speech unfolds in the human brain. Previous research showed the propagation of low-level acoustic features of speech from posterior superior temporal gyrus toward anterior superior temporal gyrus in the human brain (Hullett et al., 2016). In this study, we investigate what happens to these neural representations past the superior temporal gyrus and how they engage higher-level language processing areas such as inferior frontal gyrus. We used low-level sound features to model neural responses to speech outside of the primary auditory cortex. Two complementary imaging techniques were used with human participants (both males and females): electrocorticography (ECoG) and fMRI. Both imaging techniques showed tuning of the perisylvian cortex to low-level speech features. With ECoG, we found evidence of propagation of the temporal features of speech sounds along the ventral pathway of language processing in the brain toward inferior frontal gyrus. Increasingly coarse temporal features of speech spreading from posterior superior temporal cortex toward inferior frontal gyrus were associated with linguistic features such as voice onset time, duration of the formant transitions, and phoneme, syllable, and word boundaries. The present findings provide the groundwork for a comprehensive bottom-up account of speech comprehension in the human brain.SIGNIFICANCE STATEMENT We know that, during natural speech comprehension, a broad network of perisylvian cortical regions is involved in sound and language processing. Here, we investigated the tuning to low-level sound features within these regions using neural responses to a short feature film. We also looked at whether the tuning organization along these brain regions showed any parallel to the hierarchy of language structures in continuous speech. Our results show that low-level speech features propagate throughout the perisylvian cortex and potentially contribute to the emergence of "coarse" speech representations in inferior frontal gyrus typically associated with high-level language processing. These findings add to the previous work on auditory processing and underline a distinctive role of inferior frontal gyrus in natural speech comprehension.

Dynamic decomposition of spatiotemporal neural signals.

Neural signals are characterized by rich temporal and spatiotemporal dynamics that reflect the organization of cortical networks. Theoretical research has shown how neural networks can operate at different dynamic ranges that correspond to specific types of information processing. Here we present a data analysis framework that uses a linearized model of these dynamic states in order to decompose the measured neural signal into a series of components that capture both rhythmic and non-rhythmic neural activity. The method is based on stochastic differential equations and Gaussian process regression. Through computer simulations and analysis of magnetoencephalographic data, we demonstrate the efficacy of the method in identifying meaningful modulations of oscillatory signals corrupted by structured temporal and spatiotemporal noise. These results suggest that the method is particularly suitable for the analysis and interpretation of complex temporal and spatiotemporal neural signals.

The missing link: Predicting connectomes from noisy and partially observed tract tracing data.

Our understanding of the wiring map of the brain, known as the connectome, has increased greatly in the last decade, mostly due to technological advancements in neuroimaging techniques and improvements in computational tools to interpret the vast amount of available data. Despite this, with the exception of the C. elegans roundworm, no definitive connectome has been established for any species. In order to obtain this, tracer studies are particularly appealing, as these have proven highly reliable. The downside of tract tracing is that it is costly to perform, and can only be applied ex vivo. In this paper, we suggest that instead of probing all possible connections, hitherto unknown connections may be predicted from the data that is already available. Our approach uses a 'latent space model' that embeds the connectivity in an abstract physical space. Regions that are close in the latent space have a high chance of being connected, while regions far apart are most likely disconnected in the connectome. After learning the latent embedding from the connections that we did observe, the latent space allows us to predict connections that have not been probed previously. We apply the methodology to two connectivity data sets of the macaque, where we demonstrate that the latent space model is successful in predicting unobserved connectivity, outperforming two baselines and an alternative model in nearly all cases. Furthermore, we show how the latent spatial embedding may be used to integrate multimodal observations (i.e. anterograde and retrograde tracers) for the mouse neocortex. Finally, our probabilistic approach enables us to make explicit which connections are easy to predict and which prove difficult, allowing for informed follow-up studies.

Correction: The missing link: Predicting connectomes from noisy and partially observed tract tracing data.

[This corrects the article DOI: 10.1371/journal.pcbi.1005374.].

Increasingly complex representations of natural movies across the dorsal stream are shared between subjects.

Recently, deep neural networks (DNNs) have been shown to provide accurate predictions of neural responses across the ventral visual pathway. We here explore whether they also provide accurate predictions of neural responses across the dorsal visual pathway, which is thought to be devoted to motion processing and action recognition. This is achieved by training deep neural networks to recognize actions in videos and subsequently using them to predict neural responses while subjects are watching natural movies. Moreover, we explore whether dorsal stream representations are shared between subjects. In order to address this question, we examine if individual subject predictions can be made in a common representational space estimated via hyperalignment. Results show that a DNN trained for action recognition can be used to accurately predict how dorsal stream responds to natural movies, revealing a correspondence in representations of DNN layers and dorsal stream areas. It is also demonstrated that models operating in a common representational space can generalize to responses of multiple or even unseen individual subjects to novel spatio-temporal stimuli in both encoding and decoding settings, suggesting that a common representational space underlies dorsal stream responses across multiple subjects.

Abnormal connectivity in the sensorimotor network predicts attention deficits in traumatic brain injury.

The aim of this study was to explore modifications of functional connectivity in multiple resting-state networks (RSNs) after moderate to severe traumatic brain injury (TBI) and evaluate the relationship between functional connectivity patterns and cognitive abnormalities. Forty-three moderate/severe TBI patients and 34 healthy controls (HC) underwent resting-state fMRI. Group ICA was applied to identify RSNs. Between-subject analysis was performed using dual regression. Multiple linear regressions were used to investigate the relationship between abnormal connectivity strength and neuropsychological outcome. Forty (93%) TBI patients showed moderate disability, while 2 (5%) and 1 (2%) upper severe disability and low good recovery, respectively. TBI patients performed worse than HC on the domains attention and language. We found increased connectivity in sensorimotor, visual, default mode (DMN), executive, and cerebellar RSNs after TBI. We demonstrated an effect of connectivity in the sensorimotor RSN on attention (p < 10-3) and a trend towards a significant effect of the DMN connectivity on attention (p = 0.058). A group-by-network interaction on attention was found in the sensorimotor network (p = 0.002). In TBI, attention was positively related to abnormal connectivity within the sensorimotor RSN, while in HC this relation was negative. Our results show altered patterns of functional connectivity after TBI. Attention impairments in TBI were associated with increased connectivity in the sensorimotor network. Further research is needed to test whether attention in TBI patients is directly affected by changes in functional connectivity in the sensorimotor network or whether the effect is actually driven by changes in the DMN.

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream.

Converging evidence suggests that the primate ventral visual pathway encodes increasingly complex stimulus features in downstream areas. We quantitatively show that there indeed exists an explicit gradient for feature complexity in the ventral pathway of the human brain. This was achieved by mapping thousands of stimulus features of increasing complexity across the cortical sheet using a deep neural network. Our approach also revealed a fine-grained functional specialization of downstream areas of the ventral stream. Furthermore, it allowed decoding of representations from human brain activity at an unsurpassed degree of accuracy, confirming the quality of the developed approach. Stimulus features that successfully explained neural responses indicate that population receptive fields were explicitly tuned for object categorization. This provides strong support for the hypothesis that object categorization is a guiding principle in the functional organization of the primate ventral stream.

Preference for Audiovisual Speech Congruency in Superior Temporal Cortex.

Auditory speech perception can be altered by concurrent visual information. The superior temporal cortex is an important combining site for this integration process. This area was previously found to be sensitive to audiovisual congruency. However, the direction of this congruency effect (i.e., stronger or weaker activity for congruent compared to incongruent stimulation) has been more equivocal. Here, we used fMRI to look at the neural responses of human participants during the McGurk illusion--in which auditory /aba/ and visual /aga/ inputs are fused to perceived /ada/--in a large homogenous sample of participants who consistently experienced this illusion. This enabled us to compare the neuronal responses during congruent audiovisual stimulation with incongruent audiovisual stimulation leading to the McGurk illusion while avoiding the possible confounding factor of sensory surprise that can occur when McGurk stimuli are only occasionally perceived. We found larger activity for congruent audiovisual stimuli than for incongruent (McGurk) stimuli in bilateral superior temporal cortex, extending into the primary auditory cortex. This finding suggests that superior temporal cortex prefers when auditory and visual input support the same representation.

Bayesian Estimation of Conditional Independence Graphs Improves Functional Connectivity Estimates.

Functional connectivity concerns the correlated activity between neuronal populations in spatially segregated regions of the brain, which may be studied using functional magnetic resonance imaging (fMRI). This coupled activity is conveniently expressed using covariance, but this measure fails to distinguish between direct and indirect effects. A popular alternative that addresses this issue is partial correlation, which regresses out the signal of potentially confounding variables, resulting in a measure that reveals only direct connections. Importantly, provided the data are normally distributed, if two variables are conditionally independent given all other variables, their respective partial correlation is zero. In this paper, we propose a probabilistic generative model that allows us to estimate functional connectivity in terms of both partial correlations and a graph representing conditional independencies. Simulation results show that this methodology is able to outperform the graphical LASSO, which is the de facto standard for estimating partial correlations. Furthermore, we apply the model to estimate functional connectivity for twenty subjects using resting-state fMRI data. Results show that our model provides a richer representation of functional connectivity as compared to considering partial correlations alone. Finally, we demonstrate how our approach can be extended in several ways, for instance to achieve data fusion by informing the conditional independence graph with data from probabilistic tractography. As our Bayesian formulation of functional connectivity provides access to the posterior distribution instead of only to point estimates, we are able to quantify the uncertainty associated with our results. This reveals that while we are able to infer a clear backbone of connectivity in our empirical results, the data are not accurately described by simply looking at the mode of the distribution over connectivity. The implication of this is that deterministic alternatives may misjudge connectivity results by drawing conclusions from noisy and limited data.

Modality-specific alpha modulations facilitate long-term memory encoding in the presence of distracters.

It has been proposed that long-term memory encoding is not only dependent on engaging task-relevant regions but also on disengaging task-irrelevant regions. In particular, oscillatory alpha activity has been shown to be involved in shaping the functional architecture of the working brain because it reflects the functional disengagement of specific regions in attention and memory tasks. We here ask if such allocation of resources by alpha oscillations generalizes to long-term memory encoding in a cross-modal setting in which we acquired the ongoing brain activity using magnetoencephalography. Participants were asked to encode pictures while ignoring simultaneously presented words and vice versa. We quantified the brain activity during rehearsal reflecting subsequent memory in the different attention conditions. The key finding was that successful long-term memory encoding is reflected by alpha power decreases in the sensory region of the to-be-attended modality and increases in the sensory region of the to-be-ignored modality to suppress distraction during rehearsal period. Our results corroborate related findings from attention studies by demonstrating that alpha activity is also important for the allocation of resources during long-term memory encoding in the presence of distracters.

Predicting the semantic category of internally generated words from neuromagnetic recordings.

In this study, we explore the possibility to predict the semantic category of words from brain signals in a free word generation task. Participants produced single words from different semantic categories in a modified semantic fluency task. A Bayesian logistic regression classifier was trained to predict the semantic category of words from single-trial MEG data. Significant classification accuracies were achieved using sensor-level MEG time series at the time interval of conceptual preparation. Semantic category prediction was also possible using source-reconstructed time series, based on minimum norm estimates of cortical activity. Brain regions that contributed most to classification on the source level were identified. These were the left inferior frontal gyrus, left middle frontal gyrus, and left posterior middle temporal gyrus. Additionally, the temporal dynamics of brain activity underlying the semantic preparation during word generation was explored. These results provide important insights about central aspects of language production.

Measuring directionality between neuronal oscillations of different frequencies.

It is well established that neuronal oscillations at different frequencies interact with each other in terms of cross-frequency coupling (CFC). In particular, the phase of slower oscillations modulates the power of faster oscillations. This is referred to as phase-amplitude coupling (PAC). Examples are alpha phase to gamma power coupling as observed in humans and theta phase to gamma power coupling as observed in the rat hippocampus. We here ask if the interaction between alpha and gamma oscillations is in the direction of the phase of slower oscillations driving the power of faster oscillations or conversely from the power of faster oscillations driving the phase of slower oscillations. To answer this question, we introduce a new measure to estimate the cross-frequency directionality (CFD). This measure is based on the phase-slope index (PSI) between the phase of slower oscillations and the power envelope of faster oscillations. Further, we propose a randomization framework for statistically evaluating the coupling measures when controlling for multiple comparisons over the investigated frequency ranges. The method was firstly validated on simulated data and next applied to resting state electrocorticography (ECoG) data. These results demonstrate that the method works reliably. In particular, we found that the power envelope of gamma oscillations drives the phase of slower oscillations in the alpha band. This surprising finding is not easily reconcilable with theories suggesting that feedback controlled alpha oscillations modulate feedforward processing reflected in the gamma band.

Unsupervised feature learning improves prediction of human brain activity in response to natural images.

Encoding and decoding in functional magnetic resonance imaging has recently emerged as an area of research to noninvasively characterize the relationship between stimulus features and human brain activity. To overcome the challenge of formalizing what stimulus features should modulate single voxel responses, we introduce a general approach for making directly testable predictions of single voxel responses to statistically adapted representations of ecologically valid stimuli. These representations are learned from unlabeled data without supervision. Our approach is validated using a parsimonious computational model of (i) how early visual cortical representations are adapted to statistical regularities in natural images and (ii) how populations of these representations are pooled by single voxels. This computational model is used to predict single voxel responses to natural images and identify natural images from stimulus-evoked multiple voxel responses. We show that statistically adapted low-level sparse and invariant representations of natural images better span the space of early visual cortical representations and can be more effectively exploited in stimulus identification than hand-designed Gabor wavelets. Our results demonstrate the potential of our approach to better probe unknown cortical representations.