Convolutional neural networks develop major organizational principles of early visual cortex when enhanced with retinal sampling.

Primate visual cortex exhibits key organizational principles: cortical magnification, eccentricity-dependent receptive field size and spatial frequency tuning as well as radial bias. We provide compelling evidence that these principles arise from the interplay of the non-uniform distribution of retinal ganglion cells, and a quasi-uniform convergence rate from the retina to the cortex. We show that convolutional neural networks outfitted with a retinal sampling layer, which resamples images according to retinal ganglion cell density, develop these organizational principles. Surprisingly, our results indicate that radial bias is spatial-frequency dependent and only manifests for high spatial frequencies. For low spatial frequencies, the bias shifts towards orthogonal orientations. These findings introduce a novel hypothesis about the origin of radial bias. Quasi-uniform convergence limits the range of spatial frequencies (in retinal space) that can be resolved, while retinal sampling determines the spatial frequency content throughout the retina.

Modular-integrative modeling: a new framework for building brain models that blend biological realism and functional performance.

This Perspective presents the Modular-Integrative Modeling approach, a novel framework in neuroscience for developing brain models that blend biological realism with functional performance to provide a holistic view on brain function in interaction with the body and environment.

AngoraPy: A Python toolkit for modeling anthropomorphic goal-driven sensorimotor systems.

Goal-driven deep learning increasingly supplements classical modeling approaches in computational neuroscience. The strength of deep neural networks as models of the brain lies in their ability to autonomously learn the connectivity required to solve complex and ecologically valid tasks, obviating the need for hand-engineered or hypothesis-driven connectivity patterns. Consequently, goal-driven models can generate hypotheses about the neurocomputations underlying cortical processing that are grounded in macro- and mesoscopic anatomical properties of the network's biological counterpart. Whereas, goal-driven modeling is already becoming prevalent in the neuroscience of perception, its application to the sensorimotor domain is currently hampered by the complexity of the methods required to train models comprising the closed sensation-action loop. This paper describes AngoraPy, a Python library that mitigates this obstacle by providing researchers with the tools necessary to train complex recurrent convolutional neural networks that model the human sensorimotor system. To make the technical details of this toolkit more approachable, an illustrative example that trains a recurrent toy model on in-hand object manipulation accompanies the theoretical remarks. An extensive benchmark on various classical, 3D robotic, and anthropomorphic control tasks demonstrates AngoraPy's general applicability to a wide range of tasks. Together with its ability to adaptively handle custom architectures, the flexibility of this toolkit demonstrates its power for goal-driven sensorimotor modeling.

Editorial: Focus feature on biomarkers in network neuroscience.

There is an ongoing need for novel biomarkers in clinical neuroscience, as diagnosis of neurological and psychiatric disorders is hampered by the pronounced overlap of behavioral symptoms and other pathophysiological characteristics. The question that this Focus Feature puts center stage is whether network-based biomarkers may provide a viable tool for distinguishing between disordered populations or whether they may yield only limited differentiating power because of largely shared network characteristics across conditions.

Extremely fast pRF mapping for real-time applications.

Population receptive field (pRF) mapping is a popular tool in computational neuroimaging that allows for the investigation of receptive field properties, their topography and interrelations in health and disease. Furthermore, the possibility to invert population receptive fields provides a decoding model for constructing stimuli from observed cortical activation patterns. This has been suggested to pave the road towards pRF-based brain-computer interface (BCI) communication systems, which would be able to directly decode internally visualized letters from topographically organized brain activity. A major stumbling block for such an application is, however, that the pRF mapping procedure is computationally heavy and time consuming. To address this, we propose a novel and fast pRF mapping procedure that is suitable for real-time applications. The method is built upon hashed-Gaussian encoding of the stimulus, which tremendously reduces computational resources. After the stimulus is encoded, mapping can be performed using either ridge regression for fast offline analyses or gradient descent for real-time applications. We validate our model-agnostic approach in silico, as well as on empirical fMRI data obtained from 3T and 7T MRI scanners. Our approach is capable of estimating receptive fields and their parameters for millions of voxels in mere seconds. This method thus facilitates real-time applications of population receptive field mapping.

Cortical Synchrony as a Mechanism of Collinear Facilitation and Suppression in Early Visual Cortex.

Stimulus-induced oscillations and synchrony among neuronal populations in visual cortex are well-established phenomena. Their functional role in cognition are, however, not well-understood. Recent studies have suggested that neural synchrony may underlie perceptual grouping as stimulus-frequency relationships and stimulus-dependent lateral connectivity profiles can determine the success or failure of synchronization among neuronal groups encoding different stimulus elements. We suggest that the same mechanism accounts for collinear facilitation and suppression effects where the detectability of a target Gabor stimulus is improved or diminished by the presence of collinear flanking Gabor stimuli. We propose a model of oscillators which represent three neuronal populations in visual cortex with distinct receptive fields reflecting the target and two flankers, respectively, and whose connectivity is determined by the collinearity of the presented Gabor stimuli. Our model simulations confirm that neuronal synchrony can indeed explain known collinear facilitation and suppression effects for attended and unattended stimuli.

Investigating the Reliability of Population Receptive Field Size Estimates Using fMRI.

In functional MRI (fMRI), population receptive field (pRF) models allow a quantitative description of the response as a function of the features of the stimuli that are relevant for each voxel. The most popular pRF model used in fMRI assumes a Gaussian shape in the features space (e.g., the visual field) reducing the description of the voxel's pRF to the Gaussian mean (the pRF preferred feature) and standard deviation (the pRF size). The estimation of the pRF mean has been proven to be highly reliable. However, the estimate of the pRF size has been shown not to be consistent within and between subjects. While this issue has been noted experimentally, here we use an optimization theory perspective to describe how the inconsistency in estimating the pRF size is linked to an inherent property of the Gaussian pRF model. When fitting such models, the goodness of fit is less sensitive to variations in the pRF size than to variations in the pRF mean. We also show how the same issue can be considered from a bias-variance perspective. We compare different estimation procedures in terms of the reliability of their estimates using simulated and real fMRI data in the visual (using the Human Connectome Project database) and auditory domain. We show that, the reliability of the estimate of the pRF size can be improved considering a linear combination of those pRF models with similar goodness of fit or a permutation based approach. This increase in reliability of the pRF size estimate does not affect the reliability of the estimate of the pRF mean and the prediction accuracy.

Model-based whole-brain effective connectivity to study distributed cognition in health and disease.

Neuroimaging techniques are now widely used to study human cognition. The functional associations between brain areas have become a standard proxy to describe how cognitive processes are distributed across the brain network. Among the many analysis tools available, dynamic models of brain activity have been developed to overcome the limitations of original connectivity measures such as functional connectivity. This goes in line with the many efforts devoted to the assessment of directional interactions between brain areas from the observed neuroimaging activity. This opinion article provides an overview of our model-based whole-brain effective connectivity to analyze fMRI data, while discussing the pros and cons of our approach with respect to other established approaches. Our framework relies on the multivariate Ornstein-Uhlenbeck (MOU) process and is thus referred to as MOU-EC. Once tuned, the model provides a directed connectivity estimate that reflects the dynamical state of BOLD activity, which can be used to explore cognition. We illustrate this approach using two applications on task-evoked fMRI data. First, as a connectivity measure, MOU-EC can be used to extract biomarkers for task-specific brain coordination, understood as the patterns of areas exchanging information. The multivariate nature of connectivity measures raises several challenges for whole-brain analysis, for which machine-learning tools present some advantages over statistical testing. Second, we show how to interpret changes in MOU-EC connections in a collective and model-based manner, bridging with network analysis. Our framework provides a comprehensive set of tools that open exciting perspectives to study distributed cognition, as well as neuropathologies.

Contextual encoder-decoder network for visual saliency prediction.

Predicting salient regions in natural images requires the detection of objects that are present in a scene. To develop robust representations for this challenging task, high-level visual features at multiple spatial scales must be extracted and augmented with contextual information. However, existing models aimed at explaining human fixation maps do not incorporate such a mechanism explicitly. Here we propose an approach based on a convolutional neural network pre-trained on a large-scale image classification task. The architecture forms an encoder-decoder structure and includes a module with multiple convolutional layers at different dilation rates to capture multi-scale features in parallel. Moreover, we combine the resulting representations with global scene information for accurately predicting visual saliency. Our model achieves competitive and consistent results across multiple evaluation metrics on two public saliency benchmarks and we demonstrate the effectiveness of the suggested approach on five datasets and selected examples. Compared to state of the art approaches, the network is based on a lightweight image classification backbone and hence presents a suitable choice for applications with limited computational resources, such as (virtual) robotic systems, to estimate human fixations across complex natural scenes. Our TensorFlow implementation is openly available at https://github.com/alexanderkroner/saliency.

Interfering with a memory without erasing its trace.

Previous research has shown that performance of a novice skill can be easily interfered with by subsequent training of another skill. We address the open questions whether extensively trained skills show the same vulnerability to interference as novice skills and which memory mechanism regulates interference between expert skills. We developed a recurrent neural network model of V1 able to learn from feedback experienced over the course of a long-term orientation discrimination experiment. After first exposing the model to one discrimination task for 3480 consecutive trials, we assessed how its performance was affected by subsequent training in a second, similar task. Training the second task strongly interfered with the first (highly trained) discrimination skill. The magnitude of interference depended on the relative amounts of training devoted to the different tasks. We used these and other model outcomes as predictions for a perceptual learning experiment in which human participants underwent the same training protocol as our model. Specifically, over the course of three months participants underwent baseline training in one orientation discrimination task for 15 sessions before being trained for 15 sessions on a similar task and finally undergoing another 15 sessions of training on the first task (to assess interference). Across all conditions, the pattern of interference observed empirically closely matched model predictions. According to our model, behavioral interference can be explained by antagonistic changes in neuronal tuning induced by the two tasks. Remarkably, this did not stem from erasing connections due to earlier learning but rather from a reweighting of lateral inhibition.

The road ahead in clinical network neuroscience.

Clinical network neuroscience, the study of brain network topology in neurological and psychiatric diseases, has become a mainstay field within clinical neuroscience. Being a multidisciplinary group of clinical network neuroscience experts based in The Netherlands, we often discuss the current state of the art and possible avenues for future investigations. These discussions revolve around questions like "How do dynamic processes alter the underlying structural network?" and "Can we use network neuroscience for disease classification?" This opinion paper is an incomplete overview of these discussions and expands on ten questions that may potentially advance the field. By no means intended as a review of the current state of the field, it is instead meant as a conversation starter and source of inspiration to others.

Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception.

Visual mental imagery is the quasi-perceptual experience of "seeing in the mind's eye". While a tight correspondence between imagery and perception in terms of subjective experience is well established, their correspondence in terms of neural representations remains insufficiently understood. In the present study, we exploit the high spatial resolution of functional magnetic resonance imaging (fMRI) at 7T, the retinotopic organization of early visual cortex, and machine-learning techniques to investigate whether visual imagery of letter shapes preserves the topographic organization of perceived shapes. Sub-millimeter resolution fMRI images were obtained from early visual cortex in six subjects performing visual imagery of four different letter shapes. Predictions of imagery voxel activation patterns based on a population receptive field-encoding model and physical letter stimuli provided first evidence in favor of detailed topographic organization. Subsequent visual field reconstructions of imagery data based on the inversion of the encoding model further showed that visual imagery preserves the geometric profile of letter shapes. These results open new avenues for decoding, as we show that a denoising autoencoder can be used to pretrain a classifier purely based on perceptual data before fine-tuning it on imagery data. Finally, we show that the autoencoder can project imagery-related voxel activations onto their perceptual counterpart allowing for visually recognizable reconstructions even at the single-trial level. The latter may eventually be utilized for the development of content-based BCI letter-speller systems.

Task-related effective connectivity reveals that the cortical rich club gates cortex-wide communication.

Higher cognition may require the globally coordinated integration of specialized brain regions into functional networks. A collection of structural cortical hubs-referred to as the rich club-has been hypothesized to support task-specific functional integration. In the present paper, we use a whole-cortex model to estimate directed interactions between 68 cortical regions from functional magnetic resonance imaging activity for four different tasks (reflecting different cognitive domains) and resting state. We analyze the state-dependent input and output effective connectivity (EC) of the structural rich club and relate these to whole-cortex dynamics and network reconfigurations. We find that the cortical rich club exhibits an increase in outgoing EC during task performance as compared with rest while incoming connectivity remains constant. Increased outgoing connectivity targets a sparse set of peripheral regions with specific regions strongly overlapping between tasks. At the same time, community detection analyses reveal massive reorganizations of interactions among peripheral regions, including those serving as target of increased rich club output. This suggests that while peripheral regions may play a role in several tasks, their concrete interplay might nonetheless be task-specific. Furthermore, we observe that whole-cortex dynamics are faster during task as compared with rest. The decoupling effects usually accompanying faster dynamics appear to be counteracted by the increased rich club outgoing EC. Together our findings speak to a gating mechanism of the rich club that supports fast-paced information exchange among relevant peripheral regions in a task-specific and goal-directed fashion, while constantly listening to the whole network.

Cortical rich club regions can organize state-dependent functional network formation by engaging in oscillatory behavior.

Cognition is hypothesized to require the globally coordinated, functionally relevant integration of otherwise segregated information processing carried out by specialized brain regions. Studies of the macroscopic connectome as well as recent neuroimaging and neuromodeling research have suggested a densely connected collective of cortical hubs, termed the rich club, to provide a central workspace for such integration. In order for rich club regions to fulfill this role they must dispose of a dynamic mechanism by which they can actively shape networks of brain regions whose information processing needs to be integrated. A potential candidate for such a mechanism comes in the form of oscillations which might be employed to establish communication channels among relevant brain regions. We explore this possibility using an integrative approach combining whole-brain computational modeling with neuroimaging, wherein we investigate the local dynamics model brain regions need to exhibit in order to fit (dynamic) network behavior empirically observed for resting as well as a range of task states. We find that rich club regions largely exhibit oscillations during task performance but not during rest. Furthermore, oscillations exhibited by rich club regions can harmonize a set of asynchronous brain regions thus supporting functional coupling among them. These findings are in line with the hypothesis that the rich club can actively shape integration using oscillations.

Evaluating population receptive field estimation frameworks in terms of robustness and reproducibility.

Within vision research retinotopic mapping and the more general receptive field estimation approach constitute not only an active field of research in itself but also underlie a plethora of interesting applications. This necessitates not only good estimation of population receptive fields (pRFs) but also that these receptive fields are consistent across time rather than dynamically changing. It is therefore of interest to maximize the accuracy with which population receptive fields can be estimated in a functional magnetic resonance imaging (fMRI) setting. This, in turn, requires an adequate estimation framework providing the data for population receptive field mapping. More specifically, adequate decisions with regard to stimulus choice and mode of presentation need to be made. Additionally, it needs to be evaluated whether the stimulation protocol should entail mean luminance periods and whether it is advantageous to average the blood oxygenation level dependent (BOLD) signal across stimulus cycles or not. By systematically studying the effects of these decisions on pRF estimates in an empirical as well as simulation setting we come to the conclusion that a bar stimulus presented at random positions and interspersed with mean luminance periods is generally most favorable. Finally, using this optimal estimation framework we furthermore tested the assumption of temporal consistency of population receptive fields. We show that the estimation of pRFs from two temporally separated sessions leads to highly similar pRF parameters.

Rich club organization supports a diverse set of functional network configurations.

Brain function relies on the flexible integration of a diverse set of segregated cortical modules, with the structural connectivity of the brain being a fundamentally important factor in shaping the brain's functional dynamics. Following up on macroscopic studies showing the existence of centrally connected nodes in the mammalian brain, combined with the notion that these putative brain hubs may form a dense interconnected 'rich club' collective, we hypothesized that brain connectivity might involve a rich club type of architecture to promote a repertoire of different and flexibly accessible brain functions. With the rich club suggested to play an important role in global brain communication, examining the effects of a rich club organization on the functional repertoire of physical systems in general, and the brain in particular, is of keen interest. Here we elucidate these effects using a spin glass model of neural networks for simulating stable configurations of cortical activity. Using simulations, we show that the presence of a rich club increases the set of attractors and hence the diversity of the functional repertoire over and above the effects produced by scale free type topology alone. Within the networks' overall functional repertoire rich nodes are shown to be important for enabling a high level of dynamic integrations of low-degree nodes to form functional networks. This suggests that the rich club serves as an important backbone for numerous co-activation patterns among peripheral nodes of the network. In addition, applying the spin glass model to empirical anatomical data of the human brain, we show that the positive effects on the functional repertoire attributed to the rich club phenomenon can be observed for the brain as well. We conclude that a rich club organization in network architectures may be crucial for the facilitation and integration of a diverse number of segregated functions.

How anatomy shapes dynamics: a semi-analytical study of the brain at rest by a simple spin model.

Resting state networks (RSNs) show a surprisingly coherent and robust spatiotemporal organization. Previous theoretical studies demonstrated that these patterns can be understood as emergent on the basis of the underlying neuroanatomical connectivity skeleton. Integrating the biologically realistic DTI/DSI-(Diffusion Tensor Imaging/Diffusion Spectrum Imaging)based neuroanatomical connectivity into a brain model of Ising spin dynamics, we found a system with multiple attractors, which can be studied analytically. The multistable attractor landscape thus defines a functionally meaningful dynamic repertoire of the brain network that is inherently present in the neuroanatomical connectivity. We demonstrate that the more entropy of attractors exists, the richer is the dynamical repertoire and consequently the brain network displays more capabilities of computation. We hypothesize therefore that human brain connectivity developed a scale free type of architecture in order to be able to store a large number of different and flexibly accessible brain functions.