A large-scale neurocomputational model of spatial cognition integrating memory with vision.

We introduce a large-scale neurocomputational model of spatial cognition called 'Spacecog', which integrates recent findings from mechanistic models of visual and spatial perception. As a high-level cognitive ability, spatial cognition requires the processing of behaviourally relevant features in complex environments and, importantly, the updating of this information during processes of eye and body movement. The Spacecog model achieves this by interfacing spatial memory and imagery with mechanisms of object localisation, saccade execution, and attention through coordinate transformations in parietal areas of the brain. We evaluate the model in a realistic virtual environment where our neurocognitive model steers an agent to perform complex visuospatial tasks. Our modelling approach opens up new possibilities in the assessment of neuropsychological data and human spatial cognition.

Correction: The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach.

[This corrects the article DOI: 10.1371/journal.pcbi.1011024.].

Exploration behavior after reversals is predicted by STN-GPe synaptic plasticity in a basal ganglia model.

Humans can quickly adapt their behavior to changes in the environment. Classical reversal learning tasks mainly measure how well participants can disengage from a previously successful behavior but not how alternative responses are explored. Here, we propose a novel 5-choice reversal learning task with alternating position-reward contingencies to study exploration behavior after a reversal. We compare human exploratory saccade behavior with a prediction obtained from a neuro-computational model of the basal ganglia. A new synaptic plasticity rule for learning the connectivity between the subthalamic nucleus (STN) and external globus pallidus (GPe) results in exploration biases to previously rewarded positions. The model simulations and human data both show that during experimental experience exploration becomes limited to only those positions that have been rewarded in the past. Our study demonstrates how quite complex behavior may result from a simple sub-circuit within the basal ganglia pathways.

The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach.

Motor learning involves a widespread brain network including the basal ganglia, cerebellum, motor cortex, and brainstem. Despite its importance, little is known about how this network learns motor tasks and which role different parts of this network take. We designed a systems-level computational model of motor learning, including a cortex-basal ganglia motor loop and the cerebellum that both determine the response of central pattern generators in the brainstem. First, we demonstrate its ability to learn arm movements toward different motor goals. Second, we test the model in a motor adaptation task with cognitive control, where the model replicates human data. We conclude that the cortex-basal ganglia loop learns via a novelty-based motor prediction error to determine concrete actions given a desired outcome, and that the cerebellum minimizes the remaining aiming error.

Auto-Selection of an Optimal Sparse Matrix Format in the Neuro-Simulator ANNarchy.

Modern neuro-simulators provide efficient implementations of simulation kernels on various parallel hardware (multi-core CPUs, distributed CPUs, GPUs), thereby supporting the simulation of increasingly large and complex biologically realistic networks. However, the optimal configuration of the parallel hardware and computational kernels depends on the exact structure of the network to be simulated. For example, the computation time of rate-coded neural networks is generally limited by the available memory bandwidth, and consequently, the organization of the data in memory will strongly influence the performance for different connectivity matrices. We pinpoint the role of sparse matrix formats implemented in the neuro-simulator ANNarchy with respect to computation time. Rather than asking the user to identify the best data structures required for a given network and platform, such a decision could also be carried out by the neuro-simulator. However, it requires heuristics that need to be adapted over time for the available hardware. The present study investigates how machine learning methods can be used to identify appropriate implementations for a specific network. We employ an artificial neural network to develop a predictive model to help the developer select the optimal sparse matrix format. The model is first trained offline using a set of training examples on a particular hardware platform. The learned model can then predict the execution time of different matrix formats and decide on the best option for a specific network. Our experimental results show that using up to 3,000 examples of random network configurations (i.e., different population sizes as well as variable connectivity), our approach effectively selects the appropriate configuration, providing over 93% accuracy in predicting the suitable format on three different NVIDIA devices.

Virtual deep brain stimulation: Multiscale co-simulation of a spiking basal ganglia model and a whole-brain mean-field model with The Virtual Brain.

Deep brain stimulation (DBS) has been successfully applied in various neurodegenerative diseases as an effective symptomatic treatment. However, its mechanisms of action within the brain network are still poorly understood. Many virtual DBS models analyze a subnetwork around the basal ganglia and its dynamics as a spiking network with their details validated by experimental data. However, connectomic evidence shows widespread effects of DBS affecting many different cortical and subcortical areas. From a clinical perspective, various effects of DBS besides the motoric impact have been demonstrated. The neuroinformatics platform The Virtual Brain (TVB) offers a modeling framework allowing us to virtually perform stimulation, including DBS, and forecast the outcome from a dynamic systems perspective prior to invasive surgery with DBS lead placement. For an accurate prediction of the effects of DBS, we implement a detailed spiking model of the basal ganglia, which we combine with TVB via our previously developed co-simulation environment. This multiscale co-simulation approach builds on the extensive previous literature of spiking models of the basal ganglia while simultaneously offering a whole-brain perspective on widespread effects of the stimulation going beyond the motor circuit. In the first demonstration of our model, we show that virtual DBS can move the firing rates of a Parkinson's disease patient's thalamus - basal ganglia network towards the healthy regime while, at the same time, altering the activity in distributed cortical regions with a pronounced effect in frontal regions. Thus, we provide proof of concept for virtual DBS in a co-simulation environment with TVB. The developed modeling approach has the potential to optimize DBS lead placement and configuration and forecast the success of DBS treatment for individual patients.

BOLD Monitoring in the Neural Simulator ANNarchy.

Multi-scale network models that simultaneously simulate different measurable signals at different spatial and temporal scales, such as membrane potentials of single neurons, population firing rates, local field potentials, and blood-oxygen-level-dependent (BOLD) signals, are becoming increasingly popular in computational neuroscience. The transformation of the underlying simulated neuronal activity of these models to simulated non-invasive measurements, such as BOLD signals, is particularly relevant. The present work describes the implementation of a BOLD monitor within the neural simulator ANNarchy to allow an on-line computation of simulated BOLD signals from neural network models. An active research topic regarding the simulation of BOLD signals is the coupling of neural processes to cerebral blood flow (CBF) and cerebral metabolic rate of oxygen (CMRO2). The flexibility of ANNarchy allows users to define this coupling with a high degree of freedom and thus, not only allows to relate mesoscopic network models of populations of spiking neurons to experimental BOLD data, but also to investigate different hypotheses regarding the coupling between neural processes, CBF and CMRO2 with these models. In this study, we demonstrate how simulated BOLD signals can be obtained from a network model consisting of multiple spiking neuron populations. We first demonstrate the use of the Balloon model, the predominant model for simulating BOLD signals, as well as the possibility of using novel user-defined models, such as a variant of the Balloon model with separately driven CBF and CMRO2 signals. We emphasize how different hypotheses about the coupling between neural processes, CBF and CMRO2 can be implemented and how these different couplings affect the simulated BOLD signals. With the BOLD monitor presented here, ANNarchy provides a tool for modelers who want to relate their network models to experimental MRI data and for scientists who want to extend their studies of the coupling between neural processes and the BOLD signal by using modeling approaches. This facilitates the investigation and model-based analysis of experimental BOLD data and thus improves multi-scale understanding of neural processes in humans.

Enhanced habit formation in Tourette patients explained by shortcut modulation in a hierarchical cortico-basal ganglia model.

Devaluation protocols reveal that Tourette patients show an increased propensity to habitual behaviors as they continue to respond to devalued outcomes in a cognitive stimulus-response-outcome association task. We use a neuro-computational model of hierarchically organized cortico-basal ganglia-thalamo-cortical loops to shed more light on habit formation and its alteration in Tourette patients. In our model, habitual behavior emerges from cortico-thalamic shortcut connections, where enhanced habit formation can be linked to faster plasticity in the shortcut or to a stronger feedback from the shortcut to the basal ganglia. We explore two major hypotheses of Tourette pathophysiology-local striatal disinhibition and increased dopaminergic modulation of striatal medium spiny neurons-as causes for altered shortcut activation. Both model changes altered shortcut functioning and resulted in higher rates of responses towards devalued outcomes, similar to what is observed in Tourette patients. We recommend future experimental neuroscientific studies to locate shortcuts between cortico-basal ganglia-thalamo-cortical loops in the human brain and study their potential role in health and disease.

Building and Understanding the Minimal Self.

Within the methodologically diverse interdisciplinary research on the minimal self, we identify two movements with seemingly disparate research agendas - cognitive science and cognitive (developmental) robotics. Cognitive science, on the one hand, devises rather abstract models which can predict and explain human experimental data related to the minimal self. Incorporating the established models of cognitive science and ideas from artificial intelligence, cognitive robotics, on the other hand, aims to build embodied learning machines capable of developing a self "from scratch" similar to human infants. The epistemic promise of the latter approach is that, at some point, robotic models can serve as a testbed for directly investigating the mechanisms that lead to the emergence of the minimal self. While both approaches can be productive for creating causal mechanistic models of the minimal self, we argue that building a minimal self is different from understanding the human minimal self. Thus, one should be cautious when drawing conclusions about the human minimal self based on robotic model implementations and vice versa. We further point out that incorporating constraints arising from different levels of analysis will be crucial for creating models that can predict, generate, and causally explain behavior in the real world.

A neuro-computational model of visual attention with multiple attentional control sets.

In numerous activities, humans need to attend to multiple sources of visual information at the same time. Although several recent studies support the evidence of this ability, the mechanism of multi-item attentional processing is still a matter of debate and has not been investigated much by previous computational models. Here, we present a neuro-computational model aiming to address specifically the question of how subjects attend to two items that deviate defined by feature and location. We simulate the experiment of Adamo et al. (2010) which required subjects to use two different attentional control sets, each a combination of color and location. The structure of our model is composed of two components "attention" and "decision-making". The important aspect of our model is its dynamic equations that allow us to simulate the time course of processes at a neural level that occur during different stages until a decision is made. We analyze in detail the conditions under which our model matches the behavioral and EEG data from human subjects. Consistent with experimental findings, our model supports the hypothesis of attending to two control settings concurrently. In particular, our model proposes that initially, feature-based attention operates in parallel across the scene, and only in ongoing processing, a selection by the location takes place.

Sensory coding and contrast invariance emerge from the control of plastic inhibition over emergent selectivity.

Visual stimuli are represented by a highly efficient code in the primary visual cortex, but the development of this code is still unclear. Two distinct factors control coding efficiency: Representational efficiency, which is determined by neuronal tuning diversity, and metabolic efficiency, which is influenced by neuronal gain. How these determinants of coding efficiency are shaped during development, supported by excitatory and inhibitory plasticity, is only partially understood. We investigate a fully plastic spiking network of the primary visual cortex, building on phenomenological plasticity rules. Our results suggest that inhibitory plasticity is key to the emergence of tuning diversity and accurate input encoding. We show that inhibitory feedback (random and specific) increases the metabolic efficiency by implementing a gain control mechanism. Interestingly, this led to the spontaneous emergence of contrast-invariant tuning curves. Our findings highlight that (1) interneuron plasticity is key to the development of tuning diversity and (2) that efficient sensory representations are an emergent property of the resulting network.

Performance of biologically grounded models of the early visual system on standard object recognition tasks.

Computational neuroscience models of vision and neural network models for object recognition are often framed by different research agendas. Computational neuroscience mainly aims at replicating experimental data, while (artificial) neural networks target high performance on classification tasks. However, we propose that models of vision should be validated on object recognition tasks. At some point, mechanisms of realistic neuro-computational models of the visual cortex have to convince in object recognition as well. In order to foster this idea, we report the recognition accuracy for two different neuro-computational models of the visual cortex on several object recognition datasets. The models were trained using unsupervised Hebbian learning rules on natural scene inputs for the emergence of receptive fields comparable to their biological counterpart. We assume that the emerged receptive fields result in a general codebook of features, which should be applicable to a variety of visual scenes. We report the performances on datasets with different levels of difficulty, ranging from the simple MNIST to the more complex CIFAR-10 or ETH-80. We found that both networks show good results on simple digit recognition, comparable with previously published biologically plausible models. We also observed that our deeper layer neurons provide for naturalistic datasets a better recognition codebook. As for most datasets, recognition results of biologically grounded models are not available yet, our results provide a broad basis of performance values to compare methodologically similar models.

Optimal attention tuning in a neuro-computational model of the visual cortex-basal ganglia-prefrontal cortex loop.

Visual attention is widely considered a vital factor in the perception and analysis of a visual scene. Several studies explored the effects and mechanisms of top-down attention, but the mechanisms that determine the attentional signal are less explored. By developing a neuro-computational model of visual attention including the visual cortex-basal ganglia loop, we demonstrate how attentional alignment can evolve based on dopaminergic reward during a visual search task. Unlike most previous modeling studies of feature-based attention, we do not implement a manually predefined attention template. Dopamine-modulated covariance learning enable the basal ganglia to learn rewarded associations between the visual input and the attentional gain represented in the PFC of the model. Hence, the model shows human-like performance on a visual search task by optimally tuning the attention signal. In particular, similar as in humans, this reward-based tuning in the model leads to an attentional template that is not centered on the target feature, but a relevant feature deviating away from the target due to the presence of highly similar distractors. Further analyses of the model shows, attention is mainly guided by the signal-to-noise ratio between target and distractors.

A spiking model of basal ganglia dynamics in stopping behavior supported by arkypallidal neurons.

The common view that stopping action plans by the basal ganglia is achieved mainly by the subthalamic nucleus alone due to its direct excitatory projection onto the output nuclei of the basal ganglia has been challenged by recent findings. The proposed "pause-then-cancel" model suggests that the subthalamic nucleus provides a rapid stimulus-unspecific "pause" signal, followed by a stop-cue-specific "cancel" signal from striatum-projecting arkypallidal neurons. To determine more precisely the relative contribution of the different basal ganglia nuclei in stopping, we simulated a stop-signal task with a spiking neuron model of the basal ganglia, considering recently discovered connections from the arkypallidal neurons, and cortex-projecting GPe neurons. For the arkypallidal and prototypical GPe neurons, we obtained neuron model parameters by fitting their neuronal responses to published experimental data. Our model replicates findings of stop-signal tasks at neuronal and behavioral levels. We provide evidence for the existence of a stop-related cortical input to the arkypallidal and cortex-projecting GPe neurons such that the stop responses of the subthalamic nucleus, the arkypallidal neurons, and the cortex-projecting GPe neurons complement each other to achieve functional stopping behavior. Particularly, the cortex-projecting GPe neurons may complement the stopping within the basal ganglia caused by the arkypallidal and STN neurons by diminishing cortical go-related processes. Furthermore, we predict effects of lesions on stopping performance and propose that arkypallidal neurons mainly participate in stopping by inhibiting striatal neurons of the indirect rather than the direct pathway.

A computational model-based analysis of basal ganglia pathway changes in Parkinson's disease inferred from resting-state fMRI.

Previous computational model-based approaches for understanding the dynamic changes related to Parkinson's disease made particular assumptions about Parkinson's disease-related activity changes or specified dopamine-dependent activation or learning rules. Inspired by recent model-based analysis of resting-state fMRI, we have taken a data-driven approach. We fit the free parameters of a spiking neuro-computational model to match correlations of blood oxygen level-dependent signals between different basal ganglia nuclei and obtain subject-specific neuro-computational models of two subject groups: Parkinson patients and matched controls. When comparing mean firing rates at rest and connectivity strengths between the control and Parkinsonian model groups, several significant differences were found that are consistent with previous experimental observations. We discuss the implications of our approach and compare its results also with the popular "rate model" of the basal ganglia. Our study suggests that a model-based analysis of imaging data from healthy and Parkinsonian subjects is a promising approach for the future to better understand Parkinson-related changes in the basal ganglia and corresponding treatments.

Habit learning in hierarchical cortex-basal ganglia loops.

How do the multiple cortico-basal ganglia-thalamo-cortical loops interact? Are they parallel and fully independent or controlled by an arbitrator, or are they hierarchically organized? We introduce here a set of four key concepts, integrated and evaluated by means of a neuro-computational model, that bring together current ideas regarding cortex-basal ganglia interactions in the context of habit learning. According to key concept 1, each loop learns to select an intermediate objective at a different abstraction level, moving from goals in the ventral striatum to motor in the putamen. Key concept 2 proposes that the cortex integrates the basal ganglia selection with environmental information regarding the achieved objective. Key concept 3 claims shortcuts between loops, and key concept 4 predicts that loops compute their own prediction error signal for learning. Computational benefits of the key concepts are demonstrated. Contrasting with former concepts of habit learning, the loops collaborate to select goal-directed actions while training slower shortcuts develops habitual responses.

Spatial updating of attention across eye movements: A neuro-computational approach.

While we are scanning our environment, the retinal image changes with every saccade. Nevertheless, the visual system anticipates where an attended target will be next and attention is updated to the new location. Recently, two different types of perisaccadic attentional updates were discovered: predictive remapping of attention before saccade onset (Rolfs, Jonikaitis, Deubel, & Cavanagh, 2011) and lingering of attention after saccade (Golomb, Chun, & Mazer, 2008; Golomb, Pulido, Albrecht, Chun, & Mazer, 2010). We here propose a neuro-computational model located in lateral intraparietal cortex based on a previous model of perisaccadic space perception (Ziesche & Hamker, 2011, 2014). Our model can account for both types of updating of attention at a neural-systems level. The lingering effect originates from the late updating of the proprioceptive eye-position signal and the remapping from the early corollary-discharge signal. We put these results in relationship to predictive remapping of receptive fields and show that both phenomena arise from the same simple, recurrent neural circuit. Thus, together with the previously published results, the model provides a comprehensive framework for discussing multiple experimental observations that occur around saccades.

The effect of dopamine on response inhibition in Parkinson's disease relates to age-dependent patterns of nigrostriatal degeneration.

INTRODUCTION: Motor but also non-motor effects are modulated by dopamine (DA) in Parkinson's disease (PD). Impaired inhibition has been related to dopamine overdosing of the associative striatum. We compared effects of dopaminergic medication on inhibitory control in patients with young (age at onset <50 years, YOPD) and late onset PD (LOPD) and related them to nigrostriatal degeneration. METHODS: 27 patients (10 YOPD, 17 LOPD) underwent a Go/NoGo paradigm comprising a global and specific NoGo condition ON and OFF DA. The ratio of dopamine transporter availability (DAT) in the associative relative to the sensorimotor striatum according to [123I]FP-CIT SPECT was compared between YOPD and LOPD (n = 8/12). Neuro-computational modeling was used to identify pathway activation during Go/NoGo performance. RESULTS: Patients made more errors ON compared to OFF in the global NoGo. This DA effect on global NoGo errors correlated with disease duration (r = 0.489, p = 0.010). YOPD made more errors in the specific NoGo ON-OFF compared to LOPD (p = 0.015). YOPD showed higher associative-to-sensorimotor DAT ratios compared to LOPD (p < 0.001). Neuro-computational modeling revealed DA overdosing of the associative striatum in YOPD resulting in excess activation of the direct basal ganglia pathway triggering incorrect responses. CONCLUSIONS: Depending on the age of symptom onset, DA differentially modulated inhibition in PD with detrimental effects on specific NoGo performance in YOPD but increased performance in LOPD. YOPD showed relatively less degeneration in the associative striatum suggesting DA overdosing that is supported by our neuro-computational model. Reduced inhibition in the global NoGo condition suggests different pathway activation.

The subthalamic nucleus-external globus pallidus loop biases exploratory decisions towards known alternatives: a neuro-computational study.

Theories and models of the basal ganglia have mainly focused on the role of three different corticothalamic pathways: direct, indirect and hyperdirect. Although the indirect and the hyperdirect pathways are linked through the bidirectional connections between the subthalamic nucleus (STN) and the external globus pallidus (GPe), the role of their interactions has been mainly discussed in the context of a dysfunction (abnormal oscillations in Parkinson's disease) and not of its function. We here propose a novel role for the loop formed by the STN and the GPe. We show, through a neuro-computational model, that this loop can bias the selection of actions during the exploratory period after a change in the environmental conditions towards alternative responses. Testing well-known alternative solutions before completely random actions can reduce the time required for the search of a new response after a rule change. Our simulations further show that the knowledge acquired by the indirect pathway can be transferred into a stable memory via learning in the hyperdirect pathway to establish the blocking of unwanted responses. After a rule switch, first the indirect pathway learns to inhibit the previously correct actions. Once the new correct association is learned, the inhibition is transferred to the hyperdirect pathway through synaptic plasticity.

On the Role of Cortex-Basal Ganglia Interactions for Category Learning: A Neurocomputational Approach.

In addition to the prefrontal cortex (PFC), the basal ganglia (BG) have been increasingly often reported to play a fundamental role in category learning, but the circuit mechanisms mediating their interaction remain to be explored. We developed a novel neurocomputational model of category learning that particularly addresses the BG-PFC interplay. We propose that the BG bias PFC activity by removing the inhibition of cortico-thalamo-cortical loop and thereby provide a teaching signal to guide the acquisition of category representations in the corticocortical associations to the PFC. Our model replicates key behavioral and physiological data of macaque monkey learning a prototype distortion task from Antzoulatos and Miller (2011) Our simulations allowed us to gain a deeper insight into the observed drop of category selectivity in striatal neurons seen in the experimental data and in the model. The simulation results and a new analysis of the experimental data based on the model's predictions show that the drop in category selectivity of the striatum emerges as the variability of responses in the striatum rises when confronting the BG with an increasingly larger number of stimuli to be classified. The neurocomputational model therefore provides new testable insights of systems-level brain circuits involved in category learning that may also be generalized to better understand other cortico-BG-cortical loops.SIGNIFICANCE STATEMENT Inspired by the idea that basal ganglia (BG) teach the prefrontal cortex (PFC) to acquire category representations, we developed a novel neurocomputational model and tested it on a task that was recently applied in monkey experiments. As an advantage over previous models of category learning, our model allows to compare simulation data with single-cell recordings in PFC and BG. We not only derived model predictions, but already verified a prediction to explain the observed drop in striatal category selectivity. When testing our model with a simple, real-world face categorization task, we observed that the fast striatal learning with a performance of 85% correct responses can teach the slower PFC learning to push the model performance up to almost 100%.