Correction: The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach.

[This corrects the article DOI: 10.1371/journal.pcbi.1011024.].

The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach.

Motor learning involves a widespread brain network including the basal ganglia, cerebellum, motor cortex, and brainstem. Despite its importance, little is known about how this network learns motor tasks and which role different parts of this network take. We designed a systems-level computational model of motor learning, including a cortex-basal ganglia motor loop and the cerebellum that both determine the response of central pattern generators in the brainstem. First, we demonstrate its ability to learn arm movements toward different motor goals. Second, we test the model in a motor adaptation task with cognitive control, where the model replicates human data. We conclude that the cortex-basal ganglia loop learns via a novelty-based motor prediction error to determine concrete actions given a desired outcome, and that the cerebellum minimizes the remaining aiming error.

Sensory coding and contrast invariance emerge from the control of plastic inhibition over emergent selectivity.

Visual stimuli are represented by a highly efficient code in the primary visual cortex, but the development of this code is still unclear. Two distinct factors control coding efficiency: Representational efficiency, which is determined by neuronal tuning diversity, and metabolic efficiency, which is influenced by neuronal gain. How these determinants of coding efficiency are shaped during development, supported by excitatory and inhibitory plasticity, is only partially understood. We investigate a fully plastic spiking network of the primary visual cortex, building on phenomenological plasticity rules. Our results suggest that inhibitory plasticity is key to the emergence of tuning diversity and accurate input encoding. We show that inhibitory feedback (random and specific) increases the metabolic efficiency by implementing a gain control mechanism. Interestingly, this led to the spontaneous emergence of contrast-invariant tuning curves. Our findings highlight that (1) interneuron plasticity is key to the development of tuning diversity and (2) that efficient sensory representations are an emergent property of the resulting network.

A spiking model of basal ganglia dynamics in stopping behavior supported by arkypallidal neurons.

The common view that stopping action plans by the basal ganglia is achieved mainly by the subthalamic nucleus alone due to its direct excitatory projection onto the output nuclei of the basal ganglia has been challenged by recent findings. The proposed "pause-then-cancel" model suggests that the subthalamic nucleus provides a rapid stimulus-unspecific "pause" signal, followed by a stop-cue-specific "cancel" signal from striatum-projecting arkypallidal neurons. To determine more precisely the relative contribution of the different basal ganglia nuclei in stopping, we simulated a stop-signal task with a spiking neuron model of the basal ganglia, considering recently discovered connections from the arkypallidal neurons, and cortex-projecting GPe neurons. For the arkypallidal and prototypical GPe neurons, we obtained neuron model parameters by fitting their neuronal responses to published experimental data. Our model replicates findings of stop-signal tasks at neuronal and behavioral levels. We provide evidence for the existence of a stop-related cortical input to the arkypallidal and cortex-projecting GPe neurons such that the stop responses of the subthalamic nucleus, the arkypallidal neurons, and the cortex-projecting GPe neurons complement each other to achieve functional stopping behavior. Particularly, the cortex-projecting GPe neurons may complement the stopping within the basal ganglia caused by the arkypallidal and STN neurons by diminishing cortical go-related processes. Furthermore, we predict effects of lesions on stopping performance and propose that arkypallidal neurons mainly participate in stopping by inhibiting striatal neurons of the indirect rather than the direct pathway.

A computational model-based analysis of basal ganglia pathway changes in Parkinson's disease inferred from resting-state fMRI.

Previous computational model-based approaches for understanding the dynamic changes related to Parkinson's disease made particular assumptions about Parkinson's disease-related activity changes or specified dopamine-dependent activation or learning rules. Inspired by recent model-based analysis of resting-state fMRI, we have taken a data-driven approach. We fit the free parameters of a spiking neuro-computational model to match correlations of blood oxygen level-dependent signals between different basal ganglia nuclei and obtain subject-specific neuro-computational models of two subject groups: Parkinson patients and matched controls. When comparing mean firing rates at rest and connectivity strengths between the control and Parkinsonian model groups, several significant differences were found that are consistent with previous experimental observations. We discuss the implications of our approach and compare its results also with the popular "rate model" of the basal ganglia. Our study suggests that a model-based analysis of imaging data from healthy and Parkinsonian subjects is a promising approach for the future to better understand Parkinson-related changes in the basal ganglia and corresponding treatments.

Habit learning in hierarchical cortex-basal ganglia loops.

How do the multiple cortico-basal ganglia-thalamo-cortical loops interact? Are they parallel and fully independent or controlled by an arbitrator, or are they hierarchically organized? We introduce here a set of four key concepts, integrated and evaluated by means of a neuro-computational model, that bring together current ideas regarding cortex-basal ganglia interactions in the context of habit learning. According to key concept 1, each loop learns to select an intermediate objective at a different abstraction level, moving from goals in the ventral striatum to motor in the putamen. Key concept 2 proposes that the cortex integrates the basal ganglia selection with environmental information regarding the achieved objective. Key concept 3 claims shortcuts between loops, and key concept 4 predicts that loops compute their own prediction error signal for learning. Computational benefits of the key concepts are demonstrated. Contrasting with former concepts of habit learning, the loops collaborate to select goal-directed actions while training slower shortcuts develops habitual responses.

On the Role of Cortex-Basal Ganglia Interactions for Category Learning: A Neurocomputational Approach.

In addition to the prefrontal cortex (PFC), the basal ganglia (BG) have been increasingly often reported to play a fundamental role in category learning, but the circuit mechanisms mediating their interaction remain to be explored. We developed a novel neurocomputational model of category learning that particularly addresses the BG-PFC interplay. We propose that the BG bias PFC activity by removing the inhibition of cortico-thalamo-cortical loop and thereby provide a teaching signal to guide the acquisition of category representations in the corticocortical associations to the PFC. Our model replicates key behavioral and physiological data of macaque monkey learning a prototype distortion task from Antzoulatos and Miller (2011) Our simulations allowed us to gain a deeper insight into the observed drop of category selectivity in striatal neurons seen in the experimental data and in the model. The simulation results and a new analysis of the experimental data based on the model's predictions show that the drop in category selectivity of the striatum emerges as the variability of responses in the striatum rises when confronting the BG with an increasingly larger number of stimuli to be classified. The neurocomputational model therefore provides new testable insights of systems-level brain circuits involved in category learning that may also be generalized to better understand other cortico-BG-cortical loops.SIGNIFICANCE STATEMENT Inspired by the idea that basal ganglia (BG) teach the prefrontal cortex (PFC) to acquire category representations, we developed a novel neurocomputational model and tested it on a task that was recently applied in monkey experiments. As an advantage over previous models of category learning, our model allows to compare simulation data with single-cell recordings in PFC and BG. We not only derived model predictions, but already verified a prediction to explain the observed drop in striatal category selectivity. When testing our model with a simple, real-world face categorization task, we observed that the fast striatal learning with a performance of 85% correct responses can teach the slower PFC learning to push the model performance up to almost 100%.

The subthalamic nucleus-external globus pallidus loop biases exploratory decisions towards known alternatives: a neuro-computational study.

Theories and models of the basal ganglia have mainly focused on the role of three different corticothalamic pathways: direct, indirect and hyperdirect. Although the indirect and the hyperdirect pathways are linked through the bidirectional connections between the subthalamic nucleus (STN) and the external globus pallidus (GPe), the role of their interactions has been mainly discussed in the context of a dysfunction (abnormal oscillations in Parkinson's disease) and not of its function. We here propose a novel role for the loop formed by the STN and the GPe. We show, through a neuro-computational model, that this loop can bias the selection of actions during the exploratory period after a change in the environmental conditions towards alternative responses. Testing well-known alternative solutions before completely random actions can reduce the time required for the search of a new response after a rule change. Our simulations further show that the knowledge acquired by the indirect pathway can be transferred into a stable memory via learning in the hyperdirect pathway to establish the blocking of unwanted responses. After a rule switch, first the indirect pathway learns to inhibit the previously correct actions. Once the new correct association is learned, the inhibition is transferred to the hyperdirect pathway through synaptic plasticity.

Functional segregation of basal ganglia pathways in Parkinson's disease.

Dopamine exerts modulatory signals on cortex-basal ganglia circuits to enable flexible motor control. Parkinson's disease is characterized by a loss of dopaminergic innervation in the basal ganglia leading to complex motor and non-motor symptoms. Clinical symptom alleviation through dopaminergic medication and deep brain stimulation in the subthalamic nucleus likely depends on a complex interplay between converging basal ganglia pathways. As a unique translational research platform, deep brain stimulation allows instantaneous investigation of functional effects of subthalamic neuromodulation in human patients with Parkinson's disease. The present study aims at disentangling the role of the inhibitory basal ganglia pathways in cognitive and kinematic aspects of automatic and controlled movements in healthy and parkinsonian states by combining behavioural experiments, clinical observations, whole-brain deep brain stimulation fibre connectivity mapping and computational modelling. Twenty patients with Parkinson's disease undergoing subthalamic deep brain stimulation and 20 age-matched healthy controls participated in a visuomotor tracking task requiring normal (automatic) and inverted (controlled) reach movements. Parkinsonian patients on and off deep brain stimulation presented complex patterns of reaction time and kinematic changes, when compared to healthy controls. Stimulation of cortico-subthalamic fibres was correlated with reduced reaction time adaptation to task demand, but not kinematic aspects of motor control or alleviation of Parkinson's disease motor signs. By using clinically, behaviourally and fibre tracking informed computational models, our study reveals that loss of cognitive adaptation can be attributed to modulation of the hyperdirect pathway, while kinematic depends on suppression of indirect pathway activity. Our findings suggest that hyperdirect and indirect pathways, converging in the subthalamic nucleus, are differentially involved in cognitive aspects of cautious motor preparation and kinematic gain control during motor performance. Subthalamic deep brain stimulation modulates but does not restore these functions. Intelligent stimulation algorithms could re-enable flexible motor control in Parkinson's disease when adapted to instantaneous environmental demand. Our results may inspire new innovative pathway-specific approaches to reduce side effects and increase therapeutic efficacy of neuromodulation in patients with Parkinson's disease.

Spatial updating of attention across eye movements: A neuro-computational approach.

While we are scanning our environment, the retinal image changes with every saccade. Nevertheless, the visual system anticipates where an attended target will be next and attention is updated to the new location. Recently, two different types of perisaccadic attentional updates were discovered: predictive remapping of attention before saccade onset (Rolfs, Jonikaitis, Deubel, & Cavanagh, 2011) and lingering of attention after saccade (Golomb, Chun, & Mazer, 2008; Golomb, Pulido, Albrecht, Chun, & Mazer, 2010). We here propose a neuro-computational model located in lateral intraparietal cortex based on a previous model of perisaccadic space perception (Ziesche & Hamker, 2011, 2014). Our model can account for both types of updating of attention at a neural-systems level. The lingering effect originates from the late updating of the proprioceptive eye-position signal and the remapping from the early corollary-discharge signal. We put these results in relationship to predictive remapping of receptive fields and show that both phenomena arise from the same simple, recurrent neural circuit. Thus, together with the previously published results, the model provides a comprehensive framework for discussing multiple experimental observations that occur around saccades.

Basal Ganglia dysfunctions in movement disorders: What can be learned from computational simulations.

The basal ganglia are a complex neuronal system that is impaired in several movement disorders, including Parkinson's disease, Huntington's disease, and dystonia. Empirical studies have provided valuable insights into the brain dysfunctions underlying these disorders. The systems-level perspective, however, of how patients' motor, cognitive, and emotional impairments originate from known brain dysfunctions has been a challenge to empirical investigations. These causal relations have been analyzed via computational modeling, a method that describes the simulation of interacting brain processes in a computer system. In this article, we review computational insights into the brain dysfunctions underlying Parkinson's disease, Huntington's disease, and dystonia, with particular foci on dysfunctions of the dopamine system, basal ganglia pathways, and neuronal oscillations. © 2016 International Parkinson and Movement Disorder Society.

Suppression of displacement detection in the presence and absence of eye movements: a neuro-computational perspective.

Understanding the subjective experience of a visually stable world during eye movements has been an important research topic for many years. Various studies were conducted to reveal fundamental mechanisms of this phenomenon. For example, in the paradigm saccadic suppression of displacement (SSD), it has been observed that a small displacement of a saccade target could not easily be reported if this displacement took place during a saccade. New results from Zimmermann et al. (J Neurophysiol 112(12):3066-3076, 2014) show that the effect of being oblivious to small displacements occurs not only during saccades, but also if a mask is introduced while the target is displaced. We address the question of how neurons in the parietal cortex may be connected to each other to account for the SSD effect in experiments involving a saccade and equally well in the absence of an eye movement while perception is disrupted by a mask.

Differential contributions of the globus pallidus and ventral thalamus to stimulus-response learning in humans.

The ability to learn associations between stimuli, responses and rewards is a prerequisite for survival. Models of reinforcement learning suggest that the striatum, a basal ganglia input nucleus, vitally contributes to these learning processes. Our recently presented computational model predicts, first, that not only the striatum, but also the globus pallidus contributes to the learning (i.e., exploration) of stimulus-response associations based on rewards. Secondly, it predicts that the stable execution (i.e., exploitation) of well-learned associations involves further learning in the thalamus. To test these predictions, we postoperatively recorded local field potentials (LFPs) from patients that had undergone surgery for deep brain stimulation to treat severe movement disorders. Macroelectrodes were placed either in the globus pallidus or in the ventral thalamus. During recordings, patients performed a reward-based stimulus-response learning task that comprised periods of exploration and exploitation. We analyzed correlations between patients' LFP amplitudes and model-based estimates of their reward expectations and reward prediction errors. In line with our first prediction, pallidal LFP amplitudes during the presentation of rewards and reward omissions correlated with patients' reward prediction errors, suggesting pallidal access to reward-based teaching signals. Unexpectedly, the same was true for the thalamus. In further support of this prediction, pallidal LFP amplitudes during stimulus presentation correlated with patients' reward expectations during phases of low reward certainty - suggesting pallidal participation in the learning of stimulus-response associations. In line with our second prediction, correlations between thalamic stimulus-related LFP amplitudes and patients' reward expectations were significant within phases of already high reward certainty, suggesting thalamic participation in exploitation.

Combined lesions of direct and indirect basal ganglia pathways but not changes in dopamine levels explain learning deficits in patients with Huntington's disease.

Huntington's disease (HD) is a hereditary neurodegenerative disease of the basal ganglia that causes severe motor, cognitive and emotional dysfunctions. In the human basal ganglia, these dysfunctions are accompanied by a loss of striatal medium spiny neurons, dysfunctions of the subthalamic nucleus and globus pallidus, and changes in dopamine receptor binding. Here, we used a neuro-computational model to investigate which of these basal ganglia dysfunctions can explain patients' deficits in different stimulus-response learning paradigms. We show that these paradigms are particularly suitable for scrutinising the effects of potential changes in dopamine signaling and of potential basal ganglia lesions on overt behavior in HD. We find that combined lesions of direct and indirect basal ganglia pathways, but none of these lesions alone, reproduce patients' learning impairments. Degeneration of medium spiny neurons of the direct pathway accounts for patients' deficits in facilitating correct responses, whereas degeneration of indirect pathway medium spiny neurons explains their impairments in inhibiting dominant but incorrect responses. The empirical results cannot be explained by lesions of the subthalamic nucleus, which is part of the hyperdirect pathway, or by changes in dopamine levels. Overall, our simulations suggest combined lesions of direct and indirect pathways as a major source of HD patients' learning impairments and, tentatively, also their motor and cognitive deficits in general, whereas changes in dopamine levels are suggested to not be causally related to patients' impairments.

Open and closed cortico-subcortical loops: A neuro-computational account of access to consciousness in the distractor-induced blindness paradigm.

How the brain decides which information to process 'consciously' has been debated over for decades without a simple explanation at hand. While most experiments manipulate the perceptual energy of presented stimuli, the distractor-induced blindness task is a prototypical paradigm to investigate gating of information into consciousness without or with only minor visual manipulation. In this paradigm, subjects are asked to report intervals of coherent dot motion in a rapid serial visual presentation (RSVP) stream, whenever these are preceded by a particular color stimulus in a different RSVP stream. If distractors (i.e., intervals of coherent dot motion prior to the color stimulus) are shown, subjects' abilities to perceive and report intervals of target dot motion decrease, particularly with short delays between intervals of target color and target motion. We propose a biologically plausible neuro-computational model of how the brain controls access to consciousness to explain how distractor-induced blindness originates from information processing in the cortex and basal ganglia. The model suggests that conscious perception requires reverberation of activity in cortico-subcortical loops and that basal-ganglia pathways can either allow or inhibit this reverberation. In the distractor-induced blindness paradigm, inadequate distractor-induced response tendencies are suppressed by the inhibitory 'hyperdirect' pathway of the basal ganglia. If a target follows such a distractor closely, temporal aftereffects of distractor suppression prevent target identification. The model reproduces experimental data on how delays between target color and target motion affect the probability of target detection.

Dysfunctional and compensatory synaptic plasticity in Parkinson's disease.

In Parkinson's disease, a loss of dopamine neurons causes severe motor impairments. These motor impairments have long been thought to result exclusively from immediate effects of dopamine loss on neuronal firing in basal ganglia, causing imbalances of basal ganglia pathways. However, motor impairments and pathway imbalances may also result from dysfunctional synaptic plasticity - a novel concept of how Parkinsonian symptoms evolve. Here we built a neuro-computational model that allows us to simulate the effects of dopamine loss on synaptic plasticity in basal ganglia. Our simulations confirm that dysfunctional synaptic plasticity can indeed explain the emergence of both motor impairments and pathway imbalances in Parkinson's disease, thus corroborating the novel concept. By predicting that dysfunctional plasticity results not only in reduced activation of desired responses, but also in their active inhibition, our simulations provide novel testable predictions. When simulating dopamine replacement therapy (which is a standard treatment in clinical practice), we observe a new balance of pathway outputs, rather than a simple restoration of non-Parkinsonian states. In addition, high doses of replacement are shown to result in overshooting motor activity, in line with empirical evidence. Finally, our simulations provide an explanation for the intensely debated paradox that focused basal ganglia lesions alleviate Parkinsonian symptoms, but do not impair performance in healthy animals. Overall, our simulations suggest that the effects of dopamine loss on synaptic plasticity play an essential role in the development of Parkinsonian symptoms, thus arguing for a re-conceptualisation of Parkinsonian pathophysiology.

Exploration behavior after reversals is predicted by STN-GPe synaptic plasticity in a basal ganglia model.

Humans can quickly adapt their behavior to changes in the environment. Classical reversal learning tasks mainly measure how well participants can disengage from a previously successful behavior but not how alternative responses are explored. Here, we propose a novel 5-choice reversal learning task with alternating position-reward contingencies to study exploration behavior after a reversal. We compare human exploratory saccade behavior with a prediction obtained from a neuro-computational model of the basal ganglia. A new synaptic plasticity rule for learning the connectivity between the subthalamic nucleus (STN) and external globus pallidus (GPe) results in exploration biases to previously rewarded positions. The model simulations and human data both show that during experimental experience exploration becomes limited to only those positions that have been rewarded in the past. Our study demonstrates how quite complex behavior may result from a simple sub-circuit within the basal ganglia pathways.

A large-scale neurocomputational model of spatial cognition integrating memory with vision.

We introduce a large-scale neurocomputational model of spatial cognition called 'Spacecog', which integrates recent findings from mechanistic models of visual and spatial perception. As a high-level cognitive ability, spatial cognition requires the processing of behaviourally relevant features in complex environments and, importantly, the updating of this information during processes of eye and body movement. The Spacecog model achieves this by interfacing spatial memory and imagery with mechanisms of object localisation, saccade execution, and attention through coordinate transformations in parietal areas of the brain. We evaluate the model in a realistic virtual environment where our neurocognitive model steers an agent to perform complex visuospatial tasks. Our modelling approach opens up new possibilities in the assessment of neuropsychological data and human spatial cognition.

A computational model for the influence of corollary discharge and proprioception on the perisaccadic mislocalization of briefly presented stimuli in complete darkness.

Spatial perception, the localization of stimuli in space, can rely on visual reference stimuli or on egocentric factors such as a stimulus position relative to eye gaze. In total darkness, only an egocentric reference frame provides sufficient information. When stimuli are briefly flashed around saccades, the localization error reveals potential mechanisms of updating such reference frames as described in several theories and computational models. Recent novel experimental evidence, however, showed that the maximum amount of mislocalization does not scale linearly with saccade amplitude but rather stays below 13° even for long saccades, which is different from predicted by present models. We propose a new model of perisaccadic mislocalization in complete darkness to account for this observation. According to this model, mislocalization arises not on the motor side by comparing a retinal position signal with an extraretinal eye position related signal but by updating stimulus position in visual areas through a combination of proprioceptive eye position and corollary discharge. Simulations with realistic input signals and temporal dynamics show that both signals together are used for spatial updating and in turn bring about perisaccadic mislocalization.

Anticipatory saccade target processing and the presaccadic transfer of visual features.

As we shift our gaze to explore the visual world, information enters cortex in a sequence of successive snapshots, interrupted by phases of blur. Our experience, in contrast, appears like a movie of a continuous stream of objects embedded in a stable world. This perception of stability across eye movements has been linked to changes in spatial sensitivity of visual neurons anticipating the upcoming saccade, often referred to as shifting receptive fields (Duhamel et al., 1992; Walker et al., 1995; Umeno and Goldberg, 1997; Nakamura and Colby, 2002). How exactly these receptive field dynamics contribute to perceptual stability is currently not clear. Anticipatory receptive field shifts toward the future, postsaccadic position may bridge the transient perisaccadic epoch (Sommer and Wurtz, 2006; Wurtz, 2008; Melcher and Colby, 2008). Alternatively, a presaccadic shift of receptive fields toward the saccade target area (Tolias et al., 2001) may serve to focus visual resources onto the most relevant objects in the postsaccadic scene (Hamker et al., 2008). In this view, shifts of feature detectors serve to facilitate the processing of the peripheral visual content before it is foveated. While this conception is consistent with previous observations on receptive field dynamics and on perisaccadic compression (Ross et al., 1997; Morrone et al., 1997; Kaiser and Lappe, 2004), it predicts that receptive fields beyond the saccade target shift toward the saccade target rather than in the direction of the saccade. We have tested this prediction in human observers via the presaccadic transfer of the tilt-aftereffect (Melcher, 2007).