On the Role of Cortex-Basal Ganglia Interactions for Category Learning: A Neurocomputational Approach.

In addition to the prefrontal cortex (PFC), the basal ganglia (BG) have been increasingly often reported to play a fundamental role in category learning, but the circuit mechanisms mediating their interaction remain to be explored. We developed a novel neurocomputational model of category learning that particularly addresses the BG-PFC interplay. We propose that the BG bias PFC activity by removing the inhibition of cortico-thalamo-cortical loop and thereby provide a teaching signal to guide the acquisition of category representations in the corticocortical associations to the PFC. Our model replicates key behavioral and physiological data of macaque monkey learning a prototype distortion task from Antzoulatos and Miller (2011) Our simulations allowed us to gain a deeper insight into the observed drop of category selectivity in striatal neurons seen in the experimental data and in the model. The simulation results and a new analysis of the experimental data based on the model's predictions show that the drop in category selectivity of the striatum emerges as the variability of responses in the striatum rises when confronting the BG with an increasingly larger number of stimuli to be classified. The neurocomputational model therefore provides new testable insights of systems-level brain circuits involved in category learning that may also be generalized to better understand other cortico-BG-cortical loops.SIGNIFICANCE STATEMENT Inspired by the idea that basal ganglia (BG) teach the prefrontal cortex (PFC) to acquire category representations, we developed a novel neurocomputational model and tested it on a task that was recently applied in monkey experiments. As an advantage over previous models of category learning, our model allows to compare simulation data with single-cell recordings in PFC and BG. We not only derived model predictions, but already verified a prediction to explain the observed drop in striatal category selectivity. When testing our model with a simple, real-world face categorization task, we observed that the fast striatal learning with a performance of 85% correct responses can teach the slower PFC learning to push the model performance up to almost 100%.

Functional segregation of basal ganglia pathways in Parkinson's disease.

Dopamine exerts modulatory signals on cortex-basal ganglia circuits to enable flexible motor control. Parkinson's disease is characterized by a loss of dopaminergic innervation in the basal ganglia leading to complex motor and non-motor symptoms. Clinical symptom alleviation through dopaminergic medication and deep brain stimulation in the subthalamic nucleus likely depends on a complex interplay between converging basal ganglia pathways. As a unique translational research platform, deep brain stimulation allows instantaneous investigation of functional effects of subthalamic neuromodulation in human patients with Parkinson's disease. The present study aims at disentangling the role of the inhibitory basal ganglia pathways in cognitive and kinematic aspects of automatic and controlled movements in healthy and parkinsonian states by combining behavioural experiments, clinical observations, whole-brain deep brain stimulation fibre connectivity mapping and computational modelling. Twenty patients with Parkinson's disease undergoing subthalamic deep brain stimulation and 20 age-matched healthy controls participated in a visuomotor tracking task requiring normal (automatic) and inverted (controlled) reach movements. Parkinsonian patients on and off deep brain stimulation presented complex patterns of reaction time and kinematic changes, when compared to healthy controls. Stimulation of cortico-subthalamic fibres was correlated with reduced reaction time adaptation to task demand, but not kinematic aspects of motor control or alleviation of Parkinson's disease motor signs. By using clinically, behaviourally and fibre tracking informed computational models, our study reveals that loss of cognitive adaptation can be attributed to modulation of the hyperdirect pathway, while kinematic depends on suppression of indirect pathway activity. Our findings suggest that hyperdirect and indirect pathways, converging in the subthalamic nucleus, are differentially involved in cognitive aspects of cautious motor preparation and kinematic gain control during motor performance. Subthalamic deep brain stimulation modulates but does not restore these functions. Intelligent stimulation algorithms could re-enable flexible motor control in Parkinson's disease when adapted to instantaneous environmental demand. Our results may inspire new innovative pathway-specific approaches to reduce side effects and increase therapeutic efficacy of neuromodulation in patients with Parkinson's disease.

Cognitive performance correlates with the degree of dopaminergic degeneration in the associative part of the striatum in non-demented Parkinson's patients.

Parkinson's disease (PD) patients show cognitive deficits that are relevant in terms of prognosis and quality of life. Degeneration of striatal dopaminergic afferents proceeds from dorsal/caudal to anterior/ventral and is discussed to account for some of these symptoms. Treatment with dopamine (DA) has differential effects on cognitive dysfunctions, improving some and worsening others. We hypothesized that cognitive performance during the dopaminergic OFF state correlates with DAT availability in the associative striatum. 16 PD patients underwent motor and cognitive examination ON and OFF DA. Global cognition was measured using the Montréal Cognitive Assessment (MoCA) test and executive functioning using a Stroop test. Nigrostriatal dopaminergic innervation was characterized with [123I]FP-CIT SPECT. A connectivity atlas of the striatum was used to assess DAT availability in functionally defined striatal subregions. Correlations between imaging data and behavioral data OFF medication were calculated. Correlations between DAT availability and MoCA performance in the dopaminergic OFF state was strongest in the associative part of the striatum (r = 0.674, p = 0.004). MoCA test performance did not differ between the ON and the OFF state. There was no correlation of DAT availability with Stroop performance in the OFF state but performance was significantly better during the ON state. Not only motor but also cognitive dysfunctions in PD are associated with striatal dopaminergic depletion. Cognitive decline in non-demented PD patients goes along with nigrostriatal degeneration, most pronounced in the associative subdivision of the striatum. In addition, the present findings suggest that executive dysfunctions are ameliorated by DA whereas global cognition is not improved by dopaminergic medication.

Basal Ganglia dysfunctions in movement disorders: What can be learned from computational simulations.

The basal ganglia are a complex neuronal system that is impaired in several movement disorders, including Parkinson's disease, Huntington's disease, and dystonia. Empirical studies have provided valuable insights into the brain dysfunctions underlying these disorders. The systems-level perspective, however, of how patients' motor, cognitive, and emotional impairments originate from known brain dysfunctions has been a challenge to empirical investigations. These causal relations have been analyzed via computational modeling, a method that describes the simulation of interacting brain processes in a computer system. In this article, we review computational insights into the brain dysfunctions underlying Parkinson's disease, Huntington's disease, and dystonia, with particular foci on dysfunctions of the dopamine system, basal ganglia pathways, and neuronal oscillations. © 2016 International Parkinson and Movement Disorder Society.

Differential contributions of the globus pallidus and ventral thalamus to stimulus-response learning in humans.

The ability to learn associations between stimuli, responses and rewards is a prerequisite for survival. Models of reinforcement learning suggest that the striatum, a basal ganglia input nucleus, vitally contributes to these learning processes. Our recently presented computational model predicts, first, that not only the striatum, but also the globus pallidus contributes to the learning (i.e., exploration) of stimulus-response associations based on rewards. Secondly, it predicts that the stable execution (i.e., exploitation) of well-learned associations involves further learning in the thalamus. To test these predictions, we postoperatively recorded local field potentials (LFPs) from patients that had undergone surgery for deep brain stimulation to treat severe movement disorders. Macroelectrodes were placed either in the globus pallidus or in the ventral thalamus. During recordings, patients performed a reward-based stimulus-response learning task that comprised periods of exploration and exploitation. We analyzed correlations between patients' LFP amplitudes and model-based estimates of their reward expectations and reward prediction errors. In line with our first prediction, pallidal LFP amplitudes during the presentation of rewards and reward omissions correlated with patients' reward prediction errors, suggesting pallidal access to reward-based teaching signals. Unexpectedly, the same was true for the thalamus. In further support of this prediction, pallidal LFP amplitudes during stimulus presentation correlated with patients' reward expectations during phases of low reward certainty - suggesting pallidal participation in the learning of stimulus-response associations. In line with our second prediction, correlations between thalamic stimulus-related LFP amplitudes and patients' reward expectations were significant within phases of already high reward certainty, suggesting thalamic participation in exploitation.

Combined lesions of direct and indirect basal ganglia pathways but not changes in dopamine levels explain learning deficits in patients with Huntington's disease.

Huntington's disease (HD) is a hereditary neurodegenerative disease of the basal ganglia that causes severe motor, cognitive and emotional dysfunctions. In the human basal ganglia, these dysfunctions are accompanied by a loss of striatal medium spiny neurons, dysfunctions of the subthalamic nucleus and globus pallidus, and changes in dopamine receptor binding. Here, we used a neuro-computational model to investigate which of these basal ganglia dysfunctions can explain patients' deficits in different stimulus-response learning paradigms. We show that these paradigms are particularly suitable for scrutinising the effects of potential changes in dopamine signaling and of potential basal ganglia lesions on overt behavior in HD. We find that combined lesions of direct and indirect basal ganglia pathways, but none of these lesions alone, reproduce patients' learning impairments. Degeneration of medium spiny neurons of the direct pathway accounts for patients' deficits in facilitating correct responses, whereas degeneration of indirect pathway medium spiny neurons explains their impairments in inhibiting dominant but incorrect responses. The empirical results cannot be explained by lesions of the subthalamic nucleus, which is part of the hyperdirect pathway, or by changes in dopamine levels. Overall, our simulations suggest combined lesions of direct and indirect pathways as a major source of HD patients' learning impairments and, tentatively, also their motor and cognitive deficits in general, whereas changes in dopamine levels are suggested to not be causally related to patients' impairments.

Open and closed cortico-subcortical loops: A neuro-computational account of access to consciousness in the distractor-induced blindness paradigm.

How the brain decides which information to process 'consciously' has been debated over for decades without a simple explanation at hand. While most experiments manipulate the perceptual energy of presented stimuli, the distractor-induced blindness task is a prototypical paradigm to investigate gating of information into consciousness without or with only minor visual manipulation. In this paradigm, subjects are asked to report intervals of coherent dot motion in a rapid serial visual presentation (RSVP) stream, whenever these are preceded by a particular color stimulus in a different RSVP stream. If distractors (i.e., intervals of coherent dot motion prior to the color stimulus) are shown, subjects' abilities to perceive and report intervals of target dot motion decrease, particularly with short delays between intervals of target color and target motion. We propose a biologically plausible neuro-computational model of how the brain controls access to consciousness to explain how distractor-induced blindness originates from information processing in the cortex and basal ganglia. The model suggests that conscious perception requires reverberation of activity in cortico-subcortical loops and that basal-ganglia pathways can either allow or inhibit this reverberation. In the distractor-induced blindness paradigm, inadequate distractor-induced response tendencies are suppressed by the inhibitory 'hyperdirect' pathway of the basal ganglia. If a target follows such a distractor closely, temporal aftereffects of distractor suppression prevent target identification. The model reproduces experimental data on how delays between target color and target motion affect the probability of target detection.

Dysfunctional and compensatory synaptic plasticity in Parkinson's disease.

In Parkinson's disease, a loss of dopamine neurons causes severe motor impairments. These motor impairments have long been thought to result exclusively from immediate effects of dopamine loss on neuronal firing in basal ganglia, causing imbalances of basal ganglia pathways. However, motor impairments and pathway imbalances may also result from dysfunctional synaptic plasticity - a novel concept of how Parkinsonian symptoms evolve. Here we built a neuro-computational model that allows us to simulate the effects of dopamine loss on synaptic plasticity in basal ganglia. Our simulations confirm that dysfunctional synaptic plasticity can indeed explain the emergence of both motor impairments and pathway imbalances in Parkinson's disease, thus corroborating the novel concept. By predicting that dysfunctional plasticity results not only in reduced activation of desired responses, but also in their active inhibition, our simulations provide novel testable predictions. When simulating dopamine replacement therapy (which is a standard treatment in clinical practice), we observe a new balance of pathway outputs, rather than a simple restoration of non-Parkinsonian states. In addition, high doses of replacement are shown to result in overshooting motor activity, in line with empirical evidence. Finally, our simulations provide an explanation for the intensely debated paradox that focused basal ganglia lesions alleviate Parkinsonian symptoms, but do not impair performance in healthy animals. Overall, our simulations suggest that the effects of dopamine loss on synaptic plasticity play an essential role in the development of Parkinsonian symptoms, thus arguing for a re-conceptualisation of Parkinsonian pathophysiology.

The effect of dopamine on response inhibition in Parkinson's disease relates to age-dependent patterns of nigrostriatal degeneration.

INTRODUCTION: Motor but also non-motor effects are modulated by dopamine (DA) in Parkinson's disease (PD). Impaired inhibition has been related to dopamine overdosing of the associative striatum. We compared effects of dopaminergic medication on inhibitory control in patients with young (age at onset <50 years, YOPD) and late onset PD (LOPD) and related them to nigrostriatal degeneration. METHODS: 27 patients (10 YOPD, 17 LOPD) underwent a Go/NoGo paradigm comprising a global and specific NoGo condition ON and OFF DA. The ratio of dopamine transporter availability (DAT) in the associative relative to the sensorimotor striatum according to [123I]FP-CIT SPECT was compared between YOPD and LOPD (n = 8/12). Neuro-computational modeling was used to identify pathway activation during Go/NoGo performance. RESULTS: Patients made more errors ON compared to OFF in the global NoGo. This DA effect on global NoGo errors correlated with disease duration (r = 0.489, p = 0.010). YOPD made more errors in the specific NoGo ON-OFF compared to LOPD (p = 0.015). YOPD showed higher associative-to-sensorimotor DAT ratios compared to LOPD (p < 0.001). Neuro-computational modeling revealed DA overdosing of the associative striatum in YOPD resulting in excess activation of the direct basal ganglia pathway triggering incorrect responses. CONCLUSIONS: Depending on the age of symptom onset, DA differentially modulated inhibition in PD with detrimental effects on specific NoGo performance in YOPD but increased performance in LOPD. YOPD showed relatively less degeneration in the associative striatum suggesting DA overdosing that is supported by our neuro-computational model. Reduced inhibition in the global NoGo condition suggests different pathway activation.

Reinforcement magnitudes modulate subthalamic beta band activity in patients with Parkinson's disease.

We set out to investigate whether beta oscillations in the human basal ganglia are modulated during reinforcement learning. Based on previous research, we assumed that beta activity might either reflect the magnitudes of individuals' received reinforcements (reinforcement hypothesis), their reinforcement prediction errors (dopamine hypothesis) or their tendencies to repeat versus adapt responses based upon reinforcements (status-quo hypothesis). We tested these hypotheses by recording local field potentials (LFPs) from the subthalamic nuclei of 19 Parkinson's disease patients engaged in a reinforcement-learning paradigm. We then correlated patients' reinforcement magnitudes, reinforcement prediction errors and response repetition tendencies with task-related power changes in their LFP oscillations. During feedback presentation, activity in the frequency range of 14 to 27 Hz (beta spectrum) correlated positively with reinforcement magnitudes. During responding, alpha and low beta activity (6 to 18 Hz) was negatively correlated with previous reinforcement magnitudes. Reinforcement prediction errors and response repetition tendencies did not correlate significantly with LFP oscillations. These results suggest that alpha and beta oscillations during reinforcement learning reflect patients' observed reinforcement magnitudes, rather than their reinforcement prediction errors or their tendencies to repeat versus adapt their responses, arguing both against an involvement of phasic dopamine and against applicability of the status-quo theory.

Subthalamic nucleus stimulation impairs emotional conflict adaptation in Parkinson's disease.

The subthalamic nucleus (STN) occupies a strategic position in the motor network, slowing down responses in situations with conflicting perceptual input. Recent evidence suggests a role of the STN in emotion processing through strong connections with emotion recognition structures. As deep brain stimulation (DBS) of the STN in patients with Parkinson's disease (PD) inhibits monitoring of perceptual and value-based conflict, STN DBS may also interfere with emotional conflict processing. To assess a possible interference of STN DBS with emotional conflict processing, we used an emotional Stroop paradigm. Subjects categorized face stimuli according to their emotional expression while ignoring emotionally congruent or incongruent superimposed word labels. Eleven PD patients ON and OFF STN DBS and eleven age-matched healthy subjects conducted the task. We found conflict-induced response slowing in healthy controls and PD patients OFF DBS, but not ON DBS, suggesting STN DBS to decrease adaptation to within-trial conflict. OFF DBS, patients showed more conflict-induced slowing for negative conflict stimuli, which was diminished by STN DBS. Computational modelling of STN influence on conflict adaptation disclosed DBS to interfere via increased baseline activity.

Patients with Parkinson׳s disease are less affected than healthy persons by relevant response-unrelated features in visual search.

Patients with Parkinson׳s disease (PD) respond more readily than healthy controls to irrelevant stimuli that contain task-relevant, response-priming features. This behavior may reflect oversensitivity to response-relevant features of irrelevant stimuli or failure to select relevant stimuli. To decide between these alternatives, we investigated in a "contingent-capture" paradigm whether PD patients are also oversensitive to task-relevant features that do not prime responses. PD patients and healthy controls had to report the orientation of bars in target color, presented among bars of other colors. Critically, target arrays were preceded by arrays of rings, all gray except one which might be the target color and might be presented at the same position as the upcoming target. Replicating earlier results from young healthy participants (Eimer, Kiss, Press, & Sauter, 2009), signal rings in target color induced an N2pc component over contralateral visual cortex and some positivity at anterior sites (d-P200), both indicative of attentional capture. Correspondingly, signals in target color facilitated correct responding to upcoming targets presented at the same location and impeded correct responses otherwise. Patients with PD had diminished N2pc, lacked the frontal focus of d-P200, and their responses tended to be less affected than healthy participants' by signal position. Thus PD patients appeared less affected than healthy persons by stimuli with relevant features. This outcome is compatible with the notion that PD patients have poorer internal representations of what is relevant in a given task.

Computational models of basal-ganglia pathway functions: focus on functional neuroanatomy.

Over the past 15 years, computational models have had a considerable impact on basal-ganglia research. Most of these models implement multiple distinct basal-ganglia pathways and assume them to fulfill different functions. As there is now a multitude of different models, it has become complex to keep track of their various, sometimes just marginally different assumptions on pathway functions. Moreover, it has become a challenge to oversee to what extent individual assumptions are corroborated or challenged by empirical data. Focusing on computational, but also considering non-computational models, we review influential concepts of pathway functions and show to what extent they are compatible with or contradict each other. Moreover, we outline how empirical evidence favors or challenges specific model assumptions and propose experiments that allow testing assumptions against each other.

The unstable bridge from stimulus processing to correct responding in Parkinson's disease.

It has been assumed that the basal ganglia implement links between stimulus (S) processing and motor response (R). It has also been proposed that the P3b component of the event-related EEG potential, which is usually as large in R- as in S-locked averages over trials, is a candidate marker to reveal integrity of S-R links. Therefore, the P3 complex (consisting of P3a and P3b) was here measured in averages over trials time-locked either to S or to the key-press R. P3b was expected to be equally large in healthy participants' R- and S-locked averages but to be smaller in R-locked than S-locked averages of patients with Parkinson's disease (PD) (n=12 each), reflecting the loss of S-R links in PD. It has been further assumed that the basal ganglia extract unambiguous results from complex signal patterns thereby resolving discrepancies between competing responses. If so, signals arriving at PD patients' cortex may still be discrepant and produce conflicts, as indicated by error-negativity (NE)-type components even with PD patients' correct responses. As expected, healthy persons had equal S- and R- locked P3b amplitudes and topographies, and PD patients had smaller P3b amplitudes in R-locked than S-locked averages. This latter result was due to an R-related fronto-central negative shift, overlapping PD patients' P3b in the final 100 ms before overt responding. This negativity might indicate response conflict even with correct responses. In line with this interpretation, PD patients had an N(E)-type signal in error trials not only in R-locked averages, like healthy participants, but in S-locked averages as well. These findings support the discrepancy-resolving hypothesis of the basal ganglia and suggest that the overlap of P3b by N(E)-type components, related both to S and R, may reflect a major pathophysiological feature in Parkinson's disease.

Open and closed loops: A computational approach to attention and consciousness.

Within recent years, researchers have proposed the independence of attention and consciousness on both empirical and conceptual grounds. However, the elusive nature of these constructs complicates progress in the investigation of their interaction. We present a framework within which we conceptualize attention and consciousness in computational terms. Here, the concepts are consi-dered as large-scale, functionally and structurally different processes, embedded in a biologically inspired architecture, spanning the full arc from stimulus to response. Our architecture assumes a general independence of attention and consciousness, but supposes strong interactions. Furthermore, it addresses the developmental aspect, stressing that these functions have to gradually develop through learning.

Working memory and response selection: a computational account of interactions among cortico-basalganglio-thalamic loops.

Cortico-basalganglio-thalamic loops are involved in both cognitive processes and motor control. We present a biologically meaningful computational model of how these loops contribute to the organization of working memory and the development of response behavior. Via reinforcement learning in basal ganglia, the model develops flexible control of working memory within prefrontal loops and achieves selection of appropriate responses based on working memory content and visual stimulation within a motor loop. We show that both working memory control and response selection can evolve within parallel and interacting cortico-basalganglio-thalamic loops by Hebbian and three-factor learning rules. Furthermore, the model gives a coherent explanation for how complex strategies of working memory control and response selection can derive from basic cognitive operations that can be learned via trial and error.