How cortico-basal ganglia-thalamic subnetworks can shift decision policies to maximize reward rate.

All mammals exhibit flexible decision policies that depend, at least in part, on the cortico-basal ganglia-thalamic (CBGT) pathways. Yet understanding how the complex connectivity, dynamics, and plasticity of CBGT circuits translates into experience-dependent shifts of decision policies represents a longstanding challenge in neuroscience. Here we used a computational approach to address this problem. Specifically, we simulated decisions driven by CBGT circuits under baseline, unrewarded conditions using a spiking neural network, and fit the resulting behavior to an evidence accumulation model. Using canonical correlation analysis, we then replicated the existence of three recently identified control ensembles (responsiveness, pliancy and choice) within CBGT circuits, with each ensemble mapping to a specific configuration of the evidence accumulation process. We subsequently simulated learning in a simple two-choice task with one optimal (i.e., rewarded) target. We find that value-based learning, via dopaminergic signals acting on cortico-striatal synapses, effectively manages the speed-accuracy tradeoff so as to increase reward rate over time. Within this process, learning-related changes in decision policy can be decomposed in terms of the contributions of each control ensemble, and these changes are driven by sequential reward prediction errors on individual trials. Our results provide a clear and simple mechanism for how dopaminergic plasticity shifts specific subnetworks within CBGT circuits so as to strategically modulate decision policies in order to maximize effective reward rate.

Arkypallidal neurons in the external globus pallidus can mediate inhibitory control by altering competition in the striatum.

Reactive inhibitory control is crucial for survival. Traditionally, this control in mammals was attributed solely to the hyperdirect pathway, with cortical control signals flowing unidirectionally from the subthalamic nucleus (STN) to basal ganglia output regions. Yet recent findings have put this model into question, suggesting that the STN is assisted in stopping actions through ascending control signals to the striatum mediated by the external globus pallidus (GPe). Here we investigate this suggestion by harnessing a biologically-constrained spiking model of the corticobasal ganglia-thalamic (CBGT) circuit that includes pallidostriatal pathways originating from arkypallidal neurons. Through a series of experiments probing the interaction between three critical inhibitory nodes (the STN, arkypallidal cells, and indirect path-way spiny projection neurons), we find that the GPe acts as a critical mediator of both ascending and descending inhibitory signals in the CBGT circuit. In particular, pallidostriatal pathways regulate this process by weakening the direct pathway dominance of the evidence accumulation process driving decisions, which increases the relative suppressive influence of the indirect pathway on basal ganglia output. These findings delineate how pallidostriatal pathways can facilitate action cancellation by managing the bidirectional flow of information within CBGT circuits.

Diagnostically distinct resting state fMRI energy distributions: A subject-specific maximum entropy modeling study.

Objective: Existing neuroimaging studies of psychotic and mood disorders have reported brain activation differences (first-order properties) and altered pairwise correlation-based functional connectivity (second-order properties). However, both approaches have certain limitations that can be overcome by integrating them in a pairwise maximum entropy model (MEM) that better represents a comprehensive picture of fMRI signal patterns and provides a system-wide summary measure called energy. This study examines the applicability of individual-level MEM for psychiatry and identifies image-derived model coefficients related to model parameters. Method: MEMs are fit to resting state fMRI data from each individual with schizophrenia/schizoaffective disorder, bipolar disorder, and major depression (n=132) and demographically matched healthy controls (n=132) from the UK Biobank to different subsets of the default mode network (DMN) regions. Results: The model satisfactorily explained observed brain energy state occurrence probabilities across all participants, and model parameters were significantly correlated with image-derived coefficients for all groups. Within clinical groups, averaged energy level distributions were higher in schizophrenia/schizoaffective disorder but lower in bipolar disorder compared to controls for both bilateral and unilateral DMN. Major depression energy distributions were higher compared to controls only in the right hemisphere DMN. Conclusions: Diagnostically distinct energy states suggest that probability distributions of temporal changes in synchronously active nodes may underlie each diagnostic entity. Subject-specific MEMs allow for factoring in the individual variations compared to traditional group-level inferences, offering an improved measure of biologically meaningful correlates of brain activity that may have potential clinical utility.

Energy in functional brain states correlates with cognition in adolescent schizophrenia and healthy persons.

Adolescent-onset schizophrenia (AOS) is a relatively rare and under-studied form of schizophrenia with more severe cognitive impairments and poorer outcome compared to adult-onset schizophrenia. Several neuroimaging studies have reported alterations in regional activations that account for activity in individual regions (first-order model) and functional connectivity that reveals pairwise co-activations (second-order model) in AOS compared to controls. The pairwise maximum entropy model, also called the Ising model, can integrate both first-order and second-order terms to elucidate a comprehensive picture of neural dynamics and captures both individual and pairwise activity measures into a single quantity known as energy, which is inversely related to the probability of state occurrence. We applied the MEM framework to task functional MRI data collected on 23 AOS individuals in comparison with 53 healthy control subjects while performing the Penn Conditional Exclusion Test (PCET), which measures executive function that has been repeatedly shown to be more impaired in AOS compared to adult-onset schizophrenia. Accuracy of PCET performance was significantly reduced among AOS compared to controls as expected. Average cumulative energy achieved for a participant over the course of the fMRI negatively correlated with task performance, and the association was stronger than any first-order associations. The AOS subjects spent more time in higher energy states that represent lower probability of occurrence and were associated with impaired executive function suggesting that the neural dynamics may be less efficient compared to controls who spent more time in lower energy states occurring with higher probability and hence are more stable and efficient. The energy landscapes in both conditions featured attractors that corresponded to two distinct subnetworks, namely fronto-temporal and parieto-motor. Attractor basins were larger in the controls than in AOS; moreover, fronto-temporal basin size was significantly correlated with cognitive performance in controls but not among the AOS. The single trial trajectories for the AOS group also showed higher variability in concordance with shallow attractor basins among AOS. These findings suggest that the neural dynamics of AOS features more frequent occurrence of less probable states with narrower attractors, which lack the relation to executive function associated with attractors in control subjects suggesting a diminished capacity of AOS to generate task-effective brain states.

Competing neural representations of choice shape evidence accumulation in humans.

Making adaptive choices in dynamic environments requires flexible decision policies. Previously, we showed how shifts in outcome contingency change the evidence accumulation process that determines decision policies. Using in silico experiments to generate predictions, here we show how the cortico-basal ganglia-thalamic (CBGT) circuits can feasibly implement shifts in decision policies. When action contingencies change, dopaminergic plasticity redirects the balance of power, both within and between action representations, to divert the flow of evidence from one option to another. When competition between action representations is highest, the rate of evidence accumulation is the lowest. This prediction was validated in in vivo experiments on human participants, using fMRI, which showed that (1) evoked hemodynamic responses can reliably predict trial-wise choices and (2) competition between action representations, measured using a classifier model, tracked with changes in the rate of evidence accumulation. These results paint a holistic picture of how CBGT circuits manage and adapt the evidence accumulation process in mammals.

Different Faces of Medial Beta-Band Activity Reflect Distinct Visuomotor Feedback Signals.

Beta-band (13-35 Hz) modulations following reward, task outcome feedback, and error have been described in cognitive and/or motor adaptation tasks. Observations from different studies are, however, difficult to conciliate. Among the studies that used cognitive response selection tasks, several reported an increase in beta-band activity following reward, whereas others observed increased beta power after negative feedback. Moreover, in motor adaptation tasks, an attenuation of the postmovement beta rebound follows a movement execution error induced by visual or mechanical perturbations. Given that kinematic error typically leads to negative task-outcome feedback (e.g., target missed), one may wonder how contradictory modulations, beta power decrease with movement error versus beta power increase with negative feedback, may coexist. We designed a motor adaptation task in which female and male participants experience varied feedbacks-binary success/failure feedback, kinematic error, and sensory-prediction error-and demonstrate that beta-band modulations in opposite directions coexist at different spatial locations, time windows, and frequency ranges. First, high beta power in the medial frontal cortex showed opposite modulations well separated in time when compared in success and failure trials; that is, power was higher in success trials just after the binary success feedback, whereas it was lower in the postmovement period compared with failure trials. Second, although medial frontal high-beta activity was sensitive to task outcome, low-beta power in the medial parietal cortex was strongly attenuated following movement execution error but was not affected by either the outcome of the task or sensory-prediction error. These findings suggest that medial beta activity in different spatio-temporal-spectral configurations play a multifaceted role in encoding qualitatively distinct feedback signals.SIGNIFICANCE STATEMENT Beta-band activity reflects neural processes well beyond sensorimotor functions, including cognition and motivation. By disentangling alternative spatio-temporal-spectral patterns of possible beta-oscillatory activity, we reconcile a seemingly discrepant literature. First, high-beta power in the medial frontal cortex showed opposite modulations separated in time in success and failure trials; power was higher in success trials just after success feedback and lower in the postmovement period compared with failure trials. Second, although medial frontal high-beta activity was sensitive to task outcome, low-beta power in the medial parietal cortex was strongly attenuated following movement execution error but was not affected by the task outcome or the sensory-prediction error. We propose that medial beta activity reflects distinct feedback signals depending on its anatomic location, time window, and frequency range.

CBGTPy: An extensible cortico-basal ganglia-thalamic framework for modeling biological decision making.

Here we introduce CBGTPy, a virtual environment for designing and testing goal-directed agents with internal dynamics that are modeled off of the cortico-basal-ganglia-thalamic (CBGT) pathways in the mammalian brain. CBGTPy enables researchers to investigate the internal dynamics of the CBGT system during a variety of tasks, allowing for the formation of testable predictions about animal behavior and neural activity. The framework has been designed around the principle of flexibility, such that many experimental parameters in a decision making paradigm can be easily defined and modified. Here we demonstrate the capabilities of CBGTPy across a range of single and multi-choice tasks, highlighting the ease of set up and the biologically realistic behavior that it produces. We show that CBGTPy is extensible enough to apply to a wide range of experimental protocols and to allow for the implementation of model extensions with minimal developmental effort.

Cerebellar connectivity maps embody individual adaptive behavior in mice.

The cerebellar cortex encodes sensorimotor adaptation during skilled locomotor behaviors, however the precise relationship between synaptic connectivity and behavior is unclear. We studied synaptic connectivity between granule cells (GCs) and Purkinje cells (PCs) in murine acute cerebellar slices using photostimulation of caged glutamate combined with patch-clamp in developing or after mice adapted to different locomotor contexts. By translating individual maps into graph network entities, we found that synaptic maps in juvenile animals undergo critical period characterized by dissolution of their structure followed by the re-establishment of a patchy functional organization in adults. Although, in adapted mice, subdivisions in anatomical microzones do not fully account for the observed spatial map organization in relation to behavior, we can discriminate locomotor contexts with high accuracy. We also demonstrate that the variability observed in connectivity maps directly accounts for motor behavior traits at the individual level. Our findings suggest that, beyond general motor contexts, GC-PC networks also encode internal models underlying individual-specific motor adaptation.

Uncoupling the roles of firing rates and spike bursts in shaping the STN-GPe beta band oscillations.

The excess of 15-30 Hz (ß-band) oscillations in the basal ganglia is one of the key signatures of Parkinson's disease (PD). The STN-GPe network is integral to generation and modulation of ß band oscillations in basal ganglia. However, the role of changes in the firing rates and spike bursting of STN and GPe neurons in shaping these oscillations has remained unclear. In order to uncouple their effects, we studied the dynamics of STN-GPe network using numerical simulations. In particular, we used a neuron model, in which firing rates and spike bursting can be independently controlled. Using this model, we found that while STN firing rate is predictive of oscillations, GPe firing rate is not. The effect of spike bursting in STN and GPe neurons was state-dependent. That is, only when the network was operating in a state close to the border of oscillatory and non-oscillatory regimes, spike bursting had a qualitative effect on the ß band oscillations. In these network states, an increase in GPe bursting enhanced the oscillations whereas an equivalent proportion of spike bursting in STN suppressed the oscillations. These results provide new insights into the mechanisms underlying the transient ß bursts and how duration and power of ß band oscillations may be controlled by an interplay of GPe and STN firing rates and spike bursts.

Exploring the role of striatal D1 and D2 medium spiny neurons in action selection using a virtual robotic framework.

The basal ganglia have been hypothesized to be involved in action selection, i.e. resolving competition between simultaneously activated motor programs. It has been shown that the direct pathway facilitates action execution whereas the indirect pathway inhibits it. However, as the pathways are both active during an action, it remains unclear whether their role is co-operative or competitive. In order to investigate this issue, we developed a striatal model consisting of D1 and D2 medium spiny neurons (MSNs) and interfaced it to a simulated robot moving in an environment. We demonstrate that this model is able to reproduce key behavioral features of several experiments involving optogenetic manipulation of the striatum, such as freezing and ambulation. We then investigate the interaction of D1- and D2-MSNs. We find that their fundamental relationship is co-operative within a channel and competitive between channels; this turns out to be crucial for action selection. However, individual pairs of D1- and D2-MSNs may exhibit predominantly competition or co-operation depending on their distance, and D1- and D2-MSNs population activity can alternate between co-operation and competition modes during a stimulation. Additionally, our results show that D2-D2 connectivity between channels is necessary for effective resolution of competition; in its absence, a conflict of two motor programs typically results in neither being selected.

Homologous Basal Ganglia Network Models in Physiological and Parkinsonian Conditions.

The classical model of basal ganglia has been refined in recent years with discoveries of subpopulations within a nucleus and previously unknown projections. One such discovery is the presence of subpopulations of arkypallidal and prototypical neurons in external globus pallidus, which was previously considered to be a primarily homogeneous nucleus. Developing a computational model of these multiple interconnected nuclei is challenging, because the strengths of the connections are largely unknown. We therefore use a genetic algorithm to search for the unknown connectivity parameters in a firing rate model. We apply a binary cost function derived from empirical firing rate and phase relationship data for the physiological and Parkinsonian conditions. Our approach generates ensembles of over 1,000 configurations, or homologies, for each condition, with broad distributions for many of the parameter values and overlap between the two conditions. However, the resulting effective weights of connections from or to prototypical and arkypallidal neurons are consistent with the experimental data. We investigate the significance of the weight variability by manipulating the parameters individually and cumulatively, and conclude that the correlation observed between the parameters is necessary for generating the dynamics of the two conditions. We then investigate the response of the networks to a transient cortical stimulus, and demonstrate that networks classified as physiological effectively suppress activity in the internal globus pallidus, and are not susceptible to oscillations, whereas parkinsonian networks show the opposite tendency. Thus, we conclude that the rates and phase relationships observed in the globus pallidus are predictive of experimentally observed higher level dynamical features of the physiological and parkinsonian basal ganglia, and that the multiplicity of solutions generated by our method may well be indicative of a natural diversity in basal ganglia networks. We propose that our approach of generating and analyzing an ensemble of multiple solutions to an underdetermined network model provides greater confidence in its predictions than those derived from a unique solution, and that projecting such homologous networks on a lower dimensional space of sensibly chosen dynamical features gives a better chance than a purely structural analysis at understanding complex pathologies such as Parkinson's disease.

Activity Dynamics and Signal Representation in a Striatal Network Model with Distance-Dependent Connectivity.

The striatum is the main input nucleus of the basal ganglia. Characterizing striatal activity dynamics is crucial to understanding mechanisms underlying action selection, initiation, and execution. Here, we studied the effects of spatial network connectivity on the spatiotemporal structure of striatal activity. We show that a striatal network with nonmonotonically changing distance-dependent connectivity (according to a gamma distribution) can exhibit a wide repertoire of spatiotemporal dynamics, ranging from spatially homogeneous, asynchronous-irregular (AI) activity to a state with stable, spatially localized activity bumps, as in "winner-take-all" (WTA) dynamics. Among these regimes, the unstable activity bumps [transition activity (TA)] regime closely resembles the experimentally observed spatiotemporal activity dynamics and neuronal assemblies in the striatum. In contrast, striatal networks with monotonically decreasing distance-dependent connectivity (in a Gaussian fashion) can exhibit only an AI state. Thus, given the observation of spatially compact neuronal clusters in the striatum, our model suggests that recurrent connectivity among striatal projection neurons should vary nonmonotonically. In brain disorders such as Parkinson's disease, increased cortical inputs and high striatal firing rates are associated with a reduction in stimulus sensitivity. Consistent with this, our model suggests that strong cortical inputs drive the striatum to a WTA state, leading to low stimulus sensitivity and high variability. In contrast, the AI and TA states show high stimulus sensitivity and reliability. Thus, based on these results, we propose that in a healthy state the striatum operates in a AI/TA state and that lack of dopamine pushes it into a WTA state.

Existence and control of Go/No-Go decision transition threshold in the striatum.

A typical Go/No-Go decision is suggested to be implemented in the brain via the activation of the direct or indirect pathway in the basal ganglia. Medium spiny neurons (MSNs) in the striatum, receiving input from cortex and projecting to the direct and indirect pathways express D1 and D2 type dopamine receptors, respectively. Recently, it has become clear that the two types of MSNs markedly differ in their mutual and recurrent connectivities as well as feedforward inhibition from FSIs. Therefore, to understand striatal function in action selection, it is of key importance to identify the role of the distinct connectivities within and between the two types of MSNs on the balance of their activity. Here, we used both a reduced firing rate model and numerical simulations of a spiking network model of the striatum to analyze the dynamic balance of spiking activities in D1 and D2 MSNs. We show that the asymmetric connectivity of the two types of MSNs renders the striatum into a threshold device, indicating the state of cortical input rates and correlations by the relative activity rates of D1 and D2 MSNs. Next, we describe how this striatal threshold can be effectively modulated by the activity of fast spiking interneurons, by the dopamine level, and by the activity of the GPe via pallidostriatal backprojections. We show that multiple mechanisms exist in the basal ganglia for biasing striatal output in favour of either the `Go' or the `No-Go' pathway. This new understanding of striatal network dynamics provides novel insights into the putative role of the striatum in various behavioral deficits in patients with Parkinson's disease, including increased reaction times, L-Dopa-induced dyskinesia, and deep brain stimulation-induced impulsivity.