Striatal cholinergic interneurons exert inhibition on competing default behaviours controlled by the nucleus accumbens and dorsolateral striatum.

With repeated practice, learned actions become more skilled, and eventually highly stereotypical. This transition is accompanied by a shift in striatal control over behaviour from ventral and dorsomedial striatum to dorsolateral striatum. The cholinergic interneurons (CINs) in the striatum are central to striatal computation. Yet, their role in the transition from motivated to stereotypic behaviour is still unclear. In this study, we examined whether CINs contribute to the competition between both control systems. We selectively lesioned CINs in the nucleus accumbens (NAc) or in the dorsolateral striatum (DLS) of rats trained in a cued maze task. After obtaining skilled performance, we manipulated the motivation for reward. While sparing task acquisition, selective lesions of the CINs had a marked dissociable impact on the sensitivity to motivation in the highly skilled state. Selective lesions of CINs increased automaticity of behaviour when performed in the DLS, but increased sensitivity to motivation in the NAc. These findings indicate a central role of CINs in regulating motivational impact on striatally controlled behaviours.

The chicken and egg problem of grid cells and place cells.

Place cells and grid cells are major building blocks of the hippocampal cognitive map. The prominent forward model postulates that grid-cell modules are generated by a continuous attractor network; that a velocity signal evoked during locomotion moves entorhinal activity bumps; and that place-cell activity constitutes summation of entorhinal grid-cell modules. Experimental data support the first postulate, but not the latter two. Several families of solutions that depart from these postulates have been put forward. We suggest a modified model (spatial modulation continuous attractor network; SCAN), whereby place cells are generated from spatially selective nongrid cells. Locomotion causes these cells to move the hippocampal activity bump, leading to movement of the entorhinal manifolds. Such inversion accords with the shift of hippocampal thought from navigation to more abstract functions.

Active experience, not time, determines within-day representational drift in dorsal CA1.

Memories of past events can be recalled long after the event, indicating stability. But new experiences are also integrated into existing memories, indicating plasticity. In the hippocampus, spatial representations are known to remain stable but have also been shown to drift over long periods of time. We hypothesized that experience, more than the passage of time, is the driving force behind representational drift. We compared the within-day stability of place cells' representations in dorsal CA1 of the hippocampus of mice traversing two similar, familiar tracks for different durations. We found that the more time the animals spent actively traversing the environment, the greater the representational drift, regardless of the total elapsed time between visits. Our results suggest that spatial representation is a dynamic process, related to the ongoing experiences within a specific context, and is related to memory update rather than to passive forgetting.

Flexible coding of time or distance in hippocampal cells.

Analysis of neuronal activity in the hippocampus of behaving animals has revealed cells acting as 'Time Cells', which exhibit selective spiking patterns at specific time intervals since a triggering event, and 'Distance Cells', which encode the traversal of specific distances. Other neurons exhibit a combination of these features, alongside place selectivity. This study aims to investigate how the task performed by animals during recording sessions influences the formation of these representations. We analyzed data from a treadmill running study conducted by Kraus et al., 2013, in which rats were trained to run at different velocities. The rats were recorded in two trial contexts: a 'fixed time' condition, where the animal ran on the treadmill for a predetermined duration before proceeding, and a 'fixed distance' condition, where the animal ran a specific distance on the treadmill. Our findings indicate that the type of experimental condition significantly influenced the encoding of hippocampal cells. Specifically, distance-encoding cells dominated in fixed-distance experiments, whereas time-encoding cells dominated in fixed-time experiments. These results underscore the flexible coding capabilities of the hippocampus, which are shaped by over-representation of salient variables associated with reward conditions.

Cannabinoids disrupt hippocampal sharp wave-ripples via inhibition of glutamate release.

Cannabis consumption results in impaired learning. The proper synchronization of neuronal activity in the mammalian hippocampus gives rise to network rhythms that are implicated in memory formation. Here, we have studied the impact of cannabinoids on hippocampal sharp waves and associated ripple oscillations using field- and whole-cell voltage-clamp recordings. We demonstrate that the activation of cannabinoid receptor 1 suppresses sharp wave-ripples (SWRs) in mice in vivo and in vitro. This suppression was paralleled by a selective reduction of SWR-associated inward but not outward charge transfer, demonstrating an impairment of excitation due to cannabinoid exposure. Adenosine, a presynaptic modulator of glutamate release, mimicked and occluded the observed consequences of cannabinoids on SWRs. We conclude that inhibition of glutamatergic feed-forward excitation can explain cannabinoid-mediated disruption of SWRs and may account for cannabinoid-induced impairment of hippocampus-dependent memory.

Hippocampal sub-networks exhibit distinct spatial representation deficits in Alzheimer's disease model mice.

Not much is known about how the dentate gyrus (DG) and hippocampal CA3 networks, critical for memory and spatial processing, malfunction in Alzheimer's disease (AD). While studies of associative memory deficits in AD have focused mainly on behavior, here, we directly measured neurophysiological network dysfunction. We asked what the pattern of deterioration of different networks is during disease progression. We investigated how the associative memory-processing capabilities in different hippocampal subfields are affected by familial AD (fAD) mutations leading to amyloid-ß dyshomeostasis. Specifically, we focused on the DG and CA3, which are known to be involved in pattern completion and separation and are susceptible to pathological alterations in AD. To identify AD-related deficits in neural-ensemble dynamics, we recorded single-unit activity in wild-type (WT) and fAD model mice (APPSwe+PSEN1/ΔE9) in a novel tactile morph task, which utilizes the extremely developed somatosensory modality of mice. As expected from the sub-network regional specialization, we found that tactile changes induced lower rate map correlations in the DG than in CA3 of WT mice. This reflects DG pattern separation and CA3 pattern completion. In contrast, in fAD model mice, we observed pattern separation deficits in the DG and pattern completion deficits in CA3. This demonstration of region-dependent impairments in fAD model mice contributes to understanding of brain networks deterioration during fAD progression. Furthermore, it implies that the deterioration cannot be studied generally throughout the hippocampus but must be researched at a finer resolution of microcircuits. This opens novel systems-level approaches for analyzing AD-related neural network deficits.

Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory.

Decision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.

Striatal action-learning based on dopamine concentration.

The reinforcement learning hypothesis of dopamine function predicts that dopamine acts as a teaching signal by governing synaptic plasticity in the striatum. Induced changes in synaptic strength enable the cortico-striatal network to learn a mapping between situations and actions that lead to a reward. A review of the relevant neurophysiology of dopamine function in the cortico-striatal network and the machine reinforcement learning hypothesis reveals an apparent mismatch with recent electrophysiological studies. It was found that in addition to the well-described reward-related responses, a subpopulation of dopamine neurons also exhibits phasic responses to aversive stimuli or to cues predicting aversive stimuli. Obviously, actions that lead to aversive events should not be reinforced. However, published data suggest that the phasic responses of dopamine neurons to reward-related stimuli have a higher firing rate and have a longer duration than phasic responses of dopamine neurons to aversion-related stimuli. We propose that based on different dopamine concentrations, the target structures are able to decode reward-related dopamine from aversion-related dopamine responses. Thereby, the learning of actions in the basal-ganglia network integrates information about both costs and benefits. This hypothesis predicts that dopamine concentration should be a crucial parameter for plasticity rules at cortico-striatal synapses. Recent in vitro studies on cortico-striatal synaptic plasticity rules support a striatal action-learning scheme where during reward-related dopamine release dopamine-dependent forms of synaptic plasticity occur, while during aversion-related dopamine release the dopamine concentration only allows dopamine-independent forms of synaptic plasticity to occur.

Midbrain dopamine neurons encode decisions for future action.

Current models of the basal ganglia and dopamine neurons emphasize their role in reinforcement learning. However, the role of dopamine neurons in decision making is still unclear. We recorded from dopamine neurons in monkeys engaged in two types of trial: reference trials in an instructed-choice task and decision trials in a two-armed bandit decision task. We show that the activity of dopamine neurons in the decision setting is modulated according to the value of the upcoming action. Moreover, analysis of the probability matching strategy in the decision trials revealed that the dopamine population activity and not the reward during reference trials determines choice behavior. Because dopamine neurons do not have spatial or motor properties, we conclude that immediate decisions are likely to be generated elsewhere and conveyed to the dopamine neurons, which play a role in shaping long-term decision policy through dynamic modulation of the efficacy of basal ganglia synapses.

Odor Concentration Change Coding in the Olfactory Bulb.

Dynamical changes in the environment strongly impact our perception. Likewise, sensory systems preferentially represent stimulus changes, enhancing temporal contrast. In olfaction, odor concentration changes across consecutive inhalations (ΔCt ) can guide odor source localization, yet the neural representation of ΔCt has not been studied in vertebrates. We have found that, in the mouse olfactory bulb, a subset of mitral/tufted (M/T) cells represents ΔCt , enhancing the contrast between different concentrations. These concentration change responses are direction selective: they respond either to increments or decrements of concentration, reminiscent of ON and OFF selectivity in the retina. This contrast enhancement scales with the magnitude, but not the duration of the concentration step. Further, ΔCt can be read out from the total spike count per sniff, unlike odor identity and intensity, which are represented by fast temporal spike patterns. Our results demonstrate that a subset of M/T cells represents ΔCt , providing a signal that may instruct navigational decisions in downstream olfactory circuits.

A Cellular Mechanism Underlying Enhanced Capability for Complex Olfactory Discrimination Learning.

The biological mechanisms underlying complex forms of learning requiring the understanding of rules based on previous experience are not yet known. Previous studies have raised the intriguing possibility that improvement in complex learning tasks requires the long-term modulation of intrinsic neuronal excitability, induced by reducing the conductance of the slow calcium-dependent potassium current (sIAHP) simultaneously in most neurons in the relevant neuronal networks in several key brain areas. Such sIAHP reduction is expressed in attenuation of the postburst afterhyperpolarization (AHP) potential, and thus in enhanced repetitive action potential firing. Using complex olfactory discrimination (OD) learning as a model for complex learning, we show that brief activation of the GluK2 subtype glutamate receptor results in long-lasting enhancement of neuronal excitability in neurons from controls, but not from trained rats. Such an effect can be obtained by a brief tetanic synaptic stimulation or by direct application of kainate, both of which reduce the postburst AHP in pyramidal neurons. Induction of long-lasting enhancement of neuronal excitability is mediated via a metabotropic process that requires PKC and ERK activation. Intrinsic neuronal excitability cannot be modulated by synaptic activation in neurons from GluK2 knock-out mice. Accordingly, these mice are incapable of learning the complex OD task. Moreover, viral-induced overexpression of Gluk2 in piriform cortex pyramidal neurons results in remarkable enhancement of complex OD learning. Thus, signaling via kainate receptors has a central functional role in higher cognitive abilities.

Dissociation between Postrhinal Cortex and Downstream Parahippocampal Regions in the Representation of Egocentric Boundaries.

Navigation requires the integration of many sensory inputs to form a multi-modal cognitive map of the environment, which is believed to be implemented in the hippocampal region by spatially tuned cells [1-10]. These cells encode various aspects of the environment in a world-based (allocentric) reference frame. Although the cognitive map is represented in allocentric coordinates, the environment is sensed through diverse sensory organs, mostly situated in the animal's head, and therefore represented in sensory and parietal cortices in head-centered egocentric coordinates. Yet it is not clear how and where the brain transforms these head-centered egocentric representations to map-like allocentric representations computed in the hippocampal region. Theoretical modeling has predicted a role for both egocentric and head direction (HD) information in performing an egocentric-allocentric transformation [11-15]. Here, we recorded new data and also used data from a previous study [16]. Adapting a generalized linear model (GLM) classification [17]; we show that the postrhinal cortex (POR) contains a population of pure egocentric boundary cells (EBCs), in contrast with the conjunctive EBCs × HD cells, which we found downstream mostly in the parasubiculum (PaS) and in the medial entorhinal cortex (MEC). Our finding corroborates the idea of a brain network performing an egocentric to allocentric transformation by HD cells. This is a fundamental building block in the formation of the brain's internal cognitive map.

Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons.

Midbrain dopamine and striatal tonically active neurons (TANs, presumed acetylcholine interneurons) signal behavioral significance of environmental events. Since striatal dopamine and acetylcholine affect plasticity of cortico-striatal transmission and are both crucial to learning, they may serve as teachers in the basal ganglia circuits. We recorded from both neuronal populations in monkeys performing a probabilistic instrumental conditioning task. Both neuronal types respond robustly to reward-related events. Although different events yielded responses with different latencies, the responses of the two populations coincided, indicating integration at the target level. Yet, while the dopamine neurons' response reflects mismatch between expectation and outcome in the positive domain, the TANs are invariant to reward predictability. Finally, TAN pairs are synchronized, compared to a minority of dopamine neuron pairs. We conclude that the striatal cholinergic and dopaminergic systems carry distinct messages by different means, which can be integrated differently to shape the basal ganglia responses to reward-related events.

Statistical properties of pauses of the high-frequency discharge neurons in the external segment of the globus pallidus.

The neurons of many basal ganglia nuclei, including the external and internal globus pallidus (GPe and GPi, respectively) and the substantia nigra pars reticulata (SNr) are characterized by their high-frequency (50-100 spikes/s) tonic discharge (HFD). However, the high firing rate of GPe neurons is interrupted by long pauses. We studied the extracellularly recorded spiking activity of 212 well-isolated HFD GPe and 52 GPi/SNr neurons from five monkeys during different states of behavioral activity. An algorithm that maximizes the surprise function was used to detect pauses and pauser cells ("pausers"). Only 6% of the GPi/SNr neurons versus as many as 56% of the GPe neurons were classified as pausers. The GPe average pause duration equals 0.62 s. The interpause intervals follow a Poissonian distribution with a frequency of 13 pauses/minute. No linear relationship was found between pause parameters (duration or frequency) and the firing rate of the cell. Pauses were preceded by various changes in firing rate but not dominantly by a decrease. The average amplitude and duration of the spike waveform was modulated only after the pause but not before it. Pauses of pairs of cells that were recorded simultaneously were not correlated. The probability of GPe cells to pause spontaneously was extremely variable among monkeys (30-90%) and inversely related to the degree of the monkey's motor activity. These findings suggest that spontaneous GPe pauses are related to low-arousal periods and are generated by a process that is independent of the discharge properties of the cells.

Spatial Rule Learning and Corresponding CA1 Place Cell Reorientation Depend on Local Dopamine Release.

Incentives drive goal-directed behavior; however, how they impact the formation and stabilization of goal-relevant hippocampal maps remains unknown. Since dopamine is involved in reward processing, affects hippocampal-dependent behavior, and modulates hippocampal plasticity, we hypothesized that local dopaminergic transmission in the hippocampus serves to mold the formation and updating of hippocampal cognitive maps to adaptively represent reward-predicting space of sensory inputs. We recorded CA1 place cells of rats throughout training on a spatial extra-dimensional set-shift task. After learning to rely on one of two orthogonal sets of cues, we introduced a rule shift and infused locally the D1/5 receptor (D1/5R) antagonist SCH23390. Successful learning was accompanied by place cell reorientation to represent rule-relevant spatial dimension. SCH23390 infusion prevented this remapping and, consequently, impaired learning, causing perseveration. These findings suggest that dopaminergic innervation provides reward information to the hippocampus and is critical for the stabilization of goal-related hippocampal representation, contributing to successful goal-directed behavior.

Animal Learning in a Multidimensional Discrimination Task as Explained by Dimension-Specific Allocation of Attention.

Reinforcement learning describes the process by which during a series of trial-and-error attempts, actions that culminate in reward are reinforced, becoming more likely to be chosen in similar circumstances. When decisions are based on sensory stimuli, an association is formed between the stimulus, the action and the reward. Computational, behavioral and neurobiological accounts of this process successfully explain simple learning of stimuli that differ in one aspect, or along a single stimulus dimension. However, when stimuli may vary across several dimensions, identifying which features are relevant for the reward is not trivial, and the underlying cognitive process is poorly understood. To study this we adapted an intra-dimensional/ extra-dimensional set-shifting paradigm to train rats on a multi-sensory discrimination task. In our setup, stimuli of different modalities (spatial, olfactory and visual) are combined into complex cues and manipulated independently. In each set, only a single stimulus dimension is relevant for reward. To distinguish between learning and decision-making we suggest a weighted attention model (WAM). Our model learns by assigning a separate learning rule for the values of features of each dimension (e.g., for each color), reinforced after every experience. Decisions are made by comparing weighted averages of the learnt values, factored by dimension specific weights. Based on the observed behavior of the rats we estimated the parameters of the WAM and demonstrated that it outperforms an alternative model, in which a learnt value is assigned to each combination of features. Estimated decision weights of the WAM reveal an experience-based bias in learning. In the first experimental set the weights associated with all dimensions were similar. The extra-dimensional shift rendered this dimension irrelevant. However, its decision weight remained high for the early learning stage in this last set, providing an explanation for the poor performance of the animals. Thus, estimated weights can be viewed as a possible way to quantify the experience-based bias.

Information processing, dimensionality reduction and reinforcement learning in the basal ganglia.

Modeling of the basal ganglia has played a major role in our understanding of this elusive group of nuclei. Models of the basal ganglia have undergone evolutionary and revolutionary changes over the last 20 years, as new research in the fields of anatomy, physiology and biochemistry of these nuclei has yielded new information. Early models dealt with a single pathway through the nuclei and focused on the nature of the processing performed within it, convergence of information versus parallel processing of information. Later, the Albin-DeLong "box-and-arrow" model characterized the inter-nuclei interaction as multiple pathways while maintaining a simplistic scalar representation of the nuclei themselves. This model made a breakthrough by providing key insights into the behavior of these nuclei in hypo- and hyper-kinetic movement disorders. The next generation of models elaborated the intra-nuclei interactions and focused on the role of the basal ganglia in action selection and sequence generation which form the most current consensus regarding basal ganglia function in both normal and pathological conditions. However, new findings challenge these models and point to a different neural network approach to information processing in the basal ganglia. Here, we take an in-depth look at the reinforcement driven dimensionality reduction (RDDR) model which postulates that the basal ganglia compress cortical information according to a reinforcement signal using optimal extraction methods. The model provides new insights and experimental predictions on the computational capacity of the basal ganglia and their role in health and disease.

Independent coding of movement direction and reward prediction by single pallidal neurons.

Associating action with its reward value is a basic ability needed by adaptive organisms and requires the convergence of limbic, motor, and associative information. To chart the basal ganglia (BG) involvement in this association, we recorded the activity of 61 well isolated neurons in the external segment of the globus pallidus (GPe) of two monkeys performing a probabilistic visuomotor task. Our results indicate that most (96%) neurons responded to multiple phases of the task. The activity of many (34%) pallidal neurons was modulated solely by direction of movement, and the activity of only a few (3%) pallidal neurons was modulated exclusively by reward prediction. However, the activity of a large number (41%) of single pallidal neurons was comodulated by both expected trial outcome and direction of arm movement. The information carried by the neuronal activity of single pallidal neurons dynamically changed as the trial progressed. The activity was predominantly modulated by both outcome prediction and future movement direction at the beginning of trials and became modulated mainly by movement-direction toward the end of trials. GPe neurons can either increase or decrease their discharge rate in response to predicted future reward. The effects of movement-direction and reward probability on neural activity are linearly summed and thus reflect two independent modulations of pallidal activity. We propose that GPe neurons are uniquely suited for independent processing of a multitude of parameters. This is enabled by the funnel-structure characteristic of the BG architecture, as well as by the anatomical and physiological properties of GPe neurons.

Physiological studies of information processing in the normal and Parkinsonian basal ganglia: pallidal activity in Go/No-Go task and following MPTP treatment.

Understanding the role of the basal ganglia in day to day behavior is critical for a better understanding of the role of these structures in pathological states--such as Parkinson's disease. To elucidate this connection, we studied pallidal activity in a monkey performing a delayed release Go/No-Go task and in monkeys treated with the dopaminergic neurotoxin--MPTP. We compared the results with the predictions of the action selection and reinforcement driven dimensionality reduction models of the basal ganglia. The fraction of responding pallidal neurons, as well as the ratio of positive to negative responses, were equal in the Go and the No-Go modes. The fraction of pallidal neurons with significant responses following the trigger signal (19/26) was higher than that following the visual cue (11/26); however, the fraction of negative responses was significantly higher following the cue signal (47%) than that following the trigger signal (22%). Most (80%) of the cue responses were sensitive to the laterality of the cue, whereas only 25% of the responses following the trigger signal were sensitive to the cue or movement direction. Finally, pallidal spiking activity was not correlated in the normal behaving monkey, and became highly synchronized following MPTP treatment. We conclude that pallidal activity in the normal monkey is consistent with the model of action selection, assuming that action is selected following the visual cue. However, the reinforcement driven dimensionality reduction model is consistent with both the Go/No-Go responses and the normal/MPTP correlation studies.