Effects of Ventral Striatum Lesions on Stimulus-Based versus Action-Based Reinforcement Learning.

Learning the values of actions versus stimuli may depend on separable neural circuits. In the current study, we evaluated the performance of rhesus macaques with ventral striatum (VS) lesions on a two-arm bandit task that had randomly interleaved blocks of stimulus-based and action-based reinforcement learning (RL). Compared with controls, monkeys with VS lesions had deficits in learning to select rewarding images but not rewarding actions. We used a RL model to quantify learning and choice consistency and found that, in stimulus-based RL, the VS lesion monkeys were more influenced by negative feedback and had lower choice consistency than controls. Using a Bayesian model to parse the groups' learning strategies, we also found that VS lesion monkeys defaulted to an action-based choice strategy. Therefore, the VS is involved specifically in learning the value of stimuli, not actions.SIGNIFICANCE STATEMENT Reinforcement learning models of the ventral striatum (VS) often assume that it maintains an estimate of state value. This suggests that it plays a general role in learning whether rewards are assigned based on a chosen action or stimulus. In the present experiment, we examined the effects of VS lesions on monkeys' ability to learn that choosing a particular action or stimulus was more likely to lead to reward. We found that VS lesions caused a specific deficit in the monkeys' ability to discriminate between images with different values, whereas their ability to discriminate between actions with different values remained intact. Our results therefore suggest that the VS plays a specific role in learning to select rewarded stimuli.

Efficient viral expression of a chemogenetic receptor in the old-world monkey amygdala.

Genetically encoded synthetic receptors, such as the chemogenetic and optogenetic proteins, are powerful tools for functional brain studies in animals. In the primate brain, with its comparatively large, intricate anatomical structures, it can be challenging to express transgenes, such as the hM4Di chemogenetic receptor, in a defined anatomical structure with high penetrance. Here, we compare parameters for lentivirus vector injections in the rhesus monkey amygdala. We find that four injections of 20 µl, infused at 0.5 µl/min, can achieve neuronal hM4Di expression in 50-100% of neurons within a 60 mm3 volume, without observable damage from overexpression. Increasing the number of hM4Di_CFP lentivirus injections to up to 12 sites per hemisphere, resulted in 30%-40% neuronal coverage of the overall amygdala volume, with coverage reaching 60% in some subnuclei. Manganese Chloride was mixed with lentivirus and used as an MRI marker to verify targeting accuracy and correct unsuccessful injections in these experiments. In a separate monkey we visualized, in vivo, viral expression of the hM4Di receptor protein in the amygdala, using Positron Emission Tomography. Together, these data show efficient and verifiable expression of a chemogenetic receptor in old-world monkey amygdala.

Rare rewards amplify dopamine responses.

Dopamine prediction error responses are essential components of universal learning mechanisms. However, it is unknown whether individual dopamine neurons reflect the shape of reward distributions. Here, we used symmetrical distributions with differently weighted tails to investigate how the frequency of rewards and reward prediction errors influence dopamine signals. Rare rewards amplified dopamine responses, even when conventional prediction errors were identical, indicating a mechanism for learning the complexities of real-world incentives.

Transcriptional and anatomical diversity of medium spiny neurons in the primate striatum.

Medium spiny neurons (MSNs) constitute the vast majority of striatal neurons and the principal interface between dopamine reward signals and functionally diverse cortico-basal ganglia circuits. Information processing in these circuits is dependent on distinct MSN types: cell types that are traditionally defined according to their projection targets or dopamine receptor expression. Single-cell transcriptional studies have revealed greater MSN heterogeneity than predicted by traditional circuit models, but the transcriptional landscape in the primate striatum remains unknown. Here, we set out to establish molecular definitions for MSN subtypes in Rhesus monkeys and to explore the relationships between transcriptionally defined subtypes and anatomical subdivisions of the striatum. Our results suggest at least nine MSN subtypes, including dorsal striatum subtypes associated with striosome and matrix compartments, ventral striatum subtypes associated with the nucleus accumbens shell and olfactory tubercle, and an MSN-like cell type restricted to µ-opioid receptor rich islands in the ventral striatum. Although each subtype was demarcated by discontinuities in gene expression, continuous variation within subtypes defined gradients corresponding to anatomical locations and, potentially, functional specializations. These results lay the foundation for achieving cell-type-specific transgenesis in the primate striatum and provide a blueprint for investigating circuit-specific information processing.

Dopamine Signals Learn New Tricks.

In this issue of Neuron, Morrens et al. (2020) show that stimulus-evoked dopamine responses are enhanced by novelty and increase the rate at which animals acquire conditioned responses. These results provide a candidate neural mechanism for latent inhibition and illustrate a new role of dopamine signals in learning.