Dopamine transients encode reward prediction errors independent of learning rates.

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented by corticostriatal synaptic weights, which are updated by dopamine-dependent plasticity. This suggests that dopamine release reflects the product of the learning rate and RPE. Here, we characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc) in a volatile environment. Using a task with semi-observable states offering different rewards, we find that rats adjust how quickly they initiate trials across states using RPEs. Computational modeling and behavioral analyses show that learning rates are higher following state transitions and scale with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the NAcc encodes RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

Acetylcholine demixes heterogeneous dopamine signals for learning and moving.

Midbrain dopamine neurons promote reinforcement learning and movement vigor. A major outstanding question is how dopamine-recipient neurons in the striatum parse these heterogeneous signals. Here we characterized dopamine and acetylcholine release in the dorsomedial striatum (DMS) of rats performing a decision-making task. We found that dopamine acted as a reward prediction error (RPE), modulating behavior and DMS spiking on subsequent trials when coincident with pauses in cholinergic release. In contrast, at task events that elicited coincident bursts of acetylcholine and dopamine, dopamine preceded contralateral movements and predicted movement vigor without inducing plastic changes in DMS firing rates. Our findings provide a circuit-level mechanism by which cholinergic modulation allows the same dopamine signals to be used for either movement or learning depending on instantaneous behavioral context.

Dopamine transients encode reward prediction errors independent of learning rates.

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc). We trained rats on a task with semi-observable states offering different rewards, and rats adjusted how quickly they initiated trials across states using RPEs. Computational modeling and behavioral analyses showed that learning rates were higher following state transitions, and scaled with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the NAcc encoded RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

Estrogenic control of reward prediction errors and reinforcement learning.

Gonadal hormones act throughout the brain 1 , and neuropsychiatric disorders vary in symptom severity over the reproductive cycle, pregnancy, and perimenopause 2-4 . Yet how hormones influence cognitive processes is unclear. Exogenous 17 ß -estradiol modulates dopamine signaling in the nucleus accumbens core (NAcc) 5,6 , which instantiates reward prediction errors (RPEs) for reinforcement learning 7-16 . Here we show that endogenous 17 ß -estradiol enhances RPEs and sensitivity to previous rewards by reducing dopamine reuptake proteins in the NAcc. Rats performed a task with different reward states; they adjusted how quickly they initiated trials across states, balancing effort against expected rewards. NAcc dopamine reflected RPEs that predicted and causally influenced initiation times. Elevated endogenous 17 ß -estradiol increased sensitivity to reward states by enhancing dopaminergic RPEs in the NAcc. Proteomics revealed reduced dopamine transporter expression. Finally, knockdown of midbrain estrogen receptors suppressed reinforcement learning. 17 ß -estradiol therefore controls RPEs via dopamine reuptake, mechanistically revealing how hormones influence neural dynamics for motivation and learning.

A consensus protocol for functional connectivity analysis in the rat brain.

Task-free functional connectivity in animal models provides an experimental framework to examine connectivity phenomena under controlled conditions and allows for comparisons with data modalities collected under invasive or terminal procedures. Currently, animal acquisitions are performed with varying protocols and analyses that hamper result comparison and integration. Here we introduce StandardRat, a consensus rat functional magnetic resonance imaging acquisition protocol tested across 20 centers. To develop this protocol with optimized acquisition and processing parameters, we initially aggregated 65 functional imaging datasets acquired from rats across 46 centers. We developed a reproducible pipeline for analyzing rat data acquired with diverse protocols and determined experimental and processing parameters associated with the robust detection of functional connectivity across centers. We show that the standardized protocol enhances biologically plausible functional connectivity patterns relative to previous acquisitions. The protocol and processing pipeline described here is openly shared with the neuroimaging community to promote interoperability and cooperation toward tackling the most important challenges in neuroscience.

Reduced brain volume and white matter alterations in Shank3-deficient rats.

Mutations and deletions in the SHANK3 gene cause the major neurodevelopmental features of Phelan-McDermid syndrome (PMS), which is characterized by intellectual disability, autism spectrum disorder, and sensory hyporeactivity. SHANK3 encodes a key structural component of excitatory synapses important for synaptogenesis. Clinical assessments and limited brain imaging studies of patients with PMS have uncovered regional volume reductions and white matter thinning. While these impairments have been replicated ex vivo in pups of a rat model, brain structure has not been assessed in rats in vivo or in adults. We assessed the brain structure of heterozygous and homozygous adult Shank3-deficient male rats in comparison to wild-type littermates with magnetic resonance imaging using both anatomical assessments and diffusion tensor imaging (DTI). Shank3-deficient rats showed a reduction in overall brain size and the absolute volume of the neocortex, piriform cortex, thalamus, forebrain, inferior and superior colliculi, internal capsule, and anterior commissure. The superior colliculus was decreased in relative volume. DTI revealed that axial diffusion and fractional anisotropy were reduced in the external capsule and mean diffusion was increased in the fornix, suggesting that restriction of diffusion perpendicular to the axis of the axonal fibers was impaired in these white matter tracts. Therefore, Shank3-deficient rats replicate the reduced brain volume and altered white matter phenotypes present in PMS. Our results indicate that the loss of a glutamatergic synaptic protein, Shank3, has structural consequences at the level of the whole brain. The brain regions that were altered represent potential cross-species structural biomarkers that warrant further study.

Reduced axonal caliber and structural changes in a rat model of Fragile X syndrome with a deletion of a K-Homology domain of Fmr1.

Fragile X syndrome (FXS) is a neurodevelopmental disorder that is caused by mutations in the FMR1 gene. Neuroanatomical alterations have been reported in both male and female individuals with FXS, yet the morphological underpinnings of these alterations have not been elucidated. In the current study, we found structural changes in both male and female rats that model FXS, some of which are similarly impaired in both sexes, including the superior colliculus and periaqueductal gray, and others that show sex-specific changes. The splenium of the corpus callosum, for example, was only impaired in males. We also found reduced axonal caliber in the splenium, offering a mechanism for its structural changes. Furthermore, we found that overall, male rats have higher brain-wide diffusion than female rats. Our results provide insight into which brain regions are vulnerable to a loss of Fmr1 expression and reveal an impairment at the level of the axon that could cause structural changes in white matter regions.

Deletion of the KH1 Domain of Fmr1 Leads to Transcriptional Alterations and Attentional Deficits in Rats.

Fragile X syndrome (FXS) is a neurodevelopmental disorder caused by mutations in the FMR1 gene. It is a leading monogenic cause of autism spectrum disorder and inherited intellectual disability and is often comorbid with attention deficits. Most FXS cases are due to an expansion of CGG repeats leading to suppressed expression of fragile X mental retardation protein (FMRP), an RNA-binding protein involved in mRNA metabolism. We found that the previously published Fmr1 knockout rat model of FXS expresses an Fmr1 transcript with an in-frame deletion of exon 8, which encodes for the K-homology (KH) RNA-binding domain, KH1. Notably, 3 pathogenic missense mutations associated with FXS lie in the KH domains. We observed that the deletion of exon 8 in rats leads to attention deficits and to alterations in transcriptional profiles within the medial prefrontal cortex (mPFC), which map to 2 weighted gene coexpression network modules. These modules are conserved in human frontal cortex and enriched for known FMRP targets. Hub genes in these modules represent potential therapeutic targets for FXS. Taken together, these findings indicate that attentional testing might be a reliable cross-species tool for investigating FXS and identify dysregulated conserved gene networks in a relevant brain region.

Startling Differences: Using the Acoustic Startle Response to Study Sex Differences and Neurosteroids in Affective Disorders.

PURPOSE OF REVIEW: Neuroactive steroid hormones, such as estradiol and progesterone, likely play a role in the pathophysiology of female-specific psychiatric disorders such as premenstrual dysphoric disorder (PMDD) and postpartum depression and may contribute to the marked sex differences observed in the incidence and presentation of affective disorders. However, few tools are available to study the precise contributions of these neuroactive steroids (NSs). In this review, we propose that the acoustic startle response (ASR), an objective measure of an organism's response to an emotional context or stressor, is sensitive to NSs. As such, the ASR represents a unique translational tool that may help to elucidate the contribution of NSs to sex differences in psychiatric disorders. RECENT FINDINGS: Findings suggest that anxiety-potentiated startle (APS) and prepulse inhibition of startle (PPI) are the most robust ASR paradigms for assessing contribution of NSs to affective disorders, while affective startle response modulation (ASRM) appears less diagnostic of sex or menstrual cycle (MC) effects. However, few studies have appropriately used ASR to test a priori hypotheses about sex or MC differences. We recommend that ASR studies account for sex as a biological variable (SABV) and hormonal status to further knowledge of NS contribution to affective disorders.