Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making.

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics.

Serotonin neurons modulate learning rate through uncertainty.

Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.

Norepinephrine potentiates and serotonin depresses visual cortical responses by transforming eligibility traces.

Reinforcement allows organisms to learn which stimuli predict subsequent biological relevance. Hebbian mechanisms of synaptic plasticity are insufficient to account for reinforced learning because neuromodulators signaling biological relevance are delayed with respect to the neural activity associated with the stimulus. A theoretical solution is the concept of eligibility traces (eTraces), silent synaptic processes elicited by activity which upon arrival of a neuromodulator are converted into a lasting change in synaptic strength. Previously we demonstrated in visual cortical slices the Hebbian induction of eTraces and their conversion into LTP and LTD by the retroactive action of norepinephrine and serotonin Here we show in vivo in mouse V1 that the induction of eTraces and their conversion to LTP/D by norepinephrine and serotonin respectively potentiates and depresses visual responses. We also show that the integrity of this process is crucial for ocular dominance plasticity, a canonical model of experience-dependent plasticity.

Parallel Social Information Processing Circuits Are Differentially Impacted in Autism.

Parallel processing circuits are thought to dramatically expand the network capabilities of the nervous system. Magnocellular and parvocellular oxytocin neurons have been proposed to subserve two parallel streams of social information processing, which allow a single molecule to encode a diverse array of ethologically distinct behaviors. Here we provide the first comprehensive characterization of magnocellular and parvocellular oxytocin neurons in male mice, validated across anatomical, projection target, electrophysiological, and transcriptional criteria. We next use novel multiple feature selection tools in Fmr1-KO mice to provide direct evidence that normal functioning of the parvocellular but not magnocellular oxytocin pathway is required for autism-relevant social reward behavior. Finally, we demonstrate that autism risk genes are enriched in parvocellular compared with magnocellular oxytocin neurons. Taken together, these results provide the first evidence that oxytocin-pathway-specific pathogenic mechanisms account for social impairments across a broad range of autism etiologies.

Stable Representations of Decision Variables for Flexible Behavior.

Decisions occur in dynamic environments. In the framework of reinforcement learning, the probability of performing an action is influenced by decision variables. Discrepancies between predicted and obtained rewards (reward prediction errors) update these variables, but they are otherwise stable between decisions. Although reward prediction errors have been mapped to midbrain dopamine neurons, it is unclear how the brain represents decision variables themselves. We trained mice on a dynamic foraging task in which they chose between alternatives that delivered reward with changing probabilities. Neurons in the medial prefrontal cortex, including projections to the dorsomedial striatum, maintained persistent firing rate changes over long timescales. These changes stably represented relative action values (to bias choices) and total action values (to bias response times) with slow decay. In contrast, decision variables were weakly represented in the anterolateral motor cortex, a region necessary for generating choices. Thus, we define a stable neural mechanism to drive flexible behavior.

Nucleus Accumbens and Posterior Amygdala Mediate Cue-Triggered Alcohol Seeking and Suppress Behavior During the Omission of Alcohol-Predictive Cues.

Neurobiological mechanisms that influence behavior in the presence of alcohol-associated stimuli involve processes that organize behavior during the presence of these cues, and separately, regulation of behavior in their absence. However, little is known about anatomical structures that might mediate this regulation. Here we examined nucleus accumbens shell (AcbSh) as a possible neural substrate mediating behavior modulation triggered by the presence and absence of alcohol-associated environmental cues and contexts. We also examined subregions of basal amygdala nuclei- rostral basolateral (BLA) and basal posterior (BAP)- as they provide a major source of glutamatergic input to the AcbSh. Animals were trained to associate an auditory conditioning stimulus with alcohol in a discriminative context and then subsequently tested for conditioned port-entries across contexts either previously associated or not associated with alcohol. We found that, on test to the alcohol cue alone, AcbSh inactivation prevented conditioned port-entries in contexts that either were associated with alcohol or were novel, while also increasing unconditioned port-entries during the intertrial intervals. When tested to alcohol-reinforced cues, AcbSh inactivation produced more cue-trial omissions and elevated unconditioned port-entries. Interestingly, BLA and BAP inactivation produced dissociable effects. BAP but not BLA increased unconditioned port-entries, while both manipulations prevented conditioned port-entries during the alcohol-cue. We conclude that AcbSh is necessary for modulating control over behavior otherwise guided by the presence of alcohol-predictive environmental stimuli and contexts. Moreover, this role may involve integration of functionally segregated inputs from rostral and posterior portions of basal amygdala nuclei.