The Rescorla-Wagner model, prediction error, and fear learning.

The Rescorla-Wagner model remains one of the most important and influential theoretical accounts of the conditions under which Pavlovian learning occurs. Moreover, the experimental approaches that inspired the model continue to provide powerful behavioral tools to advance mechanistic understanding of how we and other animals learn to fear and learn to reduce fear. Here we consider key features of the Rescorla-Wagner model as applied to study of fear learning. We review evidence for key insights of the model. First, learning to fear and learning to reduce fear are governed by a common, signed prediction error. Second, this error drives variations in effectiveness of the shock US that are causal to whether and how much fear is learned or lost during a conditioning trial. We also consider behavioral and neural findings inconsistent with the model and which will be essential to understand and advance understanding of fear learning.

The activity of ventral tegmental area dopamine neurons during shock omission predicts safety learning.

We studied the role of dopamine [tyrosine hydroxylase (TH)] neurons in the rat ventral tegmental area (VTA) in safety learning. First, we used an AX +/BX-discrimination procedure to establish conditioned stimulus (CS) B as a learned safety signal that passed both summation and retardation tests of conditioned inhibition. Then, we combined this procedure with fiber photometry in TH-Cre rats to study the activity of VTA dopamine neurons during safety learning. We show that whereas footshock is associated with calcium transients in TH neurons across the VTA, shock omission during safety learning is selectively associated with calcium transients in dopamine neurons in the medial but not lateral VTA. Moreover, the magnitude of medial VTA calcium transients during shock omission accurately predicts the amount of safety that is learned and expressed during summation testing. Our findings are consistent with a common medial VTA dopamine mechanism contributing to the learned inhibition of fear in extinction and safety. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The Roles of Basolateral Amygdala Parvalbumin Neurons in Fear Learning.

The basolateral amygdala (BLA) is obligatory for fear learning. This learning is linked to BLA excitatory projection neurons whose activity is regulated by complex networks of inhibitory interneurons, dominated by parvalbumin (PV)-expressing GABAergic neurons. The roles of these GABAergic interneurons in learning to fear and learning not to fear, activity profiles of these interneurons across the course of fear learning, and whether or how these change across the course of learning all remain poorly understood. Here, we used PV cell-type-specific recording and manipulation approaches in male transgenic PV-Cre rats during pavlovian fear conditioning to address these issues. We show that activity of BLA PV neurons during the moments of aversive reinforcement controls fear learning about aversive events, but activity during moments of nonreinforcement does not control fear extinction learning. Furthermore, we show expectation-modulation of BLA PV neurons during fear learning, with greater activity to an unexpected than expected aversive unconditioned stimulus (US). This expectation-modulation was specifically because of BLA PV neuron sensitivity to aversive prediction error. Finally, we show that BLA PV neuron function in fear learning is conserved across these variations in prediction error. We suggest that aversive prediction-error modulation of PV neurons could enable BLA fear-learning circuits to retain selectivity for specific sensory features of aversive USs despite variations in the strength of US inputs, thereby permitting the rapid updating of fear associations when these sensory features change.SIGNIFICANCE STATEMENT The capacity to learn about sources of danger in the environment is essential for survival. This learning depends on complex microcircuitries of inhibitory interneurons in the basolateral amygdala. Here, we show that parvalbumin-positive GABAergic interneurons in the rat basolateral amygdala are important for fear learning during moments of danger, but not for extinction learning during moments of safety, and that the activity of these neurons is modulated by expectation of danger. This may enable fear-learning circuits to retain selectivity for specific aversive events across variations in expectation, permitting the rapid updating of learning when aversive events change.

Neural substrates of appetitive and aversive prediction error.

Prediction error, defined by the discrepancy between real and expected outcomes, lies at the core of associative learning. Behavioural investigations have provided evidence that prediction error up- and down-regulates associative relationships, and allocates attention to stimuli to enable learning. These behavioural advances have recently been followed by investigations into the neurobiological substrates of prediction error. In the present paper, we review neuroscience data obtained using causal and recording neural methods from a variety of key behavioural designs. We explore the neurobiology of both appetitive (reward) and aversive (fear) prediction error with a focus on the mesolimbic dopamine system, the amygdala, ventrolateral periaqueductal gray, hippocampus, cortex and locus coeruleus noradrenaline. New questions and avenues for research are considered.

The Mesolimbic Dopamine Activity Signatures of Relapse to Alcohol-Seeking.

The mesolimbic dopamine system comprises distinct compartments supporting different functions in learning and motivation. Less well understood is how complex addiction-related behaviors emerge from activity patterns across these compartments. Here we show how different forms of relapse to alcohol-seeking in male rats are assembled from activity across the VTA and the nucleus accumbens. First, we used chemogenetic approaches to show a causal role for VTA TH neurons in two forms of relapse to alcohol-seeking: renewal (context-induced reinstatement) and reacquisition. Then, using gCaMP fiber photometry of VTA TH neurons, we identified medial and lateral VTA TH neuron activity profiles during self-administration, renewal, and reacquisition. Next, we used optogenetic inhibition of VTA TH neurons to show distinct causal roles for VTA subregions in distinct forms of relapse. We then used dLight fiber photometry to measure dopamine binding across the ventral striatum (medial accumbens shell, accumbens core, lateral accumbens shell) and showed complex and heterogeneous profiles of dopamine binding during self-administration and relapse. Finally, we used representational similarity analysis to identify mesolimbic dopamine signatures of self-administration, extinction, and relapse. Our results show that signatures of relapse can be identified from heterogeneous activity profiles across the mesolimbic dopamine system and that these signatures are unique for different forms of relapse.SIGNIFICANCE STATEMENT It is axiomatic that the actions of dopamine are critical to drug addiction. Yet how relapse to drug-seeking is assembled from activity across the mesolimbic dopamine system is poorly understood. Here we show how relapse to alcohol-seeking relates to activity in specific VTA and accumbens compartments, how these change for different forms of relapse, and how relapse-associated activity relates to activity during self-administration and extinction. We report the mesolimbic dopamine activity signatures for relapse and show that these signatures are unique for different forms of relapse.

Pharmacogenetic excitation of dorsomedial prefrontal cortex restores fear prediction error.

Pavlovian conditioning involves encoding the predictive relationship between a conditioned stimulus (CS) and an unconditioned stimulus, so that synaptic plasticity and learning is instructed by prediction error. Here we used pharmacogenetic techniques to show a causal relation between activity of rat dorsomedial prefrontal cortex (dmPFC) neurons and fear prediction error. We expressed the excitatory hM3Dq designer receptor exclusively activated by a designer drug (DREADD) in dmPFC and isolated actions of prediction error by using an associative blocking design. Rats were trained to fear the visual CS (CSA) in stage I via pairings with footshock. Then in stage II, rats received compound presentations of visual CSA and auditory CS (CSB) with footshock. This prior fear conditioning of CSA reduced the prediction error during stage II to block fear learning to CSB. The group of rats that received AAV-hSYN-eYFP vector that was treated with clozapine-N-oxide (CNO; 3 mg/kg, i.p.) before stage II showed blocking when tested in the absence of CNO the next day. In contrast, the groups that received AAV-hSYN-hM3Dq and AAV-CaMKIIα-hM3Dq that were treated with CNO before stage II training did not show blocking; learning toward CSB was restored. This restoration of prediction error and fear learning was specific to the injection of CNO because groups that received AAV-hSYN-hM3Dq and AAV-CaMKIIα-hM3Dq that were injected with vehicle before stage II training did show blocking. These effects were not attributable to the DREADD manipulation enhancing learning or arousal, increasing fear memory strength or asymptotic levels of fear learning, or altering fear memory retrieval. Together, these results identify a causal role for dmPFC in a signature of adaptive behavior: using the past to predict future danger and learning from errors in these predictions.