Pavlovian Fear Conditioning Is More than You Think It Is.

A common neuroscience application of Pavlovian fear conditioning is to manipulate neuron-type activity, pair a cue with foot shock, then measure cue-elicited freezing in a novel context. If the manipulation reduces freezing, the neuron type is implicated in Pavlovian fear conditioning. This application reduces Pavlovian fear conditioning to a single concept. In this Viewpoint, I describe experiments supporting the view that Pavlovian fear conditioning refers to three distinct concepts: procedure, process, and behavior. An experimenter controls procedure, observes behavior, but infers process. Distinguishing these concepts is essential because: (1) a shock-paired cue can engage numerous processes and behaviors; (2) experimenter decisions about procedure influence the processes engaged and behaviors elicited; and (3) many processes are latent, imbuing the cue with properties that only manifest outside of the original conditioning setting. This means we could understand the complete neural basis of freezing, yet know little about the neural basis of fear. Neuroscientists can choose to use a variety of procedures to study a diversity of processes and behaviors. Manipulating neuron-type activity in multiple procedures can reveal specific, general, or complex neuron-type contributions to cue-elicited processes and behaviors. The results will be a broader and more detailed neural basis of fear with greater relevance to the spectrum of symptoms defining anxiety and stressor-related disorders.

Threat and Bidirectional Valence Signaling in the Nucleus Accumbens Core.

Appropriate responding to threat and reward is essential to survival. The nucleus accumbens core (NAcc) is known to support and organize reward behavior. The NAcc is also necessary to fully discriminate threat and safety cues. To directly reveal NAcc threat firing, we recorded single-unit activity from seven female rats undergoing pavlovian fear discrimination. Rats fully discriminated danger, uncertainty, and safety cues, and most NAcc neurons showed the greatest firing change to danger and uncertainty. Heterogeneity in cue and reward firing led us to identify distinct functional populations. One NAcc population signaled threat, specifically decreasing firing to danger and uncertainty cues. A separate population signaled Bidirectional Valence, decreasing firing to the danger cue (negative valence), but increasing firing to reward (positive valence). The results reveal the NAcc to be a source of threat information and a more general valence hub.SIGNIFICANCE STATEMENT The nucleus accumbens core (NAcc) is synonymous with reward. Yet, anatomy, neurotoxic lesions, and optogenetic manipulation implicate the NAcc in threat. Here, we directly revealed NAcc threat firing by recording single-unit activity during multicue fear discrimination. Most cue-responsive NAcc neurons markedly altered firing to threat cues. Finer analyses revealed a NAcc population signaling threat, specifically decreasing firing to danger and uncertainty cues; and a NAcc population signaling Bidirectional Valence, increasing firing to reward but decreasing firing to the danger cue. The results reveal the NAcc to be a source of threat information and a valence hub.

The Nucleus Accumbens Core is Necessary to Scale Fear to Degree of Threat.

Fear is adaptive when the level of the response rapidly scales to degree of threat. Using a discrimination procedure consisting of danger, uncertainty, and safety cues, we have found rapid fear scaling (within 2 s of cue presentation) in male rats. Here, we examined a possible role for the nucleus accumbens core (NAcc) in the acquisition and expression of fear scaling. In experiment 1, male Long-Evans rats received bilateral sham or neurotoxic NAcc lesions, recovered, and underwent fear discrimination. NAcc-lesioned rats were generally impaired in scaling fear to degree of threat, and specifically impaired in rapid uncertainty-safety discrimination. In experiment 2, male Long-Evans rats received NAcc transduction with halorhodopsin (Halo) or a control fluorophore. After fear scaling was established, the NAcc was illuminated during cue or control periods. NAcc-Halo rats receiving cue illumination were specifically impaired in rapid uncertainty-safety discrimination. The results reveal a general role for the NAcc in scaling fear to degree of threat, and a specific role in rapid discrimination of uncertain threat and safety.SIGNIFICANCE STATEMENT Rapidly discriminating cues for threat and safety is essential for survival and impaired threat-safety discrimination is a hallmark of stress and anxiety disorders. In two experiments, we induced nucleus accumbens core (NAcc) dysfunction in rats receiving fear discrimination consisting of cues for danger, uncertainty, and safety. Permanent NAcc dysfunction, via neurotoxic lesion, generally disrupted the ability to scale fear to degree of threat, and specifically impaired one component of scaling: rapid discrimination of uncertain threat and safety. Reversible NAcc dysfunction, via optogenetic inhibition, specifically impaired rapid discrimination of uncertain threat and safety. The results reveal that the NAcc is essential to scale fear to degree of threat, and is a plausible source of dysfunction in stress and anxiety disorders.

The ventrolateral periaqueductal grey updates fear via positive prediction error.

Aversive, positive prediction error (+PE) provides a mechanism to update and increase future fear to uncertain threat predictors. The ventrolateral periaqueductal grey (vlPAG) has been offered as a neural locus for +PE computation. Yet, a causal demonstration of vlPAG +PE activity to update fear remains elusive. We devised a fear discrimination procedure in which a danger cue predicts shock deterministically and an uncertainty cue predicts shock probabilistically, requiring prediction errors to achieve an appropriate fear response. Recording vlPAG single-unit activity during fear discrimination in Long-Evans rats, we reveal activity related to shock is consistent with +PE and updates subsequent fear to uncertainty at the trial level. We further demonstrate that vlPAG inhibition during shock selectively decreases future fear to uncertainty, but not danger, and temporal emergence of this effect is consistent with single-unit activity. These findings provide causal evidence that vlPAG +PE is necessary for fear updating.

Medial Orbitofrontal Neurons Preferentially Signal Cues Predicting Changes in Reward during Unblocking.

UNLABELLED: The orbitofrontal cortex (OFC) has been broadly implicated in the ability to use the current value of expected outcomes to guide behavior. Although value correlates have been prominently reported in lateral OFC, they are more often associated with more medial areas. Further, recent studies in primates have suggested a dissociation in which the lateral OFC is involved in credit assignment and representation of reward identity and more medial areas are critical to representing value. Previously, we used unblocking to test more specifically what information about outcomes is represented by OFC neurons in rats; consistent with the proposed dichotomy between the lateral and medial OFC, we found relatively little linear value coding in the lateral OFC (Lopatina et al., 2015). Here we have repeated this experiment, recording in the medial OFC, to test whether such value signals might be found there. Neurons were recorded in an unblocking task as rats learned about cues that signaled either more, less, or the same amount of reward. We found that medial OFC neurons acquired responses to these cues; however, these responses did not signal different reward values across cues. Surprisingly, we found that cells developed responses to cues predicting a change, particularly a decrease, in reward value. This is consistent with a special role for medial OFC in representing current value to support devaluation/revaluation sensitive changes in behavior. SIGNIFICANCE STATEMENT: This study uniquely examines encoding in rodent mOFC at the single-unit level in response to cues that predict more, less, or no change in reward in rats during training in a Pavlovian unblocking task, finding more cells responding to change-predictive cues and stronger activity in response to cues predictive of less reward.

Subsecond fear discrimination in rats: adult impairment in adolescent heavy alcohol drinkers.

Discriminating safety from danger must be accurate and rapid. Yet, the rapidity with which fear discrimination emerges remains unknown. Rapid fear discrimination in adulthood may be susceptible to impairment by adolescent heavy alcohol drinking, which increases incidence of anxiety disorders. Rats were given voluntary, adolescent alcohol access, and heavy drinkers were identified. In adulthood, rapid fear discrimination of safety, uncertainty, and danger cues was assessed. Normal rats, but not heavy drinkers, showed discriminative fear <1 sec following cue onset. This provides the first demonstration of subsecond fear discrimination and its adult impairment in adolescent heavy alcohol drinkers.

The dorsal raphe nucleus is integral to negative prediction errors in Pavlovian fear.

Prediction errors are central to modern learning theories. While brain regions contributing to reward prediction errors have been uncovered, the sources of aversive prediction errors remain largely unknown. Here we used probabilistic and deterministic reinforcement procedures, followed by extinction, to examine the contribution of the dorsal raphe nucleus to negative, aversive prediction errors in Pavlovian fear. Rats with dorsal raphe lesions were able to acquire fear and reduce fear to a non-reinforced deterministic cue. However, dorsal raphe lesions impaired the reduction of fear to a probabilistic cue and fear extinction to a deterministic cue, both of which involve the use of negative prediction errors. The results point to an integral role for the dorsal raphe nucleus in negative prediction error signaling in Pavlovian fear.

Learning theory: a driving force in understanding orbitofrontal function.

Since it was demonstrated the orbitofrontal cortex (OFC) is critical to reversal learning, there has been considerable interest in specifying its role in flexible, outcome-guided behavior. Behavioral paradigms from the learning theory tradition, such as outcome devaluation, blocking, Pavlovian to instrumental transfer, and overexpectation have been a driving force in this research. The use of these procedures has revealed OFC's unique role in forming and integrating information about specific features of events and outcomes to drive behavior and learning. These studies highlight the power and importance of learning theory principles in guiding neuroscience research.

A fear conditioned cue orchestrates a suite of behaviors in rats.

Pavlovian fear conditioning has been extensively used to study the behavioral and neural basis of defensive systems. In a typical procedure, a cue is paired with foot shock, and subsequent cue presentation elicits freezing, a behavior theoretically linked to predator detection. Studies have since shown a fear conditioned cue can elicit locomotion, a behavior that - in addition to jumping, and rearing - is theoretically linked to imminent or occurring predation. A criticism of studies observing fear conditioned cue-elicited locomotion is that responding is non-associative. We gave rats Pavlovian fear discrimination over a baseline of reward seeking. TTL-triggered cameras captured 5 behavior frames/s around cue presentation. Experiment 1 examined the emergence of danger-specific behaviors over fear acquisition. Experiment 2 examined the expression of danger-specific behaviors in fear extinction. In total, we scored 112,000 frames for nine discrete behavior categories. Temporal ethograms show that during acquisition, a fear conditioned cue suppresses reward seeking and elicits freezing, but also elicits locomotion, jumping, and rearing - all of which are maximal when foot shock is imminent. During extinction, a fear conditioned cue most prominently suppresses reward seeking, and elicits locomotion that is timed to shock delivery. The independent expression of these behaviors in both experiments reveals a fear conditioned cue to orchestrate a temporally organized suite of behaviors.

Knowing that an animal is fearful is crucial for many psychology and neuroscience studies. For instance, this knowledge allows researchers to examine the brain pathways involved in processing and responding to fear. Typically, researchers consider that a rodent is experiencing fear if it 'freezes' ­ a response which, in the wild, helps to evade detection by predators. In Pavlovian fear conditioning experiments, for example, rats and mice freeze when exposed to a stimulus (often a specific sound) previously associated with unpleasant sensations. However, rodents can also respond more actively to threats, for instance by running or jumping away. It remains unclear whether the 'fearful stimuli' used in Pavlovian approaches specifically elicits only freezing, or other fear-related behaviors as well. To investigate this, Chu et al. used high-speed cameras to record rats' responses to a sound cue they had 'learned' to associate with a mild foot shock. In addition to freezing, the animals ran, jumped, stood on their hind legs and stopped their usual reward-seeking behavior in response to the cue. Crucially, these reactions were absent when the rats were exposed to sound cues not associated with pain. Overall, these experiments demonstrate that Pavlovian conditioning can elicit a full range of fear-related behaviors beyond freezing. Understanding the neural activity behind these diverse responses could lead to more targeted therapies and interventions addressing the various ways stress and anxiety manifest in people.

Role of amygdala central nucleus in aversive learning produced by shock or by unexpected omission of food.

Many psychological learning theories have noted commonalities between aversive states produced by presentation of negative reinforcers, such as electric shock, and the omission of expected positive reinforcers, such as food. Here, three groups of rats received training with one auditory cue paired with shock and another with the omission of expected food, a shock-paired cue and a food-omission control cue, or a food-omission cue and a shock control cue. Food-omission cues were established by contrast with food delivery; after extensive light-food pairings, the light was followed by the food-omission cue instead of food. Aversiveness of the food-omission cue was assessed with a conditioned punishment procedure, in which presentation of that cue was made contingent on performance of one previously trained instrumental response, whereas a second response had no consequences. We found that rats with lesions of amygdala central nucleus (CeA) showed impaired acquisition of freezing to the cue paired with shock and no evidence for acquisition of aversive properties by the cue that accompanied the omission of expected food. Furthermore, analyses of Arc and Homer1a mRNAs after rats were exposed to a two-epoch test procedure that allowed assessment of gene expression produced by two different test stimuli showed that both food-omission and shock-paired cues generated more neuronal activity in CeA than appropriate control cues. However, the number of neurons that were activated by both shock and food-omission cues was not significantly greater than expected by chance. Thus, under these test conditions, different subsets of CeA neurons represented these two aversive states.

Effects of ventral striatal lesions on first- and second-order appetitive conditioning.

Rats with bilateral lesions of the ventral striatal nucleus accumbens failed to acquire Pavlovian second-order conditioning to auditory stimuli paired with visual stimuli that had previously received first-order pairings with food. This deficit in second-order conditioning was specific to learning driven by incentive properties of the first-order cues, and was observed whether the first-order training had occurred prior to or after lesion surgery. Lesions also produced deficits in the display of conditioned responses to the first-order conditioned stimulus, but only when they were made after first-order training. These results suggest a specific role for the ventral striatum in acquiring and expressing incentive properties of conditioned stimuli through second-order conditioning, as well as a more general role in expressing previously acquired Pavlovian conditioned responses.

Timing of behavioral responding to long-duration Pavlovian fear conditioned cues.

Behavioral responding is most beneficial when it reflects event timing. Compared to reward, there are fewer studies on timing of defensive responding. We gave female and male rats Pavlovian fear conditioning over a baseline of reward seeking. Two 100-s cues predicted foot shock at different time points. Rats acquired timing of behavioral responding to both cues. Suppression of reward seeking was minimal at cue onset and maximal before shock delivery. Rats also came to minimize suppres-sion of reward seeking following cue offset. The results reveal timing as a mechanism to focus defen-sive responding to shock-imminent, cue periods.

Optogenetic inhibition of the caudal substantia nigra inflates behavioral responding to uncertain threat and safety.

Defensive responding is adaptive when it approximates current threat, but maladaptive when it exceeds current threat. Here we asked if the substantia nigra, a region consistently implicated in reward, is necessary to show appropriate levels of defensive responding in Pavlovian fear discrimination. Rats received bilateral transduction of the caudal substantia nigra with halorhodopsin or a control fluorophore, and bilateral ferrule implants. Rats then behaviorally discriminated cues predicting unique foot shock probabilities (danger, p =1; uncertainty, p =0.25; and safety, p =0). Green-light illumination (532 nm) during cue presentation inflated defensive responding of halorhodopsin rats - measured by suppression of reward seeking - to uncertainty and safety beyond control levels. Green-light illumination outside of cue presentation had no impact on halorhodopsin or control rat responding. The results reveal caudal substantia nigra cue activity is necessary to inhibit defensive responding to non-threatening and uncertain threat cues.

Optogenetic inhibition of the caudal substantia nigra inflates behavioral responding to uncertain threat and safety.

Defensive responding is adaptive when it approximates the current threat but maladaptive when it exceeds the current threat. Here we asked if the substantia nigra, a region consistently implicated in reward, is necessary to show appropriate levels of defensive responding in Pavlovian fear discrimination. Rats received bilateral transduction of the caudal substantia nigra with halorhodopsin or a control fluorophore and bilateral ferrule implants. Rats then behaviorally discriminated cues predicting unique foot shock probabilities (danger, p = 1; uncertainty, p = .25; and safety, p = 0). Green-light illumination (532 nm) during cue presentation inflated defensive responding of halorhodopsin rats-measured by suppression of reward seeking-to uncertainty and safety beyond control levels. Green-light illumination outside of cue presentation had no impact on halorhodopsin or control rat responding. The results reveal caudal substantia nigra cue activity is necessary to inhibit defensive responding to nonthreatening and uncertain threat cues. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning.

In many cases, learning is thought to be driven by differences between the value of rewards we expect and rewards we actually receive. Yet learning can also occur when the identity of the reward we receive is not as expected, even if its value remains unchanged. Learning from changes in reward identity implies access to an internal model of the environment, from which information about the identity of the expected reward can be derived. As a result, such learning is not easily accounted for by model-free reinforcement learning theories such as temporal difference reinforcement learning (TDRL), which predicate learning on changes in reward value, but not identity. Here, we used unblocking procedures to assess learning driven by value- versus identity-based prediction errors. Rats were trained to associate distinct visual cues with different food quantities and identities. These cues were subsequently presented in compound with novel auditory cues and the reward quantity or identity was selectively changed. Unblocking was assessed by presenting the auditory cues alone in a probe test. Consistent with neural implementations of TDRL models, we found that the ventral striatum was necessary for learning in response to changes in reward value. However, this area, along with orbitofrontal cortex, was also required for learning driven by changes in reward identity. This observation requires that existing models of TDRL in the ventral striatum be modified to include information about the specific features of expected outcomes derived from model-based representations, and that the role of orbitofrontal cortex in these models be clearly delineated.

Model-based learning and the contribution of the orbitofrontal cortex to the model-free world.

Learning is proposed to occur when there is a discrepancy between reward prediction and reward receipt. At least two separate systems are thought to exist: one in which predictions are proposed to be based on model-free or cached values; and another in which predictions are model-based. A basic neural circuit for model-free reinforcement learning has already been described. In the model-free circuit the ventral striatum (VS) is thought to supply a common-currency reward prediction to midbrain dopamine neurons that compute prediction errors and drive learning. In a model-based system, predictions can include more information about an expected reward, such as its sensory attributes or current, unique value. This detailed prediction allows for both behavioral flexibility and learning driven by changes in sensory features of rewards alone. Recent evidence from animal learning and human imaging suggests that, in addition to model-free information, the VS also signals model-based information. Further, there is evidence that the orbitofrontal cortex (OFC) signals model-based information. Here we review these data and suggest that the OFC provides model-based information to this traditional model-free circuitry and offer possibilities as to how this interaction might occur.

The role of the nucleus accumbens in knowing when to respond.

While knowing what to expect is important, it is equally important to know when to expect it and to respond accordingly. This is apparent even in simple Pavlovian training situations in which animals learn to respond more strongly closer to reward delivery. Here we report that the nucleus accumbens core, an area well-positioned to represent information about the timing of impending rewards, plays a critical role in this timing function.

Brainstem networks construct threat probability and prediction error from neuronal building blocks.

When faced with potential threat we must estimate its probability, respond advantageously, and leverage experience to update future estimates. Threat estimation is the proposed domain of the forebrain, while behaviour is elicited by the brainstem. Yet, the brainstem is also a source of prediction error, a learning signal to acquire and update threat estimates. Neuropixels probes allowed us to record single-unit activity across a 21-region brainstem axis in rats receiving probabilistic fear discrimination with foot shock outcome. Against a backdrop of diffuse behaviour signaling, a brainstem network with a dorsal hub signaled threat probability. Neuronal function remapping during the outcome period gave rise to brainstem networks signaling prediction error and shock on multiple timescales. The results reveal brainstem networks construct threat probability, behaviour, and prediction error signals from neuronal building blocks.

Dorsal Raphe 5-HT Neurons Utilize, But Do Not Generate, Negative Aversive Prediction Errors.

The dorsal raphe nucleus (DRN) contains the largest population of serotonin (5-HT) neurons in the central nervous system. 5-HT, synthesized via tryptophan hydroxylase 2 (Tph2), is a widely functioning neuromodulator implicated in fear learning. Here, we sought to investigate whether DRN 5-HT is necessary to reduce fear via negative prediction error (-PE). Using male and female TPH2-cre rats, DRNtph2+ cells were selectively deleted via cre-caspase (rAAV5-Flex-taCasp3-TEVp) in experiment 1. Rats then underwent fear discrimination during which three cues were associated with unique foot shock probabilities: safety p = 0.00, uncertainty p = 0.375, and danger p = 1.00. Rats then received selective extinction to the uncertainty cue, a behavioral manipulation designed to probe -PE. Deleting DRNtph2+ cells had no impact on initial discrimination but slowed selective extinction. In experiment 2, we used a within-subjects optogenetic inhibition design to causally implicate DRNtph2+ cells in prediction error signaling. Male and female TPH2-cre rats received intra-DRN infusions of cre-dependent halorhodopsin (rAAV5-Ef1a-DIO-eNpHR3.0-eYFP) or cre-YFP. DRNtph2+ cells were inhibited specifically during the time of prediction error or a control period. Illumination during either positive prediction error (+PE) or control periods had no impact on fear to the uncertainty cue. Inhibition of DRNtph2+ cells at the time of -PE did not impact immediate fear, but facilitated selective extinction in postillumination sessions. Together, these results demonstrate a role for DRNtph2+ cells in using, but not generating, -PE to weaken cue-shock associations.

Decision making: Serotonin goes for goal.

Decision making is adaptive when our actions align with our goals. A new study shows that activity of dorsal raphe serotonin neurons is essential to adaptive decision making, permitting actions to reflect the current goal value.