Dynamic computation of incentive salience: "wanting" what was never "liked".

Pavlovian cues for rewards become endowed with incentive salience, guiding "wanting" to their learned reward. Usually, cues are "wanted" only if their rewards have ever been "liked," but here we show that mesocorticolimbic systems can recompute "wanting" de novo by integrating novel physiological signals with a cue's preexisting associations to an outcome that lacked hedonic value. That is, a cue's incentive salience can be recomputed adaptively. We demonstrate that this recomputation is encoded in neural signals coursing through the ventral pallidum. Ventral pallidum neurons do not ordinarily fire vigorously to a cue that predicts the previously "disliked" taste of intense salt, although they do fire to a cue that predicts the taste of previously "liked" sucrose. Yet we show that neural firing rises dramatically to the salt cue immediately and selectively when that cue is encountered in a never-before-experienced state of physiological salt depletion. Crucially, robust neural firing to the salt cue occurred the first time it was encountered in the new depletion state (in cue-only extinction trials), even before its associated intense saltiness has ever been tasted as positively "liked" (salt taste had always been "disliked" before). The amplification of incentive salience did not require additional learning about the cue or the newly positive salt taste. Thus dynamic recomputation of cue-triggered "wanting" signals can occur in real time at the moment of cue re-encounter by combining previously learned Pavlovian associations with novel physiological information about a current state of specific appetite.

A neural computational model of incentive salience.

Incentive salience is a motivational property with 'magnet-like' qualities. When attributed to reward-predicting stimuli (cues), incentive salience triggers a pulse of 'wanting' and an individual is pulled toward the cues and reward. A key computational question is how incentive salience is generated during a cue re-encounter, which combines both learning and the state of limbic brain mechanisms. Learning processes, such as temporal-difference models, provide one way for stimuli to acquire cached predictive values of rewards. However, empirical data show that subsequent incentive values are also modulated on the fly by dynamic fluctuation in physiological states, altering cached values in ways requiring additional motivation mechanisms. Dynamic modulation of incentive salience for a Pavlovian conditioned stimulus (CS or cue) occurs during certain states, without necessarily requiring (re)learning about the cue. In some cases, dynamic modulation of cue value occurs during states that are quite novel, never having been experienced before, and even prior to experience of the associated unconditioned reward in the new state. Such cases can include novel drug-induced mesolimbic activation and addictive incentive-sensitization, as well as natural appetite states such as salt appetite. Dynamic enhancement specifically raises the incentive salience of an appropriate CS, without necessarily changing that of other CSs. Here we suggest a new computational model that modulates incentive salience by integrating changing physiological states with prior learning. We support the model with behavioral and neurobiological data from empirical tests that demonstrate dynamic elevations in cue-triggered motivation (involving natural salt appetite, and drug-induced intoxication and sensitization). Our data call for a dynamic model of incentive salience, such as presented here. Computational models can adequately capture fluctuations in cue-triggered 'wanting' only by incorporating modulation of previously learned values by natural appetite and addiction-related states.

Ventral pallidal representation of pavlovian cues and reward: population and rate codes.

We recorded neural activity in the ventral pallidum (VP) while rats learned a pavlovian reward association. Rats learned to distinguish a tone that predicted sucrose pellets (CS+) from a different tone that predicted nothing (CS-). Many VP units became responsive to CS+, but few units responded to CS-. When two CS+ were encountered sequentially, the earliest predictor of reward became most potent. Many VP units were also activated when the sucrose reward was received [unconditioned stimulus (UCS)]. These VP units for UCS remained responsive to sucrose reward after learning, even when sucrose was already predicted by CS+. Neural representation of reward learning and reward itself was characterized by population codes. The population of units that responded to CS+ increased with learning, whereas the population that responded to UCS did not change. A relative firing rate code also represented the identities of conditioned stimuli and UCS. Firing rate differences among stimuli were acquired early and remained stable during subsequent training, whereas population codes and behavioral conditioned responses continued to develop during subsequent training. Thus, the VP makes use of dynamic CS population and rate codes to encode pavlovian reward cues in reward learning and uses stable UCS population and firing codes to encode sucrose reward itself.

Ventral pallidum roles in reward and motivation.

In recent years the ventral pallidum has become a focus of great research interest as a mechanism of reward and incentive motivation. As a major output for limbic signals, the ventral pallidum was once associated primarily with motor functions rather than regarded as a reward structure in its own right. However, ample evidence now suggests that ventral pallidum function is a major mechanism of reward in the brain. We review data indicating that (1) an intact ventral pallidum is necessary for normal reward and motivation, (2) stimulated activation of ventral pallidum is sufficient to cause reward and motivation enhancements, and (3) activation patterns in ventral pallidum neurons specifically encode reward and motivation signals via phasic bursts of excitation to incentive and hedonic stimuli. We conclude that the ventral pallidum may serve as an important 'limbic final common pathway' for mesocorticolimbic processing of many rewards.

Ventral pallidum firing codes hedonic reward: when a bad taste turns good.

The ventral pallidum (VP) is a key structure in brain mesocorticolimbic reward circuits that mediate "liking" reactions to sensory pleasures. Do firing patterns in VP actually code sensory pleasure? Strong evidence for hedonic coding requires showing that neural signals track positive increases in sensory pleasure or even reversals from bad to good. A useful test is the salt alliesthesia of physiological sodium depletion that makes even aversively intense NaCl taste become palatable and "liked." We compared VP neural firing activity in rats during aversive "disliking" reactions elicited by a noxiously intense NaCl taste (triple-seawater 1.5 M concentration) in normal homeostatic state versus in a physiological salt appetite state that made the same NaCl taste palatable and elicit positive "liking" reactions. We also compared firing elicited by palatable sucrose taste, which always elicited "liking" reactions in both states. A dramatic doubling in the amplitude of VP neural firing peaks to NaCl was caused by salt appetite that matched the affective switch from aversive ("disliking") to positive hedonic ("liking") reactions. By contrast, VP neural activity to "liked" sucrose taste was always high and never altered. In summary, VP firing activity selectively tracks the hedonic values of tastes, even across hedonic reversals caused by physiological changes. Our data provide the strongest evidence yet for neural hedonic coding of natural sensory pleasures and suggest, by extension, how abnormalities in VP firing patterns might contribute to clinical hedonic dysfunctions.

Ventral pallidal neurons code incentive motivation: amplification by mesolimbic sensitization and amphetamine.

Neurons in ventral pallidum fire to reward and its predictive cues. We tested mesolimbic activation effects on neural reward coding. Rats learned that a Pavlovian conditioned stimulus (CS+1 tone) predicted a second conditioned stimulus (CS+2 feeder click) followed by an unconditioned stimulus (UCS sucrose reward). Some rats were sensitized to amphetamine after training. Electrophysiological activity of ventral pallidal neurons to stimuli was later recorded under the influence of vehicle or acute amphetamine injection. Both sensitization and acute amphetamine increased ventral pallidum firing at CS+2 (population code and rate code). There were no changes at CS+1 and minimal changes to UCS. With a new 'Profile Analysis', we show that mesolimbic activation by sensitization/amphetamine incrementally shifted neuronal firing profiles away from prediction signal coding (maximal at CS+1) and toward incentive coding (maximal at CS+2), without changing hedonic impact coding (maximal at UCS). This pattern suggests mesolimbic activation specifically amplifies a motivational transform of CS+ predictive information into incentive salience coded by ventral pallidal neurons. Our results support incentive-sensitization predictions and suggest why cues temporally proximal to drug presentation may precipitate cue-triggered relapse in human addicts.