Reward and avoidance learning in the context of aversive environments and possible implications for depressive symptoms.

BACKGROUND: Aversive stimuli in the environment influence human actions. This includes valence-dependent influences on action selection, e.g., increased avoidance but decreased approach behavior. However, it is yet unclear how aversive stimuli interact with complex learning and decision-making in the reward and avoidance domain. Moreover, the underlying computational mechanisms of these decision-making biases are unknown. METHODS: To elucidate these mechanisms, 54 healthy young male subjects performed a two-step sequential decision-making task, which allows to computationally model different aspects of learning, e.g., model-free, habitual, and model-based, goal-directed learning. We used a within-subject design, crossing task valence (reward vs. punishment learning) with emotional context (aversive vs. neutral background stimuli). We analyzed choice data, applied a computational model, and performed simulations. RESULTS: Whereas model-based learning was not affected, aversive stimuli interacted with model-free learning in a way that depended on task valence. Thus, aversive stimuli increased model-free avoidance learning but decreased model-free reward learning. The computational model confirmed this effect: the parameter lambda that indicates the influence of reward prediction errors on decision values was increased in the punishment condition but decreased in the reward condition when aversive stimuli were present. Further, by using the inferred computational parameters to simulate choice data, our effects were captured. Exploratory analyses revealed that the observed biases were associated with subclinical depressive symptoms. CONCLUSION: Our data show that aversive environmental stimuli affect complex learning and decision-making, which depends on task valence. Further, we provide a model of the underlying computations of this affective modulation. Finally, our finding of increased decision-making biases in subjects reporting subclinical depressive symptoms matches recent reports of amplified Pavlovian influences on action selection in depression and suggests a potential vulnerability factor for mood disorders. We discuss our findings in the light of the involvement of the neuromodulators serotonin and dopamine.

Strong seduction: impulsivity and the impact of contextual cues on instrumental behavior in alcohol dependence.

Alcohol-related cues acquire incentive salience through Pavlovian conditioning and then can markedly affect instrumental behavior of alcohol-dependent patients to promote relapse. However, it is unclear whether similar effects occur with alcohol-unrelated cues. We tested 116 early-abstinent alcohol-dependent patients and 91 healthy controls who completed a delay discounting task to assess choice impulsivity, and a Pavlovian-to-instrumental transfer (PIT) paradigm employing both alcohol-unrelated and alcohol-related stimuli. To modify instrumental choice behavior, we tiled the background of the computer screen either with conditioned stimuli (CS) previously generated by pairing abstract pictures with pictures indicating monetary gains or losses, or with pictures displaying alcohol or water beverages. CS paired to money gains and losses affected instrumental choices differently. This PIT effect was significantly more pronounced in patients compared to controls, and the group difference was mainly driven by highly impulsive patients. The PIT effect was particularly strong in trials in which the instrumental stimulus required inhibition of instrumental response behavior and the background CS was associated to monetary gains. Under that condition, patients performed inappropriate approach behavior, contrary to their previously formed behavioral intention. Surprisingly, the effect of alcohol and water pictures as background stimuli resembled that of aversive and appetitive CS, respectively. These findings suggest that positively valenced background CS can provoke dysfunctional instrumental approach behavior in impulsive alcohol-dependent patients. Consequently, in real life they might be easily seduced by environmental cues to engage in actions thwarting their long-term goals. Such behaviors may include, but are not limited to, approaching alcohol.