Higher-Order Conditioning With Simultaneous and Backward Conditioned Stimulus: Implications for Models of Pavlovian Conditioning.

In a new environment, humans and animals can detect and learn that cues predict meaningful outcomes, and use this information to adapt their responses. This process is termed Pavlovian conditioning. Pavlovian conditioning is also observed for stimuli that predict outcome-associated cues; a second type of conditioning is termed higher-order Pavlovian conditioning. In this review, we will focus on higher-order conditioning studies with simultaneous and backward conditioned stimuli. We will examine how the results from these experiments pose a challenge to models of Pavlovian conditioning like the Temporal Difference (TD) models, in which learning is mainly driven by reward prediction errors. Contrasting with this view, the results suggest that humans and animals can form complex representations of the (temporal) structure of the task, and use this information to guide behavior, which seems consistent with model-based reinforcement learning. Future investigations involving these procedures could result in important new insights on the mechanisms that underlie Pavlovian conditioning.

Effect of non-instructed instrumental contingency of monetary reward and positive affect in a cognitive control task.

In recent years, we observed a strong interest in the influence of motivation and emotion on cognitive control. Prior studies suggest that the instrumental contingency between a response and a rewarding or affective stimulus is particularly important in that context-which is resonating with observations in the associative learning literature. However, despite this overlap, and the relevance of non-instructed learning in real life, the vast majority of studies investigating motivation-cognition interactions use direct instructions to inform participants about the contingencies between responses and stimuli. Thus, there is little experimental insight regarding how humans detect non-instructed contingencies between their actions and motivational or affective outcomes, and how these learned contingencies come to influence cognitive control processes. In an attempt to close this gap, the goal of the present study was to test the effect of non-instructed contingent and non-contingent outcomes (i.e. monetary reward and positive affective stimuli) on cognitive control using the AX-continuous performance task (AX-CPT) paradigm. We found that entirely non-instructed contingencies between responses and positive outcomes (both monetary and affective ones) led to significant performance improvement. The present results open new perspectives for studying the influence of motivation and emotion on cognitive control at the insertion with associative learning.

Selective reinforcement of conflict processing in the Stroop task.

Motivation signals have been shown to influence the engagement of cognitive control processes. However, most studies focus on the invigorating effect of reward prospect, rather than the reinforcing effect of reward feedback. The present study aimed to test whether people strategically adapt conflict processing when confronted with condition-specific congruency-reward contingencies in a manual Stroop task. Results show that the size of the Stroop effect can be affected by selectively rewarding responses following incongruent versus congruent trials. However, our findings also suggest important boundary conditions. Our first two experiments only show a modulation of the Stroop effect in the first half of the experimental blocks, possibly due to our adaptive threshold procedure demotivating adaptive behavior over time. The third experiment showed an overall modulation of the Stroop effect, but did not find evidence for a similar modulation on test items, leaving open whether this effect generalizes to the congruency conditions, or is stimulus-specific. More generally, our results are consistent with computational models of cognitive control and support contemporary learning perspectives on cognitive control. The findings also offer new guidelines and directions for future investigations on the selective reinforcement of cognitive control processes.

Expecting the good: Symbolic valence signals provoke action biases and undermine goal-directed behavior.

Positive and negative events are known to trigger opposing action tendencies (approach vs. avoidance). Recently, we found that advance monetary incentive cues can override such valence-action biases. In the present study we tested whether symbolic emotional valence cues can lead to similar adjustments and facilitate performance regardless of the required action. To this end, we performed three closely related experiments in which valence prospect (positive vs. neutral; indicated by stimulus color) and action requirements (approach vs. avoid; indicated by stimulus shape) were manipulated in a trial-to-trial fashion. Orthogonal to this, valence prospect was either embedded in the cue or target stimulus in discrete blocks (cue-valence vs. target-valence blocks). Actual valence was presented in the form of emotional face stimuli after response execution, which mirrors monetary incentive manipulations. In two of the experiments, we observed a positive-approach bias in form of performance benefit for positive versus neutral valence trials, which was exclusive for approach actions. Although numerically more pronounced in target-valence blocks, the bias was not significantly diminished in cue- versus target-valence blocks. This opposes our prediction that emotional valence cues can diminish such biases and instead highlights the robustness of inherent mappings between emotional valence and action tendencies - even if this goes against the task goal.

Excitatory second-order conditioning using a backward first-order conditioned stimulus: A challenge for prediction error reduction.

Prével and colleagues reported excitatory learning with a backward conditioned stimulus (CS) in a conditioned reinforcement preparation. Their results add to existing evidence of backward CSs sometimes being excitatory and were viewed as challenging the view that learning is driven by prediction error reduction, which assumes that only predictive (i.e., forward) relationships are learned. The results instead were consistent with the assumptions of both Miller's Temporal Coding Hypothesis and Wagner's Sometimes Opponent Processes (SOP) model. The present experiment extended the conditioned reinforcement preparation developed by Prével et al. to a backward second-order conditioning preparation, with the aim of discriminating between these two accounts. We tested whether a second-order CS can serve as an effective conditioned reinforcer, even when the first-order CS with which it was paired is a backward CS that elicits no responding. Evidence of conditioned reinforcement was found, despite no conditioned response (CR) being elicited by the first-order backward CS. The evidence of second-order conditioning in the absence of excitatory conditioning to the first-order CS is interpreted as a challenge to SOP. In contrast, the present results are consistent with the Temporal Coding Hypothesis and constitute a conceptual replication in humans of previous reports of excitatory second-order conditioning in rodents with a backward CS. The proposal is made that learning is driven by "discrepancy" with prior experience as opposed to " prediction error."