Task Learnability Modulates Surprise but Not Valence Processing for Reinforcement Learning in Probabilistic Choice Tasks.

The goal of temporal difference (TD) reinforcement learning is to maximize outcomes and improve future decision-making. It does so by utilizing a prediction error (PE), which quantifies the difference between the expected and the obtained outcome. In gambling tasks, however, decision-making cannot be improved because of the lack of learnability. On the basis of the idea that TD utilizes two independent bits of information from the PE (valence and surprise), we asked which of these aspects is affected when a task is not learnable. We contrasted behavioral data and ERPs in a learning variant and a gambling variant of a simple two-armed bandit task, in which outcome sequences were matched across tasks. Participants were explicitly informed that feedback could be used to improve performance in the learning task but not in the gambling task, and we predicted a corresponding modulation of the aspects of the PE. We used a model-based analysis of ERP data to extract the neural footprints of the valence and surprise information in the two tasks. Our results revealed that task learnability modulates reinforcement learning via the suppression of surprise processing but leaves the processing of valence unaffected. On the basis of our model and the data, we propose that task learnability can selectively suppress TD learning as well as alter behavioral adaptation based on a flexible cost-benefit arbitration.

Neurophysiological evidence for evaluative feedback processing depending on goal relevance.

Feedback signaling the success or failure of actions is readily exploited to implement goal-directed behavior. Two event-related brain potentials (ERPs) have been identified as reliable markers of evaluative feedback processing: the Feedback-Related Negativity (FRN) and the P3. Recent ERP studies have shown a substantial reduction of these components when the feedback's goal relevance (in terms of goal informativeness) was decreased. However, it remains unclear whether this lowering of evaluative feedback processing at the FRN and P3 levels (i) reflects a common regulation process operating across them or (ii) indirectly and mostly depends on valence processing. To address these questions, 44 participants performed a time estimation task wherein the perceived goal relevance of the feedback following each decision was manipulated via instructions in different blocks. We recorded 64-channel EEG and collected subjective ratings of feedback valence and relevance, separately for goal-relevant and irrelevant conditions. ERP results showed a substantial reduction of the FRN and P3 components for irrelevant than relevant feedback, despite the balanced task relevance between them. Moreover, a Principal Component Analysis (PCA) showed that these two successive ERP effects had dissociable spatiotemporal properties. Crucially, a multivariate multiple regression analysis revealed that goal relevance per se, but not valence, was the unique significant predictor of the amplitude reduction of the FRN and P3 when the feedback was goal irrelevant. Our results suggest that although these ERP components exhibit non-overlapping spatiotemporal properties and performance monitoring effects, they can both be modulated by a common, valence-unspecific process related to goal relevance.

Goals matter: Amplification of the motivational significance of the feedback when goal impact is increased.

Performance monitoring (PM) entails the continuous evaluation of actions and their outcomes. At the electrophysiological level, PM has been consistently related to two event-related brain potentials (ERPs): the Feedback-Related Negativity (FRN) and the P3. In a previous within-subject crossover design study, we showed that feedback's goal impact (i.e., its importance to one's goal) modulated these ERP components, yet in opposing directions. Although high goal impact was associated with a larger P3, the preceding FRN had a lower amplitude than in the low impact condition. We sought to extend these findings here by adopting a between-subjects design for a pure goal impact manipulation. Sixty-eight participants completed a Go/No Go Task while 64-channel electroencephalography was recorded concurrently. They were randomly assigned to either a high or low goal impact condition, manipulated through instructions on the supposed task's diagnosticity, while reward probability was kept similar between conditions. Replicating our previous results, we found that high goal impact yielded a marginally lower FRN, but substantially larger P3 during PM than low goal impact, without arousal or performance differences. Moreover, a principal component analysis confirmed these opposing directions of goal impact modulation. Overall, these results dovetail with the assumption that goal impact influences PM processes.

Dissociable effects of reward magnitude on fronto-medial theta and FRN during performance monitoring.

Reward processing is influenced by reward magnitude, as previous EEG studies showed changes in amplitude of the feedback-related negativity (FRN) and reward positivity (RewP), or power of fronto-medial theta (FMθ). However, it remains unclear whether these changes are driven by increased reward sensitivity, altered reward predictions, enhanced cognitive control, or a combination of these effects. To address this question, we asked 36 participants to perform a simple gambling task where feedback valence (reward vs. no-reward), its magnitude (small vs. large reward), and expectancy (expected vs. unexpected) were manipulated in a factorial design, while 64-channel EEG was recorded concurrently. We performed standard ERP analyses (FRN and RewP) as well as time-frequency decompositions (FMθ) of feedback-locked EEG data. Subjective reports showed that large rewards were more liked and expected than small ones. At the EEG level, increasing magnitude led to a larger RewP irrespective of expectancy, whereas the FRN was not influenced by this manipulation. In comparison, FMθ power was overall increased when reward magnitude was large, except if it was unexpected. These results show dissociable effects of reward magnitude on the RewP and FMθ power. Further, they suggest, that although large reward magnitude boosts reward processing (RewP), it can nonetheless undermine the need for enhanced cognitive control (FMθ) in case reward is unexpected. We discuss these new results in terms of optimistic bias or positive mood resulting from an increased reward magnitude.

When the outcome is different than expected: Subjective expectancy shapes reward prediction error at the FRN level.

Converging evidence in human electrophysiology suggests that evaluative feedback provided during performance monitoring (PM) elicits two distinctive and successive ERP components: the feedback-related negativity (FRN) and the P3b. Whereas the FRN has previously been linked to reward prediction error (RPE), the P3b has been conceived as reflecting motivational or attentional processes following the early processing of the RPE, including action value updating. However, it remains unclear whether these two consecutive neurophysiological effects depend on the direction of the unexpectedness (better- or worse-than-expected outcomes; signed RPE) or instead only on the degree of unexpectedness irrespective of direction (i.e., unsigned RPE). To address this question, we devised an experiment in which we manipulated the objective reward probability and the subjective reward expectancy (via instructions) in a factorial within-subject design and explored amplitude changes of the FRN and the P3b. A 64-channel EEG was recorded while 32 participants performed a speeded go/no-go task in which evaluative feedback based on the reward probability either violated expectancy (thereby creating a RPE) or did not. This violation corresponded either to better- or worse-than-expected events. Results showed that the FRN was larger when RPE occurred than when it did not, but irrespective of the direction of this violation. Interestingly, in these two conditions, action value was updated for the positive feedback selectively, as shown by the P3b amplitude. These results obey a two-stage model of PM assuming that unsigned RPE is first rapidly detected (FRN level) before the positive feedback's value is updated selectively (P3b effect).

Relevance and uncertainty jointly influence reward anticipation at the level of the SPN ERP component.

The stimulus-preceding negativity (SPN) component reflects the anticipatory phase of reward processing. Its amplitude is usually larger for informative compared to uninformative upcoming stimuli, as well as for uncertain relative to predictable ones. In this study, we sought to assess whether these two effects, when combined together, produced a synergistic effect or rather independent ones on the SPN during performance monitoring. Participants performed a speeded Go/NoGo task while 64-channel EEG was recorded concurrently. We focused on the SPN activity generated in anticipation of feedback, which was either positive (for correct and fast reactions) or negative (for correct but slow responses). Further, the feedback's informativeness about the satisfaction status of goals was alternated across blocks. When uncertainty about the action outcome was low (in conditions where positive feedback was either less or more frequent than negative feedback), the SPN amplitude (measured at fronto-central electrodes) did not vary as a function of feedback's relevance or valence. By comparison, when positive and negative feedback were equiprobable (uncertainty was high), the SPN was more pronounced for relevant compared to irrelevant feedback. Interestingly, in this condition, it was also larger at right fronto-central sites for positive than negative feedback. These ERP results suggest that both factors-relevance and uncertainty- combine and influence reward anticipation at the SPN level.

The causal role of the somatosensory cortex in prosocial behaviour.

Witnessing another person's suffering elicits vicarious brain activity in areas that are active when we ourselves are in pain. Whether this activity influences prosocial behavior remains the subject of debate. Here participants witnessed a confederate express pain through a reaction of the swatted hand or through a facial expression, and could decide to reduce that pain by donating money. Participants donate more money on trials in which the confederate expressed more pain. Electroencephalography shows that activity of the somatosensory cortex I (SI) hand region explains variance in donation. Transcranial magnetic stimulation (TMS) shows that altering this activity interferes with the pain-donation coupling only when pain is expressed by the hand. High-definition transcranial direct current stimulation (HD-tDCS) shows that altering SI activity also interferes with pain perception. These experiments show that vicarious somatosensory activations contribute to prosocial decision-making and suggest that they do so by helping to transform observed reactions of affected body-parts into accurate perceptions of pain that are necessary for decision-making.

Goal impact influences the evaluative component of performance monitoring: Evidence from ERPs.

Successful performance monitoring (PM) requires continuous assessment of context and action outcomes. Electrophysiological studies have reliably identified event-related potential (ERP) markers for evaluative feedback processing during PM: the Feedback-Related Negativity (FRN) and P3 components. The functional significance of FRN remains debated in the literature, with recent research suggesting that feedback's goal relevance can account for FRN (amplitude) modulation, apart from its valence or expectedness alone. Extending this account, the present study assessed whether graded differentiations in feedback's relevance or importance to one's goal (referred to as goal impact) would influence PM at the FRN (and P3) level. To this end, we ran a within-subject crossover design experiment in which 40 participants completed two standard cognitive control tasks (Go/No Go and Simon), while 64-channel electroencephalography was recorded. Critically, both tasks entailed similar reward processing but systematically varied in goal impact assignment (high vs. low), manipulated through their supposed diagnosticity for daily life functioning and activation of social comparison. ERP results showed that goal impact reliably modulated FRN in a general manner. Irrespective of feedback valence, it was overall less negative in the high compared to the low impact condition, suggesting a general decrease in feedback monitoring in the former compared to the latter condition. These findings lend support to the idea that PM is best conceived operating not solely based on motor cues, but is shaped by motivational demands.