Task Learnability Modulates Surprise but Not Valence Processing for Reinforcement Learning in Probabilistic Choice Tasks.

The goal of temporal difference (TD) reinforcement learning is to maximize outcomes and improve future decision-making. It does so by utilizing a prediction error (PE), which quantifies the difference between the expected and the obtained outcome. In gambling tasks, however, decision-making cannot be improved because of the lack of learnability. On the basis of the idea that TD utilizes two independent bits of information from the PE (valence and surprise), we asked which of these aspects is affected when a task is not learnable. We contrasted behavioral data and ERPs in a learning variant and a gambling variant of a simple two-armed bandit task, in which outcome sequences were matched across tasks. Participants were explicitly informed that feedback could be used to improve performance in the learning task but not in the gambling task, and we predicted a corresponding modulation of the aspects of the PE. We used a model-based analysis of ERP data to extract the neural footprints of the valence and surprise information in the two tasks. Our results revealed that task learnability modulates reinforcement learning via the suppression of surprise processing but leaves the processing of valence unaffected. On the basis of our model and the data, we propose that task learnability can selectively suppress TD learning as well as alter behavioral adaptation based on a flexible cost-benefit arbitration.

Neurophysiological evidence for evaluative feedback processing depending on goal relevance.

Feedback signaling the success or failure of actions is readily exploited to implement goal-directed behavior. Two event-related brain potentials (ERPs) have been identified as reliable markers of evaluative feedback processing: the Feedback-Related Negativity (FRN) and the P3. Recent ERP studies have shown a substantial reduction of these components when the feedback's goal relevance (in terms of goal informativeness) was decreased. However, it remains unclear whether this lowering of evaluative feedback processing at the FRN and P3 levels (i) reflects a common regulation process operating across them or (ii) indirectly and mostly depends on valence processing. To address these questions, 44 participants performed a time estimation task wherein the perceived goal relevance of the feedback following each decision was manipulated via instructions in different blocks. We recorded 64-channel EEG and collected subjective ratings of feedback valence and relevance, separately for goal-relevant and irrelevant conditions. ERP results showed a substantial reduction of the FRN and P3 components for irrelevant than relevant feedback, despite the balanced task relevance between them. Moreover, a Principal Component Analysis (PCA) showed that these two successive ERP effects had dissociable spatiotemporal properties. Crucially, a multivariate multiple regression analysis revealed that goal relevance per se, but not valence, was the unique significant predictor of the amplitude reduction of the FRN and P3 when the feedback was goal irrelevant. Our results suggest that although these ERP components exhibit non-overlapping spatiotemporal properties and performance monitoring effects, they can both be modulated by a common, valence-unspecific process related to goal relevance.

Goals matter: Amplification of the motivational significance of the feedback when goal impact is increased.

Performance monitoring (PM) entails the continuous evaluation of actions and their outcomes. At the electrophysiological level, PM has been consistently related to two event-related brain potentials (ERPs): the Feedback-Related Negativity (FRN) and the P3. In a previous within-subject crossover design study, we showed that feedback's goal impact (i.e., its importance to one's goal) modulated these ERP components, yet in opposing directions. Although high goal impact was associated with a larger P3, the preceding FRN had a lower amplitude than in the low impact condition. We sought to extend these findings here by adopting a between-subjects design for a pure goal impact manipulation. Sixty-eight participants completed a Go/No Go Task while 64-channel electroencephalography was recorded concurrently. They were randomly assigned to either a high or low goal impact condition, manipulated through instructions on the supposed task's diagnosticity, while reward probability was kept similar between conditions. Replicating our previous results, we found that high goal impact yielded a marginally lower FRN, but substantially larger P3 during PM than low goal impact, without arousal or performance differences. Moreover, a principal component analysis confirmed these opposing directions of goal impact modulation. Overall, these results dovetail with the assumption that goal impact influences PM processes.

Modulatory effects of happy mood on performance monitoring: Insights from error-related brain potentials.

Goal-adaptive behavior requires the rapid detection of conflicts between actions and intentions or goals. Although many studies have focused in the past on the influence of negative affect on this cognitive control process (and more specifically, on error monitoring), little is known about the possible modulatory effects of positive affect on it. To address this question, we used a standard (positive) mood induction procedure (based on guided imagery) and asked participants to carry out a speeded go/no-go task while high-density electroencephalography was recorded concurrently. As a control condition, we used a group with neutral mood. Event-related potential results showed that the error-related negativity (ERN) component, reflecting early error detection within the dorsal anterior cingulate cortex, was not influenced by happy mood. In contrast, the subsequent error positivity (Pe) component, related to the appraisal of the motivational significance of errors, was reliably smaller in the happy than in the neutral mood group. Complementing source localization analyses showed that this effect was explained by decreased activation within the posterior cingulate and insular cortices. These results were obtained in the absence of group differences regarding behavioral performance and tonic arousal. These findings suggest that happy mood likely decreases and changes the motivational significance of worse-than-expected events (Pe), while leaving their earlier automatic detection (ERN) unaltered. We discuss these new results in terms of dynamic changes in the complex interplay of performance monitoring with motivation.

When the outcome is different than expected: Subjective expectancy shapes reward prediction error at the FRN level.

Converging evidence in human electrophysiology suggests that evaluative feedback provided during performance monitoring (PM) elicits two distinctive and successive ERP components: the feedback-related negativity (FRN) and the P3b. Whereas the FRN has previously been linked to reward prediction error (RPE), the P3b has been conceived as reflecting motivational or attentional processes following the early processing of the RPE, including action value updating. However, it remains unclear whether these two consecutive neurophysiological effects depend on the direction of the unexpectedness (better- or worse-than-expected outcomes; signed RPE) or instead only on the degree of unexpectedness irrespective of direction (i.e., unsigned RPE). To address this question, we devised an experiment in which we manipulated the objective reward probability and the subjective reward expectancy (via instructions) in a factorial within-subject design and explored amplitude changes of the FRN and the P3b. A 64-channel EEG was recorded while 32 participants performed a speeded go/no-go task in which evaluative feedback based on the reward probability either violated expectancy (thereby creating a RPE) or did not. This violation corresponded either to better- or worse-than-expected events. Results showed that the FRN was larger when RPE occurred than when it did not, but irrespective of the direction of this violation. Interestingly, in these two conditions, action value was updated for the positive feedback selectively, as shown by the P3b amplitude. These results obey a two-stage model of PM assuming that unsigned RPE is first rapidly detected (FRN level) before the positive feedback's value is updated selectively (P3b effect).

Relevance and uncertainty jointly influence reward anticipation at the level of the SPN ERP component.

The stimulus-preceding negativity (SPN) component reflects the anticipatory phase of reward processing. Its amplitude is usually larger for informative compared to uninformative upcoming stimuli, as well as for uncertain relative to predictable ones. In this study, we sought to assess whether these two effects, when combined together, produced a synergistic effect or rather independent ones on the SPN during performance monitoring. Participants performed a speeded Go/NoGo task while 64-channel EEG was recorded concurrently. We focused on the SPN activity generated in anticipation of feedback, which was either positive (for correct and fast reactions) or negative (for correct but slow responses). Further, the feedback's informativeness about the satisfaction status of goals was alternated across blocks. When uncertainty about the action outcome was low (in conditions where positive feedback was either less or more frequent than negative feedback), the SPN amplitude (measured at fronto-central electrodes) did not vary as a function of feedback's relevance or valence. By comparison, when positive and negative feedback were equiprobable (uncertainty was high), the SPN was more pronounced for relevant compared to irrelevant feedback. Interestingly, in this condition, it was also larger at right fronto-central sites for positive than negative feedback. These ERP results suggest that both factors-relevance and uncertainty- combine and influence reward anticipation at the SPN level.

Goal impact influences the evaluative component of performance monitoring: Evidence from ERPs.

Successful performance monitoring (PM) requires continuous assessment of context and action outcomes. Electrophysiological studies have reliably identified event-related potential (ERP) markers for evaluative feedback processing during PM: the Feedback-Related Negativity (FRN) and P3 components. The functional significance of FRN remains debated in the literature, with recent research suggesting that feedback's goal relevance can account for FRN (amplitude) modulation, apart from its valence or expectedness alone. Extending this account, the present study assessed whether graded differentiations in feedback's relevance or importance to one's goal (referred to as goal impact) would influence PM at the FRN (and P3) level. To this end, we ran a within-subject crossover design experiment in which 40 participants completed two standard cognitive control tasks (Go/No Go and Simon), while 64-channel electroencephalography was recorded. Critically, both tasks entailed similar reward processing but systematically varied in goal impact assignment (high vs. low), manipulated through their supposed diagnosticity for daily life functioning and activation of social comparison. ERP results showed that goal impact reliably modulated FRN in a general manner. Irrespective of feedback valence, it was overall less negative in the high compared to the low impact condition, suggesting a general decrease in feedback monitoring in the former compared to the latter condition. These findings lend support to the idea that PM is best conceived operating not solely based on motor cues, but is shaped by motivational demands.

Goal relevance influences performance monitoring at the level of the FRN and P3 components.

The feedback-related negativity (FRN) provides a reliable ERP marker of performance monitoring (PM). It is usually larger for negative compared to positive feedback, and for unexpected relative to expected feedback. In two experiments, we assessed whether these effects could be modulated by goal relevance, defined as feedback informativeness (reliability) and/or impact on a person's goals. EEG (64-channel) was recorded while 30 participants (in each experiment) performed a speeded go/no-go task across blocks in which the feedback on task performance was deemed either relevant or not. At the ERP level, the FRN component was larger for (frequent) negative compared to (deviant) positive feedback exclusively when the feedback was relevant (Experiment 1). When the probability of positive and negative feedback was balanced (Experiment 2), this valence-driven FRN effect was absent. However, across these two experiments, the FRN was always larger for irrelevant than relevant feedback. Moreover, the subsequent P300 component was larger for feedback in the relevant than the irrelevant blocks. This effect was valence unspecific in Experiment 1, while in Experiment 2 larger P3 amplitudes were recorded for negative than positive (relevant) feedback. Across the two experiments, a larger correct-related negativity in the irrelevant than relevant context was also observed, suggesting that PM is flexible. These ERP findings indicate that goal relevance influences feedback (and response) processing during PM, with two nonoverlapping neurophysiological effects: It gates reward prediction error brain mechanisms (FRN effect), before enhancing subsequent motivational processes (P300 effect).

Trait anxiety and involuntary processing of facial emotions.

There is suggestion that trait anxiety influences the processing of threat-related information. To test this hypothesis we recorded ERPs in response to subliminally presented and backward masked fearful and neutral faces, and non-face objects, in the preselected low- and high-anxious individuals. The amplitude of N170 was found to be larger when elicited by faces in comparison to non-faces, however it was not found to be emotion-sensitive or modulated by anxiety level. Differences between low- and high-anxious individuals appeared in a time window of the P1 component. At later stages, within the EPN component, stronger negativity specific for fearful faces was recorded exclusively in the low-anxious participants. Our findings indicate that anxiety level modulates early stages of information processing, as reflected in the P1 component. This leads to anxiety-related differences in involuntary emotional expression detection at later stages (EPN component).

Attention modulates emotional expression processing.

To investigate the time course of emotional expression processing, we recorded ERPs to facial stimuli. The first task was to discriminate emotional expressions. Enhanced negativity of the face-specific N170 was elicited by emotional as opposed to neutral faces, followed by the occipital negativity (240-340 ms poststimulus). The second task was to classify face gender. Here, N170 was unaffected by the emotional expression. However, emotional expression effect was expressed in the anterior positivity (160-250 ms poststimulus) and subsequent occipital negativity (240-340 ms poststimulus). Results support the thesis that structural encoding relevant to gender recognition and simultaneous expression analysis are independent processes. Attention modulates facial emotion processing 140-185 ms poststimulus. Involuntary differentiation of facial expression was observed later (160-340 ms poststimulus), suggesting unintentional attention capture.