The value of confidence: Confidence prediction errors drive value-based learning in the absence of external feedback.

Reinforcement learning algorithms have a long-standing success story in explaining the dynamics of instrumental conditioning in humans and other species. While normative reinforcement learning models are critically dependent on external feedback, recent findings in the field of perceptual learning point to a crucial role of internally generated reinforcement signals based on subjective confidence, when external feedback is not available. Here, we investigated the existence of such confidence-based learning signals in a key domain of reinforcement-based learning: instrumental conditioning. We conducted a value-based decision making experiment which included phases with and without external feedback and in which participants reported their confidence in addition to choices. Behaviorally, we found signatures of self-reinforcement in phases without feedback, reflected in an increase of subjective confidence and choice consistency. To clarify the mechanistic role of confidence in value-based learning, we compared a family of confidence-based learning models with more standard models predicting either no change in value estimates or a devaluation over time when no external reward is provided. We found that confidence-based models indeed outperformed these reference models, whereby the learning signal of the winning model was based on the prediction error between current confidence and a stimulus-unspecific average of previous confidence levels. Interestingly, individuals with more volatile reward-based value updates in the presence of feedback also showed more volatile confidence-based value updates when feedback was not available. Together, our results provide evidence that confidence-based learning signals affect instrumentally learned subjective values in the absence of external feedback.

Pupil responses to pitch deviants reflect predictability of melodic sequences.

Humans automatically detect events that, in deviating from their expectations, may signal prediction failure and a need to reorient behaviour. The pupil dilation response (PDR) to violations has been associated with subcortical signals of arousal and prediction resetting. However, it is unclear how the context in which a deviant occurs affects the size of the PDR. Using ecological musical stimuli that we characterised using a computational model, we showed that the PDR to pitch deviants is sensitive to contextual uncertainty (quantified as entropy), whereby the PDR was greater in low than high entropy contexts. The PDR was also positively correlated with the unexpectedness of notes. No effects of music expertise were found, suggesting a ceiling effect due to enculturation. These results show that the same sudden environmental change can lead to differing arousal levels depending on contextual factors, providing evidence for a sensitivity of the PDR to long-term context.

How movies move us - movie preferences are linked to differences in neuronal emotion processing of fear and anger: an fMRI study.

Introduction: As a source of audio-visual stimulation, movies expose people to various emotions. Interestingly, several genres are characterized by negative emotional content. Albeit theoretical approaches exist, little is known about preferences for specific movie genres and the neuronal processing of negative emotions. Methods: We investigated associations between movie genre preference and limbic and reward-related brain reactivity to close this gap by employing an fMRI paradigm with negative emotional faces in 257 healthy participants. We compared the functional activity of the amygdala and the nucleus accumbens (NAcc) between individuals with a preference for a particular movie genre and those without such preference. Results and discussion: Amygdala activation was relatively higher in individuals with action movie preference (p TFCE-FWE = 0.013). Comedy genre preference was associated with increased amygdala (p TFCE-FWE = 0.038) and NAcc activity (p TFCE-FWE = 0.011). In contrast, crime/thriller preference (amygdala: p TFCE-FWE ≤ 0.010, NAcc: p TFCE-FWE = 0.036), as well as documentary preference, was linked to the decreased amygdala (p TFCE-FWE = 0.012) and NAcc activity (p TFCE-FWE = 0.015). The study revealed associations between participants' genre preferences and brain reactivity to negative affective stimuli. Interestingly, preferences for genres with similar emotion profiles (action, crime/thriller) were associated with oppositely directed neural activity. Potential links between brain reactivity and susceptibility to different movie-related gratifications are discussed.