Language statistical learning responds to reinforcement learning principles rooted in the striatum.

Statistical learning (SL) is the ability to extract regularities from the environment. In the domain of language, this ability is fundamental in the learning of words and structural rules. In lack of reliable online measures, statistical word and rule learning have been primarily investigated using offline (post-familiarization) tests, which gives limited insights into the dynamics of SL and its neural basis. Here, we capitalize on a novel task that tracks the online SL of simple syntactic structures combined with computational modeling to show that online SL responds to reinforcement learning principles rooted in striatal function. Specifically, we demonstrate-on 2 different cohorts-that a temporal difference model, which relies on prediction errors, accounts for participants' online learning behavior. We then show that the trial-by-trial development of predictions through learning strongly correlates with activity in both ventral and dorsal striatum. Our results thus provide a detailed mechanistic account of language-related SL and an explanation for the oft-cited implication of the striatum in SL tasks. This work, therefore, bridges the long-standing gap between language learning and reinforcement learning phenomena.

Anticipatory and consummatory neural correlates of monetary and music rewarding stimuli.

Most of the literature on the neural bases of human reward and punishment processing has used monetary gains and losses, but less is known about the neurophysiological mechanisms underlying the anticipation and consumption of other types of rewarding stimuli. In the present study, EEG was recorded from 19 participants who completed a modified version of the Monetary Incentive Delay (MID) task. During the task, cues providing information about potential future outcomes were presented to the participants. Then, they had to respond rapidly to a target stimulus to win money or listening to pleasant music, or to avoid losing money or listening to unpleasant music. Results revealed similar responses for monetary and music cues, with increased activity for cues indicating potential gains compared to losses. However, differences emerged in the outcome phase between money and music. Monetary outcomes showed an interaction between the type of the cue and the outcome in the Feedback Related Negativity and Fb-P3 ERPs and increased theta activity increased for negative feedbacks. In contrast, music outcomes showed significant interactions in the Fb-P3 and theta activities. These findings suggest similar neurophysiological mechanisms in processing cues for potential positive or negative outcomes in these two types of stimuli.

Neural predictors of cognitive-behavior therapy outcome in anxiety-related disorders: a meta-analysis of task-based fMRI studies.

BACKGROUND: Cognitive-behavior therapy (CBT) is a well-established first-line intervention for anxiety-related disorders, including specific phobia, social anxiety disorder, panic disorder/agoraphobia, generalized anxiety disorder, obsessive-compulsive disorder, and posttraumatic stress disorder. Several neural predictors of CBT outcome for anxiety-related disorders have been proposed, but previous results are inconsistent. METHODS: We conducted a systematic review and meta-analysis of task-based functional magnetic resonance imaging (fMRI) studies investigating whole-brain predictors of CBT outcome in anxiety-related disorders (17 studies, n = 442). RESULTS: Across different tasks, we observed that brain response in a network of regions involved in salience and interoception processing, encompassing fronto-insular (the right inferior frontal gyrus-anterior insular cortex) and fronto-limbic (the dorsomedial prefrontal cortex-dorsal anterior cingulate cortex) cortices was strongly associated with a positive CBT outcome. CONCLUSIONS: Our results suggest that there are robust neural predictors of CBT outcome in anxiety-related disorders that may eventually lead (probably in combination with other data) to develop personalized approaches for the treatment of these mental disorders.

Dopamine modulates the reward experiences elicited by music.

Understanding how the brain translates a structured sequence of sounds, such as music, into a pleasant and rewarding experience is a fascinating question which may be crucial to better understand the processing of abstract rewards in humans. Previous neuroimaging findings point to a challenging role of the dopaminergic system in music-evoked pleasure. However, there is a lack of direct evidence showing that dopamine function is causally related to the pleasure we experience from music. We addressed this problem through a double blind within-subject pharmacological design in which we directly manipulated dopaminergic synaptic availability while healthy participants (n = 27) were engaged in music listening. We orally administrated to each participant a dopamine precursor (levodopa), a dopamine antagonist (risperidone), and a placebo (lactose) in three different sessions. We demonstrate that levodopa and risperidone led to opposite effects in measures of musical pleasure and motivation: while the dopamine precursor levodopa, compared with placebo, increased the hedonic experience and music-related motivational responses, risperidone led to a reduction of both. This study shows a causal role of dopamine in musical pleasure and indicates that dopaminergic transmission might play different or additive roles than the ones postulated in affective processing so far, particularly in abstract cognitive activities.

Electrophysiological underpinnings of reward processing: Are we exploiting the full potential of EEG?

Understanding how the brain processes reward is an important and complex endeavor, which has involved the use of a range of complementary neuroimaging tools, including electroencephalography (EEG). EEG has been praised for its high temporal resolution but, because the signal recorded at the scalp is a mixture of brain activities, it is often considered to have poor spatial resolution. Besides, EEG data analysis has most often relied on event-related potentials (ERPs) which cancel out non-phase locked oscillatory activity, thus limiting the functional discriminative power of EEG attainable through spectral analyses. Because these three dimensions -temporal, spatial and spectral- have been unequally leveraged in reward studies, we argue that the full potential of EEG has not been exploited. To back up our claim, we first performed a systematic survey of EEG studies assessing reward processing. Specifically, we report on the nature of the cognitive processes investigated (i.e., reward anticipation or reward outcome processing) and the methods used to collect and process the EEG data (i.e., event-related potential, time-frequency or source analyses). A total of 359 studies involving healthy subjects and the delivery of monetary rewards were surveyed. We show that reward anticipation has been overlooked (88% of studies investigated reward outcome processing, while only 24% investigated reward anticipation), and that time-frequency and source analyses (respectively reported by 19% and 12% of the studies) have not been widely adopted by the field yet, with ERPs still being the dominant methodology (92% of the studies). We argue that this focus on feedback-related ERPs provides a biased perspective on reward processing, by ignoring reward anticipation processes as well as a large part of the information contained in the EEG signal. Finally, we illustrate with selected examples how addressing these issues could benefit the field, relying on approaches combining time-frequency analyses, blind source separation and source localization.

White Matter Microstructure Reflects Individual Differences in Music Reward Sensitivity.

People show considerable variability in the degree of pleasure they experience from music. These individual differences in music reward sensitivity are driven by variability in functional connectivity between the nucleus accumbens (NAcc), a key structure of the reward system, and the right superior temporal gyrus (STG). However, it is unknown whether a neuroanatomical basis exists for this variability. We used diffusion tensor imaging and probabilistic tractography to study the relationship between music reward sensitivity and white matter microstructure connecting these two regions via the orbitofrontal cortex (OFC) in 38 healthy human participants (24 females and 14 males). We found that right axial diffusivity (AD) in the STG-OFC connectivity inversely correlated with music reward sensitivity. Additionally, right mean diffusivity and left AD in the NAcc-OFC tract also showed an inverse correlation. Further, AD in this tract also correlated with previously acquired BOLD activity during music listening, but not for a control monetary reward task in the NAcc. Finally, we used mediation analysis to show that AD in the NAcc-OFC tract explains the influence of NAcc activation during a music task on music reward sensitivity. Overall, our results provide further support for the idea that the exchange of information among perceptual, integrative, and reward systems is important for musical pleasure, and that individual differences in the structure of the relevant anatomical connectivity influences the degree to which people are able to derive such pleasure.SIGNIFICANCE STATEMENT Music is one of the most important sources of pleasure for many people, but at the same time there are important individual differences in the sensitivity to musical reward. Previous studies have revealed the critical involvement of the functional connectivity between perceptual and subcortical brain areas in the enjoyment of music. However, it is unknown whether individual differences in music sensitivity might arise from variability in the structural connectivity among these areas. Here we show that structural connectivity between supratemporal and orbitofrontal cortices, and between orbitofrontal and nucleus accumbens, predict individual differences in sensibility to music reward. These results provide evidence for the critical involvement of the interaction between the subcortical reward system and higher-order cortical areas in music-induced pleasure.

Fronto-temporal theta phase-synchronization underlies music-evoked pleasantness.

Listening to pleasant music engages a complex distributed network including pivotal areas for auditory, reward, emotional and memory processing. On the other hand, frontal theta rhythms appear to be relevant in the process of giving value to music. However, it is not clear to which extent this oscillatory mechanism underlies the brain interactions that characterize music-evoked pleasantness and its related processes. The goal of the present experiment was to study brain synchronization in this oscillatory band as a function of music-evoked pleasantness. EEG was recorded from 25 healthy subjects while they were listening to music and rating the experienced degree of induced pleasantness. By using a multilevel Bayesian approach we found that phase synchronization in the theta band between right temporal and frontal signals increased with the degree of pleasure experienced by participants. These results show that slow fronto-temporal loops play a key role in music-evoked pleasantness.

Gossip information increases reward-related oscillatory activity.

Previous research has described the process by which the interaction between the firing in midbrain dopamine neurons and the hippocampus results in promoting memory for high-value motivational and rewarding events, both extrinsically and intrinsically driven (i.e. curiosity). Studies on social cognition and gossip have also revealed the activation of similar areas from the reward network. In this study we wanted to assess the electrophysiological correlates of the anticipation and processing of novel information (as an intrinsic cognitive reward) depending on the degree of elicited curiosity and the content of the information. 24 healthy volunteers participated in this EEG experiment. The task consisted of 150 questions and answers divided into three different conditions: trivia-like questions, personal-gossip information about celebrities and personal-neutral information about the same celebrities. Our main results from the ERPs and time-frequency analysis pinpointed main differences for gossip in comparison with personal-neutral and trivia-like conditions. Specifically, we found an increase in beta oscillatory activity in the outcome phase and a decrease of the same frequency band in the expectation phase. Larger amplitudes in P300 component were also found for gossip condition. Finally, gossip answers were the most remembered in a one-week memory test. The arousing value and saliency of gossip information, its rewarding effect evidenced by the increase of beta oscillatory power and the recruitment of areas from the brain reward network in previous fMRI studies, as well as its potential social value have been argued in order to explain its differential processing, encoding and recall.

Evidence for default mode network dysfunction in borderline personality disorder.

BACKGROUND: Although executive and other cognitive deficits have been found in patients with borderline personality disorder (BPD), whether these have brain functional correlates has been little studied. This study aimed to examine patterns of task-related activation and de-activation during the performance of a working memory task in patients with the disorder. METHODS: Sixty-seven DSM-IV BPD patients and 67 healthy controls underwent fMRI during the performance of the n-back task. Linear models were used to obtain maps of within-group activations and areas of differential activation between the groups. RESULTS: On corrected whole-brain analysis, there were no activation differences between the BPD patients and the healthy controls during the main 2-back v. baseline contrast, but reduced activation was seen in the precentral cortex bilaterally and the left inferior parietal cortex in the 2-back v. 1-back contrast. The patients showed failure of de-activation affecting the medial frontal cortex and the precuneus, plus in other areas. The changes did not appear to be attributable to previous history of depression, which was present in nearly half the sample. CONCLUSIONS: In this study, there was some, though limited, evidence for lateral frontal hypoactivation in BPD during the performance of an executive task. BPD also appears to be associated with failure of de-activation in key regions of the default mode network.

The contribution of striatal pseudo-reward prediction errors to value-based decision-making.

Most studies that have investigated the brain mechanisms underlying learning have focused on the ability to learn simple stimulus-response associations. However, in everyday life, outcomes are often obtained through complex behavioral patterns involving a series of actions. Parallel learning systems might be important to reduce the complexity of the learning problem in such scenarios, as proposed in the framework of hierarchical reinforcement learning (HRL). The key feature of HRL is the decomposition of complex sets of action into subgoals. These subgoals are associated with the computation of pseudo-reward prediction errors (PRPEs), which allow the reinforcement of actions that led to a subgoal before the final goal itself is achieved. Here we wanted to test the hypothesis that, despite not carrying any rewarding value per se, pseudo-rewards might generate a bias in choice behavior in the absence of any advantage. Second, we also hypothesized that this bias might be related to the strength of PRPE striatal representations. In order to test these ideas, we developed a novel decision-making paradigm to assess reward prediction errors (RPEs) and PRPEs in two studies (fMRI study: n = 20; behavioral study: n = 19). Our results show that the participants developed a preference for the most pseudo-rewarding option throughout the task, even though it did not lead to more monetary rewards. fMRI analyses revealed that this preference was predicted by individual differences in the relative striatal sensitivity to PRPEs vs RPEs. Together, our results indicate that pseudo-rewards generate learning signals in the striatum and subsequently bias choice behavior despite their lack of association with actual reward.