A Common Neural Account for Social and Nonsocial Decisions.

To date, social and nonsocial decisions have been studied largely in isolation. Consequently, the extent to which social and nonsocial forms of decision uncertainty are integrated using shared neurocomputational resources remains elusive. Here, we address this question using simultaneous electroencephalography (EEG)-functional magnetic resonance imaging (fMRI) in healthy human participants (young adults of both sexes) and a task in which decision evidence in social and nonsocial contexts varies along comparable scales. First, we identify time-resolved build-up of activity in the EEG, akin to a process of evidence accumulation (EA), across both contexts. We then use the endogenous trial-by-trial variability in the slopes of these accumulating signals to construct parametric fMRI predictors. We show that a region of the posterior-medial frontal cortex (pMFC) uniquely explains trial-wise variability in the process of evidence accumulation in both social and nonsocial contexts. We further demonstrate a task-dependent coupling between the pMFC and regions of the human valuation system in dorso-medial and ventro-medial prefrontal cortex across both contexts. Finally, we report domain-specific representations in regions known to encode the early decision evidence for each context. These results are suggestive of a domain-general decision-making architecture, whereupon domain-specific information is likely converted into a "common currency" in medial prefrontal cortex and accumulated for the decision in the pMFC.SIGNIFICANCE STATEMENT Little work has directly compared social-versus-nonsocial decisions to investigate whether they share common neurocomputational origins. Here, using combined electroencephalography (EEG)-functional magnetic resonance imaging (fMRI) and computational modeling, we offer a detailed spatiotemporal account of the neural underpinnings of social and nonsocial decisions. Specifically, we identify a comparable mechanism of temporal evidence integration driving both decisions and localize this integration process in posterior-medial frontal cortex (pMFC). We further demonstrate task-dependent coupling between the pMFC and regions of the human valuation system across both contexts. Finally, we report domain-specific representations in regions encoding the early, domain-specific, decision evidence. These results suggest a domain-general decision-making architecture, whereupon domain-specific information is converted into a common representation in the valuation system and integrated for the decision in the pMFC.

Differential functional connectivity underlying asymmetric reward-related activity in human and nonhuman primates.

The orbitofrontal cortex (OFC) is a key brain region involved in complex cognitive functions such as reward processing and decision making. Neuroimaging studies have reported unilateral OFC response to reward-related variables; however, those studies rarely discussed this observation. Nevertheless, some lesion studies suggest that the left and right OFC contribute differently to cognitive processes. We hypothesized that the OFC asymmetrical response to reward could reflect underlying hemispherical difference in OFC functional connectivity. Using resting-state and reward-related functional MRI data from humans and from rhesus macaques, we first identified an asymmetrical response of the lateral OFC to reward in both species. Crucially, the subregion showing the highest reward-related asymmetry (RRA) overlapped with the region showing the highest functional connectivity asymmetry (FCA). Furthermore, the two types of asymmetries were found to be significantly correlated across individuals. In both species, the right lateral OFC was more connected to the default mode network compared to the left lateral OFC. Altogether, our results suggest a functional specialization of the left and right lateral OFC in primates.

At-risk alcohol users have disrupted valence discrimination during reward anticipation.

Alcohol use disorder is characterised by disrupted reward learning, underpinned by dysfunctional cortico-striatal reward pathways, although relatively little is known about the biology of reward processing in populations who engage in risky alcohol use. Cues that trigger reward anticipation can be categorized according to their learnt valence (i.e., positive vs. negative outcomes) and motivational salience (i.e., incentive vs. neutral cues). Separating EEG signals associated with these dimensions is challenging because of their inherent collinearity, but the recent application of machine learning methods to single EEG trials affords a solution. Here, the Alcohol Use Disorders Identification Test (AUDIT) was used to quantify risky alcohol use, with participants split into high alcohol (HA) (n = 22, mean AUDIT score: 13.82) and low alcohol (LA) (n = 22, mean AUDIT score: 5.77) groups. We applied machine learning multivariate single-trial classification to the electroencephalography (EEG) data collected during reward anticipation. The LA group demonstrated significant valence discrimination in the early stages of reward anticipation within the cue-P3 time window (400-550 ms), whereas the HA group was insensitive to valence within this time window. Notably, the LA, but not the HA group demonstrated a relationship between single-trial variability in the early valence component and reaction times for gain and loss trials. This study evidences disrupted hypoactive valence sensitivity in the HA group, revealing potential neurophysiological markers for risky drinking behaviours which place individuals at-risk of adverse health events.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis.

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Reputational priors magnify striatal responses to violations of trust.

Humans learn to trust each other by evaluating the outcomes of repeated interpersonal interactions. However, available prior information on the reputation of traders may alter the way outcomes affect learning. Our functional magnetic resonance imaging study is the first to allow the direct comparison of interaction-based and prior-based learning. Twenty participants played repeated trust games with anonymous counterparts. We manipulated two experimental conditions: whether or not reputational priors were provided, and whether counterparts were generally trustworthy or untrustworthy. When no prior information is available our results are consistent with previous studies in showing that striatal activation patterns correlate with behaviorally estimated reinforcement learning measures. However, our study additionally shows that this correlation is disrupted when reputational priors on counterparts are provided. Indeed participants continue to rely on priors even when experience sheds doubt on their accuracy. Notably, violations of trust from a cooperative counterpart elicited stronger caudate deactivations when priors were available than when they were not. However, tolerance to such violations appeared to be mediated by prior-enhanced connectivity between the caudate nucleus and ventrolateral prefrontal cortex, which anticorrelated with retaliation rates. Moreover, on top of affecting learning mechanisms, priors also clearly oriented initial decisions to trust, reflected in medial prefrontal cortex activity.

Timing along the cardiac cycle modulates neural signals of reward-based learning.

Natural fluctuations in cardiac activity modulate brain activity associated with sensory stimuli, as well as perceptual decisions about low magnitude, near-threshold stimuli. However, little is known about the relationship between fluctuations in heart activity and other internal representations. Here we investigate whether the cardiac cycle relates to learning-related internal representations - absolute and signed prediction errors. We combined machine learning techniques with electroencephalography with both simple, direct indices of task performance and computational model-derived indices of learning. Our results demonstrate that just as people are more sensitive to low magnitude, near-threshold sensory stimuli in certain cardiac phases, so are they more sensitive to low magnitude absolute prediction errors in the same cycles. However, this occurs even when the low magnitude prediction errors are associated with clearly suprathreshold sensory events. In addition, participants exhibiting stronger differences in their prediction error representations between cardiac cycles exhibited higher learning rates and greater task accuracy.

Transcranial focused ultrasound-mediated neurochemical and functional connectivity changes in deep cortical regions in humans.

Low-intensity transcranial ultrasound stimulation (TUS) is an emerging non-invasive technique for focally modulating human brain function. The mechanisms and neurochemical substrates underlying TUS neuromodulation in humans and how these relate to excitation and inhibition are still poorly understood. In 24 healthy controls, we separately stimulated two deep cortical regions and investigated the effects of theta-burst TUS, a protocol shown to increase corticospinal excitability, on the inhibitory neurotransmitter gamma-aminobutyric acid (GABA) and functional connectivity. We show that theta-burst TUS in humans selectively reduces GABA levels in the posterior cingulate, but not the dorsal anterior cingulate cortex. Functional connectivity increased following TUS in both regions. Our findings suggest that TUS changes overall excitability by reducing GABAergic inhibition and that changes in TUS-mediated neuroplasticity last at least 50 mins after stimulation. The difference in TUS effects on the posterior and anterior cingulate could suggest state- or location-dependency of the TUS effect-both mechanisms increasingly recognized to influence the brain's response to neuromodulation.

Three-layer model with absorption for conservative estimation of the maximum acoustic transmission coefficient through the human skull for transcranial ultrasound stimulation.

Transcranial ultrasound stimulation (TUS) has been shown to be a safe and effective technique for non-invasive superficial and deep brain stimulation. Safe and efficient translation to humans requires estimating the acoustic attenuation of the human skull. Nevertheless, there are no international guidelines for estimating the impact of the skull bone. A tissue independent, arbitrary derating was developed by the U.S. Food and Drug Administration to take into account tissue absorption (0.3 dB/cm-MHz) for diagnostic ultrasound. However, for the case of transcranial ultrasound imaging, the FDA model does not take into account the insertion loss induced by the skull bone, nor the absorption by brain tissue. Therefore, the estimated absorption is overly conservative which could potentially limit TUS applications if the same guidelines were to be adopted. Here we propose a three-layer model including bone absorption to calculate the maximum pressure transmission through the human skull for frequencies ranging between 100 kHz and 1.5 MHz. The calculated pressure transmission decreases with the frequency and the thickness of the bone, with peaks for each thickness corresponding to a multiple of half the wavelength. The 95th percentile maximum transmission was calculated over the accessible surface of 20 human skulls for 12 typical diameters of the ultrasound beam on the skull surface, and varies between 40% and 78%. To facilitate the safe adjustment of the acoustic pressure for short ultrasound pulses, such as transcranial imaging or transcranial ultrasound stimulation, a table summarizes the maximum pressure transmission for each ultrasound beam diameter and each frequency.

Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex.

Environments furnish multiple information sources for making predictions about future events. Here we use behavioural modelling and functional magnetic resonance imaging to describe how humans select predictors that might be most relevant. First, during early encounters with potential predictors, participants' selections were explorative and directed towards subjectively uncertain predictors (positive uncertainty effect). This was particularly the case when many future opportunities remained to exploit knowledge gained. Then, preferences for accurate predictors increased over time, while uncertain predictors were avoided (negative uncertainty effect). The behavioural transition from positive to negative uncertainty-driven selections was accompanied by changes in the representations of belief uncertainty in ventromedial prefrontal cortex (vmPFC). The polarity of uncertainty representations (positive or negative encoding of uncertainty) changed between exploration and exploitation periods. Moreover, the two periods were separated by a third transitional period in which beliefs about predictors' accuracy predominated. The vmPFC signals a multiplicity of decision variables, the strength and polarity of which vary with behavioural context.

Ultrasound modulation of macaque prefrontal cortex selectively alters credit assignment-related activity and behavior.

Credit assignment is the association of specific instances of reward to the specific events, such as a particular choice, that caused them. Without credit assignment, choice values reflect an approximate estimate of how good the environment was when the choice was made­the global reward state­rather than exactly which outcome the choice caused. Combined transcranial ultrasound stimulation (TUS) and functional magnetic resonance imaging in macaques demonstrate credit assignment­related activity in prefrontal area 47/12o, and when this signal was disrupted with TUS, choice value representations across the brain were impaired. As a consequence, behavior was no longer guided by choice value, and decision-making was poorer. By contrast, global reward state­related activity in the adjacent anterior insula remained intact and determined decision-making after prefrontal disruption.

Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys.

People and other animals learn the values of choices by observing the contingencies between them and their outcomes. However, decisions are not guided by choice-linked reward associations alone; macaques also maintain a memory of the general, average reward rate - the global reward state - in an environment. Remarkably, global reward state affects the way that each choice outcome is valued and influences future decisions so that the impact of both choice success and failure is different in rich and poor environments. Successful choices are more likely to be repeated but this is especially the case in rich environments. Unsuccessful choices are more likely to be abandoned but this is especially likely in poor environments. Functional magnetic resonance imaging (fMRI) revealed two distinct patterns of activity, one in anterior insula and one in the dorsal raphe nucleus, that track global reward state as well as specific outcome events.

Neural correlates of weighted reward prediction error during reinforcement learning classify response to cognitive behavioral therapy in depression.

While cognitive behavioral therapy (CBT) is an effective treatment for major depressive disorder, only up to 45% of depressed patients will respond to it. At present, there is no clinically viable neuroimaging predictor of CBT response. Notably, the lack of a mechanistic understanding of treatment response has hindered identification of predictive biomarkers. To obtain mechanistically meaningful fMRI predictors of CBT response, we capitalize on pretreatment neural activity encoding a weighted reward prediction error (RPE), which is implicated in the acquisition and processing of feedback information during probabilistic learning. Using a conventional mass-univariate fMRI analysis, we demonstrate that, at the group level, responders exhibit greater pretreatment neural activity encoding a weighted RPE in the right striatum and right amygdala. Crucially, using multivariate methods, we show that this activity offers significant out-of-sample classification of treatment response. Our findings support the feasibility and validity of neurocomputational approaches to treatment prediction in psychiatry.

Manipulation of Subcortical and Deep Cortical Activity in the Primate Brain Using Transcranial Focused Ultrasound Stimulation.

The causal role of an area within a neural network can be determined by interfering with its activity and measuring the impact. Many current reversible manipulation techniques have limitations preventing their application, particularly in deep areas of the primate brain. Here, we demonstrate that a focused transcranial ultrasound stimulation (TUS) protocol impacts activity even in deep brain areas: a subcortical brain structure, the amygdala (experiment 1), and a deep cortical region, the anterior cingulate cortex (ACC, experiment 2), in macaques. TUS neuromodulatory effects were measured by examining relationships between activity in each area and the rest of the brain using functional magnetic resonance imaging (fMRI). In control conditions without sonication, activity in a given area is related to activity in interconnected regions, but such relationships are reduced after sonication, specifically for the targeted areas. Dissociable and focal effects on neural activity could not be explained by auditory confounds.

The macaque anterior cingulate cortex translates counterfactual choice value into actual behavioral change.

The neural mechanisms mediating sensory-guided decision-making have received considerable attention, but animals often pursue behaviors for which there is currently no sensory evidence. Such behaviors are guided by internal representations of choice values that have to be maintained even when these choices are unavailable. We investigated how four macaque monkeys maintained representations of the value of counterfactual choices-choices that could not be taken at the current moment but which could be taken in the future. Using functional magnetic resonance imaging, we found two different patterns of activity co-varying with values of counterfactual choices in a circuit spanning the hippocampus, the anterior lateral prefrontal cortex and the anterior cingulate cortex. Anterior cingulate cortex activity also reflected whether the internal value representations would be translated into actual behavioral change. To establish the causal importance of the anterior cingulate cortex for this translation process, we used a novel technique, transcranial focused ultrasound stimulation, to reversibly disrupt anterior cingulate cortex activity.

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans.

Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.

Neural correlates of evidence accumulation during value-based decisions revealed via simultaneous EEG-fMRI.

Current computational accounts posit that, in simple binary choices, humans accumulate evidence in favour of the different alternatives before committing to a decision. Neural correlates of this accumulating activity have been found during perceptual decisions in parietal and prefrontal cortex; however the source of such activity in value-based choices remains unknown. Here we use simultaneous EEG-fMRI and computational modelling to identify EEG signals reflecting an accumulation process and demonstrate that the within- and across-trial variability in these signals explains fMRI responses in posterior-medial frontal cortex. Consistent with its role in integrating the evidence prior to reaching a decision, this region also exhibits task-dependent coupling with the ventromedial prefrontal cortex and the striatum, brain areas known to encode the subjective value of the decision alternatives. These results further endorse the proposition of an evidence accumulation process during value-based decisions in humans and implicate the posterior-medial frontal cortex in this process.

Preference for Safe Over Risky Options in Binge Eating.

Binge eating has been usually viewed as a loss of control and an impulsive behavior. But, little is known about the actual behavior of binging patients (prevalently women) in terms of basic decision-making under risk or under uncertainty. In healthy women, stressful cues bias behavior for safer options, raising the question of whether food cues that are perceived as threatening by binging patients may modulate patients' behaviors towards safer options. A cross-sectional study was conducted with binging patients (20 bulimia nervosa (BN) and 23 anorexia nervosa binging (ANB) patients) and two control groups (22 non-binging restrictive (ANR) anorexia nervosa patients and 20 healthy participants), without any concomitant impulsive disorder. We assessed decisions under risk with a gambling task with known probabilities and decisions under uncertainty with the balloon analog risk taking task (BART) with unknown probabilities of winning, in three cued-conditions including neutral, binge food and stressful cues. In the gambling task, binging and ANR patients adopted similar safer attitudes and coherently elicited a higher aversion to losses when primed by food as compared to neutral cues. This held true for BN and ANR patients in the BART. After controlling for anxiety level, these safer attitudes in the food condition were similar to the ones under stress. In the BART, ANB patients exhibited a higher variability in their choices in the food compared to neutral condition. This higher variability was associated with higher difficulties to discard irrelevant information. All these results suggest that decision-making under risk and under uncertainty is not fundamentally altered in all these patients.

Two spatiotemporally distinct value systems shape reward-based learning in the human brain.

Avoiding repeated mistakes and learning to reinforce rewarding decisions is critical for human survival and adaptive actions. Yet, the neural underpinnings of the value systems that encode different decision-outcomes remain elusive. Here coupling single-trial electroencephalography with simultaneously acquired functional magnetic resonance imaging, we uncover the spatiotemporal dynamics of two separate but interacting value systems encoding decision-outcomes. Consistent with a role in regulating alertness and switching behaviours, an early system is activated only by negative outcomes and engages arousal-related and motor-preparatory brain structures. Consistent with a role in reward-based learning, a later system differentially suppresses or activates regions of the human reward network in response to negative and positive outcomes, respectively. Following negative outcomes, the early system interacts and downregulates the late system, through a thalamic interaction with the ventral striatum. Critically, the strength of this coupling predicts participants' switching behaviour and avoidance learning, directly implicating the thalamostriatal pathway in reward-based learning.