Neural Reward Representations Enable Utilitarian Welfare Maximization.

From deciding which meal to prepare for our guests to trading off the proenvironmental effects of climate protection measures against their economic costs, we often must consider the consequences of our actions for the well-being of others (welfare). Vexingly, the tastes and views of others can vary widely. To maximize welfare according to the utilitarian philosophical tradition, decision-makers facing conflicting preferences of others should choose the option that maximizes the sum of the subjective value (utility) of the entire group. This notion requires comparing the intensities of preferences across individuals. However, it remains unclear whether such comparisons are possible at all and (if they are possible) how they might be implemented in the brain. Here, we show that female and male participants can both learn the preferences of others by observing their choices and represent these preferences on a common scale to make utilitarian welfare decisions. On the neural level, multivariate support vector regressions revealed that a distributed activity pattern in the ventromedial prefrontal cortex (VMPFC), a brain region previously associated with reward processing, represented the preference strength of others. Strikingly, also the utilitarian welfare of others was represented in the VMPFC and relied on the same neural code as the estimated preferences of others. Together, our findings reveal that humans can behave as if they maximized utilitarian welfare using a specific utility representation and that the brain enables such choices by repurposing neural machinery processing the reward others receive.

Bayesian reinforcement learning: A basic overview.

We and other animals learn because there is some aspect of the world about which we are uncertain. This uncertainty arises from initial ignorance, and from changes in the world that we do not perfectly know; the uncertainty often becomes evident when our predictions about the world are found to be erroneous. The Rescorla-Wagner learning rule, which specifies one way that prediction errors can occasion learning, has been hugely influential as a characterization of Pavlovian conditioning and, through its equivalence to the delta rule in engineering, in a much wider class of learning problems. Here, we review the embedding of the Rescorla-Wagner rule in a Bayesian context that is precise about the link between uncertainty and learning, and thereby discuss extensions to such suggestions as the Kalman filter, structure learning, and beyond, that collectively encompass a wider range of uncertainties and accommodate a wider assortment of phenomena in conditioning.

The social transmission of empathy relies on observational reinforcement learning.

Theories of moral development propose that empathy is transmitted across individuals. However, the mechanisms through which empathy is socially transmitted remain unclear. Here, we combine computational learning models and functional MRI to investigate whether, and if so, how empathic and non-empathic responses observed in others affect the empathy of female observers. The results of three independent studies showed that watching empathic or non-empathic responses generates a learning signal that respectively increases or decreases empathy ratings of the observer. A fourth study revealed that the learning-related transmission of empathy is stronger when observing human rather than computer demonstrators. Finally, we show that the social transmission of empathy alters empathy-related responses in the anterior insula, i.e., the same region that correlated with empathy baseline ratings, as well as its functional connectivity with the temporoparietal junction. Together, our findings provide a computational and neural mechanism for the social transmission of empathy that accounts for changes in individual empathic responses in empathic and non-empathic social environments.

Reward advantage over punishment for incentivizing visual working memory.

The prospects of gaining reward and avoiding punishment widely influence human behavior. Despite of numerous attempts to investigate the influence of motivational signals on working memory (WM), whether the valence and the magnitude of motivational signals interactively influence WM performance remains unclear. To investigate this, the present study used a free-recall working memory task with EEG recording to compare the effect of incentive valence (reward or punishment), as well as the magnitude of incentives on visual WM. Behavioral results revealed that the presence of incentive signals improved WM precision when compared with no-incentive condition, and compared with punishing cues, rewarding cues led to greater facilitation in WM precision, as well as confidence ratings afterward. Moreover, event related potential (ERP) results suggested that compared with punishment, reward led to an earlier latency of late positive component (LPC), a larger amplitude of contingent negative variation (CNV) during the expectation period, and a larger P300 amplitude during the sample and delay periods. Furthermore, reward advantage over punishment in behavioral and neural results were correlated, such that individuals with larger CNV difference between reward and punishment conditions also report greater distinction in confidence ratings between the two conditions. In sum, our results demonstrate what and how rewarding cues cause more beneficial effects than punishing cues when incentivizing visual WM.

Learning from Ingroup Experiences Changes Intergroup Impressions.

Humans form impressions toward individuals of their own social groups (ingroup members) and of different social groups (outgroup members). Outgroup-focused theories predict that intergroup impressions are mainly shaped by experiences with outgroup individuals, while ingroup-focused theories predict that ingroup experiences play a dominant role. Here we test predictions from these two psychological theories by estimating how intergroup impressions are dynamically shaped when people learn from both ingroup and outgroup experiences. While undergoing fMRI, male participants had identical experiences with different ingroup or outgroup members and rated their social closeness and impressions toward the ingroup and the outgroup. Behavioral results showed an initial ingroup bias in impression ratings which was significantly reduced over the course of learning, with larger effects in individuals with stronger ingroup identification. Computational learning models revealed that these changes in intergroup impressions were predicted by the weight given to ingroup prediction errors. Neurally, the individual weight for ingroup prediction errors was related to the coupling between the left inferior parietal lobule and the left anterior insula, which, in turn, predicted learning-related changes in intergroup impressions. Our findings provide computational and neural evidence for ingroup-focused theories, highlighting the importance of ingroup experiences in shaping social impressions in intergroup settings.Significance Statement:Living in multicultural societies, humans interact with individuals of their own social groups (ingroup members) and of different social groups (outgroup members). However, little is known about how people learn from the mixture of ingroup and outgroup interactions, the most natural experiences in current societies. Here, participants had identical, intermixed experiences with different ingroup and outgroup individuals and rated their closeness and impressions toward the ingroup and the outgroup. Combining computational models and fMRI, we find that the weight given to ingroup experiences (ingroup prediction errors) is the main source of intergroup impression change, captured by changes in connectivity between the parietal lobe and insula. These findings highlight the importance of ingroup experiences in shaping intergroup impressions in complex social environments.

Why We Learn Less from Observing Outgroups.

Humans are less likely to learn from individuals belonging to a different group (outgroup) than from individuals of their own group (ingroup), yet the source of this societally relevant deficit has remained unclear. Here we used neuroimaging and computational modeling to investigate how people learn from observing the actions and outcomes of ingroup and outgroup demonstrators. Politically left-wing male and female participants performed worse when observing computer-simulated actions they believed were from a right-wing outgroup member compared with those from a left-wing ingroup member. A control experiment in which participants observed choices from a nonhuman agent confirmed that this performance difference reflected an outgroup deficit, rather than an ingroup gain. Accounting for the outgroup deficit, a computational model showed that participants relied less on information from outgroup actions compared with ingroup actions, while learning from outgroup outcomes was not impaired. At the neural level, the differences in observational ingroup versus outgroup learning were reflected in lateral prefrontal activity. The stronger the activity in this region, the more strongly participants weighed ingroup compared with outgroup learning signals (action prediction errors), which formally captured deficits in outgroup learning. Together, our work provides a computational and neural account of why people learn less from observing outgroups.SIGNIFICANCE STATEMENT Learning from observing others is an efficient way to acquire knowledge. In our globalized world, "the others" often are people from a different social group (outgroup). There is evidence that people learn less from observing outgroup individuals compared with individuals from their own group (ingroup). However, the source of this outgroup deficit in observational learning remained unknown, which limits our chances to improve intergroup learning. Our results showed that participants rely less on observed outgroup actions compared with ingroup actions, while learning from outgroup outcomes is not impaired. On the neural level, this outgroup deficit was reflected in the activation of the inferior frontal gyrus. These findings imply that intergroup learning should rely on observing outcomes, rather than actions.

Brain Stimulation Over the Frontopolar Cortex Enhances Motivation to Exert Effort for Reward.

BACKGROUND: Loss of motivation is a characteristic feature of several psychiatric and neurological disorders. However, the neural mechanisms underlying human motivation are far from being understood. Here, we investigate the role that the frontopolar cortex (FPC) plays in motivating cognitive and physical effort exertion by computing subjective effort equivalents. METHODS: We manipulated neural processing with transcranial direct current stimulation targeting the FPC while 141 healthy participants decided whether or not to engage in cognitive or physical effort to obtain rewards. RESULTS: We found that brain stimulation targeting the FPC increased the amount of both types of effort participants were willing to exert for rewards. CONCLUSIONS: Our findings provide important insights into the neural mechanisms involved in motivating effortful behavior. Moreover, they suggest that considering the motivation-related activity of the FPC could facilitate the development of treatments for the loss of motivation commonly seen in psychiatric and other neurological disorders.

Dorsomedial prefrontal cortex activity predicts the accuracy in estimating others' preferences.

The ability to accurately estimate another person's preferences is crucial for a successful social life. In daily interactions, we often do this on the basis of minimal information. The aims of the present study were (a) to examine whether people can accurately judge others based only on a brief exposure to their appearances, and (b) to reveal the underlying neural mechanisms with functional magnetic resonance imaging (fMRI). Participants were asked to make guesses about unfamiliar target individuals' preferences for various items after looking at their faces for 3 s. The behavioral results showed that participants estimated others' preferences above chance level. The fMRI data revealed that higher accuracy in preference estimation was associated with greater activity in the dorsomedial prefrontal cortex (DMPFC) when participants were guessing the targets' preferences relative to thinking about their own preferences. These findings suggest that accurate estimations of others' preferences may require increased activity in the DMPFC. A functional connectivity analysis revealed that higher accuracy in preference estimation was related to increased functional connectivity between the DMPFC and the brain regions that are known to be involved in theory of mind processing, such as the temporoparietal junction (TPJ) and the posterior cingulate cortex (PCC)/precuneus, during correct vs. incorrect guessing trials. On the contrary, the tendency to refer to self-preferences when estimating others' preference was related to greater activity in the ventromedial prefrontal cortex. These findings imply that the DMPFC may be a core region in estimating the preferences of others and that higher accuracy may require stronger communication between the DMPFC and the TPJ and PCC/precuneus, part of a neural network known to be engaged in mentalizing.

Neural evidence for individual and cultural variability in the social comparison effect.

Although several studies have investigated the neural mechanism of social comparison, it remains unclear whether and how cultural membership, particularly independent versus interdependent cultures, may differentially shape the neural processes underlying social comparison. In the present functional magnetic resonance imaging (fMRI) study, we examined the behaviors and neural response patterns of Korean (i.e., interdependent culture) and American (i.e., independent culture) participants while performing a financial gambling task simultaneously and independently with a partner. Upon seeing the partner's income, greater modulation of the activity in the ventral striatum (VS) and the ventromedial prefrontal cortex (vmPFC) by relative gain was observed in Korean than American participants, suggesting greater sensitivity of Koreans toward social comparison. The strength of functional connectivity between the VS and the vmPFC predicted individual variability in the degree to which participants' decisions were affected by relative incomes. Additional model-based fMRI analysis further confirmed the primary role of the vmPFC in biasing decisions based on relative incomes. In summary, the present study provides the first neural evidence for decision biases due to social comparison and their individual and cultural variations.

Cultural modulation of self-referential brain activity for personality traits and social identities.

Cross-cultural studies have shown that personality traits are less central and social identities are more important to the selfhood of collectivistic people. However, most cultural neuroscience studies using the self-reference effect (SRE) paradigm have only used personality traits to explore cultural differences in the neural circuits of self-referential processes. In the present study, we used both personality traits and social identities as stimuli in the SRE paradigm and investigated whether and how one's cultural orientation (i.e., individualism vs. collectivism) affects the SRE in the brain. The results showed that the medial prefrontal cortex, anterior cingulate, bilateral temporoparietal regions, and precuneus were involved in self-representation for both personality traits and social identities. Importantly, cultural orientation predicted differential activation patterns in these regions. Collectivists showed stronger activation in the left temporoparietal regions than individualists, who mainly recruited the medial prefrontal regions. Our findings suggest that the personal and social self share common neural substrates, the activation of which can be modulated by one's cultural orientation.