The social transmission of empathy relies on observational reinforcement learning.

Theories of moral development propose that empathy is transmitted across individuals. However, the mechanisms through which empathy is socially transmitted remain unclear. Here, we combine computational learning models and functional MRI to investigate whether, and if so, how empathic and non-empathic responses observed in others affect the empathy of female observers. The results of three independent studies showed that watching empathic or non-empathic responses generates a learning signal that respectively increases or decreases empathy ratings of the observer. A fourth study revealed that the learning-related transmission of empathy is stronger when observing human rather than computer demonstrators. Finally, we show that the social transmission of empathy alters empathy-related responses in the anterior insula, i.e., the same region that correlated with empathy baseline ratings, as well as its functional connectivity with the temporoparietal junction. Together, our findings provide a computational and neural mechanism for the social transmission of empathy that accounts for changes in individual empathic responses in empathic and non-empathic social environments.

Neural Reward Representations Enable Utilitarian Welfare Maximization.

From deciding which meal to prepare for our guests to trading off the proenvironmental effects of climate protection measures against their economic costs, we often must consider the consequences of our actions for the well-being of others (welfare). Vexingly, the tastes and views of others can vary widely. To maximize welfare according to the utilitarian philosophical tradition, decision-makers facing conflicting preferences of others should choose the option that maximizes the sum of the subjective value (utility) of the entire group. This notion requires comparing the intensities of preferences across individuals. However, it remains unclear whether such comparisons are possible at all and (if they are possible) how they might be implemented in the brain. Here, we show that female and male participants can both learn the preferences of others by observing their choices and represent these preferences on a common scale to make utilitarian welfare decisions. On the neural level, multivariate support vector regressions revealed that a distributed activity pattern in the ventromedial prefrontal cortex (VMPFC), a brain region previously associated with reward processing, represented the preference strength of others. Strikingly, also the utilitarian welfare of others was represented in the VMPFC and relied on the same neural code as the estimated preferences of others. Together, our findings reveal that humans can behave as if they maximized utilitarian welfare using a specific utility representation and that the brain enables such choices by repurposing neural machinery processing the reward others receive.

Neural Representation of Valenced and Generic Probability and Uncertainty.

Representing the probability and uncertainty of outcomes facilitates adaptive behavior by allowing organisms to prepare in advance and devote attention to relevant events. Probability and uncertainty are often studied only for valenced (appetitive or aversive) outcomes, raising the question of whether the identified neural machinery also processes the probability and uncertainty of motivationally neutral outcomes. Here, we aimed to dissociate valenced from valence-independent (i.e., generic) probability (p; maximum at p = 1) and uncertainty (maximum at p = 0.5) signals using human neuroimaging. In a Pavlovian task (n = 41; 19 females), different cues predicted appetitive, aversive, or neutral liquids with different probabilities (p = 0, p = 0.5, p = 1). Cue-elicited motor responses accelerated, and pupil sizes increased primarily for cues that predicted valenced liquids with higher probability. For neutral liquids, uncertainty rather than probability tended to accelerate cue-induced responding and decrease pupil size. At the neural level, generic uncertainty signals were limited to the occipital cortex, while generic probability also activated the anterior ventromedial prefrontal cortex. These generic probability and uncertainty signals contrasted with cue-induced responses that only encoded the probability and uncertainty of valenced liquids in medial prefrontal, insular, and occipital cortices. Our findings show a behavioral and neural dissociation of generic and valenced signals. Thus, some parts of the brain keep track of motivational charge while others do not, highlighting the need and usefulness of characterizing the exact nature of learned representations.

Adaptive coding of reward in schizophrenia, its change over time and relationship to apathy.

Adaptive coding of reward is the process by which neurons adapt their response to the context of available compensations. Higher rewards lead to a stronger brain response, but the increase of the response depends on the range of available rewards. A steeper increase is observed in a narrow range and a more gradual slope in a wider range. In schizophrenia, adaptive coding appears to be affected in different domains, especially in the reward domain. Here, we tested adaptive coding of reward in a large group of patients with schizophrenia (n = 86) and control subjects (n = 66). We assessed: (i) the association between adaptive coding deficits and symptoms; (ii) the longitudinal stability of deficits (the same task was performed 3 months apart); and (iii) the stability of results between two experimental sites. We used functional MRI and the monetary incentive delay task to assess adaptation of participants to two different reward ranges: a narrow range and a wide range. We used a region-of-interest analysis to evaluate adaptation within striatal and visual regions. Patients and control subjects underwent a full demographic and clinical assessment. We found reduced adaptive coding in patients, with a decreased slope in the narrow reward range with respect to that of control participants, in striatal but not visual regions. This pattern was observed at both research sites. Upon retesting, patients increased their narrow-range slopes, showing improved adaptive coding, whereas control subjects slightly reduced them. At retesting, patients with overly steep slopes in the narrow range also showed higher levels of negative symptoms. Our data confirm deficits in reward adaptation in schizophrenia and reveal an effect of practice in patients, leading to improvement, with steeper slopes upon retesting. However, in some patients, an excessively steep slope may result in poor discriminability of larger rewards, owing to early saturation of the brain response. Together, the loss of precision of reward representation in new (first exposure, underadaptation) and more familiar (retest, overadaptation) situations might contribute to the multiple motivational symptoms in schizophrenia.

Bayesian reinforcement learning: A basic overview.

We and other animals learn because there is some aspect of the world about which we are uncertain. This uncertainty arises from initial ignorance, and from changes in the world that we do not perfectly know; the uncertainty often becomes evident when our predictions about the world are found to be erroneous. The Rescorla-Wagner learning rule, which specifies one way that prediction errors can occasion learning, has been hugely influential as a characterization of Pavlovian conditioning and, through its equivalence to the delta rule in engineering, in a much wider class of learning problems. Here, we review the embedding of the Rescorla-Wagner rule in a Bayesian context that is precise about the link between uncertainty and learning, and thereby discuss extensions to such suggestions as the Kalman filter, structure learning, and beyond, that collectively encompass a wider range of uncertainties and accommodate a wider assortment of phenomena in conditioning.

Characterizing Human Habits in the Lab.

Habits pose a fundamental puzzle for those aiming to understand human behavior. They pervade our everyday lives and dominate some forms of psychopathology but are extremely hard to elicit in the lab. In this Registered Report, we developed novel experimental paradigms grounded in computational models, which suggest that habit strength should be proportional to the frequency of behavior and, in contrast to previous research, independent of value. Specifically, we manipulated how often participants performed responses in two tasks varying action repetition without, or separately from, variations in value. Moreover, we asked how this frequency-based habitization related to value-based operationalizations of habit and self-reported propensities for habitual behavior in real life. We find that choice frequency during training increases habit strength at test and that this form of habit shows little relation to value-based operationalizations of habit. Our findings empirically ground a novel perspective on the constituents of habits and suggest that habits may arise in the absence of external reinforcement. We further find no evidence for an overlap between different experimental approaches to measuring habits and no associations with self-reported real-life habits. Thus, our findings call for a rigorous reassessment of our understanding and measurement of human habitual behavior in the lab.