Learning rules of engagement for social exchange within and between groups.

Globalizing economies and long-distance trade rely on individuals from different cultural groups to negotiate agreement on what to give and take. In such settings, individuals often lack insight into what interaction partners deem fair and appropriate, potentially seeding misunderstandings, frustration, and conflict. Here, we examine how individuals decipher distinct rules of engagement and adapt their behavior to reach agreements with partners from other cultural groups. Modeling individuals as Bayesian learners with inequality aversion reveals that individuals, in repeated ultimatum bargaining with responders sampled from different groups, can be more generous than needed. While this allows them to reach agreements, it also gives rise to biased beliefs about what is required to reach agreement with members from distinct groups. Preregistered behavioral (N = 420) and neuroimaging experiments (N = 49) support model predictions: Seeking equitable agreements can lead to overly generous behavior toward partners from different groups alongside incorrect beliefs about prevailing norms of what is appropriate in groups and cultures other than one's own.

Effort avoidance as a core mechanism of apathy in frontotemporal dementia.

Apathy is a core symptom in patients with behavioural variant frontotemporal dementia (bvFTD). It is defined by the observable reduction in goal-directed behaviour, but the underlying mechanisms are poorly understood. According to decision theory, engagement in goal-directed behaviour depends on a cost-benefit optimization trading off the estimated effort (related to the behaviour) against the expected reward (related to the goal). In this framework, apathy would thus result from either a decreased appetence for reward, or from an increased aversion to effort. Here, we phenotyped the motivational state of 21 patients with bvFTD and 40 matched healthy controls using computational analyses of behavioural responses in a comprehensive series of behavioural tasks, involving both expression of preference (comparing reward value and effort cost) and optimization of performance (adjusting effort production to the reward at stake). The primary finding was an elevated aversion to effort, consistent across preference and performance tasks in patients with bvFTD compared to controls. Within the bvFTD group, effort avoidance was correlated to cortical atrophy in the dorsal anterior cingulate cortex and to apathy score measured on a clinical scale. Thus, our results highlight elevated effort aversion (not reduced reward appetence) as a core dysfunction that might generate apathy in patients with bvFTD. More broadly, they provide novel behavioural tests and computational tools to identify the dysfunctional mechanisms producing motivation deficits in patients with brain damage.

Social behavioural adaptation in Autism.

Autism is still diagnosed on the basis of subjective assessments of elusive notions such as interpersonal contact and social reciprocity. We propose to decompose reciprocal social interactions in their basic computational constituents. Specifically, we test the assumption that autistic individuals disregard information regarding the stakes of social interactions when adapting to others. We compared 24 adult autistic participants to 24 neurotypical (NT) participants engaging in a repeated dyadic competitive game against artificial agents with calibrated reciprocal adaptation capabilities. Critically, participants were framed to believe either that they were competing against somebody else or that they were playing a gambling game. Only the NT participants did alter their adaptation strategy when they held information regarding others' competitive incentives, in which case they outperformed the AS group. Computational analyses of trial-by-trial choice sequences show that the behavioural repertoire of autistic people exhibits subnormal flexibility and mentalizing sophistication, especially when information regarding opponents' incentives was available. These two computational phenotypes yield 79% diagnosis classification accuracy and explain 62% of the severity of social symptoms in autistic participants. Such computational decomposition of the autistic social phenotype may prove relevant for drawing novel diagnostic boundaries and guiding individualized clinical interventions in autism.

Sour grapes and sweet victories: How actions shape preferences.

Classical decision theory postulates that choices proceed from subjective values assigned to the probable outcomes of alternative actions. Some authors have argued that opposite causality should also be envisaged, with choices influencing subsequent values expressed in desirability ratings. The idea is that agents may increase their ratings of items that they have chosen in the first place, which has been typically explained by the need to reduce cognitive dissonance. However, evidence in favor of this reverse causality has been the topic of intense debates that have not reached consensus so far. Here, we take a novel approach using Bayesian techniques to compare models in which choices arise from stable (but noisy) underlying values (one-way causality) versus models in which values are in turn influenced by choices (two-way causality). Moreover, we examined whether in addition to choices, other components of previous actions, such as the effort invested and the eventual action outcome (success or failure), could also impact subsequent values. Finally, we assessed whether the putative changes in values were only expressed in explicit ratings, or whether they would also affect other value-related behaviors such as subsequent choices. Behavioral data were obtained from healthy participants in a rating-choice-rating-choice-rating paradigm, where the choice task involves deciding whether or not to exert a given physical effort to obtain a particular food item. Bayesian selection favored two-way causality models, where changes in value due to previous actions affected subsequent ratings, choices and action outcomes. Altogether, these findings may help explain how values and actions drift when several decisions are made successively, hence highlighting some shortcomings of classical decision theory.

Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases.

Motivation deficits, such as apathy, are pervasive in both neurological and psychiatric diseases. Even when they are not the core symptom, they reduce quality of life, compromise functional outcome and increase the burden for caregivers. They are currently assessed with clinical scales that do not give any mechanistic insight susceptible to guide therapeutic intervention. Here, we present another approach that consists of phenotyping the behaviour of patients in motivation tests, using computational models. These formal models impose a precise and operational definition of motivation that is embedded in decision theory. Motivation can be defined as the function that orients and activates the behaviour according to two attributes: a content (the goal) and a quantity (the goal value). Decision theory offers a way to quantify motivation, as the cost that patients would accept to endure in order to get the benefit of achieving their goal. We then review basic and clinical studies that have investigated the trade-off between the expected cost entailed by potential actions and the expected benefit associated with potential rewards. These studies have shown that the trade-off between effort and reward involves specific cortical, subcortical and neuromodulatory systems, such that it may be shifted in particular clinical conditions, and reinstated by appropriate treatments. Finally, we emphasize the promises of computational phenotyping for clinical purposes. Ideally, there would be a one-to-one mapping between specific neural components and distinct computational variables and processes of the decision model. Thus, fitting computational models to patients' behaviour would allow inferring of the dysfunctional mechanism in both cognitive terms (e.g. hyposensitivity to reward) and neural terms (e.g. lack of dopamine). This computational approach may therefore not only give insight into the motivation deficit but also help personalize treatment.

Learning about and from others' prudence, impatience or laziness: The computational bases of attitude alignment.

Peoples' subjective attitude towards costs such as, e.g., risk, delay or effort are key determinants of inter-individual differences in goal-directed behaviour. Thus, the ability to learn about others' prudent, impatient or lazy attitudes is likely to be critical for social interactions. Conversely, how adaptive such attitudes are in a given environment is highly uncertain. Thus, the brain may be tuned to garner information about how such costs ought to be arbitrated. In particular, observing others' attitude may change one's uncertain belief about how to best behave in related difficult decision contexts. In turn, learning from others' attitudes is determined by one's ability to learn about others' attitudes. We first derive, from basic optimality principles, the computational properties of such a learning mechanism. In particular, we predict two apparent cognitive biases that would arise when individuals are learning about others' attitudes: (i) people should overestimate the degree to which they resemble others (false-consensus bias), and (ii) they should align their own attitudes with others' (social influence bias). We show how these two biases non-trivially interact with each other. We then validate these predictions experimentally by profiling people's attitudes both before and after guessing a series of cost-benefit arbitrages performed by calibrated artificial agents (which are impersonating human individuals).

Choose, rate or squeeze: Comparison of economic value functions elicited by different behavioral tasks.

A standard view in neuroeconomics is that to make a choice, an agent first assigns subjective values to available options, and then compares them to select the best. In choice tasks, these cardinal values are typically inferred from the preference expressed by subjects between options presented in pairs. Alternatively, cardinal values can be directly elicited by asking subjects to place a cursor on an analog scale (rating task) or to exert a force on a power grip (effort task). These tasks can vary in many respects: they can notably be more or less costly and consequential. Here, we compared the value functions elicited by choice, rating and effort tasks on options composed of two monetary amounts: one for the subject (gain) and one for a charity (donation). Bayesian model selection showed that despite important differences between the three tasks, they all elicited a same value function, with similar weighting of gain and donation, but variable concavity. Moreover, value functions elicited by the different tasks could predict choices with equivalent accuracy. Our finding therefore suggests that comparable value functions can account for various motivated behaviors, beyond economic choice. Nevertheless, we report slight differences in the computational efficiency of parameter estimation that may guide the design of future studies.

Reading wild minds: A computational assay of Theory of Mind sophistication across seven primate species.

Theory of Mind (ToM), i.e. the ability to understand others' mental states, endows humans with highly adaptive social skills such as teaching or deceiving. Candidate evolutionary explanations have been proposed for the unique sophistication of human ToM among primates. For example, the Machiavellian intelligence hypothesis states that the increasing complexity of social networks may have induced a demand for sophisticated ToM. This type of scenario ignores neurocognitive constraints that may eventually be crucial limiting factors for ToM evolution. In contradistinction, the cognitive scaffolding hypothesis asserts that a species' opportunity to develop sophisticated ToM is mostly determined by its general cognitive capacity (on which ToM is scaffolded). However, the actual relationships between ToM sophistication and either brain volume (a proxy for general cognitive capacity) or social group size (a proxy for social network complexity) are unclear. Here, we let 39 individuals sampled from seven non-human primate species (lemurs, macaques, mangabeys, orangutans, gorillas and chimpanzees) engage in simple dyadic games against artificial ToM players (via a familiar human caregiver). Using computational analyses of primates' choice sequences, we found that the probability of exhibiting a ToM-compatible learning style is mainly driven by species' brain volume (rather than by social group size). Moreover, primates' social cognitive sophistication culminates in a precursor form of ToM, which still falls short of human fully-developed ToM abilities.

Computational Dissection of Dopamine Motor and Motivational Functions in Humans.

UNLABELLED: Motor dysfunction (e.g., bradykinesia) and motivational deficit (i.e., apathy) are hallmarks of Parkinson's disease (PD). Yet, it remains unclear whether these two symptoms arise from a same dopaminergic dysfunction. Here, we develop a computational model that articulates motor control to economic decision theory, to dissect the motor and motivational functions of dopamine in humans. This model can capture different aspects of the behavior: choice (which action is selected) and vigor (action speed and intensity). It was used to characterize the behavior of 24 PD patients, tested both when medicated and unmedicated, in two behavioral tasks: an incentive motivation task that involved producing a physical effort, knowing that it would be multiplied by reward level to calculate the payoff, and a binary choice task that involved choosing between high reward/high effort and low reward/low effort options. Model-free analyses in both tasks showed the same two effects when comparing unmedicated patients to medicated patients: dopamine depletion (1) decreased the amount of effort that patients were willing to produce for a given reward and (2) slowed down the production of this effort, regardless of reward level. Model-based analyses captured these effects with two independent parameters, namely reward sensitivity and motor activation rate. These two parameters were respectively predictive of medication effects on clinical measures of apathy and motor dysfunction. More generally, we suggest that such computational phenotyping might help characterizing deficits and refining treatments in neuropsychiatric disorders. SIGNIFICANCE STATEMENT: Many neurological conditions are characterized by motor and motivational deficits, which both result in reduced behavior. It remains extremely difficult to disentangle whether these patients are simply unable or do not want to produce a behavior. Here, we propose a model-based analysis of the behavior produced in tasks that involve trading physical efforts for monetary rewards, to quantify parameters that capture motor dynamics as well as sensitivity to reward, effort, and fatigue. Applied to Parkinson's disease, this computational analysis revealed two independent effects of dopamine enhancers, which predicted clinical improvement in motor and motivational deficits. Such computational profiling might provide a useful explanatory level, between neural dysfunction and clinical manifestations, for characterizing neuropsychiatric disorders and personalizing treatments.

Neurocomputational account of how the human brain decides when to have a break.

No pain, no gain: cost-benefit trade-off has been formalized in classical decision theory to account for how we choose whether to engage effort. However, how the brain decides when to have breaks in the course of effort production remains poorly understood. We propose that decisions to cease and resume work are triggered by a cost evidence accumulation signal reaching upper and lower bounds, respectively. We developed a task in which participants are free to exert a physical effort knowing that their payoff would be proportional to their effort duration. Functional MRI and magnetoencephalography recordings conjointly revealed that the theoretical cost evidence accumulation signal was expressed in proprioceptive regions (bilateral posterior insula). Furthermore, the slopes and bounds of the accumulation process were adapted to the difficulty of the task and the money at stake. Cost evidence accumulation might therefore provide a dynamical mechanistic account of how the human brain maximizes benefits while preventing exhaustion.

Neural mechanisms underlying motivation of mental versus physical effort.

Mental and physical efforts, such as paying attention and lifting weights, have been shown to involve different brain systems. These cognitive and motor systems, respectively, include cortical networks (prefronto-parietal and precentral regions) as well as subregions of the dorsal basal ganglia (caudate and putamen). Both systems appeared sensitive to incentive motivation: their activity increases when we work for higher rewards. Another brain system, including the ventral prefrontal cortex and the ventral basal ganglia, has been implicated in encoding expected rewards. How this motivational system drives the cognitive and motor systems remains poorly understood. More specifically, it is unclear whether cognitive and motor systems can be driven by a common motivational center or if they are driven by distinct, dedicated motivational modules. To address this issue, we used functional MRI to scan healthy participants while performing a task in which incentive motivation, cognitive, and motor demands were varied independently. We reasoned that a common motivational node should (1) represent the reward expected from effort exertion, (2) correlate with the performance attained, and (3) switch effective connectivity between cognitive and motor regions depending on task demand. The ventral striatum fulfilled all three criteria and therefore qualified as a common motivational node capable of driving both cognitive and motor regions of the dorsal striatum. Thus, we suggest that the interaction between a common motivational system and the different task-specific systems underpinning behavioral performance might occur within the basal ganglia.

The social Bayesian brain: does mentalizing make a difference when we learn?

When it comes to interpreting others' behaviour, we almost irrepressibly engage in the attribution of mental states (beliefs, emotions…). Such "mentalizing" can become very sophisticated, eventually endowing us with highly adaptive skills such as convincing, teaching or deceiving. Here, sophistication can be captured in terms of the depth of our recursive beliefs, as in "I think that you think that I think…" In this work, we test whether such sophisticated recursive beliefs subtend learning in the context of social interaction. We asked participants to play repeated games against artificial (Bayesian) mentalizing agents, which differ in their sophistication. Critically, we made people believe either that they were playing against each other, or that they were gambling like in a casino. Although both framings are similarly deceiving, participants win against the artificial (sophisticated) mentalizing agents in the social framing of the task, and lose in the non-social framing. Moreover, we find that participants' choice sequences are best explained by sophisticated mentalizing Bayesian learning models only in the social framing. This study is the first demonstration of the added-value of mentalizing on learning in the context of repeated social interactions. Importantly, our results show that we would not be able to decipher intentional behaviour without a priori attributing mental states to others.

VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data.

This work is in line with an on-going effort tending toward a computational (quantitative and refutable) understanding of human neuro-cognitive processes. Many sophisticated models for behavioural and neurobiological data have flourished during the past decade. Most of these models are partly unspecified (i.e. they have unknown parameters) and nonlinear. This makes them difficult to peer with a formal statistical data analysis framework. In turn, this compromises the reproducibility of model-based empirical studies. This work exposes a software toolbox that provides generic, efficient and robust probabilistic solutions to the three problems of model-based analysis of empirical data: (i) data simulation, (ii) parameter estimation/model selection, and (iii) experimental design optimization.

Inferring on the intentions of others by hierarchical Bayesian learning.

Inferring on others' (potentially time-varying) intentions is a fundamental problem during many social transactions. To investigate the underlying mechanisms, we applied computational modeling to behavioral data from an economic game in which 16 pairs of volunteers (randomly assigned to "player" or "adviser" roles) interacted. The player performed a probabilistic reinforcement learning task, receiving information about a binary lottery from a visual pie chart. The adviser, who received more predictive information, issued an additional recommendation. Critically, the game was structured such that the adviser's incentives to provide helpful or misleading information varied in time. Using a meta-Bayesian modeling framework, we found that the players' behavior was best explained by the deployment of hierarchical learning: they inferred upon the volatility of the advisers' intentions in order to optimize their predictions about the validity of their advice. Beyond learning, volatility estimates also affected the trial-by-trial variability of decisions: participants were more likely to rely on their estimates of advice accuracy for making choices when they believed that the adviser's intentions were presently stable. Finally, our model of the players' inference predicted the players' interpersonal reactivity index (IRI) scores, explicit ratings of the advisers' helpfulness and the advisers' self-reports on their chosen strategy. Overall, our results suggest that humans (i) employ hierarchical generative models to infer on the changing intentions of others, (ii) use volatility estimates to inform decision-making in social interactions, and (iii) integrate estimates of advice accuracy with non-social sources of information. The Bayesian framework presented here can quantify individual differences in these mechanisms from simple behavioral readouts and may prove useful in future clinical studies of maladaptive social cognition.

Spatial attention, precision, and Bayesian inference: a study of saccadic response speed.

Inferring the environment's statistical structure and adapting behavior accordingly is a fundamental modus operandi of the brain. A simple form of this faculty based on spatial attentional orienting can be studied with Posner's location-cueing paradigm in which a cue indicates the target location with a known probability. The present study focuses on a more complex version of this task, where probabilistic context (percentage of cue validity) changes unpredictably over time, thereby creating a volatile environment. Saccadic response speed (RS) was recorded in 15 subjects and used to estimate subject-specific parameters of a Bayesian learning scheme modeling the subjects' trial-by-trial updates of beliefs. Different response models-specifying how computational states translate into observable behavior-were compared using Bayesian model selection. Saccadic RS was most plausibly explained as a function of the precision of the belief about the causes of sensory input. This finding is in accordance with current Bayesian theories of brain function, and specifically with the proposal that spatial attention is mediated by a precision-dependent gain modulation of sensory input. Our results provide empirical support for precision-dependent changes in beliefs about saccade target locations and motivate future neuroimaging and neuropharmacological studies of how Bayesian inference may determine spatial attention.

Modelling trial-by-trial changes in the mismatch negativity.

The mismatch negativity (MMN) is a differential brain response to violations of learned regularities. It has been used to demonstrate that the brain learns the statistical structure of its environment and predicts future sensory inputs. However, the algorithmic nature of these computations and the underlying neurobiological implementation remain controversial. This article introduces a mathematical framework with which competing ideas about the computational quantities indexed by MMN responses can be formalized and tested against single-trial EEG data. This framework was applied to five major theories of the MMN, comparing their ability to explain trial-by-trial changes in MMN amplitude. Three of these theories (predictive coding, model adjustment, and novelty detection) were formalized by linking the MMN to different manifestations of the same computational mechanism: approximate Bayesian inference according to the free-energy principle. We thereby propose a unifying view on three distinct theories of the MMN. The relative plausibility of each theory was assessed against empirical single-trial MMN amplitudes acquired from eight healthy volunteers in a roving oddball experiment. Models based on the free-energy principle provided more plausible explanations of trial-by-trial changes in MMN amplitude than models representing the two more traditional theories (change detection and adaptation). Our results suggest that the MMN reflects approximate Bayesian learning of sensory regularities, and that the MMN-generating process adjusts a probabilistic model of the environment according to prediction errors.

A neurocomputational model of the mismatch negativity.

The mismatch negativity (MMN) is an event related potential evoked by violations of regularity. Here, we present a model of the underlying neuronal dynamics based upon the idea that auditory cortex continuously updates a generative model to predict its sensory inputs. The MMN is then modelled as the superposition of the electric fields evoked by neuronal activity reporting prediction errors. The process by which auditory cortex generates predictions and resolves prediction errors was simulated using generalised (Bayesian) filtering--a biologically plausible scheme for probabilistic inference on the hidden states of hierarchical dynamical models. The resulting scheme generates realistic MMN waveforms, explains the qualitative effects of deviant probability and magnitude on the MMN - in terms of latency and amplitude--and makes quantitative predictions about the interactions between deviant probability and magnitude. This work advances a formal understanding of the MMN and--more generally--illustrates the potential for developing computationally informed dynamic causal models of empirical electromagnetic responses.

Your goal is mine: unraveling mimetic desires in the human brain.

The spread of desires among individuals is widely believed to shape motivational drives in human populations. However, objective evidence for this phenomenon and insights into the underlying brain mechanisms are still lacking. Here we show that participants rated objects as more desirable once perceived as the goals of another agent's action. We then unravel the mechanisms underpinning such goal contagion, using functional neuroimaging. As expected, observing goal-directed actions activated a parietofrontal network known as the mirror neuron system (MNS), whereas subjective desirability ratings were represented in a ventral striatoprefrontal network known as the brain valuation system (BVS). Crucially, the induction of mimetic desires through action observation involved the modulation of BVS activity through MNS activity. Furthermore, MNS-BVS effective connectivity predicted individual susceptibility toward mimetic desires. We therefore suggest that MNS-BVS interaction represents a fundamental mechanism explaining how nonverbal behavior propagates desires without the need for explicit, intentional communication.

Model selection and gobbledygook: response to Lohmann et al.

Lohmann et al. (this issue) make three unremarkable observations about model selection and use them to critique dynamic causal modelling-a Bayesian model selection procedure based on causal models of dynamical systems (Marreiros et al., 2010). In this response, we unpack their misconceptions and try to answer their questions.

Variational Bayesian mixed-effects inference for classification studies.

Multivariate classification algorithms are powerful tools for predicting cognitive or pathophysiological states from neuroimaging data. Assessing the utility of a classifier in application domains such as cognitive neuroscience, brain-computer interfaces, or clinical diagnostics necessitates inference on classification performance at more than one level, i.e., both in individual subjects and in the population from which these subjects were sampled. Such inference requires models that explicitly account for both fixed-effects (within-subjects) and random-effects (between-subjects) variance components. While models of this sort are standard in mass-univariate analyses of fMRI data, they have not yet received much attention in multivariate classification studies of neuroimaging data, presumably because of the high computational costs they entail. This paper extends a recently developed hierarchical model for mixed-effects inference in multivariate classification studies and introduces an efficient variational Bayes approach to inference. Using both synthetic and empirical fMRI data, we show that this approach is equally simple to use as, yet more powerful than, a conventional t-test on subject-specific sample accuracies, and computationally much more efficient than previous sampling algorithms and permutation tests. Our approach is independent of the type of underlying classifier and thus widely applicable. The present framework may help establish mixed-effects inference as a future standard for classification group analyses.